Compare commits

..

126 Commits

Author SHA1 Message Date
Junyan Qin e8a09b7537 fix(build): install git for pinned SDK 2026-07-24 19:29:14 +08:00
Junyan Qin 98f45aa88e feat(tenancy): connect cloud workspace control plane 2026-07-24 19:11:33 +08:00
Junyan Qin d7cdd206c2 docs(tenancy): record final isolation verification 2026-07-24 16:22:45 +08:00
Junyan Qin ac72563664 fix(tenancy): close isolation and permission gaps 2026-07-24 16:22:45 +08:00
Junyan Qin 64dc887b20 docs(tenancy): record final isolation verification 2026-07-24 16:22:45 +08:00
Junyan Qin 627eb6b8ef feat(tenancy): harden shared cloud runtime boundaries 2026-07-24 16:22:45 +08:00
Junyan Qin 3f01ffe63b feat(tenancy): establish cloud isolation foundations 2026-07-24 16:22:45 +08:00
Junyan Qin d7adbeec1e docs: finalize cloud v2 multi-tenant decisions 2026-07-24 16:22:44 +08:00
Junyan Qin abf77cecfa docs(tenancy): refine architecture options 2026-07-24 16:22:44 +08:00
Junyan Qin 270622ae9d docs(tenancy): revise single-instance SaaS topology 2026-07-24 16:22:44 +08:00
Junyan Qin 30f414a534 docs(tenancy): record verification evidence 2026-07-24 16:22:44 +08:00
Junyan Qin 8b7ce77cec feat(tenancy): implement workspace isolation 2026-07-24 16:22:44 +08:00
Junyan Qin 37099ddf7e docs: redesign multi-tenant workspace architecture 2026-07-24 16:22:44 +08:00
Junyan Qin ee59e2d3fd Add OSS and commercial workspace boundaries 2026-07-24 16:22:44 +08:00
Junyan Qin a4550350c0 Document multi-tenant workspace architecture 2026-07-24 16:22:44 +08:00
Hyu 38e35d328a feat(web): add marketplace likes (#2352)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-24 15:16:37 +08:00
RockChinQ 4226f71f05 fix(runtime): make plugin and box connectors resilient (#2347)
Preserve the contributor-authored MCP timeout commit and the follow-up configurable timeout and runtime robustness fixes.
2026-07-23 20:25:21 +08:00
Junyan Qin 283c6949f4 fix(mcp): validate tool call timeout config 2026-07-23 18:26:41 +08:00
Junyan Qin 0dfae76e39 fix(mcp): make tool call timeout configurable 2026-07-23 18:26:37 +08:00
douxt 7677d1a288 fix(mcp): add 30s timeout to prevent MCP tool calls from hanging indefinitely
Previously, MCP tool calls via call_tool() had no timeout, so a hung MCP
server would block the entire session indefinitely (exacerbated by
concurrency.session=1). This wraps the call in asyncio.timeout(30) and
raises an Exception on expiry, letting the LLM recover gracefully.

Closes #2339
2026-07-23 18:26:18 +08:00
Hyu a53a41b1ca feat(web): trim dynamic form strings on save (#2349) 2026-07-23 18:14:54 +08:00
Junyan Qin 73e47ea2d4 fix(mcp): keep waiting for box recovery 2026-07-23 17:41:27 +08:00
Junyan Qin 06e03af994 fix(runtime): complete reconnect recovery paths 2026-07-23 17:28:31 +08:00
Junyan Qin 9cbbaf617b fix(runtime): consume sdk shutdown fix 2026-07-23 16:46:48 +08:00
Junyan Qin 210e5349d9 fix(runtime): preserve lockfile metadata 2026-07-23 16:34:03 +08:00
Junyan Qin a2f6814517 fix(runtime): pin robust sdk release 2026-07-23 16:28:33 +08:00
Junyan Qin 8511178666 fix(runtime): harden connector lifecycle 2026-07-23 16:03:26 +08:00
Junyan Qin 998b76d53a fix(runtime): make plugin and box connectors resilient 2026-07-21 18:43:24 +08:00
Junyan Qin 1765c43262 Revert "fix(runtime): make plugin and box connectors resilient"
This reverts commit 0b461e5830.
2026-07-21 18:43:01 +08:00
Junyan Qin 0b461e5830 fix(runtime): make plugin and box connectors resilient 2026-07-21 18:41:22 +08:00
DongXiaoming 76c5003c21 fix(localagent): stop re-seeding stream accumulator with previous round content (#2329)
When the LLM (e.g. MiniMax-M3) returns multiple rounds after tool calls, the _StreamAccumulator was initialized with initial_content=first_content, causing every subsequent round to repeat the entire first message. Remove the re-seeding so each round starts with a clean accumulator.

Add regression test verifying multi-round tool call content is not duplicated.
2026-07-20 16:38:11 +08:00
Hyu 6baeb032a7 fix(web): show page component icon on plugin cards (#2345)
Co-authored-by: dadachann <dadachann@users.noreply.github.com>
2026-07-17 16:39:38 +08:00
Hyu cb6c8d1eb6 chore(release): bump version to 4.10.6 (#2343)
Co-authored-by: dadachann <dadachann@users.noreply.github.com>
2026-07-16 21:56:54 +08:00
Hyu 48952206db fix(lark): keep connection lookup off event loop
Move the Lark SDK synchronous connection URL lookup off the main asyncio loop, serialize reconnects, and cover the incident with a non-blocking regression test.
2026-07-15 19:53:48 +08:00
hedging8563 e5f0ffd960 feat(provider): add TokenLab requester (#2326) 2026-07-15 18:10:05 +08:00
advancer-young c7720b126d fix(mcp): reconnect session when timeout (#2340) (#2340)
Co-authored-by: yang.xiang <yang.xiang@advancegroup.com>
2026-07-15 18:09:47 +08:00
advancer-young 1d15798e5f feat(lark): support render markdown table in lark message (#2338) (#2338)
Co-authored-by: yang.xiang <yang.xiang@advancegroup.com>
2026-07-15 18:09:16 +08:00
WierJZ bd87bb453c fix(dingtalk): avoid blocking event loop on connection open (#2337) 2026-07-15 18:04:42 +08:00
Hyu 0eb1bf684a docs(readme): reorder demo and remove star history (#2341)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-15 14:43:18 +08:00
DongXiaoming c53b800267 fix(provider): strip think tags for MiniMax-M3 and other OpenAI-compatible models (#2330)
* fix(provider): strip think tags for MiniMax-M3 and other OpenAI-compatible models

MiniMax-M3 (and other OpenAI-compatible providers) emit chain-of-thought
reasoning directly in the content field wrapped in  tags, instead
of using a separate reasoning_content field or the legacy CRETIRE_REASONING
markers. The existing remove_think logic only handled CRETIRE_* tags, so
think blocks leaked into user-visible output even when remove_think was enabled.

- Add _ThinkStripState: a stateful filter that correctly handles  tags
  split across streaming chunk boundaries.
- Add _strip_think classmethod with regex patterns for both  and
  CRETIRE_* tags.
- Wire think_state into invoke_llm_stream so deltas are filtered before
  reaching the accumulator.
- Add remove_think safety net in _StreamAccumulator so the final message
  from tool-call rounds also gets stripped.
- Fix remove_think resolution to use defensive nested .get() so
  pipelines missing output.misc don't raise AttributeError.

* fix(litellmchat): add missing _CLOSE_TAG class attribute on _ThinkStripState

* fix(provider): handle think stripping across LiteLLM paths

---------

Co-authored-by: WangCham <651122857@qq.com>
2026-07-15 10:54:41 +08:00
RockChinQ bd35b793e1 fix(embed): isolate browser chat sessions (#2335)
* fix(embed): isolate browser chat sessions

* fix(embed): harden reset and reconnect recovery
2026-07-13 15:08:45 +08:00
Stevenqin 2c3e52c16c fix(aiocqhttp): correct listener lifecycle handling (#2336)
* fix(aiocqhttp): correct listener lifecycle handling

* docs(aiocqhttp): add listener lifecycle evidence
2026-07-13 13:50:08 +08:00
Dongchuan Fu 0755beebcd feat: add supports for dify hitl (#2226)
* feat: Implement workflow form handling for paused workflows

- Added module-level storage for pending forms to manage state across sessions.
- Introduced functions to set, get, and clear pending forms with expiration handling.
- Enhanced DifyServiceAPIRunner to support resuming paused workflows via form actions.
- Implemented logic to yield human input requests and display appropriate messages.
- Updated workflow submission methods to handle paused states and resume actions.
- Ensured proper merging of pending form actions with user inputs for seamless interaction.

* feat: Add '_routed_by_rule' variable to form action in Lark and Telegram adapters

* feat: Enhance Lark and Telegram adapters with new form handling for paused workflows

* feat: Enhance TelegramAdapter to handle form action buttons and message threading

* feat: Improve TelegramAdapter message handling with enhanced error management and draft message support

* feat: Add the function for formatting human input text to support adapters without rich UI.

* feat(dingtalk): implement human input card support and card action handling

- Add a new module `card_callback.py` to handle card action button clicks from DingTalk.
- Introduce `DingTalkCardActionHandler` to process card action callbacks and extract parameters.
- Update `DingTalkAdapter` to manage card state and handle form input through a single card template.
- Add configuration for `human_input_card_template_id` in `dingtalk.yaml` to specify the template for human input.
- Create a new card template `dingtalk_human_input_card.json` for rendering human input prompts and buttons.

* feat(dingtalk): enhance human input card functionality with streaming support and active turn management

- Updated the DingTalk card template to enable streaming mode and multi-update configuration.
- Removed the obsolete delete_card method from DingTalkClient to streamline card management.
- Enhanced DingTalkAdapter to manage active turn cards and accumulated streaming text, ensuring a seamless user experience during human input prompts.
- Modified the create_message_card method to utilize existing active cards for resumed workflows, preventing duplication.
- Improved the _paint_form_on_card method to update existing cards with human input prompts and buttons dynamically.
- Updated the dingtalk_human_input_card.json template to reflect the new streaming capabilities and configuration options.

* feat(wecom): implement Dify human input pause handling with button interaction support

* feat(qqofficial): implement Dify human input button interaction handling and markdown keyboard support

* feat(qqofficial): implement one-click QR binding and enhance localization support

* feat(discord): implement Discord form view with button interactions for Dify actions

* fix(telegram): correct group chat type check and handle oversized callback data for Telegram actions
fix(difysvapi): ensure safe access to remove-think configuration in pipeline settings

* feat(dify): add support for chatflow app type and enhance human input handling

* feat(telegram): add action title feedback for user selections in Telegram messages

* feat(lark): enhance LarkAdapter to store form content for resume notices

* feat(dingtalk): update display formatting for card content with HTML line breaks

* feat(dingtalk): add feedback functionality to cards with 👍/👎 buttons

- Implemented feedback state management for cards, allowing users to provide feedback via thumbs up/down buttons.
- Enhanced card rendering to include feedback buttons when appropriate.
- Registered feedback listeners to handle feedback events and update card states accordingly.
- Updated the card template to support dynamic button rendering for feedback actions.
- Improved error handling and logging for feedback actions and card updates.

* fix: add Avatar component to dingtalk_human_input_card.json for enhanced user interaction

* feat(wecom): add optional source block to interactive template cards for enhanced branding

* feat(wecom): add functions for template card action extraction and update, enhance button interaction handling

* feat(qqofficial): synchronize passive-reply counter with inbound message sequence

* feat(qqofficial): add method to identify invisible form placeholder chunks in messages

* feat(dingtalk): add download link for human input card template and enhance dynamic form configuration

* feat(telegram): enhance message handling with group stream deletion and form placeholder detection

* Add unit tests for DingTalk, Lark, WeComBot, and Dify service API runners

- Implement tests for DingTalk adapter helper functions including form content cleaning, input extraction, and completed input lines.
- Create unit tests for Lark adapter helper functions focusing on input extraction and completed input lines.
- Add tests for WeComBot template card functionalities, including event extraction and payload building for human input.
- Enhance Dify service API runner tests to cover human input forms, including input collection, action handling, and form snapshot extraction.

* feat: Enhance Telegram and QQ Official adapters with select field handling and form action processing

- Added support for select fields in Telegram adapter, including option extraction and callback handling.
- Implemented form action processing for Telegram callbacks, improving user interaction feedback.
- Introduced new helper functions for building keyboards and resolving select button actions in QQ Official adapter.
- Enhanced DifyServiceAPIRunner to handle cumulative streaming responses and improve error handling during workflow resumes.
- Added unit tests for new functionalities in Telegram and QQ Official adapters, ensuring robust behavior for select fields and form actions.

* feat(lark): add functions for current input definitions and visible form content handling
feat(qqofficial): update fallback text handling for non-streaming scenarios
feat(difysvapi): enhance form content processing for interactive fields and actions
test: add unit tests for Lark and QQ Official adapter functionalities

* Add tests for DingTalk adapter content processing and markdown formatting

- Updated the assertion in `test_dingtalk_completed_input_lines_include_text_and_select_values` to remove unnecessary markdown formatting.
- Added new tests to verify that `_dingtalk_clean_form_content` maintains the order of prompts and completed values in various scenarios.
- Introduced `test_dingtalk_card_markdown_preserves_internal_line_breaks` to ensure internal line breaks are correctly converted to HTML line breaks.

* feat: Refactor input handling and feedback messages across multiple adapters

* feat: Update the human-computer interaction template cards, and optimize the prompt information and content display.

* feat: Refactor pending form handling to isolate by bot and pipeline

* feat: Enhance error handling and caching for Dify and WeCom interactions

* feat: Enhance select input handling and validation in Dify API runner and Telegram adapter

* feat: Add missing completed input lines handling in DingTalk adapter

* feat: Add pipeline_uuid handling across multiple adapters and update related tests
2026-07-13 00:42:46 +08:00
RockChinQ 3ddebd26ae fix: skip valkey-glide on Windows (#2333)
Skip the unsupported valkey-glide dependency on Windows while preserving automatic installation on supported platforms. Keep missing-client runtime and test paths safe, and update the Valkey integration documentation.
2026-07-13 00:22:12 +08:00
Guanchao Wang 2a3f7f8059 fix(mcp): harden remote SSE fallback (#2324)
* fix(mcp): harden remote SSE fallback

* fix(mcp): preserve transport errors during fallback

---------

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-07-11 22:08:37 +08:00
RockChinQ 940234a0d8 docs: update blog links to main site 2026-07-09 14:13:04 -04:00
Daria Korenieva 0c405901d2 feat(vector): add Valkey Search vector database backend (#2276)
* feat(vector): add Valkey Search vector database backend

Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.

- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
  all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
  accepted for interface parity but NOT honored (docstring + one-time warning +
  docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
  client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
  (vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
  change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
  (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.

* fix(vector): paginate Valkey Search deletes and guard delete_by_filter

Address self-review follow-ups for the Valkey Search VDB backend:

- _search_keys now paginates through the full result set in batches of
  _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
  page, so delete_by_file_id / delete_by_filter fully remove files and
  filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
  a filter referencing only non-indexed fields must skip and return 0
  (never fall back to match-all), and a supported filter still deletes
  matching keys.

* refactor(vector): harden Valkey Search backend and add adversarial tests

Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
  instead of building ServerCredentials(password=None, ...), which glide
  rejects).
- Add an async close() teardown that closes the glide client and resets
  cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
  never widens).

Tests:
- Add an adversarial-input integration test proving crafted file_id /
  query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* fix(vector): make Valkey Search file_id TAG support arbitrary characters

Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.

Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* refactor(vector): address Valkey Search review feedback

- Add configurable request_timeout (default 5000ms; glide default 250ms is
  too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
  mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
  the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
  _reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics

* fix(vector/valkey_search): address round-2 review feedback

- Serialize lazy client creation with an asyncio.Lock (double-checked) so
  concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
  than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
  non-numeric input and pre-empting a future NUMERIC-field injection surface.

* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-07-08 06:59:16 +08:00
Hyu 00e2103873 docs(config): add box.default_memory_mb to config.yaml template (#2319)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 15:18:27 +08:00
Hyu 209706b0b9 feat(mcp): add box.default_memory_mb config for nsjail memory limit (#2318)
Operators can now set a global default memory limit for all stdio MCP
servers in config.yaml or via environment variable:

  config.yaml:
    box:
      default_memory_mb: 2048  # default: 1536

  env:
    BOX__DEFAULT_MEMORY_MB=2048

The default is raised from 1024 to 1536 MB — a safer floor for
Node.js V8 + WASM (undici llhttp) under nsjail cgroup limits.
Individual MCP servers can still override via their own box.memory_mb.

Previously the fallback was hardcoded to 1024 MB, causing OOM kills
(return_code=137) on node/npx MCP servers that need more RAM.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 14:24:06 +08:00
Hyu 205404e3da fix(deps): pin langbot-plugin 0.4.13 (memory_mb removed from _COMPAT_FIELDS) (#2317)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 13:45:13 +08:00
Hyu 3e93ccfb45 fix(mcp): use uniform session memory_mb=1024 to avoid BoxSessionConflictError (#2316)
All MCPs share one Box session (mcp-shared). When session memory_mb differed
by command type (512 for python, 1024 for node), the second MCP to call
create_session raised BoxSessionConflictError. Fix: always use 1024 MB for
the shared session so python and node MCPs coexist without conflict.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:55:49 +08:00
Hyu c5d7e3dcb1 fix(deps): pin langbot-plugin 0.4.12 (BoxManagedProcessSpec.memory_mb fix) (#2315)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:31:09 +08:00
Hyu cd6d5d9c2f fix(mcp): pin langbot-plugin 0.4.10 - per-process memory_mb for node OOM fix (#2314)
* fix(mcp): bump default memory to 1024MB for node (npx) stdio MCP servers

Node.js MCP servers (npx/bunx) were being OOM-killed (return_code=137) by the
default 512MB nsjail cgroup_mem_max. Node V8 reserves large virtual address
space and instantiates WebAssembly modules (undici llhttp) on startup, easily
exceeding 512MB resident. This caused every node-based MCP (memory,
sequential-thinking, filesystem, weather, docker, excel) to crash-loop.

Fix: when the stdio command is npx/bunx/pnpm, default memory_mb to 1024 unless
the operator explicitly set a value. Python/uvx servers keep the 512MB default.

* chore(deps): pin langbot-plugin 0.4.10 (per-process memory_mb fix)

* chore: update uv.lock for langbot-plugin 0.4.10

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:03:43 +08:00
Hyu 1f7d9339fc fix(mcp): bump default memory to 1024MB for node (npx) stdio MCP servers (#2313)
Node.js MCP servers (npx/bunx) were being OOM-killed (return_code=137) by the
default 512MB nsjail cgroup_mem_max. Node V8 reserves large virtual address
space and instantiates WebAssembly modules (undici llhttp) on startup, easily
exceeding 512MB resident. This caused every node-based MCP (memory,
sequential-thinking, filesystem, weather, docker, excel) to crash-loop.

Fix: when the stdio command is npx/bunx/pnpm, default memory_mb to 1024 unless
the operator explicitly set a value. Python/uvx servers keep the 512MB default.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 08:31:08 +08:00
RockChinQ f3b5fcfb7c fix mcp sidebar status sync (#2312)
Co-authored-by: Hyu <chenhyu@proton.me>
2026-07-03 20:23:12 +08:00
Hyu ffabb91bfe fix(mcp): add missing MCPLogs import to MCPForm (#2311)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 17:59:02 +08:00
Hyu 96c84740db feat(mcp): add logs tab to MCP server detail panel (frontend) (#2310)
* feat(mcp): add logs tab to MCP server detail panel

- Add MCPLogs component mirroring PluginLogs functionality
- Add getMcpServerLogs API method to BackendClient
- Add logs tab to MCPForm detail panel (edit mode only)
- Add i18n keys (tabLogs, logsLevelAll, logsRefresh, logsAutoRefresh, logsEmpty) to all 8 locales
- Auto-refresh logs every 3s with toggle
- Filter logs by level (ALL/DEBUG/INFO/WARNING/ERROR)

* style: fix prettier formatting in MCPForm.tsx logs tab

* style: fix prettier in i18n locale files

---------

Co-authored-by: dadachann <dadachann@users.noreply.github.com>
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 16:52:15 +08:00
Hyu 85b5b5b54b feat(mcp): add MCP server log panel backend (#2308)
- Add log buffer (_log_buffer: deque with maxlen=500) to RuntimeMCPSession
- Capture stderr from Box managed process in monitor_process_health loop
- Add get_mcp_server_logs service method with limit and level filtering
- Add GET /servers/<server_name>/logs HTTP endpoint
- Parse log level from stderr lines (error/warning/debug/info)
- Tests passing: 211 passed, 12 skipped

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 16:05:23 +08:00
Hyu 28bffdef21 fix(mcp): set _preserve_managed_process before finally in cold-start path (#2309)
_ColdStartRetry was caught in _lifecycle_loop_with_retry which set
_preserve_managed_process = True — but by then the finally block inside
_lifecycle_loop had already run and called _cleanup_box_stdio_session(),
stopping the live managed process (return_code=143 SIGTERM). The cold-start
retry then restarted a fresh process, eliminating the warm-up advantage.

Fix: add an explicit except _ColdStartRetry in _lifecycle_loop that sets
_preserve_managed_process = True before re-raising. The finally block then
sees the flag and skips stop_managed_process, leaving the live process
untouched for the next handshake attempt.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 15:26:18 +08:00
Hyu bf3c96026b fix(mcp): fix cold-start connection drop by using lexically-intact owner exit stack (#2307)
The previous _TransferredStack approach broke anyio lexical context:
websocket_client/ClientSession use anyio task groups whose cancel scope is
bound to the frame that entered them. Deferring their aclose via a transferred
exit stack left the underlying memory streams closed once initialize() returned,
so the very next request (refresh -> list_tools) failed with Connection closed.

New design:
- Attach on the owner exit stack (same task as the serve loop, lexically intact)
- A cold-starting process makes initialize() fail; signal _ColdStartRetry up to
  the outer retry loop, which reuses the live process without consuming retry budget
- _lifecycle_loop_with_retry handles _ColdStartRetry like _TransportReconnect:
  preserves process, no fatal budget, backs off 2s and retries
- Two new unit tests: cold-start raises _ColdStartRetry (not fatal) when process
  is alive; raises fatal error when process has actually exited

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 14:58:40 +08:00
Hyu ed9343c686 fix(mcp): retry the stdio handshake during slow (npx) cold starts (#2306)
A node/npx stdio MCP server (e.g. firecrawl-mcp via npx -y) failed on first
connect with Connection closed / Failed after 4 attempts, even though the
process was fine. An npx cold start downloads+installs the package before the
server can answer the MCP handshake (measured ~27s for a simple official
server; longer for heavier ones). The old code attached the WS and called
session.initialize() the instant the process was started, so the handshake ran
before the process could answer and failed; the outer lifecycle retry then
rebuilt the process, churning it in a loop.

Verified decisively: attaching + initialize() against a mid-cold-start process
times out on attempt 1 (process still installing) but SUCCEEDS at t+0.6s on
attempt 2 once the process is ready. So the fix is to retry the handshake in
place, not to rebuild the process.

Changes (mcp_stdio.initialize):
- Start the managed process ONCE, then loop attach WS -> ClientSession ->
  initialize() within the startup_timeout budget, tearing down each failed
  attempt cleanly, until the handshake succeeds or the budget elapses. A
  successful transport/session is transferred into the owner exit stack via a
  small _TransferredStack adapter.
- Bound each attempt with asyncio.wait_for(initialize, _HANDSHAKE_ATTEMPT_TIMEOUT_SEC=10s)
  so a cold-starting process fails fast and retries instead of hanging until
  the transport drops.
- Stop retrying ONLY when the process has DEFINITIVELY exited: new
  _managed_process_has_exited() (checks EXITED status) replaces the previous
  not-_managed_process_is_running() test, which false-negatived on a
  just-spawned process that had not yet reported RUNNING and made the loop bail
  to the outer rebuild path (relay then rejected the early re-attach with HTTP 400).

Adds a unit test that fails the first two handshakes with the process alive and
asserts the loop retries to success while starting the process exactly once.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 14:16:57 +08:00
Hyu 9a8cdde86c refactor(mcp): make MCP test reuse the shared Box session (near-instant tests) (#2304)
* refactor(mcp): make MCP test reuse the shared Box session instead of a per-test session

Testing an MCP server (config-page "test" button) previously spun up a fresh
isolated mcp-test-<uuid> Box session every time: cold-start the container, run
the dependency bootstrap, probe, then tear the whole session down. That is slow
(tens of seconds) and, on an already-hosted server, wasteful — the server is
already running in the shared session.

Change the test to reuse the shared session / live process:

- _build_box_session_id: transient tests now use mcp-shared, the same Box
  session as live servers, so a test reuses the running container (and, for an
  existing server, its live managed process) instead of a cold per-test session.
- cleanup_session: a transient test no longer deletes the whole session (which
  under the shared model would kill every other MCP server in the container). It
  stops only its own process_id, exactly like a live server. Isolation is now at
  the process level (distinct process_id per server/test), not the session level.
- test_mcp_server (persisted server): reuse the live connection with a real
  list_tools refresh/probe; only fall back to a full start() when there is no
  live connection to probe or the refresh fails, instead of an ERROR->start()
  rebuild.

Trade-off: a failing test now shares the container with live servers rather than
a throwaway session. Accepted deliberately in favour of near-instant tests;
process-level isolation keeps a test from stopping another server's process.

* chore(deps): pin langbot-plugin 0.4.9 for the nsjail RLIMIT_AS node/npx MCP fix

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 13:14:14 +08:00
Hyu a2655e16f8 fix(mcp): survive transient WS transport drops for Box stdio MCP servers (#2303)
* fix(mcp): survive transient WS transport drops for Box stdio MCP servers

A Box-backed stdio MCP server (e.g. pab1it0/prometheus) would periodically
error on the frontend with Box managed process exited unexpectedly /
Failed after 4 attempts once the session had been alive for a while.

Root cause: the managed MCP process lives in the Box runtime and SURVIVES a
WebSocket transport drop, but _lifecycle_loop treated any monitor completion
as a fatal process death. It then ran the finally-block cleanup — which STOPS
the still-healthy managed process — and did a full 4-attempt exponential
backoff rebuild. Under an occasionally-stalled single-worker event loop the
mcp websocket client misses a ping/pong, the transport drops, and this
self-inflicted teardown loop is what the user sees.

Fixes:
- _lifecycle_loop: when the health monitor completes, re-check the real
  managed-process state. If the process is still running, the transport
  merely dropped: raise an internal _TransportReconnect signal instead of
  Box managed process exited unexpectedly.
- _lifecycle_loop_with_retry: handle _TransportReconnect as a free, uncounted
  reconnect (does not consume the fatal retry budget), so a long-lived session
  survives arbitrarily many transient drops.
- finally-block: gate managed-process teardown on a _preserve_managed_process
  flag so a transport-only reconnect closes just the WS, not the process.
- BoxStdioSessionRuntime.initialize: reuse an already-running managed process
  instead of stopping+rebuilding it (which also re-ran the slow dependency
  bootstrap); only (re)start when none is running. Adds
  _managed_process_is_running() helper.

Pairs with langbot-plugin-sdk fix adding a server-driven WS heartbeat to the
managed-process relay, which prevents most drops in the first place.

* style(mcp): ruff format

* chore(deps): pin langbot-plugin 0.4.8 for the managed-process WS heartbeat fix

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 12:24:24 +08:00
Hyu f89611f39a chore: release v4.10.5 2026-07-03 00:59:53 +08:00
RockChinQ 04a199a3b2 fix: preserve MCP server ids with slashes
Fixes #2301
2026-07-03 00:17:29 +08:00
Hyu f85278f98b Fix monitoring CI regressions 2026-07-02 17:22:33 +08:00
Hyu 0ce382fc8b Improve pipeline monitoring and AI tab resilience 2026-07-02 16:53:50 +08:00
Hyu e32e515bc9 Tone down tool call bubbles 2026-07-02 16:53:50 +08:00
Hyu c809e3d14f Add tool call observability 2026-07-02 16:53:50 +08:00
Hyu b1bc05d5d3 Improve monitoring conversation turns 2026-07-02 16:53:50 +08:00
Stevenqin 13dba887d5 fix(wecomcs): implement proactive text sends (#2300) 2026-07-02 14:39:05 +08:00
彼方 cc7a13158e fix(aiocqhttp): resolve group member metadata (#2298)
* fix(aiocqhttp): resolve group member metadata

* style: ruff format
2026-07-01 20:00:47 +08:00
RockChinQ 7de7c7f714 docs: update Product Hunt badge to featured 2026-06-30 08:23:55 -04:00
Junyan Qin 85923e3d7b fix(migrations): shorten mcp resource revision id 2026-06-30 19:39:23 +08:00
Junyan Qin c7c6c5dc51 fix(i18n): add skill tools tooltip translations 2026-06-30 19:37:57 +08:00
Junyan Qin b056646696 fix(migrations): add mcp resource preferences 2026-06-30 19:36:04 +08:00
advancer-young 096ec1a8ce feat(mcp): support mcp resources (#2215)
* feat(mcp): support mcp resources

* feat(web): split MCP resources into tab

* docs: add MCP resources PR review

* feat(mcp): productionize resource support

* feat(mcp): scope local agent tools and resources

* fix(web): gate space embedding models behind login

* fix(web): prevent clipped space model CTA

* test: update preproc resource tool expectations

* fix(web): expose skill authoring tools in selector

---------

Co-authored-by: yang.xiang <yang.xiang@advancegroup.com>
Co-authored-by: Hyu <chenhyu@proton.me>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-06-30 19:16:30 +08:00
RockChinQ 2618e06492 docs: update 302.AI referral link to share.302ai.cn 2026-06-30 05:10:17 -04:00
彼方 5c8c0eb17b test: use package import paths (#2297) 2026-06-30 10:51:34 +08:00
彼方 ccc51522cf fix(aiocqhttp): normalize base64 media payloads (#2296)
* fix(aiocqhttp): normalize image and voice base64 payloads

* fix(aiocqhttp): support file messages from base64 payloads

* test(aiocqhttp): use package import path
2026-06-30 10:51:09 +08:00
彼方 6a99a83f2d chore(scripts): mark scripts executable (#2295) 2026-06-30 10:50:26 +08:00
Hyu ebab5343cf fix(i18n): add bots.admins keys to all locale files (#2292)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-06-27 17:38:30 +08:00
Hyu b97cc800d3 feat(platform): migrate bot admins from config.yaml to database (#2289)
* feat(platform): migrate bot admins from config.yaml to database

- Add BotAdmin ORM model (bot_admins table) scoped per bot_uuid
- Add Alembic migration 0007 to create table and migrate legacy config admins
- Remove top-level admins key from config.yaml template
- Add GET/POST/DELETE /api/v1/platform/bots/<uuid>/admins endpoints
- Update cmdmgr privilege check to query bot_admins table (bot-scoped)
- Add BotAdminsPanel frontend component in bot detail sessions tab
- Add i18n keys (zh-Hans, en-US)

* fix(ci): ruff/prettier format, fix test_importutil assertion

* fix(ci): prettier format BackendClient.ts and i18n locales

* fix(ci): eslint-prettier fix BackendClient.ts single-param formatting

* refactor: move admin management into session monitor, fix session_id enum format

- Remove standalone admins tab, replace with BotAdminsDialog in session monitor
- Add admin toggle button inline in chat header next to Active status
- Add BotAdminsDialog component with useBotAdmins hook
- Fix session_id written as LauncherTypes.PERSON_xxx instead of person_xxx
  (monitoring_helper.py x4, pipelinemgr.py x1 missing .value on launcher_type)
- Fix duplicate platform/person label in session list and chat header
- Migrate existing malformed session_id records in DB

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-06-27 17:18:19 +08:00
彼方 48905ea080 feat(plugin): report deferred response delivery failures (#2287)
* feat(plugin): report deferred response delivery failures

* style: fix ruff format issues in plugin_diagnostics and test_handler_actions

---------

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-06-26 23:45:10 +08:00
Hyu ddb77fc43c fix(api): guard /set-password with allow_modify_login_info (#2288)
The /change-password and /bind-space endpoints already refuse when
system.allow_modify_login_info is false, but /set-password did not,
leaving a path to alter login credentials on locked-down deployments
(e.g. public demo instances). Apply the same guard.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-06-26 16:35:50 +08:00
huanghuoguoguo 5b2826fa49 Add performance and reliability QA gates (#2283)
* Add performance and reliability QA gates

* test(skills): prepare user path performance gate

* test(skills): add debug chat load gate

* test(skills): extend fake provider load profiles

* test(skills): add debug chat timing and isolation probes

* test(skills): clarify manual QA perf gates
2026-06-25 21:02:44 +08:00
Hyu 20636ac432 Merge pull request #2284 from langbot-app/fix/api-password-thread-offload
fix(api): offload password hashing from event loop
2026-06-25 20:31:44 +08:00
Hyu af42602547 Merge pull request #2285 from langbot-app/fix/monitoring-null-payloads
fix(monitoring): tolerate null API payloads
2026-06-25 20:26:25 +08:00
dadachann 53b20e2b13 fix(monitoring): tolerate null API payloads
Normalize monitoring API responses before rendering so empty or error payloads with data:null cannot crash the dashboard. Also guard chart, token, and box session arrays before reading length/map.
2026-06-25 08:22:01 -04:00
dadachann 1242dc2d21 fix(api): offload password hashing from event loop 2026-06-25 06:29:16 -04:00
RockChinQ 04628d93cb docs: add architecture guide for agents 2026-06-25 04:17:19 -04:00
RockChinQ 9c22a1521c fix(box): defer separated workspace ownership to runtime 2026-06-25 00:09:40 -04:00
RockChinQ c8d5039580 feat(box): expose Docker CPU limit toggle 2026-06-24 23:31:53 -04:00
dadachann 85d8d9304e fix(web): keep feedback dialog interactive 2026-06-24 10:10:19 -04:00
Hyu 76471af179 feat(web): add sidebar feedback popover
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-06-24 16:43:50 +08:00
RockChinQ 59b2a7cd51 fix(monitoring): hide disabled box status on cloud 2026-06-23 06:40:05 -04:00
RockChinQ a43978ff24 chore(release): bump version to 4.10.4 2026-06-22 21:15:53 -04:00
RockChinQ e3417dd20b fix(release): derive package version from metadata 2026-06-22 21:10:33 -04:00
RockChinQ 2982e7c553 chore(release): bump version to 4.10.3 2026-06-22 11:12:15 -04:00
RockChinQ e1e14e9269 chore(deps): bump langbot-plugin to 0.4.6 2026-06-22 11:08:08 -04:00
Junyan Chin 1c128a1524 docs(skills/langbot-plugin-dev): document marketplace README i18n convention (root README.md must be English; other langs in readme/) 2026-06-22 02:39:15 -04:00
RockChinQ 8ad1203fd5 docs(examples): add web-page-bot embed demo, drop stray test-embed.html
Move the Page Bot (web_page_bot) embed test page out of the repo root into
examples/web-page-bot/ as a proper, LangBot-styled demo: a self-contained
index.html that loads the live widget.js against a running instance, plus
bilingual READMEs mirroring examples/http-bot/.
2026-06-22 02:17:26 -04:00
Junyan Chin 144bec371c feat(platform): standalone HTTP Bot adapter (server-to-server) (#2274)
* docs(platform): add HTTP Bot adapter design (RFC)

Standalone server-to-server HTTP adapter for driving a pipeline from external
systems (LangBot Space ticketing et al). Inbound via the existing unified
webhook route; outbound via signed callback POSTs. Preserves pipeline-native
N->1 aggregation and 1->M multi-reply without a long-lived WebSocket.

No core changes required (router/aggregator/pipeline untouched).

* feat(platform): add standalone HTTP Bot adapter

A first-class, vendor-neutral message-platform adapter (http_bot) for
server-to-server integrations (LangBot Space ticketing et al). Drives a
pipeline over plain HTTP with no long-lived connection:

- Inbound: signed POST to the existing unified webhook route /bots/<uuid>,
  carrying a caller-defined session_id mapped to the LangBot launcher id via
  get_launcher_id -> per-session isolation. Preserves pipeline-native N->1
  aggregation for free.
- Outbound: each reply_message / reply_message_chunk becomes one signed
  callback POST to the config-only callback_url, delivered in per-session
  sequence order with retry/backoff -> 1->M multi-reply.
- Sub-paths: /reset (drop a session) and /sync (block for the collapsed reply).
- Auth: symmetric HMAC-SHA256 both directions (timestamp + replay window),
  no JWT/Turnstile, no socket.

Decisions: callback URL is config-only (SSRF closed); reset + sync shipped;
Python + TS reference clients shipped (signing verified byte-identical 3-way).

No core changes: the unified webhook router, aggregator, query pool and
pipeline are untouched. Adapter is auto-discovered from platform/sources/.

Adds:
  src/langbot/pkg/platform/sources/http_bot.{py,yaml,svg}
  src/langbot/pkg/platform/sources/http_bot_signing.py
  docs/platforms/http-bot.md, docs/http-bot-openapi.json
  examples/http-bot/{client.py,client.ts,README.md}
Updates docs/HTTP_BOT_ADAPTER_DESIGN.md (status: implemented).

* docs(examples): add interactive HTTP Bot playground (browser debug console)

A single-file aiohttp web app (examples/http-bot/playground.py) that lets you
chat with a RUNNING http_bot bot from the browser and watch the protocol live:
signed inbound POST -> 202 ack -> 1->M signed callbacks streamed back via SSE,
with a debug panel showing the signature, HTTP status, and per-callback
sequence/verification. Light LangBot-styled UI.

On startup it reads the API key + http_bot bot from data/langbot.db and points
the bot's callback_url + secrets back at itself via the LangBot API (live
reload, no restart). README updated with a playground section.

* docs(examples): add Chinese README for http-bot reference clients

* style(platform): use </> code icon for http_bot adapter logo

* docs(examples): point http-bot guide links to docs.langbot.app

* style(platform): make http_bot icon a transparent monochrome </> so WebUI tints it like other adapters

* Revert to colorful </> badge for http_bot icon (WebUI renders it as-is)
2026-06-22 13:38:00 +08:00
RockChinQ 74a18191dd docs(readme): default docker compose command starts the sandbox
The plain `docker compose up -d` leaves the Box sandbox runtime off
(it's gated behind the box/all profile), so sandbox tools, skill
add/edit and stdio MCP don't work out of the box. Use
`docker compose --profile all up -d` across all 9 README translations so
the default quick-start brings up the sandbox-capable stack.
2026-06-21 13:18:44 -04:00
RockChinQ a15c98eb06 fix(web): point plugin help links to working docs URL
The in-product plugin/add-extension help links went through
link.langbot.app/{lang}/docs/plugins, which now 404s (it resolved to the
removed /usage/plugin/plugin-intro path). Point them directly at the
current docs page docs.langbot.app/{lang}/plugin/plugin-intro (verified
200 for zh/en/ja).
2026-06-21 12:59:13 -04:00
RockChinQ cbe17cde6c fix(web): provider card overflow on mobile via grid/flex min-width floor
The previous truncate/shrink-0 pass only touched leaf nodes, but the
min-content floor was set by two ancestors: the flex-1 left group lacked
min-w-0, and CardHeader is a CSS grid whose implicit single column
defaults to min-content. Constrain both (min-w-0 on the header grid +
explicit grid-cols-[minmax(0,1fr)], min-w-0 on the inner flex groups) so
the provider name / base_url+key subtitle actually truncate instead of
forcing the card — and the whole settings modal — wider than the viewport.
2026-06-21 12:54:24 -04:00
RockChinQ 876e8bf804 fix(web): mobile overflow in settings panels
- PanelToolbar: allow wrapping and tighten padding on small screens so the
  primary action (e.g. "创建 API 密钥") no longer runs off the dialog edge.
- ProviderCard header: let the provider name truncate and pin the model-count
  badge and right-side action group with shrink-0 so credits / + controls stay
  inside the card on narrow viewports.
2026-06-21 12:48:18 -04:00
RockChinQ b3848c9d05 feat(web): make tooltips tap-toggleable on touch devices
Radix tooltips open on hover/focus only and stay closed on touch input,
so on mobile every hover tooltip was unreachable. Detect coarse/no-hover
pointers via matchMedia and drive the tooltip's open state ourselves so a
tap on the trigger toggles it. Desktop hover/focus behaviour is unchanged
(we only intercept the tap when the device has no hover capability). Fixes
all tooltips app-wide from the shared primitive.
2026-06-21 12:46:18 -04:00
RockChinQ 85743cc75f fix(tests): make Postgres migration head test revision-agnostic
The PostgreSQL migration test had the same hardcoded 0005 head
assertion as the SQLite one; resolve the actual head from the Alembic
ScriptDirectory so 0006 (and future migrations) don't break it.
2026-06-21 12:10:20 -04:00
RockChinQ c689b10c0d fix(mcp): ruff format remote-mode files; make migration head test revision-agnostic
CI follow-up to the local/remote MCP work:

- Apply ruff format to provider/tools/loaders/mcp.py and the 0006
  normalize-remote-mode migration (Lint job failed on formatting).
- test_migrations.py hardcoded the head revision as 0005_*, which broke
  once 0006 landed. Resolve the actual head from the Alembic
  ScriptDirectory so future migrations don't require editing the test.
2026-06-21 12:04:37 -04:00
RockChinQ 812b1fff4c fix(web): stop spurious page refresh on account menu open; plugin log auto-refresh as switch
Two unrelated frontend fixes:

- LanguageSelector mounts each time the sidebar account dropdown opens and
  unconditionally called i18n.changeLanguage() on mount, emitting a
  languageChanged event even when the language was unchanged. That handed
  every useTranslation() consumer a fresh `t` reference, re-running effects
  keyed on `t` (e.g. the plugins page system-status fetch) and surfacing as
  a page "refresh". Guard the call so it only fires on an actual change.

- Plugin logs auto-refresh control changed from a toggle Button to a
  Switch + Label; the on/off button i18n keys are replaced by a single
  static logsAutoRefresh label across all 8 locales.
2026-06-21 11:58:01 -04:00
RockChinQ 9daf22d661 feat(plugin-market): align recommendation carousel with Space (pause + countdown ring)
Port the Space marketplace recommendation carousel UX into the in-app
add-extension page: a 10s auto-advance driven by a smooth countdown ring
that doubles as a pause/resume toggle, and manual prev/next now reset the
countdown. Adds market.recommendation.{pause,resume} across 8 locales.
2026-06-21 11:48:39 -04:00
RockChinQ 42a2c70b14 style(plugin-market): widen marketplace cards via auto-fill min width
Replace fixed grid-cols breakpoints (which forced up to 4 narrow cards on
wide screens) with auto-fill columns and a 24rem minimum card width on
both the main market grid and the featured recommendation rows. The
featured rows already measure real column count via ResizeObserver, so
pagination adapts automatically.
2026-06-21 11:21:52 -04:00
RockChinQ 64ed6d994b feat(mcp): simplify external MCP server config to local/remote modes
Replace the three-way transport choice (stdio / sse / httpstream) for
connecting LangBot to external MCP servers with two modes: local (stdio)
and remote. Remote servers only require a URL; the runtime auto-detects
the transport (tries Streamable HTTP, falls back to SSE).

- provider/tools/loaders/mcp.py: add _init_remote_server() with
  Streamable-HTTP-then-SSE probing; dispatch 'remote' lifecycle, keep
  legacy sse/http branches for back-compat
- plugin/connector.py: normalize legacy http/sse marketplace modes to
  'remote' on Space install, preserving connection params
- entity/persistence/mcp.py: document mode as stdio, remote (legacy: sse, http)
- alembic 0006: idempotent data migration mapping existing sse/http rows
  to remote (downgrade maps back to http)
- api/http/service/mcp.py: stash runtime_info (status + tool list) into
  test task metadata before tearing down the temp session
- web: collapse mode dropdown to local/remote, remote renders URL+timeout
  only, edit auto-maps legacy sse/http to remote; show tools after test in
  create mode from task metadata; remove dead plugins/mcp-server/ tree
- i18n: local/remote labels + mode/url hints across 8 locales
2026-06-21 11:20:32 -04:00
RockChinQ 2ff854f79a build(Dockerfile): install Node.js LTS so sandbox can run npx-based stdio MCP servers
The final runtime image (used by langbot/plugin_runtime/box) shipped uv and
docker-cli but no node, so any npx-launched stdio MCP server inside the box
sandbox exited with return_code=127 (command not found). Install Node.js 22
LTS via NodeSource; node/npx land in /usr/bin, which is on the nsjail
read-only mount whitelist (_READONLY_SYSTEM_MOUNTS) and is bound into the
sandbox chroot automatically.
2026-06-21 08:15:02 -04:00
RockChinQ 52c096ea4c chore(deps): patch Dependabot vulns (Python + JS)
Python (pyproject.toml + uv.lock):
- aiohttp 3.14.0 -> 3.14.1 (8 alerts: medium+low)
- cryptography -> 49.0.0 (high, floor 48.0.1)
- langchain -> 1.3.10 (medium, floor 1.3.9)
- langsmith -> 0.8.18 (high)
- starlette 1.2.1 -> 1.3.1 (high+low, transitive)
- pydantic-settings 2.12.0 -> 2.14.2 (medium, transitive)
- torch 2.10.0 -> 2.12.1 (low, transitive; py>=3.14 only)

JS (web/, dual lockfile npm+pnpm in sync):
- vite ^8.0.5 -> ^8.0.16 (high+medium)
- js-yaml -> 4.2.0 (medium, override >=4.2.0 <5)
- form-data -> 4.0.6 (high, override)

Unfixable (no upstream patch, left + reported):
- chromadb critical <=1.5.9 (1.5.9 is latest)
- PyPDF2 medium (deprecated; needs pypdf migration)

Verified: uv sync + import check, pnpm frozen-lockfile, vite build.
2026-06-21 07:43:54 -04:00
Junyan Chin eda80030b5 Improve README_CN.md with clearer Star instructions
Update instructions for starring and watching the repository.
2026-06-21 19:34:34 +08:00
RockChinQ dfbd176e42 docs(readme): move Star & Watch CTA after Key Capabilities, host gif on langbot.app 2026-06-21 07:33:00 -04:00
RockChinQ 6ddd24ae68 docs(readme): restore Star & Watch CTA with star.gif across all locales 2026-06-21 07:25:46 -04:00
RockChinQ a2cdbb621b Merge branch 'fix/mobile-responsive-pages': mobile responsive fixes for dashboard, plugin detail & models dialog 2026-06-20 10:58:43 -04:00
RockChinQ b92d54254b fix(web): improve mobile responsiveness of dashboard, plugin detail & models dialog
- monitoring: stack filters full-width, scrollable tab bar, reduce card/content padding on mobile
- models dialog: provider form modal no longer overflows viewport on small screens; shared panel body padding shrinks on mobile
- plugin logs: reduce horizontal padding on mobile
2026-06-20 10:42:35 -04:00
huanghuoguoguo d22fa82d7c fix(skills): bootstrap generated lbs wrapper (#2270) 2026-06-20 17:24:04 +08:00
Junyan Chin e9dd584792 feat: MCP server + in-repo skills (agent-friendly platform) (#2269)
* feat(api): support global API key from config.yaml (api.global_api_key)

Accept a config-defined global API key anywhere a web-UI key is accepted
(X-API-Key / Bearer), with no login session and no DB record. Useful for
automated deployments and AI agents (HTTP API + MCP). Defaults to empty
(disabled); does not require the lbk_ prefix.

- templates/config.yaml: add api.global_api_key with security notes
- service/apikey.py: verify_api_key checks global key first (constant-time)
- docs/API_KEY_AUTH.md: document the global key + security guidance
- tests: cover global-key match, prefix-free, fallback-to-db, disabled

* feat(mcp): expose LangBot management as an MCP server at /mcp

Add an MCP (Model Context Protocol) server so external AI agents can manage a
LangBot instance. Reuses the same API-key auth as the HTTP API (including the
config.yaml global API key).

- pkg/api/mcp/server.py: FastMCP server wrapping the service layer; 21 curated
  tools across system/bots/pipelines/models/knowledge/mcp-servers/skills
- pkg/api/mcp/mount.py: ASGI dispatcher fronting Quart; authenticates /mcp
  requests with an API key, runs the streamable-HTTP session manager lifespan
- controller/main.py: serve the wrapped ASGI app via hypercorn (was run_task)
- web: new 'MCP' tab in the API integration dialog showing endpoint, auth, and
  client config; i18n for 8 locales
- tests/manual/mcp_smoke.py: e2e check (401 unauth, list tools, call tools)

Tool surface is intentionally curated (not all ~25 route groups) to keep the
agent surface small, safe, and maintainable. Extend deliberately.

* feat(skills): add in-repo skills/ as the single source of truth

Migrate the agent skills + QA/e2e test harness from the (now archived)
langbot-app/langbot-skills repo into LangBot/skills/, and add four new skills.

Migrated:
- langbot-plugin-dev, langbot-testing (e2e), langbot-env-setup,
  langbot-skills-maintenance, langbot-eba-adapter-dev
- the bin/lbs CLI (src/, test/, scripts/, schemas/, qa-agent-docs/)

New:
- langbot-dev      core backend + web development
- langbot-deploy   Docker/K8s deployment + config.yaml + global API key
- langbot-mcp-ops  operating the LangBot MCP server (/mcp)
- langbot-space-ops operating the Space marketplace MCP server

- src/cli.ts repoRoot(): recognize the skills assets root (skills.index.json +
  bin/lbs) so the CLI works when nested inside the LangBot repo
- README.md: unified skill catalog; skills.index.json regenerated

Parity with source verified: bin/lbs validate + node test suite match the
source repo (only the uncommitted .lbpkg build-artifact fixture differs).

* docs(agents): document agent-facing surfaces + API/MCP/skills sync rule

* docs(readme): add 'Built for AI Agents' section across all locales

Highlight MCP server, in-repo skills (single source of truth), AGENTS.md
sync rule, and llms.txt. Cross-link LangBot Space MCP marketplace.

* style(mcp): fix ruff format + prettier lint in MCP server and API panel

* style(web): prettier format MCP i18n locale entries

* docs(skills): note MCP instance control in dev/testing skills

All development-guidance skills now point to the LangBot instance MCP
server (/mcp) and the Space marketplace MCP server, reusing API keys.
2026-06-20 15:14:47 +08:00
RockChinQ 91906d73be docs(readme): add web panel dashboard screenshot to all READMEs
Place the populated management-dashboard screenshot (already used on the
docs homepage) near the top of every localized README — right after the
opening "What is LangBot?" paragraph and before the Key Capabilities
list. The image ships in res/ so it resolves on GitHub, PyPI and mirrors
without hotlinking the docs site. Alt text is localized per language and
carries product + feature keywords for SEO.

Covers: en, zh-CN, zh-TW, ja, es, fr, ko, ru, vi.
2026-06-19 23:20:29 -04:00
huanghuoguoguo acfac42107 fix(litellmchat): preserve provider_specific_fields for Gemini thought_signature (#2265)
Update _normalize_stream_tool_calls to preserve provider_specific_fields
(including thought_signature) from streaming tool call chunks. Also preserve
provider_specific_fields from delta in invoke_llm_stream.

This ensures Gemini's thought_signature is round-tripped correctly:
1. LiteLLM extracts thought_signature from Gemini response
2. It's preserved in Message/ToolCall entities (via SDK changes)
3. _convert_messages includes it in the next request

Also add unit tests for provider_specific_fields round-tripping.

Fixes: langbot-app/LangBot#1899
2026-06-19 23:26:12 +08:00
huanghuoguoguo 492827ea75 Add plugin rerank invocation action (#2242) 2026-06-19 23:25:54 +08:00
701 changed files with 123748 additions and 14606 deletions
+4
View File
@@ -37,6 +37,10 @@ jobs:
working-directory: web
run: pnpm install --frozen-lockfile
- name: Run frontend unit tests
working-directory: web
run: pnpm test:unit
- name: Install Playwright browsers
working-directory: web
run: pnpm exec playwright install --with-deps chromium
+10 -2
View File
@@ -44,7 +44,9 @@ jobs:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
# Release migration 0013 installs the pgvector extension in the shared
# business database; CI must exercise the same extension availability.
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: langbot
POSTGRES_PASSWORD: langbot
@@ -75,4 +77,10 @@ jobs:
- name: Run PostgreSQL migration tests
env:
TEST_POSTGRES_URL: postgresql+asyncpg://langbot:langbot@localhost:5432/langbot_test
run: uv run pytest tests/integration/persistence/test_migrations_postgres.py -q --tb=short
run: >-
uv run pytest
tests/integration/persistence/test_migrations_postgres.py
tests/integration/persistence/test_pgvector_postgres.py
tests/integration/persistence/test_release_migration_postgres.py
tests/integration/persistence/test_plugin_identity_migration.py
-q --tb=short
+1
View File
@@ -48,6 +48,7 @@ coverage.xml
.coverage
src/langbot/web/
testsdk/
.qa/
# Build artifacts
/dist
+83 -117
View File
@@ -1,139 +1,105 @@
# AGENTS.md
This file guides code agents (Claude Code, GitHub Copilot, OpenAI Codex, etc.) working in the LangBot project. `CLAUDE.md` is a symlink to this file.
This file guides code agents working in the LangBot main repository. `CLAUDE.md` is a symlink to this file.
## Project Overview
Read `ARCHITECTURE.md` before non-trivial backend, frontend, runtime, plugin, Box, MCP, persistence, or cross-repo SDK changes. This file is the working checklist; `ARCHITECTURE.md` is the system map.
LangBot is an open-source, LLM-native instant-messaging bot development platform. It aims to provide an out-of-the-box IM bot development experience with Agent, RAG, MCP and other LLM application capabilities, supporting mainstream global IM platforms and exposing rich APIs for custom development.
## Quick Facts
LangBot has a comprehensive web frontend — almost every operation can be performed through it.
- Python backend: `>=3.11,<4.0`, dependencies managed by `uv`.
- Frontend: `web/` is Vite + React Router 7 + shadcn/ui + Tailwind, managed by `pnpm`.
- Backend framework: Quart served by Hypercorn on `api.port`, default `5300`.
- Frontend dev server: `web/` on `3000`, with `VITE_API_BASE_URL` pointing at the backend.
- Plugin/Box/runtime contracts live in sibling repo `langbot-plugin-sdk`, pinned as `langbot-plugin` in `pyproject.toml`.
- **Python**: `>=3.11,<4.0`, dependencies managed by `uv`. Package version is in `pyproject.toml`.
- **Frontend**: `web/` is a **Vite + React Router 7 + shadcn/ui + Tailwind CSS** SPA, managed by `pnpm`. (Note: this is NOT Next.js — the `dev` script is `vite`.)
- **Backend framework**: Quart (the async flavour of Flask). The HTTP API and the pre-built web UI are both served by the backend on `http://127.0.0.1:5300`.
## Repository Layout
```
LangBot/
├── main.py # Entrypoint shim -> langbot.__main__.main()
├── pyproject.toml # Python project + deps (uv), pins langbot-plugin==<x.y.z>
├── src/langbot/
│ ├── __main__.py # Real entrypoint, CLI args (--standalone-runtime, --standalone-box, --debug)
│ ├── pkg/ # Core backend package
│ │ ├── api/ # HTTP API controllers + services (Quart)
│ │ ├── core/ # App bootstrap, stages, task manager
│ │ ├── platform/ # IM platform adapters, bot managers, session managers
│ │ ├── provider/ # LLM providers, requesters, tool providers
│ │ ├── pipeline/ # Pipelines, stages, query pool
│ │ ├── plugin/ # Bridge connecting LangBot to the plugin runtime (see below)
│ │ ├── box/ # Code-sandbox subsystem (Docker / nsjail / E2B backends)
│ │ ├── skill/ # Skill subsystem
│ │ ├── rag/ , vector/ # RAG + vector store
│ │ ├── command/ # Built-in commands
│ │ ├── persistence/ # ORM models + Alembic migrations (SQLite & PostgreSQL)
│ │ ├── storage/ # Object/file storage abstractions
│ │ ├── config/, entity/, discover/, utils/, telemetry/, survey/
│ ├── libs/ # Vendored SDKs (qq_official_api, wecom_api, etc.)
│ └── templates/ # Config/component templates (e.g. templates/config.yaml)
├── web/ # Frontend SPA (Vite + React Router 7 + shadcn + Tailwind)
└── docker/ # docker-compose deployment files
```
## Development Environment Setup
Full guide lives in the wiki: **["开发配置" / Dev Config](https://docs.langbot.app/zh/develop/dev-config)**. Summary:
### Backend
```bash
pip install uv
uv sync --dev # uv creates a .venv/ for you; point your editor's interpreter at it
uv run main.py # serves API + web UI on http://127.0.0.1:5300
```
On first run the config file is generated at `data/config.yaml`. DB is SQLite by default (zero setup); PostgreSQL is supported. Migrations run automatically on startup.
### Frontend
Requires Node.js + [pnpm](https://pnpm.io/installation).
```bash
cd web
cp .env.example .env # Windows: copy .env.example .env
pnpm install
pnpm dev # http://127.0.0.1:3000 (npm install / npm run dev also work)
```
`pnpm dev` reads `VITE_API_BASE_URL` from `web/.env` so the dev frontend can reach the backend on port `5300`. In production the frontend is pre-built into static files served by the backend on the same origin.
### Code formatting
The repo runs lint + format checks in CI. Install the pre-commit hooks so the same checks run locally before each commit:
## Essential Commands
```bash
uv sync --dev
uv run main.py
uv run pre-commit install
cd web
pnpm install
pnpm dev
pnpm build
```
## Plugin System
LangBot's plugin system (Plugin SDK, CLI `lbp`, Plugin Runtime, and the shared entity/API definitions) lives in a **separate repository**: [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk). LangBot depends on it via the pinned `langbot-plugin` package in `pyproject.toml`.
### Architecture (what to know inside this repo)
- Plugins run as independent processes managed by the **Plugin Runtime**. The Runtime supports two control transports: `stdio` and `websocket`.
- When LangBot is started directly by a user (not in a container), it spawns and connects to the Runtime over **stdio** (lightweight/personal use).
- When LangBot runs in a container, it connects to a standalone Runtime over **WebSocket** (production).
- The bridge code lives in `src/langbot/pkg/plugin/` (`connector.py`, `handler.py`).
- Relevant config (`data/config.yaml`): `plugin.runtime_ws_url` (e.g. `ws://langbot_plugin_runtime:5400/control/ws`). Start LangBot with `--standalone-runtime` to make it connect to an externally-launched Runtime over WebSocket instead of spawning one over stdio.
### Debugging the Plugin Runtime / CLI / SDK
This is documented in detail in the **SDK repo's `AGENTS.md`** and in the wiki page **["调试插件运行时、CLI、SDK" / Plugin Runtime](https://docs.langbot.app/zh/develop/plugin-runtime)**. The short version:
- Clone `LangBot` and `langbot-plugin-sdk` as siblings under one parent dir so the editor resolves shared entities.
- Start a standalone Runtime from the SDK repo: `uv run --no-sync lbp rt` (control port `5400`, debug port `5401`).
- To make LangBot use a locally-modified SDK: from the SDK dir, with LangBot's `.venv` active, run `uv pip install .`, then launch LangBot with `uv run --no-sync main.py --standalone-runtime` (keep `--no-sync` so your local SDK isn't overwritten).
### Debugging the Box (sandbox) runtime
The Box subsystem (`src/langbot/pkg/box/`) is the code sandbox. It picks the first available backend among **Docker / nsjail / E2B**. The standalone Box runtime is launched via the SDK CLI: `lbp box`. Backend selection details, the `lbp box` flags, and the SDK-side architecture are documented in the SDK repo's `AGENTS.md`.
Relevant config (`data/config.yaml`, `box:` section): `box.enabled` (master switch — disabling it also disables the native sandbox tools, skill add/edit, and stdio-mode MCP servers), `box.backend` (`'local'` = Docker/nsjail auto-pick, or `'docker'` / `'nsjail'` / `'e2b'`; also settable via `BOX__BACKEND`), and `box.runtime.endpoint` (external Box runtime base URL, e.g. `ws://127.0.0.1:5410`; empty = local auto-managed runtime). Like the plugin runtime, LangBot can connect to an externally-launched Box runtime by setting that endpoint and starting with `--standalone-box`.
> A common false "No supported sandbox backend (Docker / nsjail / E2B) is available" comes from Docker being installed and running but the current user not being in the `docker` group → `docker info` gets `permission denied` on the socket. Fix: `sudo usermod -aG docker <user>` and restart the backend in a shell that has the new group.
## Development Standards
- LangBot is a global project: **all code comments and docstrings must be in English**, and every user-facing string must support **i18n** (`en_US` + `zh_Hans` at minimum, plus `ja_JP` where the repo already has it).
- LangBot is adopted in both toC and toB scenarios — always consider compatibility and security.
- **Commit message format**: `<type>(<scope>): <subject>`
- `type`: one of `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`, etc.
- `scope`: the affected package/module/file/class.
- `subject`: concise description of the change.
### Database migrations (Alembic)
LangBot uses [Alembic](https://alembic.sqlalchemy.org/) for migrations, supporting both SQLite and PostgreSQL from a single set of scripts. Migration files live in `src/langbot/pkg/persistence/alembic/versions/`.
If you change ORM model definitions, generate a migration:
Useful focused tests:
```bash
# Run from the project root (requires data/config.yaml to exist)
uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description of your change"
uv run pytest tests/unit_tests -q
uv run pytest tests/integration -q
uv run pytest tests/integration/persistence -q
uv run pytest tests/manual/mcp_smoke.py
cd web
pnpm lint
pnpm test:e2e
```
Review and edit the generated script before committing. Migrations execute automatically on startup. `autogenerate` detects schema changes (add/drop columns, tables, type changes) but **data migrations** (e.g. mutating JSON field contents) must be hand-written into the generated script. `env.py` sets `render_as_batch=True`, so SQLite's ALTER TABLE limits are handled automatically — no need to branch per database. More in the wiki ["开发配置"](https://docs.langbot.app/zh/develop/dev-config#数据库迁移).
Run the narrowest useful test first, then broader checks when confidence is needed.
When writing a migration, follow these rules:
## Where to Look
- **Revision id ≤ 32 characters.** PostgreSQL stores `alembic_version.version_num` as `varchar(32)`; a longer id raises `StringDataRightTruncationError` at runtime. Prefer short, descriptive ids like `0005_add_llm_context_length`.
- **Guard every operation against missing tables/columns.** Fresh installs build the schema via `create_all()` and then stamp the Alembic baseline, so a migration may run against a table that already has the change — or, in tests, against an empty database. Check `inspector.get_table_names()` / `inspector.get_columns(...)` before `add_column` / `drop_column`, mirroring the existing migrations.
- **Keep a single linear head.** Chain `down_revision` to the current head; do not create branches. Run the migration tests after adding one: `uv run pytest tests/integration/persistence/ -q` (the PostgreSQL test needs a running PG via `TEST_POSTGRES_URL`).
- Architecture map: `ARCHITECTURE.md`.
- Dev environment guide: https://docs.langbot.app/zh/develop/dev-config.
- Plugin runtime / CLI / SDK debugging: https://docs.langbot.app/zh/develop/plugin-runtime.
- API-key auth: `docs/API_KEY_AUTH.md`.
- Box deep-dive notes: `docs/review/box-architecture.md` and related files.
- In-repo skills: `skills/` is the single source of truth for LangBot agent skills.
- SDK repo: `../langbot-plugin-sdk/` when changing shared entities, plugin APIs, action protocol, `lbp rt`, or `lbp box`.
> **Legacy migration system (deprecated — do not extend).** The old 3.x migration system under `src/langbot/pkg/persistence/migrations/` (`DBMigration` subclasses in `dbmXXX_*.py`, run from `pkg/persistence/mgr.py`) is **frozen**. Do **not** add new `dbmXXX_*.py` files. The chain is capped at `required_database_version = 25` (`pkg/utils/constants.py`); those files only exist to upgrade pre-existing 3.x databases up to the Alembic baseline and are kept read-only. All new schema changes go through Alembic.
## Cross-Repo SDK Work
## Some Principles
When changing SDK contracts used by LangBot:
```bash
# from langbot-plugin-sdk, with LangBot's .venv active
uv pip install .
# from LangBot, preserve the locally installed SDK
uv run --no-sync main.py
```
For standalone runtime debugging:
```bash
# in langbot-plugin-sdk
uv run --no-sync lbp rt
uv run --no-sync lbp box
# in LangBot
uv run --no-sync main.py --standalone-runtime
uv run --no-sync main.py --standalone-box
```
Config keys to verify in `data/config.yaml` / `src/langbot/templates/config.yaml`:
- Plugin runtime: `plugin.runtime_ws_url`, default Docker host `langbot_plugin_runtime:5400/control/ws`.
- Box runtime: `box.enabled`, `box.backend`, `box.runtime.endpoint`, Docker host `langbot_box:5410`.
- API/MCP auth: `api.global_api_key`.
## Change Rules
- HTTP API changes that should be agent-accessible must update the matching MCP tool in `src/langbot/pkg/api/mcp/server.py` and the relevant skill under `skills/` in the same pass.
- New schema changes use Alembic under `src/langbot/pkg/persistence/alembic/versions/`; do not add legacy `dbmXXX` migrations.
- New platform behavior belongs in platform adapters only for platform translation; pipeline/business logic belongs in `pkg/pipeline/` or services.
- User-facing strings must support i18n (`en_US`, `zh_Hans`; include `ja_JP` where the repo already does).
- Code comments and docstrings must be English.
- Keep compatibility and security in mind; LangBot is used in both self-hosted/community and toB deployments.
- Commit message format: `<type>(<scope>): <subject>`.
## Runtime Pitfalls
- Local stdio Plugin Runtime disconnects do not auto-reconnect; restart LangBot if that path breaks.
- Orphan runtime processes on `5400`/`5401` commonly break plugin debugging.
- Use `uv run --no-sync` after locally installing the SDK, or `uv` may restore the pinned package.
- A false Box “no backend” often means Docker is running but the current user lacks Docker socket permission.
- Do not confuse external MCP servers LangBot connects to (`pkg/provider/tools/loaders/mcp.py`) with LangBot's own `/mcp` server (`pkg/api/mcp/`).
- `CLAUDE.md` is a symlink to this file; edit `AGENTS.md`, not the symlink.
## Principles
- Keep it simple, stupid.
- Entities should not be multiplied unnecessarily.
+256
View File
@@ -0,0 +1,256 @@
# Architecture
This document is a map of LangBot's moving parts. It is intentionally more stable than a feature guide and more concrete than the README: when you need to change behavior, start here, then follow the file references into the code.
For agent-specific working rules, see `AGENTS.md`. For plugin-runtime and Box-runtime implementation details, also read the sibling SDK repo: [`langbot-plugin-sdk`](https://github.com/langbot-app/langbot-plugin-sdk).
## What LangBot Is
LangBot is an open-source platform for building production IM bots backed by LLMs, agents, RAG, plugins, MCP tools, and a web management panel.
At runtime, one LangBot process owns:
- a Quart/Hypercorn HTTP service and the built web UI on `:5300`;
- messaging-platform adapters such as Discord, Telegram, Slack, WeChat, QQ, WeCom, Lark, DingTalk, KOOK, LINE, Satori, Matrix, and HTTP/WebSocket bots;
- a pipeline engine that turns inbound platform messages into LLM/tool/plugin work and replies;
- persistence, storage, vector database, telemetry, monitoring, and configuration managers;
- bridges to the Plugin Runtime and Box Runtime provided by `langbot-plugin-sdk`;
- an MCP server at `/mcp` exposing a curated agent-facing subset of the service layer.
## Repository Boundary
LangBot is not a single-repo system.
- `LangBot/` is the main product: backend, web UI, platform adapters, pipeline engine, HTTP API, MCP server, RAG, persistence, skills integration, and the bridge code that talks to runtimes.
- `langbot-plugin-sdk/` is published as `langbot-plugin` and pinned in `LangBot/pyproject.toml`. It contains plugin developer APIs, shared entities, `lbp`, the Plugin Runtime (`lbp rt`), and the Box Runtime (`lbp box`).
- Plugins import SDK APIs from `langbot_plugin.*`; the LangBot main process imports the same package for shared entities and runtime protocols.
This split matters. If a change modifies SDK entities, component APIs, action protocols, `lbp rt`, or `lbp box`, verify the sibling SDK repo and install the local SDK into LangBot's virtualenv when testing cross-repo behavior.
## Startup Path
The process entrypoint is small and layered:
1. `main.py` delegates to `langbot.__main__.main()`.
2. `src/langbot/__main__.py` parses `--standalone-runtime`, `--standalone-box`, and `--debug`, checks dependencies, generates missing config/data files, and calls `pkg.core.boot.main()`.
3. `pkg/core/boot.py` executes startup stages in order: `LoadConfigStage`, `GenKeysStage`, `SetupLoggerStage`, `BuildAppStage`, `ShowNotesStage`.
4. `BuildAppStage` constructs the `Application` object by wiring managers, services, runtime connectors, and controllers.
5. `Application.run()` starts the platform manager, query controller, HTTP controller, telemetry/cleanup loops, and plugin initialization.
The central runtime object is `pkg/core/app.py::Application`. It is a service locator for long-lived managers. That is not elegant, but it is the current architectural center; most subsystems receive `ap: Application` and collaborate through it.
## Top-Level Layout
```text
LangBot/
├── main.py # Entrypoint shim
├── pyproject.toml # Python package, deps, pinned langbot-plugin
├── src/langbot/
│ ├── __main__.py # CLI entrypoint and boot handoff
│ ├── pkg/
│ │ ├── core/ # Application, boot stages, task manager
│ │ ├── api/ # HTTP API + MCP server mount
│ │ ├── platform/ # IM adapters and runtime bot manager
│ │ ├── pipeline/ # Message routing and pipeline stages
│ │ ├── provider/ # LLM runners, model manager, tools
│ │ ├── plugin/ # LangBot-side Plugin Runtime connector/handler
│ │ ├── box/ # LangBot-side Box service/connector
│ │ ├── skill/ # Skill metadata/activation integration
│ │ ├── rag/ , vector/ # Knowledge-base and vector DB integration
│ │ ├── persistence/ # SQLAlchemy/SQLModel, Alembic, legacy migrations
│ │ ├── storage/ # Local/S3 file storage abstraction
│ │ └── config/, entity/, utils/, telemetry/, survey/
│ ├── libs/ # Vendored third-party platform SDKs
│ └── templates/ # Default config and component metadata
├── web/ # Vite + React Router + shadcn/ui + Tailwind SPA
├── docker/ # Deployment manifests
├── skills/ # In-repo agent skills, single source of truth
└── tests/ # Unit/integration/e2e/manual tests
```
## The Runtime Graph
The most useful mental model is this graph:
```text
Platform adapter
→ RuntimeBot
→ MessageAggregator
→ QueryPool
→ Controller
→ RuntimePipeline
→ PipelineStage chain
→ RequestRunner / ToolManager / PluginRuntimeConnector / BoxService
→ response via adapter
```
The HTTP and MCP surfaces are parallel entrypoints into the same service layer:
```text
HTTP client / Web UI
→ Quart route group
→ api/http/service/*
→ Application managers / persistence / runtime connectors
MCP client
→ /mcp mount
→ api/mcp/server.py tools
→ the same service layer directly
```
## Message Flow
Inbound platform messages enter through adapter-specific SDK callbacks. The common path is:
1. A platform adapter under `pkg/platform/sources/` converts platform-specific events into SDK message/event entities.
2. `RuntimeBot` in `pkg/platform/botmgr.py` applies pipeline routing rules and either discards the message, pushes it to webhooks, or sends it to the message aggregator.
3. `MessageAggregator` batches/normalizes messages before adding a `Query` to `QueryPool`.
4. `Controller` in `pkg/pipeline/controller.py` selects queries subject to global pipeline concurrency and per-session concurrency.
5. `RuntimePipeline` in `pkg/pipeline/pipelinemgr.py` runs configured pipeline stages using a responsibility-chain style executor that supports generator stages.
6. The chat stage emits plugin events, calls a configured `RequestRunner`, handles streaming/non-streaming responses, records telemetry, and appends conversation history.
7. Output stages send text, cards, chunks, files, or error notices back through the original platform adapter.
Pipeline components are registered by decorators and package import side effects. When adding a new stage, loader, runner, or adapter, check the corresponding preregistration mechanism instead of inventing a second registry.
## Platform Layer
Platform code lives under `pkg/platform/`.
- `botmgr.py` owns runtime bots, routing rules, event logging, webhook pushing, and adapter lifecycle.
- `sources/` contains adapter implementations. Each adapter subclasses `langbot_plugin.api.definition.abstract.platform.adapter.AbstractMessagePlatformAdapter` from the SDK.
- Platform entities such as `MessageChain`, `Image`, `At`, `Voice`, and events come from `langbot-plugin-sdk`, not from this repo.
The platform layer should translate between external platform APIs and LangBot's shared message/event model. It should not contain LLM-provider logic or pipeline business logic.
## Pipeline Layer
Pipeline code lives under `pkg/pipeline/`.
Important pieces:
- `pool.py::QueryPool` stores pending queries and cached in-flight queries for plugin backward-compatible calls.
- `controller.py::Controller` schedules query processing and enforces concurrency.
- `pipelinemgr.py::RuntimePipeline` materializes database pipeline config into a runtime stage chain.
- `process/handlers/chat.py::ChatMessageHandler` is the main LLM conversation handler.
- Stage families include response rules, banned sessions, content filters, preprocessors, rate limits, message truncation, long text handling, response-back, command handling, and wrappers.
Pipelines are configuration-driven. Prefer adding a stage or extending an existing stage family over hard-coding behavior in platform adapters.
## Provider, RAG, and Tools
Provider code lives under `pkg/provider/`.
- `modelmgr/` manages configured model providers and requesters.
- `runners/` implements request runners such as the local agent runner and external workflow integrations.
- `tools/toolmgr.py` aggregates tools from native tools, plugin tools, external MCP servers, and skill-authoring tools.
- `tools/loaders/mcp.py` is the MCP client side: external MCP servers that LangBot connects to for agent tools.
- RAG lives across `pkg/rag/`, `pkg/vector/`, model services, and plugin KnowledgeEngine actions.
Do not confuse LangBot's MCP client side with LangBot's own MCP server at `/mcp`; they are different surfaces.
## Plugin System
The plugin system crosses the repo boundary.
In this repo:
- `pkg/plugin/connector.py` connects LangBot to the Plugin Runtime over stdio or WebSocket.
- `pkg/plugin/handler.py` exposes LangBot actions to the runtime and calls runtime actions for plugin operations.
- `pkg/provider/tools/loaders/plugin.py` exposes plugin Tool components to LLM runners.
- Pipeline handlers emit SDK events such as normal-message events and prompt-processing events.
In `langbot-plugin-sdk`:
- `src/langbot_plugin/api/` defines `BasePlugin`, component base classes, message/event entities, contexts, proxies, and manifests.
- `src/langbot_plugin/runtime/` implements `lbp rt`, plugin discovery, dependency installation, process launching, and control/debug connections.
- `src/langbot_plugin/entities/io/` defines the action protocol shared by LangBot, runtime, and plugin processes.
The Plugin Runtime supports stdio and WebSocket control transports. Direct local LangBot runs usually spawn the runtime over stdio. Containerized/standalone deployments connect over WebSocket using `plugin.runtime_ws_url` and `--standalone-runtime`.
## Box Runtime and Skills
Box is the sandbox subsystem used by native agent tools, stdio MCP servers, skill authoring, and managed processes.
In this repo:
- `pkg/box/service.py` is the application-facing facade for exec, sessions, managed processes, skill CRUD, status, reconnects, quotas, mounts, and sandbox profiles.
- `pkg/box/connector.py` connects to the Box Runtime over stdio, Windows subprocess+WebSocket, or remote WebSocket.
- `pkg/provider/tools/loaders/native.py`, `mcp_stdio.py`, and skill loaders depend on Box availability.
- `pkg/skill/manager.py` loads skills from the Box runtime, falling back to local `data/skills` when needed.
Durable Box Workspace storage is shared across placement generations, but
sandbox sessions and managed processes are generation-scoped. LangBot validates
the current execution binding before an MCP stdio relay attach and sends the
Workspace/generation binding in authenticated headers, so a placement cutover
retires stale processes and closes already-attached relays.
In `langbot-plugin-sdk`:
- `src/langbot_plugin/box/server.py` implements `lbp box` and the WebSocket endpoints on `:5410`.
- `src/langbot_plugin/box/runtime.py` owns sandbox sessions and managed processes.
- `backend.py`, `nsjail_backend.py`, and `e2b_backend.py` implement sandbox backends.
- `skill_store.py` manages skill packages from the Box side.
Important config keys live under `box:` in `src/langbot/templates/config.yaml`: `box.enabled`, `box.backend`, `box.runtime.endpoint`, and `box.local.*`. Start LangBot with `--standalone-box` when connecting to an externally launched Box runtime.
## HTTP API, Web UI, and MCP Server
`pkg/api/http/controller/main.py` builds a Quart app, registers route groups, serves the built SPA, and wraps the ASGI app with the MCP dispatcher.
- HTTP route groups live under `pkg/api/http/controller/groups/`.
- Service-layer logic lives under `pkg/api/http/service/`.
- The built web UI is served from the frontend build path with SPA fallback.
- The MCP server lives under `pkg/api/mcp/` and is mounted at `/mcp`.
The MCP server intentionally exposes a curated subset of the API. Tools call service classes directly rather than making HTTP requests back into LangBot.
Maintenance rule: when adding, removing, or changing an HTTP endpoint that should be agent-accessible, update the matching MCP tool and the relevant in-repo skill under `skills/` in the same pass.
## Persistence and Configuration
Persistence is centered on `pkg/persistence/mgr.py`.
- SQLite is the default database; PostgreSQL is supported.
- Models live under `pkg/entity/persistence/`.
- Fresh schemas are created from metadata, then legacy migrations run up to the frozen 3.x baseline, then Alembic migrations run to head.
- New schema changes should use Alembic under `pkg/persistence/alembic/versions/`; do not extend the frozen legacy migration chain.
Configuration starts from `src/langbot/templates/config.yaml` and is generated into `data/config.yaml` on first run. Most long-lived managers read from `ap.instance_config.data`.
## Frontend
The frontend lives in `web/` and is a Vite SPA using React Router 7, shadcn/ui, Tailwind CSS, and pnpm. It is not Next.js, despite some historical filenames.
In development, `pnpm dev` serves the UI on `:3000` and reads `VITE_API_BASE_URL` to call the backend on `:5300`. In production, the built frontend is packaged into the Python distribution and served by the backend.
Keep frontend API behavior aligned with `pkg/api/http/service/` and route groups. User-facing strings must go through the existing i18n setup.
## Agent-Facing Surfaces
LangBot is deliberately agent-friendly. The agent-facing surfaces are part of the architecture, not extra docs.
- `skills/` is the single source of truth for in-repo skills.
- `pkg/api/mcp/server.py` exposes the LangBot MCP server at `/mcp`.
- `api.global_api_key` authenticates API/MCP access without a browser login.
- `AGENTS.md` and `ARCHITECTURE.md` tell coding agents how the repo works.
When one of these changes, update the others if the behavior or contract changed. API, MCP tools, and skills are one system; drift is a bug.
## Where to Change Things
- New HTTP API: add/adjust a service in `pkg/api/http/service/`, a route group in `pkg/api/http/controller/groups/`, tests, and MCP/skills if agent-accessible.
- New platform adapter: add a `pkg/platform/sources/*` adapter, component metadata/templates as needed, i18n, docs, and tests/smoke coverage.
- New pipeline behavior: add or extend a pipeline stage family under `pkg/pipeline/`; avoid putting pipeline rules in adapters.
- New LLM provider/requester: work under `pkg/provider/modelmgr/` and related service/UI surfaces.
- New LLM tool source: extend `pkg/provider/tools/loaders/` and `ToolManager` intentionally.
- New plugin component/API/protocol: change `langbot-plugin-sdk` first or in lockstep, then update LangBot bridge code.
- New Box capability: change both `pkg/box/` and `langbot-plugin-sdk/src/langbot_plugin/box/`, plus config and tests.
- New database schema: add an Alembic migration, not a legacy `dbmXXX` migration.
## Design Biases
- Keep platform translation, pipeline orchestration, provider execution, and runtime protocols separate.
- Reuse existing registries and service layers instead of adding parallel paths.
- Prefer small, explicit agent surfaces over exposing every internal API.
- Treat cross-repo contracts with the SDK as public interfaces.
- Test behavior at the narrowest useful layer first, then add integration/e2e coverage for runtime or platform changes.
+11 -2
View File
@@ -38,7 +38,7 @@ COPY --from=node /app/web/dist ./web/dist
COPY --from=nsjail-build /usr/local/bin/nsjail /usr/local/bin/nsjail
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc ca-certificates curl gnupg \
&& apt-get install -y --no-install-recommends gcc ca-certificates curl git gnupg \
# nsjail runtime libraries (the build toolchain stays in the nsjail-build
# stage; only these shared libs are needed to execute the binary).
&& apt-get install -y --no-install-recommends libprotobuf32 libnl-route-3-200 \
@@ -52,9 +52,18 @@ RUN apt-get update \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" > /etc/apt/sources.list.d/docker.list \
&& apt-get update \
&& apt-get install -y --no-install-recommends docker-ce-cli \
# Install Node.js LTS so the sandbox (nsjail/Docker box) can run npx-based
# stdio MCP servers. node/npx land in /usr/bin, which is on the nsjail
# read-only mount whitelist (_READONLY_SYSTEM_MOUNTS), so they are bound
# into the sandbox chroot automatically. Without node, any npx-launched
# MCP server exits with return_code=127 (command not found).
&& curl -fsSL https://deb.nodesource.com/setup_22.x -o /tmp/nodesource_setup.sh \
&& bash /tmp/nodesource_setup.sh \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -f /tmp/nodesource_setup.sh \
&& python -m pip install --no-cache-dir uv \
&& uv sync \
&& apt-get purge -y --auto-remove curl gnupg \
&& apt-get purge -y --auto-remove curl git gnupg \
&& rm -rf /var/lib/apt/lists/* \
&& touch /.dockerenv
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>Production-grade platform for building agentic IM bots.</h3>
<h4>Quickly build, debug, and ship AI bots to Slack, Discord, Telegram, WeChat, and more.</h4>
@@ -36,6 +36,10 @@ English / [简体中文](README_CN.md) / [繁體中文](README_TW.md) / [日本
LangBot is an **open-source, production-grade platform** for building AI-powered instant messaging bots. It connects Large Language Models (LLMs) to any chat platform, enabling you to create intelligent agents that can converse, execute tasks, and integrate with your existing workflows.
<p align="center">
<img src="res/dashboard-overview.png" alt="LangBot web management dashboard — real-time monitoring of message volume, model calls, success rate and active sessions" width="720"/>
</p>
### Key Capabilities
- **AI Conversations & Agents** — Multi-turn dialogues, tool calling, multi-modal support, streaming output. Built-in RAG (knowledge base) with deep integration to [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
@@ -47,10 +51,16 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
[→ Learn more about all features](https://link.langbot.app/en/docs/features)
📍 Practical guides: [deploy a multi-platform AI bot in 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connect DeepSeek to WeChat, Discord, and Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [run a Dify Agent in Discord, Telegram, and Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), and [build an n8n-powered chatbot](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Practical guides: [deploy a multi-platform AI bot in 5 minutes](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connect DeepSeek to WeChat, Discord, and Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [run a Dify Agent in Discord, Telegram, and Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/), and [build an n8n-powered chatbot](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 Stay Updated
Click the Star and Watch buttons in the top-right corner of the repository to get the latest updates.
![star gif](https://langbot.app/star.gif)
## Quick Start
### ☁️ LangBot Cloud (Recommended)
@@ -70,7 +80,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### One-Click Cloud Deploy
@@ -82,6 +92,17 @@ docker compose up -d
---
## Live Demo
**Try it now:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Password: `langbot123456`
_Note: Public demo environment. Do not enter sensitive information._
---
## Supported Platforms
| Platform | Status | Notes |
@@ -126,7 +147,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU Platform | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPU Platform | ✅ |
| [接口 AI](https://jiekou.ai/) | Gateway | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Gateway | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | Gateway | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Gateway | ✅ |
[→ View all integrations](https://link.langbot.app/en/docs/features)
@@ -144,14 +165,16 @@ docker compose up -d
---
## Live Demo
## Built for AI Agents 🤖
**Try it now:** https://demo.langbot.dev/
LangBot is **agent-friendly by design** — your coding agents (Claude Code, Codex, Copilot, Cursor, …) can operate, extend, and deploy LangBot with first-class support:
- Email: `demo@langbot.app`
- Password: `langbot123456`
- **MCP Server** — LangBot exposes a built-in [Model Context Protocol](https://modelcontextprotocol.io/) endpoint at `/mcp`, mirroring the HTTP API so an agent can manage bots, pipelines, plugins, and models programmatically. Authenticate with the same API key (set a global key in `config.yaml` or use a per-user key) — no login flow required. Configure it in the Web panel's **API & MCP** tab.
- **In-repo Skills** — The [`skills/`](skills/) directory is the **single source of truth** for working with LangBot: plugin development, core development, end-to-end testing, deployment, and operating the LangBot / LangBot Space MCP servers. Point your agent at this directory and it knows how to build.
- **AGENTS.md** — Every repo ships an [`AGENTS.md`](AGENTS.md) (symlinked to `CLAUDE.md`) describing architecture, conventions, and the rule that API changes must keep the MCP server and skills in sync.
- **`llms.txt`** — Machine-readable project context for LLMs is published on the website.
_Note: Public demo environment. Do not enter sensitive information._
> **Cloud / Marketplace:** [LangBot Space](https://space.langbot.app) also exposes an MCP server so agents can search and inspect the plugin / MCP / skill marketplace, authenticated with a Personal Access Token.
---
@@ -163,12 +186,6 @@ _Note: Public demo environment. Do not enter sensitive information._
---
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## Contributors
Thanks to all [contributors](https://github.com/langbot-app/LangBot/graphs/contributors) who have helped make LangBot better:
+31 -14
View File
@@ -36,6 +36,10 @@
LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时通信机器人。它将大语言模型(LLM)连接到各种聊天平台,帮助你创建能够对话、执行任务、并集成到现有工作流程中的智能 Agent。
<p align="center">
<img src="res/dashboard-overview.png" alt="LangBot Web 管理面板仪表盘 — 实时监控消息量、模型调用、成功率与活跃会话" width="720"/>
</p>
### 核心能力
- **AI 对话与 Agent** — 多轮对话、工具调用、多模态、流式输出。自带 RAG(知识库),深度集成 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、[Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com)等 LLMOps 平台。
@@ -47,10 +51,16 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 实践指南:[5 分钟部署多平台 AI 机器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[将 DeepSeek 接入微信、企业微信与 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[让 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 构建多平台 AI 聊天机器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
📍 实践指南:[5 分钟部署多平台 AI 机器人](https://langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[将 DeepSeek 接入微信、企业微信与 Discord](https://langbot.app/zh/blog/connect-deepseek-to-wechat/)、[让 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 构建多平台 AI 聊天机器人](https://langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
---
## 😎 保持更新
点击[仓库首页](https://github.com/langbot-app/LangBot)右上角 Star 和 Watch 按钮,获取最新动态。
![star gif](https://langbot.app/star.gif)
## 快速开始
### ☁️ LangBot Cloud(推荐)
@@ -70,7 +80,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### 一键云部署
@@ -82,6 +92,16 @@ docker compose up -d
---
## 在线演示
**立即体验:** https://demo.langbot.dev/
- 邮箱:`demo@langbot.app`
- 密码:`langbot123456`
*注意:公开演示环境,请不要在其中填入任何敏感信息。*
---
## 支持的平台
| 平台 | 状态 | 备注 |
@@ -126,7 +146,7 @@ docker compose up -d
| [优云智算](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | GPU 平台 | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU 平台 | ✅ |
| [接口 AI](https://jiekou.ai/) | 聚合平台 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 聚合平台 | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | 聚合平台 | ✅ |
| [小马算力](https://www.tokenpony.cn/453z1) | 聚合平台 | ✅ |
| [百宝箱Tbox](https://www.tbox.cn/open) | 智能体平台 | ✅ |
| [七牛云Qiniu](https://www.qiniu.com/ai/agent) | 聚合平台 | ✅ |
@@ -160,13 +180,16 @@ docker compose up -d
---
## 在线演示
## 为 AI Agent 而生 🤖
**立即体验:** https://demo.langbot.dev/
- 邮箱:`demo@langbot.app`
- 密码:`langbot123456`
LangBot **从设计上就对 Agent 友好** —— 你的编码 AgentClaude Code、Codex、Copilot、Cursor 等)可以一等公民般地操作、扩展和部署 LangBot
*注意:公开演示环境,请不要在其中填入任何敏感信息。*
- **MCP Server** —— LangBot 内置 [Model Context Protocol](https://modelcontextprotocol.io/) 端点 `/mcp`,与 HTTP API 对齐,Agent 可编程式管理机器人、流水线、插件和模型。使用同一套 API Key 鉴权(可在 `config.yaml` 配置全局 Key,或使用用户 Key),无需登录流程。在 Web 面板的 **API 与 MCP** 标签页中配置。
- **仓库内 Skills** —— [`skills/`](skills/) 目录是使用 LangBot 的**唯一事实来源**:插件开发、核心开发、端到端测试、部署,以及操作 LangBot / LangBot Space MCP Server。把 Agent 指向这个目录,它就知道如何动手。
- **AGENTS.md** —— 每个仓库都提供 [`AGENTS.md`](AGENTS.md)(软链到 `CLAUDE.md`),描述架构、规范,以及「API 变更必须同步更新 MCP Server 和 skills」的约定。
- **`llms.txt`** —— 面向 LLM 的机器可读项目上下文已发布在官网。
> **云端 / 市场:** [LangBot Space](https://space.langbot.app) 同样开放 MCP ServerAgent 可搜索和查看插件 / MCP / Skill 市场,使用 Personal Access Token 鉴权。
---
@@ -180,12 +203,6 @@ docker compose up -d
---
## Star 趋势
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## 贡献者
感谢所有[贡献者](https://github.com/langbot-app/LangBot/graphs/contributors)对 LangBot 的帮助:
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>Plataforma de grado de producción para construir bots de mensajería instantánea con agentes de IA.</h3>
<h4>Construya, depure y despliegue bots de IA rápidamente en Slack, Discord, Telegram, WeChat y más.</h4>
@@ -35,6 +35,10 @@
LangBot es una **plataforma de código abierto y grado de producción** para construir bots de mensajería instantánea impulsados por IA. Conecta modelos de lenguaje de gran escala (LLMs) con cualquier plataforma de chat, permitiéndole crear agentes inteligentes que pueden conversar, ejecutar tareas e integrarse con sus flujos de trabajo existentes.
<p align="center">
<img src="res/dashboard-overview.png" alt="Panel de gestión web de LangBot — monitoreo en tiempo real de volumen de mensajes, llamadas a modelos, tasa de éxito y sesiones activas" width="720"/>
</p>
### Capacidades Clave
- **Conversaciones e Agentes IA** — Diálogos de múltiples turnos, llamadas a herramientas, soporte multimodal, salida en streaming. RAG (base de conocimientos) incorporado con integración profunda con [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com).
@@ -46,10 +50,16 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
[→ Conocer más sobre todas las funcionalidades](https://link.langbot.app/en/docs/features)
📍 Guías prácticas: [desplegar un bot de IA multiplataforma en 5 minutos](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [conectar DeepSeek a WeChat, Discord y Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [ejecutar un Dify Agent en Discord, Telegram y Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) y [crear un chatbot con n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Guías prácticas: [desplegar un bot de IA multiplataforma en 5 minutos](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [conectar DeepSeek a WeChat, Discord y Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [ejecutar un Dify Agent en Discord, Telegram y Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) y [crear un chatbot con n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 Manténgase Actualizado
Haga clic en los botones Star y Watch en la esquina superior derecha del repositorio para obtener las últimas actualizaciones.
![star gif](https://langbot.app/star.gif)
## Inicio Rápido
### ☁️ LangBot Cloud (Recomendado)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### Despliegue en la Nube con un Clic
@@ -81,6 +91,16 @@ docker compose up -d
---
## Demo en Vivo
**Pruébelo ahora:** https://demo.langbot.dev/
- Correo electrónico: `demo@langbot.app`
- Contraseña: `langbot123456`
*Nota: Entorno de demostración público. No ingrese información confidencial.*
---
## Plataformas Soportadas
| Plataforma | Estado | Notas |
@@ -125,7 +145,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Plataforma GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Plataforma GPU | ✅ |
| [接口 AI](https://jiekou.ai/) | Pasarela | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Pasarela | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | Pasarela | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Pasarela | ✅ |
[→ Ver todas las integraciones](https://link.langbot.app/en/docs/features)
@@ -143,13 +163,16 @@ docker compose up -d
---
## Demo en Vivo
## Diseñado para Agentes de IA 🤖
**Pruébelo ahora:** https://demo.langbot.dev/
- Correo electrónico: `demo@langbot.app`
- Contraseña: `langbot123456`
LangBot es **agent-friendly por diseño** —— tus agentes de codificación (Claude Code, Codex, Copilot, Cursor, …) pueden operar, extender y desplegar LangBot con soporte de primera clase:
*Nota: Entorno de demostración público. No ingrese información confidencial.*
- **Servidor MCP** —— LangBot expone un endpoint integrado de [Model Context Protocol](https://modelcontextprotocol.io/) en `/mcp`, replicando la API HTTP para que un agente gestione bots, pipelines, plugins y modelos de forma programática. Autentícate con la misma API key (configura una clave global en `config.yaml` o usa una clave por usuario) —— sin flujo de login. Configúralo en la pestaña **API & MCP** del panel web.
- **Skills en el repositorio** —— El directorio [`skills/`](skills/) es la **única fuente de verdad** para trabajar con LangBot: desarrollo de plugins, desarrollo del core, pruebas end-to-end, despliegue y operación de los servidores MCP de LangBot / LangBot Space. Apunta tu agente a este directorio y sabrá cómo construir.
- **AGENTS.md** —— Cada repo incluye un [`AGENTS.md`](AGENTS.md) (enlazado simbólicamente a `CLAUDE.md`) que describe la arquitectura, las convenciones y la regla de que los cambios en la API deben mantener sincronizados el servidor MCP y los skills.
- **`llms.txt`** —— El contexto del proyecto legible por máquina para LLMs está publicado en el sitio web.
> **Nube / Marketplace:** [LangBot Space](https://space.langbot.app) también expone un servidor MCP para que los agentes busquen e inspeccionen el marketplace de plugins / MCP / skills, autenticados con un Personal Access Token.
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## Historial de Stars
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## Colaboradores
Gracias a todos los [colaboradores](https://github.com/langbot-app/LangBot/graphs/contributors) que han ayudado a mejorar LangBot:
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>Plateforme de niveau production pour construire des bots de messagerie instantanée avec agents IA.</h3>
<h4>Créez, déboguez et déployez rapidement des bots IA sur Slack, Discord, Telegram, WeChat et plus.</h4>
@@ -35,6 +35,10 @@
LangBot est une **plateforme open-source de niveau production** pour créer des bots de messagerie instantanée alimentés par l'IA. Elle connecte les grands modèles de langage (LLMs) à n'importe quelle plateforme de chat, vous permettant de créer des agents intelligents capables de converser, d'exécuter des tâches et de s'intégrer à vos workflows existants.
<p align="center">
<img src="res/dashboard-overview.png" alt="Tableau de bord de gestion web LangBot — surveillance en temps réel du volume de messages, des appels de modèles, du taux de réussite et des sessions actives" width="720"/>
</p>
### Capacités Clés
- **Conversations IA & Agents** — Dialogues multi-tours, appels d'outils, support multimodal, sortie en streaming. RAG (base de connaissances) intégré avec intégration profonde de [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
@@ -46,10 +50,16 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
[→ En savoir plus sur toutes les fonctionnalités](https://link.langbot.app/en/docs/features)
📍 Guides pratiques : [déployer un bot IA multiplateforme en 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connecter DeepSeek à WeChat, Discord et Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [exécuter un Dify Agent dans Discord, Telegram et Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) et [créer un chatbot avec n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Guides pratiques : [déployer un bot IA multiplateforme en 5 minutes](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connecter DeepSeek à WeChat, Discord et Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [exécuter un Dify Agent dans Discord, Telegram et Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) et [créer un chatbot avec n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 Restez à Jour
Cliquez sur les boutons Star et Watch dans le coin supérieur droit du dépôt pour obtenir les dernières mises à jour.
![star gif](https://langbot.app/star.gif)
## Démarrage Rapide
### ☁️ LangBot Cloud (Recommandé)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### Déploiement Cloud en un Clic
@@ -81,6 +91,16 @@ docker compose up -d
---
## Démo en Ligne
**Essayez maintenant :** https://demo.langbot.dev/
- Email : `demo@langbot.app`
- Mot de passe : `langbot123456`
*Note : Environnement de démonstration public. Ne saisissez pas d'informations sensibles.*
---
## Plateformes Supportées
| Plateforme | Statut | Notes |
@@ -122,7 +142,7 @@ docker compose up -d
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | Passerelle | ✅ |
| [GiteeAI](https://ai.gitee.com/) | Passerelle | ✅ |
| [接口 AI](https://jiekou.ai/) | Passerelle | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Passerelle | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | Passerelle | ✅ |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | Plateforme GPU | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Plateforme GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Plateforme GPU | ✅ |
@@ -143,13 +163,16 @@ docker compose up -d
---
## Démo en Ligne
## Conçu pour les agents IA 🤖
**Essayez maintenant :** https://demo.langbot.dev/
- Email : `demo@langbot.app`
- Mot de passe : `langbot123456`
LangBot est **agent-friendly par conception** —— vos agents de codage (Claude Code, Codex, Copilot, Cursor, …) peuvent exploiter, étendre et déployer LangBot avec un support de premier ordre :
*Note : Environnement de démonstration public. Ne saisissez pas d'informations sensibles.*
- **Serveur MCP** —— LangBot expose un endpoint [Model Context Protocol](https://modelcontextprotocol.io/) intégré sur `/mcp`, reflétant l'API HTTP pour qu'un agent gère bots, pipelines, plugins et modèles de façon programmatique. Authentifiez-vous avec la même clé API (définissez une clé globale dans `config.yaml` ou utilisez une clé par utilisateur) —— sans flux de connexion. Configurez-le dans l'onglet **API & MCP** du panneau web.
- **Skills dans le dépôt** —— Le répertoire [`skills/`](skills/) est la **source unique de vérité** pour travailler avec LangBot : développement de plugins, développement du cœur, tests de bout en bout, déploiement et exploitation des serveurs MCP de LangBot / LangBot Space. Pointez votre agent vers ce répertoire et il saura construire.
- **AGENTS.md** —— Chaque dépôt fournit un [`AGENTS.md`](AGENTS.md) (lien symbolique vers `CLAUDE.md`) décrivant l'architecture, les conventions et la règle selon laquelle les changements d'API doivent garder le serveur MCP et les skills synchronisés.
- **`llms.txt`** —— Le contexte projet lisible par machine pour les LLM est publié sur le site web.
> **Cloud / Marketplace :** [LangBot Space](https://space.langbot.app) expose également un serveur MCP pour que les agents recherchent et inspectent le marketplace de plugins / MCP / skills, authentifiés avec un Personal Access Token.
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## Historique des Stars
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## Contributeurs
Merci à tous les [contributeurs](https://github.com/langbot-app/LangBot/graphs/contributors) qui ont aidé à améliorer LangBot :
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>AIエージェント搭載IMボットを構築するための本番グレードプラットフォーム。</h3>
<h4>Slack、Discord、Telegram、WeChat などに AI ボットを素早く構築、デバッグ、デプロイ。</h4>
@@ -35,6 +35,10 @@
LangBot は、AI搭載のインスタントメッセージングボットを構築するための**オープンソースの本番グレードプラットフォーム**です。大規模言語モデル(LLM)をあらゆるチャットプラットフォームに接続し、会話、タスク実行、既存のワークフローとの統合が可能なインテリジェントエージェントを作成できます。
<p align="center">
<img src="res/dashboard-overview.png" alt="LangBot Web 管理パネルのダッシュボード — メッセージ量、モデル呼び出し、成功率、アクティブセッションをリアルタイム監視" width="720"/>
</p>
### 主な機能
- **AI対話とエージェント** — マルチターン対話、ツール呼び出し、マルチモーダル対応、ストリーミング出力。RAG(ナレッジベース)を内蔵し、[Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、[Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com) と深く統合。
@@ -46,10 +50,16 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
[→ すべての機能について詳しく見る](https://link.langbot.app/ja/docs/features)
📍 実践ガイド: [5分でマルチプラットフォームAIボットをデプロイ](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/)、[DeepSeekをWeChat・Discord・Telegramに接続](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/)、[Dify AgentをDiscord・Telegram・Slackで動かす](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/)、[n8n連携チャットボットを構築](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/)。
📍 実践ガイド: [5分でマルチプラットフォームAIボットをデプロイ](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/)、[DeepSeekをWeChat・Discord・Telegramに接続](https://langbot.app/en/blog/connect-deepseek-to-wechat/)、[Dify AgentをDiscord・Telegram・Slackで動かす](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/)、[n8n連携チャットボットを構築](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/)。
---
## 😎 最新情報を入手
リポジトリの右上にある Star と Watch ボタンをクリックして、最新の更新を取得してください。
![star gif](https://langbot.app/star.gif)
## クイックスタート
### ☁️ LangBot Cloud(推奨)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### ワンクリッククラウドデプロイ
@@ -81,6 +91,16 @@ docker compose up -d
---
## ライブデモ
**今すぐ試す:** https://demo.langbot.dev/
- メール: `demo@langbot.app`
- パスワード: `langbot123456`
*注意: 公開デモ環境です。機密情報を入力しないでください。*
---
## 対応プラットフォーム
| プラットフォーム | ステータス | 備考 |
@@ -125,7 +145,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPUプラットフォーム | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPUプラットフォーム | ✅ |
| [接口 AI](https://jiekou.ai/) | ゲートウェイ | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | ゲートウェイ | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | ゲートウェイ | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | ゲートウェイ | ✅ |
[→ すべての統合を表示](https://link.langbot.app/en/docs/features)
@@ -143,13 +163,16 @@ docker compose up -d
---
## ライブデモ
## AI エージェントのために 🤖
**今すぐ試す:** https://demo.langbot.dev/
- メール: `demo@langbot.app`
- パスワード: `langbot123456`
LangBot は **設計段階からエージェントフレンドリー** です。お使いのコーディングエージェント(Claude Code、Codex、Copilot、Cursor など)が、ファーストクラスのサポートで LangBot を操作・拡張・デプロイできます:
*注意: 公開デモ環境です。機密情報を入力しないでください。*
- **MCP サーバー** —— LangBot は組み込みの [Model Context Protocol](https://modelcontextprotocol.io/) エンドポイント `/mcp` を公開し、HTTP API とミラーリングされているため、エージェントがボット・パイプライン・プラグイン・モデルをプログラム的に管理できます。同じ API キーで認証(`config.yaml` でグローバルキーを設定、またはユーザーキーを使用)—— ログインフロー不要。Web パネルの **API & MCP** タブで設定します。
- **リポジトリ内 Skills** —— [`skills/`](skills/) ディレクトリは LangBot を扱うための**唯一の信頼できる情報源**です:プラグイン開発、コア開発、E2E テスト、デプロイ、LangBot / LangBot Space MCP サーバーの操作。エージェントをこのディレクトリに向ければ、構築方法を理解します。
- **AGENTS.md** —— すべてのリポジトリに [`AGENTS.md`](AGENTS.md)`CLAUDE.md` へのシンボリックリンク)があり、アーキテクチャ・規約、そして「API 変更時は MCP サーバーと skills を同期する」というルールを記述しています。
- **`llms.txt`** —— LLM 向けの機械可読なプロジェクトコンテキストを公式サイトで公開しています。
> **クラウド / マーケット:** [LangBot Space](https://space.langbot.app) も MCP サーバーを公開しており、エージェントが Personal Access Token で認証してプラグイン / MCP / Skill マーケットを検索・確認できます。
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## Star 推移
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## コントリビューター
LangBot をより良くするために貢献してくださったすべての[コントリビューター](https://github.com/langbot-app/LangBot/graphs/contributors)に感謝します:
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>AI 에이전트 IM 봇 구축을 위한 프로덕션 등급 플랫폼.</h3>
<h4>Slack, Discord, Telegram, WeChat 등에 AI 봇을 빠르게 구축, 디버그 및 배포.</h4>
@@ -35,6 +35,10 @@
LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈소스 프로덕션 등급 플랫폼**입니다. 대규모 언어 모델(LLM)을 모든 채팅 플랫폼에 연결하여 대화, 작업 실행, 기존 워크플로우와의 통합이 가능한 지능형 에이전트를 만들 수 있습니다.
<p align="center">
<img src="res/dashboard-overview.png" alt="LangBot 웹 관리 패널 대시보드 — 메시지 양, 모델 호출, 성공률, 활성 세션 실시간 모니터링" width="720"/>
</p>
### 핵심 기능
- **AI 대화 및 에이전트** — 멀티턴 대화, 도구 호출, 멀티모달 지원, 스트리밍 출력. 내장 RAG(지식 베이스)와 [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com) 심층 통합.
@@ -46,10 +50,16 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
[→ 모든 기능 자세히 보기](https://link.langbot.app/en/docs/features)
📍 실전 가이드: [5분 만에 멀티 플랫폼 AI 봇 배포하기](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [DeepSeek를 WeChat, Discord, Telegram에 연결하기](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [Dify Agent를 Discord, Telegram, Slack에서 실행하기](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), [n8n 기반 챗봇 만들기](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 실전 가이드: [5분 만에 멀티 플랫폼 AI 봇 배포하기](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [DeepSeek를 WeChat, Discord, Telegram에 연결하기](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [Dify Agent를 Discord, Telegram, Slack에서 실행하기](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/), [n8n 기반 챗봇 만들기](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 최신 정보 받기
리포지토리 오른쪽 상단의 Star 및 Watch 버튼을 클릭하여 최신 업데이트를 받으세요.
![star gif](https://langbot.app/star.gif)
## 빠른 시작
### ☁️ LangBot Cloud (추천)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### 원클릭 클라우드 배포
@@ -81,6 +91,16 @@ docker compose up -d
---
## 라이브 데모
**지금 체험:** https://demo.langbot.dev/
- 이메일: `demo@langbot.app`
- 비밀번호: `langbot123456`
*참고: 공개 데모 환경입니다. 민감한 정보를 입력하지 마세요.*
---
## 지원 플랫폼
| 플랫폼 | 상태 | 비고 |
@@ -125,7 +145,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU 플랫폼 | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | GPU 플랫폼 | ✅ |
| [接口 AI](https://jiekou.ai/) | 게이트웨이 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 게이트웨이 | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | 게이트웨이 | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | 게이트웨이 | ✅ |
[→ 모든 통합 보기](https://link.langbot.app/en/docs/features)
@@ -143,13 +163,16 @@ docker compose up -d
---
## 라이브 데모
## AI 에이전트를 위한 설계 🤖
**지금 체험:** https://demo.langbot.dev/
- 이메일: `demo@langbot.app`
- 비밀번호: `langbot123456`
LangBot은 **설계 단계부터 에이전트 친화적**입니다 —— 코딩 에이전트(Claude Code, Codex, Copilot, Cursor 등)가 일급 지원으로 LangBot을 운영·확장·배포할 수 있습니다:
*참고: 공개 데모 환경입니다. 민감한 정보를 입력하지 마세요.*
- **MCP 서버** —— LangBot은 내장 [Model Context Protocol](https://modelcontextprotocol.io/) 엔드포인트 `/mcp`를 제공하며, HTTP API와 동일하게 미러링되어 에이전트가 봇·파이프라인·플러그인·모델을 프로그래밍 방식으로 관리할 수 있습니다. 동일한 API 키로 인증하며(`config.yaml`에 전역 키 설정 또는 사용자 키 사용) 로그인 절차가 필요 없습니다. 웹 패널의 **API & MCP** 탭에서 설정합니다.
- **저장소 내 Skills** —— [`skills/`](skills/) 디렉터리는 LangBot 작업의 **단일 진실 공급원**입니다: 플러그인 개발, 코어 개발, E2E 테스트, 배포, LangBot / LangBot Space MCP 서버 운영. 에이전트를 이 디렉터리로 안내하면 빌드 방법을 알게 됩니다.
- **AGENTS.md** —— 모든 저장소에는 [`AGENTS.md`](AGENTS.md)(`CLAUDE.md`로 심볼릭 링크)가 있으며 아키텍처, 규약, 그리고 API 변경 시 MCP 서버와 skills를 동기화해야 한다는 규칙을 설명합니다.
- **`llms.txt`** —— LLM을 위한 기계 판독 가능한 프로젝트 컨텍스트가 웹사이트에 게시되어 있습니다.
> **클라우드 / 마켓플레이스:** [LangBot Space](https://space.langbot.app)도 MCP 서버를 제공하여 에이전트가 Personal Access Token으로 인증해 플러그인 / MCP / Skill 마켓플레이스를 검색하고 조회할 수 있습니다.
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## Star 추이
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## 기여자
LangBot을 더 나은 프로젝트로 만들어 주신 모든 [기여자](https://github.com/langbot-app/LangBot/graphs/contributors)분들께 감사드립니다:
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>Платформа производственного уровня для создания агентных IM-ботов.</h3>
<h4>Быстро создавайте, отлаживайте и развертывайте ИИ-ботов в Slack, Discord, Telegram, WeChat и других платформах.</h4>
@@ -35,6 +35,10 @@
LangBot — это **платформа с открытым исходным кодом производственного уровня** для создания ИИ-ботов в мессенджерах. Она связывает большие языковые модели (LLM) с любой чат-платформой, позволяя создавать интеллектуальных агентов, которые могут вести диалоги, выполнять задачи и интегрироваться с вашими существующими рабочими процессами.
<p align="center">
<img src="res/dashboard-overview.png" alt="Панель веб-управления LangBot — мониторинг объёма сообщений, вызовов моделей, успешности и активных сессий в реальном времени" width="720"/>
</p>
### Ключевые возможности
- **ИИ-диалоги и агенты** — Многораундовые диалоги, вызов инструментов, мультимодальная поддержка, потоковый вывод. Встроенная реализация RAG (база знаний) с глубокой интеграцией в [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
@@ -46,10 +50,16 @@ LangBot — это **платформа с открытым исходным к
[→ Подробнее обо всех возможностях](https://link.langbot.app/en/docs/features)
📍 Практические руководства: [развернуть мультиплатформенного ИИ-бота за 5 минут](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [подключить DeepSeek к WeChat, Discord и Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [запустить Dify Agent в Discord, Telegram и Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) и [создать чат-бота на n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Практические руководства: [развернуть мультиплатформенного ИИ-бота за 5 минут](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [подключить DeepSeek к WeChat, Discord и Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [запустить Dify Agent в Discord, Telegram и Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) и [создать чат-бота на n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 Оставайтесь в курсе
Нажмите кнопки Star и Watch в правом верхнем углу репозитория, чтобы получать последние обновления.
![star gif](https://langbot.app/star.gif)
## Быстрый старт
### ☁️ LangBot Cloud (Рекомендуется)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### Облачное развертывание одним кликом
@@ -81,6 +91,16 @@ docker compose up -d
---
## Демо
**Попробуйте прямо сейчас:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Пароль: `langbot123456`
*Примечание: Публичная демо-среда. Не вводите конфиденциальную информацию.*
---
## Поддерживаемые платформы
| Платформа | Статус | Примечания |
@@ -121,7 +141,7 @@ docker compose up -d
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | Шлюз | ✅ |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | Шлюз | ✅ |
| [GiteeAI](https://ai.gitee.com/) | Шлюз | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Шлюз | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | Шлюз | ✅ |
| [接口 AI](https://jiekou.ai/) | Шлюз | ✅ |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | Платформа GPU | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Платформа GPU | ✅ |
@@ -143,13 +163,16 @@ docker compose up -d
---
## Демо
## Создано для ИИ-агентов 🤖
**Попробуйте прямо сейчас:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Пароль: `langbot123456`
LangBot **дружелюбен к агентам по своей архитектуре** —— ваши кодинг-агенты (Claude Code, Codex, Copilot, Cursor и др.) могут управлять, расширять и развёртывать LangBot с первоклассной поддержкой:
*Примечание: Публичная демо-среда. Не вводите конфиденциальную информацию.*
- **MCP-сервер** —— LangBot предоставляет встроенную конечную точку [Model Context Protocol](https://modelcontextprotocol.io/) по адресу `/mcp`, зеркалирующую HTTP API, чтобы агент мог программно управлять ботами, пайплайнами, плагинами и моделями. Аутентификация той же API-ключом (задайте глобальный ключ в `config.yaml` или используйте пользовательский ключ) —— без процедуры входа. Настраивается на вкладке **API & MCP** веб-панели.
- **Skills в репозитории** —— Каталог [`skills/`](skills/) является **единственным источником истины** для работы с LangBot: разработка плагинов, разработка ядра, сквозное тестирование, развёртывание и работа с MCP-серверами LangBot / LangBot Space. Направьте агента в этот каталог, и он будет знать, как собирать.
- **AGENTS.md** —— Каждый репозиторий содержит [`AGENTS.md`](AGENTS.md) (символическая ссылка на `CLAUDE.md`), описывающий архитектуру, соглашения и правило: изменения API должны синхронизировать MCP-сервер и skills.
- **`llms.txt`** —— Машиночитаемый контекст проекта для LLM опубликован на сайте.
> **Облако / Маркетплейс:** [LangBot Space](https://space.langbot.app) также предоставляет MCP-сервер, чтобы агенты могли искать и просматривать маркетплейс плагинов / MCP / skills, аутентифицируясь с помощью Personal Access Token.
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## История Stars
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## Участники
Спасибо всем [участникам](https://github.com/langbot-app/LangBot/graphs/contributors), которые помогли сделать LangBot лучше:
+31 -14
View File
@@ -37,6 +37,10 @@
LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時通訊機器人。它將大語言模型(LLM)連接到各種聊天平台,幫助你創建能夠對話、執行任務、並整合到現有工作流程中的智能 Agent。
<p align="center">
<img src="res/dashboard-overview.png" alt="LangBot Web 管理面板儀表板 — 即時監控訊息量、模型調用、成功率與活躍工作階段" width="720"/>
</p>
### 核心能力
- **AI 對話與 Agent** — 多輪對話、工具調用、多模態、流式輸出。自帶 RAG(知識庫),深度整合 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)、 [Deerflow](https://deerflow.tech)、[Weknora](https://weknora.weixin.qq.com)等 LLMOps 平台。
@@ -48,10 +52,16 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 實踐指南:[5 分鐘部署多平台 AI 機器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[將 DeepSeek 接入微信、企業微信與 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[讓 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 建構多平台 AI 聊天機器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
📍 實踐指南:[5 分鐘部署多平台 AI 機器人](https://langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[將 DeepSeek 接入微信、企業微信與 Discord](https://langbot.app/zh/blog/connect-deepseek-to-wechat/)、[讓 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 建構多平台 AI 聊天機器人](https://langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
---
## 😎 保持更新
點擊倉庫右上角 Star 和 Watch 按鈕,獲取最新動態。
![star gif](https://langbot.app/star.gif)
## 快速開始
### ☁️ LangBot Cloud(推薦)
@@ -71,7 +81,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### 一鍵雲端部署
@@ -83,6 +93,16 @@ docker compose up -d
---
## 線上演示
**立即體驗:** https://demo.langbot.dev/
- 信箱:`demo@langbot.app`
- 密碼:`langbot123456`
*注意:公開演示環境,請不要在其中填入任何敏感資訊。*
---
## 支援的平台
| 平台 | 狀態 | 備註 |
@@ -127,7 +147,7 @@ docker compose up -d
| [優雲智算](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | GPU 平台 | ✅ |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | GPU 平台 | ✅ |
| [接口 AI](https://jiekou.ai/) | 聚合平台 | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | 聚合平台 | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | 聚合平台 | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | 聚合平台 | ✅ |
### TTS(語音合成)
@@ -159,13 +179,16 @@ docker compose up -d
---
## 線上演示
## 為 AI Agent 而生 🤖
**立即體驗:** https://demo.langbot.dev/
- 信箱:`demo@langbot.app`
- 密碼:`langbot123456`
LangBot **從設計上就對 Agent 友善** —— 你的編碼 AgentClaude Code、Codex、Copilot、Cursor 等)可以一等公民般地操作、擴充和部署 LangBot
*注意:公開演示環境,請不要在其中填入任何敏感資訊。*
- **MCP Server** —— LangBot 內建 [Model Context Protocol](https://modelcontextprotocol.io/) 端點 `/mcp`,與 HTTP API 對齊,Agent 可程式化管理機器人、流水線、外掛和模型。使用同一套 API Key 鑑權(可在 `config.yaml` 設定全域 Key,或使用使用者 Key),無需登入流程。在 Web 面板的 **API 與 MCP** 分頁中設定。
- **倉庫內 Skills** —— [`skills/`](skills/) 目錄是使用 LangBot 的**唯一事實來源**:外掛開發、核心開發、端到端測試、部署,以及操作 LangBot / LangBot Space MCP Server。把 Agent 指向這個目錄,它就知道如何動手。
- **AGENTS.md** —— 每個倉庫都提供 [`AGENTS.md`](AGENTS.md)(軟連結到 `CLAUDE.md`),描述架構、規範,以及「API 變更必須同步更新 MCP Server 和 skills」的約定。
- **`llms.txt`** —— 面向 LLM 的機器可讀專案上下文已發布在官網。
> **雲端 / 市集:** [LangBot Space](https://space.langbot.app) 同樣開放 MCP ServerAgent 可搜尋和檢視外掛 / MCP / Skill 市集,使用 Personal Access Token 鑑權。
---
@@ -179,12 +202,6 @@ docker compose up -d
---
## Star 趨勢
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## 貢獻者
感謝所有[貢獻者](https://github.com/langbot-app/LangBot/graphs/contributors)對 LangBot 的幫助:
+32 -15
View File
@@ -5,7 +5,7 @@
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<a href="https://www.producthunt.com/products/langbot/launches/langbot?embed=true&amp;utm_source=badge-featured&amp;utm_medium=badge&amp;utm_campaign=badge-langbot" target="_blank" rel="noopener noreferrer"><img alt="LangBot - Easy-to-use global IM bot platform designed for the LLM era | Product Hunt" width="250" height="54" src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=979554&amp;theme=light&amp;t=1782822143403"></a>
<h3>Nền tảng cấp sản xuất để xây dựng bot IM với AI agent.</h3>
<h4>Xây dựng, gỡ lỗi và triển khai bot AI nhanh chóng trên Slack, Discord, Telegram, WeChat và nhiều nền tảng khác.</h4>
@@ -35,6 +35,10 @@
LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để xây dựng bot nhắn tin tức thời được hỗ trợ bởi AI. Nó kết nối các Mô hình Ngôn ngữ Lớn (LLM) với bất kỳ nền tảng chat nào, cho phép bạn tạo các agent thông minh có thể trò chuyện, thực hiện tác vụ và tích hợp với quy trình làm việc hiện có của bạn.
<p align="center">
<img src="res/dashboard-overview.png" alt="Bảng điều khiển quản lý web LangBot — giám sát thời gian thực khối lượng tin nhắn, lệnh gọi mô hình, tỷ lệ thành công và phiên hoạt động" width="720"/>
</p>
### Khả năng chính
- **Hội thoại AI & Agent** — Đối thoại nhiều lượt, gọi công cụ, hỗ trợ đa phương thức, đầu ra streaming. RAG (cơ sở kiến thức) tích hợp sẵn với tích hợp sâu vào [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org), [Deerflow](https://deerflow.tech), [Weknora](https://weknora.weixin.qq.com).
@@ -46,10 +50,16 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
[→ Tìm hiểu thêm về tất cả tính năng](https://link.langbot.app/en/docs/features)
📍 Hướng dẫn thực hành: [triển khai bot AI đa nền tảng trong 5 phút](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [kết nối DeepSeek với WeChat, Discord và Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [chạy Dify Agent trên Discord, Telegram và Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) và [xây dựng chatbot với n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Hướng dẫn thực hành: [triển khai bot AI đa nền tảng trong 5 phút](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [kết nối DeepSeek với WeChat, Discord và Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [chạy Dify Agent trên Discord, Telegram và Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) và [xây dựng chatbot với n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
## 😎 Cập nhật Mới nhất
Nhấp vào các nút Star và Watch ở góc trên bên phải của kho lưu trữ để nhận các bản cập nhật mới nhất.
![star gif](https://langbot.app/star.gif)
## Bắt đầu nhanh
### ☁️ LangBot Cloud (Khuyên dùng)
@@ -69,7 +79,7 @@ uvx langbot
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
docker compose --profile all up -d
```
### Triển khai đám mây một cú nhấp
@@ -81,6 +91,16 @@ docker compose up -d
---
## Demo trực tuyến
**Thử ngay:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Mật khẩu: `langbot123456`
*Lưu ý: Môi trường demo công khai. Không nhập thông tin nhạy cảm.*
---
## Nền tảng được hỗ trợ
| Nền tảng | Trạng thái | Ghi chú |
@@ -125,7 +145,7 @@ docker compose up -d
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | Nền tảng GPU | ✅ |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | Nền tảng GPU | ✅ |
| [接口 AI](https://jiekou.ai/) | Cổng | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Cổng | ✅ |
| [302.AI](https://share.302ai.cn/SuTG99) | Cổng | ✅ |
| [Qiniu](https://www.qiniu.com/ai/agent) | Cổng | ✅ |
[→ Xem tất cả tích hợp](https://link.langbot.app/en/docs/features)
@@ -143,13 +163,16 @@ docker compose up -d
---
## Demo trực tuyến
## Được xây dựng cho AI Agent 🤖
**Thử ngay:** https://demo.langbot.dev/
- Email: `demo@langbot.app`
- Mật khẩu: `langbot123456`
LangBot **thân thiện với agent ngay từ thiết kế** —— các coding agent của bạn (Claude Code, Codex, Copilot, Cursor, …) có thể vận hành, mở rộng và triển khai LangBot với sự hỗ trợ hạng nhất:
*Lưu ý: Môi trường demo công khai. Không nhập thông tin nhạy cảm.*
- **MCP Server** —— LangBot cung cấp endpoint [Model Context Protocol](https://modelcontextprotocol.io/) tích hợp tại `/mcp`, phản chiếu HTTP API để agent quản lý bot, pipeline, plugin và model theo cách lập trình. Xác thực bằng cùng một API key (đặt key toàn cục trong `config.yaml` hoặc dùng key theo người dùng) —— không cần luồng đăng nhập. Cấu hình tại tab **API & MCP** trong bảng điều khiển Web.
- **Skills trong repo** —— Thư mục [`skills/`](skills/) là **nguồn sự thật duy nhất** để làm việc với LangBot: phát triển plugin, phát triển core, kiểm thử end-to-end, triển khai và vận hành MCP Server của LangBot / LangBot Space. Trỏ agent của bạn vào thư mục này và nó sẽ biết cách xây dựng.
- **AGENTS.md** —— Mỗi repo đều có [`AGENTS.md`](AGENTS.md) (liên kết tượng trưng tới `CLAUDE.md`) mô tả kiến trúc, quy ước và quy tắc rằng thay đổi API phải giữ MCP Server và skills đồng bộ.
- **`llms.txt`** —— Ngữ cảnh dự án có thể đọc bằng máy dành cho LLM được công bố trên website.
> **Cloud / Marketplace:** [LangBot Space](https://space.langbot.app) cũng cung cấp MCP Server để agent tìm kiếm và kiểm tra marketplace plugin / MCP / skill, xác thực bằng Personal Access Token.
---
@@ -161,12 +184,6 @@ docker compose up -d
---
## Lịch sử Star
[![Star History Chart](https://api.star-history.com/svg?repos=langbot-app/LangBot&type=Date)](https://star-history.com/#langbot-app/LangBot&Date)
---
## Người đóng góp
Cảm ơn tất cả [người đóng góp](https://github.com/langbot-app/LangBot/graphs/contributors) đã giúp LangBot trở nên tốt hơn:
+20 -3
View File
@@ -14,6 +14,9 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
# Shared with the langbot service and sent only as a WebSocket handshake
# header. Generate with: openssl rand -hex 32
- LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN=${LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN:-}
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "rt"]
networks:
- langbot_network
@@ -40,9 +43,16 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
# Shared control-plane secret used to authenticate both the RPC socket
# and managed-process relay. Generate once (for example with
# ``openssl rand -hex 32``) and export it before enabling this profile.
# An empty value is accepted by Compose so Box can remain optional, but
# the Box runtime itself fails closed when the profile is started.
- LANGBOT_BOX_CONTROL_TOKEN=${LANGBOT_BOX_CONTROL_TOKEN:-}
# The Box runtime does NOT read box.local.* from config.yaml or env; it
# receives its configuration from LangBot via the INIT RPC action.
# Do not add LANGBOT_BOX_* / BOX__* here — they would be silently ignored.
# receives its functional configuration from LangBot via the INIT RPC
# action. LANGBOT_BOX_CONTROL_TOKEN is the intentional security-only
# exception; do not add BOX__* here because those would be ignored.
# Launched through the same CLI entry point as the plugin runtime
# (`langbot_plugin.cli.__init__ <subcommand>`). WebSocket is the default
# control transport — mirrors `rt`, which also runs with no flag. Pass
@@ -60,13 +70,20 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
# Must match langbot_plugin_runtime. Empty/missing values make the
# external control channel fail closed.
- LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN=${LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN:-}
# Must match the value supplied to langbot_box. The token is sent only
# in WebSocket handshake headers, never in URLs or action payloads.
- LANGBOT_BOX_CONTROL_TOKEN=${LANGBOT_BOX_CONTROL_TOKEN:-}
# Unified env-override convention: SECTION__SUBSECTION__KEY overrides the
# matching config.yaml field (see LoadConfigStage). These map onto
# box.local.* and are forwarded to the Box runtime via INIT RPC.
# box.* and are forwarded to the Box runtime via INIT RPC.
- BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__LOCAL__DEFAULT_WORKSPACE=default
- BOX__LOCAL__SKILLS_ROOT=skills
- BOX__LOCAL__ALLOWED_MOUNT_ROOTS=${LANGBOT_BOX_ROOT:-${PWD}/data/box}
- BOX__DOCKER__CPU_LIMIT_ENABLED=${LANGBOT_BOX_DOCKER_CPU_LIMIT_ENABLED:-true}
ports:
- 5300:5300 # For web ui and webhook callback
- 2280-2285:2280-2285 # For platform reverse connection
+40 -6
View File
@@ -4,6 +4,10 @@
# Full deployment guide (zh/en/ja): https://docs.langbot.app -> Installation -> Kubernetes
#
# Usage:
# kubectl -n langbot create secret generic langbot-plugin-runtime-control \
# --from-literal=token="$(openssl rand -hex 32)"
# kubectl -n langbot create secret generic langbot-box-control \
# --from-literal=token="$(openssl rand -hex 32)"
# kubectl apply -f kubernetes.yaml
#
# Prerequisites:
@@ -127,6 +131,11 @@ spec:
configMapKeyRef:
name: langbot-config
key: TZ
- name: LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN
valueFrom:
secretKeyRef:
name: langbot-plugin-runtime-control
key: token
volumeMounts:
- name: plugin-data
mountPath: /app/data/plugins
@@ -139,7 +148,8 @@ spec:
cpu: "1000m"
# Liveness probe to restart container if it becomes unresponsive
livenessProbe:
tcpSocket:
httpGet:
path: /healthz
port: 5400
initialDelaySeconds: 30
periodSeconds: 10
@@ -147,7 +157,8 @@ spec:
failureThreshold: 3
# Readiness probe to know when container is ready to accept traffic
readinessProbe:
tcpSocket:
httpGet:
path: /healthz
port: 5400
initialDelaySeconds: 10
periodSeconds: 5
@@ -246,9 +257,14 @@ spec:
configMapKeyRef:
name: langbot-config
key: TZ
- name: LANGBOT_BOX_CONTROL_TOKEN
valueFrom:
secretKeyRef:
name: langbot-box-control
key: token
# The Box runtime does NOT read box.local.* / BOX__* from its own env;
# it receives its configuration from LangBot via the INIT RPC action.
# Do not add BOX__* here — they would be silently ignored.
# it receives its functional configuration from LangBot via INIT.
# LANGBOT_BOX_CONTROL_TOKEN is the security-only exception.
volumeMounts:
# Box workspace root — identical path on node, box, and sandbox
# containers (see the IMPORTANT note above).
@@ -265,14 +281,16 @@ spec:
memory: "1Gi"
cpu: "1000m"
livenessProbe:
tcpSocket:
httpGet:
path: /healthz
port: 5410
initialDelaySeconds: 20
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
tcpSocket:
httpGet:
path: /healthz
port: 5410
initialDelaySeconds: 10
periodSeconds: 5
@@ -319,6 +337,10 @@ metadata:
app: langbot
spec:
replicas: 1
# Plugin Runtime has a single active LangBot control owner. Recreate avoids
# two LangBot pods fighting over that connection during a rolling update.
strategy:
type: Recreate
selector:
matchLabels:
app: langbot
@@ -352,6 +374,11 @@ spec:
configMapKeyRef:
name: langbot-config
key: PLUGIN__RUNTIME_WS_URL
- name: LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN
valueFrom:
secretKeyRef:
name: langbot-plugin-runtime-control
key: token
# Box (sandbox) runtime endpoint. Connects LangBot to the langbot-box
# Service over WebSocket. Remove this (and the langbot-box Deployment)
# and set BOX__ENABLED=false if you do not want the sandbox.
@@ -360,6 +387,13 @@ spec:
configMapKeyRef:
name: langbot-config
key: BOX__RUNTIME__ENDPOINT
# Same Secret as langbot-box. It authenticates the RPC and managed-
# process relay handshakes and is never put in a URL or RPC payload.
- name: LANGBOT_BOX_CONTROL_TOKEN
valueFrom:
secretKeyRef:
name: langbot-box-control
key: token
# box.local.* config — forwarded to the Box runtime via INIT RPC. The
# host_root MUST match the box-root hostPath mountPath below AND the box
# Deployment's box-root mountPath, so that skill package paths resolve
+55 -6
View File
@@ -8,7 +8,50 @@ API keys can be managed through the web interface:
1. Log in to the LangBot web interface
2. Click the "API Keys" button at the bottom of the sidebar
3. Create, view, copy, or delete API keys as needed
3. Create an API key and copy its secret immediately
4. Revoke keys that are no longer needed
Database-backed API-key secrets are returned exactly once. LangBot stores only
a SHA-256 lookup hash, so an existing secret cannot be displayed or recovered
later. Each key belongs to one Workspace, has explicit permission scopes, and
may have an expiry. The Workspace is derived from the authenticated key; an
`X-Workspace-Id` header cannot redirect it to another tenant.
## Global API Key (config.yaml)
In addition to web-UI-created keys (stored in the database, prefixed `lbk_`),
LangBot supports a **global API key** defined directly in `data/config.yaml`.
This is a Community-edition bootstrap option for automated deployments,
infrastructure-as-code, and AI agents
that need API/MCP access **without a login session and without creating a
database record first**.
```yaml
api:
port: 5300
# ...
global_api_key: 'your-strong-secret-here' # leave empty to disable
```
Behavior:
- In Community edition's singleton Workspace, a non-empty
`api.global_api_key` is bound to that Workspace and accepted across the HTTP
service API and the MCP server.
- The global config key is rejected when multi-Workspace SaaS mode is enabled;
SaaS automation must use a database-backed Workspace key or a closed control
plane credential.
- The global key does **not** require the `lbk_` prefix; use any sufficiently
strong secret.
- Leave it empty (`''`, the default) to disable it entirely; only database-backed
`lbk_` keys will then be accepted.
- Existing installs are unaffected until you add the key — config completion only
backfills top-level keys, and the lookup is defensive when the field is absent.
> **Security:** the global key is stored in plaintext in `config.yaml` and has
> the singleton Workspace's full fixed permission set. Only enable it on
> trusted/internal Community deployments, keep file permissions tight, always
> serve over HTTPS, and rotate it if it may have leaked.
## Using API Keys
@@ -28,7 +71,9 @@ Authorization: Bearer lbk_your_api_key_here
## Available APIs
All existing LangBot APIs now support **both user token and API key authentication**. This means you can use API keys to access:
Endpoints that declare API-key authentication accept either a user token or a
Workspace API key. The key must include the permission required by the route.
This includes:
- **Model Management** - `/api/v1/provider/models/llm` and `/api/v1/provider/models/embedding`
- **Bot Management** - `/api/v1/platform/bots`
@@ -195,6 +240,11 @@ or
}
```
### 403 Forbidden
The key is valid for its Workspace but does not include the fixed permission
required by the route.
### 500 Internal Server Error
```json
@@ -208,7 +258,7 @@ or
1. **Keep API keys secure**: Store them securely and never commit them to version control
2. **Use HTTPS**: Always use HTTPS in production to encrypt API key transmission
3. **Rotate keys regularly**: Create new API keys periodically and delete old ones
3. **Rotate keys regularly**: Create new API keys periodically and revoke old ones
4. **Use descriptive names**: Give your API keys meaningful names to track their usage
5. **Delete unused keys**: Remove API keys that are no longer needed
6. **Use X-API-Key header**: Prefer using the `X-API-Key` header for clarity
@@ -285,7 +335,6 @@ curl -X POST \
## Notes
- The same endpoints work for both the web UI (with user tokens) and external services (with API keys)
- API-key-enabled endpoints use the same resource shapes as the web UI
- No need to learn different API paths - use the existing API documentation with API key authentication
- All endpoints that previously required user authentication now also accept API keys
- API keys never select a Workspace from a request header; their persisted binding is authoritative
+575
View File
@@ -0,0 +1,575 @@
# HTTP Bot Adapter — Design Document
> Status: **Implemented** · Branch: `feat/http-bot-adapter` · Author: LangBot core
>
> A first-class, **standalone** message-platform adapter (`http_bot`) that lets
> any external system (e.g. LangBot Space ticketing, an internal back-office, a
> CRM, a custom web app) talk to a LangBot pipeline over plain HTTP — **inbound**
> by POSTing messages in, **outbound** by receiving replies on a callback URL —
> with full support for the pipeline's native N→1 aggregation and 1→M
> multi-reply semantics, and **without** holding a long-lived WebSocket
> connection.
>
> **Shipped in this branch:**
> - `src/langbot/pkg/platform/sources/http_bot.yaml` — adapter manifest (auto-discovered)
> - `src/langbot/pkg/platform/sources/http_bot.py` — `HttpBotAdapter`
> - `src/langbot/pkg/platform/sources/http_bot_signing.py` — HMAC helpers
> - `src/langbot/pkg/platform/sources/http_bot.svg` — icon
> - `docs/platforms/http-bot.md` — integration guide
> - `docs/http-bot-openapi.json` — machine-readable contract
> - `examples/http-bot/` — Python + TypeScript reference clients
>
> **Final decisions (resolving the original open questions):**
> 1. Callback URL is **config-only** — never accepted per-message (SSRF closed).
> 2. **Session reset is provided** — `POST /bots/<uuid>/reset` keyed by `session_id`.
> 3. Reference **clients are provided** — `examples/http-bot/client.py` + `client.ts`.
> 4. **Sync convenience mode is included** — `POST /bots/<uuid>/sync` (opt-in, lossy).
---
## 1. Background & Motivation
### 1.1 The concrete need
LangBot Space wants to use a LangBot pipeline as the brain for **ticket
handling**. The integration is **server-to-server**: Space's backend pushes a
user's ticket messages into LangBot and renders LangBot's replies back into the
ticket thread.
This interaction is **not** request/response shaped:
- **N → 1**: a user may fire several messages in a row ("the app crashed" …
"when I click export" … "here's a screenshot"). The pipeline's
**message aggregation** feature should debounce and merge these into one turn.
- **1 → N**: a single turn may yield **multiple** outbound messages — a tool/
function call narrating progress, a plugin emitting several cards, a streamed
answer split into chunks.
### 1.2 Why the existing options don't fit
LangBot today exposes exactly one externally-reachable way to drive a pipeline
that is **not** tied to a specific IM vendor: the **WebSocket** path
(`/api/v1/pipelines/<uuid>/ws/connect` for dashboard debug, and
`/api/v1/embed/<bot_uuid>/ws/connect` for the embeddable web widget).
For a server-to-server integration the WebSocket path has real friction:
| Problem | Detail |
|---|---|
| Long-lived connection | Caller must maintain a socket, heartbeats, and reconnect logic for what is fundamentally a fire-and-collect workload. |
| Session identity | Inbound messages are keyed by the transient `connection_id` (`websocket_{connection_id}`); the caller **cannot supply a stable, business-meaningful session id** (e.g. a ticket number). Multi-ticket isolation is not expressible. |
| Auth mismatch | The debug socket is gated by the **dashboard JWT** (must not be handed to an external service); the embed socket is gated by **Cloudflare Turnstile** (a *browser* human-check that a backend cannot satisfy). Neither is a server-to-server credential. |
| In-memory, single-process state | Session history lives in process memory and is lost on restart. |
> **Key realisation.** The N→1 / 1→M behaviour the caller wants is **not**
> provided by WebSocket — it is provided by the **pipeline** (aggregation +
> the adapter being free to call `reply_message` any number of times). It is
> therefore **transport-independent**. We can deliver the exact same semantics
> over a far lighter HTTP transport.
### 1.3 Why a *new, standalone* adapter (not a refactor of an existing one)
The brief is explicit: **do not reuse / fork an existing vendor adapter.** The
vendor adapters (`lark`, `wecom`, `qqofficial`, `slack`, …) carry vendor-specific
signature schemes, payload shapes, and message-segment mappings. Bending one of
them into a "generic" mode would couple a public integration surface to one
vendor's quirks and make the developer experience worse for everyone.
Instead we ship `http_bot` as a clean, independent adapter whose **entire
contract is LangBot's own** — documented, versioned, and designed front-to-back
around *integrator* developer experience.
---
## 2. Goals & Non-Goals
### Goals
- **G1** A standalone `http_bot` adapter, selectable like any other platform
adapter in the dashboard, with its own config schema and docs.
- **G2** **Inbound**: external systems POST messages to a stable LangBot URL,
carrying a **caller-defined `session_id`** that maps 1:1 to a LangBot session.
- **G3** **Outbound**: LangBot delivers each reply by POSTing to a
caller-configured **callback URL**; one turn may produce **many** callbacks.
- **G4** Preserve pipeline-native **N→1 aggregation** and **1→M multi-reply**.
- **G5** Server-to-server **auth**: shared-secret HMAC request signing both
directions (no JWT, no Turnstile, no long-lived socket).
- **G6** **Great DX**: copy-pasteable curl, a tiny reference client, an OpenAPI
fragment, idempotency, clear error envelope, and a local echo-server recipe.
### Non-Goals
- Not replacing or deprecating the WebSocket / embed widget path (that remains
the right tool for *browser*, real-time, streaming chat UIs).
- Not a synchronous "one request → one response" RPC (explicitly rejected: it
cannot express 1→M; see §9 for the optional sync convenience mode).
- No built-in message **persistence/replay** in v1 (callbacks are at-least-once
best-effort; durability is the caller's responsibility — see §8).
- No multi-tenant API-key management UI in v1 (one secret per bot; see §11).
---
## 3. How LangBot routes a message (the parts we plug into)
Understanding the existing flow is what makes this adapter cheap. A message
flows through these stages (verified against current `master`):
```
INBOUND OUTBOUND
external POST ─┐ ┌─ reply_message()
▼ │ reply_message_chunk()
POST /bots/<bot_uuid> (unified webhook router, AuthType.NONE)
│ webhooks.py → adapter.handle_unified_webhook(bot_uuid, path, request)
▼ │
HttpBotAdapter.handle_unified_webhook │ (called 0..N times
• verify HMAC signature │ per turn by the
• parse {session_id, message[]} │ pipeline / plugins)
• build FriendMessage / GroupMessage │
• fire registered listener ───────────────┐ │
│ │ │
▼ ▼ │
botmgr.on_friend_message / on_group_message │
• (optional) webhook_pusher fan-out │
• msg_aggregator.add_message(...) ── N→1 debounce ──►│
│ │
▼ │
query_pool → pipeline.run() ─── invokes adapter ─────┘
reply methods 1..M times
```
Two framework facts we rely on:
1. **N→1 aggregation is free.** `botmgr` hands every inbound event to
`self.ap.msg_aggregator.add_message(...)`, which debounces per
`session_id` and merges consecutive messages into one pipeline turn
(`pkg/pipeline/aggregator.py`). The adapter does nothing special.
2. **1→M is free.** The pipeline (and any plugin in the chain) calls
`adapter.reply_message()` / `reply_message_chunk()` **as many times as it
wants** per turn. The adapter's only job is to deliver each call outward.
For `http_bot` that means: **one outbound callback POST per call.**
3. **A unified inbound route already exists.** `WebhookRouterGroup`
(`pkg/api/http/controller/groups/webhooks.py`) maps
`POST /bots/<bot_uuid>[/<path>]` (auth `NONE`) to
`adapter.handle_unified_webhook(bot_uuid, path, request)`. `http_bot`
implements that method and is reachable **without registering any new
route** — it does its own signature verification, exactly like the vendor
webhook adapters do.
> Net new code is essentially: one `http_bot.py` adapter, one `http_bot.yaml`
> schema, signing helpers, and docs. No router, aggregator, or pipeline changes.
---
## 4. Architecture Overview
```
┌────────────────────┐ (1) inbound: POST signed message
│ External system │ ──────────────────────────────────────────────► ┌──────────────────────┐
│ (LangBot Space, │ POST /bots/<bot_uuid> │ LangBot │
│ CRM, web app …) │ X-LB-Signature, X-LB-Timestamp │ │
│ │ { session_id, message:[...] } │ HttpBotAdapter │
│ - callback server │ ◄────────────────────────────────────────────── │ (platform/sources) │
│ (receives │ (4) outbound: POST signed reply(s) │ │
│ replies) │ POST <callback_url> │ pipeline + aggregator│
└────────────────────┘ X-LB-Signature, X-LB-Timestamp └──────────────────────┘
{ session_id, sequence, is_final,
message:[...] } (sent 1..M times)
```
- The adapter is **stateless across requests** at the HTTP layer; session
continuity is carried by `session_id` and resolved by LangBot's normal
session manager.
- **Inbound** and **outbound** are **independent HTTP exchanges**. LangBot does
not answer the inbound POST with the pipeline result; it `202 Accepts` it and
later POSTs the reply(s) to the callback URL. This is what makes 1→M natural.
---
## 5. Configuration Schema (`http_bot.yaml`)
Follows the existing `MessagePlatformAdapter` manifest convention (cf.
`slack.yaml`). Fields:
| field | type | required | purpose |
|---|---|---|---|
| `inbound_secret` | string (secret) | yes | HMAC key the **caller** uses to sign inbound POSTs; LangBot verifies. |
| `callback_url` | string (url) | no* | Where LangBot POSTs replies. *Optional if the caller supplies `callback_url` per-message (see §6.1); a static default lives here. |
| `outbound_secret` | string (secret) | no | HMAC key LangBot uses to sign outbound callbacks; caller verifies. Defaults to `inbound_secret` if empty. |
| `default_session_type` | enum `person`/`group` | no | Default when a message omits `session_type`. Default `person`. |
| `signature_required` | bool | no | If `false`, skip inbound signature check (dev only; logs a warning). Default `true`. |
| `callback_timeout` | int (seconds) | no | Per-callback HTTP timeout. Default `15`. |
| `callback_max_retries` | int | no | Retries on 5xx/timeout with backoff. Default `3`. |
| `webhook_url` | webhook-url (display) | — | Read-only field rendering the inbound URL `…/bots/<bot_uuid>` for copy-paste, like other webhook adapters. |
Manifest sketch (i18n labels elided for brevity):
```yaml
apiVersion: v1
kind: MessagePlatformAdapter
metadata:
name: http_bot
label: { en_US: "HTTP Bot", zh_Hans: "HTTP 通用接入" }
description:
en_US: "Integrate any backend over plain HTTP. Push messages in, receive replies on a callback URL. Server-to-server, no long-lived connection."
zh_Hans: "通过 HTTP 接入任意后端系统。推入消息、在回调地址接收回复。面向服务间集成,无需长连接。"
icon: http_bot.svg
spec:
categories: [popular, global]
help_links:
zh: https://docs.langbot.app/zh/platforms/http-bot
en: https://docs.langbot.app/en/platforms/http-bot
config:
- { name: inbound_secret, type: string, required: true, default: "" }
- { name: callback_url, type: string, required: false, default: "" }
- { name: outbound_secret, type: string, required: false, default: "" }
- { name: default_session_type, type: select, required: false, default: "person",
options: [person, group] }
- { name: signature_required, type: boolean, required: false, default: true }
- { name: callback_timeout, type: integer, required: false, default: 15 }
- { name: callback_max_retries, type: integer, required: false, default: 3 }
- { name: webhook_url, type: webhook-url, required: false, default: "" }
execution:
python:
path: ./http_bot.py
attr: HttpBotAdapter
```
---
## 6. The HTTP Contract (this is the DX surface)
### 6.1 Inbound — push a message into LangBot
```
POST /bots/{bot_uuid}
Content-Type: application/json
X-LB-Timestamp: 1718000000
X-LB-Signature: sha256=<hex hmac>
X-LB-Idempotency-Key: <uuid> # optional, dedup window
```
Body:
```jsonc
{
"session_id": "ticket-10293", // REQUIRED. Caller-defined. Maps 1:1 to a LangBot session.
"session_type": "person", // optional, "person" | "group"; default from config
"sender": { // optional metadata, surfaced to pipeline/plugins
"id": "user-5567",
"name": "Alice"
},
"message": [ // REQUIRED. A LangBot MessageChain (list of segments).
{ "type": "Plain", "text": "Export keeps failing on the dashboard." },
{ "type": "Image", "url": "https://.../screenshot.png" }
]
}
```
Response (LangBot does **not** block on the pipeline):
```jsonc
// 202 Accepted
{
"code": 0,
"msg": "accepted",
"data": {
"session_id": "ticket-10293",
"accepted_message_id": "in_01H....", // server-assigned id for this inbound message
"aggregating": true // true if buffered by the aggregator
}
}
```
**N→1 in practice.** Fire three POSTs with the same `session_id` inside the
aggregation window → the pipeline runs **once** with the three messages merged.
No special flag needed; this is the aggregator's default behaviour when enabled
on the pipeline.
### 6.2 Outbound — LangBot delivers replies to your callback
For each `reply_message` / `reply_message_chunk` the pipeline emits, LangBot
POSTs to `callback_url`:
```
POST {callback_url}
Content-Type: application/json
X-LB-Timestamp: 1718000001
X-LB-Signature: sha256=<hex hmac over body>
```
Body:
```jsonc
{
"session_id": "ticket-10293", // echoes the inbound session
"reply_to": "in_01H....", // the inbound message id this answers
"sequence": 1, // 1-based ordinal within this turn (for 1→M ordering)
"is_final": false, // false for intermediate/streamed parts
"stream": false, // true when this is a streamed chunk
"message": [
{ "type": "Plain", "text": "Looking into it — checking your export logs…" }
],
"timestamp": "2026-06-22T09:00:01Z"
}
```
**1→M in practice.** A turn that fires a function call then a final answer
produces e.g.:
```
POST callback → { sequence: 1, is_final: false, message: ["Checking logs…"] }
POST callback → { sequence: 2, is_final: false, message: ["Found 2 failed exports."] }
POST callback → { sequence: 3, is_final: true, message: ["Fixed. Try again now."] }
```
The caller stitches by `session_id` + `sequence`, and knows the turn is complete
when `is_final: true` arrives.
Your callback endpoint should return `200` quickly. A non-2xx triggers retry
with backoff (`callback_max_retries`).
### 6.3 Error envelope (inbound)
Consistent, machine-readable; never leak internals:
```jsonc
{ "code": 40101, "msg": "invalid signature", "data": null }
```
| HTTP | code | meaning |
|---|---|---|
| 202 | 0 | accepted |
| 400 | 40001 | malformed body / missing `session_id` or `message` |
| 401 | 40101 | bad/expired signature |
| 403 | 40301 | bot disabled |
| 404 | 40401 | bot_uuid not found / not an `http_bot` adapter |
| 409 | 40901 | duplicate idempotency key (already accepted) |
| 413 | 41301 | message too large |
| 500 | 50001 | internal error |
---
## 7. Signing scheme (both directions)
Symmetric, dependency-free HMAC-SHA256 — trivial to implement in any language.
```
signing_string = "{timestamp}.{raw_request_body}"
signature = "sha256=" + hex(HMAC_SHA256(secret, signing_string))
```
Verification rules:
- Reject if `|now - timestamp| > 300s` (replay window).
- Constant-time compare (`hmac.compare_digest`).
- Inbound verified with `inbound_secret`; outbound signed with
`outbound_secret` (falls back to `inbound_secret`).
- `signature_required: false` bypasses verification **and logs a warning**
intended only for local development behind a trusted network.
Reference (Python, ~6 lines):
```python
import hmac, hashlib, time
def sign(secret: str, body: bytes, ts: int | None = None) -> tuple[str, str]:
ts = ts or int(time.time())
mac = hmac.new(secret.encode(), f"{ts}.".encode() + body, hashlib.sha256)
return str(ts), "sha256=" + mac.hexdigest()
```
---
## 8. Delivery semantics & reliability
- **Inbound**: `202 Accepted` means *queued*, not *processed*. Use
`X-LB-Idempotency-Key` to make client retries safe (dedup window, e.g. 10 min).
- **Outbound**: **at-least-once**, best-effort. Retries on timeout/5xx with
exponential backoff up to `callback_max_retries`. Callbacks for one
`session_id` are delivered **in `sequence` order** (serialised per session);
across sessions they may interleave.
- **No persistence in v1**: if LangBot restarts mid-turn, in-flight callbacks
may be lost. Durable replay is deferred (see §13). Callers needing exactly-once
should dedup on `(session_id, reply_to, sequence)`.
- **Backpressure**: the adapter must not block the pipeline on slow callbacks —
outbound POSTs run on a per-session ordered queue with the configured timeout.
---
## 9. Optional: synchronous convenience mode (v1.1, behind a flag)
Some simple callers genuinely want "POST a message, get the reply in the HTTP
response" and don't care about streaming/multi-part. We can offer an **opt-in**
sync endpoint that internally waits for `is_final` and **collapses** all 1→M
parts into one array:
```
POST /bots/{bot_uuid}/sync → 200 { session_id, message: [ ...all parts concatenated... ] }
```
Implemented by attaching a per-request future that resolves on the final reply,
with a hard timeout. This is a **convenience wrapper** over the same machinery,
explicitly documented as lossy for streaming/ordering. Not in v1 core.
---
## 10. Adapter implementation sketch (`platform/sources/http_bot.py`)
Implements `AbstractMessagePlatformAdapter`. Key methods:
```python
class HttpBotAdapter(AbstractMessagePlatformAdapter):
listeners: dict = pydantic.Field(default_factory=dict, exclude=True)
# --- inbound -------------------------------------------------------
async def handle_unified_webhook(self, bot_uuid, path, request):
body = await request.get_body()
if self.config.get("signature_required", True):
if not self._verify(request, body):
return jsonify({"code": 40101, "msg": "invalid signature"}), 401
data = json.loads(body)
session_id = data["session_id"] # caller-defined identity
session_type = data.get("session_type", self.config.get("default_session_type", "person"))
chain = MessageChain.model_validate(data["message"])
event = self._build_event(session_type, session_id, data.get("sender"), chain)
# remember where to send replies for this session
self._callback_for[session_id] = data.get("callback_url") or self.config.get("callback_url")
# fire the registered listener → botmgr → msg_aggregator (N→1) → pipeline
if type(event) in self.listeners:
asyncio.create_task(self.listeners[type(event)](event, self))
return jsonify({"code": 0, "msg": "accepted",
"data": {"session_id": session_id, "accepted_message_id": event.message_id}}), 202
# --- outbound (called 1..M times per turn by the pipeline) ---------
async def reply_message(self, message_source, message, quote_origin=False):
return await self._post_callback(message_source, message, is_final=True, stream=False)
async def reply_message_chunk(self, message_source, bot_message, message,
quote_origin=False, is_final=False):
return await self._post_callback(message_source, message, is_final=is_final, stream=True)
async def is_stream_output_supported(self) -> bool:
return True
def register_listener(self, event_type, func): self.listeners[event_type] = func
def unregister_listener(self, event_type, func): self.listeners.pop(event_type, None)
async def run_async(self): pass # nothing to poll; purely webhook-driven
async def kill(self): pass
```
`_post_callback` resolves the session's callback URL, assigns the next
`sequence`, signs the body, and enqueues an ordered, retrying POST.
Session→callback mapping is kept in a small in-memory dict keyed by
`session_id` (acceptable for v1; a turn's callback URL is captured at inbound
time so replies always have a destination even if config later changes).
---
## 11. Security considerations
- **Inbound route is `AuthType.NONE`** at the framework level (same as all
webhook adapters) — the adapter **must** enforce HMAC itself. Default
`signature_required: true`.
- **Timestamp window** (±300s) + idempotency key blunt replay.
- **SSRF on callback_url**: validate scheme (`https` in prod), and consider an
allow-list / block of private CIDRs since LangBot initiates the POST. Document
this; enforce in code where feasible.
- **Secret storage**: secrets live in the bot's `adapter_config` like every
other adapter credential; surfaced as `type: string`/secret in the dashboard.
- **One secret per bot** in v1. Per-caller key rotation / multiple keys is a
future enhancement (§13).
---
## 12. Developer Experience (explicit deliverables)
The whole point of a standalone adapter is that **integrating is pleasant**. v1
ships:
1. **`docs/platforms/http-bot.md`** — task-oriented integration guide:
create the bot → copy inbound URL → set secret → stand up a callback
endpoint → send first message → handle 1→M.
2. **Copy-paste curl** for the first message (with a working signing one-liner).
3. **Reference clients** (≤50 LOC each) in `examples/http-bot/`:
`client.py` (push + a Flask/Quart callback receiver) and `client.ts`.
4. **OpenAPI fragment** `docs/http-bot-openapi.json` describing inbound +
callback shapes, so integrators can codegen.
5. **Local echo recipe**: a one-command callback server that prints every
reply, so a developer sees N→1 and 1→M working in under five minutes.
6. **Postman/Hoppscotch collection** (nice-to-have).
DX acceptance check: *a developer who has never seen LangBot can, from the docs
alone, push a message and observe a multi-part reply on their callback within
10 minutes.*
### Quickstart (curl)
```bash
BOT=https://your-langbot/bots/2f1c....
SECRET=supersecret
BODY='{"session_id":"ticket-10293","message":[{"type":"Plain","text":"hello"}]}'
TS=$(date +%s)
SIG="sha256=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -r | cut -d' ' -f1)"
curl -sS -X POST "$BOT" \
-H "Content-Type: application/json" \
-H "X-LB-Timestamp: $TS" \
-H "X-LB-Signature: $SIG" \
-d "$BODY"
```
---
## 13. Future work
- **Durable outbound queue** (persist + replay across restarts; exactly-once).
- **Per-caller API keys** with rotation and scopes (multi-tenant Space usage).
- **Sync convenience endpoint** (§9) once core is stable.
- **Server-Sent Events outbound option** for callers that *do* want a stream but
not a full duplex socket — single GET, server pushes chunks.
- **Dashboard "test console"** for `http_bot` (send a message, watch callbacks)
mirroring the existing WebSocket debug panel.
---
## 14. Rollout / task breakdown
| # | Task | Touches |
|---|---|---|
| 1 | `http_bot.yaml` manifest + icon | `platform/sources/` |
| 2 | `HttpBotAdapter` (inbound verify, event build, outbound queue) | `platform/sources/http_bot.py` |
| 3 | Signing helper module (shared) | `platform/sources/` or `utils/` |
| 4 | i18n strings (en/zh/ja) | adapter yaml + web locale |
| 5 | Integration docs `docs/platforms/http-bot.md` | `docs/` |
| 6 | OpenAPI fragment + reference clients | `docs/`, `examples/http-bot/` |
| 7 | Tests: signature verify, N→1 aggregation, 1→M ordering, retry | `tests/` |
| 8 | (opt) SSRF guard for callback_url | adapter |
No changes required to: the unified webhook router, the aggregator, the query
pool, or the pipeline. That is the design's main payoff.
---
## 15. Resolved decisions
1. **Callback URL trust****config-only.** The inbound message may not carry a
`callback_url`; replies always go to the bot-config URL. Closes the SSRF
vector where a leaked inbound secret could redirect replies.
2. **Session lifecycle****`POST /bots/<uuid>/reset`** (body `{session_id,
session_type?}`) drops the matching session from the session manager; the
next message starts a fresh conversation. Implemented via sub-path routing in
`handle_unified_webhook`.
3. **Group semantics** — for `session_type: group`, `session_id` is the group/
launcher id; `sender.id` (and optional `sender.group_name`) identify the
member. A Space ticket maps to one `session_id`.
4. **Backpressure** — bounded per-session outbound queue (maxlen 1000); on
overflow the oldest reply is dropped and a warning logged, so a persistently
down callback can never exhaust memory.
### Still open / deferred (see §13)
- Durable outbound queue (persist + replay across restarts).
- Per-caller API keys with rotation/scopes for multi-tenant Space usage.
- SSE outbound option and a dashboard test console.
+171
View File
@@ -0,0 +1,171 @@
# Valkey Search Vector Database Integration
This document describes how to use **Valkey Search** (the search/vector module bundled in
`valkey/valkey-bundle`) as the vector database backend for LangBot's knowledge base (RAG)
feature.
## What is Valkey Search?
**Valkey Search** is a module that adds vector similarity search and full-text search to
[Valkey](https://valkey.io/), the open-source, BSD-licensed in-memory data store forked from
Redis OSS. It is distributed in the `valkey/valkey-bundle` image alongside other modules
(JSON, Bloom, LDAP).
LangBot talks to Valkey through the official [`valkey-glide`](https://pypi.org/project/valkey-glide/)
client (Rust core + async Python wrapper), using its native `ft` (search) command namespace.
### Key Features
- **Vector search**: ANN via HNSW or exact via FLAT, with COSINE / L2 / IP distance metrics
- **Full-text search**: term, prefix and phrase matching over indexed text fields
- **Hybrid search**: a metadata/text filter pre-selects candidates, then KNN ranks them
- **In-memory speed**: vectors and documents are stored as Valkey HASH keys
- **Auth + TLS**: optional username/password and TLS for production (toB / SaaS) deployments
### Licensing
- Valkey core and the Search module are **BSD-3-Clause**.
- The `valkey-glide` client is **Apache-2.0**.
Both are compatible with LangBot.
## Installation
Valkey Search support is included automatically on Linux and macOS. The official `valkey-glide`
client does not currently publish a Windows package, so LangBot skips this optional dependency on
Windows; LangBot remains usable there, but the Valkey Search backend is unavailable. To install the
client manually on a supported platform:
```bash
pip install 'valkey-glide>=2.4.1,<3.0.0'
```
You also need a running Valkey server with the Search module loaded. The simplest way is the
bundled image:
```bash
# Run valkey-bundle (includes the Search module) on host port 6380
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
# (docker run ... works identically)
```
`valkey-bundle` ships multi-arch images (linux/amd64 + linux/arm64), so it runs on both CI
(x86_64) and Apple-silicon dev machines.
## Configuration
Valkey Search is **opt-in and disabled by default** — the default `vdb.use` stays `chroma`,
so existing single-process deployments are unaffected. To enable it, edit your `config.yaml`:
```yaml
vdb:
use: valkey_search
valkey_search:
host: 'localhost'
port: 6379 # use 6380 if you started the container as shown above
db: 0
password: '' # optional (ACL / requirepass) — never logged
username: '' # optional (ACL user)
tls: false # optional (toB / SaaS)
index_algorithm: 'HNSW' # HNSW | FLAT
distance_metric: 'COSINE' # COSINE | L2 | IP
request_timeout: 5000 # per-request timeout in ms
```
| Option | Default | Description |
|--------|---------|-------------|
| `host` | `localhost` | Valkey host |
| `port` | `6379` | Valkey port |
| `db` | `0` | Logical database id |
| `password` | `''` | Optional auth password (empty = no auth). Never logged. |
| `username` | `''` | Optional ACL username. Configuring a username without a password fails closed (raises) rather than connecting unauthenticated. |
| `tls` | `false` | Enable TLS for the connection |
| `index_algorithm` | `HNSW` | `HNSW` (approximate) or `FLAT` (exact) |
| `distance_metric` | `COSINE` | `COSINE`, `L2`, or `IP` |
| `request_timeout` | `5000` | Per-request timeout in milliseconds. The valkey-glide default (250ms) is too low for vector KNN under load; raise it further for remote/cross-AZ Valkey. |
### Connection behavior
The backend uses a **lazy** connection (`lazy_connect=True`): the client is created on first
use and the connection is deferred to the first command. A misconfigured or unreachable Valkey
server therefore does **not** block LangBot from booting — knowledge-base operations will error
at call time instead, and you can recover by switching `vdb.use` back to another backend.
The connection sets a fixed `client_name` of `langbot_vector_client` so it is identifiable in
`CLIENT LIST` and monitoring dashboards.
## Supported search types
| Type | Behavior |
|------|----------|
| `vector` | Pure KNN over the embedding field |
| `full_text` | Term/phrase match over the indexed `document` text field |
| `hybrid` | Metadata/text filter **pre-selects** candidates, then KNN ranks them |
### ⚠️ Important: `vector_weight` is NOT honored
Valkey Search hybrid queries follow a **filter-then-KNN** model: the filter (and/or full-text
clause) narrows the candidate set, and the KNN stage ranks the survivors by vector distance.
There is **no native weighted score fusion** (unlike, e.g., SeekDB's RRF boost).
For interface compatibility the backend still accepts a `vector_weight` argument, but it is
**ignored** — passing different weights does not change result ordering. The first time a
non-default weight is supplied, the backend logs a one-time warning.
If weighted hybrid ranking is needed in the future, it can be added **application-side** (run
vector KNN and full-text search separately and blend the scores). That is intentionally out of
scope for this integration.
## Metadata & filtering
Documents are stored as Valkey HASH keys under the prefix `kb:{collection}:{id}` with fields:
- `vector` — the embedding, packed as little-endian FLOAT32
- `document` — the raw text (indexed as TEXT for full-text/hybrid search)
- `file_id` — promoted to an indexed TAG field so it is filterable
- `metadata_json` — the full metadata dict, preserved verbatim as JSON
Only **indexed** fields are filterable. Currently that is `file_id`. Filters referencing
non-indexed metadata keys are dropped with a warning (the same pragmatism used by the Milvus
and pgvector backends). All other metadata still round-trips intact via `metadata_json`.
Supported filter operators (canonical Chroma-style `where` syntax): `$eq`, `$ne`, `$gt`,
`$gte`, `$lt`, `$lte`, `$in`, `$nin`. Multiple top-level keys are AND-ed.
## Testing
Unit tests (filter mapping, float32 packing, reply parsing, import guard) run in the fast lane
with no server:
```bash
uv run pytest tests/unit_tests/vector/test_valkey_search_filter.py -q
```
Integration tests are **slow-gated** on `TEST_VALKEY_URL` and require a running server:
```bash
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
TEST_VALKEY_URL=valkey://localhost:6380 \
uv run pytest tests/integration/vector/test_valkey_search.py -m slow -q
```
The default upstream fast CI lane (`-m "not slow"`) skips these, matching the existing
PostgreSQL migration-test precedent.
## Troubleshooting
| Symptom | Cause / fix |
|---------|-------------|
| Tests skip with "Valkey Search module not available" | The server is plain Valkey without the Search module. Use the `valkey/valkey-bundle` image. |
| `ConnectionError` at call time | Check `host`/`port`/auth; remember `lazy_connect` defers errors to first use. |
| Empty search results right after insert | The Search indexer is asynchronous; results become visible within a short delay. The integration tests poll/retry to account for this. |
| Hybrid ranking ignores `vector_weight` | Expected — see the caveat above. |
## Production considerations
- **Cluster mode**: Valkey Search in cluster mode uses an additional coordination port. This
integration targets standalone mode; cluster support is a future consideration.
- **Persistence**: configure Valkey RDB/AOF persistence if the knowledge base must survive
restarts; otherwise an in-memory store is ephemeral.
- **Security**: set `password`/`username` and `tls: true` for any non-local deployment.
Credentials are never written to logs.
Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

+198
View File
@@ -0,0 +1,198 @@
{
"openapi": "3.0.3",
"info": {
"title": "LangBot HTTP Bot Adapter",
"version": "1.0.0",
"description": "Server-to-server HTTP integration for a LangBot pipeline. Inbound messages are POSTed to the unified webhook route; replies are delivered to a configured callback URL (one POST per reply part). All requests are HMAC-SHA256 signed. See docs/platforms/http-bot.md."
},
"paths": {
"/bots/{bot_uuid}": {
"post": {
"summary": "Push a message into the pipeline (fire-and-collect)",
"description": "Returns 202 immediately. Replies arrive asynchronously on the configured callback URL. Reuse the same session_id within the aggregation window to merge multiple messages into one turn (N->1).",
"parameters": [
{ "$ref": "#/components/parameters/BotUuid" },
{ "$ref": "#/components/parameters/Timestamp" },
{ "$ref": "#/components/parameters/Signature" },
{ "$ref": "#/components/parameters/Idempotency" }
],
"requestBody": {
"required": true,
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/InboundMessage" } } }
},
"responses": {
"202": {
"description": "Accepted (queued for the pipeline)",
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/AcceptedResponse" } } }
},
"400": { "$ref": "#/components/responses/Error" },
"401": { "$ref": "#/components/responses/Error" },
"409": { "$ref": "#/components/responses/Error" },
"413": { "$ref": "#/components/responses/Error" }
}
}
},
"/bots/{bot_uuid}/sync": {
"post": {
"summary": "Push a message and wait for the collapsed reply",
"description": "Blocking convenience mode. Waits for is_final and returns all reply parts collapsed into one array. Lossy (no sequence/streaming). One in-flight sync per session_id.",
"parameters": [
{ "$ref": "#/components/parameters/BotUuid" },
{ "$ref": "#/components/parameters/Timestamp" },
{ "$ref": "#/components/parameters/Signature" }
],
"requestBody": {
"required": true,
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/InboundMessage" } } }
},
"responses": {
"200": {
"description": "The collapsed reply",
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/SyncResponse" } } }
},
"400": { "$ref": "#/components/responses/Error" },
"401": { "$ref": "#/components/responses/Error" }
}
}
},
"/bots/{bot_uuid}/reset": {
"post": {
"summary": "Reset a session's conversation",
"parameters": [
{ "$ref": "#/components/parameters/BotUuid" },
{ "$ref": "#/components/parameters/Timestamp" },
{ "$ref": "#/components/parameters/Signature" }
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"required": ["session_id"],
"properties": {
"session_id": { "type": "string" },
"session_type": { "type": "string", "enum": ["person", "group"] }
}
}
}
}
},
"responses": {
"200": { "description": "Reset done" },
"400": { "$ref": "#/components/responses/Error" },
"401": { "$ref": "#/components/responses/Error" }
}
}
}
},
"components": {
"parameters": {
"BotUuid": {
"name": "bot_uuid", "in": "path", "required": true,
"schema": { "type": "string", "format": "uuid" }
},
"Timestamp": {
"name": "X-LB-Timestamp", "in": "header", "required": true,
"description": "Unix seconds; rejected if more than +/-300s from server time.",
"schema": { "type": "string" }
},
"Signature": {
"name": "X-LB-Signature", "in": "header", "required": true,
"description": "sha256=<hex> of HMAC-SHA256(secret, \"{timestamp}.\" + raw_body).",
"schema": { "type": "string" }
},
"Idempotency": {
"name": "X-LB-Idempotency-Key", "in": "header", "required": false,
"description": "Dedup key; a repeat within the dedup window returns 409.",
"schema": { "type": "string" }
}
},
"schemas": {
"Segment": {
"type": "object",
"required": ["type"],
"properties": {
"type": { "type": "string", "enum": ["Plain", "Image", "Voice", "File", "At", "Quote"] },
"text": { "type": "string", "description": "For type=Plain." },
"url": { "type": "string", "description": "For media types." },
"base64": { "type": "string", "description": "For media types (data URI or raw base64)." }
}
},
"InboundMessage": {
"type": "object",
"required": ["session_id", "message"],
"properties": {
"session_id": { "type": "string", "description": "Caller-defined; maps 1:1 to a LangBot session." },
"session_type": { "type": "string", "enum": ["person", "group"], "default": "person" },
"sender": {
"type": "object",
"properties": {
"id": { "type": "string" },
"name": { "type": "string" },
"group_name": { "type": "string", "description": "For session_type=group." }
}
},
"message": { "type": "array", "items": { "$ref": "#/components/schemas/Segment" } }
}
},
"AcceptedResponse": {
"type": "object",
"properties": {
"code": { "type": "integer", "example": 0 },
"msg": { "type": "string", "example": "accepted" },
"data": {
"type": "object",
"properties": {
"session_id": { "type": "string" },
"accepted_message_id": { "type": "string", "example": "in_01H..." },
"aggregating": { "type": "boolean" }
}
}
}
},
"SyncResponse": {
"type": "object",
"properties": {
"code": { "type": "integer", "example": 0 },
"msg": { "type": "string", "example": "ok" },
"data": {
"type": "object",
"properties": {
"session_id": { "type": "string" },
"reply_to": { "type": "string" },
"message": { "type": "array", "items": { "$ref": "#/components/schemas/Segment" } }
}
}
}
},
"Callback": {
"type": "object",
"description": "Delivered by LangBot to your callback_url, one POST per reply part. Signed with the outbound secret.",
"properties": {
"session_id": { "type": "string" },
"reply_to": { "type": "string", "description": "The accepted_message_id this answers." },
"sequence": { "type": "integer", "description": "1-based ordinal within the turn." },
"is_final": { "type": "boolean", "description": "True on the last part of the turn." },
"stream": { "type": "boolean" },
"message": { "type": "array", "items": { "$ref": "#/components/schemas/Segment" } },
"timestamp": { "type": "string", "format": "date-time" }
}
},
"ErrorEnvelope": {
"type": "object",
"properties": {
"code": { "type": "integer", "example": 40101 },
"msg": { "type": "string", "example": "invalid signature: signature_mismatch" },
"data": { "nullable": true }
}
}
},
"responses": {
"Error": {
"description": "Error envelope",
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorEnvelope" } } }
}
}
}
}
@@ -0,0 +1,292 @@
# Multi-tenant implementation checklist
This checklist turns the Workspace architecture into implementation and
verification gates. Exact commands and observed results are recorded in the
[verification report](./verification-report.md).
## Scope guard
- [x] LangBot uses branch feat/multi-tenants.
- [x] langbot-plugin-sdk uses branch feat/multi-tenants.
- [x] langbot-space implements the greenfield Cloud v2 modular-monolith control plane without extending the legacy per-account Pod topology; the old Pod UI remains available when Cloud v2 is disabled and only retained Pods appear in the v2 view.
- [x] Unrelated untracked files in either repository remain untouched.
- [x] Open-source startup cannot enable SaaS multi-workspace through edition flags or unsigned configuration.
## SaaS activation gates
These items intentionally remain incomplete. Some require additional Core
transaction/cutover primitives and others require the closed Control Plane or
deployment. The feature branch delivers the Core isolation kernel, not the
closed SaaS product or a production Cloud v2 deployment. Checked implementation
items later in this document do not supersede these gates.
- [x] The closed Control Plane owns the global Account, Workspace, Membership, and Invitation directory.
- [ ] The closed Control Plane execution-ownership module issues monotonic generations and owner leases for projected Workspaces.
- [x] Core verifies a signed `InstanceManifest` before the closed bootstrap can inject `CloudWorkspacePolicy`.
- [ ] Tenant database writes hold a generation-aware shared transaction fence through commit, while execution-owner cutovers take the exclusive fence.
- [ ] Business writes and non-transactional side effects use a generation-stamped outbox or equivalent publish fence.
- [ ] Durable object references survive an execution-generation change through stable published keys or an explicitly atomic key/reference migration.
- [ ] The SaaS runtime pools enforce tenant-safe egress and SSRF controls for Webhooks, providers, MCP servers, and every tenant-configurable outbound URL.
- [ ] Entitlement checks, usage aggregation, and subscription lifecycle are implemented in the closed Control Plane; production activation still requires provider callback amount/currency/session/expiry binding inside the locked fulfillment transaction.
- [ ] Account registration persists a `new_api.provision_account` outbox item with the Account and personal Workspace, and an in-process reconciler provisions New API idempotently after commit.
- [ ] EPay and Stripe callbacks bind provider identity, amount, currency, channel/session, payment status, and expiry to the locked order before entitlement fulfillment.
- [ ] OAuth state and directory projection use an atomic shared store suitable for horizontally scaled SaaS services.
- [ ] A greenfield Cloud v2 deployment is designed and validated independently of the legacy Space deployment scheme.
- [ ] The Plugin Runtime shared profile refuses to run without delegated cgroup v2 CPU, memory-plus-swap, and PID limits, all verified in a real Linux container; production tenant-safe egress remains incomplete.
- [x] The Plugin Runtime Supervisor automatically restores an unexpectedly exited enabled worker with bounded per-installation backoff.
- [ ] Jitter, a global restart concurrency limit, and a Runtime-level circuit breaker prevent a systemic failure from creating a cross-tenant restart storm.
- [ ] Until authenticated Runtime takeover or an owner lease/fence exists, the M0 deployment rolls Core and Plugin Runtime together and forbids an independent Core-only rollout.
- [ ] Plugin installation data has an operator-owned hard disk quota provider that atomically rejects writes over the limit; directory scans are not accepted as enforcement.
- [ ] The Box deployment provides an operator-owned quota provider that proves hard byte and inode limits for Workspace, Skill, root, tmp, and home storage.
- [ ] Core and Box Runtime mount the same durable volume and pass the authenticated marker challenge during startup and reconnect.
- [ ] Production provisions distinct migrator/runtime credentials and runs the implemented same-host/port/database release command as a one-shot Job, with tested orchestration retry, backup, and rollback procedures.
- [ ] Production PostgreSQL uses a dedicated cluster/endpoint, or a tested HBA/proxy policy proves the cluster-wide runtime credential can connect only to the target business database.
- [ ] Any future direct-migrator/pooler-runtime endpoint split is admitted only by a migrator-owned, runtime-read-only database cluster identity that the runtime role cannot spoof.
- [ ] Legacy pgvector migration failure and retry integration paths prove exact source-table RLS/FORCE restoration; the non-superuser, non-`BYPASSRLS` success path is already covered below.
- [ ] Multi-workspace is enabled in SaaS only after all closed Control Plane, deployment, and security gates pass.
## 1. Persistence foundation
### Account and directory
- [x] User has a stable, unique account UUID and explicit status.
- [x] Existing email and password behavior remains compatible during migration.
- [x] Workspace table represents the instance-local tenant.
- [x] WorkspaceMembership has a unique Workspace and Account pair.
- [x] WorkspaceInvitation stores only a token hash and supports expiry, revoke, and one-time accept.
- [x] WorkspaceExecutionState stores generation, state, source, and write fence.
- [x] OSS initialization creates exactly one Workspace and one owner membership atomically.
- [x] OSS refuses a second Workspace while allowing multiple members.
### Migration
- [x] Alembic migration upgrades SQLite.
- [x] Alembic migration upgrades PostgreSQL.
- [x] Existing first user becomes owner of the default Workspace.
- [x] Existing tenant resources are backfilled with the default Workspace UUID.
- [x] SQLite destructive boundaries create verified, revision-aware backups and atomically restore after failure.
- [x] Migration can resume safely after interruption.
- [x] New installs and upgraded installs produce the same tenancy-kernel schema.
- [x] The first Cloud release pins migrator and runtime sessions to `public` with `current_schemas(false)` containing only that business schema; runtime-role/database `search_path` overrides are rejected.
- [x] Cloud sessions require `session_replication_role=origin`, `row_security=on`, and `lo_compat_privileges=off`; every persistent `pg_db_role_setting` applicable to the runtime role or current business database is rejected.
- [x] The release migrator grants the runtime role exact business-table DML, `alembic_version` read-only access, and business-sequence `USAGE/SELECT`, with no `WITH GRANT OPTION` or non-business object grants.
- [x] Every Cloud runtime startup revalidates the login role, current user, schema, effective/direct ACLs, ownership, memberships in all directions, column ACLs, routines, extensions, foreign objects, parameter ACLs, and other-schema access before serving traffic.
- [x] The business database requires `vector`, permits only `plpgsql`/`vector` extensions, forbids runtime extension ownership, and contains no foreign data wrapper, foreign server, or user mapping.
- [x] The runtime role and `PUBLIC` have no explicit routine or parameter ACL; the runtime owns no routine and cannot effectively execute any `SECURITY DEFINER` routine, including extension-owned routines.
- [x] PostgreSQL's default `PUBLIC TEMP` is documented and tested as a dedicated-business-database v1 compatibility exception; the migrator never grants `TEMP` directly to the runtime role.
- [x] Legacy pgvector migration succeeds as a non-superuser, non-`BYPASSRLS` source-table owner and restores mixed source-table RLS/FORCE states exactly.
- [ ] Legacy pgvector migration still needs explicit failure-and-retry integration coverage before SaaS activation.
### Runtime transaction enforcement
- [x] Each tenant UoW owns one task, root transaction, database bind, and transaction-local scope.
- [x] A scoped Session and every captured bound method become permanently unusable when the owning UoW exits.
- [x] Public transaction/session control, raw/textual SQL, connection/bind escape, nested transactions, execution/loader options, foreign binds, live results, unapproved functions/operators/casts/types, `INSERT FROM SELECT`, hidden `ON CONFLICT` and batch-value expressions, forced-unquoted identifiers, and custom AST/compiler nodes fail closed and make the UoW rollback-only.
- [x] ORM `SessionEvents` fail before a registered callback can receive the synchronous Session or transaction connection; rollback cleanup cannot execute the rejected listener.
- [x] ORM flush, implicit autoflush, and commit reject SQL expressions assigned to mapped attributes before compilation.
- [x] Tenant relationship loading uses eager loading or explicit async `refresh`; synchronous object-session access and `AsyncAttrs.awaitable_attrs` are not supported tenant APIs.
- [x] The UoW guard is documented as a trusted-Core misuse boundary rather than an in-process Python sandbox; mapped metadata/compiler registration is trusted, plugins remain out of process, and SQLAlchemy upgrades must rerun the private-container regression suite.
## 2. Authentication and authorization
### Identity
- [x] JWT sub uses account UUID, with a bounded compatibility path for legacy email tokens.
- [x] Disabled or deleted accounts cannot authenticate.
- [x] Local password and Space-linked account flows support more than one local Account.
- [x] Public registration closes after initialization by default.
- [x] Invitation registration works without requiring SMTP.
- [x] An unknown Space OAuth subject cannot claim an existing Account by email; explicit account-bound binding is required.
### Request context
- [x] PrincipalContext identifies Account, API Key, or trusted runtime principal.
- [x] WorkspaceContext contains Workspace, Membership, role, permissions, and revision.
- [x] RequestContext contains instance UUID, Workspace context, auth type, request ID, and generation.
- [x] ExecutionContext propagates Workspace and generation to runtime work.
- [x] SaaS-style requests never fall back to the first or most recent Workspace.
- [x] OSS may resolve the single Workspace when the selector is omitted.
- [x] Account-token bootstrap can list only the authenticated Account's active memberships before a Workspace selector exists.
### Fixed RBAC
- [x] owner, admin, developer, operator, and viewer permissions match the architecture matrix.
- [x] Invitation cannot grant owner.
- [x] The last owner cannot be removed or demoted.
- [x] Cross-Workspace resources return 404.
- [x] Same-Workspace permission failures return 403.
## 3. Workspace and member APIs
- [x] GET /api/v1/workspaces returns the OSS singleton Workspace.
- [x] POST /api/v1/workspaces returns edition_limit in OSS.
- [x] Current Workspace endpoint returns the authenticated Membership.
- [x] Member list is permission scoped.
- [x] Invitation create, revoke, inspect, and accept are atomic.
- [x] Member role update and removal enforce owner rules.
- [x] Invitation tokens travel in a request body and are redacted from logs.
- [x] Relevant MCP tools and in-repo skills are updated with the same contract.
## 4. Tenant-scoped persistence and services
Each row type must have a non-null Workspace UUID, scoped indexes, scoped uniqueness, and scoped CRUD tests.
- [x] Bots and bot admins.
- [x] Legacy pipelines and pipeline run records.
- [x] Model providers.
- [x] LLM models.
- [x] Embedding models.
- [x] Rerank models.
- [x] Plugin installations, settings, and configuration.
- [x] MCP servers and resource preferences.
- [x] Knowledge bases, files, and chunks.
- [x] Vector collections and handles.
- [x] Monitoring messages, calls, sessions, errors, embeddings, and feedback.
- [x] API keys and scopes.
- [x] Webhooks and public route resolution.
- [x] Binary storage and Workspace storage.
- [x] Workspace metadata, separated from system metadata.
### Service and API rules
- [x] Every tenant Service receives RequestContext or an explicit Workspace UUID.
- [x] No tenant Service treats context None as global access.
- [x] Every applicable get, list, create, update, delete, copy, export, and bulk operation is scoped.
- [x] Parent-child references use the same Workspace.
- [x] API Key authentication derives Workspace from the key, not a header.
- [x] Webhook and Bot public routes derive Workspace from a trusted resource.
- [x] Background jobs carry Workspace and generation explicitly.
## 5. Runtime isolation
### Core runtime
- [x] RuntimeBot carries Workspace UUID and execution generation (currently stored in the compatibility field `placement_generation`).
- [x] RuntimePipeline carries Workspace UUID and execution generation (currently stored in the compatibility field `placement_generation`).
- [x] Query and Event carry Workspace UUID without making it an authorization source.
- [x] Session key includes Workspace UUID, Bot UUID, launcher type, and launcher ID.
- [x] QueryPool and manager indexes cannot collide across Workspaces.
- [x] Query and aggregation cache keys and locks include Workspace UUID.
- [x] Runtime transports, cached results, object operations, and long-lived tasks revalidate WorkspaceExecutionState generation at side-effect boundaries.
- [ ] Ordinary tenant database writes hold the generation fence in the same transaction until commit; this remains a SaaS activation gate.
### Plugin
- [x] Plugin installation and configuration are Workspace scoped.
- [x] Runtime control actions carry trusted Workspace binding and execution generation (wire-compatible as `placement_generation`).
- [x] The Plugin Runtime supervisor is instance-scoped and intentionally serves multiple Workspaces.
- [x] Every plugin process is bound to exactly one Workspace, installation, generation, revision, and verified artifact digest.
- [x] Same-digest plugin code may be cached once, while worker processes and writable data remain isolated.
- [x] Same-digest plugin dependencies are prepared once in a Runtime-owned immutable environment and mounted read-only into each isolated worker; dependency failure is surfaced before launch and recorded per installation without blocking other desired-state recovery.
- [x] Host API derives Workspace from the connection, installation, and trusted action context, not plugin input.
- [x] Plugin get_bots, models, tools, vector, RAG, configuration, and messaging calls are scoped.
- [x] Plugin Workspace storage no longer uses owner default.
- [x] Plugin page APIs check Membership and installation ownership.
- [x] Local plugin launches use short-lived, one-use registration capabilities bound to manifest identity.
### MCP, RAG, and Box
- [x] MCP runtime key contains instance UUID, Workspace UUID, execution generation, and server UUID.
- [x] Same-named MCP servers in two Workspaces do not share sessions.
- [x] Pipeline cannot reference another Workspace's MCP resource.
- [x] RAG collection names and handles are server-derived and Workspace scoped.
- [x] Legacy global vector migration is available only to the local OSS singleton Workspace.
- [x] Object storage paths include instance, Workspace, and execution generation for the fixed-generation OSS runtime.
- [x] Object storage revalidates generation before touching a provider or resolving an opaque key.
- [ ] Cloud cutover uses generation-scoped staging plus stable published object references, rather than making the staging generation the durable identity.
- [x] Box persistent and ephemeral namespaces include the required instance, Workspace, and generation scope.
- [x] Same-named Box sessions and processes cannot collide across Workspaces or execution generations.
- [x] Box relay and process I/O reject or retire stale generations.
- [x] External paths and privileged mounts cannot be supplied by an untrusted plugin.
- [x] Cloud attachment host I/O uses query UUIDs and link-free dirfd operations with bounded inode traversal.
- [x] Cloud Skill package paths are Runtime-owned, Workspace-scoped, read-only mounts; Python env/cache stays tenant-writable.
- [x] Skill ZIP preview/install rejects path escape, links, non-regular files, duplicate entries, excessive compression ratio, entry count, per-file size, and total size.
- [x] Cloud Box code paths and automated tests require the authenticated marker challenge before startup or reconnect can proceed.
- [x] Cloud Box readiness fails until hard Workspace, Skill, ephemeral-storage, and inode quota capabilities are available.
## 6. SDK and protocol
- [x] Public Query, Event, Session, and context entities carry backward-compatible Workspace data.
- [x] Action RPC request models carry trusted Workspace binding where required.
- [x] Action enums and callers remain consistent.
- [x] Old plugins continue to deserialize compatible events.
- [x] Plugins cannot select an arbitrary Workspace through a Host API argument.
- [x] Runtime storage uses the bound Workspace UUID.
- [x] SDK API tests pass.
- [x] Runtime tests pass.
- [x] Action consistency script passes.
## 7. Frontend
- [x] Every browser tenant API request carries the current Workspace selector after bootstrap.
- [x] OSS automatically selects the singleton Workspace.
- [x] OSS does not show Create Workspace or a misleading switcher.
- [x] Workspace settings show current Workspace information.
- [x] Members page lists roles and permissions.
- [x] Invitation creation shows a one-time link when SMTP is unavailable.
- [x] Invitation acceptance supports a signed-out user flow.
- [x] Role controls are hidden or disabled consistently with backend permissions.
- [x] Switching accounts clears stale Workspace query cache and local state.
- [x] User-facing strings support en_US, zh_Hans, and ja_JP.
## 8. Automated verification
### Persistence and authorization
- [x] SQLite fresh install.
- [x] SQLite upgrade from pre-tenant schema, including verified failure recovery.
- [x] PostgreSQL fresh install.
- [x] PostgreSQL upgrade from pre-tenant schema.
- [x] All fixed roles have positive and negative permission-matrix tests.
- [x] Concurrent invitation acceptance creates one Membership.
- [x] Concurrent owner changes never leave zero owners.
### Cross-tenant isolation
- [x] Two Workspaces are created through a test-only policy.
- [x] Applicable resource operations and parent-child references have cross-Workspace negative coverage.
- [x] Resource UUID guessing cannot cross Workspace.
- [x] API Key cannot cross Workspace.
- [x] Plugin cannot enumerate or invoke another Workspace's resources.
- [x] Sessions, caches, locks, MCP, RAG, Box, storage, and monitoring do not collide.
- [x] Background jobs cannot execute without an explicit Workspace and execution generation.
### Security and revocation
- [x] Space login and binding use purpose-bound, one-time opaque OAuth state; caller-supplied state is rejected.
- [x] OAuth redirects trust only server-configured WebUI or webhook origins, never request `Host` or `Origin` headers.
- [x] Dashboard WebSockets revalidate authentication, Membership, resource, permission, and generation per message.
- [x] Public embed WebSockets re-resolve Bot availability and execution binding per message.
- [x] Runtime, storage, Plugin Runtime, MCP, RAG, and Box reject a stale execution generation.
- [x] Unhandled API and webhook failures return a generic error plus request ID without exception text.
- [x] URL user information and sensitive query parameters are redacted before configuration is serialized or logged.
### Regression
- [x] LangBot unit tests pass.
- [x] LangBot integration tests pass.
- [x] Frontend lint completes without errors and the production build passes.
- [x] SDK focused and full relevant tests pass.
- [x] LangBot is pinned to the exact pushed SDK commit and cross-repo tests pass against that revision.
## 9. Real browser E2E
- [x] Start from a clean local data directory.
- [x] First user initializes the singleton Workspace as owner.
- [x] Owner creates an invitation link.
- [x] A second signed-out browser identity accepts the invitation and registers.
- [x] owner, admin, developer, operator, and viewer UI permissions match backend enforcement.
- [x] Direct API calls cannot bypass hidden controls.
- [x] Account switch does not expose prior account or Workspace data.
- [x] Refresh and a new browser tab recover the correct Workspace safely.
- [x] OSS rejects a second Workspace with `edition_limit`; same-name and same-identifier isolation is covered by the test-only multi-Workspace policy because OSS deliberately has no multi-Workspace browser surface.
- [x] Explicit error states are visible for expired, revoked, reused, and email-mismatched invitations.
## 10. Completion evidence
- [x] LangBot and SDK branch refs are recorded in the verification report.
- [x] Space contains the closed adapter package and Cloud v2 control plane, billing, migration, and Workspace UI changes; unrelated pre-existing files remain unstaged.
- [x] Migration output is captured for SQLite and PostgreSQL.
- [x] Test commands and results are recorded.
- [x] Browser E2E actions and observed results are recorded.
- [x] No remaining tenant table, global Service query, owner default, or unscoped runtime key is found by the final audit.
@@ -0,0 +1,272 @@
# Multi-tenant implementation decisions
This log records implementation choices made while delivering the Workspace architecture. It is intended to make trade-offs auditable without interrupting implementation for routine decisions.
> Architecture decisions, activation gates, and still-open follow-ups are tracked in
> [pending-architecture-decisions.md](./pending-architecture-decisions.md). Sections marked as decided there are authoritative;
> this file records the concrete implementation choices and compatibility names used to realize them.
## 2026-07-18
### OSS remains a singleton Workspace with multiple Accounts
- Decision: Community builds create exactly one Workspace per LangBot instance and allow multiple Accounts through invitations.
- Reason: This preserves a simple self-hosted deployment while making authorization and ownership explicit. Creating a second Workspace is an edition error, not a hidden fallback.
- SaaS boundary: Multi-Workspace directory, execution ownership, entitlement, and billing are the responsibility of a separate closed SaaS Control Plane. Core consumes a validated projection and remains the final isolation and authorization enforcement point; it does not become the SaaS system of record or billing engine.
- Deployment boundary: Cloud v2 is a greenfield deployment design. The previous per-account instance/pod scheme is not migrated or extended and remains available only for existing subscriptions. Existing OAuth, marketplace, and payment rails are reused through explicit adapters where they still fit; new Workspace, subscription, entitlement, directory, and usage modules live alongside the legacy Pod flow in `langbot-space`.
### Workspace selection is trusted only after authentication
- Decision: Browser requests carry `X-Workspace-Id`, but the server resolves it against the authenticated Account membership. API keys, public Bot routes, webhooks, jobs, and plugin calls derive Workspace from their trusted owning resource or binding instead of trusting the header.
- Reason: A selector is routing input, not authorization evidence.
- Compatibility: Community builds may select the singleton Workspace when the header is omitted. A multi-Workspace-capable build must reject an omitted selector.
### Stable Account UUID is the token subject
- Decision: New JWTs use the stable Account UUID as `sub`; a bounded compatibility path accepts legacy email-subject tokens and rotates them when checked.
- Reason: Email can change and therefore cannot be a durable authorization identity.
### Fixed roles are authoritative in Core
- Decision: `owner`, `admin`, `developer`, `operator`, and `viewer` map to a fixed permission matrix in LangBot Core. The last owner cannot be removed or demoted, and invitations cannot create an owner directly.
- Reason: Core must remain the final authorization boundary in both OSS and SaaS deployments.
### Cross-Workspace access is indistinguishable from absence
- Decision: Resource lookups always include Workspace UUID. A guessed UUID belonging to another Workspace returns 404; a visible resource with insufficient same-Workspace permission returns 403.
- Reason: This avoids leaking resource existence across tenants while preserving actionable same-tenant errors.
### Plugin Runtime is shared; every plugin process is single-Workspace
- Decision: One instance-scoped Plugin Runtime control plane serves all Workspaces in the logical LangBot instance. Each running plugin installation has its own nsjail worker with an immutable binding containing `instance_uuid`, `workspace_uuid`, `execution_generation` (stored as the compatibility field `placement_generation` until the schema rename), `installation_uuid`, `runtime_revision`, and verified artifact digest; enabled-resident is the desired semantic. A worker never routes actions for another Workspace or installation, and plugin-supplied scope fields are stripped.
- Isolation: Plugin code is mounted read-only. Home, tmp, and data paths are installation-scoped; process, file-descriptor, file-size, CPU, memory, and PID limits come only from `data/config.yaml` (including native environment overrides), never from a plugin manifest. Cloud requires nsjail and delegated cgroup v2 hard limits or fails closed.
- Cost boundary: Identical verified package bytes share one digest-addressed code cache. A dependency environment is keyed by the artifact and requirements digests, Python ABI, Runtime version, and installer schema, then atomically published read-only for reuse. Installations and processes are not merged, even for the same plugin and version. Registration creates database desired state only; a worker is launched only for an enabled installation.
- Recovery: PostgreSQL installation desired state and durable binary storage are authoritative. Runtime reconnect performs an instance-wide full reconciliation, removes stale workers, and can replay a verified package after Runtime-local cache loss. Dependency preparation failure is recorded per installation with `dependency_prepare_failed`; it prevents that worker launch without blocking recovery of other desired installations, and the same revision can be retried. The installation Supervisor now restores an unexpectedly exited enabled worker through a completion callback with bounded exponential backoff. Jitter, global restart concurrency limits, and a Runtime-wide circuit breaker are still required to prove that an infrastructure-wide failure cannot create a cross-tenant restart storm.
- Compatibility: Older SDK payloads and legacy `data/plugins` remain an OSS-only bridge. Shared mode requires complete bindings and rejects incomplete context.
- Reason: Sharing the supervisor and immutable code cache removes per-Workspace service cost without turning an untrusted plugin process into a cross-tenant router.
### Invitation delivery does not require SMTP
- Decision: Core returns an invitation secret once for copy-and-share, persists only its hash, and supports expiry, revocation, and one-time acceptance.
- Reason: Self-hosted OSS must support adding users without an email service while avoiding recoverable invitation secrets at rest.
- Browser handling: The copyable invitation URL carries the secret in its fragment, which browsers do not send in HTTP requests or Referer headers. The acceptance page immediately removes the fragment and keeps the secret only in `sessionStorage` until login or acceptance completes; it is never placed in a path, query string, analytics event, or persistent local storage.
### Schema rollout is additive before enforcement
- Decision: Add Account/Workspace directory tables first, then add non-null Workspace ownership to every tenant resource with a deterministic default-Workspace backfill. Runtime and service enforcement is enabled only with matching migration and isolation tests.
- Reason: A Workspace column alone is not isolation, and enforcing queries before data backfill would break upgraded installations.
### Login capability discovery is instance-scoped, not account-scoped
- Decision: The unauthenticated login bootstrap endpoint reports only which login mechanisms the instance supports. It does not inspect the first Account or expose whether that Account has a password. Both password and Space OAuth entry points are available on a multi-user instance; the submitted identity determines which mechanism is valid.
- Reason: This avoids projecting the original owner's authentication type onto invited users and removes a public Account-state disclosure.
### Space OAuth identity does not choose a SaaS Workspace
- Decision: Space OAuth tokens remain Account credentials. In OSS singleton mode, an OAuth refresh may update the singleton Workspace's Space provider only when that Account's role can manage provider secrets. In SaaS multi-Workspace mode an OAuth callback without an authenticated Workspace selector never guesses which Workspace to mutate; explicit Workspace configuration or the closed control plane owns that linkage.
- Reason: An Account may belong to several Workspaces, and authentication must not silently mutate a shared tenant secret.
### SaaS execution state is a validated Core projection
- Decision: Core can resolve both local and `cloud_projection` Workspaces, but only from an explicit Workspace UUID and an active, unfenced `WorkspaceExecutionState` for the current instance and matching source. OSS-only bootstrap paths additionally require `source=local`.
- Reason: The closed control plane owns execution ownership and generation decisions, while Core remains the enforcement point for instance binding, generation, and write fences.
### API-key secrets are one-time and Workspace-bound
- Decision: Database API keys persist only a globally unique SHA-256 hash, an opaque UUID, one Workspace UUID, explicit fixed-permission scopes, status, expiry, creator, and last-used time. The raw secret is returned once. Authentication derives Workspace and generation from the key record and ignores Workspace selectors. Legacy plaintext keys are hashed during migration and receive a compatibility `*` scope. The plaintext config key works only for the OSS singleton Workspace and is disabled in multi-Workspace mode.
- Reason: A bearer key is an identity and routing credential, not merely a password layered on top of caller-controlled tenant selection.
### MCP tools inherit the authenticated API-key context
- Decision: The MCP ASGI mount authenticates the API key once, binds an immutable per-request `RequestContext`, and every tool checks a fixed permission before calling tenant services with that same context.
- Reason: Authenticating the transport without propagating Workspace identity into tool calls would leave the direct service path globally scoped.
### Unreleased SDK protocol is pinned reproducibly without publishing
- Decision: The SDK tenancy protocol is versioned as 0.4.18. This task does not create a GitHub release or publish PyPI because the user authorized pushing code, not a package release. After the SDK feature branch is final, LangBot's feature branch temporarily pins the exact pushed SDK Git commit. Before merging to master, the release gate is to publish `langbot-plugin==0.4.18` and replace the Git pin with the registry pin.
- Reason: The current registry release does not contain the complete tenant action context and shared Runtime hardening. An exact Git commit is reproducible and keeps the feature branch testable without expanding release authority.
### Cloud directory writes stay outside Core
- Decision: The open-source Core startup always installs `SingleWorkspacePolicy`, creates or repairs one local Workspace, and permits local membership/invitation workflows. Changing mutable configuration such as `system.edition` cannot activate multi-Workspace routing. The future closed Cloud bootstrap will install `CloudWorkspacePolicy` only after verifying a signed `InstanceManifest`; that policy requires an explicit projected Workspace selector, does not create Workspaces, and rejects invitation or membership mutations with `control_plane_required`; member reads use the versioned local projection.
- Ownership split: The closed Control Plane owns the global Account/Workspace/Membership/Invitation directory, execution ownership and generation, entitlements, subscription state, usage aggregation, and billing decisions. Core owns request authorization, resource scoping, execution-generation validation, and fail-closed enforcement. Provisioning and invoice computation do not belong in open-source Core.
- Reason: The closed control plane is authoritative for SaaS Account, Workspace, Membership, and Invitation state. Allowing Core to mutate the same directory would create split-brain ownership and would make an ownerless compatibility Workspace a dangerous fallback.
- Release gate: Multi-Workspace activation is deliberately unavailable in the open-source bootstrap. Production Cloud v2 must implement the signed `InstanceManifest` verifier and closed bootstrap described in the architecture document before it can inject `CloudWorkspacePolicy`; `edition=cloud`, an environment variable, or any unsigned local configuration is never a valid activation credential.
### Workspace bootstrap is reactive and ordered before browser resource calls
- Decision: The web application blocks Workspace-owned pages until Account and current Workspace bootstrap completes. A `useSyncExternalStore` Workspace store publishes permission changes to React consumers; direct mutation-only routes and controls are hidden or disabled when the fixed role lacks the required permission.
- Reason: Mutating a module-level variable after the initial React render did not reliably re-render permission controls, and mounting resource pages before the selector was established could issue tenant requests without `X-Workspace-Id`.
### JWTs are bound to one LangBot instance
- Decision: New Core JWTs require `iss=langbot-core`, an audience derived from the immutable instance UUID, and an expiry. Legacy community tokens are accepted only when they have the historical issuer, carry no audience, and the active policy is the OSS singleton policy.
- Reason: A token issued by one instance must not authenticate against another instance that happens to share a secret, and a compatibility decoder must not become an alternate path around the SaaS trust boundary.
### Runtime control transports authenticate before protocol dispatch
- Decision: External Plugin Runtime and Box WebSocket control channels require independent strong shared secrets in handshake headers. Locally managed child processes receive ephemeral secrets through their environment; secrets are not placed in URLs, process arguments, request payloads, or logs. Box additionally binds the first authenticated control channel to one trusted instance. Plugin Runtime debug and control credentials remain separate.
- Reason: Workspace context inside an RPC payload is not trustworthy until the transport peer itself is authenticated. Separating control and debug credentials also limits accidental privilege reuse.
- Deployment consequence: Docker Compose and Kubernetes wire one shared secret to each host/runtime pair. An empty external-runtime secret fails startup instead of silently exposing an unauthenticated socket.
### Dashboard WebSocket sessions are tenant runtime objects
- Decision: A dashboard WebSocket sends an authentication frame immediately after upgrade. The server validates Account, Membership, permission, Pipeline ownership, instance, Workspace, and execution generation before registering the connection. Connection indexes, sessions, broadcasts, attachments, and resets include the complete execution scope.
- Reason: Browser WebSocket APIs cannot attach the normal authorization headers, and a process-global `pipeline_uuid` or `session_type` index can collide across Workspaces.
### Read permissions never imply secret permissions
- Decision: `resource.view` responses recursively redact Bot, Plugin, MCP, and provider credentials. Provider secrets require `provider_secret.manage`; Bot and Plugin configuration writes require `resource.manage`. Masked Plugin values can be round-tripped by a manager without overwriting the stored secret. Plugin Runtime debug credentials require `resource.manage`, not the operator-only `runtime.operate` permission.
- Reason: A multi-user Workspace needs useful viewer access without turning every visible configuration endpoint into credential export. Plugin debug attachment can register executable code and is therefore a resource-management operation.
### Temporary credential exchanges are bound to their initiator
- Decision: Lark, Weixin, DingTalk, WeComBot, and QQOfficial one-click registration sessions require `resource.manage` and store the initiating instance, Workspace, execution generation, and principal. Status and cancellation by any other scope return the same 404 as an unknown session.
- Reason: Random session IDs reduce guessing probability but do not authorize access to credentials returned by a completed exchange.
### Uploaded images and documents use different storage capabilities
- Decision: Browser images use the scoped `upload_image` owner type and may be resolved only through the opaque public-image route. RAG documents use `upload_document` and can be read, sized, or deleted only by an exact instance, Workspace, generation, and owner-type match. Legacy `upload` objects are cleanup-only.
- Reason: Treating every upload as a public image made a leaked document key sufficient to bypass authenticated RAG access.
## 2026-07-19
### Space OAuth state is server-issued and single-use
- Decision: Core issues an opaque, cryptographically random OAuth state for each Space login or Account-binding attempt, stores only its digest, and consumes it exactly once within a short expiry. Login and binding states are different capabilities; a binding state is additionally bound to the authenticated Account. Caller-supplied state, including a LangBot JWT, is rejected.
- Redirect boundary: Callback redirects are accepted only for the known callback path and an origin declared by the server-side `api.webui_url` or `api.webhook_prefix`. Request `Host` and `Origin` headers never expand this allowlist.
- Current deployment: The OSS state store is bounded and process-local, so a Core restart safely invalidates outstanding attempts. A horizontally scaled SaaS deployment must move this exchange to an atomic, shared Control Plane store before enabling the closed Cloud bootstrap.
- Reason: OAuth state is a narrow, one-time CSRF and flow-binding capability. Reusing a bearer JWT or trusting caller-controlled Host or Origin data would turn an authorization redirect into an Account-token theft or open-redirect primitive.
### OAuth provider subjects, not email addresses, bind Accounts
- Decision: A known Space `account_uuid` may refresh the credentials of its already-bound local Account. An unknown provider subject that presents an email belonging to an existing Account is rejected, even when the normalized emails match. The Account owner must authenticate locally and use the one-time, account-bound binding flow.
- Reason: Email is contact and display data, not a stable federated identity key. Email-only auto-linking would let provider verification drift or identity reassignment become a local Account takeover.
### Workspace discovery is an account-only bootstrap capability
- Decision: `ACCOUNT_TOKEN` validates the active Account JWT but intentionally cannot resolve a Workspace, receive `RequestContext`, or declare Workspace permissions. Its narrow bootstrap endpoint returns only active Workspace memberships belonging to that Account and never chooses the first Workspace when several exist. All tenant resource routes still require the explicit selector in multi-Workspace mode.
- Reason: Requiring a Workspace header to discover the Account's Workspaces creates an authentication deadlock; allowing the bootstrap route to perform tenant actions would create an authorization bypass. Separating the two capabilities resolves the cycle without weakening tenant routes.
### SQLite tenancy migrations have a verified recovery boundary
- Decision: Before each destructive tenant-schema boundary, a file-backed SQLite installation creates an online-consistent backup with its source and target revisions, runs `PRAGMA quick_check`, writes a durable manifest, and fsyncs restrictive-permission files and directories. A failed boundary disposes the engine, removes stale journal sidecars, atomically restores the verified source revision, and verifies the restored database before startup continues.
- Compatibility: In-memory SQLite cannot provide this recovery guarantee and is rejected for destructive production migration boundaries; it remains usable in tests that create the final schema directly.
- Reason: SQLite batch table rebuilds can leave an installation between schemas if a process or migration fails. A verified pre-boundary image makes retry behavior recoverable instead of merely idempotent in the happy path.
### Execution generation is an execution revocation capability
- Decision: RuntimeBot, RuntimePipeline, background tasks, object storage, Plugin Runtime, MCP, RAG, and Box operations carry the complete instance, Workspace, and execution-generation scope. The current schema and wire compatibility field remains `placement_generation` until a coordinated rename. They revalidate the active execution binding before accessing a provider or transport; long-running calls validate again before accepting results. A stale generation is fenced before it can read, write, or reuse a cached object.
- Plugin boundary: Each locally launched plugin receives a short-lived, one-use registration capability bound to the expected manifest identity and execution scope. The production child environment does not inherit the reusable debug credential, and Host APIs derive scope from the trusted connection and action context.
- Box boundary: Persistent skill content remains Workspace-scoped, while session/process state and relay requests also include execution generation. A generation change retires matching live sessions and closes a stale relay before further stdin, stdout, or file operations.
- Transaction boundary: Request admission and runtime side effects are fenced in this branch, but ordinary tenant database mutations do not yet hold a generation-aware lock through commit. The closed Cloud bootstrap must remain disabled until Core provides the shared-write/exclusive-cutover transaction primitive and a generation-stamped outbox (or an equivalent atomic publish fence). The OSS singleton policy has a fixed local generation and cannot trigger an execution-owner cutover.
- Durable-object boundary: Current opaque storage keys include generation and therefore fail closed after a generation change. That is safe for OSS's fixed generation, but a Cloud cutover must not strand durable KB files, images, or plugin references. Cloud v2 must publish stable final object identities from generation-scoped staging, or perform an atomic object-and-reference migration before activating the new generation.
- Reason: Workspace UUID prevents cross-tenant collisions, but it cannot revoke work after execution ownership changes or is fenced. Execution generation is the monotonic revocation value that makes old runtimes unusable; it does not express membership in a product-level deployment entity.
### Long-lived WebSockets continuously revalidate authority
- Decision: Dashboard WebSockets re-authenticate the Account, Membership, permission, resource ownership, instance, Workspace, and execution generation for every inbound message, not only during the initial frame. A changed role, removed Membership, or fenced execution binding takes effect without waiting for reconnect.
- Public embed boundary: The embed connection re-resolves its Bot before every message and rejects a Bot that was disabled, deleted, moved, or rebound. The public connection may identify a Bot, but it cannot make the initial Bot object an indefinite authorization capability.
- Reason: Authorization and resource state can change while a socket remains open. Connection-time validation alone leaves a revocation gap.
### Legacy vector migration is an OSS-local compatibility path
- Decision: Status, backup, execute, dismiss, and background entry points for legacy global vector collections require an active local Workspace binding under `SingleWorkspacePolicy`. A `cloud_projection` Workspace cannot observe or migrate the old global collection, even when it carries a legacy marker.
- Reason: The legacy collection predates tenant ownership. Treating it as a SaaS fallback would expose one installation's historical vectors to an arbitrary projected Workspace.
### External errors and persisted URLs are redacted centrally
- Decision: Unhandled HTTP and webhook failures return a stable `internal_error` response and request ID, and expose that ID in `X-Request-Id`; the detailed exception is retained only in server logs correlated by the same ID. Explicit domain and validation errors keep their documented status and code.
- Secret boundary: Shared sanitization removes URL user information and masks sensitive query parameters before provider or MCP configuration is serialized, logged, or shown to a reader. Masked placeholders can be round-tripped by an authorized manager without replacing the stored secret.
- Reason: Tenant isolation is incomplete if framework exceptions, connection URLs, or configuration reads can export credentials across otherwise authorized interfaces.
### Box is one shared control plane with one admitted sandbox per Workspace
- Decision: The logical instance has one shared Box Runtime control plane, implemented by one Runtime replica in M0. A closed entitlement adapter projects generic `managed_sandbox` capability and `managed_sandbox_sessions` limit; Core and Runtime never branch on a plan name. An eligible Workspace receives at most one persistent logical `global` session, while each ordinary command remains a one-shot nsjail process. Managed processes and network are disabled in the first Cloud release.
- Storage boundary: Core and Runtime prove they see the same durable volume with an authenticated random-marker challenge. Attachments use opaque query UUID directories and link-free dirfd operations. Skill packages remain in the Runtime-owned Workspace store and enter a sandbox only as a read-only logical-name mount; Python environments and caches live in the tenant's writable Workspace.
- Resource boundary: Cloud readiness requires cgroup v2 plus hard byte and inode limits for Workspace files, Skill storage, root, tmp, and home. The existing directory scan is only a compatibility soft check. Plain nsjail reports these storage capabilities as unavailable, so Cloud Box intentionally cannot start until the greenfield deployment supplies and verifies a real quota provider.
- Archive boundary: Skill ZIP processing is bounded by compressed input, entry count, per-entry size, total uncompressed size, and compression ratio, and rejects links, non-regular entries, duplicates, and path escape before streaming extraction.
- Reason: A shared supervisor removes per-Workspace services and idle control-plane cost, but storage and process admission must still fail closed at the untrusted execution boundary.
### Cloud business data and vectors share one PostgreSQL schema
- Decision: SaaS uses one PostgreSQL business database and shared schema. Every tenant row has an explicit Workspace key; application scope is the first boundary and precise `ENABLE` plus `FORCE ROW LEVEL SECURITY` policies are the second. The Cloud runtime role must be non-owner and have neither superuser nor `BYPASSRLS`.
- Vector boundary: pgvector is the Cloud default in the same business database. Vectors use `(workspace_uuid, knowledge_base_uuid, vector_id)` identity, an untyped vector column with explicit checked dimension, and release-created partial expression indexes for the enabled dimensions. Cloud never falls back to Chroma or performs vector DDL at runtime.
- Transaction boundary: A tenant UoW binds `SET LOCAL` and SQL to one transaction. Long-running pipeline and streaming MCP execution carry a trusted transaction-free tenant scope; each database helper opens a short scoped transaction, avoiding a held pool connection during LLM or network waits. Detached tasks start only after commit and create their own short UoW; rollback cancels them.
- Schema boundary: The first release has exactly one business schema, `public`. Both migrator and runtime sessions must report `current_schema() = 'public'` and `current_schemas(false) = ARRAY['public']`; the runtime role and business database must not carry a `search_path` override. Runtime startup validates this before using the prepared schema and reruns the complete catalog and privilege validation on every process start; it never runs DDL.
- Session boundary: Both Cloud modes require `session_replication_role = 'origin'`, `row_security = 'on'`, and `lo_compat_privileges = 'off'`. Every persistent setting applicable to the runtime role or current business database in `pg_db_role_setting` is rejected, even if its present value appears safe; tenant context remains transaction-local application state rather than a persistent role/database override.
- Grant boundary: The migrator grants the runtime role direct `CONNECT` on the dedicated business database and `USAGE` on `public`; exact `SELECT, INSERT, UPDATE, DELETE` on every allowlisted business table; `SELECT` only on `alembic_version`; and exact `USAGE, SELECT` on business-owned sequences. It grants neither `CREATE`, `TRUNCATE`, `REFERENCES`, `TRIGGER`, sequence `UPDATE`, nor any privilege with `WITH GRANT OPTION`, and grants nothing on other relations or schemas.
- Role boundary: The runtime identity is a `LOGIN` role with no superuser, `BYPASSRLS`, `CREATEDB`, `CREATEROLE`, or replication attribute; no role membership in any direction, including acting as grantor; no ownership of the business database, `public` schema, relations, sequences, routines, or extensions; no column ACLs; and no use, create, or ownership in another non-system schema. Neither the runtime role nor `PUBLIC` may have an explicit routine or parameter ACL, and the runtime role may not effectively execute any `SECURITY DEFINER` routine, including an extension-owned one. PostgreSQL's default `TEMP` privilege inherited from `PUBLIC` is an explicit first-release compatibility decision for this dedicated business database, not a direct runtime-role grant.
- Catalog boundary: The business database must contain `vector` and may contain no extension other than `plpgsql` and `vector`; the runtime role owns neither. It contains no foreign data wrapper, foreign server, or user mapping. These checks remove catalog-level escape paths without forbidding the ordinary implicit execution of non-`SECURITY DEFINER` built-in routines.
- Migration boundary: In the first release, the migrator and runtime URLs must name the same normalized PostgreSQL host, port, and database while using different roles. The migrator owns the application schema, establishes the exact allowlist above, and validates both required access and every prohibited escalation path before releasing the advisory lock. An exact Alembic head, RLS checks, and pgvector table/index/constraint validation remain mandatory; concurrent Jobs fail explicitly and are retried by orchestration.
- Deployment boundary: PostgreSQL roles are cluster-wide, while the in-database audit proves only the target business database contract. SaaS production must therefore use a dedicated PostgreSQL cluster or endpoint that exposes only this business database to the runtime credential, or enforce and test an HBA/proxy policy proving that the credential cannot connect to any other database. This external connectivity proof is still an incomplete SaaS activation gate.
- Endpoint evolution: A future deployment may use a direct endpoint for migrations and a pooler endpoint for runtime traffic. That topology may relax literal host/port equality only after both endpoints are proven to reach the same database through a database-internal, migrator-owned cluster identity that the runtime role can read but cannot create, alter, or spoof.
- Legacy pgvector boundary: Revision 0013 records the exact `ENABLE` and `FORCE ROW LEVEL SECURITY` state of each RLS-protected source table, temporarily suspends those source policies as their table owner inside the migration transaction, and restores every table to its recorded state in `finally`. The migrator does not require superuser or `BYPASSRLS` for this data move.
- Activation gate: The shared schema, pgvector adapter, and database-local runtime audit are implemented, but the external cluster/endpoint or HBA/proxy connectivity proof remains deployment work. Ordinary business writes also do not yet hold a generation-aware fence through commit; a generation-stamped outbox (or equivalent atomic publish fence) and stable durable-object references across generation cutover remain required before SaaS activation.
- Reason: Sharing one database and pool keeps marginal Workspace cost low, while transaction-local context and RLS prevent that shared storage from becoming shared authority.
### stdio MCP has an independent deployment gate
- Decision: `mcp.stdio.enabled` is independent of Box availability and entitlement. OSS defaults it on for compatibility; Cloud requires it off at bootstrap and enforces the same gate on create, update, test, startup loading, and final runtime execution.
- Reason: Treating Box availability as stdio permission would silently create another persistent `mcp-shared` sandbox for each Workspace and bypass the one-sandbox subscription and cost boundary.
## 2026-07-20
### The tenant UoW owns its task, root transaction, bind, and scope
- Decision: A tenant UoW creates one task-owned `TenantScopedAsyncSession` and one root transaction. Public commit, rollback, close, connection, bind, nested-transaction, synchronous-Session, live-streaming, raw SQL, public execution options, and public `set_config` paths fail closed and mark the transaction rollback-only. ORM objects cannot expose a usable synchronous Session, captured methods cannot run in child tasks, an explicit foreign bind is rejected, and a captured Session is permanently retired when its UoW exits rather than being reset for reuse. Tenant scope is installed only through a private UoW capability; pgvector index-plan `SET LOCAL`/`EXPLAIN` diagnostics use a test/operator connection rather than the business Session API.
- SQL boundary: Public UoW calls accept only structured SQLAlchemy query and DML trees. `TextClause`, literal SQL columns, textual labels, prefixes/suffixes/hints, statement execution options, `VALUES` roots, `INSERT FROM SELECT`, `EXTRACT`, literal-execute parameters, unknown/custom AST nodes, forced-unquoted identifiers, named `ON CONFLICT` constraints, unknown dialect post-values clauses, and untrusted casts/types fail closed. PostgreSQL/SQLite `ON CONFLICT DO UPDATE` and batch-insert containers are traversed explicitly because SQLAlchemy's standard visitor omits their executable values. Function classes are exactly allowlisted as `count`, `coalesce`, `sum`, `now`, `length`, and `nullif`; the only custom operator/cast admitted is the validated pgvector cosine operator and `Vector` cast.
- Legacy migration boundary: The local-only RAG backup restore uses explicit table and column objects, never raw SQL, but deliberately leaves legacy values untyped. This preserves SQLite's string-valued `DATETIME` rows and both the historical PostgreSQL `TEXT` and fresh-schema `JSON` settings columns while keeping every value bound rather than interpolated.
- ORM boundary: SQLAlchemy `SessionEvents` are unsupported on a tenant-scoped Session. If a listener is registered before or during a UoW, the operation fails before the callback executes and cleanup proceeds against an empty dispatch surface. Public `get`, `get_one`, `refresh`, and `merge` reject caller-supplied loader, bind, lock, shard, and execution options. Flush, implicit autoflush, and commit reject a SQL expression assigned to a mapped attribute before it can reach the compiler. Tenant code uses the async Session directly; relationships use eager loading or explicit `await session.refresh(entity, [attribute])`. LangBot's persistence base does not expose `AsyncAttrs.awaitable_attrs` as a supported tenant API.
- Compiler trust boundary: This guard prevents accidental scope/transaction escape by trusted LangBot Core code; it is not an in-process Python sandbox. Registered SQLAlchemy compilers and mapped schema metadata are trusted boot-time code. The fail-closed traversal of dialect containers necessarily covers SQLAlchemy private fields, so dependency upgrades require the regression suite and remain pinned until verified. Untrusted plugins cannot import or call this Session because they remain isolated in Plugin Runtime child processes.
- Result boundary: A caught database or boundary failure rolls back the root transaction and cancels after-commit work. Buffered results contain only already-authorized rows and no live connection; live database results cannot escape the UoW operation.
- Reason: `SET LOCAL` plus RLS protects a tenant only while every statement stays on the same owned connection and callers cannot end the transaction, replace the GUC, recover the synchronous proxy, or route a statement through another bind.
### Parallel request work re-enters tenant scope explicitly
- Decision: Child coroutines created by request-level `asyncio.gather` open their own explicit transaction-free tenant scope before calling persistence-backed Plugin, MCP, or Skill operations. They never inherit the parent's active database Session.
- Reason: Python copies ContextVars into child tasks, but SQLAlchemy Sessions are not task-safe and task identity is part of the tenant UoW boundary.
### Ordinary monitoring is readable; audit and export remain privileged
- Decision: Workspace monitoring dashboards, Bot logs, sessions, messages, calls, errors, and feedback require `resource.view`. Monitoring export requires `data.export`; system/runtime audit logs keep `audit.view`. Frontend tabs and controls use the same split.
- Reason: A Viewer needs useful read-only product observability, while bulk data extraction and privileged runtime/system logs are separate capabilities.
### Invitation failures survive login without contradictory success state
- Decision: Invitation terminal codes map to stable browser states. A login-mediated email mismatch preserves the fragment-captured secret only in session storage, returns to the acceptance page with the stable error code, and suppresses the generic login-success toast. A transient acceptance failure retains the authenticated session and offers the same one-time invitation for retry. Invalid Bearer tokens on acceptance map to the normal authentication error instead of an internal failure.
- Reason: Invitation acceptance crosses signed-out and authenticated states; losing or masking the domain error makes recovery ambiguous and can present contradictory UI feedback.
### Shared Plugin Runtime starts only from verified desired state
- Decision: SDK shared mode waits for immutable runtime configuration before inspecting plugin state, never scans or launches legacy `data/plugins`, and rejects legacy install/restart/delete/upgrade control actions. Worker RPC files use installation-private directories with aggregate size enforcement, and resident nsjail workers explicitly disable the default 600-second wall-time limit.
- Remaining gate: completion-callback recovery and bounded per-installation backoff are implemented. Cross-tenant restart-storm suppression, hard installation disk quota, and production egress policy remain Cloud activation requirements; Linux nsjail/cgroup CPU, memory-plus-swap, PID, namespace, and cgroup-reaping behavior have real-container evidence.
- Reason: A shared supervisor reduces per-Workspace services only if legacy global paths, writable transfer state, and lifecycle defaults cannot bypass installation isolation.
## 2026-07-24
### Cloud v2 remains a modular monolith with one closed adapter
- Decision: Workspace directory, plans, subscriptions, payment fulfillment, entitlements, usage, and signed runtime feeds are implemented as modules in the existing Space service and PostgreSQL database. The only separately packaged closed component is the thin Core bootstrap/control-plane adapter; it verifies signed data and owns no business state.
- Compatibility: The legacy Pod card and fulfillment path remain visible only to accounts with old subscriptions. New purchases create Workspace subscriptions and never provision a per-user LangBot Pod.
- Reason: A separate tenancy or billing service would add deployment, queue, network, and consistency cost without providing an isolation boundary. The signed adapter keeps SaaS logic closed while Core retains the ORM, RLS, authorization, and runtime enforcement boundary.
### Registration creates data, not infrastructure
- Decision: Account registration and personal Workspace creation share one transaction and include an active owner membership, Free subscription, entitlement snapshot, and outbox notifications. Repair is idempotent for older accounts. Empty Workspace creation starts no Plugin worker, Box sandbox, database, queue, bucket, or tenant service.
- Activation: This automatic Cloud v2 transaction is enabled only when Space has an explicit valid `CLOUD_V2_INSTANCE_UUID`. A legacy marketplace-only Space deployment with no Cloud v2 instance configured keeps its existing Account registration path; Cloud v2 internal endpoints still fail closed instead of inventing an instance identity.
- Billing boundary: Core and runtimes consume only generic capability and numeric-limit entitlements. They never branch on `free` or `pro`; plan names, prices, payment providers, and fulfillment stay in Space.
- Reason: This keeps the marginal cost of a new user close to a few PostgreSQL rows while preserving an immediate, usable Workspace.
### Directory bootstrap is full; steady-state projection is per Workspace
- Decision: Space generates the initial signed directory snapshot and its outbox high-water mark in one read-only PostgreSQL `REPEATABLE READ` transaction. After bootstrap, each event page names affected Workspaces and Core fetches one signed `directory.delta` for that set. Missing requested Workspaces are tombstones; unrelated Workspaces are untouched.
- Cursor boundary: A delta carries no event cursor. Each signed event page carries the transaction-consistent current high-water mark, while Core advances only to the final event it actually consumed, even when the authoritative delta already contains a later revision. Event payload Workspace and revision fields must exactly match the signed event envelope; readiness is renewed only after the replica-local cursor reaches the signed high-water and the shared projection is not ahead.
- Replica boundary: Every Core replica owns a process-local consumer cursor because its verified entitlement cache is process-local. Replicas share the PostgreSQL projection high-water mark and inbox. The state separately records which cursor range was atomically subsumed by a full snapshot, so a lagging replica can add missing receipts within that coverage and refresh its local caches without repeating tenant mutations; a missing receipt beyond snapshot coverage fails closed.
- Reason: A shared consumer cursor would let one replica starve another replica's local cache, while a full snapshot on every change would make steady-state cost grow with every registered Workspace.
### Cloud OAuth can authenticate only an existing projected Account
- Decision: In multi-Workspace Cloud mode, Space OAuth may refresh tokens only for an active `cloud_projection` Account whose Space subject UUID and normalized email exactly match. It cannot create an Account, relink by email, mutate directory identity, or choose a Workspace.
- Redirect boundary: When Cloud v2 configures its Core public URL, Space issues authorization codes only to that exact callback origin or the explicitly retained legacy managed-Pod domain. Community installations remain dynamic OAuth clients because they cannot pre-register with the public Space service: the consent screen displays their hostname, and only HTTPS or loopback HTTP with the fixed callback path is accepted. Remote HTTP, userinfo, fragments, arbitrary callback paths, and unrecognized query parameters always fail closed.
- Reason: Space is the SaaS identity and directory authority, but Core remains the authentication enforcement point. Projected-only matching avoids split-brain identities; redirect allowlisting prevents bearer authorization-code exfiltration.
@@ -0,0 +1,519 @@
# Cloud v2 多租户架构决策与待决策项
状态:`DECIDED — Core isolation kernel implemented; SaaS activation gates remain`
创建日期:2026-07-19
最近更新:2026-07-24
本文记录 Cloud v2 多租户架构中已经确认的首期决策、明确淘汰的方案和仍需在后续阶段决定的扩展项。
本文同时记录实现状态。“实现完成”仅指开源 Core/SDK 的隔离内核和 fail-closed 门禁,
不表示闭源 Control Plane、计费或 Cloud v2 部署已经可上线。最终实现选择同步记录在
[implementation-decisions.md](./implementation-decisions.md),剩余发布门禁记录在实施清单和验证报告。
## 0. 已确认的 SaaS 拓扑前提
1. SaaS 只有一个逻辑 LangBot 实例,全部 Workspace 都是该实例内的租户。
2. 产品和领域模型中不引入 Cell 内多个 CloudInstance、Workspace Placement 或 Workspace 到 CloudInstance 的路由。
3. 当前不实现分布式,但同一个逻辑实例未来可以运行多个 Core、Plugin Runtime 和 Box Runtime replica
也可以增加 PostgreSQL shard;这些只是内部实现,不成为新的租户或产品实体。
4. 所有副本共享稳定的 `instance_uuid``replica_id``worker_id` 和进程地址是短期运行身份,不能写进业务资源的永久主键。
5. `workspace_uuid` 始终是数据、任务和运行时的租户键,也是未来内部路由与分片的候选键。
6. generation/epoch 的语义是执行所有权、故障转移和任务撤销,不代表 Workspace 在多个 CloudInstance 之间 Placement。
7. 注册 Account 时自动创建 Workspace,但只新增目录记录和业务行,不创建租户专属部署、数据库、队列或 Runtime。
8. OSS 仍是单租户 LangBot 实例,但允许该 Workspace 内存在多个用户;只有 SaaS 开启多 Workspace 租户模式。
“单个 LangBot 实例”表示单个逻辑服务和安全域,不等于永远只有一个 OS 进程或一个 Kubernetes replica。
当前代码字段 `placement_generation` 在完成架构迁移前继续兼容,目标语义和候选命名是 `execution_generation`
### 0.1 本轮确认的首期决策
| 编号 | 结论 | 首期状态 |
| ----- | --------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| D-001 | 一个共享 Plugin Runtime 控制面;每个运行中的 plugin installation 独占一个 nsjail 子进程;只有 digest 相同且已验证的代码 artifact 可以只读共享 | `FOUNDATION VERIFIED — egress, total disk-quota and restart-storm gates pending` |
| D-002 | 一个共享 Box RuntimeCloud 固定使用 nsjail;符合套餐的 Workspace 最多一个持久 `global` 逻辑 sandbox,普通执行按需启动 nsjail 进程 | `IMPLEMENTED FAIL-CLOSED — hard filesystem quota provider pending` |
| D-003 | SaaS 业务数据使用 PostgreSQL shared schema、应用层作用域和 RLS 双重隔离;pgvector 使用同一 PostgreSQL,作为 SaaS 默认向量后端 | `PARTIALLY IMPLEMENTED — transaction/outbox/deployment gates remain` |
| D-004 | stdio MCP 与 Box availability 解耦;Cloud v2 首期强制关闭 stdio MCP,避免为每个 Workspace 创建额外的 `mcp-shared` persistent sandbox | `IMPLEMENTED` |
| D-005 | 目录启动使用事务一致的全量快照,运行时按事件涉及的 Workspace 拉取增量;每个 Core replica 独立消费事件,共享 PostgreSQL 投影和 inbox | `IMPLEMENTED — production fault injection pending` |
Workspace 的具体创建、释放、数据导出和单 Workspace 恢复机制不在本轮决定;本文只保证这些后续能力不会改变稳定的
`workspace_uuid`,也不会要求重建租户专属部署。
## 1. 本轮重构的最高目标
> 共享可信控制面和基础设施池,隔离不可信执行单元;减少独立部署、扩缩容和运维组件,使新增 Account 或 Workspace 的静态成本接近零。
这里的“减少组件”指减少独立 Deployment、Service、数据库、消息系统和租户专属常驻控制面,
不是通过合并安全边界来减少必要的隔离进程。
统一评估原则:
1. 注册 Account、自动创建空 Workspace 时,不启动 Plugin worker 或 Box sandbox。
2. 启用插件后,每个 installation 的常驻成本来自其独立安全边界;首次使用托管 sandbox 后,符合套餐的 Workspace 才承担一个持久逻辑 session 的成本。
3. 可信 supervisor、artifact cache、数据库连接池和 Runtime 容量可以多租户共享。
4. 一个不可信插件进程不能服务多个 installation;一个 sandbox/session 不能服务多个 Workspace。
5. 默认使用共享 Runtime;dedicated 只作为未来高隔离、大客户或合规资源等级,不建立第二套外部协议。
6. 没有明确容量证据前,不新增 Kafka、Redis、Runtime 专用数据库、Box 专用数据库或租户级调度服务。
7. 多租户隔离必须覆盖身份、路由、存储、缓存、日志、配额、撤销和故障恢复,不能只给请求增加 `workspace_uuid`
## 2. 首期部署形态与未来演进
```mermaid
flowchart LR
Traffic["SaaS traffic"] --> Core["One logical LangBot instance<br/>1 Core replica in MVP"]
Core --> PluginRuntime["Shared Plugin Runtime<br/>trusted supervisor"]
Core --> BoxRuntime["Shared Box Runtime<br/>nsjail backend"]
PluginRuntime --> PA["Workspace A / installation 1<br/>isolated nsjail process"]
PluginRuntime --> PB["Workspace B / installation 2<br/>isolated nsjail process"]
BoxRuntime --> BA["Workspace A<br/>one persistent global logical session"]
BoxRuntime --> BB["Workspace B<br/>one persistent global logical session"]
Core --> PG["Shared PostgreSQL business schema<br/>RLS + pgvector"]
```
| 档位 | 内部部署形态 | 新 Workspace 静态成本 | 启用条件 |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- | ------------------------ |
| M0. 单副本 MVP | 一个 Core、一个共享 Plugin Runtime、一个共享 Box Runtime、一个 PostgreSQL business database;插件按启用状态运行,托管 sandbox 按首次使用与 entitlement 创建 | 只新增 Workspace 业务行 | 当前已确认目标 |
| M1. 同逻辑实例内部横向扩展 | Core、Plugin Runtime、Box Runtime 按容量增加 replica;运行所有权由内部 lease 和 generation fence 决定;PostgreSQL 可增加共享 shard | 不创建 Workspace 专属部署 | 出现容量或可用性证据后 |
| M2. Dedicated 资源档位 | 特定 workload 使用独享 worker pool、sandbox class 或 PostgreSQL shard,但沿用相同身份、协议、schema 和控制面 | 仅由购买 dedicated 的客户承担 | 合规、数据驻留或超大负载 |
M1 是 M0 的透明扩容,M2 是相同架构下的资源等级;两者都不是新的 LangBot 实例、Cell 或 CloudInstance。
外部 API 只认识稳定的 `instance_uuid``workspace_uuid`,不认识 replica、worker、pool 或 shard。
Plugin Runtime 与 Core 在 M0 使用独立容器和 security contextCore 不能继承 Plugin Runtime 所需的
nsjail/cgroup 权限。当前 Runtime 在进程生命周期内绑定首次认证的 `runtime_id`,因此 M0 必须把 Core 与
Plugin Runtime 放在同一 rollout/restart unit 中协调重启;在实现受认证 takeover 或 owner lease/fencing 前,
不能单独滚动 Core 并让它接管仍存活的 Runtime。Box Runtime 同样使用独立进程身份和安全配置,
不与 Plugin Runtime 合并成一个高权限进程。
## 3. D-001Plugin Runtime 多租户控制面
状态:`FOUNDATION VERIFIED — Cloud egress, total disk-quota and restart-storm gates pending`
### 3.1 已实现的基础
- Plugin Runtime 控制连接只绑定稳定实例身份;一个逻辑共享控制面通过完整 installation binding 管理多个 WorkspaceM0 由一个 Supervisor replica 承担。
- 每个运行中的 installation 使用独立 nsjail workerenabled-resident 是 desired semantics。代码只读,home/tmp/data 私有,shared profile 不读取 artifact `.env`
- 实例级 `PluginWorkerPolicy` 由 Core 的 `data/config.yaml` 下发,支持原生环境变量覆写;manifest 不能覆盖。
- `installation_uuid``artifact_digest``runtime_revision` 已持久化并进入 desired-state、注册、Host API 和 generation/revision fence。
- 已验证 `.lbpkg` 先进入 Workspace-scoped durable binary storageRuntime 本地缓存丢失后可由 Core replay。
- 相同 digest 的代码和 Runtime 准备的只读依赖环境可以共享,但 worker、运行时写入、配置和数据不合并;
dependency preparation 在启动 worker 前完成,失败会进入明确的 installation failed 状态。
- Cloud shared profile 强制 Linux nsjail`plugin.worker.require_hard_limits=true` 时 cgroup v2 delegation 不可用会启动失败。
### 3.2 已确认的不变量
1. 每个运行中的 plugin installation 独占一个 worker process tree,任何时刻都不能与其他 installation 共用;
停用或删除的 installation 可以没有进程。
2. 插件进程只绑定一个
`(instance_uuid, workspace_uuid, execution_generation, installation_uuid, runtime_revision, artifact_digest)`,且运行期间不可重绑。
3. 插件不能通过 payload、Host API 参数、环境变量或重连选择 Workspace。
4. 插件进程的 home、tmp、可写数据、secret、进程视图和配额必须按 installation 隔离。
5. 只有 `artifact_digest` 相同且完整性已验证的代码文件和依赖环境可以只读共享;
同名同版本但 digest 不同的 artifact 不能共享。配置、持久数据和运行进程不能共享。
6. generation、installation revision 或 capability 被撤销后,旧进程必须失去 Host API 和副作用权限。
7. Supervisor 不在自身解释器中加载第三方插件代码。
### 3.3 首期执行模型
- 整个 SaaS 实例共享一个可信 Plugin Runtime 逻辑控制面,M0 运行一个 Supervisor replica;新 Workspace 不创建专属 Runtime、连接、卷或进程。
- Supervisor 的控制连接只绑定稳定 `instance_uuid` 和短期 Runtime identity,不绑定某个 Workspace。
每条 installation desired-state 命令都携带并验证完整的 installation binding;每个 worker action context 在注册后永久绑定该 tuple。
- 安装并启用插件后,Supervisor 在自己的 Runtime 容器内直接启动一个 nsjail 子进程;
不再为每个插件创建 nested container、Pod、sidecar 或租户级 Runtime service。
- desired semantics 要求 enabled installation 保持 resident,不做 idle eviction;停用、删除、revision/generation 变化或 entitlement 撤销时停止并按需重建。
当前 Supervisor 已通过 completion callback 和有界指数 backoff 恢复意外退出的 worker;尚缺 jitter、全局重启并发上限和 Runtime 级 circuit breaker
因而系统性依赖或宿主故障下的跨租户重启风暴抑制仍是 Cloud 激活门禁。
- 子进程使用一次性 registration capability 向 Supervisor 注册;capability 由可信 desired state 派生并绑定完整 installation tuple
不是插件直接建立 Core Host connection,也不能只绑定 author/name/path。Supervisor/Core 据此注入 tenant context
丢弃插件 payload 中自带的 scope 字段。
- Supervisor 的进程表、nsjail root/tmp 和 artifact cache 都是可重建运行态;PostgreSQL 中的 installation desired state 才是权威业务状态。
- M0 不增加 Runtime 专用数据库、Redis、Kafka、scheduler 或 artifact serviceCore 重连后向 Supervisor replay desired state。
### 3.4 nsjail 和文件边界
首期目标目录模型:
```text
data/plugin-runtime/
├── artifacts/sha256/<artifact_digest>/code/ # digest 校验后只读共享
├── environments/sha256/<environment_digest>/ # 原子发布、只读共享依赖环境
└── installations/<installation_uuid>/
├── home/ # 私有可写
├── tmp/ # 私有可写、可清理
└── data/ # 私有持久数据
```
- artifact 只有在内容摘要和完整性校验一致时才允许共享,不能只凭 author/name/version 复用目录;
cache 可接受的签名/来源、撤销和 GC 规范属于后续发布规则,不改变本轮基于已验证 digest 的只读共享边界。
- artifact 与按环境摘要构建的共享依赖环境以只读 mount 进入 nsjailinstallation 的 home/tmp/data 使用独立可写 mount。
环境摘要包含 artifact、requirements、Python ABI、Runtime 版本和 installer schema。依赖只能从已验证 artifact 的 PEP 508 声明构建,
index/trusted-host 只由实例配置控制;构建在独立 nsjail 的临时路径中完成并在成功后原子发布,失败或并发安装不能留下可见半成品。
- 插件 cwd 可以是其私有 mount namespace 内的只读 `/plugin`,不要求为每个 installation 复制代码;
必须私有的是 home/tmp/data 等所有可写路径。
- nsjail 必须启用 mount、PID、IPC、UTS 和 private `/proc` 等必要 namespace,插件不能枚举或 signal 其他插件及 Runtime 进程,
不能读取 Runtime 文件系统、宿主机路径、其他 installation 目录或平台 metadata endpoint。
- 公开 SaaS 禁止从插件 artifact 自动加载 `.env`。secret 只能由可信控制面按 installation 注入,且不能进入共享 artifact/cache。
- 插件需要外网时使用受控 egress;不得通过共享 host network 访问 Core loopback、Box Runtime、数据库或其他内部服务。
- Cloud 部署必须提供可用的 cgroup v2 delegation 和所需 namespace 权限;如果硬 CPU/内存/PID 限制不可用,
Plugin Runtime readiness 必须失败,不能只记录告警后降级为普通子进程。
### 3.5 统一资源上限与配置
首期资源规格完全由 LangBot 实例配置决定,manifest 不能声明、放宽或覆盖资源。以下数值是建议默认值,
最终仍由同一实例的 `data/config.yaml` 统一配置:
```yaml
plugin:
worker:
max_cpus: 1.0
max_memory_mb: 512
max_pids: 128
max_open_files: 256
max_file_size_mb: 512
require_hard_limits: true # Cloud; OSS defaults false
```
配置文件路径为 `data/config.yaml`,沿用现有原生环境变量覆写:
- `PLUGIN__WORKER__MAX_CPUS`
- `PLUGIN__WORKER__MAX_MEMORY_MB`
- `PLUGIN__WORKER__MAX_PIDS`
- `PLUGIN__WORKER__MAX_OPEN_FILES`
- `PLUGIN__WORKER__MAX_FILE_SIZE_MB`
- `PLUGIN__WORKER__REQUIRE_HARD_LIMITS`
Core 启动时校验配置并通过现有 `SET_RUNTIME_CONFIG` 下发不可变 `PluginWorkerPolicy`
Runtime 不读取另一份环境变量配置,避免两个配置源不一致。CPU、内存和 PID 使用 cgroup 硬限制,
open files/file size 使用 rlimit。Cloud deployment profile 固定使用 nsjail,不能通过插件 manifest 或 SaaS 环境变量降级为普通进程。
installation data 的总空间硬配额需要 filesystem project quota 或独立 quota volume,不能用目录扫描伪装成硬限制;
该字段在选定可原子拒绝写入的存储机制前不进入首期配置。
### 3.6 淘汰与暂缓方案
| 状态 | 方案 | 结论 |
| ---------- | -------------------------------------------------- | -------------------------------------------------------- |
| 淘汰 | 每 Workspace 一个 Plugin Runtime | 部署、连接和固定内存随 Workspace 线性增长 |
| 淘汰 | 一个插件进程服务多个 Workspace/installation | 全局状态、本地文件和依赖无法形成可信租户边界 |
| 淘汰 | 同 Workspace 多插件合并到一个 worker | 与“每 installation 独立进程”冲突,扩大故障和权限边界 |
| 淘汰 | manifest 自行声明 CPU、内存或更高限额 | 首期统一执行实例级最大值 |
| MVP 不引入 | Runtime 专用数据库、Redis、Kafka 或独立 scheduler | 当前无容量证据,会增加组件和运维面 |
| 后续演进 | 多 Supervisor replica、owner lease、dedicated pool | 保留接口,达到容量或可用性阈值后再决定具体存储与调度方式 |
架构扩展项包括:Core/Supervisor 是否共置、artifact/venv cache 的签名/来源/撤销/GC 规范、installation data hard-quota provider、
v1 connection 的兼容期限,以及进入多 replica 后的 lease TTL、fencing token 和 owner 转移顺序。
这些不改变“每个运行中的 installation 一个隔离进程”的首期边界。
### 3.7 验收条件
- 两个 Workspace 安装 digest 相同且已验证的 artifact 时,共享目录仍为只读,进程、配置、data、home、tmp、日志和 Host API 完全隔离;
同名同版本但 digest 不同的 artifact 绝不共享目录。
- 插件不能读取其他 installation 文件、枚举或 signal 其他进程,也不能修改共享代码/依赖目录。
- CPU、内存、PID、open files 和单文件上限在真实 nsjail/cgroup 环境中生效;超额只终止或拒绝对应 installation。
- 修改 manifest 不能改变任何资源上限。
- installation data 的总空间硬配额在写入边界原子拒绝超额,并证明目录扫描不是生产 enforcement。
- 旧 generation/revision 的回调、消息、副作用和存储访问全部失败关闭。
- Runtime 重启能从业务 desired state 恢复,不依赖本地进程表作为权威真相。
- 意外退出的 enabled worker 由 completion callback 触发带有界 backoff 的自动恢复;连续失败只影响对应 installation,不能形成跨租户重启风暴。
- requirements 中存在 Runtime 基础镜像未预装的包时,Supervisor 仍能先完成共享依赖环境准备再启动 worker;
安装失败不会留下持续重启的半启动进程,也不会影响同 digest 已就绪环境的其他 installation。
## 4. D-002Box 多租户控制面和套餐边界
状态:`IMPLEMENTED FAIL-CLOSED — production quota provider pending`
### 4.1 已实现的基础
- 共享 Box 控制连接可服务多个 Workspace;所有操作绑定 instance、Workspace 和 generationRuntime namespace 由可信 context 派生。
- 短期 `SandboxAdmissionGrant`、revision tombstone 和原子 session admission 强制每个合资格 Workspace 最多一个 `global` persistent sessionmanaged process 固定为零。
- Core 与 Runtime 使用认证 host-control challenge 校验同一个 durable volume,而不是比较路径字符串;不一致时启动和重连失败。
- Cloud skill 只传逻辑名称;Runtime 从 Workspace-scoped store 解析只读包路径,Python env/cache 写入租户自己的 `/workspace/.skill-envs`
- ZIP 安装限制压缩输入、条目、单项、总解压量和压缩比,采用流式解压并拒绝 link、非普通文件、重复项和路径逃逸。
- 附件 host path 使用 query UUID 和 dirfd/openat/O_NOFOLLOWCloud replica 启动不再全局清理其他请求目录,遍历和删除有 inode 预算。
- grant-enforced readiness 强制 cgroup、namespace、mount、共享卷、Workspace hard quota、Skill hard quota、ephemeral storage 和 inode quota 全部被证明。
普通 nsjail backend 对尚未实现的硬磁盘能力明确返回 false,因此当前 Cloud Box 会按设计拒绝启动,直到新部署提供真实 quota provider。
### 4.2 首期套餐与 entitlement 模型
- 闭源订阅管理/Control Plane 负责把套餐映射为版本化 entitlementCore 和 Box Runtime 不硬编码 `plan == pro`
- 首期复用 Cloud Control Plane(可结合现有 Space 的订阅模块)承载闭源套餐、计费和 entitlement 投影,
不再拆一个独立 billing/tenant microservice;开源 Core 只实现通用 capability 和数值限额。
- 首期套餐投影为:Pro 的 `managed_sandbox_sessions = 1`,其他套餐为 `0`。建议 capability 形态:
```json
{
"features": {
"managed_sandbox": true,
"external_sandbox": false,
"mcp_stdio": false
},
"limits": {
"managed_sandbox_sessions": 1
}
}
```
- `box.enabled` 只表示当前 LangBot 实例是否部署了 Box Runtime,不能替代 Workspace entitlement。
- 工具发现层根据 entitlement 隐藏/禁用托管 sandbox。Core 校验 Control Plane 的 entitlement 后,
通过受认证控制连接向 Box Runtime 下发短期 `SandboxAdmissionGrant`,绑定
`instance_uuid + workspace_uuid + execution_generation + entitlement_revision + expires_at + max_sessions + max_managed_processes`
Runtime 只验证和执行该内部 grant,不理解 Pro 等套餐名称,也不相信业务调用方提交的 plan、session ID 或 host path。
- entitlement 缺失、过期或无法验证时失败关闭。并发创建必须用原子 admission 保证同一 Workspace 永远不超过一个 managed session。
- entitlement 被撤销后停止 managed process 并关闭逻辑 sessionWorkspace 数据保留/删除策略随未来 Workspace 释放机制一并决定。
### 4.3 Cloud nsjail 执行模型
- 整个逻辑 SaaS 实例共享一个 Box Runtime 逻辑控制面,M0 运行一个 Runtime replica;不创建每 Workspace Box service、worker pool、PVC、bucket、scheduler、Redis 或 Box 数据库。
- Cloud 显式固定 `box.backend: nsjail`。sandbox 直接作为 Box Runtime 容器内的 nsjail 子进程运行,
不创建 nested Docker container、独立 Pod、microVM 或 warm pool,也不挂宿主机 `docker.sock`
- 符合 entitlement 的 Workspace 首次使用时懒创建一个逻辑 session,内部固定 ID 为 `global`,并强制 `persistent=True`
外部调用方不能选择或覆盖 session ID、persistence、host path 或 backend。
- “全局 sandbox 一直存活”在当前机制中的精确定义是:每个合资格 Workspace 最多一个稳定的 `global` 逻辑 session
它不被 TTL reaper 回收,其 `/workspace` 持久保存;普通命令仍按需启动并退出 nsjail 进程,不能承诺一个空闲 OS 进程永久驻留。
- Box Runtime 重启后,旧进程、attach token、root/tmp/home 和内存 session 状态失效;下一次使用时以相同 Workspace namespace 懒重建
`global` session。持久 `/workspace` 必须继续存在,旧 generation 权限必须失败关闭。
- 共享 Box Runtime 采用单 owner 的 M0 实现;未来多 replica 才引入 session owner lease 和跨 replica 路由,
但 session handle 永远不包含 replica 地址。
- Cloud 首期强制 `network=off`,调用方和 WebUI 不能覆盖。当前 `network=on` 会关闭 nsjail 的独立 network namespace
不能用于共享 SaaS。未来如需联网,必须先实现每 session 独立 netns 和受控 egress,再单独开放。
- Cloud 首期禁止 `START_MANAGED_PROCESS``SandboxAdmissionGrant.max_managed_processes` 固定为 `0`
普通 exec 在同一 Workspace 的 `global` session 内串行执行。未来开放 resident process 前必须增加数量和聚合 CPU/内存上限。
### 4.4 文件与资源边界
- 文件机制沿用当前 nsjail 方案:Box-owned durable volume 上的 Workspace 目录只 bind mount 到对应租户的 `/workspace`
不在首期新增对象存储双向同步服务或文件服务。
- `/workspace` 的持久性来自独立 durable host path,而不是 `persistent=True`;后者只禁止 TTL/普通 shutdown 回收逻辑 session。
Box Runtime 容器必须挂载持久卷,Workspace 数据不能只放在容器可写层。root/tmp/home 可以在 Runtime 重启时丢失。
- Cloud MVP 要求 Core 与 Box Runtime 以相同路径挂载同一持久卷并沿用直接文件读写;现有 exec/base64 fallback 的单文件上限
低于正常附件上限,不能当作等价 Cloud 文件机制。无法共享路径时必须先扩展传输协议,否则 deployment readiness 失败。
- 现有执行前后目录扫描只能提供软检查,不能阻止单次命令写满共享卷。生产 Cloud 的 Workspace 总空间上限必须由
Box-owned volume 的 filesystem project quota/subvolume quota 在写入点原子执行;Box Runtime 负责设置和验证,不能因 Core 看不到路径而跳过。
- Cloud 的 nsjail CPU、内存、PID、单文件和总空间限制使用运维配置统一设置;套餐只决定 session 数量,
不允许 Workspace 放宽 sandbox 上限。
- Box Runtime 必须在 cgroup v2 hard limit、namespace 和 mount 条件满足后才通过 readiness;不能在共享 SaaS 中告警后降级运行。
- 同 Workspace 的 `global` session 可以复用持久 `/workspace`,但不同 Workspace 即使使用相同文件名、进程名或逻辑 session ID,
物理 namespace、路径、进程和 capability 也必须完全隔离;Cloud 首期没有可暴露端口或共享网络 namespace。
### 4.5 非 Pro 和未来外部 E2B
- 非 Pro Workspace 在首期没有 Cloud managed sandbox,直接调用内部 API 也必须被拒绝。
- 未来允许 Workspace 在 WebUI 配置自己购买的远程 E2B endpoint/template/secret;它属于 tenant-owned external sandbox
不消耗 Cloud 的 `managed_sandbox_sessions` 配额,也不能读取其他 Workspace 的凭证。
- 当前 Box Runtime 只有实例级全局 backend 和 E2B credentialWebUI 也没有 Workspace 级配置,因此 BYOK E2B 明确不在首期实现。
- 未来实现时在共享 Box Runtime 内增加按可信 Workspace context 选择 backend/provider 的 registry
仍不创建租户专属 Box 控制面或新协议。
### 4.6 淘汰与暂缓方案
| 状态 | 方案 | 结论 |
| ---------- | ------------------------------------------------ | -------------------------------------------------------- |
| 淘汰 | 每 Workspace 一个 Box service | 组件和空闲成本随 Workspace 线性增长 |
| 淘汰 | 多 Workspace 共享一个活 sandbox/session | 不能承载不可信代码 |
| 淘汰为 MVP | Docker、独立 Pod、microVM 或 warm pool | Cloud v2 首期固定使用 Runtime 容器内 nsjail |
| 淘汰为 MVP | 非 Pro 使用 Cloud managed sandbox | 首期数值 entitlement 为 0 |
| 后续演进 | 多 Box Runtime replica、dedicated pool、BYOK E2B | 保留 provider/ownership 接口,有真实容量或产品需求后实现 |
### 4.7 验收条件
- Pro entitlement 首次使用时懒创建一个 persistent `global` session;重复和并发请求都不能产生第二个 session。
- 非 Pro、entitlement 缺失/过期及伪造 plan 的 API 直调全部失败关闭。
- TTL 不回收 persistent sessionRuntime 重启后进程和临时目录失效,但 `/workspace` 保留并能在下一次使用时安全重建。
- 两个 Workspace 的文件、进程、session、attach token 和 generation 完全隔离;network/managed-process 请求在首期失败关闭。
- Core 与 Box Runtime 通过随机 marker challenge 证明同一共享持久卷;只配置相同路径字符串不算通过。
- Workspace、Skill store、ephemeral root/tmp/home 的 byte quota 与 inode quota 在写入点真实生效;现有目录扫描不被当作硬配额。
- cgroup 或任一硬存储能力不可用时 Cloud Box Runtime readiness 失败;普通 nsjail 因此不会被误当成 production-ready provider。
## 5. D-003SaaS PostgreSQL 与 pgvector
状态:`PARTIALLY IMPLEMENTED — shared schema/pgvector complete; SaaS transaction and deployment gates remain`
当前分支已实现 PostgreSQL shared schema、transaction-local scope、FORCE RLS、Cloud runtime 非 DDL 模式、
同业务数据库 pgvector、显式向量维度和 tenant-scoped vector 主键。一次性 migrator 使用独立凭据、advisory lock
负责建立并校验 runtime role 的最小权限,并完成全量 schema 验证;
普通业务写入贯穿 commit 的 generation-aware fence、与外部副作用同事务的 outbox,以及 generation cutover 后稳定的 durable object 引用尚未实现。
这些 Core 事务原语与生产 Job、凭据发放、备份和回滚流程,以及 runtime credential 的跨 database 连接隔离证明一起,
都是 Cloud v2 的 SaaS activation gate。
### 5.1 已确认的数据库边界
- PostgreSQL 是 SaaS 的业务数据库,不把它扩展成通用 Runtime coordinator、Box session directory 或新控制面数据库。
- M0 使用一个 PostgreSQL business database/shared schema 承载全部 Workspace。创建 Workspace 不创建 database、schema、role 或专属连接池。
- 首版 migrator URL 和 runtime URL 必须归一化到同一 host、port 和 database,但必须使用不同 role。
未来如果 migrator 使用 direct endpoint、runtime 使用 pooler endpoint,只能在两个端点都验证同一个数据库内部 cluster identity 后放宽 host/port 相等。
该 identity 由 migrator 所有并固定,runtime role 只能读取,不能创建、修改或伪造。
- 首版唯一业务 schema 固定为 `public`。migrator 和 runtime 连接都必须满足
`current_schema() = 'public'``current_schemas(false) = ARRAY['public']`;禁止 runtime role 级和 business database 级 `search_path` 覆写。
- migrator 和 runtime session 的安全值固定为 `session_replication_role=origin``row_security=on`
`lo_compat_privileges=off`。runtime role 或当前 business database 作用域内只要存在任意 `pg_db_role_setting` 持久化设置就失败关闭,
即使该设置当前看似等于安全值也不接受;tenant context 只能通过事务内 `SET LOCAL` 建立。
- 业务行显式携带 `workspace_uuid`Repository/Service 的应用层 scope 是第一道边界,PostgreSQL RLS 是第二道边界。
- runtime role 的直接 ACL 固定为:business database 的 `CONNECT``public``USAGE`、全部 allowlisted business table 的
`SELECT/INSERT/UPDATE/DELETE``alembic_version` 的只读 `SELECT`,以及业务表自有 sequence 的 `USAGE/SELECT`
不授予 database/schema `CREATE`、table `TRUNCATE/REFERENCES/TRIGGER`、sequence `UPDATE`、其他对象权限或任何 `WITH GRANT OPTION`
- runtime role 必须是 `LOGIN`,但不得具有 superuser、`BYPASSRLS``CREATEDB``CREATEROLE` 或 replication 属性;
不得在 role membership 中以 granted role、member 或 grantor 任一方向出现;不得拥有 database、schema、table、view、sequence、routine 或 extension
不得持有 column ACL,也不得使用、创建或拥有其他非系统 schema。
- business database 必须安装 `vector`,且 extension catalog 只允许 `plpgsql``vector`;不得存在 FDW、foreign server 或 user mapping。
runtime role 和 `PUBLIC` 都不得有显式 routine ACL 或 parameter `SET/ALTER SYSTEM` ACLruntime role 不得有效执行任何
`SECURITY DEFINER` routine,包括被 allowlisted extension 收编的 routine。普通非 `SECURITY DEFINER` 内建函数的隐式执行权限不在此禁令内。
- PostgreSQL 新 database 默认向 `PUBLIC` 提供的 `TEMP` 是首版在专用业务 database 上明确接受的兼容性决定,
不是 migrator 对 runtime role 的直接 grant;首版不得据此把业务 database 与不受信任工作负载混用。
migrator 在释放 advisory lock 前建立并校验上述精确 allowlistCloud runtime 每次启动都必须重新完成 schema、身份、有效权限和 catalog 负向校验,发现 drift 立即失败关闭。
- PostgreSQL role 是 cluster-wide identity,当前 database 内的 catalog audit 不能证明同一 credential 无法连接 cluster 中的其他 database。
SaaS 生产环境必须使用仅向该 credential 暴露目标 business database 的专用 PostgreSQL cluster/endpoint
或通过已验证的 HBA/proxy policy 证明该 credential 只能连接目标 business database;这项部署隔离仍是未完成的 activation gate。
- 关键租户表使用 `FORCE ROW LEVEL SECURITY`migration/repair/audit 使用独立受控 migrator role。
- 每个租户事务通过 `SET LOCAL` 设置 tenant context,并由统一 `TenantUnitOfWork` 保证设置 context 和业务查询使用同一事务/连接。
禁止使用连接级 session variable 或 `search_path`,避免连接池、PgBouncer、异常回滚和后台任务串租户。
- 一个 `TenantUnitOfWork` 只能访问一个 Workspace。业务写入与对应 business outbox 在同一事务中提交;
写入可以校验由执行层传入的 generation/fencing token,但 Runtime owner、lease 和 Box session directory 不由业务 PostgreSQL 承担。
- SaaS schema、extension 和 policy 只由 release migration job 创建;应用启动角色不执行 `CREATE EXTENSION``create_all` 或自动 migration。
- OSS 继续默认 SQLite,并保留自托管 PostgreSQL 选项;Cloud RLS 约束不让 OSS 强制依赖 PostgreSQL。
### 5.2 pgvector 首期方案
- SaaS 使用 pgvector 作为默认向量数据库,并与业务表使用同一个 PostgreSQL cluster/database
vector schema 可以使用独立 adapter、受控 role 和有上限的 pool,但不新增 Chroma、Milvus 或独立向量数据库服务。
- OSS 默认仍是 SQLite + Chroma,用户可以显式选择 pgvectorCloud 配置 pgvector 失败时必须启动失败,不能静默回退到 Chroma。
- 向量表必须显式保存 `workspace_uuid``knowledge_base_uuid`,并至少以
`(workspace_uuid, knowledge_base_uuid, vector_id)` 建立唯一键/主键和查询条件;服务端生成的 collection name/hash 不是安全边界。
- pgvector adapter 复用相同的 tenant-context/RLS 契约,每次向量操作在自己的事务中执行 `SET LOCAL`
是否复用普通业务 UoW、role 或 connection pool 由实现决定,但 adapter 不能丢弃 tenant metadata。
- `vector(1536)` 不能继续作为无条件硬编码。首期使用无 typmod 的 `vector` 列和显式 `embedding_dimension`
`CHECK (vector_dims(embedding) = embedding_dimension)` 校验;release migration 为允许的维度创建带 dimension predicate 的 expression/partial ANN index。
知识库/model 元数据必须选择已启用维度,写入和查询 mismatch 或未启用维度时失败关闭,不能截断、补齐、退化为无界扫描或换后端。
- `vector` extension、表、索引和 RLS 由 release migration 创建。应用进程不在启动时执行 DDL。
- 0013 如需搬迁 legacy pgvector 数据,migrator 作为源表 owner 先记录每个受保护源表的 `ENABLE/FORCE RLS` 状态,
仅在同一 migration transaction 内临时暂停 RLS,并在 `finally` 中精确恢复各表原状态。
该流程不依赖 superuser 或 `BYPASSRLS`,也不允许在迁移事务外留下已禁用的 RLS。
### 5.3 候选拓扑与未来演进
| 状态 | 方案 | 结论 |
| -------- | ----------------------------------------- | --------------------------------------------------------------------------- |
| 首期决定 | P0. shared database/shared schema | 一个 pool、一套 migration;应用 scope + RLS |
| 后续演进 | P1. 多 shared database shard | 每个 shard 仍承载多个 Workspace,并使用相同 schema;有容量/地域证据后再设计 |
| 后续例外 | P2. dedicated shard | 只作为合规、驻留或超大 workload 的资源等级,不建立第二套代码路径 |
| 淘汰 | schema/database per Workspace | catalog、pool、migration、备份成本随 Workspace 线性增长 |
| 淘汰 | database/schema per replica/Cell/Instance | 把业务数据拓扑错误绑定到计算副本或已删除的产品实体 |
M0 不提前增加始终返回 `primary` 的 resolver、shard router 或 shard binding。
P1 的 resolver、映射、在线迁移、连接池预算、shard-affine replica 和 dedicated shard 细节等到出现容量、地域或合规需求时再设计。
在此之前,direct endpoint 与 pooler endpoint 分离只能通过数据库内部、runtime 不可伪造的 cluster identity 开启,不使用 DNS 名、数据库名或配置声明代替。
### 5.4 备份与生命周期边界
- PostgreSQL PITR 是 database/cluster 级恢复手段,不等同于单 Workspace 恢复。
- Workspace 创建、释放、export、delete、单 Workspace restore 和在线迁移机制本轮暂缓,后续单独决策;
首期不以尚未设计的 export 能力作为数据库架构验收条件。
- 除关系业务数据和 pgvector 的向量/检索字段外,大对象、插件 artifact 和 sandbox 文件仍存放在对象存储或 Runtime 持久卷;
PostgreSQL 保存相应业务元数据和稳定引用。
- tenant-visible usage/billing 业务行可以进入 PostgreSQL;基础设施 log/metric/trace 不进入业务数据库。
高增长业务表在有数据量证据后再决定 retention、时间分区或分析存储。
### 5.5 验收条件
- 故意遗漏应用层 Workspace filter 时,RLS 仍阻止跨租户读写。
- 连接池/事务池复用、异常回滚、并发请求和后台任务不会残留 tenant context;如部署 PgBouncer,也必须覆盖 transaction pooling。
- migration 对 shared schema 只执行一次,不产生 Workspace 级 schema drift;应用启动角色不能执行 DDL。
- 首版拒绝 host、port 或 database 不同的 migrator/runtime URL,并拒绝相同 rolemigrator/runtime 都只解析到 `public`
migrator 在迁移后完成精确 table/sequence/`alembic_version` ACL grant 和正反向 role 校验,runtime 每次启动重新校验。
- runtime role 没有任一方向的 role membership、`WITH GRANT OPTION``search_path` 覆写、对象所有权、其他 schema 访问或非业务对象权限;
专用业务 database 上可继承 PostgreSQL 默认 `PUBLIC TEMP`,但 runtime role 没有直接 `TEMP` ACL。
- migrator/runtime session GUC 保持 `session_replication_role=origin``row_security=on``lo_compat_privileges=off`
runtime role/当前 database 没有任何 `pg_db_role_setting`extension 仅为 `plpgsql/vector` 且 runtime 不拥有 extension
database 中没有 FDW/server/user mapping、runtime 或 `PUBLIC` 显式 routine/parameter ACL、runtime-owned routine 或 runtime 可执行的 `SECURITY DEFINER` routine。
- 生产 deployment 证明 cluster-wide runtime credential 只能连接目标 business database;专用 cluster/endpoint 或 HBA/proxy 隔离未经验证前不得启用 SaaS。
- legacy pgvector 搬迁在非 superuser、非 `BYPASSRLS` 的 table-owner migrator 下可成功,成功、异常和重试路径都精确恢复所有源表的 RLS/FORCE 状态。
- 业务写入和对应 business outbox 在同一事务内具备可证明的提交顺序;外部 generation/fencing token 校验失败时不产生写入。
- pgvector 使用真实 PostgreSQL 集成测试覆盖:两个 Workspace 使用相同 `vector_id`、猜测其他 Workspace ID、
故意遗漏 scope、连接复用、CRUD 和后台任务,全部不能越权。
- embedding dimension 不匹配或 pgvector extension 不可用时失败关闭,不回退到其他向量后端。
- 新建 Workspace 只新增目录与业务行,不创建 database、schema、role 或专属连接池。
## 6. D-004stdio MCP 独立开关
状态:`IMPLEMENTED`
### 6.1 已修复的原问题
- 修复前,stdio MCP 的启用条件只检查 transport 为 `stdio` 且 Box available,没有独立 feature gate。
- 修复前,所有 stdio MCP 使用固定的 `mcp-shared` 逻辑 session,并强制 `persistent=True`
- 在该旧逻辑下,如果多租户 Cloud 只通过 `box.enabled` 开放能力,每个配置 stdio MCP 的 Workspace 都会额外保留一个 persistent sandbox
绕过“每 Workspace 最多一个 managed `global` sandbox”的成本和套餐边界。
### 6.2 首期决定
新增独立实例配置:
```yaml
mcp:
stdio:
enabled: true
```
- OSS 默认 `true`,保持当前本地部署兼容;Cloud v2 通过 `MCP__STDIO__ENABLED=false` 强制关闭。
- 该开关与 `box.enabled``managed_sandbox` entitlement 和 sandbox session 数量相互独立,不能从任一条件推导。
- 后续如开放给特定套餐,可在实例开关之上再叠加 Workspace capability;实例开关为 `false` 时任何 entitlement 都不能绕过。
- HTTP/SSE/其他远程 MCP transport 不受此开关影响。
### 6.3 强制检查点
开关必须同时覆盖:
1. MCP create
2. MCP update 到 stdio
3. transient connection test
4. Core 启动时加载已有 stdio 配置;
5. RuntimeMCPSession/loader 的最终执行门禁;
6. WebUI transport selector 和错误提示。
不能只在 WebUI 隐藏选项。Cloud 配置关闭时,已有 stdio 记录保留但不自动启动,并返回明确的 feature-disabled 错误;
最终 gate 必须位于 Box 分支和 legacy host-stdio 分支之前,不能误报为 `box_unavailable`
也不得创建 `mcp-shared` session 或 stdio 子进程。
### 6.4 验收条件
- Cloud 即使 `box.enabled=true` 且 Workspace 拥有一个 managed sandbox,也无法 create/update/test/start 任何 stdio MCP。
- 直接调用 API、重放旧配置和启动 bootstrap 都失败关闭,且不会产生 `mcp-shared` session、nsjail 进程或额外配额占用。
- OSS 默认行为保持兼容;HTTP/SSE MCP 正常工作。
## 7. 五项决策之间的关系
五项决策共同遵循:
> 多租户共享可信控制面、连接池、只读 artifact 和基础容量;租户独占不可信执行进程、sandbox、secret、可写文件和数据作用域。
- Plugin Runtime 通过“共享 Supervisor + 每 installation 独立 nsjail 进程”降低控制面数量,同时保留进程级租户隔离。
- Box 通过“共享 Runtime + 每个合资格 Workspace 一个持久逻辑 session + one-shot nsjail exec”避免每租户部署服务和空闲容器。
- stdio MCP 独立关闭,防止从 Box availability 隐式产生第二套 persistent sandbox。
- PostgreSQL 和 pgvector 共享数据库组件,但使用显式 tenant key、应用层 scope 和 RLS 防止共享存储变成共享权限。
- 订阅管理只在闭源 Control Plane 维护套餐与计费规则,并向开源 Core 投影签名/版本化 entitlement
Core/Runtime 执行通用 capability 和数值限额,不复制套餐名称或计费逻辑。
- 目录同步只在启动或恢复时读取全量快照;常态变更按 Workspace 聚合为签名增量,新增租户不会使每次目录事件退化为全实例重投影。
## 8. 当前不做分布式时仍保留的能力
1. 所有运行时协议继续携带稳定 `instance_uuid``workspace_uuid` 和 execution generation;不能依赖进程地址表达身份。
2. Core、Plugin Runtime 和 Box Runtime 的本地进程表不能成为 durable desired state 或撤销状态的唯一真相。
3. 创建、重试、回调、outbox 和 worker 注册使用稳定 idempotency key;重复投递不能产生第二个 owner 或副作用。
4. Plugin installation 和 Box session 使用稳定 owner abstraction;启用第二个 replica 前再实现带 expiry、CAS 和 fencing token 的 lease。
具体 lease store 后续决定,不预设复用业务 PostgreSQL,更不因此新增 Runtime/Box 数据库。
5. Repository/UoW 不允许无边界跨 Workspace 事务;`workspace_uuid` 从第一天就是内部路由与分片候选键。
6. schema migration、任务扫描、监控聚合和运维接口不能假设永远只有一个 Core 进程。
7. 外部 API 不暴露 replica、worker 或 shard 标识;未来扩容不改变 Workspace URL、UUID 或客户端协议。
8. 只有出现容量、可用性、地域或合规需求时才增加 replica/shard;预留协议不等于现在部署额外组件。
9. 每个 Core replica 保存自己的事件消费 cursor,因为 entitlement cache 是进程本地状态;PostgreSQL 中的 projection high-water mark
、snapshot coverage 和 inbox 仍由所有 replica 共享,用于幂等投影和冲突检测。事件页携带签名 high-water,副本追平前不能续期
ready;不能让一个 replica 的共享 cursor 使其他 replica 跳过本地 cache 刷新。
## 9. 本轮明确不做的事情
- 不合并 plugin installation 进程,即使插件和版本完全相同;只允许共享摘要校验后的只读代码/依赖文件。
- 不允许插件 manifest 声明或覆盖 CPU、内存、PID、文件或存储上限。
- 不为每个 Workspace 创建 Plugin Runtime、Box service、database、schema、role、bucket、PVC 或消息队列。
- 不在 Cloud v2 首期使用 Docker sandbox、microVM、warm pool 或非 Pro managed sandbox。
- 不在首期实现 Workspace 级 BYOK E2B WebUI 配置。
- 不在 Cloud v2 首期支持 stdio MCP,也不让 Box availability 隐式开启它。
- 不把业务 PostgreSQL 用作未决定的 Runtime/Box 通用协调数据库,不在缺少容量证据时引入 Redis、Kafka 或新 scheduler。
- 不在本轮实现 Workspace export、释放、单租户恢复或在线迁移;具体生命周期另行决策。
- 不实现多个 CloudInstance、Workspace Placement 或 Cell Router;未来分布式只作为单逻辑实例内部的副本和分片能力。
- 不修改旧 Space 部署模型;Cloud v2 继续按绿地方案设计。
+252
View File
@@ -0,0 +1,252 @@
# Cloud v2 multi-tenant verification report
Date: 2026-07-24
Status: `FOUNDATION AND CONTROL PLANE VERIFIED — PRODUCTION ACTIVATION REMAINS GATED`
This report records the implementation and verification evidence for one
logical LangBot instance serving multiple Workspace tenants. It covers the
open-source Core, the shared Plugin/Box runtimes, the closed Space adapter, and
the Space Cloud v2 modular-monolith control plane. It does not claim that the
production Cloud deployment may enable `CLOUD_V2_ENABLED` yet.
## Repository refs and scope
- LangBot Core: branch `feat/multi-tenants`
- langbot-plugin-sdk: branch `feat/multi-tenants`, commit
`e7d946af4a6b1494fbe74627c1815ace19ac8991`
- langbot-space: branch `feat/cloud-v2-control-plane`
- Closed Core adapter: `langbot-space/cloud-adapter`
- SDK protocol/package version: `0.4.18`
Core pins the exact pushed SDK commit above. Cloud v2 is a greenfield
multi-Workspace deployment and does not provision one Pod, database, queue, or
Runtime per tenant. Legacy Space Pods remain a compatibility surface only.
## Implemented product and security boundaries
### Workspace and identity
- SaaS has one stable `instance_uuid`; `workspace_uuid` is the tenant key.
- Registering an Account creates its personal Workspace and owner Membership in
the same PostgreSQL transaction.
- A Workspace may contain multiple users with owner, admin, and member roles.
Invitation acceptance is token-hash based, email-bound, single-use,
concurrency-safe, and member-limit checked while locked.
- Community remains exactly one local Workspace while supporting multiple
Accounts and fixed RBAC roles. Installing the closed adapter is required to
inject the SaaS policy; configuration alone cannot activate it.
- Disabled or deleted Accounts cannot use password/code/OAuth login, sessions,
refresh/access tokens, or personal access tokens.
### Closed Space control plane
- `CLOUD_V2_ENABLED` defaults to false. Invalid or incomplete configuration
fails startup, and disabled endpoints return 503.
- Space owns Workspace/Membership/Invitation, versioned plans, subscriptions,
entitlements, usage events, directory outbox, and signed instance manifests.
- Free and Pro are Workspace plans. Pro projects one managed global sandbox;
Free projects none. Both force stdio MCP off in the first release.
- Subscription changes use Workspace advisory locking, expected entitlement
revision, one-live-order uniqueness, idempotent usage ingestion, period-end
downgrade, renewal, and lazy expiry settlement.
- Directory snapshot and per-Workspace delta reads use repeatable-read,
read-only transactions with a high-water cursor. Core stores a replica-local
consumer cursor and shared PostgreSQL projection state, snapshot coverage,
and inbox rows.
- The `/cloud` page selects a Workspace and shows its independent subscription,
entitlement, limits, and usage. When Cloud v2 is disabled or the backend does
not expose the feature field, the complete legacy Welcome/Pod UI is used.
### PostgreSQL and pgvector
- SaaS business data uses one PostgreSQL shared schema with application scope
plus forced RLS. Cloud directory writes are separated from local tenant
writes.
- Projected Account and Membership revisions are monotonic. Tombstones remove
memberships, and stale revisions cannot resurrect them.
- pgvector shares the business PostgreSQL database and remains
Workspace-scoped.
- Space runs versioned SQL migrations before application seeds. A fresh
database, a partial pre-migration database, and an existing-baseline path
converge without startup-time `AutoMigrate`.
### Shared Plugin Runtime
- One instance-scoped trusted Supervisor serves multiple Workspaces.
- Every enabled installation has its own nsjail process bound to
`(instance, workspace, execution generation, installation, runtime revision,
artifact digest)`.
- Verified same-digest code and dependency files are mounted read-only and may
be shared; home, tmp, data, process namespace, registration capability, and
cgroup are private.
- Instance configuration owns CPU, memory, PID, open-file, and file-size
limits. Plugin manifests cannot increase them.
- Memory includes swap: nsjail receives `memory.max` and `swap.max=0`.
- Unexpected worker exit is recovered by a completion callback with bounded
per-installation exponential backoff. Remove, reconcile, Runtime shutdown,
and container SIGTERM perform a graceful-to-SIGKILL bounded reap.
### Shared Box Runtime and MCP
- One shared Box control plane serves Workspace-bound logical sessions.
- Cloud grants allow at most one persistent `global` session for an entitled
Workspace and no managed processes in the first release.
- Cloud is fixed to nsjail and network-off. Core and Box prove the shared
durable Workspace mount with an authenticated marker challenge.
- stdio MCP is independently gated and forced off for Cloud v2.
- The current nsjail Box backend does not provide hard byte/inode quotas, so
Cloud readiness correctly fails closed instead of silently using a soft
directory scan.
## Automated verification
### LangBot Core
```text
uv run --no-sync pytest -q
2590 passed, 32 skipped, 177 warnings
real PostgreSQL migration, pgvector, and release-migrator suites
21 passed, 11 warnings
uv run --no-sync ruff check .
uv lock --check
git diff --check
passed
```
The full suite ran without the closed adapter installed, proving the open-source
single-Workspace/multi-user path remains standalone. Focused closed-adapter,
directory projection, runtime connector, Box cleanup, and configuration suites
also passed with the adapter installed.
### Plugin SDK and real Linux runtime
```text
SDK full suite
1226 passed, 22 existing warnings
Ruff check and format check
git diff --check
passed
```
A privileged Linux test container with host cgroup namespace ran one shared
Runtime and two Workspace installations:
- both workers referenced the same artifact inode;
- home, tmp, and data inodes were distinct;
- each plugin saw only PID 1 in its private PID namespace;
- a tampered binding and an unknown installation were rejected;
- the control token was absent from worker environments;
- cgroups were distinct with `memory.max=134217728`, `memory.swap.max=0`,
`pids.max=32`, and `cpu.max=500000 1000000`;
- touching 256 MiB exited with code 137 without swap growth;
- the 32nd fork failed with `EAGAIN`;
- reconcile and container SIGTERM removed the worker cgroups.
The same run started the Runtime from a non-root working directory, covering
absolute nsjail mount-source normalization.
### Space backend, adapter, and frontend
```text
go test ./...
passed
fresh PostgreSQL app startup, partial-baseline migration,
Cloud v2 migration rerun, and control-plane integration
passed
closed adapter pytest and Ruff
passed
pnpm exec tsc --noEmit
targeted ESLint for changed Cloud files
pnpm build
passed; 47 routes generated
```
The PostgreSQL checks started from an empty database and verified all 32
registered migrations in order, Cloud v2 Free/Pro seeds, legacy plan seeds,
Cloud columns/indexes, and repeatable reruns.
## Cross-service and browser E2E
### Signed Space-to-Core directory projection
Using an isolated Space PostgreSQL database and a migrated Core PostgreSQL
database:
1. Space issued a signed manifest for the fixed LangBot instance.
2. Space returned a directory snapshot at cursor 5 containing two Workspaces,
two projected Accounts, and owner/admin memberships.
3. Core verified the signature, instance, release, validity window, and
capability before injecting the Cloud Workspace policy.
4. Core stored both Workspaces, all active Memberships, both projected
Accounts, snapshot coverage, inbox entries, and cursor 5.
5. Account-field-only projection revisions and Workspace directory revisions
remained independent, and cross-Workspace Account conflicts failed closed.
### Real browser Cloud v2 flow
A real local browser operated the Space frontend and backend:
1. An owner logged in and saw the automatically created personal Workspace.
2. A second Account invited the owner as admin to another Workspace.
3. The selector showed both Workspaces and switched between them without stale
subscription or usage state crossing the selection.
4. Each Workspace showed its own Free entitlement and limits.
5. Pro checkout opened for the selected Workspace. With no configured payment
channels, confirmation was visibly disabled and no order was created.
6. No browser errors occurred; only the pre-existing i18next warning for the
short `en` locale was observed.
The legacy feature-flag branch was then covered by API/static checks and the
production build: false or missing `cloud_v2_enabled` renders the original
Welcome/Pod client; a failed web-config request renders an explicit retry
instead of guessing a deployment mode.
### Deliberate Core startup failure
Core completed signed manifest verification and directory projection, then
stopped at the Box readiness gate because the current nsjail backend cannot
prove hard Workspace byte/inode quota enforcement. Connector shutdown and
reconnect tasks were cleanly reaped; no event-loop or never-awaited coroutine
warning remained.
This is a successful fail-closed acceptance result, not a passing production
Cloud boot.
## Remaining production activation gates
Cloud v2 must remain disabled until these gates are closed:
- Box provides and proves hard byte and inode quotas for Workspace, Skill,
root, home, and tmp storage.
- Plugin installation writable data receives an operator-owned hard total
disk quota.
- Plugin and future networked Box workloads have tenant-safe egress/SSRF
policy.
- Plugin Runtime adds jitter, global restart concurrency limiting, and a
Runtime-level circuit breaker, then passes systemic-failure injection.
- M0 rolls Core and Plugin Runtime together until authenticated Runtime takeover
or an owner lease/fencing protocol exists.
- Account registration moves New API provisioning to an in-process,
transactional outbox reconciler keyed by Space Account UUID.
- EPay and Stripe fulfillment bind provider identity, amount, currency,
channel/session, payment status, and expiry to the locked order transaction;
provider transaction IDs are unique and replay-safe.
- Payment-session creation and stale `processing` orders gain in-process
reconciliation.
- Cloud v2 subscription service periods are stored immutably and included in
recognized-revenue reporting.
- Production migration Job, backup/rollback, PostgreSQL credential/network
boundaries, horizontal-replica fault injection, Workspace release/export,
deletion, and restore semantics are completed.
These gates intentionally add no tenant-specific service. They are implemented
inside the existing Space, Core, Plugin Runtime, Box Runtime, and PostgreSQL
components to preserve the architecture goal: near-zero static cost for a new
Workspace.
@@ -0,0 +1,941 @@
# LangBot Workspace 多用户与 SaaS 多租户架构
状态:`ARCHITECTURE BASELINE — isolation kernel implemented; SaaS activation gates remain`
本文描述 Cloud v2 的目标架构和安全边界。详细的 Runtime、Box、PostgreSQL、pgvector 与 stdio MCP 决策以
[pending-architecture-decisions.md](./pending-architecture-decisions.md) 为权威来源;已经落地的实现选择记录在
[implementation-decisions.md](./implementation-decisions.md)。
“隔离内核已实现”仅表示开源 Core/SDK 已具备多租户数据和运行时隔离所需的基础能力,
不表示闭源控制面、计费、生产部署或 Cloud v2 已经可以上线。
## 1. 架构决策摘要
Cloud v2 采用以下模型:
> SaaS 对外只有一个逻辑 LangBot 实例,全部 Workspace 都是该实例内的租户;
> 开源 Core 提供完整隔离内核,闭源 Cloud Control Plane 管理 SaaS 目录、订阅、权益和计费。
核心决策如下:
1. `Workspace` 是数据、成员、权限、用量和不可信执行的租户边界,不是一个 Pod、namespace、数据库或独立 LangBot 部署。
2. SaaS 注册 Account 时自动创建个人 Workspace;这只新增目录与业务记录,不创建租户专属服务、数据库、队列或 Runtime。
3. OSS 每个 LangBot 实例只能存在一个 Workspace,但该 Workspace 可以有多个 Account、邀请和固定角色。
4. SaaS 才允许一个 Account 拥有或加入多个 Workspace,并在 WebUI 中切换当前 Workspace。
5. MVP 可以各运行一个 Core、Plugin Runtime 和 Box Runtime 进程;未来增加副本或 PostgreSQL shard 仍属于同一个逻辑实例的内部扩展,不改变产品模型和外部 API。
6. 一个共享 Plugin Runtime 控制面管理所有 Workspace,但每个运行中的 plugin installation 独占一个 nsjail 进程;enabled-resident 是 desired semantics,只读代码和依赖可按已验证摘要共享。
7. 一个共享 Box Runtime 管理所有 Workspace;首期符合 entitlement 的 Workspace 最多拥有一个持久 `global` 逻辑 sandbox,实际命令继续以 nsjail 子进程执行。
8. SaaS 业务数据使用 PostgreSQL shared schema、应用层 scope 与 RLS 双重隔离;pgvector 位于同一个业务数据库并作为 SaaS 默认向量后端。
9. stdio MCP 有独立实例开关,Cloud v2 首期强制关闭,不能由 Box availability 或套餐能力隐式开启。
10. 闭源 Control Plane 可以作为模块化单体复用现有账户、支付和运营能力,但历史 Cloud 的租户专属部署模型不进入新架构。
11. Workspace 创建、释放、export、单 Workspace restore 和在线迁移的具体流程仍待后续决策。
本轮重构的最高目标是:
> 共享可信控制面和基础设施池,隔离不可信执行单元;减少独立部署和常驻组件,使新增 Account 或空 Workspace 的静态成本接近零。
减少组件数量不意味着合并安全边界。插件进程、sandbox、secret、可写文件和租户数据仍必须严格隔离。
## 2. 范围与非目标
### 2.1 本方案覆盖
- OSS 单 Workspace 多用户、邀请和固定 RBAC。
- SaaS 多 Workspace 账户、成员和 Workspace 切换模型。
- HTTP、WebSocket、API Key、Bot、Webhook、后台任务和内部调用的可信 Workspace 上下文。
- Bot、Pipeline、Provider、Knowledge、Plugin、MCP、RAG、Session、Storage 和 Monitoring 的租户隔离。
- Plugin Runtime 与 Box Runtime 的共享控制面和进程级隔离。
- SaaS PostgreSQL shared schema、RLS 与 pgvector 边界。
- 开源 Core 与闭源 Control Plane 的职责、协议和故障边界。
- 当前单副本运行和未来同一逻辑实例内横向扩展的兼容约束。
- 分阶段实施、激活门禁和验收策略。
### 2.2 本方案不覆盖
- 兼容或原地升级历史 Cloud 的租户专属部署方案。
- 为每个 Workspace 创建独立服务、数据库、schema、role、bucket、PVC、队列或 Runtime。
- 当前阶段实现多副本调度、跨地域 active-active 或 PostgreSQL 在线分片迁移。
- 第一版自定义角色、SAML、SCIM 或企业离线授权。
- 第一版 Workspace 级 BYOK E2B WebUI 配置。
- Cloud v2 首期 stdio MCP。
- Workspace export、释放、单租户恢复和在线迁移的具体产品流程。
历史客户数据、账户和财务记录如需迁移,应单独立项;旧部署拓扑不作为本架构的设计约束。
## 3. 术语与不变量
### 3.1 术语
| 术语 | 定义 |
| --- | --- |
| Account | 登录主体。OSS 中是实例本地账户;SaaS 中是全局账户 |
| Workspace | 逻辑 LangBot 实例内的租户,是资源、成员、权限、用量和不可信执行的首要边界 |
| Membership | Account 与 Workspace 的关系,包含固定角色、状态和权限版本 |
| Invitation | 邀请一个 Account 或邮箱加入 Workspace 的一次性凭证 |
| Logical Instance | 对外唯一的 LangBot 服务与安全域,拥有稳定 `instance_uuid`,不等同于某个进程或 Pod |
| Replica | Core、Plugin Runtime 或 Box Runtime 的短期内部运行副本,不是产品实体 |
| Execution Generation | Workspace 执行所有权和撤销的单调代数,用于隔离旧任务、旧连接和故障转移 |
| Billing Account | SaaS 付款主体,可以为一个或多个 Workspace 付费 |
| Entitlement | Control Plane 签发、Core 与 Runtime 本地执行的功能和数值额度快照 |
| Cloud Control Plane | 闭源 SaaS 控制面,管理全局身份、Workspace 目录、订阅、权益、计费和生命周期 |
| LangBot Core | 开源数据面,执行 Bot、Pipeline、Plugin、MCP、RAG 等业务并实施最终授权与隔离 |
当前代码中的 `placement_generation` 字段在迁移完成前保留兼容;其架构语义和目标命名均为
`execution_generation`,不表达 Workspace 属于某个产品级部署单元。
### 3.2 必须始终成立的不变量
1. SaaS 只有一个稳定 `instance_uuid`;所有副本共享该身份。
2. `replica_id``worker_id`、Pod 名称、进程地址和数据库连接地址都是短期运行信息,不能进入业务资源的永久主键或外部 URL。
3. `workspace_uuid` 是租户数据、任务、缓存、文件、日志、用量和运行时隔离的稳定键,也是未来内部路由与分片的候选键。
4. OSS 一个实例最多一个 WorkspaceSaaS 才能激活多个 Workspace。
5. 一个 Workspace 可以有多个 Account;一个 SaaS Account 可以加入多个 Workspace。
6. 所有租户业务资源都具有非空 `workspace_uuid`,并使用 `(workspace_uuid, resource_uuid)` 定位。
7. Workspace 选择器只是路由输入,不是授权凭证;服务端必须重新验证 Account、Membership、资源所有权和权限。
8. API Key、Bot、Webhook、后台任务、Plugin 与 Box 调用从可信所有权或绑定派生 Workspace,不能信任调用方自报 scope。
9. SaaS 缺少有效 Workspace 上下文时必须失败关闭,不能回退到第一个、最近或 OSS 默认 Workspace。
10. Core 是资源访问、运行时授权和 entitlement 执行的最后一道边界;Control Plane 不同步代理每条消息或普通资源请求。
11. 一个不可信插件进程只能属于一个 installation;一个 sandbox/session 只能属于一个 Workspace。
12. execution generation 失效后,旧任务、连接、回调和副作用必须被拒绝。
13. 本地进程表、缓存和临时目录都可重建,不能成为 desired state、撤销状态或业务数据的唯一真相。
14. 创建空 Workspace 不启动插件 worker、sandbox 或租户专属常驻组件。
15. 未来横向扩展不能改变 Workspace UUID、外部 API、权限模型或隔离语义。
## 4. 产品与部署模型
### 4.1 SaaS 逻辑拓扑
```mermaid
flowchart LR
User["Browser / API / Bot traffic"] --> Edge["SaaS Edge"]
User --> CP["Closed Cloud Control Plane<br/>directory + subscription + billing"]
Edge --> Core["One logical LangBot instance<br/>Core replica pool; MVP = 1"]
CP -->|"signed manifest, directory projection,<br/>entitlement and desired state"| Core
Core -->|"usage outbox and observed state"| CP
Core --> PG["Shared PostgreSQL business database<br/>RLS + pgvector"]
Core --> PluginRT["Shared Plugin Runtime<br/>trusted supervisor"]
Core --> BoxRT["Shared Box Runtime<br/>trusted supervisor"]
PluginRT --> PluginA["Workspace A installation<br/>isolated nsjail process"]
PluginRT --> PluginB["Workspace B installation<br/>isolated nsjail process"]
BoxRT --> SandboxA["Workspace A<br/>persistent global logical sandbox"]
BoxRT --> SandboxB["Workspace B<br/>persistent global logical sandbox"]
Core --> ObjectStore["Shared durable object storage<br/>Workspace-scoped keys"]
```
这里的“一个逻辑实例”是一个服务、安全域和稳定身份,不是“永远只有一个 OS 进程”。
MVP 不实现分布式,但从第一天保留内部扩展所需的身份、幂等、generation 和 owner 抽象。
### 4.2 容量演进
| 阶段 | 内部部署形态 | 新 Workspace 静态成本 | 启用条件 |
| --- | --- | --- | --- |
| M0 单副本 MVP | 一个 Core、一个共享 Plugin Runtime、一个共享 Box Runtime、一个 PostgreSQL business database | 只新增目录和业务行 | 当前目标 |
| M1 同逻辑实例横向扩展 | 按容量增加 Core/Runtime 副本;使用 owner lease、fencing 和 generationPostgreSQL 可增加 shared shard | 不创建 Workspace 专属部署 | 出现容量或可用性证据后 |
| M2 Dedicated 资源等级 | 特定 workload 使用独享 worker pool、sandbox class 或 database shard,但沿用相同身份、协议和 schema | 仅购买该等级的客户承担 | 合规、驻留或超大负载需求 |
M1 是 M0 的透明扩容,M2 是相同架构下的资源等级。外部 API 只认识稳定的
`instance_uuid``workspace_uuid`,不认识 replica、worker、pool 或 shard。
### 4.3 当前不做分布式时必须预留的能力
1. 运行时协议携带稳定 `instance_uuid``workspace_uuid``execution_generation`,不依赖进程地址表达身份。
2. Plugin installation 和 Box session 使用稳定 owner 抽象;启用第二个副本前再实现带 expiry、CAS 和 fencing token 的 lease。
3. 创建、重试、回调、worker 注册和 outbox 使用稳定 idempotency key,重复投递不能产生第二个 owner 或副作用。
4. Repository/UoW 不允许无边界跨 Workspace 事务;`workspace_uuid` 可直接作为未来 shard key。
5. schema migration、后台任务扫描、监控聚合和运维接口不能假设永远只有一个 Core 进程。
6. Runtime 重启通过 durable desired state reconciliation 恢复,不依赖原进程或本地 cache。
7. 只有出现容量、可用性、地域或合规证据后才增加副本、lease store 或 shard router;预留协议不等于提前部署组件。
### 4.4 组件边界
- Core、Plugin Runtime 和 Box Runtime 必须保持独立进程身份、容器和 security context。M0 中 Core 与 Plugin Runtime
需要处于同一 rollout/restart unit;在实现受认证 takeover 或 owner lease/fencing 前,Core 不能单独重启后接管仍存活的 Runtime。
- Core 不能继承 nsjail、cgroup 或 mount namespace 所需的高权限。
- Plugin Runtime 与 Box Runtime 不合并为一个高权限进程。
- MVP 不新增 Runtime 专用数据库、Box 专用数据库、Kafka、Redis、租户级 scheduler 或 artifact service。
- 可信 supervisor、数据库连接池、只读 artifact cache 和基础容量可以多租户共享。
## 5. OSS 与 SaaS 产品行为
### 5.1 能力矩阵
| 能力 | OSS | SaaS |
| --- | --- | --- |
| Workspace 数量 | 实例固定一个 | Account 可拥有或加入多个,受 ProductPolicy 约束 |
| Workspace 成员 | 多用户 | 多用户,受 entitlement 约束 |
| 邀请成员 | 支持 | 支持 |
| 固定 RBAC | 支持 | 支持 |
| 自定义角色 | 不支持 | 后续商业能力 |
| Workspace 创建 | 首次初始化创建唯一 Workspace | 注册自动创建个人 Workspace;后续创建受 ProductPolicy 约束 |
| Workspace 切换 | 无需展示 | 支持 |
| 订阅与计费 | 无远端依赖 | 闭源 Control Plane 管理 |
| 租户隔离 | 完整实现 | 完整实现 |
OSS edition policy 应表达为:
```text
workspace_limit = 1
members_enabled = true
invitations_enabled = true
fixed_rbac_enabled = true
multi_workspace_enabled = false
```
不能用 `member_limit = 1`、关闭邀请或移除 RBAC 来实现单租户限制。
### 5.2 OSS 初始化和邀请
首次初始化在一个事务中完成:
1. 创建本地 Account。
2. 创建实例唯一 Workspace。
3. 创建 owner Membership。
4. 创建默认 Pipeline、metadata 等 Workspace 初始资源。
5. 标记实例初始化完成。
初始化后默认关闭公开注册。后续用户由 owner/admin 创建一次性 Invitation,注册或登录后接受邀请并加入唯一 Workspace。
OSS 后续注册不创建第二个 Workspace。未配置 SMTP 时,系统返回只展示一次的邀请链接供管理员通过可信渠道发送。
### 5.3 SaaS 注册和邀请
普通注册由 Control Plane 通过幂等工作流完成:
1. 创建或确认全局 Account 与 AuthIdentity。
2. 创建 personal Workspace 和 owner Membership。
3. 创建初始 Subscription/Entitlement 投影。
4. 完成 verified email、速率限制和基础风控。
5. 将 Account、Workspace 和 Membership 投影到 Core。
6. Core 达到要求的目录 revision 后返回可访问 route。
注册只创建逻辑记录,不启动 Runtime 或租户专属基础设施。
通过邀请注册的新用户也创建自己的 personal Workspace,同时加入受邀 Workspace;已注册用户接受邀请时只新增目标 Membership。
个人 Workspace 与团队 Workspace 的付费关系必须由 ProductPolicy 明确,不允许代码根据名称或创建路径隐式推断。
### 5.4 Invitation 安全规则
- token 使用至少 256-bit 加密安全随机数,数据库只保存 hash。
- token 具有 `expires_at``accepted_at``revoked_at`,只能使用一次。
- Membership 创建与 token 消费在同一事务中提交。
- Invitation 不能授予 ownerowner 转移使用独立流程。
- SaaS 接受邀请时必须验证目标邮箱;OAuth 邮箱相同不能跳过 token 和显式确认。
- Workspace 必须始终至少有一个 active owneradmin 不能移除或降级 owner。
- 浏览器邀请链接把 secret 放在 URL fragment 中,页面读取后立即清除 fragment,并只短期保存在 `sessionStorage`
### 5.5 固定 RBAC
Core 权威定义 `owner``admin``developer``operator``viewer` 固定角色。
权限按能力划分,例如资源查看、资源管理、运行操作、成员管理、provider secret 管理、审计查看和数据导出。
规则:
- 普通资源可见性不自动授予 secret 可见性。
- 跨 Workspace 猜测资源 UUID 返回 404,不泄露存在性。
- 同 Workspace 资源存在但缺少权限时返回 403。
- 最后一个 owner 不能被删除或降级。
- 前端隐藏或禁用无权限入口只改善体验;后端仍必须执行所有授权检查。
## 6. 开源与闭源职责边界
### 6.1 LangBot Core OSS
Core 负责:
- 本地 Account、Workspace、Membership 和 OSS Invitation。
- 固定 RBAC 与单 Workspace edition policy。
- 业务资源及其 Workspace scope。
- HTTP、WebSocket、后台任务和运行时请求上下文。
- Plugin、MCP、RAG、Box、Session、Storage 和 Monitoring 隔离。
- SaaS Account/Workspace/Membership 的版本化执行投影。
- InstanceManifest、EntitlementSnapshot 和 Runtime 控制通道验证。
- 通用 capability 与数值 quota enforcement。
- UsageEvent/business outbox 和基础安全审计。
Core 是 Bot、Pipeline、Model、Knowledge、Plugin installation、MCP configuration 和 Monitoring 数据的权威来源,
也是每个业务和运行时请求的最终授权边界。
### 6.2 Closed Cloud Control Plane
Control Plane 负责:
- SaaS 全局 Account、AuthIdentity、Session、OIDC 和后续 SSO。
- SaaS Workspace、Membership 和 Invitation 的权威目录。
- Workspace 创建、暂停、归档和删除工作流。
- BillingAccount、Product、PlanVersion、Price、Subscription、Invoice、Refund 和 provider event。
- Entitlement 计算、签名与版本。
- Usage ledger、聚合、额度和欠费策略。
- 实例 manifest、release、capacity、内部 desired state 和 observed state。
- SaaS 运营后台、平台角色和高级审计。
首期不把这些职责拆成多个租户、计费和调度微服务。推荐以一个独立于 Core 的闭源模块化单体承载,
并通过模块边界复用已有账户、OAuth、支付、邮件和运营能力。历史 Cloud 的租户专属部署代码不复用。
Control Plane 不保存 Bot、Pipeline、Model 或 Knowledge 等业务内容,也不代理普通消息执行。
### 6.3 SaaS Adapter
Core 中只保留薄的协议适配层:
- 验证 InstanceManifest、Account token 和 JWKS。
- 消费 DirectoryEvent 并写入本地投影。
- 缓存并验证 EntitlementSnapshot。
- 将 UsageEvent 写入 durable outbox。
- 接收 execution desired state 并上报 observed state。
适配层不得 monkey patch ORM、绕过 Core 权限检查或在普通资源请求中同步调用 Control Plane。
### 6.4 Source of Truth
| 数据 | OSS | SaaS |
| --- | --- | --- |
| Account、Workspace、Membership | Core 本地数据库 | Control Plane 权威,Core 保存版本化投影 |
| Invitation | Core 本地数据库 | Control Plane 权威,不向 Core 投影 pending secret |
| Bot、Pipeline、Model、KB、Plugin、MCP | Core | Core |
| Subscription、Payment、Invoice、Usage ledger | 无远端依赖 | Control Plane |
| Feature 和 quota | 本地 edition policy | Control Plane 签发,Core/Runtime 验证执行 |
| Execution generation | OSS 固定本地值 | Control Plane desired stateCore 执行 |
| 运行时授权 | Core | Core 根据本地投影和 entitlement 执行 |
SaaS 不维护两套可写目录。Control Plane 是目录权威写模型;Core 只保存带 revision 的执行投影。
## 7. 控制面协议
### 7.1 InstanceManifest
仅设置 `system.edition=cloud`、环境变量或前端 feature flag 不得启用 SaaS 多 Workspace。
Cloud bootstrap 必须验证由预置根信任签名的 InstanceManifest,并据此安装闭源 Workspace policy。
Manifest 至少绑定:
```text
iss, aud, sub, jti, iat, nbf, exp
instance_uuid
release
capabilities
tenant_isolation_version
execution_generation
delegated issuers and keyset revision
```
签名错误、audience 不匹配、过期、generation 回滚或信任链缺失时必须失败关闭,不能降级为 OSS 默认 Workspace。
### 7.2 DirectoryEvent 与目录新鲜度
Control Plane 通过 transactional outbox 发布 Account、Workspace 和 Membership 的版本化事件。
Core 使用 inbox 按 `event_id` 去重,以 aggregate revision 拒绝旧写,并追踪连续应用水位。启动时读取一个 PostgreSQL
`REPEATABLE READ` 事务内生成的签名全量 snapshot;运行时先消费携带当前 high-water 的签名事件页,再只请求该页涉及的 Workspace 签名增量。
增量响应不携带新的事件 cursor,因此即使其内容已包含并发提交的后续 revision,也不能跳过尚未消费的事件。
要求:
- 事件和 batch 经过实例绑定的强认证与签名。
- 重复、乱序、延迟、断流和全量 replay 都安全。
- 删除使用 tombstone。
- 新实例先导入带 high watermark 的 snapshot,再消费增量。
- 常态目录更新成本与本页发生变化的 Workspace 数量相关,不得为每个 `directory.changed` 重新读取和投影全部 Workspace。
- 每个 Core replica 独立保存进程内消费 cursor,以确保各自的 entitlement cache 都看到事件;共享 PostgreSQL 保存投影
high-water mark、全量 snapshot coverage 和 inbox。同一事件被多个 replica 消费时,第二个 replica 验证已有 receipt
snapshot coverage 内缺少的 receipt 可以补写,coverage 之外缺失则失败关闭。只有本地 cursor 追平签名 high-water 后才续期 ready。
- projection 未就绪或落后于授权 lease 要求时,交互与自动化请求按策略失败关闭。
- SaaS pending Invitation、email 和 token hash 不进入 Core 投影。
MVP 可采用一个共享、原子且可恢复的 Control Plane store;未来多副本不能继续使用进程内状态承担一次性 token 或目录水位。
### 7.3 EntitlementSnapshot
Entitlement 使用版本化签名快照,至少绑定:
```text
instance_uuid
workspace_uuid
plan_revision
entitlement_revision
status
features
limits
nbf, exp, grace_until
```
Core 校验 issuer、audience、subject、instance、revision、时间和签名;旧 revision 不覆盖新快照。
套餐名称和价格规则只存在于闭源 Control PlaneCore 与 Runtime 只理解通用 capability 和数值限额。
Control Plane 故障时,已缓存且仍有效的快照可继续执行;过期后只能进入明确、有限的 grace 模式或失败关闭。
### 7.4 UsageEvent 与 outbox
用量事件 append-only、至少一次投递,Control Plane 按 `event_id` 去重。事件至少包含:
```text
event_id
instance_uuid
workspace_uuid
execution_generation
meter
quantity_integer
unit
source
occurred_at
entitlement_revision
schema_version
```
Core 不计算账单金额,也不在普通请求中同步扣费。业务写入与相应 business outbox 必须在同一事务中提交;
generation-aware write fence 与 outbox 原子性尚是 SaaS 激活门禁。
### 7.5 Desired state 与 observed state
闭源控制面发布版本化的 release、capacity 和 execution desired stateCore/Runtime 幂等 reconcile 并上报 observed state。
desired state 只描述同一逻辑实例内部的执行所有权和容量,不产生新的产品级实例或租户实体。
Workspace 安全状态由 directory revision 决定,订阅状态由 entitlement revision 决定,执行撤销由
execution generation 决定。三者取最严格有效状态,但任何通道都不能修改另一个通道的权威字段。
## 8. 身份、鉴权与请求上下文
### 8.1 上下文模型
租户业务入口统一解析不可变的 `RequestContext`
```python
@dataclass(frozen=True)
class RequestContext:
instance_uuid: str
workspace_uuid: str
execution_generation: int
principal_type: str
principal_uuid: str
permissions: frozenset[str]
auth_method: str
entitlement_revision: int | None
request_id: str
```
不同入口的 Workspace 来源:
| 入口 | Workspace 来源 |
| --- | --- |
| Browser Account token | `X-Workspace-Id` 只作候选;服务端校验 Membership |
| API Key | key 记录绑定的 Workspace,忽略 caller selector |
| Public Bot / Webhook | Bot 或 webhook route 的可信所有权 |
| Background job | durable payload 中的完整 scope,执行前重新验证 generation |
| Plugin Host API | 认证控制连接和 immutable action context |
| Box operation | 已验证 entitlement、admission grant 和 Runtime namespace |
| System operation | 显式、最小能力的 SystemContext,禁止隐式全局上下文 |
禁止从模块全局变量、进程默认 Workspace、请求 payload 或“第一个 Workspace”推断 scope。
### 8.2 Account token 与 Workspace discovery
- 新 JWT 使用稳定 Account UUID 作为 `sub`,并绑定 issuer、当前 `instance_uuid` audience 和 expiry。
- 账户级 Workspace discovery 是一个窄 bootstrap capability,只列出该 Account 的 active Membership,不能执行租户业务。
- multi-Workspace 模式下,tenant route 缺少 selector 必须拒绝;OSS singleton 模式可由 policy 选择唯一 Workspace。
- Account token 不直接证明任一 Workspace 权限;Membership 必须在服务端解析并验证状态与 revision。
### 8.3 API Key、WebSocket 与长任务
- API Key 只持久化 hashraw secret 仅返回一次;记录绑定 Workspace、固定 scopes、状态、expiry 和 creator。
- Dashboard WebSocket 在升级后认证,并在每条入站消息前重新验证 Account、Membership、权限、资源所有权和 generation。
- 长时间 LLM、MCP、Plugin 或 Box 调用在产生副作用或接受结果前再次校验 execution generation。
- 临时凭证交换绑定发起者、Workspace、instance 和 generation;其他 scope 查询返回与不存在相同的 404。
### 8.4 错误语义
| 场景 | 语义 |
| --- | --- |
| 未认证或 token 无效 | 401 |
| 同 Workspace 资源存在但权限不足 | 403 |
| 资源不存在或属于其他 Workspace | 404 |
| edition / entitlement / quota 禁止 | 稳定领域错误码,不伪装为 500 |
| execution generation 过期 | fail closed,并停止旧运行态 |
| 未处理异常 | 稳定 `internal_error` + request ID;细节只进入服务端日志 |
## 9. Core 数据模型
### 9.1 Account、Workspace 与 Membership
核心实体至少包含:
```text
Account
uuid
email_normalized
display_name
status
auth bindings
Workspace
uuid
name
status
source: local | cloud_projection
directory_revision
WorkspaceExecutionState
workspace_uuid
instance_uuid
execution_generation
status
write_fenced_at
revision
WorkspaceMembership
workspace_uuid
account_uuid
role
status
directory_revision
```
约束:
- Membership 对 `(workspace_uuid, account_uuid)` 唯一。
- Workspace 的 source 不允许通过可变本地配置从 local 升级成 cloud projection。
- Cloud projection 只有在 manifest、instance binding、目录 revision 和 execution state 均有效时才可路由。
- OSS bootstrap 只创建或修复 local singleton Workspace。
### 9.2 Invitation
OSS Invitation 存在 Core 本地数据库;SaaS Invitation 只存在于闭源目录。
```text
WorkspaceInvitation
uuid
workspace_uuid
email_normalized
role
token_hash
expires_at
accepted_at
revoked_at
created_by
```
数据库约束必须保证同一 Workspace 与邮箱只有一个有效邀请,并保证 token hash 全局唯一。
### 9.3 业务资源
所有租户资源显式包含 `workspace_uuid`,包括但不限于:
- Bot、Pipeline、Provider、Model、Knowledge Base 和 vector record。
- Plugin installation、MCP configuration、API Key 和 webhook binding。
- Query、Message、Session、Monitoring、Usage 和 AuditEvent。
- Upload、ObjectRef、Skill、Runtime desired state 和 temporary credential session。
唯一键、索引、缓存 key、object key、日志维度和幂等键都必须包含 Workspace scope。
服务层不得暴露可绕过 Workspace 条件的普通 `get(id)``list()``delete(id)`
### 9.4 防御性约束
- tenant table 的 `workspace_uuid` 非空并有外键。
- SaaS PostgreSQL 关键表启用并强制 RLS。
- 需要全局唯一的 opaque token 使用 hash 唯一索引,不依赖 Workspace 内唯一。
- owner 保底、Membership revision、invitation one-shot 等规则同时由 service 和数据库事务保护。
- 任何跨 Workspace 运维操作必须走显式受审计的 system capability,不得复用普通 repository。
## 10. PostgreSQL、pgvector 与存储
### 10.1 数据库边界
- OSS 继续默认 SQLite,并可显式选择自托管 PostgreSQL。
- SaaS 使用一个 PostgreSQL business database、一个 `public` shared schema 和共享连接池。
- 创建 Workspace 不创建 database、schema、role 或专属连接池。
- 每个 tenant transaction 使用 `SET LOCAL` 建立 scope,并由统一 TenantUnitOfWork 保证 context 与 SQL 使用同一事务和连接。
- 应用层 Workspace scope 是第一道边界,`ENABLE` + `FORCE ROW LEVEL SECURITY` 是第二道边界。
- runtime role 必须是非 owner、最小权限、无 superuser、无 `BYPASSRLS`、无 role membership 和跨 schema 权限。
- schema、extension、policy 和 ACL 只由独立 release migrator 创建与验证;Cloud runtime 不执行 DDL。
- PostgreSQL 仅承载业务数据和 pgvector,不成为 Plugin/Box 通用协调数据库、进程目录或新的控制面数据库。
首期 migrator 和 runtime URL 必须连接同一个 host、port、database,但使用不同 role。
生产部署还必须证明 runtime credential 无法连接 PostgreSQL 集群中的其他 database;专用 endpoint 或经验证的 HBA/proxy 隔离仍是激活门禁。
### 10.2 Transaction 与后台任务
- 一个 TenantUnitOfWork 只绑定一个 Workspace、一个 execution generation 和一个事务所有者任务。
- 子任务不能继承并提交、回滚或关闭父任务的 tenant session。
- 长时间 LLM 或网络等待不持有数据库连接;每次数据库 helper 打开短事务。
- detached task 只在父事务提交后启动,并自行建立新 scope;父事务回滚时取消待启动任务。
- generation-aware write fence 必须保持到 commit,并与 business outbox 原子提交;该能力完成前不得激活 SaaS 写流量。
### 10.3 pgvector
- SaaS 默认使用同一业务 PostgreSQL 中的 pgvector,不静默回退到 Chroma。
- 向量身份至少为 `(workspace_uuid, knowledge_base_uuid, vector_id)`
- 向量操作使用相同 tenant context 与 RLS 契约。
- embedding 维度显式存储和校验;不匹配时失败关闭,不截断、补齐或改用无界扫描。
- extension、表、constraint 和 ANN index 由 release migration 创建。
- OSS 默认仍可使用 SQLite + Chroma;选择 pgvector 时遵守相同 scope。
### 10.4 Object storage
- 大对象、plugin artifact、upload、knowledge 文件和 sandbox 文件不作为 PostgreSQL blob 存储。
- durable object key 和 metadata 都包含 Workspace scope;临时 staging 可包含 generation,但稳定业务引用不能因未来 generation 切换而永久失效。
- 现有 generation-scoped opaque key 在固定 generation 的 OSS 中安全,但 Cloud cutover 前必须实现稳定 final identity 或原子引用迁移。
- public image 与 private document 使用不同 capability;不能把通用 upload key 当作公开读取凭证。
## 11. Plugin Runtime
### 11.1 共享 supervisor、独立 worker
整个逻辑实例共享一个可信 Plugin Runtime 逻辑控制面;M0 由一个 supervisor replica 承担。新 Workspace 不创建专属 Runtime、连接、卷或进程。
每个运行中的 plugin installation 独占一个 nsjail worker process treeenabled-resident 是 desired semantics。worker 运行期间永久绑定:
```text
instance_uuid
workspace_uuid
execution_generation
installation_uuid
runtime_revision
artifact_digest
```
插件不能通过 payload、Host API 参数、环境变量或重连改变该绑定。Supervisor 不在自身解释器中加载第三方插件代码。
停用、删除、revision/generation 变化或 entitlement 撤销时,旧 worker 必须停止并失去 Host API 权限。
### 11.2 文件和进程边界
```text
data/plugin-runtime/
├── artifacts/sha256/<artifact_digest>/code/ # 已验证、只读共享
├── environments/sha256/<environment_digest>/ # 原子发布、只读共享
└── installations/<installation_uuid>/
├── home/ # 私有可写
├── tmp/ # 私有可写
└── data/ # 私有持久数据
```
- 同插件同版本只有在 package digest 完全相同且完整性已验证时才共享只读代码。
- dependency environment key 包含 artifact/requirements digest、Python ABI、Runtime version 和 installer schema。
- installation 进程、配置、secret、home、tmp、data 和日志永不合并。
- namespace、private `/proc`、mount、PID、IPC、UTS、cgroup 与 rlimit 阻止读取其他文件、枚举或 signal 其他进程。
- Cloud 不从 artifact 自动加载 `.env`;secret 只由可信控制面按 installation 注入。
- 插件 egress 必须阻止访问 Core loopback、Box Runtime、数据库和平台 metadata endpoint。
### 11.3 统一资源上限
资源限制只来自实例级 `data/config.yaml`,并支持现有环境变量覆写;plugin manifest 不能声明、放宽或覆盖。
```yaml
plugin:
worker:
max_cpus: 1.0
max_memory_mb: 512
max_pids: 128
max_open_files: 256
max_file_size_mb: 512
require_hard_limits: true
```
CPU、内存和 PID 使用 cgroup 硬限制,open files 和单文件大小使用 rlimit。
Cloud deployment profile 强制 nsjail;硬限制不可用时 readiness 失败,不能降级为普通子进程。
installation 总磁盘配额需要可原子拒绝写入的 quota provider,不能以目录扫描冒充硬限制。
### 11.4 Desired state 与恢复
- PostgreSQL 中的 installation desired state 与 durable binary storage 是权威状态。
- Runtime 本地进程表、nsjail 目录、artifact/venv cache 都可重建。
- Runtime 重连执行实例范围 full reconciliation,清理 stale worker 并恢复 enabled installation。
- dependency preparation 失败记录在对应 installation,不启动半就绪 worker,也不阻塞其他 installation。
- desired semantics 要求 enabled installation 常驻,不做 idle eviction;是否按负载回收以后再决定。
- 当前 Supervisor 已在意外退出时通过 completion callback 和有界指数 backoff 恢复 enabled worker。
Cloud 激活前仍需加入 jitter、全局重启并发上限和 Runtime 级 circuit breaker,并验证系统性故障不会形成跨租户重启风暴。
真实 Linux/nsjail/cgroup 与受控 egress 的 Cloud 部署验证尚未完成,是生产激活门禁。
## 12. Box Runtime 与 stdio MCP
### 12.1 共享 Box 控制面
整个逻辑实例共享一个可信 Box Runtime 逻辑控制面;M0 由一个 Runtime replica 承担。Core 与 Runtime 控制通道绑定稳定 instance identity
每个 operation 绑定 `workspace_uuid``execution_generation`、session revision 和短期 admission grant。
首期 entitlement 模型:
```json
{
"features": {
"managed_sandbox": true,
"external_sandbox": false
},
"limits": {
"managed_sandbox_sessions": 1
}
}
```
闭源订阅模块把套餐映射为该通用 capabilityCore 与 Runtime 不判断 `plan == pro`
预期 Pro 得到 `managed_sandbox_sessions = 1`,其他套餐为 `0`
### 12.2 Sandbox 模型
- 合资格 Workspace 首次使用时懒创建一个持久 `global` 逻辑 session。
- `global` 表示 Workspace 内默认逻辑 sandbox,不表示跨 Workspace 共享。
- session TTL 不自动回收;Runtime 重启后进程和临时目录失效,但 `/workspace` 持久数据保留。
- 每次普通命令在 Box Runtime 容器内启动一个 one-shot nsjail 子进程。
- 首期禁止 managed background process 和 network,避免 session 被当成常驻共享主机。
- Core 与 Runtime 通过认证 random-marker challenge 证明看到同一 durable volume,不能只比较路径字符串。
- 文件同步、attachment 和 skill mount 沿用现有 nsjail 机制,但所有 host path 解析必须由可信 Workspace context 派生并防止 symlink/path escape。
Cloud readiness 必须证明 cgroup、namespace、mount、Workspace/Skill/ephemeral byte quota 和 inode quota 均为硬限制。
当前普通 nsjail backend 不具备全部硬磁盘能力,因此 Cloud Box 应失败关闭,直到绿地部署提供并验证真实 quota provider
不能把软目录扫描写成“生产已就绪”。
### 12.3 外部 E2B
非 Pro 用户后续可在 WebUI 配置 Workspace 自有的远程 E2B sandbox。该功能尚未实现,首期不纳入。
未来 credential 必须属于 Workspace、加密存储且读取受 secret 权限保护,不消耗 Cloud managed sandbox 配额。
### 12.4 stdio MCP 独立开关
```yaml
mcp:
stdio:
enabled: true
```
- OSS 默认 `true` 保持兼容。
- Cloud v2 通过 `MCP__STDIO__ENABLED=false` 强制关闭。
- 该 gate 独立于 `box.enabled`、managed sandbox entitlement 和 session quota。
- gate 同时覆盖 create、update、test、bootstrap load 和最终 Runtime execution。
- 已有 stdio 配置在 gate 关闭时保留但不启动,并返回明确的 feature-disabled 错误。
- HTTP/SSE 等远程 MCP transport 不受影响。
## 13. HTTP API 与 WebUI
### 13.1 Core API
OSS 与 SaaS 执行面共用通用 Workspace API
```text
GET /api/v1/workspaces
GET /api/v1/workspaces/{workspace_uuid}
GET /api/v1/workspaces/{workspace_uuid}/members
POST /api/v1/workspaces/{workspace_uuid}/invitations
PATCH /api/v1/workspaces/{workspace_uuid}/members/{account_uuid}
DELETE /api/v1/workspaces/{workspace_uuid}/members/{account_uuid}
```
Cloud policy 下,目录 mutation 由闭源 Control Plane 负责;Core 对本地创建、邀请和成员修改返回稳定的
`control_plane_required`,只提供执行投影的安全读取。
所有 tenant resource route 必须经过统一 decorator/middleware
1. 认证 principal。
2. 解析可信 Workspace。
3. 校验 Workspace/ExecutionState。
4. 校验 Membership 或资源绑定。
5. 校验 permission 和 entitlement。
6. 创建 RequestContext 与 TenantUnitOfWork。
### 13.2 SaaS Control Plane API
SaaS 产品 API 包含:
```text
POST /cloud/workspaces
GET /cloud/workspaces
POST /cloud/workspaces/{workspace_uuid}/invitations
POST /cloud/invitations/{token}/accept
GET /cloud/workspaces/{workspace_uuid}/subscription
POST /cloud/workspaces/{workspace_uuid}/checkout
GET /cloud/workspaces/{workspace_uuid}/usage
```
这些 API 管理目录、产品和计费,不直接操作 Bot/Pipeline 等 Core 业务资源。
### 13.3 WebUI
OSS
- 首次注册进入唯一 Workspace。
- owner/admin 可邀请成员并管理固定角色。
- 不展示 Workspace 切换器和创建第二 Workspace 的入口。
SaaS
- 登录先获取 Account 级 Workspace 列表,再显式选择当前 Workspace。
- 当前 Workspace UUID 保存在受控客户端状态中;所有 tenant request 自动附带 selector。
- 切换 Account 或 Workspace 时清理缓存、WebSocket、上传、表单、错误和 optimistic state,不能显示前一租户数据。
- 页面 refresh、新 tab 和邀请跳转恢复同一个经过授权的 Workspace;失效 Membership 不回退到其他 Workspace。
- UI 权限变化必须响应式更新,但 API 仍是最终授权边界。
## 14. 故障、安全与降级
### 14.1 Fail-closed 场景
以下情况必须拒绝新的租户业务和副作用:
- Cloud manifest 缺失、签名失败、audience 错误或回滚。
- Account token、Membership、Workspace status 或 execution generation 无效。
- 目录投影未就绪或落后于有效 lease 要求。
- Entitlement 缺失、过期且不在明确 grace 范围内。
- Runtime 控制通道认证失败或实例绑定不一致。
- Plugin nsjail/cgroup hard limit 在 Cloud profile 下不可用。
- Box 的任一硬存储或 namespace capability 无法证明。
- PostgreSQL RLS、runtime role、schema、catalog 或 endpoint 隔离校验失败。
- stdio MCP 在 Cloud profile 下被尝试启用。
不能把上述错误静默降级为 OSS singleton、普通子进程、Chroma、软 quota 或 caller-supplied Workspace。
### 14.2 撤销语义
- Membership 删除或降权必须影响下一次 HTTP 请求,并使长连接在下一条消息前重新授权。
- Workspace 暂停禁止新交互、自动化工作负载和新副作用;恢复只允许当前 generation。
- entitlement 到期按 capability 明确停止新创建或新执行,不隐式删除已有数据。
- generation 变化使旧 worker、session、callback、cached runtime object 和 outbox publisher 失效。
- 控制面暂时不可达时,只能在有效签名快照和本地投影允许的范围内继续;过期后失败关闭。
### 14.3 安全清单
- 所有 identifier 使用不可猜 UUID,但不把随机性当成授权。
- 所有 token/secret 只存 hash 或加密值,raw secret 一次展示。
- 日志、trace、metric、cache 和 object key 都包含 Workspace 维度并过滤 secret。
- Provider、Bot、Plugin、MCP 配置的 read response 递归遮蔽 credential。
- Runtime control、debug、registration 和 attachment capability 分离,不能复用万能 secret。
- untrusted code 不访问 Core loopback、数据库、其他 Runtime、宿主文件系统或 metadata endpoint。
- bulk operation、后台扫描和 monitoring 聚合使用显式 tenant/system capability。
- 所有跨 Workspace 运维操作记录 principal、reason、scope、request ID 和结果。
## 15. 实现状态与 SaaS 激活门禁
### 15.1 已实现的隔离内核
当前分支已经实现或具备基础的部分包括:
- OSS singleton Workspace、多 Account、Invitation 和固定 RBAC。
- trusted RequestContext、Workspace-scoped repository 和资源所有权检查。
- tenant-aware Plugin SDK protocol 与 Runtime installation binding。
- shared Plugin Runtime / Box Runtime 控制协议和 execution generation fence。
- stdio MCP 独立 gate。
- PostgreSQL shared schema、transaction-local scope、FORCE RLS 与 pgvector adapter。
- Cloud bootstrap 默认不可由普通配置激活,并对缺失安全能力失败关闭。
这些是代码能力边界,不等于完成闭源 SaaS 产品或生产部署验收。
### 15.2 尚未完成的激活门禁
以下事项完成并取得真实环境证据前,不得宣称 Cloud v2 production-ready
1. 闭源 Control Plane 的全局目录、注册、邀请、订阅、计费、entitlement 签发和签名 manifest bootstrap;横向扩展前 OAuth exchange 与目录投影还必须使用原子共享存储。
2. 普通业务写入贯穿 commit 的 generation-aware fence,以及与外部副作用同事务的 business outbox。
3. generation cutover 后稳定的 durable object identity 或原子对象引用迁移。
4. 所有 tenant-configurable outbound URL 的 SSRF 防护与 tenant-safe egressPlugin Runtime 还需在真实 Linux/nsjail/cgroup v2 环境验证 namespace、资源限制和文件隔离。
5. Plugin Runtime 已实现意外退出 worker 的 completion callback、有界 backoff 和自动恢复;Cloud 激活前增加全局重启风暴抑制并完成故障注入验证。
6. Plugin installation data 的 production hard disk quota provider,能够在写入边界原子拒绝超额,不能以目录扫描代替。
7. Box Runtime 的 production hard quota provider,包括 Workspace、Skill、root/tmp/home 的 byte 与 inode quota;真实部署还必须在启动和重连时通过共享卷 marker challenge。
8. PostgreSQL runtime credential 的专用 endpoint 或 HBA/proxy 跨 database 隔离证明、生产 migration/rollback 流程,以及 legacy pgvector migration 失败后精确恢复 RLS/FORCE 并可安全重试的集成证据。
9. 闭源目录事件、lease、snapshot、entitlement 和 usage/outbox 的重放、断流与灾难恢复验证。
10. 真实浏览器多 Account/RBAC/邀请/刷新场景已完成;仍需生产 Runtime 重启、worker crash、断流、异常回滚和闭源 Control Plane 的 fault-injection 验收。
### 15.3 有意暂缓的产品决策
- Workspace 创建后的休眠、释放、删除和保留策略。
- Workspace export 与单 Workspace restore。
- 非 Pro Workspace 的 BYOK E2B WebUI。
- 多副本 owner lease 的 store、TTL、fencing token 和转移顺序。
- PostgreSQL shard resolver、在线迁移和 dedicated shard 产品规则。
- artifact/cache 的签名来源、撤销、GC 和磁盘配额机制。
- custom roles、SSO、SCIM 和企业合规能力。
暂缓项不得被实现代码用隐式默认值提前固化。
## 16. 实施顺序
### Phase 0:契约和基线
- 固定术语、RequestContext、角色矩阵、edition policy 和错误语义。
- 建立升级备份、回滚和跨租户负向测试基线。
### Phase 1OSS tenancy kernel
- Account、Workspace、Membership、Invitation。
- singleton bootstrap、多用户邀请、RBAC 和前端权限。
### Phase 2:数据与入口隔离
- 为所有资源补充 Workspace scope。
- HTTP、API Key、Bot、Webhook、WebSocket、后台任务和 storage 统一上下文。
- SQLite migration recovery 与 PostgreSQL RLS 集成测试。
### Phase 3Runtime 与 SDK 隔离
- Plugin installation binding、nsjail、资源上限和 artifact replay。
- Box admission、session namespace、skill/attachment 文件边界。
- MCP gate、RAG/vector 与 long-running generation revalidation。
### Phase 4:闭源 SaaS 控制面
- signed manifest bootstrap。
- 全局目录、注册、邀请、Subscription、Entitlement 和 Usage ledger。
- projection、lease、outbox、reconciliation 和运维后台。
### Phase 5:生产部署激活
- 真实 Linux Plugin/Box hard isolation。
- PostgreSQL credential、migration、backup 和 rollback 验证。
- 完整浏览器/API/Runtime E2E 和故障注入。
- 所有激活门禁通过后才开启多 Workspace Cloud policy。
### Phase 6:同逻辑实例内部扩展
- 有容量证据后增加副本、owner lease 和 fencing。
- 有地域、合规或规模证据后增加 shared/dedicated shard。
- 保持外部身份、API 和 Workspace URL 不变。
## 17. 测试与验收
### 17.1 数据隔离
- 两个 Workspace 使用相同 resource UUID、name、vector ID 和 cache key,不发生冲突或越权。
- 故意遗漏应用层 Workspace filter 时,PostgreSQL RLS 仍阻止跨租户读写。
- 连接池复用、异常回滚、子任务、后台任务和 transaction pooling 不残留 tenant context。
- 跨 Workspace 猜测返回 404;同租户缺权限返回 403。
### 17.2 产品行为
- OSS 首个 Account 创建唯一 Workspace;第二个 Account 只能通过邀请加入;创建第二 Workspace 返回 edition error。
- 邀请覆盖有效、已使用、撤销、过期、邮箱不匹配和并发接受。
- owner/admin/developer/operator/viewer 的 API 和 WebUI 权限一致。
- SaaS 普通注册和邀请注册都创建个人 Workspace,但不创建专属部署或 Runtime。
### 17.3 Runtime
- 两个 Workspace 安装同一已验证 artifact 时只共享只读 code/env,进程、secret、home/tmp/data、日志和 Host API 完全隔离。
- cgroup、rlimit、namespace、egress 和 generation fence 在真实 Linux 环境生效。
- Runtime restart/cache loss 通过 durable desired state 与 binary storage 恢复。
- 两个 Workspace 的 Box session、files、process、skill、attachment 和 quota 完全隔离。
- stdio MCP gate 对 UI、API、bootstrap 和最终 execution 同时生效。
### 17.4 Control Plane 与故障
- DirectoryEvent 重复、乱序、缺口、snapshot + replay 和过期 lease 均安全。
- Entitlement 旧 revision、签名错误、过期和撤销均失败关闭。
- UsageEvent 重放不重复计费;业务事务回滚不发送副作用。
- Runtime、Core 或 Control Plane 重启不创建重复 Workspace、worker 或 sandbox。
- manifest、数据库安全校验或 hard quota 缺失时实例保持不可激活,而不是静默降级。
### 17.5 浏览器端到端
真实浏览器至少覆盖:
1. clean database 首位 owner 注册与 singleton Workspace bootstrap。
2. owner 创建邀请,第二个用户注册/登录并接受。
3. 角色在 viewer/operator/developer/admin 间变化时,导航、控制项和 API 结果同步变化。
4. Account/Workspace 切换清空前一 scope 状态,refresh 和新 tab 恢复正确 Workspace。
5. 第二 Workspace edition limit,以及 invitation used/revoked/expired/email mismatch 的可见错误。
6. 直接 API 越权、伪造 selector 和跨租户 UUID 猜测不能绕过 UI。
## 18. 最终结论
Cloud v2 的产品模型只有一个逻辑 LangBot 实例和实例内多个 Workspace。
当前选择单副本 MVP 是为了减少组件和新增租户成本,不是把单进程假设写进业务身份或协议。
未来需要容量或高可用时,在同一逻辑实例内部增加 Core/Runtime 副本和 PostgreSQL shard
Workspace 的 UUID、权限、数据边界和外部 API 均保持不变。
开源 Core 必须完整实现安全的 Workspace 隔离和 OSS 单 Workspace 多用户;闭源 Control Plane
管理 SaaS 的全局目录、订阅、权益、计费和生命周期。共享可信控制面、连接池、只读 artifact 和数据库组件,
同时让每个不可信插件进程、sandbox、secret、可写文件和 tenant transaction 保持独占边界,
才能在不增加每租户部署的前提下最大化降低新增用户成本。
在闭源控制面、事务 fence/outbox、真实 Runtime hard isolation、Box hard quota 和 PostgreSQL 生产隔离等门禁完成之前,
本架构仍处于隔离内核阶段,不应被描述为可上线的 SaaS 多租户部署。
+256
View File
@@ -0,0 +1,256 @@
# HTTP Bot Adapter — Integration Guide
Integrate **any backend system** with a LangBot pipeline over plain HTTP. Push
messages in via a signed webhook; receive replies on a callback URL. No
long-lived connection, full support for message **aggregation** (many inbound
messages merged into one turn) and **multi-part replies** (one turn → many
outbound messages).
This is the right adapter for **server-to-server** integrations — ticketing
systems, CRMs, internal tools, custom web backends. (For an in-browser,
real-time chat widget, use the embeddable Web Page Bot instead.)
> **5-minute goal:** stand up a callback receiver, send a message, and watch a
> multi-part reply arrive — using the reference client in
> [`examples/http-bot/`](../../examples/http-bot/).
---
## 1. Mental model
```
Your backend ──(1) POST signed message──► LangBot /bots/<bot_uuid>
(pipeline runs: aggregate → think → reply)
Your callback ◄─(2) POST signed reply(s)── LangBot one POST per reply part
```
- **(1) Inbound** is *fire-and-collect*: LangBot answers `202 Accepted`
immediately and does **not** return the pipeline result on that response.
- **(2) Outbound** replies arrive later as separate signed POSTs to your
`callback_url`. A single turn may produce **several** callbacks (e.g. a tool
call narration followed by the final answer).
- Everything is keyed by a **`session_id` you choose** (e.g. a ticket number).
Each `session_id` maps to one isolated LangBot conversation.
---
## 2. Create the bot
1. In the LangBot dashboard, add a bot and choose the **HTTP Bot** platform.
2. Fill in the config:
| Field | Required | Notes |
|---|---|---|
| **Inbound Signing Secret** | yes | Your backend signs inbound requests with this. |
| **Outbound Callback URL** | yes | Where LangBot POSTs replies. **Config-only** — cannot be overridden per message (SSRF protection). |
| **Outbound Signing Secret** | no | LangBot signs callbacks with this; defaults to the inbound secret. |
| **Default Session Type** | no | `person` (default) or `group`. |
| **Require Inbound Signature** | no | Keep `true` in production. |
| **Callback Timeout / Max Retries** | no | Defaults: 15s, 3 retries. |
3. Bind the bot to a **pipeline** and **enable** it.
4. Copy the **Inbound Webhook URL** shown in the config — it looks like
`https://your-langbot/bots/<bot_uuid>`.
---
## 3. The signature scheme
Both directions use the same dependency-free HMAC-SHA256 scheme:
```
signing_string = "{timestamp}." + raw_body_bytes
signature = "sha256=" + hex(HMAC_SHA256(secret, signing_string))
```
Sent as headers:
| Header | Meaning |
|---|---|
| `X-LB-Timestamp` | Unix seconds. Rejected if more than **±300s** from server time. |
| `X-LB-Signature` | `sha256=<hex>` over `"{timestamp}." + body`. |
| `X-LB-Idempotency-Key` | *(optional, inbound)* dedup key; retries with the same key return `409`. |
Verify outbound callbacks the same way, using the **outbound** secret (or the
inbound secret if you left it blank).
A six-line reference implementation is in `examples/http-bot/client.py`
(`sign()` / `verify()`); a Node/TS version is in `client.ts`.
---
## 4. Send your first message (curl)
```bash
BOT="https://your-langbot/bots/<bot_uuid>"
SECRET="your-inbound-secret"
BODY='{"session_id":"ticket-10293","message":[{"type":"Plain","text":"Export keeps failing on the dashboard."}]}'
TS=$(date +%s)
SIG="sha256=$(printf '%s.%s' "$TS" "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -r | cut -d' ' -f1)"
curl -sS -X POST "$BOT" \
-H "Content-Type: application/json" \
-H "X-LB-Timestamp: $TS" \
-H "X-LB-Signature: $SIG" \
-d "$BODY"
# -> 202 {"code":0,"msg":"accepted","data":{"session_id":"ticket-10293","accepted_message_id":"in_...","aggregating":true}}
```
The reply(s) will be POSTed to your configured callback URL shortly after.
---
## 5. Inbound request format
`POST /bots/{bot_uuid}`
```jsonc
{
"session_id": "ticket-10293", // REQUIRED. Your stable id. Maps 1:1 to a LangBot session.
"session_type": "person", // optional: "person" | "group"; default from config
"sender": { // optional metadata, surfaced to the pipeline/plugins
"id": "user-5567",
"name": "Alice"
},
"message": [ // REQUIRED. A LangBot MessageChain (array of segments).
{ "type": "Plain", "text": "Export keeps failing on the dashboard." },
{ "type": "Image", "url": "https://example.com/screenshot.png" }
]
}
```
**Message segments.** Text uses `{"type":"Plain","text":"..."}`. Images use
`{"type":"Image","url":"..."}` (or `base64`). Other supported types: `Voice`,
`File`, `At`, `Quote`.
> Note: the callback URL is **not** accepted in the body — it is taken only from
> bot config. This is deliberate (prevents an attacker who obtains the inbound
> secret from redirecting replies to an arbitrary host).
### Aggregation (N → 1)
If your pipeline has **message aggregation** enabled, send several messages with
the **same `session_id`** within the aggregation window and they are merged into
**one** pipeline turn. No special flag — just reuse the `session_id`.
---
## 6. Outbound callback format
LangBot POSTs each reply part to your `callback_url`:
```jsonc
{
"session_id": "ticket-10293", // echoes the inbound session
"reply_to": "in_01H...", // the accepted_message_id this answers
"sequence": 1, // 1-based ordinal within this turn
"is_final": false, // true on the last part of the turn
"stream": false, // true for streamed chunks
"message": [ { "type": "Plain", "text": "Looking into it…" } ],
"timestamp": "2026-06-22T09:00:01Z"
}
```
Your endpoint should return `2xx` quickly. Non-2xx / timeout → LangBot retries
with exponential backoff (up to `callback_max_retries`).
### Multi-part replies (1 → M)
One turn may emit multiple callbacks, delivered **in `sequence` order** for a
given session:
```
seq=1 is_final=false "Checking your export logs…"
seq=2 is_final=false "Found 2 failed exports."
seq=3 is_final=true "Fixed — please try again."
```
Stitch by `session_id` + `sequence`; the turn is complete when
`is_final: true` arrives.
---
## 7. Reset a session
Start a fresh conversation for a `session_id` (drops history):
```
POST /bots/{bot_uuid}/reset
{ "session_id": "ticket-10293", "session_type": "person" }
→ 200 { "code":0, "msg":"reset", "data": { "session_id":"ticket-10293", "removed": true } }
```
Signed exactly like an inbound message.
---
## 8. Synchronous convenience mode
If you don't need streaming/multi-part and just want one reply back on the same
HTTP call, POST to `/sync`. LangBot waits for the turn to finish and returns all
parts **collapsed** into one array:
```
POST /bots/{bot_uuid}/sync
{ "session_id": "ticket-10293", "message": [ { "type":"Plain", "text":"hi" } ] }
→ 200 { "code":0, "msg":"ok",
"data": { "session_id":"ticket-10293", "reply_to":"in_...",
"message": [ {"type":"Plain","text":"..."}, ... ] } }
```
This is **lossy** (you lose `sequence` / streaming boundaries) and blocks up to
`callback_timeout × 4` seconds. Prefer the callback model for anything
real-time or multi-part. Only one in-flight `/sync` per `session_id`.
---
## 9. Error envelope
```jsonc
{ "code": 40101, "msg": "invalid signature: signature_mismatch", "data": null }
```
| HTTP | code | meaning |
|---|---|---|
| 202 | 0 | accepted |
| 400 | 40001 | malformed body / missing `session_id` or `message` |
| 401 | 40101 | bad/expired signature |
| 409 | 40901 | duplicate idempotency key |
| 413 | 41301 | message too large (>1 MiB) |
| 500 | 50001 | internal error |
---
## 10. Try it end-to-end in 5 minutes
```bash
cd examples/http-bot
pip install flask requests
# Terminal 1 — your callback receiver (point the bot's callback_url here, e.g. via a tunnel):
python client.py serve --port 8900 --secret SHARED_SECRET
# Terminal 2 — push a message:
python client.py push \
--url https://your-langbot/bots/<bot_uuid> \
--secret SHARED_SECRET \
--session ticket-1 \
--text "hello"
```
Watch Terminal 1 print each reply part (`[part ]` / `[FINAL]`) with its
sequence number — that's 1→M working, signatures verified.
A machine-readable contract is in
[`docs/http-bot-openapi.json`](../http-bot-openapi.json).
---
## 11. Security checklist
- Keep **Require Inbound Signature** on in production.
- Use **HTTPS** callback URLs; the URL is config-only (no per-message override).
- Treat the secrets like passwords; rotate via the dashboard.
- The inbound route is unauthenticated at the framework level **by design**
security comes entirely from the HMAC signature, so never disable it on a
public deployment.
+196
View File
@@ -0,0 +1,196 @@
# MCP Resources PR #2215 Review
> 更新日期: 2026-06-29
> 分支: `mcp_resources`
> PR: langbot-app/LangBot#2215
> 主题: MCP Resources 在 LangBot 中的产品价值、AgentRunner 集成方式与后续架构方向
## 结论
PR #2215 对 LangBot 有明确价值:它补齐了 MCP 协议中 Resources 这一重要能力,让 MCP server 不再只暴露 tools,也可以暴露文档、代码片段、配置、日志、图片等上下文资源。管理端可以发现和预览资源,Agent 也可以通过当前实现按需列出和读取资源。
但当前 AgentRunner 层的接入方式更接近一个可用的第一阶段方案,而不是最终架构。现在 MCP Resources 被包装成两个 synthetic tools
- `langbot_mcp_list_resources`
- `langbot_mcp_read_resource`
这让模型可以通过 function calling 主动探索资源,落地成本低,也复用了已有 `ToolManager` / `LocalAgentRunner` 的工具调用链路。不过从 MCP 规范和主流实现来看,Resources 更适合作为一种一等上下文来源,而不是长期隐藏在工具列表里。
建议保留当前 synthetic tools 作为探索能力,同时把后续主线设计调整为:MCP Resources 是 pipeline / conversation / message 级别可选择、可固定、可审计的上下文输入。
## 当前实现判断
当前 AgentRunner 集成路径如下:
```text
Pipeline 绑定 MCP server
-> query.variables['_pipeline_bound_mcp_servers']
-> Preproc 为 local-agent 加载工具
-> ToolManager.get_all_tools()
-> MCPLoader 注入 synthetic resource tools
-> LocalAgentRunner 将工具 schema 传给模型
-> 模型发起 list/read tool call
-> ToolManager.execute_func_call()
-> MCPLoader 调 MCP session.list_resources/read_resource
-> tool result 回灌给模型
```
这个路径的优点是:
- 复用现有工具调用机制,改动范围小。
- Agent 可以按需探索资源,不需要每轮预先读取所有资源。
- 可以沿用 pipeline 绑定的 MCP server 范围,避免越权读取未绑定 server。
- 对已有 MCP tools 行为影响较小。
主要问题是:
- Resources 在语义上被降级成 tools,和 MCP 规范里的 resource primitive 不完全一致。
- 模型必须先理解并主动调用 `list/read`,资源不会自然成为上下文。
- pipeline 不能配置“默认携带某些资源”或“本轮附加某些资源”。
- UI 资源 tab 目前是管理端预览能力,和 Agent 上下文选择没有打通。
- 对 blob、图片、大文件、结构化资源的处理还比较粗糙。
- 缺少 resource templates、订阅更新、缓存、chunk、token budget、trace 与审计策略。
## 主流项目做法
### MCP 官方规范
MCP Resources 是 server 暴露上下文数据的协议能力。规范没有要求 resources 必须以 tool call 形式给模型使用,而是把如何选择、过滤、读取和纳入上下文交给 Host application。
这意味着比较正统的集成方式是:LangBot 作为 Host,在 pipeline、会话或消息层决定哪些 resources 进入模型上下文。
参考: https://modelcontextprotocol.io/specification/2025-06-18/server/resources
### VS Code Copilot
VS Code 把 MCP Resources 做成 chat context 的一部分。用户可以通过 `Add Context > MCP Resources` 或命令浏览 MCP resources,并把选中的资源附加到一次 chat request。
这是目前最值得 LangBot 参考的产品形态:资源不是模型工具,而是用户和 Host 可控的上下文附件。
参考: https://code.visualstudio.com/docs/agent-customization/mcp-servers
### Anthropic SDK
Anthropic 的 client-side MCP helpers 提供资源读取和转换能力,例如把 MCP resource 转为 Claude message content 或 file。也就是说,应用先读取 resource,再显式放进模型消息。
这同样是 application-owned context injection,而不是把 resource 伪装成模型工具。
参考: https://platform.claude.com/docs/en/agents-and-tools/mcp-connector
### LangChain MCP Adapters
LangChain 把 MCP Resources 更像 data loader / document input 来处理,可以把资源加载成 `Blob`,再进入 LangChain 的文档、检索或上下文处理链路。
这说明 Resources 很适合作为知识源、文档源或上下文源,而不只是即时工具调用。
参考: https://docs.langchain.com/oss/python/langchain/mcp
### OpenAI Agents SDK
OpenAI Agents SDK 主路径仍偏向 MCP tools,但底层 MCP server API 已经有 `list_resources``list_resource_templates``read_resource` 等能力。当前形态说明 resources 是 client 能力,但并未默认变成 agent-visible tools。
参考: https://openai.github.io/openai-agents-python/mcp/
### Cline
Cline 会拉取 MCP tools、resources、resourceTemplates、prompts,并通过类似 `access_mcp_resource` 的内置访问方式让模型读取资源。这个方向和 LangBot 当前 synthetic tools 比较接近。
这种模式适合让 Agent 自主探索,但更像 Host 自定义的模型访问协议,不应成为唯一集成路径。
参考: https://github.com/cline/cline/blob/main/src/services/mcp/McpHub.ts
## 建议架构方向
### 1. 保留探索型工具
保留当前两个 synthetic tools
- `langbot_mcp_list_resources`
- `langbot_mcp_read_resource`
它们适合处理“用户没有显式选择资源,但 Agent 判断需要探索 MCP server 上下文”的场景。后续可以优化工具描述、返回格式、资源大小限制和错误信息。
### 2. 增加一等 Resource Context
新增一个 Host 层资源上下文概念,例如:
```text
PipelineResourceBinding
ConversationResourceAttachment
MessageResourceAttachment
```
Preproc 或独立的 `ResourceContextProvider` 在模型调用前读取这些资源,按 MIME 类型、大小、token budget 转为模型可消费的上下文。
### 3. 打通 UI 与 Agent 上下文
当前 MCP 详情页的 Resources tab 可以继续作为资源发现和预览入口。建议增加操作:
- 添加到本轮上下文
- 固定到当前 pipeline
- 固定到当前 bot / conversation
- 查看资源读取历史和错误
这样 UI 资源管理能力才能真正影响 Agent 行为。
### 4. 支持 resource templates
MCP resource templates 允许 server 暴露参数化资源,例如:
```text
repo://{owner}/{repo}/file/{path}
log://{service}/{date}
```
LangBot 后续应支持模板发现、参数填写、实例化和绑定。否则只能使用静态 resources,覆盖面会受限。
### 5. 增加资源处理策略
建议补齐:
- 文本资源 token budget 与截断策略。
- 大文件 chunk 与摘要策略。
- 图片/blob 的模型能力判断与 fallback。
- MIME 类型白名单与安全限制。
- 缓存与过期策略。
- `resources/listChanged` 或订阅更新。
- resource read trace,便于审计 Agent 读取了什么上下文。
## 推荐落地顺序
### Phase 1: 完成当前 PR 可用性
- 保留 synthetic tools。
- 明确文档说明当前 Agent 集成是 tool-mediated。
- 完善资源工具描述,降低模型误用概率。
- 给 read/list 增加大小限制和更清晰的 MIME 处理。
- 前端 Resources tab 与 Tools tab 分离,保持管理端清晰。
### Phase 2: 做 Host-owned context attachments
- 在 pipeline 或 conversation 层新增 resource attachment 配置。
- Preproc 读取已绑定 resources,注入模型上下文。
- UI 支持“添加到上下文 / 固定到 pipeline”。
- 记录每轮实际注入的 resource URI 和 token 消耗。
### Phase 3: 做完整 MCP Resources 能力
- 支持 resource templates。
- 支持资源订阅更新。
- 支持 chunk、summary、RAG 化接入。
- 为 DifyAgentRunner、LocalAgentRunner 等不同 runner 定义统一资源上下文接口。
## 最终建议
PR #2215 可以作为 MCP Resources 的第一阶段实现继续推进。它让 LangBot 快速拥有“资源发现、预览、按需读取”的闭环,也给 Agent 探索资源提供了可运行路径。
但在正式设计上,不建议把 “Resources == Tools” 固化为长期抽象。LangBot 更应该把 MCP Resources 定位为上下文来源,与 tools、prompts、knowledge base 并列:
```text
Tools -> Agent 可以执行的动作
Resources -> Host/用户/Agent 可以选择的上下文数据
Prompts -> 可复用的任务模板
Knowledge -> 可检索、可索引的长期知识
```
这样既尊重 MCP 协议语义,也能让 LangBot 在 Agent 工作流、企业知识接入和多 MCP server 管理上走得更稳。
+75
View File
@@ -0,0 +1,75 @@
# HTTP Bot Adapter — Reference Clients
> English | [中文](./README.zh.md)
Minimal, dependency-light clients for the LangBot **HTTP Bot** platform adapter.
They show the whole loop: signing a request, pushing a message, and receiving
multi-part replies on a callback endpoint.
Full guide: [docs.langbot.app — HTTP Bot](https://docs.langbot.app/en/usage/platforms/http-bot).
Machine-readable contract: [`docs/http-bot-openapi.json`](../../docs/http-bot-openapi.json).
## Files
| File | What it is |
|---|---|
| `playground.py` | **Interactive browser debug console** — a single-file web app you open in a browser to chat with a running `http_bot` bot and watch signing / 202 / callbacks live. Zero extra deps. |
| `client.py` | Python client + Flask callback receiver (`pip install flask requests`). |
| `client.ts` | TypeScript/Node 18+ client + callback receiver, **zero deps** (`npx tsx client.ts`). |
All three implement the identical HMAC-SHA256 scheme
(`sha256=hex(HMAC(secret, "{timestamp}." + body))`) — verified byte-for-byte
against the adapter.
## Interactive playground (recommended first run)
A self-contained web console: type a message in your browser, it is signed and
POSTed to a **running** `http_bot` bot, and the bot's replies stream back into
the page — with a debug panel showing the signature, the `202` ack, and each
callback's `sequence` / signature-verification.
```bash
# From the LangBot repo root, with the backend already running:
PUBLIC_IP=<your-host-ip> ./.venv/bin/python examples/http-bot/playground.py
# then open http://<your-host-ip>:8920/
```
On startup it reads the LangBot API key + the `http_bot` bot from
`data/langbot.db`, and configures that bot (inbound/outbound secret +
`callback_url`) to point back at itself via the LangBot API — the bot reloads
live, no restart needed. Requirements: an enabled `http_bot` bot bound to a
working pipeline, and port `8920` reachable from your browser.
Env knobs: `PUBLIC_IP` (default `127.0.0.1`), `PLAYGROUND_PORT` (default `8920`).
## Headless clients
```bash
# Python — Terminal 1: callback receiver (your callback_url target)
python client.py serve --port 8900 --secret SHARED_SECRET
# Python — Terminal 2: push a message
python client.py push --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1 --text "hello"
# blocking sync mode
python client.py sync --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1 --text "hello"
# reset a session
python client.py reset --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1
```
```bash
# TypeScript (Node 18+)
npx tsx client.ts serve 8900 SHARED_SECRET
npx tsx client.ts push https://your-langbot/bots/<BOT_UUID> SHARED_SECRET ticket-1 "hello"
```
When the bot replies, the receiver prints each part with its `sequence` and an
`[FINAL]` marker on the last one — that's the 1→M multi-reply model in action.
> The bot's `callback_url` must be reachable from LangBot. For local testing,
> expose your receiver with a tunnel (cloudflared / ngrok) and set that URL in
> the bot config.
+71
View File
@@ -0,0 +1,71 @@
# HTTP Bot 适配器 —— 参考客户端
> [English](./README.md) | 中文
面向 LangBot **HTTP Bot** 平台适配器的极简、低依赖客户端示例。
它们完整展示了整条链路:对请求签名、推送一条消息、在回调端点接收
1→M 的多段回复。
完整指南:[docs.langbot.app —— HTTP Bot](https://docs.langbot.app/zh/usage/platforms/http-bot)。
机器可读的接口契约:[`docs/http-bot-openapi.json`](../../docs/http-bot-openapi.json)。
## 文件清单
| 文件 | 是什么 |
|---|---|
| `playground.py` | **浏览器交互式调试台** —— 单文件 Web 应用,在浏览器里和一个运行中的 `http_bot` bot 对话,实时观察签名 / 202 / 回调。零额外依赖。 |
| `client.py` | Python 客户端 + Flask 回调接收端(`pip install flask requests`)。 |
| `client.ts` | TypeScript/Node 18+ 客户端 + 回调接收端,**零依赖**(`npx tsx client.ts`)。 |
三者实现完全一致的 HMAC-SHA256 签名方案
(`sha256=hex(HMAC(secret, "{timestamp}." + body))`)—— 已与适配器逐字节比对验证。
## 交互式 playground(推荐先跑这个)
一个自包含的 Web 控制台:在浏览器里输入消息,它会被签名并 POST 给一个
**运行中**的 `http_bot` bot,bot 的回复会流式回到页面上 —— 调试面板会显示
签名、`202` 确认,以及每条回调的 `sequence` / 签名验证结果。
```bash
# 在 LangBot 仓库根目录、后端已启动的前提下:
PUBLIC_IP=<你的主机IP> ./.venv/bin/python examples/http-bot/playground.py
# 然后打开 http://<你的主机IP>:8920/
```
启动时它会从 `data/langbot.db` 读取 LangBot API key 和 `http_bot` bot,
并通过 LangBot API 把该 bot 配好(入站/出站密钥 + `callback_url`)指回自己 ——
bot 会热加载,无需重启。前提:有一个已启用、绑定了可用 pipeline 的
`http_bot` bot,且端口 `8920` 能从你的浏览器访问到。
可调环境变量:`PUBLIC_IP`(默认 `127.0.0.1`)、`PLAYGROUND_PORT`(默认 `8920`)。
## 无头客户端
```bash
# Python —— 终端 1:回调接收端(你的 callback_url 指向它)
python client.py serve --port 8900 --secret SHARED_SECRET
# Python —— 终端 2:推送一条消息
python client.py push --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1 --text "hello"
# 阻塞式同步模式
python client.py sync --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1 --text "hello"
# 重置一个会话
python client.py reset --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-1
```
```bash
# TypeScript(Node 18+)
npx tsx client.ts serve 8900 SHARED_SECRET
npx tsx client.ts push https://your-langbot/bots/<BOT_UUID> SHARED_SECRET ticket-1 "hello"
```
当 bot 回复时,接收端会逐条打印,带上各自的 `sequence`,并在最后一条标记
`[FINAL]` —— 这就是 1→M 多段回复模型的实际效果。
> bot 的 `callback_url` 必须能从 LangBot 访问到。本地测试时,可用隧道
> (cloudflared / ngrok)把你的接收端暴露出去,并把那个 URL 填进 bot 配置。
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
"""LangBot HTTP Bot adapter — reference client (Python).
Two things in one file:
1. ``push()`` / ``push_sync()`` send a message into a LangBot ``http_bot`` bot.
2. A tiny Flask callback receiver that verifies signatures and prints replies,
so you can watch N->1 aggregation and 1->M multi-reply working live.
Usage
-----
pip install flask requests
# Terminal 1 — start the callback receiver (this is your callback_url):
python client.py serve --port 8900 --secret SHARED_SECRET
# Terminal 2 — push a message (async; reply lands on the receiver):
python client.py push \
--url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET \
--session ticket-10293 \
--text "Export keeps failing on the dashboard."
# Or push and block for the collapsed reply (sync convenience mode):
python client.py sync --url https://your-langbot/bots/<BOT_UUID> \
--secret SHARED_SECRET --session ticket-10293 --text "hi"
The signing scheme is HMAC-SHA256 over ``"{timestamp}." + raw_body``; see
``sign()`` below it is intentionally tiny and easy to port.
"""
from __future__ import annotations
import argparse
import hashlib
import hmac
import json
import sys
import time
import uuid
HEADER_TIMESTAMP = 'X-LB-Timestamp'
HEADER_SIGNATURE = 'X-LB-Signature'
HEADER_IDEMPOTENCY = 'X-LB-Idempotency-Key'
REPLAY_WINDOW = 300
def sign(secret: str, body: bytes, timestamp: int | None = None) -> tuple[str, str]:
"""Return (timestamp, signature) for *body*."""
ts = str(timestamp if timestamp is not None else int(time.time()))
mac = hmac.new(secret.encode(), f'{ts}.'.encode() + body, hashlib.sha256)
return ts, 'sha256=' + mac.hexdigest()
def verify(secret: str, body: bytes, timestamp: str | None, signature: str | None) -> bool:
"""Verify an inbound signature (used by the callback receiver)."""
if not timestamp or not signature:
return False
try:
if abs(int(time.time()) - int(float(timestamp))) > REPLAY_WINDOW:
return False
except ValueError:
return False
_, expected = sign(secret, body, int(float(timestamp)))
return hmac.compare_digest(expected, signature)
def _post(url: str, secret: str, payload: dict, idempotency: bool = True):
import requests
body = json.dumps(payload, ensure_ascii=False).encode()
ts, sig = sign(secret, body)
headers = {
'Content-Type': 'application/json',
HEADER_TIMESTAMP: ts,
HEADER_SIGNATURE: sig,
}
if idempotency:
headers[HEADER_IDEMPOTENCY] = uuid.uuid4().hex
resp = requests.post(url, data=body, headers=headers, timeout=30)
print(f'-> {resp.status_code} {resp.text}')
return resp
def push(url: str, secret: str, session: str, text: str, session_type: str = 'person'):
"""Fire-and-collect: returns 202 immediately; reply arrives on your callback."""
payload = {
'session_id': session,
'session_type': session_type,
'message': [{'type': 'Plain', 'text': text}],
}
return _post(url.rstrip('/'), secret, payload)
def push_sync(url: str, secret: str, session: str, text: str, session_type: str = 'person'):
"""Blocking convenience: POST to /sync and get the collapsed reply back."""
payload = {
'session_id': session,
'session_type': session_type,
'message': [{'type': 'Plain', 'text': text}],
}
resp = _post(url.rstrip('/') + '/sync', secret, payload, idempotency=False)
return resp
def reset(url: str, secret: str, session: str, session_type: str = 'person'):
"""Reset a session's conversation (next message starts fresh)."""
payload = {'session_id': session, 'session_type': session_type}
return _post(url.rstrip('/') + '/reset', secret, payload, idempotency=False)
def serve(port: int, secret: str):
"""Run a callback receiver that verifies signatures and prints replies."""
from flask import Flask, request
app = Flask(__name__)
@app.route('/', methods=['POST'])
def recv():
raw = request.get_data()
ok = verify(secret, raw, request.headers.get(HEADER_TIMESTAMP), request.headers.get(HEADER_SIGNATURE))
if not ok:
print('!! signature verification FAILED — rejecting')
return {'error': 'bad signature'}, 401
data = json.loads(raw)
text_parts = [c.get('text', '') for c in data.get('message', []) if c.get('type') == 'Plain']
marker = 'FINAL' if data.get('is_final') else 'part '
print(
f'[{marker}] session={data["session_id"]} seq={data["sequence"]} '
f'reply_to={data.get("reply_to")}: {" ".join(text_parts)}'
)
return {'ok': True}
print(f'callback receiver listening on http://0.0.0.0:{port}/ (Ctrl-C to stop)')
app.run(host='0.0.0.0', port=port)
def main(argv=None):
p = argparse.ArgumentParser(description='LangBot HTTP Bot reference client')
sub = p.add_subparsers(dest='cmd', required=True)
sp = sub.add_parser('serve', help='run the callback receiver')
sp.add_argument('--port', type=int, default=8900)
sp.add_argument('--secret', required=True)
for name in ('push', 'sync', 'reset'):
c = sub.add_parser(name)
c.add_argument('--url', required=True, help='https://host/bots/<BOT_UUID>')
c.add_argument('--secret', required=True)
c.add_argument('--session', required=True)
c.add_argument('--session-type', default='person', choices=['person', 'group'])
if name != 'reset':
c.add_argument('--text', required=True)
args = p.parse_args(argv)
if args.cmd == 'serve':
serve(args.port, args.secret)
elif args.cmd == 'push':
push(args.url, args.secret, args.session, args.text, args.session_type)
elif args.cmd == 'sync':
push_sync(args.url, args.secret, args.session, args.text, args.session_type)
elif args.cmd == 'reset':
reset(args.url, args.secret, args.session, args.session_type)
if __name__ == '__main__':
sys.exit(main())
+123
View File
@@ -0,0 +1,123 @@
/**
* LangBot HTTP Bot adapter reference client (TypeScript / Node 18+).
*
* Zero runtime dependencies (uses global `fetch`, `crypto`, and `http`).
*
* - `push()` : fire-and-collect; reply lands on your callback URL.
* - `pushSync()` : POST /sync and await the collapsed reply.
* - `reset()` : reset a session's conversation.
* - `startReceiver()` : a callback server that verifies signatures and logs
* replies, so you can watch N->1 and 1->M live.
*
* Run the demos:
* npx tsx client.ts serve 8900 SHARED_SECRET
* npx tsx client.ts push https://host/bots/<UUID> SHARED_SECRET ticket-1 "hello"
* npx tsx client.ts sync https://host/bots/<UUID> SHARED_SECRET ticket-1 "hello"
* npx tsx client.ts reset https://host/bots/<UUID> SHARED_SECRET ticket-1
*/
import { createHmac, randomUUID, timingSafeEqual } from 'node:crypto';
import { createServer } from 'node:http';
const HEADER_TIMESTAMP = 'X-LB-Timestamp';
const HEADER_SIGNATURE = 'X-LB-Signature';
const HEADER_IDEMPOTENCY = 'X-LB-Idempotency-Key';
const REPLAY_WINDOW = 300;
/** Compute the `sha256=<hex>` signature over `"{ts}." + body`. */
export function sign(secret: string, body: Buffer | string, timestamp?: number): [string, string] {
const ts = String(timestamp ?? Math.floor(Date.now() / 1000));
const buf = typeof body === 'string' ? Buffer.from(body) : body;
const mac = createHmac('sha256', secret).update(Buffer.concat([Buffer.from(`${ts}.`), buf])).digest('hex');
return [ts, `sha256=${mac}`];
}
/** Verify an inbound signature (used by the callback receiver). */
export function verify(secret: string, body: Buffer, timestamp?: string, signature?: string): boolean {
if (!timestamp || !signature) return false;
if (Math.abs(Math.floor(Date.now() / 1000) - Number(timestamp)) > REPLAY_WINDOW) return false;
const [, expected] = sign(secret, body, Number(timestamp));
const a = Buffer.from(expected);
const b = Buffer.from(signature);
return a.length === b.length && timingSafeEqual(a, b);
}
interface Segment { type: string; text?: string; url?: string; [k: string]: unknown }
async function post(url: string, secret: string, payload: object, idempotency = true) {
const body = Buffer.from(JSON.stringify(payload));
const [ts, sig] = sign(secret, body);
const headers: Record<string, string> = {
'Content-Type': 'application/json',
[HEADER_TIMESTAMP]: ts,
[HEADER_SIGNATURE]: sig,
};
if (idempotency) headers[HEADER_IDEMPOTENCY] = randomUUID();
const resp = await fetch(url, { method: 'POST', headers, body });
const text = await resp.text();
console.log(`-> ${resp.status} ${text}`);
return { status: resp.status, text };
}
/** Fire-and-collect: 202 now, reply later on your callback URL. */
export function push(url: string, secret: string, session: string, text: string, sessionType = 'person') {
return post(url.replace(/\/$/, ''), secret, {
session_id: session,
session_type: sessionType,
message: [{ type: 'Plain', text }] as Segment[],
});
}
/** Blocking convenience: POST /sync, get the collapsed reply. */
export function pushSync(url: string, secret: string, session: string, text: string, sessionType = 'person') {
return post(`${url.replace(/\/$/, '')}/sync`, secret, {
session_id: session,
session_type: sessionType,
message: [{ type: 'Plain', text }] as Segment[],
}, false);
}
/** Reset a session's conversation. */
export function reset(url: string, secret: string, session: string, sessionType = 'person') {
return post(`${url.replace(/\/$/, '')}/reset`, secret, { session_id: session, session_type: sessionType }, false);
}
/** Run a callback receiver that verifies signatures and prints replies. */
export function startReceiver(port: number, secret: string) {
const server = createServer((req, res) => {
if (req.method !== 'POST') { res.writeHead(405).end(); return; }
const chunks: Buffer[] = [];
req.on('data', (c) => chunks.push(c));
req.on('end', () => {
const raw = Buffer.concat(chunks);
const ok = verify(secret, raw, req.headers[HEADER_TIMESTAMP.toLowerCase()] as string,
req.headers[HEADER_SIGNATURE.toLowerCase()] as string);
if (!ok) {
console.log('!! signature verification FAILED — rejecting');
res.writeHead(401, { 'Content-Type': 'application/json' }).end(JSON.stringify({ error: 'bad signature' }));
return;
}
const data = JSON.parse(raw.toString());
const parts = (data.message as Segment[]).filter((c) => c.type === 'Plain').map((c) => c.text).join(' ');
const marker = data.is_final ? 'FINAL' : 'part ';
console.log(`[${marker}] session=${data.session_id} seq=${data.sequence} reply_to=${data.reply_to}: ${parts}`);
res.writeHead(200, { 'Content-Type': 'application/json' }).end(JSON.stringify({ ok: true }));
});
});
server.listen(port, () => console.log(`callback receiver listening on http://0.0.0.0:${port}/ (Ctrl-C to stop)`));
}
// --- CLI ---
const [cmd, ...rest] = process.argv.slice(2);
if (cmd === 'serve') {
startReceiver(Number(rest[0] ?? 8900), rest[1] ?? 'SHARED_SECRET');
} else if (cmd === 'push') {
push(rest[0], rest[1], rest[2], rest[3]);
} else if (cmd === 'sync') {
pushSync(rest[0], rest[1], rest[2], rest[3]);
} else if (cmd === 'reset') {
reset(rest[0], rest[1], rest[2]);
} else if (cmd) {
console.error(`unknown command: ${cmd}`);
process.exit(1);
}
+349
View File
@@ -0,0 +1,349 @@
#!/usr/bin/env python3
"""LangBot HTTP Bot — interactive playground (public, browser-based).
This is a REAL end-to-end demo against the RUNNING LangBot instance on this
host. It is NOT a mock and NOT an in-process import: every message you type in
the browser is signed and POSTed to the live `http_bot` bot at
http://127.0.0.1:5300/bots/<uuid>, and the bot's replies come back to this
server's /callback endpoint over real HTTP, then stream to your browser via SSE.
What it does on startup:
1. Reads the LangBot API key + the http_bot bot from data/langbot.db.
2. Configures the bot via the LangBot API (PUT /api/v1/platform/bots/<uuid>):
sets inbound_secret + outbound_secret + callback_url to point back here.
(LangBot reloads the bot live no server restart needed.)
3. Serves a chat page on 0.0.0.0:<PORT> so you can open it from the internet.
Run: ./.venv/bin/python examples/http-bot/playground.py
Then open: http://<this-host-public-ip>:<PORT>/
"""
from __future__ import annotations
import asyncio
import json
import os
import sqlite3
import sys
REPO = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..'))
sys.path.insert(0, os.path.join(REPO, 'src'))
from aiohttp import web # noqa: E402
import aiohttp # noqa: E402
from langbot.pkg.platform.sources import http_bot_signing as sg # noqa: E402
# ---- config -----------------------------------------------------------------
LANGBOT_BASE = 'http://127.0.0.1:5300'
DB_PATH = os.path.join(REPO, 'data', 'langbot.db')
PUBLIC_IP = os.environ.get('PUBLIC_IP', '127.0.0.1')
PORT = int(os.environ.get('PLAYGROUND_PORT', '8920'))
SECRET = 'playground-shared-secret'
# SSE subscribers: list of asyncio.Queue
subscribers: list[asyncio.Queue] = []
def db_lookup() -> tuple[str, str]:
"""Return (api_key, http_bot_uuid) from the LangBot DB."""
db = sqlite3.connect(DB_PATH)
db.row_factory = sqlite3.Row
api_key = db.execute('SELECT key FROM api_keys LIMIT 1').fetchone()['key']
bot = db.execute("SELECT uuid FROM bots WHERE adapter='http_bot' LIMIT 1").fetchone()
if not bot:
raise SystemExit('No http_bot bot found. Create one in the WebUI first.')
return api_key, bot['uuid']
async def configure_bot(api_key: str, bot_uuid: str, callback_url: str):
"""Point the live bot at this playground via the LangBot API.
update_bot() runs a raw SQL UPDATE with whatever keys we send, so we send a
MINIMAL payload: only adapter_config (built from scratch, not read back
the GET masks secrets). LangBot reloads + reruns the bot live.
"""
cfg = {
'inbound_secret': SECRET,
'outbound_secret': SECRET,
'callback_url': callback_url,
'signature_required': True,
'default_session_type': 'person',
'callback_timeout': 15,
'callback_max_retries': 3,
}
async with aiohttp.ClientSession() as s:
async with s.put(
f'{LANGBOT_BASE}/api/v1/platform/bots/{bot_uuid}',
headers={'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json'},
json={'adapter_config': cfg},
) as r:
txt = await r.text()
print(f'[configure] PUT adapter_config -> {r.status} {txt[:200]}')
return r.status < 400
async def broadcast(event: dict):
for q in list(subscribers):
try:
q.put_nowait(event)
except Exception:
pass
# ---- HTTP handlers ----------------------------------------------------------
async def index(request: web.Request):
return web.Response(text=PAGE, content_type='text/html')
async def send(request: web.Request):
"""Browser -> here -> signed POST -> live LangBot bot."""
body_in = await request.json()
session_id = body_in.get('session_id') or 'playground-1'
text = body_in.get('text', '')
bot_uuid = request.app['bot_uuid']
payload = {
'session_id': session_id,
'sender': {'id': 'browser-user', 'name': 'You'},
'message': [{'type': 'Plain', 'text': text}],
}
raw = json.dumps(payload, ensure_ascii=False).encode()
ts, sig = sg.sign(SECRET, raw)
url = f'{LANGBOT_BASE}/bots/{bot_uuid}'
# echo what we send to the browser timeline
await broadcast(
{'dir': 'out', 'kind': 'request', 'session_id': session_id, 'text': text, 'url': url, 'sig': sig[:24] + ''}
)
async with aiohttp.ClientSession() as s:
async with s.post(
url,
data=raw,
headers={
'Content-Type': 'application/json',
sg.HEADER_TIMESTAMP: ts,
sg.HEADER_SIGNATURE: sig,
},
) as r:
status = r.status
try:
jr = await r.json()
except Exception:
jr = {'raw': await r.text()}
await broadcast({'dir': 'in', 'kind': 'ack', 'status': status, 'data': jr})
return web.json_response({'status': status, 'data': jr})
async def callback(request: web.Request):
"""Live LangBot bot -> here. Verify signature, stream to browser."""
raw = await request.read()
ok, why = sg.verify(SECRET, raw, request.headers.get(sg.HEADER_TIMESTAMP), request.headers.get(sg.HEADER_SIGNATURE))
data = json.loads(raw)
text = ' '.join(c.get('text', '') for c in data.get('message', []) if c.get('type') == 'Plain')
await broadcast(
{
'dir': 'in',
'kind': 'reply',
'session_id': data.get('session_id'),
'sequence': data.get('sequence'),
'is_final': data.get('is_final'),
'sig_ok': ok,
'sig_why': why,
'text': text,
}
)
return web.json_response({'ok': True})
async def events(request: web.Request):
"""SSE stream to the browser."""
resp = web.StreamResponse(
headers={
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'Access-Control-Allow-Origin': '*',
}
)
await resp.prepare(request)
q: asyncio.Queue = asyncio.Queue()
subscribers.append(q)
try:
await resp.write(b': connected\n\n')
while True:
try:
ev = await asyncio.wait_for(q.get(), timeout=15)
await resp.write(f'data: {json.dumps(ev, ensure_ascii=False)}\n\n'.encode())
except asyncio.TimeoutError:
await resp.write(b': ping\n\n')
except (asyncio.CancelledError, ConnectionResetError):
pass
finally:
if q in subscribers:
subscribers.remove(q)
return resp
PAGE = r"""<!doctype html>
<html lang="zh"><head><meta charset="utf-8"/>
<meta name="viewport" content="width=device-width,initial-scale=1"/>
<title>LangBot HTTP Bot · 调试台</title>
<style>
:root{
--bg:#f7f8fa; --panel:#ffffff; --line:#e8eaed; --ink:#1f2329; --mut:#8a909a;
--brand:#2563eb; --brand-soft:#eef3ff; --ok:#16a34a; --bad:#dc2626; --code:#f3f4f6;
}
*{box-sizing:border-box}
html,body{height:100%}
body{margin:0;background:var(--bg);color:var(--ink);
font:14px/1.6 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"PingFang SC","Microsoft YaHei",sans-serif}
.top{height:52px;background:var(--panel);border-bottom:1px solid var(--line);
display:flex;align-items:center;gap:10px;padding:0 18px}
.logo{width:26px;height:26px;border-radius:7px;background:var(--brand);display:grid;place-items:center;color:#fff;font-weight:700;font-size:14px}
.top b{font-size:15px} .top .ver{font-size:12px;color:var(--mut)}
.dot{width:8px;height:8px;border-radius:50%;background:#cbd2dc;display:inline-block;margin-right:5px;vertical-align:middle}
.dot.on{background:var(--ok)} .dot.off{background:var(--bad)}
.conn{margin-left:auto;font-size:12px;color:var(--mut)}
.wrap{max-width:1080px;margin:0 auto;padding:18px;display:grid;grid-template-columns:1fr 360px;gap:16px}
@media(max-width:880px){.wrap{grid-template-columns:1fr}}
.card{background:var(--panel);border:1px solid var(--line);border-radius:12px;display:flex;flex-direction:column;min-height:0}
.card h3{margin:0;padding:12px 16px;font-size:13px;font-weight:600;color:#4b5563;border-bottom:1px solid var(--line);display:flex;align-items:center;gap:8px}
.chat{height:62vh}
.msgs{flex:1;overflow:auto;padding:16px;display:flex;flex-direction:column;gap:12px}
.row{display:flex;flex-direction:column;gap:4px;max-width:82%}
.row.me{align-self:flex-end;align-items:flex-end}
.row.bot{align-self:flex-start}
.bub{padding:9px 13px;border-radius:12px;white-space:pre-wrap;word-break:break-word}
.me .bub{background:var(--brand);color:#fff;border-bottom-right-radius:3px}
.bot .bub{background:#f1f3f6;color:var(--ink);border-bottom-left-radius:3px}
.meta{font-size:11px;color:var(--mut)}
.meta .ok{color:var(--ok)} .meta .bad{color:var(--bad)}
.sys{align-self:center;font-size:12px;color:var(--mut);background:#f1f3f6;border-radius:8px;padding:4px 12px}
.bar{display:flex;gap:8px;padding:12px;border-top:1px solid var(--line)}
.bar input{flex:1;border:1px solid var(--line);border-radius:9px;padding:10px 12px;font-size:14px;outline:none}
.bar input:focus{border-color:var(--brand);box-shadow:0 0 0 3px var(--brand-soft)}
.bar button{background:var(--brand);color:#fff;border:0;border-radius:9px;padding:0 18px;font-size:14px;font-weight:500;cursor:pointer}
.bar button:disabled{opacity:.5;cursor:default}
.side{height:62vh}
.kv{padding:12px 16px;border-bottom:1px solid var(--line);font-size:12px}
.kv .k{color:var(--mut)} .kv .v{color:var(--ink);word-break:break-all}
.kv code{background:var(--code);border-radius:5px;padding:1px 5px;font-size:11px}
.sessrow{display:flex;align-items:center;gap:8px;padding:10px 16px;border-bottom:1px solid var(--line);font-size:12px}
.sessrow input{flex:1;border:1px solid var(--line);border-radius:7px;padding:5px 8px;font-size:12px}
.sessrow button{border:1px solid var(--line);background:#fff;border-radius:7px;padding:5px 9px;font-size:12px;cursor:pointer;color:#4b5563}
.trace{flex:1;overflow:auto;padding:10px 12px;font:11px/1.55 ui-monospace,SFMono-Regular,Menlo,monospace}
.ev{padding:6px 8px;border-radius:7px;margin-bottom:6px;border:1px solid var(--line)}
.ev .t{font-weight:600;font-size:10px;letter-spacing:.3px;text-transform:uppercase}
.ev.out{background:#f5f8ff;border-color:#dbe6ff}.ev.out .t{color:var(--brand)}
.ev.ack{background:#f4f6f8}.ev.ack .t{color:#6b7280}
.ev.reply{background:#f1faf3;border-color:#cdeed6}.ev.reply .t{color:var(--ok)}
.ev pre{margin:3px 0 0;white-space:pre-wrap;word-break:break-all;color:#374151}
</style></head>
<body>
<div class="top">
<div class="logo">L</div>
<b>HTTP Bot 调试台</b><span class="ver">examples/http-bot</span>
<span class="conn"><span class="dot off" id="cdot"></span><span id="conn">连接中</span></span>
</div>
<div class="wrap">
<!-- chat -->
<div class="card chat">
<h3>对话 · 真实发往运行中的 http_bot</h3>
<div class="msgs" id="msgs"></div>
<div class="bar">
<input id="msg" placeholder="输入消息,回车发送…" autofocus/>
<button id="send">发送</button>
</div>
</div>
<!-- debug -->
<div class="card side">
<h3>调试信息</h3>
<div class="kv"><span class="k">入站地址</span><br><span class="v"><code id="endpoint">/bots/&lt;uuid&gt;</code></span></div>
<div class="kv"><span class="k">签名</span> <span class="v">HMAC-SHA256 · <code>X-LB-Signature</code></span></div>
<div class="sessrow">
<span class="k">会话</span>
<input id="sid" value="playground-1"/>
<button id="reset">新会话</button>
</div>
<div class="trace" id="trace"></div>
</div>
</div>
<script>
const $=s=>document.querySelector(s);
const msgs=$('#msgs'),trace=$('#trace'),inp=$('#msg'),btn=$('#send'),
conn=$('#conn'),cdot=$('#cdot'),sidIn=$('#sid');
function el(c){const d=document.createElement('div');d.className=c;return d}
function atBottom(n){n.scrollTop=n.scrollHeight}
function bubble(side,text,metaHtml){
const r=el('row '+side),b=el('bub');b.textContent=text;r.appendChild(b);
if(metaHtml){const m=el('meta');m.innerHTML=metaHtml;r.appendChild(m)}
msgs.appendChild(r);atBottom(msgs)}
function sys(t){const d=el('sys');d.textContent=t;msgs.appendChild(d);atBottom(msgs)}
function logEv(kind,title,obj){
const e=el('ev '+kind),t=el('t');t.textContent=title;e.appendChild(t);
if(obj!==undefined){const p=document.createElement('pre');
p.textContent=typeof obj==='string'?obj:JSON.stringify(obj,null,2);e.appendChild(p)}
trace.appendChild(e);atBottom(trace)}
const es=new EventSource('/events');
es.onopen=()=>{conn.textContent='SSE 已连接';cdot.className='dot on'};
es.onerror=()=>{conn.textContent='SSE 断开,重连…';cdot.className='dot off'};
es.onmessage=e=>{const ev=JSON.parse(e.data);
if(ev.kind==='request'){
if(ev.endpoint)$('#endpoint').textContent=ev.url||ev.endpoint;
logEv('out','出站 · 已签名 POST',{url:ev.url,session_id:ev.session_id,'X-LB-Signature':ev.sig});
}else if(ev.kind==='ack'){
const id=ev.data&&ev.data.data&&ev.data.data.accepted_message_id;
sys(`LangBot 已接收 · HTTP ${ev.status}`);
logEv('ack','入站确认 202',{status:ev.status,accepted_message_id:id||'-'});
}else if(ev.kind==='reply'){
const sig=ev.sig_ok?'<span class=ok>验签通过</span>':'<span class=bad>验签失败</span>';
bubble('bot',ev.text,`seq=${ev.sequence} · ${ev.is_final?'<b>FINAL</b>':'中间段'} · ${sig}`);
logEv('reply',`回调 · seq ${ev.sequence}${ev.is_final?' · FINAL':''}`,
{session_id:ev.session_id,sequence:ev.sequence,is_final:ev.is_final,sig_ok:ev.sig_ok,text:ev.text});
}};
async function send(){
const t=inp.value.trim();if(!t)return;inp.value='';btn.disabled=true;
bubble('me',t,'已签名 → POST /bots/&lt;uuid&gt;');
try{await fetch('/send',{method:'POST',headers:{'Content-Type':'application/json'},
body:JSON.stringify({session_id:sidIn.value.trim()||'playground-1',text:t})});}
catch(e){sys('发送失败:'+e)}
btn.disabled=false;inp.focus();}
btn.onclick=send;inp.addEventListener('keydown',e=>{if(e.key==='Enter')send()});
$('#reset').onclick=()=>{sidIn.value='playground-'+Math.random().toString(36).slice(2,7);
sys('已切换到新会话 '+sidIn.value);};
sys('调试台就绪 · 每条消息都会真实发往运行中的 http_bot,右侧可观察签名 / 202 / 回调全过程。');
</script>
</body></html>"""
async def main():
api_key, bot_uuid = db_lookup()
callback_url = f'http://{PUBLIC_IP}:{PORT}/callback'
print(f'[init] http_bot uuid = {bot_uuid}')
print(f'[init] callback_url = {callback_url}')
ok = await configure_bot(api_key, bot_uuid, callback_url)
if not ok:
print('[warn] bot config update failed; check the API key / payload shape')
app = web.Application()
app['bot_uuid'] = bot_uuid
app.router.add_get('/', index)
app.router.add_post('/send', send)
app.router.add_post('/callback', callback)
app.router.add_get('/events', events)
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, '0.0.0.0', PORT)
await site.start()
print(f'\n ▶ 打开: http://{PUBLIC_IP}:{PORT}/\n')
while True:
await asyncio.sleep(3600)
if __name__ == '__main__':
asyncio.run(main())
+48
View File
@@ -0,0 +1,48 @@
# Page Bot Adapter — Embed Demo
> English | [中文](./README.zh.md)
A single self-contained HTML page that demos the LangBot **Page Bot**
(`web_page_bot`) embeddable chat widget — the one you drop onto any website with
a single `<script>` tag.
Full guide: [docs.langbot.app — Page Bot](https://docs.langbot.app/en/usage/platforms/webpage).
## Files
| File | What it is |
|---|---|
| `index.html` | **Browser demo** — open it, point it at a running LangBot instance + a Page Bot you created, and it loads the live embed widget so you can chat with the bot exactly as a site visitor would. Zero deps, no build step. |
## How to use
1. In the LangBot WebUI, create a bot with the **Page Bot** (`页面机器人`)
adapter and bind it to a working pipeline. Copy its **bot UUID** from the
generated embed code.
2. Open `index.html` in a browser. Any of these work:
- double-click the file, or
- serve the folder: `python3 -m http.server 8930` then open
`http://localhost:8930/examples/web-page-bot/`.
3. Fill in:
- **LangBot base URL** — where your instance is reachable from the browser
(e.g. `http://localhost:5300`, or your public address).
- **Page Bot UUID** — from step 1.
- **Widget title** — optional, sets the `data-title` attribute.
4. Click **Load widget**. A floating chat bubble appears in the bottom-right
corner — click it and chat.
The page also renders the exact `<script>` snippet you'd paste into your own
site (before `</body>`), and updates it live as you edit the fields.
## What it demonstrates
- The embed contract: `<script data-title="…" src="<base>/api/v1/embed/<uuid>/widget.js"></script>`.
- `widget.js` is served by LangBot pre-configured for that bot UUID — title,
bubble icon, language and optional Cloudflare Turnstile protection all come
from the bot's config, no page changes needed.
- Messages travel over a WebSocket to the bot's bound pipeline; replies stream
back into the bubble.
> The widget loads `widget.js` from your LangBot instance, so the **base URL
> must be reachable from the browser** you open this page in. If LangBot runs on
> a server, use its public address instead of `localhost`.
+44
View File
@@ -0,0 +1,44 @@
# 页面机器人适配器 —— 嵌入演示
> [English](./README.md) | 中文
一个自包含的单文件 HTML 页面,用于演示 LangBot **页面机器人**
(`web_page_bot`) 的可嵌入聊天组件 —— 也就是你用一行 `<script>` 标签就能放到任意
网站上的那个组件。
完整指南:[docs.langbot.app —— 页面机器人](https://docs.langbot.app/zh/usage/platforms/webpage)。
## 文件清单
| 文件 | 是什么 |
|---|---|
| `index.html` | **浏览器演示页** —— 打开它,填上一个运行中的 LangBot 实例地址 + 你创建的页面机器人,它就会加载真实的嵌入组件,让你像网站访客一样和机器人对话。零依赖,无需构建。 |
## 使用方法
1. 在 LangBot WebUI 中,用 **页面机器人**`web_page_bot`)适配器创建一个机器人,
并绑定一个可用的流水线。从生成的嵌入代码里复制它的 **机器人 UUID**
2. 在浏览器中打开 `index.html`,以下任一方式皆可:
- 直接双击该文件;或
- 起一个静态服务:`python3 -m http.server 8930`,然后打开
`http://localhost:8930/examples/web-page-bot/`
3. 填写:
- **LangBot base URL** —— 你的实例在该浏览器中可访问的地址
(例如 `http://localhost:5300`,或你的公网地址)。
- **页面机器人 UUID** —— 第 1 步里复制的。
- **组件标题** —— 可选,对应 `data-title` 属性。
4. 点击 **Load widget**。页面右下角会出现一个浮动聊天气泡 —— 点开即可对话。
页面还会实时渲染出你需要粘贴到自己网站(放在 `</body>` 前)的那段 `<script>`
代码,并随着你编辑输入框同步更新。
## 它演示了什么
- 嵌入契约:`<script data-title="…" src="<base>/api/v1/embed/<uuid>/widget.js"></script>`
- `widget.js` 由 LangBot 针对该机器人 UUID 预配置后下发 —— 标题、气泡图标、语言
以及可选的 Cloudflare Turnstile 防护,全部来自机器人配置,无需改动页面。
- 消息通过 WebSocket 发往机器人绑定的流水线,回复流式回到气泡中。
> 组件会从你的 LangBot 实例加载 `widget.js`,因此 **base URL 必须能从你打开本页
> 的浏览器访问到**。如果 LangBot 部署在服务器上,请用它的公网地址而非
> `localhost`
+205
View File
@@ -0,0 +1,205 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>LangBot Page Bot · Embed Demo</title>
<style>
:root {
--bg: #f7f8fa; --panel: #ffffff; --line: #e8eaed; --ink: #1f2329;
--mut: #8a909a; --brand: #2563eb; --brand-soft: #eef3ff;
--ok: #16a34a; --bad: #dc2626; --code: #f3f4f6;
}
* { box-sizing: border-box; }
html, body { height: 100%; }
body {
margin: 0; background: var(--bg); color: var(--ink);
font: 14px/1.6 -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
"PingFang SC", "Microsoft YaHei", sans-serif;
}
.top {
height: 52px; background: var(--panel); border-bottom: 1px solid var(--line);
display: flex; align-items: center; gap: 10px; padding: 0 18px;
}
.logo {
width: 26px; height: 26px; border-radius: 7px; background: var(--brand);
display: grid; place-items: center; color: #fff; font-weight: 700; font-size: 14px;
}
.top b { font-size: 15px; }
.top .ver { font-size: 12px; color: var(--mut); }
.wrap { max-width: 760px; margin: 0 auto; padding: 28px 18px 80px; }
.hero h1 { margin: 8px 0 6px; font-size: 22px; }
.hero p { margin: 0 0 4px; color: var(--mut); }
.card {
background: var(--panel); border: 1px solid var(--line); border-radius: 12px;
padding: 20px; margin-top: 20px;
}
.card h3 {
margin: 0 0 14px; font-size: 14px; font-weight: 600; color: #4b5563;
display: flex; align-items: center; gap: 8px;
}
.card h3 .num {
width: 20px; height: 20px; border-radius: 50%; background: var(--brand-soft);
color: var(--brand); display: grid; place-items: center; font-size: 12px; font-weight: 700;
}
.field { margin-bottom: 14px; }
.field:last-child { margin-bottom: 0; }
.field label { display: block; font-size: 12px; color: var(--mut); margin-bottom: 5px; }
.field input {
width: 100%; border: 1px solid var(--line); border-radius: 9px;
padding: 10px 12px; font-size: 14px; outline: none; font-family: inherit;
}
.field input:focus { border-color: var(--brand); box-shadow: 0 0 0 3px var(--brand-soft); }
.hint { font-size: 12px; color: var(--mut); margin-top: 5px; }
.hint code { background: var(--code); border-radius: 5px; padding: 1px 5px; font-size: 11px; }
.actions { display: flex; gap: 10px; margin-top: 18px; align-items: center; }
button {
border: 0; border-radius: 9px; padding: 10px 18px; font-size: 14px;
font-weight: 500; cursor: pointer; font-family: inherit;
}
.btn-primary { background: var(--brand); color: #fff; }
.btn-primary:disabled { opacity: .5; cursor: default; }
.btn-ghost { background: #fff; border: 1px solid var(--line); color: #4b5563; }
.status { font-size: 13px; color: var(--mut); }
.status .ok { color: var(--ok); }
.status .bad { color: var(--bad); }
pre {
background: #0f172a; color: #e2e8f0; border-radius: 10px; padding: 14px 16px;
overflow: auto; font: 12px/1.6 ui-monospace, SFMono-Regular, Menlo, monospace;
margin: 0;
}
.snippet-row { position: relative; }
.snippet-row .copy {
position: absolute; top: 10px; right: 10px; background: rgba(255,255,255,.12);
color: #fff; border: 0; border-radius: 7px; padding: 5px 10px; font-size: 12px; cursor: pointer;
}
ul.steps { margin: 0; padding-left: 18px; color: #4b5563; }
ul.steps li { margin-bottom: 6px; }
</style>
</head>
<body>
<div class="top">
<div class="logo">L</div>
<b>Page Bot · Embed Demo</b>
<span class="ver">examples/web-page-bot</span>
</div>
<div class="wrap">
<div class="hero">
<h1>Try the LangBot Page Bot widget</h1>
<p>Point this page at a running LangBot instance and a <strong>Page Bot</strong> you created,</p>
<p>then load the live embed widget below to chat with it — exactly as your site visitors would.</p>
</div>
<div class="card">
<h3><span class="num">1</span> Connect your Page Bot</h3>
<div class="field">
<label for="base">LangBot base URL</label>
<input id="base" placeholder="http://localhost:5300" value="http://localhost:5300" />
<div class="hint">The address where your LangBot instance is reachable from this browser. No trailing slash.</div>
</div>
<div class="field">
<label for="uuid">Page Bot UUID</label>
<input id="uuid" placeholder="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" />
<div class="hint">Create a bot with the <code>Page Bot</code> adapter in the WebUI, then copy its UUID from the embed code.</div>
</div>
<div class="field">
<label for="title">Widget title (optional)</label>
<input id="title" placeholder="LangBot" value="LangBot" />
</div>
<div class="actions">
<button id="load" class="btn-primary">Load widget</button>
<button id="unload" class="btn-ghost">Remove widget</button>
<span class="status" id="status">Not loaded.</span>
</div>
</div>
<div class="card">
<h3><span class="num">2</span> The embed snippet</h3>
<p style="margin:0 0 12px;color:var(--mut)">This is exactly what you paste into your own site (before <code>&lt;/body&gt;</code>). It updates as you edit the fields above.</p>
<div class="snippet-row">
<button class="copy" id="copy">Copy</button>
<pre id="snippet">&lt;script data-title="LangBot" src="http://localhost:5300/api/v1/embed/&lt;bot-uuid&gt;/widget.js"&gt;&lt;/script&gt;</pre>
</div>
</div>
<div class="card">
<h3><span class="num">3</span> How it works</h3>
<ul class="steps">
<li>The <code>&lt;script&gt;</code> tag pulls <code>widget.js</code> from your LangBot instance, pre-configured for that bot UUID.</li>
<li>A floating chat bubble appears in the bottom-right corner of the page.</li>
<li>Messages travel over a WebSocket to the bot's bound pipeline; replies stream back into the bubble.</li>
<li>Title, bubble icon, language and optional Cloudflare Turnstile protection are all set in the bot's config — no page changes needed.</li>
</ul>
</div>
</div>
<script>
var $ = function (s) { return document.querySelector(s); };
var baseEl = $("#base"), uuidEl = $("#uuid"), titleEl = $("#title"),
statusEl = $("#status"), snippetEl = $("#snippet");
var WIDGET_ID = "langbot-embed-demo-script";
function clean(v) { return (v || "").trim().replace(/\/+$/, ""); }
function buildSrc() {
var base = clean(baseEl.value) || "http://localhost:5300";
var uuid = uuidEl.value.trim() || "<bot-uuid>";
return base + "/api/v1/embed/" + uuid + "/widget.js";
}
function refreshSnippet() {
var title = titleEl.value.trim() || "LangBot";
var src = buildSrc();
snippetEl.textContent =
'<script data-title="' + title + '" src="' + src + '"><\/script>';
}
function setStatus(html) { statusEl.innerHTML = html; }
function removeWidget() {
var old = document.getElementById(WIDGET_ID);
if (old) old.remove();
// The widget injects its own DOM (bubble + panel). Clear the common containers it creates.
document.querySelectorAll('[id^="langbot-"]').forEach(function (n) {
if (n.id !== WIDGET_ID) n.remove();
});
}
function loadWidget() {
var uuid = uuidEl.value.trim();
var uuidRe = /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/i;
if (!uuidRe.test(uuid)) {
setStatus('<span class="bad">Enter a valid bot UUID first.</span>');
return;
}
removeWidget();
var s = document.createElement("script");
s.id = WIDGET_ID;
s.setAttribute("data-title", titleEl.value.trim() || "LangBot");
s.src = buildSrc();
s.onload = function () {
setStatus('<span class="ok">Widget loaded — look bottom-right.</span>');
};
s.onerror = function () {
setStatus('<span class="bad">Failed to load widget.js — check the base URL and that the bot is enabled.</span>');
};
document.body.appendChild(s);
setStatus("Loading…");
}
$("#load").onclick = loadWidget;
$("#unload").onclick = function () {
removeWidget();
setStatus("Widget removed.");
};
$("#copy").onclick = function () {
navigator.clipboard.writeText(snippetEl.textContent).then(function () {
var b = $("#copy"); b.textContent = "Copied"; setTimeout(function () { b.textContent = "Copy"; }, 1200);
});
};
[baseEl, uuidEl, titleEl].forEach(function (el) { el.addEventListener("input", refreshSnippet); });
refreshSnippet();
</script>
</body>
</html>
+7 -6
View File
@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.10.2"
version = "4.10.6"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -8,7 +8,7 @@ requires-python = ">=3.11,<4.0"
dependencies = [
"aiocqhttp>=1.4.4",
"aiofiles>=24.1.0",
"aiohttp>=3.14.0",
"aiohttp>=3.14.1",
"aioshutil>=1.5",
"aiosqlite>=0.21.0",
"anthropic>=0.51.0",
@@ -16,7 +16,7 @@ dependencies = [
"async-lru>=2.0.5",
"certifi>=2025.4.26",
"colorlog~=6.6.0",
"cryptography>=46.0.7",
"cryptography>=48.0.1",
"dashscope>=1.25.10",
"dingtalk-stream>=0.24.0",
"discord-py>=2.5.2",
@@ -61,16 +61,16 @@ dependencies = [
"beautifulsoup4>=4.12.3",
"ebooklib>=0.18",
"html2text>=2024.2.26",
"langchain>=0.2.0",
"langchain>=1.3.9",
"langchain-core>=1.3.3",
"langsmith>=0.8.0",
"langsmith>=0.8.18",
"python-multipart>=0.0.27",
"Mako>=1.3.12",
"langchain-text-splitters>=1.1.2",
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.4.5",
"langbot-plugin @ git+https://github.com/langbot-app/langbot-plugin-sdk.git@e7d946af4a6b1494fbe74627c1815ace19ac8991",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2",
@@ -80,6 +80,7 @@ dependencies = [
"pgvector>=0.4.1",
"botocore>=1.42.39",
"litellm>=1.0.0",
"valkey-glide>=2.4.1,<3.0.0; sys_platform != 'win32'", # No Windows wheels are published
]
keywords = [
"bot",
Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

File diff suppressed because it is too large Load Diff
+9
View File
@@ -0,0 +1,9 @@
node_modules/
coverage/
.tap/
__pycache__/
*.pyc
skills/.env.local
reports/
skills/*/reports/
.browser/
+68
View File
@@ -0,0 +1,68 @@
# Agent Workflow
This repository stores reusable LangBot agent-testing assets. Keep changes structured so the next agent does not need to rediscover paths.
## First Steps
1. Read `skills/.env` before using local URLs, paths, browser profiles, or proxy defaults. If present, `skills/.env.local` overrides it for this machine and must not be committed. On a new machine, copy `skills/.env.example` to `skills/.env.local` first.
2. Pick the smallest relevant skill:
- `langbot-env-setup` for environment, browser, OAuth, proxy, and startup.
- `langbot-testing` for WebUI, provider, pipeline, cases, and troubleshooting.
- `langbot-skills-maintenance` for adding, deduplicating, or auditing this skills repository.
3. Prefer existing cases and troubleshooting entries before exploring from scratch.
## Editing Rules
- UI/browser testing is the primary QA path. API/curl checks are diagnostic only and cannot make a UI case pass by themselves.
- Put skills under `skills/<name>/`.
- Keep `SKILL.md` concise; move detailed workflows to `references/`.
- Put reusable test paths in `cases/*.yaml`.
- New or edited cases must include `priority`, `risk`, `ci_eligible`, and `evidence_required` so agents can select the right test set without rereading every file.
- Use `env_any` / `automation_env_any` for one-of machine inputs, such as `LANGBOT_PIPELINE_URL|LANGBOT_PIPELINE_NAME`; do not list those alternatives as separate all-required env keys.
- Put reusable groups of cases in `suites/*.yaml` rather than hardcoding test sets in docs or CLI code.
- Put growing failure knowledge in `troubleshooting/*.yaml`.
- Do not hardcode local ports in testing docs; use `skills/.env` variables and machine-local `skills/.env.local` overrides.
- Do not store secrets, API keys, OAuth tokens, or localStorage token values.
## Required Checks
After structural changes, run:
```bash
bin/lbs validate
```
After changing skills, cases, or troubleshooting assets, run:
```bash
bin/lbs index
```
Use `bin/lbs env show` to inspect defaults and `bin/lbs env doctor` when diagnosing local environment readiness. Env output is redacted by default; do not work around that by printing raw secrets.
`bin/lbs` is a generated local wrapper. If it is missing on a fresh checkout, run `npm run bootstrap` from this directory first; `npm install` also regenerates it via `prepare`.
Use `bin/lbs fixture check` before fixture-heavy cases such as MCP, RAG, multimodal, or plugin smoke tests.
Use `bin/lbs case list --ready` for cases that have no missing machine inputs and no manual preconditions. Use `bin/lbs case list --machine-ready` when you want to keep `manual-check` candidates and confirm their preconditions yourself.
Before executing a saved QA path, generate the agent-facing plan:
```bash
bin/lbs test plan <case-id>
```
Read the plan readiness sections before running the browser path. Missing env,
automation env, or fixture readiness means the case is not ready to execute and
should be marked `blocked` or fixed first.
`manual_check` means machine inputs are present but the agent must verify the
declared `preconditions` or `setup` items before executing the UI path. Do not
turn a `manual_check` case into `pass` until those items were checked in the
same run.
Before executing a group of saved QA paths, generate the suite plan:
```bash
bin/lbs suite plan <suite-id>
```
Use `bin/lbs suite start <suite-id>` to create a shared suite run id, suite evidence root, per-case evidence directories, and `suite-start.json`/`suite-start.md` handoff files. Then run `bin/lbs suite report <suite-id> --evidence-dir <dir>` to aggregate case results.
Automation scripts write `automation-result.json`; write the final per-case `result.json` with `bin/lbs test result <case-id> --result <status> --reason <text> --evidence-dir <dir> --evidence <comma-list>` after collecting the required evidence. A `pass` result must include all required evidence.
For runner-specific Debug Chat cases, prefer case-specific pipeline env keys such as `LANGBOT_LOCAL_AGENT_PIPELINE_URL` over the generic `LANGBOT_PIPELINE_URL`; otherwise an agent can accidentally test the wrong runner.
+59
View File
@@ -0,0 +1,59 @@
# LangBot Skills
This directory is the **single source of truth** for LangBot's agent skills —
reusable, on-demand instruction packs for AI agents (Claude Code, Codex, Cursor,
and LangBot's own Local Agent) working with the LangBot ecosystem.
> These skills were consolidated here from the former `langbot-app/langbot-skills`
> repository (now archived). Documentation and the landing page link here; do not
> re-copy skill content elsewhere — link to this directory instead.
## Skill catalog
| Skill | What it covers |
| --- | --- |
| [`langbot-dev`](skills/langbot-dev) | Core backend + web frontend development (Quart, Vite, API, migrations, MCP server) |
| [`langbot-plugin-dev`](skills/langbot-plugin-dev) | Plugin SDK / component development, debugging, WebSocket testing |
| [`langbot-deploy`](skills/langbot-deploy) | Docker / Compose / Kubernetes deployment, config.yaml, Box runtime, global API key |
| [`langbot-testing`](skills/langbot-testing) | WebUI / e2e QA harness, cases, fixtures, troubleshooting (the `bin/lbs` CLI) |
| [`langbot-env-setup`](skills/langbot-env-setup) | Local dev/test environment, browser access, OAuth, proxy, startup |
| [`langbot-mcp-ops`](skills/langbot-mcp-ops) | Operating a LangBot instance through its MCP server (`/mcp`) |
| [`langbot-space-ops`](skills/langbot-space-ops) | Browsing the LangBot Space marketplaces through the Space MCP server |
| [`langbot-eba-adapter-dev`](skills/langbot-eba-adapter-dev) | Building platform adapters for the Event-Based Agents architecture |
| [`langbot-skills-maintenance`](skills/langbot-skills-maintenance) | Adding, deduplicating, and auditing skills in this directory |
`skills.index.json` is the machine-readable index (regenerate with `bin/lbs index`).
## Quick start (for an AI agent)
1. Read this README, `AGENTS.md`, and `docs/user-guide.md` to understand the layout.
2. Read `skills/.env` for shared local defaults. On a new machine, copy
`skills/.env.example` to `skills/.env.local` (gitignored) and override
machine-specific values there. Never commit secrets.
3. Pick the smallest relevant skill from the catalog above and follow its
`SKILL.md`.
## The `lbs` CLI
The testing assets ship with a small CLI (`bin/lbs`, Node >= 22.6). The
`bin/lbs` wrapper is a generated local entrypoint; on a fresh checkout, run
`npm run bootstrap` once if it is missing. `npm install` also regenerates it via
the `prepare` script.
```bash
npm run bootstrap # create bin/lbs if missing
bin/lbs validate # validate skills/cases/troubleshooting structure
bin/lbs index # regenerate skills.index.json
bin/lbs env show # inspect resolved env defaults (redacted)
bin/lbs env doctor # diagnose local environment readiness
bin/lbs case list --ready
bin/lbs test plan <case-id>
bin/lbs suite plan langbot-debug-chat-load-gate
```
## Maintenance rule
When the LangBot / LangBot Space **API or MCP server changes**, the
corresponding skill here MUST be updated in the same change. The MCP tool
surface, the API, and these skills are kept in lockstep — see each repo's
`AGENTS.md`.
+171
View File
@@ -0,0 +1,171 @@
# LangBot QA Skills User Guide
Use this guide as the first operational path after reading `README.md` and
`AGENTS.md`.
## 1. Configure Local Inputs
Read `skills/.env`, then create `skills/.env.local` for machine-local values.
Do not commit `.env.local`, browser profiles, reports, tokens, API keys, OAuth
state, or provider credentials.
Minimum local fields for live browser QA:
```bash
LANGBOT_REPO=/path/to/LangBot
LANGBOT_WEB_REPO=/path/to/LangBot/web
LANGBOT_BACKEND_URL=http://127.0.0.1:5300
LANGBOT_FRONTEND_URL=http://127.0.0.1:3000
LANGBOT_DEV_FRONTEND_URL=http://127.0.0.1:3000
LANGBOT_BROWSER_PROFILE=/path/to/langbot-browser-profile
LANGBOT_CHROMIUM_EXECUTABLE=/path/to/chromium-or-playwright-chrome
LANGBOT_E2E_LOGIN_USER=qa-local@example.com
```
`LANGBOT_E2E_LOGIN_USER` is a local QA account. The setup automation uses the
LangBot recovery key from the active checkout to initialize or refresh that
local account and write a browser `localStorage` token. It does not need the
user's GitHub or Space credentials.
## 2. Check Readiness
From `skills/`:
```bash
bin/lbs env show
bin/lbs env doctor
bin/lbs validate
bin/lbs index --check
```
`env doctor` should report reachable backend and frontend URLs before live
browser cases are run. Missing Space provider credentials are not a LangBot
product pass; classify them as `env_issue` and configure the local Space
provider before measuring Debug Chat performance.
## 3. Start Services
Start the backend from `LANGBOT_REPO`:
```bash
cd "$LANGBOT_REPO"
uv run main.py
```
Start the standalone frontend from `LANGBOT_WEB_REPO` and point it at the
backend:
```bash
cd "$LANGBOT_WEB_REPO"
VITE_API_BASE_URL="$LANGBOT_BACKEND_URL" pnpm dev --host 0.0.0.0
```
If `VITE_API_BASE_URL` is missing, browser tests can load the Vite page but send
API requests to the frontend port, which produces false UI failures.
## 4. Prepare User-Path Fixtures
For local-agent Debug Chat cases and the user-path performance gate:
```bash
node scripts/e2e/ensure-local-agent-pipeline.mjs --write-env
```
The script:
- refreshes the local QA login and browser token;
- marks the local wizard as skipped;
- creates or updates a local QA pipeline;
- scans Space LLM models, tests candidates, and switches to the first working
Space model with tested fallback models;
- writes `LANGBOT_PIPELINE_URL`, `LANGBOT_PIPELINE_NAME`, and local-agent
pipeline/model variables into `skills/.env.local`;
- returns `env_issue` when no Space model can be scanned or tested.
Useful model controls:
```bash
LANGBOT_E2E_MODEL_TEST_LIMIT=8
LANGBOT_E2E_MODEL_FALLBACK_COUNT=3
LANGBOT_E2E_SKIP_MODEL_UUIDS=uuid-a,uuid-b
LANGBOT_E2E_SKIP_MODEL_NAMES=model-a,model-b
LANGBOT_E2E_SCAN_SPACE_MODELS=true
```
The setup writes a current-runtime compatibility `max-round` value into the
pipeline config because this backend still reads that field directly during
message truncation. Do not treat it as a long-term QA contract.
## 5. Run Gates
Fast contract gate, no live service required:
```bash
bin/lbs suite run langbot-performance-contract-gate --run-id langbot-contract-local
```
Live backend gate:
```bash
bin/lbs suite run langbot-live-backend-gate --run-id langbot-backend-local
```
Browser-visible user-path performance gate:
```bash
bin/lbs suite plan langbot-user-path-performance-gate
bin/lbs suite run langbot-user-path-performance-gate --run-id langbot-user-path-local --include-manual-check
```
Controlled Debug Chat message-path load gate (manual/non-required; run fake-provider cases serially when they share `LANGBOT_FAKE_PROVIDER_URL`):
```bash
bin/lbs suite plan langbot-debug-chat-load-gate
bin/lbs test run langbot-fake-provider-debug-chat-load --run-id langbot-fake-load-local
bin/lbs test run langbot-fake-provider-debug-chat-slow-load --run-id langbot-fake-slow-local
bin/lbs test run langbot-fake-provider-debug-chat-fault-recovery --run-id langbot-fake-fault-local
bin/lbs test run langbot-space-debug-chat-concurrency-smoke --run-id langbot-space-smoke-local
```
Cross-pipeline Debug Chat isolation is a separate manual regression gate because
current releases may fail it due to product bug #2286:
```bash
bin/lbs suite plan langbot-debug-chat-isolation-gate
bin/lbs suite run langbot-debug-chat-isolation-gate --run-id langbot-debug-chat-isolation-local --include-manual-check
```
Start with `langbot-fake-provider-debug-chat-load`. It launches a local
OpenAI-compatible fake provider, creates the matching provider/model/pipeline,
then sends concurrent WebSocket Debug Chat messages through the real backend.
Use `langbot-fake-provider-debug-chat-slow-load` to measure the same path under
deterministic streaming latency. Use
`langbot-fake-provider-debug-chat-fault-recovery` to inject bounded provider
HTTP failures and confirm later Debug Chat requests recover. Use the separate
`langbot-debug-chat-isolation-gate` to verify that concurrent Debug Chat traffic
on two pipelines does not leak assistant responses across pipeline boundaries;
current releases may fail that gate because of #2286, so keep it out of the
normal load gate until the product fix lands.
Use `langbot-space-debug-chat-concurrency-smoke` only as a low-volume live
provider smoke; it includes Space/model/network latency and should be compared
against the fake-provider baseline before attributing failures to LangBot.
`manual_check` means the agent must confirm the declared preconditions for that
run window. When setup automation is declared, run output may stop early with
`env_issue`; fix that environment input before treating the product path as
measured.
## 6. Read Results
Suite reports live under `skills/reports/`. Evidence lives under
`skills/reports/evidence/<run-id>/`.
For performance cases, inspect:
- `metrics.json` for p50/p95/p99, error rate, and total duration;
- `automation-result.json` for threshold decisions and artifacts;
- `console.log` and `network.log` for frontend/API failures;
- backend logs for provider, runner, WebSocket, or persistence failures.
Do not call a user-path performance result a LangBot overhead regression until
provider/tool/network time has been separated or ruled out.
+29
View File
@@ -0,0 +1,29 @@
{
"private": true,
"type": "module",
"bin": {
"lbs": "./bin/lbs"
},
"scripts": {
"bootstrap": "node scripts/bootstrap-lbs.mjs",
"prepare": "node scripts/bootstrap-lbs.mjs",
"prevalidate": "node scripts/bootstrap-lbs.mjs",
"preindex": "node scripts/bootstrap-lbs.mjs",
"preindex:check": "node scripts/bootstrap-lbs.mjs",
"pretest": "node scripts/bootstrap-lbs.mjs",
"precheck": "node scripts/bootstrap-lbs.mjs",
"lbs": "node src/lbs.ts",
"test": "node test/lbs-cli.test.ts",
"validate": "node src/lbs.ts validate",
"index": "node src/lbs.ts index",
"index:check": "node src/lbs.ts index --check",
"check:syntax": "find src test scripts -type f \\( -name '*.ts' -o -name '*.mjs' \\) -print0 | xargs -0 -n1 node --check",
"check": "npm run check:syntax && npm run validate && npm test"
},
"engines": {
"node": ">=22.6"
},
"devDependencies": {
"playwright": "^1.60.0"
}
}
@@ -0,0 +1,117 @@
# LangBot Agent Testing 技术选型
## 状态
这是技术选型背景文档,不是当前路线图。当前黑盒 E2E QA 的实施顺序见:
```text
docs/qa-agent/04-black-box-e2e-roadmap.md
```
## 目标
`langbot-skills` 的目标不是替代测试框架,而是沉淀 agent 可复用的测试资产,让开发者 clone 仓库后,可以让 Codex、Claude Code、Computer Use 或 Playwright MCP 复用已有路径完成 LangBot 功能验证。
核心原则:
- Skill 负责路由和少量规则。
- Reference 负责可读流程和背景知识。
- Case 负责结构化测试路径。
- Troubleshooting 负责结构化故障资产。
- `lbs` 负责结构校验、索引、资产创建和未来的运行/报告能力。
- UI/browser 是产品 QA 的主路径;API/curl 只用于诊断。
## 浏览器控制层
不同开发者可用的浏览器控制能力不同,所以浏览器层必须可替换。
| 方案 | 适用场景 | 优点 | 代价 |
|---|---|---|---|
| Codex / Claude Computer Use | agent 可以直接控制可见浏览器 | 登录和交互路径最自然,通常不需要额外 MCP 浏览器桥接 | 依赖具体 agent 工具能力 |
| Playwright MCP | 没有 Computer Use,但有 MCP 浏览器工具 | 稳定、可脚本化、适合回归路径 | OAuth 登录通常需要额外 visible profile |
| 直接 Playwright 脚本 | 测试路径非常稳定,适合 CI | 可重复性强 | 需要维护脚本和 selector |
| 商业 AI QA 平台 | 团队希望外包测试运行平台 | 报告和 PR 集成完整 | 成本和平台绑定 |
## 当前推荐
先采用分层降级:
```text
有 Computer Use
是 -> 使用 Computer Use 控制浏览器
否 -> 使用 Playwright MCP
需要 GitHub OAuth
是 -> 使用持久浏览器 profile,让用户手动完成登录
否 -> 直接使用已有登录态或测试账号状态
```
具体选择逻辑沉淀在:
```text
skills/langbot-env-setup/references/browser-access-selection.md
```
测试原则固定在:
```text
docs/qa-agent/03-agent-browser-qa-principles.md
```
## 环境变量层
测试文档不应写死端口。共享默认值放在:
```text
skills/.env
```
关键变量:
```text
LANGBOT_FRONTEND_URL
LANGBOT_BACKEND_URL
LANGBOT_DEV_FRONTEND_URL
LANGBOT_REPO
LANGBOT_WEB_REPO
LANGBOT_BROWSER_PROFILE
```
Agent 执行测试前应先读取 `skills/.env`,再用用户提供的当前环境或已启动服务覆盖默认值。
## 测试资产层
测试资产分两类:
```text
skills/<skill>/
references/ # Markdown 流程说明
cases/ # 结构化测试用例
troubleshooting/ # 结构化故障记录
```
当前已实现:
- `SKILL.md` 路由
- `references/*.md`
- `lbs case new/list/show`
- `lbs trouble show/search`
- `lbs test plan`
- `lbs test report`
- `lbs list / validate / index`
下一步重点:
- 日志守卫规则补充
- 报告产物管理
## 关键判断
不要强制所有内容只能通过 CLI 修改。更好的模式是:
- 新增 case/troubleshooting:优先使用 `lbs`
- 大段流程说明:允许直接编辑 Markdown
- 结构性变更后:必须运行 `lbs validate`
- 任何生成索引的变更后:运行 `lbs index`
这样既能沉淀结构化资产,又不会在 schema 未稳定时拖慢迭代。
@@ -0,0 +1,231 @@
# LangBot Skills 测试资产库规划
## 状态
这是早期测试资产库规划文档,保留用于解释 `langbot-skills` 的分层来源。
当前路线已经收敛为黑盒 E2E QA:开发者用 agent 通过浏览器测试 LangBot
稳定路径沉淀为 case,失败知识沉淀为 troubleshooting。`lbs test report`
日志守卫已有 MVP,后续重点是报告证据、case 元数据和少量稳定路径自动化。当前优先级见:
```text
docs/qa-agent/04-black-box-e2e-roadmap.md
```
本文中关于 `case list/show``trouble show/search``test plan` 的“计划实现”
内容已经部分过时,因为这些能力已经落地。
## 目标
让开发者 clone `langbot-skills` 后,可以把测试意图交给 agent,由 agent 复用已有环境配置、测试路径和故障知识完成 LangBot 功能验证。
典型场景:
- 冒烟测试:验证 pipeline Debug Chat、provider、常见页面是否正常。
- Provider 测试:添加 DeepSeek/OpenAI/Claude 等供应商并验证模型可用。
- 新 feature 测试:探索新 UI 路径,并在稳定后沉淀成 case/reference。
- 回归测试:复用旧路径,避免每个窗口重新探索登录、模型配置、pipeline 调试。
- 故障沉淀:把 runtime 超时、代理不一致、WebSocket 问题记录为可搜索资产。
核心方向见 `03-agent-browser-qa-principles.md`:agent 必须以浏览器/UI 为主路径,API/curl 只能作为诊断手段。
## 当前仓库结构
```text
skills/
.env # 共享默认变量
langbot-env-setup/ # 环境准备、浏览器控制路径、代理、登录态
langbot-testing/ # WebUI / provider / pipeline 测试入口
langbot-plugin-dev/ # 插件开发测试
langbot-eba-adapter-dev/ # 平台适配器开发测试
src/
lbs.ts # CLI 源码
bin/
lbs # CLI 入口
docs/
qa-agent/ # 规划文档,历史目录名保留
```
## 设计分层
### 1. Skill 层
`SKILL.md` 只做触发和路由,不承载大段流程。
例子:
```text
langbot-env-setup -> 选择 Computer Use / Playwright MCP / OAuth profile / proxy
langbot-testing -> 选择 WebUI / pipeline / provider / troubleshooting
```
### 2. Reference 层
Markdown 记录人和 agent 都能读的流程说明。
适合内容:
- 如何选择浏览器控制方式
- 如何启动/检查服务
- 如何执行 pipeline Debug Chat
- 如何处理 OAuth 登录态
### 3. Case 层
使用 YAML 记录可重复测试路径。
建议结构:
```text
skills/langbot-testing/cases/
pipeline-debug-chat.yaml
provider-deepseek.yaml
```
建议格式:
```yaml
id: pipeline-debug-chat
title: Pipeline Debug Chat returns a bot response
mode: agent-browser
area: pipeline
type: smoke
skills:
- langbot-env-setup
- langbot-testing
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
steps:
- Open LANGBOT_FRONTEND_URL
- Navigate to Pipelines
- Open target pipeline
- Select Debug Chat
- Send deterministic prompt
checks:
- "UI: User message appears"
- "UI: Bot message appears"
- "Console: No unexpected frontend errors"
- "Logs: Backend log includes Conversation(0) Streaming completed"
diagnostics:
- "Use API/curl only after the UI path is attempted, to distinguish frontend display failure from backend/runtime failure."
troubleshooting:
- plugin-runtime-timeout
- proxy-env-mismatch
```
### 4. Troubleshooting 层
故障资产会逐渐变大,适合结构化记录。
历史 Markdown 入口保留在:
```text
skills/langbot-testing/references/troubleshooting.md
```
当前 canonical 结构化故障资产在:
```text
skills/langbot-testing/troubleshooting/
plugin-runtime-timeout.yaml
proxy-env-mismatch.yaml
```
### 5. CLI 层
`lbs` 是统一入口,不再引入独立 `qa` 命令。
已实现或当前可用:
```bash
bin/lbs list
bin/lbs validate
bin/lbs index
bin/lbs new-skill <name>
bin/lbs new-ref <skill> <name>
bin/lbs case new pipeline-debug-chat --title "Pipeline Debug Chat"
bin/lbs case list
bin/lbs case show pipeline-debug-chat
bin/lbs trouble list <skill>
bin/lbs trouble show plugin-runtime-timeout
bin/lbs trouble search runtime
bin/lbs trouble add <skill> --title ... --symptom ... --cause ... --fix ...
bin/lbs test plan pipeline-debug-chat
bin/lbs test start pipeline-debug-chat
bin/lbs test run pipeline-debug-chat --dry-run
bin/lbs test report pipeline-debug-chat
bin/lbs test report pipeline-debug-chat --backend-log /path/to/backend.log
```
## 测试库位置
不要使用隐藏 `.qa/` 作为主测试库。测试资产应该和 skill 放在一起,便于触发和维护:
```text
skills/langbot-testing/
references/
cases/
troubleshooting/
reports/ # 可选,本地运行产物可按需忽略或输出到外部目录
```
如果未来需要项目本地测试库,可以允许 `lbs` 支持 `--workspace` 或项目根目录配置,但 canonical 资产仍保存在 `langbot-skills`
## 阶段规划
### 阶段一:环境和测试路径沉淀
状态:基本完成,持续维护。
- `skills/.env` 管共享默认变量。
- `langbot-env-setup` 拆出 Computer Use、Playwright MCP、OAuth profile、proxy、service startup。
- `langbot-testing` 记录 WebUI、pipeline、provider 测试路径。
- `lbs validate/index` 维护结构。
完成标准:
- agent 可以从 `skills/.env` 和 references 中找到当前测试入口。
- pipeline Debug Chat 这类路径不再需要从头探索。
### 阶段二:结构化 case/troubleshooting
状态:主体已完成,继续补齐元数据和资产质量。
目标:
- `lbs case new/list/show`
- `lbs trouble show/search`
- case id 去重、字段校验、索引生成
完成标准:
- 冒烟测试路径可以用结构化 case 表示。
- 下一个 agent 窗口可以直接读取 case 执行。
### 阶段三:计划和报告
状态:已有 MVP,继续完善。
目标:
- `lbs test plan <case>`
- agent 按 plan 使用浏览器执行 UI QA
- `lbs test report`
- 日志守卫集成
- 报告产物和 evidence 约定
完成标准:
- agent 可以按 case plan 执行浏览器测试。
- 结果报告包含 UI 结果、后端日志、console 错误和 troubleshooting 建议。
## 执行规则
- agent 可以直接编辑 Markdown reference。
- 新增结构化 case/troubleshooting 时,优先使用 `lbs`
- 每次结构变更后运行 `bin/lbs validate`
- 每次索引相关变更后运行 `bin/lbs index`
- 测试文档不写死端口,使用 `skills/.env` 中的 URL 变量。
- 测试 case 的 `mode` 固定为 `agent-browser`
- API/curl 只能写入 `diagnostics`,不能替代 UI 步骤和 UI 检查。
@@ -0,0 +1,161 @@
# 日志守卫规划
## 状态
这是当前活跃设计,已有第一版文件扫描 MVP。实现边界需要和黑盒 E2E 路线保持一致:
- 日志守卫服务于 `lbs test report`
- 它不替代浏览器/UI 判断。
- 它不发展成独立后端 API 测试框架。
- 第一版默认扫描 `LANGBOT_REPO/data/logs/` 下最新的 `langbot-*.log`,也可扫描 agent
显式提供的 backend/frontend/console 日志文件。
当前总体路线见:
```text
docs/qa-agent/04-black-box-e2e-roadmap.md
```
## 目标
日志守卫是 `lbs test report` 的一部分,用来在 agent 执行测试期间捕获 UI 断言之外的运行时问题。
当前命令方向已收敛为 `lbs test plan` / `lbs test report`。日志守卫服务于 agent-browser QA,不是独立的后端 API 测试入口。
LangBot 是异步且集成度高的系统,有些问题不会直接表现为页面失败:
- 后台任务异常
- 未等待的协程
- Provider 流式调用失败
- 插件 runtime 超时
- 平台发送失败
- 数据库连接问题
- 敏感信息泄露
日志守卫负责把这些信号结构化地放进测试报告,并关联到 troubleshooting 资产。
## 输入
日志守卫应从环境和运行上下文读取配置:
- `skills/.env` 中的 `LANGBOT_BACKEND_URL`
- `skills/.env` 中的 `LANGBOT_REPO`,用于自动发现 LangBot 后端日志
- `lbs test plan` / report 记录的 case id
- LangBot 后端进程输出
- 前端 dev server 输出
- 浏览器 console/network 错误
- case 声明的 success/failure patterns 和 expected failures
## MVP 范围
- 读取一个或多个日志流或日志文件。
- 检测错误模式。
- 支持按 case id 或 pattern 白名单。
- 输出 JSON/Markdown 摘要。
- 发现非预期错误时让测试报告标记失败;未来如果有自动执行器,再返回非零退出码。
## 错误分类
### 永远非预期
除非 case 明确声明,否则应失败:
- `Traceback`
- `Task exception was never retrieved`
- `RuntimeWarning: coroutine .* was never awaited`
- `Unclosed client session`
- `Unclosed connector`
- `KeyError`
- `TypeError`
- `AttributeError`
- 密钥、token、secret 明文泄露
### Case 预期错误
只有当前 case 声明时允许:
- 无效 provider key
- Provider 认证失败
- 无效 webhook payload
- 插件测试故意抛错
- 超时测试
- 限流测试
### 仅警告
报告但默认不失败:
- 可恢复重试
- 恢复的超时
- 废弃配置
- 慢请求
- 版本检查失败
## 与 Troubleshooting 集成
日志守卫不只输出错误文本,还应尽量匹配已知 troubleshooting id。
例子:
```text
Action list_plugins call timed out
Action list_agent_runners call timed out
Action invoke_llm_stream call timed out
```
可映射到:
```text
plugin-runtime-timeout
```
```text
uppercase proxy points to one host, lowercase proxy points to another
```
可映射到:
```text
proxy-env-mismatch
```
## 未来命令
```bash
bin/lbs test plan pipeline-debug-chat
bin/lbs test start pipeline-debug-chat
bin/lbs test run pipeline-debug-chat --dry-run
bin/lbs test report pipeline-debug-chat
bin/lbs test report --output report.md
bin/lbs test report pipeline-debug-chat --backend-log /path/to/backend.log --console-log /path/to/console.log
bin/lbs test report pipeline-debug-chat --since "2026-05-21T10:30:00+08:00"
bin/lbs test report pipeline-debug-chat --tail-lines 2000
bin/lbs test report pipeline-debug-chat --since "2026-05-21T10:30:00+08:00" --tail-lines 2000
bin/lbs test report pipeline-debug-chat --no-auto-log
```
运行报告应包含:
- case id
- URL 和环境变量摘要,不能包含 secrets
- 浏览器可见结果
- 后端日志摘要
- console/network 错误
- 匹配到的 troubleshooting id
- 通过/失败结论
## MVP 完成标准
- 可以自动扫描最新 LangBot 后端日志,也可以扫描前端日志和 console 日志文件。
- 可以用 `--since``--tail-lines` 把扫描范围限制到本次测试窗口。
- 可以检测明显 Python/运行时错误和 secret 泄露风险。
- 可以识别 case 声明的 success/failure patterns。
- 可以识别 troubleshooting pattern,包括 `plugin-runtime-timeout``proxy-env-mismatch`
- 支持 case 级白名单。
- 输出机器可读摘要。
- 至少一个 `langbot-testing` case 使用它。
当前 MVP 已覆盖自动发现 LangBot 后端日志、文件扫描、`--since`/`--tail-lines` 扫描窗口、
基础错误检测、case success/failure signal、troubleshooting 匹配、secret 脱敏和 `--json`
输出。仍待继续完善的是 live log 采集、更多规则、case 级 expected failure 的资产化和真实
E2E report 样例。
@@ -0,0 +1,57 @@
# Agent Browser QA Principles
This document fixes the direction of LangBot agent testing so the project does not drift into a backend API smoke-test framework.
## Primary Goal
`langbot-skills` should help an agent behave like a QA engineer using the product, not like a backend curl script.
The primary path is:
```text
developer intent -> lbs test plan -> agent controls browser -> UI result + console + logs -> report/assets
```
## Rules
1. Browser/UI interaction is the source of truth for product QA cases.
2. A backend API or curl response is never enough to mark a UI case passed.
3. API/curl/log checks are allowed as diagnostics after a UI path is attempted or when debugging environment readiness.
4. A case passes only when the user-visible UI result is correct.
5. The agent should inspect browser console/network output when available.
6. If screenshot or vision capability is available, the agent should check for blank pages, overlap, hidden actions, broken layout, and error toasts.
7. If no visual model is available, use DOM/accessibility snapshots and console output instead.
8. New stable UI paths should be added as `cases/*.yaml`.
9. New recurring failure modes should be added as `troubleshooting/*.yaml`.
10. Secrets, tokens, API keys, and localStorage token values must never be printed.
## Command Semantics
`lbs` manages assets and produces plans. It does not replace the agent's browser-control ability.
```bash
bin/lbs test plan pipeline-debug-chat
```
This command outputs:
- environment variables to use
- required skills
- browser steps
- UI/console/visual/log checks
- diagnostic options
- related troubleshooting patterns
- report template
The active agent then executes the plan with Computer Use, Playwright MCP, or another available browser-control tool.
## Diagnostics
Diagnostics can include:
- `bin/lbs env doctor`
- browser console/network inspection
- backend logs
- targeted API/curl checks
Diagnostics answer "where did it fail?" They do not replace "did the user-visible UI work?"
@@ -0,0 +1,299 @@
# 黑盒 E2E QA 路线图
## 定位
LangBot 有大量外部依赖:模型供应商、plugin runtime、浏览器登录态、
marketplace/network、RAG engine、sandbox backend、平台适配器等。单测仍然有价值,
但这个 QA 方向当前不优先解决 LangBot core 的单测覆盖率问题,因为重 mock 往往不能
真实代表产品路径。
`langbot-skills` 当前目标是让黑盒 E2E 测试变得可执行、可沉淀、可复用:
```text
开发者测试意图
-> 复用或新增 case
-> agent 通过浏览器执行
-> UI + console + network + log 证据
-> report
-> 反哺 case / troubleshooting
```
这是面向开发者的 QA 资产库。开发者可以让 agent 测一个 feature;如果路径稳定,
就把路径正规化为 case,让下一个开发者或 QA agent 继续复用。
## 非目标
- 这一阶段不优先建设 LangBot core 单测覆盖率。
- 不把 API/curl 作为 WebUI 行为的通过标准。
- 不要求每个 case 都能进 CI。
- 不在 report 和日志守卫有用之前急着做完整 browser runner。
- 不把外部 provider、OAuth、marketplace 抖动直接判成产品失败,除非证据明确。
## 当前状态
仓库已经具备第一层基础设施:
- `skills/.env``skills/.env.local` 管理测试环境;
- `langbot-env-setup``langbot-testing``langbot-plugin-dev` 等 skill
- `skills/langbot-testing/cases` 下的结构化 case
- `skills/langbot-testing/troubleshooting` 下的结构化故障资产;
- RAG、多模态、plugin、MCP 等 fixture
- `bin/lbs validate``bin/lbs index``bin/lbs case``bin/lbs trouble`
`bin/lbs test plan``bin/lbs test start``bin/lbs test report`
所以当前已经不是“先把路径写进 Markdown”的阶段,而是进入“让每次运行有证据、
有报告、能沉淀”的阶段。
## 测试模型
UI case 只有在用户可见行为正确时才能通过。辅助证据必须解释同一次运行。
通过一个 UI case 的最低证据:
- 用户可见的成功信号,例如 bot 回复、provider 保存成功、文件上传完成、plugin 页面渲染;
- 没有意外 browser console error
- 相关时间窗口内没有意外后端/runtime 错误;
- 有截图、DOM snapshot 或同等视觉/结构证据,如果当前 agent 能获取;
- API/curl 只在解释同一条 UI 路径时作为诊断证据。
失败报告需要保留足够信息,让开发者能复现或分流:
- case id 和实际测试 URL
- 使用的 browser path
- 最后可见 UI 状态;
- console/network 症状;
- 相关后端/前端日志;
- 匹配到的 troubleshooting id
- 这是产品失败、环境问题、外部依赖抖动,还是证据不足。
## 结果词汇
统一使用这些结果:
- `pass`UI 行为正确,辅助证据干净。
- `fail`:UI 行为错误,或同一次运行的 console/log 出现意外产品错误。
- `blocked`:缺登录、缺 provider credentials、服务未启动等原因导致目标路径没有跑起来。
- `env_issue`:失败在目标行为之外,例如 proxy、OAuth、provider quota、marketplace outage、
本地服务启动问题。
- `flaky`:同一环境下结果不稳定,进入门禁前需要先稳定。
做 merge/release 判断时,`env_issue``blocked` 不能算产品通过。
## 路线图
### Phase 0:对齐文档
目标:明确当前黑盒 E2E 方向。
交付物:
- `docs/qa-agent/README.md` 文档状态导航;
- 本路线图;
- 给旧规划文档加状态说明。
完成标准:
- 新贡献者不用通读所有旧文档,也能知道当前重点。
### Phase 1Test Report MVP
状态:已有第一版。
目标:让每次 agent browser 测试都有一致报告格式,即使 browser 执行还没自动化。
建议命令:
```bash
bin/lbs test start <case-id>
bin/lbs test report <case-id> --output reports/<timestamp>-<case-id>.md
```
MVP 行为:
- 读取 case 和关联 troubleshooting
- 生成 Markdown report 模板;
- 生成 run handoff,固定本次测试的 start timestamp 和推荐 report command
- 写入脱敏后的环境摘要;
- 提供 `pass/fail/blocked/env_issue/flaky` 结果选项;
- 包含 UI result、console errors、network symptoms、logs、screenshots、
diagnostics、matched troubleshooting、assets to update 等 section
- 支持 `--json`,输出机器可读报告。
第一版已经是 report generator,不急着做自动判定。先把 evidence 收集格式统一起来,
再做自动化更稳。
完成标准:
- agent 可以先跑 `lbs test start <case-id>`,用它给出的时间窗口执行浏览器路径,
然后按固定格式填写 report,不需要每次重新发明报告结构。
### Phase 2:日志守卫 MVP
状态:已有第一版文件扫描。
目标:捕获 UI 不一定明显展示的 runtime 问题。
日志守卫应集成进 `lbs test report`,不要发展成独立后端 API 测试框架。
建议命令形态:
```bash
bin/lbs test report <case-id> \
--backend-log /path/to/backend.log \
--frontend-log /path/to/frontend.log \
--console-log /path/to/console.log \
--evidence-dir reports/evidence/<run-id> \
--since "2026-05-21T10:30:00+08:00" \
--tail-lines 2000 \
--output reports/<timestamp>-<case-id>.md
```
MVP 行为:
- 默认从 `LANGBOT_REPO/data/logs/` 扫描最新 `langbot-*.log`
- 支持 agent 显式提供 backend、frontend、console 日志文件;
- 支持读取 evidence 目录下的 `automation-result.json`,把浏览器自动化脚本结论纳入报告;
- 支持 `lbs test result` 为人工/agent browser 运行写入标准 `result.json`,供 suite 聚合;
- 支持 `--since``--tail-lines`,避免历史日志污染本次报告;
- 检测默认非预期模式,例如 `Traceback`、未 await coroutine、unclosed client/connector、
`KeyError``TypeError``AttributeError`、明显 secret 泄露;
- 匹配 case 声明的 `success_patterns``failure_patterns`
- 匹配已知 troubleshooting,先支持 `plugin-runtime-timeout``proxy-env-mismatch`
- 只有 case 明确声明时,才允许 expected failure
- 将发现分类为 fail、warning、matched troubleshooting、ignored expected issue
- 永远不打印 secret 值。
完成标准:
- 至少 `pipeline-debug-chat` 能生成包含日志摘要和 troubleshooting 匹配结果的 report。
### Phase 3Case 元数据加固
状态:已有第一版。
目标:让 case 更容易选择、执行和晋级。
字段逐步补充,保持向后兼容:
```yaml
priority: p0 | p1 | p2
risk: low | medium | high
ci_eligible: false
preconditions:
- "Authenticated browser profile is available."
setup:
- "Start LangBot backend and frontend."
cleanup:
- "Remove temporary provider, plugin, or knowledge base if created."
expected_failures: []
success_patterns:
- "Conversation(0) Streaming completed"
failure_patterns:
- "Action invoke_llm_stream call timed out"
evidence:
required:
- ui
- console
- backend_log
```
当前实现采用扁平字段 `evidence_required`,避免轻量 YAML 解析器在 case 文件里承载嵌套结构。
`bin/lbs validate` 会校验 `priority``risk``ci_eligible``evidence_required`
`automation` 脚本路径、case 关联 skill 和 troubleshooting 交叉引用。`bin/lbs case list`
支持 `--json``--type``--area``--tag``--priority``--risk``--automation``--ci`
`--ready``--machine-ready` 过滤,方便 agent 快速选择测试集。
`env_any``automation_env_any` 用于表达 URL-or-name 这类 one-of 输入,避免把可替代变量误判为全部必填。
当前也有 `skills/<skill>/suites/*.yaml``bin/lbs suite plan <suite-id>`,用于组织常跑测试集,
例如 `core-smoke``local-agent-gate`
`agent-runner-release-gate`。发布门禁使用 `agent-runner-release-preflight`
先分类配置 blockers 和 runtime env issues,再运行较重的浏览器 Debug Chat case。
依赖 fixture 的 case 可以在浏览器执行前先跑 `bin/lbs fixture check`,检查
`fixtures/fixtures.json` 登记的 deterministic 文件、plugin 包和本地测试 server 是否存在。
`bin/lbs suite start <suite-id>` 会生成 suite run id、suite evidence root、per-case evidence 目录、
`suite-start.json`/`suite-start.md` handoff 文件和 per-case evidence 命令;
浏览器自动化脚本会写入 `automation-result.json`,供 `bin/lbs test report` 展示原始自动化结论;
`bin/lbs test result <case-id>` 会在人工/agent browser case 完成后写入最终 `result.json`
`bin/lbs suite report <suite-id> --evidence-dir <dir>` 会聚合各 case 的 `result.json`,并且
不会把缺少 required evidence 的 `pass` 当作 suite 通过。
Runner 专用 Debug Chat case 通过 `automation_pipeline_url_env`
`automation_pipeline_name_env` 绑定专用 pipeline 变量,避免 local-agent、Codex 或
Claude Code case 误用通用 `LANGBOT_PIPELINE_URL` 后产生假阳性。
Debug Chat case 还可以通过 `automation_stream_output` 固定流式或非流式发送路径。
多模态 Debug Chat case 可以通过 `automation_image_base64_fixture` 复用 deterministic 图片 fixture。
`test plan``suite plan` 会输出 readiness,让 agent 在执行浏览器前就看到缺失的 env、
自动化变量、fixture,以及需要人工确认的 `manual_check` 前置条件。
完成标准:
- `lbs case list` 或后续 filter 能回答“smoke 跑哪些”、“哪些适合 CI”、
“哪些需要真实 provider credentials”。
### Phase 4:开发者沉淀流程
目标:开发者让 agent 测新 feature 后,稳定路径不会丢在聊天记录里。
流程:
1. 开发者要求 agent 通过浏览器测试某个 feature。
2. agent 先按 UI 主路径探索。
3. agent 用 `lbs test start` 固定运行窗口,再用 `lbs test report` 写报告。
4. 如果路径稳定,agent 新增或更新 case。
5. 如果出现可复用故障,agent 新增或更新 troubleshooting。
6. agent 跑 `bin/lbs validate``bin/lbs index`
完成标准:
- feature QA 的结果能进入资产库,而不是只留在一次对话里。
### Phase 5:选择性浏览器自动化
状态:已有第一版 `test run` 入口和两个 Playwright 脚本。
目标:只自动化少量稳定、值得重复跑的黑盒路径。
建议顺序:
1. `webui-login-state`
2. `pipeline-debug-chat`
3. `local-agent-basic-debug-chat`
4. `local-agent-rag-debug-chat`
5. 一个基于 deterministic fixture 的 plugin 或 MCP smoke path
执行策略:
- 继续把 Computer Use 或 Playwright MCP 作为默认交互路径;
- 只给稳定、确定性的路径补直接 Playwright script
- 保存 screenshots、console logs、trace/video
- flaky 或强依赖真实 credentials 的 provider case 暂时不要进 CI。
当前已经绑定:
- `webui-login-state` -> `scripts/e2e/webui-login-state.mjs`
- `pipeline-debug-chat` -> `scripts/e2e/pipeline-debug-chat.mjs`
第一版自动化先产出 `reports/evidence/<run-id>/` 下的 console、network、screenshot 和
result JSON。真实执行后仍要用 `lbs test report --since ... --console-log ...` 做日志守卫和
最终报告。开发期间可以先用 `bin/lbs test run <case-id> --dry-run` 检查命令和 evidence 路径。
Debug Chat 类脚本应复用 `scripts/e2e/lib/debug-chat.mjs`,避免重复实现 visible response leaf
判断和已知失败信号分类。
完成标准:
- 小规模 smoke subset 可以不靠人工决定每一步点击;更大的资产库仍然服务于人工/agent
驱动的探索式 E2E。
## 下一批动工切片
在做 browser runner 之前,继续做这些:
1. 等 LangBot 当前开发状态稳定后,用一次真实 `pipeline-debug-chat` 跑通
`test start -> test run -> test report -> test result -> suite report`,产出 sample report。
2. 只给 smoke/local-agent 首批 case 补必要元数据。
3. 继续补日志守卫规则,尤其是 WebSocket、plugin runtime、provider streaming、前端
chunk/rendering failure。
4. 约定 report 产物目录、截图和 console/network 导出的命名方式。
5. 再评估是否开始给 `webui-login-state``pipeline-debug-chat` 做直接 Playwright
自动化。
这样 infra 会立刻有用,同时保留后续自动化 browser execution 的空间。
+46
View File
@@ -0,0 +1,46 @@
# LangBot QA Agent 文档导航
这个目录记录 `langbot-skills` 当前的 QA 方向和后续建设顺序。
## 当前判断
当前重点是 LangBot 的黑盒 E2E QA,不是 LangBot core 的单测覆盖率建设。
`langbot-skills` 要帮助开发者和 QA agent 做接近人工测试的 WebUI 验证:
- 打开真实 LangBot WebUI
- 按用户路径点击和输入;
- 检查用户可见的 UI 结果;
- 查看 console、network、截图、后端和前端日志;
- 输出可复用的测试报告;
- 把稳定 feature 路径沉淀为 case
- 把重复故障沉淀为 troubleshooting。
API 和 curl 只做诊断。它们可以解释失败原因,但不能让一个 UI case 通过。
## 文档状态
| 文档 | 状态 | 用途 |
| --- | --- | --- |
| `04-black-box-e2e-roadmap.md` | 当前主路线图 | 决定下一步建设什么。 |
| `03-agent-browser-qa-principles.md` | 当前原则文档 | 定义 browser-first QA 的通过标准。 |
| `02-log-guard-plan.md` | 当前活跃设计 | 设计 `lbs test report` 里的日志守卫。 |
| `../user-guide.md` | 当前使用手册 | 开发者日常使用。 |
| `00-technology-options.md` | 背景文档 | 选择 Computer Use、Playwright MCP 或未来直接 Playwright。 |
| `01-qa-agent-harness-plan.md` | 历史规划,部分过时 | 解释最初分层和目录设计;使用前先看状态说明。 |
## 已过时的点
`01-qa-agent-harness-plan.md` 还保留早期规划状态。现在结构化 cases、
结构化 troubleshooting、`validate``index``lbs test plan` 都已经落地。
已经补上第一版 `lbs test start``lbs test run``lbs test report` 和日志守卫文件扫描。
`webui-login-state``pipeline-debug-chat` 已经绑定直接 Playwright 自动化脚本。后续重点是:
- 报告 evidence 字段继续打磨;
- case success/failure signal 和日志守卫规则继续补充;
- 报告产物和 evidence 约定;
- 等 LangBot 当前开发状态稳定后跑真实 sample report。
不要再把旧阶段列表当成当前 source of truth。后续排序以
`04-black-box-e2e-roadmap.md` 为准。
+521
View File
@@ -0,0 +1,521 @@
# LangBot Skills 用户使用手册
## 这个仓库解决什么
`langbot-skills` 是给 agent 使用的 LangBot 测试资产库。开发者 clone 后,可以让 Codex、Claude Code、Computer Use 或 Playwright MCP 复用已有环境配置、测试路径和故障知识,像 QA 一样操作 LangBot WebUI。
核心目标:
- 不让下一个 agent 窗口从头探索登录、模型配置、pipeline 调试。
- 把稳定 UI 测试路径沉淀为 case。
- 把常见故障沉淀为 troubleshooting。
- 让 agent 优先通过浏览器点击验证产品行为。
- API/curl/log 只作为诊断手段,不作为 UI case 通过标准。
## 快速开始
1. Clone 仓库。
2. 检查本地默认变量:
```bash
bin/lbs env show
```
默认变量在:
```text
skills/.env
```
本机专用覆盖写到:
```text
skills/.env.local
```
它会覆盖 `skills/.env` 中的同名变量,并且不应该提交。
`skills/.env` 是共享默认值,不应写入本机绝对路径、浏览器 profile、provider key 或其他凭据。
新机器建议从模板开始:
```bash
cp skills/.env.example skills/.env.local
```
常用变量:
```text
LANGBOT_FRONTEND_URL
LANGBOT_BACKEND_URL
LANGBOT_DEV_FRONTEND_URL
LANGBOT_REPO
LANGBOT_WEB_REPO
LANGBOT_BROWSER_PROFILE
```
3. 检查环境是否就绪:
```bash
bin/lbs env doctor
bin/lbs fixture check
```
`env doctor` 会检查 URL、路径、代理变量等。代理变量是可选项;只有大小写代理变量互相冲突时才会报错。失败不一定代表仓库坏了,通常说明本地 LangBot 没启动、代理不一致或浏览器 profile 不存在。
`fixture check` 会检查仓库内测试 fixture 是否存在,例如 MCP stdio server、RAG 文档、多模态图片、qa-plugin-smoke 包和 QA AgentRunner 包。它也会校验 `.lbpkg` 是 zip 包,并检查 QA AgentRunner fixture 的入口文件未漂移。
4. 查看已有测试 case
```bash
bin/lbs case list
bin/lbs case list --json --priority p0 --automation
bin/lbs case list --ready
bin/lbs case list --machine-ready
bin/lbs suite list
bin/lbs suite plan core-smoke
bin/lbs suite plan agent-runner-release-gate
bin/lbs suite start core-smoke
bin/lbs suite start core-smoke --run-id core-smoke-local --evidence-dir reports/evidence/core-smoke-local
```
`case list` 支持按 `--type``--area``--tag``--priority``--risk``--automation`
`--ci``--ready``--machine-ready` 过滤。`--ready` 只显示没有缺机器输入且没有人工前置条件的 case;
`--machine-ready` 过滤掉缺机器输入的 case,但保留 `manual-check`,表示执行前还要确认前置条件。需要交给 agent 自动选择测试集时,优先使用 `--json`
其中包含 `priority``risk``ci_eligible``automation``evidence_required` 以及
env/automation/fixture/manual readiness。
Case metadata 中的 `env``automation_env` 表示全部必填;URL 或 name 这类二选一输入会放在
`env_any``automation_env_any`,readiness 只要求组合里至少一个变量有值。
如果要跑一组已沉淀的测试路径,优先使用 suite。Suite 位于 `skills/<skill>/suites/*.yaml`
只负责组织 case,不改变 UI/browser 作为通过标准的原则。
`suite plan` 会聚合 readiness:缺环境变量、缺自动化变量、缺 fixture 或需要
`manual_check` 时,会在执行前标出受影响的 case。`manual_check` 不是产品通过,
它表示机器配置已满足但 agent 必须先确认 case 里的 `preconditions``setup`
Runner externalization 发布判断使用 `agent-runner-release-gate`。先跑
`agent-runner-release-preflight`,把缺 pipeline、runner id 错误、插件未安装这类
`blocked`,以及 provider key、Box、插件运行时这类 `env_issue` 分开,再执行较重的
浏览器 Debug Chat case。
5. 生成 agent 执行计划:
```bash
bin/lbs test plan pipeline-debug-chat
```
然后把计划交给当前 agent 执行。agent 应使用 Computer Use、Playwright MCP 或其他浏览器控制能力去操作 UI。
`test plan` 中的 Environment、Automation Readiness、Fixture Readiness 和 Manual
Readiness 是执行前门禁;如果 readiness 缺失,应先补配置或将本次 case 标记为
`blocked`。如果状态是 `manual_check`,先确认 `preconditions``setup`,再开始 UI
执行。不要把后续 curl/API 诊断当成 UI case 通过。
## 推荐使用方式
### 冒烟测试
你可以直接对 agent 说:
```text
帮我跑一下 LangBot 新前端 smoke test。
```
agent 应该:
- 读 `skills/.env`
- 优先查看 `bin/lbs suite plan core-smoke`,或查找 `type: smoke` 的 cases
- 生成 test plan
- 用浏览器执行 UI 操作
- 检查 console、截图、后端日志
- 输出简短 QA 报告
### Runner Externalization 发布门禁
你可以直接对 agent 说:
```text
按 agent-runner release gate 跑完整矩阵,先做 preflight,再跑浏览器 case,并把 blocked/env_issue/fail 分开。
```
agent 应该先查看 `skills/langbot-testing/references/agent-runner-release-gate.md`
再执行:
```bash
bin/lbs test recommend
bin/lbs suite plan agent-runner-release-gate
bin/lbs test run agent-runner-release-preflight --dry-run
bin/lbs suite start agent-runner-release-gate --run-id agent-runner-release-local --evidence-dir reports/evidence/agent-runner-release-local
```
`test recommend` 输出的 run 命令默认带 `--dry-run`;确认 readiness 和 `manual_check` 前置条件后,再去掉 `--dry-run` 执行。
完成所有 case 后,用:
```bash
bin/lbs suite report agent-runner-release-gate --evidence-dir reports/evidence/agent-runner-release-local
```
没有最终 `result.json`、缺 required evidence、或把 `blocked`/`env_issue` 当成 pass
都不能算发布门禁通过。
### 新 Feature 测试
你可以说:
```text
我改了 provider 页面,帮我测一下 DeepSeek provider 添加、测试、绑定 pipeline 是否正常。
```
agent 应该:
- 查找相关 case 和 reference
- 如果没有稳定路径,先探索 UI
- 用浏览器执行真实交互
- 失败时用日志/API 辅助定位
- 稳定后新增或更新 case/reference
- 新故障沉淀为 troubleshooting
### 定点排错
你可以说:
```text
Debug Chat 点了没回复,帮我查是前端问题还是后端问题。
```
agent 应该:
- 先通过 UI 复现
- 看 console/network
- 看后端日志
- 必要时用 API/curl 做诊断
- 匹配 troubleshooting
- 给出修复建议或直接修复
## 重要原则
这些原则固定在:
```text
docs/qa-agent/03-agent-browser-qa-principles.md
```
简化版:
- UI/browser 是测试主路径。
- API/curl/log 只做诊断。
- 后端接口成功不等于 UI case 通过。
- case 通过必须以用户可见 UI 结果为准。
- 有视觉能力时应检查截图。
- 没有视觉能力时用 DOM/accessibility snapshot 和 console。
- 不要打印 token、API key、OAuth secret 或 localStorage token 值。
## 规划文档
如果要判断下一步建设什么,先看:
```text
docs/qa-agent/README.md
docs/qa-agent/04-black-box-e2e-roadmap.md
```
`01-qa-agent-harness-plan.md` 是早期规划,部分内容已经被当前实现和路线图替代。
## 常用命令
### 环境
```bash
bin/lbs env show
bin/lbs env show --json
bin/lbs env doctor
bin/lbs fixture list
bin/lbs fixture check
bin/lbs fixture check --json
```
`env show``env doctor` 默认会对 token、API key、password、secret 以及 URL basic auth
做脱敏。不要把 `.env.local` 里的原始凭据复制进测试报告。
### Skill 和索引
```bash
bin/lbs list
bin/lbs validate
bin/lbs index --check
bin/lbs index
```
### Case
```bash
bin/lbs case list
bin/lbs case list --type smoke
bin/lbs case list --json --priority p1 --tag local-agent
bin/lbs case list --ready
bin/lbs case list --machine-ready
bin/lbs case show pipeline-debug-chat
bin/lbs case new my-feature --title "My Feature Works"
```
### Suite
```bash
bin/lbs suite list
bin/lbs suite list --json --priority p1
bin/lbs suite show local-agent-gate
bin/lbs suite plan core-smoke
bin/lbs suite plan local-agent-gate --json
bin/lbs suite start core-smoke
bin/lbs suite start core-smoke --run-id core-smoke-local --evidence-dir reports/evidence/core-smoke-local
bin/lbs suite run core-smoke --dry-run --json
bin/lbs suite run core-smoke --run-id core-smoke-local --evidence-dir reports/evidence/core-smoke-local
bin/lbs suite start core-smoke --json
bin/lbs suite report core-smoke --evidence-dir reports/evidence/<suite-run-id>
bin/lbs suite report core-smoke --evidence-dir reports/evidence/<suite-run-id> --json
bin/lbs suite new my-feature-gate --title "My Feature Gate"
```
`suite start` 不直接控制浏览器。它生成统一 run id、suite evidence root、每个 case 的 evidence
目录、`suite-start.json`/`suite-start.md` handoff 文件,以及每个 case 的 `test run``test report`
`test result` 命令模板。需要固定路径时,使用 `--run-id``--evidence-dir`
`suite run --dry-run --json` 只预览 planned/skipped case,不创建 evidence,也不执行 automation。
`suite run` 会顺序执行 suite 中已有 automation、机器 readiness 已满足且不需要 `manual_check` 的 case,并在最后聚合 `suite report`
缺 env、automation env 或 fixture 的 case 默认会跳过;确实要强制执行时,加 `--include-not-ready`
确认前置条件后,才用 `--include-manual-check` 执行这类 case。
所有 case 执行完并写入最终 `result.json` 后,
`suite report` 会读取各 case evidence 目录并汇总为 `pass``fail``blocked``env_issue`
`flaky``incomplete` 等状态。`pass` 必须声明已经收集 case 的全部 required evidence
否则 suite 会保持 `incomplete`,避免把缺证据的运行误判成通过。
### Test Plan
```bash
bin/lbs test plan pipeline-debug-chat
bin/lbs test plan pipeline-debug-chat --json
```
### Test Start
```bash
bin/lbs test start pipeline-debug-chat
bin/lbs test start pipeline-debug-chat --json
```
`test start` 用于 agent 开始一次浏览器测试前记录 run id、开始时间和推荐 report 命令。
它会把 `--since "<started_at_local>"` 写进后续报告命令,减少历史日志污染本次判断。
如果 case 绑定了自动化脚本,输出里也会包含 `test run` 命令和 evidence 目录。
### Test Automation
```bash
bin/lbs test run webui-login-state --dry-run
bin/lbs test run pipeline-debug-chat --dry-run
bin/lbs test run webui-login-state --run-id login-smoke --output reports/evidence/login-smoke
bin/lbs test run pipeline-debug-chat --run-id pipeline-smoke --output reports/evidence/pipeline-smoke
```
查看当前所有带自动化脚本的 case
```bash
bin/lbs case list --automation
bin/lbs case list --json --automation
```
当前自动化覆盖包括登录态、通用 Pipeline Debug Chat、local-agent runner 的基础回复、
PromptPreProcessing、RAG、plugin tool、MCP stdio tool、非流式、多模态和 RAG+多模态路径。
不要在文档里手工维护静态 case 清单;以 `case list --automation` 和 suite 定义为准。
自动化脚本位于 `scripts/e2e/`。它们会保存:
- `console.log`
- `network.log`
- `screenshot.png`
- `automation-result.json`
新增 Debug Chat 类自动化时,优先复用 `scripts/e2e/lib/debug-chat.mjs` 中的 pipeline 打开、
prompt 发送、visible response leaf 判断和失败信号分类,不要在新脚本里复制 DOM 扫描逻辑。
脚本需要本地安装 Playwright 后才能真正执行:
```bash
npm install
npx playwright install chromium
```
`pipeline-debug-chat` 通用自动化建议配置 `LANGBOT_PIPELINE_URL`。如果没有 direct URL
脚本会尝试通过 `LANGBOT_PIPELINE_NAME` 从 Pipelines 页面进入目标 pipeline。两者都没有时,
该自动化会返回 `blocked`,不会伪造通过。
Runner 专用 case 不应复用通用 pipeline 变量。Local Agent、Codex AgentRunner 和
Claude Code AgentRunner 这类 case 会通过 `automation_pipeline_url_env` /
`automation_pipeline_name_env` 映射到 case-specific env,例如
`LANGBOT_LOCAL_AGENT_PIPELINE_URL`。这些 case 如果缺少专用变量,会返回 `blocked`
不会退回到 `LANGBOT_PIPELINE_URL`,避免跑错 pipeline 后产生假阳性。
如果 case 声明了 `setup_automation`,只有 `bin/lbs test run <case-id>` 的真实执行路径会先运行这些 setup;
`test plan``suite plan``case list``--dry-run` 只展示它们,不会修改本地环境。
setup 可以是 `case:<case-id>` 或仓库内 `node:scripts/... --flag`,每一步证据会写到主 evidence 目录下的
`setup/` 子目录。setup 失败时主 automation 不会继续执行;setup 写入 `.env.local` 后,主 automation
会重新读取环境。用 `setup_provides_env` 声明 setup 会生成的变量,可以让 readiness 正确显示机器可准备状态。
如果 Debug Chat case 需要固定流式/非流式路径,可以在 case 中设置
`automation_stream_output: "1"``"0"`,脚本会在发送消息前切换 Debug Chat 的 Stream 控件。
如果 case 需要上传图片,可以设置 `automation_image_base64_fixture` 指向仓库内的 base64 PNG fixture
脚本会在 evidence 目录写出临时 PNG 并通过 Debug Chat 上传控件发送。
`bin/lbs test plan <case-id> --json``bin/lbs suite plan <suite-id> --json`
都会显示这些专用变量是否已配置。
### Test Report 和日志守卫
```bash
bin/lbs test report pipeline-debug-chat
bin/lbs test report pipeline-debug-chat --output reports/pipeline-debug-chat.md
bin/lbs test report pipeline-debug-chat \
--backend-log /path/to/backend.log \
--frontend-log /path/to/frontend.log \
--console-log /path/to/console.log
bin/lbs test report pipeline-debug-chat --evidence-dir reports/evidence/pipeline-smoke
bin/lbs test report pipeline-debug-chat --backend-log /path/to/backend.log --json
bin/lbs test report pipeline-debug-chat --since "2026-05-21T10:30:00+08:00"
bin/lbs test report pipeline-debug-chat --tail-lines 2000
bin/lbs test report pipeline-debug-chat --since "2026-05-21T10:30:00+08:00" --tail-lines 2000
```
`test report` 会生成报告模板,并默认从 `LANGBOT_REPO/data/logs/` 自动选择最新的
`langbot-*.log` 作为 LangBot 后端日志扫描。也可以用 `--backend-log` 覆盖,或用
`--no-auto-log` 只生成模板。
如果提供 `--evidence-dir`,或 `--console-log` 指向 `reports/evidence/<run-id>/console.log`
报告会优先读取同目录的 `automation-result.json`,并展示自动化脚本的 `status``reason`
起止时间和目标 URL。
日志守卫会扫描常见错误、secret 泄露风险、case 声明的 success/failure patterns,以及已知
troubleshooting pattern。它不控制浏览器,也不替代 UI 通过判断。`success_patterns`
命中会作为通过证据写入 `success_signals`;声明了 success pattern 但本次扫描窗口没有命中,
会给 warning`failure_patterns` 命中会让本次日志守卫 fail。
建议在执行浏览器 case 前记录当前时间,然后在报告阶段使用 `--since`。如果只想快速看
最近日志,可以使用 `--tail-lines`
### Runtime Log Guard
如果还没有进入某个具体 UI case,只是想观察 LangBot 后端日志,可以直接使用 `log`
命令。它和 `test report` 使用同一套扫描器、secret 脱敏、troubleshooting pattern 和
case success/failure pattern。
```bash
bin/lbs log scan --tail-lines 300
bin/lbs log scan --case pipeline-debug-chat --since "2026-05-21T10:30:00+08:00"
bin/lbs log scan --backend-log /path/to/langbot.log --json
bin/lbs log scan --failure-pattern "runner.tool_loop_error|Action invoke_llm_stream call timed out" --strict
```
`log scan` 默认从 `LANGBOT_REPO/data/logs/` 自动选择最新的 `langbot-*.log`。传入
`--case <case-id>` 后,会额外应用该 case 声明的 `success_patterns``failure_patterns`
和 related troubleshooting。默认用于观察,返回码保持 0;加 `--strict` 后,`fail`
`env_issue` 会返回非 0,适合脚本门禁。
运行期观察可以用 `watch`
```bash
bin/lbs log watch --case pipeline-debug-chat
bin/lbs log watch --backend-log /path/to/langbot.log --interval-ms 500
bin/lbs log watch --duration-ms 30000 --strict --json
```
`log watch` 默认从启动时的文件末尾开始,只观察新追加的日志;加 `--from-start` 可从文件开头扫。
它会实时打印新命中的 findings 和 success signals。为了避免当前历史日志噪声影响观察,默认不因
异常返回非 0;加 `--strict` 后,退出时如果看到 `fail``env_issue` 会返回非 0。
给一次 QA 运行包日志窗口时,用 `guard start/stop`
```bash
bin/lbs log guard start --run-id local-debug --case pipeline-debug-chat
# 执行浏览器或手工测试
bin/lbs log guard stop --run-id local-debug
```
`start` 会在 `reports/log-guards/<run-id>.json` 记录起始时间、case 和当前后端日志路径;
`stop` 会用 start/stop 时间作为扫描窗口,生成 `reports/log-guards/<run-id>.md`,并默认按
strict guard 返回码处理。临时只想收集报告、不想让命令失败,可以加 `--no-strict`
当前 LangBot core 日志还不是完全结构化日志,runtime guard 主要依赖时间窗口和文本 pattern。
已支持 ISO 时间戳和 LangBot 当前的 `[MM-DD HH:mm:ss.SSS]` 前缀;没有时间戳的连续行会跟随上一条
带时间戳的日志块。如果后续 core 能提供稳定 request id、conversation id、plugin action id 或
JSON log fieldguard 可以从“时间窗口 + 文本匹配”升级为更精确的关联分析。
### Test Result
```bash
bin/lbs test result pipeline-debug-chat \
--result pass \
--reason "Debug Chat returned OK and the report log guard was clean." \
--evidence-dir reports/evidence/pipeline-smoke \
--started-at "2026-05-21T10:30:00+08:00" \
--evidence ui,screenshot,console,backend_log
```
`test result` 用于把一次人工/agent browser 运行的最终判断写成标准 `result.json`
`suite report` 聚合。它不会替代 UI 测试:如果写 `--result pass``--evidence`
必须覆盖该 case 的 `evidence_required`,否则命令会失败。自动化脚本写
`automation-result.json`;如果 case 还要求 backend log、API diagnostic 或 filesystem
evidence,agent 需要在报告和诊断完成后再用 `test result` 写最终 `result.json`
### Troubleshooting
```bash
bin/lbs trouble list langbot-testing
bin/lbs trouble show plugin-runtime-timeout
bin/lbs trouble search runtime
bin/lbs trouble add langbot-testing --title "..." --symptom "..." --cause "..." --fix "..."
```
## 目录说明
```text
skills/
.env # 共享默认变量
langbot-env-setup/ # 环境、浏览器、登录态、代理
langbot-testing/ # WebUI / provider / pipeline 测试
schemas/ # 结构化资产 schema
src/ # lbs TypeScript 源码
bin/ # lbs 入口
docs/ # 设计文档和用户手册
AGENTS.md # agent 维护协议
```
## 添加一个新测试路径
1. 先让 agent 通过浏览器探索并执行路径。
2. 稳定后创建 case
```bash
bin/lbs case new provider-xxx --title "Provider XXX can be configured" --area provider --type provider
```
3. 编辑生成的 `cases/*.yaml`,补充真实步骤、检查点和 troubleshooting。
4. 校验:
```bash
bin/lbs validate
bin/lbs index --check
bin/lbs index
```
## 添加一个新故障
```bash
bin/lbs trouble add langbot-testing \
--title "Plugin runtime actions time out" \
--symptom "Debug Chat shows Agent runner temporarily unavailable" \
--cause "Old plugin runtime survived backend restart" \
--fix "Stop old runtime processes and restart LangBot"
```
然后编辑生成的 YAML,补充 `patterns``related_cases` 和验证方式。
## 当前边界
- `lbs test plan` 只生成测试计划,不直接控制浏览器。
- `lbs test report` 生成报告,默认扫描最新 LangBot 后端日志;也可扫描显式提供的
backend/frontend/console 日志文件。
- 真正的 UI 操作由当前 agent 的浏览器能力执行。
- `env doctor` 是 readiness check,不是产品测试。
- `curl/API` 是诊断工具,不是主要测试路径。
+59
View File
@@ -0,0 +1,59 @@
# Schemas
这个目录存放 LangBot skills 结构化资产的 JSON Schema。
它们不是测试脚本,也不会执行浏览器动作。它们的作用是定义 agent 和维护者后续新增资产时应该遵守的文件结构。
## 文件说明
- `skills/<skill>/fixtures/fixtures.json`
不是 JSON Schema,但由 `bin/lbs validate` 校验。
它登记 deterministic fixture 文件、类型和关联 case,供 `bin/lbs fixture check` 做 readiness 检查。
- `case.schema.json`
约束 `skills/<skill>/cases/*.yaml` 的格式。
Case 描述 agent-browser 或 probe QA 路径,包括前置条件、步骤、检查点、诊断手段和关联故障。
- `suite.schema.json`
约束 `skills/<skill>/suites/*.yaml` 的格式。
Suite 只组织 case 集合,用于 smoke、regression 或 release gate 等测试入口。
- `troubleshooting.schema.json`
约束 `skills/<skill>/troubleshooting/*.yaml` 的格式。
Troubleshooting 条目描述症状、日志/错误模式、可能原因、修复步骤和验证信号。
- `skill-index.schema.json`
约束生成文件 `skills.index.json` 的格式。
这个索引用于让 agent 快速发现已有 skills、references、cases、suites 和 troubleshooting。
- `reports/evidence/<run-id>/result.json`
不是 catalog schema,而是执行期最终裁定产物,由 `bin/lbs test result` 写入。
`suite report` 读取其中的 `status``reason`、起止时间和 `evidence_collected`
并用 `evidence_missing` 防止缺证据的 `pass` 被当作完整通过。
- `reports/evidence/<run-id>/automation-result.json`
不是 catalog schema,而是浏览器自动化脚本的原始运行结论,供 `bin/lbs test report`
展示和推断日志扫描窗口。
## 为什么需要 schemas
Schemas 是基础设施护栏:
- 防止 case、suite 和 troubleshooting 随着增长变得格式混乱
- 让 `bin/lbs validate` 能发现缺字段和错误结构
- 为未来编辑器提示和 CI 校验留接口
- 帮助 agent 新增资产时知道应该写哪些字段
## 当前校验方式
`bin/lbs validate` 做轻量、schema 对齐的校验,不引入额外依赖。它会检查必填字段、
枚举值、boolean 字段、重复列表项、automation 脚本存在性,以及 case、suite、skill、
troubleshooting 之间的交叉引用。这里的 schema 仍是格式契约;如果未来引入正式 JSON
Schema validator,应继续保持这些本地交叉引用检查。
Case 里的 `env` / `automation_env` 表示所有列出的变量都需要配置。遇到二选一输入时,
使用 `env_any` / `automation_env_any`,每一项写成 `LANGBOT_PIPELINE_URL|LANGBOT_PIPELINE_NAME`
这类 one-of 组合,避免 agent 因为只配置了 URL 或 name 其中之一而误判未就绪。
`setup``preconditions` 是人工确认项,会让 readiness 进入 `manual_check`
`setup_automation``test run` 可以自动执行的准备步骤,配合 `setup_provides_env`
声明它会生成的机器变量。
+326
View File
@@ -0,0 +1,326 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://langbot.app/schemas/langbot-skills/case.schema.json",
"title": "LangBot Skill Test Case",
"type": "object",
"required": [
"id",
"title",
"mode",
"area",
"type",
"priority",
"risk",
"ci_eligible",
"tags",
"skills",
"steps",
"checks",
"evidence_required"
],
"allOf": [
{
"if": {
"properties": {
"mode": { "const": "agent-browser" }
}
},
"then": {
"required": ["env"]
}
}
],
"additionalProperties": true,
"properties": {
"id": {
"type": "string",
"pattern": "^[a-z0-9][a-z0-9_-]*$"
},
"title": {
"type": "string"
},
"mode": {
"type": "string",
"enum": ["agent-browser", "probe"]
},
"area": {
"type": "string"
},
"type": {
"type": "string",
"enum": [
"smoke",
"regression",
"feature",
"provider",
"exploratory",
"contract",
"performance",
"reliability",
"chaos",
"security"
]
},
"priority": {
"type": "string",
"enum": ["p0", "p1", "p2"]
},
"risk": {
"type": "string",
"enum": ["low", "medium", "high"]
},
"ci_eligible": {
"type": "boolean"
},
"tags": {
"type": "array",
"items": { "type": "string" }
},
"skills": {
"type": "array",
"items": { "type": "string" }
},
"env": {
"type": "array",
"items": { "type": "string" }
},
"env_any": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*(\\|[A-Z][A-Z0-9_]*)+$"
}
},
"steps": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"checks": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"evidence_required": {
"type": "array",
"items": {
"type": "string",
"enum": [
"ui",
"screenshot",
"console",
"network",
"backend_log",
"frontend_log",
"api_diagnostic",
"filesystem",
"metrics",
"trace",
"profile",
"resource_log"
]
},
"minItems": 1
},
"preconditions": {
"type": "array",
"items": { "type": "string" }
},
"setup": {
"type": "array",
"items": { "type": "string" }
},
"setup_automation": {
"type": "array",
"items": {
"type": "string",
"pattern": "^(?:case:[a-z0-9][a-z0-9_-]*|node:scripts/[A-Za-z0-9_./-]+\\.(?:mjs|js|ts)(?:\\s+--[A-Za-z0-9][A-Za-z0-9_-]*(?:=[A-Za-z0-9_./:@-]+)?)*)$"
}
},
"setup_provides_env": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
}
},
"cleanup": {
"type": "array",
"items": { "type": "string" }
},
"diagnostics": {
"type": "array",
"items": { "type": "string" }
},
"automation": {
"type": "string"
},
"automation_env": {
"type": "array",
"items": { "type": "string" }
},
"automation_env_any": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*(\\|[A-Z][A-Z0-9_]*)+$"
}
},
"automation_prompt": {
"type": "string"
},
"automation_prompts_json": {
"type": "string"
},
"automation_expected_text": {
"type": "string"
},
"automation_response_timeout_ms": {
"type": "string"
},
"automation_stream_output": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_image_base64_fixture": {
"type": "string"
},
"automation_runner_config_patch_json": {
"type": "string"
},
"automation_restore_runner_config": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_expected_runner_id": {
"type": "string"
},
"automation_reset_debug_chat": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_debug_chat_session_type": {
"type": "string",
"enum": ["person", "group"]
},
"automation_debug_chat_response_p95_ms": {
"type": "string"
},
"automation_debug_chat_max_error_rate": {
"type": "string"
},
"automation_debug_chat_load_requests": {
"type": "string"
},
"automation_debug_chat_load_concurrency": {
"type": "string"
},
"automation_debug_chat_load_timeout_ms": {
"type": "string"
},
"automation_debug_chat_load_response_p95_ms": {
"type": "string"
},
"automation_debug_chat_load_first_response_p95_ms": {
"type": "string"
},
"automation_debug_chat_load_max_error_rate": {
"type": "string"
},
"automation_debug_chat_load_min_error_rate": {
"type": "string"
},
"automation_debug_chat_load_min_error_count": {
"type": "string"
},
"automation_debug_chat_load_min_ok_count": {
"type": "string"
},
"automation_debug_chat_load_min_provider_fault_count": {
"type": "string"
},
"automation_debug_chat_load_expected_prefix": {
"type": "string"
},
"automation_debug_chat_load_prompt_template": {
"type": "string"
},
"automation_debug_chat_load_stream": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_debug_chat_load_reset": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_debug_chat_load_fail_on_final_mismatch": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_fake_provider_response_text": {
"type": "string"
},
"automation_fake_provider_first_token_delay_ms": {
"type": "string"
},
"automation_fake_provider_chunk_delay_ms": {
"type": "string"
},
"automation_fake_provider_chunk_count": {
"type": "string"
},
"automation_fake_provider_fail_first_n": {
"type": "string"
},
"automation_fake_provider_fail_every_n": {
"type": "string"
},
"automation_fake_provider_fault_status": {
"type": "string"
},
"automation_fake_provider_fail_after_first_chunk": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_fake_provider_dynamic_response": {
"type": "string",
"enum": ["0", "1", "false", "true"]
},
"automation_filesystem_checks_json": {
"type": "string"
},
"metrics_thresholds_json": {
"type": "string"
},
"load_profile_json": {
"type": "string"
},
"fault_model_json": {
"type": "string"
},
"automation_pipeline_url_env": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"automation_pipeline_name_env": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"success_patterns": {
"type": "array",
"items": { "type": "string" }
},
"failure_patterns": {
"type": "array",
"items": { "type": "string" }
},
"expected_failures": {
"type": "array",
"items": { "type": "string" }
},
"troubleshooting": {
"type": "array",
"items": { "type": "string" }
}
}
}
+147
View File
@@ -0,0 +1,147 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://langbot.app/schemas/langbot-skills/skill-index.schema.json",
"title": "LangBot Skills Index",
"type": "object",
"required": ["generated_by", "skills"],
"additionalProperties": false,
"properties": {
"generated_by": {
"type": "string"
},
"skills": {
"type": "array",
"items": {
"type": "object",
"required": [
"directory",
"name",
"description",
"references",
"cases",
"case_summaries",
"suites",
"suite_summaries",
"fixtures",
"troubleshooting",
"troubleshooting_summaries"
],
"additionalProperties": false,
"properties": {
"directory": { "type": "string" },
"name": { "type": "string" },
"description": { "type": "string" },
"references": {
"type": "array",
"items": { "type": "string" }
},
"cases": {
"type": "array",
"items": { "type": "string" }
},
"case_summaries": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "title", "mode", "area", "type", "priority", "risk", "ci_eligible", "tags", "automation", "setup_automation", "setup_provides_env", "evidence_required"],
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"title": { "type": "string" },
"mode": { "type": "string", "enum": ["agent-browser", "probe"] },
"area": { "type": "string" },
"type": { "type": "string" },
"priority": { "type": "string" },
"risk": { "type": "string" },
"ci_eligible": { "type": "boolean" },
"tags": {
"type": "array",
"items": { "type": "string" }
},
"automation": { "type": "string" },
"setup_automation": {
"type": "array",
"items": { "type": "string" }
},
"setup_provides_env": {
"type": "array",
"items": { "type": "string" }
},
"evidence_required": {
"type": "array",
"items": { "type": "string" }
}
}
}
},
"suites": {
"type": "array",
"items": { "type": "string" }
},
"suite_summaries": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "title", "description", "type", "priority", "tags", "cases"],
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"title": { "type": "string" },
"description": { "type": "string" },
"type": { "type": "string" },
"priority": { "type": "string" },
"tags": {
"type": "array",
"items": { "type": "string" }
},
"cases": {
"type": "array",
"items": { "type": "string" }
}
}
}
},
"fixtures": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "title", "kind", "path", "related_cases"],
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"title": { "type": "string" },
"kind": { "type": "string" },
"path": { "type": "string" },
"related_cases": {
"type": "array",
"items": { "type": "string" }
}
}
}
},
"troubleshooting": {
"type": "array",
"items": { "type": "string" }
},
"troubleshooting_summaries": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "title", "category", "related_cases"],
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"title": { "type": "string" },
"category": { "type": "string" },
"related_cases": {
"type": "array",
"items": { "type": "string" }
}
}
}
}
}
}
}
}
}
+48
View File
@@ -0,0 +1,48 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://langbot.app/schemas/langbot-skills/suite.schema.json",
"title": "LangBot Skill Test Suite",
"type": "object",
"required": ["id", "title", "description", "type", "priority", "tags", "cases"],
"additionalProperties": true,
"properties": {
"id": {
"type": "string",
"pattern": "^[a-z0-9][a-z0-9_-]*$"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
},
"type": {
"type": "string",
"enum": [
"smoke",
"regression",
"release_gate",
"exploratory",
"contract",
"performance",
"reliability",
"chaos",
"security"
]
},
"priority": {
"type": "string",
"enum": ["p0", "p1", "p2"]
},
"tags": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"cases": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
}
}
}
@@ -0,0 +1,51 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://langbot.app/schemas/langbot-skills/troubleshooting.schema.json",
"title": "LangBot Skill Troubleshooting Entry",
"type": "object",
"required": ["id", "title", "symptoms", "patterns", "likely_causes", "fix_steps", "verification"],
"additionalProperties": true,
"properties": {
"id": {
"type": "string",
"pattern": "^[a-z0-9][a-z0-9_-]*$"
},
"title": {
"type": "string"
},
"date": {
"type": "string"
},
"category": {
"type": "string",
"enum": ["product", "env_issue", "external_dependency", "blocked", "flaky"]
},
"symptoms": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"patterns": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"likely_causes": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"fix_steps": {
"type": "array",
"items": { "type": "string" },
"minItems": 1
},
"verification": {
"type": "string"
},
"related_cases": {
"type": "array",
"items": { "type": "string" }
}
}
}
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env node
import { chmod, mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const binDir = resolve(root, "bin");
const lbsPath = resolve(binDir, "lbs");
const wrapper = [
"#!/usr/bin/env bash",
"set -euo pipefail",
"",
'SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"',
'exec node "$SCRIPT_DIR/../src/lbs.ts" "$@"',
"",
].join("\n");
await mkdir(binDir, { recursive: true });
let current = "";
try {
current = await readFile(lbsPath, "utf8");
} catch {
// Missing wrapper is the normal first-run path.
}
if (current !== wrapper) {
await writeFile(lbsPath, wrapper, "utf8");
await chmod(lbsPath, 0o755);
}
+476
View File
@@ -0,0 +1,476 @@
#!/usr/bin/env node
import { existsSync, readFileSync } from "node:fs";
import { writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
localIsoWithOffset,
safeScreenshot,
writeResult,
} from "./lib/langbot-e2e.mjs";
function loadEnvDefaults(path) {
if (!existsSync(path)) return;
for (const rawLine of readFileSync(path, "utf8").split(/\r?\n/)) {
const line = rawLine.trim();
if (!line || line.startsWith("#")) continue;
const sep = line.indexOf("=");
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
}
}
function boolFromEnv(value, defaultValue) {
if (value === undefined || value === "") return defaultValue;
if (/^(0|false|no|off)$/i.test(value)) return false;
if (/^(1|true|yes|on)$/i.test(value)) return true;
return defaultValue;
}
function firstEnv(...keys) {
for (const key of keys) {
if (env[key]) return env[key];
}
return "";
}
function redactMessage(text) {
return String(text ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
}
function isEnvironmentError(message) {
return /Playwright is not installed|LANGBOT_FRONTEND_URL|LANGBOT_BACKEND_URL|ERR_CONNECTION_REFUSED|ECONNREFUSED|net::ERR_|fetch failed|timed out/i
.test(message);
}
loadEnvDefaults("skills/.env");
loadEnvDefaults("skills/.env.local");
const caseId = env.LBS_CASE_ID || "agent-runner-release-preflight";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const frontendUrl = env.LANGBOT_FRONTEND_URL || backendUrl;
const testModels = boolFromEnv(env.LANGBOT_PREFLIGHT_TEST_MODELS, true);
const requireVision = boolFromEnv(env.LANGBOT_PREFLIGHT_REQUIRE_VISION, true);
const diagnosticPath = resolve(paths.evidenceDir, "api-diagnostic.json");
const startedAt = new Date();
const targets = [
{
id: "local-agent",
expected_runner_id: "plugin:langbot/local-agent/default",
pipeline_url: firstEnv("LANGBOT_LOCAL_AGENT_PIPELINE_URL"),
pipeline_name: firstEnv("LANGBOT_LOCAL_AGENT_PIPELINE_NAME"),
require_func_call_model: true,
require_vision_model: requireVision,
require_langbot_mcp: false,
},
{
id: "acp-agent-runner",
expected_runner_id: "plugin:langbot/acp-agent-runner/default",
pipeline_url: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL", "LANGBOT_AGENT_RUNNER_PIPELINE_URL"),
pipeline_name: firstEnv("LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME", "LANGBOT_AGENT_RUNNER_PIPELINE_NAME"),
require_func_call_model: false,
require_vision_model: false,
},
];
let browser;
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
frontend_url: frontendUrl,
backend_url: backendUrl,
test_models: testModels,
require_vision_model: requireVision,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
api_diagnostic_json: diagnosticPath,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "network", "api_diagnostic"],
};
async function run() {
if (!backendUrl || !frontendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL must be configured.";
return;
}
browser = await createBrowser(paths);
const { page } = browser;
await page.goto(frontendUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const diagnostic = await page.evaluate(async ({ backendUrl, targets, testModels }) => {
const blockers = [];
const envIssues = [];
const warnings = [];
const checks = [];
const addCheck = (name, status, detail = {}) => {
checks.push({ name, status, ...detail });
if (status === "blocked") blockers.push({ name, ...detail });
if (status === "env_issue") envIssues.push({ name, ...detail });
};
const safeMessage = (value) => String(value ?? "")
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]")
.replace(/(api[_-]?key|authorization|credential|jwt|oauth|password|secret|token)\s*[:=]\s*["']?[^"',\s]+/gi, "$1=[redacted]");
const token = localStorage.getItem("token");
if (!token) {
addCheck("browser-auth", "blocked", { reason: "Browser profile has no localStorage token." });
return { authenticated: false, blockers, env_issues: envIssues, warnings, checks };
}
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const postJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "POST",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const tokenCheck = await getJson("/api/v1/user/check-token");
addCheck(
"browser-auth",
tokenCheck.status < 400 && (tokenCheck.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: tokenCheck.status, code: tokenCheck.json.code ?? null, reason: safeMessage(tokenCheck.json.msg || "") },
);
const systemInfo = await getJson("/api/v1/system/info");
addCheck(
"backend-system-info",
systemInfo.status < 400 ? "pass" : "env_issue",
{
http_status: systemInfo.status,
version: systemInfo.json.data?.version || systemInfo.json.data?.system?.version || "",
},
);
const pluginSystem = await getJson("/api/v1/system/status/plugin-system");
addCheck(
"plugin-system",
pluginSystem.status < 400 && (pluginSystem.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: pluginSystem.status,
code: pluginSystem.json.code ?? null,
status: pluginSystem.json.data?.status || pluginSystem.json.data?.state || "",
reason: safeMessage(pluginSystem.json.msg || ""),
},
);
const boxStatus = await getJson("/api/v1/box/status");
addCheck(
"box-runtime",
boxStatus.status < 400 && (boxStatus.json.code ?? 0) === 0 ? "pass" : "env_issue",
{
http_status: boxStatus.status,
code: boxStatus.json.code ?? null,
status: boxStatus.json.data?.status || "",
backend: boxStatus.json.data?.backend || "",
reason: safeMessage(boxStatus.json.msg || ""),
},
);
const plugins = await getJson("/api/v1/plugins");
const installedPluginIds = (plugins.json.data?.plugins || [])
.map((plugin) => {
const metadata = plugin.manifest?.manifest?.metadata || plugin.manifest?.metadata || plugin.metadata || {};
return metadata.author && metadata.name ? `${metadata.author}/${metadata.name}` : "";
})
.filter(Boolean);
const requiredPlugins = ["langbot/local-agent", "langbot/acp-agent-runner", "qa/plugin-smoke"];
const pluginPresence = Object.fromEntries(requiredPlugins.map((id) => [id, installedPluginIds.includes(id)]));
for (const [id, present] of Object.entries(pluginPresence)) {
addCheck(`plugin:${id}`, present ? "pass" : "blocked", { plugin_id: id, reason: present ? "" : "Required plugin is not listed by /api/v1/plugins." });
}
const tools = await getJson("/api/v1/tools");
const toolNames = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
addCheck(
"tool:qa_plugin_echo",
toolNames.includes("qa_plugin_echo") ? "pass" : "blocked",
{ reason: toolNames.includes("qa_plugin_echo") ? "" : "qa-plugin-smoke tool qa_plugin_echo is not exposed through /api/v1/tools." },
);
if (!toolNames.includes("qa_mcp_echo")) {
warnings.push({
name: "tool:qa_mcp_echo",
reason: "qa_mcp_echo is not currently exposed. This is acceptable before mcp-stdio-register, but mcp-stdio-tool-call must run after registration.",
});
}
const modelResponse = await getJson("/api/v1/provider/models/llm");
const models = (modelResponse.json.data?.models || []).map((model) => ({
uuid: model.uuid,
name: model.name,
abilities: Array.isArray(model.abilities) ? model.abilities : [],
provider_uuid: model.provider_uuid || model.provider?.uuid || "",
provider_name: model.provider_name || model.provider?.name || "",
requester: model.requester || model.provider?.requester || "",
}));
addCheck(
"llm-model-list",
modelResponse.status < 400 && (modelResponse.json.code ?? 0) === 0 ? "pass" : "env_issue",
{ http_status: modelResponse.status, model_count: models.length, reason: safeMessage(modelResponse.json.msg || "") },
);
const modelById = new Map(models.map((model) => [model.uuid, model]));
const pipelineList = await getJson("/api/v1/pipelines");
const pipelines = pipelineList.json.data?.pipelines || [];
addCheck(
"pipeline-list",
pipelineList.status < 400 && (pipelineList.json.code ?? 0) === 0 ? "pass" : "blocked",
{ http_status: pipelineList.status, pipeline_count: pipelines.length, reason: safeMessage(pipelineList.json.msg || "") },
);
const resolvedPipelines = [];
const modelTested = new Set();
for (const target of targets) {
let pipelineId = "";
let matchedBy = "";
if (target.pipeline_url) {
try {
pipelineId = new URL(target.pipeline_url).searchParams.get("id") || "";
matchedBy = pipelineId ? "url" : "";
} catch {
pipelineId = "";
}
}
if (!pipelineId && target.pipeline_name) {
const match = pipelines.find((pipeline) => pipeline.name === target.pipeline_name);
if (match) {
pipelineId = match.uuid;
matchedBy = "name";
}
}
if (!pipelineId) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
reason: "Required pipeline env is missing or could not resolve to a pipeline id.",
});
continue;
}
const response = await getJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`);
const pipeline = response.json.data?.pipeline;
if (response.status >= 400 || !pipeline) {
addCheck(`pipeline:${target.id}`, "blocked", {
target: target.id,
pipeline_id: pipelineId,
http_status: response.status,
reason: safeMessage(response.json.msg || "Could not load pipeline."),
});
continue;
}
const config = pipeline.config || {};
const aiConfig = config.ai && typeof config.ai === "object" ? config.ai : {};
const runner = aiConfig.runner && typeof aiConfig.runner === "object" ? aiConfig.runner : {};
const runnerId = runner.id || runner.runner || "";
const runnerConfigs = aiConfig.runner_config && typeof aiConfig.runner_config === "object" ? aiConfig.runner_config : {};
const runnerConfig = runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object" ? runnerConfigs[runnerId] : {};
const pipelineSummary = {
target: target.id,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: target.expected_runner_id,
runner_config_keys: Object.keys(runnerConfig).sort(),
};
resolvedPipelines.push(pipelineSummary);
addCheck(
`pipeline:${target.id}:runner`,
runnerId === target.expected_runner_id ? "pass" : "blocked",
{
...pipelineSummary,
reason: runnerId === target.expected_runner_id ? "" : `Expected ${target.expected_runner_id}, got ${runnerId || "<missing>"}.`,
},
);
if (target.require_func_call_model || target.require_vision_model || (testModels && target.id === "local-agent")) {
const modelConfig = runnerConfig.model;
const primaryModelId = typeof modelConfig === "string"
? modelConfig
: modelConfig && typeof modelConfig === "object"
? modelConfig.primary || ""
: "";
if (!primaryModelId) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
reason: "Local-agent runner config has no primary model.",
});
continue;
}
const model = modelById.get(primaryModelId);
if (!model) {
addCheck(`pipeline:${target.id}:primary-model`, "blocked", {
...pipelineSummary,
model_uuid: primaryModelId,
reason: "Primary model is not listed by /api/v1/provider/models/llm.",
});
continue;
}
addCheck(`pipeline:${target.id}:primary-model`, "pass", {
...pipelineSummary,
model: {
uuid: model.uuid,
name: model.name,
abilities: model.abilities,
provider_name: model.provider_name,
requester: model.requester,
},
});
if (target.require_func_call_model) {
addCheck(
`pipeline:${target.id}:func-call-model`,
model.abilities.includes("func_call") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("func_call") ? "" : "Release gate includes tool-call cases; the local-agent primary model must advertise func_call.",
},
);
}
if (target.require_vision_model) {
addCheck(
`pipeline:${target.id}:vision-model`,
model.abilities.includes("vision") ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
abilities: model.abilities,
reason: model.abilities.includes("vision") ? "" : "Release gate includes multimodal cases; the local-agent primary model must advertise vision.",
},
);
}
if (testModels && !modelTested.has(model.uuid)) {
modelTested.add(model.uuid);
const modelTest = await postJson(`/api/v1/provider/models/llm/${encodeURIComponent(model.uuid)}/test`, { extra_args: {} });
const passed = modelTest.status < 400 && (modelTest.json.code ?? 0) === 0;
addCheck(
`model-test:${model.name}`,
passed ? "pass" : "env_issue",
{
model_uuid: model.uuid,
model_name: model.name,
http_status: modelTest.status,
code: modelTest.json.code ?? null,
reason: passed ? "" : safeMessage(modelTest.json.msg || modelTest.json.message || "Model test failed."),
},
);
}
}
}
return {
authenticated: true,
blockers,
env_issues: envIssues,
warnings,
checks,
resolved_pipelines: resolvedPipelines,
tools: {
required: ["qa_plugin_echo"],
optional_before_register: ["qa_mcp_echo"],
present: toolNames.filter((name) => ["qa_plugin_echo", "qa_mcp_echo"].includes(name)),
},
models,
};
}, { backendUrl, targets, testModels });
diagnostic.blockers = (diagnostic.blockers || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
diagnostic.env_issues = (diagnostic.env_issues || []).map((item) => ({ ...item, reason: redactMessage(item.reason || "") }));
await writeFile(diagnosticPath, `${JSON.stringify(diagnostic, null, 2)}\n`, "utf8");
await safeScreenshot(page, paths.screenshot);
const blockers = diagnostic.blockers || [];
const envIssues = diagnostic.env_issues || [];
if (blockers.length > 0) {
result.status = "blocked";
result.reason = `Preflight blocked: ${blockers.map((item) => item.name).join(", ")}`;
} else if (envIssues.length > 0) {
result.status = "env_issue";
result.reason = `Preflight environment issue: ${envIssues.map((item) => item.name).join(", ")}`;
} else {
result.status = "pass";
result.reason = "Release gate preflight passed: auth, plugin runtime, required pipelines, runner ids, tools, and local-agent model checks are ready.";
}
result.check_count = Array.isArray(diagnostic.checks) ? diagnostic.checks.length : 0;
result.warning_count = Array.isArray(diagnostic.warnings) ? diagnostic.warnings.length : 0;
}
try {
await run();
} catch (error) {
const message = redactMessage(error instanceof Error ? error.message : String(error));
result.status = isEnvironmentError(message) ? "env_issue" : "fail";
result.reason = message;
await writeFile(diagnosticPath, `${JSON.stringify({
authenticated: false,
blockers: [],
env_issues: result.status === "env_issue" ? [{ name: "preflight-runtime", reason: message }] : [],
warnings: [],
checks: [
{
name: "preflight-runtime",
status: result.status,
reason: message,
},
],
}, null, 2)}\n`, "utf8").catch(() => {});
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
+263
View File
@@ -0,0 +1,263 @@
#!/usr/bin/env node
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
ensureEvidence,
evidencePaths,
loadEnvFiles,
resetAndAuthLocalUser,
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "plugin:langbot/acp-agent-runner/default";
const DEFAULT_PIPELINE_NAME = "Agent QA ACP Claude Debug Chat";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const caseId = "ensure-acp-agent-runner-pipeline";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const sshTarget = env.LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET || "yhh@101.34.71.12";
const sshConnectTimeout = env.LANGBOT_ACP_AGENT_RUNNER_SSH_CONNECT_TIMEOUT || "8";
const sshPort = env.LANGBOT_ACP_AGENT_RUNNER_SSH_PORT || "22";
const sshIdentityFile = env.LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE || "";
const sshExtraOptions = env.LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS || "";
const remoteWorkspace = env.LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE || "/home/yhh/langbot-e2e/acp-workspace";
const envLocalPath = resolve("skills/.env.local");
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
frontend_url: frontendUrl,
backend_url: backendUrl,
pipeline_name: pipelineName,
pipeline_id: "",
pipeline_url: "",
runner_id: RUNNER_ID,
ssh_target: sshTarget,
ssh_port: sshPort,
remote_workspace: remoteWorkspace,
wrote_env: false,
auth: null,
evidence: {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic"],
};
try {
if (!frontendUrl) throw new Error("LANGBOT_FRONTEND_URL is not configured.");
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
result.auth = {
source: "local_recovery_login",
user,
backend_token_check: auth.check,
};
const runnerConfig = {
provider: "claude-code",
location: "remote-ssh",
workspace: remoteWorkspace,
"ssh-target": sshTarget,
"ssh-port": Number.parseInt(sshPort, 10),
"ssh-identity-file": sshIdentityFile,
"ssh-connect-timeout": Number.parseInt(sshConnectTimeout, 10),
"ssh-extra-options": sshExtraOptions,
"langbot-assets-enabled": true,
"mcp-bridge-request-timeout": 90,
"reuse-session": false,
"create-session-if-missing": true,
"append-run-scope-prompt": true,
"startup-timeout": 30,
"initialize-timeout": 120,
timeout: 300,
};
const prepared = await ensurePipeline({
backendUrl,
token: auth.token,
pipelineName,
runnerId: RUNNER_ID,
runnerConfig,
});
Object.assign(result, prepared);
if (result.pipeline_id) {
result.pipeline_url = `${frontendUrl.replace(/\/$/, "")}/home/pipelines?id=${encodeURIComponent(result.pipeline_id)}`;
}
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET: sshTarget,
LANGBOT_ACP_AGENT_RUNNER_SSH_PORT: sshPort,
LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE: sshIdentityFile,
LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS: sshExtraOptions,
LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE: remoteWorkspace,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
} catch (error) {
result.reason = result.reason || error.message;
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
if (isApiFailure(pipelineList)) {
return {
status: "fail",
reason: pipelineList.json.msg || "Failed to list pipelines.",
list_status: pipelineList.status,
};
}
const pipelines = pipelineList.json.data?.pipelines || [];
let pipeline = pipelines.find((item) => item.name === pipelineName) || null;
let created = false;
if (!pipeline) {
const createdResponse = await apiJson(backendUrl, "/api/v1/pipelines", {
method: "POST",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
emoji: "QA",
},
});
if (isApiFailure(createdResponse)) {
return {
status: "fail",
reason: createdResponse.json.msg || "Failed to create pipeline.",
create_status: createdResponse.status,
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
if (!pipeline?.uuid) {
return {
status: "fail",
reason: "Pipeline was not created or resolved.",
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
reason: loaded.json.msg || "Failed to load pipeline.",
get_status: loaded.status,
pipeline_id: pipeline.uuid,
};
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const updatedConfig = {
...config,
ai: {
...ai,
runner: {
...(ai.runner && typeof ai.runner === "object" ? ai.runner : {}),
id: runnerId,
"expire-time": 0,
},
runner_config: {
...runnerConfigs,
[runnerId]: runnerConfig,
},
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for real ACP Claude AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
});
if (isApiFailure(updateResponse)) {
return {
status: "fail",
reason: updateResponse.json.msg || "Failed to update pipeline.",
update_status: updateResponse.status,
pipeline_id: pipeline.uuid,
};
}
return {
status: "pass",
reason: created ? "ACP AgentRunner pipeline created and configured." : "ACP AgentRunner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
updated: true,
};
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
}
async function upsertEnvLocal(path, values) {
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const keys = new Set(Object.keys(values));
const output = [];
for (const line of lines) {
const match = line.match(/^([A-Z][A-Z0-9_]*)=/);
if (match && keys.has(match[1])) {
output.push(`${match[1]}=${values[match[1]]}`);
keys.delete(match[1]);
} else if (line !== "" || output.length > 0) {
output.push(line);
}
}
if (keys.size > 0 && output.length > 0 && output[output.length - 1] !== "") {
output.push("");
}
for (const key of keys) {
output.push(`${key}=${values[key]}`);
}
await writeFile(path, `${output.join("\n").replace(/\n+$/, "")}\n`, "utf8");
}
+205
View File
@@ -0,0 +1,205 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { env } from "node:process";
import {
appendLine,
ensureEvidence,
evidencePaths,
loadEnvFiles,
redact,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = "ensure-fake-provider-cross-pipelines";
const DEFAULT_PIPELINE_A_NAME = "LangBot QA Fake Provider Debug Chat A";
const DEFAULT_PIPELINE_B_NAME = "LangBot QA Fake Provider Debug Chat B";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const envLocalPath = resolve("skills/.env.local");
const pipelineAName = env.LANGBOT_FAKE_PROVIDER_PIPELINE_A_NAME || DEFAULT_PIPELINE_A_NAME;
const pipelineBName = env.LANGBOT_FAKE_PROVIDER_PIPELINE_B_NAME || DEFAULT_PIPELINE_B_NAME;
const result = {
source: "setup_automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
pipeline_a: {
name: pipelineAName,
id: "",
url: "",
},
pipeline_b: {
name: pipelineBName,
id: "",
url: "",
},
fake_provider: {
url: "",
base_url: "",
pid: null,
},
wrote_env: false,
evidence: {
console_log: paths.consoleLog,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic", "filesystem"],
};
try {
console.error(`[langbot-qa] configuring cross-pipeline QA fixtures: pipeline_a=\"${pipelineAName}\", pipeline_b=\"${pipelineBName}\"`);
console.error("[langbot-qa] run these fake-provider setup/probe commands serially when they share LANGBOT_FAKE_PROVIDER_URL.");
if (pipelineAName === pipelineBName) {
throw new Error("LANGBOT_FAKE_PROVIDER_PIPELINE_A_NAME and LANGBOT_FAKE_PROVIDER_PIPELINE_B_NAME must be different.");
}
const setupA = await runPipelineSetup(pipelineAName, "A");
const setupB = await runPipelineSetup(pipelineBName, "B");
result.pipeline_a = {
name: setupA.pipeline_name || pipelineAName,
id: setupA.pipeline_id || "",
url: setupA.pipeline_url || "",
};
result.pipeline_b = {
name: setupB.pipeline_name || pipelineBName,
id: setupB.pipeline_id || "",
url: setupB.pipeline_url || "",
};
result.fake_provider = {
url: setupB.fake_provider?.url || setupA.fake_provider?.url || "",
base_url: setupB.fake_provider?.base_url || setupA.fake_provider?.base_url || "",
pid: setupB.fake_provider?.pid ?? setupA.fake_provider?.pid ?? null,
};
if (!result.pipeline_a.url || !result.pipeline_b.url || !result.fake_provider.url) {
throw new Error("Cross-pipeline fake provider setup did not return both pipeline URLs and provider URL.");
}
if (writeEnv) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_FAKE_PROVIDER_URL: result.fake_provider.url,
LANGBOT_FAKE_PROVIDER_BASE_URL: result.fake_provider.base_url,
LANGBOT_FAKE_PROVIDER_PID: result.fake_provider.pid ? String(result.fake_provider.pid) : "",
LANGBOT_FAKE_PROVIDER_PIPELINE_A_URL: result.pipeline_a.url,
LANGBOT_FAKE_PROVIDER_PIPELINE_A_NAME: result.pipeline_a.name,
LANGBOT_FAKE_PROVIDER_PIPELINE_B_URL: result.pipeline_b.url,
LANGBOT_FAKE_PROVIDER_PIPELINE_B_NAME: result.pipeline_b.name,
});
result.wrote_env = true;
}
result.status = "pass";
result.reason = "Fake provider cross-pipeline fixtures are configured.";
} catch (error) {
result.status = looksLikeEnvIssue(error) ? "env_issue" : "fail";
result.reason = safeReason(error.message);
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
function runPipelineSetup(pipelineName, label) {
return new Promise((resolvePromise, rejectPromise) => {
const child = spawn(process.execPath, ["scripts/e2e/ensure-fake-provider-pipeline.mjs"], {
cwd: resolve("."),
env: {
...env,
LANGBOT_FAKE_PROVIDER_PIPELINE_NAME: pipelineName,
LANGBOT_FAKE_PROVIDER_FIRST_TOKEN_DELAY_MS: env.LANGBOT_FAKE_PROVIDER_FIRST_TOKEN_DELAY_MS || "25",
LANGBOT_FAKE_PROVIDER_CHUNK_DELAY_MS: env.LANGBOT_FAKE_PROVIDER_CHUNK_DELAY_MS || "10",
LANGBOT_FAKE_PROVIDER_CHUNK_COUNT: env.LANGBOT_FAKE_PROVIDER_CHUNK_COUNT || "0",
LANGBOT_FAKE_PROVIDER_FAIL_FIRST_N: "0",
LANGBOT_FAKE_PROVIDER_FAIL_EVERY_N: "0",
LANGBOT_FAKE_PROVIDER_FAULT_STATUS: env.LANGBOT_FAKE_PROVIDER_FAULT_STATUS || "500",
LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK: "false",
LANGBOT_FAKE_PROVIDER_DYNAMIC_RESPONSE: "true",
},
stdio: ["ignore", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout.on("data", (chunk) => {
const text = chunk.toString();
stdout += text;
appendLine(paths.consoleLog, `[setup ${label} stdout] ${text.trimEnd()}`).catch(() => {});
});
child.stderr.on("data", (chunk) => {
const text = chunk.toString();
stderr += text;
appendLine(paths.consoleLog, `[setup ${label} stderr] ${text.trimEnd()}`).catch(() => {});
});
child.on("error", rejectPromise);
child.on("close", (code) => {
const parsed = parseJsonOutput(stdout);
if (code !== 0 || parsed.status !== "pass") {
rejectPromise(new Error(parsed.reason || stderr || `Fake provider pipeline setup ${label} exited with ${code}.`));
return;
}
resolvePromise(parsed);
});
});
}
function parseJsonOutput(text) {
const trimmed = String(text || "").trim();
if (!trimmed) return {};
try {
return JSON.parse(trimmed);
} catch {
const start = trimmed.indexOf("{");
const end = trimmed.lastIndexOf("}");
if (start >= 0 && end > start) {
try {
return JSON.parse(trimmed.slice(start, end + 1));
} catch {
return {};
}
}
return {};
}
}
async function upsertEnvLocal(path, updates) {
await mkdir(dirname(path), { recursive: true });
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const seen = new Set();
const next = lines.map((line) => {
const trimmed = line.trim();
const match = trimmed.match(/^([A-Z][A-Z0-9_]*)=/);
if (!match || updates[match[1]] === undefined) return line;
seen.add(match[1]);
return `${match[1]}=${updates[match[1]]}`;
});
for (const [key, value] of Object.entries(updates)) {
if (!seen.has(key)) next.push(`${key}=${value}`);
}
await writeFile(path, `${next.join("\n").replace(/\n+$/, "")}\n`, "utf8");
}
function looksLikeEnvIssue(error) {
const message = String(error?.message || error || "");
return /fetch failed|ECONNREFUSED|ENOTFOUND|LANGBOT_.*not configured|Could not read recovery_key|Backend did not respond/i.test(message);
}
function safeReason(value) {
return redact(String(value || "")).slice(0, 1000);
}
+635
View File
@@ -0,0 +1,635 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { open, readFile, mkdir, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
ensureEvidence,
evidencePaths,
loadEnvFiles,
redact,
resetAndAuthLocalUser,
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "local-agent";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const DEFAULT_PIPELINE_NAME = "LangBot QA Fake Provider Debug Chat";
const DEFAULT_PROVIDER_NAME = "LangBot QA Fake OpenAI Provider";
const QA_RESOURCE_DESCRIPTION = "Managed by LangBot skills QA automation for controlled fake-provider Debug Chat tests. Safe to delete when local QA fixtures are no longer needed.";
const DEFAULT_MODEL_NAME = "gpt-4o-mini";
const DEFAULT_REQUESTER = "openai-chat-completions";
const caseId = "ensure-fake-provider-pipeline";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const envLocalPath = resolve("skills/.env.local");
const repoRoot = resolve(env.LANGBOT_REPO || "..");
const fakeStateDir = resolve(env.LANGBOT_FAKE_PROVIDER_STATE_DIR || resolve(repoRoot, ".qa/fake-provider"));
const fakeStatePath = resolve(fakeStateDir, "state.json");
const fakeStdoutPath = resolve(fakeStateDir, "fake-provider.stdout.log");
const fakeStderrPath = resolve(fakeStateDir, "fake-provider.stderr.log");
const pipelineName = env.LANGBOT_FAKE_PROVIDER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const providerName = env.LANGBOT_FAKE_PROVIDER_NAME || DEFAULT_PROVIDER_NAME;
const requester = env.LANGBOT_FAKE_PROVIDER_REQUESTER || DEFAULT_REQUESTER;
const modelName = env.LANGBOT_FAKE_PROVIDER_MODEL_NAME || DEFAULT_MODEL_NAME;
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
frontend_url: frontendUrl,
backend_url: backendUrl,
fake_provider: {
url: "",
base_url: "",
pid: null,
reused: false,
config: {},
state_file: fakeStatePath,
stdout_log: fakeStdoutPath,
stderr_log: fakeStderrPath,
},
provider: {
uuid: "",
name: providerName,
requester,
created: false,
updated: false,
},
model: {
uuid: "",
name: modelName,
created: false,
updated: false,
test_status: "not_run",
test_reason: "",
},
pipeline_id: "",
pipeline_name: pipelineName,
pipeline_url: "",
created: false,
updated: false,
wrote_env: false,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic", "network", "filesystem"],
};
try {
console.error(`[langbot-qa] configuring QA-owned fake-provider fixtures: provider=\"${providerName}\", pipeline=\"${pipelineName}\"`);
console.error("[langbot-qa] this setup may create or update local QA provider/model/pipeline resources on the selected backend.");
if (!backendUrl) {
result.status = "env_issue";
throw new Error("LANGBOT_BACKEND_URL is not configured.");
}
if (!frontendUrl) {
result.status = "env_issue";
throw new Error("LANGBOT_FRONTEND_URL is not configured.");
}
const fakeProvider = await ensureFakeProvider();
const setupConfig = await configureFakeProvider(fakeProvider.url, healthyFakeProviderConfig(), true);
result.fake_provider = {
...result.fake_provider,
...fakeProvider,
config: setupConfig.config || healthyFakeProviderConfig(),
};
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
result.status = "env_issue";
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the fake provider pipeline.");
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
const wizard = await skipWizard({ backendUrl, token: auth.token });
if (wizard.status !== "pass") {
result.status = "fail";
throw new Error(wizard.reason || "Failed to mark the local QA wizard as skipped.");
}
const provider = await ensureProvider({
backendUrl,
token: auth.token,
name: providerName,
requester,
baseUrl: fakeProvider.base_url,
});
result.provider = provider;
const model = await ensureModel({
backendUrl,
token: auth.token,
providerUuid: provider.uuid,
name: modelName,
});
result.model = model;
const pipeline = await ensurePipeline({
backendUrl,
token: auth.token,
name: pipelineName,
modelUuid: model.uuid,
});
Object.assign(result, pipeline);
result.pipeline_url = `${frontendUrl.replace(/\/$/, "")}/home/pipelines?id=${encodeURIComponent(pipeline.pipeline_id)}`;
const runConfig = await configureFakeProvider(fakeProvider.url, targetFakeProviderConfig(), true);
result.fake_provider.config = runConfig.config || targetFakeProviderConfig();
if (writeEnv) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_FAKE_PROVIDER_URL: fakeProvider.url,
LANGBOT_FAKE_PROVIDER_BASE_URL: fakeProvider.base_url,
LANGBOT_FAKE_PROVIDER_PID: fakeProvider.pid ? String(fakeProvider.pid) : "",
LANGBOT_FAKE_PROVIDER_PROVIDER_UUID: provider.uuid,
LANGBOT_FAKE_PROVIDER_MODEL_UUID: model.uuid,
LANGBOT_FAKE_PROVIDER_PIPELINE_URL: result.pipeline_url,
LANGBOT_FAKE_PROVIDER_PIPELINE_NAME: pipelineName,
});
result.wrote_env = true;
}
result.status = "pass";
result.reason = `Fake provider pipeline is configured with ${requester}/${modelName}.`;
} catch (error) {
result.status = result.status === "env_issue" ? "env_issue" : "fail";
result.reason = result.reason || safeReason(error.message);
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
async function ensureFakeProvider() {
const envUrl = normalizeProviderRootUrl(env.LANGBOT_FAKE_PROVIDER_URL || "");
if (envUrl && await fakeProviderHealthy(envUrl) && await fakeProviderConfigurable(envUrl)) {
return {
url: envUrl,
base_url: `${envUrl}/v1`,
pid: null,
reused: true,
};
}
const state = await readState(fakeStatePath);
const stateUrl = normalizeProviderRootUrl(state.url || "");
if (stateUrl && await fakeProviderHealthy(stateUrl)) {
if (await fakeProviderConfigurable(stateUrl)) {
return {
url: stateUrl,
base_url: state.base_url || `${stateUrl}/v1`,
pid: Number.isInteger(state.pid) ? state.pid : null,
reused: true,
};
}
if (Number.isInteger(state.pid)) await stopProcess(state.pid);
}
await mkdir(fakeStateDir, { recursive: true });
await writeFile(fakeStatePath, `${JSON.stringify({ status: "starting", started_at: new Date().toISOString() }, null, 2)}\n`, "utf8");
const stdout = await open(fakeStdoutPath, "a");
const stderr = await open(fakeStderrPath, "a");
const scriptPath = resolve("scripts/e2e/fake-openai-provider.mjs");
const host = env.LANGBOT_FAKE_PROVIDER_HOST || "127.0.0.1";
const port = env.LANGBOT_FAKE_PROVIDER_PORT || "0";
const child = spawn(process.execPath, [
scriptPath,
`--host=${host}`,
`--port=${port}`,
`--state-file=${fakeStatePath}`,
], {
cwd: resolve("."),
detached: true,
env: {
...env,
LANGBOT_FAKE_PROVIDER_MODEL_NAME: modelName,
},
stdio: ["ignore", stdout.fd, stderr.fd],
});
child.unref();
await stdout.close();
await stderr.close();
const started = await waitForFakeProviderState(fakeStatePath, child.pid, 10_000);
if (!started.url || !await fakeProviderHealthy(started.url) || !await fakeProviderConfigurable(started.url)) {
throw new Error(`Fake provider did not become healthy. See ${fakeStderrPath}`);
}
return {
url: started.url,
base_url: started.base_url || `${started.url}/v1`,
pid: child.pid ?? started.pid ?? null,
reused: false,
};
}
async function configureFakeProvider(rootUrl, config, resetRequestCount) {
const response = await fetch(`${normalizeProviderRootUrl(rootUrl)}/__qa/config`, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({
config,
reset_request_count: resetRequestCount,
}),
signal: AbortSignal.timeout(3000),
});
const json = await response.json().catch(() => ({}));
if (!response.ok || json.ok !== true) {
throw new Error(`Fake provider config failed with HTTP ${response.status}.`);
}
return json;
}
async function fakeProviderHealthy(rootUrl) {
try {
const response = await fetch(`${rootUrl.replace(/\/$/, "")}/healthz`, {
signal: AbortSignal.timeout(2000),
});
if (!response.ok) return false;
const json = await response.json().catch(() => ({}));
return json.ok === true;
} catch {
return false;
}
}
async function fakeProviderConfigurable(rootUrl) {
try {
const response = await fetch(`${rootUrl.replace(/\/$/, "")}/__qa/config`, {
signal: AbortSignal.timeout(2000),
});
if (!response.ok) return false;
const json = await response.json().catch(() => ({}));
return json.ok === true && json.config && typeof json.config === "object";
} catch {
return false;
}
}
async function stopProcess(pid) {
try {
process.kill(pid, "SIGTERM");
} catch {
return;
}
await sleep(500);
}
async function waitForFakeProviderState(path, expectedPid, timeoutMs) {
const startedAt = Date.now();
let lastState = {};
while (Date.now() - startedAt < timeoutMs) {
const state = await readState(path);
if (state.url && (!expectedPid || state.pid === expectedPid)) return state;
lastState = state;
await sleep(150);
}
return lastState;
}
async function readState(path) {
try {
return JSON.parse(await readFile(path, "utf8"));
} catch {
return {};
}
}
function normalizeProviderRootUrl(value) {
const trimmed = String(value || "").trim().replace(/\/$/, "");
return trimmed.endsWith("/v1") ? trimmed.slice(0, -3) : trimmed;
}
function healthyFakeProviderConfig() {
return {
response_text: "OK",
first_token_delay_ms: 25,
chunk_delay_ms: 10,
chunk_count: 0,
fault_status: 500,
fail_first_n: 0,
fail_every_n: 0,
fail_after_first_chunk: false,
dynamic_response: true,
};
}
function targetFakeProviderConfig() {
return {
response_text: env.LANGBOT_FAKE_PROVIDER_RESPONSE_TEXT || "OK",
first_token_delay_ms: nonNegativeInteger(env.LANGBOT_FAKE_PROVIDER_FIRST_TOKEN_DELAY_MS, 25),
chunk_delay_ms: nonNegativeInteger(env.LANGBOT_FAKE_PROVIDER_CHUNK_DELAY_MS, 10),
chunk_count: nonNegativeInteger(env.LANGBOT_FAKE_PROVIDER_CHUNK_COUNT, 0),
fault_status: httpFaultStatus(env.LANGBOT_FAKE_PROVIDER_FAULT_STATUS, 500),
fail_first_n: nonNegativeInteger(env.LANGBOT_FAKE_PROVIDER_FAIL_FIRST_N, 0),
fail_every_n: nonNegativeInteger(env.LANGBOT_FAKE_PROVIDER_FAIL_EVERY_N, 0),
fail_after_first_chunk: envBool(env.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK, false),
dynamic_response: envBool(env.LANGBOT_FAKE_PROVIDER_DYNAMIC_RESPONSE, true),
};
}
async function skipWizard({ backendUrl, token }) {
const response = await apiJson(backendUrl, "/api/v1/system/wizard/completed", {
method: "POST",
token,
body: { status: "skipped" },
});
const ok = response.status < 400 && response.json.code === 0;
return {
status: ok ? "pass" : "fail",
http_status: response.status,
code: response.json.code ?? null,
reason: ok ? "Wizard marked skipped for local QA." : response.json.msg || "Wizard status update failed.",
};
}
async function ensureProvider({ backendUrl, token, name, requester, baseUrl }) {
const list = await apiJson(backendUrl, "/api/v1/provider/providers", { token });
if (isApiFailure(list)) {
throw new Error(list.json.msg || "Failed to list providers.");
}
const providers = list.json.data?.providers || [];
const existing = providers.find((provider) => (
provider.name === name
|| (provider.requester === requester && String(provider.base_url || "").replace(/\/$/, "") === baseUrl.replace(/\/$/, ""))
));
const body = {
name,
requester,
base_url: baseUrl,
api_keys: [env.LANGBOT_FAKE_PROVIDER_API_KEY || "langbot-fake-provider-key"],
};
if (existing?.uuid) {
const update = await apiJson(backendUrl, `/api/v1/provider/providers/${encodeURIComponent(existing.uuid)}`, {
method: "PUT",
token,
body,
});
if (isApiFailure(update)) {
throw new Error(update.json.msg || "Failed to update fake provider.");
}
return {
uuid: existing.uuid,
name,
requester,
created: false,
updated: true,
};
}
const create = await apiJson(backendUrl, "/api/v1/provider/providers", {
method: "POST",
token,
body,
});
const uuid = create.json.data?.uuid || "";
if (isApiFailure(create) || !uuid) {
throw new Error(create.json.msg || "Failed to create fake provider.");
}
return {
uuid,
name,
requester,
created: true,
updated: false,
};
}
async function ensureModel({ backendUrl, token, providerUuid, name }) {
const list = await apiJson(backendUrl, `/api/v1/provider/models/llm?provider_uuid=${encodeURIComponent(providerUuid)}`, { token });
if (isApiFailure(list)) {
throw new Error(list.json.msg || "Failed to list fake provider models.");
}
const models = list.json.data?.models || [];
const existing = models.find((model) => model.name === name);
const body = {
name,
provider_uuid: providerUuid,
abilities: [],
context_length: positiveInteger(env.LANGBOT_FAKE_PROVIDER_CONTEXT_LENGTH, 8192),
extra_args: {},
prefered_ranking: 0,
};
let modelUuid = existing?.uuid || "";
let created = false;
let updated = false;
if (modelUuid) {
const update = await apiJson(backendUrl, `/api/v1/provider/models/llm/${encodeURIComponent(modelUuid)}`, {
method: "PUT",
token,
body,
});
if (isApiFailure(update)) {
throw new Error(update.json.msg || "Failed to update fake provider model.");
}
updated = true;
} else {
const create = await apiJson(backendUrl, "/api/v1/provider/models/llm", {
method: "POST",
token,
body,
});
modelUuid = create.json.data?.uuid || "";
if (isApiFailure(create) || !modelUuid) {
throw new Error(create.json.msg || "Failed to create fake provider model.");
}
created = true;
}
const test = await apiJson(backendUrl, `/api/v1/provider/models/llm/${encodeURIComponent(modelUuid)}/test`, {
method: "POST",
token,
body: { extra_args: {} },
});
if (isApiFailure(test)) {
throw new Error(safeReason(test.json.msg || test.json.message || "Fake provider model test failed."));
}
return {
uuid: modelUuid,
name,
created,
updated,
test_status: "pass",
test_reason: "",
};
}
async function ensurePipeline({ backendUrl, token, name, modelUuid }) {
const list = await apiJson(backendUrl, "/api/v1/pipelines", { token });
if (isApiFailure(list)) {
throw new Error(list.json.msg || "Failed to list pipelines.");
}
const pipelines = list.json.data?.pipelines || [];
let pipeline = pipelines.find((item) => item.name === name) || null;
let created = false;
if (!pipeline) {
const create = await apiJson(backendUrl, "/api/v1/pipelines", {
method: "POST",
token,
body: {
name,
description: QA_RESOURCE_DESCRIPTION,
emoji: "QA",
},
});
const pipelineId = create.json.data?.uuid || "";
if (isApiFailure(create) || !pipelineId) {
throw new Error(create.json.msg || "Failed to create fake provider pipeline.");
}
created = true;
pipeline = { uuid: pipelineId };
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
pipeline = loaded.json.data?.pipeline || null;
if (isApiFailure(loaded) || !pipeline?.uuid) {
throw new Error(loaded.json.msg || "Failed to load fake provider pipeline.");
}
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const existingLocalAgentConfig = ai["local-agent"] && typeof ai["local-agent"] === "object"
? ai["local-agent"]
: {};
const localAgentConfig = {
timeout: 60,
prompt: [{ role: "system", content: "You are a deterministic QA assistant. Reply exactly as instructed." }],
"remove-think": false,
"knowledge-bases": [],
"box-session-id-template": "{launcher_type}_{launcher_id}",
"retrieval-top-k": 5,
"rerank-model": "",
"rerank-top-k": 5,
"max-tool-iterations": 20,
"tool-execution-mode": "parallel",
"max-tool-result-chars": 20000,
"context-history-fetch-limit": 20,
"context-window-tokens": 8192,
"context-reserve-tokens": 1024,
"context-keep-recent-tokens": 2048,
"context-summary-tokens": 1024,
...existingLocalAgentConfig,
// Current backend truncation still reads this field directly.
"max-round": positiveInteger(existingLocalAgentConfig["max-round"], 10),
model: {
primary: modelUuid,
fallbacks: [],
},
};
const updatedConfig = {
...config,
ai: {
...ai,
runner: {
...(ai.runner && typeof ai.runner === "object" ? ai.runner : {}),
id: RUNNER_ID,
runner: RUNNER_ID,
"expire-time": 0,
},
"local-agent": localAgentConfig,
},
};
const update = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name,
description: QA_RESOURCE_DESCRIPTION,
emoji: "QA",
config: updatedConfig,
},
});
if (isApiFailure(update)) {
throw new Error(update.json.msg || "Failed to update fake provider pipeline.");
}
return {
pipeline_id: pipeline.uuid,
pipeline_name: name,
created,
updated: true,
};
}
function isApiFailure(response) {
return response.status >= 400 || (response.json.code !== undefined && response.json.code !== 0);
}
function positiveInteger(value, fallback) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback;
}
function nonNegativeInteger(value, fallback) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed >= 0 ? parsed : fallback;
}
function httpFaultStatus(value, fallback) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed >= 400 && parsed <= 599 ? parsed : fallback;
}
function envBool(value, fallback) {
if (value === undefined || value === "") return fallback;
if (/^(1|true|yes|on)$/i.test(String(value))) return true;
if (/^(0|false|no|off)$/i.test(String(value))) return false;
return fallback;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function safeReason(value) {
return redact(String(value || "")).slice(0, 1000);
}
async function upsertEnvLocal(path, updates) {
await mkdir(dirname(path), { recursive: true });
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const seen = new Set();
const next = lines.map((line) => {
const trimmed = line.trim();
const equals = trimmed.indexOf("=");
if (equals <= 0 || trimmed.startsWith("#")) return line;
const key = trimmed.slice(0, equals).trim();
if (!(key in updates)) return line;
seen.add(key);
return `${key}=${updates[key]}`;
});
for (const [key, value] of Object.entries(updates)) {
if (!seen.has(key)) next.push(`${key}=${value}`);
}
await writeFile(path, `${next.filter((line, index) => line !== "" || index < next.length - 1).join("\n")}\n`, "utf8");
}
+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env node
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
ensureEvidence,
evidencePaths,
loadEnvFiles,
resetAndAuthLocalUser,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = env.LBS_CASE_ID || "ensure-langrag-sentinel-kb";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || "LangBotE2ELocalPass!2026";
const expectedText = env.LANGBOT_E2E_EXPECTED_TEXT || "azalea-cobalt-7421";
const query = env.LANGBOT_E2E_RETRIEVE_QUERY || "What is the local agent runner retrieval sentinel?";
const writeEnv = process.argv.includes("--write-env");
const checkOnly = process.argv.includes("--check-only");
const envLocalPath = resolve("skills/.env.local");
const kbName = env.LANGBOT_E2E_RAG_KB_NAME || "qa-local-agent-rag";
const sentinelPath = resolve(env.LANGBOT_E2E_RAG_SENTINEL_DOC || "skills/langbot-testing/fixtures/rag/sentinel-doc.txt");
const waitMs = Number(env.LANGBOT_E2E_RAG_WAIT_MS || 180_000);
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
backend_url: backendUrl,
expected_text: expectedText,
query,
kb_uuid: "",
kb_name: "",
kb_created: false,
uploaded_file_id: "",
store_task_id: "",
embedding_model_uuid: "",
engine_plugin_id: "",
checked_bases: [],
file_statuses: [],
wrote_env: false,
evidence: {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic"],
};
try {
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
if (!user) throw new Error("LANGBOT_E2E_LOGIN_USER is required.");
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
const basesResponse = await apiJson(backendUrl, "/api/v1/knowledge/bases", { token: auth.token });
if (basesResponse.status >= 400 || basesResponse.json.code !== 0) {
throw new Error(basesResponse.json.msg || `Failed to list knowledge bases: HTTP ${basesResponse.status}.`);
}
let bases = basesResponse.json.data?.bases || [];
await findSentinelBase(backendUrl, auth.token, bases, result);
if (!result.kb_uuid && !checkOnly) {
const targetBase = bases.find((base) => {
const uuid = base.uuid || base.id || "";
return (base.name || "") === kbName && !hasRetrieveFailure(result.checked_bases, uuid);
});
result.kb_uuid = targetBase?.uuid || targetBase?.id || "";
result.kb_name = targetBase?.name || kbName;
if (!result.kb_uuid) {
const setup = await createKnowledgeBase(backendUrl, auth.token, kbName);
result.kb_uuid = setup.kbUuid;
result.kb_name = kbName;
result.kb_created = true;
result.embedding_model_uuid = setup.embeddingModelUuid;
result.engine_plugin_id = setup.enginePluginId;
}
const upload = await uploadDocument(backendUrl, auth.token, sentinelPath);
result.uploaded_file_id = upload.fileId;
const store = await apiJson(backendUrl, `/api/v1/knowledge/bases/${encodeURIComponent(result.kb_uuid)}/files`, {
method: "POST",
token: auth.token,
body: { file_id: upload.fileId },
});
if (store.status >= 400 || store.json.code !== 0) {
throw new Error(store.json.msg || `Failed to store file in knowledge base: HTTP ${store.status}.`);
}
result.store_task_id = store.json.data?.task_id || "";
const ready = await waitForSentinel(backendUrl, auth.token, result.kb_uuid, query, expectedText, waitMs);
result.file_statuses = ready.fileStatuses;
if (ready.matched) {
result.checked_bases.push(ready.checked);
}
}
if (!result.kb_uuid) {
result.status = "env_issue";
result.reason = checkOnly
? `No existing knowledge base retrieved expected sentinel: ${expectedText}`
: `Could not create or verify LangRAG sentinel knowledge base: ${expectedText}`;
} else {
if (writeEnv) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_LOCAL_AGENT_RAG_KB_UUID: result.kb_uuid,
});
result.wrote_env = true;
}
result.status = "pass";
result.reason = `Found LangRAG sentinel knowledge base: ${result.kb_uuid}`;
}
} catch (error) {
result.status = /not configured|required|No existing knowledge base/.test(error.message) ? "env_issue" : "fail";
result.reason = error.message;
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
async function findSentinelBase(backendUrl, token, bases, result) {
for (const base of bases) {
const uuid = base.uuid || base.id || "";
if (!uuid) continue;
const checked = await retrieveSentinel(backendUrl, token, uuid, base.name || "", result.query, result.expected_text);
result.checked_bases.push(checked);
if (checked.matched) {
result.kb_uuid = uuid;
result.kb_name = checked.name;
return;
}
}
}
async function createKnowledgeBase(backendUrl, token, name) {
const enginesResponse = await apiJson(backendUrl, "/api/v1/knowledge/engines", { token });
if (enginesResponse.status >= 400 || enginesResponse.json.code !== 0) {
throw new Error(enginesResponse.json.msg || `Failed to list knowledge engines: HTTP ${enginesResponse.status}.`);
}
const engines = enginesResponse.json.data?.engines || [];
const engine = engines.find((item) => item.plugin_id === "langbot-team/LangRAG")
|| engines.find((item) => JSON.stringify(item.name || item.label || "").includes("LangRAG"));
const enginePluginId = engine?.plugin_id || "";
if (!enginePluginId) throw new Error("LangRAG knowledge engine is not installed.");
const embeddingModelUuid = await pickEmbeddingModel(backendUrl, token);
const create = await apiJson(backendUrl, "/api/v1/knowledge/bases", {
method: "POST",
token,
body: {
name,
description: "Automated LangBot agent-runner RAG sentinel knowledge base.",
knowledge_engine_plugin_id: enginePluginId,
creation_settings: {
embedding_model_uuid: embeddingModelUuid,
index_type: "chunk",
chunk_size: 512,
overlap: 50,
},
retrieval_settings: {
top_k: 5,
search_type: "vector",
query_rewrite: "off",
rerank: "off",
context_window: 0,
},
},
});
const kbUuid = create.json.data?.uuid || "";
if (create.status >= 400 || create.json.code !== 0 || !kbUuid) {
throw new Error(create.json.msg || `Failed to create knowledge base: HTTP ${create.status}.`);
}
return { kbUuid, embeddingModelUuid, enginePluginId };
}
async function pickEmbeddingModel(backendUrl, token) {
const configured = env.LANGBOT_LOCAL_AGENT_RAG_EMBEDDING_MODEL_UUID || env.LANGBOT_RAG_EMBEDDING_MODEL_UUID || "";
if (configured) return configured;
const modelsResponse = await apiJson(backendUrl, "/api/v1/provider/models/embedding", { token });
if (modelsResponse.status >= 400 || modelsResponse.json.code !== 0) {
throw new Error(modelsResponse.json.msg || `Failed to list embedding models: HTTP ${modelsResponse.status}.`);
}
const models = modelsResponse.json.data?.models || [];
const preferred = models.find((model) => /chroma|MiniLM/i.test(model.name || ""))
|| models.find((model) => /text-embedding-3-small/i.test(model.name || ""))
|| [...models].sort((a, b) => (a.prefered_ranking ?? 9999) - (b.prefered_ranking ?? 9999))[0];
const uuid = preferred?.uuid || "";
if (!uuid) throw new Error("No embedding model is configured.");
return uuid;
}
async function uploadDocument(backendUrl, token, path) {
const bytes = await readFile(path);
const form = new FormData();
form.append("file", new Blob([bytes], { type: "text/plain" }), "sentinel-doc.txt");
const response = await fetch(`${backendUrl.replace(/\/$/, "")}/api/v1/files/documents`, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
},
body: form,
});
const json = await response.json().catch(() => ({}));
const fileId = json.data?.file_id || "";
if (response.status >= 400 || json.code !== 0 || !fileId) {
throw new Error(json.msg || `Failed to upload sentinel document: HTTP ${response.status}.`);
}
return { fileId };
}
async function waitForSentinel(backendUrl, token, kbUuid, query, expectedText, timeoutMs) {
const started = Date.now();
let fileStatuses = [];
let lastChecked = null;
while (Date.now() - started < timeoutMs) {
const files = await apiJson(backendUrl, `/api/v1/knowledge/bases/${encodeURIComponent(kbUuid)}/files`, { token });
fileStatuses = files.json.data?.files || fileStatuses;
lastChecked = await retrieveSentinel(backendUrl, token, kbUuid, kbName, query, expectedText);
if (lastChecked.matched) {
return { matched: true, fileStatuses, checked: lastChecked };
}
if (fileStatuses.some((item) => item.status === "failed")) break;
await sleep(2_000);
}
result.reason = lastChecked?.msg
|| `LangRAG sentinel was not retrievable within ${timeoutMs}ms; file statuses: ${JSON.stringify(fileStatuses)}`;
result.kb_uuid = "";
return { matched: false, fileStatuses, checked: lastChecked };
}
async function retrieveSentinel(backendUrl, token, uuid, name, query, expectedText) {
const retrieve = await apiJson(backendUrl, `/api/v1/knowledge/bases/${encodeURIComponent(uuid)}/retrieve`, {
method: "POST",
token,
body: { query },
});
const text = JSON.stringify(retrieve.json.data?.results || []);
return {
uuid,
name,
http_status: retrieve.status,
code: retrieve.json.code ?? null,
msg: retrieve.json.msg || "",
matched: text.includes(expectedText),
};
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function hasRetrieveFailure(checkedBases, uuid) {
const checked = checkedBases.find((item) => item.uuid === uuid);
return checked && (checked.http_status >= 500 || (typeof checked.code === "number" && checked.code < 0));
}
async function upsertEnvLocal(path, values) {
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const keys = new Set(Object.keys(values));
const output = [];
for (const line of lines) {
const match = line.match(/^([A-Z][A-Z0-9_]*)=/);
if (match && keys.has(match[1])) {
output.push(`${match[1]}=${values[match[1]]}`);
keys.delete(match[1]);
} else if (line !== "" || output.length > 0) {
output.push(line);
}
}
if (keys.size > 0 && output.length > 0 && output[output.length - 1] !== "") output.push("");
for (const key of keys) output.push(`${key}=${values[key]}`);
await writeFile(path, `${output.join("\n").replace(/\n+$/, "")}\n`, "utf8");
}
+609
View File
@@ -0,0 +1,609 @@
#!/usr/bin/env node
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
bodyText,
createBrowser,
ensureEvidence,
evidencePaths,
loadEnvFiles,
redact,
resetAndAuthLocalUser,
safeScreenshot,
setBrowserToken,
verifyBrowserToken,
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "local-agent";
const SPACE_PROVIDER_UUID = "00000000-0000-0000-0000-000000000000";
const DEFAULT_PIPELINE_NAME = "Agent QA Local Agent Debug Chat";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const DEFAULT_MODEL_TEST_LIMIT = 8;
const DEFAULT_MODEL_FALLBACK_COUNT = 3;
const caseId = "ensure-local-agent-pipeline";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_LOCAL_AGENT_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const envLocalPath = resolve("skills/.env.local");
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
frontend_url: frontendUrl,
backend_url: backendUrl,
pipeline_name: pipelineName,
pipeline_id: "",
pipeline_url: "",
runner_id: RUNNER_ID,
selected_model_id: "",
selected_model_name: "",
fallback_model_ids: [],
model_count: 0,
space_model_count: 0,
scanned_space_model_count: 0,
tested_model_count: 0,
model_tests: [],
created: false,
updated: false,
wrote_env: false,
auth: null,
wizard: null,
browser_token_check: null,
page_signal: "",
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic", "console", "network", "screenshot"],
};
let browser;
try {
if (!frontendUrl) throw new Error("LANGBOT_FRONTEND_URL is not configured.");
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
result.status = "env_issue";
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
result.auth = {
source: "local_recovery_login",
user,
backend_token_check: auth.check,
};
const wizard = await skipWizard({ backendUrl, token: auth.token });
result.wizard = wizard;
if (wizard.status !== "pass") {
result.status = "fail";
throw new Error(wizard.reason || "Failed to mark the local QA wizard as skipped.");
}
const prepared = await ensureLocalAgentPipeline({
backendUrl,
token: auth.token,
pipelineName,
runnerId: RUNNER_ID,
});
Object.assign(result, prepared);
if (result.pipeline_id) {
result.pipeline_url = `${frontendUrl.replace(/\/$/, "")}/home/pipelines?id=${encodeURIComponent(result.pipeline_id)}`;
}
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_PIPELINE_URL: result.pipeline_url,
LANGBOT_PIPELINE_NAME: result.pipeline_name || pipelineName,
LANGBOT_LOCAL_AGENT_PIPELINE_URL: result.pipeline_url,
LANGBOT_LOCAL_AGENT_PIPELINE_NAME: result.pipeline_name || pipelineName,
...(result.selected_model_id ? {
LANGBOT_LOCAL_AGENT_MODEL_UUID: result.selected_model_id,
LANGBOT_E2E_MODEL_UUID: result.selected_model_id,
} : {}),
});
result.wrote_env = true;
}
browser = await createBrowser(paths);
const { page } = browser;
await setBrowserToken(page, frontendUrl, auth.token);
const browserCheck = await verifyBrowserToken(page, backendUrl);
result.browser_token_check = browserCheck;
if (!browserCheck.authenticated) {
throw new Error(browserCheck.reason || "Browser token check failed after setup.");
}
await page.goto(result.pipeline_url || frontendUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const text = await bodyText(page);
result.page_signal = ["Pipelines", "流水线", pipelineName].find((signal) => text.includes(signal)) || "";
} catch (error) {
result.status = result.status === "env_issue" ? "env_issue" : "fail";
result.reason = result.reason || error.message;
} finally {
if (browser?.page) await safeScreenshot(browser.page, paths.screenshot);
if (browser) await browser.close().catch(() => {});
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
async function skipWizard({ backendUrl, token }) {
const response = await apiJson(backendUrl, "/api/v1/system/wizard/completed", {
method: "POST",
token,
body: { status: "skipped" },
});
const ok = response.status < 400 && response.json.code === 0;
return {
status: ok ? "pass" : "fail",
http_status: response.status,
code: response.json.code ?? null,
reason: ok ? "Wizard marked skipped for local QA." : response.json.msg || "Wizard status update failed.",
};
}
async function ensureLocalAgentPipeline({ backendUrl, token, pipelineName, runnerId }) {
const [pipelineList, modelList] = await Promise.all([
apiJson(backendUrl, "/api/v1/pipelines", { token }),
apiJson(backendUrl, "/api/v1/provider/models/llm", { token }),
]);
if (isApiFailure(pipelineList)) {
return {
status: "fail",
reason: pipelineList.json.msg || "Failed to list pipelines.",
list_status: pipelineList.status,
};
}
if (isApiFailure(modelList)) {
return {
status: "fail",
reason: modelList.json.msg || "Failed to list LLM models.",
model_status: modelList.status,
};
}
const models = modelList.json.data?.models || [];
const skippedModelIds = new Set(
String(env.LANGBOT_E2E_SKIP_MODEL_UUIDS || "")
.split(",")
.map((item) => item.trim())
.filter(Boolean),
);
const skippedModelNames = new Set(
String(env.LANGBOT_E2E_SKIP_MODEL_NAMES || "")
.split(",")
.map((item) => item.trim())
.filter(Boolean),
);
const spaceModels = models.filter((model) => isSpaceModel(model) && !skippedModelIds.has(model.uuid));
const pipelines = pipelineList.json.data?.pipelines || [];
let pipeline = pipelines.find((item) => item.name === pipelineName) || null;
let created = false;
if (!pipeline) {
const createdResponse = await apiJson(backendUrl, "/api/v1/pipelines", {
method: "POST",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for AgentRunner Debug Chat smoke tests.",
emoji: "QA",
},
});
if (isApiFailure(createdResponse)) {
return {
status: "fail",
reason: createdResponse.json.msg || "Failed to create pipeline.",
create_status: createdResponse.status,
model_count: models.length,
space_model_count: spaceModels.length,
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
if (!pipeline?.uuid) {
return {
status: "fail",
reason: "Pipeline was not created or resolved.",
model_count: models.length,
space_model_count: spaceModels.length,
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
reason: loaded.json.msg || "Failed to load pipeline.",
get_status: loaded.status,
pipeline_id: pipeline.uuid,
model_count: models.length,
space_model_count: spaceModels.length,
};
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const rawExistingLocalAgentConfig = ai["local-agent"] && typeof ai["local-agent"] === "object"
? ai["local-agent"]
: {};
const existingLocalAgentConfig = rawExistingLocalAgentConfig;
const existingModel = existingLocalAgentConfig.model && typeof existingLocalAgentConfig.model === "object"
? existingLocalAgentConfig.model
: {};
const requestedModelId = env.LANGBOT_LOCAL_AGENT_MODEL_UUID || env.LANGBOT_E2E_MODEL_UUID || "";
const selected = await selectWorkingSpaceModel({
backendUrl,
token,
models,
skippedModelIds,
skippedModelNames,
requestedModelId,
existingModelId: existingModel.primary || "",
});
const selectedModelId = selected.selected_model_id || "";
const localAgentConfig = {
timeout: 300,
prompt: [{ role: "system", content: "You are a helpful assistant." }],
"remove-think": false,
"knowledge-bases": [],
"box-session-id-template": "{launcher_type}_{launcher_id}",
"retrieval-top-k": 5,
"rerank-model": "",
"rerank-top-k": 5,
"max-tool-iterations": 20,
"tool-execution-mode": "parallel",
"max-tool-result-chars": 20000,
"context-history-fetch-limit": 50,
"context-window-tokens": 200000,
"context-reserve-tokens": 16384,
"context-keep-recent-tokens": 20000,
"context-summary-tokens": 8000,
...existingLocalAgentConfig,
// Current backend truncation still reads this field directly.
"max-round": positiveInteger(existingLocalAgentConfig["max-round"], 10),
model: {
primary: selectedModelId,
fallbacks: selected.fallback_model_ids || [],
},
};
const updatedConfig = {
...config,
ai: {
...ai,
runner: {
...(ai.runner && typeof ai.runner === "object" ? ai.runner : {}),
id: runnerId,
runner: runnerId,
"expire-time": 0,
},
"local-agent": localAgentConfig,
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
});
if (isApiFailure(updateResponse)) {
return {
status: "fail",
reason: updateResponse.json.msg || "Failed to update pipeline config.",
update_status: updateResponse.status,
pipeline_id: pipeline.uuid,
model_count: models.length,
space_model_count: spaceModels.length,
scanned_space_model_count: selected.scanned_space_model_count,
tested_model_count: selected.tested_model_count,
model_tests: selected.model_tests,
selected_model_id: selectedModelId,
selected_model_name: selected.selected_model_name,
fallback_model_ids: selected.fallback_model_ids,
};
}
return {
status: selectedModelId ? "pass" : "env_issue",
reason: selectedModelId
? `Local-agent pipeline is configured for Debug Chat with Space model ${selected.selected_model_name || selectedModelId} and ${selected.fallback_model_ids.length} fallback(s).`
: selected.reason || "No working Space LLM model is configured in this LangBot instance.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
model_count: models.length,
space_model_count: spaceModels.length,
scanned_space_model_count: selected.scanned_space_model_count,
tested_model_count: selected.tested_model_count,
model_tests: selected.model_tests,
selected_model_id: selectedModelId,
selected_model_name: selected.selected_model_name,
fallback_model_ids: selected.fallback_model_ids,
created,
updated: true,
};
}
function isApiFailure(response) {
return response.status >= 400 || (response.json.code !== undefined && response.json.code !== 0);
}
function isSpaceModel(model) {
const provider = model?.provider && typeof model.provider === "object" ? model.provider : {};
return model?.provider_uuid === SPACE_PROVIDER_UUID
|| provider.uuid === SPACE_PROVIDER_UUID
|| provider.requester === "space-chat-completions"
|| provider.name === "LangBot Models";
}
async function selectWorkingSpaceModel({
backendUrl,
token,
models,
skippedModelIds,
skippedModelNames,
requestedModelId,
existingModelId,
}) {
const modelTests = [];
const testLimit = positiveInteger(env.LANGBOT_E2E_MODEL_TEST_LIMIT, DEFAULT_MODEL_TEST_LIMIT);
const fallbackCount = positiveInteger(env.LANGBOT_E2E_MODEL_FALLBACK_COUNT, DEFAULT_MODEL_FALLBACK_COUNT);
const workingModels = [];
const spaceModels = rankModels(models.filter((model) => (
model.uuid
&& isSpaceModel(model)
&& !skippedModelIds.has(model.uuid)
&& !skippedModelNames.has(model.name)
)));
const requestedModel = requestedModelId
? spaceModels.find((model) => model.uuid === requestedModelId) || null
: null;
const existingModel = existingModelId
? spaceModels.find((model) => model.uuid === existingModelId) || null
: null;
const candidates = uniqueCandidates([
...(requestedModel ? [existingCandidate(requestedModel, "requested")] : []),
...(existingModel ? [existingCandidate(existingModel, "existing-pipeline")] : []),
...spaceModels.map((model) => existingCandidate(model, "configured-space")),
]);
let scanResult = { status: "skipped", models: [], reason: "" };
if (env.LANGBOT_E2E_SCAN_SPACE_MODELS !== "false") {
scanResult = await scanSpaceModels({ backendUrl, token });
if (scanResult.status === "pass") {
const knownNames = new Set(spaceModels.map((model) => model.name));
candidates.push(...scanResult.models
.filter((model) => model.name && !knownNames.has(model.name) && !skippedModelNames.has(model.name))
.map((model) => scannedCandidate(model)));
}
}
const unique = uniqueCandidates(candidates);
for (const candidate of unique.slice(0, testLimit)) {
const test = await ensureAndTestModel({ backendUrl, token, candidate });
modelTests.push(test);
if (test.status === "pass" && test.model_uuid) {
workingModels.push(test);
if (workingModels.length >= fallbackCount + 1) break;
}
}
if (workingModels.length > 0) {
const [primary, ...fallbacks] = workingModels;
return {
status: "pass",
reason: "",
selected_model_id: primary.model_uuid,
selected_model_name: primary.model_name,
fallback_model_ids: fallbacks.map((model) => model.model_uuid),
scanned_space_model_count: scanResult.models.length,
tested_model_count: modelTests.length,
model_tests: modelTests,
};
}
const baseReason = unique.length === 0
? scanResult.reason || "No Space LLM model candidates are available."
: `No working Space LLM model found after testing ${modelTests.length} candidate(s).`;
return {
status: "env_issue",
reason: requestedModelId && !requestedModel
? `Requested Space LLM model ${requestedModelId} is missing or skipped; ${baseReason}`
: baseReason,
selected_model_id: "",
selected_model_name: "",
fallback_model_ids: [],
scanned_space_model_count: scanResult.models.length,
tested_model_count: modelTests.length,
model_tests: modelTests,
};
}
async function scanSpaceModels({ backendUrl, token }) {
const response = await apiJson(
backendUrl,
`/api/v1/provider/providers/${encodeURIComponent(SPACE_PROVIDER_UUID)}/scan-models?type=llm`,
{ token },
);
if (isApiFailure(response)) {
return {
status: "env_issue",
models: [],
reason: safeReason(response.json.msg || response.json.message || "Failed to scan Space LLM models."),
};
}
return {
status: "pass",
models: response.json.data?.models || [],
reason: "",
};
}
async function ensureAndTestModel({ backendUrl, token, candidate }) {
let modelUuid = candidate.uuid || "";
let created = false;
if (!modelUuid) {
const create = await apiJson(backendUrl, "/api/v1/provider/models/llm", {
method: "POST",
token,
body: {
name: candidate.name,
provider_uuid: SPACE_PROVIDER_UUID,
abilities: candidate.abilities || [],
context_length: candidate.context_length ?? null,
extra_args: {},
prefered_ranking: positiveInteger(candidate.prefered_ranking, 0),
},
});
modelUuid = create.json.data?.uuid || "";
if (isApiFailure(create) || !modelUuid) {
return modelTestResult(candidate, {
status: "fail",
reason: safeReason(create.json.msg || "Failed to create scanned Space model."),
http_status: create.status,
});
}
created = true;
}
const test = await apiJson(backendUrl, `/api/v1/provider/models/llm/${encodeURIComponent(modelUuid)}/test`, {
method: "POST",
token,
body: { extra_args: {} },
});
const passed = !isApiFailure(test);
if (!passed && created) {
await apiJson(backendUrl, `/api/v1/provider/models/llm/${encodeURIComponent(modelUuid)}`, {
method: "DELETE",
token,
}).catch(() => {});
}
return modelTestResult(candidate, {
status: passed ? "pass" : "fail",
reason: passed ? "" : safeReason(test.json.msg || test.json.message || "Space model test failed."),
http_status: test.status,
model_uuid: modelUuid,
created,
});
}
function modelTestResult(candidate, details) {
return {
source: candidate.source,
model_uuid: details.model_uuid || candidate.uuid || "",
model_name: candidate.name,
status: details.status,
reason: details.reason || "",
http_status: details.http_status ?? null,
created: Boolean(details.created),
};
}
function existingCandidate(model, source) {
return {
source,
uuid: model.uuid,
name: model.name,
abilities: model.abilities || [],
context_length: model.context_length,
prefered_ranking: model.prefered_ranking,
};
}
function scannedCandidate(model) {
return {
source: "scanned-space",
uuid: "",
name: model.name || model.id,
abilities: model.abilities || [],
context_length: model.context_length,
prefered_ranking: model.prefered_ranking,
};
}
function uniqueCandidates(candidates) {
const seen = new Set();
const result = [];
for (const candidate of candidates) {
const key = candidate.uuid ? `uuid:${candidate.uuid}` : `name:${candidate.name}`;
if (!candidate.name || seen.has(key)) continue;
seen.add(key);
result.push(candidate);
}
return result;
}
function rankModels(models) {
return [...models].sort((left, right) => {
const leftRank = Number.isFinite(Number(left.prefered_ranking)) ? Number(left.prefered_ranking) : 9999;
const rightRank = Number.isFinite(Number(right.prefered_ranking)) ? Number(right.prefered_ranking) : 9999;
if (leftRank !== rightRank) return leftRank - rightRank;
return String(left.name || "").localeCompare(String(right.name || ""));
});
}
function positiveInteger(value, fallback) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback;
}
function safeReason(value) {
return redact(String(value || "")).slice(0, 1000);
}
async function upsertEnvLocal(path, updates) {
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const seen = new Set();
const next = lines.map((line) => {
const trimmed = line.trim();
const equals = trimmed.indexOf("=");
if (equals <= 0 || trimmed.startsWith("#")) return line;
const key = trimmed.slice(0, equals).trim();
if (!(key in updates)) return line;
seen.add(key);
return `${key}=${updates[key]}`;
});
for (const [key, value] of Object.entries(updates)) {
if (!seen.has(key)) next.push(`${key}=${value}`);
}
await writeFile(path, `${next.filter((line, index) => line !== "" || index < next.length - 1).join("\n")}\n`, "utf8");
}
+230
View File
@@ -0,0 +1,230 @@
#!/usr/bin/env node
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
ensureEvidence,
evidencePaths,
loadEnvFiles,
resetAndAuthLocalUser,
writeResult,
} from "./lib/langbot-e2e.mjs";
const RUNNER_ID = "plugin:qa/agent-runner/default";
const DEFAULT_PIPELINE_NAME = "Agent QA Deterministic Runner Debug Chat";
const DEFAULT_LOCAL_PASSWORD = "LangBotE2ELocalPass!2026";
const caseId = "ensure-qa-agent-runner-pipeline";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const writeEnv = process.argv.includes("--write-env");
const frontendUrl = env.LANGBOT_FRONTEND_URL || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineName = env.LANGBOT_E2E_CREATE_PIPELINE_NAME || env.LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME || DEFAULT_PIPELINE_NAME;
const envLocalPath = resolve("skills/.env.local");
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
frontend_url: frontendUrl,
backend_url: backendUrl,
pipeline_name: pipelineName,
pipeline_id: "",
pipeline_url: "",
runner_id: RUNNER_ID,
wrote_env: false,
auth: null,
evidence: {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic"],
};
try {
if (!frontendUrl) throw new Error("LANGBOT_FRONTEND_URL is not configured.");
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || DEFAULT_LOCAL_PASSWORD;
if (!user) {
throw new Error("LANGBOT_E2E_LOGIN_USER is required so this setup can create/update the pipeline via backend API.");
}
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
result.auth = {
source: "local_recovery_login",
user,
backend_token_check: auth.check,
};
const prepared = await ensurePipeline({
backendUrl,
token: auth.token,
pipelineName,
runnerId: RUNNER_ID,
runnerConfig: {},
});
Object.assign(result, prepared);
if (result.pipeline_id) {
result.pipeline_url = `${frontendUrl.replace(/\/$/, "")}/home/pipelines?id=${encodeURIComponent(result.pipeline_id)}`;
}
if (writeEnv && result.pipeline_id) {
await upsertEnvLocal(envLocalPath, {
LANGBOT_E2E_LOGIN_USER: user,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_URL: result.pipeline_url,
LANGBOT_QA_AGENT_RUNNER_PIPELINE_NAME: result.pipeline_name || pipelineName,
});
result.wrote_env = true;
}
} catch (error) {
result.reason = result.reason || error.message;
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : result.status === "env_issue" ? 2 : 1);
async function ensurePipeline({ backendUrl, token, pipelineName, runnerId, runnerConfig }) {
const pipelineList = await apiJson(backendUrl, "/api/v1/pipelines", { token });
if (isApiFailure(pipelineList)) {
return {
status: "fail",
reason: pipelineList.json.msg || "Failed to list pipelines.",
list_status: pipelineList.status,
};
}
const pipelines = pipelineList.json.data?.pipelines || [];
let pipeline = pipelines.find((item) => item.name === pipelineName) || null;
let created = false;
if (!pipeline) {
const createdResponse = await apiJson(backendUrl, "/api/v1/pipelines", {
method: "POST",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
emoji: "QA",
},
});
if (isApiFailure(createdResponse)) {
return {
status: "fail",
reason: createdResponse.json.msg || "Failed to create pipeline.",
create_status: createdResponse.status,
};
}
const pipelineId = createdResponse.json.data?.uuid || "";
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, { token });
pipeline = loaded.json.data?.pipeline || null;
created = true;
}
if (!pipeline?.uuid) {
return {
status: "fail",
reason: "Pipeline was not created or resolved.",
};
}
const loaded = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, { token });
if (isApiFailure(loaded) || !loaded.json.data?.pipeline) {
return {
status: "fail",
reason: loaded.json.msg || "Failed to load pipeline.",
get_status: loaded.status,
pipeline_id: pipeline.uuid,
};
}
pipeline = loaded.json.data.pipeline;
const config = pipeline.config && typeof pipeline.config === "object" ? pipeline.config : {};
const ai = config.ai && typeof config.ai === "object" ? config.ai : {};
const runnerConfigs = ai.runner_config && typeof ai.runner_config === "object" ? ai.runner_config : {};
const updatedConfig = {
...config,
ai: {
...ai,
runner: {
...(ai.runner && typeof ai.runner === "object" ? ai.runner : {}),
id: runnerId,
"expire-time": 0,
},
runner_config: {
...runnerConfigs,
[runnerId]: runnerConfig,
},
},
};
const updateResponse = await apiJson(backendUrl, `/api/v1/pipelines/${encodeURIComponent(pipeline.uuid)}`, {
method: "PUT",
token,
body: {
name: pipelineName,
description: "Local QA pipeline for deterministic QA AgentRunner Debug Chat smoke tests.",
emoji: "QA",
config: updatedConfig,
},
});
if (isApiFailure(updateResponse)) {
return {
status: "fail",
reason: updateResponse.json.msg || "Failed to update pipeline.",
update_status: updateResponse.status,
pipeline_id: pipeline.uuid,
};
}
return {
status: "pass",
reason: created ? "QA AgentRunner pipeline created and configured." : "QA AgentRunner pipeline updated.",
pipeline_id: pipeline.uuid,
pipeline_name: pipelineName,
created,
updated: true,
};
}
function isApiFailure(response) {
return response.status >= 400 || (response.json && response.json.code !== undefined && response.json.code !== 0);
}
async function upsertEnvLocal(path, values) {
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
text = "";
}
const lines = text.split(/\r?\n/);
const keys = new Set(Object.keys(values));
const output = [];
for (const line of lines) {
const match = line.match(/^([A-Z][A-Z0-9_]*)=/);
if (match && keys.has(match[1])) {
output.push(`${match[1]}=${values[match[1]]}`);
keys.delete(match[1]);
} else if (line !== "" || output.length > 0) {
output.push(line);
}
}
if (keys.size > 0 && output.length > 0 && output[output.length - 1] !== "") {
output.push("");
}
for (const key of keys) {
output.push(`${key}=${values[key]}`);
}
await writeFile(path, `${output.join("\n").replace(/\n+$/, "")}\n`, "utf8");
}
+496
View File
@@ -0,0 +1,496 @@
#!/usr/bin/env node
import { createServer } from "node:http";
import { mkdir, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import { env, exit } from "node:process";
const args = parseArgs(process.argv.slice(2));
const host = args.host || env.LANGBOT_FAKE_PROVIDER_HOST || "127.0.0.1";
const port = integer(args.port ?? env.LANGBOT_FAKE_PROVIDER_PORT, 0);
const stateFile = args["state-file"] || env.LANGBOT_FAKE_PROVIDER_STATE_FILE || "";
const modelName = env.LANGBOT_FAKE_PROVIDER_MODEL_NAME || "gpt-4o-mini";
const config = {
response_text: env.LANGBOT_FAKE_PROVIDER_RESPONSE_TEXT || "OK",
first_token_delay_ms: integer(env.LANGBOT_FAKE_PROVIDER_FIRST_TOKEN_DELAY_MS, 25),
chunk_delay_ms: integer(env.LANGBOT_FAKE_PROVIDER_CHUNK_DELAY_MS, 10),
chunk_count: integer(env.LANGBOT_FAKE_PROVIDER_CHUNK_COUNT, 0),
fault_status: integer(env.LANGBOT_FAKE_PROVIDER_FAULT_STATUS, 500),
fail_first_n: integer(env.LANGBOT_FAKE_PROVIDER_FAIL_FIRST_N, 0),
fail_every_n: integer(env.LANGBOT_FAKE_PROVIDER_FAIL_EVERY_N, 0),
fail_after_first_chunk: bool(env.LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK, false),
dynamic_response: !/^(0|false|no|off)$/i.test(env.LANGBOT_FAKE_PROVIDER_DYNAMIC_RESPONSE || ""),
request_log_limit: integer(env.LANGBOT_FAKE_PROVIDER_REQUEST_LOG_LIMIT, 500),
};
let requestCount = 0;
const recentRequests = [];
const server = createServer(async (request, response) => {
const startedAt = Date.now();
const startedPerf = performance.now();
let requestRecord = null;
const url = new URL(request.url || "/", `http://${request.headers.host || `${host}:${port}`}`);
try {
if (request.method === "GET" && url.pathname === "/healthz") {
sendJson(response, 200, {
ok: true,
model: modelName,
config,
request_count: requestCount,
recent_request_count: recentRequests.length,
});
return;
}
if (request.method === "GET" && url.pathname === "/__qa/config") {
sendJson(response, 200, {
ok: true,
model: modelName,
config,
request_count: requestCount,
recent_requests: recentRequests,
});
return;
}
if (request.method === "POST" && url.pathname === "/__qa/config") {
const body = await readJson(request);
applyConfig(body.config && typeof body.config === "object" ? body.config : body);
if (body.reset_request_count !== false) resetRequestState();
sendJson(response, 200, {
ok: true,
model: modelName,
config,
request_count: requestCount,
});
return;
}
if (request.method === "POST" && url.pathname === "/__qa/reset") {
resetRequestState();
sendJson(response, 200, {
ok: true,
model: modelName,
config,
request_count: requestCount,
});
return;
}
if (request.method === "GET" && ["/models", "/v1/models"].includes(url.pathname)) {
sendJson(response, 200, {
object: "list",
data: [
{
id: modelName,
object: "model",
created: 1,
owned_by: "langbot-qa",
type: "llm",
},
],
});
return;
}
if (request.method === "POST" && ["/chat/completions", "/v1/chat/completions"].includes(url.pathname)) {
requestCount += 1;
const body = await readJson(request);
const requestId = `chatcmpl-langbot-fake-${requestCount}`;
const shouldFail = requestCount <= config.fail_first_n
|| (config.fail_every_n > 0 && requestCount % config.fail_every_n === 0);
const replyText = responseTextForBody(body);
requestRecord = recordRequest({
id: requestId,
request_number: requestCount,
path: url.pathname,
stream: Boolean(body.stream),
model: body.model || "",
message_count: Array.isArray(body.messages) ? body.messages.length : 0,
should_fail: shouldFail,
status: "running",
http_status: null,
expected_text: replyText,
response_text_preview: previewText(replyText),
started_at: new Date(startedAt).toISOString(),
started_epoch_ms: startedAt,
configured_first_token_delay_ms: config.first_token_delay_ms,
configured_chunk_delay_ms: config.chunk_delay_ms,
configured_chunk_count: config.chunk_count,
});
if (shouldFail) {
await sleep(config.first_token_delay_ms);
sendJson(response, config.fault_status, {
error: {
message: `LangBot fake provider injected HTTP ${config.fault_status}`,
type: "fake_provider_fault",
code: "fake_provider_fault",
},
});
finishRequestRecord(requestRecord, startedPerf, {
status: "http_fault",
http_status: config.fault_status,
});
return;
}
if (body.stream) {
await streamCompletion(response, {
requestId,
model: body.model || modelName,
content: replyText,
failAfterFirstChunk: config.fail_after_first_chunk,
requestRecord,
startedPerf,
});
} else {
await sleep(config.first_token_delay_ms + config.chunk_delay_ms);
sendJson(response, 200, completionPayload({
requestId,
model: body.model || modelName,
content: replyText,
}));
markRequestTiming(requestRecord, "first_chunk", startedPerf);
markRequestTiming(requestRecord, "first_content_chunk", startedPerf);
requestRecord.content_chunk_count = 1;
finishRequestRecord(requestRecord, startedPerf, {
status: "ok",
http_status: 200,
});
}
return;
}
sendJson(response, 404, {
error: {
message: `No fake provider route for ${request.method} ${url.pathname}`,
type: "not_found",
},
});
} catch (error) {
if (requestRecord) {
finishRequestRecord(requestRecord, startedPerf, {
status: "fake_provider_error",
http_status: 500,
error: error instanceof Error ? error.message : String(error),
});
}
sendJson(response, 500, {
error: {
message: error instanceof Error ? error.message : String(error),
type: "fake_provider_error",
},
});
} finally {
const durationMs = Date.now() - startedAt;
if (url.pathname !== "/healthz") {
console.log(JSON.stringify({
at: new Date().toISOString(),
method: request.method,
path: url.pathname,
duration_ms: durationMs,
}));
}
}
});
server.listen(port, host, async () => {
const address = server.address();
const selectedPort = typeof address === "object" && address ? address.port : port;
const url = `http://${host}:${selectedPort}`;
const state = {
status: "ready",
pid: process.pid,
url,
base_url: `${url}/v1`,
model: modelName,
started_at: new Date().toISOString(),
};
if (stateFile) {
const path = resolve(stateFile);
await mkdir(dirname(path), { recursive: true });
await writeFile(path, `${JSON.stringify(state, null, 2)}\n`, "utf8");
}
console.log(JSON.stringify(state));
});
server.on("error", (error) => {
console.error(JSON.stringify({
status: "error",
reason: error instanceof Error ? error.message : String(error),
}));
exit(1);
});
process.on("SIGTERM", () => {
server.close(() => exit(0));
});
function parseArgs(argv) {
const result = {};
for (const item of argv) {
const match = item.match(/^--([^=]+)(?:=(.*))?$/);
if (!match) continue;
result[match[1]] = match[2] ?? "1";
}
return result;
}
function integer(value, fallback) {
const parsed = Number.parseInt(String(value ?? ""), 10);
return Number.isFinite(parsed) && parsed >= 0 ? parsed : fallback;
}
function bool(value, fallback) {
if (value === undefined || value === "") return fallback;
if (/^(1|true|yes|on)$/i.test(String(value))) return true;
if (/^(0|false|no|off)$/i.test(String(value))) return false;
return fallback;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, Math.max(0, ms)));
}
async function readJson(request) {
let text = "";
for await (const chunk of request) text += chunk.toString();
if (!text) return {};
return JSON.parse(text);
}
function sendJson(response, status, payload) {
const text = `${JSON.stringify(payload)}\n`;
response.writeHead(status, {
"content-type": "application/json",
"content-length": Buffer.byteLength(text),
});
response.end(text);
}
function completionPayload({ requestId, model, content }) {
const completionTokens = tokenEstimate(content);
return {
id: requestId,
object: "chat.completion",
created: Math.floor(Date.now() / 1000),
model,
choices: [
{
index: 0,
message: {
role: "assistant",
content,
},
finish_reason: "stop",
},
],
usage: {
prompt_tokens: 8,
completion_tokens: completionTokens,
total_tokens: 8 + completionTokens,
},
};
}
async function streamCompletion(response, {
requestId,
model,
content,
failAfterFirstChunk: failMidStream,
requestRecord,
startedPerf,
}) {
response.writeHead(200, {
"content-type": "text/event-stream; charset=utf-8",
"cache-control": "no-cache",
"connection": "keep-alive",
});
await sleep(config.first_token_delay_ms);
markRequestTiming(requestRecord, "first_chunk", startedPerf);
writeSse(response, {
id: requestId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [{ index: 0, delta: { role: "assistant" }, finish_reason: null }],
});
const chunks = splitContent(content);
for (let index = 0; index < chunks.length; index += 1) {
await sleep(config.chunk_delay_ms);
if (index === 0) markRequestTiming(requestRecord, "first_content_chunk", startedPerf);
requestRecord.content_chunk_count = (requestRecord.content_chunk_count || 0) + 1;
writeSse(response, {
id: requestId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [{ index: 0, delta: { content: chunks[index] }, finish_reason: null }],
});
if (failMidStream && index === 0) {
finishRequestRecord(requestRecord, startedPerf, {
status: "mid_stream_disconnect",
http_status: 200,
});
response.destroy(new Error("LangBot fake provider injected mid-stream disconnect"));
return;
}
}
await sleep(config.chunk_delay_ms);
const completionTokens = tokenEstimate(content);
writeSse(response, {
id: requestId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [{ index: 0, delta: {}, finish_reason: "stop" }],
usage: {
prompt_tokens: 8,
completion_tokens: completionTokens,
total_tokens: 8 + completionTokens,
},
});
response.write("data: [DONE]\n\n");
response.end();
finishRequestRecord(requestRecord, startedPerf, {
status: "ok",
http_status: 200,
});
}
function writeSse(response, payload) {
response.write(`data: ${JSON.stringify(payload)}\n\n`);
}
function splitContent(content) {
const text = String(content);
const requested = config.chunk_count;
if (requested <= 1 || text.length <= 1) return [text];
const chunkSize = Math.max(1, Math.ceil(text.length / requested));
const chunks = [];
for (let index = 0; index < text.length; index += chunkSize) {
chunks.push(text.slice(index, index + chunkSize));
}
return chunks;
}
function tokenEstimate(content) {
return Math.max(1, Math.ceil(String(content || "").length / 4));
}
function responseTextForBody(body) {
if (!config.dynamic_response) {
return config.response_text;
}
const messages = Array.isArray(body.messages) ? body.messages : [];
const lastUser = [...messages].reverse().find((message) => message?.role === "user");
const text = flattenContent(lastUser?.content || "");
const quoted = text.match(/["'“”](.{1,80}?)["'“”]/);
if (quoted?.[1]) return quoted[1].trim();
const exact = text.match(/(?:reply|回复|输出|return)\s+(?:exactly\s+)?([A-Za-z0-9_.:@-]{1,80})/i);
if (exact?.[1]) return exact[1].trim().replace(/[。.!?]+$/, "");
const only = text.match(/只回复\s*([A-Za-z0-9_.:@-]{1,80})/);
if (only?.[1]) return only[1].trim().replace(/[。.!?]+$/, "");
return config.response_text;
}
function flattenContent(content) {
if (typeof content === "string") return content;
if (Array.isArray(content)) {
return content
.map((item) => {
if (typeof item === "string") return item;
if (item && typeof item === "object") return item.text || "";
return "";
})
.join("\n");
}
return "";
}
function recordRequest(entry) {
const item = {
...entry,
at: new Date().toISOString(),
finished_at: null,
finished_epoch_ms: null,
duration_ms: null,
first_chunk_at: null,
first_chunk_epoch_ms: null,
first_chunk_ms: null,
first_content_chunk_at: null,
first_content_chunk_epoch_ms: null,
first_content_chunk_ms: null,
content_chunk_count: 0,
};
recentRequests.push(item);
while (recentRequests.length > config.request_log_limit) recentRequests.shift();
return item;
}
function markRequestTiming(entry, key, startedPerf) {
if (!entry || entry[`${key}_at`]) return;
const now = Date.now();
entry[`${key}_at`] = new Date(now).toISOString();
entry[`${key}_epoch_ms`] = now;
entry[`${key}_ms`] = rounded(performance.now() - startedPerf);
}
function finishRequestRecord(entry, startedPerf, updates = {}) {
if (!entry || entry.finished_at) return;
const now = Date.now();
Object.assign(entry, updates);
entry.finished_at = new Date(now).toISOString();
entry.finished_epoch_ms = now;
entry.duration_ms = rounded(performance.now() - startedPerf);
}
function rounded(value) {
return Number(value.toFixed(3));
}
function previewText(value) {
return String(value || "").slice(0, 120);
}
function resetRequestState() {
requestCount = 0;
recentRequests.length = 0;
}
function applyConfig(updates) {
if (!updates || typeof updates !== "object") return;
assignString(updates, "response_text");
assignNonNegativeInteger(updates, "first_token_delay_ms");
assignNonNegativeInteger(updates, "chunk_delay_ms");
assignNonNegativeInteger(updates, "chunk_count");
assignNonNegativeInteger(updates, "fail_first_n");
assignNonNegativeInteger(updates, "fail_every_n");
assignNonNegativeInteger(updates, "request_log_limit");
if (updates.fault_status !== undefined) {
const parsed = Number.parseInt(String(updates.fault_status), 10);
if (Number.isInteger(parsed) && parsed >= 400 && parsed <= 599) config.fault_status = parsed;
}
assignBoolean(updates, "fail_after_first_chunk");
assignBoolean(updates, "dynamic_response");
}
function assignString(updates, key) {
if (updates[key] !== undefined) config[key] = String(updates[key]);
}
function assignNonNegativeInteger(updates, key) {
if (updates[key] === undefined) return;
const parsed = Number.parseInt(String(updates[key]), 10);
if (Number.isInteger(parsed) && parsed >= 0) config[key] = parsed;
}
function assignBoolean(updates, key) {
if (updates[key] === undefined) return;
config[key] = bool(updates[key], config[key]);
}
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env node
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
apiJson,
ensureEvidence,
evidencePaths,
loadEnvFiles,
resetAndAuthLocalUser,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = env.LBS_CASE_ID || "install-qa-plugin-smoke";
const paths = evidencePaths(caseId);
await loadEnvFiles();
await ensureEvidence(paths);
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const user = env.LANGBOT_E2E_LOGIN_USER || "";
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || "LangBotE2ELocalPass!2026";
const packagePath = resolve(
env.LANGBOT_E2E_PLUGIN_PACKAGE
|| env.LANGBOT_QA_PLUGIN_SMOKE_PACKAGE
|| "skills/langbot-testing/fixtures/plugins/qa-plugin-smoke/dist/qa-plugin-smoke-0.1.0.lbpkg",
);
const expectedPluginId = env.LANGBOT_E2E_EXPECTED_PLUGIN_ID || "qa/plugin-smoke";
const expectedTool = env.LANGBOT_E2E_EXPECTED_TOOL || (expectedPluginId === "qa/plugin-smoke" ? "qa_plugin_echo" : "");
const expectedRunnerId = env.LANGBOT_E2E_EXPECTED_RUNNER_ID || "";
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
backend_url: backendUrl,
package_path: packagePath,
package_preview: null,
task_id: null,
task: null,
plugin_present_before: false,
plugin_present_after: false,
tool_names: [],
runner_ids: [],
evidence: {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic", "filesystem"],
};
try {
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
if (!user) throw new Error("LANGBOT_E2E_LOGIN_USER is required.");
const bytes = await readFile(packagePath);
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
result.package_preview = await previewPackage(backendUrl, auth.token, bytes, packagePath);
const metadata = result.package_preview.metadata || {};
if (`${metadata.author}/${metadata.name}` !== expectedPluginId) {
throw new Error(`Fixture package metadata is ${metadata.author}/${metadata.name}, expected ${expectedPluginId}.`);
}
result.plugin_present_before = await hasPlugin(backendUrl, auth.token);
if (!result.plugin_present_before) {
const form = new FormData();
form.set("file", new Blob([bytes]), packagePath.split("/").pop());
const response = await fetch(`${backendUrl.replace(/\/$/, "")}/api/v1/plugins/install/local`, {
method: "POST",
headers: { Authorization: `Bearer ${auth.token}` },
body: form,
});
const json = await response.json().catch(() => ({}));
if (response.status >= 400 || json.code !== 0) {
throw new Error(json.msg || `Plugin install request failed with HTTP ${response.status}.`);
}
result.task_id = json.data?.task_id ?? null;
if (!result.task_id) throw new Error("Plugin install response did not include task_id.");
result.task = await waitForTask(backendUrl, auth.token, result.task_id);
if (!isTaskComplete(result.task)) {
throw new Error(`Plugin install task did not complete successfully: ${JSON.stringify(result.task)}`);
}
}
await sleep(1000);
result.plugin_present_after = await hasPlugin(backendUrl, auth.token);
if (!result.plugin_present_after) throw new Error(`${expectedPluginId} is not listed by /api/v1/plugins after install.`);
if (expectedTool) {
result.tool_names = await listToolNames(backendUrl, auth.token);
if (!result.tool_names.includes(expectedTool)) {
throw new Error(`${expectedTool} is not listed by /api/v1/tools after install.`);
}
}
if (expectedRunnerId) {
result.runner_ids = await listRunnerIds(backendUrl, auth.token);
if (!result.runner_ids.includes(expectedRunnerId)) {
throw new Error(`${expectedRunnerId} is not listed by /api/v1/pipelines/_/metadata after install.`);
}
}
result.status = "pass";
result.reason = `${expectedPluginId} is installed.`;
} catch (error) {
result.status = "fail";
result.reason = error.message;
} finally {
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : 1);
async function hasPlugin(backendUrl, token) {
const response = await apiJson(backendUrl, "/api/v1/plugins", { token });
const plugins = response.json.data?.plugins || [];
return plugins.some((plugin) => {
const metadata = plugin.manifest?.manifest?.metadata || plugin.manifest?.metadata || plugin.metadata || {};
return `${metadata.author}/${metadata.name}` === expectedPluginId;
});
}
async function previewPackage(backendUrl, token, bytes, packagePath) {
const form = new FormData();
form.set("file", new Blob([bytes]), packagePath.split("/").pop());
const response = await fetch(`${backendUrl.replace(/\/$/, "")}/api/v1/plugins/install/local/preview`, {
method: "POST",
headers: { Authorization: `Bearer ${token}` },
body: form,
});
const json = await response.json().catch(() => ({}));
if (response.status >= 400 || json.code !== 0) {
throw new Error(json.msg || `Plugin package preview failed with HTTP ${response.status}.`);
}
return {
metadata: json.data?.metadata || {},
component_types: json.data?.component_types || [],
file_count: json.data?.file_count ?? null,
};
}
async function listToolNames(backendUrl, token) {
const response = await apiJson(backendUrl, "/api/v1/tools", { token });
return (response.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
}
async function listRunnerIds(backendUrl, token) {
const response = await apiJson(backendUrl, "/api/v1/pipelines/_/metadata", { token });
const configs = response.json.data?.configs || [];
return configs
.flatMap((section) => section.stages || [])
.flatMap((stage) => stage.config || [])
.filter((item) => item.name === "id")
.flatMap((item) => item.options || [])
.map((option) => option.name || option.value || option.id || "")
.filter(Boolean)
.sort();
}
async function waitForTask(backendUrl, token, taskId) {
const deadline = Date.now() + Number(env.LANGBOT_PLUGIN_INSTALL_TIMEOUT_MS || 120000);
let last = null;
while (Date.now() < deadline) {
const response = await apiJson(backendUrl, `/api/v1/system/tasks/${encodeURIComponent(taskId)}`, { token });
last = response.json.data || response.json;
if (isTaskComplete(last) || isTaskFailed(last)) return last;
await sleep(1000);
}
return last;
}
function isTaskComplete(task) {
const status = String(task?.status || task?.state || "").toLowerCase();
const runtimeStatus = String(task?.runtime?.status || task?.runtime?.state || "").toLowerCase();
return ["done", "completed", "success", "succeeded", "finished"].includes(status)
|| ["done", "completed", "success", "succeeded", "finished"].includes(runtimeStatus)
|| task?.done === true
|| task?.completed === true
|| (task?.runtime?.done === true && !task?.runtime?.exception);
}
function isTaskFailed(task) {
const status = String(task?.status || task?.state || "").toLowerCase();
const runtimeStatus = String(task?.runtime?.status || task?.runtime?.state || "").toLowerCase();
return ["failed", "error", "cancelled", "canceled"].includes(status)
|| ["failed", "error", "cancelled", "canceled"].includes(runtimeStatus)
|| task?.failed === true
|| Boolean(task?.error)
|| Boolean(task?.runtime?.exception);
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env node
import {
bodyText,
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
isLoginUrl,
localIsoWithOffset,
safeScreenshot,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = process.env.LBS_CASE_ID || "langrag-kb-retrieve";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const startedAt = new Date();
const frontendUrl = process.env.LANGBOT_FRONTEND_URL || "";
const backendUrl = process.env.LANGBOT_BACKEND_URL || "";
const kbUuid = process.env.LANGBOT_LOCAL_AGENT_RAG_KB_UUID || process.env.LANGBOT_RAG_KB_UUID || "";
const query = process.env.LANGBOT_E2E_RETRIEVE_QUERY || "What is the local agent runner retrieval sentinel?";
const expectedText = process.env.LANGBOT_E2E_EXPECTED_TEXT || "azalea-cobalt-7421";
let browser;
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
url: "",
kb_uuid: kbUuid,
query,
expected_text: expectedText,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "network", "api_diagnostic"],
};
try {
if (!frontendUrl) throw new Error("LANGBOT_FRONTEND_URL is not configured.");
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
if (!kbUuid) throw new Error("LANGBOT_LOCAL_AGENT_RAG_KB_UUID or LANGBOT_RAG_KB_UUID is required.");
browser = await createBrowser(paths);
const { page } = browser;
await page.goto(`${frontendUrl.replace(/\/$/, "")}/home/knowledge`, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
result.url = page.url();
const text = await bodyText(page);
if (isLoginUrl(page.url()) || /登录|Login|Sign in/i.test(text)) {
result.status = "blocked";
result.reason = "Browser profile is not authenticated for LANGBOT_FRONTEND_URL.";
} else if (!/Knowledge|知识库|qa-local-agent-rag/i.test(text)) {
result.status = "fail";
result.reason = "Knowledge page opened, but no Knowledge UI signal or QA KB name was visible.";
} else {
const retrieve = await page.evaluate(async ({ backendUrl, kbUuid, query }) => {
const token = localStorage.getItem("token");
if (!token) {
return { status: "blocked", authenticated: false, reason: "Browser profile has no localStorage token." };
}
const response = await fetch(`${backendUrl}/api/v1/knowledge/bases/${encodeURIComponent(kbUuid)}/retrieve`, {
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ query }),
});
const json = await response.json().catch(() => ({}));
return {
status: response.status >= 400 ? "fail" : "ready",
authenticated: true,
http_status: response.status,
code: json.code ?? null,
msg: json.msg || "",
results: json.data?.results || [],
};
}, { backendUrl, kbUuid, query });
result.retrieve = {
...retrieve,
results: Array.isArray(retrieve.results)
? retrieve.results.map((item) => ({
score: item.score ?? item.distance ?? null,
text: String(item.text || item.content || "").slice(0, 500),
metadata: item.metadata || {},
}))
: [],
};
const resultText = JSON.stringify(result.retrieve.results || []);
if (retrieve.status === "blocked") {
result.status = "blocked";
result.reason = retrieve.reason || "Retrieve API blocked.";
} else if (retrieve.status === "fail") {
result.status = "fail";
result.reason = retrieve.msg || "Retrieve API failed.";
} else if (!resultText.includes(expectedText)) {
result.status = "fail";
result.reason = `Retrieve results did not contain expected text: ${expectedText}`;
} else {
result.status = "pass";
result.reason = `Knowledge retrieve returned expected sentinel: ${expectedText}`;
}
}
await safeScreenshot(page, paths.screenshot);
} catch (error) {
result.status = /Playwright is not installed|not configured|required/.test(error.message) ? "env_issue" : "fail";
result.reason = error.message;
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
+416
View File
@@ -0,0 +1,416 @@
import {
bodyText,
clickFirstVisible,
countOccurrences,
gotoFrontend,
isLoginUrl,
} from "./langbot-e2e.mjs";
export const DEBUG_CHAT_FAILURE_SIGNALS = [
"Agent runner temporarily unavailable",
"All models failed during streaming setup",
"调用超时",
"超时",
];
export function minExpectedOccurrences(beforeText, expectedText, prompt) {
const beforeCount = countOccurrences(beforeText, expectedText);
return beforeCount + (String(prompt).includes(expectedText) ? 2 : 1);
}
export function latestExpectedLeafMatches(latestExpectedLeaf, prompt) {
return Boolean(latestExpectedLeaf)
&& latestExpectedLeaf !== prompt
&& !String(latestExpectedLeaf).includes(prompt);
}
export function findNewFailureSignal(beforeText, afterText, failureSignals = DEBUG_CHAT_FAILURE_SIGNALS) {
return failureSignals.find((signal) => countOccurrences(afterText, signal) > countOccurrences(beforeText, signal)) || "";
}
function findFailureSignalInText(text, failureSignals = DEBUG_CHAT_FAILURE_SIGNALS) {
return failureSignals.find((signal) => String(text || "").includes(signal)) || "";
}
function countExpectedInMessages(messages, expectedText) {
return messages
.filter((message) => message.role === "assistant")
.reduce((count, message) => count + countOccurrences(message.text, expectedText), 0);
}
function debugChatInput(page) {
return page
.locator('input[placeholder*="message"], input[placeholder*="消息"], textarea[placeholder*="message"], textarea[placeholder*="消息"]')
.last();
}
async function clickDebugChatTab(page) {
const tabByRole = page.getByRole("tab", { name: /Debug Chat|调试聊天|调试对话|Debug|调试/i }).first();
if (await tabByRole.isVisible({ timeout: 3_000 }).catch(() => false)) {
await tabByRole.click();
return true;
}
const tabBySelector = page.locator('[role="tab"]').filter({ hasText: /Debug Chat|调试聊天|调试对话|Debug|调试/i }).first();
if (await tabBySelector.isVisible({ timeout: 2_000 }).catch(() => false)) {
await tabBySelector.click();
return true;
}
return Boolean(await clickFirstVisible(page, ["Debug Chat", "调试聊天", "调试对话"], 2_000));
}
async function waitForDebugChatReady(page, timeout = 20_000) {
const input = debugChatInput(page);
const visible = await input.isVisible({ timeout }).catch(() => false);
if (!visible) {
return {
ready: false,
reason: "Debug Chat tab was clicked, but the Debug Chat input did not become visible.",
};
}
const enabled = await input.isEnabled({ timeout }).catch(() => false);
if (!enabled) {
return {
ready: false,
reason: "Debug Chat input is visible but disabled; WebSocket may not be connected.",
};
}
return { ready: true, reason: "" };
}
export function classifyDebugChatResult({
beforeText,
afterText,
expectedText,
prompt,
latestExpectedLeaf,
latestFailureLeaf,
beforeMessages = null,
afterMessages = null,
latestAssistantText = "",
failureSignals = DEBUG_CHAT_FAILURE_SIGNALS,
}) {
const minExpectedCount = minExpectedOccurrences(beforeText, expectedText, prompt);
const finalCount = countOccurrences(afterText, expectedText);
const failureText = findNewFailureSignal(beforeText, afterText, failureSignals);
const promptContainsExpected = String(prompt).includes(expectedText);
const hasMessageEvidence = Array.isArray(beforeMessages) && Array.isArray(afterMessages);
const beforeAssistantExpectedCount = hasMessageEvidence
? countExpectedInMessages(beforeMessages, expectedText)
: null;
const afterAssistantExpectedCount = hasMessageEvidence
? countExpectedInMessages(afterMessages, expectedText)
: null;
const assistantExpectedIncreased = hasMessageEvidence
? afterAssistantExpectedCount > beforeAssistantExpectedCount
: false;
if (hasMessageEvidence) {
const latestAssistantFailure = findFailureSignalInText(latestAssistantText, failureSignals);
if (latestAssistantFailure) {
return {
status: "fail",
reason: `Debug Chat displayed a known failure signal in the latest assistant message: ${latestAssistantFailure}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
failure_signal: latestAssistantFailure,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
};
}
if (assistantExpectedIncreased && String(latestAssistantText).includes(expectedText)) {
return {
status: "pass",
reason: `Expected text appeared in a new assistant message: ${expectedText}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
};
}
if (failureText) {
return {
status: "fail",
reason: `Debug Chat displayed a known failure signal: ${failureText}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
failure_signal: failureText,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
};
}
return {
status: "fail",
reason: `Expected text did not appear in a new assistant message. Expected assistant occurrences to increase above ${beforeAssistantExpectedCount}, saw ${afterAssistantExpectedCount}.`,
min_expected_count: minExpectedCount,
final_count: finalCount,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
};
}
if (failureText) {
return {
status: "fail",
reason: `Debug Chat displayed a known failure signal: ${failureText}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
failure_signal: failureText,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
};
}
if (latestExpectedLeafMatches(latestExpectedLeaf, prompt) && finalCount >= minExpectedCount) {
return {
status: "pass",
reason: `Expected text appeared in the latest visible response leaf: ${expectedText}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
};
}
if (!promptContainsExpected && finalCount >= minExpectedCount) {
return {
status: "pass",
reason: `Expected text appeared enough times for user prompt plus bot response: ${expectedText}`,
min_expected_count: minExpectedCount,
final_count: finalCount,
};
}
return {
status: "fail",
reason: `Bot response did not appear. Expected ${minExpectedCount} occurrences of ${expectedText}, saw ${finalCount}.`,
min_expected_count: minExpectedCount,
final_count: finalCount,
};
}
export async function openPipelineDebugChat(page, { pipelineUrl, pipelineName, envHint = "LANGBOT_PIPELINE_URL or LANGBOT_PIPELINE_NAME" }) {
if (pipelineUrl) {
await page.goto(pipelineUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
} else {
if (!pipelineName) {
return {
opened: false,
status: "blocked",
reason: `Set ${envHint} before running pipeline-debug-chat automation.`,
};
}
await gotoFrontend(page);
if (isLoginUrl(page.url())) {
return {
opened: false,
status: "blocked",
reason: "Browser profile is not authenticated for LANGBOT_FRONTEND_URL.",
};
}
const clickedPipelines = await clickFirstVisible(page, ["Pipelines", "流水线"], 4_000);
if (!clickedPipelines) {
return { opened: false, status: "fail", reason: "Could not find Pipelines navigation." };
}
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const clickedPipeline = await clickFirstVisible(page, [pipelineName], 5_000);
if (!clickedPipeline) {
return { opened: false, status: "blocked", reason: `Could not find pipeline named ${pipelineName}.` };
}
}
if (isLoginUrl(page.url())) {
return {
opened: false,
status: "blocked",
reason: "Browser profile is not authenticated for LANGBOT_FRONTEND_URL.",
};
}
const clickedDebug = await clickDebugChatTab(page);
if (!clickedDebug) {
return { opened: false, status: "fail", reason: "Could not find the Debug Chat tab." };
}
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const ready = await waitForDebugChatReady(page);
if (!ready.ready) {
return { opened: false, status: "fail", reason: ready.reason };
}
return { opened: true };
}
export async function latestVisibleLeafText(page, needles) {
return await page.evaluate((items) => {
const isVisible = (element) => {
const style = window.getComputedStyle(element);
const rect = element.getBoundingClientRect();
return style.visibility !== "hidden"
&& style.display !== "none"
&& rect.width > 0
&& rect.height > 0;
};
const leaves = [];
for (const element of document.body.querySelectorAll("*")) {
if (!isVisible(element)) continue;
const text = element.innerText?.trim();
if (!text || text.length > 4000) continue;
const visibleChildHasText = Array.from(element.children).some((child) => (
isVisible(child) && child.innerText?.trim()
));
if (visibleChildHasText) continue;
if (!items.some((needle) => text.includes(needle))) continue;
leaves.push(text);
}
return leaves.at(-1) || "";
}, needles);
}
export async function visibleDebugChatMessages(page) {
return await page.evaluate(() => {
const isVisible = (element) => {
const style = window.getComputedStyle(element);
const rect = element.getBoundingClientRect();
return style.visibility !== "hidden"
&& style.display !== "none"
&& rect.width > 0
&& rect.height > 0;
};
const classText = (element) => String(element.getAttribute("class") || "");
return Array.from(document.querySelectorAll("div.max-w-3xl"))
.filter((element) => isVisible(element))
.map((element) => {
const row = element.parentElement;
const text = element.innerText?.trim() || "";
const isUser = classText(element).includes("user-message-bubble")
|| classText(row).includes("justify-end");
return {
role: isUser ? "user" : "assistant",
text,
};
})
.filter((message) => message.text);
});
}
export async function waitForExpectedDebugChatText(page, { expectedText, minExpectedCount, timeoutMs }) {
await page.waitForFunction(
({ expected, min }) => {
return document.body.innerText.split(expected).length - 1 >= min;
},
{ expected: expectedText, min: minExpectedCount },
{ timeout: timeoutMs },
).catch(() => {});
}
export async function waitForDebugChatTextStable(page, { timeoutMs = 5_000, quietMs = 750 } = {}) {
const startedAt = Date.now();
let lastText = await bodyText(page);
let stableSince = Date.now();
while (Date.now() - startedAt < timeoutMs) {
await page.waitForTimeout(250);
const currentText = await bodyText(page);
if (currentText !== lastText) {
lastText = currentText;
stableSince = Date.now();
continue;
}
if (Date.now() - stableSince >= quietMs) return;
}
}
export async function attachDebugChatImage(page, imagePath) {
if (!imagePath) return { status: "not_required", reason: "" };
const input = page.locator('input[type="file"][accept*="image"], input[type="file"]').first();
if (!await input.count()) {
return { status: "fail", reason: "Could not find a Debug Chat image upload input." };
}
await input.setInputFiles(imagePath);
await page.locator("img").last().waitFor({ state: "visible", timeout: 10_000 }).catch(() => {});
return { status: "ready", reason: `Attached image fixture: ${imagePath}` };
}
export async function sendDebugChatPrompt(page, prompt, imagePath = "") {
const imageResult = await attachDebugChatImage(page, imagePath);
if (imageResult.status === "fail") return imageResult;
const input = debugChatInput(page);
const inputVisible = await input.isVisible({ timeout: 5_000 }).catch(() => false);
const inputEnabled = inputVisible && await input.isEnabled({ timeout: 10_000 }).catch(() => false);
if (!inputVisible || !inputEnabled) return false;
await input.fill(prompt).catch(async () => {
await input.click();
await input.pressSequentially(prompt);
});
const clickedSend = await clickFirstVisible(page, ["Send", "发送", "提交"], 1_500);
if (!clickedSend) await page.keyboard.press("Enter");
await page.getByText(prompt, { exact: false }).last().waitFor({ state: "visible", timeout: 10_000 }).catch(() => {});
return true;
}
export async function runDebugChatPrompt(page, { prompt, expectedText, responseTimeoutMs, imagePath = "", failureSignals = DEBUG_CHAT_FAILURE_SIGNALS }) {
const beforeText = await bodyText(page);
const beforeMessages = await visibleDebugChatMessages(page);
const minExpectedCount = minExpectedOccurrences(beforeText, expectedText, prompt);
const sent = await sendDebugChatPrompt(page, prompt, imagePath);
if (sent !== true) {
if (sent && typeof sent === "object" && typeof sent.reason === "string") return sent;
return { status: "fail", reason: "Could not find a Debug Chat text input." };
}
await waitForExpectedDebugChatText(page, {
expectedText,
minExpectedCount,
prompt,
timeoutMs: responseTimeoutMs,
});
await waitForDebugChatTextStable(page);
const afterText = await bodyText(page);
const afterMessages = await visibleDebugChatMessages(page);
const latestAssistantText = afterMessages.filter((message) => message.role === "assistant").at(-1)?.text || "";
const latestExpectedLeaf = await latestVisibleLeafText(page, [expectedText]);
const failureText = findNewFailureSignal(beforeText, afterText, failureSignals);
const latestFailureLeaf = failureText ? await latestVisibleLeafText(page, [failureText]) : "";
return classifyDebugChatResult({
beforeText,
afterText,
expectedText,
prompt,
latestExpectedLeaf,
latestFailureLeaf,
beforeMessages,
afterMessages,
latestAssistantText,
failureSignals,
});
}
export async function setDebugChatStreamOutput(page, desired) {
if (desired === null || desired === undefined) return { status: "not_required", reason: "" };
const streamSwitch = page.locator('[role="switch"]').first();
if (!await streamSwitch.isVisible({ timeout: 5_000 }).catch(() => false)) {
return { status: "blocked", reason: "Debug Chat stream switch was not visible." };
}
if (!await streamSwitch.isEnabled({ timeout: 10_000 }).catch(() => false)) {
return { status: "blocked", reason: "Debug Chat stream switch was visible but disabled." };
}
const checked = (await streamSwitch.getAttribute("aria-checked").catch(() => null)) === "true";
if (checked !== desired) {
await streamSwitch.click();
await page.waitForFunction(
({ selector, expected }) => document.querySelector(selector)?.getAttribute("aria-checked") === String(expected),
{ selector: '[role="switch"]', expected: desired },
{ timeout: 5_000 },
).catch(() => {});
}
const finalChecked = (await streamSwitch.getAttribute("aria-checked").catch(() => null)) === "true";
if (finalChecked !== desired) {
return {
status: "fail",
reason: `Debug Chat stream switch did not reach requested state: ${desired ? "on" : "off"}.`,
};
}
return { status: "ready", reason: `Debug Chat stream switch is ${desired ? "on" : "off"}.` };
}
+342
View File
@@ -0,0 +1,342 @@
import { appendFile, mkdir, readFile, stat, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { env } from "node:process";
const secretRe = /(?:authorization|bearer|token|secret|password|api[_-]?key|jwt|oauth)\s*[:=]\s*["']?[^"',\s]+/gi;
export function redact(text) {
return String(text ?? "")
.replace(secretRe, (match) => match.replace(/[:=]\s*["']?.*$/, "=[redacted]"))
.replace(/\bbearer\s+[A-Za-z0-9._~+/=-]{8,}/gi, "Bearer [redacted]")
.replace(/\bsk-[A-Za-z0-9_-]{6,}\b/g, "[redacted]");
}
export function timestampSlug(date = new Date()) {
return date.toISOString().replace(/\.\d{3}Z$/, "Z").replace(/[^0-9A-Za-z]+/g, "-").replace(/^-|-$/g, "");
}
export function localIsoWithOffset(date = new Date()) {
const offsetMinutes = -date.getTimezoneOffset();
const sign = offsetMinutes >= 0 ? "+" : "-";
const absolute = Math.abs(offsetMinutes);
const pad = (value) => String(value).padStart(2, "0");
const yyyy = date.getFullYear();
const mm = pad(date.getMonth() + 1);
const dd = pad(date.getDate());
const hh = pad(date.getHours());
const mi = pad(date.getMinutes());
const ss = pad(date.getSeconds());
const ms = String(date.getMilliseconds()).padStart(3, "0");
return `${yyyy}-${mm}-${dd}T${hh}:${mi}:${ss}.${ms}${sign}${pad(Math.floor(absolute / 60))}:${pad(absolute % 60)}`;
}
export function evidencePaths(caseId) {
const runId = env.LBS_RUN_ID || `${timestampSlug()}-${caseId}`;
const evidenceDir = resolve(env.LBS_EVIDENCE_DIR || join("reports", "evidence", runId));
return {
runId,
evidenceDir,
consoleLog: join(evidenceDir, "console.log"),
networkLog: join(evidenceDir, "network.log"),
screenshot: join(evidenceDir, "screenshot.png"),
automationResultJson: join(evidenceDir, "automation-result.json"),
resultJson: join(evidenceDir, "result.json"),
};
}
export async function ensureEvidence(paths) {
await mkdir(paths.evidenceDir, { recursive: true });
await appendFile(paths.consoleLog, "", "utf8");
await appendFile(paths.networkLog, "", "utf8");
}
export async function pathExists(path) {
try {
await stat(path);
return true;
} catch {
return false;
}
}
export async function appendLine(path, line) {
await appendFile(path, `[${localIsoWithOffset()}] ${redact(line)}\n`, "utf8");
}
export async function writeResult(paths, result) {
const text = `${JSON.stringify(result, null, 2)}\n`;
if (paths.automationResultJson) await writeFile(paths.automationResultJson, text, "utf8");
if (paths.resultJson && paths.resultJson !== paths.automationResultJson) {
await writeFile(paths.resultJson, text, "utf8");
}
}
export async function loadEnvFiles(paths = ["skills/.env", "skills/.env.local"]) {
const processEnvKeys = new Set(Object.keys(env));
for (const path of paths) {
let text = "";
try {
text = await readFile(path, "utf8");
} catch {
continue;
}
for (const line of text.split(/\r?\n/)) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const equals = trimmed.indexOf("=");
if (equals <= 0) continue;
const key = trimmed.slice(0, equals).trim();
const value = trimmed.slice(equals + 1).trim().replace(/^["']|["']$/g, "");
if (!processEnvKeys.has(key)) env[key] = value;
}
}
}
export async function readRecoveryKey(repo = env.LANGBOT_REPO || "../LangBot") {
const configPath = resolve(repo, "data/config.yaml");
const config = await readFile(configPath, "utf8");
const match = config.match(/^\s*recovery_key:\s*['"]?([^'"\s#]+)['"]?\s*$/m);
return match?.[1] || "";
}
export async function apiJson(backendUrl, path, { method = "GET", token = "", body } = {}) {
const headers = { "Content-Type": "application/json" };
if (token) headers.Authorization = `Bearer ${token}`;
const response = await fetch(`${backendUrl.replace(/\/$/, "")}${path}`, {
method,
headers,
body: body === undefined ? undefined : JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
}
export async function checkBackendToken(backendUrl, token) {
if (!token) {
return { authenticated: false, http_status: 0, code: null, reason: "No token." };
}
const response = await apiJson(backendUrl, "/api/v1/user/check-token", { token });
const code = response.json.code ?? null;
const authenticated = response.status < 400 && code === 0;
return {
authenticated,
http_status: response.status,
code,
reason: authenticated ? "Token accepted by backend." : response.json.msg || "Backend rejected token.",
};
}
export async function resetAndAuthLocalUser({ backendUrl, user, password, recoveryKey = "" }) {
const key = recoveryKey || await readRecoveryKey();
if (!key) throw new Error("Could not read recovery_key from LangBot config.");
const reset = await apiJson(backendUrl, "/api/v1/user/reset-password", {
method: "POST",
body: {
user,
recovery_key: key,
new_password: password,
},
});
if (reset.status >= 400 || reset.json.code !== 0) {
throw new Error(reset.json.msg || `Password reset failed with HTTP ${reset.status}.`);
}
const auth = await apiJson(backendUrl, "/api/v1/user/auth", {
method: "POST",
body: { user, password },
});
const token = auth.json.data?.token || "";
if (auth.status >= 400 || auth.json.code !== 0 || !token) {
throw new Error(auth.json.msg || `Auth failed with HTTP ${auth.status}.`);
}
const check = await checkBackendToken(backendUrl, token);
if (!check.authenticated) {
throw new Error(check.reason || "Authenticated token failed backend token check.");
}
return { token, check };
}
export async function setBrowserToken(page, frontendUrl, token) {
await page.addInitScript((value) => {
localStorage.setItem("token", value);
}, token);
await page.goto(frontendUrl, { waitUntil: "domcontentloaded" });
await page.evaluate((value) => localStorage.setItem("token", value), token);
}
export async function verifyBrowserToken(page, backendUrl) {
return await page.evaluate(async (baseUrl) => {
const token = localStorage.getItem("token");
if (!token) {
return { authenticated: false, http_status: 0, code: null, reason: "No localStorage token." };
}
try {
const response = await fetch(`${baseUrl.replace(/\/$/, "")}/api/v1/user/check-token`, {
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
});
const json = await response.json().catch(() => ({}));
const code = json.code ?? null;
const authenticated = response.status < 400 && code === 0;
return {
authenticated,
http_status: response.status,
code,
reason: authenticated ? "Token accepted by backend." : json.msg || "Backend rejected token.",
};
} catch (error) {
return {
authenticated: false,
http_status: 0,
code: null,
reason: error.message,
};
}
}, backendUrl);
}
export function exitCode(status) {
if (status === "pass") return 0;
if (status === "blocked" || status === "env_issue") return 2;
return 1;
}
export async function loadPlaywright() {
try {
return await import("playwright");
} catch {
throw new Error(
"Playwright is not installed. Install it in this repo with `npm install --save-dev playwright`, then run `npx playwright install chromium`.",
);
}
}
export async function createBrowser(paths) {
const { chromium } = await loadPlaywright();
const headed = env.LBS_HEADED === "1";
const launchOptions = {
headless: !headed,
};
if (env.LANGBOT_CHROMIUM_EXECUTABLE && await pathExists(env.LANGBOT_CHROMIUM_EXECUTABLE)) {
launchOptions.executablePath = env.LANGBOT_CHROMIUM_EXECUTABLE;
}
let browser;
let context;
if (env.LANGBOT_BROWSER_PROFILE) {
context = await chromium.launchPersistentContext(resolve(env.LANGBOT_BROWSER_PROFILE), {
...launchOptions,
viewport: { width: 1440, height: 960 },
});
} else {
browser = await chromium.launch(launchOptions);
context = await browser.newContext({ viewport: { width: 1440, height: 960 } });
}
const page = context.pages()[0] || await context.newPage();
page.on("console", (message) => {
appendLine(paths.consoleLog, `[${message.type()}] ${message.text()}`).catch(() => {});
});
page.on("pageerror", (error) => {
appendLine(paths.consoleLog, `[pageerror] ${error.message}`).catch(() => {});
});
page.on("requestfailed", (request) => {
appendLine(paths.networkLog, `[requestfailed] ${request.method()} ${request.url()} ${request.failure()?.errorText ?? ""}`).catch(() => {});
});
page.on("response", (response) => {
if (response.status() < 400) return;
appendLine(paths.networkLog, `[response] ${response.status()} ${response.url()}`).catch(() => {});
});
return {
page,
context,
async close() {
await context.close();
if (browser) await browser.close();
},
};
}
export async function safeScreenshot(page, path) {
try {
await page.screenshot({ path, fullPage: true });
} catch {
// Screenshot evidence is useful, but a screenshot failure should not hide the real test result.
}
}
export async function gotoFrontend(page) {
const frontendUrl = env.LANGBOT_FRONTEND_URL;
if (!frontendUrl) {
throw new Error("LANGBOT_FRONTEND_URL is not configured.");
}
await page.goto(frontendUrl, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
}
export function isLoginUrl(url) {
return /\/login(?:[/?#]|$)/.test(url);
}
export async function bodyText(page) {
return await page.locator("body").innerText({ timeout: 5_000 }).catch(() => "");
}
export function countOccurrences(haystack, needle) {
if (!needle) return 0;
return String(haystack).split(needle).length - 1;
}
export async function clickFirstVisible(page, labels, timeout = 2_000) {
for (const label of labels) {
const roleButton = page.getByRole("button", { name: label }).first();
if (await roleButton.isVisible({ timeout }).catch(() => false)) {
await roleButton.click();
return label;
}
const roleLink = page.getByRole("link", { name: label }).first();
if (await roleLink.isVisible({ timeout }).catch(() => false)) {
await roleLink.click();
return label;
}
const text = page.getByText(label, { exact: false }).first();
if (await text.isVisible({ timeout }).catch(() => false)) {
await text.click();
return label;
}
}
return null;
}
export async function fillFirstTextInput(page, value) {
const candidates = [
page.getByRole("textbox").last(),
page.locator("textarea").last(),
page.locator("[contenteditable=true]").last(),
page.locator("input[type=text]").last(),
];
for (const locator of candidates) {
if (!await locator.isVisible({ timeout: 2_000 }).catch(() => false)) continue;
await locator.fill(value).catch(async () => {
await locator.click();
await locator.pressSequentially(value);
});
return true;
}
return false;
}
export async function waitForVisibleText(page, text, timeout = 20_000) {
await page.getByText(text, { exact: false }).last().waitFor({ state: "visible", timeout });
}
+565
View File
@@ -0,0 +1,565 @@
#!/usr/bin/env node
import { writeFile } from "node:fs/promises";
import { env } from "node:process";
import {
DEBUG_CHAT_FAILURE_SIGNALS,
openPipelineDebugChat,
setDebugChatStreamOutput,
visibleDebugChatMessages,
waitForDebugChatTextStable,
} from "./lib/debug-chat.mjs";
import {
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
localIsoWithOffset,
loadEnvFiles,
pathExists,
safeScreenshot,
writeResult,
} from "./lib/langbot-e2e.mjs";
await loadEnvFiles();
const caseId = env.LBS_CASE_ID || "local-agent-steering-debug-chat";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const backendUrl = (env.LANGBOT_BACKEND_URL || "").replace(/\/$/, "");
const pipelineUrl = env.LANGBOT_E2E_PIPELINE_URL || env.LANGBOT_LOCAL_AGENT_PIPELINE_URL || env.LANGBOT_PIPELINE_URL || "";
const pipelineName = env.LANGBOT_E2E_PIPELINE_NAME || env.LANGBOT_LOCAL_AGENT_PIPELINE_NAME || env.LANGBOT_PIPELINE_NAME || "";
const expectedRunnerId = env.LANGBOT_E2E_EXPECTED_RUNNER_ID || "plugin:langbot/local-agent/default";
const expectedText = env.LANGBOT_E2E_EXPECTED_TEXT || "qa_steering_sentinel_6194";
const responseTimeoutMs = positiveInt(env.LANGBOT_E2E_RESPONSE_TIMEOUT_MS, 240000);
const followupDelayMs = 1000;
const followupEnabledTimeoutMs = 1500;
const firstPrompt = env.LANGBOT_E2E_PROMPT || [
"You are running the LangBot steering E2E test.",
"First call the qa_plugin_sleep tool with seconds=8 and text=steering-e2e-anchor.",
"Do not answer before the tool result is available.",
"After the tool returns, answer the latest user follow-up.",
"If no follow-up was injected, reply only STEERING_NO_FOLLOWUP.",
].join(" ");
const followupPrompt = [
"This is a steering follow-up sent while the first tool call is still active.",
`Return only ${expectedText}.`,
].join(" ");
const pipelineConfigDiagnosticPath = `${paths.evidenceDir}/pipeline-config-diagnostic.json`;
const debugChatResetDiagnosticPath = `${paths.evidenceDir}/debug-chat-reset-diagnostic.json`;
const toolDiagnosticPath = `${paths.evidenceDir}/tool-diagnostic.json`;
let browser;
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
status: "fail",
reason: "",
started_at: new Date().toISOString(),
started_at_local: localIsoWithOffset(new Date()),
url: "",
backend_url: backendUrl,
pipeline_url: pipelineUrl,
pipeline_name: pipelineName,
expected_runner_id: expectedRunnerId,
first_prompt: firstPrompt,
followup_prompt: followupPrompt,
expected_text: expectedText,
followup_delay_ms: followupDelayMs,
followup_enabled_timeout_ms: followupEnabledTimeoutMs,
response_timeout_ms: responseTimeoutMs,
pipeline_config: null,
debug_chat_reset: null,
tool_diagnostic: null,
steering: null,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "console", "network", "screenshot"],
};
try {
if (!backendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_BACKEND_URL is required.";
throw new Error(result.reason);
}
browser = await createBrowser(paths);
const { page } = browser;
const openResult = await openPipelineDebugChat(page, {
pipelineUrl,
pipelineName,
envHint: "case-specific pipeline env mapped to LANGBOT_E2E_PIPELINE_URL or LANGBOT_E2E_PIPELINE_NAME",
});
result.url = page.url();
if (!openResult.opened) {
result.status = openResult.status;
result.reason = openResult.reason;
} else {
const pipelineDiagnostic = await inspectPipeline(page, {
backendUrl,
pipelineUrl,
pipelineName,
expectedRunnerId,
});
await writeFile(pipelineConfigDiagnosticPath, `${JSON.stringify(pipelineDiagnostic, null, 2)}\n`, "utf8");
result.evidence.pipeline_config_diagnostic_json = pipelineConfigDiagnosticPath;
result.pipeline_config = pipelineDiagnostic;
if (!result.evidence_collected.includes("api_diagnostic")) result.evidence_collected.push("api_diagnostic");
const toolDiagnostic = await inspectToolNames(page, { backendUrl });
await writeFile(toolDiagnosticPath, `${JSON.stringify(toolDiagnostic, null, 2)}\n`, "utf8");
result.evidence.tool_diagnostic_json = toolDiagnosticPath;
result.tool_diagnostic = toolDiagnostic;
if (pipelineDiagnostic.status === "fail" || pipelineDiagnostic.status === "blocked") {
result.status = pipelineDiagnostic.status;
result.reason = pipelineDiagnostic.reason || "Pipeline diagnostic failed.";
} else if (toolDiagnostic.status === "fail" || toolDiagnostic.status === "blocked") {
result.status = toolDiagnostic.status;
result.reason = toolDiagnostic.reason || "Tool diagnostic failed.";
} else if (!toolDiagnostic.tool_names.includes("qa_plugin_sleep")) {
result.status = "blocked";
result.reason = "qa_plugin_sleep is not exposed by /api/v1/tools; rebuild/reinstall qa-plugin-smoke before running steering E2E.";
} else {
const resetDiagnostic = await resetPipelineDebugChat(page, {
backendUrl,
pipelineId: pipelineDiagnostic.pipeline_id,
sessionType: "person",
});
await writeFile(debugChatResetDiagnosticPath, `${JSON.stringify(resetDiagnostic, null, 2)}\n`, "utf8");
result.evidence.debug_chat_reset_diagnostic_json = debugChatResetDiagnosticPath;
result.debug_chat_reset = resetDiagnostic;
if (resetDiagnostic.status === "fail" || resetDiagnostic.status === "blocked") {
result.status = resetDiagnostic.status;
result.reason = resetDiagnostic.reason || "Debug Chat reset failed.";
} else {
await page.waitForTimeout(1000);
const reopenResult = await openPipelineDebugChat(page, {
pipelineUrl,
pipelineName,
envHint: "case-specific pipeline env mapped to LANGBOT_E2E_PIPELINE_URL or LANGBOT_E2E_PIPELINE_NAME",
});
result.url = page.url();
if (!reopenResult.opened) {
result.status = reopenResult.status;
result.reason = reopenResult.reason;
} else {
const streamResult = await setDebugChatStreamOutput(page, true);
if (streamResult.status === "blocked" || streamResult.status === "fail") {
result.status = streamResult.status;
result.reason = streamResult.reason;
} else {
result.steering = await runSteeringProbe(page);
result.status = result.steering.status;
result.reason = result.steering.reason;
}
}
}
}
}
} catch (error) {
if (!["env_issue", "blocked", "fail", "pass"].includes(result.status) || !result.reason) {
result.status = /Playwright is not installed|LANGBOT_FRONTEND_URL/.test(error.message) ? "env_issue" : "fail";
}
result.reason = result.reason || error.message;
} finally {
if (browser?.page) await safeScreenshot(browser.page, paths.screenshot);
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
const existingEvidence = {};
for (const [key, value] of Object.entries(result.evidence)) {
if (typeof value !== "string") continue;
const isResultFile = value === paths.automationResultJson || value === paths.resultJson;
if (isResultFile || await pathExists(value)) existingEvidence[key] = value;
}
result.evidence = existingEvidence;
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
async function runSteeringProbe(page) {
const beforeMessages = await visibleDebugChatMessages(page);
const beforeAssistantCount = countRole(beforeMessages, "assistant");
const beforeUserCount = countRole(beforeMessages, "user");
const firstStartedAt = Date.now();
const firstSend = await sendPrompt(page, firstPrompt, { enabledTimeoutMs: 5000 });
if (!firstSend.sent) {
return {
status: "fail",
reason: firstSend.reason || "Could not send first Debug Chat prompt.",
first_send: firstSend,
before_assistant_count: beforeAssistantCount,
before_user_count: beforeUserCount,
};
}
await page.waitForTimeout(followupDelayMs);
const preFollowupMessages = await visibleDebugChatMessages(page);
const preFollowupAssistantCount = countRole(preFollowupMessages, "assistant");
const followupStartedAt = Date.now();
const followupSend = await sendPrompt(page, followupPrompt, { enabledTimeoutMs: followupEnabledTimeoutMs });
const followupSentAt = Date.now();
if (!followupSend.sent) {
return {
status: "fail",
reason: followupSend.reason || "Could not send steering follow-up while the first run was active.",
first_send: firstSend,
followup_send: followupSend,
first_to_followup_attempt_ms: followupStartedAt - firstStartedAt,
followup_send_latency_ms: followupSentAt - followupStartedAt,
before_assistant_count: beforeAssistantCount,
pre_followup_assistant_count: preFollowupAssistantCount,
before_user_count: beforeUserCount,
};
}
const waitResult = await waitForLatestAssistantContaining(page, {
expectedText,
beforeAssistantCount,
timeoutMs: responseTimeoutMs,
});
await waitForDebugChatTextStable(page);
const afterMessages = await visibleDebugChatMessages(page);
const afterAssistantCount = countRole(afterMessages, "assistant");
const afterUserCount = countRole(afterMessages, "user");
const latestAssistantText = latestRoleText(afterMessages, "assistant");
const failureSignal = findFailureSignal(latestAssistantText) || findFailureSignal(messagesText(afterMessages));
const newAssistantCount = afterAssistantCount - beforeAssistantCount;
const newUserCount = afterUserCount - beforeUserCount;
const base = {
first_send: firstSend,
followup_send: followupSend,
first_to_followup_attempt_ms: followupStartedAt - firstStartedAt,
followup_send_latency_ms: followupSentAt - followupStartedAt,
before_assistant_count: beforeAssistantCount,
pre_followup_assistant_count: preFollowupAssistantCount,
after_assistant_count: afterAssistantCount,
new_assistant_count: newAssistantCount,
before_user_count: beforeUserCount,
after_user_count: afterUserCount,
new_user_count: newUserCount,
latest_assistant_text: latestAssistantText,
assistant_containing_expected_seen: waitResult.seen,
failure_signal: failureSignal,
};
if (failureSignal) {
return {
...base,
status: "fail",
reason: `Debug Chat displayed a known failure signal: ${failureSignal}`,
};
}
if (!waitResult.seen) {
return {
...base,
status: "fail",
reason: `No new assistant message contained steering sentinel ${expectedText}.`,
};
}
if (!latestAssistantText.includes(expectedText)) {
return {
...base,
status: "fail",
reason: `Latest assistant message did not contain steering sentinel ${expectedText}.`,
};
}
if (newUserCount < 2) {
return {
...base,
status: "fail",
reason: `Expected two new user messages, saw ${newUserCount}.`,
};
}
if (newAssistantCount !== 1) {
return {
...base,
status: "fail",
reason: `Expected one assistant response for one claimed steering run, saw ${newAssistantCount}. More than one usually means the follow-up became a separate run.`,
};
}
if (latestAssistantText.includes("STEERING_NO_FOLLOWUP")) {
return {
...base,
status: "fail",
reason: "Runner answered the no-follow-up branch, so steering was not injected.",
};
}
return {
...base,
status: "pass",
reason: `Follow-up sentinel ${expectedText} appeared in the only new assistant response after two user messages.`,
};
}
function debugChatInput(page) {
return page
.locator('input[placeholder*="message"], input[placeholder*="消息"], textarea[placeholder*="message"], textarea[placeholder*="消息"]')
.last();
}
async function sendPrompt(page, prompt, { enabledTimeoutMs }) {
const input = debugChatInput(page);
const inputVisible = await input.isVisible({ timeout: 5000 }).catch(() => false);
if (!inputVisible) return { sent: false, reason: "Debug Chat input is not visible." };
const inputEnabled = await input.isEnabled({ timeout: enabledTimeoutMs }).catch(() => false);
if (!inputEnabled) return { sent: false, reason: `Debug Chat input was not enabled within ${enabledTimeoutMs}ms.` };
await input.fill(prompt).catch(async () => {
await input.click();
await input.pressSequentially(prompt);
});
await input.press("Enter");
await page.getByText(prompt, { exact: false }).last().waitFor({ state: "visible", timeout: 10000 }).catch(() => {});
return {
sent: true,
submitted_by: "keyboard_enter",
};
}
async function waitForLatestAssistantContaining(page, { expectedText, beforeAssistantCount, timeoutMs }) {
const deadline = Date.now() + timeoutMs;
let lastMessages = [];
let latestAssistantText = "";
while (Date.now() < deadline) {
const messages = await visibleDebugChatMessages(page);
lastMessages = messages;
latestAssistantText = latestRoleText(messages, "assistant");
if (countRole(messages, "assistant") > beforeAssistantCount && latestAssistantText.includes(expectedText)) {
return {
seen: true,
latest_assistant_text: latestAssistantText,
messages,
};
}
const failureSignal = findFailureSignal(latestAssistantText);
if (failureSignal) {
return {
seen: false,
latest_assistant_text: latestAssistantText,
messages,
failure_signal: failureSignal,
};
}
await page.waitForTimeout(500);
}
return {
seen: false,
latest_assistant_text: latestAssistantText,
messages: lastMessages,
};
}
async function inspectPipeline(page, { backendUrl, pipelineUrl, pipelineName, expectedRunnerId }) {
const pipelineIdFromUrl = pipelineIdFromUrlValue(pipelineUrl);
return await page.evaluate(async ({ backendUrl, pipelineIdFromUrl, pipelineName, expectedRunnerId }) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
reason: "Browser profile has no localStorage token.",
};
}
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, {
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
let pipelineId = pipelineIdFromUrl;
let matchedBy = pipelineId ? "url" : "";
if (!pipelineId) {
if (!pipelineName) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: false,
reason: "Set LANGBOT_LOCAL_AGENT_PIPELINE_URL or LANGBOT_LOCAL_AGENT_PIPELINE_NAME.",
};
}
const list = await getJson("/api/v1/pipelines");
const pipelines = list.json.data?.pipelines || [];
const match = pipelines.find((pipeline) => pipeline.name === pipelineName);
if (!match) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: false,
list_status: list.status,
reason: `Could not find pipeline named ${pipelineName}.`,
};
}
pipelineId = match.uuid;
matchedBy = "name";
}
const loaded = await getJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`);
const pipeline = loaded.json.data?.pipeline;
if (loaded.status >= 400 || !pipeline) {
return {
status: "fail",
authenticated: true,
pipeline_resolved: false,
pipeline_id: pipelineId,
get_status: loaded.status,
reason: loaded.json.msg || "Could not load pipeline.",
};
}
const config = pipeline.config || {};
const runner = config.ai?.runner || {};
const runnerId = runner.id || runner.runner || "";
if (!runnerId) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
reason: "Pipeline has no ai.runner.id or legacy ai.runner.runner.",
};
}
if (expectedRunnerId && runnerId !== expectedRunnerId) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: expectedRunnerId,
reason: `Pipeline runner mismatch: expected ${expectedRunnerId}, got ${runnerId}.`,
};
}
return {
status: "ready",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: expectedRunnerId || "",
};
}, { backendUrl, pipelineIdFromUrl, pipelineName, expectedRunnerId });
}
async function inspectToolNames(page, { backendUrl }) {
return await page.evaluate(async ({ backendUrl }) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
tool_names: [],
reason: "Browser profile has no localStorage token.",
};
}
const response = await fetch(`${backendUrl}/api/v1/tools`, {
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
});
const json = await response.json().catch(() => ({}));
const toolNames = (json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
return {
status: response.status >= 400 ? "fail" : "ready",
authenticated: true,
http_status: response.status,
code: json.code ?? null,
tool_names: toolNames,
reason: response.status >= 400 ? json.msg || "Could not list tools." : "Tool list loaded.",
};
}, { backendUrl });
}
async function resetPipelineDebugChat(page, { backendUrl, pipelineId, sessionType }) {
return await page.evaluate(async ({ backendUrl, pipelineId, sessionType }) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
pipeline_id: pipelineId,
session_type: sessionType,
reason: "Browser profile has no localStorage token.",
};
}
const response = await fetch(
`${backendUrl}/api/v1/pipelines/${encodeURIComponent(pipelineId)}/ws/reset/${encodeURIComponent(sessionType)}`,
{
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
},
);
const json = await response.json().catch(() => ({}));
return {
status: response.status >= 400 ? "fail" : "ready",
authenticated: true,
pipeline_id: pipelineId,
session_type: sessionType,
reset_status: response.status,
reset_code: json.code ?? null,
reason: response.status >= 400 ? json.msg || "Debug Chat reset failed." : "Debug Chat session reset.",
};
}, { backendUrl, pipelineId, sessionType });
}
function pipelineIdFromUrlValue(value) {
const match = String(value || "").match(/\/pipelines?\/([^/?#]+)/i);
return match ? decodeURIComponent(match[1]) : "";
}
function countRole(messages, role) {
return messages.filter((message) => message.role === role).length;
}
function latestRoleText(messages, role) {
return messages.filter((message) => message.role === role).at(-1)?.text || "";
}
function messagesText(messages) {
return messages.map((message) => message.text).join("\n");
}
function findFailureSignal(text) {
return DEBUG_CHAT_FAILURE_SIGNALS.find((signal) => String(text || "").includes(signal)) || "";
}
function positiveInt(value, fallback) {
const parsed = Number.parseInt(String(value || ""), 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
}
+185
View File
@@ -0,0 +1,185 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { existsSync, readFileSync } from "node:fs";
import { resolve } from "node:path";
import { env } from "node:process";
import {
ensureEvidence,
evidencePaths,
exitCode,
localIsoWithOffset,
writeResult,
} from "./lib/langbot-e2e.mjs";
function loadEnvDefaults(path) {
if (!existsSync(path)) return;
for (const rawLine of readFileSync(path, "utf8").split(/\r?\n/)) {
const line = rawLine.trim();
if (!line || line.startsWith("#")) continue;
const sep = line.indexOf("=");
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
}
}
loadEnvDefaults("skills/.env");
loadEnvDefaults("skills/.env.local");
const caseId = env.LBS_CASE_ID || "mcp-stdio-fixture-direct";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const startedAt = new Date();
const fixturePath = resolve(env.LANGBOT_MCP_FIXTURE_PATH || "skills/langbot-testing/fixtures/mcp/qa_mcp_echo_server.py");
const langbotRepo = env.LANGBOT_REPO ? resolve(env.LANGBOT_REPO) : "";
const uvCandidates = [
env.LANGBOT_MCP_FIXTURE_UV,
"uv",
].filter(Boolean);
const uv = uvCandidates.find((candidate) => candidate === "uv" || existsSync(candidate));
const pythonCandidates = [
env.LANGBOT_MCP_FIXTURE_PYTHON,
langbotRepo ? `${langbotRepo}/.venv/bin/python` : "",
"python3",
].filter(Boolean);
const python = pythonCandidates.find((candidate) => candidate === "python3" || existsSync(candidate));
const command = langbotRepo && uv
? { executable: uv, args: ["run", "python", fixturePath], cwd: langbotRepo, mode: "uv" }
: python
? { executable: python, args: [fixturePath], cwd: resolve("."), mode: "python" }
: null;
const expectedText = "qa_mcp_echo:mcp-stdio-fixture-ok";
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
fixture_path: fixturePath,
command,
expected_text: expectedText,
evidence: {
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
};
function parseJsonLines(buffer) {
return buffer
.split(/\r?\n/)
.map((line) => line.trim())
.filter(Boolean)
.map((line) => {
try {
return JSON.parse(line);
} catch {
return null;
}
})
.filter(Boolean);
}
async function request(child, id, method, params) {
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
}
async function run() {
if (!command) {
result.status = "env_issue";
result.reason = "No uv or Python interpreter found. Set LANGBOT_REPO, LANGBOT_MCP_FIXTURE_UV, or LANGBOT_MCP_FIXTURE_PYTHON.";
return;
}
if (!existsSync(fixturePath)) {
result.status = "env_issue";
result.reason = `MCP fixture not found: ${fixturePath}`;
return;
}
const child = spawn(command.executable, command.args, {
cwd: command.cwd,
stdio: ["pipe", "pipe", "pipe"],
});
let stdout = "";
let stderr = "";
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
child.stdout.on("data", (chunk) => {
stdout += chunk;
});
child.stderr.on("data", (chunk) => {
stderr += chunk;
});
const timeout = setTimeout(() => child.kill("SIGTERM"), 10_000);
try {
await new Promise((resolveReady) => setTimeout(resolveReady, 100));
await request(child, 1, "initialize", {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "langbot-skills", version: "0" },
});
await new Promise((resolveReady) => setTimeout(resolveReady, 200));
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method: "notifications/initialized", params: {} })}\n`);
await request(child, 2, "tools/list", {});
await request(child, 3, "tools/call", {
name: "qa_mcp_echo",
arguments: { text: "mcp-stdio-fixture-ok" },
});
await new Promise((resolveDone) => setTimeout(resolveDone, 1500));
} finally {
clearTimeout(timeout);
child.kill("SIGTERM");
}
const messages = parseJsonLines(stdout);
if (/No module named ['"]mcp['"]|ModuleNotFoundError/i.test(stderr)) {
result.status = "env_issue";
result.reason = `Python environment cannot import mcp. Set LANGBOT_MCP_FIXTURE_PYTHON to a LangBot venv Python. stderr=${stderr.trim()}`;
return;
}
const listResult = messages.find((message) => message.id === 2)?.result;
const callResult = messages.find((message) => message.id === 3)?.result;
const toolNames = Array.isArray(listResult?.tools)
? listResult.tools.map((tool) => tool.name)
: [];
const callText = Array.isArray(callResult?.content)
? callResult.content.map((item) => item.text || "").join("\n")
: "";
if (!toolNames.includes("qa_mcp_echo")) {
result.status = "fail";
result.reason = `MCP fixture did not list qa_mcp_echo. stderr=${stderr.trim()}`;
return;
}
if (!callText.includes(expectedText)) {
result.status = "fail";
result.reason = `MCP fixture call did not return ${expectedText}. stderr=${stderr.trim()}`;
return;
}
result.status = "pass";
result.reason = "MCP stdio fixture listed qa_mcp_echo and returned the deterministic tool result without a model provider.";
}
try {
await run();
} catch (error) {
result.status = "fail";
result.reason = error instanceof Error ? error.message : String(error);
} finally {
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
+234
View File
@@ -0,0 +1,234 @@
#!/usr/bin/env node
import { existsSync, readFileSync } from "node:fs";
import { writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
localIsoWithOffset,
safeScreenshot,
writeResult,
} from "./lib/langbot-e2e.mjs";
function loadEnvDefaults(path) {
if (!existsSync(path)) return;
for (const rawLine of readFileSync(path, "utf8").split(/\r?\n/)) {
const line = rawLine.trim();
if (!line || line.startsWith("#")) continue;
const sep = line.indexOf("=");
if (sep === -1) continue;
const key = line.slice(0, sep).trim();
if (env[key]) continue;
env[key] = line.slice(sep + 1).trim().replace(/^["']|["']$/g, "");
}
}
loadEnvDefaults("skills/.env");
loadEnvDefaults("skills/.env.local");
const caseId = env.LBS_CASE_ID || "mcp-stdio-register";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const startedAt = new Date();
const serverName = env.LANGBOT_MCP_SERVER_NAME || "qa-local-stdio";
const expectedTool = env.LANGBOT_MCP_EXPECTED_TOOL || "qa_mcp_echo";
const fixturePath = resolve(env.LANGBOT_MCP_FIXTURE_PATH || "skills/langbot-testing/fixtures/mcp/qa_mcp_echo_server.py");
const fixtureCommand = env.LANGBOT_MCP_FIXTURE_COMMAND || "python";
const fixtureArgs = env.LANGBOT_MCP_FIXTURE_ARGS
? JSON.parse(env.LANGBOT_MCP_FIXTURE_ARGS)
: [fixturePath];
const startupTimeoutSec = Number(env.LANGBOT_MCP_STARTUP_TIMEOUT_SEC || "300");
const readyTimeoutMs = Number(env.LANGBOT_MCP_READY_TIMEOUT_MS || "360000");
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const apiDiagnosticPath = resolve(paths.evidenceDir, "api-diagnostic.json");
let browser;
const result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
server_name: serverName,
fixture_path: fixturePath,
expected_tool: expectedTool,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
api_diagnostic_json: apiDiagnosticPath,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["api_diagnostic"],
};
async function run() {
if (!backendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_BACKEND_URL is not configured.";
return;
}
if (!existsSync(fixturePath)) {
result.status = "env_issue";
result.reason = `MCP fixture not found: ${fixturePath}`;
return;
}
browser = await createBrowser(paths);
const { page } = browser;
await page.goto(env.LANGBOT_FRONTEND_URL, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const diagnostic = await page.evaluate(async ({
backendUrl,
serverName,
expectedTool,
fixturePath,
fixtureCommand,
fixtureArgs,
startupTimeoutSec,
readyTimeoutMs,
}) => {
const token = localStorage.getItem("token");
if (!token) {
return {
authenticated: false,
save_status: 0,
save_code: null,
save_msg: "Browser profile has no localStorage token.",
tool_names: [],
has_expected_tool: false,
runtime_status: null,
runtime_tool_names: [],
runtime_error: "",
};
}
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
};
const serverConfig = {
name: serverName,
mode: "stdio",
enable: true,
extra_args: {
command: fixtureCommand,
args: fixtureArgs,
env: {},
box: {
startup_timeout_sec: startupTimeoutSec,
},
},
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const sendJson = async (method, path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method,
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const serverPath = `/api/v1/mcp/servers/${encodeURIComponent(serverName)}`;
const beforeServer = await getJson(serverPath);
const save = beforeServer.status === 404
? await sendJson("POST", "/api/v1/mcp/servers", serverConfig)
: await sendJson("PUT", serverPath, serverConfig);
const deadline = Date.now() + readyTimeoutMs;
let lastTools = [];
let lastRuntime = null;
while (Date.now() < deadline) {
await new Promise((resolveReady) => setTimeout(resolveReady, 500));
const tools = await getJson("/api/v1/tools");
const server = await getJson(serverPath);
lastTools = (tools.json.data?.tools || [])
.map((tool) => tool.name || tool.tool_name || tool.function?.name || "")
.filter(Boolean)
.sort();
lastRuntime = server.json.data?.server?.runtime_info || null;
if (lastTools.includes(expectedTool)) break;
}
return {
authenticated: true,
before_status: beforeServer.status,
save_status: save.status,
save_code: save.json.code ?? null,
save_msg: save.json.msg || "",
tool_names: lastTools,
has_expected_tool: lastTools.includes(expectedTool),
runtime_status: lastRuntime?.status || null,
runtime_tool_names: (lastRuntime?.tools || [])
.map((tool) => tool.name || tool.tool_name || "")
.filter(Boolean)
.sort(),
runtime_tool_count: lastRuntime?.tool_count ?? null,
runtime_error: lastRuntime?.error_message || "",
};
}, { backendUrl, serverName, expectedTool, fixturePath, fixtureCommand, fixtureArgs, startupTimeoutSec, readyTimeoutMs });
await writeFile(apiDiagnosticPath, `${JSON.stringify(diagnostic, null, 2)}\n`, "utf8");
await safeScreenshot(page, paths.screenshot);
if (!diagnostic.authenticated) {
result.status = "blocked";
result.reason = "Browser profile is not authenticated for LangBot; cannot update MCP server.";
return;
}
if (diagnostic.save_status >= 400 || diagnostic.save_code !== 0) {
result.status = "fail";
result.reason = `Failed to save MCP server ${serverName}: ${diagnostic.save_status} ${diagnostic.save_msg}`;
return;
}
if (diagnostic.runtime_status !== "connected") {
result.status = "fail";
result.reason = `MCP server ${serverName} is not connected after save: ${diagnostic.runtime_status || "missing runtime"}. ${diagnostic.runtime_error}`;
return;
}
if (!diagnostic.has_expected_tool || !diagnostic.runtime_tool_names.includes(expectedTool)) {
result.status = "fail";
result.reason = `MCP server ${serverName} did not expose ${expectedTool}. See ${apiDiagnosticPath}.`;
return;
}
result.status = "pass";
result.reason = `MCP server ${serverName} is connected and exposes ${expectedTool} through LangBot /api/v1/tools.`;
}
try {
await run();
} catch (error) {
result.status = /Playwright is not installed|LANGBOT_FRONTEND_URL/.test(error.message) ? "env_issue" : "fail";
result.reason = error instanceof Error ? error.message : String(error);
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
+806
View File
@@ -0,0 +1,806 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { resolve } from "node:path";
import { env } from "node:process";
import {
openPipelineDebugChat,
runDebugChatPrompt,
setDebugChatStreamOutput,
} from "./lib/debug-chat.mjs";
import {
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
localIsoWithOffset,
pathExists,
safeScreenshot,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = env.LBS_CASE_ID || "pipeline-debug-chat";
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const expectedText = env.LANGBOT_E2E_EXPECTED_TEXT || "OK";
const prompt = env.LANGBOT_E2E_PROMPT || `请只回复 ${expectedText},用于前端调试测试。`;
const responseTimeoutMs = Number.parseInt(env.LANGBOT_E2E_RESPONSE_TIMEOUT_MS || "120000", 10);
const safeResponseTimeoutMs = Number.isFinite(responseTimeoutMs) && responseTimeoutMs > 0 ? responseTimeoutMs : 120000;
const streamOutput = /^(0|false)$/i.test(env.LANGBOT_E2E_STREAM_OUTPUT || "")
? false
: /^(1|true)$/i.test(env.LANGBOT_E2E_STREAM_OUTPUT || "")
? true
: null;
const failureSignals = (env.LANGBOT_E2E_FAILURE_SIGNALS || "")
.split(/\r?\n/)
.map((item) => item.trim())
.filter(Boolean);
const imageBase64Path = env.LANGBOT_E2E_IMAGE_BASE64_PATH || "";
const imagePathEnv = env.LANGBOT_E2E_IMAGE_PATH || "";
const backendUrl = env.LANGBOT_BACKEND_URL || "";
const pipelineRequired = env.LANGBOT_E2E_PIPELINE_REQUIRED === "1";
const pipelineUrl = pipelineRequired
? env.LANGBOT_E2E_PIPELINE_URL
: (env.LANGBOT_E2E_PIPELINE_URL || env.LANGBOT_PIPELINE_URL);
const pipelineName = pipelineRequired
? env.LANGBOT_E2E_PIPELINE_NAME
: (env.LANGBOT_E2E_PIPELINE_NAME || env.LANGBOT_PIPELINE_NAME);
const expectedRunnerId = env.LANGBOT_E2E_EXPECTED_RUNNER_ID || "";
const resetDebugChat = boolFromEnv(env.LANGBOT_E2E_RESET_DEBUG_CHAT, false);
const restoreRunnerConfig = boolFromEnv(env.LANGBOT_E2E_RESTORE_RUNNER_CONFIG, true);
const debugChatSessionType = env.LANGBOT_E2E_DEBUG_CHAT_SESSION_TYPE || "person";
const pipelineConfigDiagnosticPath = resolve(paths.evidenceDir, "pipeline-config-diagnostic.json");
const debugChatResetDiagnosticPath = resolve(paths.evidenceDir, "debug-chat-reset-diagnostic.json");
const pipelineConfigRestoreDiagnosticPath = resolve(paths.evidenceDir, "pipeline-config-restore-diagnostic.json");
const metricsPath = resolve(paths.evidenceDir, "metrics.json");
const startedAt = new Date();
let browser;
let restorePlan = null;
let result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
url: "",
prompt,
expected_text: expectedText,
response_timeout_ms: safeResponseTimeoutMs,
stream_output: streamOutput,
image_fixture: imageBase64Path || imagePathEnv,
prompt_count: 1,
chat_results: [],
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
metrics_json: metricsPath,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "network", "metrics"],
};
function boolFromEnv(value, defaultValue) {
if (value === undefined || value === "") return defaultValue;
if (/^(0|false|no|off)$/i.test(value)) return false;
if (/^(1|true|yes|on)$/i.test(value)) return true;
return defaultValue;
}
function parseJsonEnv(key, fallback) {
const raw = env[key];
if (!raw) return fallback;
try {
return JSON.parse(raw);
} catch (error) {
throw new Error(`${key} must be valid JSON: ${error.message}`);
}
}
function positiveNumberEnv(key, fallback) {
const value = Number(env[key] || "");
return Number.isFinite(value) && value >= 0 ? value : fallback;
}
function percentile(values, percentileValue) {
if (values.length === 0) return 0;
const sorted = [...values].sort((a, b) => a - b);
const index = Math.min(sorted.length - 1, Math.ceil((percentileValue / 100) * sorted.length) - 1);
return Number(sorted[index].toFixed(3));
}
function stats(values) {
if (values.length === 0) return { min: 0, p50: 0, p95: 0, p99: 0, max: 0 };
return {
min: Number(Math.min(...values).toFixed(3)),
p50: percentile(values, 50),
p95: percentile(values, 95),
p99: percentile(values, 99),
max: Number(Math.max(...values).toFixed(3)),
};
}
function promptStepsFromEnv() {
const rawSteps = parseJsonEnv("LANGBOT_E2E_PROMPTS_JSON", null);
if (rawSteps === null) {
return [{ prompt, expectedText, responseTimeoutMs: safeResponseTimeoutMs }];
}
if (!Array.isArray(rawSteps) || rawSteps.length === 0) {
throw new Error("LANGBOT_E2E_PROMPTS_JSON must be a non-empty JSON array.");
}
return rawSteps.map((item, index) => {
if (typeof item === "string") {
return { prompt: item, expectedText, responseTimeoutMs: safeResponseTimeoutMs };
}
if (!item || typeof item !== "object" || typeof item.prompt !== "string" || !item.prompt) {
throw new Error(`LANGBOT_E2E_PROMPTS_JSON[${index}] must be a string or an object with a prompt string.`);
}
const stepTimeout = Number.parseInt(String(item.response_timeout_ms || item.responseTimeoutMs || safeResponseTimeoutMs), 10);
return {
prompt: item.prompt,
expectedText: String(item.expected_text || item.expectedText || expectedText),
responseTimeoutMs: Number.isFinite(stepTimeout) && stepTimeout > 0 ? stepTimeout : safeResponseTimeoutMs,
};
});
}
function expandEnvRefs(value) {
return String(value || "").replace(/\$\{([A-Z][A-Z0-9_]*)\}|\$([A-Z][A-Z0-9_]*)/g, (_match, braced, bare) => {
return env[braced || bare] || "";
});
}
function textList(value) {
if (value === undefined || value === null || value === "") return [];
return Array.isArray(value) ? value.map(String) : [String(value)];
}
function runArgv(argv, { cwd = "", timeoutMs = 30_000 } = {}) {
return new Promise((resolveRun) => {
if (!Array.isArray(argv) || argv.length === 0 || !argv.every((item) => typeof item === "string" && item)) {
resolveRun({
status: "fail",
reason: "Filesystem command check requires a non-empty argv string array.",
exit_code: null,
stdout: "",
stderr: "",
});
return;
}
const child = spawn(argv[0], argv.slice(1), {
cwd: cwd ? resolve(cwd) : undefined,
env,
shell: false,
});
let stdout = "";
let stderr = "";
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
child.kill("SIGTERM");
}, timeoutMs);
child.stdout.on("data", (chunk) => {
stdout += chunk.toString();
});
child.stderr.on("data", (chunk) => {
stderr += chunk.toString();
});
child.on("error", (error) => {
clearTimeout(timer);
resolveRun({
status: "fail",
reason: error.message,
exit_code: null,
stdout,
stderr,
});
});
child.on("close", (code) => {
clearTimeout(timer);
resolveRun({
status: timedOut ? "fail" : "pass",
reason: timedOut ? `Command timed out after ${timeoutMs} ms.` : "",
exit_code: code,
stdout,
stderr,
});
});
});
}
async function runFilesystemChecks(checks) {
if (!Array.isArray(checks) || checks.length === 0) {
return { status: "not_required", checks: [] };
}
const results = [];
for (let index = 0; index < checks.length; index += 1) {
const check = checks[index];
if (!check || typeof check !== "object") {
results.push({ index, status: "fail", reason: "Filesystem check must be an object." });
continue;
}
const contains = textList(check.contains);
const notContains = textList(check.not_contains || check.notContains);
const expectedExitCode = Number.isInteger(check.exit_code)
? check.exit_code
: Number.isInteger(check.expected_exit_code)
? check.expected_exit_code
: 0;
const expectedStdout = textList(check.stdout_contains || check.expected_stdout || check.expectedStdout);
if (check.path) {
const path = resolve(expandEnvRefs(check.path));
let text = "";
try {
text = await readFile(path, "utf8");
} catch (error) {
results.push({ index, status: "fail", type: "file", path, reason: error.message });
continue;
}
const missing = contains.filter((needle) => !text.includes(needle));
const forbidden = notContains.filter((needle) => text.includes(needle));
results.push({
index,
status: missing.length || forbidden.length ? "fail" : "pass",
type: "file",
path,
missing,
forbidden,
reason: missing.length
? `Missing expected text: ${missing.join(", ")}`
: forbidden.length
? `Found forbidden text: ${forbidden.join(", ")}`
: "",
});
continue;
}
if (check.argv) {
const cwd = check.cwd ? expandEnvRefs(check.cwd) : "";
const timeoutMs = Number.parseInt(String(check.timeout_ms || check.timeoutMs || "30000"), 10);
const run = await runArgv(check.argv.map(expandEnvRefs), {
cwd,
timeoutMs: Number.isFinite(timeoutMs) && timeoutMs > 0 ? timeoutMs : 30_000,
});
const missingStdout = expectedStdout.filter((needle) => !run.stdout.includes(needle));
const exitMatches = run.exit_code === expectedExitCode;
results.push({
index,
status: run.status === "pass" && exitMatches && missingStdout.length === 0 ? "pass" : "fail",
type: "command",
argv: check.argv,
cwd,
exit_code: run.exit_code,
expected_exit_code: expectedExitCode,
missing_stdout: missingStdout,
stdout_preview: run.stdout.slice(0, 2000),
stderr_preview: run.stderr.slice(0, 2000),
reason: run.reason
|| (!exitMatches ? `Expected exit code ${expectedExitCode}, saw ${run.exit_code}.` : "")
|| (missingStdout.length ? `Missing stdout text: ${missingStdout.join(", ")}` : ""),
});
continue;
}
results.push({ index, status: "fail", reason: "Filesystem check requires either path or argv." });
}
const failed = results.filter((item) => item.status !== "pass");
return {
status: failed.length ? "fail" : "pass",
checks: results,
reason: failed.length ? `Filesystem checks failed: ${failed.map((item) => item.index).join(", ")}` : "",
};
}
function pipelineIdFromUrl(url) {
if (!url) return "";
try {
const parsed = new URL(url);
return parsed.searchParams.get("id") || "";
} catch {
return "";
}
}
function sanitizePipelineDiagnostic(diagnostic) {
const { restore_config: _restoreConfig, ...safe } = diagnostic || {};
return safe;
}
async function prepareImageFixture(paths) {
if (imagePathEnv) return resolve(imagePathEnv);
if (!imageBase64Path) return "";
const source = resolve(imageBase64Path);
const target = resolve(paths.evidenceDir, "image-fixture.png");
const encoded = await readFile(source, "utf8");
await writeFile(target, Buffer.from(encoded.replace(/\s+/g, ""), "base64"));
return target;
}
async function inspectAndPatchPipelineConfig(page, {
backendUrl,
pipelineUrl,
pipelineName,
runnerConfigPatch,
expectedRunnerId,
}) {
const pipelineIdFromUrlValue = pipelineIdFromUrl(pipelineUrl) || pipelineIdFromUrl(page.url());
return await page.evaluate(async ({
backendUrl,
pipelineIdFromUrlValue,
pipelineName,
runnerConfigPatch,
expectedRunnerId,
}) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
reason: "Browser profile has no localStorage token.",
};
}
const headers = {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
};
const getJson = async (path) => {
const response = await fetch(`${backendUrl}${path}`, { headers });
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
const putJson = async (path, body) => {
const response = await fetch(`${backendUrl}${path}`, {
method: "PUT",
headers,
body: JSON.stringify(body),
});
return {
status: response.status,
json: await response.json().catch(() => ({})),
};
};
let pipelineId = pipelineIdFromUrlValue || "";
let matchedBy = pipelineId ? "url" : "";
if (!pipelineId && pipelineName) {
const list = await getJson("/api/v1/pipelines");
const pipelines = list.json.data?.pipelines || [];
const match = pipelines.find((pipeline) => pipeline.name === pipelineName);
if (!match) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: false,
list_status: list.status,
reason: `Could not find pipeline named ${pipelineName}.`,
};
}
pipelineId = match.uuid;
matchedBy = "name";
}
if (!pipelineId) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: false,
reason: "Could not resolve pipeline id from URL or pipeline name.",
};
}
const before = await getJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`);
const pipeline = before.json.data?.pipeline;
if (before.status >= 400 || !pipeline) {
return {
status: "fail",
authenticated: true,
pipeline_resolved: false,
pipeline_id: pipelineId,
get_status: before.status,
reason: before.json.msg || "Could not load pipeline.",
};
}
const config = JSON.parse(JSON.stringify(pipeline.config || {}));
const aiConfig = config.ai && typeof config.ai === "object" ? config.ai : {};
const runner = aiConfig.runner && typeof aiConfig.runner === "object" ? aiConfig.runner : {};
const runnerId = runner.id || runner.runner || "";
if (!runnerId) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
reason: "Pipeline has no ai.runner.id or legacy ai.runner.runner.",
};
}
if (expectedRunnerId && runnerId !== expectedRunnerId) {
return {
status: "blocked",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: expectedRunnerId,
reason: `Pipeline runner mismatch: expected ${expectedRunnerId}, got ${runnerId}.`,
};
}
const runnerConfigs = aiConfig.runner_config && typeof aiConfig.runner_config === "object"
? aiConfig.runner_config
: {};
const currentRunnerConfig = runnerConfigs[runnerId] && typeof runnerConfigs[runnerId] === "object"
? runnerConfigs[runnerId]
: {};
const patchKeys = Object.keys(runnerConfigPatch || {});
const baseDiagnostic = {
status: "ready",
authenticated: true,
pipeline_resolved: true,
pipeline_id: pipelineId,
pipeline_name: pipeline.name,
matched_by: matchedBy,
runner_id: runnerId,
expected_runner_id: expectedRunnerId || "",
patch_keys: patchKeys,
runner_config_before_keys: Object.keys(currentRunnerConfig),
patched: patchKeys.length > 0,
};
if (patchKeys.length === 0) {
return baseDiagnostic;
}
const updatedRunnerConfig = {
...currentRunnerConfig,
...runnerConfigPatch,
};
const updatedConfig = {
...config,
ai: {
...aiConfig,
runner: {
...runner,
id: runnerId,
},
runner_config: {
...runnerConfigs,
[runnerId]: updatedRunnerConfig,
},
},
};
const update = await putJson(`/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, {
config: updatedConfig,
});
if (update.status >= 400) {
return {
...baseDiagnostic,
status: "fail",
put_status: update.status,
put_code: update.json.code ?? null,
reason: update.json.msg || "Pipeline config update failed.",
};
}
return {
...baseDiagnostic,
put_status: update.status,
put_code: update.json.code ?? null,
runner_config_after_keys: Object.keys(updatedRunnerConfig),
restore_config: config,
};
}, {
backendUrl,
pipelineIdFromUrlValue,
pipelineName,
runnerConfigPatch,
expectedRunnerId,
});
}
async function restorePipelineConfig(page, { backendUrl, pipelineId, config }) {
return await page.evaluate(async ({ backendUrl, pipelineId, config }) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
pipeline_id: pipelineId,
reason: "Browser profile has no localStorage token.",
};
}
const response = await fetch(`${backendUrl}/api/v1/pipelines/${encodeURIComponent(pipelineId)}`, {
method: "PUT",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ config }),
});
const json = await response.json().catch(() => ({}));
return {
status: response.status >= 400 ? "fail" : "ready",
authenticated: true,
pipeline_id: pipelineId,
put_status: response.status,
put_code: json.code ?? null,
reason: response.status >= 400 ? json.msg || "Pipeline config restore failed." : "Pipeline config restored.",
};
}, { backendUrl, pipelineId, config });
}
async function resetPipelineDebugChat(page, { backendUrl, pipelineId, sessionType }) {
return await page.evaluate(async ({ backendUrl, pipelineId, sessionType }) => {
const token = localStorage.getItem("token");
if (!token) {
return {
status: "blocked",
authenticated: false,
pipeline_id: pipelineId,
session_type: sessionType,
reason: "Browser profile has no localStorage token.",
};
}
const response = await fetch(
`${backendUrl}/api/v1/pipelines/${encodeURIComponent(pipelineId)}/ws/reset/${encodeURIComponent(sessionType)}`,
{
method: "POST",
headers: {
Authorization: `Bearer ${token}`,
"Content-Type": "application/json",
},
},
);
const json = await response.json().catch(() => ({}));
return {
status: response.status >= 400 ? "fail" : "ready",
authenticated: true,
pipeline_id: pipelineId,
session_type: sessionType,
reset_status: response.status,
reset_code: json.code ?? null,
reason: response.status >= 400 ? json.msg || "Debug Chat reset failed." : "Debug Chat session reset.",
};
}, { backendUrl, pipelineId, sessionType });
}
try {
browser = await createBrowser(paths);
const { page } = browser;
const imagePath = await prepareImageFixture(paths);
const promptSteps = promptStepsFromEnv();
const filesystemChecks = parseJsonEnv("LANGBOT_E2E_FILESYSTEM_CHECKS_JSON", []);
const runnerConfigPatch = parseJsonEnv("LANGBOT_E2E_RUNNER_CONFIG_PATCH_JSON", {});
const runnerPatchKeys = Object.keys(runnerConfigPatch);
if (runnerPatchKeys.length > 0 || resetDebugChat || expectedRunnerId) {
if (!backendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_BACKEND_URL is required for runner config patch, runner assertion, or Debug Chat reset.";
throw new Error(result.reason);
}
}
result.prompt_count = promptSteps.length;
result.prompt = promptSteps.length === 1 ? promptSteps[0].prompt : `${promptSteps.length} prompts`;
result.expected_text = promptSteps.at(-1)?.expectedText || expectedText;
const openResult = await openPipelineDebugChat(page, {
pipelineUrl,
pipelineName,
envHint: pipelineRequired
? "case-specific pipeline env mapped to LANGBOT_E2E_PIPELINE_URL or LANGBOT_E2E_PIPELINE_NAME"
: "LANGBOT_PIPELINE_URL or LANGBOT_PIPELINE_NAME",
});
result.url = page.url();
if (!openResult.opened) {
result.status = openResult.status;
result.reason = openResult.reason;
} else {
result.status = "running";
result.reason = "";
if (runnerPatchKeys.length > 0 || resetDebugChat || expectedRunnerId) {
const pipelineDiagnostic = await inspectAndPatchPipelineConfig(page, {
backendUrl,
pipelineUrl,
pipelineName,
runnerConfigPatch,
expectedRunnerId,
});
const safeDiagnostic = sanitizePipelineDiagnostic(pipelineDiagnostic);
await writeFile(pipelineConfigDiagnosticPath, `${JSON.stringify(safeDiagnostic, null, 2)}\n`, "utf8");
result.evidence.pipeline_config_diagnostic_json = pipelineConfigDiagnosticPath;
result.pipeline_config = safeDiagnostic;
if (!result.evidence_collected.includes("api_diagnostic")) result.evidence_collected.push("api_diagnostic");
if (pipelineDiagnostic.status === "fail" || pipelineDiagnostic.status === "blocked") {
result.status = pipelineDiagnostic.status;
result.reason = pipelineDiagnostic.reason || "Pipeline config preparation failed.";
} else {
if (pipelineDiagnostic.restore_config && restoreRunnerConfig) {
restorePlan = {
backendUrl,
pipelineId: pipelineDiagnostic.pipeline_id,
config: pipelineDiagnostic.restore_config,
};
}
if (resetDebugChat) {
const resetDiagnostic = await resetPipelineDebugChat(page, {
backendUrl,
pipelineId: pipelineDiagnostic.pipeline_id,
sessionType: debugChatSessionType,
});
await writeFile(debugChatResetDiagnosticPath, `${JSON.stringify(resetDiagnostic, null, 2)}\n`, "utf8");
result.evidence.debug_chat_reset_diagnostic_json = debugChatResetDiagnosticPath;
result.debug_chat_reset = resetDiagnostic;
if (resetDiagnostic.status === "fail" || resetDiagnostic.status === "blocked") {
result.status = resetDiagnostic.status;
result.reason = resetDiagnostic.reason || "Debug Chat reset failed.";
} else {
await page.waitForTimeout(1000);
const reopenResult = await openPipelineDebugChat(page, {
pipelineUrl,
pipelineName,
envHint: pipelineRequired
? "case-specific pipeline env mapped to LANGBOT_E2E_PIPELINE_URL or LANGBOT_E2E_PIPELINE_NAME"
: "LANGBOT_PIPELINE_URL or LANGBOT_PIPELINE_NAME",
});
result.url = page.url();
if (!reopenResult.opened) {
result.status = reopenResult.status;
result.reason = reopenResult.reason;
}
}
}
}
}
if (result.status === "fail" || result.status === "blocked" || result.status === "env_issue") {
// Preparation already determined the outcome.
} else {
const streamResult = await setDebugChatStreamOutput(page, streamOutput);
if (streamResult.status === "blocked" || streamResult.status === "fail") {
result.status = streamResult.status;
result.reason = streamResult.reason;
} else {
for (let index = 0; index < promptSteps.length; index += 1) {
const step = promptSteps[index];
const promptStartedAt = Date.now();
const chatResult = await runDebugChatPrompt(page, {
prompt: step.prompt,
expectedText: step.expectedText,
responseTimeoutMs: step.responseTimeoutMs,
imagePath: index === 0 ? imagePath : "",
failureSignals: failureSignals.length > 0 ? failureSignals : undefined,
});
const promptDurationMs = Date.now() - promptStartedAt;
result.chat_results.push({
index,
expected_text: step.expectedText,
status: chatResult.status,
reason: chatResult.reason,
response_duration_ms: promptDurationMs,
min_expected_count: chatResult.min_expected_count,
final_count: chatResult.final_count,
before_assistant_expected_count: chatResult.before_assistant_expected_count,
after_assistant_expected_count: chatResult.after_assistant_expected_count,
failure_signal: chatResult.failure_signal || "",
});
result.status = chatResult.status;
result.reason = `Prompt ${index + 1}/${promptSteps.length}: ${chatResult.reason}`;
if (chatResult.status !== "pass") break;
}
}
}
if (result.status === "pass" && filesystemChecks.length > 0) {
const filesystemResult = await runFilesystemChecks(filesystemChecks);
result.filesystem_checks = filesystemResult;
if (!result.evidence_collected.includes("filesystem")) result.evidence_collected.push("filesystem");
if (filesystemResult.status === "fail") {
result.status = "fail";
result.reason = filesystemResult.reason || "Filesystem checks failed.";
}
}
}
} catch (error) {
if (!["env_issue", "blocked", "fail", "pass"].includes(result.status) || !result.reason) {
result.status = /Playwright is not installed|LANGBOT_FRONTEND_URL/.test(error.message) ? "env_issue" : "fail";
}
result.reason = result.reason || error.message;
} finally {
if (browser?.page) await safeScreenshot(browser.page, paths.screenshot);
if (browser?.page && restorePlan) {
const restoreDiagnostic = await restorePipelineConfig(browser.page, restorePlan).catch((error) => ({
status: "fail",
pipeline_id: restorePlan.pipelineId,
reason: error.message,
}));
await writeFile(pipelineConfigRestoreDiagnosticPath, `${JSON.stringify(restoreDiagnostic, null, 2)}\n`, "utf8");
result.evidence.pipeline_config_restore_diagnostic_json = pipelineConfigRestoreDiagnosticPath;
result.pipeline_config_restore = restoreDiagnostic;
}
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
result.duration_ms = finishedAt.getTime() - startedAt.getTime();
const responseDurations = result.chat_results
.map((item) => item.response_duration_ms)
.filter((value) => Number.isFinite(value));
const passedPrompts = result.chat_results.filter((item) => item.status === "pass").length;
const attemptedPrompts = result.chat_results.length;
const errorRate = attemptedPrompts === 0 ? 1 : Number(((attemptedPrompts - passedPrompts) / attemptedPrompts).toFixed(4));
const responseStats = stats(responseDurations);
const responseP95BudgetMs = positiveNumberEnv(
"LANGBOT_E2E_DEBUG_CHAT_RESPONSE_P95_MS",
positiveNumberEnv("LANGBOT_DEBUG_CHAT_RESPONSE_P95_MS", safeResponseTimeoutMs),
);
const maxErrorRate = positiveNumberEnv("LANGBOT_E2E_DEBUG_CHAT_MAX_ERROR_RATE", 0);
const metrics = {
probe: caseId,
url: result.url,
prompt_count: result.prompt_count,
attempted_prompt_count: attemptedPrompts,
passed_prompt_count: passedPrompts,
error_rate: errorRate,
response_duration_ms: responseStats,
total_duration_ms: result.duration_ms,
chat_results: result.chat_results,
};
result.metrics_summary = {
prompt_count: metrics.prompt_count,
attempted_prompt_count: metrics.attempted_prompt_count,
passed_prompt_count: metrics.passed_prompt_count,
error_rate: metrics.error_rate,
response_p50_ms: metrics.response_duration_ms.p50,
response_p95_ms: metrics.response_duration_ms.p95,
total_duration_ms: metrics.total_duration_ms,
};
result.thresholds_summary = {
response_p95_ms: {
actual: metrics.response_duration_ms.p95,
max: responseP95BudgetMs,
pass: attemptedPrompts > 0 && metrics.response_duration_ms.p95 <= responseP95BudgetMs,
},
error_rate: {
actual: metrics.error_rate,
max: maxErrorRate,
pass: metrics.error_rate <= maxErrorRate,
},
};
await writeFile(metricsPath, `${JSON.stringify(metrics, null, 2)}\n`, "utf8");
if (result.status === "pass" && !Object.values(result.thresholds_summary).every((item) => item.pass)) {
result.status = "fail";
result.reason = "Debug Chat performance breached response latency or error-rate thresholds.";
}
const existingEvidence = {};
for (const [key, value] of Object.entries(result.evidence)) {
if (typeof value !== "string") continue;
const isResultFile = value === paths.automationResultJson || value === paths.resultJson;
if (isResultFile || await pathExists(value)) existingEvidence[key] = value;
}
result.evidence = existingEvidence;
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env node
import { env } from "node:process";
import {
bodyText,
createBrowser,
ensureEvidence,
evidencePaths,
loadEnvFiles,
resetAndAuthLocalUser,
safeScreenshot,
setBrowserToken,
verifyBrowserToken,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = "refresh-local-login";
const paths = evidencePaths(caseId);
await loadEnvFiles();
await ensureEvidence(paths);
const result = {
source: "automation",
case_id: caseId,
status: "fail",
reason: "",
user: env.LANGBOT_E2E_LOGIN_USER || "",
frontend_url: env.LANGBOT_FRONTEND_URL || "",
backend_url: env.LANGBOT_BACKEND_URL || "",
backend_token_check: null,
browser_token_check: null,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console", "api_diagnostic"],
};
let browser;
try {
const backendUrl = env.LANGBOT_BACKEND_URL;
const frontendUrl = env.LANGBOT_FRONTEND_URL;
const user = env.LANGBOT_E2E_LOGIN_USER;
const password = env.LANGBOT_E2E_LOGIN_PASSWORD || "LangBotE2ELocalPass!2026";
if (!backendUrl) throw new Error("LANGBOT_BACKEND_URL is not configured.");
if (!frontendUrl) throw new Error("LANGBOT_FRONTEND_URL is not configured.");
if (!user) throw new Error("LANGBOT_E2E_LOGIN_USER is required.");
const auth = await resetAndAuthLocalUser({ backendUrl, user, password });
result.backend_token_check = auth.check;
browser = await createBrowser(paths);
const { page } = browser;
await setBrowserToken(page, frontendUrl, auth.token);
const browserCheck = await verifyBrowserToken(page, backendUrl);
result.browser_token_check = browserCheck;
if (!browserCheck.authenticated) {
throw new Error(browserCheck.reason || "Browser token check failed.");
}
await page.goto(`${frontendUrl.replace(/\/$/, "")}/home/monitoring`, { waitUntil: "domcontentloaded" });
await page.waitForLoadState("networkidle", { timeout: 10_000 }).catch(() => {});
const text = await bodyText(page);
if (!text.includes("Dashboard") && !text.includes("Pipelines") && !text.includes("流水线")) {
throw new Error("Token was written, but authenticated navigation was not visible.");
}
result.status = "pass";
result.reason = "Browser profile localStorage token refreshed.";
} catch (error) {
result.status = "fail";
result.reason = error.message;
} finally {
if (browser?.page) await safeScreenshot(browser.page, paths.screenshot);
if (browser) await browser.close().catch(() => {});
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(result.status === "pass" ? 0 : 1);
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env node
import {
bodyText,
createBrowser,
ensureEvidence,
evidencePaths,
exitCode,
gotoFrontend,
isLoginUrl,
loadEnvFiles,
localIsoWithOffset,
safeScreenshot,
verifyBrowserToken,
writeResult,
} from "./lib/langbot-e2e.mjs";
const caseId = "webui-login-state";
await loadEnvFiles();
const paths = evidencePaths(caseId);
await ensureEvidence(paths);
const startedAt = new Date();
let browser;
let result = {
source: "automation",
case_id: caseId,
run_id: paths.runId,
started_at: startedAt.toISOString(),
started_at_local: localIsoWithOffset(startedAt),
finished_at: "",
finished_at_local: "",
status: "fail",
reason: "",
url: "",
auth: null,
evidence: {
console_log: paths.consoleLog,
network_log: paths.networkLog,
screenshot: paths.screenshot,
automation_result_json: paths.automationResultJson,
result_json: paths.resultJson,
},
evidence_collected: ["ui", "screenshot", "console"],
};
try {
browser = await createBrowser(paths);
const { page } = browser;
await gotoFrontend(page);
result.url = page.url();
const backendUrl = process.env.LANGBOT_BACKEND_URL || "";
if (!backendUrl) {
result.status = "env_issue";
result.reason = "LANGBOT_BACKEND_URL is not configured.";
await safeScreenshot(page, paths.screenshot);
throw new Error(result.reason);
}
const auth = await verifyBrowserToken(page, backendUrl);
result.auth = auth;
const text = await bodyText(page);
const navigationSignals = [
"Dashboard",
"Bots",
"Pipelines",
"Knowledge",
"Plugins",
"首页",
"机器人",
"流水线",
"知识库",
"插件",
];
const matchedSignal = navigationSignals.find((signal) => text.includes(signal));
if (!auth.authenticated) {
result.status = "blocked";
result.reason = auth.reason || "Browser profile token was not accepted by backend.";
} else if (isLoginUrl(page.url()) || /登录|Login|Sign in/i.test(text)) {
result.status = "fail";
result.reason = "Backend accepted the token, but the WebUI still showed the login page.";
} else if (!matchedSignal) {
result.status = "fail";
result.reason = "Opened WebUI, but no known LangBot navigation signal was visible.";
} else {
result.status = "pass";
result.reason = `Authenticated navigation signal visible: ${matchedSignal}`;
}
await safeScreenshot(page, paths.screenshot);
} catch (error) {
if (!["env_issue", "blocked", "fail", "pass"].includes(result.status) || !result.reason) {
result.status = /Playwright is not installed|LANGBOT_FRONTEND_URL/.test(error.message) ? "env_issue" : "fail";
result.reason = error.message;
}
} finally {
if (browser) await browser.close().catch(() => {});
const finishedAt = new Date();
result.finished_at = finishedAt.toISOString();
result.finished_at_local = localIsoWithOffset(finishedAt);
await writeResult(paths, result);
console.log(JSON.stringify(result, null, 2));
}
process.exit(exitCode(result.status));
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
# Shared defaults for LangBot skills.
# Agents should read this file first, then load machine-local overrides from
# skills/.env.local. Do not put workstation-specific absolute paths or secrets
# in this committed file.
# The UI URL that testing skills should open.
# Default to the standalone Vite frontend. Set this to the backend WebUI URL
# instead if your LangBot checkout serves the frontend from the backend.
LANGBOT_FRONTEND_URL=http://127.0.0.1:3000
# LangBot API/backend URL.
LANGBOT_BACKEND_URL=http://127.0.0.1:5300
# Common standalone frontend dev URL. This is a candidate, not the default.
LANGBOT_DEV_FRONTEND_URL=http://127.0.0.1:3000
# Local repository paths. Copy skills/.env.example to skills/.env.local and set
# these for your checkout.
LANGBOT_REPO=
LANGBOT_WEB_REPO=
LANGBOT_RAG_PLUGIN_REPO=
LANGBOT_PARSER_PLUGIN_REPO=
# Browser profile and Playwright/Chromium paths.
LANGBOT_BROWSER_PROFILE=
LANGBOT_CHROMIUM_EXECUTABLE=
# Optional local proxy defaults. Do not store secrets here.
LANGBOT_PROXY_HTTP=
LANGBOT_PROXY_SOCKS=
LANGBOT_NO_PROXY=localhost,127.0.0.1,::1
# Optional case-specific pipeline targets. Put machine-local values in
# skills/.env.local so runner-specific cases do not accidentally reuse the
# generic LANGBOT_PIPELINE_URL.
# LANGBOT_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<generic-pipeline-uuid>
# LANGBOT_PIPELINE_NAME=Generic QA Pipeline
# LANGBOT_LOCAL_AGENT_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<local-agent-pipeline-uuid>
# LANGBOT_LOCAL_AGENT_PIPELINE_NAME=Local Agent QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL=http://127.0.0.1:3000/home/pipelines?id=<acp-agent-runner-pipeline-uuid>
# LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME=ACP AgentRunner QA Pipeline
# LANGBOT_ACP_AGENT_RUNNER_SSH_TARGET=yhh@101.34.71.12
# LANGBOT_ACP_AGENT_RUNNER_SSH_PORT=22
# LANGBOT_ACP_AGENT_RUNNER_SSH_IDENTITY_FILE=
# LANGBOT_ACP_AGENT_RUNNER_SSH_EXTRA_OPTIONS=
# LANGBOT_ACP_AGENT_RUNNER_REMOTE_WORKSPACE=/home/yhh/langbot-e2e/acp-workspace
+53
View File
@@ -0,0 +1,53 @@
# Copy this file to skills/.env.local and adjust it for your machine.
# Do not put API keys, OAuth tokens, browser localStorage tokens, or provider
# credentials in committed files.
# LangBot WebUI and backend endpoints.
LANGBOT_FRONTEND_URL=http://127.0.0.1:3000
LANGBOT_BACKEND_URL=http://127.0.0.1:5300
LANGBOT_DEV_FRONTEND_URL=http://127.0.0.1:3000
# Local repository paths.
LANGBOT_REPO=/path/to/LangBot
LANGBOT_WEB_REPO=/path/to/LangBot/web
LANGBOT_RAG_PLUGIN_REPO=/path/to/langbot-rag
LANGBOT_PARSER_PLUGIN_REPO=/path/to/langbot-parser
# Browser profile and Playwright/Chromium paths.
LANGBOT_BROWSER_PROFILE=/path/to/langbot-playwright-profile
LANGBOT_CHROMIUM_EXECUTABLE=/path/to/ms-playwright/chromium/chrome
# Optional local proxy defaults. Leave blank if not needed.
LANGBOT_PROXY_HTTP=
LANGBOT_PROXY_SOCKS=
LANGBOT_NO_PROXY=localhost,127.0.0.1,::1
# Optional generic pipeline target for generic Debug Chat smoke tests.
LANGBOT_PIPELINE_URL=
LANGBOT_PIPELINE_NAME=
# Optional fake OpenAI-compatible provider controls for Debug Chat load tests.
# Leave URL empty to let setup automation start a local provider and write the
# selected URL to skills/.env.local.
LANGBOT_FAKE_PROVIDER_URL=
LANGBOT_FAKE_PROVIDER_HOST=127.0.0.1
LANGBOT_FAKE_PROVIDER_PORT=
LANGBOT_FAKE_PROVIDER_MODEL_NAME=gpt-4o-mini
LANGBOT_FAKE_PROVIDER_RESPONSE_TEXT=OK
LANGBOT_FAKE_PROVIDER_FIRST_TOKEN_DELAY_MS=25
LANGBOT_FAKE_PROVIDER_CHUNK_DELAY_MS=10
LANGBOT_FAKE_PROVIDER_CHUNK_COUNT=0
LANGBOT_FAKE_PROVIDER_FAIL_FIRST_N=0
LANGBOT_FAKE_PROVIDER_FAIL_EVERY_N=0
LANGBOT_FAKE_PROVIDER_FAULT_STATUS=500
LANGBOT_FAKE_PROVIDER_FAIL_AFTER_FIRST_CHUNK=false
LANGBOT_FAKE_PROVIDER_DYNAMIC_RESPONSE=true
# Optional case-specific runner targets. Prefer these for runner-specific cases
# so the automation cannot silently test the wrong runner.
LANGBOT_LOCAL_AGENT_PIPELINE_URL=
LANGBOT_LOCAL_AGENT_PIPELINE_NAME=
LANGBOT_CODEX_AGENT_PIPELINE_URL=
LANGBOT_CODEX_AGENT_PIPELINE_NAME=
LANGBOT_CLAUDE_CODE_AGENT_PIPELINE_URL=
LANGBOT_CLAUDE_CODE_AGENT_PIPELINE_NAME=
+95
View File
@@ -0,0 +1,95 @@
---
name: langbot-deploy
description: Deploy and configure a LangBot instance — Docker / Docker Compose, Kubernetes, the config.yaml model, the Box sandbox runtime, the plugin runtime, and the global API key. Use when installing, deploying, upgrading, or configuring LangBot in production or self-hosted environments. Triggers on "deploy langbot", "langbot docker", "langbot compose", "langbot kubernetes", "langbot config.yaml", "langbot box runtime", "langbot global api key".
---
# LangBot Deployment & Configuration
Covers running LangBot in production. For development see `langbot-dev`.
## Docker Compose (recommended)
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
# Full stack (sandbox/Box + stdio MCP hosting + skill add/edit enabled)
docker compose --profile all up
# Basic (no Box runtime)
docker compose up
```
The `all` / `box` profile starts three services:
- `langbot` — main app, serves API + UI on `:5300`.
- `langbot_plugin_runtime` — plugin runtime (control `:5400`, debug `:5401`).
- `langbot_box` — Box sandbox runtime (`:5410`). Uses the host Docker socket to
spawn sandbox containers, so the **Box root host path and in-container path
must be identical** (`BOX__LOCAL__HOST_ROOT=${LANGBOT_BOX_ROOT:-${PWD}/data/box}`).
Its RPC and managed-process relay require a shared
`LANGBOT_BOX_CONTROL_TOKEN` (at least 32 non-whitespace characters) in both
the LangBot and Box containers. Generate it once with `openssl rand -hex 32`;
never put it in `box.runtime.endpoint` or commit it to config.
Every Compose deployment also needs one
`LANGBOT_PLUGIN_RUNTIME_CONTROL_TOKEN` shared by `langbot` and
`langbot_plugin_runtime`. Generate it with `openssl rand -hex 32` and export it
before `docker compose up`; the external Plugin Runtime fails closed when the
token is empty or weak. Kubernetes uses the `langbot-plugin-runtime-control`
Secret shown in `docker/kubernetes.yaml`.
With Box off, the dashboard/skills list stays visible (read-only) but sandbox
tools, skill add/edit, and stdio MCP are disabled. Set `box.enabled: false`
(or `BOX__ENABLED=false`) to match.
## Kubernetes
See `docker/kubernetes.yaml` and the deployment guide at
https://docs.langbot.app. `docker/deploy-k8s-test.sh` is a test helper.
## config.yaml (generated at `data/config.yaml` on first run)
Top-level sections: `api`, `system`, `command`, `concurrency`, `proxy`,
`database`, `vdb`, `storage`, `plugin`, `monitoring`, `box`, `space`.
Key settings:
| Key | Meaning |
| --- | --- |
| `api.port` | HTTP API + UI port (default 5300) |
| `api.global_api_key` | **Global API key** for the HTTP API + MCP server. Non-empty = accepted with no login/DB record; no `lbk_` prefix required. Empty = disabled. Plaintext — trusted/internal only, serve over HTTPS. |
| `plugin.runtime_ws_url` | Standalone plugin runtime WS URL (e.g. `ws://langbot_plugin_runtime:5400/control/ws`) |
| `box.enabled` | Master switch for the Box sandbox runtime |
| `box.backend` | `local` (Docker/nsjail autopick) / `docker` / `nsjail` / `e2b`; env override `BOX__BACKEND` |
| `box.runtime.endpoint` | External Box runtime URL (e.g. `ws://127.0.0.1:5410`); empty = local auto-managed |
Many keys have `ENV__SUBKEY` overrides (e.g. `BOX__BACKEND`, `BOX__ENABLED`).
## Runtimes & flags
- LangBot started directly spawns the plugin runtime over **stdio**.
- In containers it connects to a standalone runtime over **WebSocket**; start
with `--standalone-runtime`.
- Box has a parallel `--standalone-box` flag; the Docker box host is
`langbot_box:5410`.
## Global API key — enabling for agents/automation
```yaml
# data/config.yaml
api:
port: 5300
global_api_key: 'a-strong-secret' # empty disables it
```
This key authenticates both the HTTP API and the MCP server (`/mcp`) without a
login session. See `langbot-mcp-ops` for using it, and `docs/API_KEY_AUTH.md`.
## Pitfalls
- "No supported sandbox backend (Docker / nsjail / E2B)" with Docker running
usually means the user isn't in the `docker` group →
`sudo usermod -aG docker <user>` and restart in a new shell.
- Box root host/container path mismatch breaks sandbox container creation.
- Don't commit a non-empty `api.global_api_key` to version control.
+129
View File
@@ -0,0 +1,129 @@
---
name: langbot-dev
description: Develop, build, and debug the LangBot core backend and web frontend. Use when working inside the LangBot repository — backend (Python/Quart, src/langbot/pkg), the Vite/React web UI, HTTP API controllers/services, Alembic migrations, or the MCP server. Covers the dev environment (uv, pnpm), repo layout, the API auth model (user token / API key / global key), adding API endpoints, and the rule that API changes must update the MCP server and skills. Triggers on "langbot backend", "langbot dev", "langbot api", "add langbot endpoint", "langbot migration".
---
# LangBot Core Development
This skill covers developing the LangBot core (the main repo), distinct from
plugin development (see `langbot-plugin-dev`) and deployment (`langbot-deploy`).
## Stack
- **Backend**: Python `>=3.11,<4.0`, deps via `uv`. Framework: **Quart** (async
Flask). Serves the HTTP API + pre-built web UI on `http://127.0.0.1:5300`.
- **Frontend** (`web/`): **Vite + React Router 7 + shadcn/ui + Tailwind**,
managed by `pnpm`. Dev server on `:3000`. (NOT Next.js — `dev` script is `vite`.)
## Dev environment
```bash
# Backend
pip install uv
uv sync --dev
uv run main.py # API + UI on http://127.0.0.1:5300
# Frontend (separate terminal)
cd web
cp .env.example .env
pnpm install
pnpm dev # http://127.0.0.1:3000 (reads VITE_API_BASE_URL)
# Lint/format hooks (CI runs the same checks)
uv run pre-commit install
```
First run generates `data/config.yaml`; DB defaults to SQLite (PostgreSQL
supported). Migrations run automatically on startup.
## Repo layout (key paths)
```
src/langbot/
├── __main__.py # entrypoint, CLI flags (--standalone-runtime/-box/--debug)
├── pkg/
│ ├── api/
│ │ ├── http/ # Quart controllers + services
│ │ │ ├── controller/groups/ # route groups (@group.group_class)
│ │ │ └── service/ # business logic (called by controllers AND MCP)
│ │ └── mcp/ # MCP server (server.py = tools, mount.py = ASGI dispatch)
│ ├── core/ # app bootstrap, stages, task manager
│ ├── platform/ provider/ pipeline/ plugin/ box/ skill/ rag/ vector/
│ ├── command/ persistence/ storage/ config/ entity/ telemetry/
│ └── templates/config.yaml # config template (top-level: api, system, plugin, box, space...)
├── web/ # Vite SPA
└── docker/ # compose deployment
```
## HTTP API auth model
Route auth is declared per-route via `AuthType` in
`pkg/api/http/controller/group.py`:
- `NONE` — public.
- `USER_TOKEN` — web UI JWT (`Authorization: Bearer <jwt>`).
- `API_KEY``X-API-Key` or `Authorization: Bearer <key>`.
- `USER_TOKEN_OR_API_KEY` — either.
Authenticated routes receive an immutable `RequestContext` containing the
principal, authorized Workspace membership, fixed-role permissions, instance,
request id, and placement generation. A browser's `X-Workspace-Id` is only a
selector and is always checked against the Account membership. Tenant services
must accept this context (or an explicit trusted execution context) and fail
closed when it is absent.
API-key authentication accepts:
1. the **global key** from `config.yaml` `api.global_api_key` only for a
community instance with exactly one local Workspace, then
2. **web-UI keys** whose one-time `lbk_` secret is stored only as a hash and is
bound to one Workspace, explicit scopes, status, and optional expiry.
An API key derives its Workspace from the key record and ignores a caller's
Workspace selector. Public Bot/Webhook routes similarly derive Workspace from
the opaque owning resource rather than a header.
Route groups self-register via `@group.group_class(name, path)` and are
discovered by `importutil.import_modules_in_pkg`.
## Adding an API endpoint
1. Add/extend a controller in `pkg/api/http/controller/groups/` and the matching
service method in `pkg/api/http/service/`.
2. Pick the right `AuthType`.
3. **If the endpoint should be agent-accessible, add/adjust the matching MCP tool
in `pkg/api/mcp/server.py` and update the `langbot-mcp-ops` skill.** API and
MCP surface must stay aligned (see `AGENTS.md`).
4. Update `docs/service-api-openapi.json` if you maintain the OpenAPI overview.
## Database migrations (Alembic)
Single migration set supports SQLite + PostgreSQL. Files in
`src/langbot/pkg/persistence/alembic/versions/`.
```bash
# From project root (needs data/config.yaml)
uv run python -m langbot.pkg.persistence.alembic_runner autogenerate "description"
```
## Standards
- All code comments/docstrings in **English**; user-facing strings need **i18n**
(`en_US` + `zh_Hans` minimum, `ja_JP` where present).
- Consider toC and toB compatibility + security.
- Commit format: `<type>(<scope>): <subject>` (feat/fix/docs/refactor/...).
## Tests
```bash
uv run pytest tests/unit_tests -q # unit tests
uv run pytest tests/unit_tests/api -q # API service tests
uv run python tests/manual/mcp_smoke.py # MCP server e2e smoke
```
## See also
- `langbot-plugin-dev` — plugin SDK / runtime development.
- `langbot-testing` — WebUI/e2e QA harness (`bin/lbs`).
- `langbot-deploy` — Docker/compose deployment + config.
- `langbot-mcp-ops` — operating the LangBot MCP server.
@@ -0,0 +1,301 @@
---
name: langbot-eba-adapter-dev
description: Build, refactor, and test LangBot platform adapters for the Event-Based Agents architecture. Use when adding or migrating Telegram, Discord, or other messaging platform adapters to the EBA adapter layout, validating unified event/message conversion, writing live adapter probes, or using standalone plugin runtime plus Computer Use for end-to-end platform testing.
---
# LangBot EBA Adapter Development
Use this skill when implementing or reviewing a LangBot platform adapter under the Event-Based Agents architecture.
## Controlling a running instance via MCP
Beyond writing code, you can **drive a live LangBot instance over MCP** — no raw
HTTP needed. Two MCP servers exist (both reuse existing API keys; see `AGENTS.md`):
- **LangBot instance**`http://<host>:5300/mcp` (auth: web-UI `lbk_` key or the
`api.global_api_key` from `config.yaml`). Manage bots, pipelines, models,
knowledge bases, and skills. See the **`langbot-mcp-ops`** skill.
- **LangBot Space marketplace**`https://space.langbot.app/mcp` (auth: Personal
Access Token). Search plugins / MCP servers / skills. See the
**`langbot-space-ops`** skill.
> Any change to an agent-accessible HTTP API endpoint must keep the matching MCP
> tool and these skills in sync.
## Core Rule
Do not let platform-native event or message shapes leak into LangBot's common path. Each adapter must convert incoming SDK objects into unified EBA entities before dispatch:
- Events: `langbot_plugin.api.entities.builtin.platform.events`
- Message chains: `langbot_plugin.api.entities.builtin.platform.message.MessageChain`
- Users/groups/members: `langbot_plugin.api.entities.builtin.platform.entities`
- Raw platform objects may remain only in `source_platform_object` for debugging or platform-specific escape hatches.
## Start Here
1. Read the EBA design docs in `LangBot/docs/event-based-agents/`.
2. Read the architecture-level acceptance checklist before writing or validating code:
- `LangBot/docs/event-based-agents/adapters/acceptance-checklist.md`
3. Read the current reference adapter before writing code. Prefer Telegram first:
- `LangBot/src/langbot/pkg/platform/adapters/telegram/`
- `LangBot/docs/event-based-agents/adapters/telegram.md`
4. Read the legacy source adapter for the target platform:
- `LangBot/src/langbot/pkg/platform/sources/<platform>.py`
- `LangBot/src/langbot/pkg/platform/sources/<platform>.yaml`
5. Inspect SDK entity definitions in `langbot-plugin-sdk/src/langbot_plugin/api/entities/builtin/platform/`.
6. Search before assuming APIs. Platform SDKs change often.
## Adapter Layout
Create one directory per adapter:
```text
LangBot/src/langbot/pkg/platform/adapters/<platform>/
├── __init__.py
├── adapter.py
├── api_impl.py
├── event_converter.py
├── manifest.yaml
├── message_converter.py
├── platform_api.py
├── types.py
└── <platform>.svg
```
Add optional helpers such as `voice.py` only when the platform has a real domain-specific surface.
Ensure `pyproject.toml` package data includes adapter assets:
```toml
package-data = { "langbot" = ["templates/**", "pkg/platform/sources/*", "pkg/platform/adapters/**", ...] }
```
## Implementation Checklist
- `manifest.yaml` declares `metadata.name`, config schema, supported events, common APIs, and platform-specific APIs.
- `adapter.py` creates the platform client, subscribes to native events, filters self/bot loops where appropriate, calls `event_converter.target2yiri(...)`, then dispatches the EBA event.
- `event_converter.py` maps native events to EBA event classes such as `MessageReceivedEvent`, `MessageEditedEvent`, `MessageDeletedEvent`, `MessageReactionEvent`, `MemberJoinedEvent`, `BotInvitedToGroupEvent`, and `PlatformSpecificEvent`.
- `message_converter.py` maps native messages to `MessageChain`, and maps `MessageChain` back to the platform send format.
- `api_impl.py` implements common EBA APIs: send, reply, edit, delete, forward, user/group/member lookup, moderation, upload/file URL, leave group.
- `platform_api.py` keeps platform-specific calls behind `call_platform_api(action, params)`.
- Unsupported common APIs must raise explicit SDK platform errors such as `NotSupportedError`; do not silently no-op.
- Destructive APIs such as kick, ban, leave, delete, or moderation must be gated in live tests and documented.
## Conversion Contract
For message events, the common shape should look like this regardless of platform:
```python
platform_events.MessageReceivedEvent(
type="message.received",
adapter_name="<platform>",
message_id=<platform_message_id>,
message_chain=platform_message.MessageChain([...]),
sender=platform_entities.User(...),
chat_type=platform_entities.ChatType.PRIVATE or ChatType.GROUP,
chat_id=<conversation_or_channel_id>,
group=platform_entities.UserGroup(...) or None,
source_platform_object=<raw_object>,
)
```
Message content should use common components:
- `Source` for original message id/time when available.
- `Plain` for text.
- `At` / `AtAll` for mentions.
- `Image`, `Voice`, `File` for media.
- `Forward` only when the platform can represent or emulate it safely.
If a platform event cannot cleanly map to a common event, emit `PlatformSpecificEvent` with a compact `action` and structured `data`.
## Unit Tests
Add focused tests under `LangBot/tests/unit_tests/platform/test_<platform>_eba_adapter.py`.
Cover at least:
- Manifest supported events match adapter `supported_events()`.
- Manifest supported APIs match adapter `supported_apis()`.
- Platform API map matches manifest actions.
- Dispatcher chooses the most specific EBA listener.
- Message converter maps every supported common component both directions where possible:
- `Source`
- `Plain`
- `At`
- `AtAll`
- `Image`
- `Voice`
- `File`
- `Quote`
- `Face`
- `Forward`
- `Unknown`
- mixed chains preserving order
- Event converter maps message received/edited/deleted/reaction, raw uncached gateway events, member events, and bot join/leave events.
- Send/reply methods pass correct platform kwargs and return `MessageResult`.
Run the existing reference adapter tests too:
```bash
cd LangBot
uv run pytest tests/unit_tests/platform/test_<platform>_eba_adapter.py tests/unit_tests/platform/test_telegram_eba_adapter.py
uv run python -m py_compile tests/e2e/live_<platform>_eba_probe.py
git diff --check
```
## Live Test Workflow
Direct adapter live probes are useful diagnostics, but they are not sufficient acceptance evidence for EBA. Treat `tests/e2e/live_<platform>_eba_probe.py` as an auxiliary tool only. The final adapter record must distinguish:
- `plugin-e2e-ui`: real SDK plugin through standalone runtime, LangBot core, adapter, and a real/simulator UI action. This can mark an inbound UI item complete.
- `plugin-e2e-protocol`: real SDK plugin through standalone runtime, LangBot core, adapter, and a protocol-boundary injected event. This is useful evidence but must not be claimed as UI coverage.
- `plugin-e2e-outbound`: real SDK plugin calls an API and the bot output is visible in the real/simulator UI. This can mark send/API coverage complete.
- `adapter-live`: direct adapter probe connected to a real/simulator endpoint. This is auxiliary only.
- `unit`: mocked conversion/API-shape coverage. This is auxiliary only.
- `not-supported`: platform protocol or SDK has no equivalent. Must include the reason.
- `blocked`: intended capability could not be verified. This is not complete.
Write a live probe in `LangBot/tests/e2e/live_<platform>_eba_probe.py`. It should:
1. Read token/client ids from environment variables or CLI args.
2. Start the adapter directly.
3. Register an EBA listener and write JSONL evidence to `LangBot/data/temp/`.
4. Wait for a real user/platform event instead of fabricating the entrypoint.
5. Exercise common APIs and `call_platform_api` actions.
6. Observe returned gateway events for edit/delete/reaction/member/bot lifecycle where available.
7. Print a summary containing passed, failed, skipped, and observed event types.
8. Redact or avoid printing secrets.
9. Keep destructive operations behind flags and run them last.
Use Computer Use when the user asks for real platform end-to-end coverage. Actually send messages/click reactions in the platform UI or otherwise trigger real user-side events; do not replace that with unit tests.
For media/component acceptance, keep the direction and trigger source explicit:
- Real inbound media only counts when a human-side platform UI or simulator UI sends the image/file/voice to the bot and the plugin JSONL records the corresponding common component.
- Bot outbound media only proves `send_message`/adapter send conversion. It does not prove inbound conversion.
- Protocol-boundary injection, such as sending a OneBot event directly into a reverse WebSocket adapter, is useful and should be labelled `plugin-e2e-protocol`, but it must not be reported as UI-level end-to-end media upload.
- If the UI cannot send or upload the media, record the item as `blocked` with the exact client/simulator limitation.
## Standalone Runtime + Plugin Test
When validating the whole LangBot EBA path, test with the SDK standalone runtime and a real test plugin. This is the required acceptance path; direct adapter calls do not prove the EBA architecture path.
The required path is:
```text
Real platform / simulator UI
-> platform SDK native event
-> adapter event converter
-> unified EBA event/entity/message types
-> LangBot core event dispatch
-> standalone SDK runtime
-> real test plugin listener
-> plugin calls platform APIs through SDK
-> LangBot core API dispatch
-> adapter API implementation
-> real platform / simulator UI
```
Typical shape:
```bash
# Terminal 1, SDK repo
cd langbot-plugin-sdk
uv run python -m langbot_plugin.cli.__init__ rt \
--debug-only \
--ws-control-port 5400 \
--ws-debug-port 5401 \
--skip-deps-check
# Terminal 2, LangBot repo
cd LangBot
export PYTHONPATH=/absolute/path/to/langbot-plugin-sdk/src:${PYTHONPATH:-}
uv run main.py --standalone-runtime
# Terminal 3, plugin directory
export DEBUG_RUNTIME_WS_URL=ws://127.0.0.1:5401/plugin/ws
export EBA_PROBE_LOG=/absolute/path/to/LangBot/data/temp/<platform>_eba_plugin_probe.jsonl
export EBA_PROBE_API=1
export EBA_PROBE_COMPONENT_SWEEP=1
export EBA_PROBE_PLATFORM_API=1
uv --project /absolute/path/to/langbot-plugin-sdk run python -m langbot_plugin.cli.__init__ run
```
Use an EBA probe plugin that subscribes to all relevant EBA event classes and runs SDK API calls after the first `MessageReceived`.
The plugin evidence should be JSONL and include:
- event class and `event.type`
- adapter name
- chat type and chat ID
- sender/user/group IDs with secrets redacted
- `bot_uuid` and `adapter_name`, proving LangBot filled common routing fields before plugin dispatch
- received `message_chain` component list
- API action name, input summary, result or error
- unsupported or blocked reason when an item is skipped
For full adapter acceptance, enable both probe sweeps:
- `EBA_PROBE_COMPONENT_SWEEP=1` sends the required outbound message components through `send_message`.
- `EBA_PROBE_PLATFORM_API=1` calls common safe APIs plus selected `call_platform_api` actions for the adapter.
The SDK must support `plugin.call_platform_api(bot_uuid, action, params)` for platform-specific acceptance. If the SDK cannot call a platform-specific action from the plugin, the adapter cannot be fully accepted even if direct adapter probes pass.
## Required EBA Acceptance Coverage
Before marking an adapter migrated, fill out an adapter record against `LangBot/docs/event-based-agents/adapters/acceptance-checklist.md`.
At minimum, the record must cover these categories:
- Message receive component tests through `plugin-e2e-ui`: `Source`, `Plain`, `At`, `AtAll`, `Image`, `Voice`, `File`, `Quote`, `Face`, `Forward`, `Unknown`, and mixed chains where the platform supports them. Protocol-only receive evidence must be labelled `plugin-e2e-protocol`.
- Message send component tests through `plugin-e2e-outbound`: `Plain`, `At`, `AtAll`, `Image`, `Voice`, `File`, `Quote`, `Face`, `Forward`, and mixed chains where the platform supports them.
- Every event declared in `manifest.yaml -> spec.supported_events`.
- Every common API declared in `manifest.yaml -> spec.supported_apis.required` and `optional`.
- Every action declared in `manifest.yaml -> spec.platform_specific_apis`.
- Compatibility tests for manifest declarations, legacy message listener fallback, EBA listener specificity, bot self-message filtering, and `source_platform_object` reply/debug behavior.
Do not declare an event or API in the manifest unless it has an implementation path and an acceptance entry. If a platform or simulator lacks a capability, document it as `not-supported` or `blocked` rather than silently omitting the test.
## Common Pitfalls
- `get_bots()` may return bot dictionaries, not UUID strings. Probe plugins should select an enabled dict and pass `bot["uuid"]` to `get_bot_info()` and `send_message()`.
- Make sure the probe subscribes to every event you claim to verify. Missing `MessageDeleted` subscription can make a working adapter look untested.
- Some platforms emit both cached and raw gateway events, producing duplicate evidence for delete/reaction. Count this explicitly; do not treat duplicates as failure unless semantics differ.
- Self-message filtering is platform-specific. Filter bot-originated `message.received` loops, but do not accidentally filter edit/delete events needed for bot-owned API probes.
- Reaction events may be filtered for bot self reactions. To test user reaction add/remove, use real UI interaction or a real user token path if permitted.
- File uploads usually happen as message attachments. A standalone `upload_file` API may need to be `NotSupportedError`.
- Live probes should not leak bot tokens through command output, logs, docs, or final answers.
- Discord requires privileged intents for message content and members. Missing intents can look like converter bugs.
- Telegram Bot API exposes only limited member lists; document capability gaps.
- Do not mark moderation APIs verified unless they ran against a disposable target member/bot.
- If `leave_group` is tested, run it last because the test bot will be removed from the server/group.
- Restore local LangBot DB/test state after live runs if you enabled temporary bots or changed plugin settings.
## Documentation Record
Add or update `LangBot/docs/event-based-agents/adapters/<platform>.md` in the same style as Telegram:
- Status and adapter directory.
- Configuration table matching manifest fields.
- Supported EBA event list.
- Common API table with support and limitations.
- `call_platform_api` action list.
- Receive component table with evidence level per component.
- Send component table with evidence level per component.
- Event table with evidence level per event.
- Common API table with evidence level per API.
- Platform-specific API table with evidence level per action.
- Live test record with exact date, endpoint/simulator, standalone runtime command, test plugin path/name, JSONL evidence path, channel/group type, observed events, APIs exercised, destructive operations, and skipped items.
Be honest. Put untested or skipped APIs in the document with the reason. Do not imply full parity when a platform cannot provide the same information density.
## Before Finishing
- Run unit tests and compile the live probe.
- Run the standalone runtime plugin E2E path for every required acceptance item that the platform supports.
- Run `git diff --check`.
- Summarize live JSONL evidence by event type.
- Stop all long-running runtimes and probes.
- Confirm no secrets are staged.
- Leave unrelated untracked files alone.
+28
View File
@@ -0,0 +1,28 @@
---
name: langbot-env-setup
description: Prepare a local LangBot development and testing environment for an AI agent. Use when setting up WSL or Linux development, shared local URL variables, proxy variables, backend/frontend startup, Playwright MCP browser access, GitHub OAuth browser login, persisted Chrome profiles, or future Codex computer-use environment paths.
---
# LangBot Environment Setup
Use this skill when a task needs LangBot to be in a testable state before product testing or development verification.
## Routing
- **Shared local variables**: read `../.env` before using URL, path, browser profile, or proxy defaults.
- **Always start here**: read `references/browser-access-selection.md` to choose the browser-control path.
- **LangBot service checks and startup**: read `references/service-startup.md`.
- **Computer Use available**: read `references/computer-use.md`. This path usually needs less browser/MCP setup.
- **No Computer Use, browser automation required**: read `references/playwright-mcp.md`.
- **GitHub OAuth or persisted login profile**: read `references/oauth-browser-profile.md`.
- **WSL-specific notes**: read `references/wsl-notes.md` only when running under WSL.
- **Proxy setup**: read `references/proxy.md` when external login, model provider tests, or package downloads time out.
- **Headless-only automation**: use only after a profile already contains a valid LangBot login. Do not ask the agent to enter GitHub credentials or 2FA.
## Rules
- Never handle the user's GitHub password, passkey, recovery code, or 2FA secret.
- For OAuth login, open a visible browser and let the user complete the credential steps.
- Reuse a fixed browser profile path so the agent can later access the logged-in LangBot session.
- Keep environment-specific paths and commands in `references/`, not in this file.
- Treat environment setup as complete only after the target LangBot services are reachable and the browser profile can access the WebUI.
@@ -0,0 +1,15 @@
# Browser Access Selection
Choose the lightest browser-control path that can complete the task.
## Decision Order
1. If Codex Computer Use, Claude Computer Use, or another visible browser-control tool is available, use `computer-use.md`.
2. If no computer-control tool is available but Playwright MCP is available, use `playwright-mcp.md`.
3. If the browser session must survive restarts or OAuth login is required, also use `oauth-browser-profile.md`.
4. If running under WSL, add `wsl-notes.md`.
5. If external sites or model providers time out, add `proxy.md`.
## Principle
Computer Use and Playwright MCP are alternative browser-control paths. Both still need LangBot services to be reachable, so service checks stay in `service-startup.md`.
@@ -0,0 +1,20 @@
# Computer Use Browser Path
Use this path when Codex Computer Use, Claude Computer Use, or another agent-visible browser-control capability is available.
## Why This Path Is Simpler
Computer Use can interact with a visible browser directly, so it usually does not need Playwright MCP configuration or a separate MCP browser bridge.
## Workflow
1. Verify LangBot backend/frontend with `service-startup.md`.
2. Open the WebUI in the controlled browser.
3. If login is needed, let the user complete GitHub OAuth. Never handle credentials or 2FA.
4. Keep the browser/profile available for later testing.
5. Hand off to `langbot-testing` after the page shows the logged-in WebUI.
## Still Required
- Proxy may still be needed for GitHub OAuth or model provider tests. Use `proxy.md`.
- Persisted profile details may still matter if the computer-control browser is restarted. Use `oauth-browser-profile.md` if login state must survive.
@@ -0,0 +1,62 @@
# OAuth Browser Profile
Use this reference when LangBot or LangBot Space needs GitHub OAuth login and the agent must reuse the authenticated browser state later.
Read `skills/.env` first for `LANGBOT_BACKEND_URL`, `LANGBOT_FRONTEND_URL`, `LANGBOT_BROWSER_PROFILE`, `LANGBOT_CHROMIUM_EXECUTABLE`, and proxy defaults.
## Rules
- Never handle the user's GitHub password, passkey, recovery code, or 2FA secret.
- Open a visible browser and let the user complete credential steps.
- Reuse a fixed browser profile path.
- Do not print token values. It is acceptable to report localStorage key names.
## Manual Visible Login Flow
1. Verify LangBot backend is reachable with `service-startup.md`.
2. Launch a visible Chromium window with the persistent profile:
```bash
setsid "$LANGBOT_CHROMIUM_EXECUTABLE" \
--no-sandbox \
--ozone-platform=x11 \
--user-data-dir="$LANGBOT_BROWSER_PROFILE" \
--proxy-server="$LANGBOT_PROXY_SOCKS" \
--proxy-bypass-list="$LANGBOT_NO_PROXY" \
"$LANGBOT_BACKEND_URL/login" \
>/tmp/langbot-visible-chrome.log 2>&1 < /dev/null &
```
3. The user completes:
```text
Login with Space -> Login with GitHub -> GitHub credentials / 2FA -> authorize
```
4. The agent can then reuse the same profile for automated checks.
## Expected Successful State
After login, LangBot should redirect away from `/login`, for example to a `/home/...` URL on the selected origin.
Expected visible signals:
```text
LangBot
Dashboard
Home
Bots
Pipelines
Knowledge
Extensions
```
Expected localStorage key names:
```text
token
userEmail
langbot_language
```
If the user logged in on one origin but `LANGBOT_FRONTEND_URL` still shows `/login`, copy only the auth state needed between origins. Do not print token values.
@@ -0,0 +1,30 @@
# Playwright MCP Browser Path
Use this path when the agent needs browser automation but no Computer Use browser-control path is available.
## Known Paths
- Persistent browser profile: `LANGBOT_BROWSER_PROFILE` from `skills/.env.local`
- Chromium executable: `LANGBOT_CHROMIUM_EXECUTABLE` from `skills/.env.local`
- Codex MCP config: `$CODEX_HOME/config.toml` or the config path used by the active agent.
## MCP Config
Keep the profile path fixed so the agent can reuse authenticated state.
```toml
[mcp_servers.playwright]
command = "npx"
args = ["-y", "@playwright/mcp@latest", "--no-sandbox", "--executable-path", "<LANGBOT_CHROMIUM_EXECUTABLE>", "--proxy-server", "<LANGBOT_PROXY_SOCKS>", "--proxy-bypass", "localhost,127.0.0.1", "--user-data-dir", "<LANGBOT_BROWSER_PROFILE>"]
```
After changing MCP config, restart Codex so the MCP server is relaunched with the new args.
## Visible Login
For OAuth login, Playwright MCP's headless browser is not enough. Launch a visible browser with the same profile and let the user complete login. Use `oauth-browser-profile.md`.
## Common Failures
- MCP still uses old args after editing config: restart Codex or kill old `playwright-mcp` processes and restart the session.
- Browser is headless during OAuth: use the visible login command from `oauth-browser-profile.md`.
@@ -0,0 +1,30 @@
# Proxy Setup
Use this reference when GitHub OAuth, package installation, model provider tests, or external API calls time out.
Read defaults from `skills/.env` first.
## Standard Local Proxy
```bash
export HTTP_PROXY="$LANGBOT_PROXY_HTTP"
export HTTPS_PROXY="$LANGBOT_PROXY_HTTP"
export ALL_PROXY="$LANGBOT_PROXY_SOCKS"
export http_proxy="$LANGBOT_PROXY_HTTP"
export https_proxy="$LANGBOT_PROXY_HTTP"
export all_proxy="$LANGBOT_PROXY_SOCKS"
export NO_PROXY="$LANGBOT_NO_PROXY"
export no_proxy="$LANGBOT_NO_PROXY"
```
## Rule
Keep uppercase and lowercase proxy variables consistent. Different libraries read different names.
## Checks
```bash
env | rg -i '^(http|https|all|no)_?proxy='
curl -I --max-time 8 --proxy "$LANGBOT_PROXY_SOCKS" https://github.com
curl -I --max-time 3 "$LANGBOT_BACKEND_URL"
```
@@ -0,0 +1,77 @@
# Service Startup
Use this reference for LangBot backend/frontend readiness checks regardless of OS or browser-control method. Read `skills/.env` first and override those defaults with user-provided values or detected running services.
## Variables
- `LANGBOT_REPO`
- `LANGBOT_WEB_REPO`
- `LANGBOT_BACKEND_URL`
- `LANGBOT_FRONTEND_URL`
- `LANGBOT_DEV_FRONTEND_URL`
## Backend
Start LangBot from the backend repo:
```bash
cd "$LANGBOT_REPO"
uv run main.py
```
Healthy startup includes:
```text
Running on http://0.0.0.0:<backend-port>
Connected to plugin runtime.
Plugin langbot/local-agent initialized
```
Quick check:
```bash
curl -I --max-time 3 "$LANGBOT_BACKEND_URL/login"
```
If `bin/lbs env doctor` reports that `LANGBOT_BACKEND_URL` has no TCP listener,
the backend is not running at the configured host and port. A reachable
standalone frontend on `LANGBOT_FRONTEND_URL` does not prove backend readiness.
Prefer a visible terminal session while debugging backend startup. Detached
background startup methods can hide early process exits in local agent runs; if
you use one, immediately verify both the process and the listener:
```bash
ps -eo pid,cmd | rg 'main.py|uv run main|langbot'
ss -ltnp | rg ':5300'
curl -I --max-time 3 "$LANGBOT_BACKEND_URL/login"
```
## Frontend
Start the new frontend from the web repo:
```bash
cd "$LANGBOT_WEB_REPO"
VITE_API_BASE_URL="$LANGBOT_BACKEND_URL" pnpm dev --host 0.0.0.0
```
Healthy startup includes:
```text
Local: <frontend-url>
```
Quick check:
```bash
curl -I --max-time 3 "$LANGBOT_FRONTEND_URL"
```
If `VITE_API_BASE_URL` is missing, Vite still serves the page but frontend API
calls may go to the frontend port instead of the backend port. That produces
false browser failures in login, wizard, pipeline, and Debug Chat cases.
## Completion Signal
Environment setup is not complete until the required frontend/backend URLs are reachable and the chosen browser-control path can open the WebUI.
@@ -0,0 +1,36 @@
# WSL Notes
Use this reference only for WSL-specific details. Do not put generic LangBot startup or browser-login steps here.
## Network
GitHub login and model provider calls may require proxy access from WSL.
Working proxy form:
```bash
socks5://127.0.0.1:7890
```
Bypass local LangBot:
```bash
localhost,127.0.0.1
```
Quick checks:
```bash
curl -I --max-time 8 --proxy socks5h://127.0.0.1:7890 https://github.com
curl -I --max-time 3 "$LANGBOT_BACKEND_URL"
```
## Visible Browser
If OAuth requires a visible browser, WSL must have a usable display path. If a visible Chromium launch fails, check the local WSL GUI/X11 setup before changing LangBot config.
## Common Failures
- `ERR_NETWORK_CHANGED` or GitHub timeout: browser is not using the SOCKS proxy.
- LangBot connection refused: backend is not running or not reachable from WSL.
- User cannot type credentials: browser is headless or not visible to the user.
+108
View File
@@ -0,0 +1,108 @@
---
name: langbot-mcp-ops
description: Operate a LangBot instance through its built-in MCP (Model Context Protocol) server. Use when an AI agent needs to manage LangBot — list/create/update/delete bots, pipelines, models, knowledge bases, MCP servers, and skills — over MCP instead of raw HTTP. Covers the /mcp endpoint, API-key auth (web-UI lbk_ keys and the config.yaml global key), the tool surface, and client configuration. Triggers on "langbot mcp", "manage langbot via mcp", "langbot /mcp", "langbot mcp server".
---
# LangBot MCP Operations
LangBot exposes an **MCP server** so AI agents can manage an instance
programmatically. It mirrors a curated subset of the HTTP service API.
## Endpoint
```
http://<langbot-host>:5300/mcp
```
Transport: **streamable HTTP** (stateless, JSON responses). Same host/port as
the web UI and HTTP API.
## Authentication
Reuses the same API keys as the HTTP API. Send either header:
```
X-API-Key: <api-key>
# or
Authorization: Bearer <api-key>
```
Two kinds of key are accepted:
1. **Web-UI key** — created in the web UI (sidebar → API Keys), prefixed `lbk_`.
The secret is shown once; only its SHA-256 hash is stored. Each key is bound
to one Workspace and has explicit scopes, status, optional expiry, and
last-used metadata. The key determines the Workspace; callers cannot switch
it with `X-Workspace-Id`.
2. **Global API key** — set in `data/config.yaml` under `api.global_api_key`.
Requires no login session and no DB record; does not need the `lbk_` prefix.
It is accepted only by a community instance with exactly one local
Workspace and is disabled for SaaS multi-Workspace operation. Leave empty to
disable. See the `langbot-deploy` skill for config details.
Invalid, revoked, or expired keys get `401 Unauthorized`. A valid key whose
scopes do not authorize a tool gets `403 Forbidden`.
## Client configuration
```json
{
"mcpServers": {
"langbot": {
"url": "http://<langbot-host>:5300/mcp",
"headers": { "X-API-Key": "<api-key>" }
}
}
}
```
## Tool surface
The tools wrap the LangBot service layer. Current tools (v1):
| Tool | Purpose |
| --- | --- |
| `get_system_info` | Version, edition, instance id |
| `list_bots` / `get_bot` / `create_bot` / `update_bot` / `delete_bot` | Manage messaging-platform bots (secrets redacted on read) |
| `list_pipelines` / `get_pipeline` / `create_pipeline` / `update_pipeline` / `delete_pipeline` | Manage pipelines |
| `list_llm_models` / `get_llm_model` / `list_embedding_models` / `list_model_providers` | Inspect models & providers |
| `list_knowledge_bases` / `get_knowledge_base` / `retrieve_knowledge_base` | RAG knowledge bases (incl. semantic search) |
| `list_mcp_servers` | External MCP servers LangBot connects to (as a client) |
| `list_skills` / `get_skill` | Installed skills |
Mutating tools (`create_*`, `update_*`) take a JSON object matching the same
shape as the corresponding HTTP API request body. Discover resources with the
`list_*` / `get_*` tools before mutating; identifiers are UUIDs. Reads require
`resource.view`; mutations require `resource.manage`. All service calls inherit
the immutable Workspace context authenticated at the MCP transport boundary.
## How to use
1. Get an API key (web UI key, or set `api.global_api_key` in config.yaml).
2. Point your MCP client at `http://<host>:5300/mcp` with the key header.
3. Call `get_system_info` to confirm connectivity.
4. Use `list_*` tools to discover, then `get_*` / `create_*` / `update_*` /
`delete_*` as needed.
## Implementation & maintenance (for LangBot developers)
- Server: `src/langbot/pkg/api/mcp/server.py` (FastMCP). Tools call the service
layer directly, so the MCP surface stays aligned with the API.
- Mount: `src/langbot/pkg/api/mcp/mount.py` — an ASGI dispatcher fronting Quart,
authenticating `/mcp` requests, running the streamable-HTTP session manager.
- Smoke test: `tests/manual/mcp_smoke.py`.
> When you add, remove, or change an HTTP API endpoint that should be
> agent-accessible, update the corresponding MCP tool **and** this skill. The
> MCP tool surface and the API must stay aligned (see `AGENTS.md`).
## Pitfalls
- `/mcp` is the **server** LangBot exposes. The `/api/v1/mcp` routes are the
**client** side (managing external MCP servers LangBot connects to). Don't
confuse them.
- A `401` means the key is wrong, missing, revoked, expired, or (for the global
key) `api.global_api_key` is empty or the instance is not an OSS singleton.
- A `403` means the key is valid but lacks the permission required by the tool.
- The global key is plaintext in config.yaml — only enable it on trusted/internal
deployments and serve over HTTPS.
+484
View File
@@ -0,0 +1,484 @@
---
name: langbot-plugin-dev
description: Develop, debug, and test LangBot plugins. Use when creating new LangBot plugins, fixing plugin bugs, setting up a LangBot test environment, or testing plugins via WebSocket. Covers plugin component architecture (EventListener, Command, Tool), the plugin SDK API (invoke_llm, get_llm_models, send_message, plugin storage), common pitfalls, and automated WebSocket-based testing. Triggers on "langbot plugin", "lbp", "GroupChatSummary", "plugin debug", "langbot test".
---
# LangBot Plugin Development & Debugging
## Controlling a running instance via MCP
Beyond writing code, you can **drive a live LangBot instance over MCP** — no raw
HTTP needed. Two MCP servers exist (both reuse existing API keys; see `AGENTS.md`):
- **LangBot instance**`http://<host>:5300/mcp` (auth: web-UI `lbk_` key or the
`api.global_api_key` from `config.yaml`). Manage bots, pipelines, models,
knowledge bases, and skills. See the **`langbot-mcp-ops`** skill.
- **LangBot Space marketplace**`https://space.langbot.app/mcp` (auth: Personal
Access Token). Search plugins / MCP servers / skills. See the
**`langbot-space-ops`** skill.
> Any change to an agent-accessible HTTP API endpoint must keep the matching MCP
> tool and these skills in sync.
## Plugin Architecture
A LangBot plugin consists of:
```
MyPlugin/
├── manifest.yaml # Plugin metadata, config schema
├── main.py # BasePlugin subclass (entry point, shared state)
├── components/
│ ├── event_listener/ # Hook pipeline events
│ │ ├── collector.yaml
│ │ └── collector.py
│ ├── commands/ # !command handlers
│ │ ├── mycommand.yaml
│ │ └── mycommand.py
│ └── tools/ # LLM function-call tools
│ ├── mytool.yaml
│ └── mytool.py
```
Each component has a `.yaml` (metadata) and `.py` (implementation).
## README & i18n convention (enforced on the marketplace)
A plugin published to LangBot Space serves a localized README on its detail page.
The resolver (`langbot-space` `PluginService.GetPluginREADME`) works like this:
- **Root `README.md` MUST be in English.** It is the default and the fallback —
when no per-language README matches the viewer's locale, the page serves the
root `README.md`. A non-English root README makes the English/default view show
the wrong language.
- **All other languages live under `readme/README_{lang}.md`** — e.g.
`readme/README_zh_Hans.md`, `readme/README_ja_JP.md`. The 8 supported locales:
`en_US, zh_Hans, zh_Hant, ja_JP, th_TH, vi_VN, es_ES, ru_RU`.
- `manifest.yaml` `metadata.label` / `metadata.description` should carry the same
8-locale i18n set (`repository` must be a real, alive URL).
```
MyPlugin/
├── manifest.yaml
├── README.md # English (default + fallback) — REQUIRED, must be English
└── readme/
├── README_zh_Hans.md
├── README_zh_Hant.md
├── README_ja_JP.md
├── README_th_TH.md
├── README_vi_VN.md
├── README_es_ES.md
└── README_ru_RU.md
```
`manifest.yaml` (incl. `repository`) is the source of truth — the marketplace
syncs from it, so edit the package and re-publish rather than patching live data.
## Critical SDK Pitfalls
### 1. MessageChain is a RootModel — iterate directly
```python
# ❌ WRONG — MessageChain has no .components attribute
for component in event.message_chain.components:
# ✅ CORRECT — MessageChain is a Pydantic RootModel, iterate directly
for component in event.message_chain:
```
### 2. Message.content must be `list[ContentElement]` or `str`, not a single ContentElement
```python
from langbot_plugin.api.entities.builtin.provider import message as provider_message
# ❌ WRONG — single ContentElement
Message(role="user", content=ContentElement.from_text("hello"))
# ✅ CORRECT — list of ContentElement
Message(role="user", content=[ContentElement.from_text("hello")])
# ✅ ALSO CORRECT — plain string
Message(role="user", content="hello")
```
### 3. invoke_llm does NOT accept timeout
```python
# ❌ WRONG
await self.invoke_llm(llm_model_uuid=uuid, messages=msgs, timeout=60)
# ✅ CORRECT
await self.invoke_llm(llm_model_uuid=uuid, messages=msgs)
```
### 4. invoke_llm response.content can be str OR list
```python
response = await self.invoke_llm(...)
if response.content:
if isinstance(response.content, str):
return response.content
elif isinstance(response.content, list):
parts = [e.text for e in response.content if hasattr(e, "text") and e.text]
return "\n".join(parts)
```
### 5. get_llm_models() returns UUIDs
```python
# Returns list[str] of model UUIDs
models = await self.get_llm_models()
model_uuid = models[0] # First available model UUID
```
**Known bug (v4.9.3):** The host handler may return `list[dict]` instead of `list[str]`. If you hit `TypeError: unhashable type: 'dict'` in `invoke_llm`, the fix is in `LangBot/src/langbot/pkg/plugin/handler.py` — change `'llm_models': llm_models` to `'llm_models': [m['uuid'] for m in llm_models]`.
### 6. invoke_llm parameter is `llm_model_uuid`, NOT `model_uuid`
```python
# ❌ WRONG — will throw "got an unexpected keyword argument"
await self.invoke_llm(messages=msgs, model_uuid=uuid)
# ✅ CORRECT
await self.invoke_llm(messages=msgs, llm_model_uuid=uuid)
```
### 7. prevent_default() alone does NOT block LLM response
To fully prevent the default LLM pipeline from responding when your EventListener handles the message, you must call **both**:
```python
event_context.prevent_default() # Block default behavior
event_context.prevent_postorder() # Block subsequent plugins/pipeline
```
Using only `prevent_default()` still allows the LLM to generate a response.
### 8. get_plugin_storage / set_plugin_storage may throw KeyError: 'owner'
This is a version mismatch between the SDK and host. Wrap storage calls in try/except:
```python
try:
data = await self.get_plugin_storage("my_key")
except Exception:
data = None # Fallback gracefully
```
### 9. Component YAML must have full structure, not just name/description
```yaml
# ❌ WRONG — will silently fail to register the component
name: translator
description:
en_US: 'Does stuff'
# ✅ CORRECT — full component YAML
apiVersion: v1
kind: EventListener
metadata:
name: translator
label:
en_US: Translator
spec:
execution:
python:
path: translator.py
attr: Translator
```
### 10. BasePlugin import path
```python
# ❌ WRONG
from langbot_plugin.api.definition.base_plugin import BasePlugin
# ✅ CORRECT
from langbot_plugin.api.definition.plugin import BasePlugin
```
## Pipeline Events
Events the EventListener can hook (from most general to most specific):
| Event | When |
|---|---|
| `GroupMessageReceived` | **Any** group message arrives (before trigger rules) |
| `PersonMessageReceived` | **Any** private message arrives |
| `GroupNormalMessageReceived` | Group message passes trigger rules, going to LLM |
| `PersonNormalMessageReceived` | Private message going to LLM |
| `GroupCommandSent` | Group message matched as command |
| `PersonCommandSent` | Private message matched as command |
| `NormalMessageResponded` | LLM generated a response |
| `PromptPreProcessing` | About to build LLM context |
**Key insight:** `*MessageReceived` fires for ALL messages regardless of trigger rules. `*NormalMessageReceived` only fires for messages that match the pipeline's trigger rules (e.g., @bot, prefix, random%). Use `*MessageReceived` for message collection/logging.
## EventContext API
```python
@self.handler(events.GroupMessageReceived)
async def on_msg(event_context: context.EventContext):
event = event_context.event
event.launcher_id # Group ID
event.sender_id # Sender ID
event.message_chain # MessageChain (iterate directly)
# Reply to the current conversation
await event_context.reply(MessageChain([Plain(text="hello")]))
# Block default pipeline behavior
event_context.prevent_default()
# Block subsequent plugins
event_context.prevent_postorder()
```
## Setting Up a Test Environment
### Deploy via Docker (GitOps + Portainer)
See `references/test-env-setup.md` for full deployment steps.
Quick summary:
1. Create `docker-compose.yaml` in `server-deploy` repo
2. Deploy via Portainer git repository method
3. Set up admin account via `/api/v1/user/init` POST
4. Configure LLM provider and model via API
5. Copy plugin to `data/plugins/` directory
### WebSocket Testing
LangBot's WebUI chat uses WebSocket. Connect to test message flow:
```
ws://<host>:<port>/api/v1/pipelines/<pipeline_uuid>/ws/connect?session_type=group
```
- `session_type=group` for group chat simulation
- `session_type=person` for private chat (always triggers pipeline)
**Requires Origin header** to pass CORS:
```javascript
const ws = new WebSocket(url, {
headers: { Origin: 'https://your-langbot-domain' }
});
```
Send messages:
```json
{"type": "message", "message": [{"type": "Plain", "text": "hello"}]}
```
Receive:
- `{"type": "connected", ...}` — connection established
- `{"type": "user_message", "data": {...}}` — echo of sent message
- `{"type": "response", "data": {"content": "...", "is_final": true/false}}` — bot reply (streamed)
### Group Trigger Rules
Group messages only enter the pipeline if trigger rules are met:
```json
{
"group-respond-rules": {
"at": true, // Respond when @bot
"prefix": ["ai"], // Respond to messages starting with "ai"
"random": 0.0, // Probability of responding to any message (0.0-1.0)
"regexp": [] // Regex patterns
}
}
```
For testing, set `random: 1.0` via PUT `/api/v1/pipelines/<uuid>` to respond to all messages.
**Important:** EventListener hooks like `GroupMessageReceived` fire regardless of trigger rules. Only the LLM processing (`GroupNormalMessageReceived` and beyond) requires trigger rules.
### Plugin Hot-Reload
There is **no hot-reload**. After changing plugin files:
```bash
docker restart <runtime-container>
# Wait ~5 seconds for plugin to re-mount
```
The main LangBot container does NOT need restart for plugin changes — only the runtime container.
## API Quick Reference
### Admin Setup
```bash
# Initialize admin account (first time only)
curl -X POST $BASE/api/v1/user/init \
-H "Content-Type: application/json" \
-d '{"user":"admin@test.com","password":"test123"}'
# Login
curl -X POST $BASE/api/v1/user/auth \
-H "Content-Type: application/json" \
-d '{"user":"admin@test.com","password":"test123"}'
# Returns: {"data":{"token":"eyJ..."}}
```
### Provider & Model Setup
```bash
# Create provider
curl -X POST $BASE/api/v1/provider/providers \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"MyProvider","requester":"new-api-chat-completions","base_url":"https://api.example.com/v1","api_keys":["sk-xxx"]}'
# Create LLM model
curl -X POST $BASE/api/v1/provider/models/llm \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"name":"gpt-4o-mini","provider_uuid":"<uuid>","abilities":["chat","tool-use"]}'
# List models
curl $BASE/api/v1/provider/models/llm -H "Authorization: Bearer $TOKEN"
```
### Pipeline Config
```bash
# Get pipeline
curl $BASE/api/v1/pipelines -H "Authorization: Bearer $TOKEN"
# Update pipeline (e.g., set model, modify trigger rules)
curl -X PUT $BASE/api/v1/pipelines/<uuid> \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '<full pipeline JSON>'
```
## Plugin Config Types
Supported `type` values in `manifest.yaml` `spec.config`:
| Type | Description | Value |
|---|---|---|
| `string` | Text input | string |
| `int` / `integer` | Number input | int |
| `float` | Decimal input | float |
| `bool` / `boolean` | Toggle | bool |
| `select` | Dropdown (needs `options`) | string |
| `prompt-editor` | Multi-line prompt editor | string |
| `llm-model-selector` | LLM model picker UI | UUID string |
| `bot-selector` | Bot picker UI | UUID string |
Example — let users choose which model the plugin uses:
```yaml
spec:
config:
- name: model
type: llm-model-selector
label:
en_US: 'LLM Model'
zh_Hans: 'LLM 模型'
description:
en_US: 'Select the LLM model. Falls back to first available if not set.'
zh_Hans: '选择 LLM 模型。未设置时使用第一个可用模型。'
required: false
```
Read config in plugin code:
```python
model_uuid = self.get_config().get("model")
```
## Container Restart Timing
After plugin file changes, **only the runtime container needs restart**:
```bash
docker restart langbot-test-runtime
# Wait ~15 seconds before testing
```
**When to restart both (runtime first, then host):**
- Added/removed Command or Tool components (host caches component lists)
- Changed `manifest.yaml` structure
```bash
docker restart langbot-test-runtime
sleep 8
docker restart langbot-test
sleep 8
```
**⚠️ Do NOT restart both simultaneously** — the host may connect before plugins are mounted, causing 502 errors or missing plugin registrations.
## Debugging Checklist
When a plugin doesn't work:
1. **Check runtime logs**: `docker logs <runtime-container>` — look for mount/init errors
2. **Check host logs**: `docker logs <langbot-container>` — look for pipeline processing errors
3. **Verify plugin loaded**: `GET /api/v1/plugins` — should list your plugin
4. **Test person mode first**: `session_type=person` always triggers pipeline, isolating trigger rule issues
5. **Check trigger rules**: Group mode requires @bot, prefix match, or random% to enter pipeline
6. **Verify model configured**: Pipeline's `config.ai.local-agent.model.primary` must point to a valid model UUID with working API keys
## Publishing Plugins
After testing, publish via `lbp publish`:
```bash
cd /path/to/MyPlugin
lbp publish
```
This builds `.lbpkg` and uploads to Space marketplace as a draft. Then go to https://space.langbot.app/market to upload screenshots and submit for review.
**Prerequisite:** Must be logged in via `lbp login --token lbpat_xxx` (PAT from Space profile page).
## Reference: EventListener-Only Plugin Pattern
For plugins that react to messages without commands or tools (e.g., auto-summarize URLs, collect messages, translate):
```
MyPlugin/
├── manifest.yaml # Only EventListener in spec.components
├── main.py # BasePlugin with shared logic (fetch, LLM calls)
├── components/
│ └── event_listener/
│ ├── detector.yaml
│ └── detector.py
└── requirements.txt
```
**manifest.yaml** — only declare EventListener:
```yaml
spec:
components:
EventListener:
fromDirs:
- path: components/event_listener/
```
**detector.py** — hook `*MessageReceived`, extract text, process, reply:
```python
@self.handler(events.PersonMessageReceived)
async def on_msg(event_context: context.EventContext):
event = event_context.event
text_parts = []
for component in event.message_chain:
if isinstance(component, platform_message.Plain):
text_parts.append(component.text)
text = "".join(text_parts).strip()
if should_handle(text):
event_context.prevent_default()
event_context.prevent_postorder()
result = await self.plugin.process(text)
await event_context.reply(platform_message.MessageChain([
platform_message.Plain(text=result)
]))
```
**Key:** Access shared plugin logic via `self.plugin` (the BasePlugin instance).
@@ -0,0 +1,116 @@
# Test Environment Setup
## Docker Compose (GitOps)
Create in `server-deploy` repo under `servers/<hostname>/langbot-test/docker-compose.yaml`:
```yaml
version: "3"
services:
langbot_plugin_runtime:
image: rockchin/langbot:latest
container_name: langbot-test-runtime
volumes:
- /opt/docker-data/langbot-test/data/plugins:/app/data/plugins
ports:
- "5411:5401"
restart: on-failure
environment:
- TZ=Asia/Shanghai
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "rt"]
networks:
- langbot_test_network
langbot:
image: rockchin/langbot:latest
container_name: langbot-test
volumes:
- /opt/docker-data/langbot-test/data:/app/data
ports:
- "5310:5300"
restart: on-failure
depends_on:
- langbot_plugin_runtime
environment:
- TZ=Asia/Shanghai
networks:
- langbot_test_network
networks:
langbot_test_network:
driver: bridge
```
## Post-Deploy Configuration
After first start, LangBot auto-generates `data/config.yaml`. You need to update `plugin.runtime_ws_url` to match the runtime container name:
```bash
# On the host, edit config
sed -i 's|ws://localhost:5400/control/ws|ws://langbot-test-runtime:5400/control/ws|' \
/opt/docker-data/langbot-test/data/config.yaml
docker restart langbot-test
```
## Installing a Plugin
Copy plugin directory to `data/plugins/` on the host:
```bash
scp -r MyPlugin/ user@host:/opt/docker-data/langbot-test/data/plugins/MyPlugin/
docker restart langbot-test-runtime # Runtime picks up new plugins on restart
```
## Caddy Reverse Proxy (Optional)
If testing externally, add to Caddyfile on the same host:
```
langbot-test.example.com {
reverse_proxy langbot-test:5300
}
```
Then reload: `docker exec caddy caddy reload --config /etc/caddy/Caddyfile`
The WebSocket endpoint works through Caddy without special config.
## WebSocket Test Script (Node.js)
```javascript
const WebSocket = require('ws');
const PIPELINE_UUID = '<your-pipeline-uuid>';
const BASE = 'wss://langbot-test.example.com';
const URL = `${BASE}/api/v1/pipelines/${PIPELINE_UUID}/ws/connect?session_type=group`;
const ws = new WebSocket(URL, {
headers: { Origin: BASE }
});
const send = (text) => {
ws.send(JSON.stringify({
type: 'message',
message: [{ type: 'Plain', text }]
}));
console.log('[SENT]', text);
};
ws.on('message', (data) => {
const msg = JSON.parse(data.toString());
if (msg.type === 'connected') {
console.log('Connected!');
// Send test messages
send('Message 1');
setTimeout(() => send('Message 2'), 500);
setTimeout(() => send('!summary'), 2000);
} else if (msg.type === 'response' && msg.data?.is_final) {
console.log('[BOT]', msg.data.content);
}
});
ws.on('error', (e) => console.error('Error:', e.message));
setTimeout(() => { ws.close(); process.exit(); }, 60000);
```
Requires: `npm install ws`
@@ -0,0 +1,40 @@
---
name: langbot-skills-maintenance
description: Maintain the langbot-skills repository with low duplication. Use when adding, editing, or auditing LangBot skills, references, cases, troubleshooting entries, indexes, or periodic entropy-control checks for this skills repository.
---
# LangBot Skills Maintenance
Use this skill before changing reusable assets in this repository.
## Workflow
1. Read `AGENTS.md`, `skills/.env`, and the relevant existing skill files.
2. Classify the change:
- `SKILL.md` for routing and concise operating rules.
- `references/*.md` for canonical detailed workflows.
- `cases/*.yaml` for executable test-plan skeletons.
- `suites/*.yaml` for reusable groups of case ids.
- `fixtures/fixtures.json` for deterministic fixture readiness metadata.
- `reports/evidence/<run-id>/automation-result.json` as automation output and `reports/evidence/<run-id>/result.json` as final judgment output; neither is a catalog asset to commit.
- `troubleshooting/*.yaml` for one reusable failure mode.
3. Search existing assets before adding new files:
- `rg "<feature|error|case id>" skills`
- `bin/lbs case list`
- `bin/lbs suite list`
- `bin/lbs fixture list`
4. Put detail in one canonical place and link to it from cases or routing bullets.
5. Run the checks in `AGENTS.md` after edits.
## Entropy Rules
- Prefer extending an existing reference or troubleshooting entry when the root cause is the same.
- Keep cases short: setup, action, evidence, pass/fail checks. Do not paste long prompts or debug transcripts when a reference exists.
- Put machine-checkable inputs in `env`, `automation_env`, or fixtures; put operator-confirmed assumptions in `preconditions` so `test plan` can surface `manual_check`.
- Keep suites short: title, intent, tags, and ordered case ids. Do not duplicate case steps inside a suite.
- Keep fixture manifests factual: id, title, path, kind, and related case ids. Do not encode environment-specific absolute paths.
- Keep troubleshooting entries narrow: symptoms, patterns, likely causes, fixes, related assets.
- Do not hardcode local ports, browser profile paths, secrets, tokens, or provider keys.
- Use `bin/lbs index --check` to verify the committed index is current without writing it; run `bin/lbs index` when the index needs regeneration.
For periodic repository audits, read `references/curation-workflow.md`.
@@ -0,0 +1,70 @@
# Curation Workflow
Use this checklist when the repository starts accumulating repeated cases, copied steps, or overlapping troubleshooting entries.
## Audit Pass
1. Inspect the current surface:
- `bin/lbs case list`
- `bin/lbs case list --json --priority p0 --automation`
- `bin/lbs case list --ready`
- `bin/lbs case list --machine-ready`
- `bin/lbs suite list`
- `bin/lbs fixture list`
- `rg "sandbox|provider|pipeline|plugin|knowledge|mcp" skills`
- `rg "If .* fails|Known Pitfalls|Debug Chat|/api/v1" skills`
2. Group nearby assets by intent, not by file path:
- user-facing scenario
- backend or provider dependency
- failure signature
- pass/fail evidence
3. Pick one canonical owner:
- stable procedures belong in `references/`
- deterministic files and packages belong in `fixtures/` plus `fixtures/fixtures.json`
- repeated failure signatures belong in `troubleshooting/`
- runnable QA paths belong in `cases/`
- reusable groups of QA paths belong in `suites/`
- skill entry points belong in `SKILL.md`
## Merge Or Split
Merge when two files share the same trigger, root cause, and fix. Keep the stronger id and move missing patterns into it.
Split when a file mixes unrelated failure modes or requires different fixes. Each troubleshooting id should map to one diagnosis path.
Move repeated step lists out of cases and into a reference when more than one case would need the same prompt, UI path, or log interpretation.
Add or update a suite when developers repeatedly run the same ordered group of cases. Do not copy case steps into suites; use `bin/lbs suite plan <suite-id>` to expand the group.
Use `bin/lbs suite start <suite-id>` and `bin/lbs suite report <suite-id> --evidence-dir <dir>` when validating that a suite is operational end to end.
Add or update `fixtures/fixtures.json` when a case depends on a deterministic file, plugin package, or local test server. The manifest should use repo-relative paths under the owning skill and should not contain machine-local absolute paths.
When adding Debug Chat Playwright automation, reuse `scripts/e2e/lib/debug-chat.mjs` for navigation, prompt send, response leaf matching, and known failure classification. Keep case-specific prompts and expected sentinels in case YAML automation fields when possible.
## Case Review
For every changed case:
1. Ensure `steps` describe what to execute, not every command in the underlying implementation.
2. Ensure `checks` contain observable UI, log, network, or filesystem evidence.
3. Ensure `diagnostics` are fallback investigation hints, not pass criteria.
4. Ensure `priority`, `risk`, `ci_eligible`, and `evidence_required` match the actual repeatability and evidence burden.
5. Put must-have env vars in `env` / `automation_env`; put one-of choices such as URL-or-name in `env_any` / `automation_env_any`.
6. Ensure linked `skills` and `troubleshooting` ids exist.
7. Run:
```bash
bin/lbs validate
bin/lbs index --check
bin/lbs index
bin/lbs test plan <case-id>
```
## Final Gate
Before handing off:
- `git diff --stat` should show a focused change set.
- `skills.index.json` should be regenerated only by `bin/lbs index`.
- No new asset should contain local credentials, OAuth tokens, API keys, or copied localStorage values.
- The final note should say which checks ran and which cases or troubleshooting ids changed.
+79
View File
@@ -0,0 +1,79 @@
---
name: langbot-space-ops
description: Browse and search the LangBot Space marketplaces (plugins, MCP servers, skills) through the Space MCP server. Use when an AI agent needs to discover LangBot extensions on space.langbot.app over MCP. Covers the /mcp endpoint, Personal Access Token (PAT) auth, the tool surface, and client configuration. Triggers on "langbot space mcp", "search langbot plugins", "langbot marketplace mcp", "space.langbot.app mcp".
---
# LangBot Space MCP Operations
LangBot Space (space.langbot.app) exposes an **MCP server** so user-facing AI
agents can browse and search the marketplaces (plugins, MCP servers, skills).
## Endpoint
```
https://space.langbot.app/mcp
```
Transport: **streamable HTTP** (stateless, JSON responses). For a self-hosted
Space instance: `http://<host>:8383/mcp`.
## Authentication
Reuses the existing **Personal Access Token (PAT)** — the same token the `lbp`
CLI uses. Create one in your Space account (Profile → Personal Access Tokens),
then send it as a Bearer token:
```
Authorization: Bearer lbpat_...uests without a valid PAT get `401 Unauthorized`.
## Client configuration
```json
{
"mcpServers": {
"langbot-space": {
"url": "https://space.langbot.app/mcp",
"headers": { "Authorization": "Bearer <your-pat>" }
}
}
}
```
## Tool surface
| Tool | Purpose |
| --- | --- |
| `list_plugins` / `search_plugins` / `get_plugin` | Plugin marketplace |
| `list_mcp_servers` / `search_mcp_servers` / `get_mcp_server` | MCP-server marketplace |
| `list_skills` / `search_skills` / `get_skill` | Skill marketplace |
`list_*` and `search_*` are paged (`page`, `page_size`). `get_*` takes
`author` + `name`. The tool surface mirrors the REST endpoints under
`/api/v1/marketplace/*` and is read/browse only.
## How to use
1. Create a PAT in your Space account settings.
2. Point your MCP client at `https://space.langbot.app/mcp` with the Bearer PAT.
3. Use `search_plugins` / `search_mcp_servers` / `search_skills` to find items,
then `get_*` for details (e.g. to obtain author/name for installation in
LangBot itself).
## Implementation & maintenance (for Space developers)
- Server: `internal/controller/mcp/server.go` (official Go MCP SDK
`github.com/modelcontextprotocol/go-sdk`). Tools call the service layer
(`PluginService`, `MCPService`, `SkillService`) directly.
- Mount: `internal/controller/api.go` at `/mcp` and `/mcp/*any`.
- Auth: PAT via `AccountService.ValidatePersonalAccessToken`.
- Docs: `docs/MCP_SERVER.md`.
> When you add, remove, or change a marketplace API endpoint that should be
> agent-accessible, update the corresponding MCP tool **and** this skill. The
> MCP tool surface and the API must stay aligned (see `AGENTS.md`).
## Pitfalls
- The PAT prefix is `lbpat_` (Space), distinct from LangBot's `lbk_` API keys.
- This server is read/browse only; it does not publish or modify marketplace
items. Use the web UI or REST API (with appropriate auth) for that.
+44
View File
@@ -0,0 +1,44 @@
---
name: langbot-testing
description: Test LangBot WebUI and core product flows with an automated browser and backend logs. Use when validating the configured LangBot frontend, pipeline Debug Chat, model provider setup and test buttons, bot and knowledge-base UI flows, or troubleshooting failed LangBot end-to-end tests.
---
# LangBot Testing
Use this skill when an agent needs to verify LangBot behavior through the WebUI instead of only reading code.
## Routing
- **General WebUI testing**: read `references/web-ui-testing.md`.
- **Pipeline Debug Chat**: read `references/pipeline-debug-chat.md`.
- **Dify AgentRunner**: read `references/dify-agent-runner.md`.
- **Model provider setup or test button**: read `references/model-provider-testing.md`.
- **Plugin install/runtime/tool/page smoke**: read `references/plugin-e2e-smoke.md`.
- **Local Agent Runner**: read `references/local-agent-runner.md`.
- **Local Agent Runner path coverage**: read `references/local-agent-runner-coverage.md`.
- **Diff-aware AgentRunner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
- **Agent Runner release gate**: read `references/agent-runner-release-gate.md`.
- **Sandbox-backed skill authoring**: read `references/sandbox-skill-authoring.md`.
- **LangRAG knowledge bases**: read `references/langrag-knowledge-base.md`.
- **MCP stdio tool testing**: read `references/mcp-stdio-testing.md`.
- **Performance, reliability, or chaos probes**: read `references/performance-reliability-testing.md`.
- **Drive a live instance over MCP (not raw HTTP)**: use the `langbot-mcp-ops` skill — the instance exposes an MCP server at `http://<host>:5300/mcp` (reuses API keys). Useful for setting up bots/pipelines/models as test fixtures programmatically.
- **Known failures and fixes**: read `references/troubleshooting.md`.
- **Reusable test groups**: run `bin/lbs suite list` and `bin/lbs suite plan <suite-id>` before manually assembling a case set.
## Rules
- Read `../.env` first and use `LANGBOT_FRONTEND_URL` and `LANGBOT_BACKEND_URL` instead of hardcoded ports.
- If a standalone frontend dev server is running, `LANGBOT_FRONTEND_URL` may point to `LANGBOT_DEV_FRONTEND_URL`; otherwise it may point to the backend WebUI.
- Confirm the backend and frontend are actually running before testing.
- Run `bin/lbs fixture check` before fixture-heavy MCP, RAG, multimodal, or plugin smoke tests.
- For runner externalization release checks, run `bin/lbs test run agent-runner-release-preflight` before the full `agent-runner-release-gate` suite so configuration blockers are separated from product failures.
- Read `Manual Readiness` in `bin/lbs test plan <case-id>`; `manual_check` means the declared preconditions or setup still need operator confirmation for this run.
- Use an authenticated browser profile prepared by `langbot-env-setup`.
- Do not expose API keys, OAuth secrets, tokens, or localStorage token values in output.
- A WebUI test is not complete until the visible UI result is checked against backend logs or network behavior.
- A performance result is not complete without `metrics` evidence and a clear split between LangBot overhead and external provider/tool/network time.
- A chaos or reliability result is not complete until the fault scope, cleanup, and recovery checks are recorded.
- For a suite, use `bin/lbs suite start <suite-id>` to create the suite evidence root, per-case directories, and `suite-start.json`/`suite-start.md` handoff files; use `bin/lbs test result <case-id>` to write final per-case `result.json`, then run `bin/lbs suite report <suite-id> --evidence-dir <dir>`.
- Do not mark a case `pass` until `test result --evidence` covers every value in the case's `evidence_required`.
- For runner-specific Debug Chat cases, use the case-specific pipeline env declared by `automation_pipeline_url_env` / `automation_pipeline_name_env`; do not silently reuse a generic `LANGBOT_PIPELINE_URL`.
@@ -0,0 +1,79 @@
id: acp-agent-runner-debug-chat
title: "ACP AgentRunner can answer through Debug Chat using real remote Claude"
mode: agent-browser
area: pipeline
type: regression
priority: p2
risk: high
ci_eligible: false
tags:
- agent-runner
- acp
- claude
- external-runner
- pipeline
skills:
- langbot-env-setup
- langbot-testing
env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
env_any:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL|LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation: scripts/e2e/pipeline-debug-chat.mjs
automation_env:
- LANGBOT_FRONTEND_URL
- LANGBOT_BACKEND_URL
- LANGBOT_BROWSER_PROFILE
- LANGBOT_CHROMIUM_EXECUTABLE
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
- LANGBOT_E2E_PROMPT
- LANGBOT_E2E_EXPECTED_TEXT
- LANGBOT_E2E_EXPECTED_RUNNER_ID
- LANGBOT_E2E_RESPONSE_TIMEOUT_MS
automation_pipeline_url_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
automation_pipeline_name_env: LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
automation_expected_runner_id: "plugin:langbot/acp-agent-runner/default"
automation_prompt: "Use the injected LangBot MCP server tool langbot_get_current_event once. If the MCP call succeeds, reply with exactly ACP_AGENT_RUNNER_E2E_OK."
automation_expected_text: "ACP_AGENT_RUNNER_E2E_OK"
automation_response_timeout_ms: "300000"
setup_automation:
- "node:scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env"
setup_provides_env:
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_URL
- LANGBOT_ACP_AGENT_RUNNER_PIPELINE_NAME
preconditions:
- "The remote machine has a working Claude Code login and can run npx -y @agentclientprotocol/claude-agent-acp."
- "LangBot can non-interactively SSH to the remote machine; the runner opens the MCP reverse tunnel automatically."
steps:
- "Open LANGBOT_FRONTEND_URL."
- "Open the ACP AgentRunner QA pipeline."
- "Confirm the pipeline AI runner is plugin:langbot/acp-agent-runner/default."
- "Open Debug Chat."
- "Ask the real remote Claude ACP agent to call langbot_get_current_event and return ACP_AGENT_RUNNER_E2E_OK exactly."
checks:
- "UI: Debug Chat shows the user prompt."
- "UI: Debug Chat shows a Bot response containing ACP_AGENT_RUNNER_E2E_OK."
- "Logs: Backend logs include Processing request from person_websocket and Streaming completed for this run."
- "Logs: No acp runner request error appears for this run."
- "Console: No unexpected frontend errors appear during Debug Chat."
evidence_required:
- ui
- console
- backend_log
diagnostics:
- "Use scripts/e2e/ensure-acp-agent-runner-pipeline.mjs --write-env to create/update the pipeline."
- "For remote Claude on 101, verify ssh yhh@101.34.71.12 can run without password prompts; no separate ssh -R process is required."
success_patterns:
- "ACP_AGENT_RUNNER_E2E_OK"
- "Processing request from person_websocket"
- "Streaming completed"
failure_patterns:
- "acp.command_not_found"
- "acp.process_exited"
- "Agent runner plugin:langbot/acp-agent-runner/default execution failed"
troubleshooting:
- backend-not-listening
- plugin-runtime-timeout
- proxy-env-mismatch
@@ -0,0 +1,34 @@
id: agent-runner-async-db-readiness
title: "AgentRunner async DB readiness probe"
mode: probe
area: release
type: smoke
priority: p0
risk: high
ci_eligible: true
tags:
- agent-runner
- probe
- async-db
- aiosqlite
skills:
- langbot-testing
env:
automation: skills/langbot-testing/probes/agent-runner-async-db-readiness.mjs
steps:
- "Run `rtk bin/lbs test run agent-runner-async-db-readiness --dry-run` first; remove `--dry-run` after checking the planned evidence directory."
- "Automation checks whether a direct aiosqlite in-memory connection can create a table within the readiness timeout."
checks:
- "automation-result.json status is pass or env_issue."
- "pass means async SQLite tests are worth running."
- "env_issue means async SQLite-dependent Host pytest probes should be classified as environment-limited until fixed."
evidence_required:
- filesystem
diagnostics:
- "If this probe returns env_issue, run agent-runner-ledger-invariants for fast ledger coverage and skip async ledger pytest as a release blocker in this environment."
success_patterns:
- "AIOSQLITE_READY"
failure_patterns:
- "aiosqlite readiness timed out"
troubleshooting:
- aiosqlite-connect-hangs

Some files were not shown because too many files have changed in this diff Show More