Compare commits

...

28 Commits

Author SHA1 Message Date
Junyan Qin ffd4143672 Add OSS and commercial workspace boundaries 2026-07-10 12:02:31 +08:00
Junyan Qin 516e85f9e3 Document multi-tenant workspace architecture 2026-07-10 12:02:31 +08:00
RockChinQ 940234a0d8 docs: update blog links to main site 2026-07-09 14:13:04 -04:00
Daria Korenieva 0c405901d2 feat(vector): add Valkey Search vector database backend (#2276)
* feat(vector): add Valkey Search vector database backend

Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.

- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
  all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
  accepted for interface parity but NOT honored (docstring + one-time warning +
  docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
  client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
  (vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
  change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
  (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.

* fix(vector): paginate Valkey Search deletes and guard delete_by_filter

Address self-review follow-ups for the Valkey Search VDB backend:

- _search_keys now paginates through the full result set in batches of
  _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
  page, so delete_by_file_id / delete_by_filter fully remove files and
  filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
  a filter referencing only non-indexed fields must skip and return 0
  (never fall back to match-all), and a supported filter still deletes
  matching keys.

* refactor(vector): harden Valkey Search backend and add adversarial tests

Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
  instead of building ServerCredentials(password=None, ...), which glide
  rejects).
- Add an async close() teardown that closes the glide client and resets
  cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
  never widens).

Tests:
- Add an adversarial-input integration test proving crafted file_id /
  query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* fix(vector): make Valkey Search file_id TAG support arbitrary characters

Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.

Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* refactor(vector): address Valkey Search review feedback

- Add configurable request_timeout (default 5000ms; glide default 250ms is
  too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
  mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
  the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
  _reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics

* fix(vector/valkey_search): address round-2 review feedback

- Serialize lazy client creation with an asyncio.Lock (double-checked) so
  concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
  than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
  non-numeric input and pre-empting a future NUMERIC-field injection surface.

* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-07-08 06:59:16 +08:00
Hyu 00e2103873 docs(config): add box.default_memory_mb to config.yaml template (#2319)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 15:18:27 +08:00
Hyu 209706b0b9 feat(mcp): add box.default_memory_mb config for nsjail memory limit (#2318)
Operators can now set a global default memory limit for all stdio MCP
servers in config.yaml or via environment variable:

  config.yaml:
    box:
      default_memory_mb: 2048  # default: 1536

  env:
    BOX__DEFAULT_MEMORY_MB=2048

The default is raised from 1024 to 1536 MB — a safer floor for
Node.js V8 + WASM (undici llhttp) under nsjail cgroup limits.
Individual MCP servers can still override via their own box.memory_mb.

Previously the fallback was hardcoded to 1024 MB, causing OOM kills
(return_code=137) on node/npx MCP servers that need more RAM.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 14:24:06 +08:00
Hyu 205404e3da fix(deps): pin langbot-plugin 0.4.13 (memory_mb removed from _COMPAT_FIELDS) (#2317)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 13:45:13 +08:00
Hyu 3e93ccfb45 fix(mcp): use uniform session memory_mb=1024 to avoid BoxSessionConflictError (#2316)
All MCPs share one Box session (mcp-shared). When session memory_mb differed
by command type (512 for python, 1024 for node), the second MCP to call
create_session raised BoxSessionConflictError. Fix: always use 1024 MB for
the shared session so python and node MCPs coexist without conflict.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:55:49 +08:00
Hyu c5d7e3dcb1 fix(deps): pin langbot-plugin 0.4.12 (BoxManagedProcessSpec.memory_mb fix) (#2315)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:31:09 +08:00
Hyu cd6d5d9c2f fix(mcp): pin langbot-plugin 0.4.10 - per-process memory_mb for node OOM fix (#2314)
* fix(mcp): bump default memory to 1024MB for node (npx) stdio MCP servers

Node.js MCP servers (npx/bunx) were being OOM-killed (return_code=137) by the
default 512MB nsjail cgroup_mem_max. Node V8 reserves large virtual address
space and instantiates WebAssembly modules (undici llhttp) on startup, easily
exceeding 512MB resident. This caused every node-based MCP (memory,
sequential-thinking, filesystem, weather, docker, excel) to crash-loop.

Fix: when the stdio command is npx/bunx/pnpm, default memory_mb to 1024 unless
the operator explicitly set a value. Python/uvx servers keep the 512MB default.

* chore(deps): pin langbot-plugin 0.4.10 (per-process memory_mb fix)

* chore: update uv.lock for langbot-plugin 0.4.10

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 09:03:43 +08:00
Hyu 1f7d9339fc fix(mcp): bump default memory to 1024MB for node (npx) stdio MCP servers (#2313)
Node.js MCP servers (npx/bunx) were being OOM-killed (return_code=137) by the
default 512MB nsjail cgroup_mem_max. Node V8 reserves large virtual address
space and instantiates WebAssembly modules (undici llhttp) on startup, easily
exceeding 512MB resident. This caused every node-based MCP (memory,
sequential-thinking, filesystem, weather, docker, excel) to crash-loop.

Fix: when the stdio command is npx/bunx/pnpm, default memory_mb to 1024 unless
the operator explicitly set a value. Python/uvx servers keep the 512MB default.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-04 08:31:08 +08:00
RockChinQ f3b5fcfb7c fix mcp sidebar status sync (#2312)
Co-authored-by: Hyu <chenhyu@proton.me>
2026-07-03 20:23:12 +08:00
Hyu ffabb91bfe fix(mcp): add missing MCPLogs import to MCPForm (#2311)
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 17:59:02 +08:00
Hyu 96c84740db feat(mcp): add logs tab to MCP server detail panel (frontend) (#2310)
* feat(mcp): add logs tab to MCP server detail panel

- Add MCPLogs component mirroring PluginLogs functionality
- Add getMcpServerLogs API method to BackendClient
- Add logs tab to MCPForm detail panel (edit mode only)
- Add i18n keys (tabLogs, logsLevelAll, logsRefresh, logsAutoRefresh, logsEmpty) to all 8 locales
- Auto-refresh logs every 3s with toggle
- Filter logs by level (ALL/DEBUG/INFO/WARNING/ERROR)

* style: fix prettier formatting in MCPForm.tsx logs tab

* style: fix prettier in i18n locale files

---------

Co-authored-by: dadachann <dadachann@users.noreply.github.com>
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 16:52:15 +08:00
Hyu 85b5b5b54b feat(mcp): add MCP server log panel backend (#2308)
- Add log buffer (_log_buffer: deque with maxlen=500) to RuntimeMCPSession
- Capture stderr from Box managed process in monitor_process_health loop
- Add get_mcp_server_logs service method with limit and level filtering
- Add GET /servers/<server_name>/logs HTTP endpoint
- Parse log level from stderr lines (error/warning/debug/info)
- Tests passing: 211 passed, 12 skipped

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 16:05:23 +08:00
Hyu 28bffdef21 fix(mcp): set _preserve_managed_process before finally in cold-start path (#2309)
_ColdStartRetry was caught in _lifecycle_loop_with_retry which set
_preserve_managed_process = True — but by then the finally block inside
_lifecycle_loop had already run and called _cleanup_box_stdio_session(),
stopping the live managed process (return_code=143 SIGTERM). The cold-start
retry then restarted a fresh process, eliminating the warm-up advantage.

Fix: add an explicit except _ColdStartRetry in _lifecycle_loop that sets
_preserve_managed_process = True before re-raising. The finally block then
sees the flag and skips stop_managed_process, leaving the live process
untouched for the next handshake attempt.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 15:26:18 +08:00
Hyu bf3c96026b fix(mcp): fix cold-start connection drop by using lexically-intact owner exit stack (#2307)
The previous _TransferredStack approach broke anyio lexical context:
websocket_client/ClientSession use anyio task groups whose cancel scope is
bound to the frame that entered them. Deferring their aclose via a transferred
exit stack left the underlying memory streams closed once initialize() returned,
so the very next request (refresh -> list_tools) failed with Connection closed.

New design:
- Attach on the owner exit stack (same task as the serve loop, lexically intact)
- A cold-starting process makes initialize() fail; signal _ColdStartRetry up to
  the outer retry loop, which reuses the live process without consuming retry budget
- _lifecycle_loop_with_retry handles _ColdStartRetry like _TransportReconnect:
  preserves process, no fatal budget, backs off 2s and retries
- Two new unit tests: cold-start raises _ColdStartRetry (not fatal) when process
  is alive; raises fatal error when process has actually exited

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 14:58:40 +08:00
Hyu ed9343c686 fix(mcp): retry the stdio handshake during slow (npx) cold starts (#2306)
A node/npx stdio MCP server (e.g. firecrawl-mcp via npx -y) failed on first
connect with Connection closed / Failed after 4 attempts, even though the
process was fine. An npx cold start downloads+installs the package before the
server can answer the MCP handshake (measured ~27s for a simple official
server; longer for heavier ones). The old code attached the WS and called
session.initialize() the instant the process was started, so the handshake ran
before the process could answer and failed; the outer lifecycle retry then
rebuilt the process, churning it in a loop.

Verified decisively: attaching + initialize() against a mid-cold-start process
times out on attempt 1 (process still installing) but SUCCEEDS at t+0.6s on
attempt 2 once the process is ready. So the fix is to retry the handshake in
place, not to rebuild the process.

Changes (mcp_stdio.initialize):
- Start the managed process ONCE, then loop attach WS -> ClientSession ->
  initialize() within the startup_timeout budget, tearing down each failed
  attempt cleanly, until the handshake succeeds or the budget elapses. A
  successful transport/session is transferred into the owner exit stack via a
  small _TransferredStack adapter.
- Bound each attempt with asyncio.wait_for(initialize, _HANDSHAKE_ATTEMPT_TIMEOUT_SEC=10s)
  so a cold-starting process fails fast and retries instead of hanging until
  the transport drops.
- Stop retrying ONLY when the process has DEFINITIVELY exited: new
  _managed_process_has_exited() (checks EXITED status) replaces the previous
  not-_managed_process_is_running() test, which false-negatived on a
  just-spawned process that had not yet reported RUNNING and made the loop bail
  to the outer rebuild path (relay then rejected the early re-attach with HTTP 400).

Adds a unit test that fails the first two handshakes with the process alive and
asserts the loop retries to success while starting the process exactly once.

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 14:16:57 +08:00
Hyu 9a8cdde86c refactor(mcp): make MCP test reuse the shared Box session (near-instant tests) (#2304)
* refactor(mcp): make MCP test reuse the shared Box session instead of a per-test session

Testing an MCP server (config-page "test" button) previously spun up a fresh
isolated mcp-test-<uuid> Box session every time: cold-start the container, run
the dependency bootstrap, probe, then tear the whole session down. That is slow
(tens of seconds) and, on an already-hosted server, wasteful — the server is
already running in the shared session.

Change the test to reuse the shared session / live process:

- _build_box_session_id: transient tests now use mcp-shared, the same Box
  session as live servers, so a test reuses the running container (and, for an
  existing server, its live managed process) instead of a cold per-test session.
- cleanup_session: a transient test no longer deletes the whole session (which
  under the shared model would kill every other MCP server in the container). It
  stops only its own process_id, exactly like a live server. Isolation is now at
  the process level (distinct process_id per server/test), not the session level.
- test_mcp_server (persisted server): reuse the live connection with a real
  list_tools refresh/probe; only fall back to a full start() when there is no
  live connection to probe or the refresh fails, instead of an ERROR->start()
  rebuild.

Trade-off: a failing test now shares the container with live servers rather than
a throwaway session. Accepted deliberately in favour of near-instant tests;
process-level isolation keeps a test from stopping another server's process.

* chore(deps): pin langbot-plugin 0.4.9 for the nsjail RLIMIT_AS node/npx MCP fix

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 13:14:14 +08:00
Hyu a2655e16f8 fix(mcp): survive transient WS transport drops for Box stdio MCP servers (#2303)
* fix(mcp): survive transient WS transport drops for Box stdio MCP servers

A Box-backed stdio MCP server (e.g. pab1it0/prometheus) would periodically
error on the frontend with Box managed process exited unexpectedly /
Failed after 4 attempts once the session had been alive for a while.

Root cause: the managed MCP process lives in the Box runtime and SURVIVES a
WebSocket transport drop, but _lifecycle_loop treated any monitor completion
as a fatal process death. It then ran the finally-block cleanup — which STOPS
the still-healthy managed process — and did a full 4-attempt exponential
backoff rebuild. Under an occasionally-stalled single-worker event loop the
mcp websocket client misses a ping/pong, the transport drops, and this
self-inflicted teardown loop is what the user sees.

Fixes:
- _lifecycle_loop: when the health monitor completes, re-check the real
  managed-process state. If the process is still running, the transport
  merely dropped: raise an internal _TransportReconnect signal instead of
  Box managed process exited unexpectedly.
- _lifecycle_loop_with_retry: handle _TransportReconnect as a free, uncounted
  reconnect (does not consume the fatal retry budget), so a long-lived session
  survives arbitrarily many transient drops.
- finally-block: gate managed-process teardown on a _preserve_managed_process
  flag so a transport-only reconnect closes just the WS, not the process.
- BoxStdioSessionRuntime.initialize: reuse an already-running managed process
  instead of stopping+rebuilding it (which also re-ran the slow dependency
  bootstrap); only (re)start when none is running. Adds
  _managed_process_is_running() helper.

Pairs with langbot-plugin-sdk fix adding a server-driven WS heartbeat to the
managed-process relay, which prevents most drops in the first place.

* style(mcp): ruff format

* chore(deps): pin langbot-plugin 0.4.8 for the managed-process WS heartbeat fix

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-07-03 12:24:24 +08:00
Hyu f89611f39a chore: release v4.10.5 2026-07-03 00:59:53 +08:00
RockChinQ 04a199a3b2 fix: preserve MCP server ids with slashes
Fixes #2301
2026-07-03 00:17:29 +08:00
Hyu f85278f98b Fix monitoring CI regressions 2026-07-02 17:22:33 +08:00
Hyu 0ce382fc8b Improve pipeline monitoring and AI tab resilience 2026-07-02 16:53:50 +08:00
Hyu e32e515bc9 Tone down tool call bubbles 2026-07-02 16:53:50 +08:00
Hyu c809e3d14f Add tool call observability 2026-07-02 16:53:50 +08:00
Hyu b1bc05d5d3 Improve monitoring conversation turns 2026-07-02 16:53:50 +08:00
Stevenqin 13dba887d5 fix(wecomcs): implement proactive text sends (#2300) 2026-07-02 14:39:05 +08:00
65 changed files with 6763 additions and 717 deletions
+1 -1
View File
@@ -51,7 +51,7 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
[→ Learn more about all features](https://link.langbot.app/en/docs/features)
📍 Practical guides: [deploy a multi-platform AI bot in 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connect DeepSeek to WeChat, Discord, and Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [run a Dify Agent in Discord, Telegram, and Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), and [build an n8n-powered chatbot](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Practical guides: [deploy a multi-platform AI bot in 5 minutes](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connect DeepSeek to WeChat, Discord, and Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [run a Dify Agent in Discord, Telegram, and Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/), and [build an n8n-powered chatbot](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+1 -1
View File
@@ -51,7 +51,7 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 实践指南:[5 分钟部署多平台 AI 机器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[将 DeepSeek 接入微信、企业微信与 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[让 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 构建多平台 AI 聊天机器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
📍 实践指南:[5 分钟部署多平台 AI 机器人](https://langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[将 DeepSeek 接入微信、企业微信与 Discord](https://langbot.app/zh/blog/connect-deepseek-to-wechat/)、[让 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 构建多平台 AI 聊天机器人](https://langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
[→ Conocer más sobre todas las funcionalidades](https://link.langbot.app/en/docs/features)
📍 Guías prácticas: [desplegar un bot de IA multiplataforma en 5 minutos](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [conectar DeepSeek a WeChat, Discord y Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [ejecutar un Dify Agent en Discord, Telegram y Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) y [crear un chatbot con n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Guías prácticas: [desplegar un bot de IA multiplataforma en 5 minutos](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [conectar DeepSeek a WeChat, Discord y Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [ejecutar un Dify Agent en Discord, Telegram y Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) y [crear un chatbot con n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
[→ En savoir plus sur toutes les fonctionnalités](https://link.langbot.app/en/docs/features)
📍 Guides pratiques : [déployer un bot IA multiplateforme en 5 minutes](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connecter DeepSeek à WeChat, Discord et Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [exécuter un Dify Agent dans Discord, Telegram et Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) et [créer un chatbot avec n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Guides pratiques : [déployer un bot IA multiplateforme en 5 minutes](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [connecter DeepSeek à WeChat, Discord et Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [exécuter un Dify Agent dans Discord, Telegram et Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) et [créer un chatbot avec n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
[→ すべての機能について詳しく見る](https://link.langbot.app/ja/docs/features)
📍 実践ガイド: [5分でマルチプラットフォームAIボットをデプロイ](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/)、[DeepSeekをWeChat・Discord・Telegramに接続](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/)、[Dify AgentをDiscord・Telegram・Slackで動かす](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/)、[n8n連携チャットボットを構築](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/)。
📍 実践ガイド: [5分でマルチプラットフォームAIボットをデプロイ](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/)、[DeepSeekをWeChat・Discord・Telegramに接続](https://langbot.app/en/blog/connect-deepseek-to-wechat/)、[Dify AgentをDiscord・Telegram・Slackで動かす](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/)、[n8n連携チャットボットを構築](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/)。
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
[→ 모든 기능 자세히 보기](https://link.langbot.app/en/docs/features)
📍 실전 가이드: [5분 만에 멀티 플랫폼 AI 봇 배포하기](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [DeepSeek를 WeChat, Discord, Telegram에 연결하기](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [Dify Agent를 Discord, Telegram, Slack에서 실행하기](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/), [n8n 기반 챗봇 만들기](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 실전 가이드: [5분 만에 멀티 플랫폼 AI 봇 배포하기](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [DeepSeek를 WeChat, Discord, Telegram에 연결하기](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [Dify Agent를 Discord, Telegram, Slack에서 실행하기](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/), [n8n 기반 챗봇 만들기](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot — это **платформа с открытым исходным к
[→ Подробнее обо всех возможностях](https://link.langbot.app/en/docs/features)
📍 Практические руководства: [развернуть мультиплатформенного ИИ-бота за 5 минут](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [подключить DeepSeek к WeChat, Discord и Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [запустить Dify Agent в Discord, Telegram и Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) и [создать чат-бота на n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Практические руководства: [развернуть мультиплатформенного ИИ-бота за 5 минут](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [подключить DeepSeek к WeChat, Discord и Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [запустить Dify Agent в Discord, Telegram и Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) и [создать чат-бота на n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+1 -1
View File
@@ -52,7 +52,7 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
[→ 了解更多功能特性](https://link.langbot.app/zh/docs/features)
📍 實踐指南:[5 分鐘部署多平台 AI 機器人](https://blog.langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[將 DeepSeek 接入微信、企業微信與 Discord](https://blog.langbot.app/zh/blog/connect-deepseek-to-wechat/)、[讓 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://blog.langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 建構多平台 AI 聊天機器人](https://blog.langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
📍 實踐指南:[5 分鐘部署多平台 AI 機器人](https://langbot.app/zh/blog/deploy-ai-bot-in-5-minutes/)、[將 DeepSeek 接入微信、企業微信與 Discord](https://langbot.app/zh/blog/connect-deepseek-to-wechat/)、[讓 Dify Agent 跑在 Discord、Telegram 和 Slack 上](https://langbot.app/zh/blog/dify-agent-discord-telegram-slack/),以及[用 n8n 建構多平台 AI 聊天機器人](https://langbot.app/zh/blog/n8n-multi-platform-ai-chatbot/)。
---
+1 -1
View File
@@ -50,7 +50,7 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
[→ Tìm hiểu thêm về tất cả tính năng](https://link.langbot.app/en/docs/features)
📍 Hướng dẫn thực hành: [triển khai bot AI đa nền tảng trong 5 phút](https://blog.langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [kết nối DeepSeek với WeChat, Discord và Telegram](https://blog.langbot.app/en/blog/connect-deepseek-to-wechat/), [chạy Dify Agent trên Discord, Telegram và Slack](https://blog.langbot.app/en/blog/dify-agent-discord-telegram-slack/) và [xây dựng chatbot với n8n](https://blog.langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
📍 Hướng dẫn thực hành: [triển khai bot AI đa nền tảng trong 5 phút](https://langbot.app/en/blog/deploy-ai-bot-in-5-minutes/), [kết nối DeepSeek với WeChat, Discord và Telegram](https://langbot.app/en/blog/connect-deepseek-to-wechat/), [chạy Dify Agent trên Discord, Telegram và Slack](https://langbot.app/en/blog/dify-agent-discord-telegram-slack/) và [xây dựng chatbot với n8n](https://langbot.app/en/blog/n8n-multi-platform-ai-chatbot/).
---
+169
View File
@@ -0,0 +1,169 @@
# Valkey Search Vector Database Integration
This document describes how to use **Valkey Search** (the search/vector module bundled in
`valkey/valkey-bundle`) as the vector database backend for LangBot's knowledge base (RAG)
feature.
## What is Valkey Search?
**Valkey Search** is a module that adds vector similarity search and full-text search to
[Valkey](https://valkey.io/), the open-source, BSD-licensed in-memory data store forked from
Redis OSS. It is distributed in the `valkey/valkey-bundle` image alongside other modules
(JSON, Bloom, LDAP).
LangBot talks to Valkey through the official [`valkey-glide`](https://pypi.org/project/valkey-glide/)
client (Rust core + async Python wrapper), using its native `ft` (search) command namespace.
### Key Features
- **Vector search**: ANN via HNSW or exact via FLAT, with COSINE / L2 / IP distance metrics
- **Full-text search**: term, prefix and phrase matching over indexed text fields
- **Hybrid search**: a metadata/text filter pre-selects candidates, then KNN ranks them
- **In-memory speed**: vectors and documents are stored as Valkey HASH keys
- **Auth + TLS**: optional username/password and TLS for production (toB / SaaS) deployments
### Licensing
- Valkey core and the Search module are **BSD-3-Clause**.
- The `valkey-glide` client is **Apache-2.0**.
Both are compatible with LangBot.
## Installation
Valkey Search support is included when you install LangBot — the `valkey-glide` dependency is
declared in `pyproject.toml`. To install manually:
```bash
pip install 'valkey-glide>=2.4.1,<3.0.0'
```
You also need a running Valkey server with the Search module loaded. The simplest way is the
bundled image:
```bash
# Run valkey-bundle (includes the Search module) on host port 6380
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
# (docker run ... works identically)
```
`valkey-bundle` ships multi-arch images (linux/amd64 + linux/arm64), so it runs on both CI
(x86_64) and Apple-silicon dev machines.
## Configuration
Valkey Search is **opt-in and disabled by default** — the default `vdb.use` stays `chroma`,
so existing single-process deployments are unaffected. To enable it, edit your `config.yaml`:
```yaml
vdb:
use: valkey_search
valkey_search:
host: 'localhost'
port: 6379 # use 6380 if you started the container as shown above
db: 0
password: '' # optional (ACL / requirepass) — never logged
username: '' # optional (ACL user)
tls: false # optional (toB / SaaS)
index_algorithm: 'HNSW' # HNSW | FLAT
distance_metric: 'COSINE' # COSINE | L2 | IP
request_timeout: 5000 # per-request timeout in ms
```
| Option | Default | Description |
|--------|---------|-------------|
| `host` | `localhost` | Valkey host |
| `port` | `6379` | Valkey port |
| `db` | `0` | Logical database id |
| `password` | `''` | Optional auth password (empty = no auth). Never logged. |
| `username` | `''` | Optional ACL username. Configuring a username without a password fails closed (raises) rather than connecting unauthenticated. |
| `tls` | `false` | Enable TLS for the connection |
| `index_algorithm` | `HNSW` | `HNSW` (approximate) or `FLAT` (exact) |
| `distance_metric` | `COSINE` | `COSINE`, `L2`, or `IP` |
| `request_timeout` | `5000` | Per-request timeout in milliseconds. The valkey-glide default (250ms) is too low for vector KNN under load; raise it further for remote/cross-AZ Valkey. |
### Connection behavior
The backend uses a **lazy** connection (`lazy_connect=True`): the client is created on first
use and the connection is deferred to the first command. A misconfigured or unreachable Valkey
server therefore does **not** block LangBot from booting — knowledge-base operations will error
at call time instead, and you can recover by switching `vdb.use` back to another backend.
The connection sets a fixed `client_name` of `langbot_vector_client` so it is identifiable in
`CLIENT LIST` and monitoring dashboards.
## Supported search types
| Type | Behavior |
|------|----------|
| `vector` | Pure KNN over the embedding field |
| `full_text` | Term/phrase match over the indexed `document` text field |
| `hybrid` | Metadata/text filter **pre-selects** candidates, then KNN ranks them |
### ⚠️ Important: `vector_weight` is NOT honored
Valkey Search hybrid queries follow a **filter-then-KNN** model: the filter (and/or full-text
clause) narrows the candidate set, and the KNN stage ranks the survivors by vector distance.
There is **no native weighted score fusion** (unlike, e.g., SeekDB's RRF boost).
For interface compatibility the backend still accepts a `vector_weight` argument, but it is
**ignored** — passing different weights does not change result ordering. The first time a
non-default weight is supplied, the backend logs a one-time warning.
If weighted hybrid ranking is needed in the future, it can be added **application-side** (run
vector KNN and full-text search separately and blend the scores). That is intentionally out of
scope for this integration.
## Metadata & filtering
Documents are stored as Valkey HASH keys under the prefix `kb:{collection}:{id}` with fields:
- `vector` — the embedding, packed as little-endian FLOAT32
- `document` — the raw text (indexed as TEXT for full-text/hybrid search)
- `file_id` — promoted to an indexed TAG field so it is filterable
- `metadata_json` — the full metadata dict, preserved verbatim as JSON
Only **indexed** fields are filterable. Currently that is `file_id`. Filters referencing
non-indexed metadata keys are dropped with a warning (the same pragmatism used by the Milvus
and pgvector backends). All other metadata still round-trips intact via `metadata_json`.
Supported filter operators (canonical Chroma-style `where` syntax): `$eq`, `$ne`, `$gt`,
`$gte`, `$lt`, `$lte`, `$in`, `$nin`. Multiple top-level keys are AND-ed.
## Testing
Unit tests (filter mapping, float32 packing, reply parsing, import guard) run in the fast lane
with no server:
```bash
uv run pytest tests/unit_tests/vector/test_valkey_search_filter.py -q
```
Integration tests are **slow-gated** on `TEST_VALKEY_URL` and require a running server:
```bash
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
TEST_VALKEY_URL=valkey://localhost:6380 \
uv run pytest tests/integration/vector/test_valkey_search.py -m slow -q
```
The default upstream fast CI lane (`-m "not slow"`) skips these, matching the existing
PostgreSQL migration-test precedent.
## Troubleshooting
| Symptom | Cause / fix |
|---------|-------------|
| Tests skip with "Valkey Search module not available" | The server is plain Valkey without the Search module. Use the `valkey/valkey-bundle` image. |
| `ConnectionError` at call time | Check `host`/`port`/auth; remember `lazy_connect` defers errors to first use. |
| Empty search results right after insert | The Search indexer is asynchronous; results become visible within a short delay. The integration tests poll/retry to account for this. |
| Hybrid ranking ignores `vector_weight` | Expected — see the caveat above. |
## Production considerations
- **Cluster mode**: Valkey Search in cluster mode uses an additional coordination port. This
integration targets standalone mode; cluster support is a future consideration.
- **Persistence**: configure Valkey RDB/AOF persistence if the knowledge base must survive
restarts; otherwise an in-memory store is ephemeral.
- **Security**: set `password`/`username` and `tls: true` for any non-local deployment.
Credentials are never written to logs.
@@ -0,0 +1,858 @@
# LangBot 多租户与多用户改造方案
## 目标
本方案面向 LangBot 从“单实例单管理员”演进到 SaaS 友好的“多 workspace、多账户、多权限”架构。
核心定义:
- Account:登录主体。一个自然人或服务账号,可加入多个 workspace。
- Workspace:租户边界。一个 workspace 内可拥有多个用户、机器人、流水线、模型、知识库、扩展、监控数据与 API Key。
- Membership:账户与 workspace 的关系,承载角色与权限。
- Role/Permissionworkspace 内权限,不再用“是否是当前唯一用户”来决定访问能力。
目标体验:
- 新用户登录后可以创建 workspace、加入 workspace、切换 workspace。
- 同一个账户可加入多个 workspace,每个 workspace 权限不同。
- 一个 workspace 可邀请多个用户协作,并分别设置 owner/admin/editor/viewer 等权限。
- 所有业务资源默认属于某个 workspace,所有 API 默认在当前 workspace 下工作。
- Plugin SDK、MCP、知识库、模型调用、监控日志都能拿到稳定的 workspace 上下文,并且不跨租户泄露数据。
## 调研结论
### 当前 LangBot 的单用户假设
LangBot 现在已经有 `users` 表和 JWT 登录,但仍是单用户/单租户模型:
- `src/langbot/pkg/entity/persistence/user.py``User` 只保存 `user/password/account_type/space_*`,没有角色、状态、workspace 关系。
- `src/langbot/pkg/api/http/service/user.py` 通过 `is_initialized()` 判断系统是否已有用户;`create_or_update_space_user()` 在系统已初始化且邮箱不匹配时拒绝新用户,这直接限制了多用户登录。
- `src/langbot/pkg/api/http/controller/group.py``AuthType.USER_TOKEN` 验证后只向 handler 注入 `user_email`JWT payload 也只有 `user`,没有 `account_id``workspace_id``role``permissions`
- `src/langbot/pkg/api/http/service/apikey.py` 的 API Key 只验证 key 是否存在,没有 owner、scope、workspace。
- `web/src/app/infra/http/BaseHttpClient.ts``localStorage.token` 读取单个 token,并在所有请求上加 `Authorization`;前端没有 workspace selector,也没有当前 workspace 上下文。
当前登录流程更像“初始化一个本地管理账号”,而不是 SaaS 账户体系。要支持多用户,必须把“初始化状态”和“首个 workspace 创建”拆开。
### 业务资源当前都是全局资源
主要持久化表没有租户字段:
- Bot`bots`
- Pipeline`legacy_pipelines``pipeline_run_records`
- Model`model_providers``llm_models``embedding_models``rerank_models`
- Plugin`plugin_settings`
- MCP`mcp_servers`
- RAG`knowledge_bases``knowledge_base_files``knowledge_base_chunks`
- Monitoring`monitoring_messages``monitoring_llm_calls``monitoring_sessions``monitoring_errors``monitoring_embedding_calls``monitoring_feedback`
- API Key`api_keys`
- Webhook`webhooks`
- Metadata`metadata`
- Binary storage`binary_storages`
对应服务也直接 select 全表,例如:
- `BotService.get_bots()` 返回所有 bot。
- `PipelineService.get_pipelines()` 返回所有 pipeline。
- `ModelProviderService.get_providers()` 返回所有 provider。
- `MCPService.get_mcp_servers()` 返回所有 MCP server。
- 插件和二进制存储没有 workspace 维度,插件 workspace storage 在 SDK 里还硬编码为 `default`
所以改造重点不是只给用户表加字段,而是给资源访问层统一加入 workspace scope。
### 运行时也存在全局单例假设
`src/langbot/pkg/core/stages/build_app.py` 初始化的是一个全局 `Application`,其中包含单例:
- `platform_mgr`
- `pipeline_mgr`
- `model_mgr`
- `tool_mgr`
- `plugin_connector`
- `sess_mgr`
- `rag_mgr`
- `vector_db_mgr`
当前运行时把所有 bot、pipeline、model、plugin、MCP 都加载到同一套内存管理器。多租户改造需要决定:是共享运行时并在对象上带 workspace 过滤,还是每个 workspace 拆 runtime shard。第一阶段建议共享进程、强制 workspace-aware;等规模变大后再演进为按 workspace 分片。
### Plugin SDK 对 workspace 的假设
SDK 当前只认识 bot/pipeline/query/session,不认识租户:
- `src/langbot_plugin/api/entities/builtin/pipeline/query.py``Query``query_id/launcher_type/launcher_id/sender_id/bot_uuid/pipeline_uuid`,没有 `workspace_id/account_id`
- `src/langbot_plugin/api/entities/builtin/provider/session.py``Session` 只按 `launcher_type + launcher_id` 表达会话。
- `src/langbot_plugin/api/proxies/langbot_api.py` 暴露 `get_bots/get_llm_models/invoke_llm/list_tools/vector_*` 等 Host API,都是全局语义。
- `src/langbot_plugin/runtime/io/handlers/plugin.py``set_workspace_storage/get_workspace_storage``owner_type` 设为 `workspace`,但 `owner` 固定为 `default`
- LangBot 侧 `src/langbot/pkg/plugin/handler.py` 处理插件请求时,会把 `GET_BOTS``GET_LLM_MODELS``VECTOR_*` 等转到全局服务。
这意味着多租户落地时,不能只在 Web API 层过滤;插件可以通过 Host API 访问全局资源,所以 SDK/Runtime 通信也必须传递 workspace context。
## 开源版与商业版产品边界
LangBot 是开源软件,但多 workspace 能力本质上是 SaaS 控制面能力。如果把完整多 workspace、成员协作、订阅权益和配额代码都放进开源仓库,只靠本地 feature flag 或本地 license check,无法有效避免第三方 fork 后自建 SaaS。因此建议采用 open-core 架构:开源版保留单 workspace 执行能力,账户、订阅、权益和多 workspace 协作能力放到 LangBot Space/Cloud Control Plane 和商业模块中。
### 版本边界
推荐拆成三层:
- `LangBot Core OSS`:开源,自托管,默认只有一个隐式 `default workspace`。它可以在数据结构上兼容 workspace,但产品能力上不提供创建多个 workspace、切换 workspace、成员邀请、成员权限管理、审计和多租户配额。
- `LangBot Space / Cloud Control Plane`:托管控制面,负责 account、workspace、membership、subscription、billing、entitlement、license token、workspace quota、marketplace 权益等能力。
- `LangBot Commercial Module`:商业闭源或私有包,承载多 workspace、团队协作、RBAC、自定义角色、审计日志、SAML/SSO、企业私有化授权等能力。
企业私有化版本可以采用 `LangBot Core + Commercial Module + License Token` 的形式交付。开源 Core 仍然可独立运行,但只能作为单 workspace 自托管产品,不提供 SaaS 多租户控制面。
### OSS 中如何保留兼容但不开放多 workspace
为了让后续商业版复用同一套资源隔离模型,OSS 代码里可以保留 `workspace_uuid` 相关字段和默认 workspace 迁移,但应限制为单 workspace:
- 首次初始化时创建一个 `Default Workspace`
- 所有资源自动归属这个 default workspace。
- 不暴露 `POST /api/v1/workspaces`
- 不暴露 workspace switcher。
- 不暴露成员邀请和成员角色管理。
- 不支持一个 account 加入多个 workspace。
- 不支持 workspace 数量大于 1。
- 前端不展示 workspace selector。
- API 层如果收到非 default workspace 的 `X-Workspace-Id`,直接拒绝。
也就是说,OSS 可以是 workspace-aware,但不是 multi-workspace-enabled。这样做的价值是:代码结构提前适配租户隔离,未来商业版不用重写所有资源模型;同时开源版用户无法直接通过 UI/API 获得 SaaS 型多租户能力。
### 账户、订阅和权益抽到 Space
账户和订阅体系建议从 LangBot Core 中抽出,交给 Space 控制面:
```text
LangBot Space
-> Account
-> Workspace
-> Membership
-> Subscription
-> Entitlement
-> License Token
LangBot Core
-> Validate entitlement / license
-> Run bots, pipelines, plugins, MCP, RAG
-> Enforce local resource scope
-> Report usage
```
这样做有几个原因:
- 账号体系如果完全在本地,第三方容易直接改库创建 workspace/membership。
- 订阅、配额和商业权益如果完全在本地,容易绕过。
- Space 可以统一处理 OAuth、组织、邀请、付款、发票、套餐、权益、Marketplace 分发权限。
- LangBot Core 只作为执行面消费 Space 下发的 entitlement,减少商业规则暴露。
### Entitlement 设计
Space 向 LangBot Core 下发签名权益,可以是在线校验,也可以为企业版提供短期/长期离线 license token。
示例:
```json
{
"edition": "oss",
"workspace_limit": 1,
"member_limit": 1,
"multi_workspace": false,
"rbac": false,
"audit_log": false,
"custom_roles": false,
"sso": false,
"commercial_use": false,
"expires_at": 1893456000
}
```
OSS 默认权益:
- `workspace_limit = 1`
- `member_limit = 1`
- `multi_workspace = false`
- `rbac = false`
- `audit_log = false`
- `sso = false`
Cloud/Pro/Enterprise 权益:
- `workspace_limit > 1`
- `member_limit > 1`
- `multi_workspace = true`
- `rbac = true`
- 可按套餐打开 audit、custom roles、SSO、usage reporting、enterprise support 等能力。
Core 执行面需要在关键入口强制校验 entitlement
- 创建 workspace。
- 邀请成员。
- 修改成员角色。
- 切换 workspace。
- 创建超过 quota 的资源。
- 开启商业模块功能。
### 商业模块边界
以下能力不建议进入 OSS 仓库的完整实现:
- 多 workspace 创建和切换。
- Workspace 成员邀请。
- Workspace RBAC 和自定义角色。
- Workspace 审计日志。
- Workspace 级用量和配额管理。
- 订阅、账单、发票。
- 企业 SSO/SAML/OIDC。
- 在线/离线 license 管理。
- 多租户 SaaS 运营控制台。
OSS 仓库可以保留接口占位、默认 workspace 兼容和必要的数据隔离字段,但完整交互、管理 UI、权益校验器和多 workspace policy engine 应由 Space 或商业模块提供。
### 防自建 SaaS 的现实边界
技术上无法 100% 阻止别人 fork 开源代码后自行改造。更可靠的策略是组合:
- 不把完整商业多 workspace 实现放进 OSS。
- Space 控制面提供账号、订阅、权益、Marketplace 和官方托管能力。
- 商业模块闭源或私有分发。
- 使用商标、云服务条款和商业 license 限制“自称官方 LangBot SaaS”或未经授权商用托管。
- 如果当前开源 license 对托管商用限制不足,需要单独评估 license 策略,必要时引入 open-core license 或新增商业附加条款。具体 license 调整需要法律评审。
结论:多 workspace 的底层 schema 可以在 OSS 中以 default workspace 兼容方式铺路,但多 workspace 产品能力、账户订阅权益、协作管理和 SaaS 控制面应放到 Space/商业模块,不作为开源版可直接使用的能力。
## 推荐总体架构
采用“单实例多 workspace,资源行级隔离,运行时上下文隔离”的架构:
```mermaid
flowchart TD
A["Account"] --> B["WorkspaceMembership"]
B --> C["Workspace"]
C --> D["Bots"]
C --> E["Pipelines"]
C --> F["Models & Providers"]
C --> G["Knowledge Bases"]
C --> H["Extensions: Plugins / MCP"]
C --> I["API Keys & Webhooks"]
C --> J["Monitoring"]
D --> K["Runtime Query"]
E --> K
K --> L["Plugin Runtime"]
K --> M["MCP Runtime"]
L --> N["Workspace-scoped Host APIs"]
```
原则:
- 账户全局唯一,workspace 是所有业务资源的归属边界。
- 所有 HTTP handler 在进入业务服务前解析出 `RequestContext(account, workspace, membership, permissions)`
- 所有 service 方法显式接收 `ctx``workspace_id`,禁止在业务服务里无条件 select 全表。
- 运行时对象的 key 从 `uuid` 扩展为 `(workspace_id, uuid)` 或使用全局唯一 uuid 但必须记录 workspace_id 并校验。
- 插件/MCP/知识库/模型调用都按 query 所属 workspace 过滤可用资源。
## 数据模型设计
### Account
替代现有 `users` 的语义,建议保留表名但升级字段,避免过大迁移:
字段建议:
- `id`
- `uuid`
- `email`
- `password_hash`
- `display_name`
- `avatar_url`
- `account_type`: `local | space`
- `status`: `active | disabled | deleted`
- `space_account_uuid`
- `space_access_token`
- `space_refresh_token`
- `space_access_token_expires_at`
- `space_api_key`
- `created_at`
- `updated_at`
兼容策略:
- 旧字段 `user` 迁移为 `email`,可以短期保留 alias。
-`password` 迁移为 `password_hash`,也可先保持列名不变,service 层改命名。
- JWT 中不要继续只放 email,应放 `sub=account_uuid`
### Workspace
新增 `workspaces`
- `uuid`
- `name`
- `slug`
- `avatar_url`
- `type`: `personal | team`
- `status`: `active | suspended | deleted`
- `default_language`
- `created_by_account_uuid`
- `created_at`
- `updated_at`
每个账户首次登录时自动创建一个 personal workspace。旧单用户实例迁移时创建一个 `Default Workspace`
### WorkspaceMembership
新增 `workspace_memberships`
- `workspace_uuid`
- `account_uuid`
- `role`: `owner | admin | developer | operator | viewer`
- `status`: `active | invited | disabled`
- `invited_by_account_uuid`
- `joined_at`
- `created_at`
- `updated_at`
唯一索引:
- `(workspace_uuid, account_uuid)`
### WorkspaceInvitation
新增 `workspace_invitations`
- `uuid`
- `workspace_uuid`
- `email`
- `role`
- `token_hash`
- `expires_at`
- `accepted_at`
- `created_by_account_uuid`
- `created_at`
用于邀请外部用户加入 workspace。Space OAuth 登录时可以根据 email 自动匹配未接受邀请。
### Role 与 Permission
先用固定角色,后续再做自定义角色。
建议权限:
- `workspace.manage`
- `member.view`
- `member.invite`
- `member.update_role`
- `member.remove`
- `bot.view`
- `bot.manage`
- `pipeline.view`
- `pipeline.manage`
- `model.view`
- `model.manage`
- `knowledge.view`
- `knowledge.manage`
- `extension.view`
- `extension.manage`
- `monitoring.view`
- `apikey.manage`
- `webhook.manage`
- `billing.view`
- `billing.manage`
角色映射:
| Role | 说明 | 权限 |
| --- | --- | --- |
| owner | workspace 拥有者 | 全部权限;可转让 owner;不可被其他角色移除 |
| admin | 管理员 | 除 owner 转让和删除 workspace 外的全部权限 |
| developer | 构建者 | 管理 bot、pipeline、model、knowledge、extension、webhook,可看监控 |
| operator | 运营者 | 查看和启停 bot、查看监控、查看配置,不可改密钥和删除资源 |
| viewer | 只读成员 | 只读资源和监控 |
### 业务资源加 workspace_uuid
以下表需要新增 `workspace_uuid`
- `bots`
- `legacy_pipelines`
- `pipeline_run_records`
- `model_providers`
- `llm_models`
- `embedding_models`
- `rerank_models`
- `plugin_settings`
- `mcp_servers`
- `knowledge_bases`
- `knowledge_base_files`
- `knowledge_base_chunks`
- `monitoring_*`
- `api_keys`
- `webhooks`
- `binary_storages`
- `metadata`
索引建议:
- 所有资源表加 `(workspace_uuid, created_at)``(workspace_uuid, updated_at)`
- 资源唯一键从单列改为 workspace 复合唯一:
- `bots.uuid` 可保持全局唯一,但查询仍必须带 workspace。
- `plugin_settings` 主键从 `(plugin_author, plugin_name)` 改为 `(workspace_uuid, plugin_author, plugin_name)`
- `mcp_servers.name` 如果未来要求唯一,必须是 `(workspace_uuid, name)`
- `metadata.key` 改为 `(workspace_uuid, key)`,系统级 metadata 单独放 `system_metadata` 或使用 `workspace_uuid=NULL`
- `binary_storages.unique_key` 建议改为 `workspace_uuid + owner_type + owner + key` 的 hash。
### API Key
API Key 必须归属于 workspace
- `workspace_uuid`
- `created_by_account_uuid`
- `scopes`
- `expires_at`
- `last_used_at`
- `status`
验证 API Key 后生成 `RequestContext`
- `account_uuid=None` 或 service account uuid
- `workspace_uuid=key.workspace_uuid`
- `permissions=key.scopes`
这样 `/api/v1/platform/bots/<uuid>/send_message` 之类接口不会跨 workspace 操作 bot。
## 后端改造方案
### RequestContext
新增统一上下文对象,例如:
```python
class RequestContext:
account_uuid: str | None
workspace_uuid: str
role: str | None
permissions: set[str]
auth_type: Literal["user_token", "api_key"]
```
改造 `RouterGroup.route()`
- `USER_TOKEN`:验证 JWT,读取 `account_uuid`,再从 header/query/cookie 中解析 current workspace。
- `API_KEY`:验证 API Key,直接得到 workspace。
- `USER_TOKEN_OR_API_KEY`:两者都返回同一种 `RequestContext`
- handler 参数从可选 `user_email` 升级为可选 `ctx`;兼容期同时支持 `user_email`
当前 workspace 传递方式:
- 推荐 header`X-Workspace-Id: <workspace_uuid>`
- Web 前端同时把当前 workspace 存在 localStorage。
- 如果未传,后端用账户最近使用 workspace 或第一个 active membership。
JWT payload
```json
{
"sub": "account_uuid",
"email": "user@example.com",
"iss": "LangBot-...",
"exp": 1234567890
}
```
不要把 workspace 写死在 JWT 里,否则切换 workspace 需要刷新 token。可以额外支持短 TTL workspace token,但第一阶段不必。
### 服务层改造模式
所有 service 方法增加 `ctx``workspace_uuid`
```python
async def get_bots(self, ctx: RequestContext, include_secret: bool = True):
require(ctx, "bot.view")
query = sqlalchemy.select(Bot).where(Bot.workspace_uuid == ctx.workspace_uuid)
```
需要改造的服务:
- `UserService`:拆成 AccountService + WorkspaceService 更清晰。
- `ApiKeyService`:按 workspace 管理 key。
- `BotService`:所有 bot 查询/创建/更新/删除按 workspace。
- `PipelineService`:所有 pipeline 查询/默认 pipeline 按 workspace。
- `ModelProviderService` 和 model services:按 workspace 隔离 provider 和 model。
- `MCPService`:按 workspace 管理 MCP server,运行时按 workspace host。
- `KnowledgeService/RAGRuntimeService`:按 workspace 管理 KB、文件、collection。
- `MonitoringService`:记录和查询都带 workspace。
- `WebhookService`:按 workspace 管理 webhook。
- `PluginRuntimeConnector`:插件安装、设置、配置按 workspace。
### HTTP API 形态
保留现有路径,靠 `X-Workspace-Id` 表示当前 workspace,可减少前端和 SDK 破坏:
- `GET /api/v1/workspaces`
- `POST /api/v1/workspaces`
- `GET /api/v1/workspaces/current`
- `PUT /api/v1/workspaces/current`
- `GET /api/v1/workspaces/<workspace_uuid>/members`
- `POST /api/v1/workspaces/<workspace_uuid>/invitations`
- `PUT /api/v1/workspaces/<workspace_uuid>/members/<account_uuid>`
- `DELETE /api/v1/workspaces/<workspace_uuid>/members/<account_uuid>`
现有资源 API
- `/api/v1/platform/bots`
- `/api/v1/pipelines`
- `/api/v1/provider/*`
- `/api/v1/plugins`
- `/api/v1/mcp`
- `/api/v1/knowledge`
继续保留,但必须从 `RequestContext.workspace_uuid` 过滤。
对外 API Key 也使用相同路径,只是由 key 决定 workspace。
### 初始化流程
现有 `/api/v1/user/init` 含义改为“创建首个账号和首个 workspace”:
1. 如果系统没有任何 account
- 创建 account。
- 创建 personal/team workspace。
- 创建 owner membership。
- 创建默认 pipeline。
- 标记 wizard status 到该 workspace metadata。
2. 如果系统已有 account
- 禁止无邀请注册,除非配置允许公开注册。
- Space OAuth 登录后,如果没有 membership,引导创建 workspace 或接受邀请。
`/api/v1/user/account-info` 不应再只返回 first user,应返回:
- `initialized`
- `registration_mode`
- `space_enabled`
- `default_login_methods`
登录成功后前端调用 `/api/v1/workspaces` 选择 workspace。
### 运行时隔离
第一阶段采用共享进程 + workspace-aware runtime
- `RuntimeBot` 增加 `workspace_uuid`
- `RuntimePipeline` 增加 `workspace_uuid`
- `Query` 增加 `workspace_uuid`,从 bot/pipeline 派生。
- `SessionManager.get_session()` key 从 `(launcher_type, launcher_id)` 改为 `(workspace_uuid, bot_uuid, launcher_type, launcher_id)`
- `PipelineManager.pipeline_dict` key 可保持 pipeline uuid,但所有 load/get 都校验 workspace;如果 uuid 不是全局唯一则改为 `(workspace_uuid, pipeline_uuid)`
- `ModelManager` provider/model 加 workspace 过滤;`get_model_by_uuid` 必须确保 query workspace 可访问。
- `ToolManager` 中 MCP tools、plugin tools 按 query workspace 过滤。
后续规模化时可演进:
- workspace runtime shard:每个 workspace 一套 plugin runtime/MCP runtime。
- 大客户独立进程或独立数据库。
## Plugin SDK 与 Runtime 改造
### Query/Event 增加 workspace context
SDK `Query` 增加:
- `workspace_uuid: str`
- `workspace_slug: str | None`
- `account_uuid: str | None`,仅 Web/API 触发时可能有,聊天平台消息通常为空。
Event 模型通过 `event.query.workspace_uuid` 可拿到租户上下文;序列化时也应包含这些字段。
向后兼容:
- 字段可选,默认 `None`
- 老插件不感知这些字段也能跑。
- 新插件可通过 `ctx.event.query.workspace_uuid` 或新增 `ctx.get_workspace()` 访问。
### Host API 默认按当前 workspace 限制
`LangBotAPIProxy` 的以下方法必须由 Host 端按 workspace 过滤:
- `get_bots`
- `get_bot_info`
- `send_message`
- `get_llm_models`
- `invoke_llm`
- `list_plugins_manifest`
- `list_commands`
- `list_tools`
- `call_tool`
- `invoke_embedding`
- `vector_*`
- `list_knowledge_bases`
- `retrieve_knowledge`
建议新增显式方法:
- `get_workspace_info()`
- `get_current_account()`
- `get_workspace_storage(...)`
但不要让插件传入任意 workspace id 来越权访问。插件请求的 workspace 应由 Runtime 根据当前 query/plugin connection 填充。
### Workspace storage 修复
当前 SDK runtime 中:
```python
data["owner_type"] = "workspace"
data["owner"] = "default"
```
必须改为:
- 如果请求来自 query/eventowner 为 `workspace_uuid`
- 如果请求来自后台插件任务:owner 为 plugin 安装所属 workspace。
- Host 侧 `binary_storages``workspace_uuid`,并在 unique key 中包含 workspace。
Plugin storage 建议也同时加 workspace
- 现在 plugin storage owner 是 `author/name`,这会导致同一插件在不同 workspace 的私有数据冲突。
- 应改为 `(workspace_uuid, plugin_id, key)`
### 插件安装与配置
`plugin_settings` 从全局变为 workspace-scoped
- 同一个插件可安装到多个 workspace。
- 每个 workspace 有自己的 enabled、priority、config、install_source、install_info。
- 插件 runtime 列表需要能按 workspace 过滤。
实现路线有两种:
1. 共享插件进程,插件代码只加载一份,设置和调用时附带 workspace。
2. 每个 workspace 一个插件容器实例,隔离最彻底但资源占用更高。
推荐第一阶段采用方案 1,但要求:
- 所有 RuntimeToLangBot/PluginToRuntime action 都能携带 `workspace_uuid`
- 插件 config 获取时按 workspace 返回。
- 插件 page API 请求必须校验当前用户在该 workspace 有访问权限。
### MCP
MCP server 是租户资源:
- `mcp_servers.workspace_uuid`
- MCP session key 从 `server_name` 改为 `(workspace_uuid, server_name)` 或使用全局 uuid。
- Pipeline extension preferences 中绑定 MCP server uuid 时,只能绑定同 workspace 的 server。
- MCP tool 列表在 query 执行时按 query.workspace_uuid + pipeline 绑定关系过滤。
## 前端改造
### Workspace selector
Home layout 顶部或 sidebar 增加 workspace selector
- 当前 workspace 名称和头像。
- 切换 workspace 后写入 `localStorage.currentWorkspaceId`
- 所有请求自动带 `X-Workspace-Id`
- 切换后刷新 sidebar 数据和页面缓存。
`BaseHttpClient` request interceptor 增加:
```ts
const workspaceId = localStorage.getItem("currentWorkspaceId");
if (workspaceId) config.headers["X-Workspace-Id"] = workspaceId;
```
### 用户与成员管理页面
新增页面:
- `/home/workspace/settings`
- `/home/workspace/members`
- `/home/workspace/invitations`
能力:
- owner/admin 邀请成员。
- owner/admin 修改成员角色。
- owner 移除成员、转让 owner。
- 所有人可切换 workspace。
- viewer/operator 在 UI 上隐藏不可操作按钮,但后端仍做权限校验。
### 登录与注册
登录后流程:
1. `authUser` 拿 token。
2. `initializeUserInfo()` 获取 account info。
3. `GET /api/v1/workspaces`
4. 如果没有 workspace:进入创建 workspace 向导。
5. 如果有多个 workspace:默认进入最近使用 workspace,可切换。
注册页不再表达“初始化管理员账号”,而是:
- 首次系统启动:创建首个 owner + default workspace。
- 后续:根据配置允许公开注册,或只能接受邀请。
### 旧页面影响
需要逐个检查这些页面的数据加载是否都依赖当前 workspace:
- Bots
- Pipelines
- Plugins/Market/MCP
- Knowledge
- Monitoring
- Models dialog
- API integration dialog
- Wizard
## 迁移方案
### 迁移阶段 0:准备
- 引入 `workspaces``workspace_memberships``workspace_invitations`
-`users` 增加 `uuid/status/display_name` 等字段。
- 创建 `RequestContext`,但先不强制所有服务改完。
### 迁移阶段 1:默认 workspace
对现有实例执行迁移:
1. 创建 `Default Workspace`
2. 找到现有第一个 user,设为 owner。
3. 所有已有资源写入 `workspace_uuid=default_workspace_uuid`
4. `metadata` 迁入 default workspace;确实全局的配置放到 `system_metadata`
5. `binary_storages``owner_type=workspace, owner=default` 改为 owner 为 default workspace uuid。
6. 插件 `plugin_settings` 归入 default workspace。
### 迁移阶段 2:服务层强制 scope
- 改所有 service 查询,必须要求 `workspace_uuid`
- API Key 迁移为 workspace key。
- 所有写操作必须检查权限。
- 监控和任务查询按 workspace 过滤。
### 迁移阶段 3:运行时上下文
- `Query``Session``RuntimeBot``RuntimePipeline` 增加 workspace。
- Plugin/MCP/Model/RAG runtime 全部按 workspace 过滤。
- 修复 SDK workspace storage。
### 迁移阶段 4:前端多 workspace
- 登录后 workspace 选择。
- Header/sidebar workspace switcher。
- 成员和邀请管理。
- 所有 API 请求带 `X-Workspace-Id`
### 迁移阶段 5:安全收敛
- 添加跨 workspace 越权测试。
- 添加 API Key scope 测试。
- 添加插件 Host API 过滤测试。
- 添加 MCP 和 RAG 隔离测试。
## 安全边界
必须防的场景:
- 用户 A 修改 URL 中 bot uuid,访问用户 B workspace 的 bot。
- API Key 来自 workspace A,但调用 workspace B 的 bot。
- 插件通过 `get_bots()` 枚举所有 workspace 的 bot。
- 插件通过 `workspace_storage` 读取其它 workspace 的数据。
- MCP server 名称相同导致 session 复用。
- monitoring session_id 相同导致数据串租户。
- Space OAuth 登录时,同 email 账户被错误绑定到已有本地 account。
建议策略:
- 所有资源访问都使用 `workspace_uuid + resource_id`
- 所有 service 方法入口做权限检查。
- 插件 Host API 的 workspace 不信任插件入参,只信任 query/runtime connection 上下文。
- API Key 只授予最小 scope,默认不允许成员管理。
- owner 角色不能被普通 admin 移除或降权。
## 实施优先级
### P0:基础租户骨架
- Account uuid/status。
- Workspace / Membership / Invitation。
- RequestContext。
- JWT 改为 account uuid。
- 前端 current workspace header。
### P1:资源行级隔离
- Bots、Pipelines、Models、MCP、Plugins、Knowledge、Monitoring、API Keys 全部加 workspace_uuid。
- service 查询统一加 workspace filter。
- 权限矩阵落地。
### P2:运行时隔离
- Query、Session、RuntimeBot、RuntimePipeline 加 workspace。
- Plugin Host API 和 MCP tools 按 workspace 过滤。
- SDK workspace storage 从 `default` 改为真实 workspace。
### P3:协作体验
- 邀请成员。
- 成员列表和角色管理。
- workspace switcher。
- 最近使用 workspace。
### P4SaaS 运维增强
- Workspace 级用量统计。
- Workspace 级限额:max_bots/max_pipelines/max_extensions/tokens/storage。
- 审计日志。
- workspace suspend/delete。
- 可选自定义角色。
## 测试计划
后端测试:
- 账户可加入多个 workspace。
- 同账户在不同 workspace 权限不同。
- viewer 不能创建/修改资源。
- API Key 只能访问所属 workspace。
- 所有资源 list/get/update/delete 都不能跨 workspace。
- 默认 workspace 迁移后旧数据可用。
运行时测试:
- 两个 workspace 使用相同 `launcher_id` 不共享 session。
- 两个 workspace 使用相同 MCP server name 不共享 MCP session。
- 插件 `get_bots()` 只能看到当前 workspace bot。
- 插件 `workspace_storage` 在不同 workspace 读写隔离。
- Pipeline 只调用当前 workspace 绑定的插件和 MCP tools。
前端测试:
- 登录后自动进入最近 workspace。
- 切换 workspace 后列表数据变化。
- 无权限按钮隐藏,直接调用 API 也被后端拒绝。
- 邀请成员流程完整。
迁移测试:
- SQLite 老实例迁移。
- PostgreSQL 老实例迁移。
- 已有 local account 迁移为 default workspace owner。
- 已有 Space account token 和 Space model provider API key 不丢失。
## 关键实现注意事项
- 不建议在第一版做数据库 schema-per-tenant。LangBot 当前 ORM 和运行时均以单库单表为主,先做 shared schema + workspace_uuid 成本更低。
- 不建议每个 workspace 立即启动独立 plugin runtime。先共享 runtime,强制 action 带 workspace;大客户隔离可作为后续部署形态。
- 不要只在前端过滤 workspace。插件、API Key、MCP、RAG 都能绕过前端,必须在后端和运行时层过滤。
- `metadata` 要拆清楚:wizard status 属于 workspace,系统版本/迁移信息属于 system。
- `users.user` 用 email 当主键语义不稳,应尽快引入 `account_uuid` 并让 JWT 以 uuid 为准。
- `plugin_settings` 当前主键没有 workspace,改造时要先改主键/唯一约束,否则同插件无法在多个 workspace 配不同配置。
## 建议落地顺序
1. 新增 workspace/account/membership 表和 RequestContext。
2. 迁移旧数据到 default workspace。
3. 改 auth 和前端请求头,让每个请求都有 current workspace。
4. 从最核心资源开始逐个加 scopebot -> pipeline -> provider/model -> plugin/MCP -> knowledge -> monitoring。
5. 改 SDK Query/Event 和 runtime storage。
6. 上成员管理 UI 和邀请。
7. 做越权测试和迁移测试。
这个顺序的好处是可以较早让主 UI 在一个 workspace 下继续工作,同时把最危险的跨租户泄露面逐步收紧。
+3 -2
View File
@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.10.4"
version = "4.10.5"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -70,7 +70,7 @@ dependencies = [
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.4.6",
"langbot-plugin==0.4.13",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"matrix-nio>=0.25.2",
@@ -80,6 +80,7 @@ dependencies = [
"pgvector>=0.4.1",
"botocore>=1.42.39",
"litellm>=1.0.0",
"valkey-glide>=2.4.1,<3.0.0",
]
keywords = [
"bot",
@@ -138,6 +138,39 @@ class MonitoringRouterGroup(group.RouterGroup):
}
)
@self.route('/tool-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_tool_calls() -> str:
"""Get tool call records"""
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
session_ids = quart.request.args.getlist('sessionId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
tool_calls, total = await self.ap.monitoring_service.get_tool_calls(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
session_ids=session_ids if session_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'tool_calls': tool_calls,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/embedding-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_embedding_calls() -> str:
"""Get embedding call records"""
@@ -284,6 +317,16 @@ class MonitoringRouterGroup(group.RouterGroup):
offset=0,
)
# Get tool calls
tool_calls, tool_calls_total = await self.ap.monitoring_service.get_tool_calls(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=0,
)
# Get sessions
sessions, sessions_total = await self.ap.monitoring_service.get_sessions(
bot_ids=bot_ids if bot_ids else None,
@@ -318,12 +361,14 @@ class MonitoringRouterGroup(group.RouterGroup):
'overview': overview,
'messages': messages,
'llmCalls': llm_calls,
'toolCalls': tool_calls,
'embeddingCalls': embedding_calls,
'sessions': sessions,
'errors': errors,
'totalCount': {
'messages': messages_total,
'llmCalls': llm_calls_total,
'toolCalls': tool_calls_total,
'embeddingCalls': embedding_calls_total,
'sessions': sessions_total,
'errors': errors_total,
@@ -29,11 +29,11 @@ class MCPRouterGroup(group.RouterGroup):
traceback.print_exc()
return self.http_status(500, -1, f'Failed to create MCP server: {str(e)}')
@self.route('/servers/<server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN)
@self.route(
'/servers/<path:server_name>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN
)
async def _(server_name: str) -> str:
"""获取、更新或删除MCP服务器配置"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await self.ap.mcp_service.get_mcp_server_by_name(server_name)
@@ -58,17 +58,15 @@ class MCPRouterGroup(group.RouterGroup):
except Exception as e:
return self.http_status(500, -1, f'Failed to delete MCP server: {str(e)}')
@self.route('/servers/<server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/servers/<path:server_name>/test', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""测试MCP服务器连接"""
from urllib.parse import unquote
server_name = unquote(server_name)
server_data = await quart.request.json
task_id = await self.ap.mcp_service.test_mcp_server(server_name=server_name, server_data=server_data)
return self.success(data={'task_id': task_id})
@self.route('/servers/<server_name>/resources', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/servers/<path:server_name>/resources', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""Get resources from an MCP server"""
server_name = unquote(server_name)
@@ -86,7 +84,9 @@ class MCPRouterGroup(group.RouterGroup):
except Exception as e:
return self.http_status(500, -1, f'Failed to get resources: {str(e)}')
@self.route('/servers/<server_name>/resource-templates', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
@self.route(
'/servers/<path:server_name>/resource-templates', methods=['GET'], auth_type=group.AuthType.USER_TOKEN
)
async def _(server_name: str) -> str:
"""Get resource templates from an MCP server"""
server_name = unquote(server_name)
@@ -96,7 +96,20 @@ class MCPRouterGroup(group.RouterGroup):
except Exception as e:
return self.http_status(500, -1, f'Failed to get resource templates: {str(e)}')
@self.route('/servers/<server_name>/resources/read', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/servers/<path:server_name>/logs', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""Get logs from an MCP server"""
server_name = unquote(server_name)
try:
limit = int(quart.request.args.get('limit', 200))
except (TypeError, ValueError):
limit = 200
limit = min(limit, 500)
level = quart.request.args.get('level') or None
logs = await self.ap.mcp_service.get_mcp_server_logs(server_name, limit=limit, level=level)
return self.success(data={'logs': logs})
@self.route('/servers/<path:server_name>/resources/read', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(server_name: str) -> str:
"""Read a resource from an MCP server"""
server_name = unquote(server_name)
@@ -243,6 +243,7 @@ class MaintenanceService:
tables = {
'messages': persistence_monitoring.MonitoringMessage.id,
'llm_calls': persistence_monitoring.MonitoringLLMCall.id,
'tool_calls': persistence_monitoring.MonitoringToolCall.id,
'embedding_calls': persistence_monitoring.MonitoringEmbeddingCall.id,
'errors': persistence_monitoring.MonitoringError.id,
'sessions': persistence_monitoring.MonitoringSession.session_id,
+41 -2
View File
@@ -48,6 +48,17 @@ class MCPService:
if total_extensions >= max_extensions:
raise ValueError(f'Maximum number of extensions ({max_extensions}) reached')
server_name = str(server_data.get('name') or '').strip()
if not server_name:
raise ValueError('MCP server name is required')
server_data['name'] = server_name
existing_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_mcp.MCPServer).where(persistence_mcp.MCPServer.name == server_name)
)
if existing_result.first() is not None:
raise ValueError(f'MCP server already exists: {server_name}')
server_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_mcp.MCPServer).values(server_data))
@@ -177,10 +188,22 @@ class MCPService:
persisted_session = runtime_mcp_session
async def _refresh_and_report() -> None:
if persisted_session.status == MCPSessionStatus.ERROR:
# Testing a persisted server should REUSE its live shared-session
# process, not rebuild it. Try a lightweight refresh (a real
# list_tools probe over the existing connection) first; only fall
# back to a full start() when the session has no live connection
# to probe (never connected, or the process is actually gone).
needs_start = persisted_session.status == MCPSessionStatus.ERROR or persisted_session.session is None
if needs_start:
await persisted_session.start()
else:
await persisted_session.refresh()
try:
await persisted_session.refresh()
except Exception:
# The live connection was stale/dropped: reconnect once
# (reusing the live managed process where possible) and
# re-probe, instead of reporting a false failure.
await persisted_session.start()
# Surface the discovered tools so the config page can render them
# even for an already-hosted server.
ctx.metadata['runtime_info'] = persisted_session.get_runtime_info_dict()
@@ -221,3 +244,19 @@ class MCPService:
context=ctx,
)
return wrapper.id
async def get_mcp_server_logs(self, server_name: str, limit: int = 200, level: str | None = None) -> list[dict]:
"""Get recent log lines captured from the MCP server's stderr."""
session = self.ap.tool_mgr.mcp_tool_loader.get_session(server_name)
if not session:
return []
# Get logs from the session's buffer
logs = list(session._log_buffer)
# Filter by level if specified
if level:
logs = [log for log in logs if log.get('level') == level]
# Return the most recent 'limit' logs
return logs[-limit:]
@@ -2,6 +2,7 @@ from __future__ import annotations
import uuid
import datetime
import json
import sqlalchemy
from ....core import app
@@ -50,6 +51,12 @@ class MonitoringService:
persistence_monitoring.MonitoringLLMCall.timestamp,
persistence_monitoring.MonitoringLLMCall.id,
),
(
'monitoring_tool_calls',
persistence_monitoring.MonitoringToolCall,
persistence_monitoring.MonitoringToolCall.timestamp,
persistence_monitoring.MonitoringToolCall.id,
),
(
'monitoring_embedding_calls',
persistence_monitoring.MonitoringEmbeddingCall,
@@ -131,6 +138,68 @@ class MonitoringService:
await autocommit_conn.execute(sqlalchemy.text('PRAGMA wal_checkpoint(TRUNCATE)'))
await autocommit_conn.execute(sqlalchemy.text('VACUUM'))
def _serialize_tool_payload(self, payload: object, max_length: int = 20000) -> str | None:
"""Serialize tool arguments/results for monitoring storage."""
if payload is None:
return None
if isinstance(payload, str):
text = payload
else:
try:
text = json.dumps(payload, ensure_ascii=False, default=str)
except Exception:
text = str(payload)
if len(text) <= max_length:
return text
return f'{text[:max_length]}... [truncated {len(text) - max_length} chars]'
async def _get_message_for_tool_context(
self,
message_id: str | None = None,
session_id: str | None = None,
):
if message_id:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_monitoring.MonitoringMessage).where(
persistence_monitoring.MonitoringMessage.id == message_id
)
)
row = result.first()
if row:
return row[0]
if not session_id:
return None
user_query = (
sqlalchemy.select(persistence_monitoring.MonitoringMessage)
.where(
sqlalchemy.and_(
persistence_monitoring.MonitoringMessage.session_id == session_id,
persistence_monitoring.MonitoringMessage.role == 'user',
)
)
.order_by(persistence_monitoring.MonitoringMessage.timestamp.desc())
.limit(1)
)
result = await self.ap.persistence_mgr.execute_async(user_query)
row = result.first()
if row:
return row[0]
any_query = (
sqlalchemy.select(persistence_monitoring.MonitoringMessage)
.where(persistence_monitoring.MonitoringMessage.session_id == session_id)
.order_by(persistence_monitoring.MonitoringMessage.timestamp.desc())
.limit(1)
)
result = await self.ap.persistence_mgr.execute_async(any_query)
row = result.first()
return row[0] if row else None
# ========== Recording Methods ==========
async def record_message(
@@ -220,6 +289,57 @@ class MonitoringService:
return call_id
async def record_tool_call(
self,
tool_name: str,
tool_source: str,
duration: int,
status: str = 'success',
bot_id: str | None = None,
bot_name: str | None = None,
pipeline_id: str | None = None,
pipeline_name: str | None = None,
session_id: str | None = None,
message_id: str | None = None,
arguments: object | None = None,
result: object | None = None,
error_message: str | None = None,
) -> str:
"""Record a tool call."""
context_message = await self._get_message_for_tool_context(message_id=message_id, session_id=session_id)
if context_message:
bot_id = bot_id or context_message.bot_id
bot_name = bot_name or context_message.bot_name
pipeline_id = pipeline_id or context_message.pipeline_id
pipeline_name = pipeline_name or context_message.pipeline_name
session_id = session_id or context_message.session_id
message_id = message_id or context_message.id
call_id = str(uuid.uuid4())
call_data = {
'id': call_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'tool_name': tool_name,
'tool_source': tool_source,
'duration': max(0, duration),
'status': status,
'bot_id': bot_id or 'unknown',
'bot_name': bot_name or 'Unknown',
'pipeline_id': pipeline_id or 'unknown',
'pipeline_name': pipeline_name or 'Unknown',
'session_id': session_id,
'message_id': message_id,
'arguments': self._serialize_tool_payload(arguments),
'result': self._serialize_tool_payload(result),
'error_message': self._serialize_tool_payload(error_message),
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringToolCall).values(call_data)
)
return call_id
async def record_embedding_call(
self,
model_name: str,
@@ -749,6 +869,58 @@ class MonitoringService:
total,
)
async def get_tool_calls(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
session_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get tool calls with filters"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringToolCall.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringToolCall.pipeline_id.in_(pipeline_ids))
if session_ids:
conditions.append(persistence_monitoring.MonitoringToolCall.session_id.in_(session_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringToolCall.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringToolCall.timestamp <= end_time)
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringToolCall.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
query = sqlalchemy.select(persistence_monitoring.MonitoringToolCall).order_by(
persistence_monitoring.MonitoringToolCall.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
tool_calls_rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringToolCall, row[0] if isinstance(row, tuple) else row
)
for row in tool_calls_rows
],
total,
)
async def get_embedding_calls(
self,
start_time: datetime.datetime | None = None,
@@ -971,6 +1143,34 @@ class MonitoringService:
else:
error_llm_calls += 1
# Get tool calls for this session
tool_query = (
sqlalchemy.select(persistence_monitoring.MonitoringToolCall)
.where(persistence_monitoring.MonitoringToolCall.session_id == session_id)
.order_by(persistence_monitoring.MonitoringToolCall.timestamp.asc())
)
tool_result = await self.ap.persistence_mgr.execute_async(tool_query)
tool_rows = tool_result.all()
tool_calls = [
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringToolCall, row[0] if isinstance(row, tuple) else row
)
for row in tool_rows
]
total_tool_calls = len(tool_rows)
success_tool_calls = 0
error_tool_calls = 0
total_tool_duration = 0
for row in tool_rows:
tool_call = row[0] if isinstance(row, tuple) else row
total_tool_duration += tool_call.duration
if tool_call.status == 'success':
success_tool_calls += 1
else:
error_tool_calls += 1
# Get errors for this session
error_query = (
sqlalchemy.select(persistence_monitoring.MonitoringError)
@@ -1014,6 +1214,14 @@ class MonitoringService:
'total_tokens': total_tokens,
'average_duration_ms': int(total_duration / total_llm_calls) if total_llm_calls > 0 else 0,
},
'tool_calls': tool_calls,
'tool_stats': {
'total_calls': total_tool_calls,
'success_calls': success_tool_calls,
'error_calls': error_tool_calls,
'total_duration_ms': total_tool_duration,
'average_duration_ms': int(total_tool_duration / total_tool_calls) if total_tool_calls > 0 else 0,
},
'errors': errors,
'session_duration_seconds': session_duration_seconds,
}
@@ -49,6 +49,28 @@ class MonitoringLLMCall(Base):
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True) # Associated message ID
class MonitoringToolCall(Base):
"""Tool call records"""
__tablename__ = 'monitoring_tool_calls'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
tool_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
tool_source = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # native, plugin, mcp, skill
duration = sqlalchemy.Column(sqlalchemy.Integer, nullable=False) # milliseconds
status = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # success, error
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
arguments = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
result = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
error_message = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
class MonitoringSession(Base):
"""Session tracking records"""
@@ -0,0 +1,17 @@
from langbot.pkg.entity.persistence import monitoring as persistence_monitoring
from .. import migration
@migration.migration_class(26)
class DBMigrateMonitoringToolCalls(migration.DBMigration):
"""Add monitoring_tool_calls table"""
async def upgrade(self):
"""Upgrade"""
async with self.ap.persistence_mgr.get_db_engine().begin() as conn:
await conn.run_sync(persistence_monitoring.MonitoringToolCall.__table__.create, checkfirst=True)
async def downgrade(self):
"""Downgrade"""
async with self.ap.persistence_mgr.get_db_engine().begin() as conn:
await conn.run_sync(persistence_monitoring.MonitoringToolCall.__table__.drop, checkfirst=True)
+23 -1
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import typing
import asyncio
import traceback
import uuid
import datetime
import pydantic
@@ -182,7 +183,28 @@ class WecomCSAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
)
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
pass
if target_type != 'person':
raise ValueError('WeCom customer service only supports sending messages to person targets')
open_kfid = self.bot_account_id
external_userid = target_id
if '|' in target_id:
open_kfid, external_userid = target_id.split('|', 1)
if external_userid.startswith('u'):
external_userid = external_userid[1:]
if not open_kfid:
raise ValueError('WeCom customer service open_kfid is required before sending messages')
content_list = await WecomMessageConverter.yiri2target(message, self.bot)
for content in content_list:
msgid = f'langbot_{uuid.uuid4().hex}'
if content['type'] == 'text':
await self.bot.send_text_msg(
open_kfid=open_kfid,
external_userid=external_userid,
msgid=msgid,
content=content['content'],
)
def set_bot_uuid(self, bot_uuid: str):
"""设置 bot UUID(用于生成 webhook URL"""
+101 -8
View File
@@ -25,7 +25,7 @@ from ....core import app
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
import langbot_plugin.api.entities.builtin.provider.message as provider_message
from ....entity.persistence import mcp as persistence_mcp
from .mcp_stdio import BoxStdioSessionRuntime, MCPServerBoxConfig, MCPSessionErrorPhase # noqa: F401
from .mcp_stdio import BoxStdioSessionRuntime, MCPServerBoxConfig, MCPSessionErrorPhase, _ColdStartRetry # noqa: F401
# Synthesized LLM tools for MCP resources (not from server tools/list).
# Dispatched in MCPLoader.invoke_tool; placeholder func on LLMTool is never used.
@@ -185,6 +185,16 @@ class MCPSessionStatus(enum.Enum):
ERROR = 'error'
class _TransportReconnect(Exception):
"""Internal signal: the Box stdio WS transport dropped but the managed
process is still alive. Triggers a lightweight transport reconnect that
reuses the live process, instead of a full process rebuild.
Reconnect attempts are NOT counted toward the fatal retry budget, so a
long-lived session can survive arbitrarily many transient drops.
"""
class RuntimeMCPSession:
"""运行时 MCP 会话"""
@@ -254,6 +264,16 @@ class RuntimeMCPSession:
self._lifecycle_task = None
self._shutdown_event = asyncio.Event()
self._ready_event = asyncio.Event()
# Set transiently when a WS transport drop should NOT stop the managed
# process (it will be re-attached on the next initialize()).
self._preserve_managed_process = False
# Log buffer for capturing stderr from Box managed process (maxlen=500 keeps
# recent lines without unbounded memory growth)
import collections as _collections
self._log_buffer: _collections.deque = _collections.deque(maxlen=500)
self._last_stderr_text: str = ''
self._box_stdio_runtime = BoxStdioSessionRuntime(self)
self.box_config = self._box_stdio_runtime.config
@@ -399,11 +419,39 @@ class RuntimeMCPSession:
task.cancel()
for task in done:
if task is monitor_task and not self._shutdown_event.is_set():
# The monitor completed. This is EITHER the managed
# process actually exiting OR just the WS transport
# dropping while the process stays alive in the Box
# runtime. Re-check the real process state so a
# transient transport drop reconnects (reusing the live
# process) instead of tearing the process down and
# running a full rebuild+backoff cycle.
process_still_running = False
try:
process_still_running = await self._box_stdio_runtime._managed_process_is_running()
except Exception:
process_still_running = False
if process_still_running:
self.ap.logger.info(
f'MCP server {self.server_name}: transport dropped but '
f'managed process is still running; reconnecting transport'
)
self.error_phase = MCPSessionErrorPhase.RELAY_CONNECT
# Preserve the live process across the finally-block
# cleanup: only the WS transport should be torn down.
self._preserve_managed_process = True
raise _TransportReconnect('Box managed process transport dropped; reconnecting')
self.error_phase = MCPSessionErrorPhase.RUNTIME
raise Exception('Box managed process exited unexpectedly')
else:
await self._shutdown_event.wait()
except _ColdStartRetry:
# Cold-start in progress: set the preserve flag BEFORE the finally
# block runs so it does not stop the live managed process. The outer
# _lifecycle_loop_with_retry will reuse it on the next attempt.
self._preserve_managed_process = True
raise
except Exception as e:
self.status = MCPSessionStatus.ERROR
self.error_message = str(e)
@@ -424,14 +472,55 @@ class RuntimeMCPSession:
except Exception as e:
self.ap.logger.error(f'Error cleaning up MCP session {self.server_name}: {e}\n{traceback.format_exc()}')
finally:
await self._cleanup_box_stdio_session()
# On a transport-only reconnect the managed process is healthy
# and will be re-attached on the next initialize(); do NOT stop
# it. Any other exit path fully tears the session down.
if getattr(self, '_preserve_managed_process', False):
self._preserve_managed_process = False
else:
await self._cleanup_box_stdio_session()
async def _lifecycle_loop_with_retry(self):
"""Wrap _lifecycle_loop with retry and exponential backoff."""
for attempt in range(self._MAX_RETRIES + 1):
attempt = 0
while attempt <= self._MAX_RETRIES:
try:
await self._lifecycle_loop()
return # Normal shutdown, don't retry
except _TransportReconnect as e:
# Transient WS transport drop while the managed process is still
# alive. Reconnect promptly WITHOUT consuming the fatal retry
# budget and WITHOUT stopping the process — initialize() will
# re-attach to the live process. This is what lets a long-lived
# stdio MCP survive repeated brief event-loop stalls / pings.
if self._shutdown_event.is_set():
return
self.ap.logger.info(
f'MCP session {self.server_name}: reconnecting transport ({self._describe_exception(e)})'
)
self.status = MCPSessionStatus.CONNECTING
self.error_message = None
self.error_phase = None
await asyncio.sleep(1)
continue
except _ColdStartRetry as e:
# The managed process is alive but still cold-starting (e.g.
# `npx -y <pkg>` is still installing) and cannot yet answer the
# handshake. Reuse the live process and retry the attach WITHOUT
# consuming the fatal retry budget or stopping the process, so a
# slow cold start is waited out instead of failing. Preserve the
# process across the finally-block cleanup.
if self._shutdown_event.is_set():
return
self._preserve_managed_process = True
self.ap.logger.debug(
f'MCP session {self.server_name}: waiting for cold start ({self._describe_exception(e)})'
)
self.status = MCPSessionStatus.CONNECTING
self.error_message = None
self.error_phase = None
await asyncio.sleep(2)
continue
except Exception as e:
self.retry_count = attempt + 1
if self._shutdown_event.is_set():
@@ -460,6 +549,7 @@ class RuntimeMCPSession:
self.error_message = None
self.error_phase = None
await asyncio.sleep(delay)
attempt += 1
@staticmethod
def _describe_exception(exc: BaseException) -> str:
@@ -927,11 +1017,14 @@ class RuntimeMCPSession:
return self._box_stdio_runtime.uses_box_stdio()
def _build_box_session_id(self) -> str:
# Transient test sessions get their own isolated Box session so a
# failing/short-lived test can never disturb the shared session that
# hosts live, already-connected MCP servers.
if self.is_transient:
return f'mcp-test-{self.server_uuid}'
# Both live servers and transient config-page tests share ONE Box
# session ('mcp-shared'). A test therefore reuses the already-running
# container (and, for an existing server, its live managed process)
# instead of paying a full per-test session cold-start + dependency
# bootstrap. Isolation between a test and the live servers is provided
# at the *process* level: each server/test has its own process_id and a
# test only ever stops its own process_id (see cleanup_session), so it
# never disturbs another server's process or the shared session itself.
return 'mcp-shared'
def _rewrite_path(self, path: str, host_path: str | None) -> str:
@@ -6,7 +6,7 @@ import os
import shutil
import shlex
import threading
from contextlib import suppress
from contextlib import suppress, AsyncExitStack
from typing import TYPE_CHECKING, Any
import pydantic
@@ -57,6 +57,23 @@ class MCPSessionErrorPhase(enum.Enum):
BOX_UNAVAILABLE = 'box_unavailable'
def _get_default_memory_mb(ap) -> int:
"""Read box.default_memory_mb from instance config (env: BOX__DEFAULT_MEMORY_MB).
Falls back to 1536 MB a safe floor for Node.js V8 + WASM under nsjail.
Operators running memory-constrained hosts can lower this; those with large
machines can raise it. Individual MCP servers can still override via their
own box.memory_mb setting.
"""
try:
data = getattr(getattr(ap, 'instance_config', None), 'data', None)
if isinstance(data, dict):
return int(data.get('box', {}).get('default_memory_mb', 1536))
except (TypeError, ValueError):
pass
return 1536
class MCPServerBoxConfig(pydantic.BaseModel):
"""Structured configuration for running an MCP server inside a Box container."""
@@ -74,6 +91,35 @@ class MCPServerBoxConfig(pydantic.BaseModel):
model_config = pydantic.ConfigDict(extra='ignore')
_HANDSHAKE_ATTEMPT_TIMEOUT_SEC = 10.0
class _TransferredStack:
"""Adapts an already-populated AsyncExitStack into an async context manager
so ownership of its resources can be transferred into another exit stack.
Entering is a no-op; exiting closes the wrapped stack (and thus the live WS
transport + ClientSession) when the owning session shuts down."""
def __init__(self, stack: AsyncExitStack):
self._stack = stack
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
await self._stack.aclose()
return False
class _ColdStartRetry(Exception):
"""Signal: the managed process is alive but not yet answering the MCP
handshake because it is still cold-starting (e.g. `npx -y <pkg>` is still
installing). The outer lifecycle retry treats this like a transient
reconnect: it reuses the live process and does not count toward the fatal
retry budget, so a slow cold start is waited out rather than failing.
"""
class BoxStdioSessionRuntime:
"""Encapsulate Box-backed stdio MCP session orchestration."""
@@ -113,7 +159,17 @@ class BoxStdioSessionRuntime:
read_only_rootfs=self.config.read_only_rootfs if self.config.read_only_rootfs is not None else False,
image=self.config.image,
cpus=self.config.cpus,
memory_mb=self.config.memory_mb,
# Node.js runtimes (npx/bunx) reserve large virtual address space and
# load WebAssembly modules (llhttp) on startup; the default 512 MB
# cgroup_mem_max is too small and causes OOM kills (return_code=137).
# Auto-bump to 1024 MB when the runner is npx/bunx/pnpm dlx.
# Per-server override wins; global default comes from
# config.yaml box.default_memory_mb (env: BOX__DEFAULT_MEMORY_MB).
# Hard floor of 1536 MB: enough for Node.js V8 + WASM without OOM.
# Per-server override wins; global default from config.yaml
# box.default_memory_mb (env: BOX__DEFAULT_MEMORY_MB), hard floor
# of 1536 MB so Node.js V8 + WASM never OOM under nsjail.
memory_mb=(self.config.memory_mb or _get_default_memory_mb(self.ap)),
pids_limit=self.config.pids_limit,
persistent=True,
)
@@ -173,28 +229,55 @@ class BoxStdioSessionRuntime:
stderr_preview = (result.stderr or '')[:500]
raise Exception(f'Dependency install failed (exit code {result.exit_code}): {stderr_preview}')
try:
process_workspace = (
self._build_workspace(host_path=host_path, workdir=process_cwd, mount_path=process_cwd)
if host_path
else workspace
# Reuse an already-running managed process instead of rebuilding it.
# The Box runtime keeps the managed process alive across a transient
# WebSocket transport drop, so on a reconnect we only need to re-attach
# the WS below. Rebuilding here would needlessly stop a healthy process
# and re-run the (slow, network-touching) dependency bootstrap.
if not await self._managed_process_is_running():
try:
process_workspace = (
self._build_workspace(host_path=host_path, workdir=process_cwd, mount_path=process_cwd)
if host_path
else workspace
)
payload = process_workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
cwd=process_cwd,
)
if install_cmd:
payload = self._wrap_process_payload_with_python_env(payload, process_cwd)
payload['process_id'] = self.process_id
await workspace.box_service.start_managed_process(workspace.session_id, payload)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
raise
else:
self.ap.logger.info(
f'MCP server {self.server_name}: reusing live managed process '
f'process_id={self.process_id} (transport reconnect)'
)
payload = process_workspace.build_process_payload(
self.server_config['command'],
self.server_config.get('args', []),
env=self.server_config.get('env', {}),
cwd=process_cwd,
)
if install_cmd:
payload = self._wrap_process_payload_with_python_env(payload, process_cwd)
payload['process_id'] = self.process_id
await workspace.box_service.start_managed_process(workspace.session_id, payload)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
raise
websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
# Attach the WS transport + MCP session ONCE, on the owner's exit stack,
# in the same task as the serve loop that follows. websocket_client and
# ClientSession use anyio task groups whose cancel scope is bound to the
# frame/stack that entered them, so they must live on the owner exit
# stack (not a deferred/transferred one) or the streams close the moment
# initialize() returns and the next request fails with "Connection
# closed".
#
# A slow (`npx -y <pkg>`) cold start makes this single attempt fail
# while the process is still alive — the package is still installing and
# cannot answer the handshake. We surface that to the outer retry loop
# as a _ColdStartRetry: it must NOT stop the process (it is healthy and
# will be reused) and must NOT consume the fatal retry budget. The next
# attempt re-attaches to the same live process; once it has finished
# cold start the handshake succeeds and stays healthy.
try:
websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
transport = await self.owner.exit_stack.enter_async_context(websocket_client(websocket_url))
read_stream, write_stream = transport
self.owner.session = await self.owner.exit_stack.enter_async_context(
@@ -202,12 +285,19 @@ class BoxStdioSessionRuntime:
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.RELAY_CONNECT
if not await self._managed_process_has_exited():
# Process is alive but not yet serving (cold start) — reconnect.
raise _ColdStartRetry(f'{self.server_name}: transport not ready during cold start')
raise
try:
await self.owner.session.initialize()
except Exception:
await asyncio.wait_for(self.owner.session.initialize(), timeout=_HANDSHAKE_ATTEMPT_TIMEOUT_SEC)
except Exception as exc:
self.owner.error_phase = MCPSessionErrorPhase.MCP_INIT
if not await self._managed_process_has_exited():
raise _ColdStartRetry(
f'{self.server_name}: handshake not ready during cold start ({type(exc).__name__})'
)
raise
async def monitor_process_health(self) -> None:
@@ -234,8 +324,74 @@ class BoxStdioSessionRuntime:
)
if consecutive_errors >= self.owner._MONITOR_MAX_CONSECUTIVE_ERRORS:
return
# Capture stderr logs from the managed process
if isinstance(info, dict):
stderr_text = info.get('stderr', '') or info.get('stderr_preview', '')
else:
stderr_text = getattr(info, 'stderr', '') or getattr(info, 'stderr_preview', '')
if stderr_text and stderr_text != self.owner._last_stderr_text:
# Find new lines not in the previous snapshot
old_lines = set(self.owner._last_stderr_text.splitlines()) if self.owner._last_stderr_text else set()
new_lines = [l for l in stderr_text.splitlines() if l and l not in old_lines]
self.owner._last_stderr_text = stderr_text
import time as _time
for line in new_lines:
level = (
'error'
if any(k in line.upper() for k in ('ERROR', 'CRITICAL'))
else 'warning'
if 'WARNING' in line.upper()
else 'debug'
if 'DEBUG' in line.upper()
else 'info'
)
self.owner._log_buffer.append({'ts': _time.time(), 'level': level, 'text': line})
await asyncio.sleep(self.owner._MONITOR_POLL_INTERVAL)
async def _managed_process_is_running(self) -> bool:
"""Return True if this server's managed process exists and is running.
Used to decide whether initialize() must (re)start the process or can
simply re-attach the WebSocket transport to a process the Box runtime
kept alive across a transient transport drop.
"""
from langbot_plugin.box.models import BoxManagedProcessStatus
workspace = self._build_workspace()
try:
info = await workspace.get_managed_process(self.process_id)
except Exception:
return False
status = info.get('status', '') if isinstance(info, dict) else getattr(info, 'status', '')
return status in (BoxManagedProcessStatus.RUNNING.value, BoxManagedProcessStatus.RUNNING)
async def _managed_process_has_exited(self) -> bool:
"""Return True only if the process is DEFINITIVELY gone (reports EXITED).
Distinct from ``not _managed_process_is_running()``: a process that has
just been spawned may not yet report RUNNING, and a transient query
error is not proof of exit. During the cold-start handshake retry we
must NOT treat 'not yet running' or 'query failed' as a terminal
failure, or we bail out to the outer rebuild path and churn the
process (relay then rejects the early re-attach with HTTP 400). Only a
successful query that reports EXITED stops the retry loop.
"""
from langbot_plugin.box.models import BoxManagedProcessStatus
workspace = self._build_workspace()
try:
info = await workspace.get_managed_process(self.process_id)
except Exception:
# Unknown — treat as 'still coming up', not exited.
return False
status = info.get('status', '') if isinstance(info, dict) else getattr(info, 'status', '')
return status in (BoxManagedProcessStatus.EXITED.value, BoxManagedProcessStatus.EXITED)
async def _stage_host_path_to_shared_workspace(self, host_path: str) -> str:
source_path = normalize_host_path(host_path)
if not source_path:
@@ -342,16 +498,20 @@ class BoxStdioSessionRuntime:
workspace = self._build_workspace(host_path=None)
# Transient test sessions own their isolated Box session, so tear the
# whole session down rather than leaking it. This cannot affect live
# servers because they live in the separate shared session.
# Transient config-page tests now share the same 'mcp-shared' Box
# session as live servers, so we must NOT tear the session down here —
# that would kill every other MCP server in the container. A test is
# isolated at the process level: it ran under its own process_id, so we
# stop only that process, exactly like a live server does below. The
# shared session and all other servers' live processes are untouched.
# (Staged per-test workspace files are still cleaned up.)
if getattr(self.owner, 'is_transient', False):
try:
await workspace.cleanup()
await workspace.stop_managed_process(self.process_id)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to delete transient test session '
f'{self.owner._build_box_session_id()}: {type(exc).__name__}: {exc}'
f'MCP server {self.server_name}: failed to stop transient test process '
f'process_id={self.process_id}: {type(exc).__name__}: {exc}'
)
await self._cleanup_staged_workspace()
return
+114 -4
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import typing
import time
from typing import TYPE_CHECKING
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
@@ -142,21 +143,130 @@ class ToolManager:
return tools
def _get_query_session_id(self, query: pipeline_query.Query) -> str | None:
launcher_type = getattr(query, 'launcher_type', None)
launcher_id = getattr(query, 'launcher_id', None)
if launcher_type is None or launcher_id is None:
return None
launcher_type_value = launcher_type.value if hasattr(launcher_type, 'value') else launcher_type
return f'{launcher_type_value}_{launcher_id}'
async def _record_tool_call(
self,
*,
name: str,
source: str,
parameters: dict,
query: pipeline_query.Query,
duration_ms: int,
status: str,
result: typing.Any = None,
error_message: str | None = None,
) -> None:
monitoring_service = getattr(self.ap, 'monitoring_service', None)
if not monitoring_service:
return
variables = getattr(query, 'variables', {}) or {}
message_id = variables.get('_monitoring_message_id') if isinstance(variables, dict) else None
bot_name = variables.get('_monitoring_bot_name') if isinstance(variables, dict) else None
pipeline_name = variables.get('_monitoring_pipeline_name') if isinstance(variables, dict) else None
try:
await monitoring_service.record_tool_call(
tool_name=name,
tool_source=source,
duration=duration_ms,
status=status,
bot_id=getattr(query, 'bot_uuid', None),
bot_name=bot_name,
pipeline_name=pipeline_name,
session_id=self._get_query_session_id(query),
message_id=message_id,
arguments=parameters,
result=result,
error_message=error_message,
)
except Exception as e:
self.ap.logger.warning(f'Failed to record tool call: {e}')
async def _invoke_tool_with_monitoring(
self,
*,
source: str,
name: str,
parameters: dict,
query: pipeline_query.Query,
invoke: typing.Callable[[], typing.Awaitable[typing.Any]],
) -> typing.Any:
start_time = time.perf_counter()
try:
result = await invoke()
except Exception as e:
duration_ms = int((time.perf_counter() - start_time) * 1000)
await self._record_tool_call(
name=name,
source=source,
parameters=parameters,
query=query,
duration_ms=duration_ms,
status='error',
error_message=str(e),
)
raise
duration_ms = int((time.perf_counter() - start_time) * 1000)
await self._record_tool_call(
name=name,
source=source,
parameters=parameters,
query=query,
duration_ms=duration_ms,
status='success',
result=result,
)
return result
async def execute_func_call(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any:
from langbot.pkg.telemetry import features as telemetry_features
if await self.native_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'native')
return await self.native_tool_loader.invoke_tool(name, parameters, query)
return await self._invoke_tool_with_monitoring(
source='native',
name=name,
parameters=parameters,
query=query,
invoke=lambda: self.native_tool_loader.invoke_tool(name, parameters, query),
)
if await self.plugin_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'plugin')
return await self.plugin_tool_loader.invoke_tool(name, parameters, query)
return await self._invoke_tool_with_monitoring(
source='plugin',
name=name,
parameters=parameters,
query=query,
invoke=lambda: self.plugin_tool_loader.invoke_tool(name, parameters, query),
)
if await self.mcp_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'mcp')
return await self.mcp_tool_loader.invoke_tool(name, parameters, query)
return await self._invoke_tool_with_monitoring(
source='mcp',
name=name,
parameters=parameters,
query=query,
invoke=lambda: self.mcp_tool_loader.invoke_tool(name, parameters, query),
)
if await self.skill_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'skill')
return await self.skill_tool_loader.invoke_tool(name, parameters, query)
return await self._invoke_tool_with_monitoring(
source='skill',
name=name,
parameters=parameters,
query=query,
invoke=lambda: self.skill_tool_loader.invoke_tool(name, parameters, query),
)
raise ToolNotFoundError(name)
async def shutdown(self):
+6
View File
@@ -33,6 +33,12 @@ class VectorDBManager:
self.vector_db = SeekDBVectorDatabase(self.ap)
self.ap.logger.info('Initialized SeekDB vector database backend.')
elif vdb_type == 'valkey_search':
from .vdbs.valkey_search import ValkeySearchVectorDatabase
self.vector_db = ValkeySearchVectorDatabase(self.ap)
self.ap.logger.info('Initialized Valkey Search vector database backend.')
elif vdb_type == 'milvus':
from .vdbs.milvus import MilvusVectorDatabase
@@ -0,0 +1,828 @@
from __future__ import annotations
import asyncio
import json
import struct
from typing import Any
from langbot.pkg.core import app
from langbot.pkg.vector.vdb import VectorDatabase, SearchType
from langbot.pkg.vector.filter_utils import normalize_filter, strip_unsupported_fields
try:
from glide import (
Batch,
GlideClient,
GlideClientConfiguration,
NodeAddress,
RequestError,
ServerCredentials,
ft,
VectorField,
VectorFieldAttributesHnsw,
VectorFieldAttributesFlat,
VectorAlgorithm,
VectorType,
DistanceMetricType,
TagField,
TextField,
FtCreateOptions,
DataType,
FtSearchOptions,
FtSearchLimit,
ReturnField,
)
VALKEY_SEARCH_AVAILABLE = True
except ImportError:
VALKEY_SEARCH_AVAILABLE = False
# Default per-request timeout (ms) for the glide client. The glide library
# default is 250ms, which is too low for vector KNN (``FT.SEARCH ... =>[KNN]``)
# under moderate load or with large indexes and yields spurious TimeoutErrors.
# Overridable via the ``vdb.valkey_search.request_timeout`` config option.
_DEFAULT_REQUEST_TIMEOUT_MS = 5000
# Safety cap on the number of SCAN rounds when purging a collection's keys, so
# a cursor-handling bug or pathological keyspace can never spin forever.
_MAX_SCAN_ROUNDS = 100000
# Mandatory client name for production observability (CLIENT LIST / dashboards).
VALKEY_CLIENT_NAME = 'langbot_vector_client'
# Fixed, indexed metadata schema. LangBot's RAG layer stores ``file_id`` on
# every chunk; it is the only metadata field we promote to a first-class
# (filterable) index field. All other metadata is preserved verbatim inside
# the ``metadata_json`` field so it survives a round-trip, but is NOT
# filterable (the established Milvus / pgvector pragmatism).
_INDEXED_TAG_FIELDS = {'file_id'}
_SUPPORTED_FILTER_FIELDS = set(_INDEXED_TAG_FIELDS)
# Hash field names used for stored documents.
_FIELD_VECTOR = 'vector'
_FIELD_DOCUMENT = 'document'
_FIELD_FILE_ID = 'file_id'
_FIELD_METADATA = 'metadata_json'
_VEC_SCORE_ALIAS = '__vec_score'
# Valkey Search has no bare "match everything" token for non-vector queries
# (a standalone ``*`` is a syntax error). A negated match on a sentinel tag
# value that can never exist matches every key, which is the canonical
# match-all idiom for FT.SEARCH.
_MATCH_ALL = '-@file_id:{__langbot_match_all_sentinel__}'
# Page size used when enumerating matching keys for deletion. Deletes
# paginate through the full result set in batches of this size so that
# files/filters matching more than one page of chunks are fully removed
# (no silent truncation / orphaned vectors).
_DELETE_SCAN_BATCH = 10000
# Characters Valkey Search's TAG query parser cannot handle even when
# backslash-escaped (the brace delimiters and the wildcard). file_id TAG
# values are percent-encoded over this set (plus '%' itself, so the encoding
# is reversible/unambiguous) before being stored or queried, so an arbitrary
# file_id round-trips instead of producing an unparseable query. For normal
# UUID/hash file_ids none of these characters occur, so the encoding is a
# no-op and the stored value is unchanged. The original file_id is always
# preserved verbatim inside ``metadata_json``.
_FT_UNSAFE_TAG_CHARS = frozenset('{}*%')
class ValkeySearchVectorDatabase(VectorDatabase):
"""Valkey Search (valkey-bundle) vector database adapter for LangBot.
Backed by the Valkey Search module shipped in ``valkey/valkey-bundle``,
accessed through the official ``valkey-glide`` client's native ``ft``
(search) command namespace. Documents are stored as Valkey HASH keys
under a per-collection prefix and indexed by one ``FT.CREATE`` index per
collection.
Supported search types: ``VECTOR``, ``FULL_TEXT`` and ``HYBRID``.
Hybrid search semantics (IMPORTANT)
-----------------------------------
Valkey Search hybrid queries follow a *filter-then-KNN* model: the text /
metadata filter pre-selects candidate keys and the KNN stage ranks them by
vector distance. This backend does **NOT** implement application-side
weighted score fusion. The ``vector_weight`` argument is therefore
accepted for interface compatibility but is **not honored** passing
different weights does not change result ordering. A one-time warning is
emitted the first time a non-default weight is supplied. App-side score
fusion can be layered on later if weighted hybrid ranking is required.
"""
@classmethod
def supported_search_types(cls) -> list[SearchType]:
return [SearchType.VECTOR, SearchType.FULL_TEXT, SearchType.HYBRID]
def __init__(self, ap: app.Application):
if not VALKEY_SEARCH_AVAILABLE:
raise ImportError(
"valkey-glide is not installed. Install it with: pip install 'valkey-glide>=2.4.1,<3.0.0'"
)
self.ap = ap
config = self.ap.instance_config.data['vdb']['valkey_search']
self._host = config.get('host', 'localhost')
self._port = int(config.get('port', 6379))
self._db = int(config.get('db', 0))
# Auth / TLS are optional (toB / SaaS). Never logged.
self._password = config.get('password', '') or None
self._username = config.get('username', '') or None
self._tls = bool(config.get('tls', False))
self._request_timeout = int(config.get('request_timeout', _DEFAULT_REQUEST_TIMEOUT_MS))
algorithm = str(config.get('index_algorithm', 'HNSW')).upper()
self._algorithm = VectorAlgorithm.FLAT if algorithm == 'FLAT' else VectorAlgorithm.HNSW
metric = str(config.get('distance_metric', 'COSINE')).upper()
self._distance_metric = {
'COSINE': DistanceMetricType.COSINE,
'L2': DistanceMetricType.L2,
'IP': DistanceMetricType.IP,
}.get(metric, DistanceMetricType.COSINE)
# Lazily-created client (created on first use so a down Valkey does not
# block LangBot boot).
self._client: GlideClient | None = None
# Serializes lazy client creation so concurrent first-use callers do not
# each construct (and leak) a separate GlideClient.
self._client_lock = asyncio.Lock()
# Index names we have already ensured this process lifetime.
self._ensured_indexes: set[str] = set()
# Whether we have already warned about the non-honored vector_weight.
self._vector_weight_warned = False
# ------------------------------------------------------------------ #
# Client lifecycle
# ------------------------------------------------------------------ #
async def _ensure_client(self) -> GlideClient:
"""Create the glide client on first use (lazy, non-blocking boot)."""
if self._client is not None:
return self._client
# Double-checked locking: serialize creation so two concurrent
# first-use callers don't both build a client and leak one.
async with self._client_lock:
if self._client is not None:
return self._client
credentials = None
if self._password is not None:
# username is optional alongside a password (ACL "user" vs default user).
credentials = ServerCredentials(password=self._password, username=self._username)
elif self._username is not None:
# A username without a password is not a valid credential pair, and silently
# connecting unauthenticated to a potentially shared Valkey instance is a
# security footgun (e.g. an env var that failed to resolve). Fail closed.
raise ValueError(
'Valkey Search: a username was configured without a password. '
'Set both username and password to use ACL authentication, or remove both.'
)
conf = GlideClientConfiguration(
addresses=[NodeAddress(self._host, self._port)],
client_name=VALKEY_CLIENT_NAME,
database_id=self._db,
use_tls=self._tls,
lazy_connect=True,
credentials=credentials,
request_timeout=self._request_timeout,
)
self._client = await GlideClient.create(conf)
self.ap.logger.info(
f'Initialized Valkey Search client to {self._host}:{self._port} (db={self._db}, tls={self._tls})'
)
return self._client
async def close(self) -> None:
"""Close the glide client and reset state.
Safe to call when no client was created. After ``close`` the next
operation transparently re-creates the client (``_ensure_client``
guards on ``self._client is None``).
"""
if self._client is not None:
try:
await self._client.close()
except Exception:
self.ap.logger.warning('Valkey Search: error while closing client (ignored)')
finally:
self._client = None
self._ensured_indexes.clear()
# ------------------------------------------------------------------ #
# Naming helpers
# ------------------------------------------------------------------ #
@staticmethod
def _index_name(collection: str) -> str:
return f'idx:{collection}'
@staticmethod
def _key_prefix(collection: str) -> str:
return f'kb:{collection}:'
@staticmethod
def _pack_vector(vec: list[float]) -> bytes:
"""Pack a float vector into little-endian float32 bytes.
Valkey Search stores and queries vectors as FLOAT32 little-endian
blobs (per the search query-language spec).
"""
return struct.pack(f'<{len(vec)}f', *[float(x) for x in vec])
@staticmethod
def _escape_tag(value: str) -> str:
"""Escape characters that are special inside a TAG ``{...}`` clause.
The backslash is escaped first so it cannot consume a following
escape. This neutralises injection-style values (quotes, parens,
``|``, ``@``, ``:``, spaces, dashes) so a crafted ``file_id`` cannot
break out of the clause.
Note: Valkey Search's TAG query parser cannot handle a literal brace
(``{`` / ``}``) or ``*`` even when backslash-escaped. Callers that pass
a ``file_id`` route it through ``_encode_and_escape_tag`` /
``_encode_file_id`` first, which percent-encodes exactly those
characters, so an arbitrary ``file_id`` round-trips safely. This raw
escaper is still correct for all other special characters.
"""
out = []
for ch in str(value):
if ch in '\\,.<>{}[]"\':;!@#$%^&*()-+=~| ':
out.append('\\')
out.append(ch)
return ''.join(out)
@staticmethod
def _encode_file_id(value: str) -> str:
"""Make a ``file_id`` safe to use as an FT TAG token AND query value.
Percent-encodes the characters Valkey Search's TAG parser cannot handle
even when backslash-escaped (``{``, ``}``, ``*``) plus ``%`` itself for
reversibility. Applied identically at write time (the stored TAG field)
and query time (filters / ``delete_by_file_id``) so any value matches
itself. For normal UUID/hash ids none of these characters occur, so
this is a no-op. The original value is always kept verbatim in
``metadata_json``; this encoded form is only ever used for the indexed
TAG.
"""
out = []
for ch in str(value):
if ch in _FT_UNSAFE_TAG_CHARS:
out.append('%{:02X}'.format(ord(ch)))
else:
out.append(ch)
return ''.join(out)
def _encode_and_escape_tag(self, value: str) -> str:
"""Encode an FT-unsafe ``file_id`` then escape TAG special chars."""
return self._escape_tag(self._encode_file_id(value))
# ------------------------------------------------------------------ #
# Filter mapping (canonical triples -> FT query fragment)
# ------------------------------------------------------------------ #
def _triples_to_ft(self, filter: dict[str, Any] | None) -> str:
"""Translate a canonical filter dict into an FT filter expression.
Only indexed fields (``file_id``) are filterable; unsupported fields
are dropped with a warning (matching the Milvus / pgvector pattern).
Returns an empty string when there is no usable filter.
"""
triples = normalize_filter(filter)
if not triples:
return ''
triples = strip_unsupported_fields(triples, _SUPPORTED_FILTER_FIELDS)
fragments: list[str] = []
for field, op, value in triples:
# All currently-indexed fields are TAG fields; file_id values are
# encoded (FT-unsafe chars) then escaped so any value round-trips.
if op == '$eq':
fragments.append(f'@{field}:{{{self._encode_and_escape_tag(value)}}}')
elif op == '$ne':
fragments.append(f'-@{field}:{{{self._encode_and_escape_tag(value)}}}')
elif op == '$in':
joined = '|'.join(self._encode_and_escape_tag(v) for v in value)
fragments.append(f'@{field}:{{{joined}}}')
elif op == '$nin':
joined = '|'.join(self._encode_and_escape_tag(v) for v in value)
fragments.append(f'-@{field}:{{{joined}}}')
elif op == '$gt':
fragments.append(f'@{field}:[({float(value)} +inf]')
elif op == '$gte':
fragments.append(f'@{field}:[{float(value)} +inf]')
elif op == '$lt':
fragments.append(f'@{field}:[-inf ({float(value)}]')
elif op == '$lte':
fragments.append(f'@{field}:[-inf {float(value)}]')
else:
# normalize_filter() already rejects unknown operators, so this
# only triggers if SUPPORTED_OPS grows without this chain being
# updated. Fail closed (rather than silently dropping the
# condition, which would widen delete_by_filter's match set).
raise ValueError(f'Valkey Search: unhandled filter operator {op!r} on field {field!r}')
return ' '.join(fragments)
@staticmethod
def _build_text_clause(text: str) -> str:
"""Build a field-scoped full-text clause for the ``document`` field.
Each whitespace-delimited word becomes a ``@document:<term>`` term and
the terms are AND-ed (space separated). FT special characters in each
term are escaped. Returns an empty string when *text* has no words.
"""
words = [w for w in str(text).split() if w]
if not words:
return ''
terms = [f'@{_FIELD_DOCUMENT}:{ValkeySearchVectorDatabase._escape_text(w)}' for w in words]
return ' '.join(terms)
@staticmethod
def _escape_text(text: str) -> str:
"""Escape FT full-text special characters in a single term."""
out = []
for ch in str(text):
if ch in '@!{}[]()|-"~*:\\':
out.append('\\')
out.append(ch)
return ''.join(out)
# ------------------------------------------------------------------ #
# Index management
# ------------------------------------------------------------------ #
async def _ensure_index(self, client: GlideClient, collection: str, dim: int) -> None:
index = self._index_name(collection)
if index in self._ensured_indexes:
return
# ft.info is O(1) and raises RequestError when the index is absent —
# cheaper than ft.list (O(n) over all indexes) and it closes the
# check-then-create TOCTOU window.
try:
await ft.info(client, index)
self._ensured_indexes.add(index)
return
except RequestError:
pass
if self._algorithm == VectorAlgorithm.FLAT:
vector_attrs = VectorFieldAttributesFlat(
dimensions=dim,
distance_metric=self._distance_metric,
type=VectorType.FLOAT32,
)
else:
vector_attrs = VectorFieldAttributesHnsw(
dimensions=dim,
distance_metric=self._distance_metric,
type=VectorType.FLOAT32,
)
schema = [
VectorField(name=_FIELD_VECTOR, algorithm=self._algorithm, attributes=vector_attrs),
TagField(name=_FIELD_FILE_ID),
TextField(name=_FIELD_DOCUMENT),
]
options = FtCreateOptions(data_type=DataType.HASH, prefixes=[self._key_prefix(collection)])
await ft.create(client, index, schema, options)
self._ensured_indexes.add(index)
self.ap.logger.info(
f"Valkey Search index '{index}' created (dim={dim}, algo={self._algorithm.value}, "
f'metric={self._distance_metric.value})'
)
@staticmethod
def _decode(value: Any) -> str:
if isinstance(value, (bytes, bytearray, memoryview)):
return bytes(value).decode('utf-8', errors='replace')
return str(value)
# ------------------------------------------------------------------ #
# VectorDatabase ABC implementation
# ------------------------------------------------------------------ #
async def get_or_create_collection(self, collection: str):
"""Ensure a client exists.
The index itself requires the vector dimension, which is only known at
first ``add_embeddings`` (same constraint as Qdrant / SeekDB), so this
is a best-effort no-op when the index does not yet exist.
"""
await self._ensure_client()
async def add_embeddings(
self,
collection: str,
ids: list[str],
embeddings_list: list[list[float]],
metadatas: list[dict[str, Any]],
documents: list[str] | None = None,
) -> None:
if not embeddings_list:
return
client = await self._ensure_client()
dim = len(embeddings_list[0])
# The index schema is fixed to the first embedding's dimension. A later
# embedding of a different length would be packed into a wrong-sized
# blob that Valkey stores silently but that yields garbage KNN
# distances, so reject mixed dimensions up-front.
if any(len(e) != dim for e in embeddings_list[1:]):
raise ValueError(f'All embeddings must have dimension {dim}; got mixed lengths')
await self._ensure_index(client, collection, dim)
prefix = self._key_prefix(collection)
batch = Batch(is_atomic=False)
for i, _id in enumerate(ids):
key = prefix + str(_id)
metadata = metadatas[i] if i < len(metadatas) else {}
mapping: dict[str, Any] = {
_FIELD_VECTOR: self._pack_vector(embeddings_list[i]),
_FIELD_METADATA: json.dumps(metadata, ensure_ascii=False),
}
file_id = metadata.get('file_id')
if file_id is not None:
mapping[_FIELD_FILE_ID] = self._encode_file_id(str(file_id))
if documents is not None and i < len(documents) and documents[i] is not None:
mapping[_FIELD_DOCUMENT] = documents[i]
batch.hset(key, mapping)
# Pipeline all HSETs into a single round-trip (non-atomic) instead of
# one await per embedding, which is N sequential round-trips for N
# chunks.
await client.exec(batch, raise_on_error=True)
self.ap.logger.info(f"Added {len(ids)} embeddings to Valkey Search collection '{collection}'")
async def search(
self,
collection: str,
query_embedding: list[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
vector_weight: float | None = None,
) -> dict[str, Any]:
client = await self._ensure_client()
index = self._index_name(collection)
if not await self._index_exists(client, index):
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
# vector_weight is accepted for interface parity but NOT honored by this
# backend (filter-then-KNN, no weighted fusion). Warn once.
if vector_weight is not None and not self._vector_weight_warned:
self.ap.logger.warning(
'Valkey Search backend does not honor vector_weight: hybrid search uses '
'filter-then-KNN without weighted score fusion. The vector_weight value '
'is ignored. See docs/VALKEY_SEARCH_INTEGRATION.md.'
)
self._vector_weight_warned = True
filter_expr = self._triples_to_ft(filter)
if search_type == SearchType.FULL_TEXT:
if not query_text:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
text_clause = self._build_text_clause(query_text)
if not text_clause:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
query = f'{filter_expr} {text_clause}'.strip() if filter_expr else text_clause
return await self._run_text_search(client, index, query, k)
if search_type == SearchType.HYBRID:
# Filter / text pre-selects candidates; KNN ranks. No fusion.
pre = filter_expr
if query_text:
text_clause = self._build_text_clause(query_text)
if text_clause:
pre = f'{pre} {text_clause}'.strip() if pre else text_clause
pre = pre or '*'
query = f'{self._wrap_pre(pre)}=>[KNN {k} @{_FIELD_VECTOR} $BLOB AS {_VEC_SCORE_ALIAS}]'
return await self._run_knn_search(client, index, query, query_embedding, k)
# Default: pure VECTOR search.
pre = filter_expr or '*'
query = f'{self._wrap_pre(pre)}=>[KNN {k} @{_FIELD_VECTOR} $BLOB AS {_VEC_SCORE_ALIAS}]'
return await self._run_knn_search(client, index, query, query_embedding, k)
@staticmethod
def _wrap_pre(pre: str) -> str:
"""Parenthesize a multi-condition pre-filter before the ``=>`` KNN clause.
When ``pre`` combines several terms (e.g. ``@file_id:{x} @document:term``)
the Valkey Search parser can otherwise mis-associate only the last term
with the KNN clause. Wrapping the whole expression forces correct
grouping. A bare ``*`` (match-all) and single-term expressions are left
untouched.
"""
if pre and pre != '*' and ' ' in pre.strip():
return f'({pre})'
return pre
async def _run_knn_search(
self,
client: GlideClient,
index: str,
query: str,
query_embedding: list[float],
k: int,
) -> dict[str, Any]:
options = FtSearchOptions(
params={'BLOB': self._pack_vector(list(query_embedding))},
return_fields=[
ReturnField(field_identifier=_VEC_SCORE_ALIAS, alias='distance'),
ReturnField(field_identifier=_FIELD_DOCUMENT),
ReturnField(field_identifier=_FIELD_METADATA),
],
limit=FtSearchLimit(0, k),
dialect=2,
)
try:
reply = await ft.search(client, index, query, options)
except Exception as exc:
if self._is_missing_index_error(exc):
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
raise
return self._reply_to_chroma(index, reply, has_distance=True)
async def _run_text_search(
self,
client: GlideClient,
index: str,
query: str,
k: int,
) -> dict[str, Any]:
options = FtSearchOptions(
return_fields=[
ReturnField(field_identifier=_FIELD_DOCUMENT),
ReturnField(field_identifier=_FIELD_METADATA),
],
limit=FtSearchLimit(0, k),
dialect=2,
)
try:
reply = await ft.search(client, index, query, options)
except Exception as exc:
if self._is_missing_index_error(exc):
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
raise
return self._reply_to_chroma(index, reply, has_distance=False)
@staticmethod
def _is_missing_index_error(exc: Exception) -> bool:
"""Return True if *exc* indicates the FT index does not exist.
``FT.DROPINDEX`` is applied eventually, so an index can briefly still
appear in ``FT._LIST`` after being dropped; a follow-up search then
fails with a "not found" error which we treat as an empty result.
"""
message = str(exc).lower()
return 'not found' in message and 'index' in message
def _iter_reply_docs(self, reply: Any, prefix: str):
"""Yield ``(doc_id, decoded_fields)`` pairs from an FT.SEARCH reply.
glide returns ``[total, {key: {field: value}, ...}]``. This shared
iterator decodes each key, strips the per-collection prefix to recover
the original document id, and decodes the field map the logic both
``_reply_to_chroma`` and ``list_by_filter`` need.
"""
docs = reply[1] if reply and len(reply) >= 2 and isinstance(reply[1], dict) else {}
for key, fields in docs.items():
key_str = self._decode(key)
doc_id = key_str[len(prefix) :] if prefix and key_str.startswith(prefix) else key_str
decoded_fields = {self._decode(fk): fv for fk, fv in fields.items()} if isinstance(fields, dict) else {}
yield doc_id, decoded_fields
def _reply_to_chroma(self, index: str, reply: Any, has_distance: bool) -> dict[str, Any]:
"""Convert an FT.SEARCH reply into Chroma-style nested lists.
The KNN score field (aliased ``distance``) is a COSINE/L2 distance
directly, so no inversion is needed (unlike Qdrant).
"""
ids: list[str] = []
distances: list[float] = []
metadatas: list[dict[str, Any]] = []
if not reply or len(reply) < 2:
return {'ids': [ids], 'metadatas': [metadatas], 'distances': [distances]}
prefix = self._key_prefix(index[len('idx:') :]) if index.startswith('idx:') else ''
for doc_id, decoded_fields in self._iter_reply_docs(reply, prefix):
ids.append(doc_id)
if has_distance and 'distance' in decoded_fields:
try:
distances.append(float(self._decode(decoded_fields['distance'])))
except (TypeError, ValueError):
distances.append(0.0)
else:
distances.append(0.0)
metadata: dict[str, Any] = {}
raw_meta = decoded_fields.get(_FIELD_METADATA)
if raw_meta is not None:
try:
metadata = json.loads(self._decode(raw_meta))
except (TypeError, ValueError):
metadata = {}
metadatas.append(metadata)
return {'ids': [ids], 'metadatas': [metadatas], 'distances': [distances]}
async def delete_by_file_id(self, collection: str, file_id: str) -> None:
client = await self._ensure_client()
index = self._index_name(collection)
if not await self._index_exists(client, index):
self.ap.logger.warning(f"Valkey Search collection '{collection}' not found for deletion")
return
query = f'@{_FIELD_FILE_ID}:{{{self._encode_and_escape_tag(file_id)}}}'
keys = await self._search_keys(client, index, query)
if keys:
await client.delete(keys)
self.ap.logger.info(
f"Deleted {len(keys)} embeddings from Valkey Search collection '{collection}' with file_id: {file_id}"
)
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
client = await self._ensure_client()
index = self._index_name(collection)
if not await self._index_exists(client, index):
self.ap.logger.warning(f"Valkey Search collection '{collection}' not found for deletion")
return 0
# Guard against accidental mass deletion: a non-empty filter that maps
# to no usable (indexed) conditions must NOT fall back to match-all and
# wipe the whole collection. Skip instead (matching Milvus / pgvector).
query = self._triples_to_ft(filter)
if not query:
self.ap.logger.warning(
"Valkey Search delete_by_filter on '%s': filter produced no usable conditions, skipping",
collection,
)
return 0
keys = await self._search_keys(client, index, query)
if keys:
await client.delete(keys)
self.ap.logger.info(f"Deleted {len(keys)} embeddings from Valkey Search collection '{collection}' by filter")
return len(keys)
async def list_by_filter(
self,
collection: str,
filter: dict[str, Any] | None = None,
limit: int = 20,
offset: int = 0,
) -> tuple[list[dict[str, Any]], int]:
client = await self._ensure_client()
index = self._index_name(collection)
if not await self._index_exists(client, index):
return [], 0
query = self._triples_to_ft(filter) or _MATCH_ALL
options = FtSearchOptions(
return_fields=[
ReturnField(field_identifier=_FIELD_DOCUMENT),
ReturnField(field_identifier=_FIELD_METADATA),
],
limit=FtSearchLimit(offset, limit),
dialect=2,
)
try:
reply = await ft.search(client, index, query, options)
except Exception as exc:
if self._is_missing_index_error(exc):
return [], 0
raise
total = 0
if reply:
try:
total = int(reply[0])
except (TypeError, ValueError):
total = 0
prefix = self._key_prefix(collection)
items: list[dict[str, Any]] = []
for doc_id, decoded_fields in self._iter_reply_docs(reply, prefix):
document = decoded_fields.get(_FIELD_DOCUMENT)
metadata: dict[str, Any] = {}
raw_meta = decoded_fields.get(_FIELD_METADATA)
if raw_meta is not None:
try:
metadata = json.loads(self._decode(raw_meta))
except (TypeError, ValueError):
metadata = {}
items.append(
{
'id': doc_id,
'document': self._decode(document) if document is not None else None,
'metadata': metadata,
}
)
return items, total
async def delete_collection(self, collection: str):
client = await self._ensure_client()
index = self._index_name(collection)
self._ensured_indexes.discard(index)
if await self._index_exists(client, index):
try:
await ft.dropindex(client, index)
except RequestError:
# The index was already dropped (e.g. by a concurrent process)
# between the existence check and this call — benign. Other
# errors (connection / auth) must propagate so the caller knows
# the operation failed rather than silently SCAN-deleting next.
pass
# DROPINDEX does not remove the underlying hashes; delete them too.
prefix = self._key_prefix(collection)
cursor = b'0'
deleted = 0
for _ in range(_MAX_SCAN_ROUNDS):
cursor, keys = await client.scan(cursor, match=f'{prefix}*', count=500)
if keys:
await client.delete(keys)
deleted += len(keys)
if cursor in (b'0', '0', 0):
break
self.ap.logger.info(f"Valkey Search collection '{collection}' deleted ({deleted} keys removed)")
# ------------------------------------------------------------------ #
# Internal search helpers
# ------------------------------------------------------------------ #
async def _index_exists(self, client: GlideClient, index: str) -> bool:
if index in self._ensured_indexes:
return True
# ft.info is O(1) and raises RequestError when the index does not
# exist, vs ft.list which is O(n) over every index on the server and
# was being paid on the first query to each collection.
try:
await ft.info(client, index)
self._ensured_indexes.add(index)
return True
except RequestError:
return False
async def _search_keys(self, client: GlideClient, index: str, query: str) -> list[str]:
"""Return all matching document keys for a query (NOCONTENT).
Paginates through the full result set in pages of ``_DELETE_SCAN_BATCH``
so that queries matching more than one page of chunks are fully
enumerated (avoids silently truncating deletes and leaving orphaned
vectors).
"""
keys: list[str] = []
offset = 0
while True:
options = FtSearchOptions(
nocontent=True,
limit=FtSearchLimit(offset, _DELETE_SCAN_BATCH),
dialect=2,
)
try:
reply = await ft.search(client, index, query, options)
except Exception as exc:
if self._is_missing_index_error(exc):
return keys
raise
if not reply or len(reply) < 2:
break
# reply[0] is the total match count; reply[1] holds this page.
total = 0
try:
total = int(reply[0])
except (TypeError, ValueError):
total = 0
docs = reply[1]
if isinstance(docs, dict):
page = [self._decode(k) for k in docs.keys()]
elif isinstance(docs, (list, tuple)):
page = [self._decode(k) for k in docs]
else:
page = []
if not page:
break
keys.extend(page)
offset += len(page)
if offset >= total or len(page) < _DELETE_SCAN_BATCH:
break
return keys
+19
View File
@@ -87,6 +87,16 @@ vdb:
database: 'langbot'
user: 'postgres'
password: 'postgres'
valkey_search:
host: 'localhost'
port: 6379 # integration tests use 6380 -> valkey/valkey-bundle:9.1.0
db: 0
password: '' # optional (toB auth)
username: '' # optional (ACL user, toB)
tls: false # optional (toB/SaaS)
index_algorithm: 'HNSW' # HNSW | FLAT
distance_metric: 'COSINE' # COSINE | L2 | IP
request_timeout: 5000 # per-request timeout in ms (glide default 250ms is too low for KNN)
storage:
use: local
cleanup:
@@ -143,6 +153,15 @@ box:
- './data/box'
- '/tmp'
workspace_quota_mb: null # Optional disk quota override (>= 0). null = profile default.
# Default nsjail cgroup memory limit for each MCP stdio server process, in MB.
# Node.js MCP servers (npx/bunx) need more memory than Python ones because V8
# and WebAssembly modules (e.g. undici llhttp) reserve large virtual address
# space at startup. Setting this too low causes processes to be killed with
# return_code=137 (OOM kill); the symptom is "Box managed process exited
# unexpectedly" in the logs. Raise on machines with ample RAM; lower only if
# you run exclusively Python (uvx) MCP servers.
# Can also be set via BOX__DEFAULT_MEMORY_MB. Default: 1536.
default_memory_mb: 1536
docker:
cpu_limit_enabled: true # When false, Docker sandbox containers are started without --cpus. Memory and PID limits still apply.
e2b:
+11
View File
@@ -104,6 +104,17 @@ def create_minimal_config(tmpdir: Path, port: int = 15300) -> Path:
'user': 'postgres',
'password': 'postgres',
},
'valkey_search': {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': '',
'username': '',
'tls': False,
'index_algorithm': 'HNSW',
'distance_metric': 'COSINE',
'request_timeout': 5000,
},
},
'storage': {
'use': 'local',
+1
View File
@@ -81,6 +81,7 @@ def fake_monitoring_app():
)
app.monitoring_service.get_messages = AsyncMock(return_value=([{'id': 'msg-1', 'content': 'test'}], 100))
app.monitoring_service.get_llm_calls = AsyncMock(return_value=([{'id': 'llm-1'}], 50))
app.monitoring_service.get_tool_calls = AsyncMock(return_value=([{'id': 'tool-1'}], 5))
app.monitoring_service.get_embedding_calls = AsyncMock(return_value=([{'id': 'emb-1'}], 10))
app.monitoring_service.get_sessions = AsyncMock(return_value=([{'session_id': 'sess-1'}], 20))
app.monitoring_service.get_errors = AsyncMock(return_value=([{'id': 'err-1'}], 2))
@@ -0,0 +1,343 @@
"""Integration tests for the Valkey Search VDB backend.
These are SLOW, real-server tests. They are gated on ``TEST_VALKEY_URL`` and
skipped when it is unset (same precedent as the PostgreSQL migration tests).
Run locally against valkey/valkey-bundle:9.1.0::
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
TEST_VALKEY_URL=valkey://localhost:6380 \\
uv run pytest tests/integration/vector/test_valkey_search.py -m slow -q
The default upstream fast CI lane (``-m "not slow"``) skips these; the local
supervisor validator MUST run them.
"""
from __future__ import annotations
import asyncio
import os
import uuid
from types import SimpleNamespace
from urllib.parse import urlparse
import pytest
pytestmark = [pytest.mark.integration, pytest.mark.slow]
def _parse_valkey_url(url: str) -> tuple[str, int, int]:
"""Parse ``valkey://host:port/db`` into ``(host, port, db)``."""
parsed = urlparse(url)
host = parsed.hostname or 'localhost'
port = parsed.port or 6379
db = 0
if parsed.path and parsed.path.strip('/'):
try:
db = int(parsed.path.strip('/'))
except ValueError:
db = 0
return host, port, db
@pytest.fixture
def valkey_config():
url = os.environ.get('TEST_VALKEY_URL')
if not url:
pytest.skip('TEST_VALKEY_URL not set')
host, port, db = _parse_valkey_url(url)
return {
'host': host,
'port': port,
'db': db,
'password': '',
'username': '',
'tls': False,
'index_algorithm': 'HNSW',
'distance_metric': 'COSINE',
}
def _make_ap(valkey_config):
"""Build a minimal fake ``ap`` with the config + a no-op logger."""
logger = SimpleNamespace(
info=lambda *a, **k: None,
warning=lambda *a, **k: None,
error=lambda *a, **k: None,
debug=lambda *a, **k: None,
)
instance_config = SimpleNamespace(data={'vdb': {'valkey_search': valkey_config}})
return SimpleNamespace(instance_config=instance_config, logger=logger)
@pytest.fixture
async def backend(valkey_config):
"""Create a Valkey Search backend, skip if module/server unavailable."""
from langbot.pkg.vector.vdbs.valkey_search import (
ValkeySearchVectorDatabase,
VALKEY_SEARCH_AVAILABLE,
)
from glide import ft
if not VALKEY_SEARCH_AVAILABLE:
pytest.skip('valkey-glide not installed')
ap = _make_ap(valkey_config)
db = ValkeySearchVectorDatabase(ap)
client = await db._ensure_client()
# Module-presence gate: FT.LIST must be available (Search module loaded).
try:
await ft.list(client)
except Exception as exc: # noqa: BLE001
await client.close()
pytest.skip(f'Valkey Search module not available: {exc}')
collection = f'test_{uuid.uuid4().hex[:12]}'
yield db, collection
# Cleanup
try:
await db.delete_collection(collection)
except Exception:
pass
if db._client is not None:
await db._client.close()
async def _poll_until(coro_factory, predicate, timeout=5.0, interval=0.2):
"""Poll an async result until predicate is true (indexer is async)."""
deadline = asyncio.get_event_loop().time() + timeout
result = await coro_factory()
while not predicate(result) and asyncio.get_event_loop().time() < deadline:
await asyncio.sleep(interval)
result = await coro_factory()
return result
def _sample_docs():
ids = ['d1', 'd2', 'd3']
embeddings = [
[1.0, 0.0, 0.0, 0.0],
[0.0, 1.0, 0.0, 0.0],
[0.9, 0.1, 0.0, 0.0],
]
metadatas = [
{'file_id': 'fileA', 'topic': 'cats'},
{'file_id': 'fileB', 'topic': 'dogs'},
{'file_id': 'fileA', 'topic': 'cats'},
]
documents = [
'the quick brown fox',
'lazy dogs sleeping',
'foxes and cats playing',
]
return ids, embeddings, metadatas, documents
@pytest.mark.asyncio
async def test_add_and_vector_search(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
result = await _poll_until(
lambda: db.search(collection, [1.0, 0.0, 0.0, 0.0], k=3, search_type='vector'),
lambda r: len(r['ids'][0]) >= 1,
)
assert len(result['ids'][0]) >= 1
# Closest to [1,0,0,0] should be d1.
assert result['ids'][0][0] == 'd1'
assert all(isinstance(d, float) for d in result['distances'][0])
@pytest.mark.asyncio
async def test_full_text_search(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
result = await _poll_until(
lambda: db.search(collection, [0.0, 0.0, 0.0, 0.0], k=5, search_type='full_text', query_text='dogs'),
lambda r: len(r['ids'][0]) >= 1,
)
assert 'd2' in result['ids'][0]
@pytest.mark.asyncio
async def test_hybrid_filter_then_knn(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
result = await _poll_until(
lambda: db.search(
collection,
[1.0, 0.0, 0.0, 0.0],
k=5,
search_type='hybrid',
query_text='cats',
filter={'file_id': 'fileA'},
),
lambda r: len(r['ids'][0]) >= 1,
)
# Only fileA docs (d1, d3) should be candidates.
assert set(result['ids'][0]).issubset({'d1', 'd3'})
@pytest.mark.asyncio
async def test_vector_weight_not_honored(backend):
"""Passing different vector_weight values must NOT change ranking."""
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
common = dict(
collection=collection, query_embedding=[1.0, 0.0, 0.0, 0.0], k=3, search_type='hybrid', query_text='cats'
)
await _poll_until(lambda: db.search(**common), lambda r: len(r['ids'][0]) >= 1)
r_low = await db.search(**common, vector_weight=0.1)
r_high = await db.search(**common, vector_weight=0.9)
assert r_low['ids'][0] == r_high['ids'][0]
@pytest.mark.asyncio
async def test_filter_operators(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
# Wait for indexing.
await _poll_until(
lambda: db.list_by_filter(collection, limit=10),
lambda r: r[1] >= 3,
)
# $eq
items, total = await db.list_by_filter(collection, filter={'file_id': 'fileA'})
assert total == 2
assert {it['id'] for it in items} == {'d1', 'd3'}
# $ne
items, total = await db.list_by_filter(collection, filter={'file_id': {'$ne': 'fileA'}})
assert {it['id'] for it in items} == {'d2'}
# $in
items, total = await db.list_by_filter(collection, filter={'file_id': {'$in': ['fileA', 'fileB']}})
assert total == 3
# $nin
items, total = await db.list_by_filter(collection, filter={'file_id': {'$nin': ['fileB']}})
assert {it['id'] for it in items} == {'d1', 'd3'}
@pytest.mark.asyncio
async def test_delete_by_file_id(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
await _poll_until(lambda: db.list_by_filter(collection, limit=10), lambda r: r[1] >= 3)
await db.delete_by_file_id(collection, 'fileA')
items, total = await _poll_until(
lambda: db.list_by_filter(collection, limit=10),
lambda r: r[1] <= 1,
)
assert {it['id'] for it in items} == {'d2'}
@pytest.mark.asyncio
async def test_delete_by_filter_returns_count(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
await _poll_until(lambda: db.list_by_filter(collection, limit=10), lambda r: r[1] >= 3)
deleted = await db.delete_by_filter(collection, filter={'file_id': 'fileA'})
assert deleted == 2
@pytest.mark.asyncio
async def test_list_by_filter_pagination(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
await _poll_until(lambda: db.list_by_filter(collection, limit=10), lambda r: r[1] >= 3)
page1, total = await db.list_by_filter(collection, limit=2, offset=0)
assert total == 3
assert len(page1) == 2
page2, total = await db.list_by_filter(collection, limit=2, offset=2)
assert total == 3
assert len(page2) == 1
@pytest.mark.asyncio
async def test_delete_collection(backend):
db, collection = backend
ids, embeddings, metadatas, documents = _sample_docs()
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
await _poll_until(lambda: db.list_by_filter(collection, limit=10), lambda r: r[1] >= 3)
await db.delete_collection(collection)
# After dropping, search on a missing index returns empty.
result = await db.search(collection, [1.0, 0.0, 0.0, 0.0], k=3, search_type='vector')
assert result['ids'][0] == []
@pytest.mark.asyncio
async def test_adversarial_filter_and_query_input(backend):
"""Crafted FT special chars in file_id / query_text must not break out.
Guarantees locked in here:
* A file_id full of injection-style chars (quotes, parens, ``|``, ``@``,
``:``, spaces, dashes) only ever matches its own row the payload is
escaped to literal TAG content, never interpreted as extra clauses.
* A query_text full of FT operators does not raise and does not widen the
result set.
* A file_id containing FT-unsafe chars (``{`` / ``}`` / ``*``) is
percent-encoded, so it round-trips correctly: an exact match returns ONLY
its own row and never widens to an unrelated row, and the query does not
raise.
"""
db, collection = backend
# Injection-style file_id WITHOUT FT-unsafe chars (the realistic surface).
injection_fid = 'evil") @file_id (".id|x-y:z'
# file_id WITH FT-unsafe chars that previously could not be queried.
brace_fid = 'x} @file_id:{*'
ids = ['adv1', 'benign2', 'brace3']
embeddings = [[1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 1.0, 0.0]]
metadatas = [{'file_id': injection_fid}, {'file_id': 'plainB'}, {'file_id': brace_fid}]
documents = ['payload row content', 'unrelated benign content', 'brace row content']
await db.add_embeddings(collection, ids, embeddings, metadatas, documents)
await _poll_until(lambda: db.list_by_filter(collection, limit=10), lambda r: r[1] >= 3)
# Exact-match on the crafted file_id returns ONLY its own row.
items, total = await db.list_by_filter(collection, filter={'file_id': injection_fid})
assert total == 1
assert {it['id'] for it in items} == {'adv1'}
# A query_text packed with FT operators must not raise and must not match
# the benign row (escaped to literal terms, none of which it contains).
result = await db.search(
collection,
[0.0, 0.0, 0.0, 0.0],
k=5,
search_type='full_text',
query_text='@document:{*} | -()~ "evil"',
)
assert 'benign2' not in result['ids'][0]
# The brace/star-bearing file_id is encoded, so it round-trips: exact match
# returns ONLY its own row and never widens. No RequestError is raised.
b_items, b_total = await db.list_by_filter(collection, filter={'file_id': brace_fid})
assert b_total == 1
assert {it['id'] for it in b_items} == {'brace3'}
# And deletion by that file_id removes exactly its own row.
deleted = await db.delete_by_filter(collection, filter={'file_id': brace_fid})
assert deleted == 1
+1 -1
View File
@@ -27,7 +27,7 @@
### 4. 向量数据库 (`vector/vdbs/`)
- **路径**: `src/langbot/pkg/vector/vdbs/`
- **模块**: chroma, milvus, pgvector, qdrant, seekdb
- **模块**: chroma, milvus, pgvector, qdrant, seekdb, valkey_search
- **排除原因**: 需要真实向量数据库实例运行
- **测试方式**: 需要 Docker 启动测试数据库或 mock
- **状态**: 后续可补充 mock 测试
@@ -280,6 +280,25 @@ class TestMCPServiceCreateMCPServer:
assert server_uuid is not None
assert len(server_uuid) == 36 # UUID format
async def test_create_mcp_server_duplicate_name_raises(self):
"""Rejects duplicate MCP server names."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.instance_config = SimpleNamespace()
ap.instance_config.data = {'system': {'limitation': {'max_extensions': -1}}}
ap.tool_mgr = None
existing_server = _create_mock_mcp_server(name='Existing Server')
ap.persistence_mgr.execute_async = AsyncMock(return_value=_create_mock_result(first_item=existing_server))
ap.persistence_mgr.serialize_model = Mock(return_value={})
service = MCPService(ap)
# Execute & Verify
with pytest.raises(ValueError, match='MCP server already exists: Existing Server'):
await service.create_mcp_server({'name': 'Existing Server'})
async def test_create_mcp_server_loads_server(self):
"""Loads server into tool_mgr when enabled."""
# Setup
@@ -301,7 +320,7 @@ class TestMCPServiceCreateMCPServer:
nonlocal call_count
call_count += 1
if call_count == 1:
return _create_mock_result([]) # Empty list for limit check
return _create_mock_result([]) # Empty result for duplicate-name check
elif call_count == 2:
return Mock() # Insert
return _create_mock_result(first_item=server_entity) # Select created
@@ -0,0 +1,76 @@
from __future__ import annotations
import sys
import types
from importlib import import_module
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
import quart
core_app_module = types.ModuleType('langbot.pkg.core.app')
core_app_module.Application = object
sys.modules.setdefault('langbot.pkg.core.app', core_app_module)
pytestmark = pytest.mark.asyncio
async def _create_test_client(mcp_service: SimpleNamespace):
app = quart.Quart(__name__)
user_service = SimpleNamespace(
verify_jwt_token=AsyncMock(return_value='test@example.com'),
get_user_by_email=AsyncMock(return_value=SimpleNamespace(user='test@example.com')),
)
ap = SimpleNamespace(mcp_service=mcp_service, user_service=user_service)
MCPRouterGroup = import_module('langbot.pkg.api.http.controller.groups.resources.mcp').MCPRouterGroup
group = MCPRouterGroup(ap, app)
await group.initialize()
return app.test_client()
async def test_mcp_server_route_accepts_encoded_slash_name():
mcp_service = SimpleNamespace(
get_mcp_server_by_name=AsyncMock(
return_value={
'uuid': 'test-uuid',
'name': 'pab1it0/prometheus',
'enable': True,
'mode': 'stdio',
'extra_args': {},
}
)
)
client = await _create_test_client(mcp_service)
response = await client.get(
'/api/v1/mcp/servers/pab1it0%2Fprometheus',
headers={'Authorization': 'Bearer test-token'},
)
assert response.status_code == 200
mcp_service.get_mcp_server_by_name.assert_awaited_once_with('pab1it0/prometheus')
payload = await response.get_json()
assert payload['data']['server']['name'] == 'pab1it0/prometheus'
async def test_mcp_resource_route_accepts_encoded_slash_name():
mcp_service = SimpleNamespace(
get_mcp_server_by_name=AsyncMock(),
get_mcp_server_resources=AsyncMock(return_value=[]),
get_mcp_server_resource_templates=AsyncMock(return_value=[]),
get_runtime_info=AsyncMock(return_value={'resource_capabilities': {'subscribe': False}}),
)
client = await _create_test_client(mcp_service)
response = await client.get(
'/api/v1/mcp/servers/pab1it0%2Fprometheus/resources',
headers={'Authorization': 'Bearer test-token'},
)
assert response.status_code == 200
mcp_service.get_mcp_server_by_name.assert_not_awaited()
mcp_service.get_mcp_server_resources.assert_awaited_once_with('pab1it0/prometheus')
payload = await response.get_json()
assert payload['data']['resource_capabilities'] == {'subscribe': False}
@@ -0,0 +1,91 @@
from types import SimpleNamespace
from unittest.mock import AsyncMock
import pytest
import langbot_plugin.api.definition.abstract.platform.event_logger as abstract_platform_logger
import langbot_plugin.api.entities.builtin.platform.message as platform_message
from langbot.pkg.platform.sources.wecomcs import WecomCSAdapter
class DummyLogger(abstract_platform_logger.AbstractEventLogger):
async def info(self, *args, **kwargs):
pass
async def debug(self, *args, **kwargs):
pass
async def warning(self, *args, **kwargs):
pass
async def error(self, *args, **kwargs):
pass
def make_adapter():
return WecomCSAdapter(
config={
'corpid': 'corp-id',
'secret': 'secret',
'token': 'token',
'EncodingAESKey': 'encoding-key',
},
logger=DummyLogger(),
)
@pytest.mark.asyncio
async def test_send_message_sends_text_to_customer_service_user():
adapter = make_adapter()
adapter.bot_account_id = 'kf-test'
adapter.bot = SimpleNamespace(send_text_msg=AsyncMock())
message = platform_message.MessageChain([platform_message.Plain(text='hello')])
await adapter.send_message('person', 'uexternal-user', message)
adapter.bot.send_text_msg.assert_awaited_once()
kwargs = adapter.bot.send_text_msg.await_args.kwargs
assert kwargs['open_kfid'] == 'kf-test'
assert kwargs['external_userid'] == 'external-user'
assert kwargs['content'] == 'hello'
assert kwargs['msgid'].startswith('langbot_')
@pytest.mark.asyncio
async def test_send_message_allows_explicit_open_kfid_in_target_id():
adapter = make_adapter()
adapter.bot = SimpleNamespace(send_text_msg=AsyncMock())
message = platform_message.MessageChain([platform_message.Plain(text='hello')])
await adapter.send_message('person', 'kf-explicit|uexternal-user', message)
kwargs = adapter.bot.send_text_msg.await_args.kwargs
assert kwargs['open_kfid'] == 'kf-explicit'
assert kwargs['external_userid'] == 'external-user'
@pytest.mark.asyncio
async def test_send_message_requires_open_kfid():
adapter = make_adapter()
adapter.bot = SimpleNamespace(send_text_msg=AsyncMock())
message = platform_message.MessageChain([platform_message.Plain(text='hello')])
with pytest.raises(ValueError, match='open_kfid is required'):
await adapter.send_message('person', 'uexternal-user', message)
adapter.bot.send_text_msg.assert_not_called()
@pytest.mark.asyncio
async def test_send_message_rejects_group_targets():
adapter = make_adapter()
adapter.bot_account_id = 'kf-test'
adapter.bot = SimpleNamespace(send_text_msg=AsyncMock())
message = platform_message.MessageChain([platform_message.Plain(text='hello')])
with pytest.raises(ValueError, match='only supports sending messages to person'):
await adapter.send_message('group', 'group-id', message)
adapter.bot.send_text_msg.assert_not_called()
@@ -417,7 +417,7 @@ class TestBuildBoxSessionPayload:
payload = s._build_box_session_payload('session-123')
assert payload['image'] == 'node:20'
assert payload['cpus'] == 2.0
assert payload['memory_mb'] == 1024
assert payload["memory_mb"] == 1024
assert payload['pids_limit'] == 256
def test_none_fields_excluded(self, mcp_module):
@@ -639,10 +639,13 @@ class TestGetRuntimeInfoDict:
assert info['box_session_id'] == 'mcp-shared'
assert info['box_enabled'] is True
def test_transient_test_session_is_isolated_from_shared(self, mcp_module):
"""A transient test session (config-page "test", no persisted UUID)
must NOT share the live "mcp-shared" Box session. Regression: a failing
test churned the shared session and tore down healthy live servers."""
def test_transient_test_shares_session_but_isolated_by_process(self, mcp_module):
"""A transient config-page "test" now shares the same 'mcp-shared' Box
session as live servers (so a test reuses the running container / live
process instead of a cold per-test session bootstrap). Isolation is at
the PROCESS level: the test runs under its own process_id and only ever
stops that process_id, so it cannot disturb another server's live
process or the shared session itself."""
ap = _make_ap()
ap.box_service.available = True
transient = _make_session(
@@ -670,10 +673,12 @@ class TestGetRuntimeInfoDict:
)
assert transient.is_transient is True
assert live.is_transient is False
# Isolated session id for the test, shared for the live server.
assert transient._build_box_session_id() == 'mcp-test-gen-uuid-123'
# Both share ONE Box session ...
assert transient._build_box_session_id() == 'mcp-shared'
assert live._build_box_session_id() == 'mcp-shared'
assert transient._build_box_session_id() != live._build_box_session_id()
assert transient._build_box_session_id() == live._build_box_session_id()
# ... but are isolated by distinct process_ids within that session.
assert transient._box_stdio_runtime.process_id != live._box_stdio_runtime.process_id
def test_stdio_session_refuses_when_box_unavailable(self, mcp_module):
"""Policy: when Box is configured but unavailable (disabled in config
@@ -824,3 +829,129 @@ async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_mo
assert process_payload['command'] == 'python'
assert process_payload['args'] == ['/workspace/.mcp/u1/workspace/server.py']
assert process_payload['cwd'] == '/workspace/.mcp/u1/workspace'
@pytest.mark.asyncio
async def test_stdio_handshake_raises_coldstart_retry_while_process_alive(mcp_module, tmp_path, monkeypatch):
"""During a slow (npx) cold start the handshake fails while the managed
process is still alive. initialize() must raise _ColdStartRetry (so the
outer lifecycle loop reuses the live process and retries without stopping it
or consuming the fatal budget), NOT a fatal error."""
from contextlib import asynccontextmanager
mcp_stdio_module = sys.modules['langbot.pkg.provider.tools.loaders.mcp_stdio']
class ColdClientSession:
def __init__(self, *_args):
pass
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def initialize(self):
# Process still cold-starting: handshake fails.
raise Exception('Connection closed')
@asynccontextmanager
async def fake_websocket_client(_url: str):
yield ('read-stream', 'write-stream')
monkeypatch.setattr(mcp_stdio_module, 'ClientSession', ColdClientSession)
monkeypatch.setattr(mcp_stdio_module, 'websocket_client', fake_websocket_client)
monkeypatch.setattr(mcp_stdio_module, '_HANDSHAKE_ATTEMPT_TIMEOUT_SEC', 1.0, raising=False)
ap = _make_ap()
ap.box_service.available = True
ap.box_service.create_session = AsyncMock(return_value={})
ap.box_service.start_managed_process = AsyncMock(return_value={})
ap.box_service.get_managed_process_websocket_url = Mock(return_value='ws://box/p')
session = _make_session(
mcp_module,
{
'name': 'slow',
'uuid': 'slow-uuid',
'mode': 'stdio',
'command': 'npx',
'args': ['-y', 'some-mcp'],
},
ap=ap,
)
# Process is NOT exited (still cold-starting) and not yet running for reuse.
async def _not_exited():
return False
session._box_stdio_runtime._managed_process_has_exited = _not_exited
async def _not_running():
return False
session._box_stdio_runtime._managed_process_is_running = _not_running
with pytest.raises(mcp_stdio_module._ColdStartRetry):
await session._init_box_stdio_server()
# Process was started exactly once (the retry will reuse it, not rebuild).
assert ap.box_service.start_managed_process.await_count == 1
await session.exit_stack.aclose()
@pytest.mark.asyncio
async def test_stdio_handshake_raises_fatal_when_process_exited(mcp_module, tmp_path, monkeypatch):
"""If the handshake fails AND the process has definitively exited, that is a
real failure initialize() must NOT swallow it as a cold-start retry."""
from contextlib import asynccontextmanager
mcp_stdio_module = sys.modules['langbot.pkg.provider.tools.loaders.mcp_stdio']
class DeadClientSession:
def __init__(self, *_args):
pass
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def initialize(self):
raise Exception('Connection closed')
@asynccontextmanager
async def fake_websocket_client(_url: str):
yield ('read-stream', 'write-stream')
monkeypatch.setattr(mcp_stdio_module, 'ClientSession', DeadClientSession)
monkeypatch.setattr(mcp_stdio_module, 'websocket_client', fake_websocket_client)
monkeypatch.setattr(mcp_stdio_module, '_HANDSHAKE_ATTEMPT_TIMEOUT_SEC', 1.0, raising=False)
ap = _make_ap()
ap.box_service.available = True
ap.box_service.create_session = AsyncMock(return_value={})
ap.box_service.start_managed_process = AsyncMock(return_value={})
ap.box_service.get_managed_process_websocket_url = Mock(return_value='ws://box/p')
session = _make_session(
mcp_module,
{'name': 'dead', 'uuid': 'dead-uuid', 'mode': 'stdio', 'command': 'npx', 'args': ['-y', 'x']},
ap=ap,
)
async def _exited():
return True
session._box_stdio_runtime._managed_process_has_exited = _exited
async def _not_running():
return False
session._box_stdio_runtime._managed_process_is_running = _not_running
with pytest.raises(Exception) as ei:
await session._init_box_stdio_server()
assert not isinstance(ei.value, mcp_stdio_module._ColdStartRetry)
await session.exit_stack.aclose()
+20 -1
View File
@@ -33,7 +33,7 @@ class TestVectorDBManagerInitialization:
mocks['langbot.pkg.core.app'] = MagicMock()
# Mock all VDB backend implementations
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db']:
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db', 'valkey_search']:
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
return mocks
@@ -123,6 +123,25 @@ class TestVectorDBManagerInitialization:
mock_seekdb_class.assert_called_once_with(mock_app)
def test_initialize_valkey_search_backend(self):
"""Valkey Search config uses ValkeySearchVectorDatabase backend."""
vdb_config = {'use': 'valkey_search'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_valkey_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.valkey_search'].ValkeySearchVectorDatabase = mock_valkey_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_valkey_class.assert_called_once_with(mock_app)
def test_initialize_milvus_backend_with_uri(self):
"""Milvus config with custom URI."""
vdb_config = {
@@ -0,0 +1,388 @@
"""Unit tests for the Valkey Search VDB backend's pure helpers.
These tests exercise the filter-to-FT mapping, float32 packing, tag/text
escaping, FT.SEARCH reply parsing and the import guard. They run in the fast
CI lane and require NO running Valkey server.
"""
from __future__ import annotations
import asyncio
import struct
from importlib import import_module
from unittest.mock import AsyncMock
import pytest
def get_valkey_module():
"""Lazy import of the valkey_search backend module."""
return import_module('langbot.pkg.vector.vdbs.valkey_search')
def make_backend():
"""Construct a backend instance without running its __init__.
The constructor needs a live ``ap`` + config; for pure-helper tests we
only need a bare instance with the attributes the helpers touch.
"""
mod = get_valkey_module()
backend = object.__new__(mod.ValkeySearchVectorDatabase)
# _ensure_client serializes creation through this lock; set it here since
# __init__ (which normally creates it) is bypassed.
backend._client_lock = asyncio.Lock()
return backend
class TestFloat32Packing:
"""Tests for _pack_vector little-endian float32 packing."""
def test_pack_round_trips(self):
mod = get_valkey_module()
vec = [0.1, -2.5, 3.0, 4.25]
packed = mod.ValkeySearchVectorDatabase._pack_vector(vec)
assert isinstance(packed, bytes)
assert len(packed) == 4 * len(vec)
unpacked = list(struct.unpack(f'<{len(vec)}f', packed))
for original, restored in zip(vec, unpacked):
assert restored == pytest.approx(original, rel=1e-6)
def test_pack_is_little_endian(self):
mod = get_valkey_module()
packed = mod.ValkeySearchVectorDatabase._pack_vector([1.0])
assert packed == struct.pack('<f', 1.0)
class TestTagEscaping:
"""Tests for _escape_tag."""
def test_escapes_special_chars(self):
mod = get_valkey_module()
escaped = mod.ValkeySearchVectorDatabase._escape_tag('a-b c.d')
assert '\\-' in escaped
assert '\\ ' in escaped
assert '\\.' in escaped
def test_plain_value_unchanged(self):
mod = get_valkey_module()
assert mod.ValkeySearchVectorDatabase._escape_tag('abc123') == 'abc123'
class TestFileIdEncoding:
"""Tests for _encode_file_id (FT-unsafe char percent-encoding)."""
def test_uuid_is_noop(self):
mod = get_valkey_module()
fid = '550e8400-e29b-41d4-a716-446655440000'
assert mod.ValkeySearchVectorDatabase._encode_file_id(fid) == fid
def test_encodes_braces_star_and_percent(self):
mod = get_valkey_module()
enc = mod.ValkeySearchVectorDatabase._encode_file_id('a{b}c*d%e')
# '{'=7B '}'=7D '*'=2A '%'=25
assert enc == 'a%7Bb%7Dc%2Ad%25e'
# No raw FT-unsafe char survives.
assert all(ch not in enc for ch in '{}*') or '%' in enc
def test_encoding_is_deterministic_and_collision_safe(self):
mod = get_valkey_module()
enc = mod.ValkeySearchVectorDatabase._encode_file_id
# A literal "%7B" must not collide with an encoded "{".
assert enc('{') != enc('%7B')
assert enc('{') == '%7B'
assert enc('%7B') == '%257B'
def test_filter_encodes_unsafe_chars_in_tag_query(self):
backend = make_backend()
# The emitted TAG query must contain the encoded form, never raw braces.
frag = backend._triples_to_ft({'file_id': 'x}y{z*'})
assert '7D' in frag and '7B' in frag and '2A' in frag
# No raw '*' from the value, and exactly one opening/closing brace (the
# TAG-clause delimiters) — the value's own braces were encoded away.
assert '*' not in frag
assert frag.count('{') == 1 and frag.count('}') == 1
assert frag.startswith('@file_id:{') and frag.endswith('}')
def test_filter_in_operator_encodes_each_value(self):
backend = make_backend()
frag = backend._triples_to_ft({'file_id': {'$in': ['a*b', 'c}d']}})
assert '2A' in frag and '7D' in frag
assert '*' not in frag
class TestFilterToFt:
"""Tests for _triples_to_ft filter mapping (all 8 operators)."""
def test_empty_filter_returns_empty_string(self):
backend = make_backend()
assert backend._triples_to_ft(None) == ''
assert backend._triples_to_ft({}) == ''
def test_eq_tag(self):
backend = make_backend()
assert backend._triples_to_ft({'file_id': 'abc'}) == '@file_id:{abc}'
def test_explicit_eq_tag(self):
backend = make_backend()
assert backend._triples_to_ft({'file_id': {'$eq': 'abc'}}) == '@file_id:{abc}'
def test_ne_tag(self):
backend = make_backend()
assert backend._triples_to_ft({'file_id': {'$ne': 'abc'}}) == '-@file_id:{abc}'
def test_in_tag(self):
backend = make_backend()
assert backend._triples_to_ft({'file_id': {'$in': ['a', 'b']}}) == '@file_id:{a|b}'
def test_nin_tag(self):
backend = make_backend()
assert backend._triples_to_ft({'file_id': {'$nin': ['a', 'b']}}) == '-@file_id:{a|b}'
def test_numeric_range_operators(self):
backend = make_backend()
# file_id is the only indexed field; numeric ops still render via the
# generic range fragment, so use file_id to keep the field supported.
# Values are cast to float (defensive against non-numeric input and a
# future NUMERIC field becoming an injection surface).
assert backend._triples_to_ft({'file_id': {'$gt': 5}}) == '@file_id:[(5.0 +inf]'
assert backend._triples_to_ft({'file_id': {'$gte': 5}}) == '@file_id:[5.0 +inf]'
assert backend._triples_to_ft({'file_id': {'$lt': 5}}) == '@file_id:[-inf (5.0]'
assert backend._triples_to_ft({'file_id': {'$lte': 5}}) == '@file_id:[-inf 5.0]'
def test_numeric_range_rejects_non_numeric(self):
backend = make_backend()
# A non-numeric range value fails closed rather than interpolating raw.
with pytest.raises((ValueError, TypeError)):
backend._triples_to_ft({'file_id': {'$gt': 'not-a-number'}})
def test_unsupported_field_dropped(self):
backend = make_backend()
# Non-indexed fields are dropped (returns empty expression).
assert backend._triples_to_ft({'some_other_field': 'x'}) == ''
def test_multiple_supported_keys_anded(self):
backend = make_backend()
# Two conditions on the same indexed field are joined with a space (AND).
result = backend._triples_to_ft({'file_id': {'$in': ['a', 'b']}})
assert result == '@file_id:{a|b}'
class TestTextEscaping:
"""Tests for _escape_text full-text escaping."""
def test_escapes_ft_special_chars(self):
mod = get_valkey_module()
escaped = mod.ValkeySearchVectorDatabase._escape_text('hello@world|test')
assert '\\@' in escaped
assert '\\|' in escaped
class TestReplyToChroma:
"""Tests for _reply_to_chroma FT.SEARCH reply parsing."""
def test_parses_knn_reply(self):
backend = make_backend()
# glide returns [total, {key: {field: value}}]
reply = [
2,
{
b'kb:col1:id1': {
b'distance': b'0.10',
b'document': b'hello',
b'metadata_json': b'{"file_id": "f1"}',
},
b'kb:col1:id2': {
b'distance': b'0.25',
b'document': b'world',
b'metadata_json': b'{"file_id": "f2"}',
},
},
]
result = backend._reply_to_chroma('idx:col1', reply, has_distance=True)
assert result['ids'][0] == ['id1', 'id2']
assert result['distances'][0] == [pytest.approx(0.10), pytest.approx(0.25)]
assert result['metadatas'][0][0] == {'file_id': 'f1'}
assert result['metadatas'][0][1] == {'file_id': 'f2'}
def test_empty_reply(self):
backend = make_backend()
result = backend._reply_to_chroma('idx:col1', [0, {}], has_distance=True)
assert result == {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
def test_malformed_reply(self):
backend = make_backend()
result = backend._reply_to_chroma('idx:col1', [], has_distance=True)
assert result == {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
def test_text_search_reply_no_distance(self):
backend = make_backend()
reply = [
1,
{
b'kb:col1:id1': {
b'document': b'hello',
b'metadata_json': b'{"file_id": "f1"}',
},
},
]
result = backend._reply_to_chroma('idx:col1', reply, has_distance=False)
assert result['ids'][0] == ['id1']
assert result['distances'][0] == [0.0]
class TestImportGuard:
"""Tests for the ImportError guard when glide is unavailable."""
def test_constructor_raises_when_unavailable(self, monkeypatch):
mod = get_valkey_module()
monkeypatch.setattr(mod, 'VALKEY_SEARCH_AVAILABLE', False)
with pytest.raises(ImportError, match='valkey-glide'):
mod.ValkeySearchVectorDatabase(ap=None)
class TestSupportedSearchTypes:
"""Tests for supported_search_types."""
def test_supports_vector_full_text_hybrid(self):
mod = get_valkey_module()
from langbot.pkg.vector.vdb import SearchType
types = mod.ValkeySearchVectorDatabase.supported_search_types()
assert SearchType.VECTOR in types
assert SearchType.FULL_TEXT in types
assert SearchType.HYBRID in types
class TestDeleteByFilterGuard:
"""Regression tests for the delete_by_filter mass-deletion guard.
A non-empty filter referencing only non-indexed fields must NOT fall back
to match-all and wipe the whole collection: it must skip and return 0.
"""
async def test_unsupported_only_filter_skips_and_returns_zero(self):
backend = make_backend()
# Make the client/index lookups succeed without a real server.
backend._client = AsyncMock()
backend.ap = type('Ap', (), {'logger': AsyncMock()})()
backend._ensure_client = AsyncMock(return_value=backend._client)
backend._index_exists = AsyncMock(return_value=True)
# _search_keys must never be reached for an unusable filter.
backend._search_keys = AsyncMock(
side_effect=AssertionError('_search_keys must not be called for an unusable filter')
)
# Filter references only a non-indexed field -> maps to no FT conditions.
deleted = await backend.delete_by_filter('col1', {'some_other_field': 'x'})
assert deleted == 0
backend._client.delete.assert_not_called()
async def test_supported_filter_deletes_matching_keys(self):
backend = make_backend()
backend._client = AsyncMock()
backend.ap = type('Ap', (), {'logger': AsyncMock()})()
backend._ensure_client = AsyncMock(return_value=backend._client)
backend._index_exists = AsyncMock(return_value=True)
backend._search_keys = AsyncMock(return_value=['kb:col1:id1', 'kb:col1:id2'])
deleted = await backend.delete_by_filter('col1', {'file_id': 'f1'})
assert deleted == 2
backend._client.delete.assert_awaited_once_with(['kb:col1:id1', 'kb:col1:id2'])
class TestClose:
"""Tests for the close() teardown."""
async def test_close_resets_client_and_indexes(self):
backend = make_backend()
client = AsyncMock()
backend._client = client
backend.ap = type('Ap', (), {'logger': AsyncMock()})()
backend._ensured_indexes = {'idx:col1'}
await backend.close()
client.close.assert_awaited_once()
assert backend._client is None
assert backend._ensured_indexes == set()
async def test_close_is_noop_when_no_client(self):
backend = make_backend()
backend._client = None
backend.ap = type('Ap', (), {'logger': AsyncMock()})()
backend._ensured_indexes = set()
# Should not raise.
await backend.close()
assert backend._client is None
class TestCredentialsBuild:
"""Tests for the auth-credential construction in _ensure_client."""
def _prep_backend(self, mod, monkeypatch, *, username, password):
backend = make_backend()
backend._client = None
backend._host = 'localhost'
backend._port = 6379
backend._db = 0
backend._tls = False
backend._username = username
backend._password = password
backend._request_timeout = 5000
backend._ensured_indexes = set()
warnings: list[str] = []
backend.ap = type(
'Ap',
(),
{
'logger': type(
'L', (), {'info': lambda self, *a, **k: None, 'warning': lambda s, m, *a, **k: warnings.append(m)}
)()
},
)()
created = {}
class _FakeClient:
@staticmethod
async def create(conf):
created['conf'] = conf
return AsyncMock()
cred_calls: list[dict] = []
def _fake_credentials(**kwargs):
cred_calls.append(kwargs)
return ('CRED', kwargs)
monkeypatch.setattr(mod, 'GlideClient', _FakeClient)
monkeypatch.setattr(mod, 'ServerCredentials', _fake_credentials)
monkeypatch.setattr(mod, 'GlideClientConfiguration', lambda **kw: kw)
monkeypatch.setattr(mod, 'NodeAddress', lambda *a, **k: ('node', a, k))
return backend, created, cred_calls, warnings
async def test_username_without_password_fails_closed(self, monkeypatch):
mod = get_valkey_module()
backend, created, cred_calls, warnings = self._prep_backend(mod, monkeypatch, username='acluser', password=None)
# A username without a password must fail closed rather than silently
# connecting unauthenticated to a (potentially shared) Valkey instance.
with pytest.raises(ValueError, match='without a password'):
await backend._ensure_client()
assert cred_calls == [] # ServerCredentials NOT constructed
assert 'conf' not in created # client never created
async def test_password_builds_credentials(self, monkeypatch):
mod = get_valkey_module()
backend, created, cred_calls, warnings = self._prep_backend(
mod, monkeypatch, username='acluser', password='secret'
)
await backend._ensure_client()
assert len(cred_calls) == 1
assert cred_calls[0] == {'password': 'secret', 'username': 'acluser'}
assert created['conf']['credentials'] == ('CRED', {'password': 'secret', 'username': 'acluser'})
Generated
+40 -5
View File
@@ -2008,7 +2008,7 @@ wheels = [
[[package]]
name = "langbot"
version = "4.10.4"
version = "4.10.5"
source = { editable = "." }
dependencies = [
{ name = "aiocqhttp" },
@@ -2084,6 +2084,7 @@ dependencies = [
{ name = "tiktoken" },
{ name = "urllib3" },
{ name = "uv" },
{ name = "valkey-glide" },
{ name = "websockets" },
]
@@ -2123,7 +2124,7 @@ requires-dist = [
{ name = "ebooklib", specifier = ">=0.18" },
{ name = "gewechat-client", specifier = ">=0.1.5" },
{ name = "html2text", specifier = ">=2024.2.26" },
{ name = "langbot-plugin", specifier = "==0.4.6" },
{ name = "langbot-plugin", specifier = "==0.4.13" },
{ name = "langchain", specifier = ">=1.3.9" },
{ name = "langchain-core", specifier = ">=1.3.3" },
{ name = "langchain-text-splitters", specifier = ">=1.1.2" },
@@ -2172,6 +2173,7 @@ requires-dist = [
{ name = "tiktoken", specifier = ">=0.9.0" },
{ name = "urllib3", specifier = ">=2.7.0" },
{ name = "uv", specifier = ">=0.11.15" },
{ name = "valkey-glide", specifier = ">=2.4.1,<3.0.0" },
{ name = "websockets", specifier = ">=15.0.1" },
]
@@ -2187,7 +2189,7 @@ dev = [
[[package]]
name = "langbot-plugin"
version = "0.4.6"
version = "0.4.13"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiofiles" },
@@ -2208,9 +2210,9 @@ dependencies = [
{ name = "watchdog" },
{ name = "websockets" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b4/6a/5fdb5365ad04aaa61344e92578d73eb1577af35783b80767c7d6c51cb8b9/langbot_plugin-0.4.6.tar.gz", hash = "sha256:838e3cd45ed795ed4c3299c73f141b217adfa05f09937a01694e7158619e4f6e", size = 334171, upload-time = "2026-06-22T15:06:56.565Z" }
sdist = { url = "https://files.pythonhosted.org/packages/40/a6/1eaf77c3b81e9de3390c504c5f627dc41f43bff6df9aff0e1e31d796b6f0/langbot_plugin-0.4.13.tar.gz", hash = "sha256:f936340e67679c21f1e7e7f1447339f31a0a2c965db060ecfbd9d0c51bb0d6fe", size = 334887, upload-time = "2026-07-04T05:38:59.942Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6b/55/7adc2e180a299ed58613e159c64195477e09c05136f949942c6cec5219e8/langbot_plugin-0.4.6-py3-none-any.whl", hash = "sha256:30eb47efc0b703818ac003a5cd67caf720d9749dd503155eb65cce0c28b194a7", size = 217434, upload-time = "2026-06-22T15:06:55.237Z" },
{ url = "https://files.pythonhosted.org/packages/e9/bf/fc9671a7afbd933440c38403c84d918c1022fdeed16e22a6ab3b2aec83ff/langbot_plugin-0.4.13-py3-none-any.whl", hash = "sha256:9d45ebc7a7ee0413d6db9baa009fcbf0ad07e2e1753a6f0a27f37b8b665cd1ee", size = 221884, upload-time = "2026-07-04T05:38:58.525Z" },
]
[[package]]
@@ -5984,6 +5986,39 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e4/16/c1fd27e9549f3c4baf1dc9c20c456cd2f822dbf8de9f463824b0c0357e06/uvloop-0.22.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6cde23eeda1a25c75b2e07d39970f3374105d5eafbaab2a4482be82f272d5a5e", size = 4296730, upload-time = "2025-10-16T22:17:00.744Z" },
]
[[package]]
name = "valkey-glide"
version = "2.4.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "anyio" },
{ name = "protobuf" },
{ name = "sniffio" },
]
sdist = { url = "https://files.pythonhosted.org/packages/72/a2/582b34c6acc8dc857c537f6007459cba48dfa0dc404789a657e5c1a998c0/valkey_glide-2.4.1.tar.gz", hash = "sha256:f1155d84156d11b90488aa67e90102f0bf98a45314f5b99308ac9074c05f7241", size = 898030, upload-time = "2026-05-28T21:41:55.881Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/60/60/961ce40492a56ef831a905dfe03df4a81c0705152f6a8e49c541c634f49e/valkey_glide-2.4.1-cp311-cp311-macosx_10_7_x86_64.whl", hash = "sha256:d7285d03c2df040f26874b7f4ae96f040da2daecc9a34fa99da6f4e6ce5149c8", size = 7482152, upload-time = "2026-05-28T21:41:02.205Z" },
{ url = "https://files.pythonhosted.org/packages/a4/b2/5a05567f0fc385dcbbbf6ab1061f0bc00443d51c2996e95eed45feaedda9/valkey_glide-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:5d2e82b74127897ccb7a957ad455787816a75fdc8c60a5e8004aef65ea93e99c", size = 6928601, upload-time = "2026-05-28T21:41:04.543Z" },
{ url = "https://files.pythonhosted.org/packages/c5/d9/7ea2b47cff0a2f99921eb0db404215f828ced7814bd09ede9c93b65d20bc/valkey_glide-2.4.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4094128cb07e06e87013b7afab1e9388f8f5aeebe48ea6cbd54de15bd772e644", size = 7236977, upload-time = "2026-05-28T21:41:06.055Z" },
{ url = "https://files.pythonhosted.org/packages/00/7a/6cda6b42156ed260e765e4ad2d6ab831607775e218a00fbb0d93411c4e8f/valkey_glide-2.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9f8dc0f3a36adb1cbe4e167972ca4758acdfed6baf58a4db94bbb713df56c8f5", size = 7691446, upload-time = "2026-05-28T21:41:07.833Z" },
{ url = "https://files.pythonhosted.org/packages/c5/b4/da8c058baaee414a6bb2450742359f3b3b6993b23281bf227c5089f0099c/valkey_glide-2.4.1-cp312-cp312-macosx_10_7_x86_64.whl", hash = "sha256:5f8df64f6a4f0fd7203113103101fdf0aaa7ff0e7557312611de11ab89c6db75", size = 7472646, upload-time = "2026-05-28T21:41:09.451Z" },
{ url = "https://files.pythonhosted.org/packages/f5/94/e1e311cb56597272b9cb69afb3fe8e2e7dd3371f88c92836015deddc6f49/valkey_glide-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b45e35f44c17e88f8cd8082f8d8061a9763238c44ef20b11b615f6d87235864a", size = 6943375, upload-time = "2026-05-28T21:41:11.079Z" },
{ url = "https://files.pythonhosted.org/packages/76/00/0e42e2f6866ebf0de552e076dc585a487b488b5b818c52460d28b50de65b/valkey_glide-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cf812b498925a30abab6e1a9f82f5eb821e967904fe7724729b2c82c47e29edf", size = 7237469, upload-time = "2026-05-28T21:41:12.733Z" },
{ url = "https://files.pythonhosted.org/packages/f5/4c/c5dd9a1ed995453b0d9ca75a5af87e881c14e6eebdbf5a5fa78c3bae23fc/valkey_glide-2.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:214e2faca98966eea3eaf9e09de616862423815a5059843a9884125e2427a344", size = 7678744, upload-time = "2026-05-28T21:41:14.634Z" },
{ url = "https://files.pythonhosted.org/packages/6a/2f/3df5702fc68684cef3e09f9cb6ed85578ddb08dc43593b1694c977f396fa/valkey_glide-2.4.1-cp313-cp313-macosx_10_7_x86_64.whl", hash = "sha256:c18976553ba663c03f7cc18c7e6075f4cbd2236c18b051e3d55bb213c6c44cb4", size = 7472972, upload-time = "2026-05-28T21:41:16.063Z" },
{ url = "https://files.pythonhosted.org/packages/54/a3/6a74c6f996fa9e411e66b6f0e645fead2e0a341f1371e4cf3212efa54412/valkey_glide-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:43006e19cd63d66051263fa34a8ad47ba7d08a199585689b3f12f56ed6c9a005", size = 6943012, upload-time = "2026-05-28T21:41:17.492Z" },
{ url = "https://files.pythonhosted.org/packages/fc/e7/d10ec41dca703f8c5dcbcba2b905e660c1cf56be53c4d5e368d7aa23d220/valkey_glide-2.4.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b652a2a62aad87738e8f0e0aa5bf660ba91449c9fdb88550ccbc42e5fec08fe7", size = 7237842, upload-time = "2026-05-28T21:41:18.995Z" },
{ url = "https://files.pythonhosted.org/packages/0a/a3/8916a9ed9e871686db444c86e601773245852ba1ad451ce1bb06f7aed91d/valkey_glide-2.4.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fbd27d26947fd9f1b6e9eaf0abce4bccfde779c1e618b310c4d725424b609793", size = 7678919, upload-time = "2026-05-28T21:41:20.502Z" },
{ url = "https://files.pythonhosted.org/packages/05/35/6d39ec3cbd24d85ad8e1051e29e6509c0999f760aff5af7851c1a1981471/valkey_glide-2.4.1-cp314-cp314-macosx_10_7_x86_64.whl", hash = "sha256:91fb7ff97acdabc8f641255b548a48627bb731e65037b1126745bf8a0022e87d", size = 7471906, upload-time = "2026-05-28T21:41:22.135Z" },
{ url = "https://files.pythonhosted.org/packages/ab/fc/3c28f794b7d35e13101598669c1d249c0a9f0408c545c87212e364c6ee4e/valkey_glide-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:d49a2537c2de44b0fc57691b1ae6c3d6f481e6f7f7eb879c0d28921d0aaec67d", size = 6943495, upload-time = "2026-05-28T21:41:23.783Z" },
{ url = "https://files.pythonhosted.org/packages/2e/15/fb884631f5df78dc538c56bca9391165e40906b9b63ca65633d1be5bf980/valkey_glide-2.4.1-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cded9f14e448da5a96f61c066395f2c7e2846f2afe74cacc8634da0ae0c3425f", size = 7257720, upload-time = "2026-05-28T21:41:25.361Z" },
{ url = "https://files.pythonhosted.org/packages/73/79/0b881017194386d21812b929a81dd8afd51d6b8d92280895b45913854785/valkey_glide-2.4.1-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5f249ab5bd0d69befe35897cf51a8fc9e01e9c8c9fe03087a68e6fe6d3e31d0d", size = 7682318, upload-time = "2026-05-28T21:41:26.996Z" },
{ url = "https://files.pythonhosted.org/packages/7a/4d/f2b4e508692fcd21e76c7cbdc4f988bec7f4675e60f4f35ef482a826f6ae/valkey_glide-2.4.1-pp311-pypy311_pp73-macosx_10_7_x86_64.whl", hash = "sha256:775df9c7421a187c41caf003e4af5f073ed7e4b8abe50f8b9bec712cb03e12bf", size = 7479155, upload-time = "2026-05-28T21:41:42.399Z" },
{ url = "https://files.pythonhosted.org/packages/52/d8/8a3495f5582dccb4c8e7faf6a73baf3dbc4580701923f06d8abf210ff22d/valkey_glide-2.4.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:0d87f21c77004240189cc3c5aab156966487afd81ffdee04225a52c7bd7132e4", size = 6938571, upload-time = "2026-05-28T21:41:44.078Z" },
{ url = "https://files.pythonhosted.org/packages/f3/5a/a70077f76c2f18e94ec4309857b248beb7a8c7a3a50e30242abde2c3827d/valkey_glide-2.4.1-pp311-pypy311_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:44376ef5fe7a25287095b073d8abde510a50b1ead0143662394b3da9717863ef", size = 7260021, upload-time = "2026-05-28T21:41:45.837Z" },
{ url = "https://files.pythonhosted.org/packages/aa/12/72d31522e06fcc9b391118c1f69a09002224e78114b1db0d01b96008dc59/valkey_glide-2.4.1-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a59cc0a21d7a8b1b3caeb299f23817429b5fe6579bd4cb016382e6b7a10de984", size = 7693093, upload-time = "2026-05-28T21:41:47.617Z" },
]
[[package]]
name = "virtualenv"
version = "20.36.1"
+41 -39
View File
@@ -191,45 +191,47 @@ export default function BotDetailContent({ id }: { id: string }) {
onValueChange={setActiveTab}
className="flex flex-1 flex-col min-h-0"
>
<TabsList className="shrink-0">
<TabsTrigger value="config" className="gap-1.5">
<Settings className="size-3.5" />
{t('bots.configuration')}
</TabsTrigger>
<TabsTrigger value="logs" className="gap-1.5">
<FileText className="size-3.5" />
{t('bots.logs')}
</TabsTrigger>
<TabsTrigger value="sessions" className="gap-1.5">
<Users className="size-3.5" />
{t('bots.sessionMonitor.title')}
{activeTab === 'sessions' && (
<button
type="button"
className="inline-flex items-center justify-center ml-0.5"
onPointerDown={(e) => e.stopPropagation()}
onClick={(e) => {
e.stopPropagation();
e.preventDefault();
if (isRefreshingSessions) return;
setIsRefreshingSessions(true);
const minDelay = new Promise((r) => setTimeout(r, 500));
Promise.all([
sessionMonitorRef.current?.refreshSessions(),
minDelay,
]).finally(() => setIsRefreshingSessions(false));
}}
>
<RefreshCw
className={cn(
'size-3 text-muted-foreground hover:text-foreground transition-colors',
isRefreshingSessions && 'animate-spin',
)}
/>
</button>
)}
</TabsTrigger>
</TabsList>
<div className="flex shrink-0 items-center gap-1">
<TabsList>
<TabsTrigger value="config" className="gap-1.5">
<Settings className="size-3.5" />
{t('bots.configuration')}
</TabsTrigger>
<TabsTrigger value="logs" className="gap-1.5">
<FileText className="size-3.5" />
{t('bots.logs')}
</TabsTrigger>
<TabsTrigger value="sessions" className="gap-1.5">
<Users className="size-3.5" />
{t('bots.sessionMonitor.title')}
</TabsTrigger>
</TabsList>
{activeTab === 'sessions' && (
<button
type="button"
aria-label={t('bots.sessionMonitor.refresh')}
title={t('bots.sessionMonitor.refresh')}
className="inline-flex h-8 w-8 items-center justify-center rounded-md text-muted-foreground transition-colors hover:bg-accent hover:text-foreground disabled:pointer-events-none disabled:opacity-50"
disabled={isRefreshingSessions}
onClick={() => {
if (isRefreshingSessions) return;
setIsRefreshingSessions(true);
const minDelay = new Promise((r) => setTimeout(r, 500));
Promise.all([
sessionMonitorRef.current?.refreshSessions(),
minDelay,
]).finally(() => setIsRefreshingSessions(false));
}}
>
<RefreshCw
className={cn(
'size-3.5',
isRefreshingSessions && 'animate-spin',
)}
/>
</button>
)}
</div>
{/* Tab: Configuration */}
<TabsContent
@@ -3,6 +3,7 @@ import React, {
useEffect,
useRef,
useCallback,
useMemo,
forwardRef,
useImperativeHandle,
} from 'react';
@@ -15,11 +16,14 @@ import {
Bot,
Copy,
Check,
ChevronDown,
ChevronRight,
Workflow,
ThumbsUp,
ThumbsDown,
ShieldCheck,
ShieldOff,
Wrench,
} from 'lucide-react';
import { toast } from 'sonner';
import BotAdminsDialog, {
@@ -76,6 +80,35 @@ interface SessionFeedback {
stream_id?: string | null;
}
interface SessionToolCall {
id: string;
timestamp: string;
tool_name: string;
tool_source: string;
duration: number;
status: string;
message_id?: string | null;
arguments?: string | null;
result?: string | null;
error_message?: string | null;
}
type SessionTimelineItem =
| {
id: string;
type: 'message';
timestamp: number;
order: number;
message: SessionMessage;
}
| {
id: string;
type: 'tool';
timestamp: number;
order: number;
toolCall: SessionToolCall;
};
export interface BotSessionMonitorHandle {
refreshSessions: () => Promise<void>;
}
@@ -100,6 +133,10 @@ const BotSessionMonitor = forwardRef<
const [feedbackMap, setFeedbackMap] = useState<
Record<string, SessionFeedback>
>({});
const [toolCalls, setToolCalls] = useState<SessionToolCall[]>([]);
const [expandedToolCallIds, setExpandedToolCallIds] = useState<
Record<string, boolean>
>({});
const messagesContainerRef = useRef<HTMLDivElement>(null);
const { admins, reload: reloadAdmins } = useBotAdmins(botId);
const [adminsDialogOpen, setAdminsDialogOpen] = useState(false);
@@ -189,6 +226,7 @@ const BotSessionMonitor = forwardRef<
const loadMessages = useCallback(
async (sessionId: string) => {
setLoadingMessages(true);
setExpandedToolCallIds({});
try {
const messagesRes = await httpClient.getSessionMessages(sessionId);
const sorted = (messagesRes.messages ?? []).sort(
@@ -197,6 +235,18 @@ const BotSessionMonitor = forwardRef<
);
setMessages(sorted);
try {
const analysisRes = await httpClient.get<{
tool_calls?: SessionToolCall[];
}>(
`/api/v1/monitoring/sessions/${encodeURIComponent(sessionId)}/analysis`,
);
setToolCalls(analysisRes?.tool_calls ?? []);
} catch (analysisError) {
console.error('Failed to load session tool calls:', analysisError);
setToolCalls([]);
}
// Collect user message IDs for feedback matching
const userMsgIds = new Set(
sorted.filter((m) => !m.role || m.role === 'user').map((m) => m.id),
@@ -240,11 +290,14 @@ const BotSessionMonitor = forwardRef<
loadMessages(selectedSessionId);
} else {
setMessages([]);
setToolCalls([]);
setExpandedToolCallIds({});
setFeedbackMap({});
}
}, [selectedSessionId, loadMessages]);
useEffect(() => {
if (messages.length === 0) return;
if (messages.length === 0 && toolCalls.length === 0) return;
// Wait for DOM to render the new messages before scrolling
requestAnimationFrame(() => {
const container = messagesContainerRef.current;
@@ -256,7 +309,7 @@ const BotSessionMonitor = forwardRef<
scrollTarget.scrollTop = scrollTarget.scrollHeight;
}
});
}, [messages]);
}, [messages, toolCalls]);
const parseMessageChain = (content: string): MessageChainComponent[] => {
try {
@@ -431,6 +484,71 @@ const BotSessionMonitor = forwardRef<
return `${diffDays}d`;
};
const formatDuration = (durationMs: number): string => {
if (!durationMs) return '0ms';
if (durationMs < 1000) return `${durationMs}ms`;
return `${(durationMs / 1000).toFixed(2)}s`;
};
const truncateToolDetail = (value?: string | null): string => {
if (!value) return '';
return value.length > 600 ? `${value.slice(0, 600)}...` : value;
};
const toggleToolCallDetails = (toolCallId: string) => {
setExpandedToolCallIds((previous) => ({
...previous,
[toolCallId]: !previous[toolCallId],
}));
};
const feedbackByMessageId = useMemo(() => {
const map: Record<string, SessionFeedback> = {};
for (let index = 0; index < messages.length; index++) {
const msg = messages[index];
if (isUserMessage(msg)) continue;
for (let previousIndex = index - 1; previousIndex >= 0; previousIndex--) {
const previousMessage = messages[previousIndex];
if (isUserMessage(previousMessage)) {
const feedback = feedbackMap[previousMessage.id];
if (feedback) {
map[msg.id] = feedback;
}
break;
}
}
}
return map;
}, [feedbackMap, messages]);
const timelineItems = useMemo<SessionTimelineItem[]>(() => {
const messageItems: SessionTimelineItem[] = messages.map(
(message, index) => ({
id: `message-${message.id}`,
type: 'message',
timestamp: parseTimestamp(message.timestamp).getTime(),
order: index * 2,
message,
}),
);
const toolItems: SessionTimelineItem[] = toolCalls.map(
(toolCall, index) => ({
id: `tool-${toolCall.id}`,
type: 'tool',
timestamp: parseTimestamp(toolCall.timestamp).getTime(),
order: index * 2 + 1,
toolCall,
}),
);
return [...messageItems, ...toolItems].sort(
(a, b) => a.timestamp - b.timestamp || a.order - b.order,
);
}, [messages, toolCalls]);
const selectedSession = sessions.find(
(s) => s.session_id === selectedSessionId,
);
@@ -612,29 +730,162 @@ const BotSessionMonitor = forwardRef<
<div className="text-center text-muted-foreground py-12 text-sm">
{t('bots.sessionMonitor.loading')}
</div>
) : messages.length === 0 ? (
) : timelineItems.length === 0 ? (
<div className="text-center text-muted-foreground py-12 text-sm">
{t('bots.sessionMonitor.noMessages')}
</div>
) : (
messages.map((msg, msgIndex) => {
timelineItems.map((item) => {
if (item.type === 'tool') {
const call = item.toolCall;
const hasToolDetails = Boolean(
call.arguments || call.result || call.error_message,
);
const expandedToolCall = Boolean(
expandedToolCallIds[call.id],
);
const detailsId = `tool-call-details-${call.id}`;
return (
<div key={item.id} className="flex justify-start">
<div className="max-w-2xl rounded-xl rounded-bl-sm border border-border/60 bg-muted/25 px-2.5 py-1.5 text-xs text-muted-foreground">
<button
type="button"
className={cn(
'flex w-full items-center justify-between gap-3 rounded-md text-left outline-none transition-colors',
hasToolDetails &&
'cursor-pointer hover:bg-muted/40 focus-visible:ring-2 focus-visible:ring-ring',
)}
aria-expanded={
hasToolDetails ? expandedToolCall : undefined
}
aria-controls={
hasToolDetails ? detailsId : undefined
}
aria-disabled={!hasToolDetails}
onClick={() =>
hasToolDetails &&
toggleToolCallDetails(call.id)
}
>
<div className="flex min-w-0 flex-wrap items-center gap-1.5">
{hasToolDetails &&
(expandedToolCall ? (
<ChevronDown className="h-3.5 w-3.5 shrink-0 text-muted-foreground/70" />
) : (
<ChevronRight className="h-3.5 w-3.5 shrink-0 text-muted-foreground/70" />
))}
<Wrench className="h-3.5 w-3.5 shrink-0 text-muted-foreground/70" />
<span className="min-w-0 max-w-[18rem] truncate text-[13px] font-medium text-foreground/75">
{call.tool_name}
</span>
<span className="rounded border border-border/50 bg-background/60 px-1.5 py-0.5 text-[10px] leading-none text-muted-foreground">
{call.tool_source}
</span>
<span
className={cn(
'rounded px-1.5 py-0.5 text-[10px] font-medium leading-none',
call.status === 'success'
? 'bg-green-100/70 text-green-700 dark:bg-green-950/60 dark:text-green-300'
: 'bg-red-100/70 text-red-700 dark:bg-red-950/60 dark:text-red-300',
)}
>
{call.status}
</span>
</div>
<span className="shrink-0 text-[11px] tabular-nums text-muted-foreground/80">
{formatDuration(call.duration)}
</span>
</button>
{hasToolDetails && expandedToolCall && (
<div
id={detailsId}
className="mt-2 space-y-1.5"
>
{(call.arguments || call.result) && (
<div className="space-y-1.5">
{call.arguments && (
<div>
<div className="mb-1 text-[11px] font-medium text-muted-foreground">
{t(
'monitoring.toolCalls.arguments',
{
defaultValue: '参数',
},
)}
</div>
<pre className="whitespace-pre-wrap break-words rounded bg-background/80 p-2 font-mono text-[11px] leading-4 text-muted-foreground">
{truncateToolDetail(call.arguments)}
</pre>
</div>
)}
{call.result && (
<div>
<div className="mb-1 text-[11px] font-medium text-muted-foreground">
{t('monitoring.toolCalls.result', {
defaultValue: '结果',
})}
</div>
<pre className="whitespace-pre-wrap break-words rounded bg-background/80 p-2 font-mono text-[11px] leading-4 text-muted-foreground">
{truncateToolDetail(call.result)}
</pre>
</div>
)}
</div>
)}
{call.error_message && (
<div className="whitespace-pre-wrap break-words rounded bg-red-50 p-2 text-[11px] text-red-600 dark:bg-red-950/40 dark:text-red-400">
{call.error_message}
</div>
)}
</div>
)}
<div className="mt-1.5 flex items-center gap-1.5 text-[11px] text-muted-foreground">
<span>
{t('monitoring.toolCalls.title', {
defaultValue: '工具调用',
})}
</span>
<span className="tabular-nums">
{formatTime(call.timestamp)}
</span>
{hasToolDetails && (
<>
<span>·</span>
<span>
{expandedToolCall
? t(
'monitoring.toolCalls.hideDetails',
{
defaultValue: '隐藏详情',
},
)
: t(
'monitoring.toolCalls.showDetails',
{
defaultValue: '查看详情',
},
)}
</span>
</>
)}
</div>
</div>
</div>
);
}
const msg = item.message;
const isUser = isUserMessage(msg);
const isDiscarded =
msg.status === 'discarded' ||
msg.pipeline_id === PIPELINE_DISCARD;
// For bot replies, find feedback linked to the preceding user message
let msgFeedback: SessionFeedback | undefined;
if (!isUser) {
for (let i = msgIndex - 1; i >= 0; i--) {
if (isUserMessage(messages[i])) {
msgFeedback = feedbackMap[messages[i].id];
break;
}
}
}
const msgFeedback = feedbackByMessageId[msg.id];
return (
<div
key={msg.id}
key={item.id}
className={cn(
'flex',
isUser ? 'justify-end' : 'justify-start',
@@ -67,6 +67,16 @@ import SettingsDialog, {
import ToolResourceSelectors from '@/app/home/components/dynamic-form/ToolResourceSelectors';
import { LANGBOT_MODELS_PROVIDER_REQUESTER } from '@/app/home/components/models-dialog/types';
function hasUsableUuid<T extends { uuid?: string | null }>(
item: T,
): item is T & { uuid: string } {
return typeof item.uuid === 'string' && item.uuid.trim().length > 0;
}
function hasUsableOptionName(option: { name?: string | null }): boolean {
return typeof option.name === 'string' && option.name.trim().length > 0;
}
export default function DynamicFormItemComponent({
config,
field,
@@ -104,7 +114,7 @@ export default function DynamicFormItemComponent({
httpClient
.getProviderLLMModels()
.then((resp) => {
setLlmModels(resp.models);
setLlmModels(resp.models.filter(hasUsableUuid));
})
.catch((err) => {
toast.error(t('models.getModelListError') + err.msg);
@@ -115,7 +125,7 @@ export default function DynamicFormItemComponent({
httpClient
.getProviderEmbeddingModels()
.then((resp) => {
setEmbeddingModels(resp.models);
setEmbeddingModels(resp.models.filter(hasUsableUuid));
})
.catch((err) => {
toast.error(t('embedding.getModelListError') + err.msg);
@@ -126,7 +136,7 @@ export default function DynamicFormItemComponent({
httpClient
.getProviderRerankModels()
.then((resp) => {
setRerankModels(resp.models);
setRerankModels(resp.models.filter(hasUsableUuid));
})
.catch((err) => {
toast.error('Failed to load rerank models: ' + err.msg);
@@ -230,7 +240,7 @@ export default function DynamicFormItemComponent({
httpClient
.getKnowledgeBases()
.then((resp) => {
setKnowledgeBases(resp.bases);
setKnowledgeBases(resp.bases.filter(hasUsableUuid));
})
.catch((err) => {
toast.error(t('knowledge.getKnowledgeBaseListError') + err.msg);
@@ -243,7 +253,7 @@ export default function DynamicFormItemComponent({
httpClient
.getBots()
.then((resp) => {
setBots(resp.bots);
setBots(resp.bots.filter(hasUsableUuid));
})
.catch((err) => {
toast.error(t('bots.getBotListError') + err.msg);
@@ -388,7 +398,7 @@ export default function DynamicFormItemComponent({
</SelectTrigger>
<SelectContent>
<SelectGroup>
{config.options?.map((option) => (
{config.options?.filter(hasUsableOptionName).map((option) => (
<SelectItem
key={option.name}
value={option.name}
@@ -1176,7 +1186,8 @@ export default function DynamicFormItemComponent({
case DynamicFormItemType.KNOWLEDGE_BASE_SELECTOR:
// Group KBs by Knowledge Engine name
const kbsByEngine = knowledgeBases.reduce(
const validKnowledgeBases = knowledgeBases.filter(hasUsableUuid);
const kbsByEngine = validKnowledgeBases.reduce(
(acc, kb) => {
const engineName = kb.knowledge_engine?.name
? extractI18nObject(kb.knowledge_engine.name)
@@ -1187,7 +1198,7 @@ export default function DynamicFormItemComponent({
acc[engineName].push(kb);
return acc;
},
{} as Record<string, typeof knowledgeBases>,
{} as Record<string, typeof validKnowledgeBases>,
);
return (
@@ -1195,7 +1206,7 @@ export default function DynamicFormItemComponent({
<SelectTrigger className="min-w-0 bg-[#ffffff] dark:bg-[#2a2a2e]">
{field.value && field.value !== '__none__' ? (
(() => {
const selectedKb = knowledgeBases.find(
const selectedKb = validKnowledgeBases.find(
(kb) => kb.uuid === field.value,
);
return (
@@ -1224,7 +1235,7 @@ export default function DynamicFormItemComponent({
<SelectGroup key={engineName}>
<SelectLabel>{engineName}</SelectLabel>
{kbs.map((base) => (
<SelectItem key={base.uuid} value={base.uuid ?? ''}>
<SelectItem key={base.uuid} value={base.uuid}>
<div className="flex items-center gap-2">
{base.emoji && (
<span className="text-sm shrink-0">{base.emoji}</span>
@@ -1241,7 +1252,8 @@ export default function DynamicFormItemComponent({
case DynamicFormItemType.KNOWLEDGE_BASE_MULTI_SELECTOR:
// Group KBs by Knowledge Engine name for multi-selector
const multiKbsByEngine = knowledgeBases.reduce(
const validMultiKnowledgeBases = knowledgeBases.filter(hasUsableUuid);
const multiKbsByEngine = validMultiKnowledgeBases.reduce(
(acc, kb) => {
const engineName = kb.knowledge_engine?.name
? extractI18nObject(kb.knowledge_engine.name)
@@ -1252,7 +1264,7 @@ export default function DynamicFormItemComponent({
acc[engineName].push(kb);
return acc;
},
{} as Record<string, typeof knowledgeBases>,
{} as Record<string, typeof validMultiKnowledgeBases>,
);
return (
@@ -1261,7 +1273,7 @@ export default function DynamicFormItemComponent({
{field.value && field.value.length > 0 ? (
<div className="min-w-0 space-y-2">
{field.value.map((kbId: string) => {
const currentKb = knowledgeBases.find(
const currentKb = validMultiKnowledgeBases.find(
(base) => base.uuid === kbId,
);
if (!currentKb) return null;
@@ -1347,15 +1359,13 @@ export default function DynamicFormItemComponent({
{engineName}
</div>
{kbs.map((base) => {
const isSelected = tempSelectedKBIds.includes(
base.uuid ?? '',
);
const isSelected = tempSelectedKBIds.includes(base.uuid);
return (
<div
key={base.uuid}
className="flex items-center gap-3 rounded-lg border p-3 hover:bg-accent cursor-pointer"
onClick={() => {
const kbId = base.uuid ?? '';
const kbId = base.uuid;
setTempSelectedKBIds((prev) =>
prev.includes(kbId)
? prev.filter((id) => id !== kbId)
@@ -1417,8 +1427,8 @@ export default function DynamicFormItemComponent({
</SelectTrigger>
<SelectContent>
<SelectGroup>
{bots.map((bot) => (
<SelectItem key={bot.uuid} value={bot.uuid ?? ''}>
{bots.filter(hasUsableUuid).map((bot) => (
<SelectItem key={bot.uuid} value={bot.uuid}>
{bot.name}
</SelectItem>
))}
@@ -264,6 +264,48 @@ function saveListExpansionState(state: SidebarListExpansionState) {
// Maximum number of entity sub-items visible before "More" toggle
const MAX_VISIBLE_ITEMS = 5;
const MCP_REFRESH_POLL_INTERVAL_MS = 1000;
const MCP_REFRESH_TIMEOUT_MS = 60000;
function sleep(ms: number) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function waitForMCPRefreshTask(taskId: number) {
const deadline = Date.now() + MCP_REFRESH_TIMEOUT_MS;
while (Date.now() < deadline) {
const task = await httpClient.getAsyncTask(taskId);
if (task.runtime.done) return task;
await sleep(MCP_REFRESH_POLL_INTERVAL_MS);
}
throw new Error(`Timed out waiting for MCP refresh task ${taskId}`);
}
async function refreshEnabledMCPConnections() {
const resp = await httpClient.getMCPServers();
const enabledServers = resp.servers.filter((server) => server.enable);
if (enabledServers.length === 0) return;
const taskResults = await Promise.allSettled(
enabledServers.map((server) => httpClient.testMCPServer(server.name, {})),
);
const taskIds: number[] = [];
for (const result of taskResults) {
if (
result.status === 'fulfilled' &&
typeof result.value.task_id === 'number'
) {
taskIds.push(result.value.task_id);
} else if (result.status === 'rejected') {
console.error('Failed to start MCP refresh task:', result.reason);
}
}
await Promise.allSettled(taskIds.map(waitForMCPRefreshTask));
}
// Sort entity items by updatedAt descending (most recent first), items without updatedAt go last
function sortByRecent(items: SidebarEntityItem[]): SidebarEntityItem[] {
@@ -352,11 +394,19 @@ function NavItems({
if (extRefreshing) return;
setExtRefreshing(true);
try {
await Promise.all([
const results = await Promise.allSettled([
sidebarData.refreshPlugins(),
sidebarData.refreshMCPServers(),
sidebarData.refreshSkills(),
refreshEnabledMCPConnections(),
]);
const mcpRefreshResult = results[2];
if (mcpRefreshResult.status === 'rejected') {
console.error(
'Failed to refresh MCP connections:',
mcpRefreshResult.reason,
);
}
await sidebarData.refreshMCPServers();
} finally {
setExtRefreshing(false);
}
@@ -157,6 +157,10 @@ export default function MCPDetailContent({ id }: { id: string }) {
navigate(`/home/mcp?id=${encodeURIComponent(serverName)}`);
}
const handlePersistedTestComplete = useCallback(async () => {
await refreshMCPServers();
}, [refreshMCPServers]);
function confirmDelete() {
httpClient
.deleteMCPServer(id)
@@ -364,6 +368,7 @@ export default function MCPDetailContent({ id }: { id: string }) {
onRuntimeInfoChange={(runtimeInfo) =>
setDetailRuntimeStatus(runtimeInfo?.status ?? null)
}
onPersistedTestComplete={handlePersistedTestComplete}
/>
</div>
</div>
@@ -41,6 +41,7 @@ import {
} from '@/components/ui/card';
import { httpClient } from '@/app/infra/http/HttpClient';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import MCPLogs from '@/app/home/mcp/components/mcp-form/MCPLogs';
import MCPReadme from '@/app/home/mcp/components/mcp-form/MCPReadme';
import {
MCPServerRuntimeInfo,
@@ -487,6 +488,7 @@ interface MCPFormProps {
onDirtyChange?: (dirty: boolean) => void;
onTestingChange?: (testing: boolean) => void;
onRuntimeInfoChange?: (runtimeInfo: MCPServerRuntimeInfo | null) => void;
onPersistedTestComplete?: (serverName: string) => void | Promise<void>;
/** Reported when the form cannot be saved because the current mode is
* ``stdio`` and the Box sandbox is disabled/unavailable. Parents that
* render the Save button outside this component should disable it. */
@@ -511,6 +513,7 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
onDirtyChange,
onTestingChange,
onRuntimeInfoChange,
onPersistedTestComplete,
onSaveBlockedChange,
layout = 'stacked',
sideHeader,
@@ -750,6 +753,8 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
}
try {
let serverConfig: MCPServer;
const serverName =
isEditMode && initServerName ? initServerName : value.name;
if (value.mode === 'remote') {
const headers: Record<string, string> = {};
@@ -758,7 +763,7 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
});
serverConfig = {
name: value.name,
name: serverName,
mode: 'remote',
enable: true,
extra_args: {
@@ -774,7 +779,7 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
});
serverConfig = {
name: value.name,
name: serverName,
mode: 'stdio',
enable: true,
extra_args: {
@@ -818,6 +823,10 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
// `uvx` with no package (exit 2 / "Connection closed", no detail).
// The form values are kept in sync on every edit and on load, so they
// are always current.
const serverName =
isEditMode && initServerName ? initServerName : form.getValues('name');
const shouldTestPersistedServer =
isEditMode && !!initServerName && !form.formState.isDirty;
const formExtraArgs = form.getValues('extra_args') ?? [];
const formStdioArgs = form.getValues('args') ?? [];
let extraArgsData: MCPServerExtraArgsRemote | MCPServerExtraArgsStdio;
@@ -840,12 +849,20 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
};
}
const { task_id } = await httpClient.testMCPServer('_', {
name: form.getValues('name'),
mode,
enable: true,
extra_args: extraArgsData,
} as MCPServer);
const testTarget = shouldTestPersistedServer ? serverName : '_';
const testPayload = shouldTestPersistedServer
? {}
: ({
name: serverName,
mode,
enable: true,
extra_args: extraArgsData,
} as MCPServer);
const { task_id } = await httpClient.testMCPServer(
testTarget,
testPayload,
);
if (!task_id) {
throw new Error(t('mcp.noTaskId'));
@@ -871,14 +888,18 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
resource_count: 0,
resources: [],
});
if (shouldTestPersistedServer) {
await onPersistedTestComplete?.(serverName);
}
} else {
if (isEditMode) {
await loadServerForEdit(form.getValues('name'));
if (shouldTestPersistedServer) {
await loadServerForEdit(serverName);
await onPersistedTestComplete?.(serverName);
} else {
// Create mode has no persisted server to reload tools from.
// Transient tests have no persisted server to reload tools from.
// The backend stashes the discovered runtime info (status +
// tools) in the test task's metadata before tearing the
// transient session down — surface it so a successful test
// tools) in the task metadata before tearing the transient
// session down — surface it so a successful test
// shows the tool list instead of "no tools found".
const runtimeInfoFromTest = taskResp.task_context?.metadata
?.runtime_info as MCPServerRuntimeInfo | undefined;
@@ -1163,11 +1184,14 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
</Card>
);
const persistedServerName =
isEditMode && initServerName ? initServerName : form.getValues('name');
const runtimePanel = (
<RuntimePanel
mcpTesting={mcpTesting}
runtimeInfo={runtimeInfo}
serverName={form.getValues('name')}
serverName={persistedServerName}
t={t}
/>
);
@@ -1200,6 +1224,9 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
<TabsTrigger value="resources" className="flex-none px-4">
{resourcesTabLabel}
</TabsTrigger>
<TabsTrigger value="logs" className="flex-none px-4">
{t('mcp.tabLogs')}
</TabsTrigger>
</TabsList>
<TabsContent value="docs" className="mt-4 min-h-0 flex-1 overflow-y-auto">
<MCPReadme readme={readme} />
@@ -1211,7 +1238,7 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
<RuntimePanel
mcpTesting={mcpTesting}
runtimeInfo={runtimeInfo}
serverName={form.getValues('name')}
serverName={persistedServerName}
content="tools"
t={t}
/>
@@ -1223,11 +1250,14 @@ const MCPForm = forwardRef<MCPFormHandle, MCPFormProps>(function MCPForm(
<RuntimePanel
mcpTesting={mcpTesting}
runtimeInfo={runtimeInfo}
serverName={form.getValues('name')}
serverName={persistedServerName}
content="resources"
t={t}
/>
</TabsContent>
<TabsContent value="logs" className="mt-4 min-h-0 flex-1 overflow-y-auto">
{persistedServerName && <MCPLogs serverName={persistedServerName} />}
</TabsContent>
</Tabs>
) : (
runtimePanel
@@ -0,0 +1,149 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient';
import { useTranslation } from 'react-i18next';
import { PluginLogEntry } from '@/app/infra/entities/plugin';
import { Button } from '@/components/ui/button';
import { Switch } from '@/components/ui/switch';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { RefreshCw } from 'lucide-react';
const LEVEL_OPTIONS = ['ALL', 'DEBUG', 'INFO', 'WARNING', 'ERROR'] as const;
function levelClassName(level: string): string {
switch (level) {
case 'ERROR':
case 'CRITICAL':
return 'text-red-500';
case 'WARNING':
return 'text-amber-500';
case 'DEBUG':
return 'text-gray-400 dark:text-gray-500';
default:
return 'text-gray-700 dark:text-gray-300';
}
}
export default function MCPLogs({ serverName }: { serverName: string }) {
const { t } = useTranslation();
const [logs, setLogs] = useState<PluginLogEntry[]>([]);
const [isLoading, setIsLoading] = useState(false);
const [level, setLevel] = useState<string>('ALL');
const [autoRefresh, setAutoRefresh] = useState(true);
const scrollRef = useRef<HTMLDivElement>(null);
const atBottomRef = useRef(true);
const fetchLogs = useCallback(() => {
setIsLoading(true);
httpClient
.getMcpServerLogs(serverName, 500, level === 'ALL' ? undefined : level)
.then((res) => {
setLogs(res.logs ?? []);
})
.catch(() => {
setLogs([]);
})
.finally(() => {
setIsLoading(false);
});
}, [serverName, level]);
useEffect(() => {
fetchLogs();
}, [fetchLogs]);
// Auto-refresh poll loop.
useEffect(() => {
if (!autoRefresh) return;
const timer = setInterval(fetchLogs, 3000);
return () => clearInterval(timer);
}, [autoRefresh, fetchLogs]);
// Keep view pinned to bottom when the user is already at the bottom.
useEffect(() => {
const el = scrollRef.current;
if (el && atBottomRef.current) {
el.scrollTop = el.scrollHeight;
}
}, [logs]);
function handleScroll() {
const el = scrollRef.current;
if (!el) return;
atBottomRef.current = el.scrollHeight - el.scrollTop - el.clientHeight < 40;
}
return (
<div className="flex h-full flex-col">
<div className="flex shrink-0 flex-wrap items-center gap-2 px-1 pb-3 sm:px-6">
<Select value={level} onValueChange={setLevel}>
<SelectTrigger className="h-8 w-[130px]">
<SelectValue />
</SelectTrigger>
<SelectContent>
{LEVEL_OPTIONS.map((opt) => (
<SelectItem key={opt} value={opt}>
{opt === 'ALL' ? t('mcp.logsLevelAll') : opt}
</SelectItem>
))}
</SelectContent>
</Select>
<Button
type="button"
variant="outline"
size="sm"
className="h-8"
onClick={fetchLogs}
disabled={isLoading}
>
<RefreshCw
className={`mr-1.5 size-3.5 ${isLoading ? 'animate-spin' : ''}`}
/>
{t('mcp.logsRefresh')}
</Button>
<div className="flex items-center gap-2">
<Switch
id="mcp-logs-auto-refresh"
checked={autoRefresh}
onCheckedChange={setAutoRefresh}
/>
<Label
htmlFor="mcp-logs-auto-refresh"
className="cursor-pointer text-sm font-normal text-muted-foreground"
>
{t('mcp.logsAutoRefresh')}
</Label>
</div>
</div>
<div
ref={scrollRef}
onScroll={handleScroll}
className="min-h-0 flex-1 overflow-auto bg-gray-50 px-3 py-3 font-mono text-xs leading-relaxed dark:bg-gray-900/40 sm:px-6"
>
{logs.length === 0 ? (
<div className="py-8 text-center text-sm text-gray-500 dark:text-gray-400">
{t('mcp.logsEmpty')}
</div>
) : (
logs.map((entry, idx) => (
<div
key={`${entry.ts}-${idx}`}
className={`whitespace-pre-wrap break-all ${levelClassName(
entry.level,
)}`}
>
{entry.text}
</div>
))
)}
</div>
</div>
);
}
@@ -0,0 +1,649 @@
import React from 'react';
import { useTranslation } from 'react-i18next';
import {
AlertCircle,
Bot,
ChevronDown,
ChevronRight,
Clock,
Cpu,
Hash,
User,
Wrench,
} from 'lucide-react';
import { cn } from '@/lib/utils';
import { MessageContentRenderer } from './MessageContentRenderer';
import {
ConversationTurn,
hasRenderableMessageContent,
} from '../utils/conversationTurns';
import { MonitoringMessage } from '../types/monitoring';
interface ConversationTurnListProps {
turns: ConversationTurn[];
expandedTurnId: string | null;
onToggleTurn: (turnId: string) => void;
}
function shortId(id?: string) {
if (!id) return '-';
if (id.length <= 12) return id;
return `${id.slice(0, 8)}...${id.slice(-4)}`;
}
function formatDuration(ms: number) {
if (!ms) return '0ms';
if (ms < 1000) return `${ms}ms`;
return `${(ms / 1000).toFixed(2)}s`;
}
function truncateDetail(value?: string) {
if (!value) return '';
return value.length > 1200 ? `${value.slice(0, 1200)}...` : value;
}
function roleLabel(message: MonitoringMessage | undefined) {
const role = message?.role?.toLowerCase();
if (role === 'assistant') return 'assistant';
if (role === 'user') return 'user';
return 'message';
}
function statusClass(level: ConversationTurn['level']) {
if (level === 'error') {
return 'border-red-200 bg-red-50 text-red-700 dark:border-red-900 dark:bg-red-950/40 dark:text-red-300';
}
if (level === 'warning') {
return 'border-yellow-200 bg-yellow-50 text-yellow-700 dark:border-yellow-900 dark:bg-yellow-950/40 dark:text-yellow-300';
}
return 'border-green-200 bg-green-50 text-green-700 dark:border-green-900 dark:bg-green-950/40 dark:text-green-300';
}
function Metric({
icon,
label,
tone = 'default',
}: {
icon: React.ReactNode;
label: string;
tone?: 'default' | 'error';
}) {
return (
<span
className={cn(
'inline-flex h-7 items-center gap-1.5 rounded-md border px-2 text-xs font-medium',
tone === 'error'
? 'border-red-200 bg-red-50 text-red-700 dark:border-red-900 dark:bg-red-950/40 dark:text-red-300'
: 'border-border bg-background text-muted-foreground',
)}
>
{icon}
{label}
</span>
);
}
function MetaItem({ label, value }: { label: string; value?: string }) {
return (
<div className="min-w-0 rounded-md bg-background px-3 py-2">
<div className="text-xs text-muted-foreground">{label}</div>
<div className="truncate text-sm font-medium text-foreground">
{value || '-'}
</div>
</div>
);
}
function MessageLane({
label,
icon,
content,
empty,
maxLines,
}: {
label: string;
icon: React.ReactNode;
content?: string;
empty: string;
maxLines: number;
}) {
return (
<div className="grid grid-cols-[5.25rem_minmax(0,1fr)] items-start gap-3 text-sm sm:grid-cols-[6rem_minmax(0,1fr)]">
<div className="flex h-7 items-center gap-1.5 text-xs font-medium text-muted-foreground">
{icon}
<span>{label}</span>
</div>
<div className="min-w-0 rounded-md bg-muted/45 px-3 py-2 text-foreground">
{content && hasRenderableMessageContent(content) ? (
<MessageContentRenderer content={content} maxLines={maxLines} />
) : (
<span className="italic text-muted-foreground">{empty}</span>
)}
</div>
</div>
);
}
function ExpandedMessage({
message,
label,
}: {
message: MonitoringMessage;
label: string;
}) {
return (
<div className="border-t border-border/70 py-3 first:border-t-0 first:pt-0 last:pb-0">
<div className="mb-2 flex flex-wrap items-center gap-2 text-xs text-muted-foreground">
<span className="rounded-md bg-muted px-2 py-1 font-medium text-foreground">
{label}
</span>
<span>{message.timestamp.toLocaleString()}</span>
<span className="font-mono">ID: {shortId(message.id)}</span>
</div>
<div className="text-sm leading-6 text-foreground">
<MessageContentRenderer content={message.messageContent} maxLines={4} />
</div>
</div>
);
}
export function ConversationTurnList({
turns,
expandedTurnId,
onToggleTurn,
}: ConversationTurnListProps) {
const { t } = useTranslation();
const [expandedToolCallIds, setExpandedToolCallIds] = React.useState<
Record<string, boolean>
>({});
const toggleToolCallDetails = (toolCallKey: string) => {
setExpandedToolCallIds((previous) => ({
...previous,
[toolCallKey]: !previous[toolCallKey],
}));
};
return (
<div className="space-y-4">
<div className="flex items-center justify-between text-sm text-muted-foreground">
<span className="font-medium text-foreground">
{t('monitoring.messageList.turns', {
defaultValue: '{{count}} 轮对话',
count: turns.length,
})}
</span>
</div>
{turns.map((turn) => {
const expanded = expandedTurnId === turn.id;
const firstAssistant = turn.assistantMessages[0];
const assistantOverflow = Math.max(
turn.assistantMessages.length - 1,
0,
);
return (
<div
key={turn.id}
className={cn(
'overflow-hidden rounded-xl border bg-card transition-colors',
turn.level === 'error' && 'border-red-200 dark:border-red-900',
)}
>
<div
role="button"
tabIndex={0}
className="cursor-pointer p-3 outline-none transition-colors hover:bg-accent/60 focus-visible:ring-2 focus-visible:ring-ring sm:p-5"
onClick={() => onToggleTurn(turn.id)}
onKeyDown={(event) => {
if (event.key === 'Enter' || event.key === ' ') {
event.preventDefault();
onToggleTurn(turn.id);
}
}}
>
<div className="flex flex-col gap-4 lg:flex-row lg:items-start lg:justify-between">
<div className="min-w-0 flex-1">
<div className="mb-2 flex min-w-0 items-center gap-2">
{expanded ? (
<ChevronDown className="h-5 w-5 shrink-0 text-muted-foreground" />
) : (
<ChevronRight className="h-5 w-5 shrink-0 text-muted-foreground" />
)}
<span className="truncate font-mono text-xs text-muted-foreground">
Turn: {shortId(turn.id)}
</span>
</div>
<div className="mb-3 flex min-w-0 flex-wrap items-center gap-2">
<span className="truncate text-sm font-medium text-foreground">
{turn.botName}
</span>
<span className="text-muted-foreground"></span>
<span className="truncate text-sm text-muted-foreground">
{turn.pipelineName}
</span>
{turn.runnerName && (
<>
<span className="text-muted-foreground"></span>
<span className="truncate text-sm text-muted-foreground">
{turn.runnerName}
</span>
</>
)}
</div>
<div className="space-y-2">
<MessageLane
label={t('monitoring.messageList.userMessage', {
defaultValue: '用户',
})}
icon={<User className="h-3.5 w-3.5" />}
content={turn.userMessage?.messageContent}
empty={t('monitoring.messageList.noUserMessage', {
defaultValue: '未记录用户输入',
})}
maxLines={2}
/>
<MessageLane
label={
assistantOverflow > 0
? t('monitoring.messageList.assistantMessageCount', {
defaultValue: '助手 +{{count}}',
count: assistantOverflow,
})
: t('monitoring.messageList.assistantMessage', {
defaultValue: '助手',
})
}
icon={<Bot className="h-3.5 w-3.5" />}
content={firstAssistant?.messageContent}
empty={t('monitoring.messageList.noAssistantMessage', {
defaultValue: '未记录助手回复',
})}
maxLines={2}
/>
</div>
</div>
<div className="flex shrink-0 flex-col gap-2 lg:items-end">
<div className="text-xs text-muted-foreground">
{turn.lastActivityAt.toLocaleString()}
</div>
<div
className={cn(
'inline-flex h-7 items-center rounded-md border px-2 text-xs font-medium',
statusClass(turn.level),
)}
>
{turn.level}
</div>
<div className="flex flex-wrap gap-2 lg:justify-end">
<Metric
icon={<Cpu className="h-3.5 w-3.5" />}
label={`${turn.llmCalls.length} LLM`}
/>
{turn.toolCalls.length > 0 && (
<Metric
icon={<Wrench className="h-3.5 w-3.5" />}
label={`${turn.toolCalls.length} tools`}
/>
)}
<Metric
icon={<Hash className="h-3.5 w-3.5" />}
label={`${turn.totalTokens.toLocaleString()} tokens`}
/>
<Metric
icon={<Clock className="h-3.5 w-3.5" />}
label={formatDuration(turn.totalDuration)}
/>
{turn.errors.length > 0 && (
<Metric
icon={<AlertCircle className="h-3.5 w-3.5" />}
label={`${turn.errors.length} errors`}
tone="error"
/>
)}
</div>
</div>
</div>
</div>
{expanded && (
<div className="border-t bg-muted/40 p-3 sm:p-5">
<div className="space-y-5 border-l-2 border-border pl-4 sm:pl-6">
<div className="grid grid-cols-2 gap-2 lg:grid-cols-5">
<MetaItem
label={t('monitoring.messageList.platform', {
defaultValue: '平台',
})}
value={turn.platform}
/>
<MetaItem
label={t('monitoring.messageList.user', {
defaultValue: '用户',
})}
value={turn.userName || turn.userId}
/>
<MetaItem
label={t('monitoring.messageList.runner', {
defaultValue: '执行器',
})}
value={turn.runnerName}
/>
<MetaItem
label={t('monitoring.sessions.sessionId', {
defaultValue: '会话 ID',
})}
value={turn.sessionId}
/>
<MetaItem
label={t('monitoring.messageList.messageCount', {
defaultValue: '消息数',
})}
value={String(turn.messages.length)}
/>
</div>
<section>
<h4 className="mb-3 flex items-center gap-2 text-sm font-semibold text-foreground">
<Bot className="h-4 w-4" />
{t('monitoring.messageList.conversationTrace', {
defaultValue: '消息链路',
})}
</h4>
<div className="rounded-lg bg-background px-3 py-3">
{turn.messages.map((message) => (
<ExpandedMessage
key={message.id}
message={message}
label={t(
`monitoring.messageList.roles.${roleLabel(message)}`,
{
defaultValue:
roleLabel(message) === 'assistant'
? '助手'
: roleLabel(message) === 'user'
? '用户'
: '消息',
},
)}
/>
))}
</div>
</section>
<section>
<h4 className="mb-3 flex items-center gap-2 text-sm font-semibold text-foreground">
<Cpu className="h-4 w-4" />
{t('monitoring.llmCalls.title', {
defaultValue: 'LLM 调用',
})}{' '}
({turn.llmCalls.length})
</h4>
<div className="grid grid-cols-3 gap-2">
<MetaItem
label={t('monitoring.llmCalls.totalTokens', {
defaultValue: '总 Token',
})}
value={turn.totalTokens.toLocaleString()}
/>
<MetaItem
label={t('monitoring.llmCalls.inputTokens', {
defaultValue: '输入 Token',
})}
value={turn.inputTokens.toLocaleString()}
/>
<MetaItem
label={t('monitoring.llmCalls.duration', {
defaultValue: '耗时',
})}
value={formatDuration(turn.totalDuration)}
/>
</div>
<div className="mt-3 rounded-lg bg-background px-3 py-3">
{turn.llmCalls.length > 0 ? (
turn.llmCalls.map((call, index) => (
<div
key={call.id}
className="border-t border-border/70 py-3 first:border-t-0 first:pt-0 last:pb-0"
>
<div className="mb-2 flex flex-wrap items-center justify-between gap-2">
<div className="flex min-w-0 flex-wrap items-center gap-2">
<span className="text-sm font-medium text-foreground">
#{index + 1} {call.modelName}
</span>
<span
className={cn(
'rounded-md px-2 py-1 text-xs font-medium',
call.status === 'success'
? 'bg-green-100 text-green-700 dark:bg-green-950 dark:text-green-300'
: 'bg-red-100 text-red-700 dark:bg-red-950 dark:text-red-300',
)}
>
{call.status}
</span>
</div>
<span className="text-xs text-muted-foreground">
{formatDuration(call.duration)}
</span>
</div>
<div className="flex flex-wrap gap-x-8 gap-y-1 text-xs text-muted-foreground">
<span>In: {call.tokens.input}</span>
<span>Out: {call.tokens.output}</span>
<span>Total: {call.tokens.total}</span>
<span className="font-mono">
ID: {shortId(call.id)}
</span>
</div>
{call.errorMessage && (
<div className="mt-2 whitespace-pre-wrap break-words text-xs text-red-600 dark:text-red-400">
{call.errorMessage}
</div>
)}
</div>
))
) : (
<div className="py-4 text-center text-sm text-muted-foreground">
{t('monitoring.messageList.noLlmCalls', {
defaultValue: '未记录模型调用',
})}
</div>
)}
</div>
</section>
<section>
<h4 className="mb-3 flex items-center gap-2 text-sm font-semibold text-foreground">
<Wrench className="h-4 w-4" />
{t('monitoring.toolCalls.title', {
defaultValue: '工具调用',
})}{' '}
({turn.toolCalls.length})
</h4>
<div className="grid grid-cols-2 gap-2 lg:grid-cols-3">
<MetaItem
label={t('monitoring.toolCalls.totalCalls', {
defaultValue: '调用次数',
})}
value={String(turn.toolCalls.length)}
/>
<MetaItem
label={t('monitoring.toolCalls.duration', {
defaultValue: '工具耗时',
})}
value={formatDuration(turn.totalToolDuration)}
/>
<MetaItem
label={t('monitoring.toolCalls.errorCalls', {
defaultValue: '失败次数',
})}
value={String(
turn.toolCalls.filter(
(call) => call.status === 'error',
).length,
)}
/>
</div>
<div className="mt-3 rounded-lg bg-background px-3 py-3">
{turn.toolCalls.length > 0 ? (
turn.toolCalls.map((call, index) => {
const toolCallKey = `${turn.id}:${call.id}`;
const hasToolDetails = Boolean(
call.arguments || call.result || call.errorMessage,
);
const expandedToolCall = Boolean(
expandedToolCallIds[toolCallKey],
);
const detailsId = `monitoring-tool-call-details-${call.id}`;
return (
<div
key={call.id}
className="border-t border-border/70 py-2 first:border-t-0 first:pt-0 last:pb-0"
>
<button
type="button"
className={cn(
'flex w-full items-start justify-between gap-3 rounded-md px-2 py-2 text-left outline-none transition-colors',
hasToolDetails &&
'cursor-pointer hover:bg-muted/60 focus-visible:ring-2 focus-visible:ring-ring',
)}
aria-expanded={
hasToolDetails ? expandedToolCall : undefined
}
aria-controls={
hasToolDetails ? detailsId : undefined
}
aria-disabled={!hasToolDetails}
onClick={() =>
hasToolDetails &&
toggleToolCallDetails(toolCallKey)
}
>
<div className="flex min-w-0 flex-wrap items-center gap-2">
{hasToolDetails &&
(expandedToolCall ? (
<ChevronDown className="mt-0.5 h-4 w-4 shrink-0 text-muted-foreground" />
) : (
<ChevronRight className="mt-0.5 h-4 w-4 shrink-0 text-muted-foreground" />
))}
<span className="min-w-0 truncate text-sm font-medium text-foreground">
#{index + 1} {call.toolName}
</span>
<span className="rounded-md bg-muted px-2 py-1 text-xs font-medium text-muted-foreground">
{call.toolSource}
</span>
<span
className={cn(
'rounded-md px-2 py-1 text-xs font-medium',
call.status === 'success'
? 'bg-green-100 text-green-700 dark:bg-green-950 dark:text-green-300'
: 'bg-red-100 text-red-700 dark:bg-red-950 dark:text-red-300',
)}
>
{call.status}
</span>
<span className="font-mono text-xs text-muted-foreground">
ID: {shortId(call.id)}
</span>
</div>
<span className="shrink-0 text-xs text-muted-foreground">
{formatDuration(call.duration)}
</span>
</button>
{hasToolDetails && expandedToolCall && (
<div
id={detailsId}
className="mt-1 grid gap-2 px-2 pb-2 text-xs lg:grid-cols-2"
>
{call.arguments && (
<div className="min-w-0 rounded-md bg-muted/50 p-2">
<div className="mb-1 font-medium text-foreground">
{t('monitoring.toolCalls.arguments', {
defaultValue: '参数',
})}
</div>
<pre className="whitespace-pre-wrap break-words font-mono text-muted-foreground">
{truncateDetail(call.arguments)}
</pre>
</div>
)}
{call.result && (
<div className="min-w-0 rounded-md bg-muted/50 p-2">
<div className="mb-1 font-medium text-foreground">
{t('monitoring.toolCalls.result', {
defaultValue: '结果',
})}
</div>
<pre className="whitespace-pre-wrap break-words font-mono text-muted-foreground">
{truncateDetail(call.result)}
</pre>
</div>
)}
{call.errorMessage && (
<div className="min-w-0 whitespace-pre-wrap break-words rounded-md bg-red-50 p-2 text-red-600 dark:bg-red-950/40 dark:text-red-400 lg:col-span-2">
{call.errorMessage}
</div>
)}
</div>
)}
</div>
);
})
) : (
<div className="py-4 text-center text-sm text-muted-foreground">
{t('monitoring.toolCalls.noToolCalls', {
defaultValue: '未记录工具调用',
})}
</div>
)}
</div>
</section>
{turn.errors.length > 0 && (
<section>
<h4 className="mb-3 flex items-center gap-2 text-sm font-semibold text-red-700 dark:text-red-300">
<AlertCircle className="h-4 w-4" />
{t('monitoring.errors.title', {
defaultValue: '错误日志',
})}{' '}
({turn.errors.length})
</h4>
<div className="rounded-lg bg-background px-3 py-3">
{turn.errors.map((error) => (
<div
key={error.id}
className="border-t border-red-200/80 py-3 first:border-t-0 first:pt-0 last:pb-0 dark:border-red-900"
>
<div className="mb-2 flex flex-wrap items-center justify-between gap-2">
<span className="text-sm font-medium text-red-700 dark:text-red-300">
{error.errorType}
</span>
<span className="text-xs text-muted-foreground">
{error.timestamp.toLocaleString()}
</span>
</div>
<div className="whitespace-pre-wrap break-words text-sm text-red-600 dark:text-red-400">
{error.errorMessage}
</div>
</div>
))}
</div>
</section>
)}
</div>
</div>
)}
</div>
);
})}
</div>
);
}
@@ -106,6 +106,9 @@ export function useMonitoringData(filterState: FilterState) {
const llmCalls = Array.isArray(response?.llmCalls)
? response.llmCalls
: [];
const toolCalls = Array.isArray(response?.toolCalls)
? response.toolCalls
: [];
const embeddingCalls = Array.isArray(response?.embeddingCalls)
? response.embeddingCalls
: [];
@@ -116,6 +119,7 @@ export function useMonitoringData(filterState: FilterState) {
const totalCount = response?.totalCount ?? {
messages: messages.length,
llmCalls: llmCalls.length,
toolCalls: toolCalls.length,
embeddingCalls: embeddingCalls.length,
sessions: sessions.length,
errors: errors.length,
@@ -145,8 +149,10 @@ export function useMonitoringData(filterState: FilterState) {
level: string;
platform?: string;
user_id?: string;
user_name?: string;
runner_name?: string;
variables?: string;
role?: string;
}) => ({
id: msg.id,
timestamp: parseUTCTimestamp(msg.timestamp),
@@ -160,8 +166,10 @@ export function useMonitoringData(filterState: FilterState) {
level: msg.level as 'info' | 'warning' | 'error' | 'debug',
platform: msg.platform,
userId: msg.user_id,
userName: msg.user_name,
runnerName: msg.runner_name,
variables: msg.variables,
role: msg.role,
}),
),
llmCalls: llmCalls.map(
@@ -179,6 +187,7 @@ export function useMonitoringData(filterState: FilterState) {
bot_name: string;
pipeline_id: string;
pipeline_name: string;
session_id?: string;
error_message?: string;
message_id?: string;
}) => ({
@@ -197,10 +206,46 @@ export function useMonitoringData(filterState: FilterState) {
botName: call.bot_name,
pipelineId: call.pipeline_id,
pipelineName: call.pipeline_name,
sessionId: call.session_id,
errorMessage: call.error_message,
messageId: call.message_id,
}),
),
toolCalls: toolCalls.map(
(call: {
id: string;
timestamp: string;
tool_name: string;
tool_source: string;
duration: number;
status: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
session_id?: string;
message_id?: string;
arguments?: string;
result?: string;
error_message?: string;
}) => ({
id: call.id,
timestamp: parseUTCTimestamp(call.timestamp),
toolName: call.tool_name,
toolSource: call.tool_source,
duration: call.duration,
status: call.status as 'success' | 'error',
botId: call.bot_id,
botName: call.bot_name,
pipelineId: call.pipeline_id,
pipelineName: call.pipeline_name,
sessionId: call.session_id,
messageId: call.message_id,
arguments: call.arguments,
result: call.result,
errorMessage: call.error_message,
}),
),
embeddingCalls: embeddingCalls.map(
(call: {
id: string;
@@ -294,6 +339,7 @@ export function useMonitoringData(filterState: FilterState) {
totalCount: {
messages: totalCount.messages,
llmCalls: totalCount.llmCalls,
toolCalls: totalCount.toolCalls ?? toolCalls.length,
embeddingCalls: totalCount.embeddingCalls || 0,
sessions: totalCount.sessions,
errors: totalCount.errors,
@@ -317,6 +363,7 @@ export function useMonitoringData(filterState: FilterState) {
botName: call.botName,
pipelineId: call.pipelineId,
pipelineName: call.pipelineName,
sessionId: call.sessionId,
}),
);
+33 -264
View File
@@ -18,60 +18,12 @@ import { ExportDropdown } from './components/ExportDropdown';
import { useMonitoringFilters } from './hooks/useMonitoringFilters';
import { useMonitoringData } from './hooks/useMonitoringData';
import { useFeedbackData } from './hooks/useFeedbackData';
import { MessageDetailsCard } from './components/MessageDetailsCard';
import { MessageContentRenderer } from './components/MessageContentRenderer';
import { ConversationTurnList } from './components/ConversationTurnList';
import { FeedbackStatsCards } from './components/FeedbackCard';
import { FeedbackList } from './components/FeedbackList';
import { MessageDetails } from './types/monitoring';
import { httpClient } from '@/app/infra/http/HttpClient';
import { buildConversationTurns } from './utils/conversationTurns';
import { LoadingSpinner, LoadingPage } from '@/components/ui/loading-spinner';
interface RawMessageData {
id: string;
timestamp: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
message_content: string;
session_id: string;
status: string;
level: string;
platform: string;
user_id: string;
runner_name: string;
variables: Record<string, unknown>;
}
interface RawLLMCallData {
id: string;
timestamp: string;
model_name: string;
status: string;
duration: number;
error_message: string | null;
input_tokens: number;
output_tokens: number;
total_tokens: number;
}
interface RawLLMStatsData {
total_calls: number;
total_input_tokens: number;
total_output_tokens: number;
total_tokens: number;
total_duration_ms: number;
average_duration_ms: number;
}
interface RawErrorData {
id: string;
timestamp: string;
error_type: string;
error_message: string;
stack_trace: string | null;
}
function MonitoringPageContent() {
const { t } = useTranslation();
const { filterState, setSelectedBots, setSelectedPipelines, setTimeRange } =
@@ -146,115 +98,37 @@ function MonitoringPageContent() {
setFeedbackRefreshKey((k) => k + 1);
}, [refetch]);
const [expandedMessageId, setExpandedMessageId] = useState<string | null>(
null,
);
const [messageDetails, setMessageDetails] = useState<
Record<string, MessageDetails>
>({});
const [loadingDetails, setLoadingDetails] = useState<Record<string, boolean>>(
{},
const conversationTurns = useMemo(
() =>
buildConversationTurns(
data?.messages || [],
data?.llmCalls || [],
data?.errors || [],
data?.toolCalls || [],
),
[data?.messages, data?.llmCalls, data?.errors, data?.toolCalls],
);
// State for expanded errors
const [expandedErrorId, setExpandedErrorId] = useState<string | null>(null);
const [expandedTurnId, setExpandedTurnId] = useState<string | null>(null);
// State for controlled tabs
const [activeTab, setActiveTab] = useState<string>('messages');
// Function to jump to a message record
const jumpToMessage = async (messageId: string) => {
const jumpToMessage = (messageId: string) => {
setActiveTab('messages');
// Small delay to ensure tab switch completes
setTimeout(() => {
toggleMessageExpand(messageId);
const turn = conversationTurns.find((item) =>
item.messages.some((message) => message.id === messageId),
);
setExpandedTurnId(turn?.id ?? messageId);
}, 100);
};
const toggleMessageExpand = async (messageId: string) => {
if (expandedMessageId === messageId) {
// Collapse
setExpandedMessageId(null);
} else {
// Expand
setExpandedMessageId(messageId);
// Fetch details if not already loaded
if (!messageDetails[messageId]) {
setLoadingDetails({ ...loadingDetails, [messageId]: true });
try {
// httpClient.get() returns the inner data directly (response.data.data)
const result = await httpClient.get<{
message_id: string;
found: boolean;
message: RawMessageData | null;
llm_calls: RawLLMCallData[];
llm_stats: RawLLMStatsData;
errors: RawErrorData[];
}>(`/api/v1/monitoring/messages/${messageId}/details`);
if (result) {
setMessageDetails((prev) => ({
...prev,
[messageId]: {
messageId: result.message_id,
found: result.found,
message: result.message
? {
id: result.message.id,
timestamp: new Date(result.message.timestamp),
botId: result.message.bot_id,
botName: result.message.bot_name,
pipelineId: result.message.pipeline_id,
pipelineName: result.message.pipeline_name,
messageContent: result.message.message_content,
sessionId: result.message.session_id,
status: result.message.status,
level: result.message.level,
platform: result.message.platform,
userId: result.message.user_id,
runnerName: result.message.runner_name,
variables: result.message.variables,
}
: undefined,
llmCalls: result.llm_calls.map((call: RawLLMCallData) => ({
id: call.id,
timestamp: new Date(call.timestamp),
modelName: call.model_name,
status: call.status,
duration: call.duration,
errorMessage: call.error_message,
tokens: {
input: call.input_tokens || 0,
output: call.output_tokens || 0,
total: call.total_tokens || 0,
},
})),
errors: result.errors.map((error: RawErrorData) => ({
id: error.id,
timestamp: new Date(error.timestamp),
errorType: error.error_type,
errorMessage: error.error_message,
stackTrace: error.stack_trace,
})),
llmStats: {
totalCalls: result.llm_stats.total_calls,
totalInputTokens: result.llm_stats.total_input_tokens,
totalOutputTokens: result.llm_stats.total_output_tokens,
totalTokens: result.llm_stats.total_tokens,
totalDurationMs: result.llm_stats.total_duration_ms,
averageDurationMs: result.llm_stats.average_duration_ms,
},
} as MessageDetails,
}));
}
} catch (error) {
console.error('Failed to fetch message details:', error);
} finally {
setLoadingDetails({ ...loadingDetails, [messageId]: false });
}
}
}
const toggleTurnExpand = (turnId: string) => {
setExpandedTurnId((current) => (current === turnId ? null : turnId));
};
const toggleErrorExpand = (errorId: string) => {
@@ -342,127 +216,22 @@ function MonitoringPageContent() {
</div>
)}
{!loading &&
data &&
data.messages &&
data.messages.length > 0 && (
<div className="space-y-4">
{data.messages
.filter((msg) => {
// Filter out messages with empty content
const content = msg.messageContent?.trim();
return (
content && content !== '[]' && content !== '""'
);
})
.map((msg) => (
<div
key={msg.id}
className="border rounded-xl overflow-hidden transition-all duration-200"
>
{/* Message Header - Always Visible */}
<div
className="p-3 cursor-pointer hover:bg-accent transition-colors sm:p-5"
onClick={() => toggleMessageExpand(msg.id)}
>
<div className="flex items-start justify-between">
<div className="flex items-start flex-1">
{/* Expand Icon */}
<div className="mr-3 mt-0.5">
{expandedMessageId === msg.id ? (
<ChevronDown className="w-5 h-5 text-muted-foreground" />
) : (
<ChevronRight className="w-5 h-5 text-muted-foreground" />
)}
</div>
{!loading && data && conversationTurns.length > 0 && (
<ConversationTurnList
turns={conversationTurns}
expandedTurnId={expandedTurnId}
onToggleTurn={toggleTurnExpand}
/>
)}
{/* Message Info */}
<div className="flex-1">
<div className="flex items-center gap-2 mb-1">
<span className="text-xs text-muted-foreground font-mono">
ID: {msg.id}
</span>
</div>
<div className="flex items-center gap-2 mb-2">
<span className="font-medium text-sm text-foreground">
{msg.botName}
</span>
<span className="text-muted-foreground">
</span>
<span className="text-sm text-muted-foreground">
{msg.pipelineName}
</span>
{msg.runnerName && (
<>
<span className="text-muted-foreground">
</span>
<span className="text-sm text-muted-foreground">
{msg.runnerName}
</span>
</>
)}
</div>
<div className="text-base text-foreground">
<MessageContentRenderer
content={msg.messageContent}
maxLines={3}
/>
</div>
</div>
</div>
{/* Status and Timestamp */}
<div className="flex flex-col items-end gap-2 ml-4">
<span className="text-xs text-muted-foreground whitespace-nowrap">
{msg.timestamp.toLocaleString()}
</span>
<span
className={`text-xs px-2 py-1 rounded ${
msg.level === 'error'
? 'bg-red-100 text-red-800 dark:bg-red-900 dark:text-red-200'
: msg.level === 'warning'
? 'bg-yellow-100 text-yellow-800 dark:bg-yellow-900 dark:text-yellow-200'
: 'bg-green-100 text-green-800 dark:bg-green-900 dark:text-green-200'
}`}
>
{msg.level}
</span>
</div>
</div>
</div>
{/* Expanded Details */}
{expandedMessageId === msg.id && (
<div className="border-t p-4 bg-muted">
{loadingDetails[msg.id] && (
<div className="py-4 flex justify-center">
<LoadingSpinner size="sm" text="" />
</div>
)}
{!loadingDetails[msg.id] &&
messageDetails[msg.id] && (
<MessageDetailsCard
details={messageDetails[msg.id]}
/>
)}
</div>
)}
</div>
))}
{!loading && (!data || conversationTurns.length === 0) && (
<div className="flex flex-col items-center justify-center text-muted-foreground py-16 gap-2">
<MessageSquare className="h-[3rem] w-[3rem]" />
<div className="text-sm">
{t('monitoring.messageList.noMessages')}
</div>
)}
{!loading &&
(!data || !data.messages || data.messages.length === 0) && (
<div className="flex flex-col items-center justify-center text-muted-foreground py-16 gap-2">
<MessageSquare className="h-[3rem] w-[3rem]" />
<div className="text-sm">
{t('monitoring.messageList.noMessages')}
</div>
</div>
)}
</div>
)}
</div>
</TabsContent>
@@ -11,8 +11,10 @@ export interface MonitoringMessage {
level: 'info' | 'warning' | 'error' | 'debug';
platform?: string;
userId?: string;
userName?: string;
runnerName?: string;
variables?: string;
role?: 'user' | 'assistant' | string;
}
export interface LLMCall {
@@ -31,10 +33,29 @@ export interface LLMCall {
botName: string;
pipelineId: string;
pipelineName: string;
sessionId?: string;
errorMessage?: string;
messageId?: string;
}
export interface ToolCall {
id: string;
timestamp: Date;
toolName: string;
toolSource: 'native' | 'plugin' | 'mcp' | 'skill' | string;
duration: number;
status: 'success' | 'error';
botId: string;
botName: string;
pipelineId: string;
pipelineName: string;
sessionId?: string;
messageId?: string;
arguments?: string;
result?: string;
errorMessage?: string;
}
export interface EmbeddingCall {
id: string;
timestamp: Date;
@@ -199,6 +220,7 @@ export interface MonitoringData {
overview: OverviewMetrics;
messages: MonitoringMessage[];
llmCalls: LLMCall[];
toolCalls: ToolCall[];
embeddingCalls: EmbeddingCall[];
modelCalls: ModelCall[];
sessions: SessionInfo[];
@@ -208,6 +230,7 @@ export interface MonitoringData {
totalCount: {
messages: number;
llmCalls: number;
toolCalls?: number;
embeddingCalls: number;
sessions: number;
errors: number;
@@ -0,0 +1,294 @@
import {
ErrorLog,
LLMCall,
MonitoringMessage,
ToolCall,
} from '../types/monitoring';
type MessageRole = 'user' | 'assistant' | 'unknown';
export interface ConversationTurn {
id: string;
sessionId: string;
startedAt: Date;
lastActivityAt: Date;
botId: string;
botName: string;
pipelineId: string;
pipelineName: string;
runnerName?: string;
platform?: string;
userId?: string;
userName?: string;
userMessage?: MonitoringMessage;
assistantMessages: MonitoringMessage[];
messages: MonitoringMessage[];
llmCalls: LLMCall[];
toolCalls: ToolCall[];
errors: ErrorLog[];
status: 'success' | 'error' | 'pending';
level: 'info' | 'warning' | 'error' | 'debug';
inputTokens: number;
outputTokens: number;
totalTokens: number;
totalDuration: number;
totalToolDuration: number;
}
function normalizeRole(
message: MonitoringMessage,
llmMessageIds: Set<string>,
): MessageRole {
const role = message.role?.toLowerCase();
if (role === 'user' || role === 'assistant') {
return role;
}
if (llmMessageIds.has(message.id)) {
return 'user';
}
return 'unknown';
}
export function hasRenderableMessageContent(content?: string): boolean {
const trimmed = content?.trim();
if (!trimmed || trimmed === '[]' || trimmed === '""') {
return false;
}
try {
const parsed = JSON.parse(trimmed);
if (typeof parsed === 'string') {
return parsed.trim().length > 0;
}
if (Array.isArray(parsed)) {
return parsed.some(
(component) =>
typeof component !== 'object' ||
component === null ||
component.type !== 'Source',
);
}
} catch {
return true;
}
return true;
}
function createTurn(message: MonitoringMessage): ConversationTurn {
return {
id: message.id,
sessionId: message.sessionId,
startedAt: message.timestamp,
lastActivityAt: message.timestamp,
botId: message.botId,
botName: message.botName,
pipelineId: message.pipelineId,
pipelineName: message.pipelineName,
runnerName: message.runnerName,
platform: message.platform,
userId: message.userId,
userName: message.userName,
assistantMessages: [],
messages: [],
llmCalls: [],
toolCalls: [],
errors: [],
status: message.status,
level: message.level,
inputTokens: 0,
outputTokens: 0,
totalTokens: 0,
totalDuration: 0,
totalToolDuration: 0,
};
}
function updateTurnActivity(turn: ConversationTurn, timestamp: Date) {
if (timestamp.getTime() > turn.lastActivityAt.getTime()) {
turn.lastActivityAt = timestamp;
}
}
function addMessageToTurn(
turn: ConversationTurn,
message: MonitoringMessage,
role: MessageRole,
) {
turn.messages.push(message);
updateTurnActivity(turn, message.timestamp);
if (message.level === 'error') {
turn.level = 'error';
} else if (message.level === 'warning' && turn.level !== 'error') {
turn.level = 'warning';
}
if (message.status === 'error') {
turn.status = 'error';
} else if (message.status === 'pending' && turn.status !== 'error') {
turn.status = 'pending';
}
if (role === 'assistant') {
turn.assistantMessages.push(message);
return;
}
if (!turn.userMessage) {
turn.userMessage = message;
turn.userId = message.userId ?? turn.userId;
turn.userName = message.userName ?? turn.userName;
return;
}
turn.assistantMessages.push(message);
}
function findTurnBySessionTime(
sessionTurns: Map<string, ConversationTurn[]>,
sessionId: string | undefined,
timestamp: Date,
): ConversationTurn | undefined {
if (!sessionId) {
return undefined;
}
const turns = sessionTurns.get(sessionId);
if (!turns?.length) {
return undefined;
}
let nearest = turns[0];
const targetTime = timestamp.getTime();
for (const turn of turns) {
if (turn.startedAt.getTime() <= targetTime) {
nearest = turn;
} else {
break;
}
}
return nearest;
}
export function buildConversationTurns(
messages: MonitoringMessage[],
llmCalls: LLMCall[],
errors: ErrorLog[],
toolCalls: ToolCall[] = [],
): ConversationTurn[] {
const activityMessageIds = new Set([
...llmCalls
.map((call) => call.messageId)
.filter((messageId): messageId is string => Boolean(messageId)),
...toolCalls
.map((call) => call.messageId)
.filter((messageId): messageId is string => Boolean(messageId)),
]);
const visibleMessages = messages
.filter((message) => hasRenderableMessageContent(message.messageContent))
.sort((a, b) => a.timestamp.getTime() - b.timestamp.getTime());
const sessionTurns = new Map<string, ConversationTurn[]>();
const lastTurnBySession = new Map<string, ConversationTurn>();
const messageIdToTurn = new Map<string, ConversationTurn>();
for (const message of visibleMessages) {
const role = normalizeRole(message, activityMessageIds);
const previousTurn = lastTurnBySession.get(message.sessionId);
const shouldStartTurn = role === 'user' || !previousTurn;
const turn = shouldStartTurn ? createTurn(message) : previousTurn;
if (shouldStartTurn) {
const turns = sessionTurns.get(message.sessionId) ?? [];
turns.push(turn);
sessionTurns.set(message.sessionId, turns);
lastTurnBySession.set(message.sessionId, turn);
}
addMessageToTurn(turn, message, role);
messageIdToTurn.set(message.id, turn);
}
const allTurns = Array.from(sessionTurns.values()).flat();
for (const call of llmCalls) {
const turn =
(call.messageId ? messageIdToTurn.get(call.messageId) : undefined) ??
findTurnBySessionTime(sessionTurns, call.sessionId, call.timestamp);
if (!turn) {
continue;
}
turn.llmCalls.push(call);
turn.inputTokens += call.tokens.input;
turn.outputTokens += call.tokens.output;
turn.totalTokens += call.tokens.total;
turn.totalDuration += call.duration;
updateTurnActivity(turn, call.timestamp);
if (call.status === 'error') {
turn.status = 'error';
turn.level = 'error';
}
}
for (const call of toolCalls) {
const turn =
(call.messageId ? messageIdToTurn.get(call.messageId) : undefined) ??
findTurnBySessionTime(sessionTurns, call.sessionId, call.timestamp);
if (!turn) {
continue;
}
turn.toolCalls.push(call);
turn.totalToolDuration += call.duration;
updateTurnActivity(turn, call.timestamp);
if (call.status === 'error') {
turn.status = 'error';
turn.level = 'error';
}
}
for (const error of errors) {
const turn =
(error.messageId ? messageIdToTurn.get(error.messageId) : undefined) ??
findTurnBySessionTime(sessionTurns, error.sessionId, error.timestamp);
if (!turn) {
continue;
}
turn.errors.push(error);
turn.status = 'error';
turn.level = 'error';
updateTurnActivity(turn, error.timestamp);
}
for (const turn of allTurns) {
turn.messages.sort((a, b) => a.timestamp.getTime() - b.timestamp.getTime());
turn.assistantMessages.sort(
(a, b) => a.timestamp.getTime() - b.timestamp.getTime(),
);
turn.llmCalls.sort((a, b) => a.timestamp.getTime() - b.timestamp.getTime());
turn.toolCalls.sort(
(a, b) => a.timestamp.getTime() - b.timestamp.getTime(),
);
turn.errors.sort((a, b) => a.timestamp.getTime() - b.timestamp.getTime());
}
return allTurns.sort(
(a, b) => b.lastActivityAt.getTime() - a.lastActivityAt.getTime(),
);
}
@@ -12,63 +12,15 @@ import {
Monitor,
} from 'lucide-react';
import { useMonitoringData } from '@/app/home/monitoring/hooks/useMonitoringData';
import { MessageContentRenderer } from '@/app/home/monitoring/components/MessageContentRenderer';
import { ConversationTurnList } from '@/app/home/monitoring/components/ConversationTurnList';
import { buildConversationTurns } from '@/app/home/monitoring/utils/conversationTurns';
import { LoadingSpinner } from '@/components/ui/loading-spinner';
import { httpClient } from '@/app/infra/http/HttpClient';
import { MessageDetails } from '@/app/home/monitoring/types/monitoring';
import { parseUTCTimestamp } from '@/app/home/monitoring/utils/dateUtils';
interface PipelineMonitoringTabProps {
pipelineId: string;
onNavigateToMonitoring?: () => void;
}
interface RawMessageData {
id: string;
timestamp: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
message_content: string;
session_id: string;
status: string;
level: string;
platform: string;
user_id: string;
runner_name: string;
variables: Record<string, unknown>;
}
interface RawLLMCallData {
id: string;
timestamp: string;
model_name: string;
status: string;
duration: number;
error_message: string | null;
input_tokens: number;
output_tokens: number;
total_tokens: number;
}
interface RawLLMStatsData {
total_calls: number;
total_input_tokens: number;
total_output_tokens: number;
total_tokens: number;
total_duration_ms: number;
average_duration_ms: number;
}
interface RawErrorData {
id: string;
timestamp: string;
error_type: string;
error_message: string;
stack_trace: string | null;
}
export default function PipelineMonitoringTab({
pipelineId,
onNavigateToMonitoring,
@@ -88,98 +40,24 @@ export default function PipelineMonitoringTab({
const { data, loading, refetch } = useMonitoringData(filterState);
const [expandedMessageId, setExpandedMessageId] = useState<string | null>(
null,
);
const [messageDetails, setMessageDetails] = useState<
Record<string, MessageDetails>
>({});
const [loadingDetails, setLoadingDetails] = useState<Record<string, boolean>>(
{},
const conversationTurns = useMemo(
() =>
data
? buildConversationTurns(
data.messages,
data.llmCalls,
data.errors,
data.toolCalls,
)
: [],
[data],
);
const [expandedTurnId, setExpandedTurnId] = useState<string | null>(null);
const [expandedErrorId, setExpandedErrorId] = useState<string | null>(null);
const [activeTab, setActiveTab] = useState<string>('messages');
const toggleMessageExpand = async (messageId: string) => {
if (expandedMessageId === messageId) {
setExpandedMessageId(null);
} else {
setExpandedMessageId(messageId);
if (!messageDetails[messageId]) {
setLoadingDetails((prev) => ({ ...prev, [messageId]: true }));
try {
const result = await httpClient.get<{
message_id: string;
found: boolean;
message: RawMessageData | null;
llm_calls: RawLLMCallData[];
llm_stats: RawLLMStatsData;
errors: RawErrorData[];
}>(`/api/v1/monitoring/messages/${messageId}/details`);
if (result) {
setMessageDetails((prev) => ({
...prev,
[messageId]: {
messageId: result.message_id,
found: result.found,
message: result.message
? {
id: result.message.id,
timestamp: parseUTCTimestamp(result.message.timestamp),
botId: result.message.bot_id,
botName: result.message.bot_name,
pipelineId: result.message.pipeline_id,
pipelineName: result.message.pipeline_name,
messageContent: result.message.message_content,
sessionId: result.message.session_id,
status: result.message.status,
level: result.message.level,
platform: result.message.platform,
userId: result.message.user_id,
runnerName: result.message.runner_name,
variables: result.message.variables,
}
: undefined,
llmCalls: result.llm_calls.map((call: RawLLMCallData) => ({
id: call.id,
timestamp: parseUTCTimestamp(call.timestamp),
modelName: call.model_name,
status: call.status,
duration: call.duration,
errorMessage: call.error_message,
tokens: {
input: call.input_tokens || 0,
output: call.output_tokens || 0,
total: call.total_tokens || 0,
},
})),
errors: result.errors.map((error: RawErrorData) => ({
id: error.id,
timestamp: parseUTCTimestamp(error.timestamp),
errorType: error.error_type,
errorMessage: error.error_message,
stackTrace: error.stack_trace,
})),
llmStats: {
totalCalls: result.llm_stats.total_calls,
totalInputTokens: result.llm_stats.total_input_tokens,
totalOutputTokens: result.llm_stats.total_output_tokens,
totalTokens: result.llm_stats.total_tokens,
totalDurationMs: result.llm_stats.total_duration_ms,
averageDurationMs: result.llm_stats.average_duration_ms,
},
} as MessageDetails,
}));
}
} catch (error) {
console.error('Failed to fetch message details:', error);
} finally {
setLoadingDetails((prev) => ({ ...prev, [messageId]: false }));
}
}
}
const toggleTurnExpand = (turnId: string) => {
setExpandedTurnId((current) => (current === turnId ? null : turnId));
};
const toggleErrorExpand = (errorId: string) => {
@@ -190,12 +68,16 @@ export default function PipelineMonitoringTab({
}
};
const jumpToMessage = async (messageId: string) => {
const jumpToMessage = (messageId: string) => {
setActiveTab('messages');
// Small delay to ensure tab transition completes before expanding
setTimeout(() => {
toggleMessageExpand(messageId);
}, 100);
const turn = conversationTurns.find((item) =>
item.messages.some((message) => message.id === messageId),
);
if (turn) {
setExpandedTurnId(turn.id);
}
};
return (
@@ -295,142 +177,22 @@ export default function PipelineMonitoringTab({
</div>
)}
{!loading && data && data.messages && data.messages.length > 0 && (
<div className="space-y-3">
{data.messages
.filter((msg) => {
const content = msg.messageContent?.trim();
return content && content !== '[]' && content !== '""';
})
.map((msg) => (
<div
key={msg.id}
className="border border-gray-200 dark:border-gray-700 rounded-lg overflow-hidden hover:shadow-md transition-all duration-200"
>
<div
className="p-4 cursor-pointer hover:bg-gray-50 dark:hover:bg-gray-800/50 transition-colors"
onClick={() => toggleMessageExpand(msg.id)}
>
<div className="flex items-start justify-between">
<div className="flex items-start flex-1">
<div className="mr-2 mt-0.5">
{expandedMessageId === msg.id ? (
<ChevronDown className="w-4 h-4 text-gray-500" />
) : (
<ChevronRight className="w-4 h-4 text-gray-500" />
)}
</div>
<div className="flex-1">
<div className="flex items-center gap-2 mb-1">
<span
className={`text-xs px-2 py-0.5 rounded ${
msg.status === 'success'
? 'bg-green-100 text-green-800 dark:bg-green-900 dark:text-green-200'
: msg.status === 'error'
? 'bg-red-100 text-red-800 dark:bg-red-900 dark:text-red-200'
: 'bg-yellow-100 text-yellow-800 dark:bg-yellow-900 dark:text-yellow-200'
}`}
>
{msg.status}
</span>
<span className="text-xs text-gray-500 dark:text-gray-400">
{msg.botName}
</span>
</div>
<div className="text-sm text-gray-700 dark:text-gray-300 line-clamp-2">
<MessageContentRenderer
content={msg.messageContent}
/>
</div>
</div>
</div>
<span className="text-xs text-gray-500 dark:text-gray-400 whitespace-nowrap ml-4">
{msg.timestamp.toLocaleString()}
</span>
</div>
</div>
{expandedMessageId === msg.id && (
<div className="border-t border-gray-200 dark:border-gray-700 p-4 bg-gray-50 dark:bg-gray-900">
{loadingDetails[msg.id] && (
<div className="flex justify-center py-8">
<LoadingSpinner
text={t('monitoring.messageList.loading')}
/>
</div>
)}
{!loadingDetails[msg.id] &&
messageDetails[msg.id] && (
<div className="space-y-4">
{messageDetails[msg.id].errors.length > 0 && (
<div className="bg-red-50 dark:bg-red-900/20 rounded-lg p-3">
<h4 className="text-sm font-semibold text-red-700 dark:text-red-400 mb-2">
{t('monitoring.errors.errorMessage')}
</h4>
{messageDetails[msg.id].errors.map(
(error) => (
<div
key={error.id}
className="text-sm space-y-2"
>
<div className="text-red-600 dark:text-red-400">
{error.errorType}:{' '}
{error.errorMessage}
</div>
{error.stackTrace && (
<pre className="text-xs text-gray-600 dark:text-gray-400 overflow-auto max-h-40 bg-white dark:bg-gray-900 p-2 rounded whitespace-pre-wrap break-words">
{error.stackTrace}
</pre>
)}
</div>
),
)}
</div>
)}
{messageDetails[msg.id].llmCalls.length > 0 && (
<div className="bg-blue-50 dark:bg-blue-900/20 rounded-lg p-3">
<h4 className="text-sm font-semibold text-blue-700 dark:text-blue-400 mb-2">
{t('monitoring.tabs.modelCalls')} (
{messageDetails[msg.id].llmCalls.length})
</h4>
<div className="text-xs text-gray-600 dark:text-gray-400 space-y-1">
<div>
{t('monitoring.llmCalls.totalTokens')}:{' '}
{
messageDetails[msg.id].llmStats
.totalTokens
}
</div>
<div>
{t('monitoring.llmCalls.duration')}:{' '}
{messageDetails[
msg.id
].llmStats.totalDurationMs.toFixed(0)}
ms
</div>
</div>
</div>
)}
</div>
)}
</div>
)}
</div>
))}
</div>
{!loading && data && conversationTurns.length > 0 && (
<ConversationTurnList
turns={conversationTurns}
expandedTurnId={expandedTurnId}
onToggleTurn={toggleTurnExpand}
/>
)}
{!loading &&
(!data || !data.messages || data.messages.length === 0) && (
<div className="text-center text-gray-500 dark:text-gray-400 py-16">
<MessageCircle className="w-16 h-16 mx-auto mb-4 text-gray-300 dark:text-gray-600" />
<p className="text-base font-medium">
{t('monitoring.messageList.noMessages')}
</p>
</div>
)}
{!loading && (!data || conversationTurns.length === 0) && (
<div className="text-center text-gray-500 dark:text-gray-400 py-16">
<MessageCircle className="w-16 h-16 mx-auto mb-4 text-gray-300 dark:text-gray-600" />
<p className="text-base font-medium">
{t('monitoring.messageList.noMessages')}
</p>
</div>
)}
</TabsContent>
{/* Errors Tab */}
+36
View File
@@ -671,6 +671,21 @@ export class BackendClient extends BaseHttpClient {
);
}
public getMcpServerLogs(
serverName: string,
limit: number = 200,
level?: string,
): Promise<{ logs: PluginLogEntry[] }> {
const params = new URLSearchParams();
params.set('limit', String(limit));
if (level) {
params.set('level', level);
}
return this.get(
`/api/v1/mcp/servers/${encodeURIComponent(serverName)}/logs?${params.toString()}`,
);
}
public getPluginAssetURL(
author: string,
name: string,
@@ -1199,8 +1214,10 @@ export class BackendClient extends BaseHttpClient {
level: string;
platform?: string;
user_id?: string;
user_name?: string;
runner_name?: string;
variables?: string;
role?: string;
}>;
llmCalls: Array<{
id: string;
@@ -1216,9 +1233,27 @@ export class BackendClient extends BaseHttpClient {
bot_name: string;
pipeline_id: string;
pipeline_name: string;
session_id?: string;
error_message?: string;
message_id?: string;
}>;
toolCalls: Array<{
id: string;
timestamp: string;
tool_name: string;
tool_source: string;
duration: number;
status: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
session_id?: string;
message_id?: string;
arguments?: string;
result?: string;
error_message?: string;
}>;
embeddingCalls: Array<{
id: string;
timestamp: string;
@@ -1263,6 +1298,7 @@ export class BackendClient extends BaseHttpClient {
totalCount: {
messages: number;
llmCalls: number;
toolCalls?: number;
embeddingCalls: number;
sessions: number;
errors: number;
+29
View File
@@ -828,6 +828,12 @@ const enUS = {
tabTools: 'Tools',
tabResources: 'Resources',
tabDocs: 'Docs',
tabLogs: 'Logs',
logsLevelAll: 'All levels',
logsRefresh: 'Refresh',
logsAutoRefresh: 'Auto refresh',
logsEmpty:
'No logs yet. Runtime logs from the MCP server will appear here.',
noReadme: 'No documentation available',
parseResultFailed: 'Failed to parse test result',
noResultReturned: 'Test returned no result',
@@ -1329,6 +1335,20 @@ const enUS = {
level: 'Level',
runner: 'Runner',
viewConversation: 'View Conversation',
turns: '{{count}} conversation turns',
userMessage: 'User',
noUserMessage: 'No user input recorded',
assistantMessage: 'Assistant',
assistantMessageCount: 'Assistant +{{count}}',
noAssistantMessage: 'No assistant reply recorded',
messageCount: 'Messages',
conversationTrace: 'Conversation Trace',
noLlmCalls: 'No model calls recorded',
roles: {
user: 'User',
assistant: 'Assistant',
message: 'Message',
},
},
llmCalls: {
title: 'LLM Calls',
@@ -1343,6 +1363,15 @@ const enUS = {
avgDuration: 'Avg Duration',
calls: 'Calls',
},
toolCalls: {
title: 'Tool Calls',
totalCalls: 'Calls',
duration: 'Tool Duration',
errorCalls: 'Failed Calls',
arguments: 'Arguments',
result: 'Result',
noToolCalls: 'No tool calls recorded',
},
tokens: {
totalTokens: 'Total Tokens',
inputTokens: 'Input Tokens',
+29
View File
@@ -842,6 +842,12 @@ const esES = {
tabTools: 'Herramientas',
tabResources: 'Recursos',
tabDocs: 'Documentación',
tabLogs: 'Registros',
logsLevelAll: 'Todos los niveles',
logsRefresh: 'Actualizar',
logsAutoRefresh: 'Actualización automática',
logsEmpty:
'Aún no hay registros. Los registros de ejecución del servidor MCP aparecerán aquí.',
noReadme: 'No hay documentación disponible',
parseResultFailed: 'Error al analizar el resultado de la prueba',
noResultReturned: 'La prueba no devolvió resultados',
@@ -1363,6 +1369,20 @@ const esES = {
level: 'Nivel',
runner: 'Ejecutor',
viewConversation: 'Ver conversación',
turns: '{{count}} turnos de conversación',
userMessage: 'Usuario',
noUserMessage: 'No se registró entrada del usuario',
assistantMessage: 'Asistente',
assistantMessageCount: 'Asistente +{{count}}',
noAssistantMessage: 'No se registró respuesta del asistente',
messageCount: 'Mensajes',
conversationTrace: 'Flujo de conversación',
noLlmCalls: 'No se registraron llamadas al modelo',
roles: {
user: 'Usuario',
assistant: 'Asistente',
message: 'Mensaje',
},
},
llmCalls: {
title: 'Llamadas LLM',
@@ -1377,6 +1397,15 @@ const esES = {
avgDuration: 'Duración promedio',
calls: 'Llamadas',
},
toolCalls: {
title: 'Llamadas de herramientas',
totalCalls: 'Llamadas',
duration: 'Duración de herramientas',
errorCalls: 'Llamadas fallidas',
arguments: 'Argumentos',
result: 'Resultado',
noToolCalls: 'No se registraron llamadas de herramientas',
},
tokens: {
totalTokens: 'Tokens totales',
inputTokens: 'Tokens de entrada',
+28
View File
@@ -834,6 +834,11 @@ const jaJP = {
tabTools: 'ツール',
tabResources: 'リソース',
tabDocs: 'ドキュメント',
tabLogs: 'ログ',
logsLevelAll: 'すべてのレベル',
logsRefresh: '更新',
logsAutoRefresh: '自動更新',
logsEmpty: 'ログはありません。MCPサーバーの実行ログがここに表示されます。',
noReadme: 'ドキュメントがありません',
parseResultFailed: 'テスト結果の解析に失敗しました',
noResultReturned: 'テスト結果が返されませんでした',
@@ -1336,6 +1341,20 @@ const jaJP = {
level: 'レベル',
runner: 'ランナー',
viewConversation: '会話詳細を表示',
turns: '{{count}} 会話ターン',
userMessage: 'ユーザー',
noUserMessage: 'ユーザー入力は記録されていません',
assistantMessage: 'アシスタント',
assistantMessageCount: 'アシスタント +{{count}}',
noAssistantMessage: 'アシスタントの返信は記録されていません',
messageCount: 'メッセージ数',
conversationTrace: '会話トレース',
noLlmCalls: 'モデル呼び出しは記録されていません',
roles: {
user: 'ユーザー',
assistant: 'アシスタント',
message: 'メッセージ',
},
},
llmCalls: {
title: 'LLM呼び出し',
@@ -1350,6 +1369,15 @@ const jaJP = {
avgDuration: '平均期間',
calls: '呼び出し',
},
toolCalls: {
title: 'ツール呼び出し',
totalCalls: '呼び出し',
duration: 'ツール時間',
errorCalls: '失敗した呼び出し',
arguments: '引数',
result: '結果',
noToolCalls: 'ツール呼び出しは記録されていません',
},
tokens: {
totalTokens: '総トークン数',
inputTokens: '入力トークン',
+29
View File
@@ -839,6 +839,12 @@ const ruRU = {
tabTools: 'Инструменты',
tabResources: 'Ресурсы',
tabDocs: 'Документация',
tabLogs: 'Журнал',
logsLevelAll: 'Все уровни',
logsRefresh: 'Обновить',
logsAutoRefresh: 'Автообновление',
logsEmpty:
'Журналов пока нет. Здесь будут отображаться журналы выполнения MCP-сервера.',
noReadme: 'Документация отсутствует',
parseResultFailed: 'Не удалось разобрать результат теста',
noResultReturned: 'Тест не вернул результат',
@@ -1339,6 +1345,20 @@ const ruRU = {
level: 'Уровень',
runner: 'Обработчик',
viewConversation: 'Просмотр диалога',
turns: '{{count}} диалоговых ходов',
userMessage: 'Пользователь',
noUserMessage: 'Ввод пользователя не записан',
assistantMessage: 'Ассистент',
assistantMessageCount: 'Ассистент +{{count}}',
noAssistantMessage: 'Ответ ассистента не записан',
messageCount: 'Сообщения',
conversationTrace: 'Ход диалога',
noLlmCalls: 'Вызовы модели не записаны',
roles: {
user: 'Пользователь',
assistant: 'Ассистент',
message: 'Сообщение',
},
},
llmCalls: {
title: 'Вызовы LLM',
@@ -1353,6 +1373,15 @@ const ruRU = {
avgDuration: 'Средняя длительность',
calls: 'Вызовы',
},
toolCalls: {
title: 'Вызовы инструментов',
totalCalls: 'Вызовы',
duration: 'Длительность инструментов',
errorCalls: 'Неудачные вызовы',
arguments: 'Аргументы',
result: 'Результат',
noToolCalls: 'Вызовы инструментов не записаны',
},
tokens: {
totalTokens: 'Всего токенов',
inputTokens: 'Входные токены',
+28
View File
@@ -817,6 +817,11 @@ const thTH = {
tabTools: 'เครื่องมือ',
tabResources: 'ทรัพยากร',
tabDocs: 'เอกสาร',
tabLogs: 'บันทึก',
logsLevelAll: 'ทุกระดับ',
logsRefresh: 'รีเฟรช',
logsAutoRefresh: 'รีเฟรชอัตโนมัติ',
logsEmpty: 'ยังไม่มีบันทึก บันทึกการทำงานของ MCP Server จะแสดงที่นี่',
noReadme: 'ไม่มีเอกสาร',
parseResultFailed: 'ไม่สามารถแยกวิเคราะห์ผลการทดสอบได้',
noResultReturned: 'การทดสอบไม่ส่งผลลัพธ์กลับมา',
@@ -1307,6 +1312,20 @@ const thTH = {
level: 'ระดับ',
runner: 'ตัวประมวลผล',
viewConversation: 'ดูการสนทนา',
turns: '{{count}} รอบการสนทนา',
userMessage: 'ผู้ใช้',
noUserMessage: 'ยังไม่มีการบันทึกข้อความจากผู้ใช้',
assistantMessage: 'ผู้ช่วย',
assistantMessageCount: 'ผู้ช่วย +{{count}}',
noAssistantMessage: 'ยังไม่มีการบันทึกคำตอบจากผู้ช่วย',
messageCount: 'จำนวนข้อความ',
conversationTrace: 'ลำดับการสนทนา',
noLlmCalls: 'ยังไม่มีการบันทึกการเรียกโมเดล',
roles: {
user: 'ผู้ใช้',
assistant: 'ผู้ช่วย',
message: 'ข้อความ',
},
},
llmCalls: {
title: 'การเรียก LLM',
@@ -1321,6 +1340,15 @@ const thTH = {
avgDuration: 'ระยะเวลาเฉลี่ย',
calls: 'การเรียก',
},
toolCalls: {
title: 'การเรียกใช้เครื่องมือ',
totalCalls: 'การเรียก',
duration: 'ระยะเวลาเครื่องมือ',
errorCalls: 'การเรียกที่ล้มเหลว',
arguments: 'อาร์กิวเมนต์',
result: 'ผลลัพธ์',
noToolCalls: 'ยังไม่มีการบันทึกการเรียกใช้เครื่องมือ',
},
tokens: {
totalTokens: 'Token ทั้งหมด',
inputTokens: 'Token อินพุต',
+29
View File
@@ -832,6 +832,12 @@ const viVN = {
tabTools: 'Công cụ',
tabResources: 'Tài nguyên',
tabDocs: 'Tài liệu',
tabLogs: 'Nhật ký',
logsLevelAll: 'Tất cả cấp độ',
logsRefresh: 'Làm mới',
logsAutoRefresh: 'Tự động làm mới',
logsEmpty:
'Chưa có nhật ký. Nhật ký chạy của MCP Server sẽ hiển thị ở đây.',
noReadme: 'Không có tài liệu',
parseResultFailed: 'Phân tích kết quả kiểm tra thất bại',
noResultReturned: 'Kiểm tra không trả về kết quả',
@@ -1332,6 +1338,20 @@ const viVN = {
level: 'Mức',
runner: 'Trình chạy',
viewConversation: 'Xem cuộc trò chuyện',
turns: '{{count}} lượt hội thoại',
userMessage: 'Người dùng',
noUserMessage: 'Chưa ghi nhận đầu vào người dùng',
assistantMessage: 'Trợ lý',
assistantMessageCount: 'Trợ lý +{{count}}',
noAssistantMessage: 'Chưa ghi nhận phản hồi của trợ lý',
messageCount: 'Số tin nhắn',
conversationTrace: 'Luồng hội thoại',
noLlmCalls: 'Chưa ghi nhận lệnh gọi mô hình',
roles: {
user: 'Người dùng',
assistant: 'Trợ lý',
message: 'Tin nhắn',
},
},
llmCalls: {
title: 'Cuộc gọi LLM',
@@ -1346,6 +1366,15 @@ const viVN = {
avgDuration: 'Thời lượng trung bình',
calls: 'Cuộc gọi',
},
toolCalls: {
title: 'Lượt gọi công cụ',
totalCalls: 'Lượt gọi',
duration: 'Thời lượng công cụ',
errorCalls: 'Lượt gọi thất bại',
arguments: 'Tham số',
result: 'Kết quả',
noToolCalls: 'Chưa ghi nhận lượt gọi công cụ',
},
tokens: {
totalTokens: 'Tổng số Token',
inputTokens: 'Token đầu vào',
+28
View File
@@ -794,6 +794,11 @@ const zhHans = {
tabTools: '工具',
tabResources: '资源',
tabDocs: '文档',
tabLogs: '日志',
logsLevelAll: '全部级别',
logsRefresh: '刷新',
logsAutoRefresh: '自动刷新',
logsEmpty: '暂无日志。MCP 服务器的运行日志会显示在这里。',
noReadme: '暂无文档',
parseResultFailed: '解析测试结果失败',
noResultReturned: '测试未返回结果',
@@ -1266,6 +1271,20 @@ const zhHans = {
level: '级别',
runner: '执行器',
viewConversation: '显示对话详情',
turns: '{{count}} 轮对话',
userMessage: '用户',
noUserMessage: '未记录用户输入',
assistantMessage: '助手',
assistantMessageCount: '助手 +{{count}}',
noAssistantMessage: '未记录助手回复',
messageCount: '消息数',
conversationTrace: '消息链路',
noLlmCalls: '未记录模型调用',
roles: {
user: '用户',
assistant: '助手',
message: '消息',
},
},
llmCalls: {
title: 'LLM调用',
@@ -1280,6 +1299,15 @@ const zhHans = {
avgDuration: '平均耗时',
calls: '调用次数',
},
toolCalls: {
title: '工具调用',
totalCalls: '调用次数',
duration: '工具耗时',
errorCalls: '失败次数',
arguments: '参数',
result: '结果',
noToolCalls: '未记录工具调用',
},
tokens: {
totalTokens: '总 Token 数',
inputTokens: '输入 Token',
+28
View File
@@ -793,6 +793,11 @@ const zhHant = {
tabTools: '工具',
tabResources: '資源',
tabDocs: '文件',
tabLogs: '日誌',
logsLevelAll: '全部級別',
logsRefresh: '重新整理',
logsAutoRefresh: '自動重新整理',
logsEmpty: '暫無日誌。MCP 服務器的運行日誌會顯示在這裡。',
noReadme: '暫無文件',
parseResultFailed: '解析測試結果失敗',
noResultReturned: '測試未返回結果',
@@ -1264,6 +1269,20 @@ const zhHant = {
level: '級別',
runner: '執行器',
viewConversation: '顯示對話詳情',
turns: '{{count}} 輪對話',
userMessage: '使用者',
noUserMessage: '未記錄使用者輸入',
assistantMessage: '助手',
assistantMessageCount: '助手 +{{count}}',
noAssistantMessage: '未記錄助手回覆',
messageCount: '訊息數',
conversationTrace: '訊息鏈路',
noLlmCalls: '未記錄模型呼叫',
roles: {
user: '使用者',
assistant: '助手',
message: '訊息',
},
},
llmCalls: {
title: 'LLM呼叫',
@@ -1278,6 +1297,15 @@ const zhHant = {
avgDuration: '平均持續時間',
calls: '呼叫次數',
},
toolCalls: {
title: '工具呼叫',
totalCalls: '呼叫次數',
duration: '工具耗時',
errorCalls: '失敗次數',
arguments: '參數',
result: '結果',
noToolCalls: '未記錄工具呼叫',
},
tokens: {
totalTokens: '總 Token 數',
inputTokens: '輸入 Token',
@@ -0,0 +1,179 @@
import { expect, test } from '@playwright/test';
import { installLangBotApiMocks } from './fixtures/langbot-api';
const botId = 'bot-tool-timeline';
const sessionId = 'person-tool-timeline-user';
const botName = 'Tool Timeline Bot';
const pipelineId = 'pipeline-tool-timeline';
const pipelineName = 'Tool Timeline Pipeline';
function at(minute: number, second = 0) {
return `2026-07-02T10:${String(minute).padStart(2, '0')}:${String(
second,
).padStart(2, '0')}Z`;
}
function sessionMessage(
id: string,
role: 'user' | 'assistant',
minute: number,
content: string,
) {
return {
id,
timestamp: at(minute),
bot_id: botId,
bot_name: botName,
pipeline_id: pipelineId,
pipeline_name: pipelineName,
message_content: content,
session_id: sessionId,
status: 'success',
level: 'info',
platform: role === 'user' ? 'person' : 'bot',
user_id: 'timeline-user',
user_name: 'Timeline User',
runner_name: role === 'assistant' ? 'local-agent' : null,
variables: '{}',
role,
};
}
function toolCall(
id: string,
minute: number,
toolName: string,
duration: number,
status: 'success' | 'error' = 'success',
) {
return {
id,
timestamp: at(minute, 30),
tool_name: toolName,
tool_source: 'native',
duration,
status,
bot_id: botId,
bot_name: botName,
pipeline_id: pipelineId,
pipeline_name: pipelineName,
session_id: sessionId,
message_id: 'user-message',
arguments: JSON.stringify({ target: toolName }),
result: status === 'success' ? JSON.stringify({ ok: true }) : null,
error_message: status === 'error' ? 'Tool execution failed' : null,
};
}
test.describe('bot session monitor tool timeline', () => {
test('renders tool calls as left-side agent events interleaved with messages', async ({
page,
}) => {
await installLangBotApiMocks(page, {
authenticated: true,
monitoringSessions: [
{
session_id: sessionId,
bot_id: botId,
bot_name: botName,
pipeline_id: pipelineId,
pipeline_name: pipelineName,
message_count: 3,
start_time: at(0),
last_activity: at(4),
is_active: true,
platform: 'person',
user_id: 'timeline-user',
user_name: 'Timeline User',
},
],
sessionMessages: {
[sessionId]: [
sessionMessage('user-message', 'user', 0, 'Need a timeline check'),
sessionMessage(
'assistant-step-1',
'assistant',
2,
'Agent step 1: inspected repository files',
),
sessionMessage(
'assistant-step-2',
'assistant',
4,
'Agent step 2: test suite finished',
),
],
},
sessionAnalyses: {
[sessionId]: {
session_id: sessionId,
found: true,
tool_calls: [
toolCall('tool-repo-read', 1, 'repo_file_read', 80),
toolCall('tool-test-run', 3, 'run_test_suite', 140),
],
},
},
});
await page.goto(`/home/bots?id=${botId}`);
await page.getByRole('tab', { name: /Sessions/ }).click();
await page.getByRole('button', { name: /Timeline User/ }).click();
await expect(page.getByText('Need a timeline check')).toBeVisible();
await expect(
page.getByText('repo_file_read', { exact: true }),
).toBeVisible();
await expect(
page.getByText('Agent step 1: inspected repository files'),
).toBeVisible();
await expect(
page.getByText('run_test_suite', { exact: true }),
).toBeVisible();
await expect(
page.getByText('Agent step 2: test suite finished'),
).toBeVisible();
await expect(page.getByText('{"target":"repo_file_read"}')).toHaveCount(0);
await expect(page.getByText('{"ok":true}')).toHaveCount(0);
await expect(
page.locator('div.flex.justify-start').filter({
hasText: 'repo_file_read',
}),
).toHaveCount(1);
await expect(
page.locator('div.flex.justify-start').filter({
hasText: 'run_test_suite',
}),
).toHaveCount(1);
await expect(
page.locator('div.flex.justify-end').filter({
hasText: 'repo_file_read',
}),
).toHaveCount(0);
await expect(
page.locator('div.flex.justify-end').filter({
hasText: 'run_test_suite',
}),
).toHaveCount(0);
const text = await page.locator('body').innerText();
expect(text.indexOf('Need a timeline check')).toBeLessThan(
text.indexOf('repo_file_read'),
);
expect(text.indexOf('repo_file_read')).toBeLessThan(
text.indexOf('Agent step 1: inspected repository files'),
);
expect(
text.indexOf('Agent step 1: inspected repository files'),
).toBeLessThan(text.indexOf('run_test_suite'));
expect(text.indexOf('run_test_suite')).toBeLessThan(
text.indexOf('Agent step 2: test suite finished'),
);
await page.getByText('repo_file_read', { exact: true }).click();
await expect(page.getByText('{"target":"repo_file_read"}')).toBeVisible();
await expect(page.getByText('{"ok":true}').first()).toBeVisible();
});
});
+25
View File
@@ -88,6 +88,31 @@ test.describe('frontend CRUD smoke flows', () => {
).toBeVisible();
});
test('opens pipeline AI capabilities with malformed model options', async ({
page,
}) => {
await installLangBotApiMocks(page, { authenticated: true });
await page.goto('/home/pipelines?id=pipeline-ai');
await expect(page.locator('input[name="basic.name"]')).toBeVisible();
await page.getByRole('button', { name: /^AI$/ }).click();
await expect(page.getByText('Runtime')).toBeVisible();
await expect(
page.locator('[data-slot="card-title"]').filter({
hasText: 'Built-in Agent',
}),
).toBeVisible();
await expect(
page.locator('label').filter({
hasText: 'Model',
}),
).toBeVisible();
await expect(page.getByText('A <Select.Item')).toHaveCount(0);
await expect(page.getByText('500')).toHaveCount(0);
});
test('creates, edits, and deletes a knowledge base', async ({ page }) => {
await installLangBotApiMocks(page, { authenticated: true });
+169 -5
View File
@@ -72,7 +72,11 @@ interface LangBotApiMockState {
counters: Record<string, number>;
knowledgeBases: KnowledgeBaseMock[];
mcpServers: MCPServerMock[];
monitoringData: unknown;
monitoringSessions: unknown[];
pipelines: PipelineMock[];
sessionAnalyses: Record<string, unknown>;
sessionMessages: Record<string, unknown[]>;
skills: SkillMock[];
}
@@ -122,12 +126,14 @@ function emptyMonitoringData() {
},
messages: [],
llmCalls: [],
toolCalls: [],
embeddingCalls: [],
sessions: [],
errors: [],
totalCount: {
messages: 0,
llmCalls: 0,
toolCalls: 0,
embeddingCalls: 0,
sessions: 0,
errors: 0,
@@ -188,6 +194,102 @@ function makePipeline(
};
}
function pipelineMetadata() {
return {
configs: [
{
name: 'ai',
label: {
en_US: 'AI Capabilities',
zh_Hans: 'AI 能力',
},
stages: [
{
name: 'runner',
label: {
en_US: 'Runtime',
zh_Hans: '运行方式',
},
config: [
{
id: 'runner',
name: 'runner',
label: {
en_US: 'Runner',
zh_Hans: '运行器',
},
type: 'select',
required: true,
default: 'local-agent',
options: [
{
name: 'local-agent',
label: {
en_US: 'Built-in Agent',
zh_Hans: '内置 Agent',
},
},
],
},
],
},
{
name: 'local-agent',
label: {
en_US: 'Built-in Agent',
zh_Hans: '内置 Agent',
},
config: [
{
id: 'model',
name: 'model',
label: {
en_US: 'Model',
zh_Hans: '模型',
},
type: 'model-fallback-selector',
required: true,
default: {
primary: 'llm-valid',
fallbacks: [],
},
},
],
},
],
},
],
};
}
function providerModelList() {
return {
models: [
{
uuid: '',
name: 'Broken Empty UUID Model',
provider_uuid: 'provider-empty',
provider: {
uuid: 'provider-empty',
name: 'Broken Provider',
requester: 'mock-provider',
},
},
{
uuid: 'llm-valid',
name: 'Valid Mock Model',
provider_uuid: 'provider-valid',
provider: {
uuid: 'provider-valid',
name: 'Mock Provider',
requester: 'mock-provider',
},
abilities: ['func_call'],
},
],
};
}
function knowledgeEngine() {
return {
plugin_id: 'builtin/minimal-knowledge',
@@ -389,8 +491,20 @@ async function handleBackendApi(route: Route, state: LangBotApiMockState) {
});
}
if (path === '/api/v1/provider/models/llm') {
return fulfillJson(route, providerModelList());
}
if (path === '/api/v1/provider/models/embedding') {
return fulfillJson(route, { models: [] });
}
if (path === '/api/v1/provider/models/rerank') {
return fulfillJson(route, { models: [] });
}
if (path === '/api/v1/pipelines/_/metadata') {
return fulfillJson(route, { configs: [] });
return fulfillJson(route, pipelineMetadata());
}
if (path === '/api/v1/pipelines') {
@@ -689,11 +803,43 @@ async function handleBackendApi(route: Route, state: LangBotApiMockState) {
}
if (path === '/api/v1/monitoring/data') {
return fulfillJson(route, emptyMonitoringData());
return fulfillJson(route, state.monitoringData);
}
if (path === '/api/v1/monitoring/sessions') {
return fulfillJson(route, {
sessions: state.monitoringSessions,
total: state.monitoringSessions.length,
});
}
if (path === '/api/v1/monitoring/messages') {
const sessionId = url.searchParams.get('sessionId') || '';
const messages = state.sessionMessages[sessionId] || [];
return fulfillJson(route, {
messages,
total: messages.length,
});
}
const sessionAnalysisMatch = path.match(
/^\/api\/v1\/monitoring\/sessions\/([^/]+)\/analysis$/,
);
if (sessionAnalysisMatch) {
const sessionId = decodeURIComponent(sessionAnalysisMatch[1]);
return fulfillJson(
route,
state.sessionAnalyses[sessionId] || {
session_id: sessionId,
found: true,
tool_calls: [],
},
);
}
if (path === '/api/v1/monitoring/overview') {
return fulfillJson(route, emptyMonitoringData().overview);
const data = state.monitoringData as { overview?: unknown };
return fulfillJson(route, data.overview || emptyMonitoringData().overview);
}
if (path === '/api/v1/monitoring/token-statistics') {
@@ -798,15 +944,33 @@ async function handleCloudApi(route: Route) {
export async function installLangBotApiMocks(
page: Page,
options: { authenticated?: boolean; storage?: JsonRecord } = {},
options: {
authenticated?: boolean;
monitoringData?: unknown;
monitoringSessions?: unknown[];
sessionAnalyses?: Record<string, unknown>;
sessionMessages?: Record<string, unknown[]>;
storage?: JsonRecord;
} = {},
) {
const { authenticated = false, storage = {} } = options;
const {
authenticated = false,
monitoringData,
monitoringSessions,
sessionAnalyses,
sessionMessages,
storage = {},
} = options;
const state: LangBotApiMockState = {
bots: [],
counters: {},
knowledgeBases: [],
mcpServers: [],
monitoringData: monitoringData || emptyMonitoringData(),
monitoringSessions: monitoringSessions || [],
pipelines: [],
sessionAnalyses: sessionAnalyses || {},
sessionMessages: sessionMessages || {},
skills: [],
};
+453
View File
@@ -0,0 +1,453 @@
import { expect, test } from '@playwright/test';
import { installLangBotApiMocks } from './fixtures/langbot-api';
import { buildConversationTurns } from '../../src/app/home/monitoring/utils/conversationTurns';
import {
ErrorLog,
LLMCall,
MonitoringMessage,
ToolCall,
} from '../../src/app/home/monitoring/types/monitoring';
const bot = {
id: 'bot-monitoring',
name: 'Monitoring Bot',
};
const pipeline = {
id: 'pipeline-monitoring',
name: 'Monitoring Pipeline',
};
function time(minute: number) {
return new Date(`2026-07-02T10:${String(minute).padStart(2, '0')}:00Z`);
}
function message(
id: string,
role: 'user' | 'assistant',
minute: number,
content: string,
sessionId = 'session-agent',
): MonitoringMessage {
return {
id,
timestamp: time(minute),
botId: bot.id,
botName: bot.name,
pipelineId: pipeline.id,
pipelineName: pipeline.name,
messageContent: content,
sessionId,
status: 'success',
level: 'info',
platform: role === 'user' ? 'person' : 'bot',
userId: 'user-1',
userName: 'Playwright User',
runnerName: 'local-agent',
variables: '{}',
role,
};
}
function llmCall(
id: string,
minute: number,
messageId: string | undefined,
input: number,
output: number,
duration: number,
sessionId = 'session-agent',
): LLMCall {
return {
id,
timestamp: time(minute),
modelName: 'gpt-5.5',
tokens: {
input,
output,
total: input + output,
},
duration,
status: 'success',
botId: bot.id,
botName: bot.name,
pipelineId: pipeline.id,
pipelineName: pipeline.name,
sessionId,
messageId,
};
}
function errorLog(id: string, minute: number, messageId: string): ErrorLog {
return {
id,
timestamp: time(minute),
errorType: 'ToolExecutionError',
errorMessage: 'Tool retry failed',
botId: bot.id,
botName: bot.name,
pipelineId: pipeline.id,
pipelineName: pipeline.name,
sessionId: 'session-agent',
messageId,
};
}
function toolCall(
id: string,
minute: number,
messageId: string | undefined,
toolName: string,
duration: number,
sessionId = 'session-agent',
status: 'success' | 'error' = 'success',
): ToolCall {
return {
id,
timestamp: time(minute),
toolName,
toolSource: 'native',
duration,
status,
botId: bot.id,
botName: bot.name,
pipelineId: pipeline.id,
pipelineName: pipeline.name,
sessionId,
messageId,
arguments: JSON.stringify({ query: toolName }),
result: status === 'success' ? JSON.stringify({ ok: true }) : undefined,
errorMessage: status === 'error' ? 'Tool failed' : undefined,
};
}
function rawMessage(message: MonitoringMessage) {
return {
id: message.id,
timestamp: message.timestamp.toISOString(),
bot_id: message.botId,
bot_name: message.botName,
pipeline_id: message.pipelineId,
pipeline_name: message.pipelineName,
message_content: message.messageContent,
session_id: message.sessionId,
status: message.status,
level: message.level,
platform: message.platform,
user_id: message.userId,
user_name: message.userName,
runner_name: message.runnerName,
variables: message.variables,
role: message.role,
};
}
function rawLlmCall(call: LLMCall) {
return {
id: call.id,
timestamp: call.timestamp.toISOString(),
model_name: call.modelName,
input_tokens: call.tokens.input,
output_tokens: call.tokens.output,
total_tokens: call.tokens.total,
duration: call.duration,
cost: call.cost,
status: call.status,
bot_id: call.botId,
bot_name: call.botName,
pipeline_id: call.pipelineId,
pipeline_name: call.pipelineName,
session_id: call.sessionId,
error_message: call.errorMessage,
message_id: call.messageId,
};
}
function rawError(error: ErrorLog) {
return {
id: error.id,
timestamp: error.timestamp.toISOString(),
error_type: error.errorType,
error_message: error.errorMessage,
bot_id: error.botId,
bot_name: error.botName,
pipeline_id: error.pipelineId,
pipeline_name: error.pipelineName,
session_id: error.sessionId,
stack_trace: error.stackTrace,
message_id: error.messageId,
};
}
function rawToolCall(call: ToolCall) {
return {
id: call.id,
timestamp: call.timestamp.toISOString(),
tool_name: call.toolName,
tool_source: call.toolSource,
duration: call.duration,
status: call.status,
bot_id: call.botId,
bot_name: call.botName,
pipeline_id: call.pipelineId,
pipeline_name: call.pipelineName,
session_id: call.sessionId,
message_id: call.messageId,
arguments: call.arguments,
result: call.result,
error_message: call.errorMessage,
};
}
function monitoringScenario() {
const messages = [
message(
'single-user',
'user',
1,
'Standalone question with no reply',
'session-single',
),
message('agent-user-1', 'user', 10, 'Need deployment plan'),
message('agent-assistant-1', 'assistant', 11, 'Agent step 1: inspect repo'),
message('agent-assistant-2', 'assistant', 12, 'Agent step 2: run tests'),
message(
'agent-assistant-3',
'assistant',
13,
'Final answer: deployment plan ready',
),
message('agent-user-2', 'user', 20, 'Continue with rollback plan'),
message('agent-assistant-4', 'assistant', 21, 'Rollback plan ready'),
];
const llmCalls = [
llmCall('agent-call-1', 10, 'agent-user-1', 100, 40, 120),
llmCall('agent-call-2', 11, 'agent-user-1', 200, 60, 220),
llmCall('agent-call-3', 12, 'agent-user-1', 300, 90, 260),
llmCall('agent-call-4', 20, 'agent-user-2', 50, 25, 80),
];
const errors = [errorLog('agent-error-1', 12, 'agent-user-1')];
const toolCalls = [
toolCall('agent-tool-1', 11, 'agent-user-1', 'repo_search', 90),
toolCall('agent-tool-2', 12, 'agent-user-1', 'run_tests', 150),
toolCall('agent-tool-3', 20, 'agent-user-2', 'rollback_lookup', 70),
];
return {
messages,
llmCalls,
toolCalls,
errors,
};
}
function rawMonitoringData() {
const scenario = monitoringScenario();
return {
overview: {
total_messages: scenario.messages.length,
llm_calls: scenario.llmCalls.length,
embedding_calls: 0,
model_calls: scenario.llmCalls.length,
success_rate: 100,
active_sessions: 2,
},
messages: scenario.messages.map(rawMessage),
llmCalls: scenario.llmCalls.map(rawLlmCall),
toolCalls: scenario.toolCalls.map(rawToolCall),
embeddingCalls: [],
sessions: [],
errors: scenario.errors.map(rawError),
totalCount: {
messages: scenario.messages.length,
llmCalls: scenario.llmCalls.length,
toolCalls: scenario.toolCalls.length,
embeddingCalls: 0,
sessions: 0,
errors: scenario.errors.length,
},
};
}
test.describe('monitoring conversation turn grouping', () => {
test('keeps a single user message as one observable turn', () => {
const userOnly = message(
'single-user-only',
'user',
1,
'No answer yet',
'session-user-only',
);
const turns = buildConversationTurns([userOnly], [], []);
expect(turns).toHaveLength(1);
expect(turns[0].id).toBe(userOnly.id);
expect(turns[0].userMessage?.messageContent).toBe('No answer yet');
expect(turns[0].assistantMessages).toHaveLength(0);
expect(turns[0].llmCalls).toHaveLength(0);
expect(turns[0].totalTokens).toBe(0);
});
test('groups multi-step agent execution and multiple replies into one user turn', () => {
const scenario = monitoringScenario();
const turns = buildConversationTurns(
scenario.messages,
scenario.llmCalls,
scenario.errors,
scenario.toolCalls,
);
const agentTurn = turns.find((turn) => turn.id === 'agent-user-1');
expect(agentTurn).toBeTruthy();
expect(agentTurn?.userMessage?.messageContent).toBe('Need deployment plan');
expect(
agentTurn?.assistantMessages.map((item) => item.messageContent),
).toEqual([
'Agent step 1: inspect repo',
'Agent step 2: run tests',
'Final answer: deployment plan ready',
]);
expect(agentTurn?.llmCalls).toHaveLength(3);
expect(agentTurn?.toolCalls).toHaveLength(2);
expect(agentTurn?.errors).toHaveLength(1);
expect(agentTurn?.totalTokens).toBe(790);
expect(agentTurn?.totalDuration).toBe(600);
expect(agentTurn?.totalToolDuration).toBe(240);
});
test('starts a new turn for each later user message in the same session', () => {
const firstUser = message('same-session-user-1', 'user', 1, 'First');
const firstReply = message(
'same-session-reply-1',
'assistant',
2,
'First reply',
);
const secondUser = message('same-session-user-2', 'user', 3, 'Second');
const secondReply = message(
'same-session-reply-2',
'assistant',
4,
'Second reply',
);
const turns = buildConversationTurns(
[firstUser, firstReply, secondUser, secondReply],
[
llmCall('same-session-call-1', 1, firstUser.id, 10, 5, 40),
llmCall('same-session-call-2', 3, secondUser.id, 20, 10, 50),
],
[],
);
expect(turns.map((turn) => turn.id)).toEqual([
'same-session-user-2',
'same-session-user-1',
]);
expect(
turns[0].assistantMessages.map((item) => item.messageContent),
).toEqual(['Second reply']);
expect(
turns[1].assistantMessages.map((item) => item.messageContent),
).toEqual(['First reply']);
});
test('attaches calls without message ids by session time', () => {
const user = message('fallback-user', 'user', 1, 'Use session fallback');
const assistant = message(
'fallback-assistant',
'assistant',
2,
'Fallback reply',
);
const call = llmCall('fallback-call', 2, undefined, 25, 5, 70);
const turns = buildConversationTurns([user, assistant], [call], []);
expect(turns).toHaveLength(1);
expect(turns[0].llmCalls).toHaveLength(1);
expect(turns[0].llmCalls[0].id).toBe(call.id);
expect(turns[0].totalTokens).toBe(30);
});
test('attaches tool calls without message ids by session time', () => {
const user = message('tool-fallback-user', 'user', 1, 'Use tool fallback');
const assistant = message(
'tool-fallback-assistant',
'assistant',
2,
'Tool fallback reply',
);
const call = toolCall(
'tool-fallback-call',
2,
undefined,
'memory_lookup',
45,
);
const turns = buildConversationTurns([user, assistant], [], [], [call]);
expect(turns).toHaveLength(1);
expect(turns[0].toolCalls).toHaveLength(1);
expect(turns[0].toolCalls[0].id).toBe(call.id);
expect(turns[0].totalToolDuration).toBe(45);
});
test('renders user-only, multi-agent, and multi-turn cases in the monitoring page', async ({
page,
}) => {
await installLangBotApiMocks(page, {
authenticated: true,
monitoringData: rawMonitoringData(),
});
await page.goto('/home/monitoring');
await expect(page.getByText('3 conversation turns')).toBeVisible();
await expect(
page.getByText('Standalone question with no reply'),
).toBeVisible();
await expect(page.getByText('No assistant reply recorded')).toBeVisible();
await expect(page.getByText('Need deployment plan')).toBeVisible();
await expect(page.getByText('Agent step 1: inspect repo')).toBeVisible();
await expect(page.getByText('Assistant +2')).toBeVisible();
await expect(page.getByText('3 LLM')).toBeVisible();
await expect(page.getByText('2 tools')).toBeVisible();
await expect(page.getByText('790 tokens')).toBeVisible();
await expect(page.getByText('1 errors')).toBeVisible();
await expect(page.getByText('Continue with rollback plan')).toBeVisible();
await expect(page.getByText('Rollback plan ready')).toBeVisible();
const agentTurn = page
.locator('div[role="button"]')
.filter({ hasText: 'Need deployment plan' });
await expect(agentTurn).toHaveCount(1);
await agentTurn.click();
await expect(page.getByText('Conversation Trace')).toBeVisible();
await expect(page.getByText('Agent step 2: run tests')).toBeVisible();
await expect(
page.getByText('Final answer: deployment plan ready'),
).toBeVisible();
await expect(page.getByText('LLM Calls (3)')).toBeVisible();
await expect(page.getByText('#3 gpt-5.5')).toBeVisible();
await expect(page.getByText('In: 300')).toBeVisible();
await expect(page.getByText('Out: 90')).toBeVisible();
await expect(page.getByText('Total: 390')).toBeVisible();
await expect(page.getByText('Tool Calls (2)')).toBeVisible();
await expect(page.getByText('#1 repo_search')).toBeVisible();
await expect(page.getByText('#2 run_tests')).toBeVisible();
await expect(page.getByText('Arguments')).toHaveCount(0);
await expect(page.getByText('Result')).toHaveCount(0);
await page.getByText('#1 repo_search').click();
await expect(page.getByText('Arguments').first()).toBeVisible();
await expect(page.getByText('Result').first()).toBeVisible();
await expect(page.getByText('Tool retry failed')).toBeVisible();
});
});
@@ -0,0 +1,195 @@
import { expect, test } from '@playwright/test';
import { installLangBotApiMocks } from './fixtures/langbot-api';
const bot = {
id: 'bot-pipeline-monitoring',
name: 'Pipeline Bot',
};
const pipeline = {
id: 'pipeline-monitoring',
name: 'Pipeline Under Test',
};
function at(minute: number) {
return `2026-07-02T10:${String(minute).padStart(2, '0')}:00Z`;
}
function message(
id: string,
role: 'user' | 'assistant',
minute: number,
content: string,
sessionId = 'session-pipeline-agent',
) {
return {
id,
timestamp: at(minute),
bot_id: bot.id,
bot_name: bot.name,
pipeline_id: pipeline.id,
pipeline_name: pipeline.name,
message_content: content,
session_id: sessionId,
status: 'success',
level: 'info',
platform: role === 'user' ? 'person' : 'bot',
user_id: 'pipeline-user',
user_name: 'Pipeline User',
runner_name: 'local-agent',
variables: '{}',
role,
};
}
function llmCall(
id: string,
minute: number,
messageId: string,
input: number,
output: number,
duration: number,
) {
return {
id,
timestamp: at(minute),
model_name: 'gpt-5.5',
input_tokens: input,
output_tokens: output,
total_tokens: input + output,
duration,
cost: 0,
status: 'success',
bot_id: bot.id,
bot_name: bot.name,
pipeline_id: pipeline.id,
pipeline_name: pipeline.name,
session_id: 'session-pipeline-agent',
message_id: messageId,
};
}
function toolCall(id: string, minute: number, messageId: string, name: string) {
return {
id,
timestamp: at(minute),
tool_name: name,
tool_source: 'native',
duration: 120,
status: 'success',
bot_id: bot.id,
bot_name: bot.name,
pipeline_id: pipeline.id,
pipeline_name: pipeline.name,
session_id: 'session-pipeline-agent',
message_id: messageId,
arguments: JSON.stringify({ query: name }),
result: JSON.stringify({ ok: true }),
};
}
function monitoringData() {
const messages = [
message(
'single-user',
'user',
1,
'Pipeline single user message without reply',
'session-pipeline-single',
),
message('agent-user', 'user', 10, 'Pipeline needs a deployment plan'),
message(
'agent-assistant-1',
'assistant',
11,
'Pipeline agent step 1: inspect repository',
),
message(
'agent-assistant-2',
'assistant',
12,
'Pipeline agent step 2: run tests',
),
message(
'agent-assistant-3',
'assistant',
13,
'Pipeline final answer: deployment ready',
),
];
const llmCalls = [
llmCall('pipeline-call-1', 10, 'agent-user', 100, 40, 180),
llmCall('pipeline-call-2', 11, 'agent-user', 140, 50, 220),
];
const toolCalls = [
toolCall('pipeline-tool-1', 11, 'agent-user', 'repo_search'),
toolCall('pipeline-tool-2', 12, 'agent-user', 'run_tests'),
];
return {
overview: {
total_messages: messages.length,
llm_calls: llmCalls.length,
embedding_calls: 0,
model_calls: llmCalls.length,
success_rate: 100,
active_sessions: 2,
},
messages,
llmCalls,
toolCalls,
embeddingCalls: [],
sessions: [],
errors: [],
totalCount: {
messages: messages.length,
llmCalls: llmCalls.length,
toolCalls: toolCalls.length,
embeddingCalls: 0,
sessions: 0,
errors: 0,
},
};
}
test.describe('pipeline monitoring conversation turns', () => {
test('uses conversation turns and folded tool calls in the pipeline dashboard', async ({
page,
}) => {
await installLangBotApiMocks(page, {
authenticated: true,
monitoringData: monitoringData(),
});
await page.goto(`/home/pipelines?id=${pipeline.id}`);
await page.getByRole('tab', { name: 'Dashboard' }).click();
await expect(page.getByText('2 conversation turns')).toBeVisible();
await expect(
page.getByText('Pipeline single user message without reply'),
).toBeVisible();
await expect(
page.getByText('Pipeline needs a deployment plan'),
).toBeVisible();
await expect(
page.getByText('Pipeline agent step 1: inspect repository'),
).toBeVisible();
await expect(page.getByText('Assistant +2')).toBeVisible();
await expect(page.getByText('2 tools')).toBeVisible();
const agentTurn = page
.locator('div[role="button"]')
.filter({ hasText: 'Pipeline needs a deployment plan' });
await expect(agentTurn).toHaveCount(1);
await agentTurn.click();
await expect(page.getByText('Tool Calls (2)')).toBeVisible();
await expect(page.getByText('#1 repo_search')).toBeVisible();
await expect(page.getByText('#2 run_tests')).toBeVisible();
await expect(page.getByText('Arguments')).toHaveCount(0);
await page.getByText('#1 repo_search').click();
await expect(page.getByText('Arguments')).toBeVisible();
await expect(page.getByText('Result')).toBeVisible();
});
});