- Replace legacy pipeline binding card + RoutingRulesEditor with unified
EventBindingsEditor; remove use_pipeline_uuid/pipeline_routing_rules
from bot form schema and API update handler
- Add _augment_event_data() to botmgr for filter virtual fields
(message_text, message_element_types, chat_type)
- Add alembic migration 0009: migrate use_pipeline_uuid and
pipeline_routing_rules into event_bindings on first run
- Fix command.tsx: data-[disabled] -> data-[disabled=true] so cmdk 1.x
items (data-disabled=false) are not pointer-events:none
- EventBindingsEditor: onSelect on CommandItems, filter conditions panel,
disabled bindings section, dnd reorder
- i18n: add filter/condition keys for zh-Hans and en-US
- Update tests to match new bot service behavior
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the PluginToolLoader.get_tool() override that returned a raw
ComponentManifest, so every loader's get_tool() now returns a uniform
resource_tool.LLMTool (PluginToolLoader.get_tools() already did this
conversion). This removes the only source of tool-shape heterogeneity.
- ToolManager.get_tool_schema(): drop the ComponentManifest-vs-LLMTool branch
- ToolManager.get_tool_detail(): new host-level shape {name, description,
human_desc, parameters}
- handler.py GET_TOOL_DETAIL: call tool_mgr.get_tool_detail(); delete the
handler-local _build_tool_detail + _i18n_to_dict/_i18n_to_text adapters and
the litellm TODO
- ToolLookupResult is now just LLMTool
The dropped label/spec fields were not consumed by any runner (local-agent
build_llm_tool and external harnesses use only name/description/parameters).
Extract the AgentRunner Protocol v1 host-side surface from the giant
RuntimeConnectionHandler.__init__ into sibling modules using a registration-
function pattern (behavior-preserving; @h.action == @self.action):
- agent_run_support.py: shared constants + authorization/scope/projection helpers
- agent_pull_actions.py: register(h) for history/event pull APIs
- agent_runner_actions.py: register(h) for run/runtime/stats/claim lifecycle
- agent_state_actions.py: register(h) for steering/state APIs
__init__ now calls the three register(self) functions. handler.py keeps the
pre-existing plugin/llm/vector/knowledge handlers, get_prompt/call_tool/
get_tool_detail (coupled to retained helpers), shared helpers, and outbound
methods; it re-imports _validate_agent_run_session so external imports keep
working. handler.py: 4066 -> 1871 lines.
test_state_api_auth.py: repoint get_session_registry patch targets to
agent_run_support (the lookup moved modules). 385 agent unit tests pass; ruff clean.
Expose skill tools (activate/register_skill/native exec) like native tools
instead of gating them behind the skill_authoring capability:
- toolmgr.get_all_tools drops include_skill_authoring; SkillToolLoader
self-gates on sandbox + skill_mgr
- preproc drops the include_skill_authoring branch; pipeline-bound skills
and the skills resource gate on skill_mgr presence
Persist activated skills into host.activated_skills conversation state so
they survive across runs (host writes at activate; last-write-wins); drop
the dead restore_activated_skills helper.
Prefill ToolResource.parameters host-side (tool_mgr.get_tool_schema) so
runners build LLM tools without per-tool get_tool_detail round-trips.
Align agent-runner-pluginization design docs to the all-tool model.
* feat(vector): add Valkey Search vector database backend
Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.
- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
accepted for interface parity but NOT honored (docstring + one-time warning +
docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
(vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
(TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.
* fix(vector): paginate Valkey Search deletes and guard delete_by_filter
Address self-review follow-ups for the Valkey Search VDB backend:
- _search_keys now paginates through the full result set in batches of
_DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
page, so delete_by_file_id / delete_by_filter fully remove files and
filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
a filter referencing only non-indexed fields must skip and return 0
(never fall back to match-all), and a supported filter still deletes
matching keys.
* refactor(vector): harden Valkey Search backend and add adversarial tests
Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
instead of building ServerCredentials(password=None, ...), which glide
rejects).
- Add an async close() teardown that closes the glide client and resets
cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
never widens).
Tests:
- Add an adversarial-input integration test proving crafted file_id /
query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
* fix(vector): make Valkey Search file_id TAG support arbitrary characters
Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.
Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
* refactor(vector): address Valkey Search review feedback
- Add configurable request_timeout (default 5000ms; glide default 250ms is
too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
_reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics
* fix(vector/valkey_search): address round-2 review feedback
- Serialize lazy client creation with an asyncio.Lock (double-checked) so
concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
non-numeric input and pre-empting a future NUMERIC-field injection surface.
* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
---------
Signed-off-by: Daria Korenieva <daric2612@gmail.com>
Operators can now set a global default memory limit for all stdio MCP
servers in config.yaml or via environment variable:
config.yaml:
box:
default_memory_mb: 2048 # default: 1536
env:
BOX__DEFAULT_MEMORY_MB=2048
The default is raised from 1024 to 1536 MB — a safer floor for
Node.js V8 + WASM (undici llhttp) under nsjail cgroup limits.
Individual MCP servers can still override via their own box.memory_mb.
Previously the fallback was hardcoded to 1024 MB, causing OOM kills
(return_code=137) on node/npx MCP servers that need more RAM.
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
The previous _TransferredStack approach broke anyio lexical context:
websocket_client/ClientSession use anyio task groups whose cancel scope is
bound to the frame that entered them. Deferring their aclose via a transferred
exit stack left the underlying memory streams closed once initialize() returned,
so the very next request (refresh -> list_tools) failed with Connection closed.
New design:
- Attach on the owner exit stack (same task as the serve loop, lexically intact)
- A cold-starting process makes initialize() fail; signal _ColdStartRetry up to
the outer retry loop, which reuses the live process without consuming retry budget
- _lifecycle_loop_with_retry handles _ColdStartRetry like _TransportReconnect:
preserves process, no fatal budget, backs off 2s and retries
- Two new unit tests: cold-start raises _ColdStartRetry (not fatal) when process
is alive; raises fatal error when process has actually exited
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
A node/npx stdio MCP server (e.g. firecrawl-mcp via npx -y) failed on first
connect with Connection closed / Failed after 4 attempts, even though the
process was fine. An npx cold start downloads+installs the package before the
server can answer the MCP handshake (measured ~27s for a simple official
server; longer for heavier ones). The old code attached the WS and called
session.initialize() the instant the process was started, so the handshake ran
before the process could answer and failed; the outer lifecycle retry then
rebuilt the process, churning it in a loop.
Verified decisively: attaching + initialize() against a mid-cold-start process
times out on attempt 1 (process still installing) but SUCCEEDS at t+0.6s on
attempt 2 once the process is ready. So the fix is to retry the handshake in
place, not to rebuild the process.
Changes (mcp_stdio.initialize):
- Start the managed process ONCE, then loop attach WS -> ClientSession ->
initialize() within the startup_timeout budget, tearing down each failed
attempt cleanly, until the handshake succeeds or the budget elapses. A
successful transport/session is transferred into the owner exit stack via a
small _TransferredStack adapter.
- Bound each attempt with asyncio.wait_for(initialize, _HANDSHAKE_ATTEMPT_TIMEOUT_SEC=10s)
so a cold-starting process fails fast and retries instead of hanging until
the transport drops.
- Stop retrying ONLY when the process has DEFINITIVELY exited: new
_managed_process_has_exited() (checks EXITED status) replaces the previous
not-_managed_process_is_running() test, which false-negatived on a
just-spawned process that had not yet reported RUNNING and made the loop bail
to the outer rebuild path (relay then rejected the early re-attach with HTTP 400).
Adds a unit test that fails the first two handshakes with the process alive and
asserts the loop retries to success while starting the process exactly once.
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
* refactor(mcp): make MCP test reuse the shared Box session instead of a per-test session
Testing an MCP server (config-page "test" button) previously spun up a fresh
isolated mcp-test-<uuid> Box session every time: cold-start the container, run
the dependency bootstrap, probe, then tear the whole session down. That is slow
(tens of seconds) and, on an already-hosted server, wasteful — the server is
already running in the shared session.
Change the test to reuse the shared session / live process:
- _build_box_session_id: transient tests now use mcp-shared, the same Box
session as live servers, so a test reuses the running container (and, for an
existing server, its live managed process) instead of a cold per-test session.
- cleanup_session: a transient test no longer deletes the whole session (which
under the shared model would kill every other MCP server in the container). It
stops only its own process_id, exactly like a live server. Isolation is now at
the process level (distinct process_id per server/test), not the session level.
- test_mcp_server (persisted server): reuse the live connection with a real
list_tools refresh/probe; only fall back to a full start() when there is no
live connection to probe or the refresh fails, instead of an ERROR->start()
rebuild.
Trade-off: a failing test now shares the container with live servers rather than
a throwaway session. Accepted deliberately in favour of near-instant tests;
process-level isolation keeps a test from stopping another server's process.
* chore(deps): pin langbot-plugin 0.4.9 for the nsjail RLIMIT_AS node/npx MCP fix
---------
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
* feat(api): support global API key from config.yaml (api.global_api_key)
Accept a config-defined global API key anywhere a web-UI key is accepted
(X-API-Key / Bearer), with no login session and no DB record. Useful for
automated deployments and AI agents (HTTP API + MCP). Defaults to empty
(disabled); does not require the lbk_ prefix.
- templates/config.yaml: add api.global_api_key with security notes
- service/apikey.py: verify_api_key checks global key first (constant-time)
- docs/API_KEY_AUTH.md: document the global key + security guidance
- tests: cover global-key match, prefix-free, fallback-to-db, disabled
* feat(mcp): expose LangBot management as an MCP server at /mcp
Add an MCP (Model Context Protocol) server so external AI agents can manage a
LangBot instance. Reuses the same API-key auth as the HTTP API (including the
config.yaml global API key).
- pkg/api/mcp/server.py: FastMCP server wrapping the service layer; 21 curated
tools across system/bots/pipelines/models/knowledge/mcp-servers/skills
- pkg/api/mcp/mount.py: ASGI dispatcher fronting Quart; authenticates /mcp
requests with an API key, runs the streamable-HTTP session manager lifespan
- controller/main.py: serve the wrapped ASGI app via hypercorn (was run_task)
- web: new 'MCP' tab in the API integration dialog showing endpoint, auth, and
client config; i18n for 8 locales
- tests/manual/mcp_smoke.py: e2e check (401 unauth, list tools, call tools)
Tool surface is intentionally curated (not all ~25 route groups) to keep the
agent surface small, safe, and maintainable. Extend deliberately.
* feat(skills): add in-repo skills/ as the single source of truth
Migrate the agent skills + QA/e2e test harness from the (now archived)
langbot-app/langbot-skills repo into LangBot/skills/, and add four new skills.
Migrated:
- langbot-plugin-dev, langbot-testing (e2e), langbot-env-setup,
langbot-skills-maintenance, langbot-eba-adapter-dev
- the bin/lbs CLI (src/, test/, scripts/, schemas/, qa-agent-docs/)
New:
- langbot-dev core backend + web development
- langbot-deploy Docker/K8s deployment + config.yaml + global API key
- langbot-mcp-ops operating the LangBot MCP server (/mcp)
- langbot-space-ops operating the Space marketplace MCP server
- src/cli.ts repoRoot(): recognize the skills assets root (skills.index.json +
bin/lbs) so the CLI works when nested inside the LangBot repo
- README.md: unified skill catalog; skills.index.json regenerated
Parity with source verified: bin/lbs validate + node test suite match the
source repo (only the uncommitted .lbpkg build-artifact fixture differs).
* docs(agents): document agent-facing surfaces + API/MCP/skills sync rule
* docs(readme): add 'Built for AI Agents' section across all locales
Highlight MCP server, in-repo skills (single source of truth), AGENTS.md
sync rule, and llms.txt. Cross-link LangBot Space MCP marketplace.
* style(mcp): fix ruff format + prettier lint in MCP server and API panel
* style(web): prettier format MCP i18n locale entries
* docs(skills): note MCP instance control in dev/testing skills
All development-guidance skills now point to the LangBot instance MCP
server (/mcp) and the Space marketplace MCP server, reusing API keys.
Update _normalize_stream_tool_calls to preserve provider_specific_fields
(including thought_signature) from streaming tool call chunks. Also preserve
provider_specific_fields from delta in invoke_llm_stream.
This ensures Gemini's thought_signature is round-tripped correctly:
1. LiteLLM extracts thought_signature from Gemini response
2. It's preserved in Message/ToolCall entities (via SDK changes)
3. _convert_messages includes it in the next request
Also add unit tests for provider_specific_fields round-tripping.
Fixes: langbot-app/LangBot#1899
huanghuoguoguo