The previous _TransferredStack approach broke anyio lexical context:
websocket_client/ClientSession use anyio task groups whose cancel scope is
bound to the frame that entered them. Deferring their aclose via a transferred
exit stack left the underlying memory streams closed once initialize() returned,
so the very next request (refresh -> list_tools) failed with Connection closed.
New design:
- Attach on the owner exit stack (same task as the serve loop, lexically intact)
- A cold-starting process makes initialize() fail; signal _ColdStartRetry up to
the outer retry loop, which reuses the live process without consuming retry budget
- _lifecycle_loop_with_retry handles _ColdStartRetry like _TransportReconnect:
preserves process, no fatal budget, backs off 2s and retries
- Two new unit tests: cold-start raises _ColdStartRetry (not fatal) when process
is alive; raises fatal error when process has actually exited
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
A node/npx stdio MCP server (e.g. firecrawl-mcp via npx -y) failed on first
connect with Connection closed / Failed after 4 attempts, even though the
process was fine. An npx cold start downloads+installs the package before the
server can answer the MCP handshake (measured ~27s for a simple official
server; longer for heavier ones). The old code attached the WS and called
session.initialize() the instant the process was started, so the handshake ran
before the process could answer and failed; the outer lifecycle retry then
rebuilt the process, churning it in a loop.
Verified decisively: attaching + initialize() against a mid-cold-start process
times out on attempt 1 (process still installing) but SUCCEEDS at t+0.6s on
attempt 2 once the process is ready. So the fix is to retry the handshake in
place, not to rebuild the process.
Changes (mcp_stdio.initialize):
- Start the managed process ONCE, then loop attach WS -> ClientSession ->
initialize() within the startup_timeout budget, tearing down each failed
attempt cleanly, until the handshake succeeds or the budget elapses. A
successful transport/session is transferred into the owner exit stack via a
small _TransferredStack adapter.
- Bound each attempt with asyncio.wait_for(initialize, _HANDSHAKE_ATTEMPT_TIMEOUT_SEC=10s)
so a cold-starting process fails fast and retries instead of hanging until
the transport drops.
- Stop retrying ONLY when the process has DEFINITIVELY exited: new
_managed_process_has_exited() (checks EXITED status) replaces the previous
not-_managed_process_is_running() test, which false-negatived on a
just-spawned process that had not yet reported RUNNING and made the loop bail
to the outer rebuild path (relay then rejected the early re-attach with HTTP 400).
Adds a unit test that fails the first two handshakes with the process alive and
asserts the loop retries to success while starting the process exactly once.
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
* refactor(mcp): make MCP test reuse the shared Box session instead of a per-test session
Testing an MCP server (config-page "test" button) previously spun up a fresh
isolated mcp-test-<uuid> Box session every time: cold-start the container, run
the dependency bootstrap, probe, then tear the whole session down. That is slow
(tens of seconds) and, on an already-hosted server, wasteful — the server is
already running in the shared session.
Change the test to reuse the shared session / live process:
- _build_box_session_id: transient tests now use mcp-shared, the same Box
session as live servers, so a test reuses the running container (and, for an
existing server, its live managed process) instead of a cold per-test session.
- cleanup_session: a transient test no longer deletes the whole session (which
under the shared model would kill every other MCP server in the container). It
stops only its own process_id, exactly like a live server. Isolation is now at
the process level (distinct process_id per server/test), not the session level.
- test_mcp_server (persisted server): reuse the live connection with a real
list_tools refresh/probe; only fall back to a full start() when there is no
live connection to probe or the refresh fails, instead of an ERROR->start()
rebuild.
Trade-off: a failing test now shares the container with live servers rather than
a throwaway session. Accepted deliberately in favour of near-instant tests;
process-level isolation keeps a test from stopping another server's process.
* chore(deps): pin langbot-plugin 0.4.9 for the nsjail RLIMIT_AS node/npx MCP fix
---------
Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
- Replace legacy pipeline binding card + RoutingRulesEditor with unified
EventBindingsEditor; remove use_pipeline_uuid/pipeline_routing_rules
from bot form schema and API update handler
- Add _augment_event_data() to botmgr for filter virtual fields
(message_text, message_element_types, chat_type)
- Add alembic migration 0009: migrate use_pipeline_uuid and
pipeline_routing_rules into event_bindings on first run
- Fix command.tsx: data-[disabled] -> data-[disabled=true] so cmdk 1.x
items (data-disabled=false) are not pointer-events:none
- EventBindingsEditor: onSelect on CommandItems, filter conditions panel,
disabled bindings section, dnd reorder
- i18n: add filter/condition keys for zh-Hans and en-US
- Update tests to match new bot service behavior
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the PluginToolLoader.get_tool() override that returned a raw
ComponentManifest, so every loader's get_tool() now returns a uniform
resource_tool.LLMTool (PluginToolLoader.get_tools() already did this
conversion). This removes the only source of tool-shape heterogeneity.
- ToolManager.get_tool_schema(): drop the ComponentManifest-vs-LLMTool branch
- ToolManager.get_tool_detail(): new host-level shape {name, description,
human_desc, parameters}
- handler.py GET_TOOL_DETAIL: call tool_mgr.get_tool_detail(); delete the
handler-local _build_tool_detail + _i18n_to_dict/_i18n_to_text adapters and
the litellm TODO
- ToolLookupResult is now just LLMTool
The dropped label/spec fields were not consumed by any runner (local-agent
build_llm_tool and external harnesses use only name/description/parameters).
Extract the AgentRunner Protocol v1 host-side surface from the giant
RuntimeConnectionHandler.__init__ into sibling modules using a registration-
function pattern (behavior-preserving; @h.action == @self.action):
- agent_run_support.py: shared constants + authorization/scope/projection helpers
- agent_pull_actions.py: register(h) for history/event pull APIs
- agent_runner_actions.py: register(h) for run/runtime/stats/claim lifecycle
- agent_state_actions.py: register(h) for steering/state APIs
__init__ now calls the three register(self) functions. handler.py keeps the
pre-existing plugin/llm/vector/knowledge handlers, get_prompt/call_tool/
get_tool_detail (coupled to retained helpers), shared helpers, and outbound
methods; it re-imports _validate_agent_run_session so external imports keep
working. handler.py: 4066 -> 1871 lines.
test_state_api_auth.py: repoint get_session_registry patch targets to
agent_run_support (the lookup moved modules). 385 agent unit tests pass; ruff clean.
The PostgreSQL migration test had the same hardcoded 0005 head
assertion as the SQLite one; resolve the actual head from the Alembic
ScriptDirectory so 0006 (and future migrations) don't break it.
CI follow-up to the local/remote MCP work:
- Apply ruff format to provider/tools/loaders/mcp.py and the 0006
normalize-remote-mode migration (Lint job failed on formatting).
- test_migrations.py hardcoded the head revision as 0005_*, which broke
once 0006 landed. Resolve the actual head from the Alembic
ScriptDirectory so future migrations don't require editing the test.
Expose skill tools (activate/register_skill/native exec) like native tools
instead of gating them behind the skill_authoring capability:
- toolmgr.get_all_tools drops include_skill_authoring; SkillToolLoader
self-gates on sandbox + skill_mgr
- preproc drops the include_skill_authoring branch; pipeline-bound skills
and the skills resource gate on skill_mgr presence
Persist activated skills into host.activated_skills conversation state so
they survive across runs (host writes at activate; last-write-wins); drop
the dead restore_activated_skills helper.
Prefill ToolResource.parameters host-side (tool_mgr.get_tool_schema) so
runners build LLM tools without per-tool get_tool_detail round-trips.
Align agent-runner-pluginization design docs to the all-tool model.
* feat(api): support global API key from config.yaml (api.global_api_key)
Accept a config-defined global API key anywhere a web-UI key is accepted
(X-API-Key / Bearer), with no login session and no DB record. Useful for
automated deployments and AI agents (HTTP API + MCP). Defaults to empty
(disabled); does not require the lbk_ prefix.
- templates/config.yaml: add api.global_api_key with security notes
- service/apikey.py: verify_api_key checks global key first (constant-time)
- docs/API_KEY_AUTH.md: document the global key + security guidance
- tests: cover global-key match, prefix-free, fallback-to-db, disabled
* feat(mcp): expose LangBot management as an MCP server at /mcp
Add an MCP (Model Context Protocol) server so external AI agents can manage a
LangBot instance. Reuses the same API-key auth as the HTTP API (including the
config.yaml global API key).
- pkg/api/mcp/server.py: FastMCP server wrapping the service layer; 21 curated
tools across system/bots/pipelines/models/knowledge/mcp-servers/skills
- pkg/api/mcp/mount.py: ASGI dispatcher fronting Quart; authenticates /mcp
requests with an API key, runs the streamable-HTTP session manager lifespan
- controller/main.py: serve the wrapped ASGI app via hypercorn (was run_task)
- web: new 'MCP' tab in the API integration dialog showing endpoint, auth, and
client config; i18n for 8 locales
- tests/manual/mcp_smoke.py: e2e check (401 unauth, list tools, call tools)
Tool surface is intentionally curated (not all ~25 route groups) to keep the
agent surface small, safe, and maintainable. Extend deliberately.
* feat(skills): add in-repo skills/ as the single source of truth
Migrate the agent skills + QA/e2e test harness from the (now archived)
langbot-app/langbot-skills repo into LangBot/skills/, and add four new skills.
Migrated:
- langbot-plugin-dev, langbot-testing (e2e), langbot-env-setup,
langbot-skills-maintenance, langbot-eba-adapter-dev
- the bin/lbs CLI (src/, test/, scripts/, schemas/, qa-agent-docs/)
New:
- langbot-dev core backend + web development
- langbot-deploy Docker/K8s deployment + config.yaml + global API key
- langbot-mcp-ops operating the LangBot MCP server (/mcp)
- langbot-space-ops operating the Space marketplace MCP server
- src/cli.ts repoRoot(): recognize the skills assets root (skills.index.json +
bin/lbs) so the CLI works when nested inside the LangBot repo
- README.md: unified skill catalog; skills.index.json regenerated
Parity with source verified: bin/lbs validate + node test suite match the
source repo (only the uncommitted .lbpkg build-artifact fixture differs).
* docs(agents): document agent-facing surfaces + API/MCP/skills sync rule
* docs(readme): add 'Built for AI Agents' section across all locales
Highlight MCP server, in-repo skills (single source of truth), AGENTS.md
sync rule, and llms.txt. Cross-link LangBot Space MCP marketplace.
* style(mcp): fix ruff format + prettier lint in MCP server and API panel
* style(web): prettier format MCP i18n locale entries
* docs(skills): note MCP instance control in dev/testing skills
All development-guidance skills now point to the LangBot instance MCP
server (/mcp) and the Space marketplace MCP server, reusing API keys.
Update _normalize_stream_tool_calls to preserve provider_specific_fields
(including thought_signature) from streaming tool call chunks. Also preserve
provider_specific_fields from delta in invoke_llm_stream.
This ensures Gemini's thought_signature is round-tripped correctly:
1. LiteLLM extracts thought_signature from Gemini response
2. It's preserved in Message/ToolCall entities (via SDK changes)
3. _convert_messages includes it in the next request
Also add unit tests for provider_specific_fields round-tripping.
Fixes: langbot-app/LangBot#1899
Ollama's OpenAI-compatible streaming endpoint emits a tool-call delta
carrying an `index` and a `function` payload but never an OpenAI-style
`id`. `_normalize_stream_tool_calls` dropped any tool call without an
`id`, so a tool-only turn yielded neither content nor a tool call: the
stream "completed" with 0 chars, the tool never ran, and the chat
appeared stuck. Models on standard OpenAI APIs (e.g. SiliconFlow) were
unaffected because they always send a `call_...` id.
Synthesize a stable per-index id (`call_<index>`) when the provider
omits one but a function name is present. Providers that do send ids
keep theirs, and parallel id-less calls keep distinct ids.
Adds regression tests for the single and multi id-less tool-call cases.
Fixes#2261
Outbound attachment collection (pipeline wrapper) runs on every turn
regardless of inbound files, but the agent was only told the per-query
outbox path inside the inbound-attachment note in LocalAgentRunner. So on
pure-generation turns (e.g. "generate a QR code"/chart/mermaid where the
user sent no file), the agent never learned the outbox path or the
query_id, wrote the generated file nowhere deliverable, and it was
silently dropped.
Move the outbox instruction into BoxService.get_system_guidance(query_id),
which is injected as a system message on every turn the exec tool is
available. The inbound note keeps its own (now redundant but harmless)
outbox line. Add unit tests asserting the outbox path is present with a
query_id and absent without one.
The agent attachment outbox is written by the sandbox container as root over
the bind-mount, so the LangBot host process (non-root) cannot rmtree those
files — the host-side delete failed silently and stale files were re-collected
on a later turn that reused the same query_id (the query_id counter resets to 0
on every restart).
- BoxService.initialize now purges leftover inbox/outbox after the runtime is
available: host rmtree first, then an in-sandbox 'rm -rf' via exec for any
root-owned survivors.
- _clear_outbox now falls back to exec when the host delete leaves root-owned
files behind, instead of silently failing.
- collect_outbound_attachments clears the outbox unconditionally (even on an
empty collection) so a reused query_id never inherits stale files.
- Tests: startup purge (host-owned + root-owned exec fallback + no-workspace
noop) and empty-collection-still-clears.
* feat(box): bidirectional attachment transfer for sandbox
Materialize inbound attachments into the sandbox workspace so agents can
process user-sent files, and collect agent-produced files from the outbox
to attach them back to the reply.
- box(service): add materialize_inbound_attachments / collect_outbound
attachments. Prefer direct host-filesystem read/write on the bind-mounted
workspace (no size limit), falling back to chunked exec only for
non-shared backends (e2b/remote). Clear per-query inbox/outbox dirs at
turn start to avoid query_id-reuse collisions.
- provider(localagent): inject inbound attachment descriptors into the
sandbox and append a system note telling the agent the inbox/outbox paths.
- pipeline(wrapper): collect outbox files on the final stream chunk and
append them as attachment components to the response chain.
- web(debug-dialog): render File components with a download link when
base64/url is present; add base64/path fields to the File entity.
- tests: cover inbound/outbound, large-file transfer without truncation,
and stale-dir clearing (86 passing).
* feat(box): support voice/file attachment round-trip end-to-end
Extends the bidirectional attachment transfer to audio and arbitrary files
through the real webchat UI, and fixes the model-payload errors that
non-image attachments triggered.
- platform(websocket_adapter): resolve Voice/File component storage keys to
base64 (previously only Image), so audio/documents reach the sandbox inbox.
- web(debug-dialog): accept audio/* and any file in the uploader (was
image-only), classify by mimetype, upload Voice/File via the documents
endpoint, and render non-image staged attachments as a chip.
- provider(litellmchat): drop non-image file parts (file_base64 / file_url)
when building the OpenAI/LiteLLM payload. These come from Voice/File
attachments — including ones replayed from conversation history — and the
agent reads their bytes from the sandbox, not the model. Without this the
provider rejects the request: 'invalid content type=file_base64'.
- provider(localagent): also strip those parts from the current user message
alongside the sandbox-path note (model-facing clarity; the requester is the
real safety net for history).
- tests: cover the requester strip/keep behavior (file dropped, image kept and
reshaped to image_url, mixed history, plain-string content).
* test(box): cover inbound/outbound attachment helpers; fix ruff format
- ruff format localagent.py (CI ruff format --check was failing)
- add unit tests for ResponseWrapper outbound-attachment helpers (wrapper.py 78%->98%)
- add unit tests for LocalAgentRunner._inject_inbound_attachments
- add unit tests for WebSocketAdapter._process_image_components (0%->covered)
Lifts PR patch coverage from 68.97% to ~88% (>75% target).
huanghuoguoguo