Commit Graph

125 Commits

Author SHA1 Message Date
Junyan Qin 1c9b54d02c Merge remote-tracking branch 'origin/master' into dev/4.11.x
# Conflicts:
#	src/langbot/pkg/pipeline/preproc/preproc.py
#	src/langbot/pkg/pipeline/process/handlers/chat.py
#	src/langbot/pkg/provider/runners/localagent.py
#	src/langbot/pkg/provider/tools/toolmgr.py
#	src/langbot/templates/metadata/pipeline/ai.yaml
#	tests/unit_tests/test_preproc.py
#	web/src/app/home/components/dynamic-form/DynamicFormComponent.tsx
#	web/src/app/home/pipelines/components/pipeline-form/PipelineFormComponent.tsx
2026-06-30 21:07:13 +08:00
advancer-young 096ec1a8ce feat(mcp): support mcp resources (#2215)
* feat(mcp): support mcp resources

* feat(web): split MCP resources into tab

* docs: add MCP resources PR review

* feat(mcp): productionize resource support

* feat(mcp): scope local agent tools and resources

* fix(web): gate space embedding models behind login

* fix(web): prevent clipped space model CTA

* test: update preproc resource tool expectations

* fix(web): expose skill authoring tools in selector

---------

Co-authored-by: yang.xiang <yang.xiang@advancegroup.com>
Co-authored-by: Hyu <chenhyu@proton.me>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-06-30 19:16:30 +08:00
彼方 5c8c0eb17b test: use package import paths (#2297) 2026-06-30 10:51:34 +08:00
彼方 ccc51522cf fix(aiocqhttp): normalize base64 media payloads (#2296)
* fix(aiocqhttp): normalize image and voice base64 payloads

* fix(aiocqhttp): support file messages from base64 payloads

* test(aiocqhttp): use package import path
2026-06-30 10:51:09 +08:00
huanghuoguoguo db1879164d test(provider): support fake token counting 2026-06-29 23:15:51 +08:00
wangcham a1066d502c fix(platform): align eba event declarations 2026-06-28 17:44:12 +08:00
Hyu b97cc800d3 feat(platform): migrate bot admins from config.yaml to database (#2289)
* feat(platform): migrate bot admins from config.yaml to database

- Add BotAdmin ORM model (bot_admins table) scoped per bot_uuid
- Add Alembic migration 0007 to create table and migrate legacy config admins
- Remove top-level admins key from config.yaml template
- Add GET/POST/DELETE /api/v1/platform/bots/<uuid>/admins endpoints
- Update cmdmgr privilege check to query bot_admins table (bot-scoped)
- Add BotAdminsPanel frontend component in bot detail sessions tab
- Add i18n keys (zh-Hans, en-US)

* fix(ci): ruff/prettier format, fix test_importutil assertion

* fix(ci): prettier format BackendClient.ts and i18n locales

* fix(ci): eslint-prettier fix BackendClient.ts single-param formatting

* refactor: move admin management into session monitor, fix session_id enum format

- Remove standalone admins tab, replace with BotAdminsDialog in session monitor
- Add admin toggle button inline in chat header next to Active status
- Add BotAdminsDialog component with useBotAdmins hook
- Fix session_id written as LauncherTypes.PERSON_xxx instead of person_xxx
  (monitoring_helper.py x4, pipelinemgr.py x1 missing .value on launcher_type)
- Fix duplicate platform/person label in session list and chat header
- Migrate existing malformed session_id records in DB

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-06-27 17:18:19 +08:00
huanghuoguoguo 3de16bbbae fix(debug-chat): preserve websocket pipeline routing 2026-06-27 01:31:18 +08:00
huanghuoguoguo d0f6fe2cec feat(agent-runner): support scoped token counting 2026-06-27 01:31:08 +08:00
彼方 48905ea080 feat(plugin): report deferred response delivery failures (#2287)
* feat(plugin): report deferred response delivery failures

* style: fix ruff format issues in plugin_diagnostics and test_handler_actions

---------

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-06-26 23:45:10 +08:00
Junyan Qin 1577567a78 feat(eba): consolidate event bindings, fix command.tsx pointer-events
- Replace legacy pipeline binding card + RoutingRulesEditor with unified
  EventBindingsEditor; remove use_pipeline_uuid/pipeline_routing_rules
  from bot form schema and API update handler
- Add _augment_event_data() to botmgr for filter virtual fields
  (message_text, message_element_types, chat_type)
- Add alembic migration 0009: migrate use_pipeline_uuid and
  pipeline_routing_rules into event_bindings on first run
- Fix command.tsx: data-[disabled] -> data-[disabled=true] so cmdk 1.x
  items (data-disabled=false) are not pointer-events:none
- EventBindingsEditor: onSelect on CommandItems, filter conditions panel,
  disabled bindings section, dnd reorder
- i18n: add filter/condition keys for zh-Hans and en-US
- Update tests to match new bot service behavior

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 16:27:25 +08:00
RockChinQ 9c22a1521c fix(box): defer separated workspace ownership to runtime 2026-06-25 00:09:40 -04:00
huanghuoguoguo d02e1a14a3 test(agent): cover agent service event bindings 2026-06-25 00:04:59 +08:00
huanghuoguoguo cd6a39d3a2 test(agent): cover pluginized agent runner runtime 2026-06-24 20:46:32 +08:00
Junyan Qin bf61547083 Merge remote-tracking branch 'origin/master' into pr-2277
# Conflicts:
#	pyproject.toml
#	uv.lock
2026-06-23 19:56:23 +08:00
RockChinQ 59b2a7cd51 fix(monitoring): hide disabled box status on cloud 2026-06-23 06:40:05 -04:00
huanghuoguoguo a332583750 test(agent): align tests with agent runner plugin path 2026-06-22 23:48:02 +08:00
huanghuoguoguo 55b098bf39 test(agent): remove obsolete local agent runner attachment tests 2026-06-22 23:26:11 +08:00
huanghuoguoguo 1c4713138f Merge remote-tracking branch 'origin/feat/agent-runner-plugin' into dev_4_11
# Conflicts:
#	src/langbot/pkg/pipeline/preproc/preproc.py
2026-06-22 23:06:45 +08:00
huanghuoguoguo dbb39663bc Merge remote-tracking branch 'origin/refactor/eba' into dev_4_11
# Conflicts:
#	pyproject.toml
#	src/langbot/pkg/pipeline/preproc/preproc.py
#	uv.lock
2026-06-22 23:06:02 +08:00
huanghuoguoguo 2b03095d4e refactor(tools): unify tool-detail normalization in ToolManager
Drop the PluginToolLoader.get_tool() override that returned a raw
ComponentManifest, so every loader's get_tool() now returns a uniform
resource_tool.LLMTool (PluginToolLoader.get_tools() already did this
conversion). This removes the only source of tool-shape heterogeneity.

- ToolManager.get_tool_schema(): drop the ComponentManifest-vs-LLMTool branch
- ToolManager.get_tool_detail(): new host-level shape {name, description,
  human_desc, parameters}
- handler.py GET_TOOL_DETAIL: call tool_mgr.get_tool_detail(); delete the
  handler-local _build_tool_detail + _i18n_to_dict/_i18n_to_text adapters and
  the litellm TODO
- ToolLookupResult is now just LLMTool

The dropped label/spec fields were not consumed by any runner (local-agent
build_llm_tool and external harnesses use only name/description/parameters).
2026-06-22 13:39:45 +08:00
huanghuoguoguo c7d4885bfc refactor(plugin): split agent-runner action handlers out of handler.py
Extract the AgentRunner Protocol v1 host-side surface from the giant
RuntimeConnectionHandler.__init__ into sibling modules using a registration-
function pattern (behavior-preserving; @h.action == @self.action):

- agent_run_support.py: shared constants + authorization/scope/projection helpers
- agent_pull_actions.py: register(h) for history/event pull APIs
- agent_runner_actions.py: register(h) for run/runtime/stats/claim lifecycle
- agent_state_actions.py: register(h) for steering/state APIs

__init__ now calls the three register(self) functions. handler.py keeps the
pre-existing plugin/llm/vector/knowledge handlers, get_prompt/call_tool/
get_tool_detail (coupled to retained helpers), shared helpers, and outbound
methods; it re-imports _validate_agent_run_session so external imports keep
working. handler.py: 4066 -> 1871 lines.

test_state_api_auth.py: repoint get_session_registry patch targets to
agent_run_support (the lookup moved modules). 385 agent unit tests pass; ruff clean.
2026-06-22 13:08:34 +08:00
RockChinQ 85743cc75f fix(tests): make Postgres migration head test revision-agnostic
The PostgreSQL migration test had the same hardcoded 0005 head
assertion as the SQLite one; resolve the actual head from the Alembic
ScriptDirectory so 0006 (and future migrations) don't break it.
2026-06-21 12:10:20 -04:00
RockChinQ c689b10c0d fix(mcp): ruff format remote-mode files; make migration head test revision-agnostic
CI follow-up to the local/remote MCP work:

- Apply ruff format to provider/tools/loaders/mcp.py and the 0006
  normalize-remote-mode migration (Lint job failed on formatting).
- test_migrations.py hardcoded the head revision as 0005_*, which broke
  once 0006 landed. Resolve the actual head from the Alembic
  ScriptDirectory so future migrations don't require editing the test.
2026-06-21 12:04:37 -04:00
huanghuoguoguo 190028d5ab feat(skill): unify skill activation as authorized tools
Expose skill tools (activate/register_skill/native exec) like native tools
instead of gating them behind the skill_authoring capability:
- toolmgr.get_all_tools drops include_skill_authoring; SkillToolLoader
  self-gates on sandbox + skill_mgr
- preproc drops the include_skill_authoring branch; pipeline-bound skills
  and the skills resource gate on skill_mgr presence

Persist activated skills into host.activated_skills conversation state so
they survive across runs (host writes at activate; last-write-wins); drop
the dead restore_activated_skills helper.

Prefill ToolResource.parameters host-side (tool_mgr.get_tool_schema) so
runners build LLM tools without per-tool get_tool_detail round-trips.

Align agent-runner-pluginization design docs to the all-tool model.
2026-06-21 09:27:05 +08:00
huanghuoguoguo cede35b31b feat(agent-runner): add plugin runner host integration 2026-06-20 20:12:02 +08:00
Junyan Chin e9dd584792 feat: MCP server + in-repo skills (agent-friendly platform) (#2269)
* feat(api): support global API key from config.yaml (api.global_api_key)

Accept a config-defined global API key anywhere a web-UI key is accepted
(X-API-Key / Bearer), with no login session and no DB record. Useful for
automated deployments and AI agents (HTTP API + MCP). Defaults to empty
(disabled); does not require the lbk_ prefix.

- templates/config.yaml: add api.global_api_key with security notes
- service/apikey.py: verify_api_key checks global key first (constant-time)
- docs/API_KEY_AUTH.md: document the global key + security guidance
- tests: cover global-key match, prefix-free, fallback-to-db, disabled

* feat(mcp): expose LangBot management as an MCP server at /mcp

Add an MCP (Model Context Protocol) server so external AI agents can manage a
LangBot instance. Reuses the same API-key auth as the HTTP API (including the
config.yaml global API key).

- pkg/api/mcp/server.py: FastMCP server wrapping the service layer; 21 curated
  tools across system/bots/pipelines/models/knowledge/mcp-servers/skills
- pkg/api/mcp/mount.py: ASGI dispatcher fronting Quart; authenticates /mcp
  requests with an API key, runs the streamable-HTTP session manager lifespan
- controller/main.py: serve the wrapped ASGI app via hypercorn (was run_task)
- web: new 'MCP' tab in the API integration dialog showing endpoint, auth, and
  client config; i18n for 8 locales
- tests/manual/mcp_smoke.py: e2e check (401 unauth, list tools, call tools)

Tool surface is intentionally curated (not all ~25 route groups) to keep the
agent surface small, safe, and maintainable. Extend deliberately.

* feat(skills): add in-repo skills/ as the single source of truth

Migrate the agent skills + QA/e2e test harness from the (now archived)
langbot-app/langbot-skills repo into LangBot/skills/, and add four new skills.

Migrated:
- langbot-plugin-dev, langbot-testing (e2e), langbot-env-setup,
  langbot-skills-maintenance, langbot-eba-adapter-dev
- the bin/lbs CLI (src/, test/, scripts/, schemas/, qa-agent-docs/)

New:
- langbot-dev      core backend + web development
- langbot-deploy   Docker/K8s deployment + config.yaml + global API key
- langbot-mcp-ops  operating the LangBot MCP server (/mcp)
- langbot-space-ops operating the Space marketplace MCP server

- src/cli.ts repoRoot(): recognize the skills assets root (skills.index.json +
  bin/lbs) so the CLI works when nested inside the LangBot repo
- README.md: unified skill catalog; skills.index.json regenerated

Parity with source verified: bin/lbs validate + node test suite match the
source repo (only the uncommitted .lbpkg build-artifact fixture differs).

* docs(agents): document agent-facing surfaces + API/MCP/skills sync rule

* docs(readme): add 'Built for AI Agents' section across all locales

Highlight MCP server, in-repo skills (single source of truth), AGENTS.md
sync rule, and llms.txt. Cross-link LangBot Space MCP marketplace.

* style(mcp): fix ruff format + prettier lint in MCP server and API panel

* style(web): prettier format MCP i18n locale entries

* docs(skills): note MCP instance control in dev/testing skills

All development-guidance skills now point to the LangBot instance MCP
server (/mcp) and the Space marketplace MCP server, reusing API keys.
2026-06-20 15:14:47 +08:00
huanghuoguoguo acfac42107 fix(litellmchat): preserve provider_specific_fields for Gemini thought_signature (#2265)
Update _normalize_stream_tool_calls to preserve provider_specific_fields
(including thought_signature) from streaming tool call chunks. Also preserve
provider_specific_fields from delta in invoke_llm_stream.

This ensures Gemini's thought_signature is round-tripped correctly:
1. LiteLLM extracts thought_signature from Gemini response
2. It's preserved in Message/ToolCall entities (via SDK changes)
3. _convert_messages includes it in the next request

Also add unit tests for provider_specific_fields round-tripping.

Fixes: langbot-app/LangBot#1899
2026-06-19 23:26:12 +08:00
huanghuoguoguo 492827ea75 Add plugin rerank invocation action (#2242) 2026-06-19 23:25:54 +08:00
Junyan Chin 3d5b70cc5d fix(modelmgr): keep id-less streamed tool calls (Ollama) (#2262)
Ollama's OpenAI-compatible streaming endpoint emits a tool-call delta
carrying an `index` and a `function` payload but never an OpenAI-style
`id`. `_normalize_stream_tool_calls` dropped any tool call without an
`id`, so a tool-only turn yielded neither content nor a tool call: the
stream "completed" with 0 chars, the tool never ran, and the chat
appeared stuck. Models on standard OpenAI APIs (e.g. SiliconFlow) were
unaffected because they always send a `call_...` id.

Synthesize a stable per-index id (`call_<index>`) when the provider
omits one but a function name is present. Providers that do send ids
keep theirs, and parallel id-less calls keep distinct ids.

Adds regression tests for the single and multi id-less tool-call cases.

Fixes #2261
2026-06-19 18:07:25 +08:00
RockChinQ 83623f6afe fix(box): always advertise outbox path in exec guidance
Outbound attachment collection (pipeline wrapper) runs on every turn
regardless of inbound files, but the agent was only told the per-query
outbox path inside the inbound-attachment note in LocalAgentRunner. So on
pure-generation turns (e.g. "generate a QR code"/chart/mermaid where the
user sent no file), the agent never learned the outbox path or the
query_id, wrote the generated file nowhere deliverable, and it was
silently dropped.

Move the outbox instruction into BoxService.get_system_guidance(query_id),
which is injected as a system message on every turn the exec tool is
available. The inbound note keeps its own (now redundant but harmless)
outbox line. Add unit tests asserting the outbox path is present with a
query_id and absent without one.
2026-06-19 04:09:45 -04:00
huanghuoguoguo a020ca680f Harden agent runner tool runtimes (#2247)
* fix(tools): harden agent runner tool runtimes

* fix(tools): bootstrap Python workspaces with available interpreter

* fix(tools): clear stale Python workspace env locks

* fix(tools): decouple runtime from agent runner

* test(tools): cover runtime hardening edge cases

* fix(tools): support binary workspace file chunks
2026-06-18 14:06:04 +00:00
huanghuoguoguo 5fe63ce822 Bound Space model sync startup wait (#2248)
* fix(modelmgr): bound Space model sync startup wait

* style(provider): format model manager
2026-06-18 22:00:33 +08:00
Junyan Chin 6b15a732e4 fix(box): purge leftover inbox/outbox on startup; clear root-owned outbox via exec (#2259)
The agent attachment outbox is written by the sandbox container as root over
the bind-mount, so the LangBot host process (non-root) cannot rmtree those
files — the host-side delete failed silently and stale files were re-collected
on a later turn that reused the same query_id (the query_id counter resets to 0
on every restart).

- BoxService.initialize now purges leftover inbox/outbox after the runtime is
  available: host rmtree first, then an in-sandbox 'rm -rf' via exec for any
  root-owned survivors.
- _clear_outbox now falls back to exec when the host delete leaves root-owned
  files behind, instead of silently failing.
- collect_outbound_attachments clears the outbox unconditionally (even on an
  empty collection) so a reused query_id never inherits stale files.
- Tests: startup purge (host-owned + root-owned exec fallback + no-workspace
  noop) and empty-collection-still-clears.
2026-06-18 21:59:48 +08:00
Junyan Chin a1e6eccdeb feat(box): bidirectional attachment transfer for sandbox (#2257)
* feat(box): bidirectional attachment transfer for sandbox

Materialize inbound attachments into the sandbox workspace so agents can
process user-sent files, and collect agent-produced files from the outbox
to attach them back to the reply.

- box(service): add materialize_inbound_attachments / collect_outbound
  attachments. Prefer direct host-filesystem read/write on the bind-mounted
  workspace (no size limit), falling back to chunked exec only for
  non-shared backends (e2b/remote). Clear per-query inbox/outbox dirs at
  turn start to avoid query_id-reuse collisions.
- provider(localagent): inject inbound attachment descriptors into the
  sandbox and append a system note telling the agent the inbox/outbox paths.
- pipeline(wrapper): collect outbox files on the final stream chunk and
  append them as attachment components to the response chain.
- web(debug-dialog): render File components with a download link when
  base64/url is present; add base64/path fields to the File entity.
- tests: cover inbound/outbound, large-file transfer without truncation,
  and stale-dir clearing (86 passing).

* feat(box): support voice/file attachment round-trip end-to-end

Extends the bidirectional attachment transfer to audio and arbitrary files
through the real webchat UI, and fixes the model-payload errors that
non-image attachments triggered.

- platform(websocket_adapter): resolve Voice/File component storage keys to
  base64 (previously only Image), so audio/documents reach the sandbox inbox.
- web(debug-dialog): accept audio/* and any file in the uploader (was
  image-only), classify by mimetype, upload Voice/File via the documents
  endpoint, and render non-image staged attachments as a chip.
- provider(litellmchat): drop non-image file parts (file_base64 / file_url)
  when building the OpenAI/LiteLLM payload. These come from Voice/File
  attachments — including ones replayed from conversation history — and the
  agent reads their bytes from the sandbox, not the model. Without this the
  provider rejects the request: 'invalid content type=file_base64'.
- provider(localagent): also strip those parts from the current user message
  alongside the sandbox-path note (model-facing clarity; the requester is the
  real safety net for history).
- tests: cover the requester strip/keep behavior (file dropped, image kept and
  reshaped to image_url, mixed history, plain-string content).

* test(box): cover inbound/outbound attachment helpers; fix ruff format

- ruff format localagent.py (CI ruff format --check was failing)
- add unit tests for ResponseWrapper outbound-attachment helpers (wrapper.py 78%->98%)
- add unit tests for LocalAgentRunner._inject_inbound_attachments
- add unit tests for WebSocketAdapter._process_image_components (0%->covered)

Lifts PR patch coverage from 68.97% to ~88% (>75% target).
2026-06-18 21:40:31 +08:00
huanghuoguoguo f390980d0a test: format test suite (#2252) 2026-06-16 11:22:29 +08:00
huanghuoguoguo 1ae5aacc00 test: add frontend smoke and backend e2e CI (#2251) 2026-06-16 11:09:55 +08:00
huanghuoguoguo e9fe2f2d43 feat(agent-runner): support host tool lookup (#2244) 2026-06-14 11:29:57 +08:00
huanghuoguoguo 27be09ab15 fix(provider): preserve litellm usage details (#2246) 2026-06-14 11:12:29 +08:00
RockChinQ b7d8332cb0 feat(telemetry): include instance_create_ts in heartbeat payload
Load the instance creation timestamp from data/labels/instance_id.json
(backfilling+persisting it for instances created before the field existed),
expose it as constants.instance_create_ts, and include it in the heartbeat
payload so Space can anchor Time-To-Value / onboarding analytics on real
install time rather than first-heartbeat.

Verified: py_compile, ruff, pytest tests/unit_tests/telemetry/ (37 passed).
2026-06-13 11:13:18 -04:00
huanghuoguoguo 7fe3eedeea fix(provider): use LiteLLM input window for context length (#2243) 2026-06-13 21:27:47 +08:00
RockChinQ 672abfe95d refactor(core): remove pre-3.x legacy config migration system
The pkg/core/migrations system (m001-m043 DBMigration-style config
migrations, MigrationStage, and the core.migration base class) only ever
ran when upgrading from LangBot 3.x. The last 3.x release is over a year
old and is no longer supported, so this dead code is removed entirely:

- delete pkg/core/migrations/ (43 mXXX_*.py + __init__)
- delete pkg/core/migration.py (base class + registry)
- delete pkg/core/stages/migrate.py (MigrationStage)
- drop 'MigrationStage' from boot.py stage_order
- delete tests/unit_tests/core/test_migration.py (tested the removed base class)
2026-06-13 05:26:01 -04:00
huanghuoguoguo 9ecb587ac0 refactor(provider): use LiteLLM as unified LLM requester backend (#2150)
* refactor(provider): use LiteLLM as unified LLM requester backend

  - Replace 23+ individual requester implementations with unified litellmchat.py
  - Add litellm_provider field to 27 YAML manifests for provider routing
  - Delete redundant requester subclasses
  - Add unit tests for LiteLLMRequester (29 tests)
  - Fix num_retries parameter name (was max_retries)
  - Fix exception handling order for subclass exceptions

  LiteLLM provides unified API for 100+ providers, eliminating need for
  provider-specific requesters.

* fix: ruff format provider.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(provider): simplify LiteLLM requester usage handling

  - Remove unused Anthropic-specific tool schema generation
  - Share completion argument construction between normal and streaming calls
  - Use LiteLLM/OpenAI native usage fields for monitoring
  - Collect stream token usage from LiteLLM stream_options
  - Update LiteLLM requester tests for unified usage fields

* restore: restore deleted provider requester files

Restore individual provider requester implementations that were
removed in de61b5d3. These files coexist with the unified
litellmchat.py backend.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: update requesters and improve provider selection UI

- Added `litellm_provider` field to various requesters' YAML configurations.
- Removed obsolete Python requester files for OpenRouter, PPIO, QHAIGC, ShengSuanYun, SiliconFlow, Space, TokenPony, VolcArk, and Xai.
- Introduced new requesters for Tencent and Together AI with corresponding YAML configurations and SVG icons.
- Enhanced the ProviderForm component to include a searchable dropdown for selecting providers, improving user experience.
- Updated localization files to include search provider text for both English and Chinese.

* fix(provider): align litellm rebase with master

* fix(provider): capture streaming token usage; add token observability

The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).

- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
  endpoint: summary, per-model breakdown, token timeseries, and a
  zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
  per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization

Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.

* refactor(provider): simplify litellm capabilities

* style: simplify wrapped expressions

* feat(models): persist context metadata

* fix(provider): handle dict embeddings and openai-compatible rerank in LiteLLMRequester

- invoke_embedding: support both object- and dict-shaped response.data
  entries (OpenAI-compatible gateways like new-api return dicts)
- invoke_rerank: litellm.arerank rejects the 'openai' provider, so for
  openai-compatible (or unspecified) providers call the standard
  Jina/Cohere-style POST /v1/rerank endpoint directly over HTTP
- accept both 'relevance_score' and 'score' fields in rerank results
- add unit tests for the openai-compatible HTTP rerank path

* feat(provider): enforce requester support_type when adding models

- frontend: AddModelPopover only shows model-type tabs (llm/embedding/
  rerank) that the provider's requester declares in its manifest
  support_type; ModelsDialog fetches requester manifests and maps
  requester -> support_type, passed down through ProviderCard
- backend: add _validate_provider_supports guard in create_llm_model /
  create_embedding_model / create_rerank_model so a model cannot be
  attached to a provider whose requester does not support that type,
  even if the frontend restriction is bypassed (manifests without
  support_type are allowed for backward compatibility)
- manifests: correct support_type for providers that do not offer all
  three model types:
  - llm only: anthropic, deepseek, groq, moonshot, openrouter, xai
  - llm + text-embedding: openai, gemini, mistral
  - add rerank to new-api (verified working via /v1/rerank)
  - set llm + text-embedding + rerank for aggregator/unknown gateways

* feat(provider): add searchable alias to requester manifests

- add a free-text 'alias' field to every requester manifest spec,
  containing the vendor's English/Chinese names, pinyin, common
  nicknames and flagship model-series names (e.g. moonshot -> kimi,
  月之暗面; zhipu -> glm, 智谱清言)
- frontend: ProviderForm requester search now also matches against
  alias (substring/contains), so searching 'kimi' surfaces Moonshot,
  '硅基' surfaces SiliconFlow, etc.
- also fix support_type: openrouter (relay) supports embedding+rerank;
  LangBot Space gains rerank (coming soon)

* fix(provider): make support_type guard defensive against incomplete model_mgr

- _validate_provider_supports now uses getattr to gracefully skip when
  model_mgr / provider_dict / manifest lookup is unavailable, instead of
  raising AttributeError (fixes unit tests that mock ap.model_mgr as a
  bare SimpleNamespace)
- add TestValidateProviderSupports covering: allow supported type,
  reject unsupported type, allow when support_type missing, allow when
  provider unknown, degrade safely when model_mgr is incomplete

* fix(persistence): guard 0004 migration against missing llm_models table

The 0004_add_llm_model_context_length migration called
inspector.get_columns('llm_models') unconditionally, raising
NoSuchTableError when the table does not exist (e.g. migrating a
fresh/empty DB, as exercised by the integration tests where
create_all() registers no tables because the ORM models are not
imported). Every other migration guards with a table-existence check
first; add the same guard here for both upgrade and downgrade.

Also restore the test head assertion to 0004 (it had been lowered to
0003 to mask this failure).

* Merge branch 'master' into feat/litellm

Resolve conflicts:
- uv.lock: regenerated via 'uv lock' to reconcile litellm/fastuuid
  (ours) with openai bump (master).
- Alembic migrations: master added 0004_add_mcp_readme while this
  branch added 0004_add_llm_model_context_length, both as children of
  0003 (would create multiple heads). Re-chain the litellm migration as
  0005_add_llm_model_context_length with down_revision=0004_add_mcp_readme
  for a single linear head. Update test head assertion accordingly.

* fix(persistence): shorten migration revision id to fit varchar(32)

PostgreSQL stores alembic_version.version_num as varchar(32).
'0005_add_llm_model_context_length' (33 chars) overflowed it, raising
StringDataRightTruncationError in the PG migration tests. Rename the
revision (and file) to '0005_add_llm_context_length' (27 chars) and
update the head assertions in both SQLite and PostgreSQL migration
tests.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: fdc310 <2213070223@qq.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-06-13 16:59:48 +08:00
RockChinQ 2b6dcfe9c7 feat(survey): add bot_response_success_100 milestone trigger event
Counts successful non-WebSocket bot responses (persisted in the metadata
table as survey_bot_response_count, survives restarts) and fires the
bot_response_success_100 survey event once the instance reaches 100
responses. Counting stops after the milestone has been triggered.

Existing first_bot_response_success behavior unchanged. 6 new unit tests.
2026-06-12 09:40:07 -04:00
RockChinQ dd96da895c feat(telemetry): payload v2 with feature usage counters and instance heartbeat
Per-query events now carry event_type='query' and a features JSON object:
- tool_calls by source (native/plugin/mcp/skill) via ToolManager
- tool_call_rounds, kb usage (count/engine plugins/retrieved entries) via local-agent
- sandbox execs/errors via BoxService
- activated_skills and bound mcp_servers snapshots

New instance_heartbeat event (startup + daily) reports anonymous instance
profile: deploy platform, database/vdb kind, box backend/availability,
adapter type names, and resource counts. Respects space.disable_telemetry.

All collection helpers are defensive and never break the pipeline.
Verified: ruff, 37 telemetry unit tests (13 new), 504 box/provider/pipeline tests.
2026-06-12 08:11:43 -04:00
Junyan Qin b0b9221495 test(api): adapt bot service webhook tests to manifest-driven detection 2026-06-11 01:13:06 +08:00
Junyan Qin f4b3b87d7a Merge remote-tracking branch 'origin/master' into refactor/eba
# Conflicts:
#	pyproject.toml
#	uv.lock
2026-06-11 01:05:14 +08:00
RockChinQ 47ade18596 fix(log): roll daily log file at midnight for long-running processes
The log filename was computed once at init_logging() startup and the
RotatingFileHandler only rotated by size, so a process running across
midnight kept appending every subsequent day's logs to the start-day
file (langbot-<start date>.log). No file ever appeared for the current
day until the process was restarted, confusing users into thinking
logging had stopped.

Replace RotatingFileHandler with DailyGroupedRotatingFileHandler, which
switches to langbot-<current date>.log when the local date changes while
still doing size-based numbered rotation within a day. On-disk naming
stays compatible with the maintenance log-retention cleanup
(LOG_FILE_PATTERN). Adds regression tests.
2026-06-10 04:58:11 -04:00
Junyan Chin 8e558ad3a1 Feat/saas sandbox adaptation (#2234)
* fix(box): trust Box-reported skill paths when filesystem is not shared

In separated deployments (Docker Compose, k8s sidecar, --standalone-box,
remote runtime.endpoint) the Box runtime owns its own filesystem, so the
skill package_root it reports via list_skills is not resolvable on the
LangBot side. LangBot's reload_skills and build_skill_extra_mounts
validated those paths with os.path.isdir() against its own filesystem,
which silently dropped every skill in such deployments — breaking the
sandbox skill feature for the nsjail/SaaS backend.

Add BoxService.shares_filesystem_with_box, derived from the connector
transport (stdio = shared, WebSocket = separated), with an explicit
override seam for tests/embedders. Gate both isdir() guards on it: keep
local validation in shared-fs stdio mode, trust Box-reported paths
otherwise. The Box runtime only reports skills found on its own
filesystem, so those paths are valid there by construction.

Adds topology-derivation tests (real connector, no mocks) and
skill-retention tests for both shared and separated filesystems.

* build(docker): ship a self-contained nsjail sandbox backend in the image

Compile nsjail 3.6 from source in a dedicated multi-stage build and carry
only the binary plus its runtime libs (libprotobuf32, libnl-route-3-200)
into the final image. This lets the Box runtime isolate sandboxed code via
nsjail user/mount/pid/net namespaces without a host Docker socket — the
prerequisite for running Box on LangBot Cloud (k8s), where mounting
docker.sock would grant node root and is not acceptable for multi-tenant.

The build toolchain (build-essential/bison/flex/protobuf-dev/libnl-dev)
stays in the nsjail-build stage and is not present in the shipped image.

Verified: image builds (583MB), nsjail --help exits 0, libraries resolve,
and the real NsjailBackend executes an isolated command end-to-end on a
v6.1/cgroup2 host matching LangBot Cloud prod (rlimit fallback path, since
container /sys/fs/cgroup is read-only; PID-namespace isolation confirmed).

* feat(box): SaaS guard to force a single global sandbox scope

Add system.limitation.force_box_session_id_template: when non-empty it
overrides every pipeline's box-session-id-template at resolve time, pinning
all queries to one shared sandbox (e.g. {global}). This is the authoritative,
unbypassable guard — it runs on every exec call, so editing the pipeline
config via API cannot escape it. The web UI locks the Sandbox Scope selector
via a combined box_scope_editable flag (box available AND not forced).

* build(deps): pin langbot-plugin==0.4.2b1 (nsjail cgroup container-safety beta)

* fix(web): show forced sandbox scope + make disabled tooltip tap-friendly

When a SaaS deployment pins every pipeline to a fixed sandbox scope via
system.limitation.force_box_session_id_template, the Sandbox Scope selector was
correctly locked but still displayed the pipeline's stored value (e.g. the
per-chat default), misrepresenting the scope that the runtime actually enforces
on every exec. Coerce the displayed/saved value to the forced template so the
locked selector truthfully shows the active scope (e.g. Global).

Also fix the disabled_tooltip being invisible on touch devices: hover-only Radix
tooltips never open without a pointer, so the explanation of why the field is
locked could not be read on mobile. Wrap the info icon so a tap toggles the
tooltip while desktop hover still works.

* feat(web): hide sidebar new-version prompt for edition=cloud

Cloud instances are upgraded centrally by the operator, so surfacing a GitHub
'new version available' badge to tenants is misleading and actionable only by
the operator. Skip the release check entirely when edition=cloud.

* style(web): prettier formatting for DisabledTooltipIcon ternary

* chore(deps): bump langbot-plugin to 0.4.2b2

Picks up the SDK fix that creates a read-write host_path before the
nsjail bind-mount, fixing the SaaS MCP shared-workspace sandbox failure
(exec exit 255 with empty output when host_path didn't exist).

* chore(deps): bump langbot-plugin to 0.4.2b3

Picks up the nsjail /dev-node fix so stdio MCP servers (uvx-launched) can
start under force_global_sandbox instead of failing with 'Connection closed
/ please check URL'.

* fix(web): show real MCP runtime status on installed extensions list

The installed-extensions list badge keyed solely off the enable flag, so a
server that was still CONNECTING (or in ERROR) was shown as 'Connected'.
Reflect the actual runtime_info.status (connecting/connected/error/disabled)
with matching colors, and poll quietly every 3s while any MCP server is
connecting so the badge transitions without a manual refresh.

* chore(deps): bump langbot-plugin to 0.4.2b4

Picks up the 30s start_managed_process timeout so cold uvx MCP bootstraps
don't get torn down mid-install.

* style(web): satisfy prettier — parenthesize nullish-coalescing in ternary

* fix(mcp): isolate transient test sessions from the shared Box session

A config-page 'test' (server_name='_', no persisted UUID) ran in the same
shared 'mcp-shared' Box session as live MCP servers. A failing test (e.g.
empty args) churned that shared session and tore down healthy, already-
connected servers — leaving them stuck after exhausting their retries.

Mark UUID-less sessions as transient, give them their own isolated Box
session ('mcp-test-<uuid>'), and fully delete that session on cleanup so
tests can never disturb live servers and don't leak sessions.

* fix(mcp): tear down transient test session after test completes

A successful config-page test left its isolated 'mcp-test-<uuid>' Box
session running (the lifecycle task blocks until shutdown). Wrap the
transient test coroutine so it always shuts the session down afterward,
preventing isolated test sessions from leaking.
2026-06-09 19:30:17 +08:00
RockChinQ 7330732f62 fix(ci): bump migration head assertion to 0004, apply prettier
- Update test_migrations / test_migrations_postgres head assertion from
  0003 to 0004 after adding the mcp readme migration.
- Reformat MCPForm.tsx / MCPReadme.tsx to satisfy prettier/prettier.
2026-06-06 03:56:14 -04:00