Load the instance creation timestamp from data/labels/instance_id.json
(backfilling+persisting it for instances created before the field existed),
expose it as constants.instance_create_ts, and include it in the heartbeat
payload so Space can anchor Time-To-Value / onboarding analytics on real
install time rather than first-heartbeat.
Verified: py_compile, ruff, pytest tests/unit_tests/telemetry/ (37 passed).
The legacy pkg/persistence/migrations (DBMigration / dbmXXX) system now
coexists with Alembic but accepts no new migrations — all new schema
changes go through Alembic.
- remove dbm026_llm_model_context_length (superseded by Alembic
0005_add_llm_context_length, which makes the identical change)
- cap required_database_version at 25 (legacy chain dbm001-025 kept
read-only to upgrade pre-existing 3.x DBs to the Alembic baseline)
- add migrations/README.md documenting the freeze
- document the Alembic-only policy and revision-id/idempotency rules in
AGENTS.md
* refactor(provider): use LiteLLM as unified LLM requester backend
- Replace 23+ individual requester implementations with unified litellmchat.py
- Add litellm_provider field to 27 YAML manifests for provider routing
- Delete redundant requester subclasses
- Add unit tests for LiteLLMRequester (29 tests)
- Fix num_retries parameter name (was max_retries)
- Fix exception handling order for subclass exceptions
LiteLLM provides unified API for 100+ providers, eliminating need for
provider-specific requesters.
* fix: ruff format provider.py
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(provider): simplify LiteLLM requester usage handling
- Remove unused Anthropic-specific tool schema generation
- Share completion argument construction between normal and streaming calls
- Use LiteLLM/OpenAI native usage fields for monitoring
- Collect stream token usage from LiteLLM stream_options
- Update LiteLLM requester tests for unified usage fields
* restore: restore deleted provider requester files
Restore individual provider requester implementations that were
removed in de61b5d3. These files coexist with the unified
litellmchat.py backend.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat: update requesters and improve provider selection UI
- Added `litellm_provider` field to various requesters' YAML configurations.
- Removed obsolete Python requester files for OpenRouter, PPIO, QHAIGC, ShengSuanYun, SiliconFlow, Space, TokenPony, VolcArk, and Xai.
- Introduced new requesters for Tencent and Together AI with corresponding YAML configurations and SVG icons.
- Enhanced the ProviderForm component to include a searchable dropdown for selecting providers, improving user experience.
- Updated localization files to include search provider text for both English and Chinese.
* fix(provider): align litellm rebase with master
* fix(provider): capture streaming token usage; add token observability
The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).
- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
endpoint: summary, per-model breakdown, token timeseries, and a
zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization
Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.
* refactor(provider): simplify litellm capabilities
* style: simplify wrapped expressions
* feat(models): persist context metadata
* fix(provider): handle dict embeddings and openai-compatible rerank in LiteLLMRequester
- invoke_embedding: support both object- and dict-shaped response.data
entries (OpenAI-compatible gateways like new-api return dicts)
- invoke_rerank: litellm.arerank rejects the 'openai' provider, so for
openai-compatible (or unspecified) providers call the standard
Jina/Cohere-style POST /v1/rerank endpoint directly over HTTP
- accept both 'relevance_score' and 'score' fields in rerank results
- add unit tests for the openai-compatible HTTP rerank path
* feat(provider): enforce requester support_type when adding models
- frontend: AddModelPopover only shows model-type tabs (llm/embedding/
rerank) that the provider's requester declares in its manifest
support_type; ModelsDialog fetches requester manifests and maps
requester -> support_type, passed down through ProviderCard
- backend: add _validate_provider_supports guard in create_llm_model /
create_embedding_model / create_rerank_model so a model cannot be
attached to a provider whose requester does not support that type,
even if the frontend restriction is bypassed (manifests without
support_type are allowed for backward compatibility)
- manifests: correct support_type for providers that do not offer all
three model types:
- llm only: anthropic, deepseek, groq, moonshot, openrouter, xai
- llm + text-embedding: openai, gemini, mistral
- add rerank to new-api (verified working via /v1/rerank)
- set llm + text-embedding + rerank for aggregator/unknown gateways
* feat(provider): add searchable alias to requester manifests
- add a free-text 'alias' field to every requester manifest spec,
containing the vendor's English/Chinese names, pinyin, common
nicknames and flagship model-series names (e.g. moonshot -> kimi,
月之暗面; zhipu -> glm, 智谱清言)
- frontend: ProviderForm requester search now also matches against
alias (substring/contains), so searching 'kimi' surfaces Moonshot,
'硅基' surfaces SiliconFlow, etc.
- also fix support_type: openrouter (relay) supports embedding+rerank;
LangBot Space gains rerank (coming soon)
* fix(provider): make support_type guard defensive against incomplete model_mgr
- _validate_provider_supports now uses getattr to gracefully skip when
model_mgr / provider_dict / manifest lookup is unavailable, instead of
raising AttributeError (fixes unit tests that mock ap.model_mgr as a
bare SimpleNamespace)
- add TestValidateProviderSupports covering: allow supported type,
reject unsupported type, allow when support_type missing, allow when
provider unknown, degrade safely when model_mgr is incomplete
* fix(persistence): guard 0004 migration against missing llm_models table
The 0004_add_llm_model_context_length migration called
inspector.get_columns('llm_models') unconditionally, raising
NoSuchTableError when the table does not exist (e.g. migrating a
fresh/empty DB, as exercised by the integration tests where
create_all() registers no tables because the ORM models are not
imported). Every other migration guards with a table-existence check
first; add the same guard here for both upgrade and downgrade.
Also restore the test head assertion to 0004 (it had been lowered to
0003 to mask this failure).
* Merge branch 'master' into feat/litellm
Resolve conflicts:
- uv.lock: regenerated via 'uv lock' to reconcile litellm/fastuuid
(ours) with openai bump (master).
- Alembic migrations: master added 0004_add_mcp_readme while this
branch added 0004_add_llm_model_context_length, both as children of
0003 (would create multiple heads). Re-chain the litellm migration as
0005_add_llm_model_context_length with down_revision=0004_add_mcp_readme
for a single linear head. Update test head assertion accordingly.
* fix(persistence): shorten migration revision id to fit varchar(32)
PostgreSQL stores alembic_version.version_num as varchar(32).
'0005_add_llm_model_context_length' (33 chars) overflowed it, raising
StringDataRightTruncationError in the PG migration tests. Rename the
revision (and file) to '0005_add_llm_context_length' (27 chars) and
update the head assertions in both SQLite and PostgreSQL migration
tests.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: fdc310 <2213070223@qq.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
* refactor: pipeline routing rules - add routed_by_rule bypass and diagnostic logging
- Add routing rules editor (RoutingRulesEditor component)
- Add routed_by_rule bypass logic in response rules
- Add diagnostic logging for pipeline routing
- Database migration for bot pipeline routing rules
- Extract RoutingRulesEditor component from BotForm
- Revert log levels to debug
* feat: add message_has_element routing rule type
Support routing by message element type (Image, Voice, File, Forward,
Face, At, AtAll, Quote) with eq/neq operators.
* test: add unit tests for pipeline routing rules
20 tests covering _match_operator (eq/neq/contains/not_contains/
starts_with/regex/invalid) and resolve_pipeline_uuid (launcher_type/
launcher_id/message_content/message_has_element/first-match-wins/
skip-invalid/default-operator).
* fix(web): add missing 'message_has_element' to routing rule type validation
The Zod schema and TypeScript type for PipelineRoutingRule.type were
missing the 'message_has_element' variant, causing silent form validation
failure when saving routing rules with this type.
* feat: add pipeline discard functionality and localization support
* feat(web): improve drag-and-drop with DragOverlay, add discard monitoring and pipeline icons
- Add DragOverlay for smooth cursor-following drag in routing rules editor
- Remove transition to eliminate redundant swap animation on drop
- Record discarded messages in monitoring system via _record_discarded_message
- Display pipeline name (Workflow icon) and runner name (Play icon) on session monitor messages
- Show discard badge on discarded messages in session monitor
- Add i18n translations for discarded/userMessage/botMessage
* fix: ensure discarded messages appear in session monitor and improve icons
- Create/update monitoring session for discarded messages so they show in
the bot session monitor (was only inserting message rows, not sessions)
- Use human-readable 'Discarded' as pipeline_name instead of '__discard__'
- Change runner icon from Play to Bot for better AI Agent semantics
* fix: merge discarded messages into same session and remove session-level pipeline name
- Use LauncherTypes enum for session_id in discarded messages to match
the format used by monitoring_helper (fixes duplicate sessions)
- Don't overwrite session pipeline info on discard — a session can have
messages from multiple pipelines
- Remove pipeline_name from session list and chat header since it's
now shown per-message and a session is no longer single-pipeline
* fix(web): only show save button on config tab in bot detail page
* fix(web): scroll to bottom after messages render in session monitor
---------
Co-authored-by: RockChinQ <rockchinq@gmail.com>
* feat: Implement WebSocket long connection client for WeChat Work AI Bot
- Added WecomBotWsClient to handle WebSocket connections for receiving messages and sending replies.
- Introduced a new migration (dbm022) to add 'enable-webhook' field to existing wecombot adapter configs, ensuring backward compatibility.
- Updated WecomBotAdapter to support both WebSocket and webhook modes based on the new configuration.
- Enhanced YAML configuration for WecomBot to include 'enable-webhook' and 'Secret' fields, adjusting requirements accordingly.
- Incremented database version to 22 to reflect schema changes.
* fix:db enable-webhook is false
* fix:add logic
* fix:Removed an unnecessary configuration check
* fix: migration
* fix: update migration
* fix:migration
* feat(rag): add knowledge base migration from v4.9.0 to plugin architecture
Rewrite dbm020 to backup old knowledge_bases data and preserve
external_knowledge_bases table. Add migration API endpoints and
frontend dialog so users can opt-in to auto-install LangRAG plugin
and restore their knowledge bases with original UUIDs preserved.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(rag): query marketplace for actual plugin version instead of 'latest'
The marketplace API does not support 'latest' as a version string.
Fetch the plugin info first to get latest_version, then use that
concrete version for installation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(rag): add data-only migration option and fix dialog width
Add option to migrate knowledge base data without auto-installing
the LangRAG plugin (for offline/intranet environments). Also
narrow the migration dialog to match other confirmation dialogs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: to red and no more
* fix lint
* fix ruff lint
* feat: add external migration
* fix: show
* feat: add external plugin auto download
* feat: update migration messages for knowledge base in multiple languages
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
* feat: add session message monitoring tab to bot detail dialog
Add a new "Sessions" tab in the bot detail dialog that displays
sent & received messages grouped by sessions. Users can select
any session to view its messages in a chat-bubble style layout.
Backend changes:
- Add sessionId filter to monitoring messages endpoint
- Add role column to MonitoringMessage (user/assistant)
- Record bot responses in monitoring via record_query_response()
- Add DB migration (dbm019) for the new role column
Frontend changes:
- New BotSessionMonitor component with session list + message viewer
- Add Sessions sidebar tab to BotDetailDialog
- Add getBotSessions/getSessionMessages API methods to BackendClient
- Add i18n translations (en-US, zh-Hans, zh-Hant, ja-JP)
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
* refactor: remove outdated version comment from PipelineManager class
* fix: bump required_database_version to 19 to trigger monitoring_messages.role migration
* fix: prevent session message auto-scroll from pushing dialog content out of view
Replace scrollIntoView (which scrolls all ancestor containers) with
direct scrollTop manipulation on the ScrollArea viewport. This keeps
the scroll contained within the messages panel only.
* ui: redesign BotSessionMonitor with polished chat UI
- Wider session list (w-72) with avatar circles and cleaner layout
- Richer chat header with avatar, platform info, and active indicator
- User messages now use blue-500 (solid) instead of blue-100 for
clear visual distinction
- Metadata (time, runner) shown on hover below bubbles, not inside
- Proper empty state illustrations for both panels
- Better spacing, rounded corners, and shadow treatment
- Consistent dark mode styling
* fix: infinite re-render loop in DynamicFormComponent
The useEffect depended on onSubmit which was a new closure every
parent render. Calling onSubmit inside the effect triggered parent
state update → re-render → new onSubmit ref → effect re-runs → loop.
Fix: use useRef to hold a stable reference to onSubmit, removing it
from the useEffect dependency array.
Also add DialogDescription to BotDetailDialog to suppress Radix
aria-describedby warning.
* fix: remove .html suffix from docs.langbot.app links (Mintlify migration)
* style: fix prettier and ruff formatting
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
* feat: add emoji support to knowledge bases and pipelines
* feat: add optional emoji property to ExternalKBCardVO for enhanced knowledge base representation
* Initial plan
* Add package structure and resource path utilities
- Created langbot/ package with __init__.py and __main__.py entry point
- Added paths utility to find frontend and resource files from package installation
- Updated config loading to use resource paths
- Updated frontend serving to use resource paths
- Added MANIFEST.in for package data inclusion
- Updated pyproject.toml with build system and entry points
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* Add PyPI publishing workflow and update license
- Created GitHub Actions workflow to build frontend and publish to PyPI
- Added license field to pyproject.toml to fix deprecation warning
- Updated .gitignore to exclude build artifacts
- Tested package building successfully
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* Add PyPI installation documentation
- Created PYPI_INSTALLATION.md with detailed installation and usage instructions
- Updated README.md to feature uvx/pip installation as recommended method
- Updated README_EN.md with same changes for English documentation
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* Address code review feedback
- Made package-data configuration more specific to langbot package only
- Improved path detection with caching to avoid repeated file I/O
- Removed sys.path searching which was incorrect for package data
- Removed interactive input() call for non-interactive environment compatibility
- Simplified error messages for version check
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* Fix code review issues
- Use specific exception types instead of bare except
- Fix misleading comments about directory levels
- Remove redundant existence check before makedirs with exist_ok=True
- Use context manager for file opening to ensure proper cleanup
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* Simplify package configuration and document behavioral differences
- Removed redundant package-data configuration, relying on MANIFEST.in
- Added documentation about behavioral differences between package and source installation
- Clarified that include-package-data=true uses MANIFEST.in for data files
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
* chore: update pyproject.toml
* chore: try pack templates in langbot/
* chore: update
* chore: update
* chore: update
* chore: update
* chore: update
* chore: adjust dir structure
* chore: fix imports
* fix: read default-pipeline-config.json
* fix: read default-pipeline-config.json
* fix: tests
* ci: publish pypi
* chore: bump version 4.6.0-beta.1 for testing
* chore: add templates/**
* fix: send adapters and requesters icons
* chore: bump version 4.6.0b2 for testing
* chore: add platform field for docker-compose.yaml
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
