Compare commits

...

16 Commits

Author SHA1 Message Date
WangCham
bbe019f0c6 fix: wrong agentid 2026-03-23 14:02:10 +08:00
copilot-swe-agent[bot]
def798bf1f fix: WeCom sender_name shows user ID instead of actual username
- Add get_user_info() to WecomClient to fetch user name via /user/get API
- Update WecomEventConverter.target2yiri to accept bot param and fetch real user name
- Update register_listener call to pass self.bot for user name lookup
- URL-encode userid parameter for safety

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-03-11 17:52:43 +00:00
copilot-swe-agent[bot]
5290834b8b Initial plan 2026-03-11 17:48:12 +00:00
Guanchao Wang
89064a9d5b feat: add support for username (#2047)
* feat: add support for username

* fix: lint

* fix: migerations

* fix: change to version 21

* fix: remove duplicate dbm021 migration and rename dbm022

* feat: add user_id and user_name display with copy functionality in BotSessionMonitor

---------

Co-authored-by: wangcham <wangcham@gmail.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-12 01:27:22 +08:00
RockChinQ
8c2aef3734 fix: prettier formatting for long URL strings 2026-03-11 07:05:45 -04:00
RockChinQ
3fb9e542b6 fix(web): use locale-aware data collection policy URL 2026-03-11 07:03:52 -04:00
RockChinQ
01844d8687 feat(web): add privacy & data collection policy consent to login/register pages 2026-03-11 06:50:54 -04:00
Copilot
2655425fbe fix: deduplicate final chunk yield in Dify chatflow streaming (#2049)
* Initial plan

* fix: prevent duplicate messages when Dify chatflow sends both workflow_finished and message_end events

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* style: apply ruff formatting to difysvapi.py

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-03-11 14:45:55 +08:00
youhuanghe
bd15b630b0 fix: chroma ruff lint 2026-03-11 04:07:21 +00:00
youhuanghe
fe5ce68436 feat(vector): add full-text and hybrid search support for Chroma backend
- Implement full-text search via Chroma's $contains filter
  - Implement hybrid search with RRF (Reciprocal Rank Fusion) combining
    vector and full-text results, with min-max normalized distances
  - Fix add_embeddings to use col.upsert instead of col.add for idempotency
  - Bump chromadb dependency to >=1.0.0,<2.0.0
  - Re-lock uv.lock with official PyPI source
2026-03-11 03:59:14 +00:00
Typer_Body
0541b05966 refactor: optimized error handling (#2020)
* Update output.yaml

* Update default-pipeline-config.json

* Update chat.py

* Add files via upload

* Update chat.py

* Update default-pipeline-config.json

* Update output.yaml

* Update constants.py

* feat: update logic

* fix: update required database version to 21

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-10 22:01:23 +08:00
youhuanghe
13cb0aa9be bugfix: rollback filter, add to retrive settings 2026-03-10 12:49:24 +00:00
youhuanghe
a048369b38 feat: Pass session context (session_name) to knowledge engine retrieval filters.
Allow KnowledgeEngine plugins to filter retrieval results by session,enabling per-session memory isolation in plugin-based knowledge bases
2026-03-10 12:27:50 +00:00
Junyan Qin
9ae0c263dc fix: update documentation links and translations for knowledge engine 2026-03-09 20:31:50 +08:00
Junyan Qin
a4e66f6459 feat: update version to 4.9.0 in pyproject.toml, __init__.py, and uv.lock 2026-03-09 20:10:01 +08:00
huanghuoguoguo
2a74a8d6ae Feat/dbm20 rag (#2037)
* feat(rag): add knowledge base migration from v4.9.0 to plugin architecture

Rewrite dbm020 to backup old knowledge_bases data and preserve
external_knowledge_bases table. Add migration API endpoints and
frontend dialog so users can opt-in to auto-install LangRAG plugin
and restore their knowledge bases with original UUIDs preserved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): query marketplace for actual plugin version instead of 'latest'

The marketplace API does not support 'latest' as a version string.
Fetch the plugin info first to get latest_version, then use that
concrete version for installation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(rag): add data-only migration option and fix dialog width

Add option to migrate knowledge base data without auto-installing
the LangRAG plugin (for offline/intranet environments). Also
narrow the migration dialog to match other confirmation dialogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: to red and no more

* fix lint

* fix ruff lint

* feat: add external migration

* fix: show

* feat: add external plugin auto download

* feat: update migration messages for knowledge base in multiple languages

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-09 20:05:38 +08:00
41 changed files with 1738 additions and 445 deletions

View File

@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.8.7"
version = "4.9.0"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -61,7 +61,7 @@ dependencies = [
"html2text>=2024.2.26",
"langchain>=0.2.0",
"langchain-text-splitters>=0.0.1",
"chromadb>=0.4.24",
"chromadb>=1.0.0,<2.0.0",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.3.0",

View File

@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.8.7'
__version__ = '4.9.0'

View File

@@ -4,6 +4,7 @@ import base64
import binascii
import httpx
import traceback
from urllib.parse import quote
from quart import Quart
import xml.etree.ElementTree as ET
from typing import Callable, Dict, Any
@@ -67,6 +68,31 @@ class WecomClient:
await self.logger.error(f'获取accesstoken失败:{response.json()}')
raise Exception(f'未获取access token: {data}')
async def get_user_info(self, userid: str) -> dict:
"""
Get user information by user ID using the application secret.
Args:
userid: The user ID to look up.
Returns:
dict: User information including 'name' field.
"""
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/user/get?access_token=' + self.access_token + '&userid=' + quote(userid)
async with httpx.AsyncClient() as client:
response = await client.get(url)
data = response.json()
if data.get('errcode') == 40014 or data.get('errcode') == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.get_user_info(userid)
if data.get('errcode', 0) != 0:
await self.logger.error(f'获取用户信息失败:{data}')
return {}
return data
async def get_users(self):
if not self.check_access_token_for_contacts():
self.access_token_for_contacts = await self.get_access_token(self.secret_for_contacts)

View File

@@ -10,6 +10,7 @@ from typing import Callable
from .wecomcsevent import WecomCSEvent
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import aiofiles
import time
class WecomCSClient:
@@ -34,6 +35,10 @@ class WecomCSClient:
self.unified_mode = unified_mode
self.app = Quart(__name__)
# Customer info cache: {external_userid: (info_dict, timestamp)}
self._customer_cache: dict[str, tuple[dict, float]] = {}
self._cache_ttl = 60 # Cache TTL in seconds (1 minute)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
@@ -378,3 +383,53 @@ class WecomCSClient:
async def get_media_id(self, image: platform_message.Image):
media_id = await self.upload_to_work(image=image)
return media_id
async def get_customer_info(self, external_userid: str) -> dict | None:
"""
Get customer information by external_userid with caching.
Uses a 1-minute cache to avoid repeated API calls for the same user.
Args:
external_userid: The external user ID of the customer.
Returns:
Customer info dict with 'nickname', 'avatar', etc., or None if not found.
"""
# Check cache first
current_time = time.time()
if external_userid in self._customer_cache:
cached_info, cached_time = self._customer_cache[external_userid]
if current_time - cached_time < self._cache_ttl:
return cached_info
# Cache miss or expired, fetch from API
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = f'{self.base_url}/kf/customer/batchget?access_token={self.access_token}'
payload = {
'external_userid_list': [external_userid],
}
async with httpx.AsyncClient() as client:
response = await client.post(url, json=payload)
data = response.json()
if data.get('errcode') in [40014, 42001]:
self.access_token = await self.get_access_token(self.secret)
return await self.get_customer_info(external_userid)
if data.get('errcode', 0) != 0:
if self.logger:
await self.logger.warning(f'Failed to get customer info: {data}')
return None
customer_list = data.get('customer_list', [])
if customer_list:
customer_info = customer_list[0]
# Store in cache
self._customer_cache[external_userid] = (customer_info, current_time)
return customer_info
return None

View File

@@ -0,0 +1,372 @@
import asyncio
import json
import httpx
import quart
import sqlalchemy
from ... import group
from ......core import taskmgr
from ......entity.persistence import metadata as persistence_metadata
from langbot_plugin.runtime.plugin.mgr import PluginInstallSource
LANGRAG_PLUGIN_AUTHOR = 'langbot-team'
LANGRAG_PLUGIN_NAME = 'LangRAG'
LANGRAG_PLUGIN_ID = f'{LANGRAG_PLUGIN_AUTHOR}/{LANGRAG_PLUGIN_NAME}'
DEFAULT_SPACE_URL = 'https://space.langbot.app'
# Old Retriever plugin_name -> New Connector plugin_name
EXTERNAL_PLUGIN_NAME_MAPPING = {
'DifyDatasetsRetriever': 'DifyDatasetsConnector',
'RAGFlowRetriever': 'RAGFlowConnector',
'FastGPTRetriever': 'FastGPTConnector',
}
# Per-plugin: which old retriever_config fields belong to creation_settings.
# Remaining fields go to retrieval_settings.
# None means ALL fields go to creation_settings (no retrieval_schema).
EXTERNAL_PLUGIN_CREATION_FIELDS: dict[str, set[str] | None] = {
'langbot-team/DifyDatasetsConnector': {'api_base_url', 'dify_apikey', 'dataset_id'},
'langbot-team/RAGFlowConnector': {'api_base_url', 'api_key', 'dataset_ids'},
'langbot-team/FastGPTConnector': None, # all fields -> creation_settings
}
@group.group_class('knowledge/migration', '/api/v1/knowledge/migration')
class KnowledgeMigrationRouterGroup(group.RouterGroup):
async def _get_migration_flag(self) -> bool:
"""Check if rag_plugin_migration_needed flag is set."""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_metadata.Metadata).where(
persistence_metadata.Metadata.key == 'rag_plugin_migration_needed'
)
)
row = result.first()
return row is not None and row.value == 'true'
async def _set_migration_flag(self, value: str):
"""Set rag_plugin_migration_needed flag."""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_metadata.Metadata)
.where(persistence_metadata.Metadata.key == 'rag_plugin_migration_needed')
.values(value=value)
)
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _install_plugin_from_marketplace(
self, plugin_id: str, task_context: taskmgr.TaskContext, space_url: str
) -> None:
"""Install a single plugin from the marketplace."""
p_author, p_name = plugin_id.split('/', 1)
self.ap.logger.info(f'RAG migration: installing plugin {plugin_id} from marketplace...')
task_context.trace(f'Installing plugin {plugin_id} from marketplace...')
async with httpx.AsyncClient(trust_env=True, timeout=15) as client:
resp = await client.get(f'{space_url}/api/v1/marketplace/plugins/{p_author}/{p_name}')
resp.raise_for_status()
p_data = resp.json().get('data', {}).get('plugin', {})
p_version = p_data.get('latest_version')
if not p_version:
raise Exception(f'Could not determine latest version for {plugin_id}')
await self.ap.plugin_connector.install_plugin(
PluginInstallSource.MARKETPLACE,
{
'plugin_author': p_author,
'plugin_name': p_name,
'plugin_version': p_version,
},
task_context=task_context,
)
self.ap.logger.info(f'RAG migration: plugin {plugin_id} install request sent.')
async def _execute_rag_migration(self, task_context: taskmgr.TaskContext, install_plugin: bool = True):
"""Execute RAG migration: install required plugins and restore backup data."""
warnings = []
# Collect all plugins we need: LangRAG (always) + connector plugins (from external KBs)
needed_plugins: dict[str, str] = {
LANGRAG_PLUGIN_ID: LANGRAG_PLUGIN_NAME,
}
has_external = await self._table_exists('external_knowledge_bases')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT DISTINCT plugin_author, plugin_name FROM external_knowledge_bases;')
)
for row in result.fetchall():
plugin_author = row[0] or ''
plugin_name = row[1] or ''
mapped_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
plugin_id = f'{plugin_author}/{mapped_name}'
if plugin_id not in needed_plugins:
needed_plugins[plugin_id] = mapped_name
self.ap.logger.info(f'RAG migration: plugins needed: {list(needed_plugins.keys())}')
if install_plugin:
# Step 1: Install all required plugins from marketplace
task_context.trace('Installing required plugins...', action='install-plugin')
space_url = self.ap.instance_config.data.get('space', {}).get('url', DEFAULT_SPACE_URL).rstrip('/')
for plugin_id in needed_plugins:
try:
await self._install_plugin_from_marketplace(plugin_id, task_context, space_url)
except Exception as e:
self.ap.logger.warning(f'RAG migration: plugin {plugin_id} install returned: {e}')
task_context.trace(f'Plugin install note ({plugin_id}): {e}')
# Step 2: Wait for all plugins to become available as knowledge engines
task_context.trace(
f'Waiting for plugins to become available: {list(needed_plugins.keys())}...',
action='wait-plugin',
)
max_retries = 30
engine_id_set: set[str] = set()
for i in range(max_retries):
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
pass
if all(pid in engine_id_set for pid in needed_plugins):
self.ap.logger.info(f'RAG migration: all plugins ready: {engine_id_set}')
task_context.trace('All required plugins are ready.')
break
if i == max_retries - 1:
still_missing = [pid for pid in needed_plugins if pid not in engine_id_set]
warning = f'Plugin(s) {still_missing} did not become available after {max_retries} retries'
self.ap.logger.warning(f'RAG migration: {warning}')
warnings.append(warning)
task_context.trace(warning)
await asyncio.sleep(2)
else:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
engine_id_set = set()
# Step 3: Restore internal knowledge bases from backup
task_context.trace('Restoring internal knowledge bases...', action='restore-internal')
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM knowledge_bases_backup;')
)
rows = result.fetchall()
columns = result.keys()
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f4da')
embedding_model_uuid = row_dict.get('embedding_model_uuid', '')
top_k = row_dict.get('top_k', 5)
created_at = row_dict.get('created_at')
updated_at = row_dict.get('updated_at')
creation_settings = json.dumps({'embedding_model_uuid': embedding_model_uuid})
retrieval_settings = json.dumps({'top_k': top_k})
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=updated_at,
plugin_id=LANGRAG_PLUGIN_ID,
collection_id=kb_uuid,
creation_settings=creation_settings,
retrieval_settings=retrieval_settings,
)
)
try:
config = {'embedding_model_uuid': embedding_model_uuid}
await self.ap.plugin_connector.rag_on_kb_create(LANGRAG_PLUGIN_ID, kb_uuid, config)
task_context.trace(f'Restored internal KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 4: Restore external knowledge bases
task_context.trace('Restoring external knowledge bases...', action='restore-external')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM external_knowledge_bases;')
)
rows = result.fetchall()
columns = result.keys()
self.ap.logger.info(
f'RAG migration: {len(rows)} external KB(s) to restore. Available engines: {engine_id_set}'
)
task_context.trace(f'Found {len(rows)} external KB(s). Available engines: {engine_id_set}')
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f517')
plugin_author = row_dict.get('plugin_author', '')
plugin_name = row_dict.get('plugin_name', '')
retriever_config = row_dict.get('retriever_config', {})
created_at = row_dict.get('created_at')
mapped_plugin_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
external_plugin_id = f'{plugin_author}/{mapped_plugin_name}'
self.ap.logger.info(
f'RAG migration: processing external KB "{name}" ({kb_uuid}), '
f'plugin: {plugin_author}/{plugin_name} -> {external_plugin_id}'
)
if isinstance(retriever_config, str):
try:
retriever_config = json.loads(retriever_config)
except (json.JSONDecodeError, TypeError):
retriever_config = {}
creation_fields = EXTERNAL_PLUGIN_CREATION_FIELDS.get(external_plugin_id)
if creation_fields is None:
creation_settings_dict = retriever_config
retrieval_settings_dict = {}
else:
creation_settings_dict = {k: v for k, v in retriever_config.items() if k in creation_fields}
retrieval_settings_dict = {k: v for k, v in retriever_config.items() if k not in creation_fields}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=created_at,
plugin_id=external_plugin_id,
collection_id=kb_uuid,
creation_settings=json.dumps(creation_settings_dict),
retrieval_settings=json.dumps(retrieval_settings_dict),
)
)
if external_plugin_id not in engine_id_set:
warning = (
f'External KB "{name}" ({kb_uuid}) record saved, but plugin {external_plugin_id} '
f'is not installed yet. Install the connector plugin to use it.'
)
warnings.append(warning)
task_context.trace(warning)
else:
try:
await self.ap.plugin_connector.rag_on_kb_create(
external_plugin_id, kb_uuid, creation_settings_dict
)
task_context.trace(f'Restored external KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for external KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 5: Clear migration flag
await self._set_migration_flag('false')
task_context.trace('RAG migration completed.', action='done')
if warnings:
task_context.trace(f'Completed with {len(warnings)} warning(s).')
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
internal_kb_count = 0
external_kb_count = 0
if needed:
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases_backup;')
)
internal_kb_count = result.scalar() or 0
if await self._table_exists('external_knowledge_bases'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
external_kb_count = result.scalar() or 0
return self.success(
data={
'needed': needed,
'internal_kb_count': internal_kb_count,
'external_kb_count': external_kb_count,
}
)
@self.route('/execute', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
data = await quart.request.get_json(silent=True) or {}
install_plugin = data.get('install_plugin', True)
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self._execute_rag_migration(task_context=ctx, install_plugin=install_plugin),
kind='rag-migration',
name='rag-migration-execute',
label='Migrating knowledge bases to plugin architecture',
context=ctx,
)
return self.success(data={'task_id': wrapper.id})
@self.route('/dismiss', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
await self._set_migration_flag('false')
return self.success()

View File

@@ -30,6 +30,7 @@ class MonitoringService:
level: str = 'info',
platform: str | None = None,
user_id: str | None = None,
user_name: str | None = None,
runner_name: str | None = None,
variables: str | None = None,
role: str = 'user',
@@ -49,6 +50,7 @@ class MonitoringService:
'level': level,
'platform': platform,
'user_id': user_id,
'user_name': user_name,
'runner_name': runner_name,
'variables': variables,
'role': role,
@@ -152,6 +154,7 @@ class MonitoringService:
pipeline_name: str,
platform: str | None = None,
user_id: str | None = None,
user_name: str | None = None,
) -> None:
"""Record a new session"""
session_data = {
@@ -166,6 +169,7 @@ class MonitoringService:
'is_active': True,
'platform': platform,
'user_id': user_id,
'user_name': user_name,
}
await self.ap.persistence_mgr.execute_async(

View File

@@ -20,6 +20,7 @@ class MonitoringMessage(Base):
level = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # info, warning, error, debug
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # User display name
runner_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # Runner name for this query
variables = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # Query variables as JSON string
role = sqlalchemy.Column(sqlalchemy.String(50), nullable=True, default='user') # user, assistant
@@ -64,6 +65,7 @@ class MonitoringSession(Base):
is_active = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True, index=True)
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # User display name
class MonitoringError(Base):

View File

@@ -1,5 +1,3 @@
import json
import sqlalchemy
from .. import migration
@@ -9,20 +7,22 @@ class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
"""Migrate to unified Knowledge Engine plugin architecture.
Changes:
- Add knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings columns to knowledge_bases
- Migrate existing top_k values into retrieval_settings JSON
- Migrate existing embedding_model_uuid into creation_settings JSON
- Drop embedding_model_uuid and top_k columns (PostgreSQL only; SQLite leaves them unmapped)
- Drop external_knowledge_bases table (no longer needed; external KB data is not migrated)
- Backup existing knowledge_bases data to knowledge_bases_backup
- Clear knowledge_bases table and add new plugin architecture columns
- Drop old columns (PostgreSQL only; SQLite leaves them unmapped)
- Preserve external_knowledge_bases table as-is for future migration
- Set rag_plugin_migration_needed flag in metadata if old data exists
"""
async def upgrade(self):
"""Upgrade"""
has_internal_data = await self._backup_knowledge_bases()
has_external_data = await self._check_external_knowledge_bases()
await self._clear_knowledge_bases()
await self._add_columns_to_knowledge_bases()
await self._migrate_top_k_to_retrieval_settings()
await self._migrate_embedding_model_uuid_to_creation_settings()
await self._drop_old_columns()
await self._drop_external_knowledge_bases_table()
if has_internal_data or has_external_data:
await self._set_migration_flag()
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
@@ -57,6 +57,50 @@ class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
)
return result.first() is not None
async def _backup_knowledge_bases(self) -> bool:
"""Backup knowledge_bases data. Returns True if data was backed up."""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases;'))
count = result.scalar()
if count == 0:
return False
# Drop backup table if it already exists (from a previous failed migration)
if await self._table_exists('knowledge_bases_backup'):
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE knowledge_bases_backup;'))
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE TABLE knowledge_bases_backup AS SELECT * FROM knowledge_bases;')
)
self.ap.logger.info(
'Backed up %d knowledge base(s) to knowledge_bases_backup table.',
count,
)
return True
async def _check_external_knowledge_bases(self) -> bool:
"""Check if external_knowledge_bases table exists and has data.
The table is preserved as-is (not dropped) for future migration.
"""
if not await self._table_exists('external_knowledge_bases'):
return False
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
count = result.scalar()
if count > 0:
self.ap.logger.info(
'Found %d external knowledge base(s) in external_knowledge_bases table. '
'Table preserved for future migration.',
count,
)
return count > 0
async def _clear_knowledge_bases(self):
"""Clear all rows from knowledge_bases table (preserve table structure)."""
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DELETE FROM knowledge_bases;'))
async def _add_columns_to_knowledge_bases(self):
"""Add new RAG plugin architecture columns to knowledge_bases table."""
columns = await self._get_table_columns('knowledge_bases')
@@ -74,73 +118,6 @@ class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
sqlalchemy.text(f'ALTER TABLE knowledge_bases ADD COLUMN {col_name} {col_type};')
)
# For existing knowledge bases without knowledge_engine_plugin_id,
# set collection_id = uuid (same default as new KBs)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE knowledge_bases SET collection_id = uuid WHERE collection_id IS NULL;')
)
async def _migrate_top_k_to_retrieval_settings(self):
"""Migrate existing top_k values into retrieval_settings JSON."""
columns = await self._get_table_columns('knowledge_bases')
if 'top_k' not in columns:
return
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT uuid, top_k FROM knowledge_bases WHERE top_k IS NOT NULL AND retrieval_settings IS NULL;'
)
)
rows = result.fetchall()
for row in rows:
kb_uuid = row[0]
top_k = row[1]
retrieval_settings = json.dumps({'top_k': top_k})
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE knowledge_bases SET retrieval_settings = :rs WHERE uuid = :uuid;').bindparams(
rs=retrieval_settings, uuid=kb_uuid
)
)
async def _migrate_embedding_model_uuid_to_creation_settings(self):
"""Migrate existing embedding_model_uuid into creation_settings JSON."""
columns = await self._get_table_columns('knowledge_bases')
if 'embedding_model_uuid' not in columns:
return
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT uuid, embedding_model_uuid, creation_settings FROM knowledge_bases '
"WHERE embedding_model_uuid IS NOT NULL AND embedding_model_uuid != '';"
)
)
rows = result.fetchall()
for row in rows:
kb_uuid = row[0]
emb_uuid = row[1]
existing_settings = row[2]
if existing_settings and isinstance(existing_settings, str):
try:
settings = json.loads(existing_settings)
except (json.JSONDecodeError, TypeError):
settings = {}
elif isinstance(existing_settings, dict):
settings = existing_settings
else:
settings = {}
if 'embedding_model_uuid' not in settings:
settings['embedding_model_uuid'] = emb_uuid
new_settings = json.dumps(settings)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE knowledge_bases SET creation_settings = :cs WHERE uuid = :uuid;'
).bindparams(cs=new_settings, uuid=kb_uuid)
)
async def _drop_old_columns(self):
"""Drop embedding_model_uuid and top_k columns (PostgreSQL only).
@@ -162,22 +139,22 @@ class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN top_k;')
)
async def _drop_external_knowledge_bases_table(self):
"""Drop the external_knowledge_bases table if it exists."""
if await self._table_exists('external_knowledge_bases'):
# Log existing external KBs before dropping, so users are aware of data loss
rows = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM external_knowledge_bases;')
async def _set_migration_flag(self):
"""Set rag_plugin_migration_needed flag in metadata table."""
# Check if the key already exists
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT value FROM metadata WHERE key = 'rag_plugin_migration_needed';")
)
row = result.first()
if row is not None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("UPDATE metadata SET value = 'true' WHERE key = 'rag_plugin_migration_needed';")
)
existing = rows.fetchall()
if existing:
self.ap.logger.warning(
'Dropping external_knowledge_bases table with %d existing record(s). '
'These external KB configurations will be removed: %s',
len(existing),
[dict(row._mapping) for row in existing],
)
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE external_knowledge_bases;'))
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("INSERT INTO metadata (key, value) VALUES ('rag_plugin_migration_needed', 'true');")
)
self.ap.logger.info('Set rag_plugin_migration_needed=true in metadata.')
async def downgrade(self):
"""Downgrade"""

View File

@@ -0,0 +1,74 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(21)
class DBMigrateMergeExceptionHandling(migration.DBMigration):
"""Merge hide-exception and block-failed-request-output into a single exception-handling select option,
and add failure-hint field.
Conversion logic:
- block-failed-request-output=true -> exception-handling: hide
- hide-exception=true -> exception-handling: show-hint
- hide-exception=false -> exception-handling: show-error
"""
async def upgrade(self):
"""Upgrade"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
if 'output' not in config:
config['output'] = {}
if 'misc' not in config['output']:
config['output']['misc'] = {}
misc = config['output']['misc']
# Determine new exception-handling value from legacy fields
hide_exception = misc.get('hide-exception', True)
block_failed = misc.get('block-failed-request-output', False)
if block_failed:
exception_handling = 'hide'
elif hide_exception:
exception_handling = 'show-hint'
else:
exception_handling = 'show-error'
misc['exception-handling'] = exception_handling
# Add failure-hint with default value
misc['failure-hint'] = 'Request failed.'
# Remove legacy fields
misc.pop('hide-exception', None)
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,73 @@
import sqlalchemy
from .. import migration
@migration.migration_class(22)
class DBMigrateMonitoringUserId(migration.DBMigration):
"""Add user_id and user_name columns to monitoring_sessions table
This migration adds the missing user_id column and also ensures user_name
column exists (in case migration 21 failed or was skipped).
"""
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return bool(result.scalar())
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _add_column_if_not_exists(self, table_name: str, column_name: str, column_type: str):
"""Add a column to a table if it does not already exist."""
columns = await self._get_table_columns(table_name)
if column_name in columns:
self.ap.logger.debug('%s column already exists in %s.', column_name, table_name)
return
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE {table_name} ADD COLUMN {column_name} {column_type};')
)
self.ap.logger.info('Added %s column to %s table.', column_name, table_name)
async def upgrade(self):
# Check if monitoring_sessions table exists
if not await self._table_exists('monitoring_sessions'):
self.ap.logger.warning('monitoring_sessions table does not exist, skipping migration.')
return
# Add user_id column to monitoring_sessions table
await self._add_column_if_not_exists('monitoring_sessions', 'user_id', 'VARCHAR(255)')
# Add user_name column to monitoring_sessions table (in case migration 21 failed)
await self._add_column_if_not_exists('monitoring_sessions', 'user_name', 'VARCHAR(255)')
# Add user_name column to monitoring_messages table (in case migration 21 failed)
if await self._table_exists('monitoring_messages'):
await self._add_column_if_not_exists('monitoring_messages', 'user_name', 'VARCHAR(255)')
async def downgrade(self):
pass

View File

@@ -34,6 +34,15 @@ class MonitoringHelper:
# Check if session exists, if not, record session start
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Try to record message
# Use JSON serialization to preserve message chain structure (including image URLs, etc.)
if hasattr(query, 'message_chain') and hasattr(query.message_chain, 'model_dump'):
@@ -57,6 +66,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
variables=None, # Will be updated in record_query_success
)
@@ -80,6 +90,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
)
return message_id
@@ -128,6 +139,15 @@ class MonitoringHelper:
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Extract response content from resp_message_chain
if hasattr(query, 'resp_message_chain') and query.resp_message_chain:
# Serialize the last response message chain
@@ -162,6 +182,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
role='assistant',
)
@@ -183,6 +204,15 @@ class MonitoringHelper:
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Get sender name from message event
sender_name = None
if hasattr(query, 'message_event'):
if hasattr(query.message_event, 'sender'):
if hasattr(query.message_event.sender, 'nickname'):
sender_name = query.message_event.sender.nickname
elif hasattr(query.message_event.sender, 'member_name'):
sender_name = query.message_event.sender.member_name
# Record error message
message_id = await ap.monitoring_service.record_message(
bot_id=bot_id,
@@ -197,6 +227,7 @@ class MonitoringHelper:
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
user_name=sender_name,
runner_name=runner_name,
)

View File

@@ -149,12 +149,19 @@ class ChatMessageHandler(handler.MessageHandler):
self.ap.logger.error(f'Conversation({query.query_id}) Request Failed: {error_info}')
traceback.print_exc()
hide_exception_info = query.pipeline_config['output']['misc']['hide-exception']
exception_handling = query.pipeline_config['output']['misc'].get('exception-handling', 'show-hint')
if exception_handling == 'show-error':
user_notice = f'{e}'
elif exception_handling == 'show-hint':
user_notice = query.pipeline_config['output']['misc'].get('failure-hint', 'Request failed.')
else: # hide
user_notice = None
yield entities.StageProcessResult(
result_type=entities.ResultType.INTERRUPT,
new_query=query,
user_notice='请求失败' if hide_exception_info else f'{e}',
user_notice=user_notice,
error_notice=f'{e}',
debug_notice=traceback.format_exc(),
)

View File

@@ -148,51 +148,54 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
pass
if type(event) is platform_events.FriendMessage:
payload = {
'MsgType': 'text',
'Content': '',
'FromUserName': event.sender.id,
'ToUserName': bot_account_id,
'CreateTime': int(datetime.datetime.now().timestamp()),
'AgentID': event.sender.nickname,
}
wecom_event = WecomEvent.from_payload(payload=payload)
if not wecom_event:
raise ValueError('无法从 message_data 构造 WecomEvent 对象')
return wecom_event
return event.source_platform_object
@staticmethod
async def target2yiri(event: WecomEvent):
async def target2yiri(event: WecomEvent, bot: WecomClient = None):
"""
将 WecomEvent 转换为平台的 FriendMessage 对象。
Args:
event (WecomEvent): 企业微信事件。
bot (WecomClient): 企业微信客户端,用于获取用户信息。
Returns:
platform_events.FriendMessage: 转换后的 FriendMessage 对象。
"""
# Try to get the user's real name from the WeCom API
nickname = str(event.user_id)
if bot and event.user_id:
try:
user_info = await bot.get_user_info(event.user_id)
if user_info and user_info.get('name'):
nickname = user_info.get('name')
except Exception:
pass # Fall back to user_id as nickname
# 转换消息链
if event.type == 'text':
yiri_chain = await WecomMessageConverter.target2yiri(event.message, event.message_id)
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.agent_id),
nickname=nickname,
remark='',
)
return platform_events.FriendMessage(sender=friend, message_chain=yiri_chain, time=event.timestamp)
return platform_events.FriendMessage(
sender=friend, message_chain=yiri_chain, time=event.timestamp, source_platform_object=event
)
elif event.type == 'image':
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.agent_id),
nickname=nickname,
remark='',
)
yiri_chain = await WecomMessageConverter.target2yiri_image(picurl=event.picurl, message_id=event.message_id)
return platform_events.FriendMessage(sender=friend, message_chain=yiri_chain, time=event.timestamp)
return platform_events.FriendMessage(
sender=friend, message_chain=yiri_chain, time=event.timestamp, source_platform_object=event
)
class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
@@ -210,7 +213,6 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
'secret',
'token',
'EncodingAESKey',
'contacts_secret',
]
missing_keys = [key for key in required_keys if key not in config]
@@ -223,7 +225,7 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
secret=config['secret'],
token=config['token'],
EncodingAESKey=config['EncodingAESKey'],
contacts_secret=config['contacts_secret'],
contacts_secret=config.get('contacts_secret', ''), # Optional, kept for backward compatibility
logger=logger,
unified_mode=True,
api_base_url=config.get('api_base_url', 'https://qyapi.weixin.qq.com/cgi-bin'),
@@ -248,18 +250,17 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
):
Wecom_event = await WecomEventConverter.yiri2target(message_source, self.bot_account_id, self.bot)
content_list = await WecomMessageConverter.yiri2target(message, self.bot)
fixed_user_id = Wecom_event.user_id
# 删掉开头的u
fixed_user_id = fixed_user_id[1:]
# user_id is the original FromUserName from WecomEvent
user_id = Wecom_event.user_id
for content in content_list:
if content['type'] == 'text':
await self.bot.send_private_msg(fixed_user_id, Wecom_event.agent_id, content['content'])
await self.bot.send_private_msg(user_id, Wecom_event.agent_id, content['content'])
elif content['type'] == 'image':
await self.bot.send_image(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_image(user_id, Wecom_event.agent_id, content['media_id'])
elif content['type'] == 'voice':
await self.bot.send_voice(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_voice(user_id, Wecom_event.agent_id, content['media_id'])
elif content['type'] == 'file':
await self.bot.send_file(fixed_user_id, Wecom_event.agent_id, content['media_id'])
await self.bot.send_file(user_id, Wecom_event.agent_id, content['media_id'])
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
content_list = await WecomMessageConverter.yiri2target(message, self.bot)
@@ -287,7 +288,7 @@ class WecomAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def on_message(event: WecomEvent):
self.bot_account_id = event.receiver_id
try:
return await callback(await self.event_converter.target2yiri(event), self)
return await callback(await self.event_converter.target2yiri(event, self.bot), self)
except Exception:
await self.logger.error(f'Error in wecom callback: {traceback.format_exc()}')

View File

@@ -39,13 +39,6 @@ spec:
type: string
required: true
default: ""
- name: contacts_secret
label:
en_US: Contacts Secret
zh_Hans: 通讯录密钥
type: string
required: true
default: ""
- name: api_base_url
label:
en_US: API Base URL

View File

@@ -81,22 +81,33 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
return event.source_platform_object
@staticmethod
async def target2yiri(event: WecomCSEvent):
async def target2yiri(event: WecomCSEvent, bot: WecomCSClient = None):
"""
将 WecomEvent 转换为平台的 FriendMessage 对象。
Args:
event (WecomEvent): 企业微信客服事件。
bot (WecomCSClient): 企业微信客服客户端,用于获取用户信息。
Returns:
platform_events.FriendMessage: 转换后的 FriendMessage 对象。
"""
# Try to get customer nickname from WeChat API
nickname = str(event.user_id)
if bot and event.user_id:
try:
customer_info = await bot.get_customer_info(event.user_id)
if customer_info and customer_info.get('nickname'):
nickname = customer_info.get('nickname')
except Exception:
pass # Fall back to user_id as nickname
# 转换消息链
if event.type == 'text':
yiri_chain = await WecomMessageConverter.target2yiri(event.message, event.message_id)
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.user_id),
nickname=nickname,
remark='',
)
@@ -106,7 +117,7 @@ class WecomEventConverter(abstract_platform_adapter.AbstractEventConverter):
elif event.type == 'image':
friend = platform_entities.Friend(
id=f'u{event.user_id}',
nickname=str(event.user_id),
nickname=nickname,
remark='',
)
@@ -187,7 +198,7 @@ class WecomCSAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def on_message(event: WecomCSEvent):
self.bot_account_id = event.receiver_id
try:
return await callback(await self.event_converter.target2yiri(event), self)
return await callback(await self.event_converter.target2yiri(event, self.bot), self)
except Exception:
await self.logger.error(f'Error in wecomcs callback: {traceback.format_exc()}')

View File

@@ -441,6 +441,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
is_final = False
think_start = False
think_end = False
yielded_final = False
remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think')
@@ -493,13 +494,19 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if answer:
basic_mode_pending_chunk = answer
if (is_final or message_idx % 8 == 0) and (basic_mode_pending_chunk != '' or is_final):
if (
not yielded_final
and (is_final or message_idx % 8 == 0)
and (basic_mode_pending_chunk != '' or is_final)
):
# content, _ = self._process_thinking_content(basic_mode_pending_chunk)
yield provider_message.MessageChunk(
role='assistant',
content=basic_mode_pending_chunk,
is_final=is_final,
)
if is_final:
yielded_final = True
if chunk is None:
raise errors.DifyAPIError('Dify API 没有返回任何响应请检查网络连接和API配置')

View File

@@ -74,7 +74,13 @@ class LocalAgentRunner(runner.RequestRunner):
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found, skipping')
continue
result = await kb.retrieve(user_message_text)
result = await kb.retrieve(
user_message_text,
settings={
'sender_id': str(query.sender_id),
'session_name': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
},
)
if result:
all_results.extend(result)

View File

@@ -321,13 +321,19 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
if not plugin_id:
raise ValueError(f'No RAG plugin ID configured for KB {kb.uuid}. Retrieval failed.')
# Session context (e.g. session_name) stays in retrieval_settings
# for plugins that need it. Do NOT move them into filters, as filters
# are passed directly to vector_search by some plugins (e.g. LangRAG)
# and would cause empty results when the metadata field doesn't exist.
filters = settings.pop('filters', {})
retrieval_context = {
'query': query,
'knowledge_base_id': kb.uuid,
'collection_id': kb.collection_id or kb.uuid,
'retrieval_settings': settings,
'creation_settings': kb.creation_settings or {},
'filters': settings.pop('filters', {}),
'filters': filters,
}
result = await self.ap.plugin_connector.call_rag_retrieve(

View File

@@ -2,7 +2,7 @@ import langbot
semantic_version = f'v{langbot.__version__}'
required_database_version = 19
required_database_version = 22
"""Tag the version of the database schema, used to check if the database needs to be migrated"""
debug_mode = False

View File

@@ -2,11 +2,14 @@ from __future__ import annotations
import asyncio
from typing import Any
from chromadb import PersistentClient
from langbot.pkg.vector.vdb import VectorDatabase
from langbot.pkg.vector.vdb import VectorDatabase, SearchType
from langbot.pkg.core import app
import chromadb
import chromadb.errors
# RRF smoothing constant (standard value from the literature)
_RRF_K = 60
class ChromaVectorDatabase(VectorDatabase):
def __init__(self, ap: app.Application, base_path: str = './data/chroma'):
@@ -14,6 +17,10 @@ class ChromaVectorDatabase(VectorDatabase):
self.client = PersistentClient(path=base_path)
self._collections = {}
@classmethod
def supported_search_types(cls) -> list[SearchType]:
return [SearchType.VECTOR, SearchType.FULL_TEXT, SearchType.HYBRID]
async def get_or_create_collection(self, collection: str) -> chromadb.Collection:
if collection not in self._collections:
self._collections[collection] = await asyncio.to_thread(
@@ -34,8 +41,8 @@ class ChromaVectorDatabase(VectorDatabase):
kwargs: dict[str, Any] = dict(embeddings=embeddings_list, ids=ids, metadatas=metadatas)
if documents is not None:
kwargs['documents'] = documents
await asyncio.to_thread(col.add, **kwargs)
self.ap.logger.info(f"Added {len(ids)} embeddings to Chroma collection '{collection}'.")
await asyncio.to_thread(col.upsert, **kwargs)
self.ap.logger.info(f"Upserted {len(ids)} embeddings to Chroma collection '{collection}'.")
async def search(
self,
@@ -47,6 +54,23 @@ class ChromaVectorDatabase(VectorDatabase):
filter: dict[str, Any] | None = None,
) -> dict[str, Any]:
col = await self.get_or_create_collection(collection)
if search_type == SearchType.FULL_TEXT:
return await self._full_text_search(col, collection, k, query_text, filter)
elif search_type == SearchType.HYBRID:
return await self._hybrid_search(col, collection, query_embedding, k, query_text, filter)
# Default: vector search
return await self._vector_search(col, collection, query_embedding, k, filter)
async def _vector_search(
self,
col: chromadb.Collection,
collection: str,
query_embedding: list[float],
k: int,
filter: dict[str, Any] | None,
) -> dict[str, Any]:
query_kwargs: dict[str, Any] = dict(
query_embeddings=query_embedding,
n_results=k,
@@ -55,9 +79,137 @@ class ChromaVectorDatabase(VectorDatabase):
if filter:
query_kwargs['where'] = filter
results = await asyncio.to_thread(col.query, **query_kwargs)
self.ap.logger.info(f"Chroma search in '{collection}' returned {len(results.get('ids', [[]])[0])} results.")
self.ap.logger.info(
f"Chroma vector search in '{collection}' returned {len(results.get('ids', [[]])[0])} results."
)
return results
async def _full_text_search(
self,
col: chromadb.Collection,
collection: str,
k: int,
query_text: str,
filter: dict[str, Any] | None,
) -> dict[str, Any]:
if not query_text:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]], 'documents': [[]]}
get_kwargs: dict[str, Any] = dict(
where_document={'$contains': query_text},
include=['metadatas', 'documents'],
limit=k,
)
if filter:
get_kwargs['where'] = filter
results = await asyncio.to_thread(col.get, **get_kwargs)
# col.get returns flat lists; wrap into column-major format.
# Distances are all 0.0 because Chroma's local $contains is a boolean
# filter with no relevance scoring. Chroma's BM25 sparse embedding
# function (ChromaBm25EmbeddingFunction) can generate scored sparse
# vectors, but sparse vector *indexing* is only available on Chroma
# Cloud, not locally. For ranked results, use hybrid mode or apply a
# reranker in a downstream stage.
ids = results.get('ids', [])
metadatas = results.get('metadatas', []) or [None] * len(ids)
documents = results.get('documents', []) or [None] * len(ids)
distances = [0.0] * len(ids)
self.ap.logger.info(f"Chroma full-text search in '{collection}' returned {len(ids)} results.")
return {'ids': [ids], 'metadatas': [metadatas], 'distances': [distances], 'documents': [documents]}
async def _hybrid_search(
self,
col: chromadb.Collection,
collection: str,
query_embedding: list[float],
k: int,
query_text: str,
filter: dict[str, Any] | None,
) -> dict[str, Any]:
# Fall back to pure vector search when no text is provided
if not query_text:
return await self._vector_search(col, collection, query_embedding, k, filter)
# Run vector search and full-text search in parallel
vector_task = self._vector_search(col, collection, query_embedding, k, filter)
text_task = self._full_text_search(col, collection, k, query_text, filter)
vector_results, text_results = await asyncio.gather(vector_task, text_task)
vector_ids = vector_results.get('ids', [[]])[0]
text_ids = text_results.get('ids', [[]])[0]
if not vector_ids and not text_ids:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]], 'documents': [[]]}
# RRF fusion
fused = self._rrf_fuse([vector_ids, text_ids], k)
if not fused:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]], 'documents': [[]]}
fused_ids = [doc_id for doc_id, _ in fused]
# Fetch full metadata and documents for fused results
fetched = await asyncio.to_thread(col.get, ids=fused_ids, include=['metadatas', 'documents'])
# col.get returns results in arbitrary order; re-order to match fused ranking
fetched_map: dict[str, tuple] = {}
for i, fid in enumerate(fetched.get('ids', [])):
meta = (fetched.get('metadatas') or [None] * len(fetched['ids']))[i]
doc = (fetched.get('documents') or [None] * len(fetched['ids']))[i]
fetched_map[fid] = (meta, doc)
ordered_ids = []
ordered_metas = []
ordered_docs = []
ordered_dists = []
# Normalize RRF scores to 0~1 distances via min-max scaling.
# Raw RRF scores are tiny (e.g. 0.016~0.033 with k=60) so a naive
# ``1 - score`` would compress all distances into a narrow 0.96~0.98
# band with almost no discriminative power. Min-max normalization
# spreads them across the full 0~1 range (0.0 = best match).
max_score = fused[0][1]
min_score = fused[-1][1]
score_range = max_score - min_score
for doc_id, score in fused:
if doc_id in fetched_map:
meta, doc = fetched_map[doc_id]
ordered_ids.append(doc_id)
ordered_metas.append(meta)
ordered_docs.append(doc)
if score_range > 0:
ordered_dists.append(1.0 - (score - min_score) / score_range)
else:
ordered_dists.append(0.0)
self.ap.logger.info(
f"Chroma hybrid search in '{collection}' returned {len(ordered_ids)} results "
f'(vector={len(vector_ids)}, text={len(text_ids)}).'
)
return {
'ids': [ordered_ids],
'metadatas': [ordered_metas],
'distances': [ordered_dists],
'documents': [ordered_docs],
}
@staticmethod
def _rrf_fuse(result_lists: list[list[str]], k: int) -> list[tuple[str, float]]:
"""Reciprocal Rank Fusion over multiple ranked ID lists.
Returns a list of (doc_id, rrf_score) sorted by descending score,
truncated to *k* entries.
"""
scores: dict[str, float] = {}
for ranked_ids in result_lists:
for rank, doc_id in enumerate(ranked_ids):
scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (_RRF_K + rank + 1)
sorted_results = sorted(scores.items(), key=lambda x: x[1], reverse=True)
return sorted_results[:k]
async def delete_by_file_id(self, collection: str, file_id: str) -> None:
col = await self.get_or_create_collection(collection)
await asyncio.to_thread(col.delete, where={'file_id': file_id})

View File

@@ -95,11 +95,12 @@
"max": 0
},
"misc": {
"hide-exception": true,
"exception-handling": "show-hint",
"failure-hint": "Request failed.",
"at-sender": true,
"quote-origin": true,
"track-function-calls": false,
"remove-think": false
}
}
}
}

View File

@@ -78,13 +78,39 @@ stages:
en_US: Misc
zh_Hans: 杂项
config:
- name: hide-exception
- name: exception-handling
label:
en_US: Hide Exception
zh_Hans: 不输出异常信息给用户
type: boolean
en_US: Exception Handling Strategy
zh_Hans: 异常处理策略
description:
en_US: Controls how error messages are displayed to the user when an AI request fails
zh_Hans: 控制 AI 请求失败时向用户展示错误信息的方式
type: select
required: true
default: true
default: show-hint
options:
- name: show-error
label:
en_US: Show Full Error
zh_Hans: 显示完整报错信息
- name: show-hint
label:
en_US: Show Failure Hint
zh_Hans: 仅文字提示
- name: hide
label:
en_US: Hide All
zh_Hans: 不显示任何异常信息
- name: failure-hint
label:
en_US: Failure Hint Text
zh_Hans: 失败提示文本
description:
en_US: The text to display when a request fails. Only effective when Exception Handling Strategy is set to "Show Failure Hint"
zh_Hans: 请求失败时显示的提示文本,仅在异常处理策略设置为"仅文字提示"时生效
type: string
required: false
default: 'Request failed.'
- name: at-sender
label:
en_US: At Sender
@@ -119,3 +145,4 @@ stages:
type: boolean
required: true
default: false

522
uv.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,7 @@ import { httpClient } from '@/app/infra/http/HttpClient';
import { ScrollArea } from '@/components/ui/scroll-area';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils';
import { Copy, Check } from 'lucide-react';
import {
MessageChainComponent,
Plain,
@@ -27,6 +28,7 @@ interface SessionInfo {
is_active: boolean;
platform?: string | null;
user_id?: string | null;
user_name?: string | null;
}
interface SessionMessage {
@@ -60,8 +62,29 @@ export default function BotSessionMonitor({ botId }: BotSessionMonitorProps) {
const [messages, setMessages] = useState<SessionMessage[]>([]);
const [loadingSessions, setLoadingSessions] = useState(false);
const [loadingMessages, setLoadingMessages] = useState(false);
const [copiedUserId, setCopiedUserId] = useState(false);
const messagesContainerRef = useRef<HTMLDivElement>(null);
const parseSessionType = (sessionId: string): string | null => {
const idx = sessionId.indexOf('_');
if (idx === -1) return null;
const type = sessionId.slice(0, idx);
if (type === 'person' || type === 'group') return type;
return null;
};
const abbreviateId = (id: string): string => {
if (id.length <= 10) return id;
return `${id.slice(0, 4)}..${id.slice(-4)}`;
};
const copyUserId = (userId: string) => {
navigator.clipboard.writeText(userId).then(() => {
setCopiedUserId(true);
setTimeout(() => setCopiedUserId(false), 2000);
});
};
const loadSessions = useCallback(async () => {
setLoadingSessions(true);
try {
@@ -338,24 +361,36 @@ export default function BotSessionMonitor({ botId }: BotSessionMonitorProps) {
>
<div className="flex items-center justify-between mb-0.5">
<span className="text-sm font-medium truncate mr-2">
{session.user_id || session.session_id.slice(0, 12)}
{session.user_name ||
session.user_id ||
session.session_id.slice(0, 12)}
</span>
<span className="text-[11px] text-muted-foreground tabular-nums flex-shrink-0">
{formatRelativeTime(session.last_activity)}
</span>
</div>
<div className="flex items-center gap-1.5 text-xs text-muted-foreground">
{parseSessionType(session.session_id) && (
<span className="px-1 py-0.5 rounded bg-muted text-[10px]">
{parseSessionType(session.session_id)}
</span>
)}
{session.platform && (
<span className="px-1 py-0.5 rounded bg-muted text-[10px]">
{session.platform}
</span>
)}
{session.user_id && (
<span className="truncate text-[10px]">
{abbreviateId(session.user_id)}
</span>
)}
{session.is_active && (
<span className="flex items-center gap-0.5 text-green-600 dark:text-green-400">
<span className="w-1.5 h-1.5 rounded-full bg-green-500 inline-block" />
</span>
)}
<span>{session.pipeline_name}</span>
<span className="truncate">{session.pipeline_name}</span>
</div>
</button>
);
@@ -377,15 +412,42 @@ export default function BotSessionMonitor({ botId }: BotSessionMonitorProps) {
<div className="px-6 py-3 border-b shrink-0 flex items-center justify-between">
<div className="min-w-0">
<div className="text-sm font-medium truncate">
{selectedSession?.user_id || selectedSessionId.slice(0, 20)}
{selectedSession?.user_name ||
selectedSession?.user_id ||
selectedSessionId.slice(0, 20)}
</div>
<div className="flex items-center gap-2 text-xs text-muted-foreground">
{parseSessionType(selectedSessionId) && (
<span>{parseSessionType(selectedSessionId)}</span>
)}
{selectedSession?.platform && (
<span>{selectedSession.platform}</span>
<>
{parseSessionType(selectedSessionId) && <span>·</span>}
<span>{selectedSession.platform}</span>
</>
)}
{selectedSession?.user_id && (
<>
<span>·</span>
<span className="font-mono">
{selectedSession.user_id}
</span>
<button
onClick={() => copyUserId(selectedSession.user_id!)}
className="inline-flex items-center text-muted-foreground hover:text-foreground transition-colors"
title={t('common.copy')}
>
{copiedUserId ? (
<Check className="w-3 h-3 text-green-600" />
) : (
<Copy className="w-3 h-3" />
)}
</button>
</>
)}
{selectedSession?.pipeline_name && (
<>
{selectedSession?.platform && <span>·</span>}
<span>·</span>
<span>{selectedSession.pipeline_name}</span>
</>
)}

View File

@@ -422,12 +422,12 @@ export default function HomeSidebar({
const language = localStorage.getItem('langbot_language');
if (language === 'zh-Hans' || language === 'zh-Hant') {
window.open(
'https://docs.langbot.app/zh/insight/guide.html',
'https://docs.langbot.app/zh/insight/guide',
'_blank',
);
} else {
window.open(
'https://docs.langbot.app/en/insight/guide.html',
'https://docs.langbot.app/en/insight/guide',
'_blank',
);
}

View File

@@ -23,9 +23,9 @@ export const sidebarConfigList = [
route: '/home/bots',
description: t('bots.description'),
helpLink: {
en_US: 'https://docs.langbot.app/en/usage/platforms/readme.html',
zh_Hans: 'https://docs.langbot.app/zh/usage/platforms/readme.html',
ja_JP: 'https://docs.langbot.app/ja/usage/platforms/readme.html',
en_US: 'https://docs.langbot.app/en/usage/platforms/readme',
zh_Hans: 'https://docs.langbot.app/zh/usage/platforms/readme',
ja_JP: 'https://docs.langbot.app/ja/usage/platforms/readme',
},
}),
new SidebarChildVO({
@@ -44,9 +44,9 @@ export const sidebarConfigList = [
route: '/home/pipelines',
description: t('pipelines.description'),
helpLink: {
en_US: 'https://docs.langbot.app/en/usage/pipelines/readme.html',
zh_Hans: 'https://docs.langbot.app/zh/usage/pipelines/readme.html',
ja_JP: 'https://docs.langbot.app/ja/usage/pipelines/readme.html',
en_US: 'https://docs.langbot.app/en/usage/pipelines/readme',
zh_Hans: 'https://docs.langbot.app/zh/usage/pipelines/readme',
ja_JP: 'https://docs.langbot.app/ja/usage/pipelines/readme',
},
}),
new SidebarChildVO({
@@ -65,8 +65,8 @@ export const sidebarConfigList = [
route: '/home/monitoring',
description: t('monitoring.description'),
helpLink: {
en_US: 'https://docs.langbot.app/en/features/monitoring.html',
zh_Hans: 'https://docs.langbot.app/zh/features/monitoring.html',
en_US: '',
zh_Hans: '',
},
}),
new SidebarChildVO({
@@ -84,9 +84,9 @@ export const sidebarConfigList = [
route: '/home/knowledge',
description: t('knowledge.description'),
helpLink: {
en_US: 'https://docs.langbot.app/en/usage/knowledge/readme.html',
zh_Hans: 'https://docs.langbot.app/zh/usage/knowledge/readme.html',
ja_JP: 'https://docs.langbot.app/ja/usage/knowledge/readme.html',
en_US: 'https://docs.langbot.app/en/usage/knowledge/readme',
zh_Hans: 'https://docs.langbot.app/zh/usage/knowledge/readme',
ja_JP: 'https://docs.langbot.app/ja/usage/knowledge/readme',
},
}),
new SidebarChildVO({
@@ -105,9 +105,9 @@ export const sidebarConfigList = [
route: '/home/plugins',
description: t('plugins.description'),
helpLink: {
en_US: 'https://docs.langbot.app/en/usage/plugin/plugin-intro.html',
zh_Hans: 'https://docs.langbot.app/zh/usage/plugin/plugin-intro.html',
ja_JP: 'https://docs.langbot.app/ja/usage/plugin/plugin-intro.html',
en_US: 'https://docs.langbot.app/en/usage/plugin/plugin-intro',
zh_Hans: 'https://docs.langbot.app/zh/usage/plugin/plugin-intro',
ja_JP: 'https://docs.langbot.app/ja/usage/plugin/plugin-intro',
},
}),
];

View File

@@ -36,11 +36,11 @@ export default function NewVersionDialog({
const getUpdateDocsUrl = () => {
const language = i18n.language;
if (language === 'zh-Hans' || language === 'zh-Hant') {
return 'https://docs.langbot.app/zh/deploy/update.html';
return 'https://docs.langbot.app/zh/deploy/update';
} else if (language === 'ja-JP') {
return 'https://docs.langbot.app/ja/deploy/update.html';
return 'https://docs.langbot.app/ja/deploy/update';
} else {
return 'https://docs.langbot.app/en/deploy/update.html';
return 'https://docs.langbot.app/en/deploy/update';
}
};

View File

@@ -1,4 +1,5 @@
import { useEffect, useState } from 'react';
import Link from 'next/link';
import { useForm } from 'react-hook-form';
import { zodResolver } from '@hookform/resolvers/zod';
import { z } from 'zod';
@@ -264,9 +265,12 @@ export default function KBForm({
<p className="text-muted-foreground">
{t('knowledge.noEnginesAvailable')}
</p>
<p className="text-sm text-muted-foreground">
<Link
href="/home/plugins"
className="text-sm text-primary hover:underline"
>
{t('knowledge.installEngineHint')}
</p>
</Link>
</div>
);
}

View File

@@ -0,0 +1,157 @@
'use client';
import { useState } from 'react';
import {
Dialog,
DialogContent,
DialogHeader,
DialogTitle,
DialogDescription,
DialogFooter,
} from '@/components/ui/dialog';
import { Button } from '@/components/ui/button';
import { useTranslation } from 'react-i18next';
import { httpClient } from '@/app/infra/http/HttpClient';
import { useAsyncTask, AsyncTaskStatus } from '@/hooks/useAsyncTask';
import { toast } from 'sonner';
import { Loader2 } from 'lucide-react';
interface KBMigrationDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
internalKbCount: number;
externalKbCount: number;
onMigrationComplete: () => void;
}
export default function KBMigrationDialog({
open,
onOpenChange,
internalKbCount,
externalKbCount,
onMigrationComplete,
}: KBMigrationDialogProps) {
const { t } = useTranslation();
const [dismissing, setDismissing] = useState(false);
const asyncTask = useAsyncTask({
onSuccess: () => {
toast.success(t('knowledge.migration.success'));
onOpenChange(false);
onMigrationComplete();
},
onError: (error) => {
toast.error(`${t('knowledge.migration.error')}${error}`);
},
});
const handleMigration = async (installPlugin: boolean) => {
try {
const resp = await httpClient.executeRagMigration(installPlugin);
asyncTask.startTask(resp.task_id);
} catch {
toast.error(t('knowledge.migration.error'));
}
};
const handleDismiss = async () => {
setDismissing(true);
try {
await httpClient.dismissRagMigration();
onOpenChange(false);
} catch {
toast.error(t('knowledge.migration.dismissError'));
} finally {
setDismissing(false);
}
};
const isRunning = asyncTask.status === AsyncTaskStatus.RUNNING;
const isError = asyncTask.status === AsyncTaskStatus.ERROR;
const totalCount = internalKbCount + externalKbCount;
return (
<Dialog
open={open}
onOpenChange={(v) => {
if (!isRunning) onOpenChange(v);
}}
>
<DialogContent className="sm:max-w-md">
<DialogHeader>
<DialogTitle>{t('knowledge.migration.title')}</DialogTitle>
<DialogDescription>
{t('knowledge.migration.description')}
</DialogDescription>
</DialogHeader>
<div className="py-4 space-y-3">
{!isRunning && !isError && (
<p className="text-sm text-muted-foreground">
{t('knowledge.migration.detected', {
total: totalCount,
internal: internalKbCount,
external: externalKbCount,
})}
</p>
)}
{isRunning && (
<div className="flex items-center gap-3">
<Loader2 className="h-5 w-5 animate-spin text-primary" />
<p className="text-sm">{t('knowledge.migration.running')}</p>
</div>
)}
{isError && (
<div className="space-y-2">
<p className="text-sm text-destructive">
{t('knowledge.migration.error')}
</p>
{asyncTask.error && (
<p className="text-xs text-muted-foreground bg-muted p-2 rounded">
{asyncTask.error}
</p>
)}
</div>
)}
</div>
<DialogFooter className="flex flex-col gap-2 sm:flex-col">
{!isRunning && !isError && (
<>
<Button onClick={() => handleMigration(true)} className="w-full">
{t('knowledge.migration.startWithInstall')}
</Button>
<Button
variant="outline"
onClick={() => handleMigration(false)}
className="w-full"
>
{t('knowledge.migration.startDataOnly')}
</Button>
<p className="text-xs text-muted-foreground text-center">
{t('knowledge.migration.dataOnlyHint')}
</p>
</>
)}
{isError && (
<Button onClick={() => handleMigration(true)} className="w-full">
{t('knowledge.migration.retry')}
</Button>
)}
{!isRunning && (
<Button
variant="ghost"
onClick={handleDismiss}
disabled={dismissing}
className="w-full text-destructive hover:text-destructive"
>
{t('knowledge.migration.dismiss')}
</Button>
)}
</DialogFooter>
</DialogContent>
</Dialog>
);
}

View File

@@ -5,7 +5,6 @@
.knowledgeListContainer {
width: 100%;
margin-top: 2rem;
padding-left: 0.8rem;
padding-right: 0.8rem;
display: grid;

View File

@@ -7,6 +7,7 @@ import { useEffect, useState } from 'react';
import { KnowledgeBaseVO } from '@/app/home/knowledge/components/kb-card/KBCardVO';
import KBCard from '@/app/home/knowledge/components/kb-card/KBCard';
import KBDetailDialog from '@/app/home/knowledge/KBDetailDialog';
import KBMigrationDialog from '@/app/home/knowledge/components/kb-migration-dialog/KBMigrationDialog';
import { httpClient } from '@/app/infra/http/HttpClient';
import { KnowledgeBase } from '@/app/infra/entities/api';
@@ -18,10 +19,29 @@ export default function KnowledgePage() {
const [selectedKbId, setSelectedKbId] = useState<string>('');
const [detailDialogOpen, setDetailDialogOpen] = useState(false);
// Migration dialog state
const [migrationDialogOpen, setMigrationDialogOpen] = useState(false);
const [migrationInternalCount, setMigrationInternalCount] = useState(0);
const [migrationExternalCount, setMigrationExternalCount] = useState(0);
useEffect(() => {
getKnowledgeBaseList();
checkMigrationStatus();
}, []);
async function checkMigrationStatus() {
try {
const resp = await httpClient.getRagMigrationStatus();
if (resp.needed) {
setMigrationInternalCount(resp.internal_kb_count);
setMigrationExternalCount(resp.external_kb_count);
setMigrationDialogOpen(true);
}
} catch {
// Silently ignore - migration check is non-critical
}
}
async function getKnowledgeBaseList() {
const resp = await httpClient.getKnowledgeBases();
@@ -85,8 +105,20 @@ export default function KnowledgePage() {
getKnowledgeBaseList();
};
const handleMigrationComplete = () => {
getKnowledgeBaseList();
};
return (
<div>
<KBMigrationDialog
open={migrationDialogOpen}
onOpenChange={setMigrationDialogOpen}
internalKbCount={migrationInternalCount}
externalKbCount={migrationExternalCount}
onMigrationComplete={handleMigrationComplete}
/>
<KBDetailDialog
open={detailDialogOpen}
onOpenChange={setDetailDialogOpen}

View File

@@ -562,8 +562,7 @@ function MarketPageContent({
{/* Recommendation Lists */}
{!searchQuery &&
componentFilter === 'all' &&
selectedTags.length === 0 &&
currentPage === 1 && (
selectedTags.length === 0 && (
<div className="pt-4">
<RecommendationLists
lists={recommendationLists}

View File

@@ -1,6 +1,12 @@
import { PluginMarketCardVO } from './PluginMarketCardVO';
import { useTranslation } from 'react-i18next';
import { Badge } from '@/components/ui/badge';
import {
Tooltip,
TooltipContent,
TooltipProvider,
TooltipTrigger,
} from '@/components/ui/tooltip';
import {
Wrench,
AudioWaveform,
@@ -9,6 +15,7 @@ import {
ExternalLink,
Book,
FileText,
Info,
} from 'lucide-react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
@@ -46,6 +53,13 @@ export default function PluginMarketCardComponent({
Parser: <FileText className="w-4 h-4" />,
};
// Plugins that only contain KnowledgeRetriever components are deprecated
const isDeprecated = (() => {
if (!cardVO.components) return false;
const keys = Object.keys(cardVO.components);
return keys.length > 0 && keys.every((k) => k === 'KnowledgeRetriever');
})();
return (
<div
className="w-[100%] h-auto min-h-[8rem] sm:min-h-[9rem] bg-white rounded-[10px] shadow-[0px_0px_4px_0_rgba(0,0,0,0.2)] p-3 sm:p-[1rem] hover:shadow-[0px_3px_6px_0_rgba(0,0,0,0.12)] transition-all duration-200 hover:scale-[1.005] dark:bg-[#1f1f22] relative"
@@ -66,8 +80,34 @@ export default function PluginMarketCardComponent({
<div className="text-[0.65rem] sm:text-[0.7rem] text-[#666] dark:text-[#999] truncate w-full">
{cardVO.pluginId}
</div>
<div className="text-base sm:text-[1.2rem] text-black dark:text-[#f0f0f0] truncate w-full">
{cardVO.label}
<div className="flex items-center gap-1.5 w-full min-w-0">
<div className="text-base sm:text-[1.2rem] text-black dark:text-[#f0f0f0] truncate">
{cardVO.label}
</div>
{isDeprecated && (
<TooltipProvider delayDuration={200}>
<Tooltip>
<TooltipTrigger
asChild
onClick={(e) => e.preventDefault()}
>
<Badge
variant="outline"
className="text-[0.6rem] px-1.5 py-0 h-4 flex-shrink-0 border-red-400 text-red-500 dark:border-red-500 dark:text-red-400 gap-0.5 cursor-help"
>
{t('market.deprecated')}
<Info className="w-2.5 h-2.5" />
</Badge>
</TooltipTrigger>
<TooltipContent
side="top"
className="max-w-[240px] text-xs"
>
{t('market.deprecatedTooltip')}
</TooltipContent>
</Tooltip>
</TooltipProvider>
)}
</div>
</div>

View File

@@ -262,6 +262,12 @@ export interface ApiRespSystemInfo {
limitation: SystemLimitation;
}
export interface RagMigrationStatusResp {
needed: boolean;
internal_kb_count: number;
external_kb_count: number;
}
export interface ApiRespPluginSystemStatus {
is_enable: boolean;
is_connected: boolean;

View File

@@ -40,6 +40,7 @@ import {
ModelProvider,
ApiRespKnowledgeEngines,
ApiRespParsers,
RagMigrationStatusResp,
} from '@/app/infra/entities/api';
import { Plugin } from '@/app/infra/entities/plugin';
import { GetBotLogsRequest } from '@/app/infra/http/requestParam/bots/GetBotLogsRequest';
@@ -355,6 +356,7 @@ export class BackendClient extends BaseHttpClient {
is_active: boolean;
platform: string | null;
user_id: string | null;
user_name: string | null;
}>;
total: number;
}> {
@@ -383,6 +385,7 @@ export class BackendClient extends BaseHttpClient {
level: string;
platform: string | null;
user_id: string | null;
user_name: string | null;
runner_name: string | null;
variables: string | null;
role: string | null;
@@ -710,6 +713,23 @@ export class BackendClient extends BaseHttpClient {
return this.get('/api/v1/system/status/plugin-system');
}
// ============ RAG Migration API ============
public getRagMigrationStatus(): Promise<RagMigrationStatusResp> {
return this.get('/api/v1/knowledge/migration/status');
}
public executeRagMigration(
installPlugin: boolean = true,
): Promise<AsyncTaskCreatedResp> {
return this.post('/api/v1/knowledge/migration/execute', {
install_plugin: installPlugin,
});
}
public dismissRagMigration(): Promise<object> {
return this.post('/api/v1/knowledge/migration/dismiss');
}
public getPluginDebugInfo(): Promise<{
debug_url: string;
plugin_debug_key: string;

View File

@@ -284,6 +284,27 @@ export default function Login() {
</form>
</Form>
)}
<p className="text-xs text-center text-muted-foreground">
{t('common.agreementNotice')}{' '}
<a
href="https://langbot.app/privacy"
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-foreground transition-colors"
>
{t('common.privacyPolicy')}
</a>{' '}
{t('common.and')}{' '}
<a
href={t('common.dataCollectionPolicyUrl')}
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-foreground transition-colors"
>
{t('common.dataCollectionPolicy')}
</a>
</p>
</CardContent>
</Card>
</div>

View File

@@ -253,6 +253,27 @@ export default function Register() {
</Button>
</form>
</Form>
<p className="text-xs text-center text-muted-foreground">
{t('common.agreementNotice')}{' '}
<a
href="https://langbot.app/privacy"
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-foreground transition-colors"
>
{t('common.privacyPolicy')}
</a>{' '}
{t('common.and')}{' '}
<a
href={t('common.dataCollectionPolicyUrl')}
target="_blank"
rel="noopener noreferrer"
className="underline hover:text-foreground transition-colors"
>
{t('common.dataCollectionPolicy')}
</a>
</p>
</CardContent>
</Card>
</div>

View File

@@ -47,6 +47,12 @@ const enUS = {
copyFailed: 'Copy Failed',
test: 'Test',
forgotPassword: 'Forgot Password?',
agreementNotice: 'By continuing, you agree to our',
privacyPolicy: 'Privacy Policy',
and: 'and',
dataCollectionPolicy: 'Data Collection Policy',
dataCollectionPolicyUrl:
'https://docs.langbot.app/en/insight/data-collection-policy',
loading: 'Loading...',
fieldRequired: 'This field is required',
or: 'or',
@@ -483,6 +489,9 @@ const enUS = {
allComponents: 'All Components',
requestPlugin: 'Request Plugin',
viewDetails: 'View Details',
deprecated: 'Deprecated',
deprecatedTooltip:
'Please install the corresponding Knowledge Engine plugin.',
tags: {
filterByTags: 'Filter by Tags',
selected: 'selected',
@@ -757,7 +766,7 @@ const enUS = {
engineSettingsReadonly: 'read-only in edit mode',
retrievalSettings: 'Retrieval Settings',
noEnginesAvailable: 'No knowledge base engines available',
installEngineHint: 'Please install a knowledge base plugin first',
installEngineHint: 'Please install a "Knowledge Engine" plugin first',
createKnowledgeBaseFailed: 'Failed to create knowledge base: ',
loadKnowledgeBaseFailed: 'Failed to load knowledge base: ',
deleteKnowledgeBaseFailed: 'Failed to delete knowledge base: ',
@@ -773,6 +782,23 @@ const enUS = {
retrieverConfiguration: 'Retriever Configuration',
retrieverInstallInfo: 'You can install Knowledge Retriever plugins from',
retrieverMarketLink: 'here',
migration: {
title: 'Knowledge Base Migration',
description:
'The new version has refactored the knowledge base into a plugin-based architecture, unifying built-in and external knowledge bases as "Knowledge Engine" plugins. Migration of legacy knowledge base data is required. Your old data has been automatically backed up in the database.',
detected:
'Found {{total}} knowledge base(s) to migrate ({{internal}} internal, {{external}} external).',
startWithInstall: 'Auto-install Plugin & Migrate',
startDataOnly: 'Migrate Data Only',
dataOnlyHint:
'"Migrate Data Only" is for offline/intranet environments. Please install the corresponding plugin manually after migration.',
dismiss: 'Discard Original Data',
running: 'Migrating knowledge bases, please wait...',
success: 'Knowledge base migration completed',
error: 'Knowledge base migration failed: ',
dismissError: 'Operation failed',
retry: 'Retry',
},
},
register: {
title: 'Initialize LangBot 👋',

View File

@@ -48,6 +48,12 @@ const jaJP = {
copyFailed: 'コピーに失敗しました',
test: 'テスト',
forgotPassword: 'パスワードを忘れた?',
agreementNotice: '続行することで、以下に同意したものとみなされます:',
privacyPolicy: 'プライバシーポリシー',
and: 'および',
dataCollectionPolicy: 'データ収集ポリシー',
dataCollectionPolicyUrl:
'https://docs.langbot.app/ja/insight/data-collection-policy',
loading: '読み込み中...',
fieldRequired: 'この項目は必須です',
or: 'または',
@@ -491,6 +497,9 @@ const jaJP = {
noTags: 'タグがありません',
},
viewDetails: '詳細を表示',
deprecated: '非推奨',
deprecatedTooltip:
'対応する「ナレッジエンジン」プラグインをインストールしてください。',
},
mcp: {
title: 'MCP',
@@ -749,6 +758,9 @@ const jaJP = {
fileName: 'ファイル名',
noResults: '検索結果がありません',
retrieveError: '検索に失敗しました:',
noEnginesAvailable: '利用可能なナレッジエンジンがありません',
installEngineHint:
'先に「ナレッジエンジン」プラグインをインストールしてください',
unknownEngine: '不明なエンジン',
loadKnowledgeBaseFailed: 'ナレッジベースの読み込みに失敗しました:',
deleteKnowledgeBaseFailed: 'ナレッジベースの削除に失敗しました:',
@@ -762,6 +774,23 @@ const jaJP = {
retrieverConfiguration: '検索器設定',
retrieverInstallInfo: 'ナレッジ検索器プラグインは',
retrieverMarketLink: 'こちらからインストールできます',
migration: {
title: 'ナレッジベースの移行',
description:
'新バージョンではナレッジベースをプラグインベースのアーキテクチャに再構築し、内蔵ナレッジベースと外部ナレッジベースを「ナレッジエンジン」プラグインとして統合しました。旧ナレッジベースデータの移行が必要です。旧データはデータベースに自動的にバックアップされています。',
detected:
'移行が必要なナレッジベースが{{total}}件見つかりました(内部{{internal}}件、外部{{external}}件)。',
startWithInstall: 'プラグインを自動インストールして移行',
startDataOnly: 'データのみ移行',
dataOnlyHint:
'「データのみ移行」はオフライン環境向けです。移行完了後に対応するプラグインを手動でインストールしてください。',
dismiss: '元データを破棄',
running: 'ナレッジベースを移行中です。しばらくお待ちください...',
success: 'ナレッジベースの移行が完了しました',
error: 'ナレッジベースの移行に失敗しました:',
dismissError: '操作に失敗しました',
retry: 'リトライ',
},
},
register: {
title: 'LangBot を初期化 👋',

View File

@@ -47,6 +47,12 @@ const zhHans = {
copyFailed: '复制失败',
test: '测试',
forgotPassword: '忘记密码?',
agreementNotice: '继续即表示您同意我们的',
privacyPolicy: '隐私政策',
and: '和',
dataCollectionPolicy: '数据收集政策',
dataCollectionPolicyUrl:
'https://docs.langbot.app/zh/insight/data-collection-policy',
loading: '加载中...',
fieldRequired: '此字段为必填项',
or: '或',
@@ -468,6 +474,8 @@ const zhHans = {
noTags: '暂无标签',
},
viewDetails: '查看详情',
deprecated: '已弃用',
deprecatedTooltip: '请安装对应「知识引擎」插件',
},
mcp: {
title: 'MCP',
@@ -726,7 +734,7 @@ const zhHans = {
engineSettingsReadonly: '编辑模式下不可修改',
retrievalSettings: '检索设置',
noEnginesAvailable: '没有可用的知识库引擎',
installEngineHint: '请先安装知识插件',
installEngineHint: '请先安装知识引擎」插件',
createKnowledgeBaseFailed: '知识库创建失败:',
loadKnowledgeBaseFailed: '知识库加载失败:',
deleteKnowledgeBaseFailed: '知识库删除失败:',
@@ -742,6 +750,23 @@ const zhHans = {
retrieverConfiguration: '检索器配置',
retrieverInstallInfo: '您可以从',
retrieverMarketLink: '此处安装知识检索器插件',
migration: {
title: '知识库迁移',
description:
'新版本已将知识库重构为插件化架构,并统一内置知识库和外部知识库为「知识引擎」插件,需要对旧知识库数据进行迁移。您的旧数据已自动备份在数据库中。',
detected:
'共检测到 {{total}} 个知识库需要迁移({{internal}} 个内置知识库,{{external}} 个外部知识库)。',
startWithInstall: '自动安装插件并迁移',
startDataOnly: '仅迁移数据',
dataOnlyHint:
'「仅迁移数据」适合内网环境使用,请在迁移完成后自行安装对应插件',
dismiss: '丢弃原数据',
running: '正在迁移知识库,请稍候...',
success: '知识库迁移完成',
error: '知识库迁移失败:',
dismissError: '操作失败',
retry: '重试',
},
},
register: {
title: '初始化 LangBot 👋',

View File

@@ -47,6 +47,12 @@ const zhHant = {
copyFailed: '複製失敗',
test: '測試',
forgotPassword: '忘記密碼?',
agreementNotice: '繼續即表示您同意我們的',
privacyPolicy: '隱私政策',
and: '和',
dataCollectionPolicy: '數據收集政策',
dataCollectionPolicyUrl:
'https://docs.langbot.app/zh/insight/data-collection-policy',
loading: '載入中...',
fieldRequired: '此欄位為必填',
or: '或',
@@ -461,6 +467,8 @@ const zhHant = {
noTags: '暫無標籤',
},
viewDetails: '查看詳情',
deprecated: '已棄用',
deprecatedTooltip: '請安裝對應「知識引擎」插件',
},
mcp: {
title: 'MCP',
@@ -709,6 +717,8 @@ const zhHant = {
fileName: '文檔名稱',
noResults: '暫無結果',
retrieveError: '檢索失敗:',
noEnginesAvailable: '沒有可用的知識庫引擎',
installEngineHint: '請先安裝「知識引擎」插件',
unknownEngine: '未知引擎',
loadKnowledgeBaseFailed: '知識庫載入失敗:',
deleteKnowledgeBaseFailed: '知識庫刪除失敗:',
@@ -722,6 +732,23 @@ const zhHant = {
retrieverConfiguration: '檢索器配置',
retrieverInstallInfo: '您可以從',
retrieverMarketLink: '此處安裝知識檢索器插件',
migration: {
title: '知識庫遷移',
description:
'新版本已將知識庫重構為插件化架構,並統一內建知識庫和外部知識庫為「知識引擎」插件,需要對舊知識庫資料進行遷移。您的舊資料已自動備份在資料庫中。',
detected:
'共檢測到 {{total}} 個知識庫需要遷移({{internal}} 個內建知識庫,{{external}} 個外部知識庫)。',
startWithInstall: '自動安裝插件並遷移',
startDataOnly: '僅遷移資料',
dataOnlyHint:
'「僅遷移資料」適合內網環境使用,請在遷移完成後自行安裝對應插件',
dismiss: '丟棄原數據',
running: '正在遷移知識庫,請稍候...',
success: '知識庫遷移完成',
error: '知識庫遷移失敗:',
dismissError: '操作失敗',
retry: '重試',
},
},
register: {
title: '初始化 LangBot 👋',