Compare commits

...

49 Commits

Author SHA1 Message Date
Junyan Qin
2e061a8713 feat: update version to 4.9.0 in pyproject.toml, __init__.py, and uv.lock 2026-03-09 20:09:00 +08:00
Junyan Qin
2cd8c56fe8 feat: update migration messages for knowledge base in multiple languages 2026-03-09 19:57:13 +08:00
youhuanghe
e09ae8f1a8 feat: add external plugin auto download 2026-03-09 09:55:12 +00:00
youhuanghe
aa7b0deb2b fix: show 2026-03-09 09:26:29 +00:00
youhuanghe
1c9a800f9d wq
Merge branch 'master' into feat/dbm20-rag
2026-03-09 08:26:05 +00:00
youhuanghe
96f24d73d5 feat: add external migration 2026-03-09 08:18:23 +00:00
Guanchao Wang
d31f25c8df Merge pull request #2041 from langbot-app/fix/websocket-chat-bug
Fix/websocket chat bug
2026-03-09 16:11:17 +08:00
WangCham
11c05ea8db style(format): fix ruff formatting issues 2026-03-09 16:04:38 +08:00
WangCham
2b8bd1cc71 fix: invoke_llm failed when use plugin 2026-03-09 16:01:45 +08:00
doujianghub
9148e02679 fix: centralized pipeline config type coercion to prevent string-type crashes (#2031)
* fix: coerce pipeline config types at load time using metadata definitions

Pipeline configs stored in SQLAlchemy JSON columns can have values turned
into strings after UI edits (e.g. "120" instead of 120), causing runtime
arithmetic/logic errors. Add centralized type coercion in load_pipeline()
that leverages existing metadata YAML type definitions (integer, number,
float, boolean) to convert values before they reach downstream stages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review - defensive getattr + add unit tests for config_coercion

- Use getattr with defaults for pipeline_config_meta_* attributes to
  avoid AttributeError when MockApplication lacks these fields
- Add 18 unit tests for config_coercion module covering all code paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add dynamic form stage tracking and snapshot management

* fix: standardize string formatting in config coercion and improve logging messages

---------

Co-authored-by: KPC <kpc@kpc.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-09 14:30:07 +08:00
fdc310
fd15284d91 fix(platform): websocket send_message not delivering to webchat frontend (#2039)
- Include websocket_proxy_bot in get_bot_by_uuid lookup so plugins can
  find it by uuid
- Rewrite send_message to broadcast directly via ws_connection_manager
  using the correct pipeline_uuid instead of misusing target_id
- Save messages to session history with unique IDs so they persist
  across page reloads and don't overwrite each other

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 13:22:03 +08:00
youhuanghe
14ea8ca7b6 fix ruff lint 2026-03-09 01:26:39 +00:00
youhuanghe
f0093dab69 fix lint 2026-03-09 01:23:56 +00:00
youhuanghe
c29e6586b3 refactor: to red and no more 2026-03-09 01:08:56 +00:00
youhuanghe
1b37dababa feat(rag): add data-only migration option and fix dialog width
Add option to migrate knowledge base data without auto-installing
the LangRAG plugin (for offline/intranet environments). Also
narrow the migration dialog to match other confirmation dialogs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:05:05 +00:00
youhuanghe
8da52b6dc7 fix(rag): query marketplace for actual plugin version instead of 'latest'
The marketplace API does not support 'latest' as a version string.
Fetch the plugin info first to get latest_version, then use that
concrete version for installation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:41:43 +00:00
youhuanghe
67c5c3af20 feat(rag): add knowledge base migration from v4.9.0 to plugin architecture
Rewrite dbm020 to backup old knowledge_bases data and preserve
external_knowledge_bases table. Add migration API endpoints and
frontend dialog so users can opt-in to auto-install LangRAG plugin
and restore their knowledge bases with original UUIDs preserved.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:17:34 +00:00
Junyan Qin
8c7a0ec027 fix: update langbot-plugin version to 0.3.0 2026-03-08 21:08:08 +08:00
youhuanghe
a1cef5c9bf bugfix: update uv.lock 2026-03-08 11:10:03 +00:00
youhuanghe
90438cec36 lint: update web knowledge pnpm lint 2026-03-08 11:05:00 +00:00
youhuanghe
95dd19f4d7 bugfix: now knowledge toast right msg 2026-03-08 11:01:13 +00:00
youhuanghe
c64eb58cf8 feat: update pyseekdb version to 1.1.0.post3 2026-03-08 10:42:20 +00:00
Junyan Qin
fbd3d7ae3a feat: enhance RecommendationLists component with responsive pagination and auto-advance functionality
- Added dynamic column measurement to adjust the number of visible plugins based on the grid layout.
- Implemented auto-advance feature for pagination every 5 seconds when there are more plugins than the visible count.
- Updated pagination controls to reflect the current page accurately.
- Refactored code to improve readability and maintainability.
2026-03-08 17:35:30 +08:00
youhuanghe
40c7b0f731 fix(web): display document_name instead of file_id in retrieval results
The getTitle fallback order was reversed, always showing the UUID
(file_id) since it's always truthy. Swap priority to document_name
first.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 04:24:41 +00:00
huanghuoguoguo
cadcf10047 Feat/rag plugin (#1995)
* [issue:1933] RAG engine plugin architecture (#1967)

* refactor: migrate RAG knowledge services to a plugin-oriented host service architecture.

* feat(rag): phase 2 core refactor with RPC Action handlers

* feat: 为 RAG 插件添加知识库创建和删除事件通知,并优化了 RAG 动作的参数传递和枚举使用。

* feat: 统一知识库管理为RAG引擎,支持动态配置并移除旧的外部知识库组件。

* refactor(rag): remove plugin_adapter, inline logic into RuntimeKnowledgeBase

BREAKING CHANGE: RAGPluginAdapter has been removed. All plugin
communication is now handled directly by RuntimeKnowledgeBase.

Architecture change:
- Before: RuntimeKnowledgeBase → RAGPluginAdapter → plugin_connector
- After:  RuntimeKnowledgeBase → plugin_connector (direct)

Changes to kbmgr.py (RuntimeKnowledgeBase):
- Remove RAGPluginAdapter import and usage
- Inline plugin communication methods:
  - _on_kb_create(): Notify plugin when KB is created
  - _on_kb_delete(): Notify plugin when KB is deleted
  - _ingest_document(): Call plugin for document ingestion
  - _retrieve(): Call plugin for retrieval
  - _delete_document(): Call plugin to delete document
- Simplify dispose(): Only notify plugin, no built-in VDB assumption

Changes to base.py (KnowledgeBaseInterface):
- Remove get_type() abstract method (outdated internal/external concept)
- Add get_rag_engine_plugin_id() abstract method

Changes to localagent.py:
- Remove get_type() call
- Simplify top_k retrieval from KB entity

Deleted files:
- pkg/rag/knowledge/plugin_adapter.py

Benefits:
- Reduced abstraction layer, simpler code
- Plugin communication logic centralized in RuntimeKnowledgeBase
- Easier to understand and maintain

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(api): remove ExternalKnowledgeBase infrastructure

BREAKING CHANGE: ExternalKnowledgeBase has been completely removed.
All knowledge bases are now unified under the single KnowledgeBase model,
differentiated by their rag_engine_plugin_id.

Deleted files:
- pkg/api/http/controller/groups/knowledge/external.py
  (ExternalKBController with /external-bases routes)
- pkg/api/http/service/external_kb.py
  (ExternalKnowledgeBaseService)
- pkg/rag/knowledge/external.py
  (ExternalKnowledgeBase implementation)

Modified files:
- pkg/entity/persistence/rag.py:
  Remove ExternalKnowledgeBase SQLAlchemy table definition
- pkg/core/app.py:
  Remove external_kb_service attribute from LangBotApplication
- pkg/core/stages/build_app.py:
  Remove external_kb_service initialization

Migration notes:
- Existing external knowledge base data should be migrated manually
- API consumers should use /api/v1/knowledge/bases for all KB operations
- Use /api/v1/knowledge/engines to discover available RAG engines

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(plugin): remove list_knowledge_retrievers from connector

Remove deprecated list_knowledge_retrievers functionality from the
plugin communication layer. This aligns with the SDK change that
removed the LIST_KNOWLEDGE_RETRIEVERS action.

Changes:
- connector.py: Remove list_knowledge_retrievers() method
- handler.py: Remove list_knowledge_retrievers() handler

The functionality is replaced by the new /api/v1/knowledge/engines
endpoint which lists available RAGEngine components with their
capabilities and configuration schemas.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(service): update knowledge service with capability-based checks

Replace type-based checks with capability-based checks for file
operations, aligning with the unified knowledge base architecture.

Changes to knowledge.py:
- store_file(): Replace get_type() check with doc_ingestion capability check
- delete_file(): Replace get_type() check with doc_ingestion capability check
- list_rag_engines(): Remove list_knowledge_retrievers call, simplify to
  only list RAGEngine components (KnowledgeRetriever type removed)

Changes to pipelines.py:
- Minor cleanup related to knowledge base references

The capability-based approach allows RAG engines to declare their
supported features (doc_ingestion, chunking_config, rerank, hybrid_search)
and the system responds accordingly, rather than hardcoding behavior
based on internal/external type distinction.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(web): unify knowledge base UI, remove external KB components

BREAKING CHANGE: The internal/external knowledge base distinction
has been removed from the frontend. All knowledge bases are now
displayed in a unified list, differentiated by their RAG engine.

Changes to page.tsx:
- Remove Tab component (内置/外置 tabs)
- Remove selectedKbType state
- Unified knowledge base list display
- Single "Create Knowledge Base" button for all types

Changes to KBDetailDialog.tsx:
- Remove kbType prop
- Simplify dialog logic for unified KB handling
- Documents menu item conditionally shown based on doc_ingestion capability

Changes to KBForm.tsx:
- Remove retriever type handling code
- Simplify form for unified KB creation
- Dynamic form rendering based on RAG engine's creation_schema

Changes to KBCardVO.ts:
- Remove 'type' field from KBCardVO interface

Changes to BackendClient.ts:
- Remove all external KB related methods:
  - getExternalKnowledgeBases()
  - getExternalKnowledgeBase()
  - createExternalKnowledgeBase()
  - updateExternalKnowledgeBase()
  - deleteExternalKnowledgeBase()
  - retrieveFromExternalKnowledgeBase()

Changes to api/index.ts:
- Remove ExternalKnowledgeBase interface definition

UI/UX improvements:
- Users no longer need to understand internal vs external distinction
- RAG engine selection is now the primary differentiator
- Documents panel visibility is capability-driven (doc_ingestion)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(plugin): code review improvements for RAG handlers

- Unify embed_model field naming to embedding_model_uuid only
- Add structured error responses with error_type for RAG actions
- Fix file_size and mime_type detection in _store_file_task
- Improve error handling with detailed error context (error_type, original_error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(rag): refactor KB dynamic form and vector manager

- Frontend: Refactor Knowledge Base form using DynamicForm components.
- Frontend: Remove obsolete jsonSchemaConverter utility.
- Backend: Update VectorManager and PluginHandler to support new RAG architecture.
- Chore: Update dependencies in pyproject.toml.

* fix: code review fixes for RAG refactor

- Remove DEBUG stderr outputs in handler.py
- Move repeated `import json` to file top
- Add warning log for unimplemented delete_by_filter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(rag): consolidate valid_fields into entity constants

Define MUTABLE_FIELDS, CREATE_FIELDS, ALL_DB_FIELDS as class
constants in KnowledgeBase entity to eliminate duplication.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: 将知识库获取和RAG引擎信息丰富逻辑移至知识库管理器。

* refactor(rag): introduce RAGRuntimeService and clean up plugin handler

- Create RAGRuntimeService to encapsulate RAG capability implementation (Embedding, VectorOps).
- Refactor PluginHandler to delegate RAG actions to RAGRuntimeService.
- Move KnowledgeService enrichment and creation logic to RAGManager.
- Register RAGRuntimeService in Application and BuildAppStage.
- Clean up legacy code in KnowledgeService.

* refactor(rag): standardize logger and fix type hints

- Use self.ap.logger consistently in kbmgr.py and runtime.py, removing module-level loggers.
- Fix type hints for retrieve_knowledge in handler.py and connector.py to match implementation returning dict.

* refactor: 将引擎徽章的样式从 Tailwind CSS 类迁移到 CSS 模块。

* fix(web): resolve React rendering errors in plugins page

- Fix missing key prop in PluginComponentList by using ternary instead of Fragment
- Fix RAGEngine.name type to I18nObject and use extractI18nObject() for rendering
- Preserves multi-language support

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(rag): update runtime service and web components

* refactor: 优化知识库设置结构并增强前端距离显示健壮性。

* fix: 处理前端距离显示中的空值。

* fix(rag): document retrieve ui and kbmgr top_k validation

* 更新 uv.lock 中的 PyPI 镜像源为官方地址。

* fix: address code review issues for RAG engine plugin architecture

P0 fixes:
- Fix ALL_DB_FIELDS missing collection_id and emoji fields
- Move rag_engine_plugin_id to CREATE_FIELDS (immutable after creation)
- Fix creation_settings mutable default value (dict -> None)
- Rename vector delete method to delete_by_file_id for correct semantics
- Fix delete_by_filter to raise NotImplementedError instead of silent no-op
- Add database migration script (dbm019) for new columns and table cleanup

P1 fixes:
- Clean up design-hesitation comments in connector.py
- Add _parse_plugin_id() with format validation for all RAG methods
- Make _retrieve() raise exceptions instead of silently returning empty results
- Extract _make_rag_error_response() helper for clean error formatting
- Remove unused imports from handler.py

P2 fixes:
- Fix runtime.py indentation inconsistencies
- Simplify get_file_stream to use storage abstraction uniformly
- Reduce redundant DB queries in knowledge service (extract _check_doc_capability)
- Fix engines.py URL encoding: use <path:plugin_id> instead of __ replacement
- Add read-only mode for engine settings in KBForm edit mode
- Simplify page.tsx handleKBCardClick to pass only kbId string

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: address code review findings for RAG plugin architecture

- Frontend: add retrieval_settings param to retrieveKnowledgeBase API call
- Backend: return {uuid} from PUT knowledge base to match frontend expectation
- Backend: validate query is non-empty in retrieve endpoint (400 on empty)
- Backend: rename vector_delete ids→file_ids for semantic clarity, keep
  backward compat by accepting both 'file_ids' and 'ids' in RPC handler
- Backend: ensure rag_engine.name fallback is always I18nObject-compatible
  dict, preventing frontend extractI18nObject from receiving plain strings
- Migration: fix misleading docstring about external_kb data migration

Co-authored-by: Cursor <cursoragent@cursor.com>

* Update langbot-plugin version to 0.2.6

* chore: update required database version from 18 to 19

* refactor: remove unused polymorphic component framework

* chore: fix lint and format issues for python and frontend

* fix(plugin): remove legacy `ids` fallback in rag_vector_delete handler

SDK now sends `file_ids` directly, the `ids` backward-compat fallback
is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): deep review fixes for critical bugs, security and quality

Critical:
- Fix StorageMgr.load() -> storage_provider.load() (C1, AttributeError)
- Update required_database_version 18 -> 19 (C2, migration never runs)

Security:
- Add path traversal validation in get_file_stream (C11)
- Add vectors/ids/metadata length validation in rag_vector_upsert (C12)

Logic fixes:
- Legacy KBs: set capabilities to [] instead of ['doc_ingestion'] (C4)
- Fix store_file return type int -> str (C5)
- Fix retrieve_knowledge return [] -> {'results': []} when disabled (C6)
- Re-raise exception in _on_kb_create instead of silently swallowing (C7)
- Log warning when KB not found in memory during delete (C8)

API fixes:
- Catch ValueError as 400 in create_knowledge_base endpoint (C15)
- Validate plugin_id format in engines endpoints (C16)

Quality:
- Remove dead if/else in migration with identical branches (C17)
- Fix variable shadowing: rag_context -> rag_context_text (C18)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove unused os import to fix ruff lint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(plugin): remove PolymorphicComponent sync from LangBot side

Remove sync_polymorphic_component_instances() from connector and handler,
and the post-connection sync call in initialize(). This dead code synced
an always-empty list of polymorphic instances that were never created.

Companion change to langbot-plugin-sdk PolymorphicComponent removal.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): fix vector_delete count bug and remove vestigial instance_id parameter

1. vector_delete: assign return value from delete_by_filter to count
   instead of silently returning 0 for filter-based deletion.

2. Remove instance_id parameter from the entire retrieve_knowledge
   call chain (kbmgr → connector → handler → runtime). This parameter
   was a remnant of the PolymorphicComponent mechanism and is no longer
   used — RAGEngine operates as a stateless singleton.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): 支持 creation_schema 字段级别的 editable 属性控制编辑模式可修改性

- IDynamicFormItemSchema 添加 editable 可选属性
- DynamicFormItemConfig 透传 editable 属性
- DynamicFormComponent 接收 isEditing prop,按字段 editable 值控制禁用
- KBForm 解析 editable 并传递 isEditing 给动态表单组件
- editable 未指定时默认可编辑,editable: false 时编辑模式下禁用该字段

* feat(storage): 添加 size() 抽象方法及 LocalStorage/S3 实现

支持获取存储对象大小,S3 使用 head_object 避免下载整个文件

* fix(migration): 删除 external_knowledge_bases 表前记录日志警告

- 迁移时如果表中存在数据,先 warning 日志记录避免无感数据丢失
- 添加 chunk 清理注释说明:仅对旧版非插件架构 KB 有效

* fix(web): 修复检索结果长文本撑大容器导致查询按钮不可见

KBDetailDialog 的 main 容器添加 min-w-0 overflow-x-hidden,
限制 flex-1 子容器宽度,防止 Dify RAG 长文本撑出 Dialog 边界

* fix(rag): address code review issues for plugin architecture PR

- Fix SQL injection in migration helpers by using bind parameters
- Move numpy import to module level in vector/mgr.py
- Improve path traversal validation using posixpath.normpath
- Add call_rag_retrieve to connector, eliminating duplicate plugin_id
  parsing in kbmgr.py _retrieve
- Normalize typing style to modern dict/list/None syntax

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(web): fix prettier formatting errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(rag): update embedding handling in RuntimeConnectionHandler

- Renamed RAG_EMBED_DOCUMENTS and RAG_EMBED_QUERY actions to INVOKE_EMBEDDING for clarity.
- Removed embed_documents and embed_query methods from RuntimeEmbeddingModel and RAGRuntimeService.
- Integrated embedding model retrieval directly in the invoke_embedding method, improving error handling for missing models.
- Updated the embedding invocation logic to streamline the process and enhance error reporting.

* refactor(web): replace KnowledgeRetriever with RAGEngine across frontend and tests

KnowledgeRetriever component type has been removed in favor of the new
RAGEngine architecture. Update all remaining references in i18n locales,
plugin component icon mappings, marketplace filter, and unit tests.

Addresses reviewer notes from RockChinQ on PR #1967.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): address critical bugs found in deep review

- Fix path traversal bypass in runtime.py (check all path components for '..')
- Use normalized path for file loading instead of raw user input
- Change knowledge_bases from list to dict for O(1) lookup and race safety
- Add rollback on KB creation failure (clean up DB + runtime on plugin error)
- Add null check after KB update in knowledge service
- Fix file extension parsing to use os.path.splitext instead of split('.')
  (handles multi-dot filenames like 'report.v2.pdf' correctly)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): address remaining review issues across frontend and backend

Frontend:
- Fix KB delete: use async/await with error handling instead of fire-and-forget
- Fix capabilities null check: add optional chaining to prevent crash
- Add toast.error on KB info load failure instead of silent console.error
- Replace hard-coded Chinese validation message with i18n key
- Replace hard-coded English error messages in DynamicFormItemComponent with i18n
- Optimize document polling: stop when all documents reach terminal state
- Add i18n keys (fieldRequired, loadKnowledgeBaseFailed,
  deleteKnowledgeBaseFailed, getKnowledgeBaseListError) to all 4 locales

Backend:
- Fix KB delete atomicity: delete from DB first, then notify plugin
- Add RAG engine plugin existence validation before creating KB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(rag): fix ruff formatting in kbmgr.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>

* chore: bump langbot-plugin to 0.3.0 (#1992)

* chore: correct sdk version to 0.3.0a1

* feat: normalize rag related actions' names

* refactor(rag): align IngestionContext fields with SDK changes

Remove redundant `chunking_strategy` field and rename `custom_settings`
to `creation_settings` to match the updated SDK entity definitions
(langbot-plugin-sdk#36).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix ruff formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): enforce immutability of embedding_model_uuid and non-editable creation_settings fields

Remove embedding_model_uuid from MUTABLE_FIELDS to prevent post-creation
modification via API. Add backend validation for creation_settings to
preserve fields marked editable:false in the plugin's creation schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(rag): fix ruff formatting in knowledge service

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(rag): split settings into immutable creation_settings and mutable retrieval_settings

- Remove standalone embedding_model_uuid and top_k columns from KB entity
- Add retrieval_settings column; update MUTABLE_FIELDS/CREATE_FIELDS accordingly
- Merge migration logic into dbm019 (add retrieval_settings, migrate top_k
  and embedding_model_uuid into JSON settings, drop old columns on PostgreSQL)
- Remove _filter_creation_settings and per-field editable concept
- Frontend: creation_settings fields are all disabled when editing,
  retrieval_settings fields are always editable via a second DynamicFormComponent
- Remove editable from IDynamicFormItemSchema, DynamicFormItemConfig
- Clean up KBCardVO, KnowledgeBase API type, and localagent runner

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* bugfix: if ingest_document failed,not raise exep

* fix: ruff lint

* refactor(rag): remove unused _get_kb_entity method from RAGRuntimeService

* feat(vector): implement metadata filters for vector_search and vector_delete (#1997)

Add functional metadata filter support across all 5 VDB backends using
Chroma-style where syntax as the canonical format. Previously the filters
parameter existed throughout the stack but was entirely ignored.

- Add filter_utils.py with normalize_filter() and strip_unsupported_fields()
- Implement filter in search() and add delete_by_filter() for all backends:
  Chroma/SeekDB (native passthrough), Qdrant (translated to models.Filter),
  Milvus (translated to expr string), pgvector (translated to SQLAlchemy conditions)
- Milvus/pgvector limited to {text, file_id, chunk_uuid}; other fields logged and ignored
- Replace delete_by_filter() NotImplementedError with backend delegation in mgr.py
- Populate retrieval_context['filters'] from settings in kbmgr._retrieve()
- Pass search_type/query_text/documents through handler and runtime service

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(vector): fix ruff formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): remove numpy dependency and fix SeekDB search modes

- Remove numpy array conversion for query vectors; all VDB backends
  accept list[float] directly
- Remove redundant get_or_create_collection call from upsert; backends
  handle collection creation internally in add_embeddings
- Fix SeekDB to raise ValueError when vector dimension is unknown
  instead of defaulting to 384
- Use hybrid_search() for full-text and hybrid search modes in SeekDB,
  since pyseekdb's query() always requires embeddings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): escape single quotes in SeekDB documents and metadata

Document text containing apostrophes (e.g. "don't", "it's") causes
SQL syntax errors in OceanBase because single quotes were not in the
escape table. Add single-quote escaping and apply the escape table to
the documents parameter in add_embeddings(), not just metadata.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(vector): use standard SQL escaping for single quotes in SeekDB

Change single quote escaping from MySQL-style \' to standard SQL ''
(doubled quote). The backslash escape is not recognized by OceanBase
in NO_BACKSLASH_ESCAPES mode, causing SQL syntax errors when metadata
text contains apostrophes (e.g. O'Shea in academic citations).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): persist retrieval_settings on knowledge base creation

retrieval_settings was not being passed from the service layer to
RAGManager.create_knowledge_base(), causing retrieval schema fields
(e.g. query_rewrite) to be lost on initial KB creation. They only
took effect after a subsequent edit/update.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): add show_if conditional rendering for dynamic forms

Support conditional field visibility in plugin-defined forms via
show_if rules (eq, neq, in operators). Fields can depend on values
from the same form or cross-reference between creation and retrieval
settings via externalDependentValues.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(rag): replace base64 with chunked file transfer for get_rag_file_stream

Use send_file() instead of base64 encoding for returning file content
in the GET_RAG_FILE_STREAM handler, avoiding memory issues with large files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(parser): add parser plugin integration and capability-aware upload UI (#2000)

* feat(parser): add parser plugin integration and capability-aware upload UI

Backend: add parser plugin API endpoints (list/invoke), connector and
handler support for parser actions, and KB manager passthrough.

Frontend: thread ragEngineCapabilities prop to FileUploadZone and use
doc_parsing capability to conditionally show the RAG engine option in
the parser selector. When no parser is available, show a warning
prompting users to install a parser plugin.

Update i18n: rename builtInParser to "Provided by RAG engine" and add
noParserAvailable warning message in all 4 locales.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(parser): replace base64 with chunked file transfer and remove stale cache

- Remove @alru_cache from list_parsers() and list_rag_engines()
- Replace inline base64 file content with send_file/read_local_file
  chunked transfer pattern in parse_document and invoke_parser flows
- Remove unused base64 import from kbmgr.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat(web): add Parser component kind to plugin market UI and i18n

Add Parser to kindIconMap, market filter toggle, and all 4 locale files
so parser plugins are properly displayed and filterable in the plugin
market, matching the existing RAGEngine treatment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style(web): fix prettier formatting from merge

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename RAGEngine to KnowledgeEngine across frontend and backend

* fix(web): fix I18nObject import path in FileUploadZone and KBDoc

* chore: format files involved in RAGEngine to KnowledgeEngine refactor

* refactor: change rag engine to knowledge engine

* fix: update langbot-plugin version to 0.3.0rc1

* chore: disable migration 20 for now

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-03-06 21:54:38 +08:00
fdc310
3e8f47fd97 feat: judge and send runner category (local or cloud) for telemetry
* feat(chat): add runner_url to payload for telemetry tracking

* feat(telemetry): add runner_url to sanitized fields in telemetry payload

* feat(telemetry): replace runner_url with runner_category in telemetry payload and add runner utility functions

* fix:ruff
2026-03-06 00:44:09 +08:00
youhuanghe
b11ae55c6e fix: update web/lint src 2026-03-05 15:02:03 +00:00
marun
2d63d528c6 refactor(dify): Optimize the Dify API output parsing and workflow processing logic (#2027)
- Add the _extract_dify_text_output method to uniformly handle the parsing of Dify output content

- Modify the content extraction method for the answer node in workflow mode

- Add workflow mode detection logic to support the workflow_started event

- Handle error state checks upon completion of the workflow

- Improve the message chunking logic for both basic and workflow modes

- Add a mechanism to capture answer content upon completion of a workflow node
2026-03-05 15:15:40 +08:00
fdc310
10f253015d Fix/tg send msg chunk (#2021)
* feat(telegram): enhance message handling with markdown support and draft messages

* fix(telegram): update draft message ID generation to use current timestamp
2026-03-04 20:42:33 +08:00
RockChinQ
b34ebf85a6 fix: update version to 4.8.7 in pyproject.toml, __init__.py, and uv.lock 2026-03-04 18:30:53 +08:00
RockChinQ
06d3298cde fix: update pnpm-lock.yaml for rehype-sanitize 2026-03-01 04:12:27 -05:00
Junyan Chin
614621ab7b Merge commit from fork
Add rehype-sanitize after rehypeRaw in all ReactMarkdown usages:
- PluginReadme.tsx (plugin README rendering)
- DebugDialog.tsx (debug chat message rendering)
- NewVersionDialog.tsx (release notes rendering)

This prevents injection of raw HTML (e.g. <iframe srcdoc>) that
could steal session tokens and API credentials from localStorage.

Fixes GHSA-w8gq-g4pc-xh3h
2026-03-01 17:01:23 +08:00
Junyan Qin
8600d0a8e7 chore: add botocore dependency to pyproject.toml and uv.lock
- Included botocore>=1.42.39 in dependencies to ensure compatibility with boto3.
- Updated lock file to reflect the new botocore dependency.
2026-02-28 19:26:50 +08:00
RockChinQ
b83e6a53be fix(storage): lazy import s3storage to avoid boto3 dependency for local storage
Fixes #2014

When using default local storage, the s3storage module was imported
at the top level, which triggered boto3/botocore import and caused
ModuleNotFoundError if those packages weren't installed.

Now s3storage is only imported when S3 storage is actually configured.
2026-02-28 06:02:41 -05:00
Junyan Chin
88132dff8a perf: reduce memory usage by ~200MB+ at startup (#2013)
* perf: reduce memory usage by ~200MB+ at startup

Two key optimizations:

1. Use importlib.util.find_spec() instead of __import__() in dependency
   checking. find_spec() only locates modules without executing them,
   avoiding loading all 36 dependencies (~222MB) into memory at startup.

2. Introduce shared aiohttp.ClientSession via httpclient module.
   Previously, every HTTP request created a new ClientSession, which
   creates a new TCPConnector and SSL context, loading system root
   certificates each time (~270MB total allocations observed via memray).
   Now all HTTP client code reuses shared sessions.

   - satori.py and coze_server_api/client.py are left unchanged as they
     create one session per adapter lifecycle (not per-request).

Profiling data (memray):
- Peak memory: 403MB
- SSL context creation: 270MB / 6.7M allocations (67% of total)
- Dependency import: 222MB (55% of peak)
- Expected reduction: 150-350MB at startup

* fix: remove unused aiohttp imports (ruff F401)

* style: ruff format
2026-02-27 20:09:03 +08:00
Junyan Qin
2dc5999583 fix: handle undefined values in DynamicFormItemComponent
- Updated BOOLEAN case to default to false when field.value is undefined.
- Updated SELECT case to default to an empty string when field.value is undefined.
2026-02-27 10:55:28 +08:00
Junyan Qin
73461814c9 fix: prevent infinite re-render loop in BotForm and DynamicFormComponent
- Updated BotForm to serialize adapter_config for stable useEffect dependency.
- Refactored DynamicFormComponent to track last emitted values, avoiding unnecessary re-renders when form values remain unchanged.
2026-02-27 10:52:19 +08:00
Guanchao Wang
210e5e50d3 fix: telegram send messsage (#2010) 2026-02-27 00:40:19 +08:00
Junyan Qin
4fd488b97a chore: Bump version to 4.8.6 in pyproject.toml, uv.lock, and __init__.py 2026-02-26 22:54:13 +08:00
Junyan Qin
422a34ead4 fix: plugins in recommendation cannot be installed 2026-02-26 22:53:29 +08:00
Junyan Qin
02a1036d63 chore: Bump version to 4.8.5 in pyproject.toml and __init__.py 2026-02-26 14:34:23 +08:00
Junyan Chin
2d837c9cb4 feat: add in-product survey system (#2008)
* feat: add in-product survey system

- SurveyManager: event-based trigger, Space API communication
- Trigger on first successful non-WebSocket response
- Backend API: /api/v1/survey/{pending,respond,dismiss}
- Frontend: floating survey widget with progressive questions
- Flat radio/checkbox style (not dropdown Select)

* fix: persist triggered survey events to disk across restarts

Store triggered events in data/survey_triggered_events.json so that
restarting the process doesn't re-query Space for already-triggered events.

* fix: use metadata table for survey event persistence instead of file

Store triggered events in the existing metadata KV table
(key='survey_triggered_events') instead of a standalone JSON file.

* fix: ruff format and prettier fixes
2026-02-26 13:50:14 +08:00
Junyan Chin
2ded774747 docs: add LangBot Cloud references to all READMEs (#2007) 2026-02-25 22:18:22 +08:00
Junyan Chin
d9a630b8c1 feat: add session message monitoring tab to bot detail dialog (#2005)
* feat: add session message monitoring tab to bot detail dialog

Add a new "Sessions" tab in the bot detail dialog that displays
sent & received messages grouped by sessions. Users can select
any session to view its messages in a chat-bubble style layout.

Backend changes:
- Add sessionId filter to monitoring messages endpoint
- Add role column to MonitoringMessage (user/assistant)
- Record bot responses in monitoring via record_query_response()
- Add DB migration (dbm019) for the new role column

Frontend changes:
- New BotSessionMonitor component with session list + message viewer
- Add Sessions sidebar tab to BotDetailDialog
- Add getBotSessions/getSessionMessages API methods to BackendClient
- Add i18n translations (en-US, zh-Hans, zh-Hant, ja-JP)

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* refactor: remove outdated version comment from PipelineManager class

* fix: bump required_database_version to 19 to trigger monitoring_messages.role migration

* fix: prevent session message auto-scroll from pushing dialog content out of view

Replace scrollIntoView (which scrolls all ancestor containers) with
direct scrollTop manipulation on the ScrollArea viewport. This keeps
the scroll contained within the messages panel only.

* ui: redesign BotSessionMonitor with polished chat UI

- Wider session list (w-72) with avatar circles and cleaner layout
- Richer chat header with avatar, platform info, and active indicator
- User messages now use blue-500 (solid) instead of blue-100 for
  clear visual distinction
- Metadata (time, runner) shown on hover below bubbles, not inside
- Proper empty state illustrations for both panels
- Better spacing, rounded corners, and shadow treatment
- Consistent dark mode styling

* fix: infinite re-render loop in DynamicFormComponent

The useEffect depended on onSubmit which was a new closure every
parent render. Calling onSubmit inside the effect triggered parent
state update → re-render → new onSubmit ref → effect re-runs → loop.

Fix: use useRef to hold a stable reference to onSubmit, removing it
from the useEffect dependency array.

Also add DialogDescription to BotDetailDialog to suppress Radix
aria-describedby warning.

* fix: remove .html suffix from docs.langbot.app links (Mintlify migration)

* style: fix prettier and ruff formatting

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2026-02-25 21:56:24 +08:00
Guanchao Wang
b8df0dbd7f feat: message aggregator (#1985)
* feat: aggregator

* fix: resolve deadlock, mutation, and safety issues in message aggregator

- Fix deadlock: don't await cancelled timer tasks inside the lock;
  _flush_buffer acquires the same lock, causing a deadlock cycle
- Fix message_event mutation: keep original message_event unmodified
  to preserve message_id/metadata for reply/quote; only pass merged
  message_chain separately
- Fix Plain positional arg: Plain('\n') → Plain(text='\n')
- Fix float() ValueError: wrap delay cast in try/except
- Add MAX_BUFFER_MESSAGES (10) cap to prevent unbounded buffer growth
- Default enabled to false to avoid surprising latency on upgrade
- Fix flush_all: cancel all timers under one lock acquisition, then
  flush outside the lock to avoid deadlock

---------

Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-02-25 14:20:34 +08:00
Dongze Yang
298437f352 feat(platform): add Forward message support for aiocqhttp adapter (#2003)
* feat(platform): add Forward message support for aiocqhttp adapter

- Add _send_forward_message method to send merged forward cards via OneBot API
- Support NapCat's send_forward_msg API with fallback to send_group_forward_msg
- Fix MessageChain deserialization for Forward messages in handler.py
- Properly deserialize nested ForwardMessageNode.message_chain to preserve data

This enables plugins to send QQ merged forward cards through the standard
LangBot send_message API using the Forward message component.

* style: fix ruff lint and format issues

- Remove f-string prefix from log message without placeholders
- Apply ruff format to aiocqhttp.py and handler.py

* refactor: remove custom deserializer, rely on SDK for Forward deserialization

- Remove _deserialize_message_chain from handler.py; use standard
  MessageChain.model_validate() (Forward handling fixed in SDK via
  langbot-app/langbot-plugin-sdk#38)
- Fix group_id type: use int instead of str for OneBot compatibility
- Add warning log when Forward message is used with non-group target

* chore: bump langbot-plugin to 0.2.7 (Forward deserialization fix)

---------

Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-02-25 14:03:17 +08:00
Dongze Yang
94d72c378c fix(web): emit initial form values on mount to prevent saving empty config (#2004)
DynamicFormComponent uses form.watch(callback) to notify parent of form
values, but react-hook-form's watch callback only fires on subsequent
changes, not on mount. This causes PluginForm's currentFormValues to
remain as {} if the user saves without modifying any field, overwriting
the existing plugin config with an empty object in the database.
2026-02-25 13:34:52 +08:00
fdc310
f09ba6a0e3 fix: Add the file upload function and optimize the media message proc… (#2002)
* fix: Add the file upload function and optimize the media message processing

* fix: Optimize the message processing logic, improve the concatenation of text elements and the sending of media messages

* fix: Simplify the file request construction and message processing logic to enhance code readability
2026-02-25 12:24:16 +08:00
Junyan Chin
1eda076b93 feat: add plugin recommendation lists to market page (#2001) 2026-02-24 21:24:36 +08:00
128 changed files with 7539 additions and 3259 deletions

View File

@@ -19,9 +19,10 @@ English / [简体中文](README_CN.md) / [繁體中文](README_TW.md) / [日本
[![GitHub stars](https://img.shields.io/github/stars/langbot-app/LangBot?style=social)](https://github.com/langbot-app/LangBot/stargazers)
<a href="https://langbot.app">Website</a>
<a href="https://docs.langbot.app/en/insight/features.html">Features</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Docs</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API</a>
<a href="https://docs.langbot.app/en/insight/features">Features</a>
<a href="https://docs.langbot.app/en/insight/guide">Docs</a>
<a href="https://docs.langbot.app/en/tags/readme">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">Plugin Market</a>
<a href="https://langbot.featurebase.app/roadmap">Roadmap</a>
@@ -44,12 +45,16 @@ LangBot is an **open-source, production-grade platform** for building AI-powered
- **Web Management Panel** — Configure, manage, and monitor your bots through an intuitive browser interface. No YAML editing required.
- **Multi-Pipeline Architecture** — Different bots for different scenarios, with comprehensive monitoring and exception handling.
[→ Learn more about all features](https://docs.langbot.app/en/insight/features.html)
[→ Learn more about all features](https://docs.langbot.app/en/insight/features)
---
## Quick Start
### ☁️ LangBot Cloud (Recommended)
**[LangBot Cloud](https://space.langbot.app/cloud)** — Zero deployment, ready to use.
### One-Line Launch
```bash
@@ -71,7 +76,7 @@ docker compose up -d
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
**More options:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker.html) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual.html) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) · [Kubernetes](./docker/README_K8S.md)
**More options:** [Docker](https://docs.langbot.app/en/deploy/langbot/docker) · [Manual](https://docs.langbot.app/en/deploy/langbot/manual) · [BTPanel](https://docs.langbot.app/en/deploy/langbot/one-click/bt) · [Kubernetes](./docker/README_K8S.md)
---
@@ -119,7 +124,7 @@ docker compose up -d
| [接口 AI](https://jiekou.ai/) | Gateway | ✅ |
| [302.AI](https://share.302.ai/SuTG99) | Gateway | ✅ |
[→ View all integrations](https://docs.langbot.app/en/insight/features.html)
[→ View all integrations](https://docs.langbot.app/en/insight/features)
---

View File

@@ -24,6 +24,7 @@
<a href="https://docs.langbot.app/zh/insight/features.html">特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">文档</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API</a>
<a href="https://space.langbot.app/cloud">Cloud</a>
<a href="https://space.langbot.app">插件市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a>
@@ -52,6 +53,10 @@ LangBot 是一个**开源的生产级平台**,用于构建 AI 驱动的即时
## 快速开始
### ☁️ LangBot Cloud推荐
**[LangBot Cloud](https://space.langbot.app/cloud)** — 免部署,开箱即用。
### 一键启动
```bash

View File

@@ -50,6 +50,10 @@ LangBot es una **plataforma de código abierto y grado de producción** para con
## Inicio Rápido
### ☁️ LangBot Cloud (Recomendado)
**[LangBot Cloud](https://space.langbot.app/cloud)** — Sin despliegue, listo para usar.
### Lanzamiento en una línea
```bash

View File

@@ -50,6 +50,10 @@ LangBot est une **plateforme open-source de niveau production** pour créer des
## Démarrage Rapide
### ☁️ LangBot Cloud (Recommandé)
**[LangBot Cloud](https://space.langbot.app/cloud)** — Sans déploiement, prêt à utiliser.
### Lancement en une ligne
```bash

View File

@@ -50,6 +50,10 @@ LangBot は、AI搭載のインスタントメッセージングボットを構
## クイックスタート
### ☁️ LangBot Cloud推奨
**[LangBot Cloud](https://space.langbot.app/cloud)** — デプロイ不要、すぐに使えます。
### ワンライン起動
```bash

View File

@@ -50,6 +50,10 @@ LangBot은 AI 기반 인스턴트 메시징 봇을 구축하기 위한 **오픈
## 빠른 시작
### ☁️ LangBot Cloud (추천)
**[LangBot Cloud](https://space.langbot.app/cloud)** — 배포 없이 바로 사용.
### 원라인 실행
```bash

View File

@@ -50,6 +50,10 @@ LangBot — это **платформа с открытым исходным к
## Быстрый старт
### ☁️ LangBot Cloud (Рекомендуется)
**[LangBot Cloud](https://space.langbot.app/cloud)** — Без развёртывания, готово к использованию.
### Запуск одной командой
```bash

View File

@@ -52,6 +52,10 @@ LangBot 是一個**開源的生產級平台**,用於建構 AI 驅動的即時
## 快速開始
### ☁️ LangBot Cloud推薦
**[LangBot Cloud](https://space.langbot.app/cloud)** — 免部署,開箱即用。
### 一鍵啟動
```bash

View File

@@ -50,6 +50,10 @@ LangBot là một **nền tảng mã nguồn mở, cấp sản xuất** để x
## Bắt đầu nhanh
### ☁️ LangBot Cloud (Khuyên dùng)
**[LangBot Cloud](https://space.langbot.app/cloud)** — Không cần triển khai, sẵn sàng sử dụng.
### Khởi chạy một dòng
```bash

View File

@@ -1,6 +1,6 @@
[project]
name = "langbot"
version = "4.8.4"
version = "4.9.0"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
@@ -63,14 +63,15 @@ dependencies = [
"langchain-text-splitters>=0.0.1",
"chromadb>=0.4.24",
"qdrant-client (>=1.15.1,<2.0.0)",
"pyseekdb==1.0.0b7",
"langbot-plugin==0.2.6",
"pyseekdb==1.1.0.post3",
"langbot-plugin==0.3.0",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"tboxsdk>=0.0.10",
"boto3>=1.35.0",
"pymilvus>=2.6.4",
"pgvector>=0.4.1",
"botocore>=1.42.39",
]
keywords = [
"bot",

View File

@@ -1,3 +1,3 @@
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.8.4'
__version__ = '4.9.0'

View File

@@ -1,5 +1,5 @@
import requests
import aiohttp
from langbot.pkg.utils import httpclient
def post_json(base_url, token, data=None):
@@ -63,16 +63,16 @@ async def async_request(
"""
headers = {'Content-Type': 'application/json'}
url = f'{base_url}?key={token_key}'
async with aiohttp.ClientSession() as session:
async with session.request(
method=method, url=url, params=params, headers=headers, data=data, json=json
) as response:
response.raise_for_status() # 如果状态码不是200抛出异常
result = await response.json()
# print(result)
return result
# if result.get('Code') == 200:
#
# return await result
# else:
# raise RuntimeError("请求失败",response.text)
session = httpclient.get_session()
async with session.request(
method=method, url=url, params=params, headers=headers, data=data, json=json
) as response:
response.raise_for_status() # 如果状态码不是200抛出异常
result = await response.json()
# print(result)
return result
# if result.get('Code') == 200:
#
# return await result
# else:
# raise RuntimeError("请求失败",response.text)

View File

@@ -13,7 +13,10 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
elif quart.request.method == 'POST':
json_data = await quart.request.json
knowledge_base_uuid = await self.ap.knowledge_service.create_knowledge_base(json_data)
try:
knowledge_base_uuid = await self.ap.knowledge_service.create_knowledge_base(json_data)
except ValueError as e:
return self.http_status(400, -1, str(e))
return self.success(data={'uuid': knowledge_base_uuid})
return self.http_status(405, -1, 'Method not allowed')
@@ -39,7 +42,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.knowledge_service.update_knowledge_base(knowledge_base_uuid, json_data)
return self.success({})
return self.success(data={'uuid': knowledge_base_uuid})
elif quart.request.method == 'DELETE':
await self.ap.knowledge_service.delete_knowledge_base(knowledge_base_uuid)
@@ -65,8 +68,12 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
if not file_id:
return self.http_status(400, -1, 'File ID is required')
parser_plugin_id = json_data.get('parser_plugin_id')
# 调用服务层方法将文件与知识库关联
task_id = await self.ap.knowledge_service.store_file(knowledge_base_uuid, file_id)
task_id = await self.ap.knowledge_service.store_file(
knowledge_base_uuid, file_id, parser_plugin_id=parser_plugin_id
)
return self.success(
{
'task_id': task_id,
@@ -90,5 +97,13 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
async def retrieve_knowledge_base(knowledge_base_uuid: str) -> str:
json_data = await quart.request.json
query = json_data.get('query')
results = await self.ap.knowledge_service.retrieve_knowledge_base(knowledge_base_uuid, query)
if not query or not query.strip():
return self.http_status(400, -1, 'Query is required and cannot be empty')
# Extract retrieval_settings to allow dynamic control over Knowledge Engine behavior (e.g. top_k, filters)
retrieval_settings = json_data.get('retrieval_settings', {})
results = await self.ap.knowledge_service.retrieve_knowledge_base(
knowledge_base_uuid, query, retrieval_settings
)
return self.success(data={'results': results})

View File

@@ -0,0 +1,45 @@
import quart
from urllib.parse import unquote
from ... import group
@group.group_class('knowledge_engines', '/api/v1/knowledge/engines')
class KnowledgeEnginesRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_knowledge_engines() -> quart.Response:
"""List all available Knowledge Engines from plugins.
Returns a list of Knowledge Engines with their capabilities and configuration schemas.
This is used by the frontend to render the knowledge base creation wizard.
"""
engines = await self.ap.knowledge_service.list_knowledge_engines()
return self.success(data={'engines': engines})
@self.route(
'/<path:plugin_id>/creation-schema', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def get_engine_creation_schema(plugin_id: str) -> quart.Response:
"""Get creation settings schema for a specific Knowledge Engine.
plugin_id is in 'author/name' format, captured via <path:> converter.
"""
plugin_id = unquote(plugin_id)
if '/' not in plugin_id:
return self.http_status(400, -1, 'Invalid plugin_id format. Expected author/name.')
schema = await self.ap.knowledge_service.get_engine_creation_schema(plugin_id)
return self.success(data={'schema': schema})
@self.route(
'/<path:plugin_id>/retrieval-schema', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def get_engine_retrieval_schema(plugin_id: str) -> quart.Response:
"""Get retrieval settings schema for a specific Knowledge Engine.
plugin_id is in 'author/name' format, captured via <path:> converter.
"""
plugin_id = unquote(plugin_id)
if '/' not in plugin_id:
return self.http_status(400, -1, 'Invalid plugin_id format. Expected author/name.')
schema = await self.ap.knowledge_service.get_engine_retrieval_schema(plugin_id)
return self.success(data={'schema': schema})

View File

@@ -1,61 +0,0 @@
import quart
from ... import group
@group.group_class('external_knowledge_base', '/api/v1/knowledge/external-bases')
class ExternalKnowledgeBaseRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/retrievers', methods=['GET'])
async def list_knowledge_retrievers() -> quart.Response:
"""List all available knowledge retrievers from plugins."""
retrievers = await self.ap.plugin_connector.list_knowledge_retrievers()
return self.success(data={'retrievers': retrievers})
@self.route('', methods=['POST', 'GET'])
async def handle_external_knowledge_bases() -> quart.Response:
if quart.request.method == 'GET':
external_kbs = await self.ap.external_kb_service.get_external_knowledge_bases()
return self.success(data={'bases': external_kbs})
elif quart.request.method == 'POST':
json_data = await quart.request.json
kb_uuid = await self.ap.external_kb_service.create_external_knowledge_base(json_data)
return self.success(data={'uuid': kb_uuid})
return self.http_status(405, -1, 'Method not allowed')
@self.route(
'/<kb_uuid>',
methods=['GET', 'DELETE', 'PUT'],
)
async def handle_specific_external_knowledge_base(kb_uuid: str) -> quart.Response:
if quart.request.method == 'GET':
external_kb = await self.ap.external_kb_service.get_external_knowledge_base(kb_uuid)
if external_kb is None:
return self.http_status(404, -1, 'external knowledge base not found')
return self.success(
data={
'base': external_kb,
}
)
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.external_kb_service.update_external_knowledge_base(kb_uuid, json_data)
return self.success({})
elif quart.request.method == 'DELETE':
await self.ap.external_kb_service.delete_external_knowledge_base(kb_uuid)
return self.success({})
@self.route(
'/<kb_uuid>/retrieve',
methods=['POST'],
)
async def retrieve_external_knowledge_base(kb_uuid: str) -> str:
json_data = await quart.request.json
query = json_data.get('query')
results = await self.ap.external_kb_service.retrieve_external_knowledge_base(kb_uuid, query)
return self.success(data={'results': results})

View File

@@ -0,0 +1,372 @@
import asyncio
import json
import httpx
import quart
import sqlalchemy
from ... import group
from ......core import taskmgr
from ......entity.persistence import metadata as persistence_metadata
from langbot_plugin.runtime.plugin.mgr import PluginInstallSource
LANGRAG_PLUGIN_AUTHOR = 'langbot-team'
LANGRAG_PLUGIN_NAME = 'LangRAG'
LANGRAG_PLUGIN_ID = f'{LANGRAG_PLUGIN_AUTHOR}/{LANGRAG_PLUGIN_NAME}'
DEFAULT_SPACE_URL = 'https://space.langbot.app'
# Old Retriever plugin_name -> New Connector plugin_name
EXTERNAL_PLUGIN_NAME_MAPPING = {
'DifyDatasetsRetriever': 'DifyDatasetsConnector',
'RAGFlowRetriever': 'RAGFlowConnector',
'FastGPTRetriever': 'FastGPTConnector',
}
# Per-plugin: which old retriever_config fields belong to creation_settings.
# Remaining fields go to retrieval_settings.
# None means ALL fields go to creation_settings (no retrieval_schema).
EXTERNAL_PLUGIN_CREATION_FIELDS: dict[str, set[str] | None] = {
'langbot-team/DifyDatasetsConnector': {'api_base_url', 'dify_apikey', 'dataset_id'},
'langbot-team/RAGFlowConnector': {'api_base_url', 'api_key', 'dataset_ids'},
'langbot-team/FastGPTConnector': None, # all fields -> creation_settings
}
@group.group_class('knowledge/migration', '/api/v1/knowledge/migration')
class KnowledgeMigrationRouterGroup(group.RouterGroup):
async def _get_migration_flag(self) -> bool:
"""Check if rag_plugin_migration_needed flag is set."""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_metadata.Metadata).where(
persistence_metadata.Metadata.key == 'rag_plugin_migration_needed'
)
)
row = result.first()
return row is not None and row.value == 'true'
async def _set_migration_flag(self, value: str):
"""Set rag_plugin_migration_needed flag."""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_metadata.Metadata)
.where(persistence_metadata.Metadata.key == 'rag_plugin_migration_needed')
.values(value=value)
)
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _install_plugin_from_marketplace(
self, plugin_id: str, task_context: taskmgr.TaskContext, space_url: str
) -> None:
"""Install a single plugin from the marketplace."""
p_author, p_name = plugin_id.split('/', 1)
self.ap.logger.info(f'RAG migration: installing plugin {plugin_id} from marketplace...')
task_context.trace(f'Installing plugin {plugin_id} from marketplace...')
async with httpx.AsyncClient(trust_env=True, timeout=15) as client:
resp = await client.get(f'{space_url}/api/v1/marketplace/plugins/{p_author}/{p_name}')
resp.raise_for_status()
p_data = resp.json().get('data', {}).get('plugin', {})
p_version = p_data.get('latest_version')
if not p_version:
raise Exception(f'Could not determine latest version for {plugin_id}')
await self.ap.plugin_connector.install_plugin(
PluginInstallSource.MARKETPLACE,
{
'plugin_author': p_author,
'plugin_name': p_name,
'plugin_version': p_version,
},
task_context=task_context,
)
self.ap.logger.info(f'RAG migration: plugin {plugin_id} install request sent.')
async def _execute_rag_migration(self, task_context: taskmgr.TaskContext, install_plugin: bool = True):
"""Execute RAG migration: install required plugins and restore backup data."""
warnings = []
# Collect all plugins we need: LangRAG (always) + connector plugins (from external KBs)
needed_plugins: dict[str, str] = {
LANGRAG_PLUGIN_ID: LANGRAG_PLUGIN_NAME,
}
has_external = await self._table_exists('external_knowledge_bases')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT DISTINCT plugin_author, plugin_name FROM external_knowledge_bases;')
)
for row in result.fetchall():
plugin_author = row[0] or ''
plugin_name = row[1] or ''
mapped_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
plugin_id = f'{plugin_author}/{mapped_name}'
if plugin_id not in needed_plugins:
needed_plugins[plugin_id] = mapped_name
self.ap.logger.info(f'RAG migration: plugins needed: {list(needed_plugins.keys())}')
if install_plugin:
# Step 1: Install all required plugins from marketplace
task_context.trace('Installing required plugins...', action='install-plugin')
space_url = self.ap.instance_config.data.get('space', {}).get('url', DEFAULT_SPACE_URL).rstrip('/')
for plugin_id in needed_plugins:
try:
await self._install_plugin_from_marketplace(plugin_id, task_context, space_url)
except Exception as e:
self.ap.logger.warning(f'RAG migration: plugin {plugin_id} install returned: {e}')
task_context.trace(f'Plugin install note ({plugin_id}): {e}')
# Step 2: Wait for all plugins to become available as knowledge engines
task_context.trace(
f'Waiting for plugins to become available: {list(needed_plugins.keys())}...',
action='wait-plugin',
)
max_retries = 30
engine_id_set: set[str] = set()
for i in range(max_retries):
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
pass
if all(pid in engine_id_set for pid in needed_plugins):
self.ap.logger.info(f'RAG migration: all plugins ready: {engine_id_set}')
task_context.trace('All required plugins are ready.')
break
if i == max_retries - 1:
still_missing = [pid for pid in needed_plugins if pid not in engine_id_set]
warning = f'Plugin(s) {still_missing} did not become available after {max_retries} retries'
self.ap.logger.warning(f'RAG migration: {warning}')
warnings.append(warning)
task_context.trace(warning)
await asyncio.sleep(2)
else:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_id_set = {e.get('plugin_id') for e in engines}
except Exception:
engine_id_set = set()
# Step 3: Restore internal knowledge bases from backup
task_context.trace('Restoring internal knowledge bases...', action='restore-internal')
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM knowledge_bases_backup;')
)
rows = result.fetchall()
columns = result.keys()
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f4da')
embedding_model_uuid = row_dict.get('embedding_model_uuid', '')
top_k = row_dict.get('top_k', 5)
created_at = row_dict.get('created_at')
updated_at = row_dict.get('updated_at')
creation_settings = json.dumps({'embedding_model_uuid': embedding_model_uuid})
retrieval_settings = json.dumps({'top_k': top_k})
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=updated_at,
plugin_id=LANGRAG_PLUGIN_ID,
collection_id=kb_uuid,
creation_settings=creation_settings,
retrieval_settings=retrieval_settings,
)
)
try:
config = {'embedding_model_uuid': embedding_model_uuid}
await self.ap.plugin_connector.rag_on_kb_create(LANGRAG_PLUGIN_ID, kb_uuid, config)
task_context.trace(f'Restored internal KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 4: Restore external knowledge bases
task_context.trace('Restoring external knowledge bases...', action='restore-external')
if has_external:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT * FROM external_knowledge_bases;')
)
rows = result.fetchall()
columns = result.keys()
self.ap.logger.info(
f'RAG migration: {len(rows)} external KB(s) to restore. Available engines: {engine_id_set}'
)
task_context.trace(f'Found {len(rows)} external KB(s). Available engines: {engine_id_set}')
for row in rows:
row_dict = dict(zip(columns, row))
kb_uuid = row_dict.get('uuid')
name = row_dict.get('name', 'Untitled')
description = row_dict.get('description', '')
emoji = row_dict.get('emoji', '\U0001f517')
plugin_author = row_dict.get('plugin_author', '')
plugin_name = row_dict.get('plugin_name', '')
retriever_config = row_dict.get('retriever_config', {})
created_at = row_dict.get('created_at')
mapped_plugin_name = EXTERNAL_PLUGIN_NAME_MAPPING.get(plugin_name, plugin_name)
external_plugin_id = f'{plugin_author}/{mapped_plugin_name}'
self.ap.logger.info(
f'RAG migration: processing external KB "{name}" ({kb_uuid}), '
f'plugin: {plugin_author}/{plugin_name} -> {external_plugin_id}'
)
if isinstance(retriever_config, str):
try:
retriever_config = json.loads(retriever_config)
except (json.JSONDecodeError, TypeError):
retriever_config = {}
creation_fields = EXTERNAL_PLUGIN_CREATION_FIELDS.get(external_plugin_id)
if creation_fields is None:
creation_settings_dict = retriever_config
retrieval_settings_dict = {}
else:
creation_settings_dict = {k: v for k, v in retriever_config.items() if k in creation_fields}
retrieval_settings_dict = {k: v for k, v in retriever_config.items() if k not in creation_fields}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'INSERT INTO knowledge_bases '
'(uuid, name, description, emoji, created_at, updated_at, '
'knowledge_engine_plugin_id, collection_id, creation_settings, retrieval_settings) '
'VALUES (:uuid, :name, :description, :emoji, :created_at, :updated_at, '
':plugin_id, :collection_id, :creation_settings, :retrieval_settings);'
).bindparams(
uuid=kb_uuid,
name=name,
description=description,
emoji=emoji,
created_at=created_at,
updated_at=created_at,
plugin_id=external_plugin_id,
collection_id=kb_uuid,
creation_settings=json.dumps(creation_settings_dict),
retrieval_settings=json.dumps(retrieval_settings_dict),
)
)
if external_plugin_id not in engine_id_set:
warning = (
f'External KB "{name}" ({kb_uuid}) record saved, but plugin {external_plugin_id} '
f'is not installed yet. Install the connector plugin to use it.'
)
warnings.append(warning)
task_context.trace(warning)
else:
try:
await self.ap.plugin_connector.rag_on_kb_create(
external_plugin_id, kb_uuid, creation_settings_dict
)
task_context.trace(f'Restored external KB: {name} ({kb_uuid})')
except Exception as e:
warning = f'Failed to notify plugin for external KB {name} ({kb_uuid}): {e}'
warnings.append(warning)
task_context.trace(warning)
await self.ap.rag_mgr.load_knowledge_bases_from_db()
# Step 5: Clear migration flag
await self._set_migration_flag('false')
task_context.trace('RAG migration completed.', action='done')
if warnings:
task_context.trace(f'Completed with {len(warnings)} warning(s).')
async def initialize(self) -> None:
@self.route('/status', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
internal_kb_count = 0
external_kb_count = 0
if needed:
if await self._table_exists('knowledge_bases_backup'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases_backup;')
)
internal_kb_count = result.scalar() or 0
if await self._table_exists('external_knowledge_bases'):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
external_kb_count = result.scalar() or 0
return self.success(
data={
'needed': needed,
'internal_kb_count': internal_kb_count,
'external_kb_count': external_kb_count,
}
)
@self.route('/execute', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
data = await quart.request.get_json(silent=True) or {}
install_plugin = data.get('install_plugin', True)
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self._execute_rag_migration(task_context=ctx, install_plugin=install_plugin),
kind='rag-migration',
name='rag-migration-execute',
label='Migrating knowledge bases to plugin architecture',
context=ctx,
)
return self.success(data={'task_id': wrapper.id})
@self.route('/dismiss', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
needed = await self._get_migration_flag()
if not needed:
return self.http_status(400, -1, 'RAG migration is not needed')
await self._set_migration_flag('false')
return self.success()

View File

@@ -0,0 +1,16 @@
import quart
from ... import group
@group.group_class('parsers', '/api/v1/knowledge/parsers')
class ParsersRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def list_parsers() -> quart.Response:
"""List all available parsers from plugins.
Optional query parameter `mime_type` to filter parsers by supported MIME type.
"""
mime_type = quart.request.args.get('mime_type')
parsers = await self.ap.knowledge_service.list_parsers(mime_type)
return self.success(data={'parsers': parsers})

View File

@@ -52,6 +52,7 @@ class MonitoringRouterGroup(group.RouterGroup):
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
session_ids = quart.request.args.getlist('sessionId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
@@ -64,6 +65,7 @@ class MonitoringRouterGroup(group.RouterGroup):
messages, total = await self.ap.monitoring_service.get_messages(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
session_ids=session_ids if session_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,

View File

@@ -68,7 +68,7 @@ class PipelinesRouterGroup(group.RouterGroup):
return self.http_status(404, -1, 'pipeline not found')
# Only include plugins with pipeline-related components (Command, EventListener, Tool)
# Plugins that only have KnowledgeRetriever components are not suitable for pipeline extensions
# Plugins that only have KnowledgeEngine components are not suitable for pipeline extensions
pipeline_component_kinds = ['Command', 'EventListener', 'Tool']
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)

View File

@@ -0,0 +1,47 @@
import quart
from .. import group
@group.group_class('survey', '/api/v1/survey')
class SurveyRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/pending', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _get_pending() -> str:
"""Get pending survey for the frontend to display."""
survey = self.ap.survey.get_pending_survey() if self.ap.survey else None
return self.success(data={'survey': survey})
@self.route('/respond', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _respond() -> str:
"""Submit survey response."""
json_data = await quart.request.json
survey_id = json_data.get('survey_id')
answers = json_data.get('answers', {})
completed = json_data.get('completed', True)
if not survey_id:
return self.fail(1, 'survey_id required')
if self.ap.survey:
ok = await self.ap.survey.submit_response(survey_id, answers, completed)
if ok:
return self.success()
return self.fail(2, 'Failed to submit response')
return self.fail(3, 'Survey not available')
@self.route('/dismiss', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _dismiss() -> str:
"""Dismiss survey."""
json_data = await quart.request.json
survey_id = json_data.get('survey_id')
if not survey_id:
return self.fail(1, 'survey_id required')
if self.ap.survey:
ok = await self.ap.survey.dismiss_survey(survey_id)
if ok:
return self.success()
return self.fail(2, 'Failed to dismiss')
return self.fail(3, 'Survey not available')

View File

@@ -1,80 +0,0 @@
from __future__ import annotations
from ....core import app
import sqlalchemy
from langbot.pkg.entity.persistence import rag as persistence_rag
import uuid
class ExternalKBService:
"""External KB service"""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
# External Knowledge Base methods
async def get_external_knowledge_bases(self) -> list[dict]:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.ExternalKnowledgeBase))
external_kbs = result.all()
return [
self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
for external_kb in external_kbs
]
async def get_external_knowledge_base(self, kb_uuid: str) -> dict | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
external_kb = result.first()
if external_kb is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
async def create_external_knowledge_base(self, kb_data: dict) -> str:
kb_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_rag.ExternalKnowledgeBase).values(kb_data)
)
kb = await self.get_external_knowledge_base(kb_data['uuid'])
await self.ap.rag_mgr.load_external_knowledge_base(kb)
return kb_data['uuid']
async def retrieve_external_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
"""Retrieve external knowledge base"""
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return [
result.model_dump() for result in await runtime_kb.retrieve(query, 5)
] # top_k is just a placeholder for external knowledge base
async def update_external_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
if 'uuid' in kb_data:
del kb_data['uuid']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.ExternalKnowledgeBase)
.values(kb_data)
.where(persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid)
)
await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid)
kb = await self.get_external_knowledge_base(kb_uuid)
await self.ap.rag_mgr.load_external_knowledge_base(kb)
async def delete_external_knowledge_base(self, kb_uuid: str) -> None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)

View File

@@ -1,6 +1,5 @@
from __future__ import annotations
import uuid
import sqlalchemy
from ....core import app
@@ -17,64 +16,77 @@ class KnowledgeService:
async def get_knowledge_bases(self) -> list[dict]:
"""获取所有知识库"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
return [
self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base)
for knowledge_base in knowledge_bases
]
return await self.ap.rag_mgr.get_all_knowledge_base_details()
async def get_knowledge_base(self, kb_uuid: str) -> dict | None:
"""获取知识库"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
knowledge_base = result.first()
if knowledge_base is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base)
return await self.ap.rag_mgr.get_knowledge_base_details(kb_uuid)
async def create_knowledge_base(self, kb_data: dict) -> str:
"""创建知识库"""
kb_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.KnowledgeBase).values(kb_data))
# In new architecture, we delegate entirely to RAGManager which uses plugins.
# Legacy internal KB creation is removed.
kb = await self.get_knowledge_base(kb_data['uuid'])
knowledge_engine_plugin_id = kb_data.get('knowledge_engine_plugin_id')
if not knowledge_engine_plugin_id:
raise ValueError('knowledge_engine_plugin_id is required')
await self.ap.rag_mgr.load_knowledge_base(kb)
return kb_data['uuid']
kb = await self.ap.rag_mgr.create_knowledge_base(
name=kb_data.get('name', 'Untitled'),
knowledge_engine_plugin_id=knowledge_engine_plugin_id,
creation_settings=kb_data.get('creation_settings', {}),
retrieval_settings=kb_data.get('retrieval_settings', {}),
description=kb_data.get('description', ''),
)
return kb.uuid
async def update_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
"""更新知识库"""
if 'uuid' in kb_data:
del kb_data['uuid']
# Filter to only mutable fields
filtered_data = {k: v for k, v in kb_data.items() if k in persistence_rag.KnowledgeBase.MUTABLE_FIELDS}
if 'embedding_model_uuid' in kb_data:
del kb_data['embedding_model_uuid']
if not filtered_data:
return
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.KnowledgeBase)
.values(kb_data)
.values(filtered_data)
.where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid)
kb = await self.get_knowledge_base(kb_uuid)
if kb is None:
raise Exception('Knowledge base not found after update')
await self.ap.rag_mgr.load_knowledge_base(kb)
async def store_file(self, kb_uuid: str, file_id: str) -> int:
async def _check_doc_capability(self, kb_uuid: str, operation: str) -> None:
"""Check if the KB's Knowledge Engine supports document operations.
Args:
kb_uuid: Knowledge base UUID.
operation: Human-readable operation name for error messages.
Raises:
Exception: If the KB does not support doc_ingestion.
"""
kb_info = await self.ap.rag_mgr.get_knowledge_base_details(kb_uuid)
if not kb_info:
raise Exception('Knowledge base not found')
capabilities = kb_info.get('knowledge_engine', {}).get('capabilities', [])
if 'doc_ingestion' not in capabilities:
raise Exception(f'This knowledge base does not support {operation}')
async def store_file(self, kb_uuid: str, file_id: str, parser_plugin_id: str | None = None) -> str:
"""存储文件"""
# await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.File).values(kb_id=kb_uuid, file_id=file_id))
# await self.ap.rag_mgr.store_file(file_id)
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
# Only internal KBs support file storage
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file storage')
result = await runtime_kb.store_file(file_id)
await self._check_doc_capability(kb_uuid, 'document upload')
result = await runtime_kb.store_file(file_id, parser_plugin_id=parser_plugin_id)
# Update the KB's updated_at timestamp
await self.ap.persistence_mgr.execute_async(
@@ -85,14 +97,18 @@ class KnowledgeService:
return result
async def retrieve_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
async def retrieve_knowledge_base(
self, kb_uuid: str, query: str, retrieval_settings: dict | None = None
) -> list[dict]:
"""检索知识库"""
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return [
result.model_dump() for result in await runtime_kb.retrieve(query, runtime_kb.knowledge_base_entity.top_k)
]
# Pass retrieval_settings
results = await runtime_kb.retrieve(query, settings=retrieval_settings)
return [result.model_dump() for result in results]
async def get_files_by_knowledge_base(self, kb_uuid: str) -> list[dict]:
"""获取知识库文件"""
@@ -107,9 +123,9 @@ class KnowledgeService:
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
# Only internal KBs support file deletion
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file deletion')
await self._check_doc_capability(kb_uuid, 'document deletion')
await runtime_kb.delete_file(file_id)
# Update the KB's updated_at timestamp
@@ -121,13 +137,14 @@ class KnowledgeService:
async def delete_knowledge_base(self, kb_uuid: str) -> None:
"""删除知识库"""
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)
# Delete from DB first to commit the deletion, then clean up runtime/plugin (best-effort)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
# delete files
# NOTE: Chunk cleanup is for legacy (pre-plugin) KBs that stored chunks locally.
# For plugin-based Knowledge Engines, the Chunk table is not populated, so this is a no-op.
files = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.File).where(persistence_rag.File.kb_id == kb_uuid)
)
@@ -140,3 +157,53 @@ class KnowledgeService:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file.uuid)
)
# Remove from runtime and notify plugin (best-effort, DB is already cleaned up)
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)
# ================= Knowledge Engine Discovery =================
async def list_knowledge_engines(self) -> list[dict]:
"""List all available Knowledge Engines from plugins."""
engines = []
if not self.ap.plugin_connector.is_enable_plugin:
return engines
# Get KnowledgeEngine plugins
try:
knowledge_engines = await self.ap.plugin_connector.list_knowledge_engines()
engines.extend(knowledge_engines)
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines from plugins: {e}')
return engines
async def list_parsers(self, mime_type: str | None = None) -> list[dict]:
"""List available parsers, optionally filtered by MIME type."""
if not self.ap.plugin_connector.is_enable_plugin:
return []
try:
parsers = await self.ap.plugin_connector.list_parsers()
if mime_type:
parsers = [p for p in parsers if mime_type in p.get('supported_mime_types', [])]
return parsers
except Exception as e:
self.ap.logger.warning(f'Failed to list parsers: {e}')
return []
async def get_engine_creation_schema(self, plugin_id: str) -> dict:
"""Get creation settings schema for a specific Knowledge Engine."""
try:
return await self.ap.plugin_connector.get_rag_creation_schema(plugin_id)
except Exception as e:
self.ap.logger.warning(f'Failed to get creation schema for {plugin_id}: {e}')
return {}
async def get_engine_retrieval_schema(self, plugin_id: str) -> dict:
"""Get retrieval settings schema for a specific Knowledge Engine."""
try:
return await self.ap.plugin_connector.get_rag_retrieval_schema(plugin_id)
except Exception as e:
self.ap.logger.warning(f'Failed to get retrieval schema for {plugin_id}: {e}')
return {}

View File

@@ -32,6 +32,7 @@ class MonitoringService:
user_id: str | None = None,
runner_name: str | None = None,
variables: str | None = None,
role: str = 'user',
) -> str:
"""Record a message"""
message_id = str(uuid.uuid4())
@@ -50,6 +51,7 @@ class MonitoringService:
'user_id': user_id,
'runner_name': runner_name,
'variables': variables,
'role': role,
}
await self.ap.persistence_mgr.execute_async(
@@ -355,6 +357,7 @@ class MonitoringService:
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
session_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
@@ -367,6 +370,8 @@ class MonitoringService:
conditions.append(persistence_monitoring.MonitoringMessage.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringMessage.pipeline_id.in_(pipeline_ids))
if session_ids:
conditions.append(persistence_monitoring.MonitoringMessage.session_id.in_(session_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringMessage.timestamp >= start_time)
if end_time:

View File

@@ -1,6 +1,6 @@
from __future__ import annotations
import aiohttp
from langbot.pkg.utils import httpclient
import typing
import datetime
import time
@@ -99,49 +99,49 @@ class SpaceService:
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.post(
f'{space_url}/api/v1/accounts/oauth/token',
json={'code': code, 'instance_id': constants.instance_id},
) as response:
if response.status != 200:
raise ValueError(f'Failed to exchange OAuth code: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to exchange OAuth code: {data.get("msg")}')
return data.get('data', {})
session = httpclient.get_session()
async with session.post(
f'{space_url}/api/v1/accounts/oauth/token',
json={'code': code, 'instance_id': constants.instance_id},
) as response:
if response.status != 200:
raise ValueError(f'Failed to exchange OAuth code: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to exchange OAuth code: {data.get("msg")}')
return data.get('data', {})
async def refresh_token(self, refresh_token: str) -> typing.Dict:
"""Refresh Space access token"""
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.post(
f'{space_url}/api/v1/accounts/token/refresh', json={'refresh_token': refresh_token}
) as response:
if response.status != 200:
raise ValueError(f'Failed to refresh token: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to refresh token: {data.get("msg")}')
return data.get('data', {})
session = httpclient.get_session()
async with session.post(
f'{space_url}/api/v1/accounts/token/refresh', json={'refresh_token': refresh_token}
) as response:
if response.status != 200:
raise ValueError(f'Failed to refresh token: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to refresh token: {data.get("msg")}')
return data.get('data', {})
async def get_user_info_raw(self, access_token: str) -> typing.Dict:
"""Get user info from Space using access token (no validation)"""
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.get(
f'{space_url}/api/v1/accounts/me', headers={'Authorization': f'Bearer {access_token}'}
) as response:
if response.status != 200:
raise ValueError(f'Failed to get user info: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get user info: {data.get("msg")}')
return data.get('data', {})
session = httpclient.get_session()
async with session.get(
f'{space_url}/api/v1/accounts/me', headers={'Authorization': f'Bearer {access_token}'}
) as response:
if response.status != 200:
raise ValueError(f'Failed to get user info: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get user info: {data.get("msg")}')
return data.get('data', {})
# === API calls with token validation ===
@@ -178,12 +178,12 @@ class SpaceService:
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.get(f'{space_url}/api/v1/models') as response:
if response.status != 200:
raise ValueError(f'Failed to get models: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get models: {data.get("msg")}')
models_data = data.get('data', {}).get('models', [])
return [SpaceModel.model_validate(model_dict) for model_dict in models_data]
session = httpclient.get_session()
async with session.get(f'{space_url}/api/v1/models') as response:
if response.status != 200:
raise ValueError(f'Failed to get models: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get models: {data.get("msg")}')
models_data = data.get('data', {}).get('models', [])
return [SpaceModel.model_validate(model_dict) for model_dict in models_data]

View File

@@ -15,6 +15,7 @@ from ..command import cmdmgr
from ..plugin import connector as plugin_connector
from ..pipeline import pool
from ..pipeline import controller, pipelinemgr
from ..pipeline import aggregator as message_aggregator
from ..utils import version as version_mgr, proxy as proxy_mgr
from ..persistence import mgr as persistencemgr
from ..api.http.controller import main as http_controller
@@ -28,7 +29,6 @@ from ..api.http.service import knowledge as knowledge_service
from ..api.http.service import mcp as mcp_service
from ..api.http.service import apikey as apikey_service
from ..api.http.service import webhook as webhook_service
from ..api.http.service import external_kb as external_kb_service
from ..api.http.service import monitoring as monitoring_service
from ..discover import engine as discover_engine
from ..storage import mgr as storagemgr
@@ -36,8 +36,10 @@ from ..utils import logcache
from . import taskmgr
from . import entities as core_entities
from ..rag.knowledge import kbmgr as rag_mgr
from ..rag.service import RAGRuntimeService
from ..vector import mgr as vectordb_mgr
from ..telemetry import telemetry as telemetry_module
from ..survey import manager as survey_module
class Application:
@@ -61,6 +63,7 @@ class Application:
model_mgr: llm_model_mgr.ModelManager = None
rag_mgr: rag_mgr.RAGManager = None
rag_runtime_service: RAGRuntimeService = None
# TODO move to pipeline
tool_mgr: llm_tool_mgr.ToolManager = None
@@ -96,6 +99,8 @@ class Application:
query_pool: pool.QueryPool = None
msg_aggregator: message_aggregator.MessageAggregator = None
ctrl: controller.Controller = None
pipeline_mgr: pipelinemgr.PipelineManager = None
@@ -134,8 +139,6 @@ class Application:
knowledge_service: knowledge_service.KnowledgeService = None
external_kb_service: external_kb_service.ExternalKBService = None
mcp_service: mcp_service.MCPService = None
apikey_service: apikey_service.ApiKeyService = None
@@ -144,6 +147,8 @@ class Application:
telemetry: telemetry_module.TelemetryManager = None
survey: survey_module.SurveyManager = None
monitoring_service: monitoring_service.MonitoringService = None
def __init__(self):

View File

@@ -1,3 +1,4 @@
import importlib.util
import pip
import os
from ...utils import pkgmgr
@@ -49,9 +50,10 @@ async def check_deps() -> list[str]:
missing_deps = []
for dep in required_deps:
try:
__import__(dep)
except ImportError:
# Use find_spec instead of __import__ to avoid actually loading
# all modules into memory. find_spec only checks if the module
# can be found, without executing module-level code.
if importlib.util.find_spec(dep) is None:
missing_deps.append(dep)
return missing_deps

View File

@@ -5,12 +5,14 @@ import asyncio
from .. import stage, app
from ...utils import version, proxy
from ...pipeline import pool, controller, pipelinemgr
from ...pipeline import aggregator as message_aggregator
from ...plugin import connector as plugin_connector
from ...command import cmdmgr
from ...provider.session import sessionmgr as llm_session_mgr
from ...provider.modelmgr import modelmgr as llm_model_mgr
from ...provider.tools import toolmgr as llm_tool_mgr
from ...rag.knowledge import kbmgr as rag_mgr
from ...rag.service import RAGRuntimeService
from ...platform import botmgr as im_mgr
from ...platform.webhook_pusher import WebhookPusher
from ...persistence import mgr as persistencemgr
@@ -25,7 +27,6 @@ from ...api.http.service import knowledge as knowledge_service
from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service
from ...api.http.service import external_kb as external_kb_service
from ...api.http.service import monitoring as monitoring_service
from ...discover import engine as discover_engine
from ...storage import mgr as storagemgr
@@ -33,6 +34,7 @@ from ...utils import logcache
from ...vector import mgr as vectordb_mgr
from .. import taskmgr
from ...telemetry import telemetry as telemetry_module
from ...survey import manager as survey_module
@stage.stage_class('BuildAppStage')
@@ -71,9 +73,6 @@ class BuildAppStage(stage.BootingStage):
knowledge_service_inst = knowledge_service.KnowledgeService(ap)
ap.knowledge_service = knowledge_service_inst
external_kb_service_inst = external_kb_service.ExternalKBService(ap)
ap.external_kb_service = external_kb_service_inst
mcp_service_inst = mcp_service.MCPService(ap)
ap.mcp_service = mcp_service_inst
@@ -109,6 +108,11 @@ class BuildAppStage(stage.BootingStage):
await telemetry_inst.initialize()
ap.telemetry = telemetry_inst
# Survey manager
survey_inst = survey_module.SurveyManager(ap)
await survey_inst.initialize()
ap.survey = survey_inst
cmd_mgr_inst = cmdmgr.CommandManager(ap)
await cmd_mgr_inst.initialize()
ap.cmd_mgr = cmd_mgr_inst
@@ -137,10 +141,17 @@ class BuildAppStage(stage.BootingStage):
await pipeline_mgr.initialize()
ap.pipeline_mgr = pipeline_mgr
# Initialize message aggregator (after pipeline_mgr, as it needs pipeline config)
msg_aggregator_inst = message_aggregator.MessageAggregator(ap)
ap.msg_aggregator = msg_aggregator_inst
rag_mgr_inst = rag_mgr.RAGManager(ap)
await rag_mgr_inst.initialize()
ap.rag_mgr = rag_mgr_inst
# Initialize RAG Runtime Service for plugins
ap.rag_runtime_service = RAGRuntimeService(ap)
# 初始化向量数据库管理器
vectordb_mgr_inst = vectordb_mgr.VectorDBManager(ap)
await vectordb_mgr_inst.initialize()

View File

@@ -22,6 +22,7 @@ class MonitoringMessage(Base):
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
runner_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # Runner name for this query
variables = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # Query variables as JSON string
role = sqlalchemy.Column(sqlalchemy.String(50), nullable=True, default='user') # user, assistant
class MonitoringLLMCall(Base):

View File

@@ -10,8 +10,21 @@ class KnowledgeBase(Base):
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='📚')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now(), onupdate=sqlalchemy.func.now())
embedding_model_uuid = sqlalchemy.Column(sqlalchemy.String, default='')
top_k = sqlalchemy.Column(sqlalchemy.Integer, default=5)
# New fields for plugin-based RAG
knowledge_engine_plugin_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)
collection_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)
creation_settings = sqlalchemy.Column(sqlalchemy.JSON, nullable=True, default=None)
retrieval_settings = sqlalchemy.Column(sqlalchemy.JSON, nullable=True, default=None)
# Field sets for different operations
MUTABLE_FIELDS = {'name', 'description', 'retrieval_settings'}
"""Fields that can be updated after creation."""
CREATE_FIELDS = MUTABLE_FIELDS | {'uuid', 'knowledge_engine_plugin_id', 'collection_id', 'creation_settings'}
"""Fields used when creating a new knowledge base."""
ALL_DB_FIELDS = CREATE_FIELDS | {'emoji', 'created_at', 'updated_at'}
"""All fields stored in database (for loading from DB row)."""
class File(Base):
@@ -29,16 +42,3 @@ class Chunk(Base):
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
file_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
text = sqlalchemy.Column(sqlalchemy.Text)
class ExternalKnowledgeBase(Base):
__tablename__ = 'external_knowledge_bases'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String, index=True)
description = sqlalchemy.Column(sqlalchemy.Text)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='🔗')
plugin_author = sqlalchemy.Column(sqlalchemy.String, nullable=False)
plugin_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())

View File

@@ -0,0 +1,24 @@
import sqlalchemy
from .. import migration
@migration.migration_class(19)
class DBMigrateMonitoringMessageRole(migration.DBMigration):
"""Add role column to monitoring_messages table"""
async def upgrade(self):
"""Upgrade"""
try:
sql_text = sqlalchemy.text("ALTER TABLE monitoring_messages ADD COLUMN role VARCHAR(50) DEFAULT 'user'")
await self.ap.persistence_mgr.execute_async(sql_text)
except Exception:
# Column may already exist
pass
async def downgrade(self):
"""Downgrade"""
try:
sql_text = sqlalchemy.text('ALTER TABLE monitoring_messages DROP COLUMN role')
await self.ap.persistence_mgr.execute_async(sql_text)
except Exception:
pass

View File

@@ -0,0 +1,161 @@
import sqlalchemy
from .. import migration
@migration.migration_class(20)
class DBMigrateKnowledgeEnginePluginArchitecture(migration.DBMigration):
"""Migrate to unified Knowledge Engine plugin architecture.
Changes:
- Backup existing knowledge_bases data to knowledge_bases_backup
- Clear knowledge_bases table and add new plugin architecture columns
- Drop old columns (PostgreSQL only; SQLite leaves them unmapped)
- Preserve external_knowledge_bases table as-is for future migration
- Set rag_plugin_migration_needed flag in metadata if old data exists
"""
async def upgrade(self):
"""Upgrade"""
has_internal_data = await self._backup_knowledge_bases()
has_external_data = await self._check_external_knowledge_bases()
await self._clear_knowledge_bases()
await self._add_columns_to_knowledge_bases()
await self._drop_old_columns()
if has_internal_data or has_external_data:
await self._set_migration_flag()
async def _get_table_columns(self, table_name: str) -> list[str]:
"""Get column names from a table (works for both SQLite and PostgreSQL)."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT column_name FROM information_schema.columns WHERE table_name = :table_name;'
).bindparams(table_name=table_name)
)
return [row[0] for row in result.fetchall()]
else:
# SQLite PRAGMA does not support bind parameters; validate identifier.
if not table_name.isidentifier():
raise ValueError(f'Invalid table name: {table_name}')
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
return [row[1] for row in result.fetchall()]
async def _table_exists(self, table_name: str) -> bool:
"""Check if a table exists."""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = :table_name);'
).bindparams(table_name=table_name)
)
return result.scalar()
else:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT name FROM sqlite_master WHERE type='table' AND name=:table_name;").bindparams(
table_name=table_name
)
)
return result.first() is not None
async def _backup_knowledge_bases(self) -> bool:
"""Backup knowledge_bases data. Returns True if data was backed up."""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('SELECT COUNT(*) FROM knowledge_bases;'))
count = result.scalar()
if count == 0:
return False
# Drop backup table if it already exists (from a previous failed migration)
if await self._table_exists('knowledge_bases_backup'):
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DROP TABLE knowledge_bases_backup;'))
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('CREATE TABLE knowledge_bases_backup AS SELECT * FROM knowledge_bases;')
)
self.ap.logger.info(
'Backed up %d knowledge base(s) to knowledge_bases_backup table.',
count,
)
return True
async def _check_external_knowledge_bases(self) -> bool:
"""Check if external_knowledge_bases table exists and has data.
The table is preserved as-is (not dropped) for future migration.
"""
if not await self._table_exists('external_knowledge_bases'):
return False
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT COUNT(*) FROM external_knowledge_bases;')
)
count = result.scalar()
if count > 0:
self.ap.logger.info(
'Found %d external knowledge base(s) in external_knowledge_bases table. '
'Table preserved for future migration.',
count,
)
return count > 0
async def _clear_knowledge_bases(self):
"""Clear all rows from knowledge_bases table (preserve table structure)."""
await self.ap.persistence_mgr.execute_async(sqlalchemy.text('DELETE FROM knowledge_bases;'))
async def _add_columns_to_knowledge_bases(self):
"""Add new RAG plugin architecture columns to knowledge_bases table."""
columns = await self._get_table_columns('knowledge_bases')
new_columns = {
'knowledge_engine_plugin_id': 'VARCHAR',
'collection_id': 'VARCHAR',
'creation_settings': 'TEXT', # JSON stored as TEXT for SQLite compatibility
'retrieval_settings': 'TEXT',
}
for col_name, col_type in new_columns.items():
if col_name not in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE knowledge_bases ADD COLUMN {col_name} {col_type};')
)
async def _drop_old_columns(self):
"""Drop embedding_model_uuid and top_k columns (PostgreSQL only).
SQLite does not support DROP COLUMN in older versions, so we leave the
columns in place — the SQLAlchemy entity simply won't map them.
"""
if self.ap.persistence_mgr.db.name != 'postgresql':
return
columns = await self._get_table_columns('knowledge_bases')
if 'embedding_model_uuid' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN embedding_model_uuid;')
)
if 'top_k' in columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases DROP COLUMN top_k;')
)
async def _set_migration_flag(self):
"""Set rag_plugin_migration_needed flag in metadata table."""
# Check if the key already exists
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT value FROM metadata WHERE key = 'rag_plugin_migration_needed';")
)
row = result.first()
if row is not None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("UPDATE metadata SET value = 'true' WHERE key = 'rag_plugin_migration_needed';")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("INSERT INTO metadata (key, value) VALUES ('rag_plugin_migration_needed', 'true');")
)
self.ap.logger.info('Set rag_plugin_migration_needed=true in metadata.')
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,289 @@
"""Message Aggregator Module
This module provides message aggregation/debounce functionality.
When users send multiple messages consecutively, the aggregator will wait
for a configurable delay period and merge them into a single message
before processing.
"""
from __future__ import annotations
import asyncio
import time
import typing
from dataclasses import dataclass, field
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
if typing.TYPE_CHECKING:
from ..core import app
# Maximum number of messages to buffer before forcing a flush
MAX_BUFFER_MESSAGES = 10
@dataclass
class PendingMessage:
"""A pending message waiting to be aggregated"""
bot_uuid: str
launcher_type: provider_session.LauncherTypes
launcher_id: typing.Union[int, str]
sender_id: typing.Union[int, str]
message_event: platform_events.MessageEvent
message_chain: platform_message.MessageChain
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter
pipeline_uuid: typing.Optional[str]
timestamp: float = field(default_factory=time.time)
@dataclass
class SessionBuffer:
"""Buffer for a single session's pending messages"""
session_id: str
messages: list[PendingMessage] = field(default_factory=list)
timer_task: typing.Optional[asyncio.Task] = None
last_message_time: float = field(default_factory=time.time)
class MessageAggregator:
"""Message aggregator that buffers and merges consecutive messages
This class implements a debounce mechanism for incoming messages.
When a message arrives, it starts a timer. If more messages arrive
before the timer expires, they are buffered. When the timer expires,
all buffered messages are merged and sent to the query pool.
"""
ap: app.Application
buffers: dict[str, SessionBuffer]
"""Session ID -> SessionBuffer mapping"""
lock: asyncio.Lock
"""Lock for thread-safe buffer operations"""
def __init__(self, ap: app.Application):
self.ap = ap
self.buffers = {}
self.lock = asyncio.Lock()
def _get_session_id(
self,
bot_uuid: str,
launcher_type: provider_session.LauncherTypes,
launcher_id: typing.Union[int, str],
) -> str:
"""Generate a unique session ID"""
return f'{bot_uuid}:{launcher_type.value}:{launcher_id}'
async def _get_aggregation_config(self, pipeline_uuid: typing.Optional[str]) -> tuple[bool, float]:
"""Get aggregation configuration for a pipeline
Returns:
tuple: (enabled, delay_seconds)
"""
default_enabled = False
default_delay = 1.5
if pipeline_uuid is None:
return default_enabled, default_delay
# Get pipeline from pipeline manager
pipeline = await self.ap.pipeline_mgr.get_pipeline_by_uuid(pipeline_uuid)
if pipeline is None:
return default_enabled, default_delay
config = pipeline.pipeline_entity.config or {}
trigger_config = config.get('trigger', {})
aggregation_config = trigger_config.get('message-aggregation', {})
enabled = aggregation_config.get('enabled', default_enabled)
delay_raw = aggregation_config.get('delay', default_delay)
try:
delay = float(delay_raw)
except (TypeError, ValueError):
delay = default_delay
# Clamp delay to valid range
delay = max(1.0, min(10.0, delay))
return enabled, delay
async def add_message(
self,
bot_uuid: str,
launcher_type: provider_session.LauncherTypes,
launcher_id: typing.Union[int, str],
sender_id: typing.Union[int, str],
message_event: platform_events.MessageEvent,
message_chain: platform_message.MessageChain,
adapter: abstract_platform_adapter.AbstractMessagePlatformAdapter,
pipeline_uuid: typing.Optional[str] = None,
) -> None:
"""Add a message to the aggregation buffer
If aggregation is disabled for the pipeline, the message is sent
directly to the query pool. Otherwise, it's buffered and will be
merged with other messages from the same session.
"""
enabled, delay = await self._get_aggregation_config(pipeline_uuid)
if not enabled:
# Aggregation disabled, send directly to query pool
await self.ap.query_pool.add_query(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
message_event=message_event,
message_chain=message_chain,
adapter=adapter,
pipeline_uuid=pipeline_uuid,
)
return
session_id = self._get_session_id(bot_uuid, launcher_type, launcher_id)
pending_msg = PendingMessage(
bot_uuid=bot_uuid,
launcher_type=launcher_type,
launcher_id=launcher_id,
sender_id=sender_id,
message_event=message_event,
message_chain=message_chain,
adapter=adapter,
pipeline_uuid=pipeline_uuid,
)
force_flush = False
async with self.lock:
if session_id in self.buffers:
buffer = self.buffers[session_id]
# Cancel existing timer (just cancel, don't await inside lock)
if buffer.timer_task and not buffer.timer_task.done():
buffer.timer_task.cancel()
buffer.messages.append(pending_msg)
else:
buffer = SessionBuffer(
session_id=session_id,
messages=[pending_msg],
)
self.buffers[session_id] = buffer
buffer.last_message_time = time.time()
# Check if buffer reached max capacity
if len(buffer.messages) >= MAX_BUFFER_MESSAGES:
force_flush = True
else:
# Start new timer
buffer.timer_task = asyncio.create_task(self._delayed_flush(session_id, delay))
if force_flush:
await self._flush_buffer(session_id)
async def _delayed_flush(self, session_id: str, delay: float) -> None:
"""Wait for delay then flush the buffer"""
try:
await asyncio.sleep(delay)
await self._flush_buffer(session_id)
except asyncio.CancelledError:
# Timer was cancelled, new message arrived
pass
async def _flush_buffer(self, session_id: str) -> None:
"""Flush the buffer for a session, merging all messages"""
async with self.lock:
buffer = self.buffers.pop(session_id, None)
if buffer is None or not buffer.messages:
return
if len(buffer.messages) == 1:
# Only one message, no need to merge
msg = buffer.messages[0]
await self.ap.query_pool.add_query(
bot_uuid=msg.bot_uuid,
launcher_type=msg.launcher_type,
launcher_id=msg.launcher_id,
sender_id=msg.sender_id,
message_event=msg.message_event,
message_chain=msg.message_chain,
adapter=msg.adapter,
pipeline_uuid=msg.pipeline_uuid,
)
return
# Merge multiple messages
merged_msg = self._merge_messages(buffer.messages)
await self.ap.query_pool.add_query(
bot_uuid=merged_msg.bot_uuid,
launcher_type=merged_msg.launcher_type,
launcher_id=merged_msg.launcher_id,
sender_id=merged_msg.sender_id,
message_event=merged_msg.message_event,
message_chain=merged_msg.message_chain,
adapter=merged_msg.adapter,
pipeline_uuid=merged_msg.pipeline_uuid,
)
def _merge_messages(self, messages: list[PendingMessage]) -> PendingMessage:
"""Merge multiple messages into one
The merged message uses the first message as base and combines
all message chains with newline separators.
The original message_event is kept unmodified to preserve
message metadata (message_id, etc.) for reply/quote.
"""
if len(messages) == 1:
return messages[0]
base_msg = messages[0]
# Build merged message chain
merged_chain = platform_message.MessageChain([])
for i, msg in enumerate(messages):
if i > 0:
# Add newline separator between messages
merged_chain.append(platform_message.Plain(text='\n'))
# Copy all components from this message
for component in msg.message_chain:
merged_chain.append(component)
# Keep message_event unmodified (preserves original message_id and
# metadata for reply/quote), only pass merged chain separately
return PendingMessage(
bot_uuid=base_msg.bot_uuid,
launcher_type=base_msg.launcher_type,
launcher_id=base_msg.launcher_id,
sender_id=base_msg.sender_id,
message_event=base_msg.message_event,
message_chain=merged_chain,
adapter=base_msg.adapter,
pipeline_uuid=base_msg.pipeline_uuid,
)
async def flush_all(self) -> None:
"""Flush all pending buffers immediately
This is useful during shutdown to ensure no messages are lost.
"""
# Snapshot session IDs and cancel all timers under lock
async with self.lock:
session_ids = list(self.buffers.keys())
for sid in session_ids:
buffer = self.buffers.get(sid)
if buffer and buffer.timer_task and not buffer.timer_task.done():
buffer.timer_task.cancel()
# Flush each buffer outside the lock
for session_id in session_ids:
await self._flush_buffer(session_id)

View File

@@ -1,10 +1,9 @@
from __future__ import annotations
import aiohttp
from .. import entities
from .. import filter as filter_model
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.pkg.utils import httpclient
BAIDU_EXAMINE_URL = 'https://aip.baidubce.com/rest/2.0/solution/v1/text_censor/v2/user_defined?access_token={}'
BAIDU_EXAMINE_TOKEN_URL = 'https://aip.baidubce.com/oauth/2.0/token'
@@ -15,50 +14,50 @@ class BaiduCloudExamine(filter_model.ContentFilter):
"""百度云内容审核"""
async def _get_token(self) -> str:
async with aiohttp.ClientSession() as session:
async with session.post(
BAIDU_EXAMINE_TOKEN_URL,
params={
'grant_type': 'client_credentials',
'client_id': self.ap.pipeline_cfg.data['baidu-cloud-examine']['api-key'],
'client_secret': self.ap.pipeline_cfg.data['baidu-cloud-examine']['api-secret'],
},
) as resp:
return (await resp.json())['access_token']
session = httpclient.get_session()
async with session.post(
BAIDU_EXAMINE_TOKEN_URL,
params={
'grant_type': 'client_credentials',
'client_id': self.ap.pipeline_cfg.data['baidu-cloud-examine']['api-key'],
'client_secret': self.ap.pipeline_cfg.data['baidu-cloud-examine']['api-secret'],
},
) as resp:
return (await resp.json())['access_token']
async def process(self, query: pipeline_query.Query, message: str) -> entities.FilterResult:
async with aiohttp.ClientSession() as session:
async with session.post(
BAIDU_EXAMINE_URL.format(await self._get_token()),
headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Accept': 'application/json',
},
data=f'text={message}'.encode('utf-8'),
) as resp:
result = await resp.json()
session = httpclient.get_session()
async with session.post(
BAIDU_EXAMINE_URL.format(await self._get_token()),
headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Accept': 'application/json',
},
data=f'text={message}'.encode('utf-8'),
) as resp:
result = await resp.json()
if 'error_code' in result:
if 'error_code' in result:
return entities.FilterResult(
level=entities.ResultLevel.BLOCK,
replacement=message,
user_notice='',
console_notice=f'百度云判定出错,错误信息:{result["error_msg"]}',
)
else:
conclusion = result['conclusion']
if conclusion in ('合规'):
return entities.FilterResult(
level=entities.ResultLevel.PASS,
replacement=message,
user_notice='',
console_notice=f'百度云判定结果:{conclusion}',
)
else:
return entities.FilterResult(
level=entities.ResultLevel.BLOCK,
replacement=message,
user_notice='',
console_notice=f'百度云判定出错,错误信息:{result["error_msg"]}',
user_notice='消息中存在不合适的内容, 请修改',
console_notice=f'百度云判定结果:{conclusion}',
)
else:
conclusion = result['conclusion']
if conclusion in ('合规'):
return entities.FilterResult(
level=entities.ResultLevel.PASS,
replacement=message,
user_notice='',
console_notice=f'百度云判定结果:{conclusion}',
)
else:
return entities.FilterResult(
level=entities.ResultLevel.BLOCK,
replacement=message,
user_notice='消息中存在不合适的内容, 请修改',
console_notice=f'百度云判定结果:{conclusion}',
)

View File

@@ -0,0 +1,105 @@
from __future__ import annotations
import logging
logger = logging.getLogger(__name__)
# metadata type -> coercion function
_COERCE_MAP = {
'integer': lambda v: int(v),
'number': lambda v: float(v),
'float': lambda v: float(v),
}
def _coerce_bool(v):
if isinstance(v, bool):
return v
if isinstance(v, str):
if v.lower() == 'true':
return True
if v.lower() == 'false':
return False
raise ValueError(f'Cannot convert string {v!r} to bool')
return bool(v)
def _coerce_value(value, expected_type: str):
"""Convert a single value to the expected type.
Returns the converted value, or the original value if no conversion needed.
"""
if value is None:
return value
if expected_type == 'boolean':
if isinstance(value, bool):
return value
return _coerce_bool(value)
coerce_fn = _COERCE_MAP.get(expected_type)
if coerce_fn is None:
return value
# Already the correct type
if expected_type == 'integer' and isinstance(value, int) and not isinstance(value, bool):
return value
if expected_type in ('number', 'float') and isinstance(value, (int, float)) and not isinstance(value, bool):
return float(value)
return coerce_fn(value)
def coerce_pipeline_config(
config: dict,
*metadata_list: dict,
) -> None:
"""Coerce pipeline config values according to metadata type definitions.
Walks each metadata dict (trigger, safety, ai, output) and converts
config values in-place so that strings coming from the JSON column are
cast to their declared types (integer, number/float, boolean).
Args:
config: The pipeline config dict to modify in-place.
*metadata_list: Metadata dicts loaded from the YAML templates.
"""
for meta in metadata_list:
section_name = meta.get('name')
if not section_name or section_name not in config:
continue
section = config[section_name]
if not isinstance(section, dict):
continue
for stage_def in meta.get('stages', []):
stage_name = stage_def.get('name')
if not stage_name or stage_name not in section:
continue
stage_config = section[stage_name]
if not isinstance(stage_config, dict):
continue
for field_def in stage_def.get('config', []):
field_name = field_def.get('name')
field_type = field_def.get('type')
if not field_name or not field_type or field_name not in stage_config:
continue
old_value = stage_config[field_name]
try:
new_value = _coerce_value(old_value, field_type)
if new_value is not old_value:
stage_config[field_name] = new_value
except (ValueError, TypeError) as e:
logger.warning(
'Failed to coerce config %s.%s.%s (%r) to %s: %s',
section_name,
stage_name,
field_name,
old_value,
field_type,
e,
)

View File

@@ -114,6 +114,60 @@ class MonitoringHelper:
except Exception as e:
ap.logger.error(f'Failed to record query success: {e}')
@staticmethod
async def record_query_response(
ap: app.Application,
query: pipeline_query.Query,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
runner_name: str | None = None,
):
"""Record bot response message to monitoring"""
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Extract response content from resp_message_chain
if hasattr(query, 'resp_message_chain') and query.resp_message_chain:
# Serialize the last response message chain
last_resp = query.resp_message_chain[-1]
if hasattr(last_resp, 'model_dump'):
message_content = json.dumps(last_resp.model_dump(), ensure_ascii=False)
else:
message_content = str(last_resp)
elif hasattr(query, 'resp_messages') and query.resp_messages:
last_resp = query.resp_messages[-1]
if hasattr(last_resp, 'get_content_platform_message_chain'):
chain = last_resp.get_content_platform_message_chain()
if hasattr(chain, 'model_dump'):
message_content = json.dumps(chain.model_dump(), ensure_ascii=False)
else:
message_content = str(chain)
else:
message_content = str(last_resp)
else:
return # No response to record
await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
message_content=message_content,
session_id=session_id,
status='success',
level='info',
platform=query.launcher_type.value
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
runner_name=runner_name,
role='assistant',
)
except Exception as e:
ap.logger.error(f'Failed to record query response: {e}')
@staticmethod
async def record_query_error(
ap: app.Application,

View File

@@ -13,6 +13,7 @@ import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events
import langbot_plugin.api.entities.events as events
from ..utils import importutil
from .config_coercion import coerce_pipeline_config
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -339,6 +340,20 @@ class RuntimePipeline:
except Exception as e:
self.ap.logger.error(f'Failed to record query success: {e}')
# Record bot response message
try:
await monitoring_helper.MonitoringHelper.record_query_response(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
bot_name=bot_name,
pipeline_id=self.pipeline_entity.uuid,
pipeline_name=pipeline_name,
runner_name=runner_name,
)
except Exception as e:
self.ap.logger.error(f'Failed to record query response: {e}')
except Exception as e:
inst_name = query.current_stage_name if query.current_stage_name else 'unknown'
self.ap.logger.error(f'Error processing query {query.query_id} stage={inst_name} : {e}')
@@ -369,8 +384,6 @@ class RuntimePipeline:
class PipelineManager:
"""流水线管理器"""
# ====== 4.0 ======
ap: app.Application
pipelines: list[RuntimePipeline]
@@ -408,6 +421,14 @@ class PipelineManager:
elif isinstance(pipeline_entity, dict):
pipeline_entity = persistence_pipeline.LegacyPipeline(**pipeline_entity)
coerce_pipeline_config(
pipeline_entity.config,
getattr(self.ap, 'pipeline_config_meta_trigger', {'name': 'trigger', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_safety', {'name': 'safety', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_ai', {'name': 'ai', 'stages': []}),
getattr(self.ap, 'pipeline_config_meta_output', {'name': 'output', 'stages': []}),
)
# initialize stage containers according to pipeline_entity.stages
stage_containers: list[StageInstContainer] = []
for stage_name in pipeline_entity.stages:

View File

@@ -12,7 +12,7 @@ from ... import entities
from ....provider import runner as runner_module
import langbot_plugin.api.entities.events as events
from ....utils import importutil, constants
from ....utils import importutil, constants, runner as runner_utils
from ....provider import runners
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -185,10 +185,15 @@ class ChatMessageHandler(handler.MessageHandler):
pipeline_plugins = query.variables.get('_pipeline_bound_plugins', None)
runner_category = runner_utils.get_runner_category_from_runner(
runner_name, runner, query.pipeline_config
)
payload = {
'query_id': query.query_id,
'adapter': adapter_name,
'runner': runner_name,
'runner_category': runner_category,
'duration_ms': duration_ms,
'model_name': model_name,
'version': constants.semantic_version,
@@ -200,6 +205,11 @@ class ChatMessageHandler(handler.MessageHandler):
# Send telemetry asynchronously and do not block pipeline via app's telemetry manager
await self.ap.telemetry.start_send_task(payload)
# Trigger survey event on first successful non-WebSocket response
if not locals().get('error_info') and adapter_name and 'WebSocket' not in adapter_name:
if self.ap.survey:
await self.ap.survey.trigger_event('first_bot_response_success')
except Exception as ex:
# Ensure telemetry issues do not affect normal flow
self.ap.logger.warning(f'Failed to send telemetry: {ex}')

View File

@@ -82,7 +82,7 @@ class RuntimeBot:
if custom_launcher_id:
launcher_id = custom_launcher_id
await self.ap.query_pool.add_query(
await self.ap.msg_aggregator.add_message(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.PERSON,
launcher_id=launcher_id,
@@ -125,7 +125,7 @@ class RuntimeBot:
if custom_launcher_id:
launcher_id = custom_launcher_id
await self.ap.query_pool.add_query(
await self.ap.msg_aggregator.add_message(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.GROUP,
launcher_id=launcher_id,
@@ -282,6 +282,8 @@ class PlatformManager:
return runtime_bot
async def get_bot_by_uuid(self, bot_uuid: str) -> RuntimeBot | None:
if self.websocket_proxy_bot and self.websocket_proxy_bot.bot_entity.uuid == bot_uuid:
return self.websocket_proxy_bot
for bot in self.bots:
if bot.bot_entity.uuid == bot_uuid:
return bot

View File

@@ -375,6 +375,18 @@ class AiocqhttpAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
self.bot = aiocqhttp.CQHttp()
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
# Check if message contains a Forward component
forward_msg = message.get_first(platform_message.Forward)
if forward_msg:
if target_type == 'group':
# Send as merged forward message via OneBot API
await self._send_forward_message(int(target_id), forward_msg)
return
else:
await self.logger.warning(
f'Forward message is only supported for group targets, got target_type={target_type}. Falling through to normal send.'
)
aiocq_msg = (await AiocqhttpMessageConverter.yiri2target(message))[0]
if target_type == 'group':
@@ -382,6 +394,90 @@ class AiocqhttpAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
elif target_type == 'person':
await self.bot.send_private_msg(user_id=int(target_id), message=aiocq_msg)
async def _send_forward_message(self, group_id: int, forward: platform_message.Forward):
"""Send a merged forward message to a group using NapCat extended API."""
messages = []
for node in forward.node_list:
# Build content for each node
content = []
if node.message_chain:
for component in node.message_chain:
if isinstance(component, platform_message.Plain):
if component.text:
content.append({'type': 'text', 'data': {'text': component.text}})
elif isinstance(component, platform_message.Image):
img_data = {}
if component.base64:
b64 = component.base64
if b64.startswith('data:'):
b64 = b64.split(',', 1)[-1] if ',' in b64 else b64
img_data['file'] = f'base64://{b64}'
elif component.url:
img_data['file'] = component.url
elif component.path:
img_data['file'] = str(component.path)
if img_data:
content.append({'type': 'image', 'data': img_data})
if not content:
continue
# Build node data - use user_id and nickname format for NapCat
user_id = str(node.sender_id) if node.sender_id else str(self.bot_account_id or '10000')
node_data = {
'type': 'node',
'data': {
'user_id': user_id,
'nickname': node.sender_name or '未知',
'content': content,
},
}
messages.append(node_data)
if not messages:
return
# Build the full message payload for NapCat's send_forward_msg API
# This matches the format used by GiveMeSetuPlugin
bot_id = str(self.bot_account_id) if self.bot_account_id else '10000'
payload = {
'group_id': group_id,
'user_id': bot_id, # Required by NapCat for display
'messages': messages,
}
# Add display settings if available
if forward.display:
if forward.display.title:
payload['news'] = [{'text': forward.display.title}]
if forward.display.brief:
payload['prompt'] = forward.display.brief
if forward.display.summary:
payload['summary'] = forward.display.summary
if forward.display.source:
payload['source'] = forward.display.source
try:
# Use send_forward_msg (NapCat extended API) instead of send_group_forward_msg
await self.logger.info(
f'Sending forward message to group {group_id} with {len(messages)} nodes, payload keys: {list(payload.keys())}'
)
result = await self.bot.call_action('send_forward_msg', **payload)
await self.logger.info(f'Forward message sent to group {group_id}, result: {result}')
except Exception as e:
await self.logger.error(f'Failed to send forward message to group {group_id}: {e}')
# Fallback: try standard OneBot API with integer group_id
try:
await self.logger.info('Trying fallback API send_group_forward_msg')
await self.bot.call_action('send_group_forward_msg', group_id=group_id, messages=messages)
await self.logger.info(f'Forward message sent via fallback API to group {group_id}')
except Exception as e2:
await self.logger.error(f'Fallback also failed: {e2}')
raise
async def reply_message(
self,
message_source: platform_events.MessageEvent,

View File

@@ -14,7 +14,7 @@ import io
import asyncio
from enum import Enum
import aiohttp
from langbot.pkg.utils import httpclient
import pydantic
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
@@ -622,23 +622,23 @@ class DiscordMessageConverter(abstract_platform_adapter.AbstractMessageConverter
image_bytes = base64.b64decode(base64_data)
elif ele.url:
# 从URL下载图片
async with aiohttp.ClientSession() as session:
async with session.get(ele.url) as response:
image_bytes = await response.read()
# 从URL或Content-Type推断文件类型
content_type = response.headers.get('Content-Type', '')
if 'jpeg' in content_type or 'jpg' in content_type:
filename = f'{uuid.uuid4()}.jpg'
elif 'gif' in content_type:
filename = f'{uuid.uuid4()}.gif'
elif 'webp' in content_type:
filename = f'{uuid.uuid4()}.webp'
elif ele.url.lower().endswith(('.jpg', '.jpeg')):
filename = f'{uuid.uuid4()}.jpg'
elif ele.url.lower().endswith('.gif'):
filename = f'{uuid.uuid4()}.gif'
elif ele.url.lower().endswith('.webp'):
filename = f'{uuid.uuid4()}.webp'
session = httpclient.get_session()
async with session.get(ele.url) as response:
image_bytes = await response.read()
# 从URL或Content-Type推断文件类型
content_type = response.headers.get('Content-Type', '')
if 'jpeg' in content_type or 'jpg' in content_type:
filename = f'{uuid.uuid4()}.jpg'
elif 'gif' in content_type:
filename = f'{uuid.uuid4()}.gif'
elif 'webp' in content_type:
filename = f'{uuid.uuid4()}.webp'
elif ele.url.lower().endswith(('.jpg', '.jpeg')):
filename = f'{uuid.uuid4()}.jpg'
elif ele.url.lower().endswith('.gif'):
filename = f'{uuid.uuid4()}.gif'
elif ele.url.lower().endswith('.webp'):
filename = f'{uuid.uuid4()}.webp'
elif ele.path:
# 从文件路径读取图片
# 确保路径没有空字节
@@ -702,9 +702,9 @@ class DiscordMessageConverter(abstract_platform_adapter.AbstractMessageConverter
file_base64 = ele.base64.split(',')[-1]
file_bytes = base64.b64decode(file_base64)
elif ele.url:
async with aiohttp.ClientSession() as session:
async with session.get(ele.url) as response:
file_bytes = await response.read()
session = httpclient.get_session()
async with session.get(ele.url) as response:
file_bytes = await response.read()
if file_bytes:
files.append(discord.File(fp=io.BytesIO(file_bytes), filename=filename))
elif isinstance(ele, platform_message.File):
@@ -717,9 +717,9 @@ class DiscordMessageConverter(abstract_platform_adapter.AbstractMessageConverter
else:
file_bytes = base64.b64decode(ele.base64)
elif ele.url:
async with aiohttp.ClientSession() as session:
async with session.get(ele.url) as response:
file_bytes = await response.read()
session = httpclient.get_session()
async with session.get(ele.url) as response:
file_bytes = await response.read()
if file_bytes:
files.append(discord.File(fp=io.BytesIO(file_bytes), filename=filename))
elif isinstance(ele, platform_message.Forward):
@@ -775,12 +775,12 @@ class DiscordMessageConverter(abstract_platform_adapter.AbstractMessageConverter
# attachments
for attachment in message.attachments:
async with aiohttp.ClientSession(trust_env=True) as session:
async with session.get(attachment.url) as response:
image_data = await response.read()
image_base64 = base64.b64encode(image_data).decode('utf-8')
image_format = response.headers['Content-Type']
element_list.append(platform_message.Image(base64=f'data:{image_format};base64,{image_base64}'))
session = httpclient.get_session(trust_env=True)
async with session.get(attachment.url) as response:
image_data = await response.read()
image_base64 = base64.b64encode(image_data).decode('utf-8')
image_format = response.headers['Content-Type']
element_list.append(platform_message.Image(base64=f'data:{image_format};base64,{image_base64}'))
return platform_message.MessageChain(element_list)

View File

@@ -9,6 +9,8 @@ import traceback
import time
import aiohttp
from langbot.pkg.utils import httpclient
import websockets
import pydantic
@@ -120,16 +122,16 @@ class KookMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
if content:
# Download image and convert to base64
try:
async with aiohttp.ClientSession() as session:
async with session.get(content) as response:
if response.status == 200:
image_bytes = await response.read()
image_base64 = base64.b64encode(image_bytes).decode('utf-8')
# Detect image format
content_type = response.headers.get('Content-Type', 'image/png')
components.append(
platform_message.Image(base64=f'data:{content_type};base64,{image_base64}')
)
session = httpclient.get_session()
async with session.get(content) as response:
if response.status == 200:
image_bytes = await response.read()
image_base64 = base64.b64encode(image_bytes).decode('utf-8')
# Detect image format
content_type = response.headers.get('Content-Type', 'image/png')
components.append(
platform_message.Image(base64=f'data:{content_type};base64,{image_base64}')
)
except Exception:
# If download fails, just add as plain text
components.append(platform_message.Plain(text=f'[Image: {content}]'))
@@ -295,17 +297,17 @@ class KookAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
'Authorization': f'Bot {self.config["token"]}',
}
async with aiohttp.ClientSession() as session:
async with session.get(base_url, params=params, headers=headers) as response:
if response.status == 200:
data = await response.json()
if data.get('code') == 0:
gateway_url = data['data']['url']
return gateway_url
else:
raise Exception(f'Failed to get gateway URL: {data.get("message")}')
session = httpclient.get_session()
async with session.get(base_url, params=params, headers=headers) as response:
if response.status == 200:
data = await response.json()
if data.get('code') == 0:
gateway_url = data['data']['url']
return gateway_url
else:
raise Exception(f'Failed to get gateway URL: HTTP {response.status}')
raise Exception(f'Failed to get gateway URL: {data.get("message")}')
else:
raise Exception(f'Failed to get gateway URL: HTTP {response.status}')
async def _get_bot_user_info(self) -> dict:
"""Get bot's own user information from KOOK API"""
@@ -315,17 +317,17 @@ class KookAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
'Authorization': f'Bot {self.config["token"]}',
}
async with aiohttp.ClientSession() as session:
async with session.get(base_url, headers=headers) as response:
if response.status == 200:
data = await response.json()
if data.get('code') == 0:
user_info = data['data']
return user_info
else:
raise Exception(f'Failed to get bot user info: {data.get("message")}')
session = httpclient.get_session()
async with session.get(base_url, headers=headers) as response:
if response.status == 200:
data = await response.json()
if data.get('code') == 0:
user_info = data['data']
return user_info
else:
raise Exception(f'Failed to get bot user info: HTTP {response.status}')
raise Exception(f'Failed to get bot user info: {data.get("message")}')
else:
raise Exception(f'Failed to get bot user info: HTTP {response.status}')
async def _handle_hello(self, data: dict):
"""Handle HELLO signal (signal 1)"""
@@ -510,7 +512,7 @@ class KookAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
try:
if not self.http_session:
self.http_session = aiohttp.ClientSession()
self.http_session = httpclient.get_session()
async with self.http_session.post(url, json=payload, headers=headers) as response:
if response.status == 200:
@@ -576,7 +578,7 @@ class KookAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
try:
if not self.http_session:
self.http_session = aiohttp.ClientSession()
self.http_session = httpclient.get_session()
async with self.http_session.post(url, json=payload, headers=headers) as response:
if response.status == 200:
@@ -624,7 +626,7 @@ class KookAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
try:
# Create HTTP session
self.http_session = aiohttp.ClientSession()
self.http_session = httpclient.get_session()
await self.logger.info('Starting KOOK adapter')

View File

@@ -1,7 +1,7 @@
from __future__ import annotations
import lark_oapi
from lark_oapi.api.im.v1 import CreateImageRequest, CreateImageRequestBody
from lark_oapi.api.im.v1 import CreateImageRequest, CreateImageRequestBody, CreateFileRequest, CreateFileRequestBody
import traceback
import typing
import asyncio
@@ -17,7 +17,7 @@ import tempfile
import os
import mimetypes
import aiohttp
from langbot.pkg.utils import httpclient
import lark_oapi.ws.exception
import quart
from lark_oapi.api.im.v1 import *
@@ -78,13 +78,13 @@ class LarkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
return None
elif msg.url:
try:
async with aiohttp.ClientSession() as session:
async with session.get(msg.url) as response:
if response.status == 200:
image_bytes = await response.read()
else:
print(f'Failed to download image from {msg.url}: HTTP {response.status}')
return None
session = httpclient.get_session()
async with session.get(msg.url) as response:
if response.status == 200:
image_bytes = await response.read()
else:
print(f'Failed to download image from {msg.url}: HTTP {response.status}')
return None
except Exception as e:
print(f'Failed to download image from {msg.url}: {e}')
traceback.print_exc()
@@ -141,6 +141,88 @@ class LarkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
traceback.print_exc()
return None
@staticmethod
async def upload_file_to_lark(
file_bytes: bytes,
api_client: lark_oapi.Client,
file_type: str,
file_name: str = 'file',
duration: typing.Optional[int] = None,
) -> typing.Optional[str]:
"""Upload a file to Lark and return the file_key, or None if upload fails.
Args:
file_bytes: Raw file bytes.
api_client: Lark API client.
file_type: Lark file type, e.g. 'opus', 'mp4', 'pdf', 'doc', etc.
file_name: Display name for the file.
duration: Duration in milliseconds (for audio files).
"""
try:
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
temp_file.write(file_bytes)
temp_file_path = temp_file.name
try:
body_builder = (
CreateFileRequestBody.builder()
.file_type(file_type)
.file_name(file_name)
.file(open(temp_file_path, 'rb'))
)
if duration is not None:
body_builder = body_builder.duration(duration)
request = CreateFileRequest.builder().request_body(body_builder.build()).build()
response = await api_client.im.v1.file.acreate(request)
if not response.success():
print(
f'client.im.v1.file.create failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}'
)
return None
return response.data.file_key
finally:
os.unlink(temp_file_path)
except Exception as e:
print(f'Failed to upload file to Lark: {e}')
traceback.print_exc()
return None
@staticmethod
async def _get_media_bytes(
msg: typing.Union[platform_message.Voice, platform_message.File],
) -> typing.Optional[bytes]:
"""Get bytes from a Voice or File message (base64, url, or path)."""
data = None
if msg.base64:
try:
base64_str = msg.base64
if ',' in base64_str:
base64_str = base64_str.split(',', 1)[1]
data = base64.b64decode(base64_str)
except Exception:
pass
elif msg.url:
try:
session = httpclient.get_session()
async with session.get(msg.url) as resp:
if resp.status == 200:
data = await resp.read()
except Exception:
pass
elif msg.path:
try:
with open(msg.path, 'rb') as f:
data = f.read()
except Exception:
pass
return data
@staticmethod
async def yiri2target(
message_chain: platform_message.MessageChain, api_client: lark_oapi.Client
@@ -150,10 +232,10 @@ class LarkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
Returns:
Tuple of (text_elements, image_keys):
- text_elements: List of paragraphs for post message format
- image_keys: List of image_key strings for separate image messages
- media_items: List of dicts with 'msg_type' and 'content' for separate media messages
"""
message_elements = []
image_keys = []
media_items = []
pending_paragraph = []
# Regex pattern to match Markdown image syntax: ![alt](url)
@@ -196,40 +278,77 @@ class LarkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
# Check for and extract Markdown images from text
cleaned_text, extracted_urls = await process_text_with_images(text)
# Add cleaned text if not empty
# Split by blank lines to create separate paragraphs for Lark post format.
# Lark truncates md elements at the first \n\n, so we must use the
# post format's native paragraph structure instead.
if cleaned_text:
pending_paragraph.append({'tag': 'md', 'text': cleaned_text})
segments = re.split(r'\n\s*\n', cleaned_text)
for i, segment in enumerate(segments):
segment = segment.strip()
if not segment:
continue
if i > 0 and pending_paragraph:
message_elements.append(pending_paragraph)
pending_paragraph = []
pending_paragraph.append({'tag': 'md', 'text': segment})
# Process extracted image URLs
for url in extracted_urls:
# Create a temporary Image message to upload
temp_image = platform_message.Image(url=url)
image_key = await LarkMessageConverter.upload_image_to_lark(temp_image, api_client)
if image_key:
image_keys.append(image_key)
media_items.append({'msg_type': 'image', 'content': {'image_key': image_key}})
elif isinstance(msg, platform_message.At):
pending_paragraph.append({'tag': 'at', 'user_id': msg.target, 'style': []})
elif isinstance(msg, platform_message.AtAll):
pending_paragraph.append({'tag': 'at', 'user_id': 'all', 'style': []})
elif isinstance(msg, platform_message.Image):
# Upload image and get image_key
image_key = await LarkMessageConverter.upload_image_to_lark(msg, api_client)
if image_key:
# Store image_key for separate image message
image_keys.append(image_key)
media_items.append({'msg_type': 'image', 'content': {'image_key': image_key}})
elif isinstance(msg, platform_message.Voice):
data = await LarkMessageConverter._get_media_bytes(msg)
if data:
duration = int(msg.length * 1000) if msg.length else None
file_key = await LarkMessageConverter.upload_file_to_lark(
data, api_client, file_type='opus', file_name='voice.opus', duration=duration
)
if file_key:
media_items.append({'msg_type': 'audio', 'content': {'file_key': file_key}})
elif isinstance(msg, platform_message.File):
data = await LarkMessageConverter._get_media_bytes(msg)
if data:
file_name = msg.name or 'file'
# Guess file_type from extension
ext = os.path.splitext(file_name)[1].lstrip('.').lower() if file_name else ''
file_type_map = {
'opus': 'opus',
'mp4': 'mp4',
'pdf': 'pdf',
'doc': 'doc',
'docx': 'doc',
'xls': 'xls',
'xlsx': 'xls',
'ppt': 'ppt',
'pptx': 'ppt',
}
file_type = file_type_map.get(ext, 'stream')
file_key = await LarkMessageConverter.upload_file_to_lark(
data, api_client, file_type=file_type, file_name=file_name
)
if file_key:
media_items.append({'msg_type': 'file', 'content': {'file_key': file_key}})
elif isinstance(msg, platform_message.Forward):
for node in msg.node_list:
sub_elements, sub_image_keys = await LarkMessageConverter.yiri2target(
node.message_chain, api_client
)
sub_elements, sub_media = await LarkMessageConverter.yiri2target(node.message_chain, api_client)
message_elements.extend(sub_elements)
image_keys.extend(sub_image_keys)
media_items.extend(sub_media)
if pending_paragraph:
message_elements.append(pending_paragraph)
return message_elements, image_keys
return message_elements, media_items
@staticmethod
async def target2yiri(
@@ -917,23 +1036,40 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
):
# 不再需要了因为message_id已经被包含到message_chain中
# lark_event = await self.event_converter.yiri2target(message_source)
text_elements, image_keys = await self.message_converter.yiri2target(message, self.api_client)
text_elements, media_items = await self.message_converter.yiri2target(message, self.api_client)
# Send text message if there are text elements
if text_elements:
final_content = {
'zh_Hans': {
'title': '',
'content': text_elements,
},
}
# Determine msg_type based on content: use 'post' if at mentions
# are present (requires post paragraph structure), otherwise 'text'
needs_post = any(ele['tag'] == 'at' for paragraph in text_elements for ele in paragraph)
if needs_post:
msg_type = 'post'
final_content = json.dumps(
{
'zh_Hans': {
'title': '',
'content': text_elements,
},
}
)
else:
msg_type = 'text'
parts = []
for paragraph in text_elements:
para_text = ''.join(ele.get('text', '') for ele in paragraph)
if para_text:
parts.append(para_text)
final_content = json.dumps({'text': '\n\n'.join(parts)})
request: ReplyMessageRequest = (
ReplyMessageRequest.builder()
.message_id(message_source.message_chain.message_id)
.request_body(
ReplyMessageRequestBody.builder()
.content(json.dumps(final_content))
.msg_type('post')
.content(final_content)
.msg_type(msg_type)
.reply_in_thread(False)
.uuid(str(uuid.uuid4()))
.build()
@@ -963,17 +1099,15 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
f'client.im.v1.message.reply failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}'
)
# Send image messages separately using msg_type='image'
for image_key in image_keys:
image_content = json.dumps({'image_key': image_key})
# Send media messages separately (image, audio, file, etc.)
for media in media_items:
request: ReplyMessageRequest = (
ReplyMessageRequest.builder()
.message_id(message_source.message_chain.message_id)
.request_body(
ReplyMessageRequestBody.builder()
.content(image_content)
.msg_type('image')
.content(json.dumps(media['content']))
.msg_type(media['msg_type'])
.reply_in_thread(False)
.uuid(str(uuid.uuid4()))
.build()
@@ -1000,7 +1134,7 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
if not response.success():
raise Exception(
f'client.im.v1.message.reply (image) failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}'
f'client.im.v1.message.reply ({media["msg_type"]}) failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}'
)
async def reply_message_chunk(
@@ -1018,15 +1152,16 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
message_id = bot_message.resp_message_id
msg_seq = bot_message.msg_sequence
if msg_seq % 8 == 0 or is_final:
text_elements, image_keys = await self.message_converter.yiri2target(message, self.api_client)
text_elements, media_items = await self.message_converter.yiri2target(message, self.api_client)
text_message = ''
if text_elements:
for ele in text_elements[0]:
if ele['tag'] == 'text':
text_message += ele['text']
elif ele['tag'] == 'md':
text_message += ele['text']
parts = []
for paragraph in text_elements:
para_text = ''.join(ele['text'] for ele in paragraph if ele['tag'] in ('text', 'md'))
if para_text:
parts.append(para_text)
text_message = '\n\n'.join(parts)
# content = {
# 'type': 'card_json',
@@ -1076,6 +1211,30 @@ class LarkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
)
return
# Send media messages when streaming is done
if is_final and media_items:
for media in media_items:
media_request: ReplyMessageRequest = (
ReplyMessageRequest.builder()
.message_id(message_source.message_chain.message_id)
.request_body(
ReplyMessageRequestBody.builder()
.content(json.dumps(media['content']))
.msg_type(media['msg_type'])
.reply_in_thread(False)
.uuid(str(uuid.uuid4()))
.build()
)
.build()
)
media_response: ReplyMessageResponse = await self.api_client.im.v1.message.areply(
media_request, req_opt
)
if not media_response.success():
raise Exception(
f'client.im.v1.message.reply ({media["msg_type"]}) failed, code: {media_response.code}, msg: {media_response.msg}, log_id: {media_response.get_log_id()}'
)
async def is_muted(self, group_id: int) -> bool:
return False

View File

@@ -9,7 +9,7 @@ import copy
import threading
import quart
import aiohttp
from langbot.pkg.utils import httpclient
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
from ....core import app
@@ -639,14 +639,14 @@ class GeWeChatAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
async def run_async(self):
if not self.config['token']:
async with aiohttp.ClientSession() as session:
async with session.post(
f'{self.config["gewechat_url"]}/v2/api/tools/getTokenId',
json={'app_id': self.config['app_id']},
) as response:
if response.status != 200:
raise Exception(f'获取gewechat token失败: {await response.text()}')
self.config['token'] = (await response.json())['data']
session = httpclient.get_session()
async with session.post(
f'{self.config["gewechat_url"]}/v2/api/tools/getTokenId',
json={'app_id': self.config['app_id']},
) as response:
if response.status != 200:
raise Exception(f'获取gewechat token失败: {await response.text()}')
self.config['token'] = (await response.json())['data']
self.bot = gewechat_client.GewechatClient(f'{self.config["gewechat_url"]}/v2/api', self.config['token'])

View File

@@ -1,4 +1,5 @@
from __future__ import annotations
import time
import telegram
@@ -9,9 +10,9 @@ import telegramify_markdown
import typing
import traceback
import base64
import aiohttp
import pydantic
from langbot.pkg.utils import httpclient
import langbot_plugin.api.definition.abstract.platform.adapter as abstract_platform_adapter
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.platform.events as platform_events
@@ -33,9 +34,9 @@ class TelegramMessageConverter(abstract_platform_adapter.AbstractMessageConverte
if component.base64:
photo_bytes = base64.b64decode(component.base64)
elif component.url:
async with aiohttp.ClientSession() as session:
async with session.get(component.url) as response:
photo_bytes = await response.read()
session = httpclient.get_session()
async with session.get(component.url) as response:
photo_bytes = await response.read()
elif component.path:
with open(component.path, 'rb') as f:
photo_bytes = f.read()
@@ -74,10 +75,9 @@ class TelegramMessageConverter(abstract_platform_adapter.AbstractMessageConverte
file_bytes = None
file_format = ''
async with aiohttp.ClientSession(trust_env=True) as session:
async with session.get(file.file_path) as response:
file_bytes = await response.read()
file_format = 'image/jpeg'
async with httpclient.get_session(trust_env=True).get(file.file_path) as response:
file_bytes = await response.read()
file_format = 'image/jpeg'
message_components.append(
platform_message.Image(
@@ -94,9 +94,8 @@ class TelegramMessageConverter(abstract_platform_adapter.AbstractMessageConverte
file_bytes = None
file_format = message.voice.mime_type or 'audio/ogg'
async with aiohttp.ClientSession(trust_env=True) as session:
async with session.get(file.file_path) as response:
file_bytes = await response.read()
async with httpclient.get_session(trust_env=True).get(file.file_path) as response:
file_bytes = await response.read()
message_components.append(
platform_message.Voice(
@@ -194,7 +193,31 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
)
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
pass
components = await TelegramMessageConverter.yiri2target(message, self.bot)
chat_id_str, _, thread_id_str = str(target_id).partition('#')
chat_id: int | str = int(chat_id_str) if chat_id_str.lstrip('-').isdigit() else chat_id_str
message_thread_id = int(thread_id_str) if thread_id_str and thread_id_str.isdigit() else None
for component in components:
component_type = component.get('type')
args = {'chat_id': chat_id}
if message_thread_id is not None:
args['message_thread_id'] = message_thread_id
if component_type == 'text':
text = component.get('text', '')
if self.config['markdown_card'] is True:
text = telegramify_markdown.markdownify(content=text)
args['parse_mode'] = 'MarkdownV2'
args['text'] = text
await self.bot.send_message(**args)
elif component_type == 'photo':
photo = component.get('photo')
if photo is None:
continue
args['photo'] = telegram.InputFile(photo)
await self.bot.send_photo(**args)
async def reply_message(
self,
@@ -228,6 +251,39 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
await self.bot.send_message(**args)
def _process_markdown(self, text: str) -> str:
if self.config.get('markdown_card', False):
return telegramify_markdown.markdownify(content=text)
return text
def _build_message_args(self, chat_id: int, text: str, message_thread_id: int = None, **extra_args) -> dict:
args = {'chat_id': chat_id, 'text': self._process_markdown(text), **extra_args}
if message_thread_id:
args['message_thread_id'] = message_thread_id
if self.config.get('markdown_card', False):
args['parse_mode'] = 'MarkdownV2'
return args
async def create_message_card(self, message_id, event):
assert isinstance(event.source_platform_object, Update)
update = event.source_platform_object
chat_id = update.effective_chat.id
chat_type = update.effective_chat.type
message_thread_id = update.message.message_thread_id
if chat_type == 'private':
draft_id = int(time.time() * 1000)
self.msg_stream_id[message_id] = ('private', draft_id)
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id, draft_id=draft_id)
await self.bot.send_message_draft(**args)
else:
args = self._build_message_args(chat_id, 'Thinking...', message_thread_id)
send_msg = await self.bot.send_message(**args)
self.msg_stream_id[message_id] = ('group', send_msg.message_id)
return True
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
@@ -236,59 +292,47 @@ class TelegramAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
quote_origin: bool = False,
is_final: bool = False,
):
message_id = bot_message.resp_message_id
msg_seq = bot_message.msg_sequence
if (msg_seq - 1) % 8 == 0 or is_final:
assert isinstance(message_source.source_platform_object, Update)
components = await TelegramMessageConverter.yiri2target(message, self.bot)
args = {}
message_id = message_source.source_platform_object.message.id
assert isinstance(message_source.source_platform_object, Update)
update = message_source.source_platform_object
chat_id = update.effective_chat.id
message_thread_id = update.message.message_thread_id
component = components[0]
if message_id not in self.msg_stream_id: # 当消息回复第一次时,发送新消息
# time.sleep(0.6)
if component['type'] == 'text':
if self.config['markdown_card'] is True:
content = telegramify_markdown.markdownify(
content=component['text'],
)
else:
content = component['text']
args = {
'chat_id': message_source.source_platform_object.effective_chat.id,
'text': content,
}
if message_source.source_platform_object.message.message_thread_id:
args['message_thread_id'] = message_source.source_platform_object.message.message_thread_id
if message_id not in self.msg_stream_id:
return
if quote_origin:
args['reply_to_message_id'] = message_source.source_platform_object.message.id
chat_mode, draft_id = self.msg_stream_id[message_id]
components = await TelegramMessageConverter.yiri2target(message, self.bot)
if self.config['markdown_card'] is True:
args['parse_mode'] = 'MarkdownV2'
send_msg = await self.bot.send_message(**args)
send_msg_id = send_msg.message_id
self.msg_stream_id[message_id] = send_msg_id
else: # 存在消息的时候直接编辑消息1
if component['type'] == 'text':
if self.config['markdown_card'] is True:
content = telegramify_markdown.markdownify(
content=component['text'],
)
else:
content = component['text']
args = {
'message_id': self.msg_stream_id[message_id],
'chat_id': message_source.source_platform_object.effective_chat.id,
'text': content,
}
if self.config['markdown_card'] is True:
args['parse_mode'] = 'MarkdownV2'
await self.bot.edit_message_text(**args)
if not components or components[0]['type'] != 'text':
if is_final and bot_message.tool_calls is None:
# self.seq = 1 # 消息回复结束之后重置seq
self.msg_stream_id.pop(message_id) # 消息回复结束之后删除流式消息id
self.msg_stream_id.pop(message_id)
return
content = components[0]['text']
if chat_mode == 'private':
args = self._build_message_args(chat_id, content, message_thread_id, draft_id=draft_id)
await self.bot.send_message_draft(**args)
if is_final and bot_message.tool_calls is None:
del args['draft_id']
await self.bot.send_message(**args)
self.msg_stream_id.pop(message_id)
else:
stream_id = draft_id
if (msg_seq - 1) % 8 == 0 or is_final:
args = {
'message_id': stream_id,
'chat_id': chat_id,
'text': self._process_markdown(content),
}
if self.config.get('markdown_card', False):
args['parse_mode'] = 'MarkdownV2'
await self.bot.edit_message_text(**args)
if is_final and bot_message.tool_calls is None:
self.msg_stream_id.pop(message_id)
def get_launcher_id(self, event: platform_events.MessageEvent) -> str | None:
if not isinstance(event.source_platform_object, Update):

View File

@@ -37,16 +37,24 @@ class WebSocketSession:
id: str
message_lists: dict[str, list[WebSocketMessage]] = {}
"""消息列表 {pipeline_uuid: [messages]}"""
stream_message_indexes: dict[str, dict[str, int]] = {}
"""流式消息索引 {pipeline_uuid: {resp_message_id: message_index}}"""
def __init__(self, id: str):
self.id = id
self.message_lists = {}
self.stream_message_indexes = {}
def get_message_list(self, pipeline_uuid: str) -> list[WebSocketMessage]:
if pipeline_uuid not in self.message_lists:
self.message_lists[pipeline_uuid] = []
return self.message_lists[pipeline_uuid]
def get_stream_message_indexes(self, pipeline_uuid: str) -> dict[str, int]:
if pipeline_uuid not in self.stream_message_indexes:
self.stream_message_indexes[pipeline_uuid] = {}
return self.stream_message_indexes[pipeline_uuid]
class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
"""WebSocket适配器 - 支持双向实时通信"""
@@ -89,20 +97,46 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
target_id: str,
message: platform_message.MessageChain,
) -> dict:
"""发送消息 - 这里用于主动推送消息到前端"""
message_data = {
'type': 'bot_message',
'target_type': target_type,
'target_id': target_id,
'content': str(message),
'message_chain': [component.__dict__ for component in message],
'timestamp': datetime.now().isoformat(),
}
"""发送消息 - 这里用于主动推送消息到前端
# 推送到所有相关连接
await self.outbound_message_queue.put(message_data)
对于 WebSocket 适配器,我们需要将消息广播到正确的 pipeline 连接。
target_id 可能是 launcher_id如 websocket_xxx或 pipeline_uuid。
我们需要尝试两种方式来确保消息能够送达。
"""
# 获取当前的 pipeline_uuid
pipeline_uuid = self.ap.platform_mgr.websocket_proxy_bot.bot_entity.use_pipeline_uuid
session_type = 'group' if target_type == 'group' else 'person'
return message_data
# 选择会话
session = self.websocket_group_session if session_type == 'group' else self.websocket_person_session
# 生成唯一消息ID
msg_id = len(session.get_message_list(pipeline_uuid)) + 1
message_data = WebSocketMessage(
id=msg_id,
role='assistant',
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=datetime.now().isoformat(),
is_final=True,
)
# 保存到历史记录
session.get_message_list(pipeline_uuid).append(message_data)
# 直接广播到当前pipeline的连接
await ws_connection_manager.broadcast_to_pipeline(
pipeline_uuid,
{
'type': 'response',
'session_type': session_type,
'data': message_data.model_dump(),
},
session_type=session_type,
)
return message_data.model_dump()
async def reply_message(
self,
@@ -169,10 +203,16 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
pipeline_uuid = self.ap.platform_mgr.websocket_proxy_bot.bot_entity.use_pipeline_uuid
session_type = 'group' if isinstance(message_source, platform_events.GroupMessage) else 'person'
message_list = session.get_message_list(pipeline_uuid)
stream_message_indexes = session.get_stream_message_indexes(pipeline_uuid)
# 检查是否是新的流式消息通过bot_message对象判断
# 如果列表为空或者最后一条消息已经is_final=True则创建新消息
if not message_list or message_list[-1].is_final:
# Streaming messages in LangBot have a stable resp_message_id during the same assistant reply.
# Use it as the primary key to avoid overwriting an old card from a previous reply.
resp_message_id = str(getattr(bot_message, 'resp_message_id', '') or '')
existing_index = stream_message_indexes.get(resp_message_id) if resp_message_id else None
message_is_final = is_final and bot_message.tool_calls is None
if existing_index is None or existing_index >= len(message_list):
# 创建新消息
msg_id = len(message_list) + 1
message_data = WebSocketMessage(
@@ -181,27 +221,31 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=datetime.now().isoformat(),
is_final=is_final and bot_message.tool_calls is None,
is_final=message_is_final,
)
# 只有在is_final时才保存到历史记录
if is_final and bot_message.tool_calls is None:
message_list.append(message_data)
# 立即添加到历史记录即使is_final=False以便后续块可以更新它
message_list.append(message_data)
if resp_message_id:
stream_message_indexes[resp_message_id] = len(message_list) - 1
else:
# 更新最后一条消息
msg_id = message_list[-1].id
# 更新同一条流式消息
old_message = message_list[existing_index]
msg_id = old_message.id
message_data = WebSocketMessage(
id=msg_id,
role='assistant',
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=message_list[-1].timestamp, # 保持原始时间戳
is_final=is_final and bot_message.tool_calls is None,
timestamp=old_message.timestamp, # 保持原始时间戳
is_final=message_is_final,
)
# 如果是final更新历史记录中的最后一条
if is_final and bot_message.tool_calls is None:
message_list[-1] = message_data
# 更新历史记录中的对应消息
message_list[existing_index] = message_data
if message_is_final and resp_message_id:
stream_message_indexes.pop(resp_message_id, None)
# 直接广播到所有该pipeline的连接包含session_type信息
await ws_connection_manager.broadcast_to_pipeline(
@@ -410,6 +454,10 @@ class WebSocketAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter)
if session_type == 'person':
if pipeline_uuid in self.websocket_person_session.message_lists:
self.websocket_person_session.message_lists[pipeline_uuid] = []
if pipeline_uuid in self.websocket_person_session.stream_message_indexes:
self.websocket_person_session.stream_message_indexes[pipeline_uuid] = {}
else:
if pipeline_uuid in self.websocket_group_session.message_lists:
self.websocket_group_session.message_lists[pipeline_uuid] = []
if pipeline_uuid in self.websocket_group_session.stream_message_indexes:
self.websocket_group_session.stream_message_indexes[pipeline_uuid] = {}

View File

@@ -3,6 +3,8 @@ from __future__ import annotations
import asyncio
import logging
import aiohttp
from langbot.pkg.utils import httpclient
import uuid
from typing import TYPE_CHECKING
@@ -119,23 +121,23 @@ class WebhookPusher:
dict | None: The response JSON if successful, None otherwise
"""
try:
async with aiohttp.ClientSession() as session:
async with session.post(
url,
json=payload,
headers={'Content-Type': 'application/json'},
timeout=aiohttp.ClientTimeout(total=15),
) as response:
if response.status >= 400:
self.logger.warning(f'Webhook {url} returned status {response.status}')
session = httpclient.get_session()
async with session.post(
url,
json=payload,
headers={'Content-Type': 'application/json'},
timeout=aiohttp.ClientTimeout(total=15),
) as response:
if response.status >= 400:
self.logger.warning(f'Webhook {url} returned status {response.status}')
return None
else:
self.logger.debug(f'Successfully pushed to webhook {url}')
try:
return await response.json()
except Exception as json_error:
self.logger.debug(f'Failed to parse JSON response from webhook {url}: {json_error}')
return None
else:
self.logger.debug(f'Successfully pushed to webhook {url}')
try:
return await response.json()
except Exception as json_error:
self.logger.debug(f'Failed to parse JSON response from webhook {url}: {json_error}')
return None
except asyncio.TimeoutError:
self.logger.warning(f'Timeout pushing to webhook {url}')
return None

View File

@@ -7,7 +7,6 @@ import typing
import os
import sys
import httpx
import traceback
import sqlalchemy
from async_lru import alru_cache
from langbot_plugin.api.entities.builtin.pipeline.query import provider_session
@@ -102,12 +101,6 @@ class PluginRuntimeConnector:
self.handler_task = asyncio.create_task(self.handler.run())
_ = await self.handler.ping()
self.ap.logger.info('Connected to plugin runtime.')
# Sync polymorphic component instances after connection
try:
await self.sync_polymorphic_component_instances()
except Exception as e:
traceback.print_exc()
self.ap.logger.error(f'Failed to sync polymorphic component instances: {e}')
await self.handler_task
task: asyncio.Task | None = None
@@ -463,30 +456,18 @@ class PluginRuntimeConnector:
yield cmd_ret
# KnowledgeRetriever methods
async def list_knowledge_retrievers(self, bound_plugins: list[str] | None = None) -> list[dict[str, Any]]:
"""List all available KnowledgeRetriever components."""
if not self.is_enable_plugin:
return []
retrievers_data = await self.handler.list_knowledge_retrievers(include_plugins=bound_plugins)
return retrievers_data
async def retrieve_knowledge(
self,
plugin_author: str,
plugin_name: str,
retriever_name: str,
instance_id: str,
retrieval_context: dict[str, Any],
) -> list[dict[str, Any]]:
"""Retrieve knowledge using a KnowledgeRetriever instance."""
) -> dict[str, Any]:
"""Retrieve knowledge using a KnowledgeEngine instance."""
if not self.is_enable_plugin:
return []
return {'results': []}
return await self.handler.retrieve_knowledge(
plugin_author, plugin_name, retriever_name, instance_id, retrieval_context
)
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, retriever_name, retrieval_context)
def dispose(self):
# No need to consider the shutdown on Windows
@@ -500,41 +481,84 @@ class PluginRuntimeConnector:
self.heartbeat_task.cancel()
self.heartbeat_task = None
async def sync_polymorphic_component_instances(self) -> dict[str, Any]:
"""Sync polymorphic component instances with runtime.
@staticmethod
def _parse_plugin_id(plugin_id: str) -> tuple[str, str]:
"""Parse a plugin ID string into (author, name).
This collects all external knowledge bases from database and sends to runtime
to ensure instance integrity across restarts.
Args:
plugin_id: Plugin ID in 'author/name' format.
Returns:
Tuple of (plugin_author, plugin_name).
Raises:
ValueError: If plugin_id is not in the expected 'author/name' format.
"""
if '/' not in plugin_id:
raise ValueError(
f"Invalid plugin_id format: '{plugin_id}'. Expected 'author/name' format (e.g. 'langbot/rag-engine')."
)
return plugin_id.split('/', 1)
async def call_rag_ingest(self, plugin_id: str, context_data: dict[str, Any]) -> dict[str, Any]:
"""Call plugin to ingest document.
Args:
plugin_id: Target plugin ID (author/name).
context_data: IngestionContext data.
"""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_ingest_document(plugin_author, plugin_name, context_data)
async def call_rag_delete_document(self, plugin_id: str, document_id: str, kb_id: str) -> bool:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_delete_document(plugin_author, plugin_name, document_id, kb_id)
async def get_rag_creation_schema(self, plugin_id: str) -> dict[str, Any]:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.get_rag_creation_schema(plugin_author, plugin_name)
async def get_rag_retrieval_schema(self, plugin_id: str) -> dict[str, Any]:
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.get_rag_retrieval_schema(plugin_author, plugin_name)
async def rag_on_kb_create(self, plugin_id: str, kb_id: str, config: dict[str, Any]) -> dict[str, Any]:
"""Notify plugin about KB creation."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_on_kb_create(plugin_author, plugin_name, kb_id, config)
async def rag_on_kb_delete(self, plugin_id: str, kb_id: str) -> dict[str, Any]:
"""Notify plugin about KB deletion."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.rag_on_kb_delete(plugin_author, plugin_name, kb_id)
async def call_rag_retrieve(self, plugin_id: str, retrieval_context: dict[str, Any]) -> dict[str, Any]:
"""Call plugin to retrieve knowledge.
Args:
plugin_id: Target plugin ID (author/name).
retrieval_context: RetrievalContext data.
"""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.retrieve_knowledge(plugin_author, plugin_name, '', retrieval_context)
async def list_knowledge_engines(self) -> list[dict[str, Any]]:
"""List all available Knowledge Engines from plugins.
Returns a list of Knowledge Engines with their capabilities and configuration schemas.
"""
if not self.is_enable_plugin:
return {}
return []
# ===== external knowledge bases =====
return await self.handler.list_knowledge_engines()
external_kbs = await self.ap.external_kb_service.get_external_knowledge_bases()
async def list_parsers(self) -> list[dict[str, Any]]:
"""List all available parsers from plugins."""
if not self.is_enable_plugin:
return []
return await self.handler.list_parsers()
# Build required_instances list
required_instances = []
for kb in external_kbs:
required_instances.append(
{
'instance_id': kb['uuid'],
'plugin_author': kb['plugin_author'],
'plugin_name': kb['plugin_name'],
'component_kind': 'KnowledgeRetriever',
'component_name': kb['retriever_name'],
'config': kb['retriever_config'],
}
)
self.ap.logger.info(f'Syncing {len(required_instances)} polymorphic component instances to runtime')
# Send to runtime
sync_result = await self.handler.sync_polymorphic_component_instances(required_instances)
self.ap.logger.info(
f'Sync complete: {len(sync_result.get("success_instances", []))} succeeded, '
f'{len(sync_result.get("failed_instances", []))} failed'
)
return sync_result
async def call_parser(self, plugin_id: str, context_data: dict[str, Any], file_bytes: bytes) -> dict[str, Any]:
"""Call plugin to parse a document."""
plugin_author, plugin_name = self._parse_plugin_id(plugin_id)
return await self.handler.parse_document(plugin_author, plugin_name, context_data, file_bytes)

View File

@@ -26,6 +26,20 @@ from ..core import app
from ..utils import constants
def _make_rag_error_response(error: Exception, error_type: str, **extra_context) -> handler.ActionResponse:
"""Create a clean error response for RAG operations.
Args:
error: The caught exception.
error_type: A category string like 'EmbeddingError', 'VectorStoreError'.
**extra_context: Additional context fields for the error message.
"""
context_parts = [f'{k}={v}' for k, v in extra_context.items()]
context_str = f' [{", ".join(context_parts)}]' if context_parts else ''
message = f'[{error_type}/{type(error).__name__}]{context_str} {str(error)}'
return handler.ActionResponse.error(message=message)
class RuntimeConnectionHandler(handler.Handler):
"""Runtime connection handler"""
@@ -279,6 +293,7 @@ class RuntimeConnectionHandler(handler.Handler):
target_id = data['target_id']
message_chain = data['message_chain']
# Use custom deserializer that properly handles Forward messages
message_chain_obj = platform_message.MessageChain.model_validate(message_chain)
bot = await self.ap.platform_mgr.get_bot_by_uuid(bot_uuid)
@@ -322,7 +337,14 @@ class RuntimeConnectionHandler(handler.Handler):
)
messages_obj = [provider_message.Message.model_validate(message) for message in messages]
funcs_obj = [resource_tool.LLMTool.model_validate(func) for func in funcs]
# The func field is excluded during model_dump() in plugin side (marked as exclude=True),
# but it's a required field for LLMTool validation. We need to provide a placeholder
# function when reconstructing the LLMTool objects from serialized data.
async def _placeholder_func(**kwargs):
pass
funcs_obj = [resource_tool.LLMTool.model_validate({**func, 'func': _placeholder_func}) for func in funcs]
result = await llm_model.provider.invoke_llm(
query=None,
@@ -438,7 +460,7 @@ class RuntimeConnectionHandler(handler.Handler):
},
)
@self.action(RuntimeToLangBotAction.GET_CONFIG_FILE)
@self.action(PluginToRuntimeAction.GET_CONFIG_FILE)
async def get_config_file(data: dict[str, Any]) -> handler.ActionResponse:
"""Get a config file by file key"""
file_key = data['file_key']
@@ -457,6 +479,125 @@ class RuntimeConnectionHandler(handler.Handler):
message=f'Failed to load config file {file_key}: {e}',
)
# ================= RAG Capability Handlers =================
@self.action(PluginToRuntimeAction.INVOKE_EMBEDDING)
async def invoke_embedding(data: dict[str, Any]) -> handler.ActionResponse:
embedding_model_uuid = data['embedding_model_uuid']
texts = data['texts']
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(embedding_model_uuid)
if embedding_model is None:
return handler.ActionResponse.error(
message=f'Embedding model with embedding_model_uuid {embedding_model_uuid} not found',
)
try:
vectors = await embedding_model.provider.invoke_embedding(embedding_model, texts)
return handler.ActionResponse.success(data={'vectors': vectors})
except Exception as e:
return _make_rag_error_response(e, 'EmbeddingError', embedding_model_uuid=embedding_model_uuid)
@self.action(PluginToRuntimeAction.VECTOR_UPSERT)
async def vector_upsert(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
vectors = data['vectors']
ids = data['ids']
metadata = data.get('metadata')
documents = data.get('documents')
if len(vectors) != len(ids):
return handler.ActionResponse.error(message='vectors and ids must have same length')
if metadata and len(metadata) != len(vectors):
return handler.ActionResponse.error(message='metadata must match vectors length')
if documents and len(documents) != len(vectors):
return handler.ActionResponse.error(message='documents must match vectors length')
try:
await self.ap.rag_runtime_service.vector_upsert(
collection_id,
vectors,
ids,
metadata,
documents,
)
return handler.ActionResponse.success(data={})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.VECTOR_SEARCH)
async def vector_search(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
query_vector = data['query_vector']
top_k = data['top_k']
filters = data.get('filters')
search_type = data.get('search_type', 'vector')
query_text = data.get('query_text', '')
try:
results = await self.ap.rag_runtime_service.vector_search(
collection_id,
query_vector,
top_k,
filters,
search_type,
query_text,
)
return handler.ActionResponse.success(data={'results': results})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.VECTOR_DELETE)
async def vector_delete(data: dict[str, Any]) -> handler.ActionResponse:
collection_id = data['collection_id']
file_ids = data.get('file_ids')
filters = data.get('filters')
try:
count = await self.ap.rag_runtime_service.vector_delete(collection_id, file_ids, filters)
return handler.ActionResponse.success(data={'count': count})
except Exception as e:
return _make_rag_error_response(e, 'VectorStoreError', collection_id=collection_id)
@self.action(PluginToRuntimeAction.GET_KNOWLEDEGE_FILE_STREAM)
async def get_knowledge_file_stream(data: dict[str, Any]) -> handler.ActionResponse:
storage_path = data['storage_path']
try:
content_bytes = await self.ap.rag_runtime_service.get_file_stream(storage_path)
file_key = await self.send_file(content_bytes, '')
return handler.ActionResponse.success(data={'file_key': file_key})
except Exception as e:
return _make_rag_error_response(e, 'FileServiceError', storage_path=storage_path)
@self.action(PluginToRuntimeAction.INVOKE_PARSER)
async def invoke_parser(data: dict[str, Any]) -> handler.ActionResponse:
"""Plugin requests host to invoke a parser plugin."""
plugin_author = data['plugin_author']
plugin_name = data['plugin_name']
storage_path = data['storage_path']
mime_type = data.get('mime_type', 'application/octet-stream')
filename = data.get('filename', '')
metadata = data.get('metadata', {})
try:
# Read file from storage
file_bytes = await self.ap.rag_runtime_service.get_file_stream(storage_path)
context_data = {
'mime_type': mime_type,
'filename': filename,
'metadata': metadata,
}
result = await self.ap.plugin_connector.call_parser(
f'{plugin_author}/{plugin_name}', context_data, file_bytes
)
return handler.ActionResponse.success(data=result)
except Exception as e:
return _make_rag_error_response(e, 'ParserError')
@self.action(CommonAction.PING)
async def ping(data: dict[str, Any]) -> handler.ActionResponse:
"""Ping"""
return handler.ActionResponse.success(
data={
'pong': 'pong',
},
)
async def ping(self) -> dict[str, Any]:
"""Ping the runtime"""
return await self.call_action(
@@ -716,26 +857,13 @@ class RuntimeConnectionHandler(handler.Handler):
async for ret in gen:
yield ret
# KnowledgeRetriever methods
async def list_knowledge_retrievers(self, include_plugins: list[str] | None = None) -> list[dict[str, Any]]:
"""List knowledge retrievers"""
result = await self.call_action(
LangBotToRuntimeAction.LIST_KNOWLEDGE_RETRIEVERS,
{
'include_plugins': include_plugins,
},
timeout=10,
)
return result['retrievers']
async def retrieve_knowledge(
self,
plugin_author: str,
plugin_name: str,
retriever_name: str,
instance_id: str,
retrieval_context: dict[str, Any],
) -> list[dict[str, Any]]:
) -> dict[str, Any]:
"""Retrieve knowledge"""
result = await self.call_action(
LangBotToRuntimeAction.RETRIEVE_KNOWLEDGE,
@@ -743,22 +871,10 @@ class RuntimeConnectionHandler(handler.Handler):
'plugin_author': plugin_author,
'plugin_name': plugin_name,
'retriever_name': retriever_name,
'instance_id': instance_id,
'retrieval_context': retrieval_context,
},
timeout=30,
)
return result['retrieval_results']
async def sync_polymorphic_component_instances(self, required_instances: list[dict[str, Any]]) -> dict[str, Any]:
"""Sync polymorphic component instances with runtime"""
result = await self.call_action(
LangBotToRuntimeAction.SYNC_POLYMORPHIC_COMPONENT_INSTANCES,
{
'required_instances': required_instances,
},
timeout=30,
)
return result
async def get_debug_info(self) -> dict[str, Any]:
@@ -769,3 +885,91 @@ class RuntimeConnectionHandler(handler.Handler):
timeout=10,
)
return result
# ================= RAG Capability Callers (LangBot -> Runtime) =================
async def rag_ingest_document(
self, plugin_author: str, plugin_name: str, context_data: dict[str, Any]
) -> dict[str, Any]:
"""Send INGEST_DOCUMENT action to runtime."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_INGEST_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'context': context_data},
timeout=300, # Ingestion can be slow
)
return result
async def rag_delete_document(self, plugin_author: str, plugin_name: str, document_id: str, kb_id: str) -> bool:
result = await self.call_action(
LangBotToRuntimeAction.RAG_DELETE_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'document_id': document_id, 'kb_id': kb_id},
timeout=30,
)
return result.get('success', False)
async def rag_on_kb_create(
self, plugin_author: str, plugin_name: str, kb_id: str, config: dict[str, Any]
) -> dict[str, Any]:
"""Notify plugin about KB creation."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_ON_KB_CREATE,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'kb_id': kb_id, 'config': config},
timeout=30,
)
return result
async def rag_on_kb_delete(self, plugin_author: str, plugin_name: str, kb_id: str) -> dict[str, Any]:
"""Notify plugin about KB deletion."""
result = await self.call_action(
LangBotToRuntimeAction.RAG_ON_KB_DELETE,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'kb_id': kb_id},
timeout=30,
)
return result
async def get_rag_creation_schema(self, plugin_author: str, plugin_name: str) -> dict[str, Any]:
return await self.call_action(
LangBotToRuntimeAction.GET_RAG_CREATION_SETTINGS_SCHEMA,
{'plugin_author': plugin_author, 'plugin_name': plugin_name},
timeout=10,
)
async def get_rag_retrieval_schema(self, plugin_author: str, plugin_name: str) -> dict[str, Any]:
return await self.call_action(
LangBotToRuntimeAction.GET_RAG_RETRIEVAL_SETTINGS_SCHEMA,
{'plugin_author': plugin_author, 'plugin_name': plugin_name},
timeout=10,
)
async def list_knowledge_engines(self) -> list[dict[str, Any]]:
"""List all available Knowledge Engines from plugins."""
result = await self.call_action(LangBotToRuntimeAction.LIST_KNOWLEDGE_ENGINES, {}, timeout=60)
return result.get('engines', [])
# ================= Parser Capability Callers (LangBot -> Runtime) =================
async def list_parsers(self) -> list[dict[str, Any]]:
"""List all available parsers from plugins."""
result = await self.call_action(LangBotToRuntimeAction.LIST_PARSERS, {}, timeout=60)
return result.get('parsers', [])
async def parse_document(
self, plugin_author: str, plugin_name: str, context_data: dict[str, Any], file_bytes: bytes
) -> dict[str, Any]:
"""Send PARSE_DOCUMENT action to runtime.
Sends file content via chunked FILE_CHUNK transfer, then invokes
the PARSE_DOCUMENT action with a file_key reference.
"""
# Send file to runtime via chunked transfer
file_key = await self.send_file(file_bytes, '')
# Include file_key in context_data for the runtime to read
context_data['file_key'] = file_key
result = await self.call_action(
LangBotToRuntimeAction.PARSE_DOCUMENT,
{'plugin_author': plugin_author, 'plugin_name': plugin_name, 'context': context_data},
timeout=300,
)
return result

View File

@@ -72,6 +72,28 @@ class DifyServiceAPIRunner(runner.RequestRunner):
content = f'<think>\n{thinking_content}\n</think>\n{content}'.strip()
return content, thinking_content
def _extract_dify_text_output(self, value: typing.Any) -> str:
"""Extract text content from Dify output payload."""
if value is None:
return ''
if isinstance(value, dict):
content = value.get('content')
if isinstance(content, str):
return content
return json.dumps(value, ensure_ascii=False)
if isinstance(value, str):
text = value.strip()
if not text:
return ''
try:
parsed = json.loads(text)
except json.JSONDecodeError:
return value
if isinstance(parsed, dict) and isinstance(parsed.get('content'), str):
return parsed['content']
return value
return str(value)
async def _preprocess_user_message(self, query: pipeline_query.Query) -> tuple[str, list[dict]]:
"""预处理用户消息,提取纯文本,并将图片/文件上传到 Dify 服务
@@ -192,7 +214,8 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if mode == 'workflow':
if chunk['event'] == 'node_finished':
if chunk['data']['node_type'] == 'answer':
content, _ = self._process_thinking_content(chunk['data']['outputs']['answer'])
answer = self._extract_dify_text_output(chunk['data']['outputs'].get('answer'))
content, _ = self._process_thinking_content(answer)
yield provider_message.Message(
role='assistant',
@@ -405,6 +428,7 @@ class DifyServiceAPIRunner(runner.RequestRunner):
for f in upload_files
]
mode = 'basic'
basic_mode_pending_chunk = ''
inputs = {}
@@ -430,11 +454,12 @@ class DifyServiceAPIRunner(runner.RequestRunner):
):
self.ap.logger.debug('dify-chat-chunk: ' + str(chunk))
# if chunk['event'] == 'workflow_started':
# mode = 'workflow'
# if mode == 'workflow':
# elif mode == 'basic':
# 因为都只是返回的 message也没有工具调用什么的暂时不分类
if chunk['event'] == 'workflow_started':
mode = 'workflow'
elif chunk['event'] in ('node_started', 'node_finished', 'workflow_finished'):
# Some Dify deployments may omit workflow_started in streamed chunks.
mode = 'workflow'
if chunk['event'] == 'message':
message_idx += 1
if remove_think:
@@ -457,8 +482,18 @@ class DifyServiceAPIRunner(runner.RequestRunner):
if chunk['event'] == 'message_end':
is_final = True
elif chunk['event'] == 'workflow_finished':
is_final = True
if chunk['data'].get('error'):
raise errors.DifyAPIError(chunk['data']['error'])
if is_final or message_idx % 8 == 0:
if mode == 'workflow' and chunk['event'] == 'node_finished':
if chunk['data'].get('node_type') == 'answer':
answer = self._extract_dify_text_output(chunk['data'].get('outputs', {}).get('answer'))
if answer:
basic_mode_pending_chunk = answer
if (is_final or message_idx % 8 == 0) and (basic_mode_pending_chunk != '' or is_final):
# content, _ = self._process_thinking_content(basic_mode_pending_chunk)
yield provider_message.MessageChunk(
role='assistant',

View File

@@ -74,15 +74,7 @@ class LocalAgentRunner(runner.RequestRunner):
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found, skipping')
continue
# Get top_k based on KB type
if kb.get_type() == 'internal':
top_k = kb.knowledge_base_entity.top_k
elif kb.get_type() == 'external':
top_k = 5 # external kb's top_k is managed by plugin config
else:
top_k = 5 # default fallback
result = await kb.retrieve(user_message_text, top_k)
result = await kb.retrieve(user_message_text)
if result:
all_results.extend(result)
@@ -97,9 +89,9 @@ class LocalAgentRunner(runner.RequestRunner):
if content.type == 'text' and content.text is not None:
texts.append(f'[{idx}] {content.text}')
idx += 1
rag_context = '\n\n'.join(texts)
rag_context_text = '\n\n'.join(texts)
final_user_message_text = rag_combined_prompt_template.format(
rag_context=rag_context, user_message=user_message_text
rag_context=rag_context_text, user_message=user_message_text
)
else:

View File

@@ -5,6 +5,8 @@ import json
import uuid
import aiohttp
from langbot.pkg.utils import httpclient
from .. import runner
from ...core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -217,50 +219,50 @@ class N8nServiceAPIRunner(runner.RequestRunner):
self.ap.logger.debug('no auth')
# 调用webhook
async with aiohttp.ClientSession() as session:
if is_stream:
# 流式请求
async with session.post(
self.webhook_url, json=payload, headers=headers, auth=auth, timeout=self.timeout
) as response:
session = httpclient.get_session()
if is_stream:
# 流式请求
async with session.post(
self.webhook_url, json=payload, headers=headers, auth=auth, timeout=self.timeout
) as response:
if response.status != 200:
error_text = await response.text()
self.ap.logger.error(f'n8n webhook call failed: {response.status}, {error_text}')
raise Exception(f'n8n webhook call failed: {response.status}, {error_text}')
# 处理流式响应
async for chunk in self._process_stream_response(response):
yield chunk
else:
async with session.post(
self.webhook_url, json=payload, headers=headers, auth=auth, timeout=self.timeout
) as response:
try:
async for chunk in self._process_stream_response(response):
output_content = chunk.content if chunk.is_final else ''
except:
# 非流式请求(保持原有逻辑)
if response.status != 200:
error_text = await response.text()
self.ap.logger.error(f'n8n webhook call failed: {response.status}, {error_text}')
raise Exception(f'n8n webhook call failed: {response.status}, {error_text}')
# 处理流式响应
async for chunk in self._process_stream_response(response):
yield chunk
else:
async with session.post(
self.webhook_url, json=payload, headers=headers, auth=auth, timeout=self.timeout
) as response:
try:
async for chunk in self._process_stream_response(response):
output_content = chunk.content if chunk.is_final else ''
except:
# 非流式请求(保持原有逻辑)
if response.status != 200:
error_text = await response.text()
self.ap.logger.error(f'n8n webhook call failed: {response.status}, {error_text}')
raise Exception(f'n8n webhook call failed: {response.status}, {error_text}')
# 解析响应
response_data = await response.json()
self.ap.logger.debug(f'n8n webhook response: {response_data}')
# 解析响应
response_data = await response.json()
self.ap.logger.debug(f'n8n webhook response: {response_data}')
# 从响应中提取输出
if self.output_key in response_data:
output_content = response_data[self.output_key]
else:
# 如果没有指定的输出键,则使用整个响应
output_content = json.dumps(response_data, ensure_ascii=False)
# 从响应中提取输出
if self.output_key in response_data:
output_content = response_data[self.output_key]
else:
# 如果没有指定的输出键,则使用整个响应
output_content = json.dumps(response_data, ensure_ascii=False)
# 返回消息
yield provider_message.Message(
role='assistant',
content=output_content,
)
# 返回消息
yield provider_message.Message(
role='assistant',
content=output_content,
)
except Exception as e:
self.ap.logger.error(f'n8n webhook call exception: {str(e)}')
raise N8nAPIError(f'n8n webhook call exception: {str(e)}')

View File

@@ -22,12 +22,12 @@ class KnowledgeBaseInterface(metaclass=abc.ABCMeta):
pass
@abc.abstractmethod
async def retrieve(self, query: str, top_k: int) -> list[rag_context.RetrievalResultEntry]:
async def retrieve(self, query: str, settings: dict | None = None) -> list[rag_context.RetrievalResultEntry]:
"""Retrieve relevant documents from the knowledge base
Args:
query: The query string
top_k: Number of top results to return
settings: Optional per-request retrieval settings overrides
Returns:
List of retrieve result entries
@@ -45,8 +45,8 @@ class KnowledgeBaseInterface(metaclass=abc.ABCMeta):
pass
@abc.abstractmethod
def get_type(self) -> str:
"""Get the type of knowledge base (internal/external)"""
def get_knowledge_engine_plugin_id(self) -> str:
"""Get the Knowledge Engine plugin ID"""
pass
@abc.abstractmethod

View File

@@ -1,85 +0,0 @@
"""External knowledge base implementation"""
from __future__ import annotations
from langbot.pkg.core import app
from langbot.pkg.entity.persistence import rag as persistence_rag
from langbot_plugin.api.entities.builtin.rag import context as rag_context
from .base import KnowledgeBaseInterface
class ExternalKnowledgeBase(KnowledgeBaseInterface):
"""External knowledge base that queries via HTTP API or plugin retriever"""
external_kb_entity: persistence_rag.ExternalKnowledgeBase
# Plugin retriever instance ID
retriever_instance_id: str | None
def __init__(self, ap: app.Application, external_kb_entity: persistence_rag.ExternalKnowledgeBase):
super().__init__(ap)
self.external_kb_entity = external_kb_entity
self.retriever_instance_id = None
async def initialize(self):
"""Initialize the external knowledge base"""
# Use KB UUID as instance ID
# Instance creation is now handled by the unified sync mechanism
# when LangBot connects to runtime
self.retriever_instance_id = self.external_kb_entity.uuid
self.ap.logger.info(
f'Initialized external KB {self.external_kb_entity.uuid}, instance will be created by sync mechanism'
)
async def retrieve(self, query: str, top_k: int = 5) -> list[rag_context.RetrievalResultEntry]:
"""Retrieve documents from external knowledge base via plugin retriever"""
if not self.retriever_instance_id:
self.ap.logger.error(f'No retriever instance for KB {self.external_kb_entity.uuid}')
return []
try:
results = await self.ap.plugin_connector.retrieve_knowledge(
self.external_kb_entity.plugin_author,
self.external_kb_entity.plugin_name,
self.external_kb_entity.retriever_name,
self.retriever_instance_id,
{'query': query},
)
# Convert plugin results to RetrievalResultEntry
retrieval_entries = []
for result in results:
retrieval_entries.append(rag_context.RetrievalResultEntry(**result))
return retrieval_entries
except Exception as e:
self.ap.logger.error(f'Plugin retriever error: {e}')
import traceback
traceback.print_exc()
return []
def get_uuid(self) -> str:
"""Get the UUID of the external knowledge base"""
return self.external_kb_entity.uuid
def get_name(self) -> str:
"""Get the name of the external knowledge base"""
return self.external_kb_entity.name
def get_type(self) -> str:
"""Get the type of knowledge base"""
return 'external'
async def dispose(self):
"""Clean up resources"""
# Trigger sync to immediately delete the instance from plugin process
# This ensures instance is cleaned up without waiting for next LangBot restart
try:
await self.ap.plugin_connector.sync_polymorphic_component_instances()
self.ap.logger.info(
f'Disposed external KB {self.external_kb_entity.uuid}, triggered sync to delete instance'
)
except Exception as e:
self.ap.logger.error(f'Failed to sync after disposing KB: {e}')

View File

@@ -1,18 +1,19 @@
from __future__ import annotations
import mimetypes
import os.path
import traceback
import uuid
import zipfile
import io
from .services import parser, chunker
from typing import Any
from langbot.pkg.core import app
from langbot.pkg.rag.knowledge.services.embedder import Embedder
from langbot.pkg.rag.knowledge.services.retriever import Retriever
import sqlalchemy
from langbot.pkg.entity.persistence import rag as persistence_rag
from langbot.pkg.core import taskmgr
from langbot_plugin.api.entities.builtin.rag import context as rag_context
from .base import KnowledgeBaseInterface
from .external import ExternalKnowledgeBase
class RuntimeKnowledgeBase(KnowledgeBaseInterface):
@@ -20,28 +21,16 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
knowledge_base_entity: persistence_rag.KnowledgeBase
parser: parser.FileParser
chunker: chunker.Chunker
embedder: Embedder
retriever: Retriever
def __init__(self, ap: app.Application, knowledge_base_entity: persistence_rag.KnowledgeBase):
super().__init__(ap)
self.knowledge_base_entity = knowledge_base_entity
self.parser = parser.FileParser(ap=self.ap)
self.chunker = chunker.Chunker(ap=self.ap)
self.embedder = Embedder(ap=self.ap)
self.retriever = Retriever(ap=self.ap)
# 传递kb_id给retriever
self.retriever.kb_id = knowledge_base_entity.uuid
async def initialize(self):
pass
async def _store_file_task(self, file: persistence_rag.File, task_context: taskmgr.TaskContext):
async def _store_file_task(
self, file: persistence_rag.File, task_context: taskmgr.TaskContext, parser_plugin_id: str | None = None
):
try:
# set file status to processing
await self.ap.persistence_mgr.execute_async(
@@ -50,31 +39,46 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
.values(status='processing')
)
task_context.set_current_action('Parsing file')
# parse file
text = await self.parser.parse(file.file_name, file.extension)
if not text:
raise Exception(f'No text extracted from file {file.file_name}')
task_context.set_current_action('Processing file')
task_context.set_current_action('Chunking file')
# chunk file
chunks_texts = await self.chunker.chunk(text)
if not chunks_texts:
raise Exception(f'No chunks extracted from file {file.file_name}')
# Get file size from storage
file_size = await self.ap.storage_mgr.storage_provider.size(file.file_name)
task_context.set_current_action('Embedding chunks')
# Detect MIME type from extension
mime_type, _ = mimetypes.guess_type(file.file_name)
if mime_type is None:
mime_type = 'application/octet-stream'
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(
self.knowledge_base_entity.embedding_model_uuid
)
# embed chunks
await self.embedder.embed_and_store(
kb_id=self.knowledge_base_entity.uuid,
file_id=file.uuid,
chunks=chunks_texts,
embedding_model=embedding_model,
# If a parser plugin is specified, call it before ingestion
parsed_content = None
if parser_plugin_id:
task_context.set_current_action('Parsing file')
file_bytes = await self.ap.storage_mgr.storage_provider.load(file.file_name)
parse_context = {
'mime_type': mime_type,
'filename': file.file_name,
'metadata': {},
}
parsed_content = await self.ap.plugin_connector.call_parser(parser_plugin_id, parse_context, file_bytes)
# Call plugin to ingest document
result = await self._ingest_document(
{
'document_id': file.uuid,
'filename': file.file_name,
'extension': file.extension,
'file_size': file_size,
'mime_type': mime_type,
},
file.file_name, # storage path
parsed_content=parsed_content,
)
# Check plugin result status
if result.get('status') == 'failed':
error_msg = result.get('error_message', 'Plugin ingestion returned failed status')
raise Exception(error_msg)
# set file status to completed
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.File)
@@ -97,16 +101,17 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
# delete file from storage
await self.ap.storage_mgr.storage_provider.delete(file.file_name)
async def store_file(self, file_id: str) -> str:
async def store_file(self, file_id: str, parser_plugin_id: str | None = None) -> str:
# pre checking
if not await self.ap.storage_mgr.storage_provider.exists(file_id):
raise Exception(f'File {file_id} not found')
file_name = file_id
extension = file_name.split('.')[-1].lower()
_, ext = os.path.splitext(file_name)
extension = ext.lstrip('.').lower() if ext else ''
if extension == 'zip':
return await self._store_zip_file(file_id)
return await self._store_zip_file(file_id, parser_plugin_id=parser_plugin_id)
file_uuid = str(uuid.uuid4())
kb_id = self.knowledge_base_entity.uuid
@@ -126,7 +131,7 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
# run background task asynchronously
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self._store_file_task(file_obj, task_context=ctx),
self._store_file_task(file_obj, task_context=ctx, parser_plugin_id=parser_plugin_id),
kind='knowledge-operation',
name=f'knowledge-store-file-{file_id}',
label=f'Store file {file_id}',
@@ -134,7 +139,7 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
)
return wrapper.id
async def _store_zip_file(self, zip_file_id: str) -> str:
async def _store_zip_file(self, zip_file_id: str, parser_plugin_id: str | None = None) -> str:
"""Handle ZIP file by extracting each document and storing them separately."""
self.ap.logger.info(f'Processing ZIP file: {zip_file_id}')
@@ -150,7 +155,8 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
if file_info.is_dir() or file_info.filename.startswith('.'):
continue
file_extension = file_info.filename.split('.')[-1].lower()
_, file_ext = os.path.splitext(file_info.filename)
file_extension = file_ext.lstrip('.').lower()
if file_extension not in supported_extensions:
self.ap.logger.debug(f'Skipping unsupported file in ZIP: {file_info.filename}')
continue
@@ -159,18 +165,18 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
file_content = zip_ref.read(file_info.filename)
base_name = file_info.filename.replace('/', '_').replace('\\', '_')
extension = base_name.split('.')[-1]
file_name = base_name.split('.')[0]
file_stem, file_ext = os.path.splitext(base_name)
extension = file_ext.lstrip('.')
if file_name.startswith('__MACOSX'):
if file_stem.startswith('__MACOSX'):
continue
extracted_file_id = file_name + '_' + str(uuid.uuid4())[:8] + '.' + extension
extracted_file_id = file_stem + '_' + str(uuid.uuid4())[:8] + '.' + extension
# save file to storage
await self.ap.storage_mgr.storage_provider.save(extracted_file_id, file_content)
task_id = await self.store_file(extracted_file_id)
task_id = await self.store_file(extracted_file_id, parser_plugin_id=parser_plugin_id)
stored_file_tasks.append(task_id)
self.ap.logger.info(
@@ -189,21 +195,28 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
return stored_file_tasks[0] if stored_file_tasks else ''
async def retrieve(self, query: str, top_k: int) -> list[rag_context.RetrievalResultEntry]:
embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid(
self.knowledge_base_entity.embedding_model_uuid
)
return await self.retriever.retrieve(self.knowledge_base_entity.uuid, query, embedding_model, top_k)
async def retrieve(self, query: str, settings: dict | None = None) -> list[rag_context.RetrievalResultEntry]:
# Merge stored retrieval_settings with per-request overrides
stored = self.knowledge_base_entity.retrieval_settings or {}
merged = {**stored, **(settings or {})}
if 'top_k' not in merged:
merged['top_k'] = 5 # fallback default
response = await self._retrieve(query, merged)
results_data = response.get('results', [])
entries = []
for r in results_data:
if isinstance(r, dict):
entries.append(rag_context.RetrievalResultEntry(**r))
elif isinstance(r, rag_context.RetrievalResultEntry):
entries.append(r)
return entries
async def delete_file(self, file_id: str):
# delete vector
await self.ap.vector_db_mgr.vector_db.delete_by_file_id(self.knowledge_base_entity.uuid, file_id)
# delete chunk
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.Chunk).where(persistence_rag.Chunk.file_id == file_id)
)
await self._delete_document(file_id)
# Also cleanup DB record
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file_id)
)
@@ -216,32 +229,289 @@ class RuntimeKnowledgeBase(KnowledgeBaseInterface):
"""Get the name of the knowledge base"""
return self.knowledge_base_entity.name
def get_type(self) -> str:
"""Get the type of knowledge base"""
return 'internal'
def get_knowledge_engine_plugin_id(self) -> str:
"""Get the Knowledge Engine plugin ID"""
return self.knowledge_base_entity.knowledge_engine_plugin_id or ''
async def dispose(self):
await self.ap.vector_db_mgr.vector_db.delete_collection(self.knowledge_base_entity.uuid)
"""Dispose the knowledge base, notifying the plugin to cleanup."""
await self._on_kb_delete()
# ========== Plugin Communication Methods ==========
async def _on_kb_create(self) -> None:
"""Notify plugin about KB creation."""
plugin_id = self.knowledge_base_entity.knowledge_engine_plugin_id
if not plugin_id:
return
try:
config = self.knowledge_base_entity.creation_settings or {}
self.ap.logger.info(
f'Calling RAG plugin {plugin_id}: on_knowledge_base_create(kb_id={self.knowledge_base_entity.uuid})'
)
await self.ap.plugin_connector.rag_on_kb_create(plugin_id, self.knowledge_base_entity.uuid, config)
except Exception as e:
self.ap.logger.error(f'Failed to notify plugin {plugin_id} on KB create: {e}')
raise
async def _on_kb_delete(self) -> None:
"""Notify plugin about KB deletion."""
plugin_id = self.knowledge_base_entity.knowledge_engine_plugin_id
if not plugin_id:
return
try:
self.ap.logger.info(
f'Calling RAG plugin {plugin_id}: on_knowledge_base_delete(kb_id={self.knowledge_base_entity.uuid})'
)
await self.ap.plugin_connector.rag_on_kb_delete(plugin_id, self.knowledge_base_entity.uuid)
except Exception as e:
self.ap.logger.error(f'Failed to notify plugin {plugin_id} on KB delete: {e}')
async def _ingest_document(
self,
file_metadata: dict[str, Any],
storage_path: str,
parsed_content: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Call plugin to ingest document."""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
self.ap.logger.error(f'No RAG plugin ID configured for KB {kb.uuid}. Ingestion failed.')
raise ValueError('RAG Plugin ID required')
self.ap.logger.info(f'Calling RAG plugin {plugin_id}: ingest(doc={file_metadata.get("filename")})')
# Inject knowledge_base_id into file metadata as required by SDK schema
file_metadata['knowledge_base_id'] = kb.uuid
context_data = {
'file_object': {
'metadata': file_metadata,
'storage_path': storage_path,
},
'knowledge_base_id': kb.uuid,
'collection_id': kb.collection_id or kb.uuid,
'creation_settings': kb.creation_settings or {},
'parsed_content': parsed_content,
}
try:
result = await self.ap.plugin_connector.call_rag_ingest(plugin_id, context_data)
return result
except Exception as e:
self.ap.logger.error(f'Plugin ingestion failed: {e}')
raise
async def _retrieve(
self,
query: str,
settings: dict[str, Any],
) -> dict[str, Any]:
"""Call plugin to retrieve documents.
Raises:
ValueError: If no RAG plugin is configured for this KB.
Exception: If the plugin retrieval call fails.
"""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
raise ValueError(f'No RAG plugin ID configured for KB {kb.uuid}. Retrieval failed.')
retrieval_context = {
'query': query,
'knowledge_base_id': kb.uuid,
'collection_id': kb.collection_id or kb.uuid,
'retrieval_settings': settings,
'creation_settings': kb.creation_settings or {},
'filters': settings.pop('filters', {}),
}
result = await self.ap.plugin_connector.call_rag_retrieve(
plugin_id,
retrieval_context,
)
return result
async def _delete_document(self, document_id: str) -> bool:
"""Call plugin to delete document."""
kb = self.knowledge_base_entity
plugin_id = kb.knowledge_engine_plugin_id
if not plugin_id:
return False
self.ap.logger.info(f'Calling RAG plugin {plugin_id}: delete_document(doc_id={document_id})')
try:
return await self.ap.plugin_connector.call_rag_delete_document(plugin_id, document_id, kb.uuid)
except Exception as e:
self.ap.logger.error(f'Plugin document deletion failed: {e}')
return False
class RAGManager:
ap: app.Application
knowledge_bases: list[KnowledgeBaseInterface]
knowledge_bases: dict[str, KnowledgeBaseInterface]
def __init__(self, ap: app.Application):
self.ap = ap
self.knowledge_bases = []
self.knowledge_bases = {}
async def initialize(self):
await self.load_knowledge_bases_from_db()
async def get_all_knowledge_base_details(self) -> list[dict]:
"""Get all knowledge bases with enriched Knowledge Engine details."""
# 1. Get raw KBs from DB
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
# 2. Get all available Knowledge Engines for enrichment
engine_map = {}
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_map = {e['plugin_id']: e for e in engines}
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines: {e}')
# 3. Serialize and enrich
kb_list = []
for kb in knowledge_bases:
kb_dict = self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, kb)
self._enrich_kb_dict(kb_dict, engine_map)
kb_list.append(kb_dict)
return kb_list
async def get_knowledge_base_details(self, kb_uuid: str) -> dict | None:
"""Get specific knowledge base with enriched Knowledge Engine details."""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
kb = result.first()
if not kb:
return None
kb_dict = self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, kb)
# Fetch engines
engine_map = {}
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_map = {e['plugin_id']: e for e in engines}
except Exception as e:
self.ap.logger.warning(f'Failed to list Knowledge Engines: {e}')
self._enrich_kb_dict(kb_dict, engine_map)
return kb_dict
@staticmethod
def _to_i18n_name(name) -> dict:
"""Ensure name is always an I18nObject-compatible dict.
If *name* is already a dict (with ``en_US`` / ``zh_Hans`` keys) it is
returned as-is. A plain string is wrapped into an I18nObject so the
frontend ``extractI18nObject`` helper never receives an unexpected type.
"""
if isinstance(name, dict):
return name
return {'en_US': str(name), 'zh_Hans': str(name)}
def _enrich_kb_dict(self, kb_dict: dict, engine_map: dict) -> None:
"""Helper to inject engine info into KB dict."""
plugin_id = kb_dict.get('knowledge_engine_plugin_id')
# Default fallback structure — name must be I18nObject for frontend compatibility
fallback_name = self._to_i18n_name(plugin_id or 'Internal (Legacy)')
fallback_info = {
'plugin_id': plugin_id,
'name': fallback_name,
'capabilities': [],
}
if not plugin_id:
kb_dict['knowledge_engine'] = fallback_info
return
engine_info = engine_map.get(plugin_id)
if engine_info:
kb_dict['knowledge_engine'] = {
'plugin_id': plugin_id,
'name': self._to_i18n_name(engine_info.get('name', plugin_id)),
'capabilities': engine_info.get('capabilities', []),
}
else:
kb_dict['knowledge_engine'] = fallback_info
async def create_knowledge_base(
self,
name: str,
knowledge_engine_plugin_id: str,
creation_settings: dict,
retrieval_settings: dict | None = None,
description: str = '',
) -> persistence_rag.KnowledgeBase:
"""Create a new knowledge base using a RAG plugin."""
# Validate that the Knowledge Engine plugin exists
if self.ap.plugin_connector.is_enable_plugin:
try:
engines = await self.ap.plugin_connector.list_knowledge_engines()
engine_ids = [e.get('plugin_id') for e in engines]
if knowledge_engine_plugin_id not in engine_ids:
raise ValueError(f'Knowledge Engine plugin {knowledge_engine_plugin_id} not found')
except ValueError:
raise
except Exception as e:
self.ap.logger.warning(f'Failed to validate Knowledge Engine plugin existence: {e}')
kb_uuid = str(uuid.uuid4())
# Use UUID as collection ID by default for isolation
collection_id = kb_uuid
kb_data = {
'uuid': kb_uuid,
'name': name,
'description': description,
'knowledge_engine_plugin_id': knowledge_engine_plugin_id,
'collection_id': collection_id,
'creation_settings': creation_settings,
'retrieval_settings': retrieval_settings or {},
}
# Create Entity
kb = persistence_rag.KnowledgeBase(**kb_data)
# Persist
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.KnowledgeBase).values(kb_data))
# Load into Runtime
runtime_kb = await self.load_knowledge_base(kb)
# Notify Plugin — rollback DB record and runtime entry on failure
try:
await runtime_kb._on_kb_create()
except Exception:
self.knowledge_bases.pop(kb_uuid, None)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
raise
self.ap.logger.info(f'Created new Knowledge Base {name} ({kb_uuid}) using plugin {knowledge_engine_plugin_id}')
return kb
async def load_knowledge_bases_from_db(self):
self.ap.logger.info('Loading knowledge bases from db...')
self.knowledge_bases = []
self.knowledge_bases = {}
# Load internal knowledge bases
# Load knowledge bases
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase))
knowledge_bases = result.all()
@@ -253,86 +523,37 @@ class RAGManager:
f'Error loading knowledge base {knowledge_base.uuid}: {e}\n{traceback.format_exc()}'
)
# Load external knowledge bases
external_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.ExternalKnowledgeBase)
)
external_kbs = external_result.all()
for external_kb in external_kbs:
try:
# Don't trigger sync during batch loading - will sync once after LangBot connects to runtime
await self.load_external_knowledge_base(external_kb, trigger_sync=False)
except Exception as e:
self.ap.logger.error(
f'Error loading external knowledge base {external_kb.uuid}: {e}\n{traceback.format_exc()}'
)
async def load_knowledge_base(
self,
knowledge_base_entity: persistence_rag.KnowledgeBase | sqlalchemy.Row | dict,
) -> RuntimeKnowledgeBase:
if isinstance(knowledge_base_entity, sqlalchemy.Row):
# Safe access to _mapping for SQLAlchemy 1.4+
knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity._mapping)
elif isinstance(knowledge_base_entity, dict):
knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity)
# Filter out non-database fields (like knowledge_engine which is computed)
filtered_dict = {
k: v for k, v in knowledge_base_entity.items() if k in persistence_rag.KnowledgeBase.ALL_DB_FIELDS
}
knowledge_base_entity = persistence_rag.KnowledgeBase(**filtered_dict)
runtime_knowledge_base = RuntimeKnowledgeBase(ap=self.ap, knowledge_base_entity=knowledge_base_entity)
await runtime_knowledge_base.initialize()
self.knowledge_bases.append(runtime_knowledge_base)
self.knowledge_bases[runtime_knowledge_base.get_uuid()] = runtime_knowledge_base
return runtime_knowledge_base
async def load_external_knowledge_base(
self,
external_kb_entity: persistence_rag.ExternalKnowledgeBase | sqlalchemy.Row | dict,
trigger_sync: bool = True,
) -> ExternalKnowledgeBase:
"""Load external knowledge base into runtime
Args:
external_kb_entity: External KB entity to load
trigger_sync: Whether to trigger sync after loading (default True for manual creation, False for batch loading)
"""
if isinstance(external_kb_entity, sqlalchemy.Row):
external_kb_entity = persistence_rag.ExternalKnowledgeBase(**external_kb_entity._mapping)
elif isinstance(external_kb_entity, dict):
external_kb_entity = persistence_rag.ExternalKnowledgeBase(**external_kb_entity)
external_kb = ExternalKnowledgeBase(ap=self.ap, external_kb_entity=external_kb_entity)
await external_kb.initialize()
self.knowledge_bases.append(external_kb)
# Trigger sync to create the instance immediately (for manual creation)
# Skip sync during batch loading from DB to avoid multiple sync calls
if trigger_sync:
try:
await self.ap.plugin_connector.sync_polymorphic_component_instances()
self.ap.logger.info(f'Triggered sync after loading external KB {external_kb_entity.uuid}')
except Exception as e:
self.ap.logger.error(f'Failed to sync after loading external KB: {e}')
return external_kb
async def get_knowledge_base_by_uuid(self, kb_uuid: str) -> KnowledgeBaseInterface | None:
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
return kb
return None
return self.knowledge_bases.get(kb_uuid)
async def remove_knowledge_base_from_runtime(self, kb_uuid: str):
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
self.knowledge_bases.remove(kb)
return
self.knowledge_bases.pop(kb_uuid, None)
async def delete_knowledge_base(self, kb_uuid: str):
for kb in self.knowledge_bases:
if kb.get_uuid() == kb_uuid:
await kb.dispose()
self.knowledge_bases.remove(kb)
return
kb = self.knowledge_bases.pop(kb_uuid, None)
if kb is not None:
await kb.dispose()
else:
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found in runtime, skipping plugin notification')

View File

@@ -1,15 +0,0 @@
# 封装异步操作
import asyncio
class BaseService:
def __init__(self):
pass
async def _run_sync(self, func, *args, **kwargs):
"""
在单独的线程中运行同步函数。
如果第一个参数是 session则在 to_thread 中获取新的 session。
"""
return await asyncio.to_thread(func, *args, **kwargs)

View File

@@ -1,49 +0,0 @@
from __future__ import annotations
import json
from typing import List
from langbot.pkg.rag.knowledge.services import base_service
from langbot.pkg.core import app
from langchain_text_splitters import RecursiveCharacterTextSplitter
class Chunker(base_service.BaseService):
"""
A class for splitting long texts into smaller, overlapping chunks.
"""
def __init__(self, ap: app.Application, chunk_size: int = 500, chunk_overlap: int = 50):
self.ap = ap
self.chunk_size = chunk_size
self.chunk_overlap = chunk_overlap
if self.chunk_overlap >= self.chunk_size:
self.ap.logger.warning(
'Chunk overlap is greater than or equal to chunk size. This may lead to empty or malformed chunks.'
)
def _split_text_sync(self, text: str) -> List[str]:
"""
Synchronously splits a long text into chunks with specified overlap.
This is a CPU-bound operation, intended to be run in a separate thread.
"""
if not text:
return []
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=self.chunk_size,
chunk_overlap=self.chunk_overlap,
length_function=len,
is_separator_regex=False,
)
return text_splitter.split_text(text)
async def chunk(self, text: str) -> List[str]:
"""
Asynchronously chunks a given text into smaller pieces.
"""
self.ap.logger.info(f'Chunking text (length: {len(text)})...')
# Run the synchronous splitting logic in a separate thread
chunks = await self._run_sync(self._split_text_sync, text)
self.ap.logger.info(f'Text chunked into {len(chunks)} pieces.')
self.ap.logger.debug(f'Chunks: {json.dumps(chunks, indent=4, ensure_ascii=False)}')
return chunks

View File

@@ -1,55 +0,0 @@
from __future__ import annotations
import uuid
from typing import List
from langbot.pkg.rag.knowledge.services.base_service import BaseService
from langbot.pkg.entity.persistence import rag as persistence_rag
from langbot.pkg.core import app
from langbot.pkg.provider.modelmgr.requester import RuntimeEmbeddingModel
import sqlalchemy
class Embedder(BaseService):
def __init__(self, ap: app.Application) -> None:
super().__init__()
self.ap = ap
async def embed_and_store(
self, kb_id: str, file_id: str, chunks: List[str], embedding_model: RuntimeEmbeddingModel
) -> list[persistence_rag.Chunk]:
# save chunk to db
chunk_entities: list[persistence_rag.Chunk] = []
chunk_ids: list[str] = []
for chunk_text in chunks:
chunk_uuid = str(uuid.uuid4())
chunk_ids.append(chunk_uuid)
chunk_entity = persistence_rag.Chunk(uuid=chunk_uuid, file_id=file_id, text=chunk_text)
chunk_entities.append(chunk_entity)
chunk_dicts = [
self.ap.persistence_mgr.serialize_model(persistence_rag.Chunk, chunk) for chunk in chunk_entities
]
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.Chunk).values(chunk_dicts))
# get embeddings (batch size limit: 64 for OpenAI)
MAX_BATCH_SIZE = 64
embeddings_list: list[list[float]] = []
for i in range(0, len(chunks), MAX_BATCH_SIZE):
batch = chunks[i : i + MAX_BATCH_SIZE]
batch_embeddings = await embedding_model.provider.invoke_embedding(
model=embedding_model,
input_text=batch,
extra_args={}, # TODO: add extra args
knowledge_base_id=kb_id,
call_type='embedding',
)
embeddings_list.extend(batch_embeddings)
# save embeddings to vdb
await self.ap.vector_db_mgr.vector_db.add_embeddings(kb_id, chunk_ids, embeddings_list, chunk_dicts)
self.ap.logger.info(f'Successfully saved {len(chunk_entities)} embeddings to Knowledge Base.')
return chunk_entities

View File

@@ -1,291 +0,0 @@
from __future__ import annotations
import PyPDF2
import io
from docx import Document
import chardet
from typing import Union, Callable, Any
import markdown
from bs4 import BeautifulSoup
import re
import asyncio # Import asyncio for async operations
from langbot.pkg.core import app
class FileParser:
"""
A robust file parser class to extract text content from various document formats.
It supports TXT, PDF, DOCX, XLSX, CSV, Markdown, HTML, and EPUB files.
All core file reading operations are designed to be run synchronously in a thread pool
to avoid blocking the asyncio event loop.
"""
def __init__(self, ap: app.Application):
self.ap = ap
async def _run_sync(self, sync_func: Callable, *args: Any, **kwargs: Any) -> Any:
"""
Runs a synchronous function in a separate thread to prevent blocking the event loop.
This is a general utility method for wrapping blocking I/O operations.
"""
try:
return await asyncio.to_thread(sync_func, *args, **kwargs)
except Exception as e:
self.ap.logger.error(f'Error running synchronous function {sync_func.__name__}: {e}')
raise
async def parse(self, file_name: str, extension: str) -> Union[str, None]:
"""
Parses the file based on its extension and returns the extracted text content.
This is the main asynchronous entry point for parsing.
Args:
file_name (str): The name of the file to be parsed, get from ap.storage_mgr
Returns:
Union[str, None]: The extracted text content as a single string, or None if parsing fails.
"""
file_extension = extension.lower()
parser_method = getattr(self, f'_parse_{file_extension}', None)
if parser_method is None:
self.ap.logger.error(f'Unsupported file format: {file_extension} for file {file_name}')
return None
try:
# Pass file_path to the specific parser methods
return await parser_method(file_name)
except Exception as e:
self.ap.logger.error(f'Failed to parse {file_extension} file {file_name}: {e}')
return None
# --- Helper for reading files with encoding detection ---
async def _read_file_content(self, file_name: str) -> Union[str, bytes]:
"""
Reads a file with automatic encoding detection, ensuring the synchronous
file read operation runs in a separate thread.
"""
# def _read_sync():
# with open(file_path, 'rb') as file:
# raw_data = file.read()
# detected = chardet.detect(raw_data)
# encoding = detected['encoding'] or 'utf-8'
# if mode == 'r':
# return raw_data.decode(encoding, errors='ignore')
# return raw_data # For binary mode
# return await self._run_sync(_read_sync)
file_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
detected = chardet.detect(file_bytes)
encoding = detected['encoding'] or 'utf-8'
return file_bytes.decode(encoding, errors='ignore')
# --- Specific Parser Methods ---
async def _parse_txt(self, file_name: str) -> str:
"""Parses a TXT file and returns its content."""
self.ap.logger.info(f'Parsing TXT file: {file_name}')
return await self._read_file_content(file_name)
async def _parse_pdf(self, file_name: str) -> str:
"""Parses a PDF file and returns its text content."""
self.ap.logger.info(f'Parsing PDF file: {file_name}')
# def _parse_pdf_sync():
# text_content = []
# with open(file_name, 'rb') as file:
# pdf_reader = PyPDF2.PdfReader(file)
# for page in pdf_reader.pages:
# text = page.extract_text()
# if text:
# text_content.append(text)
# return '\n'.join(text_content)
# return await self._run_sync(_parse_pdf_sync)
pdf_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
def _parse_pdf_sync():
pdf_reader = PyPDF2.PdfReader(io.BytesIO(pdf_bytes))
text_content = []
for page in pdf_reader.pages:
text = page.extract_text()
if text:
text_content.append(text)
return '\n'.join(text_content)
return await self._run_sync(_parse_pdf_sync)
async def _parse_docx(self, file_name: str) -> str:
"""Parses a DOCX file and returns its text content."""
self.ap.logger.info(f'Parsing DOCX file: {file_name}')
docx_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
def _parse_docx_sync():
doc = Document(io.BytesIO(docx_bytes))
text_content = [paragraph.text for paragraph in doc.paragraphs if paragraph.text.strip()]
return '\n'.join(text_content)
return await self._run_sync(_parse_docx_sync)
async def _parse_doc(self, file_name: str) -> str:
"""Handles .doc files, explicitly stating lack of direct support."""
self.ap.logger.warning(f'Direct .doc parsing is not supported for {file_name}. Please convert to .docx first.')
raise NotImplementedError('Direct .doc parsing not supported. Please convert to .docx first.')
# async def _parse_xlsx(self, file_name: str) -> str:
# """Parses an XLSX file, returning text from all sheets."""
# self.ap.logger.info(f'Parsing XLSX file: {file_name}')
# xlsx_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
# def _parse_xlsx_sync():
# excel_file = pd.ExcelFile(io.BytesIO(xlsx_bytes))
# all_sheet_content = []
# for sheet_name in excel_file.sheet_names:
# df = pd.read_excel(io.BytesIO(xlsx_bytes), sheet_name=sheet_name)
# sheet_text = f'--- Sheet: {sheet_name} ---\n{df.to_string(index=False)}\n'
# all_sheet_content.append(sheet_text)
# return '\n'.join(all_sheet_content)
# return await self._run_sync(_parse_xlsx_sync)
# async def _parse_csv(self, file_name: str) -> str:
# """Parses a CSV file and returns its content as a string."""
# self.ap.logger.info(f'Parsing CSV file: {file_name}')
# csv_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
# def _parse_csv_sync():
# # pd.read_csv can often detect encoding, but explicit detection is safer
# # raw_data = self._read_file_content(
# # file_name, mode='rb'
# # ) # Note: this will need to be await outside this sync function
# # _ = raw_data
# # For simplicity, we'll let pandas handle encoding internally after a raw read.
# # A more robust solution might pass encoding directly to pd.read_csv after detection.
# detected = chardet.detect(io.BytesIO(csv_bytes))
# encoding = detected['encoding'] or 'utf-8'
# df = pd.read_csv(io.BytesIO(csv_bytes), encoding=encoding)
# return df.to_string(index=False)
# return await self._run_sync(_parse_csv_sync)
async def _parse_md(self, file_name: str) -> str:
"""Parses a Markdown file, converting it to structured plain text."""
self.ap.logger.info(f'Parsing Markdown file: {file_name}')
md_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
def _parse_markdown_sync():
md_content = io.BytesIO(md_bytes).read().decode('utf-8', errors='ignore')
html_content = markdown.markdown(
md_content, extensions=['extra', 'codehilite', 'tables', 'toc', 'fenced_code']
)
soup = BeautifulSoup(html_content, 'html.parser')
text_parts = []
for element in soup.children:
if element.name in ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']:
level = int(element.name[1])
text_parts.append('#' * level + ' ' + element.get_text().strip())
elif element.name == 'p':
text = element.get_text().strip()
if text:
text_parts.append(text)
elif element.name in ['ul', 'ol']:
for li in element.find_all('li'):
text_parts.append(f'* {li.get_text().strip()}')
elif element.name == 'pre':
code_block = element.get_text().strip()
if code_block:
text_parts.append(f'```\n{code_block}\n```')
elif element.name == 'table':
table_str = self._extract_table_to_markdown_sync(element) # Call sync helper
if table_str:
text_parts.append(table_str)
elif element.name:
text = element.get_text(separator=' ', strip=True)
if text:
text_parts.append(text)
cleaned_text = re.sub(r'\n\s*\n', '\n\n', '\n'.join(text_parts))
return cleaned_text.strip()
return await self._run_sync(_parse_markdown_sync)
async def _parse_html(self, file_name: str) -> str:
"""Parses an HTML file, extracting structured plain text."""
self.ap.logger.info(f'Parsing HTML file: {file_name}')
html_bytes = await self.ap.storage_mgr.storage_provider.load(file_name)
def _parse_html_sync():
html_content = io.BytesIO(html_bytes).read().decode('utf-8', errors='ignore')
soup = BeautifulSoup(html_content, 'html.parser')
for script_or_style in soup(['script', 'style']):
script_or_style.decompose()
text_parts = []
for element in soup.body.children if soup.body else soup.children:
if element.name in ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']:
level = int(element.name[1])
text_parts.append('#' * level + ' ' + element.get_text().strip())
elif element.name == 'p':
text = element.get_text().strip()
if text:
text_parts.append(text)
elif element.name in ['ul', 'ol']:
for li in element.find_all('li'):
text = li.get_text().strip()
if text:
text_parts.append(f'* {text}')
elif element.name == 'table':
table_str = self._extract_table_to_markdown_sync(element) # Call sync helper
if table_str:
text_parts.append(table_str)
elif element.name:
text = element.get_text(separator=' ', strip=True)
if text:
text_parts.append(text)
cleaned_text = re.sub(r'\n\s*\n', '\n\n', '\n'.join(text_parts))
return cleaned_text.strip()
return await self._run_sync(_parse_html_sync)
def _add_toc_items_sync(self, toc_list: list, text_content: list, level: int):
"""Recursively adds TOC items to text_content (synchronous helper)."""
indent = ' ' * level
for item in toc_list:
if isinstance(item, tuple):
chapter, subchapters = item
text_content.append(f'{indent}- {chapter.title}')
self._add_toc_items_sync(subchapters, text_content, level + 1)
else:
text_content.append(f'{indent}- {item.title}')
def _extract_table_to_markdown_sync(self, table_element: BeautifulSoup) -> str:
"""Helper to convert a BeautifulSoup table element into a Markdown table string (synchronous)."""
headers = [th.get_text().strip() for th in table_element.find_all('th')]
rows = []
for tr in table_element.find_all('tr'):
cells = [td.get_text().strip() for td in tr.find_all('td')]
if cells:
rows.append(cells)
if not headers and not rows:
return ''
table_lines = []
if headers:
table_lines.append(' | '.join(headers))
table_lines.append(' | '.join(['---'] * len(headers)))
for row_cells in rows:
padded_cells = row_cells + [''] * (len(headers) - len(row_cells)) if headers else row_cells
table_lines.append(' | '.join(padded_cells))
return '\n'.join(table_lines)

View File

@@ -1,53 +0,0 @@
from __future__ import annotations
from . import base_service
from ....core import app
from ....provider.modelmgr.requester import RuntimeEmbeddingModel
from langbot_plugin.api.entities.builtin.rag import context as rag_context
from langbot_plugin.api.entities.builtin.provider.message import ContentElement
class Retriever(base_service.BaseService):
def __init__(self, ap: app.Application):
super().__init__()
self.ap = ap
async def retrieve(
self, kb_id: str, query: str, embedding_model: RuntimeEmbeddingModel, k: int = 5
) -> list[rag_context.RetrievalResultEntry]:
self.ap.logger.info(
f"Retrieving for query: '{query[:10]}' with k={k} using {embedding_model.model_entity.uuid}"
)
query_embedding: list[float] = await embedding_model.provider.invoke_embedding(
model=embedding_model,
input_text=[query],
extra_args={}, # TODO: add extra args
knowledge_base_id=kb_id,
query_text=query,
call_type='retrieve',
)
vector_results = await self.ap.vector_db_mgr.vector_db.search(kb_id, query_embedding[0], k)
# 'ids' shape mirrors the Chroma-style response contract for compatibility
matched_vector_ids = vector_results.get('ids', [[]])[0]
distances = vector_results.get('distances', [[]])[0]
vector_metadatas = vector_results.get('metadatas', [[]])[0]
if not matched_vector_ids:
self.ap.logger.info('No relevant chunks found in vector database.')
return []
result: list[rag_context.RetrievalResultEntry] = []
for i, id in enumerate(matched_vector_ids):
entry = rag_context.RetrievalResultEntry(
id=id,
content=[ContentElement.from_text(vector_metadatas[i].get('text', ''))],
metadata=vector_metadatas[i],
distance=distances[i],
)
result.append(entry)
return result

View File

@@ -0,0 +1 @@
from .runtime import RAGRuntimeService as RAGRuntimeService

View File

@@ -0,0 +1,89 @@
from __future__ import annotations
import posixpath
from typing import Any
from langbot.pkg.core import app
class RAGRuntimeService:
"""Service to handle RAG-related requests from plugins (Runtime).
This service acts as the bridge between plugin RPC requests and
LangBot's infrastructure (embedding models, vector databases, file storage).
"""
def __init__(self, ap: app.Application):
self.ap = ap
async def vector_upsert(
self,
collection_id: str,
vectors: list[list[float]],
ids: list[str],
metadata: list[dict[str, Any]] | None = None,
documents: list[str] | None = None,
) -> None:
"""Handle VECTOR_UPSERT action."""
metadatas = metadata if metadata else [{} for _ in vectors]
await self.ap.vector_db_mgr.upsert(
collection_name=collection_id,
vectors=vectors,
ids=ids,
metadata=metadatas,
documents=documents,
)
async def vector_search(
self,
collection_id: str,
query_vector: list[float],
top_k: int,
filters: dict[str, Any] | None = None,
search_type: str = 'vector',
query_text: str = '',
) -> list[dict[str, Any]]:
"""Handle VECTOR_SEARCH action."""
return await self.ap.vector_db_mgr.search(
collection_name=collection_id,
query_vector=query_vector,
limit=top_k,
filter=filters,
search_type=search_type,
query_text=query_text,
)
async def vector_delete(
self, collection_id: str, file_ids: list[str] | None = None, filters: dict[str, Any] | None = None
) -> int:
"""Handle VECTOR_DELETE action.
Deletes vectors associated with the given file IDs from the collection.
Each file_id corresponds to a document whose vectors will be removed.
Args:
collection_id: The collection to delete from.
file_ids: File IDs whose associated vectors should be deleted.
Each file_id maps to a set of vectors stored with that file_id
in their metadata.
filters: Filter-based deletion (not yet supported, will raise).
"""
count = 0
if file_ids:
await self.ap.vector_db_mgr.delete_by_file_id(collection_name=collection_id, file_ids=file_ids)
count = len(file_ids)
elif filters:
count = await self.ap.vector_db_mgr.delete_by_filter(collection_name=collection_id, filter=filters)
return count
async def get_file_stream(self, storage_path: str) -> bytes:
"""Handle GET_KNOWLEDEGE_FILE_STREAM action.
Uses the storage manager abstraction to load file content,
regardless of the underlying storage provider.
"""
# Validate storage_path to prevent path traversal
normalized = posixpath.normpath(storage_path)
if normalized.startswith('/') or '..' in normalized.split('/'):
raise ValueError('Invalid storage path')
content_bytes = await self.ap.storage_mgr.storage_provider.load(normalized)
return content_bytes if content_bytes else b''

View File

@@ -3,7 +3,7 @@ from __future__ import annotations
from ..core import app
from . import provider
from .providers import localstorage, s3storage
from .providers import localstorage
class StorageMgr:
@@ -21,6 +21,8 @@ class StorageMgr:
storage_type = storage_config.get('use', 'local')
if storage_type == 's3':
from .providers import s3storage
self.storage_provider = s3storage.S3StorageProvider(self.ap)
self.ap.logger.info('Initialized S3 storage backend.')
else:

View File

@@ -43,6 +43,13 @@ class StorageProvider(abc.ABC):
):
pass
@abc.abstractmethod
async def size(
self,
key: str,
) -> int:
pass
@abc.abstractmethod
async def delete_dir_recursive(
self,

View File

@@ -47,6 +47,12 @@ class LocalStorageProvider(provider.StorageProvider):
):
os.remove(os.path.join(LOCAL_STORAGE_PATH, f'{key}'))
async def size(
self,
key: str,
) -> int:
return os.path.getsize(os.path.join(LOCAL_STORAGE_PATH, f'{key}'))
async def delete_dir_recursive(
self,
dir_path: str,

View File

@@ -117,6 +117,21 @@ class S3StorageProvider(provider.StorageProvider):
self.ap.logger.error(f'Failed to delete from S3: {e}')
raise
async def size(
self,
key: str,
) -> int:
"""Get object size from S3 without downloading it"""
try:
response = self.s3_client.head_object(
Bucket=self.bucket_name,
Key=key,
)
return response['ContentLength']
except Exception as e:
self.ap.logger.error(f'Failed to get size from S3: {e}')
raise
async def delete_dir_recursive(
self,
dir_path: str,

View File

@@ -0,0 +1 @@
"""Survey module for in-product surveys triggered by events."""

View File

@@ -0,0 +1,148 @@
"""Survey manager: tracks events, communicates with Space to fetch/submit surveys."""
from __future__ import annotations
import asyncio
import json
import typing
import httpx
import sqlalchemy
from ..core import app as core_app
from ..entity.persistence.metadata import Metadata
from ..utils import constants
SURVEY_TRIGGERED_KEY = 'survey_triggered_events'
class SurveyManager:
"""Manages survey lifecycle: event tracking, pending survey fetch, submission."""
def __init__(self, ap: core_app.Application):
self.ap = ap
self._triggered_events: set[str] = set()
self._pending_survey: typing.Optional[dict] = None
self._space_url: str = ''
async def initialize(self):
space_config = self.ap.instance_config.data.get('space', {})
self._space_url = space_config.get('url', '').rstrip('/')
await self._load_triggered_events()
async def _load_triggered_events(self):
"""Load previously triggered events from metadata table."""
try:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == SURVEY_TRIGGERED_KEY)
)
row = result.first()
if row:
self._triggered_events = set(json.loads(row[0].value))
except Exception:
self._triggered_events = set()
async def _save_triggered_events(self):
"""Persist triggered events to metadata table."""
try:
value = json.dumps(list(self._triggered_events))
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(Metadata).where(Metadata.key == SURVEY_TRIGGERED_KEY)
)
if result.first():
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(Metadata).where(Metadata.key == SURVEY_TRIGGERED_KEY).values(value=value)
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(Metadata).values(key=SURVEY_TRIGGERED_KEY, value=value)
)
except Exception as e:
self.ap.logger.debug(f'Failed to save survey triggered events: {e}')
def _is_space_configured(self) -> bool:
space_config = self.ap.instance_config.data.get('space', {})
if space_config.get('disable_telemetry', False):
return False
return bool(self._space_url)
async def trigger_event(self, event: str):
"""Called when an event occurs. Checks Space for a pending survey."""
if event in self._triggered_events:
return
if not self._is_space_configured():
return
self._triggered_events.add(event)
await self._save_triggered_events()
# Check for pending survey asynchronously
asyncio.create_task(self._fetch_pending_survey(event))
async def _fetch_pending_survey(self, event: str):
"""Fetch pending survey from Space for this event."""
try:
url = f'{self._space_url}/api/v1/survey/pending'
payload = {
'instance_id': constants.instance_id,
'event': event,
}
async with httpx.AsyncClient(timeout=httpx.Timeout(10)) as client:
resp = await client.post(url, json=payload)
if resp.status_code == 200:
data = resp.json()
if data.get('code') == 0 and data.get('data', {}).get('survey'):
self._pending_survey = data['data']['survey']
self.ap.logger.info(f'Survey pending: {self._pending_survey.get("survey_id")}')
except Exception as e:
self.ap.logger.debug(f'Failed to fetch pending survey: {e}')
def get_pending_survey(self) -> typing.Optional[dict]:
"""Return the current pending survey (if any) for the frontend to display."""
return self._pending_survey
def clear_pending_survey(self):
"""Clear the pending survey (after user responds or dismisses)."""
self._pending_survey = None
async def submit_response(self, survey_id: str, answers: dict, completed: bool = True) -> bool:
"""Submit a survey response to Space."""
if not self._is_space_configured():
return False
try:
url = f'{self._space_url}/api/v1/survey/respond'
payload = {
'survey_id': survey_id,
'instance_id': constants.instance_id,
'answers': answers,
'metadata': {
'version': constants.semantic_version,
},
'completed': completed,
}
async with httpx.AsyncClient(timeout=httpx.Timeout(10)) as client:
resp = await client.post(url, json=payload)
if resp.status_code == 200:
self.clear_pending_survey()
return True
except Exception as e:
self.ap.logger.warning(f'Failed to submit survey response: {e}')
return False
async def dismiss_survey(self, survey_id: str) -> bool:
"""Dismiss a survey."""
if not self._is_space_configured():
return False
try:
url = f'{self._space_url}/api/v1/survey/dismiss'
payload = {
'survey_id': survey_id,
'instance_id': constants.instance_id,
}
async with httpx.AsyncClient(timeout=httpx.Timeout(10)) as client:
resp = await client.post(url, json=payload)
if resp.status_code == 200:
self.clear_pending_survey()
return True
except Exception as e:
self.ap.logger.warning(f'Failed to dismiss survey: {e}')
return False

View File

@@ -60,7 +60,7 @@ class TelemetryManager:
except Exception:
sanitized['query_id'] = str(sanitized.get('query_id', ''))
for sfield in ('adapter', 'runner', 'model_name', 'version', 'error', 'timestamp'):
for sfield in ('adapter', 'runner', 'runner_category', 'model_name', 'version', 'error', 'timestamp'):
v = sanitized.get(sfield)
sanitized[sfield] = '' if v is None else str(v)

View File

@@ -2,7 +2,7 @@ import langbot
semantic_version = f'v{langbot.__version__}'
required_database_version = 18
required_database_version = 20
"""Tag the version of the database schema, used to check if the database needs to be migrated"""
debug_mode = False

View File

@@ -0,0 +1,43 @@
"""Shared aiohttp.ClientSession to avoid repeated SSL context creation.
Each call to `aiohttp.ClientSession()` creates a new `TCPConnector` which in turn
creates a new `ssl.SSLContext` and loads all system root certificates. This is
extremely expensive in both CPU and memory (~270MB total allocations observed via
memray profiling).
This module provides a shared session pool so that all HTTP client code in LangBot
reuses the same underlying SSL context and connection pool.
"""
from __future__ import annotations
import aiohttp
_sessions: dict[str, aiohttp.ClientSession] = {}
def get_session(*, trust_env: bool = False) -> aiohttp.ClientSession:
"""Get or create a shared aiohttp.ClientSession.
Args:
trust_env: Whether to trust environment variables for proxy settings.
Returns:
A shared aiohttp.ClientSession instance.
"""
key = f'trust_env={trust_env}'
session = _sessions.get(key)
if session is None or session.closed:
session = aiohttp.ClientSession(trust_env=trust_env)
_sessions[key] = session
return session
async def close_all():
"""Close all shared sessions. Call on application shutdown."""
for session in _sessions.values():
if not session.closed:
await session.close()
_sessions.clear()

View File

@@ -5,6 +5,8 @@ from urllib.parse import urlparse, parse_qs
import ssl
import aiohttp
from langbot.pkg.utils import httpclient
import PIL.Image
import httpx
@@ -47,53 +49,54 @@ async def get_gewechat_image_base64(
)
try:
async with aiohttp.ClientSession(timeout=timeout) as session:
# 获取图片下载链接
try:
async with session.post(
f'{gewechat_url}/v2/api/message/downloadImage',
headers=headers,
json={'appId': app_id, 'type': image_type, 'xml': xml_content},
) as response:
if response.status != 200:
# print(response)
raise Exception(f'获取gewechat图片下载失败: {await response.text()}')
session = httpclient.get_session()
# 获取图片下载链接
try:
async with session.post(
f'{gewechat_url}/v2/api/message/downloadImage',
headers=headers,
json={'appId': app_id, 'type': image_type, 'xml': xml_content},
timeout=timeout,
) as response:
if response.status != 200:
# print(response)
raise Exception(f'获取gewechat图片下载失败: {await response.text()}')
resp_data = await response.json()
if resp_data.get('ret') != 200:
raise Exception(f'获取gewechat图片下载链接失败: {resp_data}')
resp_data = await response.json()
if resp_data.get('ret') != 200:
raise Exception(f'获取gewechat图片下载链接失败: {resp_data}')
file_url = resp_data['data']['fileUrl']
except asyncio.TimeoutError:
raise Exception('获取图片下载链接超时')
except aiohttp.ClientError as e:
raise Exception(f'获取图片下载链接网络错误: {str(e)}')
file_url = resp_data['data']['fileUrl']
except asyncio.TimeoutError:
raise Exception('获取图片下载链接超时')
except aiohttp.ClientError as e:
raise Exception(f'获取图片下载链接网络错误: {str(e)}')
# 解析原始URL并替换端口
base_url = gewechat_file_url
download_url = f'{base_url}/download/{file_url}'
# 解析原始URL并替换端口
base_url = gewechat_file_url
download_url = f'{base_url}/download/{file_url}'
# 下载图片
try:
async with session.get(download_url) as img_response:
if img_response.status != 200:
raise Exception(f'下载图片失败: {await img_response.text()}, URL: {download_url}')
# 下载图片
try:
async with session.get(download_url) as img_response:
if img_response.status != 200:
raise Exception(f'下载图片失败: {await img_response.text()}, URL: {download_url}')
image_data = await img_response.read()
image_data = await img_response.read()
content_type = img_response.headers.get('Content-Type', '')
if content_type:
image_format = content_type.split('/')[-1]
else:
image_format = file_url.split('.')[-1]
content_type = img_response.headers.get('Content-Type', '')
if content_type:
image_format = content_type.split('/')[-1]
else:
image_format = file_url.split('.')[-1]
base64_str = base64.b64encode(image_data).decode('utf-8')
base64_str = base64.b64encode(image_data).decode('utf-8')
return base64_str, image_format
except asyncio.TimeoutError:
raise Exception(f'下载图片超时, URL: {download_url}')
except aiohttp.ClientError as e:
raise Exception(f'下载图片网络错误: {str(e)}, URL: {download_url}')
return base64_str, image_format
except asyncio.TimeoutError:
raise Exception(f'下载图片超时, URL: {download_url}')
except aiohttp.ClientError as e:
raise Exception(f'下载图片网络错误: {str(e)}, URL: {download_url}')
except Exception as e:
raise Exception(f'获取图片失败: {str(e)}') from e
@@ -104,24 +107,24 @@ async def get_wecom_image_base64(pic_url: str) -> tuple[str, str]:
:param pic_url: 企业微信图片URL
:return: (base64_str, image_format)
"""
async with aiohttp.ClientSession() as session:
async with session.get(pic_url) as response:
if response.status != 200:
raise Exception(f'Failed to download image: {response.status}')
session = httpclient.get_session()
async with session.get(pic_url) as response:
if response.status != 200:
raise Exception(f'Failed to download image: {response.status}')
# 读取图片数据
image_data = await response.read()
# 读取图片数据
image_data = await response.read()
# 获取图片格式
content_type = response.headers.get('Content-Type', '')
image_format = content_type.split('/')[-1] # 例如 'image/jpeg' -> 'jpeg'
# 获取图片格式
content_type = response.headers.get('Content-Type', '')
image_format = content_type.split('/')[-1] # 例如 'image/jpeg' -> 'jpeg'
# 转换为 base64
import base64
# 转换为 base64
import base64
image_base64 = base64.b64encode(image_data).decode('utf-8')
image_base64 = base64.b64encode(image_data).decode('utf-8')
return image_base64, image_format
return image_base64, image_format
async def get_qq_official_image_base64(pic_url: str, content_type: str) -> tuple[str, str]:
@@ -152,21 +155,19 @@ async def get_qq_image_bytes(image_url: str, query: dict = {}) -> tuple[bytes, s
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
async with aiohttp.ClientSession(trust_env=False) as session:
async with session.get(
image_url, params=query, ssl=ssl_context, timeout=aiohttp.ClientTimeout(total=30.0)
) as resp:
resp.raise_for_status()
file_bytes = await resp.read()
content_type = resp.headers.get('Content-Type')
if not content_type:
image_format = 'jpeg'
elif not content_type.startswith('image/'):
pil_img = PIL.Image.open(io.BytesIO(file_bytes))
image_format = pil_img.format.lower()
else:
image_format = content_type.split('/')[-1]
return file_bytes, image_format
session = httpclient.get_session()
async with session.get(image_url, params=query, ssl=ssl_context, timeout=aiohttp.ClientTimeout(total=30.0)) as resp:
resp.raise_for_status()
file_bytes = await resp.read()
content_type = resp.headers.get('Content-Type')
if not content_type:
image_format = 'jpeg'
elif not content_type.startswith('image/'):
pil_img = PIL.Image.open(io.BytesIO(file_bytes))
image_format = pil_img.format.lower()
else:
image_format = content_type.split('/')[-1]
return file_bytes, image_format
async def qq_image_url_to_base64(image_url: str) -> typing.Tuple[str, str]:
@@ -204,11 +205,11 @@ async def extract_b64_and_format(image_base64_data: str) -> typing.Tuple[str, st
async def get_slack_image_to_base64(pic_url: str, bot_token: str):
headers = {'Authorization': f'Bearer {bot_token}'}
try:
async with aiohttp.ClientSession() as session:
async with session.get(pic_url, headers=headers) as resp:
mime_type = resp.headers.get('Content-Type', 'application/octet-stream')
file_bytes = await resp.read()
base64_str = base64.b64encode(file_bytes).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
session = httpclient.get_session()
async with session.get(pic_url, headers=headers) as resp:
mime_type = resp.headers.get('Content-Type', 'application/octet-stream')
file_bytes = await resp.read()
base64_str = base64.b64encode(file_bytes).decode('utf-8')
return f'data:{mime_type};base64,{base64_str}'
except Exception as e:
raise (e)

View File

@@ -0,0 +1,105 @@
from __future__ import annotations
from urllib.parse import urlparse
class RunnerCategory:
LOCAL = 'local'
CLOUD = 'cloud'
UNKNOWN = 'unknown'
CLOUD_DOMAINS = [
'.n8n.cloud',
'.n8n.io',
'api.dify.ai',
'cloud.dify.ai',
'.coze.com',
'.coze.cn',
'cloud.langflow.ai',
'.langflow.org',
]
LOCAL_PATTERNS = [
'localhost',
'127.0.0.1',
'0.0.0.0',
'192.168.',
'10.',
'172.16.',
'172.17.',
'172.18.',
'172.19.',
'172.20.',
'172.21.',
'172.22.',
'172.23.',
'172.24.',
'172.25.',
'172.26.',
'172.27.',
'172.28.',
'172.29.',
'172.30.',
'172.31.',
]
def get_runner_category(runner_name: str, runner_url: str) -> str:
if not runner_url:
return RunnerCategory.UNKNOWN
try:
parsed_url = urlparse(runner_url)
host = parsed_url.hostname.lower() if parsed_url.hostname else ''
except Exception:
return RunnerCategory.UNKNOWN
for pattern in LOCAL_PATTERNS:
if host.startswith(pattern):
return RunnerCategory.LOCAL
for domain in CLOUD_DOMAINS:
if host.endswith(domain):
return RunnerCategory.CLOUD
return RunnerCategory.CLOUD
def get_runner_info(runner_name: str, runner_url: str) -> dict:
return {
'name': runner_name,
'url': runner_url,
'category': get_runner_category(runner_name, runner_url),
}
def is_cloud_runner(runner_name: str, runner_url: str) -> bool:
return get_runner_category(runner_name, runner_url) == RunnerCategory.CLOUD
def is_local_runner(runner_name: str, runner_url: str) -> bool:
return get_runner_category(runner_name, runner_url) == RunnerCategory.LOCAL
def extract_runner_url(runner_name: str, runner, pipeline_config: dict | None) -> str | None:
if not runner or not hasattr(runner, 'pipeline_config'):
return None
ai_config = pipeline_config.get('ai', {}) if pipeline_config else {}
if runner_name == 'dify-service-api':
return ai_config.get('dify-service-api', {}).get('base-url')
elif runner_name == 'n8n-service-api':
return ai_config.get('n8n-service-api', {}).get('webhook-url')
elif runner_name == 'coze-api':
return ai_config.get('coze-api', {}).get('api-base')
elif runner_name == 'langflow-api':
return ai_config.get('langflow-api', {}).get('base-url')
return None
def get_runner_category_from_runner(runner_name: str, runner, pipeline_config: dict | None) -> str:
runner_url = extract_runner_url(runner_name, runner, pipeline_config)
return get_runner_category(runner_name, runner_url)

View File

@@ -0,0 +1,69 @@
"""Shared utilities for metadata filter handling across VDB backends.
Canonical filter format (Chroma-style ``where`` syntax):
{"file_id": "abc"} # implicit $eq
{"file_id": {"$eq": "abc"}} # explicit $eq
{"created_at": {"$gte": 1700000000}} # comparison
{"file_type": {"$in": ["pdf", "docx"]}} # in-list
Multiple top-level keys are AND-ed. Supported operators:
``$eq``, ``$ne``, ``$gt``, ``$gte``, ``$lt``, ``$lte``, ``$in``, ``$nin``.
"""
from __future__ import annotations
import logging
from typing import Any
SUPPORTED_OPS = frozenset({'$eq', '$ne', '$gt', '$gte', '$lt', '$lte', '$in', '$nin'})
logger = logging.getLogger(__name__)
def normalize_filter(
raw: dict[str, Any] | None,
) -> list[tuple[str, str, Any]]:
"""Parse a canonical filter dict into ``[(field, op, value)]`` triples.
Returns an empty list when *raw* is ``None`` or empty.
Raises ``ValueError`` on unsupported operators or malformed entries.
"""
if not raw:
return []
triples: list[tuple[str, str, Any]] = []
for field, condition in raw.items():
if isinstance(condition, dict):
for op, value in condition.items():
if op not in SUPPORTED_OPS:
raise ValueError(f'Unsupported filter operator: {op}')
triples.append((field, op, value))
else:
# Bare value -> implicit $eq
triples.append((field, '$eq', condition))
return triples
def strip_unsupported_fields(
triples: list[tuple[str, str, Any]],
supported_fields: set[str],
) -> list[tuple[str, str, Any]]:
"""Return only triples whose field is in *supported_fields*.
Dropped fields are logged at WARNING level so the caller knows they were
silently ignored (useful for Milvus / pgvector which only store a fixed
schema).
"""
kept: list[tuple[str, str, Any]] = []
for field, op, value in triples:
if field in supported_fields:
kept.append((field, op, value))
else:
logger.warning(
'Filter field %r is not supported by this backend and will be ignored (supported: %s)',
field,
', '.join(sorted(supported_fields)),
)
return kept

View File

@@ -1,7 +1,7 @@
from __future__ import annotations
from ..core import app
from .vdb import VectorDatabase
from .vdb import VectorDatabase, SearchType
from .vdbs.chroma import ChromaVectorDatabase
from .vdbs.qdrant import QdrantVectorDatabase
from .vdbs.seekdb import SeekDBVectorDatabase
@@ -65,3 +65,95 @@ class VectorDBManager:
else:
self.vector_db = ChromaVectorDatabase(self.ap)
self.ap.logger.warning('No vector database backend configured, defaulting to Chroma.')
def get_supported_search_types(self) -> list[str]:
"""Return the search types supported by the current VDB backend."""
if self.vector_db is None:
return [SearchType.VECTOR.value]
return [st.value for st in self.vector_db.supported_search_types()]
async def upsert(
self,
collection_name: str,
vectors: list[list[float]],
ids: list[str],
metadata: list[dict] | None = None,
documents: list[str] | None = None,
):
"""Proxy: Upsert vectors"""
await self.vector_db.add_embeddings(
collection=collection_name,
ids=ids,
embeddings_list=vectors,
metadatas=metadata or [{} for _ in vectors],
documents=documents,
)
async def search(
self,
collection_name: str,
query_vector: list[float],
limit: int,
filter: dict | None = None,
search_type: str = 'vector',
query_text: str = '',
) -> list[dict]:
"""Proxy: Search vectors.
Returns a list of dicts with keys: 'id', 'score', 'metadata'.
The underlying VectorDatabase.search returns Chroma-style format:
{ 'ids': [['id1']], 'distances': [[0.1]], 'metadatas': [[{}]] }
"""
results = await self.vector_db.search(
collection=collection_name,
query_embedding=query_vector,
k=limit,
search_type=search_type,
query_text=query_text,
filter=filter,
)
if not results or 'ids' not in results or not results['ids']:
return []
# Flatten nested lists (Chroma returns batch-style: list of lists)
raw_ids = results['ids']
raw_dists = results.get('distances', [])
raw_metas = results.get('metadatas', [])
r_ids = raw_ids[0] if raw_ids and isinstance(raw_ids[0], list) else raw_ids
r_dists = raw_dists[0] if raw_dists and isinstance(raw_dists[0], list) else raw_dists
r_metas = raw_metas[0] if raw_metas and isinstance(raw_metas[0], list) else raw_metas
parsed_results = []
for i, id_val in enumerate(r_ids):
parsed_results.append(
{
'id': id_val,
'score': r_dists[i] if r_dists and i < len(r_dists) else 0.0,
'metadata': r_metas[i] if r_metas and i < len(r_metas) else {},
}
)
return parsed_results
async def delete_by_file_id(self, collection_name: str, file_ids: list[str]):
"""Proxy: Delete vectors by file_id (metadata-level identifier).
This delegates to VectorDatabase.delete_by_file_id which removes
all vectors associated with the given file IDs.
"""
for file_id in file_ids:
await self.vector_db.delete_by_file_id(collection_name, file_id)
async def delete_collection(self, collection_name: str):
"""Proxy: Delete an entire collection."""
await self.vector_db.delete_collection(collection_name)
async def delete_by_filter(self, collection_name: str, filter: dict) -> int:
"""Proxy: Delete vectors by metadata filter.
Returns:
Number of deleted vectors (best-effort; some backends return 0).
"""
return await self.vector_db.delete_by_filter(collection_name, filter)

View File

@@ -1,10 +1,28 @@
from __future__ import annotations
import abc
import enum
from typing import Any, Dict
import numpy as np
class SearchType(str, enum.Enum):
"""Supported search types for vector databases."""
VECTOR = 'vector'
FULL_TEXT = 'full_text'
HYBRID = 'hybrid'
class VectorDatabase(abc.ABC):
@classmethod
def supported_search_types(cls) -> list[SearchType]:
"""Return the search types supported by this VDB backend.
Default: vector search only. Override in subclasses that support
full-text or hybrid search.
"""
return [SearchType.VECTOR]
@abc.abstractmethod
async def add_embeddings(
self,
@@ -12,14 +30,47 @@ class VectorDatabase(abc.ABC):
ids: list[str],
embeddings_list: list[list[float]],
metadatas: list[dict[str, Any]],
documents: list[str],
documents: list[str] | None = None,
) -> None:
"""Add vector data to the specified collection."""
"""Add vector data to the specified collection.
Args:
collection: Collection name.
ids: Unique IDs for each vector.
embeddings_list: List of embedding vectors.
metadatas: List of metadata dicts.
documents: Optional raw text documents. Required for full-text
and hybrid search in backends that support them.
"""
pass
@abc.abstractmethod
async def search(self, collection: str, query_embedding: np.ndarray, k: int = 5) -> Dict[str, Any]:
"""Search for the most similar vectors in the specified collection."""
async def search(
self,
collection: str,
query_embedding: np.ndarray,
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
) -> Dict[str, Any]:
"""Search for the most similar vectors in the specified collection.
Args:
collection: Collection name.
query_embedding: Query vector for similarity search.
k: Number of results to return.
search_type: One of 'vector', 'full_text', 'hybrid'.
query_text: Raw query text, used for full_text and hybrid search.
filter: Optional metadata filters using Chroma-style ``where``
syntax. Multiple top-level keys are AND-ed. Supported
operators: ``$eq``, ``$ne``, ``$gt``, ``$gte``, ``$lt``,
``$lte``, ``$in``, ``$nin``. Example::
{"file_id": "abc"}
{"created_at": {"$gte": 1700000000}}
{"file_type": {"$in": ["pdf", "docx"]}}
"""
pass
@abc.abstractmethod
@@ -27,6 +78,20 @@ class VectorDatabase(abc.ABC):
"""Delete vectors from the specified collection by file_id."""
pass
@abc.abstractmethod
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
"""Delete vectors matching the given metadata filter.
Args:
collection: Collection name.
filter: Metadata filter dict in canonical format (see ``search``).
Returns:
Number of deleted vectors (best-effort; backends that cannot
report an exact count may return 0).
"""
pass
@abc.abstractmethod
async def get_or_create_collection(self, collection: str):
"""Get or create collection."""

View File

@@ -28,19 +28,33 @@ class ChromaVectorDatabase(VectorDatabase):
ids: list[str],
embeddings_list: list[list[float]],
metadatas: list[dict[str, Any]],
documents: list[str] | None = None,
) -> None:
col = await self.get_or_create_collection(collection)
await asyncio.to_thread(col.add, embeddings=embeddings_list, ids=ids, metadatas=metadatas)
kwargs: dict[str, Any] = dict(embeddings=embeddings_list, ids=ids, metadatas=metadatas)
if documents is not None:
kwargs['documents'] = documents
await asyncio.to_thread(col.add, **kwargs)
self.ap.logger.info(f"Added {len(ids)} embeddings to Chroma collection '{collection}'.")
async def search(self, collection: str, query_embedding: list[float], k: int = 5) -> dict[str, Any]:
async def search(
self,
collection: str,
query_embedding: list[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
) -> dict[str, Any]:
col = await self.get_or_create_collection(collection)
results = await asyncio.to_thread(
col.query,
query_kwargs: dict[str, Any] = dict(
query_embeddings=query_embedding,
n_results=k,
include=['metadatas', 'distances', 'documents'],
)
if filter:
query_kwargs['where'] = filter
results = await asyncio.to_thread(col.query, **query_kwargs)
self.ap.logger.info(f"Chroma search in '{collection}' returned {len(results.get('ids', [[]])[0])} results.")
return results
@@ -49,6 +63,12 @@ class ChromaVectorDatabase(VectorDatabase):
await asyncio.to_thread(col.delete, where={'file_id': file_id})
self.ap.logger.info(f"Deleted embeddings from Chroma collection '{collection}' with file_id: {file_id}")
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
col = await self.get_or_create_collection(collection)
await asyncio.to_thread(col.delete, where=filter)
self.ap.logger.info(f"Deleted embeddings from Chroma collection '{collection}' by filter")
return 0 # Chroma delete does not return a count
async def delete_collection(self, collection: str):
if collection in self._collections:
del self._collections[collection]

View File

@@ -4,8 +4,51 @@ from typing import Any, Dict
from pymilvus import MilvusClient, DataType, CollectionSchema, FieldSchema
from pymilvus.milvus_client.index import IndexParams
from langbot.pkg.vector.vdb import VectorDatabase
from langbot.pkg.vector.filter_utils import normalize_filter, strip_unsupported_fields
from langbot.pkg.core import app
# Milvus schema only stores these metadata fields; filter on other fields is
# silently dropped with a warning.
_MILVUS_SUPPORTED_FIELDS = {'text', 'file_id', 'chunk_uuid'}
def _build_milvus_expr(filter_dict: dict[str, Any]) -> str:
"""Translate canonical filter dict into a Milvus boolean expression string."""
triples = normalize_filter(filter_dict)
triples = strip_unsupported_fields(triples, _MILVUS_SUPPORTED_FIELDS)
if not triples:
return ''
parts: list[str] = []
for field, op, value in triples:
if op == '$eq':
parts.append(f'{field} == {_milvus_literal(value)}')
elif op == '$ne':
parts.append(f'{field} != {_milvus_literal(value)}')
elif op == '$gt':
parts.append(f'{field} > {_milvus_literal(value)}')
elif op == '$gte':
parts.append(f'{field} >= {_milvus_literal(value)}')
elif op == '$lt':
parts.append(f'{field} < {_milvus_literal(value)}')
elif op == '$lte':
parts.append(f'{field} <= {_milvus_literal(value)}')
elif op == '$in':
items = ', '.join(_milvus_literal(v) for v in value)
parts.append(f'{field} in [{items}]')
elif op == '$nin':
items = ', '.join(_milvus_literal(v) for v in value)
parts.append(f'{field} not in [{items}]')
return ' and '.join(parts)
def _milvus_literal(value: Any) -> str:
"""Format a Python value as a Milvus expression literal."""
if isinstance(value, str):
escaped = value.replace('\\', '\\\\').replace('"', '\\"')
return f'"{escaped}"'
return str(value)
class MilvusVectorDatabase(VectorDatabase):
"""Milvus vector database implementation"""
@@ -155,6 +198,7 @@ class MilvusVectorDatabase(VectorDatabase):
ids: list[str],
embeddings_list: list[list[float]],
metadatas: list[dict[str, Any]],
documents: list[str] | None = None,
) -> None:
"""Add vector embeddings to Milvus collection
@@ -200,7 +244,15 @@ class MilvusVectorDatabase(VectorDatabase):
self.ap.logger.info(f"Added {len(ids)} embeddings to Milvus collection '{collection}'")
async def search(self, collection: str, query_embedding: list[float], k: int = 5) -> Dict[str, Any]:
async def search(
self,
collection: str,
query_embedding: list[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
) -> Dict[str, Any]:
"""Search for similar vectors in Milvus collection
Args:
@@ -217,14 +269,19 @@ class MilvusVectorDatabase(VectorDatabase):
# Perform search
search_params = {'metric_type': 'COSINE', 'params': {}}
results = await asyncio.to_thread(
self.client.search,
search_kwargs: dict[str, Any] = dict(
collection_name=collection,
data=[query_embedding],
limit=k,
search_params=search_params,
output_fields=['text', 'file_id', 'chunk_uuid'],
)
if filter:
expr = _build_milvus_expr(filter)
if expr:
search_kwargs['filter'] = expr
results = await asyncio.to_thread(self.client.search, **search_kwargs)
# Convert results to Chroma-compatible format
# Milvus returns: [[ {id, distance, entity: {...}} ]]
@@ -268,6 +325,21 @@ class MilvusVectorDatabase(VectorDatabase):
await asyncio.to_thread(self.client.delete, collection_name=collection, filter=f'file_id == "{file_id}"')
self.ap.logger.info(f"Deleted embeddings from Milvus collection '{collection}' with file_id: {file_id}")
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
collection = self._normalize_collection_name(collection)
await self.get_or_create_collection(collection)
expr = _build_milvus_expr(filter)
if not expr:
self.ap.logger.warning(
f"Milvus delete_by_filter on '{collection}': filter produced empty expression, skipping"
)
return 0
await asyncio.to_thread(self.client.delete, collection_name=collection, filter=expr)
self.ap.logger.info(f"Deleted embeddings from Milvus collection '{collection}' by filter")
return 0 # Milvus delete does not return a count
async def delete_collection(self, collection: str):
"""Delete a Milvus collection

View File

@@ -5,10 +5,21 @@ from sqlalchemy.orm import declarative_base
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
from pgvector.sqlalchemy import Vector
from langbot.pkg.vector.vdb import VectorDatabase
from langbot.pkg.vector.filter_utils import normalize_filter, strip_unsupported_fields
from langbot.pkg.core import app
Base = declarative_base()
# pgvector schema only stores these metadata fields.
_PG_SUPPORTED_FIELDS = {'text', 'file_id', 'chunk_uuid'}
# Map schema field names to SQLAlchemy columns (resolved lazily from PgVectorEntry).
_PG_COLUMN_MAP = {
'text': 'text',
'file_id': 'file_id',
'chunk_uuid': 'chunk_uuid',
}
class PgVectorEntry(Base):
"""SQLAlchemy model for pgvector entries"""
@@ -23,6 +34,33 @@ class PgVectorEntry(Base):
chunk_uuid = Column(String)
def _build_pg_conditions(filter_dict: dict[str, Any]) -> list:
"""Translate canonical filter dict into a list of SQLAlchemy conditions."""
triples = normalize_filter(filter_dict)
triples = strip_unsupported_fields(triples, _PG_SUPPORTED_FIELDS)
conditions = []
for field, op, value in triples:
col = getattr(PgVectorEntry, _PG_COLUMN_MAP[field])
if op == '$eq':
conditions.append(col == value)
elif op == '$ne':
conditions.append(col != value)
elif op == '$gt':
conditions.append(col > value)
elif op == '$gte':
conditions.append(col >= value)
elif op == '$lt':
conditions.append(col < value)
elif op == '$lte':
conditions.append(col <= value)
elif op == '$in':
conditions.append(col.in_(value))
elif op == '$nin':
conditions.append(col.notin_(value))
return conditions
class PgVectorDatabase(VectorDatabase):
"""PostgreSQL with pgvector extension database implementation"""
@@ -109,6 +147,7 @@ class PgVectorDatabase(VectorDatabase):
ids: list[str],
embeddings_list: list[list[float]],
metadatas: list[dict[str, Any]],
documents: list[str] | None = None,
) -> None:
"""Add vector embeddings to pgvector
@@ -142,7 +181,15 @@ class PgVectorDatabase(VectorDatabase):
self.ap.logger.error(f'Error adding embeddings to pgvector: {e}')
raise
async def search(self, collection: str, query_embedding: list[float], k: int = 5) -> Dict[str, Any]:
async def search(
self,
collection: str,
query_embedding: list[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
) -> Dict[str, Any]:
"""Search for similar vectors using cosine distance
Args:
@@ -174,6 +221,10 @@ class PgVectorDatabase(VectorDatabase):
.limit(k)
)
if filter:
for cond in _build_pg_conditions(filter):
stmt = stmt.filter(cond)
result = await session.execute(stmt)
rows = result.fetchall()
@@ -225,6 +276,39 @@ class PgVectorDatabase(VectorDatabase):
self.ap.logger.error(f'Error deleting from pgvector: {e}')
raise
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
"""Delete vectors matching a metadata filter.
Args:
collection: Collection name
filter: Canonical metadata filter dict
"""
conditions = _build_pg_conditions(filter)
if not conditions:
self.ap.logger.warning(
f"pgvector delete_by_filter on '{collection}': filter produced no conditions, skipping"
)
return 0
await self.get_or_create_collection(collection)
async with self.AsyncSessionLocal() as session:
try:
from sqlalchemy import delete
stmt = delete(PgVectorEntry).where(PgVectorEntry.collection == collection)
for cond in conditions:
stmt = stmt.where(cond)
result = await session.execute(stmt)
await session.commit()
deleted = result.rowcount
self.ap.logger.info(f"Deleted {deleted} embeddings from pgvector collection '{collection}' by filter")
return deleted
except Exception as e:
await session.rollback()
self.ap.logger.error(f'Error deleting from pgvector by filter: {e}')
raise
async def delete_collection(self, collection: str):
"""Delete all vectors in a collection

View File

@@ -5,6 +5,37 @@ from typing import Any, Dict, List
from qdrant_client import AsyncQdrantClient, models
from langbot.pkg.core import app
from langbot.pkg.vector.vdb import VectorDatabase
from langbot.pkg.vector.filter_utils import normalize_filter
def _build_qdrant_filter(filter_dict: dict[str, Any]) -> models.Filter:
"""Translate canonical filter dict into a Qdrant ``models.Filter``."""
triples = normalize_filter(filter_dict)
must: list[models.Condition] = []
must_not: list[models.Condition] = []
for field, op, value in triples:
if op == '$eq':
must.append(models.FieldCondition(key=field, match=models.MatchValue(value=value)))
elif op == '$ne':
must_not.append(models.FieldCondition(key=field, match=models.MatchValue(value=value)))
elif op == '$in':
must.append(models.FieldCondition(key=field, match=models.MatchAny(any=value)))
elif op == '$nin':
must_not.append(models.FieldCondition(key=field, match=models.MatchAny(any=value)))
elif op in ('$gt', '$gte', '$lt', '$lte'):
range_kwargs: dict[str, Any] = {}
if op == '$gt':
range_kwargs['gt'] = value
elif op == '$gte':
range_kwargs['gte'] = value
elif op == '$lt':
range_kwargs['lt'] = value
elif op == '$lte':
range_kwargs['lte'] = value
must.append(models.FieldCondition(key=field, range=models.Range(**range_kwargs)))
return models.Filter(must=must or None, must_not=must_not or None)
class QdrantVectorDatabase(VectorDatabase):
@@ -48,6 +79,7 @@ class QdrantVectorDatabase(VectorDatabase):
ids: List[str],
embeddings_list: List[List[float]],
metadatas: List[Dict[str, Any]],
documents: List[str] | None = None,
) -> None:
if not embeddings_list:
return
@@ -60,19 +92,29 @@ class QdrantVectorDatabase(VectorDatabase):
await self.client.upsert(collection_name=collection, points=points)
self.ap.logger.info(f"Added {len(ids)} embeddings to Qdrant collection '{collection}'.")
async def search(self, collection: str, query_embedding: list[float], k: int = 5) -> dict[str, Any]:
async def search(
self,
collection: str,
query_embedding: list[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: dict[str, Any] | None = None,
) -> dict[str, Any]:
exists = await self.client.collection_exists(collection)
if not exists:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
hits = (
await self.client.query_points(
collection_name=collection,
query=query_embedding,
limit=k,
with_payload=True,
)
).points
query_kwargs: dict[str, Any] = dict(
collection_name=collection,
query=query_embedding,
limit=k,
with_payload=True,
)
if filter:
query_kwargs['query_filter'] = _build_qdrant_filter(filter)
hits = (await self.client.query_points(**query_kwargs)).points
ids = [str(hit.id) for hit in hits]
metadatas = [hit.payload or {} for hit in hits]
# Qdrant's score is similarity; convert to a pseudo-distance for consistency
@@ -95,6 +137,19 @@ class QdrantVectorDatabase(VectorDatabase):
)
self.ap.logger.info(f"Deleted embeddings from Qdrant collection '{collection}' with file_id: {file_id}")
async def delete_by_filter(self, collection: str, filter: dict[str, Any]) -> int:
exists = await self.client.collection_exists(collection)
if not exists:
return 0
qdrant_filter = _build_qdrant_filter(filter)
await self.client.delete(
collection_name=collection,
points_selector=qdrant_filter,
)
self.ap.logger.info(f"Deleted embeddings from Qdrant collection '{collection}' by filter")
return 0 # Qdrant delete does not return a count
async def delete_collection(self, collection: str):
try:
await self.client.delete_collection(collection)

View File

@@ -5,7 +5,7 @@ from typing import Any, Dict, List
from langbot.pkg.core import app
from langbot.pkg.vector.vdb import VectorDatabase
from langbot.pkg.vector.vdb import VectorDatabase, SearchType
try:
import pyseekdb
@@ -25,9 +25,13 @@ class SeekDBVectorDatabase(VectorDatabase):
SeekDB is an AI-native search database by OceanBase that unifies
relational, vector, text, JSON and GIS in a single engine.
Supports both embedded mode and remote server mode.
Supports embedded mode, remote server mode, and full-text/hybrid search.
"""
@classmethod
def supported_search_types(cls) -> list[SearchType]:
return [SearchType.VECTOR, SearchType.FULL_TEXT, SearchType.HYBRID]
def __init__(self, ap: app.Application):
if not SEEKDB_AVAILABLE:
raise ImportError('pyseekdb is not installed. Install it with: pip install pyseekdb')
@@ -89,6 +93,7 @@ class SeekDBVectorDatabase(VectorDatabase):
{
'\x00': '',
'\\': '\\\\',
"'": "''", # Standard SQL escaping (OceanBase NO_BACKSLASH_ESCAPES)
'"': '\\"',
'\n': '\\n',
'\r': '\\r',
@@ -111,8 +116,10 @@ class SeekDBVectorDatabase(VectorDatabase):
# Collection doesn't exist, create it
if vector_size is None:
# Default dimension if not specified
vector_size = 384
raise ValueError(
f"Cannot create SeekDB collection '{collection}' without knowing the vector dimension. "
'Ensure add_embeddings is called before any standalone get_or_create_collection.'
)
# Create HNSW configuration
config = HNSWConfiguration(dimension=vector_size, distance='cosine')
@@ -147,7 +154,12 @@ class SeekDBVectorDatabase(VectorDatabase):
return await self._get_or_create_collection_internal(collection)
async def add_embeddings(
self, collection: str, ids: List[str], embeddings_list: List[List[float]], metadatas: List[Dict[str, Any]]
self,
collection: str,
ids: List[str],
embeddings_list: List[List[float]],
metadatas: List[Dict[str, Any]],
documents: List[str] | None = None,
) -> None:
"""Add vector embeddings to the specified collection.
@@ -156,6 +168,7 @@ class SeekDBVectorDatabase(VectorDatabase):
ids: List of document IDs
embeddings_list: List of embedding vectors
metadatas: List of metadata dictionaries
documents: Optional raw text documents for full-text search support
"""
if not embeddings_list:
return
@@ -166,17 +179,33 @@ class SeekDBVectorDatabase(VectorDatabase):
cleaned_metadatas = [self._clean_metadata(meta) for meta in metadatas]
await asyncio.to_thread(coll.add, ids=ids, embeddings=embeddings_list, metadatas=cleaned_metadatas)
kwargs: Dict[str, Any] = dict(ids=ids, embeddings=embeddings_list, metadatas=cleaned_metadatas)
if documents is not None:
kwargs['documents'] = [doc.translate(self._escape_table) for doc in documents]
await asyncio.to_thread(coll.add, **kwargs)
self.ap.logger.info(f"Added {len(ids)} embeddings to SeekDB collection '{collection}'")
async def search(self, collection: str, query_embedding: List[float], k: int = 5) -> Dict[str, Any]:
async def search(
self,
collection: str,
query_embedding: List[float],
k: int = 5,
search_type: str = 'vector',
query_text: str = '',
filter: Dict[str, Any] | None = None,
) -> Dict[str, Any]:
"""Search for the most similar vectors in the specified collection.
SeekDB supports vector, full-text, and hybrid search modes.
Args:
collection: Collection name
query_embedding: Query vector
query_embedding: Query vector (used for vector and hybrid modes)
k: Number of results to return
search_type: One of 'vector', 'full_text', 'hybrid'
query_text: Raw query text (used for full_text and hybrid modes)
filter: Optional metadata filters (Chroma-style ``where`` syntax).
Returns:
Dictionary with 'ids', 'metadatas', 'distances' keys
@@ -193,11 +222,73 @@ class SeekDBVectorDatabase(VectorDatabase):
else:
coll = self._collections[collection]
# Perform query
# SeekDB's query() returns: {'ids': [[...]], 'metadatas': [[...]], 'distances': [[...]]}
results = await asyncio.to_thread(coll.query, query_embeddings=query_embedding, n_results=k)
# Route by search type.
# pyseekdb's query() always requires embeddings, so full-text and
# hybrid modes use hybrid_search() which supports text-only queries
# and returns the same nested-list format with distances.
if search_type == SearchType.FULL_TEXT:
if not query_text:
return {'ids': [[]], 'metadatas': [[]], 'distances': [[]]}
self.ap.logger.info(f"SeekDB search in '{collection}' returned {len(results.get('ids', [[]])[0])} results")
query_cfg: Dict[str, Any] = {
'where_document': {'$contains': query_text},
'n_results': k,
}
if filter:
query_cfg['where'] = filter
# TODO: pyseekdb hybrid_search with query-only (no knn) returns None
# for IDs due to column name mismatch (*/_id vs _id).
# See: https://github.com/oceanbase/pyseekdb/issues/171
results = await asyncio.to_thread(
coll.hybrid_search,
query=query_cfg,
knn=None,
n_results=k,
include=['documents', 'metadatas'],
)
elif search_type == SearchType.HYBRID:
if not query_text:
# Fall back to pure vector search when no text is provided
query_kwargs: Dict[str, Any] = {
'n_results': k,
'query_embeddings': query_embedding,
}
if filter:
query_kwargs['where'] = filter
results = await asyncio.to_thread(coll.query, **query_kwargs)
else:
query_cfg = {
'where_document': {'$contains': query_text},
'n_results': k,
}
knn_cfg: Dict[str, Any] = {
'query_embeddings': query_embedding,
'n_results': k,
}
if filter:
query_cfg['where'] = filter
knn_cfg['where'] = filter
results = await asyncio.to_thread(
coll.hybrid_search,
query=query_cfg,
knn=knn_cfg,
rank={'rrf': {}},
n_results=k,
include=['documents', 'metadatas'],
)
else:
# Default: vector search via query()
query_kwargs = {'n_results': k, 'query_embeddings': query_embedding}
if filter:
query_kwargs['where'] = filter
results = await asyncio.to_thread(coll.query, **query_kwargs)
self.ap.logger.info(
f"SeekDB {search_type} search in '{collection}' returned {len(results.get('ids', [[]])[0])} results"
)
return results
@@ -227,6 +318,28 @@ class SeekDBVectorDatabase(VectorDatabase):
self.ap.logger.info(f"Deleted embeddings from SeekDB collection '{collection}' with file_id: {file_id}")
async def delete_by_filter(self, collection: str, filter: Dict[str, Any]) -> int:
"""Delete vectors from the collection by metadata filter.
Args:
collection: Collection name
filter: Chroma-style ``where`` filter dict
"""
exists = await asyncio.to_thread(self.client.has_collection, collection)
if not exists:
self.ap.logger.warning(f"SeekDB collection '{collection}' not found for deletion")
return 0
if collection not in self._collections:
coll = await asyncio.to_thread(self.client.get_collection, collection, embedding_function=None)
self._collections[collection] = coll
else:
coll = self._collections[collection]
await asyncio.to_thread(coll.delete, where=filter)
self.ap.logger.info(f"Deleted embeddings from SeekDB collection '{collection}' by filter")
return 0 # SeekDB delete does not return a count
async def delete_collection(self, collection: str):
"""Delete the entire collection.

View File

@@ -17,6 +17,10 @@
"prefix": [],
"regexp": []
},
"message-aggregation": {
"enabled": false,
"delay": 1.5
},
"misc": {
"combine-quote-message": true
}

View File

@@ -123,6 +123,34 @@ stages:
type: array[string]
required: true
default: []
- name: message-aggregation
label:
en_US: Message Aggregation
zh_Hans: 消息聚合
description:
en_US: When a user sends multiple messages consecutively, wait for a period and merge them into one before processing
zh_Hans: 当用户连续发送多条消息时,等待一段时间后合并为一条消息再处理(防抖)
config:
- name: enabled
label:
en_US: Enable Message Aggregation
zh_Hans: 启用消息聚合
description:
en_US: If enabled, consecutive messages from the same user will be merged after a delay
zh_Hans: 如果启用,同一用户连续发送的消息将在延迟后合并处理
type: boolean
required: true
default: false
- name: delay
label:
en_US: Aggregation Delay (seconds)
zh_Hans: 聚合延迟(秒)
description:
en_US: 'Wait time before merging messages. Range: 1.0-10.0 seconds.'
zh_Hans: '合并消息前的等待时间。范围1.0-10.0 秒。'
type: float
required: true
default: 1.5
- name: misc
label:
en_US: Misc

View File

@@ -0,0 +1,113 @@
"""Unit tests for config_coercion module"""
from __future__ import annotations
import pytest
from langbot.pkg.pipeline.config_coercion import _coerce_value, coerce_pipeline_config
class TestCoerceValue:
"""Tests for _coerce_value function"""
def test_none_passthrough(self):
assert _coerce_value(None, 'integer') is None
assert _coerce_value(None, 'boolean') is None
def test_string_to_integer(self):
assert _coerce_value('120', 'integer') == 120
assert _coerce_value('0', 'integer') == 0
assert _coerce_value('-5', 'integer') == -5
def test_integer_passthrough(self):
assert _coerce_value(42, 'integer') == 42
def test_string_to_float(self):
assert _coerce_value('3.14', 'number') == 3.14
assert _coerce_value('3.14', 'float') == 3.14
def test_int_to_float(self):
assert _coerce_value(3, 'number') == 3.0
assert isinstance(_coerce_value(3, 'number'), float)
def test_float_passthrough(self):
assert _coerce_value(3.14, 'float') == 3.14
def test_string_to_bool(self):
assert _coerce_value('true', 'boolean') is True
assert _coerce_value('True', 'boolean') is True
assert _coerce_value('false', 'boolean') is False
assert _coerce_value('False', 'boolean') is False
def test_bool_passthrough(self):
assert _coerce_value(True, 'boolean') is True
assert _coerce_value(False, 'boolean') is False
def test_invalid_bool_string_raises(self):
with pytest.raises(ValueError):
_coerce_value('notabool', 'boolean')
def test_unknown_type_passthrough(self):
assert _coerce_value('hello', 'string') == 'hello'
assert _coerce_value('hello', 'unknown') == 'hello'
def test_invalid_integer_raises(self):
with pytest.raises(ValueError):
_coerce_value('abc', 'integer')
class TestCoercePipelineConfig:
"""Tests for coerce_pipeline_config function"""
def _make_meta(self, section_name: str, stage_name: str, fields: list[dict]) -> dict:
return {
'name': section_name,
'stages': [{'name': stage_name, 'config': fields}],
}
def test_coerce_integer_in_config(self):
config = {'trigger': {'misc': {'timeout': '120'}}}
meta = self._make_meta('trigger', 'misc', [{'name': 'timeout', 'type': 'integer'}])
coerce_pipeline_config(config, meta)
assert config['trigger']['misc']['timeout'] == 120
def test_coerce_boolean_in_config(self):
config = {'output': {'misc': {'at-sender': 'true'}}}
meta = self._make_meta('output', 'misc', [{'name': 'at-sender', 'type': 'boolean'}])
coerce_pipeline_config(config, meta)
assert config['output']['misc']['at-sender'] is True
def test_missing_section_skipped(self):
config = {'ai': {}}
meta = self._make_meta('trigger', 'misc', [{'name': 'x', 'type': 'integer'}])
coerce_pipeline_config(config, meta) # should not raise
def test_missing_field_skipped(self):
config = {'trigger': {'misc': {}}}
meta = self._make_meta('trigger', 'misc', [{'name': 'nonexistent', 'type': 'integer'}])
coerce_pipeline_config(config, meta) # should not raise
def test_invalid_value_logs_warning(self, caplog):
config = {'trigger': {'misc': {'timeout': 'abc'}}}
meta = self._make_meta('trigger', 'misc', [{'name': 'timeout', 'type': 'integer'}])
import logging
with caplog.at_level(logging.WARNING):
coerce_pipeline_config(config, meta)
assert config['trigger']['misc']['timeout'] == 'abc' # unchanged
assert 'Failed to coerce' in caplog.text
def test_empty_metadata(self):
config = {'trigger': {'misc': {'timeout': '120'}}}
coerce_pipeline_config(config) # no metadata args, should not raise
def test_multiple_metadata(self):
config = {
'trigger': {'misc': {'timeout': '120'}},
'output': {'misc': {'at-sender': 'false'}},
}
meta_trigger = self._make_meta('trigger', 'misc', [{'name': 'timeout', 'type': 'integer'}])
meta_output = self._make_meta('output', 'misc', [{'name': 'at-sender', 'type': 'boolean'}])
coerce_pipeline_config(config, meta_trigger, meta_output)
assert config['trigger']['misc']['timeout'] == 120
assert config['output']['misc']['at-sender'] is False

View File

@@ -38,13 +38,11 @@ async def test_plugin_list_filter_by_component_kinds():
'manifest': {
'metadata': {
'author': 'author2',
'name': 'plugin_with_knowledge_retriever_only',
'name': 'plugin_with_knowledge_engine_only',
}
}
},
'components': [
{'manifest': {'manifest': {'kind': 'KnowledgeRetriever', 'metadata': {'name': 'retriever1'}}}}
],
'components': [{'manifest': {'manifest': {'kind': 'KnowledgeEngine', 'metadata': {'name': 'retriever1'}}}}],
},
{
'debug': False,
@@ -81,7 +79,7 @@ async def test_plugin_list_filter_by_component_kinds():
}
},
'components': [
{'manifest': {'manifest': {'kind': 'KnowledgeRetriever', 'metadata': {'name': 'retriever2'}}}},
{'manifest': {'manifest': {'kind': 'KnowledgeEngine', 'metadata': {'name': 'retriever2'}}}},
{'manifest': {'manifest': {'kind': 'Tool', 'metadata': {'name': 'tool2'}}}},
],
},
@@ -108,8 +106,8 @@ async def test_plugin_list_filter_by_component_kinds():
assert 'plugin_with_command' in plugin_names
assert 'plugin_with_event_listener' in plugin_names
assert 'plugin_with_mixed_components' in plugin_names
# Plugin with only KnowledgeRetriever should NOT be included
assert 'plugin_with_knowledge_retriever_only' not in plugin_names
# Plugin with only KnowledgeEngine should NOT be included
assert 'plugin_with_knowledge_engine_only' not in plugin_names
@pytest.mark.asyncio
@@ -150,9 +148,7 @@ async def test_plugin_list_filter_no_filter():
}
}
},
'components': [
{'manifest': {'manifest': {'kind': 'KnowledgeRetriever', 'metadata': {'name': 'retriever1'}}}}
],
'components': [{'manifest': {'manifest': {'kind': 'KnowledgeEngine', 'metadata': {'name': 'retriever1'}}}}],
},
]
@@ -189,7 +185,7 @@ async def test_plugin_list_filter_empty_result():
connector = PluginRuntimeConnector(mock_app, AsyncMock())
connector.handler = MagicMock()
# Mock plugin data - only KnowledgeRetriever plugins
# Mock plugin data - only KnowledgeEngine plugins
mock_plugins = [
{
'debug': False,
@@ -201,9 +197,7 @@ async def test_plugin_list_filter_empty_result():
}
}
},
'components': [
{'manifest': {'manifest': {'kind': 'KnowledgeRetriever', 'metadata': {'name': 'retriever1'}}}}
],
'components': [{'manifest': {'manifest': {'kind': 'KnowledgeEngine', 'metadata': {'name': 'retriever1'}}}}],
},
]

473
uv.lock generated
View File

@@ -964,6 +964,30 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/bc/58/6b3d24e6b9bc474a2dcdee65dfd1f008867015408a271562e4b690561a4d/cryptography-46.0.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7", size = 3407605, upload-time = "2026-02-10T19:18:29.233Z" },
]
[[package]]
name = "cuda-bindings"
version = "12.9.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cuda-pathfinder", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/45/e7/b47792cc2d01c7e1d37c32402182524774dadd2d26339bd224e0e913832e/cuda_bindings-12.9.4-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c912a3d9e6b6651853eed8eed96d6800d69c08e94052c292fec3f282c5a817c9", size = 12210593, upload-time = "2025-10-21T14:51:36.574Z" },
{ url = "https://files.pythonhosted.org/packages/a9/c1/dabe88f52c3e3760d861401bb994df08f672ec893b8f7592dc91626adcf3/cuda_bindings-12.9.4-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fda147a344e8eaeca0c6ff113d2851ffca8f7dfc0a6c932374ee5c47caa649c8", size = 12151019, upload-time = "2025-10-21T14:51:43.167Z" },
{ url = "https://files.pythonhosted.org/packages/63/56/e465c31dc9111be3441a9ba7df1941fe98f4aa6e71e8788a3fb4534ce24d/cuda_bindings-12.9.4-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:32bdc5a76906be4c61eb98f546a6786c5773a881f3b166486449b5d141e4a39f", size = 11906628, upload-time = "2025-10-21T14:51:49.905Z" },
{ url = "https://files.pythonhosted.org/packages/a3/84/1e6be415e37478070aeeee5884c2022713c1ecc735e6d82d744de0252eee/cuda_bindings-12.9.4-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:56e0043c457a99ac473ddc926fe0dc4046694d99caef633e92601ab52cbe17eb", size = 11925991, upload-time = "2025-10-21T14:51:56.535Z" },
{ url = "https://files.pythonhosted.org/packages/d1/af/6dfd8f2ed90b1d4719bc053ff8940e494640fe4212dc3dd72f383e4992da/cuda_bindings-12.9.4-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8b72ee72a9cc1b531db31eebaaee5c69a8ec3500e32c6933f2d3b15297b53686", size = 11922703, upload-time = "2025-10-21T14:52:03.585Z" },
{ url = "https://files.pythonhosted.org/packages/6c/19/90ac264acc00f6df8a49378eedec9fd2db3061bf9263bf9f39fd3d8377c3/cuda_bindings-12.9.4-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d80bffc357df9988dca279734bc9674c3934a654cab10cadeed27ce17d8635ee", size = 11924658, upload-time = "2025-10-21T14:52:10.411Z" },
]
[[package]]
name = "cuda-pathfinder"
version = "1.4.1"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/07/02/59a5bc738a09def0b49aea0e460bdf97f65206d0d041246147cf6207e69c/cuda_pathfinder-1.4.1-py3-none-any.whl", hash = "sha256:40793006082de88e0950753655e55558a446bed9a7d9d0bcb48b2506d50ed82a", size = 43903, upload-time = "2026-03-06T21:05:24.372Z" },
]
[[package]]
name = "dashscope"
version = "1.25.10"
@@ -1729,6 +1753,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/14/2f/967ba146e6d58cf6a652da73885f52fc68001525b4197effc174321d70b4/jmespath-1.1.0-py3-none-any.whl", hash = "sha256:a5663118de4908c91729bea0acadca56526eb2698e83de10cd116ae0f4e97c64", size = 20419, upload-time = "2026-01-22T16:35:24.919Z" },
]
[[package]]
name = "joblib"
version = "1.5.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/41/f2/d34e8b3a08a9cc79a50b2208a93dce981fe615b64d5a4d4abee421d898df/joblib-1.5.3.tar.gz", hash = "sha256:8561a3269e6801106863fd0d6d84bb737be9e7631e33aaed3fb9ce5953688da3", size = 331603, upload-time = "2025-12-15T08:41:46.427Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7b/91/984aca2ec129e2757d1e4e3c81c3fcda9d0f85b74670a094cc443d9ee949/joblib-1.5.3-py3-none-any.whl", hash = "sha256:5fc3c5039fc5ca8c0276333a188bbd59d6b7ab37fe6632daa76bc7f9ec18e713", size = 309071, upload-time = "2025-12-15T08:41:44.973Z" },
]
[[package]]
name = "jsonpatch"
version = "1.33"
@@ -1799,7 +1832,7 @@ wheels = [
[[package]]
name = "langbot"
version = "4.8.3"
version = "4.9.0"
source = { editable = "." }
dependencies = [
{ name = "aiocqhttp" },
@@ -1813,6 +1846,7 @@ dependencies = [
{ name = "asyncpg" },
{ name = "beautifulsoup4" },
{ name = "boto3" },
{ name = "botocore" },
{ name = "certifi" },
{ name = "chardet" },
{ name = "chromadb" },
@@ -1891,6 +1925,7 @@ requires-dist = [
{ name = "asyncpg", specifier = ">=0.30.0" },
{ name = "beautifulsoup4", specifier = ">=4.12.3" },
{ name = "boto3", specifier = ">=1.35.0" },
{ name = "botocore", specifier = ">=1.42.39" },
{ name = "certifi", specifier = ">=2025.4.26" },
{ name = "chardet", specifier = ">=5.2.0" },
{ name = "chromadb", specifier = ">=0.4.24" },
@@ -1902,7 +1937,7 @@ requires-dist = [
{ name = "ebooklib", specifier = ">=0.18" },
{ name = "gewechat-client", specifier = ">=0.1.5" },
{ name = "html2text", specifier = ">=2024.2.26" },
{ name = "langbot-plugin", specifier = "==0.2.5" },
{ name = "langbot-plugin", specifier = "==0.3.0" },
{ name = "langchain", specifier = ">=0.2.0" },
{ name = "langchain-text-splitters", specifier = ">=0.0.1" },
{ name = "lark-oapi", specifier = ">=1.4.15" },
@@ -1925,7 +1960,7 @@ requires-dist = [
{ name = "pymilvus", specifier = ">=2.6.4" },
{ name = "pynacl", specifier = ">=1.5.0" },
{ name = "pypdf2", specifier = ">=3.0.1" },
{ name = "pyseekdb", specifier = "==1.0.0b7" },
{ name = "pyseekdb", specifier = "==1.1.0.post3" },
{ name = "python-docx", specifier = ">=1.1.0" },
{ name = "python-socks", specifier = ">=2.7.1" },
{ name = "python-telegram-bot", specifier = ">=22.0" },
@@ -1958,7 +1993,7 @@ dev = [
[[package]]
name = "langbot-plugin"
version = "0.2.5"
version = "0.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiofiles" },
@@ -1976,9 +2011,9 @@ dependencies = [
{ name = "watchdog" },
{ name = "websockets" },
]
sdist = { url = "https://files.pythonhosted.org/packages/28/0e/117dfc00f36323cce2369be5176d5cd5247ff52edb34791413af9623f290/langbot_plugin-0.2.5.tar.gz", hash = "sha256:a1bf04c1c07b30c72fb9b28e1330372bb4a43ae2db309394435fc088c513cfd5", size = 103910, upload-time = "2026-01-29T13:55:34.328Z" }
sdist = { url = "https://files.pythonhosted.org/packages/8d/e5/3686b3225e5f2ee6e19a6050bb981b49a91f2450dff83deb5dfba13b3a2a/langbot_plugin-0.3.0.tar.gz", hash = "sha256:9add2d6e81c8cc7281863e4a92a33ed6228dcc0243f4327ac4062edc962dbf98", size = 169751, upload-time = "2026-03-08T09:54:27.102Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b1/0e/19b9a427206fa46aafbff03437296e38f425365c9ea6a97cbcfa791da2f8/langbot_plugin-0.2.5-py3-none-any.whl", hash = "sha256:b784248fc1f4754cd143bd9a16a7abd89a5c9735a4aa2b03c1c1e771b7d361e9", size = 133362, upload-time = "2026-01-29T13:55:32.486Z" },
{ url = "https://files.pythonhosted.org/packages/72/51/18f0c1446bcb6712ff3d31d81ea708e3f0e671fde5da69598204a1df977d/langbot_plugin-0.3.0-py3-none-any.whl", hash = "sha256:37bfd3ce507448a6ec4444bec1bc6da1c9911c9df144dfd428febb71122077a6", size = 144096, upload-time = "2026-03-08T09:54:25.581Z" },
]
[[package]]
@@ -2814,6 +2849,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/4a/67/5c9c8f1ba4a599e35a77ca7e0a0210ab6cd732f719bc3b0fc95c69aaca10/nakuru_project_idk-0.0.2.1-py3-none-any.whl", hash = "sha256:bddd8af8a46ef381bd05b806d6c07bd8ba407c58b47ce6148d750bd77c4420bc", size = 24281, upload-time = "2023-05-07T15:00:25.094Z" },
]
[[package]]
name = "networkx"
version = "3.6.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/6a/51/63fe664f3908c97be9d2e4f1158eb633317598cfa6e1fc14af5383f17512/networkx-3.6.1.tar.gz", hash = "sha256:26b7c357accc0c8cde558ad486283728b65b6a95d85ee1cd66bafab4c8168509", size = 2517025, upload-time = "2025-12-08T17:02:39.908Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/9e/c9/b2622292ea83fbb4ec318f5b9ab867d0a28ab43c5717bb85b0a5f6b3b0a4/networkx-3.6.1-py3-none-any.whl", hash = "sha256:d47fbf302e7d9cbbb9e2555a0d267983d2aa476bac30e90dfbe5669bd57f3762", size = 2068504, upload-time = "2025-12-08T17:02:38.159Z" },
]
[[package]]
name = "nodeenv"
version = "1.10.0"
@@ -2902,6 +2946,140 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/de/e5/b7d20451657664b07986c2f6e3be564433f5dcaf3482d68eaecd79afaf03/numpy-2.4.2-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:be71bf1edb48ebbbf7f6337b5bfd2f895d1902f6335a5830b20141fc126ffba0", size = 12502577, upload-time = "2026-01-31T23:13:07.08Z" },
]
[[package]]
name = "nvidia-cublas-cu12"
version = "12.8.4.1"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/dc/61/e24b560ab2e2eaeb3c839129175fb330dfcfc29e5203196e5541a4c44682/nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:8ac4e771d5a348c551b2a426eda6193c19aa630236b418086020df5ba9667142", size = 594346921, upload-time = "2025-03-07T01:44:31.254Z" },
]
[[package]]
name = "nvidia-cuda-cupti-cu12"
version = "12.8.90"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f8/02/2adcaa145158bf1a8295d83591d22e4103dbfd821bcaf6f3f53151ca4ffa/nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ea0cb07ebda26bb9b29ba82cda34849e73c166c18162d3913575b0c9db9a6182", size = 10248621, upload-time = "2025-03-07T01:40:21.213Z" },
]
[[package]]
name = "nvidia-cuda-nvrtc-cu12"
version = "12.8.93"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/05/6b/32f747947df2da6994e999492ab306a903659555dddc0fbdeb9d71f75e52/nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:a7756528852ef889772a84c6cd89d41dfa74667e24cca16bb31f8f061e3e9994", size = 88040029, upload-time = "2025-03-07T01:42:13.562Z" },
]
[[package]]
name = "nvidia-cuda-runtime-cu12"
version = "12.8.90"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0d/9b/a997b638fcd068ad6e4d53b8551a7d30fe8b404d6f1804abf1df69838932/nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:adade8dcbd0edf427b7204d480d6066d33902cab2a4707dcfc48a2d0fd44ab90", size = 954765, upload-time = "2025-03-07T01:40:01.615Z" },
]
[[package]]
name = "nvidia-cudnn-cu12"
version = "9.10.2.21"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/ba/51/e123d997aa098c61d029f76663dedbfb9bc8dcf8c60cbd6adbe42f76d049/nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:949452be657fa16687d0930933f032835951ef0892b37d2d53824d1a84dc97a8", size = 706758467, upload-time = "2025-06-06T21:54:08.597Z" },
]
[[package]]
name = "nvidia-cufft-cu12"
version = "11.3.3.83"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/1f/13/ee4e00f30e676b66ae65b4f08cb5bcbb8392c03f54f2d5413ea99a5d1c80/nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d2dd21ec0b88cf61b62e6b43564355e5222e4a3fb394cac0db101f2dd0d4f74", size = 193118695, upload-time = "2025-03-07T01:45:27.821Z" },
]
[[package]]
name = "nvidia-cufile-cu12"
version = "1.13.1.3"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bb/fe/1bcba1dfbfb8d01be8d93f07bfc502c93fa23afa6fd5ab3fc7c1df71038a/nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1d069003be650e131b21c932ec3d8969c1715379251f8d23a1860554b1cb24fc", size = 1197834, upload-time = "2025-03-07T01:45:50.723Z" },
]
[[package]]
name = "nvidia-curand-cu12"
version = "10.3.9.90"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/aa/6584b56dc84ebe9cf93226a5cde4d99080c8e90ab40f0c27bda7a0f29aa1/nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:b32331d4f4df5d6eefa0554c565b626c7216f87a06a4f56fab27c3b68a830ec9", size = 63619976, upload-time = "2025-03-07T01:46:23.323Z" },
]
[[package]]
name = "nvidia-cusolver-cu12"
version = "11.7.3.90"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "nvidia-cusparse-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "nvidia-nvjitlink-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/85/48/9a13d2975803e8cf2777d5ed57b87a0b6ca2cc795f9a4f59796a910bfb80/nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl", hash = "sha256:4376c11ad263152bd50ea295c05370360776f8c3427b30991df774f9fb26c450", size = 267506905, upload-time = "2025-03-07T01:47:16.273Z" },
]
[[package]]
name = "nvidia-cusparse-cu12"
version = "12.5.8.93"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink-cu12", marker = "python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/c2/f5/e1854cb2f2bcd4280c44736c93550cc300ff4b8c95ebe370d0aa7d2b473d/nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ec05d76bbbd8b61b06a80e1eaf8cf4959c3d4ce8e711b65ebd0443bb0ebb13b", size = 288216466, upload-time = "2025-03-07T01:48:13.779Z" },
]
[[package]]
name = "nvidia-cusparselt-cu12"
version = "0.7.1"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/56/79/12978b96bd44274fe38b5dde5cfb660b1d114f70a65ef962bcbbed99b549/nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f1bb701d6b930d5a7cea44c19ceb973311500847f81b634d802b7b539dc55623", size = 287193691, upload-time = "2025-02-26T00:15:44.104Z" },
]
[[package]]
name = "nvidia-nccl-cu12"
version = "2.27.5"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6e/89/f7a07dc961b60645dbbf42e80f2bc85ade7feb9a491b11a1e973aa00071f/nvidia_nccl_cu12-2.27.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:ad730cf15cb5d25fe849c6e6ca9eb5b76db16a80f13f425ac68d8e2e55624457", size = 322348229, upload-time = "2025-06-26T04:11:28.385Z" },
]
[[package]]
name = "nvidia-nvjitlink-cu12"
version = "12.8.93"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f6/74/86a07f1d0f42998ca31312f998bd3b9a7eff7f52378f4f270c8679c77fb9/nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl", hash = "sha256:81ff63371a7ebd6e6451970684f916be2eab07321b73c9d244dc2b4da7f73b88", size = 39254836, upload-time = "2025-03-07T01:49:55.661Z" },
]
[[package]]
name = "nvidia-nvshmem-cu12"
version = "3.4.5"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/09/6ea3ea725f82e1e76684f0708bbedd871fc96da89945adeba65c3835a64c/nvidia_nvshmem_cu12-3.4.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:042f2500f24c021db8a06c5eec2539027d57460e1c1a762055a6554f72c369bd", size = 139103095, upload-time = "2025-09-06T00:32:31.266Z" },
]
[[package]]
name = "nvidia-nvtx-cu12"
version = "12.8.90"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a2/eb/86626c1bbc2edb86323022371c39aa48df6fd8b0a1647bc274577f72e90b/nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5b17e2001cc0d751a5bc2c6ec6d26ad95913324a4adb86788c944f8ce9ba441f", size = 89954, upload-time = "2025-03-07T01:42:44.131Z" },
]
[[package]]
name = "oauthlib"
version = "3.3.1"
@@ -3922,12 +4100,16 @@ name = "pylibseekdb"
version = "1.1.0"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1c/b8/c226744a7a1da9295725920a36867ee5665f2617972c7881d5ed4cbd45c8/pylibseekdb-1.1.0-cp311-cp311-macosx_15_0_arm64.whl", hash = "sha256:0a0ad03d87f1db1a7087ba89e398ce1ee00496e977d38c493104d0d517590968", size = 148743770, upload-time = "2026-01-30T05:26:14.275Z" },
{ url = "https://files.pythonhosted.org/packages/51/4d/57151735afc29039f4ed680256012a33dd719ba3fd84d7c33a9bd260fc8a/pylibseekdb-1.1.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:e272bee013aabab152c4795676b3b0ba1107a8058f29a07d2a803168faea090c", size = 147132528, upload-time = "2026-01-30T03:40:10.878Z" },
{ url = "https://files.pythonhosted.org/packages/88/d7/5583fbf27e89952cda52bb9b1919229bd652d02aafac156758ac862c48e7/pylibseekdb-1.1.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:116a28356532705ed262e2a7951ac8221ae8c97ade866fdab2df521dcca62530", size = 170696822, upload-time = "2026-01-30T03:40:18.417Z" },
{ url = "https://files.pythonhosted.org/packages/5d/2b/150592287119f80cff9b025d59879a561a0cca80e71cecbf74a41af6220b/pylibseekdb-1.1.0-cp312-cp312-macosx_15_0_arm64.whl", hash = "sha256:d6ae33353e833cb56a7ce2cdb0305b872cdac9467eb79c277f82479c529b38ef", size = 148734111, upload-time = "2026-01-30T05:26:56.906Z" },
{ url = "https://files.pythonhosted.org/packages/b8/a3/b55087293115ecbe22313b40533fd67b0192c36e6bedb05aa7058a83a86a/pylibseekdb-1.1.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:9e2f8240b08a93e347d32534e7c394b7a151b67555a384eb88d73d4b0f8b9d14", size = 147137592, upload-time = "2026-01-30T03:40:26.087Z" },
{ url = "https://files.pythonhosted.org/packages/04/31/c0979960d790621dec277f64b5d6c70932f8bb9adb59029d7b481cfe9c30/pylibseekdb-1.1.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:4d8615471bac39b1980951cbce0d742fa7bec676f28eb95f4db687fdd1e9c71b", size = 170681044, upload-time = "2026-01-30T03:40:34.276Z" },
{ url = "https://files.pythonhosted.org/packages/33/7d/8acbf3eca93905c1b13b015a9e02b426fc69c10e7c162be96b35a2b1c7a4/pylibseekdb-1.1.0-cp313-cp313-macosx_15_0_arm64.whl", hash = "sha256:d5688a0fe6fc703e5a707cbe0e139d570f1d34daff1491304d6b43154f2e12d9", size = 148743750, upload-time = "2026-01-30T05:27:39.832Z" },
{ url = "https://files.pythonhosted.org/packages/c8/24/7f510ad13ad129a691fa965dc5bce874320b682674cbf12fc2e35310719b/pylibseekdb-1.1.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:1e53d171246239bd526d1a1f9b3abef1ad9b10597bc1c0a2acf7e65afbd7d844", size = 147136041, upload-time = "2026-01-30T03:40:41.782Z" },
{ url = "https://files.pythonhosted.org/packages/ed/eb/c5988e1ad72233a920f4e444d8d866c42363220b340d78a7525307922f35/pylibseekdb-1.1.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:66d01ee9c0ad4a2e88ea2420f9c4d1ee9bb011b70c553a654c8a4e230e920ad7", size = 170684140, upload-time = "2026-01-30T03:40:49.351Z" },
{ url = "https://files.pythonhosted.org/packages/9a/6f/b4a619c3a1b937fb080aa977b1d4011a1e587255707d54856188e5359a4c/pylibseekdb-1.1.0-cp314-cp314-macosx_15_0_arm64.whl", hash = "sha256:11d2fbc98dcb8ec97257b949184dc09d9ba693811e77457bba9c8f80d282c265", size = 148745880, upload-time = "2026-01-30T05:38:26.631Z" },
{ url = "https://files.pythonhosted.org/packages/0c/94/534359608571d08825ac21e709aa680b559989c905f99e273d82d5b17db2/pylibseekdb-1.1.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:ff05ac4bb13a4b5f9dd03771ded866beed72562ea497f68a4ae897c226afc446", size = 147132460, upload-time = "2026-01-30T03:40:56.684Z" },
{ url = "https://files.pythonhosted.org/packages/19/5e/7588a06918ac145fb69e57ae372b72d6fc713b9263c29eb7268f8a4edbef/pylibseekdb-1.1.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:065158b79192cce7635995a7599e99b21a3ff729cd6f68e31a65ed62f830bd3a", size = 170677921, upload-time = "2026-01-30T03:41:03.783Z" },
]
@@ -4041,20 +4223,21 @@ wheels = [
[[package]]
name = "pyseekdb"
version = "1.0.0b7"
version = "1.1.0.post3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "httpx" },
{ name = "httpx", marker = "python_full_version < '3.14'" },
{ name = "numpy" },
{ name = "onnxruntime" },
{ name = "pylibseekdb", marker = "sys_platform == 'linux'" },
{ name = "onnxruntime", marker = "python_full_version < '3.14'" },
{ name = "pylibseekdb", marker = "(platform_machine == 'arm64' and sys_platform == 'darwin') or sys_platform == 'linux'" },
{ name = "pymysql" },
{ name = "sentence-transformers", marker = "python_full_version >= '3.14'" },
{ name = "tenacity" },
{ name = "tokenizers" },
{ name = "tqdm" },
{ name = "tokenizers", marker = "python_full_version < '3.14'" },
{ name = "tqdm", marker = "python_full_version < '3.14'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/92/6a/a0d4728de90e028a60a3583e6e96579087f0cf793e705ea7898a1490541c/pyseekdb-1.0.0b7-py3-none-any.whl", hash = "sha256:e32920636c345bc73adf03040f9bcb1ecc420d652cedae1558999cce19a67d52", size = 60927, upload-time = "2025-12-29T13:19:04.669Z" },
{ url = "https://files.pythonhosted.org/packages/58/6e/2373239ab80c35a17aa14e8219727f06567e91d3b7f1b8c36d28ce94d04b/pyseekdb-1.1.0.post3-py3-none-any.whl", hash = "sha256:0437c9a4de72be44eb24b070b2b8099086467c08af10a57191498a61257a4bfb", size = 110985, upload-time = "2026-02-12T14:19:05.402Z" },
]
[[package]]
@@ -4634,6 +4817,168 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fc/51/727abb13f44c1fcf6d145979e1535a35794db0f6e450a0cb46aa24732fe2/s3transfer-0.16.0-py3-none-any.whl", hash = "sha256:18e25d66fed509e3868dc1572b3f427ff947dd2c56f844a5bf09481ad3f3b2fe", size = 86830, upload-time = "2025-12-01T02:30:57.729Z" },
]
[[package]]
name = "safetensors"
version = "0.7.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/29/9c/6e74567782559a63bd040a236edca26fd71bc7ba88de2ef35d75df3bca5e/safetensors-0.7.0.tar.gz", hash = "sha256:07663963b67e8bd9f0b8ad15bb9163606cd27cc5a1b96235a50d8369803b96b0", size = 200878, upload-time = "2025-11-19T15:18:43.199Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fa/47/aef6c06649039accf914afef490268e1067ed82be62bcfa5b7e886ad15e8/safetensors-0.7.0-cp38-abi3-macosx_10_12_x86_64.whl", hash = "sha256:c82f4d474cf725255d9e6acf17252991c3c8aac038d6ef363a4bf8be2f6db517", size = 467781, upload-time = "2025-11-19T15:18:35.84Z" },
{ url = "https://files.pythonhosted.org/packages/e8/00/374c0c068e30cd31f1e1b46b4b5738168ec79e7689ca82ee93ddfea05109/safetensors-0.7.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:94fd4858284736bb67a897a41608b5b0c2496c9bdb3bf2af1fa3409127f20d57", size = 447058, upload-time = "2025-11-19T15:18:34.416Z" },
{ url = "https://files.pythonhosted.org/packages/f1/06/578ffed52c2296f93d7fd2d844cabfa92be51a587c38c8afbb8ae449ca89/safetensors-0.7.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e07d91d0c92a31200f25351f4acb2bc6aff7f48094e13ebb1d0fb995b54b6542", size = 491748, upload-time = "2025-11-19T15:18:09.79Z" },
{ url = "https://files.pythonhosted.org/packages/ae/33/1debbbb70e4791dde185edb9413d1fe01619255abb64b300157d7f15dddd/safetensors-0.7.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8469155f4cb518bafb4acf4865e8bb9d6804110d2d9bdcaa78564b9fd841e104", size = 503881, upload-time = "2025-11-19T15:18:16.145Z" },
{ url = "https://files.pythonhosted.org/packages/8e/1c/40c2ca924d60792c3be509833df711b553c60effbd91da6f5284a83f7122/safetensors-0.7.0-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:54bef08bf00a2bff599982f6b08e8770e09cc012d7bba00783fc7ea38f1fb37d", size = 623463, upload-time = "2025-11-19T15:18:21.11Z" },
{ url = "https://files.pythonhosted.org/packages/9b/3a/13784a9364bd43b0d61eef4bea2845039bc2030458b16594a1bd787ae26e/safetensors-0.7.0-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:42cb091236206bb2016d245c377ed383aa7f78691748f3bb6ee1bfa51ae2ce6a", size = 532855, upload-time = "2025-11-19T15:18:25.719Z" },
{ url = "https://files.pythonhosted.org/packages/a0/60/429e9b1cb3fc651937727befe258ea24122d9663e4d5709a48c9cbfceecb/safetensors-0.7.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dac7252938f0696ddea46f5e855dd3138444e82236e3be475f54929f0c510d48", size = 507152, upload-time = "2025-11-19T15:18:33.023Z" },
{ url = "https://files.pythonhosted.org/packages/3c/a8/4b45e4e059270d17af60359713ffd83f97900d45a6afa73aaa0d737d48b6/safetensors-0.7.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:1d060c70284127fa805085d8f10fbd0962792aed71879d00864acda69dbab981", size = 541856, upload-time = "2025-11-19T15:18:31.075Z" },
{ url = "https://files.pythonhosted.org/packages/06/87/d26d8407c44175d8ae164a95b5a62707fcc445f3c0c56108e37d98070a3d/safetensors-0.7.0-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:cdab83a366799fa730f90a4ebb563e494f28e9e92c4819e556152ad55e43591b", size = 674060, upload-time = "2025-11-19T15:18:37.211Z" },
{ url = "https://files.pythonhosted.org/packages/11/f5/57644a2ff08dc6325816ba7217e5095f17269dada2554b658442c66aed51/safetensors-0.7.0-cp38-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:672132907fcad9f2aedcb705b2d7b3b93354a2aec1b2f706c4db852abe338f85", size = 771715, upload-time = "2025-11-19T15:18:38.689Z" },
{ url = "https://files.pythonhosted.org/packages/86/31/17883e13a814bd278ae6e266b13282a01049b0c81341da7fd0e3e71a80a3/safetensors-0.7.0-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:5d72abdb8a4d56d4020713724ba81dac065fedb7f3667151c4a637f1d3fb26c0", size = 714377, upload-time = "2025-11-19T15:18:40.162Z" },
{ url = "https://files.pythonhosted.org/packages/4a/d8/0c8a7dc9b41dcac53c4cbf9df2b9c83e0e0097203de8b37a712b345c0be5/safetensors-0.7.0-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b0f6d66c1c538d5a94a73aa9ddca8ccc4227e6c9ff555322ea40bdd142391dd4", size = 677368, upload-time = "2025-11-19T15:18:41.627Z" },
{ url = "https://files.pythonhosted.org/packages/05/e5/cb4b713c8a93469e3c5be7c3f8d77d307e65fe89673e731f5c2bfd0a9237/safetensors-0.7.0-cp38-abi3-win32.whl", hash = "sha256:c74af94bf3ac15ac4d0f2a7c7b4663a15f8c2ab15ed0fc7531ca61d0835eccba", size = 326423, upload-time = "2025-11-19T15:18:45.74Z" },
{ url = "https://files.pythonhosted.org/packages/5d/e6/ec8471c8072382cb91233ba7267fd931219753bb43814cbc71757bfd4dab/safetensors-0.7.0-cp38-abi3-win_amd64.whl", hash = "sha256:d1239932053f56f3456f32eb9625590cc7582e905021f94636202a864d470755", size = 341380, upload-time = "2025-11-19T15:18:44.427Z" },
]
[[package]]
name = "scikit-learn"
version = "1.8.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "joblib", marker = "python_full_version >= '3.14'" },
{ name = "numpy", marker = "python_full_version >= '3.14'" },
{ name = "scipy", marker = "python_full_version >= '3.14'" },
{ name = "threadpoolctl", marker = "python_full_version >= '3.14'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0e/d4/40988bf3b8e34feec1d0e6a051446b1f66225f8529b9309becaeef62b6c4/scikit_learn-1.8.0.tar.gz", hash = "sha256:9bccbb3b40e3de10351f8f5068e105d0f4083b1a65fa07b6634fbc401a6287fd", size = 7335585, upload-time = "2025-12-10T07:08:53.618Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c9/92/53ea2181da8ac6bf27170191028aee7251f8f841f8d3edbfdcaf2008fde9/scikit_learn-1.8.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:146b4d36f800c013d267b29168813f7a03a43ecd2895d04861f1240b564421da", size = 8595835, upload-time = "2025-12-10T07:07:39.385Z" },
{ url = "https://files.pythonhosted.org/packages/01/18/d154dc1638803adf987910cdd07097d9c526663a55666a97c124d09fb96a/scikit_learn-1.8.0-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:f984ca4b14914e6b4094c5d52a32ea16b49832c03bd17a110f004db3c223e8e1", size = 8080381, upload-time = "2025-12-10T07:07:41.93Z" },
{ url = "https://files.pythonhosted.org/packages/8a/44/226142fcb7b7101e64fdee5f49dbe6288d4c7af8abf593237b70fca080a4/scikit_learn-1.8.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e30adb87f0cc81c7690a84f7932dd66be5bac57cfe16b91cb9151683a4a2d3b", size = 8799632, upload-time = "2025-12-10T07:07:43.899Z" },
{ url = "https://files.pythonhosted.org/packages/36/4d/4a67f30778a45d542bbea5db2dbfa1e9e100bf9ba64aefe34215ba9f11f6/scikit_learn-1.8.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ada8121bcb4dac28d930febc791a69f7cb1673c8495e5eee274190b73a4559c1", size = 9103788, upload-time = "2025-12-10T07:07:45.982Z" },
{ url = "https://files.pythonhosted.org/packages/89/3c/45c352094cfa60050bcbb967b1faf246b22e93cb459f2f907b600f2ceda5/scikit_learn-1.8.0-cp311-cp311-win_amd64.whl", hash = "sha256:c57b1b610bd1f40ba43970e11ce62821c2e6569e4d74023db19c6b26f246cb3b", size = 8081706, upload-time = "2025-12-10T07:07:48.111Z" },
{ url = "https://files.pythonhosted.org/packages/3d/46/5416595bb395757f754feb20c3d776553a386b661658fb21b7c814e89efe/scikit_learn-1.8.0-cp311-cp311-win_arm64.whl", hash = "sha256:2838551e011a64e3053ad7618dda9310175f7515f1742fa2d756f7c874c05961", size = 7688451, upload-time = "2025-12-10T07:07:49.873Z" },
{ url = "https://files.pythonhosted.org/packages/90/74/e6a7cc4b820e95cc38cf36cd74d5aa2b42e8ffc2d21fe5a9a9c45c1c7630/scikit_learn-1.8.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:5fb63362b5a7ddab88e52b6dbb47dac3fd7dafeee740dc6c8d8a446ddedade8e", size = 8548242, upload-time = "2025-12-10T07:07:51.568Z" },
{ url = "https://files.pythonhosted.org/packages/49/d8/9be608c6024d021041c7f0b3928d4749a706f4e2c3832bbede4fb4f58c95/scikit_learn-1.8.0-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:5025ce924beccb28298246e589c691fe1b8c1c96507e6d27d12c5fadd85bfd76", size = 8079075, upload-time = "2025-12-10T07:07:53.697Z" },
{ url = "https://files.pythonhosted.org/packages/dd/47/f187b4636ff80cc63f21cd40b7b2d177134acaa10f6bb73746130ee8c2e5/scikit_learn-1.8.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4496bb2cf7a43ce1a2d7524a79e40bc5da45cf598dbf9545b7e8316ccba47bb4", size = 8660492, upload-time = "2025-12-10T07:07:55.574Z" },
{ url = "https://files.pythonhosted.org/packages/97/74/b7a304feb2b49df9fafa9382d4d09061a96ee9a9449a7cbea7988dda0828/scikit_learn-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a0bcfe4d0d14aec44921545fd2af2338c7471de9cb701f1da4c9d85906ab847a", size = 8931904, upload-time = "2025-12-10T07:07:57.666Z" },
{ url = "https://files.pythonhosted.org/packages/9f/c4/0ab22726a04ede56f689476b760f98f8f46607caecff993017ac1b64aa5d/scikit_learn-1.8.0-cp312-cp312-win_amd64.whl", hash = "sha256:35c007dedb2ffe38fe3ee7d201ebac4a2deccd2408e8621d53067733e3c74809", size = 8019359, upload-time = "2025-12-10T07:07:59.838Z" },
{ url = "https://files.pythonhosted.org/packages/24/90/344a67811cfd561d7335c1b96ca21455e7e472d281c3c279c4d3f2300236/scikit_learn-1.8.0-cp312-cp312-win_arm64.whl", hash = "sha256:8c497fff237d7b4e07e9ef1a640887fa4fb765647f86fbe00f969ff6280ce2bb", size = 7641898, upload-time = "2025-12-10T07:08:01.36Z" },
{ url = "https://files.pythonhosted.org/packages/03/aa/e22e0768512ce9255eba34775be2e85c2048da73da1193e841707f8f039c/scikit_learn-1.8.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0d6ae97234d5d7079dc0040990a6f7aeb97cb7fa7e8945f1999a429b23569e0a", size = 8513770, upload-time = "2025-12-10T07:08:03.251Z" },
{ url = "https://files.pythonhosted.org/packages/58/37/31b83b2594105f61a381fc74ca19e8780ee923be2d496fcd8d2e1147bd99/scikit_learn-1.8.0-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:edec98c5e7c128328124a029bceb09eda2d526997780fef8d65e9a69eead963e", size = 8044458, upload-time = "2025-12-10T07:08:05.336Z" },
{ url = "https://files.pythonhosted.org/packages/2d/5a/3f1caed8765f33eabb723596666da4ebbf43d11e96550fb18bdec42b467b/scikit_learn-1.8.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:74b66d8689d52ed04c271e1329f0c61635bcaf5b926db9b12d58914cdc01fe57", size = 8610341, upload-time = "2025-12-10T07:08:07.732Z" },
{ url = "https://files.pythonhosted.org/packages/38/cf/06896db3f71c75902a8e9943b444a56e727418f6b4b4a90c98c934f51ed4/scikit_learn-1.8.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8fdf95767f989b0cfedb85f7ed8ca215d4be728031f56ff5a519ee1e3276dc2e", size = 8900022, upload-time = "2025-12-10T07:08:09.862Z" },
{ url = "https://files.pythonhosted.org/packages/1c/f9/9b7563caf3ec8873e17a31401858efab6b39a882daf6c1bfa88879c0aa11/scikit_learn-1.8.0-cp313-cp313-win_amd64.whl", hash = "sha256:2de443b9373b3b615aec1bb57f9baa6bb3a9bd093f1269ba95c17d870422b271", size = 7989409, upload-time = "2025-12-10T07:08:12.028Z" },
{ url = "https://files.pythonhosted.org/packages/49/bd/1f4001503650e72c4f6009ac0c4413cb17d2d601cef6f71c0453da2732fc/scikit_learn-1.8.0-cp313-cp313-win_arm64.whl", hash = "sha256:eddde82a035681427cbedded4e6eff5e57fa59216c2e3e90b10b19ab1d0a65c3", size = 7619760, upload-time = "2025-12-10T07:08:13.688Z" },
{ url = "https://files.pythonhosted.org/packages/d2/7d/a630359fc9dcc95496588c8d8e3245cc8fd81980251079bc09c70d41d951/scikit_learn-1.8.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:7cc267b6108f0a1499a734167282c00c4ebf61328566b55ef262d48e9849c735", size = 8826045, upload-time = "2025-12-10T07:08:15.215Z" },
{ url = "https://files.pythonhosted.org/packages/cc/56/a0c86f6930cfcd1c7054a2bc417e26960bb88d32444fe7f71d5c2cfae891/scikit_learn-1.8.0-cp313-cp313t-macosx_12_0_arm64.whl", hash = "sha256:fe1c011a640a9f0791146011dfd3c7d9669785f9fed2b2a5f9e207536cf5c2fd", size = 8420324, upload-time = "2025-12-10T07:08:17.561Z" },
{ url = "https://files.pythonhosted.org/packages/46/1e/05962ea1cebc1cf3876667ecb14c283ef755bf409993c5946ade3b77e303/scikit_learn-1.8.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72358cce49465d140cc4e7792015bb1f0296a9742d5622c67e31399b75468b9e", size = 8680651, upload-time = "2025-12-10T07:08:19.952Z" },
{ url = "https://files.pythonhosted.org/packages/fe/56/a85473cd75f200c9759e3a5f0bcab2d116c92a8a02ee08ccd73b870f8bb4/scikit_learn-1.8.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:80832434a6cc114f5219211eec13dcbc16c2bac0e31ef64c6d346cde3cf054cb", size = 8925045, upload-time = "2025-12-10T07:08:22.11Z" },
{ url = "https://files.pythonhosted.org/packages/cc/b7/64d8cfa896c64435ae57f4917a548d7ac7a44762ff9802f75a79b77cb633/scikit_learn-1.8.0-cp313-cp313t-win_amd64.whl", hash = "sha256:ee787491dbfe082d9c3013f01f5991658b0f38aa8177e4cd4bf434c58f551702", size = 8507994, upload-time = "2025-12-10T07:08:23.943Z" },
{ url = "https://files.pythonhosted.org/packages/5e/37/e192ea709551799379958b4c4771ec507347027bb7c942662c7fbeba31cb/scikit_learn-1.8.0-cp313-cp313t-win_arm64.whl", hash = "sha256:bf97c10a3f5a7543f9b88cbf488d33d175e9146115a451ae34568597ba33dcde", size = 7869518, upload-time = "2025-12-10T07:08:25.71Z" },
{ url = "https://files.pythonhosted.org/packages/24/05/1af2c186174cc92dcab2233f327336058c077d38f6fe2aceb08e6ab4d509/scikit_learn-1.8.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c22a2da7a198c28dd1a6e1136f19c830beab7fdca5b3e5c8bba8394f8a5c45b3", size = 8528667, upload-time = "2025-12-10T07:08:27.541Z" },
{ url = "https://files.pythonhosted.org/packages/a8/25/01c0af38fe969473fb292bba9dc2b8f9b451f3112ff242c647fee3d0dfe7/scikit_learn-1.8.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:6b595b07a03069a2b1740dc08c2299993850ea81cce4fe19b2421e0c970de6b7", size = 8066524, upload-time = "2025-12-10T07:08:29.822Z" },
{ url = "https://files.pythonhosted.org/packages/be/ce/a0623350aa0b68647333940ee46fe45086c6060ec604874e38e9ab7d8e6c/scikit_learn-1.8.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:29ffc74089f3d5e87dfca4c2c8450f88bdc61b0fc6ed5d267f3988f19a1309f6", size = 8657133, upload-time = "2025-12-10T07:08:31.865Z" },
{ url = "https://files.pythonhosted.org/packages/b8/cb/861b41341d6f1245e6ca80b1c1a8c4dfce43255b03df034429089ca2a2c5/scikit_learn-1.8.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fb65db5d7531bccf3a4f6bec3462223bea71384e2cda41da0f10b7c292b9e7c4", size = 8923223, upload-time = "2025-12-10T07:08:34.166Z" },
{ url = "https://files.pythonhosted.org/packages/76/18/a8def8f91b18cd1ba6e05dbe02540168cb24d47e8dcf69e8d00b7da42a08/scikit_learn-1.8.0-cp314-cp314-win_amd64.whl", hash = "sha256:56079a99c20d230e873ea40753102102734c5953366972a71d5cb39a32bc40c6", size = 8096518, upload-time = "2025-12-10T07:08:36.339Z" },
{ url = "https://files.pythonhosted.org/packages/d1/77/482076a678458307f0deb44e29891d6022617b2a64c840c725495bee343f/scikit_learn-1.8.0-cp314-cp314-win_arm64.whl", hash = "sha256:3bad7565bc9cf37ce19a7c0d107742b320c1285df7aab1a6e2d28780df167242", size = 7754546, upload-time = "2025-12-10T07:08:38.128Z" },
{ url = "https://files.pythonhosted.org/packages/2d/d1/ef294ca754826daa043b2a104e59960abfab4cf653891037d19dd5b6f3cf/scikit_learn-1.8.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:4511be56637e46c25721e83d1a9cea9614e7badc7040c4d573d75fbe257d6fd7", size = 8848305, upload-time = "2025-12-10T07:08:41.013Z" },
{ url = "https://files.pythonhosted.org/packages/5b/e2/b1f8b05138ee813b8e1a4149f2f0d289547e60851fd1bb268886915adbda/scikit_learn-1.8.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:a69525355a641bf8ef136a7fa447672fb54fe8d60cab5538d9eb7c6438543fb9", size = 8432257, upload-time = "2025-12-10T07:08:42.873Z" },
{ url = "https://files.pythonhosted.org/packages/26/11/c32b2138a85dcb0c99f6afd13a70a951bfdff8a6ab42d8160522542fb647/scikit_learn-1.8.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c2656924ec73e5939c76ac4c8b026fc203b83d8900362eb2599d8aee80e4880f", size = 8678673, upload-time = "2025-12-10T07:08:45.362Z" },
{ url = "https://files.pythonhosted.org/packages/c7/57/51f2384575bdec454f4fe4e7a919d696c9ebce914590abf3e52d47607ab8/scikit_learn-1.8.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15fc3b5d19cc2be65404786857f2e13c70c83dd4782676dd6814e3b89dc8f5b9", size = 8922467, upload-time = "2025-12-10T07:08:47.408Z" },
{ url = "https://files.pythonhosted.org/packages/35/4d/748c9e2872637a57981a04adc038dacaa16ba8ca887b23e34953f0b3f742/scikit_learn-1.8.0-cp314-cp314t-win_amd64.whl", hash = "sha256:00d6f1d66fbcf4eba6e356e1420d33cc06c70a45bb1363cd6f6a8e4ebbbdece2", size = 8774395, upload-time = "2025-12-10T07:08:49.337Z" },
{ url = "https://files.pythonhosted.org/packages/60/22/d7b2ebe4704a5e50790ba089d5c2ae308ab6bb852719e6c3bd4f04c3a363/scikit_learn-1.8.0-cp314-cp314t-win_arm64.whl", hash = "sha256:f28dd15c6bb0b66ba09728cf09fd8736c304be29409bd8445a080c1280619e8c", size = 8002647, upload-time = "2025-12-10T07:08:51.601Z" },
]
[[package]]
name = "scipy"
version = "1.17.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy", marker = "python_full_version >= '3.14'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/7a/97/5a3609c4f8d58b039179648e62dd220f89864f56f7357f5d4f45c29eb2cc/scipy-1.17.1.tar.gz", hash = "sha256:95d8e012d8cb8816c226aef832200b1d45109ed4464303e997c5b13122b297c0", size = 30573822, upload-time = "2026-02-23T00:26:24.851Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/df/75/b4ce781849931fef6fd529afa6b63711d5a733065722d0c3e2724af9e40a/scipy-1.17.1-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:1f95b894f13729334fb990162e911c9e5dc1ab390c58aa6cbecb389c5b5e28ec", size = 31613675, upload-time = "2026-02-23T00:16:00.13Z" },
{ url = "https://files.pythonhosted.org/packages/f7/58/bccc2861b305abdd1b8663d6130c0b3d7cc22e8d86663edbc8401bfd40d4/scipy-1.17.1-cp311-cp311-macosx_12_0_arm64.whl", hash = "sha256:e18f12c6b0bc5a592ed23d3f7b891f68fd7f8241d69b7883769eb5d5dfb52696", size = 28162057, upload-time = "2026-02-23T00:16:09.456Z" },
{ url = "https://files.pythonhosted.org/packages/6d/ee/18146b7757ed4976276b9c9819108adbc73c5aad636e5353e20746b73069/scipy-1.17.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:a3472cfbca0a54177d0faa68f697d8ba4c80bbdc19908c3465556d9f7efce9ee", size = 20334032, upload-time = "2026-02-23T00:16:17.358Z" },
{ url = "https://files.pythonhosted.org/packages/ec/e6/cef1cf3557f0c54954198554a10016b6a03b2ec9e22a4e1df734936bd99c/scipy-1.17.1-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:766e0dc5a616d026a3a1cffa379af959671729083882f50307e18175797b3dfd", size = 22709533, upload-time = "2026-02-23T00:16:25.791Z" },
{ url = "https://files.pythonhosted.org/packages/4d/60/8804678875fc59362b0fb759ab3ecce1f09c10a735680318ac30da8cd76b/scipy-1.17.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:744b2bf3640d907b79f3fd7874efe432d1cf171ee721243e350f55234b4cec4c", size = 33062057, upload-time = "2026-02-23T00:16:36.931Z" },
{ url = "https://files.pythonhosted.org/packages/09/7d/af933f0f6e0767995b4e2d705a0665e454d1c19402aa7e895de3951ebb04/scipy-1.17.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:43af8d1f3bea642559019edfe64e9b11192a8978efbd1539d7bc2aaa23d92de4", size = 35349300, upload-time = "2026-02-23T00:16:49.108Z" },
{ url = "https://files.pythonhosted.org/packages/b4/3d/7ccbbdcbb54c8fdc20d3b6930137c782a163fa626f0aef920349873421ba/scipy-1.17.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:cd96a1898c0a47be4520327e01f874acfd61fb48a9420f8aa9f6483412ffa444", size = 35127333, upload-time = "2026-02-23T00:17:01.293Z" },
{ url = "https://files.pythonhosted.org/packages/e8/19/f926cb11c42b15ba08e3a71e376d816ac08614f769b4f47e06c3580c836a/scipy-1.17.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4eb6c25dd62ee8d5edf68a8e1c171dd71c292fdae95d8aeb3dd7d7de4c364082", size = 37741314, upload-time = "2026-02-23T00:17:12.576Z" },
{ url = "https://files.pythonhosted.org/packages/95/da/0d1df507cf574b3f224ccc3d45244c9a1d732c81dcb26b1e8a766ae271a8/scipy-1.17.1-cp311-cp311-win_amd64.whl", hash = "sha256:d30e57c72013c2a4fe441c2fcb8e77b14e152ad48b5464858e07e2ad9fbfceff", size = 36607512, upload-time = "2026-02-23T00:17:23.424Z" },
{ url = "https://files.pythonhosted.org/packages/68/7f/bdd79ceaad24b671543ffe0ef61ed8e659440eb683b66f033454dcee90eb/scipy-1.17.1-cp311-cp311-win_arm64.whl", hash = "sha256:9ecb4efb1cd6e8c4afea0daa91a87fbddbce1b99d2895d151596716c0b2e859d", size = 24599248, upload-time = "2026-02-23T00:17:34.561Z" },
{ url = "https://files.pythonhosted.org/packages/35/48/b992b488d6f299dbe3f11a20b24d3dda3d46f1a635ede1c46b5b17a7b163/scipy-1.17.1-cp312-cp312-macosx_10_14_x86_64.whl", hash = "sha256:35c3a56d2ef83efc372eaec584314bd0ef2e2f0d2adb21c55e6ad5b344c0dcb8", size = 31610954, upload-time = "2026-02-23T00:17:49.855Z" },
{ url = "https://files.pythonhosted.org/packages/b2/02/cf107b01494c19dc100f1d0b7ac3cc08666e96ba2d64db7626066cee895e/scipy-1.17.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:fcb310ddb270a06114bb64bbe53c94926b943f5b7f0842194d585c65eb4edd76", size = 28172662, upload-time = "2026-02-23T00:18:01.64Z" },
{ url = "https://files.pythonhosted.org/packages/cf/a9/599c28631bad314d219cf9ffd40e985b24d603fc8a2f4ccc5ae8419a535b/scipy-1.17.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:cc90d2e9c7e5c7f1a482c9875007c095c3194b1cfedca3c2f3291cdc2bc7c086", size = 20344366, upload-time = "2026-02-23T00:18:12.015Z" },
{ url = "https://files.pythonhosted.org/packages/35/f5/906eda513271c8deb5af284e5ef0206d17a96239af79f9fa0aebfe0e36b4/scipy-1.17.1-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:c80be5ede8f3f8eded4eff73cc99a25c388ce98e555b17d31da05287015ffa5b", size = 22704017, upload-time = "2026-02-23T00:18:21.502Z" },
{ url = "https://files.pythonhosted.org/packages/da/34/16f10e3042d2f1d6b66e0428308ab52224b6a23049cb2f5c1756f713815f/scipy-1.17.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e19ebea31758fac5893a2ac360fedd00116cbb7628e650842a6691ba7ca28a21", size = 32927842, upload-time = "2026-02-23T00:18:35.367Z" },
{ url = "https://files.pythonhosted.org/packages/01/8e/1e35281b8ab6d5d72ebe9911edcdffa3f36b04ed9d51dec6dd140396e220/scipy-1.17.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:02ae3b274fde71c5e92ac4d54bc06c42d80e399fec704383dcd99b301df37458", size = 35235890, upload-time = "2026-02-23T00:18:49.188Z" },
{ url = "https://files.pythonhosted.org/packages/c5/5c/9d7f4c88bea6e0d5a4f1bc0506a53a00e9fcb198de372bfe4d3652cef482/scipy-1.17.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8a604bae87c6195d8b1045eddece0514d041604b14f2727bbc2b3020172045eb", size = 35003557, upload-time = "2026-02-23T00:18:54.74Z" },
{ url = "https://files.pythonhosted.org/packages/65/94/7698add8f276dbab7a9de9fb6b0e02fc13ee61d51c7c3f85ac28b65e1239/scipy-1.17.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:f590cd684941912d10becc07325a3eeb77886fe981415660d9265c4c418d0bea", size = 37625856, upload-time = "2026-02-23T00:19:00.307Z" },
{ url = "https://files.pythonhosted.org/packages/a2/84/dc08d77fbf3d87d3ee27f6a0c6dcce1de5829a64f2eae85a0ecc1f0daa73/scipy-1.17.1-cp312-cp312-win_amd64.whl", hash = "sha256:41b71f4a3a4cab9d366cd9065b288efc4d4f3c0b37a91a8e0947fb5bd7f31d87", size = 36549682, upload-time = "2026-02-23T00:19:07.67Z" },
{ url = "https://files.pythonhosted.org/packages/bc/98/fe9ae9ffb3b54b62559f52dedaebe204b408db8109a8c66fdd04869e6424/scipy-1.17.1-cp312-cp312-win_arm64.whl", hash = "sha256:f4115102802df98b2b0db3cce5cb9b92572633a1197c77b7553e5203f284a5b3", size = 24547340, upload-time = "2026-02-23T00:19:12.024Z" },
{ url = "https://files.pythonhosted.org/packages/76/27/07ee1b57b65e92645f219b37148a7e7928b82e2b5dbeccecb4dff7c64f0b/scipy-1.17.1-cp313-cp313-macosx_10_14_x86_64.whl", hash = "sha256:5e3c5c011904115f88a39308379c17f91546f77c1667cea98739fe0fccea804c", size = 31590199, upload-time = "2026-02-23T00:19:17.192Z" },
{ url = "https://files.pythonhosted.org/packages/ec/ae/db19f8ab842e9b724bf5dbb7db29302a91f1e55bc4d04b1025d6d605a2c5/scipy-1.17.1-cp313-cp313-macosx_12_0_arm64.whl", hash = "sha256:6fac755ca3d2c3edcb22f479fceaa241704111414831ddd3bc6056e18516892f", size = 28154001, upload-time = "2026-02-23T00:19:22.241Z" },
{ url = "https://files.pythonhosted.org/packages/5b/58/3ce96251560107b381cbd6e8413c483bbb1228a6b919fa8652b0d4090e7f/scipy-1.17.1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:7ff200bf9d24f2e4d5dc6ee8c3ac64d739d3a89e2326ba68aaf6c4a2b838fd7d", size = 20325719, upload-time = "2026-02-23T00:19:26.329Z" },
{ url = "https://files.pythonhosted.org/packages/b2/83/15087d945e0e4d48ce2377498abf5ad171ae013232ae31d06f336e64c999/scipy-1.17.1-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:4b400bdc6f79fa02a4d86640310dde87a21fba0c979efff5248908c6f15fad1b", size = 22683595, upload-time = "2026-02-23T00:19:30.304Z" },
{ url = "https://files.pythonhosted.org/packages/b4/e0/e58fbde4a1a594c8be8114eb4aac1a55bcd6587047efc18a61eb1f5c0d30/scipy-1.17.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2b64ca7d4aee0102a97f3ba22124052b4bd2152522355073580bf4845e2550b6", size = 32896429, upload-time = "2026-02-23T00:19:35.536Z" },
{ url = "https://files.pythonhosted.org/packages/f5/5f/f17563f28ff03c7b6799c50d01d5d856a1d55f2676f537ca8d28c7f627cd/scipy-1.17.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:581b2264fc0aa555f3f435a5944da7504ea3a065d7029ad60e7c3d1ae09c5464", size = 35203952, upload-time = "2026-02-23T00:19:42.259Z" },
{ url = "https://files.pythonhosted.org/packages/8d/a5/9afd17de24f657fdfe4df9a3f1ea049b39aef7c06000c13db1530d81ccca/scipy-1.17.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:beeda3d4ae615106d7094f7e7cef6218392e4465cc95d25f900bebabfded0950", size = 34979063, upload-time = "2026-02-23T00:19:47.547Z" },
{ url = "https://files.pythonhosted.org/packages/8b/13/88b1d2384b424bf7c924f2038c1c409f8d88bb2a8d49d097861dd64a57b2/scipy-1.17.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6609bc224e9568f65064cfa72edc0f24ee6655b47575954ec6339534b2798369", size = 37598449, upload-time = "2026-02-23T00:19:53.238Z" },
{ url = "https://files.pythonhosted.org/packages/35/e5/d6d0e51fc888f692a35134336866341c08655d92614f492c6860dc45bb2c/scipy-1.17.1-cp313-cp313-win_amd64.whl", hash = "sha256:37425bc9175607b0268f493d79a292c39f9d001a357bebb6b88fdfaff13f6448", size = 36510943, upload-time = "2026-02-23T00:20:50.89Z" },
{ url = "https://files.pythonhosted.org/packages/2a/fd/3be73c564e2a01e690e19cc618811540ba5354c67c8680dce3281123fb79/scipy-1.17.1-cp313-cp313-win_arm64.whl", hash = "sha256:5cf36e801231b6a2059bf354720274b7558746f3b1a4efb43fcf557ccd484a87", size = 24545621, upload-time = "2026-02-23T00:20:55.871Z" },
{ url = "https://files.pythonhosted.org/packages/6f/6b/17787db8b8114933a66f9dcc479a8272e4b4da75fe03b0c282f7b0ade8cd/scipy-1.17.1-cp313-cp313t-macosx_10_14_x86_64.whl", hash = "sha256:d59c30000a16d8edc7e64152e30220bfbd724c9bbb08368c054e24c651314f0a", size = 31936708, upload-time = "2026-02-23T00:19:58.694Z" },
{ url = "https://files.pythonhosted.org/packages/38/2e/524405c2b6392765ab1e2b722a41d5da33dc5c7b7278184a8ad29b6cb206/scipy-1.17.1-cp313-cp313t-macosx_12_0_arm64.whl", hash = "sha256:010f4333c96c9bb1a4516269e33cb5917b08ef2166d5556ca2fd9f082a9e6ea0", size = 28570135, upload-time = "2026-02-23T00:20:03.934Z" },
{ url = "https://files.pythonhosted.org/packages/fd/c3/5bd7199f4ea8556c0c8e39f04ccb014ac37d1468e6cfa6a95c6b3562b76e/scipy-1.17.1-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:2ceb2d3e01c5f1d83c4189737a42d9cb2fc38a6eeed225e7515eef71ad301dce", size = 20741977, upload-time = "2026-02-23T00:20:07.935Z" },
{ url = "https://files.pythonhosted.org/packages/d9/b8/8ccd9b766ad14c78386599708eb745f6b44f08400a5fd0ade7cf89b6fc93/scipy-1.17.1-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:844e165636711ef41f80b4103ed234181646b98a53c8f05da12ca5ca289134f6", size = 23029601, upload-time = "2026-02-23T00:20:12.161Z" },
{ url = "https://files.pythonhosted.org/packages/6d/a0/3cb6f4d2fb3e17428ad2880333cac878909ad1a89f678527b5328b93c1d4/scipy-1.17.1-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:158dd96d2207e21c966063e1635b1063cd7787b627b6f07305315dd73d9c679e", size = 33019667, upload-time = "2026-02-23T00:20:17.208Z" },
{ url = "https://files.pythonhosted.org/packages/f3/c3/2d834a5ac7bf3a0c806ad1508efc02dda3c8c61472a56132d7894c312dea/scipy-1.17.1-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74cbb80d93260fe2ffa334efa24cb8f2f0f622a9b9febf8b483c0b865bfb3475", size = 35264159, upload-time = "2026-02-23T00:20:23.087Z" },
{ url = "https://files.pythonhosted.org/packages/4d/77/d3ed4becfdbd217c52062fafe35a72388d1bd82c2d0ba5ca19d6fcc93e11/scipy-1.17.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:dbc12c9f3d185f5c737d801da555fb74b3dcfa1a50b66a1a93e09190f41fab50", size = 35102771, upload-time = "2026-02-23T00:20:28.636Z" },
{ url = "https://files.pythonhosted.org/packages/bd/12/d19da97efde68ca1ee5538bb261d5d2c062f0c055575128f11a2730e3ac1/scipy-1.17.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:94055a11dfebe37c656e70317e1996dc197e1a15bbcc351bcdd4610e128fe1ca", size = 37665910, upload-time = "2026-02-23T00:20:34.743Z" },
{ url = "https://files.pythonhosted.org/packages/06/1c/1172a88d507a4baaf72c5a09bb6c018fe2ae0ab622e5830b703a46cc9e44/scipy-1.17.1-cp313-cp313t-win_amd64.whl", hash = "sha256:e30bdeaa5deed6bc27b4cc490823cd0347d7dae09119b8803ae576ea0ce52e4c", size = 36562980, upload-time = "2026-02-23T00:20:40.575Z" },
{ url = "https://files.pythonhosted.org/packages/70/b0/eb757336e5a76dfa7911f63252e3b7d1de00935d7705cf772db5b45ec238/scipy-1.17.1-cp313-cp313t-win_arm64.whl", hash = "sha256:a720477885a9d2411f94a93d16f9d89bad0f28ca23c3f8daa521e2dcc3f44d49", size = 24856543, upload-time = "2026-02-23T00:20:45.313Z" },
{ url = "https://files.pythonhosted.org/packages/cf/83/333afb452af6f0fd70414dc04f898647ee1423979ce02efa75c3b0f2c28e/scipy-1.17.1-cp314-cp314-macosx_10_14_x86_64.whl", hash = "sha256:a48a72c77a310327f6a3a920092fa2b8fd03d7deaa60f093038f22d98e096717", size = 31584510, upload-time = "2026-02-23T00:21:01.015Z" },
{ url = "https://files.pythonhosted.org/packages/ed/a6/d05a85fd51daeb2e4ea71d102f15b34fedca8e931af02594193ae4fd25f7/scipy-1.17.1-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:45abad819184f07240d8a696117a7aacd39787af9e0b719d00285549ed19a1e9", size = 28170131, upload-time = "2026-02-23T00:21:05.888Z" },
{ url = "https://files.pythonhosted.org/packages/db/7b/8624a203326675d7746a254083a187398090a179335b2e4a20e2ddc46e83/scipy-1.17.1-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:3fd1fcdab3ea951b610dc4cef356d416d5802991e7e32b5254828d342f7b7e0b", size = 20342032, upload-time = "2026-02-23T00:21:09.904Z" },
{ url = "https://files.pythonhosted.org/packages/c9/35/2c342897c00775d688d8ff3987aced3426858fd89d5a0e26e020b660b301/scipy-1.17.1-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:7bdf2da170b67fdf10bca777614b1c7d96ae3ca5794fd9587dce41eb2966e866", size = 22678766, upload-time = "2026-02-23T00:21:14.313Z" },
{ url = "https://files.pythonhosted.org/packages/ef/f2/7cdb8eb308a1a6ae1e19f945913c82c23c0c442a462a46480ce487fdc0ac/scipy-1.17.1-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:adb2642e060a6549c343603a3851ba76ef0b74cc8c079a9a58121c7ec9fe2350", size = 32957007, upload-time = "2026-02-23T00:21:19.663Z" },
{ url = "https://files.pythonhosted.org/packages/0b/2e/7eea398450457ecb54e18e9d10110993fa65561c4f3add5e8eccd2b9cd41/scipy-1.17.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:eee2cfda04c00a857206a4330f0c5e3e56535494e30ca445eb19ec624ae75118", size = 35221333, upload-time = "2026-02-23T00:21:25.278Z" },
{ url = "https://files.pythonhosted.org/packages/d9/77/5b8509d03b77f093a0d52e606d3c4f79e8b06d1d38c441dacb1e26cacf46/scipy-1.17.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:d2650c1fb97e184d12d8ba010493ee7b322864f7d3d00d3f9bb97d9c21de4068", size = 35042066, upload-time = "2026-02-23T00:21:31.358Z" },
{ url = "https://files.pythonhosted.org/packages/f9/df/18f80fb99df40b4070328d5ae5c596f2f00fffb50167e31439e932f29e7d/scipy-1.17.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:08b900519463543aa604a06bec02461558a6e1cef8fdbb8098f77a48a83c8118", size = 37612763, upload-time = "2026-02-23T00:21:37.247Z" },
{ url = "https://files.pythonhosted.org/packages/4b/39/f0e8ea762a764a9dc52aa7dabcfad51a354819de1f0d4652b6a1122424d6/scipy-1.17.1-cp314-cp314-win_amd64.whl", hash = "sha256:3877ac408e14da24a6196de0ddcace62092bfc12a83823e92e49e40747e52c19", size = 37290984, upload-time = "2026-02-23T00:22:35.023Z" },
{ url = "https://files.pythonhosted.org/packages/7c/56/fe201e3b0f93d1a8bcf75d3379affd228a63d7e2d80ab45467a74b494947/scipy-1.17.1-cp314-cp314-win_arm64.whl", hash = "sha256:f8885db0bc2bffa59d5c1b72fad7a6a92d3e80e7257f967dd81abb553a90d293", size = 25192877, upload-time = "2026-02-23T00:22:39.798Z" },
{ url = "https://files.pythonhosted.org/packages/96/ad/f8c414e121f82e02d76f310f16db9899c4fcde36710329502a6b2a3c0392/scipy-1.17.1-cp314-cp314t-macosx_10_14_x86_64.whl", hash = "sha256:1cc682cea2ae55524432f3cdff9e9a3be743d52a7443d0cba9017c23c87ae2f6", size = 31949750, upload-time = "2026-02-23T00:21:42.289Z" },
{ url = "https://files.pythonhosted.org/packages/7c/b0/c741e8865d61b67c81e255f4f0a832846c064e426636cd7de84e74d209be/scipy-1.17.1-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:2040ad4d1795a0ae89bfc7e8429677f365d45aa9fd5e4587cf1ea737f927b4a1", size = 28585858, upload-time = "2026-02-23T00:21:47.706Z" },
{ url = "https://files.pythonhosted.org/packages/ed/1b/3985219c6177866628fa7c2595bfd23f193ceebbe472c98a08824b9466ff/scipy-1.17.1-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:131f5aaea57602008f9822e2115029b55d4b5f7c070287699fe45c661d051e39", size = 20757723, upload-time = "2026-02-23T00:21:52.039Z" },
{ url = "https://files.pythonhosted.org/packages/c0/19/2a04aa25050d656d6f7b9e7b685cc83d6957fb101665bfd9369ca6534563/scipy-1.17.1-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:9cdc1a2fcfd5c52cfb3045feb399f7b3ce822abdde3a193a6b9a60b3cb5854ca", size = 23043098, upload-time = "2026-02-23T00:21:56.185Z" },
{ url = "https://files.pythonhosted.org/packages/86/f1/3383beb9b5d0dbddd030335bf8a8b32d4317185efe495374f134d8be6cce/scipy-1.17.1-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e3dcd57ab780c741fde8dc68619de988b966db759a3c3152e8e9142c26295ad", size = 33030397, upload-time = "2026-02-23T00:22:01.404Z" },
{ url = "https://files.pythonhosted.org/packages/41/68/8f21e8a65a5a03f25a79165ec9d2b28c00e66dc80546cf5eb803aeeff35b/scipy-1.17.1-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a9956e4d4f4a301ebf6cde39850333a6b6110799d470dbbb1e25326ac447f52a", size = 35281163, upload-time = "2026-02-23T00:22:07.024Z" },
{ url = "https://files.pythonhosted.org/packages/84/8d/c8a5e19479554007a5632ed7529e665c315ae7492b4f946b0deb39870e39/scipy-1.17.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:a4328d245944d09fd639771de275701ccadf5f781ba0ff092ad141e017eccda4", size = 35116291, upload-time = "2026-02-23T00:22:12.585Z" },
{ url = "https://files.pythonhosted.org/packages/52/52/e57eceff0e342a1f50e274264ed47497b59e6a4e3118808ee58ddda7b74a/scipy-1.17.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a77cbd07b940d326d39a1d1b37817e2ee4d79cb30e7338f3d0cddffae70fcaa2", size = 37682317, upload-time = "2026-02-23T00:22:18.513Z" },
{ url = "https://files.pythonhosted.org/packages/11/2f/b29eafe4a3fbc3d6de9662b36e028d5f039e72d345e05c250e121a230dd4/scipy-1.17.1-cp314-cp314t-win_amd64.whl", hash = "sha256:eb092099205ef62cd1782b006658db09e2fed75bffcae7cc0d44052d8aa0f484", size = 37345327, upload-time = "2026-02-23T00:22:24.442Z" },
{ url = "https://files.pythonhosted.org/packages/07/39/338d9219c4e87f3e708f18857ecd24d22a0c3094752393319553096b98af/scipy-1.17.1-cp314-cp314t-win_arm64.whl", hash = "sha256:200e1050faffacc162be6a486a984a0497866ec54149a01270adc8a59b7c7d21", size = 25489165, upload-time = "2026-02-23T00:22:29.563Z" },
]
[[package]]
name = "sentence-transformers"
version = "5.2.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "huggingface-hub", marker = "python_full_version >= '3.14'" },
{ name = "numpy", marker = "python_full_version >= '3.14'" },
{ name = "scikit-learn", marker = "python_full_version >= '3.14'" },
{ name = "scipy", marker = "python_full_version >= '3.14'" },
{ name = "torch", marker = "python_full_version >= '3.14'" },
{ name = "tqdm", marker = "python_full_version >= '3.14'" },
{ name = "transformers", marker = "python_full_version >= '3.14'" },
{ name = "typing-extensions", marker = "python_full_version >= '3.14'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5b/30/21664028fc0776eb1ca024879480bbbab36f02923a8ff9e4cae5a150fa35/sentence_transformers-5.2.3.tar.gz", hash = "sha256:3cd3044e1f3fe859b6a1b66336aac502eaae5d3dd7d5c8fc237f37fbf58137c7", size = 381623, upload-time = "2026-02-17T14:05:20.238Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/46/9f/dba4b3e18ebbe1eaa29d9f1764fbc7da0cd91937b83f2b7928d15c5d2d36/sentence_transformers-5.2.3-py3-none-any.whl", hash = "sha256:6437c62d4112b615ddebda362dfc16a4308d604c5b68125ed586e3e95d5b2e30", size = 494225, upload-time = "2026-02-17T14:05:18.596Z" },
]
[[package]]
name = "setuptools"
version = "80.10.2"
@@ -4852,6 +5197,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/9c/78/96ddb99933e11d91bc6e05edae23d2687e44213066bcbaca338898c73c47/textual-7.5.0-py3-none-any.whl", hash = "sha256:849dfee9d705eab3b2d07b33152b7bd74fb1f5056e002873cc448bce500c6374", size = 718164, upload-time = "2026-01-30T13:46:37.635Z" },
]
[[package]]
name = "threadpoolctl"
version = "3.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b7/4d/08c89e34946fce2aec4fbb45c9016efd5f4d7f24af8e5d93296e935631d8/threadpoolctl-3.6.0.tar.gz", hash = "sha256:8ab8b4aa3491d812b623328249fab5302a68d2d71745c8a4c719a2fcaba9f44e", size = 21274, upload-time = "2025-03-13T13:49:23.031Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/32/d5/f9a850d79b0851d1d4ef6456097579a9005b31fea68726a4ae5f2d82ddd9/threadpoolctl-3.6.0-py3-none-any.whl", hash = "sha256:43a0b8fd5a2928500110039e43a5eed8480b918967083ea48dc3ab9f13c4a7fb", size = 18638, upload-time = "2025-03-13T13:49:21.846Z" },
]
[[package]]
name = "tiktoken"
version = "0.12.0"
@@ -4986,6 +5340,66 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/23/d1/136eb2cb77520a31e1f64cbae9d33ec6df0d78bdf4160398e86eec8a8754/tomli-2.4.0-py3-none-any.whl", hash = "sha256:1f776e7d669ebceb01dee46484485f43a4048746235e683bcdffacdf1fb4785a", size = 14477, upload-time = "2026-01-11T11:22:37.446Z" },
]
[[package]]
name = "torch"
version = "2.10.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cuda-bindings", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "filelock", marker = "python_full_version >= '3.14'" },
{ name = "fsspec", marker = "python_full_version >= '3.14'" },
{ name = "jinja2", marker = "python_full_version >= '3.14'" },
{ name = "networkx", marker = "python_full_version >= '3.14'" },
{ name = "nvidia-cublas-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cuda-cupti-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cuda-nvrtc-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cuda-runtime-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cudnn-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cufft-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cufile-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-curand-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cusolver-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cusparse-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-cusparselt-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-nccl-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-nvjitlink-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-nvshmem-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "nvidia-nvtx-cu12", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "setuptools", marker = "python_full_version >= '3.14'" },
{ name = "sympy", marker = "python_full_version >= '3.14'" },
{ name = "triton", marker = "python_full_version >= '3.14' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "typing-extensions", marker = "python_full_version >= '3.14'" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/0f/8b/4b61d6e13f7108f36910df9ab4b58fd389cc2520d54d81b88660804aad99/torch-2.10.0-2-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:418997cb02d0a0f1497cf6a09f63166f9f5df9f3e16c8a716ab76a72127c714f", size = 79423467, upload-time = "2026-02-10T21:44:48.711Z" },
{ url = "https://files.pythonhosted.org/packages/d3/54/a2ba279afcca44bbd320d4e73675b282fcee3d81400ea1b53934efca6462/torch-2.10.0-2-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:13ec4add8c3faaed8d13e0574f5cd4a323c11655546f91fbe6afa77b57423574", size = 79498202, upload-time = "2026-02-10T21:44:52.603Z" },
{ url = "https://files.pythonhosted.org/packages/ec/23/2c9fe0c9c27f7f6cb865abcea8a4568f29f00acaeadfc6a37f6801f84cb4/torch-2.10.0-2-cp313-none-macosx_11_0_arm64.whl", hash = "sha256:e521c9f030a3774ed770a9c011751fb47c4d12029a3d6522116e48431f2ff89e", size = 79498254, upload-time = "2026-02-10T21:44:44.095Z" },
{ url = "https://files.pythonhosted.org/packages/78/89/f5554b13ebd71e05c0b002f95148033e730d3f7067f67423026cc9c69410/torch-2.10.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:3282d9febd1e4e476630a099692b44fdc214ee9bf8ee5377732d9d9dfe5712e4", size = 145992610, upload-time = "2026-01-21T16:25:26.327Z" },
{ url = "https://files.pythonhosted.org/packages/ae/30/a3a2120621bf9c17779b169fc17e3dc29b230c29d0f8222f499f5e159aa8/torch-2.10.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:a2f9edd8dbc99f62bc4dfb78af7bf89499bca3d753423ac1b4e06592e467b763", size = 915607863, upload-time = "2026-01-21T16:25:06.696Z" },
{ url = "https://files.pythonhosted.org/packages/6f/3d/c87b33c5f260a2a8ad68da7147e105f05868c281c63d65ed85aa4da98c66/torch-2.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:29b7009dba4b7a1c960260fc8ac85022c784250af43af9fb0ebafc9883782ebd", size = 113723116, upload-time = "2026-01-21T16:25:21.916Z" },
{ url = "https://files.pythonhosted.org/packages/61/d8/15b9d9d3a6b0c01b883787bd056acbe5cc321090d4b216d3ea89a8fcfdf3/torch-2.10.0-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:b7bd80f3477b830dd166c707c5b0b82a898e7b16f59a7d9d42778dd058272e8b", size = 79423461, upload-time = "2026-01-21T16:24:50.266Z" },
{ url = "https://files.pythonhosted.org/packages/cc/af/758e242e9102e9988969b5e621d41f36b8f258bb4a099109b7a4b4b50ea4/torch-2.10.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:5fd4117d89ffd47e3dcc71e71a22efac24828ad781c7e46aaaf56bf7f2796acf", size = 145996088, upload-time = "2026-01-21T16:24:44.171Z" },
{ url = "https://files.pythonhosted.org/packages/23/8e/3c74db5e53bff7ed9e34c8123e6a8bfef718b2450c35eefab85bb4a7e270/torch-2.10.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:787124e7db3b379d4f1ed54dd12ae7c741c16a4d29b49c0226a89bea50923ffb", size = 915711952, upload-time = "2026-01-21T16:23:53.503Z" },
{ url = "https://files.pythonhosted.org/packages/6e/01/624c4324ca01f66ae4c7cd1b74eb16fb52596dce66dbe51eff95ef9e7a4c/torch-2.10.0-cp312-cp312-win_amd64.whl", hash = "sha256:2c66c61f44c5f903046cc696d088e21062644cbe541c7f1c4eaae88b2ad23547", size = 113757972, upload-time = "2026-01-21T16:24:39.516Z" },
{ url = "https://files.pythonhosted.org/packages/c9/5c/dee910b87c4d5c0fcb41b50839ae04df87c1cfc663cf1b5fca7ea565eeaa/torch-2.10.0-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:6d3707a61863d1c4d6ebba7be4ca320f42b869ee657e9b2c21c736bf17000294", size = 79498198, upload-time = "2026-01-21T16:24:34.704Z" },
{ url = "https://files.pythonhosted.org/packages/c9/6f/f2e91e34e3fcba2e3fc8d8f74e7d6c22e74e480bbd1db7bc8900fdf3e95c/torch-2.10.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:5c4d217b14741e40776dd7074d9006fd28b8a97ef5654db959d8635b2fe5f29b", size = 146004247, upload-time = "2026-01-21T16:24:29.335Z" },
{ url = "https://files.pythonhosted.org/packages/98/fb/5160261aeb5e1ee12ee95fe599d0541f7c976c3701d607d8fc29e623229f/torch-2.10.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:6b71486353fce0f9714ca0c9ef1c850a2ae766b409808acd58e9678a3edb7738", size = 915716445, upload-time = "2026-01-21T16:22:45.353Z" },
{ url = "https://files.pythonhosted.org/packages/6a/16/502fb1b41e6d868e8deb5b0e3ae926bbb36dab8ceb0d1b769b266ad7b0c3/torch-2.10.0-cp313-cp313-win_amd64.whl", hash = "sha256:c2ee399c644dc92ef7bc0d4f7e74b5360c37cdbe7c5ba11318dda49ffac2bc57", size = 113757050, upload-time = "2026-01-21T16:24:19.204Z" },
{ url = "https://files.pythonhosted.org/packages/1a/0b/39929b148f4824bc3ad6f9f72a29d4ad865bcf7ebfc2fa67584773e083d2/torch-2.10.0-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:3202429f58309b9fa96a614885eace4b7995729f44beb54d3e4a47773649d382", size = 79851305, upload-time = "2026-01-21T16:24:09.209Z" },
{ url = "https://files.pythonhosted.org/packages/d8/14/21fbce63bc452381ba5f74a2c0a959fdf5ad5803ccc0c654e752e0dbe91a/torch-2.10.0-cp313-cp313t-manylinux_2_28_aarch64.whl", hash = "sha256:aae1b29cd68e50a9397f5ee897b9c24742e9e306f88a807a27d617f07adb3bd8", size = 146005472, upload-time = "2026-01-21T16:22:29.022Z" },
{ url = "https://files.pythonhosted.org/packages/54/fd/b207d1c525cb570ef47f3e9f836b154685011fce11a2f444ba8a4084d042/torch-2.10.0-cp313-cp313t-manylinux_2_28_x86_64.whl", hash = "sha256:6021db85958db2f07ec94e1bc77212721ba4920c12a18dc552d2ae36a3eb163f", size = 915612644, upload-time = "2026-01-21T16:21:47.019Z" },
{ url = "https://files.pythonhosted.org/packages/36/53/0197f868c75f1050b199fe58f9bf3bf3aecac9b4e85cc9c964383d745403/torch-2.10.0-cp313-cp313t-win_amd64.whl", hash = "sha256:ff43db38af76fda183156153983c9a096fc4c78d0cd1e07b14a2314c7f01c2c8", size = 113997015, upload-time = "2026-01-21T16:23:00.767Z" },
{ url = "https://files.pythonhosted.org/packages/0e/13/e76b4d9c160e89fff48bf16b449ea324bda84745d2ab30294c37c2434c0d/torch-2.10.0-cp313-none-macosx_11_0_arm64.whl", hash = "sha256:cdf2a523d699b70d613243211ecaac14fe9c5df8a0b0a9c02add60fb2a413e0f", size = 79498248, upload-time = "2026-01-21T16:23:09.315Z" },
{ url = "https://files.pythonhosted.org/packages/4f/93/716b5ac0155f1be70ed81bacc21269c3ece8dba0c249b9994094110bfc51/torch-2.10.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:bf0d9ff448b0218e0433aeb198805192346c4fd659c852370d5cc245f602a06a", size = 79464992, upload-time = "2026-01-21T16:23:05.162Z" },
{ url = "https://files.pythonhosted.org/packages/69/2b/51e663ff190c9d16d4a8271203b71bc73a16aa7619b9f271a69b9d4a936b/torch-2.10.0-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:233aed0659a2503b831d8a67e9da66a62c996204c0bba4f4c442ccc0c68a3f60", size = 146018567, upload-time = "2026-01-21T16:22:23.393Z" },
{ url = "https://files.pythonhosted.org/packages/5e/cd/4b95ef7f293b927c283db0b136c42be91c8ec6845c44de0238c8c23bdc80/torch-2.10.0-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:682497e16bdfa6efeec8cde66531bc8d1fbbbb4d8788ec6173c089ed3cc2bfe5", size = 915721646, upload-time = "2026-01-21T16:21:16.983Z" },
{ url = "https://files.pythonhosted.org/packages/56/97/078a007208f8056d88ae43198833469e61a0a355abc0b070edd2c085eb9a/torch-2.10.0-cp314-cp314-win_amd64.whl", hash = "sha256:6528f13d2a8593a1a412ea07a99812495bec07e9224c28b2a25c0a30c7da025c", size = 113752373, upload-time = "2026-01-21T16:22:13.471Z" },
{ url = "https://files.pythonhosted.org/packages/d8/94/71994e7d0d5238393df9732fdab607e37e2b56d26a746cb59fdb415f8966/torch-2.10.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:f5ab4ba32383061be0fb74bda772d470140a12c1c3b58a0cfbf3dae94d164c28", size = 79850324, upload-time = "2026-01-21T16:22:09.494Z" },
{ url = "https://files.pythonhosted.org/packages/e2/65/1a05346b418ea8ccd10360eef4b3e0ce688fba544e76edec26913a8d0ee0/torch-2.10.0-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:716b01a176c2a5659c98f6b01bf868244abdd896526f1c692712ab36dbaf9b63", size = 146006482, upload-time = "2026-01-21T16:22:18.42Z" },
{ url = "https://files.pythonhosted.org/packages/1d/b9/5f6f9d9e859fc3235f60578fa64f52c9c6e9b4327f0fe0defb6de5c0de31/torch-2.10.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:d8f5912ba938233f86361e891789595ff35ca4b4e2ac8fe3670895e5976731d6", size = 915613050, upload-time = "2026-01-21T16:20:49.035Z" },
{ url = "https://files.pythonhosted.org/packages/66/4d/35352043ee0eaffdeff154fad67cd4a31dbed7ff8e3be1cc4549717d6d51/torch-2.10.0-cp314-cp314t-win_amd64.whl", hash = "sha256:71283a373f0ee2c89e0f0d5f446039bdabe8dbc3c9ccf35f0f784908b0acd185", size = 113995816, upload-time = "2026-01-21T16:22:05.312Z" },
]
[[package]]
name = "tqdm"
version = "4.67.2"
@@ -4998,6 +5412,39 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f5/e2/31eac96de2915cf20ccaed0225035db149dfb9165a9ed28d4b252ef3f7f7/tqdm-4.67.2-py3-none-any.whl", hash = "sha256:9a12abcbbff58b6036b2167d9d3853042b9d436fe7330f06ae047867f2f8e0a7", size = 78354, upload-time = "2026-01-30T23:12:04.368Z" },
]
[[package]]
name = "transformers"
version = "5.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "huggingface-hub", marker = "python_full_version >= '3.14'" },
{ name = "numpy", marker = "python_full_version >= '3.14'" },
{ name = "packaging", marker = "python_full_version >= '3.14'" },
{ name = "pyyaml", marker = "python_full_version >= '3.14'" },
{ name = "regex", marker = "python_full_version >= '3.14'" },
{ name = "safetensors", marker = "python_full_version >= '3.14'" },
{ name = "tokenizers", marker = "python_full_version >= '3.14'" },
{ name = "tqdm", marker = "python_full_version >= '3.14'" },
{ name = "typer", marker = "python_full_version >= '3.14'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/fc/1a/70e830d53ecc96ce69cfa8de38f163712d2b43ac52fbd743f39f56025c31/transformers-5.3.0.tar.gz", hash = "sha256:009555b364029da9e2946d41f1c5de9f15e6b1df46b189b7293f33a161b9c557", size = 8830831, upload-time = "2026-03-04T17:41:46.119Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b8/88/ae8320064e32679a5429a2c9ebbc05c2bf32cefb6e076f9b07f6d685a9b4/transformers-5.3.0-py3-none-any.whl", hash = "sha256:50ac8c89c3c7033444fb3f9f53138096b997ebb70d4b5e50a2e810bf12d3d29a", size = 10661827, upload-time = "2026-03-04T17:41:42.722Z" },
]
[[package]]
name = "triton"
version = "3.6.0"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e0/12/b05ba554d2c623bffa59922b94b0775673de251f468a9609bc9e45de95e9/triton-3.6.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8e323d608e3a9bfcc2d9efcc90ceefb764a82b99dea12a86d643c72539ad5d3", size = 188214640, upload-time = "2026-01-20T16:00:35.869Z" },
{ url = "https://files.pythonhosted.org/packages/ab/a8/cdf8b3e4c98132f965f88c2313a4b493266832ad47fb52f23d14d4f86bb5/triton-3.6.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:74caf5e34b66d9f3a429af689c1c7128daba1d8208df60e81106b115c00d6fca", size = 188266850, upload-time = "2026-01-20T16:00:43.041Z" },
{ url = "https://files.pythonhosted.org/packages/f9/0b/37d991d8c130ce81a8728ae3c25b6e60935838e9be1b58791f5997b24a54/triton-3.6.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:10c7f76c6e72d2ef08df639e3d0d30729112f47a56b0c81672edc05ee5116ac9", size = 188289450, upload-time = "2026-01-20T16:00:49.136Z" },
{ url = "https://files.pythonhosted.org/packages/35/f8/9c66bfc55361ec6d0e4040a0337fb5924ceb23de4648b8a81ae9d33b2b38/triton-3.6.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d002e07d7180fd65e622134fbd980c9a3d4211fb85224b56a0a0efbd422ab72f", size = 188400296, upload-time = "2026-01-20T16:00:56.042Z" },
{ url = "https://files.pythonhosted.org/packages/df/3d/9e7eee57b37c80cec63322c0231bb6da3cfe535a91d7a4d64896fcb89357/triton-3.6.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a17a5d5985f0ac494ed8a8e54568f092f7057ef60e1b0fa09d3fd1512064e803", size = 188273063, upload-time = "2026-01-20T16:01:07.278Z" },
{ url = "https://files.pythonhosted.org/packages/f6/56/6113c23ff46c00aae423333eb58b3e60bdfe9179d542781955a5e1514cb3/triton-3.6.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:46bd1c1af4b6704e554cad2eeb3b0a6513a980d470ccfa63189737340c7746a7", size = 188397994, upload-time = "2026-01-20T16:01:14.236Z" },
]
[[package]]
name = "typer"
version = "0.21.1"

42
web/package-lock.json generated
View File

@@ -32,7 +32,7 @@
"@radix-ui/react-tooltip": "^1.2.7",
"@tailwindcss/postcss": "^4.1.5",
"@tanstack/react-table": "^8.21.3",
"axios": "^1.12.0",
"axios": "^1.13.5",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"highlight.js": "^11.11.1",
@@ -56,6 +56,7 @@
"rehype-autolink-headings": "^7.1.0",
"rehype-highlight": "^7.0.2",
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"rehype-slug": "^6.0.0",
"remark-gfm": "^4.0.1",
"sonner": "^2.0.3",
@@ -3798,13 +3799,13 @@
}
},
"node_modules/axios": {
"version": "1.13.4",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.13.4.tgz",
"integrity": "sha512-1wVkUaAO6WyaYtCkcYCOx12ZgpGf9Zif+qXa4n+oYzK558YryKqiL6UWwd5DqiH3VRW0GYhTZQ/vlgJrCoNQlg==",
"version": "1.13.6",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.13.6.tgz",
"integrity": "sha512-ChTCHMouEe2kn713WHbQGcuYrr6fXTBiu460OTwWrWob16g1bXn4vtz07Ope7ewMozJAnEquLk5lWQWtBig9DQ==",
"license": "MIT",
"dependencies": {
"follow-redirects": "^1.15.6",
"form-data": "^4.0.4",
"follow-redirects": "^1.15.11",
"form-data": "^4.0.5",
"proxy-from-env": "^1.1.0"
}
},
@@ -5970,6 +5971,21 @@
"url": "https://opencollective.com/unified"
}
},
"node_modules/hast-util-sanitize": {
"version": "5.0.2",
"resolved": "https://registry.npmjs.org/hast-util-sanitize/-/hast-util-sanitize-5.0.2.tgz",
"integrity": "sha512-3yTWghByc50aGS7JlGhk61SPenfE/p1oaFeNwkOOyrscaOkMGrcW9+Cy/QAIOBpZxP1yqDIzFMR0+Np0i0+usg==",
"license": "MIT",
"dependencies": {
"@types/hast": "^3.0.0",
"@ungap/structured-clone": "^1.0.0",
"unist-util-position": "^5.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/unified"
}
},
"node_modules/hast-util-to-jsx-runtime": {
"version": "2.3.6",
"resolved": "https://registry.npmjs.org/hast-util-to-jsx-runtime/-/hast-util-to-jsx-runtime-2.3.6.tgz",
@@ -9392,6 +9408,20 @@
"url": "https://opencollective.com/unified"
}
},
"node_modules/rehype-sanitize": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/rehype-sanitize/-/rehype-sanitize-6.0.0.tgz",
"integrity": "sha512-CsnhKNsyI8Tub6L4sm5ZFsme4puGfc6pYylvXo1AeqaGbjOYyzNv3qZPwvs0oMJ39eryyeOdmxwUIo94IpEhqg==",
"license": "MIT",
"dependencies": {
"@types/hast": "^3.0.0",
"hast-util-sanitize": "^5.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/unified"
}
},
"node_modules/rehype-slug": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/rehype-slug/-/rehype-slug-6.0.0.tgz",

View File

@@ -6,8 +6,8 @@
"dev": "next dev --turbopack",
"build": "next build",
"start": "next start",
"lint": "eslint .",
"lint:fix": "eslint . --fix",
"lint": "eslint src",
"lint:fix": "eslint src --fix",
"lint-staged": "lint-staged"
},
"lint-staged": {
@@ -68,6 +68,7 @@
"rehype-autolink-headings": "^7.1.0",
"rehype-highlight": "^7.0.2",
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"rehype-slug": "^6.0.0",
"remark-gfm": "^4.0.1",
"sonner": "^2.0.3",
@@ -102,4 +103,4 @@
"typescript-eslint": "^8.31.1"
},
"packageManager": "pnpm@8.9.2+sha512.b9d35fe91b2a5854dadc43034a3e7b2e675fa4b56e20e8e09ef078fa553c18f8aed44051e7b36e8b8dd435f97eb0c44c4ff3b44fc7c6fa7d21e1fac17bbe661e"
}
}

54
web/pnpm-lock.yaml generated
View File

@@ -149,6 +149,9 @@ dependencies:
rehype-raw:
specifier: ^7.0.0
version: 7.0.0
rehype-sanitize:
specifier: ^6.0.0
version: 6.0.0
rehype-slug:
specifier: ^6.0.0
version: 6.0.0
@@ -505,6 +508,7 @@ packages:
resolution: {integrity: sha512-excjX8DfsIcJ10x1Kzr4RcWe1edC9PquDRRPx3YVCvQv+U5p7Yin2s32ftzikXojb1PIFc/9Mt28/y+iRklkrw==}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -513,6 +517,7 @@ packages:
resolution: {integrity: sha512-bFI7xcKFELdiNCVov8e44Ia4u2byA+l3XtsAj+Q8tfCwO6BQ8iDojYdvoPMqsKDkuoOo+X6HZA0s0q11ANMQ8A==}
cpu: [arm]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -521,6 +526,7 @@ packages:
resolution: {integrity: sha512-FMuvGijLDYG6lW+b/UvyilUWu5Ayu+3r2d1S8notiGCIyYU/76eig1UfMmkZ7vwgOrzKzlQbFSuQfgm7GYUPpA==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -529,6 +535,7 @@ packages:
resolution: {integrity: sha512-oVDbcR4zUC0ce82teubSm+x6ETixtKZBh/qbREIOcI3cULzDyb18Sr/Wcyx7NRQeQzOiHTNbZFF1UwPS2scyGA==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -537,6 +544,7 @@ packages:
resolution: {integrity: sha512-qmp9VrzgPgMoGZyPvrQHqk02uyjA0/QrTO26Tqk6l4ZV0MPWIW6LTkqOIov+J1yEu7MbFQaDpwdwJKhbJvuRxQ==}
cpu: [s390x]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -545,6 +553,7 @@ packages:
resolution: {integrity: sha512-tJxiiLsmHc9Ax1bz3oaOYBURTXGIRDODBqhveVHonrHJ9/+k89qbLl0bcJns+e4t4rvaNBxaEZsFtSfAdquPrw==}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -553,6 +562,7 @@ packages:
resolution: {integrity: sha512-FVQHuwx1IIuNow9QAbYUzJ+En8KcVm9Lk5+uGUQJHaZmMECZmOlix9HnH7n1TRkXMS0pGxIJokIVB9SuqZGGXw==}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -561,6 +571,7 @@ packages:
resolution: {integrity: sha512-+LpyBk7L44ZIXwz/VYfglaX/okxezESc6UxDSoyo2Ks6Jxc4Y7sGjpgU9s4PMgqgjj1gZCylTieNamqA1MF7Dg==}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -570,6 +581,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-arm64': 1.2.4
@@ -581,6 +593,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [arm]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-arm': 1.2.4
@@ -592,6 +605,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [ppc64]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-ppc64': 1.2.4
@@ -603,6 +617,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [riscv64]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-riscv64': 1.2.4
@@ -614,6 +629,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [s390x]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-s390x': 1.2.4
@@ -625,6 +641,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linux-x64': 1.2.4
@@ -636,6 +653,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linuxmusl-arm64': 1.2.4
@@ -647,6 +665,7 @@ packages:
engines: {node: ^18.17.0 || ^20.3.0 || >=21.0.0}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
optionalDependencies:
'@img/sharp-libvips-linuxmusl-x64': 1.2.4
@@ -763,6 +782,7 @@ packages:
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -772,6 +792,7 @@ packages:
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -781,6 +802,7 @@ packages:
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -790,6 +812,7 @@ packages:
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -1889,6 +1912,7 @@ packages:
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -1898,6 +1922,7 @@ packages:
engines: {node: '>= 10'}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -1907,6 +1932,7 @@ packages:
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -1916,6 +1942,7 @@ packages:
engines: {node: '>= 10'}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -2331,6 +2358,7 @@ packages:
resolution: {integrity: sha512-34gw7PjDGB9JgePJEmhEqBhWvCiiWCuXsL9hYphDF7crW7UgI05gyBAi6MF58uGcMOiOqSJ2ybEeCvHcq0BCmQ==}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: true
optional: true
@@ -2339,6 +2367,7 @@ packages:
resolution: {integrity: sha512-RyMIx6Uf53hhOtJDIamSbTskA99sPHS96wxVE/bJtePJJtpdKGXO1wY90oRdXuYOGOTuqjT8ACccMc4K6QmT3w==}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: true
optional: true
@@ -2347,6 +2376,7 @@ packages:
resolution: {integrity: sha512-D8Vae74A4/a+mZH0FbOkFJL9DSK2R6TFPC9M+jCWYia/q2einCubX10pecpDiTmkJVUH+y8K3BZClycD8nCShA==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: true
optional: true
@@ -2355,6 +2385,7 @@ packages:
resolution: {integrity: sha512-frxL4OrzOWVVsOc96+V3aqTIQl1O2TjgExV4EKgRY09AJ9leZpEg8Ak9phadbuX0BA4k8U5qtvMSQQGGmaJqcQ==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: true
optional: true
@@ -2363,6 +2394,7 @@ packages:
resolution: {integrity: sha512-mJ5vuDaIZ+l/acv01sHoXfpnyrNKOk/3aDoEdLO/Xtn9HuZlDD6jKxHlkN8ZhWyLJsRBxfv9GYM2utQ1SChKew==}
cpu: [riscv64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: true
optional: true
@@ -2371,6 +2403,7 @@ packages:
resolution: {integrity: sha512-kELo8ebBVtb9sA7rMe1Cph4QHreByhaZ2QEADd9NzIQsYNQpt9UkM9iqr2lhGr5afh885d/cB5QeTXSbZHTYPg==}
cpu: [s390x]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: true
optional: true
@@ -2379,6 +2412,7 @@ packages:
resolution: {integrity: sha512-C3ZAHugKgovV5YvAMsxhq0gtXuwESUKc5MhEtjBpLoHPLYM+iuwSj3lflFwK3DPm68660rZ7G8BMcwSro7hD5w==}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: true
optional: true
@@ -2387,6 +2421,7 @@ packages:
resolution: {integrity: sha512-rV0YSoyhK2nZ4vEswT/QwqzqQXw5I6CjoaYMOX0TqBlWhojUf8P94mvI7nuJTeaCkkds3QE4+zS8Ko+GdXuZtA==}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: true
optional: true
@@ -3873,6 +3908,14 @@ packages:
zwitch: 2.0.4
dev: false
/hast-util-sanitize@5.0.2:
resolution: {integrity: sha512-3yTWghByc50aGS7JlGhk61SPenfE/p1oaFeNwkOOyrscaOkMGrcW9+Cy/QAIOBpZxP1yqDIzFMR0+Np0i0+usg==}
dependencies:
'@types/hast': 3.0.4
'@ungap/structured-clone': 1.3.0
unist-util-position: 5.0.0
dev: false
/hast-util-to-jsx-runtime@2.3.6:
resolution: {integrity: sha512-zl6s8LwNyo1P9uw+XJGvZtdFF1GdAkOg8ujOw+4Pyb76874fLps4ueHXDhXWdk6YHQ6OgUtinliG7RsYvCbbBg==}
dependencies:
@@ -4413,6 +4456,7 @@ packages:
engines: {node: '>= 12.0.0'}
cpu: [arm64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -4422,6 +4466,7 @@ packages:
engines: {node: '>= 12.0.0'}
cpu: [arm64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -4431,6 +4476,7 @@ packages:
engines: {node: '>= 12.0.0'}
cpu: [x64]
os: [linux]
libc: [glibc]
requiresBuild: true
dev: false
optional: true
@@ -4440,6 +4486,7 @@ packages:
engines: {node: '>= 12.0.0'}
cpu: [x64]
os: [linux]
libc: [musl]
requiresBuild: true
dev: false
optional: true
@@ -5713,6 +5760,13 @@ packages:
vfile: 6.0.3
dev: false
/rehype-sanitize@6.0.0:
resolution: {integrity: sha512-CsnhKNsyI8Tub6L4sm5ZFsme4puGfc6pYylvXo1AeqaGbjOYyzNv3qZPwvs0oMJ39eryyeOdmxwUIo94IpEhqg==}
dependencies:
'@types/hast': 3.0.4
hast-util-sanitize: 5.0.2
dev: false
/rehype-slug@6.0.0:
resolution: {integrity: sha512-lWyvf/jwu+oS5+hL5eClVd3hNdmwM1kAC0BUvEGD19pajQMIzcNUd/k9GsfQ+FfECvX+JE+e9/btsKH0EjJT6A==}
dependencies:

View File

@@ -5,6 +5,7 @@ import {
Dialog,
DialogContent,
DialogHeader,
DialogDescription,
DialogTitle,
DialogFooter,
} from '@/components/ui/dialog';
@@ -21,6 +22,7 @@ import {
import { Button } from '@/components/ui/button';
import BotForm from '@/app/home/bots/components/bot-form/BotForm';
import { BotLogListComponent } from '@/app/home/bots/components/bot-log/view/BotLogListComponent';
import BotSessionMonitor from '@/app/home/bots/components/bot-session/BotSessionMonitor';
import { useTranslation } from 'react-i18next';
import { z } from 'zod';
import { httpClient } from '@/app/infra/http/HttpClient';
@@ -82,6 +84,19 @@ export default function BotDetailDialog({
</svg>
),
},
{
key: 'sessions',
label: t('bots.sessionMonitor.title'),
icon: (
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
>
<path d="M2 22C2 17.5817 5.58172 14 10 14C14.4183 14 18 17.5817 18 22H16C16 18.6863 13.3137 16 10 16C6.68629 16 4 18.6863 4 22H2ZM10 13C6.685 13 4 10.315 4 7C4 3.685 6.685 1 10 1C13.315 1 16 3.685 16 7C16 10.315 13.315 13 10 13ZM10 11C12.21 11 14 9.21 14 7C14 4.79 12.21 3 10 3C7.79 3 6 4.79 6 7C6 9.21 7.79 11 10 11ZM18.2837 14.7028C21.0644 15.9561 23 18.752 23 22H21C21 19.564 19.5483 17.4671 17.4628 16.5271L18.2837 14.7028ZM17.5962 3.41321C19.5944 4.23703 21 6.20361 21 8.5C21 11.3702 18.8042 13.7252 16 13.9776V11.9646C17.6967 11.7222 19 10.264 19 8.5C19 7.11935 18.2016 5.92603 17.041 5.35635L17.5962 3.41321Z"></path>
</svg>
),
},
];
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -122,6 +137,9 @@ export default function BotDetailDialog({
<main className="flex flex-1 flex-col h-[70vh]">
<DialogHeader className="px-6 pt-6 pb-4 shrink-0">
<DialogTitle>{t('bots.createBot')}</DialogTitle>
<DialogDescription className="sr-only">
{t('bots.createBot')}
</DialogDescription>
</DialogHeader>
<div className="flex-1 overflow-y-auto px-6 pb-6">
<BotForm
@@ -155,7 +173,7 @@ export default function BotDetailDialog({
return (
<>
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="overflow-hidden p-0 !max-w-[50rem] max-h-[75vh] flex">
<DialogContent className="overflow-hidden p-0 !max-w-[70rem] max-h-[75vh] flex">
<SidebarProvider className="items-start w-full flex">
<Sidebar
collapsible="none"
@@ -189,10 +207,25 @@ export default function BotDetailDialog({
<DialogTitle>
{activeMenu === 'config'
? t('bots.editBot')
: t('bots.botLogTitle')}
: activeMenu === 'logs'
? t('bots.botLogTitle')
: t('bots.sessionMonitor.title')}
</DialogTitle>
<DialogDescription className="sr-only">
{activeMenu === 'config'
? t('bots.editBot')
: activeMenu === 'logs'
? t('bots.botLogTitle')
: t('bots.sessionMonitor.title')}
</DialogDescription>
</DialogHeader>
<div className="flex-1 overflow-y-auto px-6 pb-6">
<div
className={
activeMenu === 'sessions'
? 'flex-1 min-h-0'
: 'flex-1 overflow-y-auto px-6 pb-6'
}
>
{activeMenu === 'config' && (
<BotForm
initBotId={botId}
@@ -204,6 +237,9 @@ export default function BotDetailDialog({
{activeMenu === 'logs' && botId && (
<BotLogListComponent botId={botId} />
)}
{activeMenu === 'sessions' && botId && (
<BotSessionMonitor botId={botId} />
)}
</div>
{activeMenu === 'config' && (
<DialogFooter className="px-6 py-4 border-t shrink-0">
@@ -238,6 +274,9 @@ export default function BotDetailDialog({
<DialogContent>
<DialogHeader>
<DialogTitle>{t('common.confirmDelete')}</DialogTitle>
<DialogDescription className="sr-only">
{t('bots.deleteConfirmation')}
</DialogDescription>
</DialogHeader>
<div className="py-4">{t('bots.deleteConfirmation')}</div>
<DialogFooter>

View File

@@ -124,6 +124,12 @@ export default function BotForm({
const currentAdapter = form.watch('adapter');
const currentAdapterConfig = form.watch('adapter_config');
// Serialize adapter_config to a stable string so it can be used as a
// useEffect dependency without triggering on every render. form.watch()
// returns a new object reference each time, which would otherwise cause
// the filtering effect below to loop indefinitely.
const adapterConfigJson = JSON.stringify(currentAdapterConfig);
useEffect(() => {
setBotFormValues();
}, []);
@@ -147,7 +153,7 @@ export default function BotForm({
// For non-Lark adapters, show all fields
setFilteredDynamicFormConfigList(dynamicFormConfigList);
}
}, [currentAdapter, currentAdapterConfig, dynamicFormConfigList]);
}, [currentAdapter, adapterConfigJson, dynamicFormConfigList]);
// 复制到剪贴板的辅助函数 - 使用页面上的真实input元素
const copyToClipboard = () => {
@@ -313,6 +319,7 @@ export default function BotForm({
required: item.required,
type: parseDynamicFormItemType(item.type),
options: item.options,
show_if: item.show_if,
}),
),
);

View File

@@ -0,0 +1,502 @@
'use client';
import React, { useState, useEffect, useRef, useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import { httpClient } from '@/app/infra/http/HttpClient';
import { ScrollArea } from '@/components/ui/scroll-area';
import { Button } from '@/components/ui/button';
import { cn } from '@/lib/utils';
import {
MessageChainComponent,
Plain,
At,
Image,
Quote,
Voice,
} from '@/app/infra/entities/message';
interface SessionInfo {
session_id: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
message_count: number;
start_time: string;
last_activity: string;
is_active: boolean;
platform?: string | null;
user_id?: string | null;
}
interface SessionMessage {
id: string;
timestamp: string;
bot_id: string;
bot_name: string;
pipeline_id: string;
pipeline_name: string;
message_content: string;
session_id: string;
status: string;
level: string;
platform?: string | null;
user_id?: string | null;
runner_name?: string | null;
variables?: string | null;
role?: string | null;
}
interface BotSessionMonitorProps {
botId: string;
}
export default function BotSessionMonitor({ botId }: BotSessionMonitorProps) {
const { t } = useTranslation();
const [sessions, setSessions] = useState<SessionInfo[]>([]);
const [selectedSessionId, setSelectedSessionId] = useState<string | null>(
null,
);
const [messages, setMessages] = useState<SessionMessage[]>([]);
const [loadingSessions, setLoadingSessions] = useState(false);
const [loadingMessages, setLoadingMessages] = useState(false);
const messagesContainerRef = useRef<HTMLDivElement>(null);
const loadSessions = useCallback(async () => {
setLoadingSessions(true);
try {
const response = await httpClient.getBotSessions(botId);
setSessions(response.sessions ?? []);
} catch (error) {
console.error('Failed to load sessions:', error);
} finally {
setLoadingSessions(false);
}
}, [botId]);
const loadMessages = useCallback(async (sessionId: string) => {
setLoadingMessages(true);
try {
const response = await httpClient.getSessionMessages(sessionId);
const sorted = (response.messages ?? []).sort(
(a, b) =>
new Date(a.timestamp).getTime() - new Date(b.timestamp).getTime(),
);
setMessages(sorted);
} catch (error) {
console.error('Failed to load session messages:', error);
} finally {
setLoadingMessages(false);
}
}, []);
useEffect(() => {
loadSessions();
}, [loadSessions]);
useEffect(() => {
if (selectedSessionId) {
loadMessages(selectedSessionId);
} else {
setMessages([]);
}
}, [selectedSessionId, loadMessages]);
useEffect(() => {
const container = messagesContainerRef.current;
if (container) {
const viewport = container.querySelector(
'[data-radix-scroll-area-viewport]',
);
const scrollTarget = viewport || container;
scrollTarget.scrollTop = scrollTarget.scrollHeight;
}
}, [messages]);
const parseMessageChain = (content: string): MessageChainComponent[] => {
try {
const parsed = JSON.parse(content);
if (Array.isArray(parsed)) {
return parsed as MessageChainComponent[];
}
} catch {
// Not JSON, return as plain text
}
return [{ type: 'Plain', text: content } as Plain];
};
const isUserMessage = (msg: SessionMessage): boolean => {
if (msg.role === 'assistant') return false;
if (msg.role === 'user') return true;
return !msg.runner_name;
};
const renderMessageComponent = (
component: MessageChainComponent,
index: number,
) => {
switch (component.type) {
case 'Plain':
return <span key={index}>{(component as Plain).text}</span>;
case 'At': {
const atComponent = component as At;
const displayName =
atComponent.display || atComponent.target?.toString() || '';
return (
<span
key={index}
className="inline-flex align-middle mx-0.5 px-1.5 py-0.5 bg-blue-200/60 dark:bg-blue-800/60 text-blue-700 dark:text-blue-300 rounded-md text-xs font-medium"
>
@{displayName}
</span>
);
}
case 'AtAll':
return (
<span
key={index}
className="inline-flex align-middle mx-0.5 px-1.5 py-0.5 bg-blue-200/60 dark:bg-blue-800/60 text-blue-700 dark:text-blue-300 rounded-md text-xs font-medium"
>
@All
</span>
);
case 'Image': {
const img = component as Image;
const imageUrl = img.url || (img.base64 ? img.base64 : '');
if (!imageUrl) {
return (
<span
key={index}
className="inline-flex items-center gap-1 text-muted-foreground text-xs"
>
[Image]
</span>
);
}
return (
<div key={index} className="my-1.5">
<img
src={imageUrl}
alt="Image"
className="max-w-full max-h-52 rounded-lg"
/>
</div>
);
}
case 'Voice': {
const voice = component as Voice;
const voiceUrl = voice.url || (voice.base64 ? voice.base64 : '');
if (!voiceUrl) {
return (
<span
key={index}
className="inline-flex items-center gap-1 text-muted-foreground text-xs"
>
🎙 [Voice]
</span>
);
}
return (
<div key={index} className="my-1">
<audio controls src={voiceUrl} className="h-8 max-w-[220px]" />
</div>
);
}
case 'Quote': {
const quote = component as Quote;
return (
<div
key={index}
className="mb-2 pl-2.5 border-l-2 border-gray-300 dark:border-gray-600 opacity-80"
>
<div className="text-sm">
{quote.origin?.map((comp, idx) =>
renderMessageComponent(comp as MessageChainComponent, idx),
)}
</div>
</div>
);
}
case 'Source':
return null;
case 'File': {
const file = component as MessageChainComponent & { name?: string };
return (
<span key={index} className="text-muted-foreground text-xs">
📎 {file.name || 'File'}
</span>
);
}
default:
return (
<span key={index} className="text-muted-foreground text-xs">
[{component.type}]
</span>
);
}
};
const renderMessageContent = (msg: SessionMessage) => {
const chain = parseMessageChain(msg.message_content);
return (
<div className="whitespace-pre-wrap break-words">
{chain.map((component, index) =>
renderMessageComponent(component, index),
)}
</div>
);
};
const formatTime = (timestamp: string): string => {
if (!timestamp) return '';
const date = new Date(timestamp);
const hours = date.getHours().toString().padStart(2, '0');
const minutes = date.getMinutes().toString().padStart(2, '0');
return `${hours}:${minutes}`;
};
const formatRelativeTime = (timestamp: string): string => {
if (!timestamp) return '';
const date = new Date(timestamp);
const now = new Date();
const diffMs = now.getTime() - date.getTime();
const diffMins = Math.floor(diffMs / 60000);
const diffHours = Math.floor(diffMs / 3600000);
const diffDays = Math.floor(diffMs / 86400000);
if (diffMins < 1) return '<1m';
if (diffMins < 60) return `${diffMins}m`;
if (diffHours < 24) return `${diffHours}h`;
return `${diffDays}d`;
};
const selectedSession = sessions.find(
(s) => s.session_id === selectedSessionId,
);
return (
<div className="flex h-full min-h-0">
{/* Left Panel: Session List */}
<div className="w-64 flex-shrink-0 border-r flex flex-col min-h-0">
{/* Refresh Button */}
<div className="px-2 py-2 border-b shrink-0">
<Button
variant="ghost"
className="w-full h-9 text-sm text-muted-foreground"
onClick={loadSessions}
disabled={loadingSessions}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
className={cn(
'w-3.5 h-3.5 mr-1.5',
loadingSessions && 'animate-spin',
)}
>
<path d="M21.5 2v6h-6M2.5 22v-6h6M2 11.5a10 10 0 0 1 18.8-4.3M22 12.5a10 10 0 0 1-18.8 4.2" />
</svg>
{t('bots.sessionMonitor.refresh')}
</Button>
</div>
{/* Session List */}
<ScrollArea className="flex-1 min-h-0">
{loadingSessions && sessions.length === 0 ? (
<div className="flex items-center justify-center py-12 text-sm text-muted-foreground">
{t('bots.sessionMonitor.loading')}
</div>
) : sessions.length === 0 ? (
<div className="text-center text-muted-foreground py-12 text-sm">
{t('bots.sessionMonitor.noSessions')}
</div>
) : (
<div className="p-1">
{sessions.map((session) => {
const isSelected = selectedSessionId === session.session_id;
return (
<button
key={session.session_id}
className={cn(
'w-full text-left px-3 py-2.5 rounded-md transition-colors',
isSelected ? 'bg-accent' : 'hover:bg-accent/50',
)}
onClick={() => setSelectedSessionId(session.session_id)}
>
<div className="flex items-center justify-between mb-0.5">
<span className="text-sm font-medium truncate mr-2">
{session.user_id || session.session_id.slice(0, 12)}
</span>
<span className="text-[11px] text-muted-foreground tabular-nums flex-shrink-0">
{formatRelativeTime(session.last_activity)}
</span>
</div>
<div className="flex items-center gap-1.5 text-xs text-muted-foreground">
{session.platform && (
<span className="px-1 py-0.5 rounded bg-muted text-[10px]">
{session.platform}
</span>
)}
{session.is_active && (
<span className="flex items-center gap-0.5 text-green-600 dark:text-green-400">
<span className="w-1.5 h-1.5 rounded-full bg-green-500 inline-block" />
</span>
)}
<span>{session.pipeline_name}</span>
</div>
</button>
);
})}
</div>
)}
</ScrollArea>
</div>
{/* Right Panel: Messages */}
<div className="flex-1 flex flex-col min-h-0 min-w-0">
{!selectedSessionId ? (
<div className="text-center text-muted-foreground py-12 text-lg flex-1 flex items-center justify-center">
{t('bots.sessionMonitor.selectSession')}
</div>
) : (
<>
{/* Chat Header */}
<div className="px-6 py-3 border-b shrink-0 flex items-center justify-between">
<div className="min-w-0">
<div className="text-sm font-medium truncate">
{selectedSession?.user_id || selectedSessionId.slice(0, 20)}
</div>
<div className="flex items-center gap-2 text-xs text-muted-foreground">
{selectedSession?.platform && (
<span>{selectedSession.platform}</span>
)}
{selectedSession?.pipeline_name && (
<>
{selectedSession?.platform && <span>·</span>}
<span>{selectedSession.pipeline_name}</span>
</>
)}
{selectedSession?.is_active && (
<>
<span>·</span>
<span className="flex items-center gap-1 text-green-600 dark:text-green-400">
<span className="w-1.5 h-1.5 rounded-full bg-green-500 inline-block" />
Active
</span>
</>
)}
</div>
</div>
<Button
variant="ghost"
size="icon"
className="w-8 h-8"
onClick={() => loadMessages(selectedSessionId)}
disabled={loadingMessages}
>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth={2}
strokeLinecap="round"
strokeLinejoin="round"
className={cn('w-4 h-4', loadingMessages && 'animate-spin')}
>
<path d="M21.5 2v6h-6M2.5 22v-6h6M2 11.5a10 10 0 0 1 18.8-4.3M22 12.5a10 10 0 0 1-18.8 4.2" />
</svg>
</Button>
</div>
{/* Messages Area — matches DebugDialog style */}
<ScrollArea
ref={messagesContainerRef}
className="flex-1 p-6 overflow-y-auto min-h-0 bg-white dark:bg-black"
>
<div className="space-y-6">
{loadingMessages ? (
<div className="text-center text-muted-foreground py-12 text-lg">
{t('bots.sessionMonitor.loading')}
</div>
) : messages.length === 0 ? (
<div className="text-center text-muted-foreground py-12 text-lg">
{t('bots.sessionMonitor.noMessages')}
</div>
) : (
messages.map((msg) => {
const isUser = isUserMessage(msg);
return (
<div
key={msg.id}
className={cn(
'flex',
isUser ? 'justify-end' : 'justify-start',
)}
>
<div
className={cn(
'max-w-3xl px-5 py-3 rounded-2xl',
isUser
? 'bg-blue-100 dark:bg-blue-900 text-gray-900 dark:text-gray-100 rounded-br-none'
: 'bg-gray-100 dark:bg-gray-800 text-gray-900 dark:text-gray-100 rounded-bl-none',
msg.status === 'error' && 'ring-1 ring-red-400/50',
)}
>
{renderMessageContent(msg)}
{/* Role label + timestamp inside bubble, matching DebugDialog */}
<div
className={cn(
'text-xs mt-2 flex items-center gap-2',
isUser
? 'text-gray-600 dark:text-gray-300'
: 'text-gray-500 dark:text-gray-400',
)}
>
<span>
{isUser
? t('bots.sessionMonitor.userMessage', {
defaultValue: 'User',
})
: t('bots.sessionMonitor.botMessage', {
defaultValue: 'Assistant',
})}
</span>
<span className="tabular-nums">
{formatTime(msg.timestamp)}
</span>
{msg.status === 'error' && (
<span className="text-red-500">error</span>
)}
{msg.runner_name && (
<span className="opacity-70">
{msg.runner_name}
</span>
)}
</div>
</div>
</div>
);
})
)}
</div>
</ScrollArea>
</>
)}
</div>
</div>
);
}

View File

@@ -11,22 +11,28 @@ import {
FormMessage,
} from '@/components/ui/form';
import DynamicFormItemComponent from '@/app/home/components/dynamic-form/DynamicFormItemComponent';
import { useEffect, useRef } from 'react';
import { useCallback, useEffect, useRef } from 'react';
import { extractI18nObject } from '@/i18n/I18nProvider';
import { useTranslation } from 'react-i18next';
export default function DynamicFormComponent({
itemConfigList,
onSubmit,
initialValues,
onFileUploaded,
isEditing,
externalDependentValues,
}: {
itemConfigList: IDynamicFormItemSchema[];
onSubmit?: (val: object) => unknown;
initialValues?: Record<string, object>;
onFileUploaded?: (fileKey: string) => void;
isEditing?: boolean;
externalDependentValues?: Record<string, unknown>;
}) {
const isInitialMount = useRef(true);
const previousInitialValues = useRef(initialValues);
const { t } = useTranslation();
// 根据 itemConfigList 动态生成 zod schema
const formSchema = z.object(
@@ -55,6 +61,9 @@ export default function DynamicFormComponent({
case 'llm-model-selector':
fieldSchema = z.string();
break;
case 'embedding-model-selector':
fieldSchema = z.string();
break;
case 'knowledge-base-selector':
fieldSchema = z.string();
break;
@@ -81,7 +90,9 @@ export default function DynamicFormComponent({
(fieldSchema instanceof z.ZodString ||
fieldSchema instanceof z.ZodArray)
) {
fieldSchema = fieldSchema.min(1, { message: '此字段为必填项' });
fieldSchema = fieldSchema.min(1, {
message: t('common.fieldRequired'),
});
}
return {
@@ -141,54 +152,120 @@ export default function DynamicFormComponent({
}
}, [initialValues, form, itemConfigList]);
// Get reactive form values for conditional rendering
const watchedValues = form.watch();
// Stable ref for onSubmit to avoid re-triggering the effect when the
// parent passes a new closure on every render.
const onSubmitRef = useRef(onSubmit);
onSubmitRef.current = onSubmit;
// Track the last emitted values to avoid emitting identical snapshots,
// which would cause the parent to call setValue with an equivalent object,
// triggering a re-render loop.
const lastEmittedRef = useRef<string>('');
const emitValues = useCallback(() => {
const formValues = form.getValues();
const finalValues = itemConfigList.reduce(
(acc, item) => {
acc[item.name] = formValues[item.name] ?? item.default;
return acc;
},
{} as Record<string, object>,
);
const serialized = JSON.stringify(finalValues);
if (serialized !== lastEmittedRef.current) {
lastEmittedRef.current = serialized;
onSubmitRef.current?.(finalValues);
}
}, [form, itemConfigList]);
// 监听表单值变化
useEffect(() => {
// Emit initial form values immediately so the parent always has a valid snapshot,
// even if the user saves without modifying any field.
// form.watch(callback) only fires on subsequent changes, not on mount.
emitValues();
const subscription = form.watch(() => {
// 获取完整的表单值,确保包含所有默认值
const formValues = form.getValues();
const finalValues = itemConfigList.reduce(
(acc, item) => {
acc[item.name] = formValues[item.name] ?? item.default;
return acc;
},
{} as Record<string, object>,
);
onSubmit?.(finalValues);
emitValues();
});
return () => subscription.unsubscribe();
}, [form, onSubmit, itemConfigList]);
}, [form, itemConfigList, emitValues]);
return (
<Form {...form}>
<div className="space-y-4">
{itemConfigList.map((config) => (
<FormField
key={config.id}
control={form.control}
name={config.name as keyof FormValues}
render={({ field }) => (
<FormItem>
<FormLabel>
{extractI18nObject(config.label)}{' '}
{config.required && <span className="text-red-500">*</span>}
</FormLabel>
<FormControl>
<DynamicFormItemComponent
config={config}
field={field}
onFileUploaded={onFileUploaded}
/>
</FormControl>
{config.description && (
<p className="text-sm text-muted-foreground">
{extractI18nObject(config.description)}
</p>
)}
<FormMessage />
</FormItem>
)}
/>
))}
{itemConfigList.map((config) => {
if (config.show_if) {
const dependValue =
watchedValues[
config.show_if.field as keyof typeof watchedValues
] !== undefined
? watchedValues[
config.show_if.field as keyof typeof watchedValues
]
: externalDependentValues?.[config.show_if.field];
if (
config.show_if.operator === 'eq' &&
dependValue !== config.show_if.value
) {
return null;
}
if (
config.show_if.operator === 'neq' &&
dependValue === config.show_if.value
) {
return null;
}
if (
config.show_if.operator === 'in' &&
Array.isArray(config.show_if.value) &&
!config.show_if.value.includes(dependValue)
) {
return null;
}
}
// All fields are disabled when editing (creation_settings are immutable)
const isFieldDisabled = !!isEditing;
return (
<FormField
key={config.id}
control={form.control}
name={config.name as keyof FormValues}
render={({ field }) => (
<FormItem>
<FormLabel>
{extractI18nObject(config.label)}{' '}
{config.required && <span className="text-red-500">*</span>}
</FormLabel>
<FormControl>
<div
className={
isFieldDisabled ? 'pointer-events-none opacity-60' : ''
}
>
<DynamicFormItemComponent
config={config}
field={field}
onFileUploaded={onFileUploaded}
/>
</div>
</FormControl>
{config.description && (
<p className="text-sm text-muted-foreground">
{extractI18nObject(config.description)}
</p>
)}
<FormMessage />
</FormItem>
)}
/>
);
})}
</div>
</Form>
);

View File

@@ -22,8 +22,7 @@ import {
LLMModel,
Bot,
KnowledgeBase,
ExternalKnowledgeBase,
ApiRespPluginSystemStatus,
EmbeddingModel,
} from '@/app/infra/entities/api';
import { toast } from 'sonner';
import { useTranslation } from 'react-i18next';
@@ -51,16 +50,12 @@ export default function DynamicFormItemComponent({
onFileUploaded?: (fileKey: string) => void;
}) {
const [llmModels, setLlmModels] = useState<LLMModel[]>([]);
const [embeddingModels, setEmbeddingModels] = useState<EmbeddingModel[]>([]);
const [knowledgeBases, setKnowledgeBases] = useState<KnowledgeBase[]>([]);
const [externalKnowledgeBases, setExternalKnowledgeBases] = useState<
ExternalKnowledgeBase[]
>([]);
const [bots, setBots] = useState<Bot[]>([]);
const [uploading, setUploading] = useState<boolean>(false);
const [kbDialogOpen, setKbDialogOpen] = useState(false);
const [tempSelectedKBIds, setTempSelectedKBIds] = useState<string[]>([]);
const [pluginSystemStatus, setPluginSystemStatus] =
useState<ApiRespPluginSystemStatus | null>(null);
const { t } = useTranslation();
const handleFileUpload = async (file: File): Promise<IFileConfig | null> => {
@@ -111,7 +106,20 @@ export default function DynamicFormItemComponent({
setLlmModels(models);
})
.catch((err) => {
toast.error('Failed to get LLM model list: ' + err.msg);
toast.error(t('models.getModelListError') + err.msg);
});
}
}, [config.type]);
useEffect(() => {
if (config.type === DynamicFormItemType.EMBEDDING_MODEL_SELECTOR) {
httpClient
.getProviderEmbeddingModels()
.then((resp) => {
setEmbeddingModels(resp.models);
})
.catch((err) => {
toast.error(t('embedding.getModelListError') + err.msg);
});
}
}, [config.type]);
@@ -127,39 +135,11 @@ export default function DynamicFormItemComponent({
setKnowledgeBases(resp.bases);
})
.catch((err) => {
toast.error('Failed to get knowledge base list: ' + err.msg);
});
// Fetch plugin system status
httpClient
.getPluginSystemStatus()
.then((status) => {
setPluginSystemStatus(status);
})
.catch((err) => {
console.error('Failed to get plugin system status:', err);
toast.error(t('knowledge.getKnowledgeBaseListError') + err.msg);
});
}
}, [config.type]);
useEffect(() => {
if (
(config.type === DynamicFormItemType.KNOWLEDGE_BASE_SELECTOR ||
config.type === DynamicFormItemType.KNOWLEDGE_BASE_MULTI_SELECTOR) &&
pluginSystemStatus?.is_enable &&
pluginSystemStatus?.is_connected
) {
httpClient
.getExternalKnowledgeBases()
.then((resp) => {
setExternalKnowledgeBases(resp.bases);
})
.catch((err) => {
console.error('Failed to get external knowledge base list:', err);
});
}
}, [config.type, pluginSystemStatus]);
useEffect(() => {
if (config.type === DynamicFormItemType.BOT_SELECTOR) {
httpClient
@@ -168,7 +148,7 @@ export default function DynamicFormItemComponent({
setBots(resp.bots);
})
.catch((err) => {
toast.error('Failed to get bot list: ' + err.msg);
toast.error(t('bots.getBotListError') + err.msg);
});
}
}, [config.type]);
@@ -191,7 +171,12 @@ export default function DynamicFormItemComponent({
return <Textarea {...field} className="min-h-[120px]" />;
case DynamicFormItemType.BOOLEAN:
return <Switch checked={field.value} onCheckedChange={field.onChange} />;
return (
<Switch
checked={field.value ?? false}
onCheckedChange={field.onChange}
/>
);
case DynamicFormItemType.STRING_ARRAY:
return (
@@ -242,7 +227,7 @@ export default function DynamicFormItemComponent({
case DynamicFormItemType.SELECT:
return (
<Select value={field.value} onValueChange={field.onChange}>
<Select value={field.value ?? ''} onValueChange={field.onChange}>
<SelectTrigger className="bg-[#ffffff] dark:bg-[#2a2a2e]">
<SelectValue placeholder={t('common.select')} />
</SelectTrigger>
@@ -299,7 +284,56 @@ export default function DynamicFormItemComponent({
</Select>
);
case DynamicFormItemType.EMBEDDING_MODEL_SELECTOR:
// Group embedding models by provider
const groupedEmbeddingModels = embeddingModels.reduce(
(acc, model) => {
const providerName = model.provider?.name || 'Unknown';
if (!acc[providerName]) acc[providerName] = [];
acc[providerName].push(model);
return acc;
},
{} as Record<string, EmbeddingModel[]>,
);
return (
<Select value={field.value} onValueChange={field.onChange}>
<SelectTrigger className="bg-[#ffffff] dark:bg-[#2a2a2e]">
<SelectValue placeholder={t('knowledge.selectEmbeddingModel')} />
</SelectTrigger>
<SelectContent>
{Object.entries(groupedEmbeddingModels).map(
([providerName, models]) => (
<SelectGroup key={providerName}>
<SelectLabel>{providerName}</SelectLabel>
{models.map((model) => (
<SelectItem key={model.uuid} value={model.uuid}>
{model.name}
</SelectItem>
))}
</SelectGroup>
),
)}
</SelectContent>
</Select>
);
case DynamicFormItemType.KNOWLEDGE_BASE_SELECTOR:
// Group KBs by Knowledge Engine name
const kbsByEngine = knowledgeBases.reduce(
(acc, kb) => {
const engineName = kb.knowledge_engine?.name
? extractI18nObject(kb.knowledge_engine.name)
: t('knowledge.unknownEngine');
if (!acc[engineName]) {
acc[engineName] = [];
}
acc[engineName].push(kb);
return acc;
},
{} as Record<string, typeof knowledgeBases>,
);
return (
<Select value={field.value} onValueChange={field.onChange}>
<SelectTrigger className="bg-[#ffffff] dark:bg-[#2a2a2e]">
@@ -310,53 +344,45 @@ export default function DynamicFormItemComponent({
<SelectItem value="__none__">{t('knowledge.empty')}</SelectItem>
</SelectGroup>
{knowledgeBases.length > 0 && (
<SelectGroup>
<SelectLabel>{t('knowledge.builtIn')}</SelectLabel>
{knowledgeBases.map((base) => (
{Object.entries(kbsByEngine).map(([engineName, kbs]) => (
<SelectGroup key={engineName}>
<SelectLabel>{engineName}</SelectLabel>
{kbs.map((base) => (
<SelectItem key={base.uuid} value={base.uuid ?? ''}>
{base.name}
</SelectItem>
))}
</SelectGroup>
)}
{externalKnowledgeBases.length > 0 && (
<SelectGroup>
<SelectLabel>{t('knowledge.external')}</SelectLabel>
{externalKnowledgeBases.map((base) => (
<SelectItem key={base.uuid} value={base.uuid ?? ''}>
<div className="flex items-center gap-2">
<img
src={httpClient.getPluginIconURL(
base.plugin_author,
base.plugin_name,
)}
alt="plugin icon"
className="w-4 h-4 rounded-[8%] flex-shrink-0"
/>
<span>{base.name}</span>
</div>
</SelectItem>
))}
</SelectGroup>
)}
))}
</SelectContent>
</Select>
);
case DynamicFormItemType.KNOWLEDGE_BASE_MULTI_SELECTOR:
// Group KBs by Knowledge Engine name for multi-selector
const multiKbsByEngine = knowledgeBases.reduce(
(acc, kb) => {
const engineName = kb.knowledge_engine?.name
? extractI18nObject(kb.knowledge_engine.name)
: t('knowledge.unknownEngine');
if (!acc[engineName]) {
acc[engineName] = [];
}
acc[engineName].push(kb);
return acc;
},
{} as Record<string, typeof knowledgeBases>,
);
return (
<>
<div className="space-y-2">
{field.value && field.value.length > 0 ? (
<div className="space-y-2">
{field.value.map((kbId: string) => {
const kb = knowledgeBases.find((base) => base.uuid === kbId);
const externalKb = externalKnowledgeBases.find(
const currentKb = knowledgeBases.find(
(base) => base.uuid === kbId,
);
const currentKb = kb || externalKb;
if (!currentKb) return null;
return (
@@ -365,18 +391,17 @@ export default function DynamicFormItemComponent({
className="flex items-center justify-between rounded-lg border p-3 hover:bg-accent"
>
<div className="flex items-center gap-2 flex-1">
{externalKb && (
<img
src={httpClient.getPluginIconURL(
externalKb.plugin_author,
externalKb.plugin_name,
)}
alt="plugin icon"
className="w-8 h-8 rounded-[8%] flex-shrink-0"
/>
)}
<div className="flex-1 min-w-0">
<div className="font-medium">{currentKb.name}</div>
<div className="font-medium flex items-center gap-2">
{currentKb.name}
{currentKb.knowledge_engine?.name && (
<span className="text-xs px-2 py-0.5 rounded-full bg-purple-100 text-purple-700 dark:bg-purple-900 dark:text-purple-300">
{extractI18nObject(
currentKb.knowledge_engine.name,
)}
</span>
)}
</div>
{currentKb.description && (
<div className="text-sm text-muted-foreground">
{currentKb.description}
@@ -430,13 +455,12 @@ export default function DynamicFormItemComponent({
<DialogTitle>{t('knowledge.selectKnowledgeBases')}</DialogTitle>
</DialogHeader>
<div className="flex-1 overflow-y-auto space-y-4 pr-2">
{/* Built-in Knowledge Bases */}
{knowledgeBases.length > 0 && (
<div className="space-y-2">
{Object.entries(multiKbsByEngine).map(([engineName, kbs]) => (
<div key={engineName} className="space-y-2">
<div className="text-sm font-semibold text-muted-foreground px-2">
{t('knowledge.builtIn')}
{engineName}
</div>
{knowledgeBases.map((base) => {
{kbs.map((base) => {
const isSelected = tempSelectedKBIds.includes(
base.uuid ?? '',
);
@@ -469,56 +493,7 @@ export default function DynamicFormItemComponent({
);
})}
</div>
)}
{/* External Knowledge Bases */}
{externalKnowledgeBases.length > 0 && (
<div className="space-y-2">
<div className="text-sm font-semibold text-muted-foreground px-2">
{t('knowledge.external')}
</div>
{externalKnowledgeBases.map((base) => {
const isSelected = tempSelectedKBIds.includes(
base.uuid ?? '',
);
return (
<div
key={base.uuid}
className="flex items-center gap-3 rounded-lg border p-3 hover:bg-accent cursor-pointer"
onClick={() => {
const kbId = base.uuid ?? '';
setTempSelectedKBIds((prev) =>
prev.includes(kbId)
? prev.filter((id) => id !== kbId)
: [...prev, kbId],
);
}}
>
<Checkbox
checked={isSelected}
aria-label={`Select ${base.name}`}
/>
<img
src={httpClient.getPluginIconURL(
base.plugin_author,
base.plugin_name,
)}
alt="plugin icon"
className="w-8 h-8 rounded-[8%] flex-shrink-0"
/>
<div className="flex-1">
<div className="font-medium">{base.name}</div>
{base.description && (
<div className="text-sm text-muted-foreground">
{base.description}
</div>
)}
</div>
</div>
);
})}
</div>
)}
))}
</div>
<DialogFooter>
<Button

View File

@@ -2,6 +2,7 @@ import {
IDynamicFormItemSchema,
DynamicFormItemType,
IDynamicFormItemOption,
IShowIfCondition,
} from '@/app/infra/entities/form/dynamic';
import { I18nObject } from '@/app/infra/entities/common';
@@ -14,6 +15,7 @@ export class DynamicFormItemConfig implements IDynamicFormItemSchema {
type: DynamicFormItemType;
description?: I18nObject;
options?: IDynamicFormItemOption[];
show_if?: IShowIfCondition;
constructor(params: IDynamicFormItemSchema) {
this.id = params.id;
@@ -24,6 +26,7 @@ export class DynamicFormItemConfig implements IDynamicFormItemSchema {
this.type = params.type;
this.description = params.description;
this.options = params.options;
this.show_if = params.show_if;
}
}

View File

@@ -4,6 +4,7 @@ import { useTranslation } from 'react-i18next';
import ReactMarkdown from 'react-markdown';
import remarkGfm from 'remark-gfm';
import rehypeRaw from 'rehype-raw';
import rehypeSanitize from 'rehype-sanitize';
import rehypeHighlight from 'rehype-highlight';
import i18n from 'i18next';
import { ExternalLink } from 'lucide-react';
@@ -62,7 +63,7 @@ export default function NewVersionDialog({
<div className="markdown-body max-w-none text-sm">
<ReactMarkdown
remarkPlugins={[remarkGfm]}
rehypePlugins={[rehypeRaw, rehypeHighlight]}
rehypePlugins={[rehypeRaw, rehypeSanitize, rehypeHighlight]}
components={{
ul: ({ children }) => <ul className="list-disc">{children}</ul>,
ol: ({ children }) => (

View File

@@ -0,0 +1,391 @@
'use client';
import React, { useState, useEffect, useCallback } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient';
import type {
SurveyQuestion,
SurveyOption,
} from '@/app/infra/http/BackendClient';
import { X, ChevronRight, ChevronLeft, MessageSquare } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Checkbox } from '@/components/ui/checkbox';
/**
* Get i18n text from a Record<string, string> based on browser locale.
*/
function getI18nText(obj?: Record<string, string> | null): string {
if (!obj) return '';
const lang = typeof navigator !== 'undefined' ? navigator.language : 'en';
if (lang.startsWith('zh'))
return obj['zh_Hans'] || obj['en_US'] || Object.values(obj)[0] || '';
if (lang.startsWith('ja'))
return obj['ja_JP'] || obj['en_US'] || Object.values(obj)[0] || '';
return obj['en_US'] || Object.values(obj)[0] || '';
}
interface SurveyData {
survey_id: string;
version: number;
title: Record<string, string>;
description: Record<string, string>;
questions: SurveyQuestion[];
}
export default function SurveyWidget() {
const [survey, setSurvey] = useState<SurveyData | null>(null);
const [visible, setVisible] = useState(false);
const [currentStep, setCurrentStep] = useState(0);
const [answers, setAnswers] = useState<Record<string, unknown>>({});
const [otherInputs, setOtherInputs] = useState<Record<string, string>>({});
const [submitted, setSubmitted] = useState(false);
const [collapsed, setCollapsed] = useState(false);
// Poll for pending survey
useEffect(() => {
let timer: NodeJS.Timeout;
let cancelled = false;
const checkSurvey = async () => {
try {
const resp = await httpClient.getSurveyPending();
if (!cancelled && resp?.survey) {
setSurvey(resp.survey);
setVisible(true);
}
} catch {
// Silently ignore
}
};
// Check after 5 seconds, then every 60 seconds
timer = setTimeout(() => {
checkSurvey();
timer = setInterval(checkSurvey, 60000) as unknown as NodeJS.Timeout;
}, 5000);
return () => {
cancelled = true;
clearTimeout(timer);
clearInterval(timer);
};
}, []);
const handleDismiss = useCallback(async () => {
if (survey) {
try {
await httpClient.dismissSurvey(survey.survey_id);
} catch {
/* ignore */
}
}
setVisible(false);
}, [survey]);
const handleSubmit = useCallback(async () => {
if (!survey) return;
// Merge "other" text inputs into answers
const finalAnswers = { ...answers };
for (const [qId, text] of Object.entries(otherInputs)) {
if (text.trim()) {
const current = finalAnswers[qId];
if (Array.isArray(current)) {
// Replace 'other' with the text
finalAnswers[qId] = (current as string[]).map((v) =>
v === 'other' ? `other:${text}` : v,
);
} else if (current === 'other') {
finalAnswers[qId] = `other:${text}`;
}
}
}
try {
await httpClient.submitSurveyResponse(
survey.survey_id,
finalAnswers,
true,
);
setSubmitted(true);
setTimeout(() => setVisible(false), 2000);
} catch {
/* ignore */
}
}, [survey, answers, otherInputs]);
const setAnswer = useCallback((qId: string, value: unknown) => {
setAnswers((prev) => ({ ...prev, [qId]: value }));
}, []);
if (!visible || !survey) return null;
const questions = survey.questions || [];
const totalSteps = questions.length;
const currentQuestion = questions[currentStep];
if (submitted) {
return (
<div className="fixed bottom-6 right-6 z-50 w-80 bg-card border rounded-xl shadow-lg p-6 animate-in slide-in-from-bottom-4">
<div className="text-center">
<div className="text-3xl mb-2">🎉</div>
<p className="text-sm font-medium">
{getI18nText({
zh_Hans: '感谢你的反馈!',
en_US: 'Thanks for your feedback!',
})}
</p>
</div>
</div>
);
}
if (collapsed) {
return (
<button
onClick={() => setCollapsed(false)}
className="fixed bottom-6 right-6 z-50 w-12 h-12 bg-primary text-primary-foreground rounded-full shadow-lg flex items-center justify-center hover:scale-105 transition-transform"
>
<MessageSquare className="w-5 h-5" />
</button>
);
}
return (
<div className="fixed bottom-6 right-6 z-50 w-[340px] bg-card border rounded-xl shadow-lg animate-in slide-in-from-bottom-4">
{/* Header */}
<div className="flex items-center justify-between px-4 py-3 border-b">
<div className="flex items-center gap-2">
<MessageSquare className="w-4 h-4 text-primary" />
<span className="text-sm font-medium">
{getI18nText(survey.title)}
</span>
</div>
<div className="flex items-center gap-1">
<button
onClick={() => setCollapsed(true)}
className="p-1 hover:bg-accent rounded"
>
<ChevronRight className="w-4 h-4 text-muted-foreground" />
</button>
<button
onClick={handleDismiss}
className="p-1 hover:bg-accent rounded"
>
<X className="w-4 h-4 text-muted-foreground" />
</button>
</div>
</div>
{/* Progress */}
<div className="px-4 pt-3">
<div className="flex gap-1">
{questions.map((_, i) => (
<div
key={i}
className={`h-1 flex-1 rounded-full transition-colors ${
i <= currentStep ? 'bg-primary' : 'bg-secondary'
}`}
/>
))}
</div>
<span className="text-xs text-muted-foreground mt-1 block">
{currentStep + 1} / {totalSteps}
</span>
</div>
{/* Question */}
<div className="px-4 py-3">
<p className="text-sm font-medium mb-1">
{getI18nText(currentQuestion?.title)}
</p>
{currentQuestion?.subtitle && (
<p className="text-xs text-muted-foreground mb-3">
{getI18nText(currentQuestion.subtitle)}
</p>
)}
<div className="space-y-2 max-h-[260px] overflow-y-auto">
{currentQuestion?.type === 'single_select' &&
currentQuestion.options && (
<SingleSelectField
options={currentQuestion.options}
value={answers[currentQuestion.id] as string}
onChange={(v) => setAnswer(currentQuestion.id, v)}
otherText={otherInputs[currentQuestion.id] || ''}
onOtherChange={(t) =>
setOtherInputs((prev) => ({
...prev,
[currentQuestion.id]: t,
}))
}
/>
)}
{currentQuestion?.type === 'multi_select' &&
currentQuestion.options && (
<MultiSelectField
options={currentQuestion.options}
value={(answers[currentQuestion.id] as string[]) || []}
onChange={(v) => setAnswer(currentQuestion.id, v)}
otherText={otherInputs[currentQuestion.id] || ''}
onOtherChange={(t) =>
setOtherInputs((prev) => ({
...prev,
[currentQuestion.id]: t,
}))
}
/>
)}
{currentQuestion?.type === 'text' && (
<textarea
className="w-full h-20 text-sm border rounded-lg p-2 bg-background resize-none focus:outline-none focus:ring-1 focus:ring-primary"
placeholder={getI18nText(currentQuestion.placeholder)}
maxLength={currentQuestion.max_length || 500}
value={(answers[currentQuestion.id] as string) || ''}
onChange={(e) => setAnswer(currentQuestion.id, e.target.value)}
/>
)}
</div>
</div>
{/* Footer */}
<div className="flex items-center justify-between px-4 py-3 border-t">
<Button
variant="ghost"
size="sm"
onClick={() => setCurrentStep(Math.max(0, currentStep - 1))}
disabled={currentStep === 0}
>
<ChevronLeft className="w-4 h-4" />
</Button>
<div className="flex gap-2">
{!currentQuestion?.required && currentStep < totalSteps - 1 && (
<Button
variant="ghost"
size="sm"
onClick={() => setCurrentStep(currentStep + 1)}
>
{getI18nText({ zh_Hans: '跳过', en_US: 'Skip' })}
</Button>
)}
{currentStep < totalSteps - 1 ? (
<Button
size="sm"
onClick={() => setCurrentStep(currentStep + 1)}
disabled={
currentQuestion?.required && !answers[currentQuestion?.id]
}
>
{getI18nText({ zh_Hans: '下一题', en_US: 'Next' })}
</Button>
) : (
<Button size="sm" onClick={handleSubmit}>
{getI18nText({ zh_Hans: '提交', en_US: 'Submit' })}
</Button>
)}
</div>
</div>
</div>
);
}
// ---- Sub-components for flat radio/checkbox style ----
function SingleSelectField({
options,
value,
onChange,
otherText,
onOtherChange,
}: {
options: SurveyOption[];
value?: string;
onChange: (v: string) => void;
otherText: string;
onOtherChange: (t: string) => void;
}) {
return (
<div className="space-y-1.5">
{options.map((opt) => (
<div key={opt.id}>
<button
onClick={() => onChange(opt.id)}
className={`w-full text-left text-sm px-3 py-2 rounded-lg border transition-colors ${
value === opt.id
? 'border-primary bg-primary/5 text-primary'
: 'border-border hover:bg-accent'
}`}
>
{getI18nText(opt.label)}
</button>
{opt.has_input && value === opt.id && (
<input
type="text"
className="mt-1 w-full text-sm border rounded-lg px-3 py-1.5 bg-background focus:outline-none focus:ring-1 focus:ring-primary"
placeholder="..."
value={otherText}
onChange={(e) => onOtherChange(e.target.value)}
/>
)}
</div>
))}
</div>
);
}
function MultiSelectField({
options,
value,
onChange,
otherText,
onOtherChange,
}: {
options: SurveyOption[];
value: string[];
onChange: (v: string[]) => void;
otherText: string;
onOtherChange: (t: string) => void;
}) {
const toggle = (id: string) => {
if (value.includes(id)) {
onChange(value.filter((v) => v !== id));
} else {
onChange([...value, id]);
}
};
return (
<div className="space-y-1.5">
{options.map((opt) => {
const selected = value.includes(opt.id);
return (
<div key={opt.id}>
<button
onClick={() => toggle(opt.id)}
className={`w-full text-left text-sm px-3 py-2 rounded-lg border transition-colors flex items-center gap-2 ${
selected
? 'border-primary bg-primary/5 text-primary'
: 'border-border hover:bg-accent'
}`}
>
<Checkbox checked={selected} className="pointer-events-none" />
{getI18nText(opt.label)}
</button>
{opt.has_input && selected && (
<input
type="text"
className="mt-1 w-full text-sm border rounded-lg px-3 py-1.5 bg-background focus:outline-none focus:ring-1 focus:ring-primary"
placeholder="..."
value={otherText}
onChange={(e) => onOtherChange(e.target.value)}
/>
)}
</div>
);
})}
</div>
);
}

View File

@@ -21,18 +21,17 @@ import {
import { Button } from '@/components/ui/button';
import { useTranslation } from 'react-i18next';
import { httpClient } from '@/app/infra/http/HttpClient';
// import { KnowledgeBase } from '@/app/infra/entities/api';
import { KnowledgeBase } from '@/app/infra/entities/api';
import { CustomApiError } from '@/app/infra/entities/common';
import { toast } from 'sonner';
import KBForm from '@/app/home/knowledge/components/kb-form/KBForm';
import KBDoc from '@/app/home/knowledge/components/kb-docs/KBDoc';
import KBRetrieve from '@/app/home/knowledge/components/kb-retrieve/KBRetrieve';
import ExternalKBForm from '@/app/home/knowledge/components/external-kb-form/ExternalKBForm';
import ExternalKBRetrieve from '@/app/home/knowledge/components/kb-retrieve/ExternalKBRetrieve';
import KBRetrieveGeneric from '@/app/home/knowledge/components/kb-retrieve/KBRetrieveGeneric';
interface KBDetailDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
kbId?: string;
kbType: 'builtin' | 'external';
onFormCancel: () => void;
onKbDeleted: () => void;
onNewKbCreated: (kbId: string) => void;
@@ -43,7 +42,6 @@ export default function KBDetailDialog({
open,
onOpenChange,
kbId: propKbId,
kbType,
onFormCancel,
onKbDeleted,
onNewKbCreated,
@@ -53,13 +51,41 @@ export default function KBDetailDialog({
const [kbId, setKbId] = useState<string | undefined>(propKbId);
const [activeMenu, setActiveMenu] = useState('metadata');
const [showDeleteConfirm, setShowDeleteConfirm] = useState(false);
const [kbInfo, setKbInfo] = useState<KnowledgeBase | null>(null);
useEffect(() => {
setKbId(propKbId);
setActiveMenu('metadata');
if (propKbId) {
loadKbInfo(propKbId);
} else {
setKbInfo(null);
}
}, [propKbId, open]);
// Build menu based on KB type
async function loadKbInfo(id: string) {
try {
const resp = await httpClient.getKnowledgeBase(id);
setKbInfo(resp.base);
} catch (e) {
console.error('Failed to load KB info:', e);
toast.error(
t('knowledge.loadKnowledgeBaseFailed') + (e as CustomApiError).msg,
);
}
}
// Check if this KB supports document management
const hasDocumentCapability = (): boolean => {
if (!kbInfo || !kbInfo.knowledge_engine) {
return false;
}
return (
kbInfo.knowledge_engine.capabilities?.includes('doc_ingestion') ?? false
);
};
// Build menu based on KB capabilities
const menu = [
{
key: 'metadata',
@@ -74,8 +100,8 @@ export default function KBDetailDialog({
</svg>
),
},
// Only show documents for builtin KB
...(kbType === 'builtin'
// Show documents only if capability is present
...(hasDocumentCapability()
? [
{
key: 'documents',
@@ -107,66 +133,51 @@ export default function KBDetailDialog({
},
];
const confirmDelete = () => {
const deletePromise =
kbType === 'builtin'
? httpClient.deleteKnowledgeBase(kbId ?? '')
: httpClient.deleteExternalKnowledgeBase(kbId ?? '');
deletePromise.then(() => {
const confirmDelete = async () => {
try {
await httpClient.deleteKnowledgeBase(kbId ?? '');
onKbDeleted();
});
setShowDeleteConfirm(false);
} catch (e) {
console.error('Failed to delete KB:', e);
toast.error(
t('knowledge.deleteKnowledgeBaseFailed') + (e as CustomApiError).msg,
);
} finally {
setShowDeleteConfirm(false);
}
};
// Retrieve function
const retrieveFunction = async (id: string, query: string) => {
return await httpClient.retrieveKnowledgeBase(id, query);
};
if (!kbId) {
// new kb
// New KB creation
return (
<Dialog open={open} onOpenChange={onOpenChange}>
<DialogContent className="overflow-hidden p-0 !max-w-[40vw] max-h-[70vh] flex">
<main className="flex flex-1 flex-col h-[70vh]">
<DialogHeader className="px-6 pt-6 pb-4 shrink-0">
<DialogTitle>
{kbType === 'builtin'
? t('knowledge.createKnowledgeBase')
: t('knowledge.addExternal')}
</DialogTitle>
<DialogTitle>{t('knowledge.createKnowledgeBase')}</DialogTitle>
</DialogHeader>
<div className="flex-1 overflow-y-auto px-6 pb-6">
{kbType === 'builtin' ? (
<KBForm
initKbId={undefined}
onNewKbCreated={onNewKbCreated}
onKbUpdated={onKbUpdated}
/>
) : (
<ExternalKBForm
initKBId={undefined}
onFormSubmit={() => onOpenChange(false)}
onKBDeleted={() => {}}
onNewKBCreated={onNewKbCreated}
/>
)}
<KBForm
initKbId={undefined}
onNewKbCreated={onNewKbCreated}
onKbUpdated={onKbUpdated}
/>
</div>
{activeMenu === 'metadata' && (
<DialogFooter className="px-6 py-4 border-t shrink-0">
<div className="flex justify-end gap-2">
<Button
type="submit"
form={kbType === 'builtin' ? 'kb-form' : 'external-kb-form'}
>
{t('common.save')}
</Button>
<Button
type="button"
variant="outline"
onClick={onFormCancel}
>
{t('common.cancel')}
</Button>
</div>
</DialogFooter>
)}
<DialogFooter className="px-6 py-4 border-t shrink-0">
<div className="flex justify-end gap-2">
<Button type="submit" form="kb-form">
{t('common.save')}
</Button>
<Button type="button" variant="outline" onClick={onFormCancel}>
{t('common.cancel')}
</Button>
</div>
</DialogFooter>
</main>
</DialogContent>
</Dialog>
@@ -205,7 +216,7 @@ export default function KBDetailDialog({
</SidebarGroup>
</SidebarContent>
</Sidebar>
<main className="flex flex-1 flex-col h-[75vh]">
<main className="flex flex-1 flex-col h-[75vh] min-w-0 overflow-x-hidden">
<DialogHeader className="px-6 pt-6 pb-4 shrink-0">
<DialogTitle>
{activeMenu === 'metadata'
@@ -216,33 +227,28 @@ export default function KBDetailDialog({
</DialogTitle>
</DialogHeader>
<div className="flex-1 overflow-y-auto px-6 pb-6">
{activeMenu === 'metadata' &&
(kbType === 'builtin' ? (
<KBForm
initKbId={kbId}
onNewKbCreated={onNewKbCreated}
onKbUpdated={onKbUpdated}
/>
) : (
<ExternalKBForm
initKBId={kbId}
onFormSubmit={() => onOpenChange(false)}
onKBDeleted={() => {
onKbDeleted();
onOpenChange(false);
}}
onNewKBCreated={onNewKbCreated}
/>
))}
{activeMenu === 'documents' && kbType === 'builtin' && (
<KBDoc kbId={kbId} />
{activeMenu === 'metadata' && (
<KBForm
initKbId={kbId}
onNewKbCreated={onNewKbCreated}
onKbUpdated={onKbUpdated}
/>
)}
{activeMenu === 'documents' && hasDocumentCapability() && (
<KBDoc
kbId={kbId}
ragEngineName={kbInfo?.knowledge_engine?.name}
ragEngineCapabilities={
kbInfo?.knowledge_engine?.capabilities
}
/>
)}
{activeMenu === 'retrieve' && (
<KBRetrieveGeneric
kbId={kbId}
retrieveFunction={retrieveFunction}
/>
)}
{activeMenu === 'retrieve' &&
(kbType === 'builtin' ? (
<KBRetrieve kbId={kbId} />
) : (
<ExternalKBRetrieve kbId={kbId} />
))}
</div>
{activeMenu === 'metadata' && (
<DialogFooter className="px-6 py-4 border-t shrink-0">
@@ -254,12 +260,7 @@ export default function KBDetailDialog({
>
{t('common.delete')}
</Button>
<Button
type="submit"
form={
kbType === 'builtin' ? 'kb-form' : 'external-kb-form'
}
>
<Button type="submit" form="kb-form">
{t('common.save')}
</Button>
<Button
@@ -277,7 +278,7 @@ export default function KBDetailDialog({
</DialogContent>
</Dialog>
{/* 删除确认对话框 */}
{/* Delete confirmation dialog */}
<Dialog open={showDeleteConfirm} onOpenChange={setShowDeleteConfirm}>
<DialogContent>
<DialogHeader>

View File

@@ -1,59 +0,0 @@
import { ExternalKBCardVO } from '@/app/home/knowledge/components/external-kb-card/ExternalKBCardVO';
import { useTranslation } from 'react-i18next';
import styles from '../kb-card/KBCard.module.css';
import { httpClient } from '@/app/infra/http/HttpClient';
export default function ExternalKBCard({
kbCardVO,
}: {
kbCardVO: ExternalKBCardVO;
}) {
const { t } = useTranslation();
return (
<div className={`${styles.cardContainer}`}>
<div className={`${styles.basicInfoContainer}`}>
<div className={`${styles.iconBasicInfoContainer}`}>
{/* Emoji with plugin icon badge */}
<div className="relative">
<div className={`${styles.iconEmoji}`}>
{kbCardVO.emoji || '🔗'}
</div>
{/* Plugin icon badge at bottom right */}
<img
src={httpClient.getPluginIconURL(
kbCardVO.pluginAuthor,
kbCardVO.pluginName,
)}
alt="plugin icon"
className="absolute -bottom-1 -right-1 w-5 h-5 rounded-[20%]"
/>
</div>
<div className={`${styles.basicInfoNameContainer}`}>
<div className={`${styles.basicInfoNameText} ${styles.bigText}`}>
{kbCardVO.name}
</div>
<div className={`${styles.basicInfoDescriptionText}`}>
{kbCardVO.description}
</div>
</div>
</div>
<div className={`${styles.basicInfoLastUpdatedTimeContainer}`}>
<svg
className={`${styles.basicInfoUpdateTimeIcon}`}
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24"
fill="currentColor"
>
<path d="M12 22C6.47715 22 2 17.5228 2 12C2 6.47715 6.47715 2 12 2C17.5228 2 22 6.47715 22 12C22 17.5228 17.5228 22 12 22ZM12 20C16.4183 20 20 16.4183 20 12C20 7.58172 16.4183 4 12 4C7.58172 4 4 7.58172 4 12C4 16.4183 7.58172 20 12 20ZM13 12H17V14H11V7H13V12Z"></path>
</svg>
<div className={`${styles.basicInfoUpdateTimeText}`}>
{t('knowledge.updateTime')}
{kbCardVO.lastUpdatedTimeAgo}
</div>
</div>
</div>
</div>
);
}

View File

@@ -1,43 +0,0 @@
export class ExternalKBCardVO {
id: string;
name: string;
description: string;
emoji?: string;
retrieverName: string;
retrieverConfig: Record<string, unknown>;
lastUpdatedTimeAgo: string;
pluginAuthor: string;
pluginName: string;
constructor({
id,
name,
description,
emoji,
retrieverName,
retrieverConfig,
lastUpdatedTimeAgo,
pluginAuthor,
pluginName,
}: {
id: string;
name: string;
description: string;
emoji?: string;
retrieverName: string;
retrieverConfig: Record<string, unknown>;
lastUpdatedTimeAgo: string;
pluginAuthor: string;
pluginName: string;
}) {
this.id = id;
this.name = name;
this.description = description;
this.emoji = emoji;
this.retrieverName = retrieverName;
this.retrieverConfig = retrieverConfig;
this.lastUpdatedTimeAgo = lastUpdatedTimeAgo;
this.pluginAuthor = pluginAuthor;
this.pluginName = pluginName;
}
}

Some files were not shown because too many files have changed in this diff Show More