Compare commits

..

203 Commits

Author SHA1 Message Date
copilot-swe-agent[bot]
1e0275ee08 Improve code clarity and add documentation about in-place modification
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-01-30 15:53:40 +00:00
copilot-swe-agent[bot]
3fa7389ba9 Add defensive checks for thought_signature modification
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-01-30 15:49:25 +00:00
copilot-swe-agent[bot]
84a3b1f465 Add thought_signature to Gemini function calls for API compatibility
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2026-01-30 15:47:56 +00:00
copilot-swe-agent[bot]
377d011da2 Initial plan 2026-01-30 15:44:33 +00:00
Junyan Qin
e0d72969e3 chore(deps): update langbot-plugin version to 0.2.5 in pyproject.toml 2026-01-30 17:31:21 +08:00
Junyan Qin
a65b7ad413 chore(deps): update pyseekdb version to 1.0.0b7 in pyproject.toml 2026-01-30 13:39:36 +08:00
Junyan Qin
45df44e01b chore: update uv.lock 2026-01-30 12:42:21 +08:00
Junyan Qin
d8addb105a chore: update .gitignore and add uv.lock for dependency management 2026-01-30 12:32:39 +08:00
Junyan Qin
f17ccad665 chore: update TypeScript configuration for improved compatibility and structure 2026-01-30 12:15:19 +08:00
Junyan Qin
120ceb0b55 chore: update linting configuration to use eslint directly 2026-01-30 12:03:43 +08:00
dependabot[bot]
8a6f80a181 chore(deps): bump lodash from 4.17.21 to 4.17.23 in /web (#1944)
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.21 to 4.17.23.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.21...4.17.23)

---
updated-dependencies:
- dependency-name: lodash
  dependency-version: 4.17.23
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-30 11:25:16 +08:00
dependabot[bot]
b19e468668 chore(deps): bump next from 15.5.9 to 16.1.5 in /web (#1943)
Bumps [next](https://github.com/vercel/next.js) from 15.5.9 to 16.1.5.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.9...v16.1.5)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 16.1.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-30 11:20:08 +08:00
Junyan Qin
aeac79e1b3 feat: add tag filtering functionality to Plugin Market
- Introduced TagsFilter component for selecting and filtering plugins by tags.
- Updated PluginMarketComponent to handle tag selection and display.
- Enhanced PluginMarketCardComponent to show selected tags.
- Modified CloudServiceClient to fetch available tags from the API.
- Updated localization files to support new tag-related strings.
2026-01-29 16:08:05 +08:00
Junyan Qin
b89a240250 feat: implement LoadingSpinner component and replace existing loaders across the application 2026-01-29 15:24:23 +08:00
Junyan Qin
13f42857f5 perf: detailed control of models service displaying 2026-01-27 22:44:58 +08:00
Junyan Qin
61f3f31edc chore: bump version to 4.8.1 2026-01-27 20:33:55 +08:00
Junyan Qin
3663d9dc10 style: adjust margin in PipelineDetailDialog for improved button alignment 2026-01-27 20:33:17 +08:00
Guanchao Wang
89ec86c530 fix: issue 1936 (#1937) 2026-01-27 20:28:19 +08:00
Junyan Qin
d9ba2a17ff chore: bump version to 4.8.0 2026-01-26 21:12:56 +08:00
Junyan Qin
c4ea6188f9 chore: update layout description to reflect production-grade capabilities for IM bot integration 2026-01-26 21:09:59 +08:00
Guanchao Wang
5d9f6ec763 Feat/monitor (#1928)
* feat: add monitor

* feat: fix tab

* feat: work

* feat: not reliable monitor

* feat: enhance monitoring page layout with integrated filters and refresh button

* feat: add support for runner recording

* feat: add jump button & alignment

* feat: new

* fix: not show query variables in local agent

* fix: pnpm lint and python ruff check

* fix: ruff fromat

* chore: remove unnecessary migration

* style: optimize monitoring page layout and fix sticky filter issues

- Enhanced metric cards with gradient backgrounds and hover effects
- Increased traffic chart height from 200px to 300px
- Adjusted grid layout and spacing for better visual appeal
- Fixed sticky filter area to properly cover parent padding without transparent gaps
- Used negative margins and positioning to eliminate scrolling artifacts
- Matched padding/margins with other pages (pipelines, bots) for consistency
- Removed duplicate title/subtitle from page content
- Added cursor-pointer styling to tab triggers
- Removed border between tab list and tab content

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* fix: apply prettier formatting to monitoring components

- Fixed indentation and spacing in MetricCard.tsx
- Fixed formatting in TrafficChart.tsx
- Applied prettier formatting to page.tsx

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

* feat: update HomeSidebar to trigger action on child selection and localize monitoring titles

* refactor: streamline LLM and embedding invocation methods

* feat: add embedding model monitor

* fix: database version

* chore: simplify pnpm-lock.yaml formatting

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-26 21:08:23 +08:00
Junyan Qin (Chin)
b73847f1a6 feat: add emoji support to knowledge bases and pipelines (#1935)
* feat: add emoji support to knowledge bases and pipelines

* feat: add optional emoji property to ExternalKBCardVO for enhanced knowledge base representation
2026-01-26 17:37:35 +08:00
Typer_Body
d6e1e79f07 fix: potential copy action bug on windows (#1931)
* fix a bag updata

* Update page.tsx

* Update page.tsx

* Append text area to body for selection

* Update page.tsx

* Update mcp.py
2026-01-25 15:40:11 +08:00
Junyan Qin
525008b8b2 docs: update feature descriptions in multiple language READMEs to include Langflow integration and enhance clarity on production-grade features 2026-01-25 15:28:15 +08:00
Junyan Qin (Chin)
bbf77bac4c feat(user): update Space model provider API keys in UserService (#1932) 2026-01-25 14:15:25 +08:00
Junyan Qin (Chin)
fc6e414be4 feat: add GitHub Actions workflow for linting with Ruff (#1929)
* feat: add GitHub Actions workflow for linting with Ruff

* refactor: rename lint job and add formatting step to Ruff workflow

* chore: run ruff format

* chore: rename Ruff lint job to 'Lint' and add frontend linting workflow
2026-01-23 13:43:12 +08:00
Junyan Qin
e60cb6ad0e fix: ruff check errors 2026-01-23 13:30:44 +08:00
Junyan Qin
c90f2d6a12 chore: update mcp dependency version to 1.25.0 2026-01-20 01:59:19 +08:00
Junyan Qin
fe8a738cd7 fix(i18n): update apiKeyCreatedMessage for clarity across multiple languages 2026-01-20 01:53:49 +08:00
Tiankai Ma
604cc53973 fix(localagent): allow empty func arg (#1921) 2026-01-19 23:42:47 +08:00
Tiankai Ma
195b694ecc feat(telegram): threaded mode support (#1920)
* feat(telegram): reply in threaded mode

* feat(telegram): thread-level isolation
2026-01-19 23:42:17 +08:00
Tiankai Ma
d21f23beee fix(telegram): set reply_to_message_id correctly (#1918) 2026-01-15 18:09:57 +08:00
Junyan Qin
558587883b chore: update project version to 4.7.2 2026-01-13 14:02:00 +08:00
Junyan Qin
2e6a1daf4f feat(mcp): extend mode options in MCPCardVO to include 'http' 2026-01-13 13:59:59 +08:00
Tiankai Ma
1fc5e75f93 feat(mcp): add streamable HTTP and stdio (#1911)
* feat(mcp): add streamable HTTP

alongside with frontend UI change, w/ support for stdio

* fix(mcp): address copilot reviews

* Update src/langbot/pkg/provider/tools/loaders/mcp.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: resolve copilot reviews

* fix: Message -> MessageChunk

* feat: upgrade mcp module

* feat: add i18n

* feat(mcp): enhance MCPCardComponent with mode badge and reorder select items in MCPFormDialog

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: WangCham <651122857@qq.com>
Co-authored-by: Junyan Qin (Chin) <rockchinq@gmail.com>
2026-01-13 13:50:06 +08:00
fdc310
a332206ba3 fix: When the deletion of the thinking chain is activated, since the "continue" is triggered as soon as the thinking begins, it causes a bug in the subsequent judgment that breaks out of the loop impression. (#1913) 2026-01-12 00:14:39 +08:00
Junyan Qin
8e620dc635 fix: remove unreachable assertion in ChatMessageHandler to improve error handling 2026-01-09 23:46:43 +08:00
Junyan Qin
c9a21ebace fix: improve error handling in ChatMessageHandler 2026-01-09 23:23:53 +08:00
Junyan Qin
a05cdcac50 chore: update project version to 4.7.1 2026-01-09 21:52:08 +08:00
Junyan Qin
ecfb2bfb34 chore: add type hints for ap in telemetry.py 2026-01-09 21:50:43 +08:00
Guanchao Wang
e17dba0a98 fix: testing mcp server (#1912) 2026-01-09 18:39:40 +08:00
Hadong
6b138943ce feat(milvus): milvus related updates (#1908)
- Add Milvus db_name configuration and client parameter support.
- change kb_data uuid for Milvus. 3. add MAX_BATCH_SIZE for openai.
- support more vector_size.
2026-01-09 16:03:43 +08:00
fdc310
eb0e6aff68 feat: add telemetry support for query execution tracking and configur… (#1900)
* feat: add telemetry support for query execution tracking and configuration

* feat: integrate telemetry manager and enable telemetry data sending

* feat: integrate telemetry manager and enhance error handling for telemetry sending

* feat: update telemetry configuration to use 'space' instead of 'telemetry' and adjust related parameters

* feat: integrate telemetry manager and enable telemetry data sending

* feat: integrate telemetry manager and enhance error handling for telemetry sending

* feat: add instance id

* feat: enhance telemetry management with asynchronous task handling and improve model retrieval caching

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-01-09 15:50:44 +08:00
Junyan Qin
4d0095626a fix: update docker-compose command to include --no-sync option for improved runtime behavior 2026-01-08 11:30:25 +08:00
Junyan Qin
aa0a501ade fix: bug in bind space account in models dialog 2026-01-05 20:53:35 +08:00
Junyan Qin
68ef7bd2c4 chore: update project version to 4.7.0 and revise description for clarity 2026-01-05 20:06:01 +08:00
Junyan Qin
61dc5de085 fix: update help links in sidebar configuration to reflect new usage paths and add Japanese translations 2026-01-05 18:45:35 +08:00
Junyan Qin
63bdd71e22 fix: update models_gateway_api_url to include version in cloud service configuration 2026-01-05 17:58:50 +08:00
Junyan Qin
9ea5b50802 refactor: enhance layout and styling of ModelsDialog component for improved usability 2026-01-05 17:58:01 +08:00
Jinzhe Zeng
1cd586634d fix: split Wecom messages exceeding 2048-byte limit (#1901)
Co-authored-by: Oracle Public Cloud User <opc@arm1.subnet.vcn.oraclevcn.com>
2026-01-05 15:04:46 +08:00
Junyan Qin
45bedbe70e fix: update QQ Group link in README to the new group ID 2026-01-05 10:20:42 +08:00
Junyan Qin (Chin)
f7f1dde7b5 Merge pull request #1894 from langbot-app/feat/maas-support
refactor: model config dialog and introduce LangBot Models service integration
2026-01-03 15:47:23 +08:00
Junyan Qin
ba06555078 refactor: remove SQLite compatibility check for column cleanup in DB migration script 2026-01-03 15:43:40 +08:00
Junyan Qin
840fa39979 feat: add informational popover to registration page with tips on using Space for account authentication 2026-01-03 15:26:24 +08:00
Junyan Qin
b295416e6c fix: adjust ModelsDialog component to set a maximum width for better layout consistency 2026-01-03 01:06:17 +08:00
Junyan Qin
914f77ff37 refactor: standardize error handling across components by utilizing CustomApiError for improved error messaging 2026-01-03 00:56:25 +08:00
Junyan Qin
b0b7b914d8 feat: update README files to include new links for API integration, plugin market, and roadmap across multiple languages 2026-01-01 22:11:43 +08:00
Junyan Qin
12713aad45 feat: migrate cloud service URL configuration and update database version to 17 2026-01-01 21:40:55 +08:00
Junyan Qin
02e12cc1e4 feat: implement account email mismatch error handling and improve user feedback in authentication flows 2026-01-01 17:01:32 +08:00
Junyan Qin
61f08f3218 feat: add disable_models_service configuration to manage model service availability and update related components 2026-01-01 15:40:39 +08:00
Junyan Qin
75c2a063cc refactor: remove providerUuid prop from model components and enhance provider deletion confirmation UI 2026-01-01 15:07:37 +08:00
Junyan Qin
b4773c4e48 refactor: update model management components and enhance provider functionality 2026-01-01 14:58:06 +08:00
Junyan Qin (Chin)
fb73da8735 Merge branch 'master' into feat/maas-support 2026-01-01 13:07:45 +08:00
Junyan Qin
679e549b1d feat: implement loading states in SpaceOAuthCallback and HomeSidebar components using Suspense 2026-01-01 13:06:04 +08:00
Junyan Qin
898144e9f4 fix: remove unused HoverCard imports from DynamicFormItemComponent and clean up ModelsDialog constants 2026-01-01 12:53:39 +08:00
Junyan Qin
b99c5561fc fix: update cloud service URL retrieval and enhance model synchronization error handling 2026-01-01 12:50:26 +08:00
Copilot
b2f4b91979 perf: replace copy button toast notifications with checkmark feedback (#1898)
* Initial plan

* Replace copy button toast notifications with checkmark visual feedback

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Complete copy button checkmark feedback implementation

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* revert pnpm-lock.yaml

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2026-01-01 11:53:13 +08:00
Junyan Qin
4528000fc4 refactor: model management 2026-01-01 02:00:24 +08:00
Junyan Qin
96e40eaf25 feat: enhance model creation with UUID preservation option and implement Space model synchronization in ModelManager 2025-12-31 22:25:07 +08:00
Junyan Qin
197258ae91 feat: add LangBot Space ChatCompletions requester and integrate with ModelsDialog and EmbeddingForm components 2025-12-30 21:52:52 +08:00
Junyan Qin
19f417174c feat: implement SpaceService for OAuth handling and user management, refactor UserService to utilize new service methods 2025-12-29 22:43:19 +08:00
Junyan Qin
9c82eeddeb feat: add endpoint for retrieving user space credits and implement caching mechanism in UserService 2025-12-29 22:23:11 +08:00
Junyan Qin
f11e01b549 refactor: rename 'allow_change_password' to 'allow_modify_login_info' and update related logic across the application 2025-12-29 21:14:05 +08:00
Junyan Qin
863b26c3fa refactor: update column drop logic in DBMigrateModelProviderRefactor for PostgreSQL compatibility 2025-12-29 20:42:06 +08:00
Junyan Qin
b788858f9e fix: handle case of empty token list in TokenManager to prevent errors 2025-12-29 12:18:45 +08:00
Junyan Qin
de8a7df6c2 feat: implement instance ID management and integrate with OAuth token exchange 2025-12-29 00:35:31 +08:00
Junyan Qin
ba5b481617 refactor: simplify theme toggle implementation in HomeSidebar and ThemeToggle components 2025-12-28 22:43:05 +08:00
Junyan Qin
07ad846e96 feat: update dependencies and enhance account settings dialog with password management and improved UI elements 2025-12-28 22:38:11 +08:00
Copilot
30945aafdd feat: support configurable WeCom API base URL for reverse proxy deployment (#1890)
* Initial plan

* Add api_base_url support to WeCom API libraries and adapters

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Add api_base_url parameter to OAClient and adapters for Official Account and WeCom APIs

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-28 21:04:55 +08:00
Junyan Qin
24c15b4479 feat: implement account settings dialog for managing user passwords and binding Space accounts 2025-12-26 23:20:51 +08:00
Junyan Qin
1d4c5bbdf1 feat: enhance model abilities display in DynamicFormItem and ModelsDialog components with icons for vision and function call 2025-12-26 20:57:12 +08:00
Junyan Qin
57fcec011d feat: refactor model management to introduce provider structure, enhancing model organization and retrieval 2025-12-26 20:27:33 +08:00
Junyan Qin
455e3db28d feat: add Radix UI collapsible component for enhanced UI interactions 2025-12-26 00:49:35 +08:00
Junyan Qin
8caab43b00 feat: add Space integration for user authentication and model management with OAuth support 2025-12-26 00:35:47 +08:00
Junyan Qin
7479545339 feat: implement models dialog for managing LLM and embedding models with dynamic URL handling 2025-12-25 20:54:00 +08:00
Junyan Qin
10ee30695a feat: add error handling and alert display for model testing in EmbeddingForm and LLMForm 2025-12-24 16:12:41 +08:00
Junyan Qin
a9a262eaae feat: add new version notification dialog and version comparison logic 2025-12-24 12:43:52 +08:00
Junyan Qin
a8594b76cd fix: enable extra_args in LLMModelsService for model testing 2025-12-23 21:03:45 +08:00
Junyan Qin
11ee0fef5d chore: update Python versions in CI workflow 2025-12-23 14:27:09 +08:00
Junyan Qin
9a9ba34717 chore: bump version v4.6.5 2025-12-23 14:26:52 +08:00
Junyan Qin
312e47bf46 chore: bump langbot-plugin to 0.2.4 2025-12-23 14:22:13 +08:00
Junyan Qin
628865fd06 fix: add timeout to image fetching in get_qq_image_bytes function (#1859) 2025-12-23 14:17:16 +08:00
Junyan Qin
806a03cd53 fix: dingtalk adapter lifecycle mgm issues (#1844, #1853) 2025-12-23 14:00:41 +08:00
Junyan Qin
24bd90fcf6 fix: alter_user_message typing issues 2025-12-23 13:24:52 +08:00
Junyan Qin
d2765577c8 chore: provide '--no-sync' arg in dockerfile 2025-12-23 12:39:42 +08:00
fdc310
60ca688bcb Fix/Incomplete JSON data returned by N8N streaming data causes the loss of chunks. (#1880)
* fix: Incomplete JSON data returned by N8N streaming data causes the loss of chunks.
2025-12-23 09:42:26 +08:00
ICE
76d8eea41d fix: group bot at rule (#1882) 2025-12-22 20:20:41 +08:00
Junyan Qin
635c3a04d8 perf: ja-JP translation for New 2025-12-22 18:46:15 +08:00
Junyan Qin
dde97abe38 feat: enhance HomeSidebar with new integration options and updated translations 2025-12-22 18:43:19 +08:00
Copilot
90a22d894d fix: prevent memory overflow from excessive logging in streaming and query processing (#1879)
* Initial plan

* fix: reduce excessive logging to prevent memory overflow

- Add log file rotation (10MB max per file, 5 backups)
- Reduce streaming response logging (every 10th chunk instead of every chunk)
- Remove debug logging from controller tight loop
- Add summary logging after streaming completes

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* refactor: address code review feedback

- Extract log rotation config to module-level constants
- Keep first streaming chunk at INFO level for connection debugging
- Use DEBUG level for subsequent chunks

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* style: fix code formatting whitespace

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-12-22 18:25:24 +08:00
Junyan Qin
88ef9cd6ae chore: remove platform field from docker-compose.yaml 2025-12-21 20:31:09 +08:00
fdc310
e3595b5c57 Feat/lark file and audio (#1874)
* fix: n8n streaming no sequence bug

* feat:add lark file and audio
fix: webhook

* feat:add lark file and audio
fix: webhook

* 更新 n8nsvapi.py

* del : print and log
2025-12-21 01:30:05 +08:00
Junyan Qin (Chin)
ce82f87e43 feat: add SeekDB vector database support for knowledge bases (#1814)
* feat: add SeekDB vector database support for knowledge bases

This commit adds complete integration of OceanBase's SeekDB as a vector
database option for LangBot's knowledge base feature.

## Changes

### Core Implementation
- Add SeekDB adapter implementing VectorDatabase interface
  - Support both embedded and server deployment modes
  - HNSW indexing with cosine similarity
  - Async operations with error handling
  - Comprehensive logging

### System Integration
- Register SeekDB in VectorDBManager
- Add pyseekdb>=0.1.0 dependency
- Add SeekDB configuration template
- Update README with vector database section

### Documentation
- Complete integration guide with platform compatibility warnings
- Configuration examples for all deployment modes
- Troubleshooting guide for common issues
- Code examples demonstrating usage patterns
- Comprehensive test reports and status documentation

## Testing

Architecture validated end-to-end using ChromaDB:
- File upload → parsing → chunking → embedding → storage
- 828 bytes → 3 chunks → 3 vectors stored successfully
- BGE-M3 model (384 dimensions)
- Status: Completed 

## Platform Compatibility

### Embedded Mode
-  Linux: Fully supported
-  macOS: Not supported (pylibseekdb is Linux-only)
-  Windows: Not supported (pylibseekdb is Linux-only)

### Server Mode
-  Linux: Fully supported
- ⚠️ macOS: Known issue (oceanbase/seekdb#36)
- ⚠️ Windows: Untested

### Remote Connection
-  All platforms supported

## Known Issues

macOS Docker server mode affected by upstream bug:
https://github.com/oceanbase/seekdb/issues/36

Workaround: Use ChromaDB/Qdrant or connect to remote SeekDB server.

## Files Added
- src/langbot/pkg/vector/vdbs/seekdb.py
- docs/SEEKDB_INTEGRATION.md
- examples/seekdb_example.py
- SEEKDB_INTEGRATION_SUMMARY.md
- SEEKDB_INTEGRATION_COMPLETE.md
- SEEKDB_TEST_STATUS.md
- SEEKDB_FINAL_SUMMARY.md
- SEEKDB_INTEGRATION_DONE.md
- GITHUB_ISSUE_36_COMMENT.md

## Files Modified
- src/langbot/pkg/vector/mgr.py
- src/langbot/pkg/vector/vdbs/__init__.py
- pyproject.toml
- src/langbot/templates/config.yaml
- README.md
- README_EN.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* chore: remove unused docs

* feature: minimal seekdb change (#1866)

* feat: add SeekDB embedding requester and configuration

This commit introduces a new SeekDB embedding requester, which utilizes the local embedding function from pyseekdb. It includes the necessary Python implementation and a corresponding YAML configuration file for integration. Additionally, a new SVG icon for SeekDB is added to enhance the visual representation in the UI.

* fix: update EmbeddingForm to conditionally render URL field based on model provider

This commit modifies the EmbeddingForm component to conditionally display the URL input field only when the current model provider is not 'seekdb-embedding'. Additionally, it updates the condition for rendering the API key field to exclude both 'ollama-chat' and 'seekdb-embedding' providers.

* chore: update Python version requirement in pyproject.toml to support Python 3.11

* fix: add config default value, when it makes fronted not show spec

* fix: seekdb.py clean metadata. change api

* fix: enhance error handling in SeekDB embedding initialization

This commit adds improved error handling to the SeekDB embedding function. It ensures that a RuntimeError is raised if the embedding function fails to initialize, and wraps the embedding call in a try-except block to catch and raise a RequesterError with a descriptive message in case of failure.

* refactor: update SeekDB database management to use AdminClient

This commit refactors the SeekDB database management logic to utilize the AdminClient for database operations. It replaces the previous temp_client with admin_client for listing and creating databases, ensuring a more robust interaction with the SeekDB API.

* refactor: update SeekDB embedding model initialization to use task manager

This commit refactors the SeekDB embedding model initialization by replacing the direct asyncio task creation with the task manager's create_task method. This change enhances task management and provides a clearer naming convention for the embedding model initialization task.

* perf: integration

* chore: remove unnecessary files

* fix: linter errors

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
Co-authored-by: 名为a的全局变量 <1051233107@qq.com>
2025-12-20 23:40:30 +08:00
fdc310
854b291c5a fix: n8n streaming no sequence bug (#1873) 2025-12-20 00:03:05 +08:00
Junyan Qin
9780fd059c chore: add back arm64 docker image (#1871) 2025-12-19 23:44:28 +08:00
Junyan Qin
adc65f66eb fix: pipeline duplication bug 2025-12-19 23:27:18 +08:00
Copilot
ae772074a1 feat: Add configurable password change toggle via system.allow_change_password (#1869)
* Initial plan

* Add password change toggle feature with config flag

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Feature implementation complete and validated

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* chore: remove lock

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-18 15:14:03 +08:00
dependabot[bot]
16c1e9edd1 chore(deps): bump next from 15.5.7 to 15.5.9 in /web (#1868)
Bumps [next](https://github.com/vercel/next.js) from 15.5.7 to 15.5.9.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.7...v15.5.9)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.5.9
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-18 12:21:02 +08:00
sheetung
3ab9ffb7b7 feat(plugins): add plugin new version detection (#1865)
* feat(plugins): 添加插件更新检测功能

* perf: card style

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-18 12:17:25 +08:00
Copilot
82e2123fe7 Fix Dify v1.11.0 conversation_id UUID validation error (#1860)
* Initial plan

* Fix Dify v1.11.0 conversation_id UUID validation error

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-12-12 18:35:47 +08:00
Junyan Qin
7a65f3d2f4 chore: update AGENTS.md 2025-12-12 17:35:02 +08:00
Junyan Qin
b5b5d499e5 feat: add back streaming switch for web chat 2025-12-11 18:54:16 +08:00
Hadong
173f9e9c30 feat(lark): 支持商店应用机器人 (#1855)
* feat(lark): 支持商店应用机器人

* feat(lark): app_type改成select模式,修复select配置无效,按照copilot建议隐藏log敏感信息

* fix: KeyError for backward compatibility

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-11 16:54:28 +08:00
Junyan Qin
a610c72067 chore: bump version 4.6.4 2025-12-10 14:22:57 +08:00
Junyan Qin
d210a49fae fix: react cve 2025-12-10 14:21:41 +08:00
Junyan Qin
b015c248ea chore: bump langbot-plugin to 0.2.3 2025-12-10 14:02:23 +08:00
Hadong
4a559ea770 feat: 飞书适配器加入“机器人进群欢迎语”配置 (#1852)
* feat(lark): 支持机器人进群发送欢迎消息

* perf: existence check and indent

---------

Co-authored-by: donghao <donghao@patsnap.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-09 16:37:03 +08:00
fdc310
e306751863 feat:add lark ubified_webhook and The configuration for the front-end regarding whether to enable webhooks for Lark is displayed. (#1850) 2025-12-09 13:30:45 +08:00
Junyan Qin
2f51f5f33e docs: apply README changes to all languages 2025-12-06 22:34:48 +08:00
Junyan Qin (Chin)
74a2a61fc1 Update README with new features and headings
Added a new heading and additional features to the README.
2025-12-06 22:21:49 +08:00
Junyan Qin
b6c0345b3e chore: bump version 4.6.3 2025-12-06 21:29:28 +08:00
Junyan Qin (Chin)
6421a6f5cb Feat/complete adapter features (#1849)
* feat: add voice and file supports for wecom

* feat: add   and  in query variables

* feat: supports for lark recv file message

* feat: kook recv voice msg

* feat: supports for Voice and File in discord

* chore: remove debug msg

* perf: remove unnecessary bot logs

* feat: implement bot log filtering and per label color (#1839)

* feat: add sender_name and group_name in query variables
2025-12-06 21:11:01 +08:00
Junyan Qin
daf56e5dc2 fix: test failed 2025-12-05 22:54:13 +08:00
Yaguang.Wang
cb7c9af25c feat: Expanded WeCom message parsing to capture msgtype, inline voice/video… (#1843)
* Expanded WeCom message parsing to capture msgtype, inline voice/video/file/link data, bounded base64 downloads, and richer mixed-message attachments (src/langbot/libs/wecom_ai_bot_api/api.py); added event accessors for new fields (src/langbot/libs/wecom_ai_bot_api/wecombotevent.py).
Converter now maps richer WeCom payloads (text, images, files, voice, video, links) into platform message chain with fallbacks when nothing parsable is present (src/langbot/pkg/platform/sources/wecombot.py).
Preprocessor now turns voice inputs into file URLs for downstream runners (src/langbot/pkg/pipeline/preproc/preproc.py).
Dify runner uploads all incoming files (images/audio/video/docs) after downloading or decoding data URLs, infers MIME types, and passes typed file descriptors into chat/workflow calls (src/langbot/pkg/provider/runners/difysvapi.py).

* Update src/langbot/pkg/platform/sources/wecombot.py

Fixed the issue of duplicate text in the comments.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/langbot/libs/wecom_ai_bot_api/api.py

Modify the way you approach challenges.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/langbot/pkg/platform/sources/wecombot.py

Changing the variable names makes more sense.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: use from_base64 for the voice file converting

---------

Co-authored-by: tabriswang <tabriswang@finecomn.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-05 22:33:15 +08:00
Junyan Qin
45e61befac fix: test failed 2025-12-05 22:30:44 +08:00
Junyan Qin
ea50ba10e6 perf: add en name in the wecom manifest 2025-12-05 21:28:56 +08:00
Junyan Qin
5c4a727e74 feat: make all db migrations SQL-only 2025-12-05 21:00:04 +08:00
Junyan Qin
867f05c4ad perf: make the timeout of emit_event 180s 2025-12-05 20:59:37 +08:00
Junyan Qin
b06b32306f feat: remove all unnecessary fields in GroupMember and implement MessageEvent field for pipeline events 2025-12-05 17:24:58 +08:00
Junyan Qin
dbfcb70f8d fix: sender_id not presented to Session 2025-12-05 17:13:30 +08:00
Junyan Qin
e64d56c4ac fix: bad protocol of default plugin debug url 2025-12-05 16:06:56 +08:00
Bruce
8f0da7943c Remove plugins volume from docker-compose (#1842) 2025-12-05 11:28:04 +08:00
Junyan Qin
e62ff7e520 fix: deps issues 2025-12-04 23:07:55 +08:00
Junyan Qin (Chin)
86e951916e feat: add milvus and pgvector as vector db (#1840)
* feat: add milvus and pgvector as vector db

* chore: update config.yaml template delete comments
2025-12-04 22:34:49 +08:00
Junyan Qin
6bf08466de chore: bump version 4.6.2 2025-12-04 20:30:02 +08:00
Junyan Qin
5e36dd480d docs: add KOOK in README 2025-12-04 13:56:56 +08:00
Junyan Qin (Chin)
0e2cd8c018 Feat/kook (#1834)
* feat: add adapter file

* fix: style for bot log

* fix: kook bugs
2025-12-04 13:40:38 +08:00
Junyan Qin (Chin)
b4f92eba38 feat(platform): add skip_pipeline parameter for webhook responses (#1837)
* feat(platform): add skip_pipeline parameter for webhook responses

Add support for skip_pipeline parameter in webhook responses, allowing
webhook targets to instruct LangBot to skip pipeline processing for
specific messages. When a webhook responds with skip_pipeline=true,
the message is treated as a notification only and bypasses the query pool.

Changes:
- webhook_pusher.py: Parse JSON responses and return skip_pipeline flag
- botmgr.py: Check skip_pipeline before adding messages to query pool
- docker-compose.yaml: Add DNS configuration to fix container networking

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: webhook crud bug

* chore: revert docker-compose.yaml

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-04 13:40:26 +08:00
dependabot[bot]
905e48c8ed chore(deps): bump next from 15.4.7 to 15.4.8 in /web (#1836)
Bumps [next](https://github.com/vercel/next.js) from 15.4.7 to 15.4.8.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.4.7...v15.4.8)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.4.8
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-04 11:33:55 +08:00
Junyan Qin
10ec79312e chore: bump version 4.6.1 2025-12-02 17:43:38 +08:00
Junyan Qin
24f779ff95 fix: websocket connect failed in prod env 2025-12-02 17:41:31 +08:00
Junyan Qin
08c0677de9 chore: bump version 4.6.0 2025-12-02 13:58:08 +08:00
Junyan Qin
cc5d32cf8a chore: bump langbot-plugin to 0.2.0 2025-12-01 22:15:38 +08:00
Junyan Qin
01a5133396 chore: update docker-compose.yaml 2025-12-01 22:14:38 +08:00
Guanchao Wang
0aa5188b29 Feat/unified webhook (#1793)
* fix: wecombot id

* feat: add unified webhook for wecom

* feat: add support for wecombot,wxoa,slack and qqo

* fix: slack adapter

* feat: qqo

* fix: errors when npm lint

* fix: qqo webhook

* feat: add wecomcs

* fix: modify wecomcs

* fix: import errors

* feat: add configurable webhook display prefix (#1797)

* Initial plan

* Add webhook_display_prefix configuration option

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* perf: change config field name

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>

* feat: finish the fxxking line adapter

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-12-01 22:09:20 +08:00
Junyan Qin (Chin)
e49a161d0a feat: displaying plugin debug info (#1828) 2025-12-01 17:59:49 +08:00
Junyan Qin
0ddc3d60e7 fix: incorrect update date in kb card 2025-12-01 14:35:41 +08:00
Junyan Qin
51794176af perf: add comment for installing KB retriever plugins 2025-12-01 14:04:32 +08:00
Copilot
b634aa48dc feat(web): Add markdown rendering support to pipeline chat messages with toggle (#1826)
* Initial plan

* Add markdown rendering support to pipeline debug dialog messages with toggle button

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Fix code review feedback: remove conflicting styles and imports

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* perf: styles

* fix: websocket message broadcasting cross-contamination between person and group channels

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-12-01 13:44:01 +08:00
Junyan Qin
16ae8ac546 chore: remove NotFound component from the application 2025-11-30 21:58:28 +08:00
Copilot
1ecb0735cb perf: Filter plugins by component types in pipeline extensions (#1821)
* Initial plan

* Add component-kind filtering to list_plugins and filter pipeline extensions to only show plugins with Command, EventListener, or Tool components

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* fix: testing path

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-30 20:55:48 +08:00
Junyan Qin
c368d828c9 fix: linter error 2025-11-30 14:27:53 +08:00
Junyan Qin
019ae9c216 refactor: remove debug plugin success message from PluginForm and localization files 2025-11-30 14:20:40 +08:00
Copilot
580d9441a4 fix: increase execute_command timeout from 60s to 180s (#1813)
* Initial plan

* fix: increase execute_command timeout from 60s to 180s

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>
2025-11-28 23:07:55 +08:00
Junyan Qin (Chin)
b5d192425e perf: advanced web chat (#1811)
* perf: supports for quoting message

* feat: add supports for Voice and File

* perf: reply button
2025-11-28 22:25:06 +08:00
Copilot
58312deb8c fix: command return value image_url handling for DingTalk, Slack, LINE, and Lark adapters (#1810)
* Initial plan

* Fix command return value image_url handling for DingTalk, Slack, and LINE adapters

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

* Refactor DingTalk image handling into helper method and add clarifying comment

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

* Fix Lark adapter to not append empty paragraph before images

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

* Improve Lark adapter image handling with better error logging

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

* Fix Lark adapter to send images as separate image messages instead of embedded in post

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

* Parse Markdown image syntax in Lark adapter and render as separate image messages

Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: the-lazy-me <52873503+the-lazy-me@users.noreply.github.com>
2025-11-28 22:24:42 +08:00
Junyan Qin
cf646752c5 feat: add more service api supports 2025-11-28 20:13:58 +08:00
Junyan Qin
b53750fde4 feat: add KnowledgeRetriever supports in plugin market 2025-11-28 16:47:55 +08:00
Junyan Qin
52e6135ae8 chore: i18n for knowledge retriever component name 2025-11-28 15:48:27 +08:00
Junyan Qin
f4eb59e2ad fix: deleted external kb not destoryed 2025-11-28 15:37:55 +08:00
Junyan Qin
34d84590e2 chore: tidy files 2025-11-28 15:01:54 +08:00
Junyan Qin (Chin)
d09b823c49 refactor: switch webchat from sse to websocket (#1808)
* refactor: switch webchat from sse to websocket

* perf: image preview dialog

* chore: remove console.log
2025-11-28 14:54:01 +08:00
Junyan Qin
348620ac0a chore: remove unused code 2025-11-27 23:39:01 +08:00
Copilot
a8481e43f0 feat: external knowledge bases (#1783)
* Initial plan

* Add backend support for external knowledge bases

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Add frontend support for external knowledge bases with tabs UI

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Add i18n translations for all languages (Traditional Chinese and Japanese)

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Update knowledge base tab list styling to match plugins page

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* perf: margin-top for kb page

* refactor: switch RetrievalResultEntry to langbot_plugin pkg ones

* feat: knowledge retriever listing and creating

* stash

* refactor: unify sync mechanism for polymorphic components

* feat: use unified retireval result struct in retrieval test page

* chore: remove unused methods

* feat: retriever icon displaying

* feat: localagent retrieval with external kbs

* chore: bump version of langbot-plugin to 0.2.0b1

* fix: i18n

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-27 23:19:43 +08:00
Copilot
3c04eeaff9 perf: API integration dialog height and enable table scrolling (#1805)
* Initial plan

* Fix API integration dialog height and make tables scrollable

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Update API integration dialog height for improved layout

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-27 11:59:46 +08:00
Junyan Qin (Chin)
87131cf03b Feat/pipeline enable all extensions (#1807)
* feat: 添加流水线扩展集成"启用所有"选项

为流水线的扩展集成配置添加独立的"启用所有插件"和"启用所有MCP服务器"选项。

主要变更:
- 数据模型:在 extensions_preferences 中添加 enable_all_plugins 和 enable_all_mcp_servers 字段
- 后端逻辑:修改 RuntimePipeline 以支持独立的启用所有选项,当启用时设置为 None 表示使用所有可用资源
- API 接口:更新 GET/PUT /api/v1/pipelines/{uuid}/extensions 以支持新字段
- 前端 UI:为插件和 MCP 服务器分别添加独立的开关控件
- 国际化:添加对应的中文翻译文本
- 默认行为:新创建的流水线默认启用所有插件和 MCP 服务器

🤖 Generated with [Claude Code](https://claude.com/claude-code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* fix(i18n): add missing translations for pipeline extensions

Added translations for enable all plugins/MCP servers feature:
- en-US: English translations
- ja-JP: Japanese translations
- zh-Hant: Traditional Chinese translations

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

* chore: add migration for enable all extensions config

* fix: bad renaming

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Happy <yesreply@happy.engineering>
2025-11-27 11:52:15 +08:00
Junyan Qin
7d51293594 chore: adjust star component style 2025-11-25 10:08:11 +08:00
Junyan Qin
b78b0e50bb perf: plugin list padding bottom 2025-11-25 09:50:51 +08:00
Copilot
6b4c1a7dee fix: plugin card source badge blocked by hover overlay (#1802)
* Initial plan

* Add View Source menu item and remove clickable source badges

- Add "viewSource" translation key to all language files
- Add View Source menu item to plugin card dropdown (only for GitHub/marketplace plugins)
- Remove onClick handlers and ExternalLink icons from source badges
- Keep the badges themselves for visual indication of plugin source

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Fix prettier formatting issue in PluginCardComponent

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-25 09:46:37 +08:00
Xiaoyu Su
2e1f16d7b4 feat: improvements for installed plugin card
* feat:Add README display to installed plugins

* chore: Increase the timeout of call_tool

* perf: smaller animation

* fix: add endpiont for readme

* feat: supports for multilingual READMEs

* feat: supports for getting readme img

* chore: bump langbot-plugin to 0.1.13b1

* perf: plugin card layout

* fix: import useTranslation linter error

---------

Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-25 00:12:03 +08:00
Tigrex Dai
50c33c5213 Fix typo for variable and comment 'Quote' (#1800) 2025-11-24 23:09:31 +08:00
Copilot
ace6d62d76 perf: Sort installed plugins: debug plugins first, then by installation time (#1798)
* Initial plan

* Implement plugin list sorting: debug plugins first, then by installation time

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Apply ruff formatting

* Add unit tests for plugin list sorting functionality

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Optimize database query to avoid N+1 problem and update tests

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Remove redundant assertion in test

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* perf: plugin list sorting

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-23 13:46:45 +08:00
Junyan Qin
b7c4c21796 feat: add message_chain field to *NormalMessageReceived events 2025-11-22 14:59:12 +08:00
Copilot
66602da9cb feat: add model_config parameter support for Dify assistant type apps (#1796)
* Initial plan

* feat: add model_config parameter support for Dify assistant type

- Add model_config parameter to AsyncDifyServiceClient.chat_messages method
- Add _get_model_config helper method to DifyServiceAPIRunner
- Pass model_config from pipeline configuration to all chat_messages calls
- Add model-config configuration field to dify-service-api schema in ai.yaml
- Support optional model configuration for assistant type apps in open-source Dify

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* refactor: improve model_config implementation based on code review

- Simplify _get_model_config method logic
- Add more descriptive comment about model_config usage
- Clarify when model_config is used (assistant type apps)

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* feat: only modify client.py

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-22 14:38:26 +08:00
wrj97
31b483509c fix: fix n8n streaming support issue (#1787)
* fix: fix n8n streaming support issue

Add streaming support detection and proper message type handling for
n8n service API runner. Previously, when streaming was enabled, n8n
integration would fail due to incorrect message type usage.

1. Added streaming capability detection by checking adapter's
is_stream_output_supported method
2. Implemented conditional message generation using MessageChunk for
streaming mode and Message for non-streaming mode
3. Added proper error handling for adapters that don't support streaming
detection

* fix: add n8n webhook streaming model ,Optimized the streaming output when calling n8n.

---------

Co-authored-by: Dong_master <2213070223@qq.com>
2025-11-22 14:17:46 +08:00
Junyan Qin
ba7cf69c9d doc: update READMEs 2025-11-22 00:20:39 +08:00
Junyan Qin
37296be67e feat: refactor plugin market interaction and migrate to LangBot Space
- Replace plugin detail dialog with hover buttons interaction
- Add "Install" and "View Details" hover buttons on plugin cards
- Remove PluginDetailDialog component
- Update plugin marketplace URL format to /market/{author}/{plugin}
- Redirect all plugin detail views to LangBot Space
- Add i18n support for 4 languages (zh-Hans, en-US, zh-Hant, ja-JP)
- Optimize hover overlay styles for light/dark theme

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-21 14:23:22 +08:00
Junyan Qin
6c03a1dd31 perf: add supports for showing multilingual plugin README 2025-11-21 12:14:04 +08:00
Junyan Qin
b75ec9e989 doc: add product hunt badges 2025-11-21 10:31:57 +08:00
Copilot
5c8523e4ef docs: Add multilingual README files (Spanish, French, Korean, Russian, Vietnamese) (#1794)
* Initial plan

* Add multilingual README files (ES, FR, KO, RU, VI)

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-21 00:36:35 +08:00
Junyan Qin
9802a42a9e perf: add request plugin button to marketplace 2025-11-20 23:41:45 +08:00
Junyan Qin
99e3abec72 chore: bump version 4.5.4 2025-11-20 23:19:37 +08:00
Junyan Qin
fc2efdf994 chore: bump langbot-plugin 0.1.12 2025-11-20 21:51:44 +08:00
Junyan Qin
6ed672d996 perf: tips msg for tool call 2025-11-20 21:45:22 +08:00
Junyan Qin
2bf593fa6b feat: pass session and query_id to tool call 2025-11-20 21:17:47 +08:00
Junyan Qin
3182214663 fix: linter errors 2025-11-20 19:48:34 +08:00
Junyan Qin
20614b20b7 feat: add component filter to marketplace page 2025-11-20 19:46:33 +08:00
Junyan Qin
da323817f7 feat: add plugin components displaying in marketplace page 2025-11-20 18:50:00 +08:00
Junyan Qin
763c1a885c perf: url display in webhook dialog 2025-11-20 16:48:06 +08:00
Junyan Qin
dbc09f46f4 perf: provider icon rounded in hovercard 2025-11-20 10:25:29 +08:00
Junyan Qin
cf43f09aff perf: auto refresh logic in market 2025-11-20 10:18:28 +08:00
Copilot
c3c51b0fbf perf: Add "Select All" checkbox to Plugin and MCP Server selection dialogs (#1790)
* Initial plan

* Add "Select All" checkbox to Plugin and MCP Server selection dialogs

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Make "Select All" text clickable by adding onClick handler to container

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-18 17:00:05 +08:00
Duke
8a42daa63f Fix wecom image message send fail issue (#1789)
* Fix wecom image upload issue

* Fix log
2025-11-18 16:02:13 +08:00
Junyan Qin
d91d98c9d4 chore: bump version 4.5.3 2025-11-18 11:31:28 +08:00
Copilot
2e82f2b2d1 fix: plugin pages scroll entire viewport instead of content area only (#1788)
* Initial plan

* Fix scroll behavior in plugin pages - only content areas scroll now

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-18 11:16:41 +08:00
Junyan Qin
f459c7017a chore: update pr template 2025-11-17 16:02:39 +08:00
Copilot
c27ccb8475 feat(web): Add centered empty state messages to pipeline extension dialogs (#1784)
* Initial plan

* feat: add empty state messages in pipeline extension dialogs

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* fix: center empty state messages in dialog content area

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-16 23:37:40 +08:00
Copilot
abb2f7ae05 feat(web): Move Get Help button to account menu (#1782)
* Initial plan

* feat: Move Get Help button to account options menu

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
2025-11-16 22:44:46 +08:00
Junyan Qin
80606ed32c docs: update README_JP 2025-11-16 20:44:33 +08:00
Junyan Qin
bc7c5fa864 chore: push first pypi package 2025-11-16 20:25:48 +08:00
Junyan Qin
ed0ea68037 doc: add uv link to READMEs 2025-11-16 20:04:34 +08:00
Junyan Qin
6ac4dbc011 doc: update README 2025-11-16 20:00:43 +08:00
Copilot
e642ffa5b3 chore: Add PyPI package support for uvx/pip installation (#1764)
* Initial plan

* Add package structure and resource path utilities

- Created langbot/ package with __init__.py and __main__.py entry point
- Added paths utility to find frontend and resource files from package installation
- Updated config loading to use resource paths
- Updated frontend serving to use resource paths
- Added MANIFEST.in for package data inclusion
- Updated pyproject.toml with build system and entry points

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Add PyPI publishing workflow and update license

- Created GitHub Actions workflow to build frontend and publish to PyPI
- Added license field to pyproject.toml to fix deprecation warning
- Updated .gitignore to exclude build artifacts
- Tested package building successfully

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Add PyPI installation documentation

- Created PYPI_INSTALLATION.md with detailed installation and usage instructions
- Updated README.md to feature uvx/pip installation as recommended method
- Updated README_EN.md with same changes for English documentation

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Address code review feedback

- Made package-data configuration more specific to langbot package only
- Improved path detection with caching to avoid repeated file I/O
- Removed sys.path searching which was incorrect for package data
- Removed interactive input() call for non-interactive environment compatibility
- Simplified error messages for version check

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Fix code review issues

- Use specific exception types instead of bare except
- Fix misleading comments about directory levels
- Remove redundant existence check before makedirs with exist_ok=True
- Use context manager for file opening to ensure proper cleanup

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* Simplify package configuration and document behavioral differences

- Removed redundant package-data configuration, relying on MANIFEST.in
- Added documentation about behavioral differences between package and source installation
- Clarified that include-package-data=true uses MANIFEST.in for data files

Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>

* chore: update pyproject.toml

* chore: try pack templates in langbot/

* chore: update

* chore: update

* chore: update

* chore: update

* chore: update

* chore: adjust dir structure

* chore: fix imports

* fix: read default-pipeline-config.json

* fix: read default-pipeline-config.json

* fix: tests

* ci: publish pypi

* chore: bump version 4.6.0-beta.1 for testing

* chore: add templates/**

* fix: send adapters and requesters icons

* chore: bump version 4.6.0b2 for testing

* chore: add platform field for docker-compose.yaml

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: RockChinQ <45992437+RockChinQ@users.noreply.github.com>
Co-authored-by: Junyan Qin <rockchinq@gmail.com>
2025-11-16 19:53:01 +08:00
305 changed files with 40292 additions and 15030 deletions

8
.dockerignore Normal file
View File

@@ -0,0 +1,8 @@
.github
.venv
.vscode
.data
.temp
web/.next
web/node_modules
web/.env

View File

@@ -2,6 +2,17 @@
> 请在此部分填写你实现/解决/优化的内容:
> Summary of what you implemented/solved/optimized:
>
### 更改前后对比截图 / Screenshots
> 请在此部分粘贴更改前后对比截图(可以是界面截图、控制台输出、对话截图等):
> Please paste the screenshots of changes before and after here (can be interface screenshots, console output, conversation screenshots, etc.):
>
> 修改前 / Before:
>
> 修改后 / After:
>
## 检查清单 / Checklist

View File

@@ -3,7 +3,6 @@ on:
## 发布release的时候会自动构建
release:
types: [published]
workflow_dispatch:
jobs:
publish-docker-image:
runs-on: ubuntu-latest
@@ -42,7 +41,7 @@ jobs:
run: docker buildx create --name mybuilder --use
- name: Build for Release # only relase, exlude pre-release
if: ${{ github.event.release.prerelease == false }}
run: docker buildx build --platform linux/amd64 -t rockchin/langbot:${{ steps.check_version.outputs.version }} -t rockchin/langbot:latest . --push
run: docker buildx build --platform linux/arm64,linux/amd64 -t rockchin/langbot:${{ steps.check_version.outputs.version }} -t rockchin/langbot:latest . --push
- name: Build for Pre-release # no update for latest tag
if: ${{ github.event.release.prerelease == true }}
run: docker buildx build --platform linux/amd64 -t rockchin/langbot:${{ steps.check_version.outputs.version }} . --push
run: docker buildx build --platform linux/arm64,linux/amd64 -t rockchin/langbot:${{ steps.check_version.outputs.version }} . --push

60
.github/workflows/lint.yml vendored Normal file
View File

@@ -0,0 +1,60 @@
name: Lint
on:
push:
branches:
- main
- master
- dev
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
jobs:
ruff:
name: Ruff Lint & Format
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Install dependencies
run: uv sync --dev
- name: Run ruff check
run: uv run ruff check src
- name: Run ruff format
run: uv run ruff format src --check
frontend:
name: Frontend Lint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '25'
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 9
- name: Install dependencies
working-directory: web
run: pnpm install
- name: Run lint
working-directory: web
run: pnpm lint

View File

@@ -26,7 +26,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12']
python-version: ['3.11', '3.12', '3.13']
fail-fast: false
steps:

1
.gitignore vendored
View File

@@ -42,7 +42,6 @@ botpy.log*
test.py
/web_ui
.venv/
uv.lock
/test
plugins.bak
coverage.xml

37
.mcp.json Normal file
View File

@@ -0,0 +1,37 @@
{
"mcpServers": {
"shadcn": {
"command": "npx",
"args": [
"shadcn@latest",
"mcp"
]
},
"sequential-thinking": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-sequential-thinking"],
"env": {}
},
"github": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_PERSONAL_ACCESS_TOKEN}"
}
},
"fetch": {
"type": "stdio",
"command": "uvx",
"args": ["mcp-server-fetch"],
"env": {}
},
"playwright": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@playwright/mcp@latest"],
"env": {}
}
}
}

View File

@@ -8,16 +8,17 @@ LangBot is a open-source LLM native instant messaging bot development platform,
LangBot has a comprehensive frontend, all operations can be performed through the frontend. The project splited into these major parts:
- `./pkg`: The core python package of the project backend.
- `./pkg/platform`: The platform module of the project, containing the logic of message platform adapters, bot managers, message session managers, etc.
- `./pkg/provider`: The provider module of the project, containing the logic of LLM providers, tool providers, etc.
- `./pkg/pipeline`: The pipeline module of the project, containing the logic of pipelines, stages, query pool, etc.
- `./pkg/api`: The api module of the project, containing the http api controllers and services.
- `./pkg/plugin`: LangBot bridge for connecting with plugin system.
- `./libs`: Some SDKs we previously developed for the project, such as `qq_official_api`, `wecom_api`, etc.
- `./templates`: Templates of config files, components, etc.
- `./web`: Frontend codebase, built with Next.js + **shadcn** + **Tailwind CSS**.
- `./docker`: docker-compose deployment files.
- `./src/langbot`: The main python package of the project, below are the main modules in this package:
- `./pkg`: The core python package of the project backend.
- `./pkg/platform`: The platform module of the project, containing the logic of message platform adapters, bot managers, message session managers, etc.
- `./pkg/provider`: The provider module of the project, containing the logic of LLM providers, tool providers, etc.
- `./pkg/pipeline`: The pipeline module of the project, containing the logic of pipelines, stages, query pool, etc.
- `./pkg/api`: The api module of the project, containing the http api controllers and services.
- `./pkg/plugin`: LangBot bridge for connecting with plugin system.
- `./libs`: Some SDKs we previously developed for the project, such as `qq_official_api`, `wecom_api`, etc.
- `./templates`: Templates of config files, components, etc.
- `./web`: Frontend codebase, built with Next.js + **shadcn** + **Tailwind CSS**.
- `./docker`: docker-compose deployment files.
## Backend Development
@@ -69,6 +70,7 @@ Plugin Runtime automatically starts each installed plugin and interacts through
- type: must be a specific type, such as feat (new feature), fix (bug fix), docs (documentation), style (code style), refactor (refactoring), perf (performance optimization), etc.
- scope: the scope of the commit, such as the package name, the file name, the function name, the class name, the module name, etc.
- subject: the subject of the commit, such as the description of the commit, the reason for the commit, the impact of the commit, etc.
- If you changed the definition of database entities, please update the migration file in `src/langbot/pkg/persistence/migrations/` and update the constants.py file in `src/langbot/pkg/utils/constants.py` with the new migration number.
## Some Principles

View File

@@ -20,4 +20,4 @@ RUN apt update \
&& uv sync \
&& touch /.dockerenv
CMD [ "uv", "run", "main.py" ]
CMD [ "uv", "run", "--no-sync", "main.py" ]

View File

@@ -1,55 +1,48 @@
<p align="center">
<a href="https://langbot.app">
<img src="https://docs.langbot.app/social_zh.png" alt="LangBot"/>
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://hellogithub.com/repository/langbot-app/LangBot" target="_blank"><img src="https://abroad.hellogithub.com/v1/widgets/recommend.svg?rid=5ce8ae2aa4f74316bf393b57b952433c&claim_uid=gtmc6YWjMZkT21R" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
[English](README_EN.md) / 简体中文 / [繁體中文](README_TW.md) / [日本語](README_JP.md) / (PR for your language)
<h3>使用 LangBot 快速构建、调试、部署即时通信机器人。</h3>
[English](README_EN.md) / 简体中文 / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-966235608-blue)](https://qm.qq.com/q/JLi38whHum)
[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-1030838208-blue)](https://qm.qq.com/q/DxZZcNxM1W)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">项目主页</a>
<a href="https://docs.langbot.app/zh/insight/features.html">规格特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">部署文档</a>
<a href="https://docs.langbot.app/zh/plugin/plugin-intro.html">插件介绍</a>
<a href="https://github.com/langbot-app/LangBot/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">提交插件</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API 集成</a>
<a href="https://space.langbot.app">插件市场</a>
<a href="https://langbot.featurebase.app/roadmap">路线图</a>
</div>
</p>
LangBot 是一个开源的大语言模型原生即时通信机器人开发平台,旨在提供开箱即用的 IM 机器人开发体验,具有 Agent、RAG、MCP 等多种 LLM 应用功能,适配全球主流即时通信平台,并提供丰富的 API 接口,支持自定义开发。
## 📦 开始使用
#### 快速体验(推荐)
#### 快速部署
使用 `uvx` 一键启动(无需安装
使用 `uvx` 一键启动(需要先安装 [uv](https://docs.astral.sh/uv/getting-started/installation/)
```bash
uvx langbot
```
或使用 `pip` 安装后运行:
```bash
pip install langbot
langbot
```
访问 http://localhost:5300 即可开始使用。
详细文档[PyPI 安装](docs/PYPI_INSTALLATION.md)。
#### Docker Compose 部署
```bash
@@ -92,11 +85,15 @@ docker compose up -d
## ✨ 特性
- 💬 大模型对话、Agent支持多种大模型适配群聊和私聊具有多轮对话、工具调用、多模态、流式输出能力自带 RAG知识库实现并深度适配 [Dify](https://dify.ai)。
- 🤖 多平台支持:目前支持 QQ、QQ频道、企业微信、个人微信、飞书、Discord、Telegram 等平台。
- 🛠️ 高稳定性、功能完备:原生支持访问控制、限速、敏感词过滤等机制;配置简单,支持多种部署方式。支持多流水线配置,不同机器人用于不同应用场景。
- 🧩 插件扩展、活跃社区:支持事件驱动、组件扩展等插件机制;适配 Anthropic [MCP 协议](https://modelcontextprotocol.io/);目前已有数百个插件
- 😻 Web 管理面板:支持通过浏览器管理 LangBot 实例,不再需要手动编写配置文件
<img width="500" src="https://docs.langbot.app/ui/bot-page-zh-rounded.png" />
- 💬 大模型对话、Agent支持多种大模型适配群聊和私聊具有多轮对话、工具调用、多模态、流式输出能力自带 RAG知识库实现并深度适配 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)等 LLMOps 平台
- 🤖 多平台支持:目前支持 QQ、QQ频道、企业微信、个人微信、飞书、Discord、Telegram、KOOK、Slack、LINE 等平台
- 🛠️ 高稳定性、功能完备:原生支持访问控制、限速、敏感词过滤等机制;配置简单,支持多种部署方式。
- 🧩 插件扩展、活跃社区:高稳定性、高安全性的生产级插件系统,支持事件驱动、组件扩展等插件机制;适配 Anthropic [MCP 协议](https://modelcontextprotocol.io/);目前已有数百个插件。
- 😻 Web 管理面板:提供先进的 WebUI 管理面板,用最直观的方式配置、管理、监控机器人。
- 📊 生产级特性:支持多流水线配置,不同机器人用于不同应用场景。具有全面的监控和异常处理能力。已被多家企业采用。
详细规格特性请访问[文档](https://docs.langbot.app/zh/insight/features.html)。
@@ -117,6 +114,7 @@ docker compose up -d
| 微信公众号 | ✅ | |
| 飞书 | ✅ | |
| 钉钉 | ✅ | |
| KOOK | ✅ | |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |

View File

@@ -1,11 +1,15 @@
<p align="center">
<a href="https://langbot.app">
<img src="https://docs.langbot.app/social_en.png" alt="LangBot"/>
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
English / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / (PR for your language)
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>Quickly build, debug, and ship IM bots with LangBot.</h3>
English / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
@@ -13,37 +17,29 @@ English / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語]
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">Home</a>
<a href="https://docs.langbot.app/en/insight/features.html">Features</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Deployment</a>
<a href="https://docs.langbot.app/en/plugin/plugin-intro.html">Plugin</a>
<a href="https://github.com/langbot-app/LangBot/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">Submit Plugin</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API Integration</a>
<a href="https://space.langbot.app">Plugin Market</a>
<a href="https://langbot.featurebase.app/roadmap">Roadmap</a>
</div>
</p>
LangBot is an open-source LLM native instant messaging robot development platform, aiming to provide out-of-the-box IM robot development experience, with Agent, RAG, MCP and other LLM application functions, adapting to global instant messaging platforms, and providing rich API interfaces, supporting custom development.
## 📦 Getting Started
#### Quick Start (Recommended)
#### Quick Start
Use `uvx` to start with one command (no installation required):
Use `uvx` to start with one command (need to install [uv](https://docs.astral.sh/uv/getting-started/installation/)):
```bash
uvx langbot
```
Or install with `pip` and run:
```bash
pip install langbot
langbot
```
Visit http://localhost:5300 to start using it.
Detailed documentation [PyPI Installation](docs/PYPI_INSTALLATION.md).
#### Docker Compose Deployment
```bash
@@ -86,11 +82,15 @@ Click the Star and Watch button in the upper right corner of the repository to g
## ✨ Features
- 💬 Chat with LLM / Agent: Supports multiple LLMs, adapt to group chats and private chats; Supports multi-round conversations, tool calls, multi-modal, and streaming output capabilities. Built-in RAG (knowledge base) implementation, and deeply integrates with [Dify](https://dify.ai).
- 🤖 Multi-platform Support: Currently supports QQ, QQ Channel, WeCom, personal WeChat, Lark, DingTalk, Discord, Telegram, etc.
- 🛠️ High Stability, Feature-rich: Native access control, rate limiting, sensitive word filtering, etc. mechanisms; Easy to use, supports multiple deployment methods. Supports multiple pipeline configurations, different bots can be used for different scenarios.
- 🧩 Plugin Extension, Active Community: Support event-driven, component extension, etc. plugin mechanisms; Integrate Anthropic [MCP protocol](https://modelcontextprotocol.io/); Currently has hundreds of plugins.
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 Chat with LLM / Agent: Supports multiple LLMs, adapt to group chats and private chats; Supports multi-round conversations, tool calls, multi-modal, and streaming output capabilities. Built-in RAG (knowledge base) implementation, and deeply integrates with [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) etc. LLMOps platforms.
- 🤖 Multi-platform Support: Currently supports QQ, QQ Channel, WeCom, personal WeChat, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE, etc.
- 🛠️ High Stability, Feature-rich: Native access control, rate limiting, sensitive word filtering, etc. mechanisms; Easy to use, supports multiple deployment methods.
- 🧩 Plugin Extension, Active Community: High stability, high security production-level plugin system; Support event-driven, component extension, etc. plugin mechanisms; Integrate Anthropic [MCP protocol](https://modelcontextprotocol.io/); Currently has hundreds of plugins.
- 😻 Web UI: Support management LangBot instance through the browser. No need to manually write configuration files.
- 📊 Production-grade Features: Supports multiple pipeline configurations, different bots can be used for different scenarios. Has comprehensive monitoring and exception handling capabilities.
For more detailed specifications, please refer to the [documentation](https://docs.langbot.app/en/insight/features.html).
@@ -114,6 +114,7 @@ Or visit the demo environment: https://demo.langbot.dev/
| Personal WeChat | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
### LLMs

151
README_ES.md Normal file
View File

@@ -0,0 +1,151 @@
<p align="center">
<a href="https://langbot.app">
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>Cree, depure y despliegue bots de mensajería instantánea rápidamente con LangBot.</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / Español / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">Inicio</a>
<a href="https://docs.langbot.app/en/insight/features.html">Características</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Despliegue</a>
<a href="https://docs.langbot.app/en/tags/readme.html">Integración API</a>
<a href="https://space.langbot.app">Mercado de Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Hoja de Ruta</a>
</div>
</p>
## 📦 Comenzar
#### Inicio Rápido
Use `uvx` para iniciar con un comando (necesita instalar [uv](https://docs.astral.sh/uv/getting-started/installation/)):
```bash
uvx langbot
```
Visite http://localhost:5300 para comenzar a usarlo.
#### Despliegue con Docker Compose
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
```
Visite http://localhost:5300 para comenzar a usarlo.
Documentación detallada [Despliegue con Docker](https://docs.langbot.app/en/deploy/langbot/docker.html).
#### Despliegue con un clic en BTPanel
LangBot ha sido listado en BTPanel. Si tiene BTPanel instalado, puede usar la [documentación](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) para usarlo.
#### Despliegue en la Nube Zeabur
Plantilla de Zeabur contribuida por la comunidad.
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
#### Despliegue en la Nube Railway
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
#### Otros Métodos de Despliegue
Use directamente la versión publicada para ejecutar, consulte la documentación de [Despliegue Manual](https://docs.langbot.app/en/deploy/langbot/manual.html).
#### Despliegue en Kubernetes
Consulte la documentación de [Despliegue en Kubernetes](./docker/README_K8S.md).
## 😎 Manténgase Actualizado
Haga clic en los botones Star y Watch en la esquina superior derecha del repositorio para obtener las últimas actualizaciones.
![star gif](https://docs.langbot.app/star.gif)
## ✨ Características
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 Chat con LLM / Agent: Compatible con múltiples LLMs, adaptado para chats grupales y privados; Admite conversaciones de múltiples rondas, llamadas a herramientas, capacidades multimodales y de salida en streaming. Implementación RAG (base de conocimientos) incorporada, e integración profunda con [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) etc. LLMOps platforms.
- 🤖 Soporte Multiplataforma: Actualmente compatible con QQ, QQ Channel, WeCom, WeChat personal, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE, etc.
- 🛠️ Alta Estabilidad, Rico en Funciones: Control de acceso nativo, limitación de velocidad, filtrado de palabras sensibles, etc.; Fácil de usar, admite múltiples métodos de despliegue.
- 🧩 Extensión de Plugin, Comunidad Activa: Sistema de plugin de alta estabilidad, alta seguridad de nivel de producción; Compatible con mecanismos de plugin impulsados por eventos, extensión de componentes, etc.; Integración del protocolo [MCP](https://modelcontextprotocol.io/) de Anthropic; Actualmente cuenta con cientos de plugins.
- 😻 Interfaz Web: Admite la gestión de instancias de LangBot a través del navegador. No es necesario escribir archivos de configuración manualmente.
- 📊 Características de Nivel de Producción: Compatible con múltiples configuraciones de pipeline, diferentes bots para diferentes escenarios. Cuenta con capacidades completas de monitoreo y manejo de excepciones.
Para especificaciones más detalladas, consulte la [documentación](https://docs.langbot.app/en/insight/features.html).
O visite el entorno de demostración: https://demo.langbot.dev/
- Información de inicio de sesión: Correo electrónico: `demo@langbot.app` Contraseña: `langbot123456`
- Nota: Solo para demostración de WebUI, por favor no ingrese información confidencial en el entorno público.
### Plataformas de Mensajería
| Plataforma | Estado | Observaciones |
| --- | --- | --- |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ Personal | ✅ | |
| QQ API Oficial | ✅ | |
| WeCom | ✅ | |
| WeComCS | ✅ | |
| WeCom AI Bot | ✅ | |
| WeChat Personal | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
### LLMs
| LLM | Estado | Observaciones |
| --- | --- | --- |
| [OpenAI](https://platform.openai.com/) | ✅ | Disponible para cualquier modelo con formato de interfaz OpenAI |
| [DeepSeek](https://www.deepseek.com/) | ✅ | |
| [Moonshot](https://www.moonshot.cn/) | ✅ | |
| [Anthropic](https://www.anthropic.com/) | ✅ | |
| [xAI](https://x.ai/) | ✅ | |
| [Zhipu AI](https://open.bigmodel.cn/) | ✅ | |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | Plataforma de recursos LLM y GPU |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | Plataforma de recursos LLM y GPU |
| [接口 AI](https://jiekou.ai/) | ✅ | Plataforma de agregación LLM |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | ✅ | Plataforma de recursos LLM y GPU |
| [302.AI](https://share.302.ai/SuTG99) | ✅ | Gateway LLM (MaaS) |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | |
| [Dify](https://dify.ai) | ✅ | Plataforma LLMOps |
| [Ollama](https://ollama.com/) | ✅ | Plataforma de ejecución de LLM local |
| [LMStudio](https://lmstudio.ai/) | ✅ | Plataforma de ejecución de LLM local |
| [GiteeAI](https://ai.gitee.com/) | ✅ | Gateway de interfaz LLM (MaaS) |
| [SiliconFlow](https://siliconflow.cn/) | ✅ | Gateway LLM (MaaS) |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | ✅ | Gateway LLM (MaaS), plataforma LLMOps |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | Gateway LLM (MaaS), plataforma LLMOps |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | Gateway LLM (MaaS) |
| [MCP](https://modelcontextprotocol.io/) | ✅ | Compatible con acceso a herramientas a través del protocolo MCP |
## 🤝 Contribución de la Comunidad
Gracias a los siguientes [contribuidores de código](https://github.com/langbot-app/LangBot/graphs/contributors) y otros miembros de la comunidad por sus contribuciones a LangBot:
<a href="https://github.com/langbot-app/LangBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=langbot-app/LangBot" />
</a>

150
README_FR.md Normal file
View File

@@ -0,0 +1,150 @@
<p align="center">
<a href="https://langbot.app">
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>Créez, déboguez et déployez rapidement des bots de messagerie instantanée avec LangBot.</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / Français / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">Accueil</a>
<a href="https://docs.langbot.app/en/insight/features.html">Fonctionnalités</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Déploiement</a>
<a href="https://docs.langbot.app/en/tags/readme.html">Intégration API</a>
<a href="https://space.langbot.app">Marché des Plugins</a>
<a href="https://langbot.featurebase.app/roadmap">Feuille de Route</a>
</div>
</p>
## 📦 Commencer
#### Démarrage Rapide
Utilisez `uvx` pour démarrer avec une commande (besoin d'installer [uv](https://docs.astral.sh/uv/getting-started/installation/)) :
```bash
uvx langbot
```
Visitez http://localhost:5300 pour commencer à l'utiliser.
#### Déploiement avec Docker Compose
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
```
Visitez http://localhost:5300 pour commencer à l'utiliser.
Documentation détaillée [Déploiement Docker](https://docs.langbot.app/en/deploy/langbot/docker.html).
#### Déploiement en un clic sur BTPanel
LangBot a été répertorié sur BTPanel. Si vous avez installé BTPanel, vous pouvez utiliser la [documentation](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) pour l'utiliser.
#### Déploiement Cloud Zeabur
Modèle Zeabur contribué par la communauté.
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
#### Déploiement Cloud Railway
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
#### Autres Méthodes de Déploiement
Utilisez directement la version publiée pour exécuter, consultez la documentation de [Déploiement Manuel](https://docs.langbot.app/en/deploy/langbot/manual.html).
#### Déploiement Kubernetes
Consultez la documentation de [Déploiement Kubernetes](./docker/README_K8S.md).
## 😎 Restez à Jour
Cliquez sur les boutons Star et Watch dans le coin supérieur droit du dépôt pour obtenir les dernières mises à jour.
![star gif](https://docs.langbot.app/star.gif)
## ✨ Fonctionnalités
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 Chat avec LLM / Agent : Prend en charge plusieurs LLM, adapté aux chats de groupe et privés ; Prend en charge les conversations multi-tours, les appels d'outils, les capacités multimodales et de sortie en streaming. Implémentation RAG (base de connaissances) intégrée, et intégration profonde avec [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) etc. LLMOps platforms.
- 🤖 Support Multi-plateforme : Actuellement compatible avec QQ, QQ Channel, WeCom, WeChat personnel, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE, etc.
- 🛠️ Haute Stabilité, Riche en Fonctionnalités : Contrôle d'accès natif, limitation de débit, filtrage de mots sensibles, etc. ; Facile à utiliser, prend en charge plusieurs méthodes de déploiement.
- 🧩 Extension de Plugin, Communauté Active : Système de plugin de haute stabilité, haute sécurité de niveau production; Prend en charge les mécanismes de plugin pilotés par événements, l'extension de composants, etc. ; Intégration du protocole [MCP](https://modelcontextprotocol.io/) d'Anthropic ; Dispose actuellement de centaines de plugins.
- 😻 Interface Web : Prend en charge la gestion des instances LangBot via le navigateur. Pas besoin d'écrire manuellement les fichiers de configuration.
- 📊 Fonctionnalités de Niveau Production : Prend en charge plusieurs configurations de pipeline, différents bots pour différents scénarios. Dispose de capacités complètes de surveillance et de gestion des exceptions.
Pour des spécifications plus détaillées, veuillez consulter la [documentation](https://docs.langbot.app/en/insight/features.html).
Ou visitez l'environnement de démonstration : https://demo.langbot.dev/
- Informations de connexion : Email : `demo@langbot.app` Mot de passe : `langbot123456`
- Note : Pour la démonstration WebUI uniquement, veuillez ne pas entrer d'informations sensibles dans l'environnement public.
### Plateformes de Messagerie
| Plateforme | Statut | Remarques |
| --- | --- | --- |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ Personnel | ✅ | |
| API Officielle QQ | ✅ | |
| WeCom | ✅ | |
| WeComCS | ✅ | |
| WeCom AI Bot | ✅ | |
| WeChat Personnel | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
### LLMs
| LLM | Statut | Remarques |
| --- | --- | --- |
| [OpenAI](https://platform.openai.com/) | ✅ | Disponible pour tout modèle au format d'interface OpenAI |
| [DeepSeek](https://www.deepseek.com/) | ✅ | |
| [Moonshot](https://www.moonshot.cn/) | ✅ | |
| [Anthropic](https://www.anthropic.com/) | ✅ | |
| [xAI](https://x.ai/) | ✅ | |
| [Zhipu AI](https://open.bigmodel.cn/) | ✅ | |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | Plateforme de ressources LLM et GPU |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | Plateforme de ressources LLM et GPU |
| [接口 AI](https://jiekou.ai/) | ✅ | Plateforme d'agrégation LLM |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | ✅ | Plateforme de ressources LLM et GPU |
| [302.AI](https://share.302.ai/SuTG99) | ✅ | Passerelle LLM (MaaS) |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | |
| [Dify](https://dify.ai) | ✅ | Plateforme LLMOps |
| [Ollama](https://ollama.com/) | ✅ | Plateforme d'exécution LLM locale |
| [LMStudio](https://lmstudio.ai/) | ✅ | Plateforme d'exécution LLM locale |
| [GiteeAI](https://ai.gitee.com/) | ✅ | Passerelle d'interface LLM (MaaS) |
| [SiliconFlow](https://siliconflow.cn/) | ✅ | Passerelle LLM (MaaS) |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | ✅ | Passerelle LLM (MaaS), plateforme LLMOps |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | Passerelle LLM (MaaS), plateforme LLMOps |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | Passerelle LLM (MaaS) |
| [MCP](https://modelcontextprotocol.io/) | ✅ | Prend en charge l'accès aux outils via le protocole MCP |
## 🤝 Contribution de la Communauté
Merci aux [contributeurs de code](https://github.com/langbot-app/LangBot/graphs/contributors) suivants et aux autres membres de la communauté pour leurs contributions à LangBot :
<a href="https://github.com/langbot-app/LangBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=langbot-app/LangBot" />
</a>

View File

@@ -1,11 +1,15 @@
<p align="center">
<a href="https://langbot.app">
<img src="https://docs.langbot.app/social_en.png" alt="LangBot"/>
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / 日本語 / (PR for your language)
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>LangBotでIMボットを素早く構築、デバッグ、デプロイ。</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / 日本語 / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
@@ -13,18 +17,28 @@
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">ホーム</a>
<a href="https://docs.langbot.app/en/insight/guide.html">デプロイ</a>
<a href="https://docs.langbot.app/en/plugin/plugin-intro.html">プラグイン</a>
<a href="https://github.com/langbot-app/LangBot/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">プラグインの提出</a>
<a href="https://docs.langbot.app/ja/insight/features.html">機能仕様</a>
<a href="https://docs.langbot.app/ja/insight/guide.html">デプロイ</a>
<a href="https://docs.langbot.app/ja/tags/readme.html">API統合</a>
<a href="https://space.langbot.app">プラグインマーケット</a>
<a href="https://langbot.featurebase.app/roadmap">ロードマップ</a>
</div>
</p>
LangBot は、エージェント、RAG、MCP などの LLM アプリケーション機能を備えた、オープンソースの LLM ネイティブのインスタントメッセージングロボット開発プラットフォームです。世界中のインスタントメッセージングプラットフォームに適応し、豊富な API インターフェースを提供し、カスタム開発をサポートします。
## 📦 始め方
#### クイックスタート
`uvx` を使用した迅速なデプロイ([uv](https://docs.astral.sh/uv/getting-started/installation/) が必要です):
```bash
uvx langbot
```
http://localhost:5300 にアクセスして使用を開始します。
#### Docker Compose デプロイ
```bash
@@ -67,11 +81,15 @@ LangBotはBTPanelにリストされています。BTPanelをインストール
## ✨ 機能
- 💬 LLM / エージェントとのチャット: 複数のLLMをサポートし、グループチャットとプライベートチャットに対応。マルチラウンドの会話、ツールの呼び出し、マルチモーダル、ストリーミング出力機能をサポート、RAG知識ベースを組み込み、[Dify](https://dify.ai) と深く統合。
- 🤖 多プラットフォーム対応: 現在、QQ、QQ チャンネル、WeChat、個人 WeChat、Lark、DingTalk、Discord、Telegram など、複数のプラットフォームをサポートしています。
- 🛠️ 高い安定性、豊富な機能: ネイティブのアクセス制御、レート制限、敏感な単語のフィルタリングなどのメカニズムをサポート。使いやすく、複数のデプロイ方法をサポート。複数のパイプライン設定をサポートし、異なるボットを異なる用途に使用できます。
- 🧩 プラグイン拡張、活発なコミュニティ: イベント駆動、コンポーネント拡張などのプラグインメカニズムをサポート。適配 Anthropic [MCP プロトコル](https://modelcontextprotocol.io/);豊富なエコシステム、現在数百のプラグインが存在
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 LLM / エージェントとのチャット: 複数のLLMをサポートし、グループチャットとプライベートチャットに対応。マルチラウンドの会話、ツールの呼び出し、マルチモーダル、ストリーミング出力機能をサポート、RAG知識ベースを組み込み、[Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)などの LLMOps プラットフォームと深く統合
- 🤖 多プラットフォーム対応: 現在、QQ、QQ チャンネル、WeChat、個人 WeChat、Lark、DingTalk、Discord、Telegram、KOOK、Slack、LINE など、複数のプラットフォームをサポートしています。
- 🛠️ 高い安定性、豊富な機能: ネイティブのアクセス制御、レート制限、敏感な単語のフィルタリングなどのメカニズムをサポート。使いやすく、複数のデプロイ方法をサポート。
- 🧩 プラグイン拡張、活発なコミュニティ: 高い安定性、高いセキュリティの生産レベルのプラグインシステム;イベント駆動、コンポーネント拡張などのプラグインメカニズムをサポート。適配 Anthropic [MCP プロトコル](https://modelcontextprotocol.io/);豊富なエコシステム、現在数百のプラグインが存在。
- 😻 Web UI: ブラウザを通じてLangBotインスタンスを管理することをサポート。
- 📊 生産レベルの機能: 複数のパイプライン設定をサポートし、異なるボットを異なる用途に使用できます。包括的な監視と例外処理機能を備えています。
詳細な仕様については、[ドキュメント](https://docs.langbot.app/en/insight/features.html)を参照してください。
@@ -95,6 +113,7 @@ LangBotはBTPanelにリストされています。BTPanelをインストール
| 個人WeChat | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
| KOOK | ✅ | |
### LLMs

150
README_KO.md Normal file
View File

@@ -0,0 +1,150 @@
<p align="center">
<a href="https://langbot.app">
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>LangBot으로 IM 봇을 빠르게 구축, 디버그 및 배포하세요.</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / 한국어 / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">홈</a>
<a href="https://docs.langbot.app/en/insight/features.html">기능 사양</a>
<a href="https://docs.langbot.app/en/insight/guide.html">배포</a>
<a href="https://docs.langbot.app/en/tags/readme.html">API 통합</a>
<a href="https://space.langbot.app">플러그인 마켓</a>
<a href="https://langbot.featurebase.app/roadmap">로드맵</a>
</div>
</p>
## 📦 시작하기
#### 빠른 시작
`uvx`를 사용하여 한 명령으로 시작하세요 ([uv](https://docs.astral.sh/uv/getting-started/installation/) 설치 필요):
```bash
uvx langbot
```
http://localhost:5300을 방문하여 사용을 시작하세요.
#### Docker Compose 배포
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
```
http://localhost:5300을 방문하여 사용을 시작하세요.
자세한 문서는 [Docker 배포](https://docs.langbot.app/en/deploy/langbot/docker.html)를 참조하세요.
#### BTPanel 원클릭 배포
LangBot은 BTPanel에 등록되어 있습니다. BTPanel을 설치한 경우 [문서](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html)를 사용하여 사용할 수 있습니다.
#### Zeabur 클라우드 배포
커뮤니티에서 제공하는 Zeabur 템플릿입니다.
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
#### Railway 클라우드 배포
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
#### 기타 배포 방법
릴리스 버전을 직접 사용하여 실행하려면 [수동 배포](https://docs.langbot.app/en/deploy/langbot/manual.html) 문서를 참조하세요.
#### Kubernetes 배포
[Kubernetes 배포](./docker/README_K8S.md) 문서를 참조하세요.
## 😎 최신 정보 받기
리포지토리 오른쪽 상단의 Star 및 Watch 버튼을 클릭하여 최신 업데이트를 받으세요.
![star gif](https://docs.langbot.app/star.gif)
## ✨ 기능
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 LLM / Agent와 채팅: 여러 LLM을 지원하며 그룹 채팅 및 개인 채팅에 적응; 멀티 라운드 대화, 도구 호출, 멀티모달, 스트리밍 출력 기능을 지원합니다. 내장된 RAG(지식 베이스) 구현 및 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)등의 LLMOps 플랫폼과 깊이 통합됩니다.
- 🤖 다중 플랫폼 지원: 현재 QQ, QQ Channel, WeCom, 개인 WeChat, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE 등을 지원합니다.
- 🛠️ 높은 안정성, 풍부한 기능: 네이티브 액세스 제어, 속도 제한, 민감한 단어 필터링 등의 메커니즘; 사용하기 쉽고 여러 배포 방법을 지원합니다.
- 🧩 플러그인 확장, 활발한 커뮤니티: 고안정성, 고보안 생산 수준의 플러그인 시스템; 이벤트 기반, 컴포넌트 확장 등의 플러그인 메커니즘을 지원; Anthropic [MCP 프로토콜](https://modelcontextprotocol.io/) 통합; 현재 수백 개의 플러그인이 있습니다.
- 😻 웹 UI: 브라우저를 통해 LangBot 인스턴스 관리를 지원합니다. 구성 파일을 수동으로 작성할 필요가 없습니다.
- 📊 생산 수준의 기능: 여러 파이프라인 구성을 지원하며 다양한 시나리오에 대해 다른 봇을 사용할 수 있습니다. 포괄적인 모니터링 및 예외 처리 기능을 갖추고 있습니다.
더 자세한 사양은 [문서](https://docs.langbot.app/en/insight/features.html)를 참조하세요.
또는 데모 환경을 방문하세요: https://demo.langbot.dev/
- 로그인 정보: 이메일: `demo@langbot.app` 비밀번호: `langbot123456`
- 참고: WebUI 데모 전용이므로 공개 환경에서는 민감한 정보를 입력하지 마세요.
### 메시징 플랫폼
| 플랫폼 | 상태 | 비고 |
| --- | --- | --- |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| 개인 QQ | ✅ | |
| QQ 공식 API | ✅ | |
| WeCom | ✅ | |
| WeComCS | ✅ | |
| WeCom AI Bot | ✅ | |
| 개인 WeChat | ✅ | |
| KOOK | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
### LLMs
| LLM | 상태 | 비고 |
| --- | --- | --- |
| [OpenAI](https://platform.openai.com/) | ✅ | 모든 OpenAI 인터페이스 형식 모델에 사용 가능 |
| [DeepSeek](https://www.deepseek.com/) | ✅ | |
| [Moonshot](https://www.moonshot.cn/) | ✅ | |
| [Anthropic](https://www.anthropic.com/) | ✅ | |
| [xAI](https://x.ai/) | ✅ | |
| [Zhipu AI](https://open.bigmodel.cn/) | ✅ | |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | LLM 및 GPU 리소스 플랫폼 |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | LLM 및 GPU 리소스 플랫폼 |
| [接口 AI](https://jiekou.ai/) | ✅ | LLM 집계 플랫폼 |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | ✅ | LLM 및 GPU 리소스 플랫폼 |
| [302.AI](https://share.302.ai/SuTG99) | ✅ | LLM 게이트웨이(MaaS) |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | |
| [Dify](https://dify.ai) | ✅ | LLMOps 플랫폼 |
| [Ollama](https://ollama.com/) | ✅ | 로컬 LLM 실행 플랫폼 |
| [LMStudio](https://lmstudio.ai/) | ✅ | 로컬 LLM 실행 플랫폼 |
| [GiteeAI](https://ai.gitee.com/) | ✅ | LLM 인터페이스 게이트웨이(MaaS) |
| [SiliconFlow](https://siliconflow.cn/) | ✅ | LLM 게이트웨이(MaaS) |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | ✅ | LLM 게이트웨이(MaaS), LLMOps 플랫폼 |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | LLM 게이트웨이(MaaS), LLMOps 플랫폼 |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | LLM 게이트웨이(MaaS) |
| [MCP](https://modelcontextprotocol.io/) | ✅ | MCP 프로토콜을 통한 도구 액세스 지원 |
## 🤝 커뮤니티 기여
다음 [코드 기여자](https://github.com/langbot-app/LangBot/graphs/contributors) 및 커뮤니티의 다른 구성원들의 LangBot 기여에 감사드립니다:
<a href="https://github.com/langbot-app/LangBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=langbot-app/LangBot" />
</a>

150
README_RU.md Normal file
View File

@@ -0,0 +1,150 @@
<p align="center">
<a href="https://langbot.app">
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>Быстро создавайте, отлаживайте и развертывайте IM-ботов с LangBot.</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / Русский / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">Главная</a>
<a href="https://docs.langbot.app/en/insight/features.html">Характеристики</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Развертывание</a>
<a href="https://docs.langbot.app/en/tags/readme.html">Интеграция API</a>
<a href="https://space.langbot.app">Магазин плагинов</a>
<a href="https://langbot.featurebase.app/roadmap">Дорожная карта</a>
</div>
</p>
## 📦 Начало работы
#### Быстрый старт
Используйте `uvx` для запуска одной командой (требуется установка [uv](https://docs.astral.sh/uv/getting-started/installation/)):
```bash
uvx langbot
```
Посетите http://localhost:5300, чтобы начать использование.
#### Развертывание с Docker Compose
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
```
Посетите http://localhost:5300, чтобы начать использование.
Подробная документация [Развертывание Docker](https://docs.langbot.app/en/deploy/langbot/docker.html).
#### Развертывание одним кликом на BTPanel
LangBot добавлен в BTPanel. Если у вас установлен BTPanel, вы можете использовать [документацию](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) для его использования.
#### Облачное развертывание Zeabur
Шаблон Zeabur, предоставленный сообществом.
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
#### Облачное развертывание Railway
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
#### Другие методы развертывания
Используйте выпущенную версию напрямую для запуска, см. документацию [Ручное развертывание](https://docs.langbot.app/en/deploy/langbot/manual.html).
#### Развертывание Kubernetes
См. документацию [Развертывание Kubernetes](./docker/README_K8S.md).
## 😎 Оставайтесь в курсе
Нажмите кнопки Star и Watch в правом верхнем углу репозитория, чтобы получать последние обновления.
![star gif](https://docs.langbot.app/star.gif)
## ✨ Функции
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 Чат с LLM / Agent: Поддержка нескольких LLM, адаптация к групповым и личным чатам; Поддержка многораундовых разговоров, вызовов инструментов, мультимодальных возможностей и потоковой передачи. Встроенная реализация RAG (база знаний) и глубокая интеграция с [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) и др. LLMOps платформами.
- 🤖 Многоплатформенная поддержка: В настоящее время поддерживает QQ, QQ Channel, WeCom, личный WeChat, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE и т.д.
- 🛠️ Высокая стабильность, богатство функций: Нативный контроль доступа, ограничение скорости, фильтрация чувствительных слов и т.д.; Простота в использовании, поддержка нескольких методов развертывания.
- 🧩 Расширение плагинов, активное сообщество: Высокая стабильность, высокая безопасность уровня производства; Поддержка механизмов плагинов, управляемых событиями, расширения компонентов и т.д.; Интеграция протокола [MCP](https://modelcontextprotocol.io/) от Anthropic; В настоящее время сотни плагинов.
- 😻 Веб-интерфейс: Поддержка управления экземплярами LangBot через браузер. Нет необходимости вручную писать конфигурационные файлы.
- 📊 Функции уровня производства: Поддержка нескольких конфигураций конвейера, разные боты для разных сценариев. Имеет комплексные возможности мониторинга и обработки исключений.
Для более подробных спецификаций обратитесь к [документации](https://docs.langbot.app/en/insight/features.html).
Или посетите демонстрационную среду: https://demo.langbot.dev/
- Информация для входа: Email: `demo@langbot.app` Пароль: `langbot123456`
- Примечание: Только для демонстрации WebUI, пожалуйста, не вводите конфиденциальную информацию в общедоступной среде.
### Платформы обмена сообщениями
| Платформа | Статус | Примечания |
| --- | --- | --- |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| Личный QQ | ✅ | |
| Официальный API QQ | ✅ | |
| WeCom | ✅ | |
| WeComCS | ✅ | |
| WeCom AI Bot | ✅ | |
| Личный WeChat | ✅ | |
| KOOK | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
### LLMs
| LLM | Статус | Примечания |
| --- | --- | --- |
| [OpenAI](https://platform.openai.com/) | ✅ | Доступна для любой модели формата интерфейса OpenAI |
| [DeepSeek](https://www.deepseek.com/) | ✅ | |
| [Moonshot](https://www.moonshot.cn/) | ✅ | |
| [Anthropic](https://www.anthropic.com/) | ✅ | |
| [xAI](https://x.ai/) | ✅ | |
| [Zhipu AI](https://open.bigmodel.cn/) | ✅ | |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | Платформа ресурсов LLM и GPU |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | Платформа ресурсов LLM и GPU |
| [接口 AI](https://jiekou.ai/) | ✅ | Платформа агрегации LLM |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | ✅ | Платформа ресурсов LLM и GPU |
| [302.AI](https://share.302.ai/SuTG99) | ✅ | Шлюз LLM (MaaS) |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | |
| [Dify](https://dify.ai) | ✅ | Платформа LLMOps |
| [Ollama](https://ollama.com/) | ✅ | Платформа локального запуска LLM |
| [LMStudio](https://lmstudio.ai/) | ✅ | Платформа локального запуска LLM |
| [GiteeAI](https://ai.gitee.com/) | ✅ | Шлюз интерфейса LLM (MaaS) |
| [SiliconFlow](https://siliconflow.cn/) | ✅ | Шлюз LLM (MaaS) |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | ✅ | Шлюз LLM (MaaS), платформа LLMOps |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | Шлюз LLM (MaaS), платформа LLMOps |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | Шлюз LLM (MaaS) |
| [MCP](https://modelcontextprotocol.io/) | ✅ | Поддержка доступа к инструментам через протокол MCP |
## 🤝 Вклад сообщества
Спасибо следующим [контрибьюторам кода](https://github.com/langbot-app/LangBot/graphs/contributors) и другим членам сообщества за их вклад в LangBot:
<a href="https://github.com/langbot-app/LangBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=langbot-app/LangBot" />
</a>

View File

@@ -1,11 +1,13 @@
<p align="center">
<a href="https://langbot.app">
<img src="https://docs.langbot.app/social_zh.png" alt="LangBot"/>
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center"><a href="https://hellogithub.com/repository/langbot-app/LangBot" target="_blank"><img src="https://abroad.hellogithub.com/v1/widgets/recommend.svg?rid=5ce8ae2aa4f74316bf393b57b952433c&claim_uid=gtmc6YWjMZkT21R" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
[English](README_EN.md) / [简体中文](README.md) / 繁體中文 / [日本語](README_JP.md) / (PR for your language)
<h3>使用 LangBot 快速建構、除錯和部署 IM 機器人。</h3>
[English](README_EN.md) / [简体中文](README.md) / 繁體中文 / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / [Tiếng Việt](README_VI.md)
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-966235608-blue)](https://qm.qq.com/q/JLi38whHum)
@@ -15,18 +17,28 @@
[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot)
<a href="https://langbot.app">主頁</a>
<a href="https://docs.langbot.app/zh/insight/features.html">規格特性</a>
<a href="https://docs.langbot.app/zh/insight/guide.html">部署文件</a>
<a href="https://docs.langbot.app/zh/plugin/plugin-intro.html">外掛介紹</a>
<a href="https://github.com/langbot-app/LangBot/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">提交外掛</a>
<a href="https://docs.langbot.app/zh/tags/readme.html">API 整合</a>
<a href="https://space.langbot.app">外掛市場</a>
<a href="https://langbot.featurebase.app/roadmap">路線圖</a>
</div>
</p>
LangBot 是一個開源的大語言模型原生即時通訊機器人開發平台,旨在提供開箱即用的 IM 機器人開發體驗,具有 Agent、RAG、MCP 等多種 LLM 應用功能,適配全球主流即時通訊平台,並提供豐富的 API 介面,支援自定義開發。
## 📦 開始使用
#### 快速部署
使用 `uvx` 一鍵啟動(需要先安裝 [uv](https://docs.astral.sh/uv/getting-started/installation/)
```bash
uvx langbot
```
訪問 http://localhost:5300 即可開始使用。
#### Docker Compose 部署
```bash
@@ -69,11 +81,15 @@ docker compose up -d
## ✨ 特性
- 💬 大模型對話、Agent支援多種大模型適配群聊和私聊具有多輪對話、工具調用、多模態、流式輸出能力自帶 RAG知識庫實現並深度適配 [Dify](https://dify.ai)。
- 🤖 多平台支援:目前支援 QQ、QQ頻道、企業微信、個人微信、飛書、Discord、Telegram 等平台。
- 🛠️ 高穩定性、功能完備:原生支援訪問控制、限速、敏感詞過濾等機制;配置簡單,支援多種部署方式。支援多流水線配置,不同機器人用於不同應用場景。
- 🧩 外掛擴展、活躍社群:支援事件驅動、組件擴展等外掛機制;適配 Anthropic [MCP 協議](https://modelcontextprotocol.io/);目前已有數百個外掛
- 😻 Web 管理面板:支援通過瀏覽器管理 LangBot 實例,不再需要手動編寫配置文件
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 大模型對話、Agent支援多種大模型適配群聊和私聊具有多輪對話、工具調用、多模態、流式輸出能力自帶 RAG知識庫實現並深度適配 [Dify](https://dify.ai)、[Coze](https://coze.com)、[n8n](https://n8n.io)、[Langflow](https://langflow.org)等 LLMOps 平台
- 🤖 多平台支援:目前支援 QQ、QQ頻道、企業微信、個人微信、飛書、Discord、Telegram、KOOK、Slack、LINE 等平台
- 🛠️ 高穩定性、功能完備:原生支援訪問控制、限速、敏感詞過濾等機制;配置簡單,支援多種部署方式。
- 🧩 外掛擴展、活躍社群:高穩定性、高安全性的生產級外掛系統;支援事件驅動、組件擴展等外掛機制;適配 Anthropic [MCP 協議](https://modelcontextprotocol.io/);目前已有數百個外掛。
- 😻 Web 管理面板:提供先進的 WebUI 管理面板,用最直觀的方式配置、管理、監控機器人。
- 📊 生產級特性:支援多流水線配置,不同機器人用於不同應用場景。具有全面的監控和異常處理能力。
詳細規格特性請訪問[文件](https://docs.langbot.app/zh/insight/features.html)。
@@ -95,6 +111,7 @@ docker compose up -d
| 企微對外客服 | ✅ | |
| 企微智能機器人 | ✅ | |
| 微信公眾號 | ✅ | |
| KOOK | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |

150
README_VI.md Normal file
View File

@@ -0,0 +1,150 @@
<p align="center">
<a href="https://langbot.app">
<img width="130" src="https://docs.langbot.app/langbot-logo.png" alt="LangBot"/>
</a>
<div align="center">
<a href="https://www.producthunt.com/products/langbot?utm_source=badge-follow&utm_medium=badge&utm_source=badge-langbot" target="_blank"><img src="https://api.producthunt.com/widgets/embed-image/v1/follow.svg?product_id=1077185&theme=light" alt="LangBot - Production&#0045;grade&#0032;IM&#0032;bot&#0032;made&#0032;easy&#0046; | Product Hunt" style="width: 250px; height: 54px;" width="250" height="54" /></a>
<h3>Xây dựng, gỡ lỗi và triển khai bot IM nhanh chóng với LangBot.</h3>
[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / [Español](README_ES.md) / [Français](README_FR.md) / [한국어](README_KO.md) / [Русский](README_RU.md) / Tiếng Việt
[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot)
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10 ~ 3.13 -blue.svg" alt="python">
<a href="https://langbot.app">Trang chủ</a>
<a href="https://docs.langbot.app/en/insight/features.html">Tính năng</a>
<a href="https://docs.langbot.app/en/insight/guide.html">Triển khai</a>
<a href="https://docs.langbot.app/en/tags/readme.html">Tích hợp API</a>
<a href="https://space.langbot.app">Chợ Plugin</a>
<a href="https://langbot.featurebase.app/roadmap">Lộ trình</a>
</div>
</p>
## 📦 Bắt đầu
#### Khởi động Nhanh
Sử dụng `uvx` để khởi động bằng một lệnh (cần cài đặt [uv](https://docs.astral.sh/uv/getting-started/installation/)):
```bash
uvx langbot
```
Truy cập http://localhost:5300 để bắt đầu sử dụng.
#### Triển khai Docker Compose
```bash
git clone https://github.com/langbot-app/LangBot
cd LangBot/docker
docker compose up -d
```
Truy cập http://localhost:5300 để bắt đầu sử dụng.
Tài liệu chi tiết [Triển khai Docker](https://docs.langbot.app/en/deploy/langbot/docker.html).
#### Triển khai Một cú nhấp chuột trên BTPanel
LangBot đã được liệt kê trên BTPanel. Nếu bạn đã cài đặt BTPanel, bạn có thể sử dụng [tài liệu](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html) để sử dụng nó.
#### Triển khai Cloud Zeabur
Mẫu Zeabur được đóng góp bởi cộng đồng.
[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/en-US/templates/ZKTBDH)
#### Triển khai Cloud Railway
[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF)
#### Các Phương pháp Triển khai Khác
Sử dụng trực tiếp phiên bản phát hành để chạy, xem tài liệu [Triển khai Thủ công](https://docs.langbot.app/en/deploy/langbot/manual.html).
#### Triển khai Kubernetes
Tham khảo tài liệu [Triển khai Kubernetes](./docker/README_K8S.md).
## 😎 Cập nhật Mới nhất
Nhấp vào các nút Star và Watch ở góc trên bên phải của kho lưu trữ để nhận các bản cập nhật mới nhất.
![star gif](https://docs.langbot.app/star.gif)
## ✨ Tính năng
<img width="500" src="https://docs.langbot.app/ui/bot-page-en-rounded.png" />
- 💬 Chat với LLM / Agent: Hỗ trợ nhiều LLM, thích ứng với chat nhóm và chat riêng tư; Hỗ trợ các cuộc trò chuyện nhiều vòng, gọi công cụ, khả năng đa phương thức và đầu ra streaming. Triển khai RAG (cơ sở kiến thức) tích hợp sẵn và tích hợp sâu với [Dify](https://dify.ai), [Coze](https://coze.com), [n8n](https://n8n.io), [Langflow](https://langflow.org) v.v. LLMOps platforms.
- 🤖 Hỗ trợ Đa nền tảng: Hiện hỗ trợ QQ, QQ Channel, WeCom, WeChat cá nhân, Lark, DingTalk, Discord, Telegram, KOOK, Slack, LINE, v.v.
- 🛠️ Độ ổn định Cao, Tính năng Phong phú: Kiểm soát truy cập gốc, giới hạn tốc độ, lọc từ nhạy cảm, v.v.; Dễ sử dụng, hỗ trợ nhiều phương pháp triển khai.
- 🧩 Mở rộng Plugin, Cộng đồng Hoạt động: Hỗ trợ các cơ chế plugin hướng sự kiện, mở rộng thành phần, v.v.; Tích hợp giao thức [MCP](https://modelcontextprotocol.io/) của Anthropic; Hiện có hàng trăng plugin.
- 😻 Giao diện Web: Hỗ trợ quản lý các phiên bản LangBot thông qua trình duyệt. Không cần viết tệp cấu hình thủ công.
- 📊 Tính năng Cấp sản xuất: Hỗ trợ nhiều cấu hình pipeline, các bot khác nhau cho các kịch bản khác nhau. Có khả năng giám sát toàn diện và xử lý ngoại lệ.
Để biết thêm thông số kỹ thuật chi tiết, vui lòng tham khảo [tài liệu](https://docs.langbot.app/en/insight/features.html).
Hoặc truy cập môi trường demo: https://demo.langbot.dev/
- Thông tin đăng nhập: Email: `demo@langbot.app` Mật khẩu: `langbot123456`
- Lưu ý: Chỉ dành cho demo WebUI, vui lòng không nhập bất kỳ thông tin nhạy cảm nào trong môi trường công cộng.
### Nền tảng Nhắn tin
| Nền tảng | Trạng thái | Ghi chú |
| --- | --- | --- |
| Discord | ✅ | |
| Telegram | ✅ | |
| Slack | ✅ | |
| LINE | ✅ | |
| QQ Cá nhân | ✅ | |
| QQ API Chính thức | ✅ | |
| WeCom | ✅ | |
| WeComCS | ✅ | |
| WeCom AI Bot | ✅ | |
| WeChat Cá nhân | ✅ | |
| KOOK | ✅ | |
| Lark | ✅ | |
| DingTalk | ✅ | |
### LLMs
| LLM | Trạng thái | Ghi chú |
| --- | --- | --- |
| [OpenAI](https://platform.openai.com/) | ✅ | Có sẵn cho bất kỳ mô hình định dạng giao diện OpenAI nào |
| [DeepSeek](https://www.deepseek.com/) | ✅ | |
| [Moonshot](https://www.moonshot.cn/) | ✅ | |
| [Anthropic](https://www.anthropic.com/) | ✅ | |
| [xAI](https://x.ai/) | ✅ | |
| [Zhipu AI](https://open.bigmodel.cn/) | ✅ | |
| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | Nền tảng tài nguyên LLM và GPU |
| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | Nền tảng tài nguyên LLM và GPU |
| [接口 AI](https://jiekou.ai/) | ✅ | Nền tảng tổng hợp LLM |
| [ShengSuanYun](https://www.shengsuanyun.com/?from=CH_KYIPP758) | ✅ | Nền tảng tài nguyên LLM và GPU |
| [302.AI](https://share.302.ai/SuTG99) | ✅ | Cổng LLM (MaaS) |
| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | |
| [Dify](https://dify.ai) | ✅ | Nền tảng LLMOps |
| [Ollama](https://ollama.com/) | ✅ | Nền tảng chạy LLM cục bộ |
| [LMStudio](https://lmstudio.ai/) | ✅ | Nền tảng chạy LLM cục bộ |
| [GiteeAI](https://ai.gitee.com/) | ✅ | Cổng giao diện LLM (MaaS) |
| [SiliconFlow](https://siliconflow.cn/) | ✅ | Cổng LLM (MaaS) |
| [Aliyun Bailian](https://bailian.console.aliyun.com/) | ✅ | Cổng LLM (MaaS), nền tảng LLMOps |
| [Volc Engine Ark](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | Cổng LLM (MaaS), nền tảng LLMOps |
| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | Cổng LLM (MaaS) |
| [MCP](https://modelcontextprotocol.io/) | ✅ | Hỗ trợ truy cập công cụ qua giao thức MCP |
## 🤝 Đóng góp Cộng đồng
Cảm ơn các [người đóng góp mã](https://github.com/langbot-app/LangBot/graphs/contributors) sau đây và các thành viên khác trong cộng đồng vì những đóng góp của họ cho LangBot:
<a href="https://github.com/langbot-app/LangBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=langbot-app/LangBot" />
</a>

View File

@@ -14,7 +14,7 @@ services:
restart: on-failure
environment:
- TZ=Asia/Shanghai
command: ["uv", "run", "-m", "langbot_plugin.cli.__init__", "rt"]
command: ["uv", "run", "--no-sync", "-m", "langbot_plugin.cli.__init__", "rt"]
networks:
- langbot_network
@@ -23,13 +23,12 @@ services:
container_name: langbot
volumes:
- ./data:/app/data
- ./plugins:/app/plugins
restart: on-failure
environment:
- TZ=Asia/Shanghai
ports:
- 5300:5300 # For web ui
- 2280-2290:2280-2290 # For platform webhook
- 5300:5300 # For web ui and webhook callback
- 2280-2285:2280-2285 # For platform reverse connection
networks:
- langbot_network

412
docs/MIGRATION_SUMMARY.md Normal file
View File

@@ -0,0 +1,412 @@
# WebChat 到 WebSocket 迁移总结
## 概述
已完全移除旧的基于SSE的WebChat系统并替换为基于WebSocket的双向实时通信系统。这是一个内置在LangBot中的完整IM系统支持流式输出。
## 已删除的文件
### 后端
-`src/langbot/pkg/api/http/controller/groups/pipelines/webchat.py` - 旧的SSE路由
-`src/langbot/pkg/platform/sources/webchat.py` - 旧的WebChat适配器
-`src/langbot/pkg/platform/sources/webchat.yaml` - 旧的配置文件
### 前端
- ❌ BackendClient中所有SSE相关代码已完全移除
- ❌ DebugDialog中所有SSE相关逻辑已完全替换
## 新增的文件
### 后端核心文件
**1. WebSocket连接管理器**
```
src/langbot/pkg/platform/sources/websocket_manager.py
```
- 管理所有并发WebSocket连接
- 线程安全的连接池
- 按流水线、会话类型分组
- 广播和单播消息功能
- 连接统计和监控
**2. WebSocket适配器**
```
src/langbot/pkg/platform/sources/websocket_adapter.py
```
- 实现平台适配器接口
- **完整流式支持** (`reply_message_chunk` 方法)
- 双向消息流处理
- 消息历史管理
- 会话管理
**3. WebSocket路由控制器**
```
src/langbot/pkg/api/http/controller/groups/pipelines/websocket_chat.py
```
- WebSocket端点处理
- REST API接口
- 心跳机制
- 连接生命周期管理
**4. 配置文件**
```
src/langbot/pkg/platform/sources/websocket.yaml
```
- WebSocket适配器元数据
### 前端核心文件
**1. WebSocket客户端**
```
web/src/app/infra/websocket/WebSocketClient.ts
```
- WebSocket连接管理
- 自动重连最多5次
- 心跳机制30秒
- 事件回调系统
**2. 更新的组件**
```
web/src/app/home/pipelines/components/debug-dialog/DebugDialog.tsx
```
- 完全重写使用WebSocket
- 实时连接状态显示
- 流式消息支持
- 自动重连
**3. HTTP客户端更新**
```
web/src/app/infra/http/BackendClient.ts
```
- 移除所有旧的WebChat API
- 仅保留WebSocket API
### 测试工具
**Python测试客户端**
```
test_websocket_client.py
```
- 单连接交互测试
- 多连接并发测试
- 命令行工具
### 文档
**使用文档**
```
WEBSOCKET_README.md
```
- 完整的API文档
- 架构说明
- 使用示例
- 故障排查
## 核心变更
### 后端变更
**1. botmgr.py**
- ❌ 移除 `webchat_proxy_bot`
- ✅ 仅保留 `websocket_proxy_bot`
- ✅ 更新适配器过滤逻辑(排除`websocket`而非`webchat`
**2. 适配器注册**
```python
# 旧代码(已删除)
webchat_adapter_class = self.adapter_dict['webchat']
self.webchat_proxy_bot = RuntimeBot(...)
# 新代码
websocket_adapter_class = self.adapter_dict['websocket']
self.websocket_proxy_bot = RuntimeBot(
uuid='websocket-proxy-bot',
name='WebSocket',
adapter='websocket',
...
)
```
### 前端变更
**1. API调用完全更换**
旧代码(已删除):
```typescript
// SSE流式请求
await fetch(url, {
method: 'POST',
body: JSON.stringify({ is_stream: true })
})
// 手动解析 text/event-stream
```
新代码:
```typescript
// WebSocket实时通信
const wsClient = new WebSocketClient(pipelineId, sessionType);
await wsClient.connect();
wsClient.onMessage((message) => {
// 流式消息自动处理
setMessages(prev => [...prev, message]);
});
wsClient.sendMessage(messageChain);
```
**2. 连接状态管理**
新增功能:
- ✅ 实时连接状态指示器(绿色/红色圆点)
- ✅ 连接/断开toast提示
- ✅ 自动重连逻辑
- ✅ 心跳保活
**3. 流式支持**
完整的流式消息处理:
```typescript
wsClient.onMessage((message) => {
if (message.is_final) {
// 最终消息
finalizeBotMessage(message);
} else {
// 中间消息块实时更新UI
updateBotMessage(message);
}
});
```
## API对比
### WebSocket端点
**连接**
```
ws://localhost:8000/api/v1/pipelines/<pipeline_uuid>/ws/connect?session_type=<person|group>
```
**消息格式**
客户端发送:
```json
{
"type": "message",
"message": [
{"type": "Plain", "text": "你好"}
]
}
```
服务器响应(流式):
```json
{
"type": "response",
"data": {
"id": 1,
"role": "assistant",
"content": "你好,我是...",
"is_final": false,
"timestamp": "2025-01-28T..."
}
}
```
### REST API
| 端点 | 方法 | 说明 |
|------|------|------|
| `/api/v1/pipelines/<uuid>/ws/messages/<type>` | GET | 获取消息历史 |
| `/api/v1/pipelines/<uuid>/ws/reset/<type>` | POST | 重置会话 |
| `/api/v1/pipelines/<uuid>/ws/connections` | GET | 获取连接统计 |
| `/api/v1/pipelines/<uuid>/ws/broadcast` | POST | 广播消息 |
## 流式支持详解
### 后端流式实现
**WebSocket Adapter**
```python
async def reply_message_chunk(
self,
message_source: platform_events.MessageEvent,
bot_message,
message: platform_message.MessageChain,
quote_origin: bool = False,
is_final: bool = False,
) -> dict:
"""回复消息块 - 流式"""
message_data = WebSocketMessage(
id=-1,
role='assistant',
content=str(message),
message_chain=[component.__dict__ for component in message],
timestamp=datetime.now().isoformat(),
is_final=is_final and bot_message.tool_calls is None,
)
# 发送到队列由WebSocket连接处理发送
await session.resp_queues[message_id].put(message_data)
return message_data.model_dump()
async def is_stream_output_supported(self) -> bool:
"""WebSocket始终支持流式输出"""
return True
```
### 前端流式处理
**DebugDialog组件**
```typescript
wsClient.onMessage((message) => {
setMessages((prevMessages) => {
const existingIndex = prevMessages.findIndex(
(msg) => msg.role === 'assistant' && msg.content === 'Generating...'
);
if (existingIndex !== -1) {
// 更新正在生成的消息
const updatedMessages = [...prevMessages];
updatedMessages[existingIndex] = message;
return updatedMessages;
} else {
// 添加新消息
return [...prevMessages, message];
}
});
});
```
## 兼容性说明
### ⚠️ 不兼容旧版本
此次迁移**完全不兼容**旧的WebChat系统
1. **API端点变更**
- 旧: `/api/v1/pipelines/<uuid>/chat/send`
- 新: `ws://.../<uuid>/ws/connect`
2. **通信协议变更**
- 旧: HTTP + SSE (Server-Sent Events)
- 新: WebSocket (双向)
3. **流式实现变更**
- 旧: `text/event-stream` 格式
- 新: WebSocket JSON消息
### 迁移要求
使用新系统需要:
1. ✅ 前端必须支持WebSocket
2. ✅ 后端必须运行新的WebSocket适配器
3. ✅ 清除旧的WebChat相关配置
## 优势对比
| 特性 | 旧WebChat (SSE) | 新WebSocket |
|------|----------------|-------------|
| 双向通信 | ❌ 单向(服务器→客户端) | ✅ 双向 |
| 主动推送 | ❌ 不支持 | ✅ 支持 |
| 连接管理 | ❌ 无状态 | ✅ 有状态,完整生命周期 |
| 流式输出 | ✅ 支持 | ✅ 支持(更优) |
| 心跳机制 | ❌ 无 | ✅ 30秒心跳 |
| 自动重连 | ❌ 无 | ✅ 最多5次 |
| 多连接 | ⚠️ 难以管理 | ✅ 完整支持 |
| 连接状态 | ❌ 不可见 | ✅ 实时显示 |
| 广播功能 | ❌ 不支持 | ✅ 支持 |
## 测试方式
### 1. Python测试客户端
```bash
# 单连接测试
python test_websocket_client.py <pipeline_uuid>
# 指定会话类型
python test_websocket_client.py <pipeline_uuid> --session-type group
# 多连接并发测试5个连接
python test_websocket_client.py <pipeline_uuid> --multi 5
```
### 2. 前端测试
1. 启动LangBot服务器
2. 访问前端界面
3. 打开流水线调试对话框
4. 观察连接状态指示器(左下角圆点)
5. 发送消息测试流式响应
### 3. 浏览器控制台测试
```javascript
const ws = new WebSocket('ws://localhost:8000/api/v1/pipelines/<uuid>/ws/connect?session_type=person');
ws.onopen = () => {
console.log('已连接');
ws.send(JSON.stringify({
type: 'message',
message: [{type: 'Plain', text: '你好'}]
}));
};
ws.onmessage = (event) => {
console.log('收到:', JSON.parse(event.data));
};
```
## 常见问题
### Q: 为什么完全删除旧代码而不保留兼容性?
A: 根据需求,不需要考虑任何对老版本的兼容性,彻底迁移可以避免代码冗余和维护负担。
### Q: 流式输出如何工作?
A:
1. 后端通过`reply_message_chunk`发送消息块
2. 消息块放入队列
3. WebSocket连接从队列取出并发送
4. 前端实时更新UI
5. `is_final=true`表示最后一块
### Q: 如何确保连接不断开?
A:
1. 客户端每30秒发送心跳ping
2. 服务器响应pong
3. 连接断开时自动重连最多5次
### Q: 如何实现后端主动推送?
A:
1. 调用 `/api/v1/pipelines/<uuid>/ws/broadcast` API
2. 消息会被推送到该流水线的所有连接
3. 前端通过`onBroadcast`回调接收
## 总结
**完成的工作**
- 完全移除旧的WebChat/SSE系统
- 实现完整的WebSocket双向通信系统
- 支持流式输出
- 支持多连接并发
- 实现自动重连和心跳机制
- 提供完整的测试工具和文档
**核心特性**
- 双向实时通信
- 流式消息支持
- 多连接管理
- 自动重连
- 心跳保活
- 连接状态可视化
- 广播消息
**技术亮点**
- 异步架构asyncio
- 线程安全的连接管理
- 独立的消息队列
- 完整的错误处理
- 模块化设计
🎉 系统已完全迁移到WebSocket无任何旧代码遗留

259
docs/SEEKDB_INTEGRATION.md Normal file
View File

@@ -0,0 +1,259 @@
# SeekDB Vector Database Integration
This document describes how to use OceanBase SeekDB as the vector database backend for LangBot's knowledge base feature.
## What is SeekDB?
**OceanBase SeekDB** is an AI-native search database that unifies relational, vector, text, JSON and GIS in a single engine, enabling hybrid search and in-database AI workflows. It's developed by OceanBase and released under Apache 2.0 license.
### Key Features
- **Hybrid Search**: Combine vector search, full-text search and relational query in a single statement
- **Multi-Model Support**: Support relational, vector, text, JSON and GIS in a single engine
- **Lightweight**: Requires as little as 1 CPU core and 2 GB of memory
- **Multiple Deployment Modes**: Supports both embedded mode and client/server mode
- **MySQL Compatible**: Powered by OceanBase engine with full ACID compliance and MySQL compatibility
## Installation
SeekDB support is automatically included when you install LangBot. The required dependency `pyseekdb` is listed in `pyproject.toml`.
If you need to install it manually:
```bash
pip install pyseekdb
```
## ⚠️ Platform Compatibility
### Embedded Mode
| Platform | Status | Notes |
|----------|--------|-------|
| Linux | ✅ Supported | Full embedded mode support via `pylibseekdb` |
| macOS | ❌ Not Supported | `pylibseekdb` is Linux-only; use server mode instead |
| Windows | ❌ Not Supported | `pylibseekdb` is Linux-only; use server mode instead |
**Important**: Embedded mode requires the `pylibseekdb` library, which is only available on Linux. If you're on macOS or Windows, you must use server mode.
### Server Mode (Docker)
| Platform | Status | Notes |
|----------|--------|-------|
| Linux | ✅ Supported | Full Docker support |
| macOS | ⚠️ Known Issue | Docker container initialization failure - [See Issue #36](https://github.com/oceanbase/seekdb/issues/36) |
| Windows | ⚠️ Untested | Should work but not yet tested |
**macOS Users**: Currently, SeekDB Docker containers have an initialization issue on macOS ([oceanbase/seekdb#36](https://github.com/oceanbase/seekdb/issues/36)). Until this is resolved, we recommend:
- Using ChromaDB or Qdrant as alternatives
- Connecting to a remote SeekDB server on Linux if available
### Server Mode (Remote Connection)
| Platform | Status | Notes |
|----------|--------|-------|
| All Platforms | ✅ Supported | Connect to SeekDB running on a remote Linux server |
**Recommendation for macOS/Windows users**: Deploy SeekDB on a Linux server and connect via server mode configuration.
## Configuration
### Embedded Mode (Recommended for Development)
Embedded mode runs SeekDB directly within the LangBot process, storing data locally. This is the simplest setup and requires no external services.
Edit your `config.yaml`:
```yaml
vdb:
use: seekdb
seekdb:
mode: embedded
path: './data/seekdb' # Path to store SeekDB data
database: 'langbot' # Database name
```
### Server Mode (For Production)
Server mode connects to a remote SeekDB server or OceanBase server. This is recommended for production deployments.
#### SeekDB Server
```yaml
vdb:
use: seekdb
seekdb:
mode: server
host: 'localhost'
port: 2881
database: 'langbot'
user: 'root'
password: '' # Can also use SEEKDB_PASSWORD env var
```
#### OceanBase Server
If you're using OceanBase with seekdb capabilities:
```yaml
vdb:
use: seekdb
seekdb:
mode: server
host: 'localhost'
port: 2881
tenant: 'sys' # OceanBase tenant name
database: 'langbot'
user: 'root'
password: ''
```
## Configuration Parameters
| Parameter | Required | Default | Description |
|-----------|----------|--------------|-------------|
| `mode` | No | `embedded` | Deployment mode: `embedded` or `server` |
| `path` | No | `./data/seekdb` | Data directory for embedded mode |
| `database` | No | `langbot` | Database name |
| `host` | No | `localhost` | Server host (server mode only) |
| `port` | No | `2881` | Server port (server mode only) |
| `user` | No | `root` | Username (server mode only) |
| `password` | No | `''` | Password (server mode only) |
| `tenant` | No | None | OceanBase tenant (optional, server mode only) |
## Usage
Once configured, SeekDB will be used automatically for all knowledge base operations in LangBot:
1. **Creating Knowledge Bases**: Vectors will be stored in SeekDB collections
2. **Adding Documents**: Document embeddings will be indexed in SeekDB
3. **Searching**: Vector similarity search will use SeekDB's efficient indexing
4. **Deleting**: Document removal will delete vectors from SeekDB
No code changes are required - just update your configuration!
## Architecture Details
### Implementation
The SeekDB adapter is implemented in `src/langbot/pkg/vector/vdbs/seekdb.py` and follows the same `VectorDatabase` interface as Chroma and Qdrant adapters.
Key methods:
- `add_embeddings()`: Add vectors with metadata to a collection
- `search()`: Perform vector similarity search
- `delete_by_file_id()`: Delete vectors by file ID metadata
- `get_or_create_collection()`: Manage collections
- `delete_collection()`: Remove entire collections
### Vector Storage
- Collections are created with HNSW (Hierarchical Navigable Small World) index
- Default distance metric: Cosine similarity
- Default vector dimension: 384 (adjusts automatically based on embeddings)
- Metadata is stored alongside vectors for filtering
## Advantages Over Other Vector Databases
### vs. ChromaDB
- ✅ Better MySQL compatibility
- ✅ Hybrid search capabilities (vector + full-text + SQL)
- ✅ Production-grade distributed mode support
- ✅ Lightweight embedded mode
### vs. Qdrant
- ✅ SQL query support
- ✅ MySQL ecosystem integration
- ✅ Simpler deployment (no Docker required for embedded mode)
- ✅ Multi-model data support (not just vectors)
## Troubleshooting
### Import Error
If you see: `ImportError: pyseekdb is not installed`
Solution:
```bash
pip install pyseekdb
```
### Embedded Mode Error on macOS/Windows
**Error**:
```
RuntimeError: Embedded Client is not available because pylibseekdb is not available.
Please install pylibseekdb (Linux only) or use RemoteServerClient (host/port) instead.
```
**Cause**: `pylibseekdb` is only available on Linux platforms.
**Solution**: Use server mode instead:
1. Deploy SeekDB on a Linux server or VM
2. Configure LangBot to use server mode:
```yaml
vdb:
use: seekdb
seekdb:
mode: server
host: 'your-seekdb-server-ip'
port: 2881
database: 'langbot'
user: 'root'
password: ''
```
**Alternative**: Use ChromaDB or Qdrant, which work on all platforms:
```yaml
vdb:
use: chroma # or qdrant
```
### Docker Container Fails on macOS
**Symptoms**:
```bash
docker run -d -p 2881:2881 oceanbase/seekdb:latest
# Container exits immediately with code 30
```
**Error in logs**:
```
[ERROR] Code: Agent.SeekDB.Not.Exists
Message: initialize failed: init agent failed: SeekDB not exists in current directory.
```
**Cause**: This is a known issue with SeekDB Docker containers on macOS. See [oceanbase/seekdb#36](https://github.com/oceanbase/seekdb/issues/36).
**Status**: Under investigation by OceanBase team.
**Workaround Options**:
1. **Use alternatives**: ChromaDB or Qdrant work perfectly on macOS
2. **Remote server**: Deploy SeekDB on a Linux server and connect remotely
3. **Wait for fix**: Monitor the GitHub issue for updates
### Connection Error (Server Mode)
If SeekDB server is not reachable, check:
1. Server is running: `ps aux | grep observer`
2. Port is accessible: `nc -zv localhost 2881`
3. Credentials are correct in config
4. Firewall allows connections on port 2881
### Performance Issues
For large datasets:
- Use server mode instead of embedded mode
- Ensure adequate memory allocation
- Consider using OceanBase distributed mode for very large scale
- Adjust HNSW index parameters if needed
## Resources
- SeekDB GitHub: https://github.com/oceanbase/seekdb
- pyseekdb SDK: https://github.com/oceanbase/pyseekdb
- OceanBase Documentation: https://oceanbase.ai
- LangBot Documentation: https://docs.langbot.app
## License
SeekDB is licensed under Apache License 2.0.

394
docs/WEBSOCKET_README.md Normal file
View File

@@ -0,0 +1,394 @@
# LangBot WebSocket 双向通信系统
## 概述
这是一个内置在 LangBot 中的完整 IM (即时通讯) 系统,支持:
- ✅ WebSocket 双向实时通信
- ✅ 多个客户端并发连接
- ✅ 前端到后端的消息发送
- ✅ 后端到前端的主动推送
- ✅ 流式响应支持
- ✅ 连接管理和会话隔离
- ✅ 心跳机制
- ✅ 广播消息功能
## 架构设计
### 核心组件
1. **WebSocketConnectionManager** (`websocket_manager.py`)
- 管理所有活跃的 WebSocket 连接
- 支持按流水线、会话类型查询连接
- 提供广播和单播功能
- 线程安全的并发访问控制
2. **WebSocketAdapter** (`websocket_adapter.py`)
- 实现平台适配器接口
- 处理消息的接收和发送
- 支持流式输出
- 管理消息历史
3. **WebSocketChatRouterGroup** (`websocket_chat.py`)
- WebSocket 路由控制器
- 处理连接建立、消息收发
- 实现心跳机制
- 提供 REST API 接口
## API 接口
### WebSocket 连接
#### 建立连接
```
ws://localhost:8000/api/v1/pipelines/<pipeline_uuid>/ws/connect?session_type=<person|group>
```
**参数:**
- `pipeline_uuid`: 流水线 UUID (必需)
- `session_type`: 会话类型,可选 `person``group` (默认: `person`)
**连接成功响应:**
```json
{
"type": "connected",
"connection_id": "550e8400-e29b-41d4-a716-446655440000",
"pipeline_uuid": "your-pipeline-uuid",
"session_type": "person",
"timestamp": "2025-01-28T12:00:00"
}
```
### 消息格式
#### 客户端发送消息
**发送聊天消息:**
```json
{
"type": "message",
"message": [
{
"type": "Plain",
"text": "你好,这是一条测试消息"
}
]
}
```
**发送心跳:**
```json
{
"type": "ping"
}
```
**主动断开连接:**
```json
{
"type": "disconnect"
}
```
#### 服务器响应消息
**聊天响应 (流式):**
```json
{
"type": "response",
"data": {
"id": 1,
"role": "assistant",
"content": "这是机器人的回复",
"message_chain": [...],
"timestamp": "2025-01-28T12:00:00",
"is_final": false,
"connection_id": "..."
}
}
```
**心跳响应:**
```json
{
"type": "pong",
"timestamp": "2025-01-28T12:00:00"
}
```
**广播消息:**
```json
{
"type": "broadcast",
"message": "这是一条广播消息",
"timestamp": "2025-01-28T12:00:00"
}
```
**错误消息:**
```json
{
"type": "error",
"message": "错误描述"
}
```
### REST API 接口
#### 1. 获取消息历史
```http
GET /api/v1/pipelines/<pipeline_uuid>/ws/messages/<session_type>
```
**响应:**
```json
{
"code": 0,
"msg": "ok",
"data": {
"messages": [...]
}
}
```
#### 2. 重置会话
```http
POST /api/v1/pipelines/<pipeline_uuid>/ws/reset/<session_type>
```
**响应:**
```json
{
"code": 0,
"msg": "ok",
"data": {
"message": "Session reset successfully"
}
}
```
#### 3. 获取连接统计
```http
GET /api/v1/pipelines/<pipeline_uuid>/ws/connections
```
**响应:**
```json
{
"code": 0,
"msg": "ok",
"data": {
"stats": {
"total_connections": 5,
"pipelines": 2,
"connections_by_pipeline": {
"pipeline-1": 3,
"pipeline-2": 2
},
"connections_by_session_type": {
"person": 4,
"group": 1
}
},
"connections": [
{
"connection_id": "...",
"session_type": "person",
"created_at": "2025-01-28T12:00:00",
"last_active": "2025-01-28T12:05:00",
"is_active": true
}
]
}
}
```
#### 4. 广播消息 (后端主动推送)
```http
POST /api/v1/pipelines/<pipeline_uuid>/ws/broadcast
Content-Type: application/json
{
"message": "广"
}
```
**响应:**
```json
{
"code": 0,
"msg": "ok",
"data": {
"message": "Broadcast sent successfully"
}
}
```
## 使用示例
### Python 客户端示例
使用提供的测试客户端:
```bash
# 安装依赖
pip install websockets
# 单个连接测试
python test_websocket_client.py <pipeline_uuid>
# 指定会话类型
python test_websocket_client.py <pipeline_uuid> --session-type group
# 多连接并发测试
python test_websocket_client.py <pipeline_uuid> --multi 5
```
### JavaScript 客户端示例
```javascript
// 建立 WebSocket 连接
const ws = new WebSocket('ws://localhost:8000/api/v1/pipelines/your-pipeline-uuid/ws/connect?session_type=person');
// 连接建立
ws.onopen = () => {
console.log('WebSocket 连接已建立');
// 发送消息
ws.send(JSON.stringify({
type: 'message',
message: [
{
type: 'Plain',
text: '你好'
}
]
}));
};
// 接收消息
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.type === 'connected') {
console.log('连接成功:', data.connection_id);
} else if (data.type === 'response') {
console.log('机器人回复:', data.data.content);
if (data.data.is_final) {
console.log('响应完成');
}
} else if (data.type === 'broadcast') {
console.log('收到广播:', data.message);
}
};
// 连接关闭
ws.onclose = () => {
console.log('WebSocket 连接已关闭');
};
// 错误处理
ws.onerror = (error) => {
console.error('WebSocket 错误:', error);
};
// 发送心跳
setInterval(() => {
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify({ type: 'ping' }));
}
}, 30000); // 每 30 秒发送一次心跳
```
## 特性说明
### 1. 多连接支持
系统支持同时建立多个 WebSocket 连接,每个连接都有唯一的 `connection_id`。连接按照流水线和会话类型进行分组管理。
### 2. 双向通信
- **前端 → 后端**: 客户端可以主动发送消息给服务器
- **后端 → 前端**: 服务器可以通过广播 API 主动推送消息给客户端
### 3. 流式响应
支持流式输出,机器人的响应会分块发送,客户端可以实时显示部分响应内容。
### 4. 会话隔离
支持 `person``group` 两种会话类型,不同类型的会话消息历史互不影响。
### 5. 连接管理
- 自动追踪连接状态
- 记录最后活跃时间
- 支持连接统计查询
- 连接断开时自动清理资源
### 6. 心跳机制
客户端可以定期发送 `ping` 消息,服务器会响应 `pong`,用于保持连接活跃和检测连接状态。
## 架构优势
1. **高并发**: 使用 asyncio 异步架构,支持大量并发连接
2. **可扩展**: 模块化设计,易于扩展新功能
3. **线程安全**: 连接管理器使用锁机制保证并发安全
4. **消息队列**: 每个连接独立的发送队列,避免消息混乱
5. **灵活路由**: 支持按流水线、会话类型灵活路由消息
## 注意事项
1. **认证**: 当前 WebSocket 连接不需要认证,生产环境建议添加认证机制
2. **心跳**: 建议客户端实现心跳机制,避免连接超时
3. **重连**: 客户端应实现断线重连逻辑
4. **消息大小**: 注意控制单条消息大小,避免内存溢出
5. **连接数限制**: 生产环境建议设置最大连接数限制
## 故障排查
### 连接失败
1. 检查流水线 UUID 是否正确
2. 检查服务器是否正常运行
3. 检查防火墙设置
### 消息发送失败
1. 检查消息格式是否正确
2. 检查连接是否仍然活跃
3. 查看服务器日志获取详细错误信息
### 性能问题
1. 检查并发连接数是否过多
2. 检查消息处理速度
3. 考虑使用连接池或负载均衡
## 开发调试
启用详细日志:
```python
import logging
logging.getLogger('langbot.pkg.platform.sources.websocket_adapter').setLevel(logging.DEBUG)
logging.getLogger('langbot.pkg.platform.sources.websocket_manager').setLevel(logging.DEBUG)
logging.getLogger('langbot.pkg.api.http.controller.groups.pipelines.websocket_chat').setLevel(logging.DEBUG)
```
## 后续改进建议
1. 添加用户认证和授权机制
2. 实现消息持久化
3. 添加消息加密
4. 实现更丰富的消息类型 (图片、文件等)
5. 添加消息已读/未读状态
6. 实现群组聊天功能
7. 添加在线状态显示
8. 实现消息撤回功能

View File

@@ -1,10 +1,10 @@
[project]
name = "langbot"
version = "4.6.0-beta.2"
description = "Easy-to-use global IM bot platform designed for LLM era"
version = "4.8.1"
description = "Production-grade platform for building agentic IM bots"
readme = "README.md"
license-files = ["LICENSE"]
requires-python = ">=3.10.1,<4.0"
requires-python = ">=3.11,<4.0"
dependencies = [
"aiocqhttp>=1.4.4",
"aiofiles>=24.1.0",
@@ -23,7 +23,7 @@ dependencies = [
"pynacl>=1.5.0", # Required for Discord voice support
"gewechat-client>=0.1.5",
"lark-oapi>=1.4.15",
"mcp>=1.8.1",
"mcp>=1.25.0",
"nakuru-project-idk>=0.0.2.1",
"ollama>=0.4.8",
"openai>1.0.0",
@@ -63,11 +63,14 @@ dependencies = [
"langchain-text-splitters>=0.0.1",
"chromadb>=0.4.24",
"qdrant-client (>=1.15.1,<2.0.0)",
"langbot-plugin==0.1.11b1",
"pyseekdb==1.0.0b7",
"langbot-plugin==0.2.5",
"asyncpg>=0.30.0",
"line-bot-sdk>=3.19.0",
"tboxsdk>=0.0.10",
"boto3>=1.35.0",
"pymilvus>=2.6.4",
"pgvector>=0.4.1",
]
keywords = [
"bot",

View File

@@ -26,7 +26,7 @@ markers =
# Coverage options (when using pytest-cov)
[coverage:run]
source = langbot.pkg
source = langbot
omit =
*/tests/*
*/test_*.py

View File

@@ -22,7 +22,7 @@ echo "Running all unit tests..."
# Run tests with coverage
pytest tests/unit_tests/ -v --tb=short \
--cov=pkg \
--cov=langbot \
--cov-report=xml \
"$@"

View File

@@ -1,3 +1,3 @@
"""LangBot - Easy-to-use global IM bot platform designed for LLM era"""
"""LangBot - Production-grade platform for building agentic IM bots"""
__version__ = '4.6.0-beta.2'
__version__ = '4.8.1'

View File

@@ -32,6 +32,7 @@ class AsyncDifyServiceClient:
conversation_id: str = '',
files: list[dict[str, typing.Any]] = [],
timeout: float = 30.0,
model_config: dict[str, typing.Any] | None = None,
) -> typing.AsyncGenerator[dict[str, typing.Any], None]:
"""发送消息"""
if response_mode != 'streaming':
@@ -42,6 +43,16 @@ class AsyncDifyServiceClient:
trust_env=True,
timeout=timeout,
) as client:
payload = {
'inputs': inputs,
'query': query,
'user': user,
'response_mode': response_mode,
'conversation_id': conversation_id,
'files': files,
'model_config': model_config or {},
}
async with client.stream(
'POST',
'/chat-messages',
@@ -49,14 +60,7 @@ class AsyncDifyServiceClient:
'Authorization': f'Bearer {self.api_key}',
'Content-Type': 'application/json',
},
json={
'inputs': inputs,
'query': query,
'user': user,
'response_mode': response_mode,
'conversation_id': conversation_id,
'files': files,
},
json=payload,
) as r:
async for chunk in r.aiter_lines():
if r.status_code != 200:

View File

@@ -1,8 +1,11 @@
import asyncio
import base64
import json
import time
import urllib.parse
from typing import Callable
import dingtalk_stream # type: ignore
import websockets
from .EchoHandler import EchoTextHandler
from .dingtalkevent import DingTalkEvent
import httpx
@@ -36,6 +39,7 @@ class DingTalkClient:
self.access_token_expiry_time = ''
self.markdown_card = markdown_card
self.logger = logger
self._stopped = False # Flag to control the event loop
async def get_access_token(self):
url = 'https://api.dingtalk.com/v1.0/oauth2/accessToken'
@@ -170,6 +174,9 @@ class DingTalkClient:
"""
处理消息事件。
"""
# Skip message handling if stopped
if self._stopped:
return
msg_type = event.conversation
if msg_type in self._message_handlers:
for handler in self._message_handlers[msg_type]:
@@ -378,4 +385,70 @@ class DingTalkClient:
async def start(self):
"""启动 WebSocket 连接,监听消息"""
await self.client.start()
self._stopped = False
self.client.pre_start()
while not self._stopped:
try:
connection = self.client.open_connection()
if not connection:
if self.logger:
await self.logger.error('DingTalk: open connection failed')
await asyncio.sleep(10)
continue
uri = '%s?ticket=%s' % (connection['endpoint'], urllib.parse.quote_plus(connection['ticket']))
async with websockets.connect(uri) as websocket:
self.client.websocket = websocket
keepalive_task = asyncio.create_task(self._keepalive(websocket))
try:
async for raw_message in websocket:
if self._stopped:
break
json_message = json.loads(raw_message)
asyncio.create_task(self.client.background_task(json_message))
finally:
keepalive_task.cancel()
try:
await keepalive_task
except asyncio.CancelledError:
pass
except asyncio.CancelledError:
# Properly exit when task is cancelled
break
except websockets.exceptions.ConnectionClosedError as e:
if self._stopped:
break
if self.logger:
await self.logger.error(f'DingTalk: connection closed, reconnecting... error={e}')
await asyncio.sleep(5)
continue
except Exception as e:
if self._stopped:
break
if self.logger:
await self.logger.error(f'DingTalk: unknown exception, reconnecting... error={e}')
await asyncio.sleep(3)
continue
async def _keepalive(self, ws, ping_interval=60):
"""Keep WebSocket connection alive"""
while not self._stopped:
await asyncio.sleep(ping_interval)
try:
await ws.ping()
except websockets.exceptions.ConnectionClosed:
break
async def stop(self):
"""停止 WebSocket 连接"""
self._stopped = True
# Close WebSocket connection if exists
if self.client.websocket:
try:
await self.client.websocket.close()
except Exception:
pass
# Clear message handlers to prevent stale callbacks
self._message_handlers = {'example': []}

View File

@@ -23,20 +23,34 @@ xml_template = """
class OAClient:
def __init__(self, token: str, EncodingAESKey: str, AppID: str, Appsecret: str, logger: None):
def __init__(
self,
token: str,
EncodingAESKey: str,
AppID: str,
Appsecret: str,
logger: None,
unified_mode: bool = False,
api_base_url: str = 'https://api.weixin.qq.com',
):
self.token = token
self.aes = EncodingAESKey
self.appid = AppID
self.appsecret = Appsecret
self.base_url = 'https://api.weixin.qq.com'
self.base_url = api_base_url
self.access_token = ''
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
self._message_handlers = {
'example': [],
}
@@ -46,19 +60,39 @@ class OAClient:
self.logger = logger
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""处理回调请求的内部实现,包括 GET 验证和 POST 消息接收。
Args:
req: Quart Request 对象
"""
try:
# 每隔100毫秒查询是否生成ai回答
start_time = time.time()
signature = request.args.get('signature', '')
timestamp = request.args.get('timestamp', '')
nonce = request.args.get('nonce', '')
echostr = request.args.get('echostr', '')
msg_signature = request.args.get('msg_signature', '')
signature = req.args.get('signature', '')
timestamp = req.args.get('timestamp', '')
nonce = req.args.get('nonce', '')
echostr = req.args.get('echostr', '')
msg_signature = req.args.get('msg_signature', '')
if msg_signature is None:
await self.logger.error('msg_signature不在请求体中')
raise Exception('msg_signature不在请求体中')
if request.method == 'GET':
if req.method == 'GET':
# 校验签名
check_str = ''.join(sorted([self.token, timestamp, nonce]))
check_signature = hashlib.sha1(check_str.encode('utf-8')).hexdigest()
@@ -68,8 +102,8 @@ class OAClient:
else:
await self.logger.error('拒绝请求')
raise Exception('拒绝请求')
elif request.method == 'POST':
encryt_msg = await request.data
elif req.method == 'POST':
encryt_msg = await req.data
wxcpt = WXBizMsgCrypt(self.token, self.aes, self.appid)
ret, xml_msg = wxcpt.DecryptMsg(encryt_msg, msg_signature, timestamp, nonce)
xml_msg = xml_msg.decode('utf-8')
@@ -182,20 +216,27 @@ class OAClientForLongerResponse:
Appsecret: str,
LoadingMessage: str,
logger: None,
unified_mode: bool = False,
api_base_url: str = 'https://api.weixin.qq.com',
):
self.token = token
self.aes = EncodingAESKey
self.appid = AppID
self.appsecret = Appsecret
self.base_url = 'https://api.weixin.qq.com'
self.base_url = api_base_url
self.access_token = ''
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
self._message_handlers = {
'example': [],
}
@@ -206,24 +247,44 @@ class OAClientForLongerResponse:
self.logger = logger
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""处理回调请求的内部实现,包括 GET 验证和 POST 消息接收。
Args:
req: Quart Request 对象
"""
try:
signature = request.args.get('signature', '')
timestamp = request.args.get('timestamp', '')
nonce = request.args.get('nonce', '')
echostr = request.args.get('echostr', '')
msg_signature = request.args.get('msg_signature', '')
signature = req.args.get('signature', '')
timestamp = req.args.get('timestamp', '')
nonce = req.args.get('nonce', '')
echostr = req.args.get('echostr', '')
msg_signature = req.args.get('msg_signature', '')
if msg_signature is None:
await self.logger.error('msg_signature不在请求体中')
raise Exception('msg_signature不在请求体中')
if request.method == 'GET':
if req.method == 'GET':
check_str = ''.join(sorted([self.token, timestamp, nonce]))
check_signature = hashlib.sha1(check_str.encode('utf-8')).hexdigest()
return echostr if check_signature == signature else '拒绝请求'
elif request.method == 'POST':
encryt_msg = await request.data
elif req.method == 'POST':
encryt_msg = await req.data
wxcpt = WXBizMsgCrypt(self.token, self.aes, self.appid)
ret, xml_msg = wxcpt.DecryptMsg(encryt_msg, msg_signature, timestamp, nonce)
xml_msg = xml_msg.decode('utf-8')

View File

@@ -10,38 +10,20 @@ import traceback
from cryptography.hazmat.primitives.asymmetric import ed25519
def handle_validation(body: dict, bot_secret: str):
# bot正确的secert是32位的此处仅为了适配演示demo
while len(bot_secret) < 32:
bot_secret = bot_secret * 2
bot_secret = bot_secret[:32]
# 实际使用场景中以上三行内容可清除
seed_bytes = bot_secret.encode()
signing_key = ed25519.Ed25519PrivateKey.from_private_bytes(seed_bytes)
msg = body['d']['event_ts'] + body['d']['plain_token']
msg_bytes = msg.encode()
signature = signing_key.sign(msg_bytes)
signature_hex = signature.hex()
response = {'plain_token': body['d']['plain_token'], 'signature': signature_hex}
return response
class QQOfficialClient:
def __init__(self, secret: str, token: str, app_id: str, logger: None):
def __init__(self, secret: str, token: str, app_id: str, logger: None, unified_mode: bool = False):
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
self.secret = secret
self.token = token
self.app_id = app_id
@@ -82,18 +64,43 @@ class QQOfficialClient:
raise Exception(f'获取access_token失败: {e}')
async def handle_callback_request(self):
"""处理回调请求"""
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""处理回调请求的内部实现。
Args:
req: Quart Request 对象
"""
try:
# 读取请求数据
body = await request.get_data()
body = await req.get_data()
print(f'[QQ Official] Received request, body length: {len(body)}')
if not body or len(body) == 0:
print('[QQ Official] Received empty body, might be health check or GET request')
return {'code': 0, 'message': 'ok'}, 200
payload = json.loads(body)
# 验证是否为回调验证请求
if payload.get('op') == 13:
# 生成签名
response = handle_validation(payload, self.secret)
return response
validation_data = payload.get('d')
if not validation_data:
return {'error': "missing 'd' field"}, 400
response = await self.verify(validation_data)
return response, 200
if payload.get('op') == 0:
message_data = await self.get_message(payload)
@@ -104,6 +111,7 @@ class QQOfficialClient:
return {'code': 0, 'message': 'success'}
except Exception as e:
print(f'[QQ Official] ERROR: {traceback.format_exc()}')
await self.logger.error(f'Error in handle_callback_request: {traceback.format_exc()}')
return {'error': str(e)}, 400
@@ -261,3 +269,26 @@ class QQOfficialClient:
if self.access_token_expiry_time is None:
return True
return time.time() > self.access_token_expiry_time
async def repeat_seed(self, bot_secret: str, target_size: int = 32) -> bytes:
seed = bot_secret
while len(seed) < target_size:
seed *= 2
return seed[:target_size].encode('utf-8')
async def verify(self, validation_payload: dict):
seed = await self.repeat_seed(self.secret)
private_key = ed25519.Ed25519PrivateKey.from_private_bytes(seed)
event_ts = validation_payload.get('event_ts', '')
plain_token = validation_payload.get('plain_token', '')
msg = event_ts + plain_token
# sign
signature = private_key.sign(msg.encode()).hex()
response = {
'plain_token': plain_token,
'signature': signature,
}
return response

View File

@@ -8,14 +8,19 @@ import langbot_plugin.api.entities.builtin.platform.events as platform_events
class SlackClient:
def __init__(self, bot_token: str, signing_secret: str, logger: None):
def __init__(self, bot_token: str, signing_secret: str, logger: None, unified_mode: bool = False):
self.bot_token = bot_token
self.signing_secret = signing_secret
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.client = AsyncWebClient(self.bot_token)
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['GET', 'POST']
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['GET', 'POST']
)
self._message_handlers = {
'example': [],
}
@@ -23,8 +28,28 @@ class SlackClient:
self.logger = logger
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""处理回调请求的内部实现。
Args:
req: Quart Request 对象
"""
try:
body = await request.get_data()
body = await req.get_data()
data = json.loads(body)
if 'type' in data:
if data['type'] == 'url_verification':

View File

@@ -200,7 +200,7 @@ class StreamSessionManager:
class WecomBotClient:
def __init__(self, Token: str, EnCodingAESKey: str, Corpid: str, logger: EventLogger):
def __init__(self, Token: str, EnCodingAESKey: str, Corpid: str, logger: EventLogger, unified_mode: bool = False):
"""企业微信智能机器人客户端。
Args:
@@ -208,6 +208,7 @@ class WecomBotClient:
EnCodingAESKey: 企业微信消息加解密密钥。
Corpid: 企业 ID。
logger: 日志记录器。
unified_mode: 是否使用统一 webhook 模式(默认 False
Example:
>>> client = WecomBotClient(Token='token', EnCodingAESKey='aeskey', Corpid='corp', logger=logger)
@@ -217,10 +218,15 @@ class WecomBotClient:
self.EnCodingAESKey = EnCodingAESKey
self.Corpid = Corpid
self.ReceiveId = ''
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['POST', 'GET']
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['POST', 'GET']
)
self._message_handlers = {
'example': [],
}
@@ -359,7 +365,7 @@ class WecomBotClient:
return await self._encrypt_and_reply(payload, nonce)
async def handle_callback_request(self):
"""企业微信回调入口。
"""企业微信回调入口(独立端口模式,使用全局 request
Returns:
Quart Response: 根据请求类型返回验证、首包或刷新结果。
@@ -367,15 +373,33 @@ class WecomBotClient:
Example:
作为 Quart 路由处理函数直接注册并使用。
"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""处理回调请求的内部实现,包括 GET 验证和 POST 消息接收。
Args:
req: Quart Request 对象
"""
try:
self.wxcpt = WXBizMsgCrypt(self.Token, self.EnCodingAESKey, '')
await self.logger.info(f'{request.method} {request.url} {str(request.args)}')
if request.method == 'GET':
return await self._handle_get_callback()
if req.method == 'GET':
return await self._handle_get_callback(req)
if request.method == 'POST':
return await self._handle_post_callback()
if req.method == 'POST':
return await self._handle_post_callback(req)
return Response('', status=405)
@@ -383,13 +407,13 @@ class WecomBotClient:
await self.logger.error(traceback.format_exc())
return Response('Internal Server Error', status=500)
async def _handle_get_callback(self) -> tuple[Response, int] | Response:
async def _handle_get_callback(self, req) -> tuple[Response, int] | Response:
"""处理企业微信的 GET 验证请求。"""
msg_signature = unquote(request.args.get('msg_signature', ''))
timestamp = unquote(request.args.get('timestamp', ''))
nonce = unquote(request.args.get('nonce', ''))
echostr = unquote(request.args.get('echostr', ''))
msg_signature = unquote(req.args.get('msg_signature', ''))
timestamp = unquote(req.args.get('timestamp', ''))
nonce = unquote(req.args.get('nonce', ''))
echostr = unquote(req.args.get('echostr', ''))
if not all([msg_signature, timestamp, nonce, echostr]):
await self.logger.error('请求参数缺失')
@@ -402,16 +426,16 @@ class WecomBotClient:
return Response(decrypted_str, mimetype='text/plain')
async def _handle_post_callback(self) -> tuple[Response, int] | Response:
async def _handle_post_callback(self, req) -> tuple[Response, int] | Response:
"""处理企业微信的 POST 回调请求。"""
self.stream_sessions.cleanup()
msg_signature = unquote(request.args.get('msg_signature', ''))
timestamp = unquote(request.args.get('timestamp', ''))
nonce = unquote(request.args.get('nonce', ''))
msg_signature = unquote(req.args.get('msg_signature', ''))
timestamp = unquote(req.args.get('timestamp', ''))
nonce = unquote(req.args.get('nonce', ''))
encrypted_json = await request.get_json()
encrypted_json = await req.get_json()
encrypted_msg = (encrypted_json or {}).get('encrypt', '')
if not encrypted_msg:
await self.logger.error("请求体中缺少 'encrypt' 字段")
@@ -433,32 +457,174 @@ class WecomBotClient:
async def get_message(self, msg_json):
message_data = {}
msg_type = msg_json.get('msgtype', '')
if msg_type:
message_data['msgtype'] = msg_type
if msg_json.get('chattype', '') == 'single':
message_data['type'] = 'single'
elif msg_json.get('chattype', '') == 'group':
message_data['type'] = 'group'
if msg_json.get('msgtype') == 'text':
max_inline_file_size = 5 * 1024 * 1024 # avoid decoding very large payloads by default
async def _safe_download(url: str):
if not url:
return None
return await self.download_url_to_base64(url, self.EnCodingAESKey)
if msg_type == 'text':
message_data['content'] = msg_json.get('text', {}).get('content')
elif msg_json.get('msgtype') == 'image':
elif msg_type == 'markdown':
message_data['content'] = msg_json.get('markdown', {}).get('content') or msg_json.get('text', {}).get(
'content', ''
)
elif msg_type == 'image':
picurl = msg_json.get('image', {}).get('url', '')
base64 = await self.download_url_to_base64(picurl, self.EnCodingAESKey)
message_data['picurl'] = base64
elif msg_json.get('msgtype') == 'mixed':
base64_data = await _safe_download(picurl)
if base64_data:
message_data['picurl'] = base64_data
message_data['images'] = [base64_data]
elif msg_type == 'voice':
voice_info = msg_json.get('voice', {}) or {}
download_url = voice_info.get('url')
message_data['voice'] = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
# 企业微信智能转写文本(如果已有)直接复用,避免重复转写
if voice_info.get('content'):
message_data['content'] = voice_info.get('content')
if (message_data['voice'].get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download(download_url)
if voice_base64:
message_data['voice']['base64'] = voice_base64
elif msg_type == 'video':
video_info = msg_json.get('video', {}) or {}
download_url = video_info.get('url')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
message_data['video'] = video_data
elif msg_type == 'file':
file_info = msg_json.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
message_data['file'] = file_data
elif msg_type == 'link':
message_data['link'] = msg_json.get('link', {})
if not message_data.get('content'):
title = message_data['link'].get('title', '')
desc = message_data['link'].get('description') or message_data['link'].get('digest', '')
message_data['content'] = '\n'.join(filter(None, [title, desc]))
elif msg_type == 'mixed':
items = msg_json.get('mixed', {}).get('msg_item', [])
texts = []
picurl = None
images = []
files = []
voices = []
videos = []
links = []
for item in items:
if item.get('msgtype') == 'text':
item_type = item.get('msgtype')
if item_type == 'text':
texts.append(item.get('text', {}).get('content', ''))
elif item.get('msgtype') == 'image' and picurl is None:
picurl = item.get('image', {}).get('url')
elif item_type == 'image':
img_url = item.get('image', {}).get('url')
base64_data = await _safe_download(img_url)
if base64_data:
images.append(base64_data)
elif item_type == 'file':
file_info = item.get('file', {}) or {}
download_url = file_info.get('url') or file_info.get('fileurl')
file_data = {
'filename': file_info.get('filename') or file_info.get('name'),
'filesize': file_info.get('filesize') or file_info.get('size'),
'md5sum': file_info.get('md5sum') or file_info.get('md5'),
'sdkfileid': file_info.get('sdkfileid') or file_info.get('fileid'),
'download_url': download_url,
'extra': file_info,
}
if (file_data.get('filesize') or 0) <= max_inline_file_size:
file_base64 = await _safe_download(download_url)
if file_base64:
file_data['base64'] = file_base64
files.append(file_data)
elif item_type == 'voice':
voice_info = item.get('voice', {}) or {}
download_url = voice_info.get('url')
voice_data = {
'url': download_url,
'md5sum': voice_info.get('md5sum') or voice_info.get('md5'),
'filesize': voice_info.get('filesize') or voice_info.get('size'),
'sdkfileid': voice_info.get('sdkfileid') or voice_info.get('fileid'),
}
if voice_info.get('content'):
texts.append(voice_info.get('content'))
if (voice_data.get('filesize') or 0) <= max_inline_file_size:
voice_base64 = await _safe_download(download_url)
if voice_base64:
voice_data['base64'] = voice_base64
voices.append(voice_data)
elif item_type == 'video':
video_info = item.get('video', {}) or {}
download_url = video_info.get('url')
video_data = {
'url': download_url,
'filesize': video_info.get('filesize') or video_info.get('size'),
'sdkfileid': video_info.get('sdkfileid') or video_info.get('fileid'),
'md5sum': video_info.get('md5sum') or video_info.get('md5'),
'filename': video_info.get('filename') or video_info.get('name'),
}
if (video_data.get('filesize') or 0) <= max_inline_file_size:
video_base64 = await _safe_download(download_url)
if video_base64:
video_data['base64'] = video_base64
videos.append(video_data)
elif item_type == 'link':
links.append(item.get('link', {}))
if texts:
message_data['content'] = ''.join(texts) # 拼接所有 text
if picurl:
base64 = await self.download_url_to_base64(picurl, self.EnCodingAESKey)
message_data['picurl'] = base64 # 只保留第一个 image
message_data['content'] = ' '.join(texts) # 拼接所有 text
if images:
message_data['images'] = images
message_data['picurl'] = images[0] # 只保留第一个 image
if files:
message_data['files'] = files
message_data['file'] = files[0]
if voices:
message_data['voices'] = voices
message_data['voice'] = voices[0]
if videos:
message_data['videos'] = videos
message_data['video'] = videos[0]
if links:
message_data['link'] = links[0]
if items:
message_data['attachments'] = items
else:
message_data['raw_msg'] = msg_json
# Extract user information
from_info = msg_json.get('from', {})

View File

@@ -17,6 +17,13 @@ class WecomBotEvent(dict):
"""
return self.get('type', '')
@property
def msgtype(self) -> str:
"""
消息 msgtype
"""
return self.get('msgtype', '')
@property
def userid(self) -> str:
"""
@@ -57,6 +64,55 @@ class WecomBotEvent(dict):
"""
return self.get('picurl', '')
@property
def images(self):
"""
图片列表(兼容 mixed
"""
return self.get('images', [])
@property
def file(self):
"""
文件信息
"""
return self.get('file', {})
@property
def voice(self):
"""
语音信息
"""
return self.get('voice', {})
@property
def video(self):
"""
视频信息
"""
return self.get('video', {})
@property
def link(self):
"""
链接消息信息
"""
return self.get('link', {})
@property
def location(self):
"""
位置信息
"""
return self.get('location', {})
@property
def attachments(self):
"""
原始 mixed 中的附件项
"""
return self.get('attachments', [])
@property
def chatid(self) -> str:
"""

View File

@@ -21,23 +21,30 @@ class WecomClient:
EncodingAESKey: str,
contacts_secret: str,
logger: None,
unified_mode: bool = False,
api_base_url: str = 'https://qyapi.weixin.qq.com/cgi-bin',
):
self.corpid = corpid
self.secret = secret
self.access_token_for_contacts = ''
self.token = token
self.aes = EncodingAESKey
self.base_url = 'https://qyapi.weixin.qq.com/cgi-bin'
self.base_url = api_base_url
self.access_token = ''
self.secret_for_contacts = contacts_secret
self.logger = logger
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command',
'handle_callback',
self.handle_callback_request,
methods=['GET', 'POST'],
)
self._message_handlers = {
'example': [],
}
@@ -50,7 +57,7 @@ class WecomClient:
return bool(self.access_token_for_contacts and self.access_token_for_contacts.strip())
async def get_access_token(self, secret):
url = f'https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid={self.corpid}&corpsecret={secret}'
url = f'{self.base_url}/gettoken?corpid={self.corpid}&corpsecret={secret}'
async with httpx.AsyncClient() as client:
response = await client.get(url)
data = response.json()
@@ -109,14 +116,13 @@ class WecomClient:
async def send_image(self, user_id: str, agent_id: int, media_id: str):
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/media/upload?access_token=' + self.access_token
url = self.base_url + '/message/send?access_token=' + self.access_token
async with httpx.AsyncClient() as client:
params = {
'touser': user_id,
'toparty': '',
'totag': '',
'agentid': agent_id,
'msgtype': 'image',
'agentid': agent_id,
'image': {
'media_id': media_id,
},
@@ -125,27 +131,73 @@ class WecomClient:
'enable_duplicate_check': 0,
'duplicate_check_interval': 1800,
}
try:
response = await client.post(url, json=params)
data = response.json()
except Exception as e:
await self.logger.error(f'发送图片失败:{data}')
raise Exception('Failed to send image: ' + str(e))
# 企业微信错误码40014和42001代表accesstoken问题
response = await client.post(url, json=params)
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.send_image(user_id, agent_id, media_id)
if data['errcode'] != 0:
await self.logger.error(f'发送图片失败:{data}')
raise Exception('Failed to send image: ' + str(data))
async def send_voice(self, user_id: str, agent_id: int, media_id: str):
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/message/send?access_token=' + self.access_token
async with httpx.AsyncClient() as client:
params = {
'touser': user_id,
'msgtype': 'voice',
'agentid': agent_id,
'voice': {
'media_id': media_id,
},
'safe': 0,
'enable_id_trans': 0,
'enable_duplicate_check': 0,
'duplicate_check_interval': 1800,
}
response = await client.post(url, json=params)
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.send_voice(user_id, agent_id, media_id)
if data['errcode'] != 0:
await self.logger.error(f'发送语音失败:{data}')
raise Exception('Failed to send voice: ' + str(data))
async def send_file(self, user_id: str, agent_id: int, media_id: str):
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/message/send?access_token=' + self.access_token
async with httpx.AsyncClient() as client:
params = {
'touser': user_id,
'msgtype': 'file',
'agentid': agent_id,
'file': {
'media_id': media_id,
},
'safe': 0,
'enable_id_trans': 0,
'enable_duplicate_check': 0,
'duplicate_check_interval': 1800,
}
response = await client.post(url, json=params)
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
return await self.send_file(user_id, agent_id, media_id)
if data['errcode'] != 0:
await self.logger.error(f'发送文件失败:{data}')
raise Exception('Failed to send file: ' + str(data))
async def send_private_msg(self, user_id: str, agent_id: int, content: str):
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/message/send?access_token=' + self.access_token
async with httpx.AsyncClient() as client:
async with httpx.AsyncClient(timeout=None) as client:
params = {
'touser': user_id,
'msgtype': 'text',
@@ -168,25 +220,43 @@ class WecomClient:
raise Exception('Failed to send message: ' + str(data))
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
处理回调请求,包括 GET 验证和 POST 消息接收。
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""
处理回调请求的内部实现,包括 GET 验证和 POST 消息接收。
Args:
req: Quart Request 对象
"""
try:
msg_signature = request.args.get('msg_signature')
timestamp = request.args.get('timestamp')
nonce = request.args.get('nonce')
msg_signature = req.args.get('msg_signature')
timestamp = req.args.get('timestamp')
nonce = req.args.get('nonce')
wxcpt = WXBizMsgCrypt(self.token, self.aes, self.corpid)
if request.method == 'GET':
echostr = request.args.get('echostr')
if req.method == 'GET':
echostr = req.args.get('echostr')
ret, reply_echo_str = wxcpt.VerifyURL(msg_signature, timestamp, nonce, echostr)
if ret != 0:
await self.logger.error('验证失败')
raise Exception(f'验证失败,错误码: {ret}')
return reply_echo_str
elif request.method == 'POST':
encrypt_msg = await request.data
elif req.method == 'POST':
encrypt_msg = await req.data
ret, xml_msg = wxcpt.DecryptMsg(encrypt_msg, msg_signature, timestamp, nonce)
if ret != 0:
await self.logger.error('消息解密失败')
@@ -270,7 +340,7 @@ class WecomClient:
return ext
return 'jpg' # 默认返回jpg
async def upload_to_work(self, image: platform_message.Image):
async def upload_image_to_work(self, image: platform_message.Image):
"""
获取 media_id
"""
@@ -287,7 +357,7 @@ class WecomClient:
file_bytes = await f.read()
file_name = image.path.split('/')[-1]
elif image.url:
file_bytes = await self.download_image_to_bytes(image.url)
file_bytes = await self.download_media_to_bytes(image.url)
file_name = image.url.split('/')[-1]
elif image.base64:
try:
@@ -322,7 +392,7 @@ class WecomClient:
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
media_id = await self.upload_to_work(image)
media_id = await self.upload_image_to_work(image)
if data.get('errcode', 0) != 0:
await self.logger.error(f'上传图片失败:{data}')
raise Exception('failed to upload file')
@@ -330,13 +400,128 @@ class WecomClient:
media_id = data.get('media_id')
return media_id
async def download_image_to_bytes(self, url: str) -> bytes:
async def upload_voice_to_work(self, voice: platform_message.Voice):
"""
上传语音文件到企业微信
"""
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/media/upload?access_token=' + self.access_token + '&type=file'
file_bytes = None
file_name = 'voice.mp3'
if voice.path:
async with aiofiles.open(voice.path, 'rb') as f:
file_bytes = await f.read()
file_name = voice.path.split('/')[-1]
elif voice.url:
file_bytes = await self.download_media_to_bytes(voice.url)
file_name = voice.url.split('/')[-1]
elif voice.base64:
try:
base64_data = voice.base64
if ',' in base64_data:
base64_data = base64_data.split(',', 1)[1]
padding = 4 - (len(base64_data) % 4) if len(base64_data) % 4 else 0
padded_base64 = base64_data + '=' * padding
file_bytes = base64.b64decode(padded_base64)
except binascii.Error as e:
raise ValueError(f'Invalid base64 string: {str(e)}')
else:
await self.logger.error('Voice对象出错')
raise ValueError('voice对象出错')
boundary = '-------------------------acebdf13572468'
headers = {'Content-Type': f'multipart/form-data; boundary={boundary}'}
body = (
(
f'--{boundary}\r\n'
f'Content-Disposition: form-data; name="media"; filename="{file_name}"; filelength={len(file_bytes)}\r\n'
f'Content-Type: application/octet-stream\r\n\r\n'
).encode('utf-8')
+ file_bytes
+ f'\r\n--{boundary}--\r\n'.encode('utf-8')
)
# print(body)
async with httpx.AsyncClient() as client:
response = await client.post(url, headers=headers, content=body)
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
media_id = await self.upload_voice_to_work(voice)
if data.get('errcode', 0) != 0:
await self.logger.error(f'上传语音文件失败:{data}')
raise Exception('failed to upload file')
media_id = data.get('media_id')
return media_id
async def upload_file_to_work(self, file: platform_message.File):
"""
上传文件到企业微信
"""
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = self.base_url + '/media/upload?access_token=' + self.access_token + '&type=file'
file_bytes = None
file_name = 'file.txt'
if file.path:
async with aiofiles.open(file.path, 'rb') as f:
file_bytes = await f.read()
file_name = file.path.split('/')[-1]
elif file.url:
file_bytes = await self.download_media_to_bytes(file.url)
file_name = file.url.split('/')[-1]
elif file.base64:
try:
base64_data = file.base64
if ',' in base64_data:
base64_data = base64_data.split(',', 1)[1]
padding = 4 - (len(base64_data) % 4) if len(base64_data) % 4 else 0
padded_base64 = base64_data + '=' * padding
file_bytes = base64.b64decode(padded_base64)
except binascii.Error as e:
raise ValueError(f'Invalid base64 string: {str(e)}')
else:
await self.logger.error('File对象出错')
raise ValueError('file对象出错')
boundary = '-------------------------acebdf13572468'
headers = {'Content-Type': f'multipart/form-data; boundary={boundary}'}
body = (
(
f'--{boundary}\r\n'
f'Content-Disposition: form-data; name="media"; filename="{file_name}"; filelength={len(file_bytes)}\r\n'
f'Content-Type: application/octet-stream\r\n\r\n'
).encode('utf-8')
+ file_bytes
+ f'\r\n--{boundary}--\r\n'.encode('utf-8')
)
async with httpx.AsyncClient() as client:
response = await client.post(url, headers=headers, content=body)
data = response.json()
if data['errcode'] == 40014 or data['errcode'] == 42001:
self.access_token = await self.get_access_token(self.secret)
media_id = await self.upload_file_to_work(file)
if data.get('errcode', 0) != 0:
await self.logger.error(f'上传文件失败:{data}')
raise Exception('failed to upload file')
media_id = data.get('media_id')
return media_id
async def download_media_to_bytes(self, url: str) -> bytes:
async with httpx.AsyncClient() as client:
response = await client.get(url)
response.raise_for_status()
return response.content
# 进行media_id的获取
async def get_media_id(self, image: platform_message.Image):
media_id = await self.upload_to_work(image=image)
async def get_media_id(self, media: platform_message.Image | platform_message.Voice | platform_message.File):
if isinstance(media, platform_message.Image):
media_id = await self.upload_image_to_work(image=media)
elif isinstance(media, platform_message.Voice):
media_id = await self.upload_voice_to_work(voice=media)
elif isinstance(media, platform_message.File):
media_id = await self.upload_file_to_work(file=media)
else:
raise ValueError('Unsupported media type')
return media_id

View File

@@ -13,19 +13,33 @@ import aiofiles
class WecomCSClient:
def __init__(self, corpid: str, secret: str, token: str, EncodingAESKey: str, logger: None):
def __init__(
self,
corpid: str,
secret: str,
token: str,
EncodingAESKey: str,
logger: None,
unified_mode: bool = False,
api_base_url: str = 'https://qyapi.weixin.qq.com/cgi-bin',
):
self.corpid = corpid
self.secret = secret
self.access_token_for_contacts = ''
self.token = token
self.aes = EncodingAESKey
self.base_url = 'https://qyapi.weixin.qq.com/cgi-bin'
self.base_url = api_base_url
self.access_token = ''
self.logger = logger
self.unified_mode = unified_mode
self.app = Quart(__name__)
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['GET', 'POST']
)
# 只有在非统一模式下才注册独立路由
if not self.unified_mode:
self.app.add_url_rule(
'/callback/command', 'handle_callback', self.handle_callback_request, methods=['GET', 'POST']
)
self._message_handlers = {
'example': [],
}
@@ -61,7 +75,7 @@ class WecomCSClient:
return bool(self.access_token_for_contacts and self.access_token_for_contacts.strip())
async def get_access_token(self, secret):
url = f'https://qyapi.weixin.qq.com/cgi-bin/gettoken?corpid={self.corpid}&corpsecret={secret}'
url = f'{self.base_url}/gettoken?corpid={self.corpid}&corpsecret={secret}'
async with httpx.AsyncClient() as client:
response = await client.get(url)
data = response.json()
@@ -167,7 +181,7 @@ class WecomCSClient:
if not await self.check_access_token():
self.access_token = await self.get_access_token(self.secret)
url = f'https://qyapi.weixin.qq.com/cgi-bin/kf/send_msg?access_token={self.access_token}'
url = f'{self.base_url}/kf/send_msg?access_token={self.access_token}'
payload = {
'touser': external_userid,
@@ -192,27 +206,45 @@ class WecomCSClient:
return data
async def handle_callback_request(self):
"""处理回调请求(独立端口模式,使用全局 request"""
return await self._handle_callback_internal(request)
async def handle_unified_webhook(self, req):
"""处理回调请求(统一 webhook 模式,显式传递 request
Args:
req: Quart Request 对象
Returns:
响应数据
"""
处理回调请求,包括 GET 验证和 POST 消息接收。
return await self._handle_callback_internal(req)
async def _handle_callback_internal(self, req):
"""
处理回调请求的内部实现,包括 GET 验证和 POST 消息接收。
Args:
req: Quart Request 对象
"""
try:
msg_signature = request.args.get('msg_signature')
timestamp = request.args.get('timestamp')
nonce = request.args.get('nonce')
msg_signature = req.args.get('msg_signature')
timestamp = req.args.get('timestamp')
nonce = req.args.get('nonce')
try:
wxcpt = WXBizMsgCrypt(self.token, self.aes, self.corpid)
except Exception as e:
raise Exception(f'初始化失败,错误码: {e}')
if request.method == 'GET':
echostr = request.args.get('echostr')
if req.method == 'GET':
echostr = req.args.get('echostr')
ret, reply_echo_str = wxcpt.VerifyURL(msg_signature, timestamp, nonce, echostr)
if ret != 0:
raise Exception(f'验证失败,错误码: {ret}')
return reply_echo_str
elif request.method == 'POST':
encrypt_msg = await request.data
elif req.method == 'POST':
encrypt_msg = await req.data
ret, xml_msg = wxcpt.DecryptMsg(encrypt_msg, msg_signature, timestamp, nonce)
if ret != 0:
raise Exception(f'消息解密失败,错误码: {ret}')

View File

@@ -28,8 +28,56 @@ class FilesRouterGroup(group.RouterGroup):
return quart.Response(image_bytes, mimetype=mime_type)
@self.route('/documents', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> quart.Response:
@self.route('/images', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def upload_image() -> quart.Response:
request = quart.request
# Check file size limit before reading the file
content_length = request.content_length
if content_length and content_length > group.MAX_FILE_SIZE:
return self.fail(400, 'Image size exceeds 10MB limit.')
# get file bytes from 'file'
files = await request.files
if 'file' not in files:
return self.fail(400, 'No image file provided')
file = files['file']
assert isinstance(file, quart.datastructures.FileStorage)
file_bytes = await asyncio.to_thread(file.stream.read)
# Double-check actual file size after reading
if len(file_bytes) > group.MAX_FILE_SIZE:
return self.fail(400, 'Image size exceeds 10MB limit.')
# Validate image file extension
allowed_extensions = {'jpg', 'jpeg', 'png', 'gif', 'webp'}
if '.' in file.filename:
file_name, extension = file.filename.rsplit('.', 1)
extension = extension.lower()
else:
return self.fail(400, 'Invalid image file: no file extension')
if extension not in allowed_extensions:
return self.fail(400, f'Invalid image format. Allowed formats: {", ".join(allowed_extensions)}')
# check if file name contains '/' or '\'
if '/' in file_name or '\\' in file_name:
return self.fail(400, 'File name contains invalid characters')
file_key = file_name + '_' + str(uuid.uuid4())[:8] + '.' + extension
# save file to storage
await self.ap.storage_mgr.storage_provider.save(file_key, file_bytes)
return self.success(
data={
'file_key': file_key,
}
)
@self.route('/documents', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def upload_document() -> quart.Response:
request = quart.request
# Check file size limit before reading the file

View File

@@ -5,7 +5,7 @@ from ... import group
@group.group_class('knowledge_base', '/api/v1/knowledge/bases')
class KnowledgeBaseRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['POST', 'GET'])
@self.route('', methods=['POST', 'GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def handle_knowledge_bases() -> quart.Response:
if quart.request.method == 'GET':
knowledge_bases = await self.ap.knowledge_service.get_knowledge_bases()
@@ -21,6 +21,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
@self.route(
'/<knowledge_base_uuid>',
methods=['GET', 'DELETE', 'PUT'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def handle_specific_knowledge_base(knowledge_base_uuid: str) -> quart.Response:
if quart.request.method == 'GET':
@@ -47,6 +48,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
@self.route(
'/<knowledge_base_uuid>/files',
methods=['GET', 'POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def get_knowledge_base_files(knowledge_base_uuid: str) -> str:
if quart.request.method == 'GET':
@@ -74,6 +76,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
@self.route(
'/<knowledge_base_uuid>/files/<file_id>',
methods=['DELETE'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def delete_specific_file_in_kb(file_id: str, knowledge_base_uuid: str) -> str:
await self.ap.knowledge_service.delete_file(knowledge_base_uuid, file_id)
@@ -82,6 +85,7 @@ class KnowledgeBaseRouterGroup(group.RouterGroup):
@self.route(
'/<knowledge_base_uuid>/retrieve',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def retrieve_knowledge_base(knowledge_base_uuid: str) -> str:
json_data = await quart.request.json

View File

@@ -0,0 +1,61 @@
import quart
from ... import group
@group.group_class('external_knowledge_base', '/api/v1/knowledge/external-bases')
class ExternalKnowledgeBaseRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/retrievers', methods=['GET'])
async def list_knowledge_retrievers() -> quart.Response:
"""List all available knowledge retrievers from plugins."""
retrievers = await self.ap.plugin_connector.list_knowledge_retrievers()
return self.success(data={'retrievers': retrievers})
@self.route('', methods=['POST', 'GET'])
async def handle_external_knowledge_bases() -> quart.Response:
if quart.request.method == 'GET':
external_kbs = await self.ap.external_kb_service.get_external_knowledge_bases()
return self.success(data={'bases': external_kbs})
elif quart.request.method == 'POST':
json_data = await quart.request.json
kb_uuid = await self.ap.external_kb_service.create_external_knowledge_base(json_data)
return self.success(data={'uuid': kb_uuid})
return self.http_status(405, -1, 'Method not allowed')
@self.route(
'/<kb_uuid>',
methods=['GET', 'DELETE', 'PUT'],
)
async def handle_specific_external_knowledge_base(kb_uuid: str) -> quart.Response:
if quart.request.method == 'GET':
external_kb = await self.ap.external_kb_service.get_external_knowledge_base(kb_uuid)
if external_kb is None:
return self.http_status(404, -1, 'external knowledge base not found')
return self.success(
data={
'base': external_kb,
}
)
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.external_kb_service.update_external_knowledge_base(kb_uuid, json_data)
return self.success({})
elif quart.request.method == 'DELETE':
await self.ap.external_kb_service.delete_external_knowledge_base(kb_uuid)
return self.success({})
@self.route(
'/<kb_uuid>/retrieve',
methods=['POST'],
)
async def retrieve_external_knowledge_base(kb_uuid: str) -> str:
json_data = await quart.request.json
query = json_data.get('query')
results = await self.ap.external_kb_service.retrieve_external_knowledge_base(kb_uuid, query)
return self.success(data={'results': results})

View File

@@ -0,0 +1,325 @@
from __future__ import annotations
import datetime
import quart
from .. import group
def parse_iso_datetime(datetime_str: str | None) -> datetime.datetime | None:
"""Parse ISO 8601 datetime string, handling 'Z' suffix for UTC timezone"""
if not datetime_str:
return None
# Replace 'Z' with '+00:00' for Python 3.10 compatibility
if datetime_str.endswith('Z'):
datetime_str = datetime_str[:-1] + '+00:00'
dt = datetime.datetime.fromisoformat(datetime_str)
# Convert to UTC and remove timezone info to match database storage (which stores UTC as naive datetime)
if dt.tzinfo is not None:
# Convert to UTC and remove timezone info
dt = dt.astimezone(datetime.timezone.utc).replace(tzinfo=None)
return dt
@group.group_class('monitoring', '/api/v1/monitoring')
class MonitoringRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/overview', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_overview() -> str:
"""Get overview metrics"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
metrics = await self.ap.monitoring_service.get_overview_metrics(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
)
return self.success(data=metrics)
@self.route('/messages', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_messages() -> str:
"""Get message logs"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
messages, total = await self.ap.monitoring_service.get_messages(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'messages': messages,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/llm-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_llm_calls() -> str:
"""Get LLM call records"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
llm_calls, total = await self.ap.monitoring_service.get_llm_calls(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'llm_calls': llm_calls,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/embedding-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_embedding_calls() -> str:
"""Get embedding call records"""
# Parse query parameters
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
knowledge_base_id = quart.request.args.get('knowledgeBaseId')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
embedding_calls, total = await self.ap.monitoring_service.get_embedding_calls(
start_time=start_time,
end_time=end_time,
knowledge_base_id=knowledge_base_id if knowledge_base_id else None,
limit=limit,
offset=offset,
)
return self.success(
data={
'embedding_calls': embedding_calls,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/sessions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_sessions() -> str:
"""Get session information"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
is_active_str = quart.request.args.get('isActive')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
# Parse is_active
is_active = None
if is_active_str:
is_active = is_active_str.lower() == 'true'
sessions, total = await self.ap.monitoring_service.get_sessions(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
is_active=is_active,
limit=limit,
offset=offset,
)
return self.success(
data={
'sessions': sessions,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/errors', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_errors() -> str:
"""Get error logs"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 100))
offset = int(quart.request.args.get('offset', 0))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
errors, total = await self.ap.monitoring_service.get_errors(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=offset,
)
return self.success(
data={
'errors': errors,
'total': total,
'limit': limit,
'offset': offset,
}
)
@self.route('/data', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_all_data() -> str:
"""Get all monitoring data in a single request"""
# Parse query parameters
bot_ids = quart.request.args.getlist('botId')
pipeline_ids = quart.request.args.getlist('pipelineId')
start_time_str = quart.request.args.get('startTime')
end_time_str = quart.request.args.get('endTime')
limit = int(quart.request.args.get('limit', 50))
# Parse datetime
start_time = parse_iso_datetime(start_time_str)
end_time = parse_iso_datetime(end_time_str)
# Get overview metrics
overview = await self.ap.monitoring_service.get_overview_metrics(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
)
# Get messages
messages, messages_total = await self.ap.monitoring_service.get_messages(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=0,
)
# Get LLM calls
llm_calls, llm_calls_total = await self.ap.monitoring_service.get_llm_calls(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=0,
)
# Get sessions
sessions, sessions_total = await self.ap.monitoring_service.get_sessions(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
is_active=None,
limit=limit,
offset=0,
)
# Get errors
errors, errors_total = await self.ap.monitoring_service.get_errors(
bot_ids=bot_ids if bot_ids else None,
pipeline_ids=pipeline_ids if pipeline_ids else None,
start_time=start_time,
end_time=end_time,
limit=limit,
offset=0,
)
# Get embedding calls
embedding_calls, embedding_calls_total = await self.ap.monitoring_service.get_embedding_calls(
start_time=start_time,
end_time=end_time,
limit=limit,
offset=0,
)
return self.success(
data={
'overview': overview,
'messages': messages,
'llmCalls': llm_calls,
'embeddingCalls': embedding_calls,
'sessions': sessions,
'errors': errors,
'totalCount': {
'messages': messages_total,
'llmCalls': llm_calls_total,
'embeddingCalls': embedding_calls_total,
'sessions': sessions_total,
'errors': errors_total,
},
}
)
@self.route('/sessions/<session_id>/analysis', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_session_analysis(session_id: str) -> str:
"""Get detailed analysis for a specific session"""
analysis = await self.ap.monitoring_service.get_session_analysis(session_id)
# Always return success with the analysis data
# The frontend will handle the 'found: false' case
return self.success(data=analysis)
@self.route('/messages/<message_id>/details', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def get_message_details(message_id: str) -> str:
"""Get detailed information for a specific message"""
details = await self.ap.monitoring_service.get_message_details(message_id)
if not details.get('found'):
return self.error(message=f'Message {message_id} not found', code=404)
return self.success(data=details)

View File

@@ -49,6 +49,14 @@ class PipelinesRouterGroup(group.RouterGroup):
return self.success()
@self.route('/<pipeline_uuid>/copy', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(pipeline_uuid: str) -> str:
try:
new_uuid = await self.ap.pipeline_service.copy_pipeline(pipeline_uuid)
return self.success(data={'uuid': new_uuid})
except ValueError as e:
return self.http_status(404, -1, str(e))
@self.route(
'/<pipeline_uuid>/extensions', methods=['GET', 'PUT'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
@@ -59,25 +67,33 @@ class PipelinesRouterGroup(group.RouterGroup):
if pipeline is None:
return self.http_status(404, -1, 'pipeline not found')
plugins = await self.ap.plugin_connector.list_plugins()
# Only include plugins with pipeline-related components (Command, EventListener, Tool)
# Plugins that only have KnowledgeRetriever components are not suitable for pipeline extensions
pipeline_component_kinds = ['Command', 'EventListener', 'Tool']
plugins = await self.ap.plugin_connector.list_plugins(component_kinds=pipeline_component_kinds)
mcp_servers = await self.ap.mcp_service.get_mcp_servers(contain_runtime_info=True)
extensions_prefs = pipeline.get('extensions_preferences', {})
return self.success(
data={
'bound_plugins': pipeline.get('extensions_preferences', {}).get('plugins', []),
'enable_all_plugins': extensions_prefs.get('enable_all_plugins', True),
'enable_all_mcp_servers': extensions_prefs.get('enable_all_mcp_servers', True),
'bound_plugins': extensions_prefs.get('plugins', []),
'available_plugins': plugins,
'bound_mcp_servers': pipeline.get('extensions_preferences', {}).get('mcp_servers', []),
'bound_mcp_servers': extensions_prefs.get('mcp_servers', []),
'available_mcp_servers': mcp_servers,
}
)
elif quart.request.method == 'PUT':
# Update bound plugins and MCP servers for this pipeline
json_data = await quart.request.json
enable_all_plugins = json_data.get('enable_all_plugins', True)
enable_all_mcp_servers = json_data.get('enable_all_mcp_servers', True)
bound_plugins = json_data.get('bound_plugins', [])
bound_mcp_servers = json_data.get('bound_mcp_servers', [])
await self.ap.pipeline_service.update_pipeline_extensions(
pipeline_uuid, bound_plugins, bound_mcp_servers
pipeline_uuid, bound_plugins, bound_mcp_servers, enable_all_plugins, enable_all_mcp_servers
)
return self.success()

View File

@@ -1,109 +0,0 @@
import json
import quart
from ... import group
@group.group_class('webchat', '/api/v1/pipelines/<pipeline_uuid>/chat')
class WebChatDebugRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/send', methods=['POST'])
async def send_message(pipeline_uuid: str) -> str:
"""Send a message to the pipeline for debugging"""
async def stream_generator(generator):
yield 'data: {"type": "start"}\n\n'
async for message in generator:
yield f'data: {json.dumps({"message": message})}\n\n'
yield 'data: {"type": "end"}\n\n'
try:
data = await quart.request.get_json()
session_type = data.get('session_type', 'person')
message_chain_obj = data.get('message', [])
is_stream = data.get('is_stream', False)
if not message_chain_obj:
return self.http_status(400, -1, 'message is required')
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
webchat_adapter = self.ap.platform_mgr.webchat_proxy_bot.adapter
if not webchat_adapter:
return self.http_status(404, -1, 'WebChat adapter not found')
if is_stream:
generator = webchat_adapter.send_webchat_message(
pipeline_uuid, session_type, message_chain_obj, is_stream
)
# 设置正确的响应头
headers = {
'Content-Type': 'text/event-stream',
'Transfer-Encoding': 'chunked',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
}
return quart.Response(stream_generator(generator), mimetype='text/event-stream', headers=headers)
else: # non-stream
result = None
async for message in webchat_adapter.send_webchat_message(
pipeline_uuid, session_type, message_chain_obj
):
result = message
if result is not None:
return self.success(
data={
'message': result,
}
)
else:
return self.http_status(400, -1, 'message is required')
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/messages/<session_type>', methods=['GET'])
async def get_messages(pipeline_uuid: str, session_type: str) -> str:
"""Get the message history of the pipeline for debugging"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
webchat_adapter = self.ap.platform_mgr.webchat_proxy_bot.adapter
if not webchat_adapter:
return self.http_status(404, -1, 'WebChat adapter not found')
messages = webchat_adapter.get_webchat_messages(pipeline_uuid, session_type)
return self.success(data={'messages': messages})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/reset/<session_type>', methods=['POST'])
async def reset_session(session_type: str) -> str:
"""Reset the debug session"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
webchat_adapter = None
for bot in self.ap.platform_mgr.bots:
if hasattr(bot.adapter, '__class__') and bot.adapter.__class__.__name__ == 'WebChatAdapter':
webchat_adapter = bot.adapter
break
if not webchat_adapter:
return self.http_status(404, -1, 'WebChat adapter not found')
webchat_adapter.reset_debug_session(session_type)
return self.success(data={'message': 'Session reset successfully'})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')

View File

@@ -0,0 +1,243 @@
"""WebSocket聊天路由 - 支持双向实时通信"""
import asyncio
import datetime
import json
import logging
import quart
from ... import group
from ......platform.sources.websocket_manager import ws_connection_manager
logger = logging.getLogger(__name__)
@group.group_class('websocket_chat', '/api/v1/pipelines/<pipeline_uuid>/ws')
class WebSocketChatRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
# 直接使用 quart_app 注册 WebSocket 路由
@self.quart_app.websocket(self.path + '/connect')
async def websocket_connect(pipeline_uuid: str):
"""
建立WebSocket连接
URL参数:
- pipeline_uuid: 流水线UUID
- session_type: 会话类型 (person/group)
"""
try:
# 获取参数 - 在WebSocket上下文中使用 quart.websocket.args
session_type = quart.websocket.args.get('session_type', 'person')
if session_type not in ['person', 'group']:
await quart.websocket.send(
json.dumps({'type': 'error', 'message': 'session_type must be person or group'})
)
return
# 获取WebSocket适配器
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
await quart.websocket.send(json.dumps({'type': 'error', 'message': 'WebSocket adapter not found'}))
return
# 注册连接
connection = await ws_connection_manager.add_connection(
websocket=quart.websocket._get_current_object(),
pipeline_uuid=pipeline_uuid,
session_type=session_type,
metadata={'user_agent': quart.websocket.headers.get('User-Agent', '')},
)
# 发送连接成功消息
await quart.websocket.send(
json.dumps(
{
'type': 'connected',
'connection_id': connection.connection_id,
'pipeline_uuid': pipeline_uuid,
'session_type': session_type,
'timestamp': connection.created_at.isoformat(),
}
)
)
logger.debug(
f'WebSocket connection established: {connection.connection_id} '
f'(pipeline={pipeline_uuid}, session_type={session_type})'
)
# 创建接收和发送任务
receive_task = asyncio.create_task(self._handle_receive(connection, websocket_adapter))
send_task = asyncio.create_task(self._handle_send(connection))
# 等待任务完成
try:
await asyncio.gather(receive_task, send_task)
except Exception as e:
logger.error(f'WebSocket task execution error: {e}')
finally:
# 清理连接
await ws_connection_manager.remove_connection(connection.connection_id)
logger.debug(f'WebSocket connection cleaned: {connection.connection_id}')
except Exception as e:
logger.error(f'WebSocket connection error: {e}', exc_info=True)
try:
await quart.websocket.send(json.dumps({'type': 'error', 'message': str(e)}))
except:
pass
@self.route('/messages/<session_type>', methods=['GET'])
async def get_messages(pipeline_uuid: str, session_type: str) -> str:
"""获取消息历史"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
messages = websocket_adapter.get_websocket_messages(pipeline_uuid, session_type)
return self.success(data={'messages': messages})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/reset/<session_type>', methods=['POST'])
async def reset_session(pipeline_uuid: str, session_type: str) -> str:
"""重置会话"""
try:
if session_type not in ['person', 'group']:
return self.http_status(400, -1, 'session_type must be person or group')
websocket_adapter = self.ap.platform_mgr.websocket_proxy_bot.adapter
if not websocket_adapter:
return self.http_status(404, -1, 'WebSocket adapter not found')
websocket_adapter.reset_session(pipeline_uuid, session_type)
return self.success(data={'message': 'Session reset successfully'})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/connections', methods=['GET'])
async def get_connections(pipeline_uuid: str) -> str:
"""获取当前连接统计"""
try:
stats = ws_connection_manager.get_stats()
connections = await ws_connection_manager.get_connections_by_pipeline(pipeline_uuid)
return self.success(
data={
'stats': stats,
'connections': [
{
'connection_id': conn.connection_id,
'session_type': conn.session_type,
'created_at': conn.created_at.isoformat(),
'last_active': conn.last_active.isoformat(),
'is_active': conn.is_active,
}
for conn in connections
],
}
)
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
@self.route('/broadcast', methods=['POST'])
async def broadcast_message(pipeline_uuid: str) -> str:
"""向所有连接广播消息(后端主动推送)"""
try:
data = await quart.request.get_json()
message = data.get('message')
if not message:
return self.http_status(400, -1, 'message is required')
# 广播消息
broadcast_data = {
'type': 'broadcast',
'message': message,
'timestamp': datetime.datetime.now().isoformat(),
}
await ws_connection_manager.broadcast_to_pipeline(pipeline_uuid, broadcast_data)
return self.success(data={'message': 'Broadcast sent successfully'})
except Exception as e:
return self.http_status(500, -1, f'Internal server error: {str(e)}')
async def _handle_receive(self, connection, websocket_adapter):
"""处理接收消息的任务"""
try:
while connection.is_active:
# 接收消息
message = await quart.websocket.receive()
# 更新活跃时间
await ws_connection_manager.update_activity(connection.connection_id)
try:
data = json.loads(message)
message_type = data.get('type', 'message')
if message_type == 'ping':
# 心跳响应
await connection.send_queue.put(
{'type': 'pong', 'timestamp': datetime.datetime.now().isoformat()}
)
elif message_type == 'message':
# 处理用户消息
logger.debug(f'收到消息: {data} from {connection.connection_id}')
# 处理消息不等待响应响应会通过broadcast异步发送
await websocket_adapter.handle_websocket_message(connection, data)
elif message_type == 'disconnect':
# 客户端主动断开
logger.debug(f'Client disconnected: {connection.connection_id}')
break
else:
logger.warning(f'Unknown message type: {message_type}')
except json.JSONDecodeError:
logger.error(f'Invalid JSON message: {message}')
await connection.send_queue.put({'type': 'error', 'message': 'Invalid JSON format'})
except Exception as e:
logger.error(f'Receive message error: {e}', exc_info=True)
finally:
connection.is_active = False
async def _handle_send(self, connection):
"""处理发送消息的任务"""
try:
while connection.is_active:
# 从队列获取消息
try:
message = await asyncio.wait_for(connection.send_queue.get(), timeout=1.0)
# 发送消息
await quart.websocket.send(json.dumps(message))
except asyncio.TimeoutError:
# 超时继续循环
continue
except Exception as e:
logger.error(f'Send message error: {e}', exc_info=True)
finally:
connection.is_active = False

View File

@@ -18,7 +18,8 @@ class BotsRouterGroup(group.RouterGroup):
@self.route('/<bot_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _(bot_uuid: str) -> str:
if quart.request.method == 'GET':
bot = await self.ap.bot_service.get_bot(bot_uuid)
# 返回运行时信息包括webhook地址等
bot = await self.ap.bot_service.get_runtime_bot_info(bot_uuid)
if bot is None:
return self.http_status(404, -1, 'bot not found')
return self.success(data={'bot': bot})
@@ -42,3 +43,32 @@ class BotsRouterGroup(group.RouterGroup):
'total_count': total_count,
}
)
@self.route('/<bot_uuid>/send_message', methods=['POST'], auth_type=group.AuthType.API_KEY)
async def _(bot_uuid: str) -> str:
"""Send message to a specific target via bot"""
json_data = await quart.request.json
target_type = json_data.get('target_type')
target_id = json_data.get('target_id')
message_chain_data = json_data.get('message_chain')
# Validate required fields
if not target_type:
return self.http_status(400, -1, 'target_type is required')
if not target_id:
return self.http_status(400, -1, 'target_id is required')
if not message_chain_data:
return self.http_status(400, -1, 'message_chain is required')
# Validate target_type
if target_type not in ['person', 'group']:
return self.http_status(400, -1, 'target_type must be either "person" or "group"')
try:
await self.ap.bot_service.send_message(bot_uuid, target_type, target_id, message_chain_data)
return self.success(data={'sent': True})
except Exception as e:
import traceback
traceback.print_exc()
return self.http_status(500, -1, f'Failed to send message: {str(e)}')

View File

@@ -15,16 +15,32 @@ from langbot_plugin.runtime.plugin.mgr import PluginInstallSource
@group.group_class('plugins', '/api/v1/plugins')
class PluginsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
@self.route('', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
plugins = await self.ap.plugin_connector.list_plugins()
return self.success(data={'plugins': plugins})
@self.route('/debug-info', methods=['GET'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
"""Get plugin debug information including debug URL and key"""
debug_info = await self.ap.plugin_connector.get_debug_info()
# Get debug URL from config
plugin_config = self.ap.instance_config.data.get('plugin', {})
debug_url = plugin_config.get('display_plugin_debug_url', 'http://localhost:5401')
return self.success(
data={
'debug_url': debug_url,
'plugin_debug_key': debug_info.get('plugin_debug_key', ''),
}
)
@self.route(
'/<author>/<plugin_name>/upgrade',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN,
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(author: str, plugin_name: str) -> str:
ctx = taskmgr.TaskContext.new()
@@ -40,7 +56,7 @@ class PluginsRouterGroup(group.RouterGroup):
@self.route(
'/<author>/<plugin_name>',
methods=['GET', 'DELETE'],
auth_type=group.AuthType.USER_TOKEN,
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(author: str, plugin_name: str) -> str:
if quart.request.method == 'GET':
@@ -66,7 +82,7 @@ class PluginsRouterGroup(group.RouterGroup):
@self.route(
'/<author>/<plugin_name>/config',
methods=['GET', 'PUT'],
auth_type=group.AuthType.USER_TOKEN,
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(author: str, plugin_name: str) -> quart.Response:
plugin = await self.ap.plugin_connector.get_plugin_info(author, plugin_name)
@@ -82,6 +98,16 @@ class PluginsRouterGroup(group.RouterGroup):
return self.success(data={})
@self.route(
'/<author>/<plugin_name>/readme',
methods=['GET'],
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _(author: str, plugin_name: str) -> quart.Response:
language = quart.request.args.get('language', 'en')
readme = await self.ap.plugin_connector.get_plugin_readme(author, plugin_name, language=language)
return self.success(data={'readme': readme})
@self.route(
'/<author>/<plugin_name>/icon',
methods=['GET'],
@@ -96,7 +122,18 @@ class PluginsRouterGroup(group.RouterGroup):
return quart.Response(icon_data, mimetype=mime_type)
@self.route('/github/releases', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
@self.route(
'/<author>/<plugin_name>/assets/<filepath>',
methods=['GET'],
auth_type=group.AuthType.NONE,
)
async def _(author: str, plugin_name: str, filepath: str) -> quart.Response:
asset_data = await self.ap.plugin_connector.get_plugin_assets(author, plugin_name, filepath)
asset_bytes = base64.b64decode(asset_data['asset_base64'])
mime_type = asset_data['mime_type']
return quart.Response(asset_bytes, mimetype=mime_type)
@self.route('/github/releases', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
"""Get releases from a GitHub repository URL"""
data = await quart.request.json
@@ -145,7 +182,7 @@ class PluginsRouterGroup(group.RouterGroup):
@self.route(
'/github/release-assets',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN,
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _() -> str:
"""Get assets from a specific GitHub release"""
@@ -199,7 +236,7 @@ class PluginsRouterGroup(group.RouterGroup):
except httpx.RequestError as e:
return self.http_status(500, -1, f'Failed to fetch release assets: {str(e)}')
@self.route('/install/github', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/install/github', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
"""Install plugin from GitHub release asset"""
data = await quart.request.json
@@ -233,7 +270,7 @@ class PluginsRouterGroup(group.RouterGroup):
@self.route(
'/install/marketplace',
methods=['POST'],
auth_type=group.AuthType.USER_TOKEN,
auth_type=group.AuthType.USER_TOKEN_OR_API_KEY,
)
async def _() -> str:
data = await quart.request.json
@@ -249,7 +286,7 @@ class PluginsRouterGroup(group.RouterGroup):
return self.success(data={'task_id': wrapper.id})
@self.route('/install/local', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
@self.route('/install/local', methods=['POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
file = (await quart.request.files).get('file')
if file is None:

View File

@@ -9,12 +9,15 @@ class LLMModelsRouterGroup(group.RouterGroup):
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
if quart.request.method == 'GET':
provider_uuid = quart.request.args.get('provider_uuid')
if provider_uuid:
return self.success(
data={'models': await self.ap.llm_model_service.get_llm_models_by_provider(provider_uuid)}
)
return self.success(data={'models': await self.ap.llm_model_service.get_llm_models()})
elif quart.request.method == 'POST':
json_data = await quart.request.json
model_uuid = await self.ap.llm_model_service.create_llm_model(json_data)
return self.success(data={'uuid': model_uuid})
@self.route('/<model_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
@@ -52,12 +55,19 @@ class EmbeddingModelsRouterGroup(group.RouterGroup):
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
if quart.request.method == 'GET':
provider_uuid = quart.request.args.get('provider_uuid')
if provider_uuid:
return self.success(
data={
'models': await self.ap.embedding_models_service.get_embedding_models_by_provider(
provider_uuid
)
}
)
return self.success(data={'models': await self.ap.embedding_models_service.get_embedding_models()})
elif quart.request.method == 'POST':
json_data = await quart.request.json
model_uuid = await self.ap.embedding_models_service.create_embedding_model(json_data)
return self.success(data={'uuid': model_uuid})
@self.route('/<model_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)

View File

@@ -0,0 +1,45 @@
import quart
from ... import group
@group.group_class('models/providers', '/api/v1/provider/providers')
class ModelProvidersRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY)
async def _() -> str:
if quart.request.method == 'GET':
providers = await self.ap.provider_service.get_providers()
# Add model counts
for provider in providers:
counts = await self.ap.provider_service.get_provider_model_counts(provider['uuid'])
provider['llm_count'] = counts['llm_count']
provider['embedding_count'] = counts['embedding_count']
return self.success(data={'providers': providers})
elif quart.request.method == 'POST':
json_data = await quart.request.json
provider_uuid = await self.ap.provider_service.create_provider(json_data)
return self.success(data={'uuid': provider_uuid})
@self.route(
'/<provider_uuid>', methods=['GET', 'PUT', 'DELETE'], auth_type=group.AuthType.USER_TOKEN_OR_API_KEY
)
async def _(provider_uuid: str) -> str:
if quart.request.method == 'GET':
provider = await self.ap.provider_service.get_provider(provider_uuid)
if provider is None:
return self.http_status(404, -1, 'provider not found')
counts = await self.ap.provider_service.get_provider_model_counts(provider_uuid)
provider['llm_count'] = counts['llm_count']
provider['embedding_count'] = counts['embedding_count']
return self.success(data={'provider': provider})
elif quart.request.method == 'PUT':
json_data = await quart.request.json
await self.ap.provider_service.update_provider(provider_uuid, json_data)
return self.success()
elif quart.request.method == 'DELETE':
try:
await self.ap.provider_service.delete_provider(provider_uuid)
return self.success()
except ValueError as e:
return self.http_status(400, -1, str(e))

View File

@@ -17,11 +17,13 @@ class SystemRouterGroup(group.RouterGroup):
'enable_marketplace', True
),
'cloud_service_url': (
self.ap.instance_config.data.get('plugin', {}).get(
'cloud_service_url', 'https://space.langbot.app'
)
if 'cloud_service_url' in self.ap.instance_config.data.get('plugin', {})
else 'https://space.langbot.app'
self.ap.instance_config.data.get('space', {}).get('url', 'https://space.langbot.app')
),
'allow_modify_login_info': self.ap.instance_config.data.get('system', {}).get(
'allow_modify_login_info', True
),
'disable_models_service': self.ap.instance_config.data.get('space', {}).get(
'disable_models_service', False
),
}
)
@@ -55,17 +57,6 @@ class SystemRouterGroup(group.RouterGroup):
return self.success(data=exec(py_code, {'ap': ap}))
@self.route('/debug/tools/call', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _() -> str:
if not constants.debug_mode:
return self.http_status(403, 403, 'Forbidden')
data = await quart.request.json
return self.success(
data=await self.ap.tool_mgr.execute_func_call(data['tool_name'], data['tool_parameters'])
)
@self.route(
'/debug/plugin/action',
methods=['POST'],

View File

@@ -1,8 +1,10 @@
import quart
import argon2
import asyncio
import traceback
from .. import group
from .....entity.errors import account as account_errors
@group.group_class('user', '/api/v1/user')
@@ -33,6 +35,8 @@ class UserRouterGroup(group.RouterGroup):
token = await self.ap.user_service.authenticate(json_data['user'], json_data['password'])
except argon2.exceptions.VerifyMismatchError:
return self.fail(1, 'Invalid username or password')
except ValueError as e:
return self.fail(1, str(e))
return self.success(data={'token': token})
@@ -70,6 +74,13 @@ class UserRouterGroup(group.RouterGroup):
@self.route('/change-password', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(user_email: str) -> str:
# Check if password change is allowed
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
'allow_modify_login_info', True
)
if not allow_modify_login_info:
return self.http_status(403, -1, 'Modifying login info is disabled')
json_data = await quart.request.json
current_password = json_data['current_password']
@@ -83,3 +94,169 @@ class UserRouterGroup(group.RouterGroup):
return self.http_status(400, -1, str(e))
return self.success(data={'user': user_email})
# Space OAuth endpoints (redirect flow)
@self.route('/space/authorize-url', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str:
"""Get Space OAuth authorization URL for redirect"""
redirect_uri = quart.request.args.get('redirect_uri', '')
state = quart.request.args.get('state', '')
if not redirect_uri:
return self.fail(1, 'Missing redirect_uri parameter')
try:
authorize_url = self.ap.space_service.get_oauth_authorize_url(redirect_uri, state)
return self.success(data={'authorize_url': authorize_url})
except Exception as e:
return self.fail(1, str(e))
@self.route('/space/callback', methods=['POST'], auth_type=group.AuthType.NONE)
async def _() -> str:
"""Handle OAuth callback - exchange code for tokens and authenticate"""
json_data = await quart.request.json
code = json_data.get('code')
if not code:
return self.fail(1, 'Missing authorization code')
try:
# Exchange code for tokens
token_data = await self.ap.space_service.exchange_oauth_code(code)
access_token = token_data.get('access_token')
refresh_token = token_data.get('refresh_token')
expires_in = token_data.get('expires_in', 0)
if not access_token:
return self.fail(1, 'Failed to get access token from Space')
# Authenticate and create/update local user
jwt_token, user_obj = await self.ap.user_service.authenticate_space_user(
access_token, refresh_token, expires_in
)
return self.success(
data={
'token': jwt_token,
'user': user_obj.user,
}
)
except account_errors.AccountEmailMismatchError as e:
return self.fail(3, str(e))
except ValueError as e:
traceback.print_exc()
return self.fail(1, str(e))
except Exception as e:
traceback.print_exc()
return self.fail(2, f'OAuth callback failed: {str(e)}')
@self.route('/info', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(user_email: str) -> str:
"""Get current user information including account type"""
user_obj = await self.ap.user_service.get_user_by_email(user_email)
if user_obj is None:
return self.http_status(404, -1, 'User not found')
return self.success(
data={
'user': user_obj.user,
'account_type': user_obj.account_type,
'has_password': bool(user_obj.password and user_obj.password.strip()),
}
)
@self.route('/space-credits', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
async def _(user_email: str) -> str:
"""Get Space credits balance for current user"""
credits = await self.ap.space_service.get_credits(user_email)
return self.success(data={'credits': credits})
@self.route('/account-info', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str:
"""Get account info for login page (account type and has_password)"""
if not await self.ap.user_service.is_initialized():
return self.success(data={'initialized': False})
user_obj = await self.ap.user_service.get_first_user()
if user_obj is None:
return self.success(data={'initialized': False})
return self.success(
data={
'initialized': True,
'account_type': user_obj.account_type,
'has_password': bool(user_obj.password and user_obj.password.strip()),
}
)
@self.route('/set-password', methods=['POST'], auth_type=group.AuthType.USER_TOKEN)
async def _(user_email: str) -> str:
"""Set password for Space account (first time) or change password"""
json_data = await quart.request.json
new_password = json_data.get('new_password')
current_password = json_data.get('current_password')
if not new_password:
return self.http_status(400, -1, 'New password is required')
user_obj = await self.ap.user_service.get_user_by_email(user_email)
if user_obj is None:
return self.http_status(404, -1, 'User not found')
try:
await self.ap.user_service.set_password(user_email, new_password, current_password)
return self.success(data={'user': user_email})
except ValueError as e:
return self.http_status(400, -1, str(e))
except argon2.exceptions.VerifyMismatchError:
return self.http_status(400, -1, 'Current password is incorrect')
@self.route('/bind-space', methods=['POST'], auth_type=group.AuthType.NONE)
async def _() -> str:
"""Bind Space account to existing local account"""
# Check if modifying login info is allowed
allow_modify_login_info = self.ap.instance_config.data.get('system', {}).get(
'allow_modify_login_info', True
)
if not allow_modify_login_info:
return self.http_status(403, -1, 'Modifying login info is disabled')
json_data = await quart.request.json
code = json_data.get('code')
state = json_data.get('state') # JWT token passed as state
if not code:
return self.http_status(400, -1, 'Missing authorization code')
if not state:
return self.http_status(400, -1, 'Missing state parameter')
# Verify state is a valid JWT token
try:
user_email = await self.ap.user_service.verify_jwt_token(state)
except Exception:
return self.http_status(401, -1, 'Invalid or expired state')
user_obj = await self.ap.user_service.get_user_by_email(user_email)
if user_obj is None:
return self.http_status(404, -1, 'User not found')
if user_obj.account_type != 'local':
return self.http_status(400, -1, 'Only local accounts can bind to Space')
try:
updated_user = await self.ap.user_service.bind_space_account(user_email, code)
jwt_token = await self.ap.user_service.generate_jwt_token(updated_user.user)
return self.success(
data={
'token': jwt_token,
'user': updated_user.user,
'account_type': updated_user.account_type,
}
)
except ValueError as e:
return self.http_status(400, -1, str(e))
except Exception as e:
return self.http_status(500, -1, f'Failed to bind Space account: {str(e)}')

View File

@@ -0,0 +1,49 @@
import quart
from .. import group
@group.group_class('webhook_mgmt', '/api/v1/webhooks')
class WebhookManagementRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'])
async def _() -> str:
if quart.request.method == 'GET':
webhooks = await self.ap.webhook_service.get_webhooks()
return self.success(data={'webhooks': webhooks})
elif quart.request.method == 'POST':
json_data = await quart.request.json
name = json_data.get('name', '')
url = json_data.get('url', '')
description = json_data.get('description', '')
enabled = json_data.get('enabled', True)
if not name:
return self.http_status(400, -1, 'Name is required')
if not url:
return self.http_status(400, -1, 'URL is required')
webhook = await self.ap.webhook_service.create_webhook(name, url, description, enabled)
return self.success(data={'webhook': webhook})
@self.route('/<int:webhook_id>', methods=['GET', 'PUT', 'DELETE'])
async def _(webhook_id: int) -> str:
if quart.request.method == 'GET':
webhook = await self.ap.webhook_service.get_webhook(webhook_id)
if webhook is None:
return self.http_status(404, -1, 'Webhook not found')
return self.success(data={'webhook': webhook})
elif quart.request.method == 'PUT':
json_data = await quart.request.json
name = json_data.get('name')
url = json_data.get('url')
description = json_data.get('description')
enabled = json_data.get('enabled')
await self.ap.webhook_service.update_webhook(webhook_id, name, url, description, enabled)
return self.success()
elif quart.request.method == 'DELETE':
await self.ap.webhook_service.delete_webhook(webhook_id)
return self.success()

View File

@@ -1,49 +1,54 @@
from __future__ import annotations
import quart
import traceback
from .. import group
@group.group_class('webhooks', '/api/v1/webhooks')
class WebhooksRouterGroup(group.RouterGroup):
@group.group_class('webhooks', '/bots')
class WebhookRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET', 'POST'])
async def _() -> str:
if quart.request.method == 'GET':
webhooks = await self.ap.webhook_service.get_webhooks()
return self.success(data={'webhooks': webhooks})
elif quart.request.method == 'POST':
json_data = await quart.request.json
name = json_data.get('name', '')
url = json_data.get('url', '')
description = json_data.get('description', '')
enabled = json_data.get('enabled', True)
@self.route('/<bot_uuid>', methods=['GET', 'POST'], auth_type=group.AuthType.NONE)
async def handle_webhook(bot_uuid: str):
"""处理 bot webhook 回调(无子路径)"""
return await self._dispatch_webhook(bot_uuid, '')
if not name:
return self.http_status(400, -1, 'Name is required')
if not url:
return self.http_status(400, -1, 'URL is required')
@self.route('/<bot_uuid>/<path:path>', methods=['GET', 'POST'], auth_type=group.AuthType.NONE)
async def handle_webhook_with_path(bot_uuid: str, path: str):
"""处理 bot webhook 回调(带子路径)"""
return await self._dispatch_webhook(bot_uuid, path)
webhook = await self.ap.webhook_service.create_webhook(name, url, description, enabled)
return self.success(data={'webhook': webhook})
async def _dispatch_webhook(self, bot_uuid: str, path: str):
"""分发 webhook 请求到对应的 bot adapter
@self.route('/<int:webhook_id>', methods=['GET', 'PUT', 'DELETE'])
async def _(webhook_id: int) -> str:
if quart.request.method == 'GET':
webhook = await self.ap.webhook_service.get_webhook(webhook_id)
if webhook is None:
return self.http_status(404, -1, 'Webhook not found')
return self.success(data={'webhook': webhook})
Args:
bot_uuid: Bot 的 UUID
path: 子路径(如果有的话)
elif quart.request.method == 'PUT':
json_data = await quart.request.json
name = json_data.get('name')
url = json_data.get('url')
description = json_data.get('description')
enabled = json_data.get('enabled')
Returns:
适配器返回的响应
"""
try:
runtime_bot = await self.ap.platform_mgr.get_bot_by_uuid(bot_uuid)
await self.ap.webhook_service.update_webhook(webhook_id, name, url, description, enabled)
return self.success()
if not runtime_bot:
return quart.jsonify({'error': 'Bot not found'}), 404
elif quart.request.method == 'DELETE':
await self.ap.webhook_service.delete_webhook(webhook_id)
return self.success()
if not runtime_bot.enable:
return quart.jsonify({'error': 'Bot is disabled'}), 403
if not hasattr(runtime_bot.adapter, 'handle_unified_webhook'):
return quart.jsonify({'error': 'Adapter does not support unified webhook'}), 501
response = await runtime_bot.adapter.handle_unified_webhook(
bot_uuid=bot_uuid,
path=path,
request=quart.request,
)
return response
except Exception as e:
self.ap.logger.error(f'Webhook dispatch error for bot {bot_uuid}: {traceback.format_exc()}')
return quart.jsonify({'error': str(e)}), 500

View File

@@ -92,7 +92,11 @@ class HTTPController:
@self.quart_app.route('/')
async def index():
return await quart.send_from_directory(frontend_path, 'index.html', mimetype='text/html')
response = await quart.send_from_directory(frontend_path, 'index.html', mimetype='text/html')
response.headers['Cache-Control'] = 'no-cache, no-store, must-revalidate'
response.headers['Pragma'] = 'no-cache'
response.headers['Expires'] = '0'
return response
@self.quart_app.route('/<path:path>')
async def static_file(path: str):

View File

@@ -58,6 +58,25 @@ class BotService:
if runtime_bot is not None:
adapter_runtime_values['bot_account_id'] = runtime_bot.adapter.bot_account_id
# Webhook URL for unified webhook adapters (independent of bot running state)
if persistence_bot['adapter'] in [
'wecom',
'wecombot',
'officialaccount',
'qqofficial',
'slack',
'wecomcs',
'LINE',
'lark',
]:
webhook_prefix = self.ap.instance_config.data['api'].get('webhook_prefix', 'http://127.0.0.1:5300')
webhook_url = f'/bots/{bot_uuid}'
adapter_runtime_values['webhook_url'] = webhook_url
adapter_runtime_values['webhook_full_url'] = f'{webhook_prefix}{webhook_url}'
else:
adapter_runtime_values['webhook_url'] = None
adapter_runtime_values['webhook_full_url'] = None
persistence_bot['adapter_runtime_values'] = adapter_runtime_values
return persistence_bot
@@ -139,3 +158,29 @@ class BotService:
logs, total_count = await runtime_bot.logger.get_logs(from_index, max_count)
return [log.to_json() for log in logs], total_count
async def send_message(self, bot_uuid: str, target_type: str, target_id: str, message_chain_data: dict) -> None:
"""Send message to a specific target via bot
Args:
bot_uuid: The UUID of the bot
target_type: The type of the target, can be "group", "person"
target_id: The ID of the target
message_chain_data: The message chain data in dict format
"""
# Import here to avoid circular imports
import langbot_plugin.api.entities.builtin.platform.message as platform_message
# Get runtime bot
runtime_bot = await self.ap.platform_mgr.get_bot_by_uuid(bot_uuid)
if runtime_bot is None:
raise Exception(f'Bot not found: {bot_uuid}')
# Validate and convert message chain
try:
message_chain = platform_message.MessageChain.model_validate(message_chain_data)
except Exception as e:
raise Exception(f'Invalid message_chain format: {str(e)}')
# Send message via adapter
await runtime_bot.adapter.send_message(target_type, str(target_id), message_chain)

View File

@@ -0,0 +1,80 @@
from __future__ import annotations
from ....core import app
import sqlalchemy
from langbot.pkg.entity.persistence import rag as persistence_rag
import uuid
class ExternalKBService:
"""External KB service"""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
# External Knowledge Base methods
async def get_external_knowledge_bases(self) -> list[dict]:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.ExternalKnowledgeBase))
external_kbs = result.all()
return [
self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
for external_kb in external_kbs
]
async def get_external_knowledge_base(self, kb_uuid: str) -> dict | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
external_kb = result.first()
if external_kb is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_rag.ExternalKnowledgeBase, external_kb)
async def create_external_knowledge_base(self, kb_data: dict) -> str:
kb_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_rag.ExternalKnowledgeBase).values(kb_data)
)
kb = await self.get_external_knowledge_base(kb_data['uuid'])
await self.ap.rag_mgr.load_external_knowledge_base(kb)
return kb_data['uuid']
async def retrieve_external_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
"""Retrieve external knowledge base"""
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return [
result.model_dump() for result in await runtime_kb.retrieve(query, 5)
] # top_k is just a placeholder for external knowledge base
async def update_external_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None:
if 'uuid' in kb_data:
del kb_data['uuid']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.ExternalKnowledgeBase)
.values(kb_data)
.where(persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid)
)
await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid)
kb = await self.get_external_knowledge_base(kb_uuid)
await self.ap.rag_mgr.load_external_knowledge_base(kb)
async def delete_external_knowledge_base(self, kb_uuid: str) -> None:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_rag.ExternalKnowledgeBase).where(
persistence_rag.ExternalKnowledgeBase.uuid == kb_uuid
)
)
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)

View File

@@ -71,7 +71,19 @@ class KnowledgeService:
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
return await runtime_kb.store_file(file_id)
# Only internal KBs support file storage
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file storage')
result = await runtime_kb.store_file(file_id)
# Update the KB's updated_at timestamp
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.KnowledgeBase)
.values(updated_at=sqlalchemy.func.now())
.where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
return result
async def retrieve_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]:
"""检索知识库"""
@@ -95,8 +107,18 @@ class KnowledgeService:
runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
if runtime_kb is None:
raise Exception('Knowledge base not found')
# Only internal KBs support file deletion
if runtime_kb.get_type() != 'internal':
raise Exception('Only internal knowledge bases support file deletion')
await runtime_kb.delete_file(file_id)
# Update the KB's updated_at timestamp
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.KnowledgeBase)
.values(updated_at=sqlalchemy.func.now())
.where(persistence_rag.KnowledgeBase.uuid == kb_uuid)
)
async def delete_knowledge_base(self, kb_uuid: str) -> None:
"""删除知识库"""
await self.ap.rag_mgr.delete_knowledge_base(kb_uuid)

View File

@@ -11,6 +11,18 @@ from ....entity.persistence import pipeline as persistence_pipeline
from ....provider.modelmgr import requester as model_requester
def _parse_provider_api_keys(provider_dict: dict) -> dict:
"""Parse api_keys if it's a JSON string"""
if isinstance(provider_dict.get('api_keys'), str):
import json
try:
provider_dict['api_keys'] = json.loads(provider_dict['api_keys'])
except Exception:
provider_dict['api_keys'] = []
return provider_dict
class LLMModelsService:
ap: app.Application
@@ -18,29 +30,72 @@ class LLMModelsService:
self.ap = ap
async def get_llm_models(self, include_secret: bool = True) -> list[dict]:
"""Get all LLM models with provider info"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.LLMModel))
models = result.all()
masked_columns = []
if not include_secret:
masked_columns = ['api_keys']
# Get all providers for lookup
providers_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider)
)
providers = {p.uuid: p for p in providers_result.all()}
return [
self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model, masked_columns)
for model in models
]
models_list = []
for model in models:
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
provider = providers.get(model.provider_uuid)
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
provider_dict = _parse_provider_api_keys(provider_dict)
if not include_secret:
provider_dict['api_keys'] = ['***'] * len(provider_dict.get('api_keys', []))
model_dict['provider'] = provider_dict
models_list.append(model_dict)
async def create_llm_model(self, model_data: dict) -> str:
model_data['uuid'] = str(uuid.uuid4())
return models_list
async def get_llm_models_by_provider(self, provider_uuid: str) -> list[dict]:
"""Get LLM models by provider UUID"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.LLMModel).where(
persistence_model.LLMModel.provider_uuid == provider_uuid
)
)
models = result.all()
return [self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, m) for m in models]
async def create_llm_model(self, model_data: dict, preserve_uuid: bool = False) -> str:
"""Create a new LLM model"""
if not preserve_uuid:
model_data['uuid'] = str(uuid.uuid4())
# Handle provider creation if needed
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
# Create new provider
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_model.LLMModel).values(**model_data))
llm_model = await self.get_llm_model(model_data['uuid'])
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
await self.ap.model_mgr.load_llm_model(llm_model)
runtime_llm_model = await self.ap.model_mgr.load_llm_model_with_provider(
persistence_model.LLMModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.llm_models.append(runtime_llm_model)
# check if default pipeline has no model bound
# set the default pipeline model to this model
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.is_default == True
@@ -56,21 +111,47 @@ class LLMModelsService:
return model_data['uuid']
async def get_llm_model(self, model_uuid: str) -> dict | None:
"""Get a single LLM model with provider info"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.LLMModel).where(persistence_model.LLMModel.uuid == model_uuid)
)
model = result.first()
if model is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.LLMModel, model)
# Get provider
provider_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.uuid == model.provider_uuid
)
)
provider = provider_result.first()
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
model_dict['provider'] = _parse_provider_api_keys(provider_dict)
return model_dict
async def update_llm_model(self, model_uuid: str, model_data: dict) -> None:
"""Update an existing LLM model"""
if 'uuid' in model_data:
del model_data['uuid']
# Handle provider update if needed
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_model.LLMModel)
.where(persistence_model.LLMModel.uuid == model_uuid)
@@ -79,18 +160,25 @@ class LLMModelsService:
await self.ap.model_mgr.remove_llm_model(model_uuid)
llm_model = await self.get_llm_model(model_uuid)
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
await self.ap.model_mgr.load_llm_model(llm_model)
runtime_llm_model = await self.ap.model_mgr.load_llm_model_with_provider(
persistence_model.LLMModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.llm_models.append(runtime_llm_model)
async def delete_llm_model(self, model_uuid: str) -> None:
"""Delete an LLM model"""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_model.LLMModel).where(persistence_model.LLMModel.uuid == model_uuid)
)
await self.ap.model_mgr.remove_llm_model(model_uuid)
async def test_llm_model(self, model_uuid: str, model_data: dict) -> None:
"""Test an LLM model"""
runtime_llm_model: model_requester.RuntimeLLMModel | None = None
if model_uuid != '_':
@@ -98,25 +186,18 @@ class LLMModelsService:
if model.model_entity.uuid == model_uuid:
runtime_llm_model = model
break
if runtime_llm_model is None:
raise Exception('model not found')
else:
runtime_llm_model = await self.ap.model_mgr.init_runtime_llm_model(model_data)
runtime_llm_model = await self.ap.model_mgr.init_temporary_runtime_llm_model(model_data)
# Mon Nov 10 2025: Commented for some providers may not support thinking parameter
# # 有些模型厂商默认开启了思考功能,测试容易延迟
# extra_args = model_data.get('extra_args', {})
# if not extra_args or 'thinking' not in extra_args:
# extra_args['thinking'] = {'type': 'disabled'}
await runtime_llm_model.requester.invoke_llm(
extra_args = model_data.get('extra_args', {})
await runtime_llm_model.provider.invoke_llm(
query=None,
model=runtime_llm_model,
messages=[provider_message.Message(role='user', content='Hello, world! Please just reply a "Hello".')],
funcs=[],
# extra_args=extra_args,
extra_args=extra_args,
)
@@ -127,42 +208,111 @@ class EmbeddingModelsService:
self.ap = ap
async def get_embedding_models(self) -> list[dict]:
"""Get all embedding models with provider info"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.EmbeddingModel))
models = result.all()
return [self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model) for model in models]
async def create_embedding_model(self, model_data: dict) -> str:
model_data['uuid'] = str(uuid.uuid4())
providers_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider)
)
providers = {p.uuid: p for p in providers_result.all()}
models_list = []
for model in models:
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model)
provider = providers.get(model.provider_uuid)
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
model_dict['provider'] = _parse_provider_api_keys(provider_dict)
models_list.append(model_dict)
return models_list
async def get_embedding_models_by_provider(self, provider_uuid: str) -> list[dict]:
"""Get embedding models by provider UUID"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.EmbeddingModel).where(
persistence_model.EmbeddingModel.provider_uuid == provider_uuid
)
)
models = result.all()
return [self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, m) for m in models]
async def create_embedding_model(self, model_data: dict, preserve_uuid: bool = False) -> str:
"""Create a new embedding model"""
if not preserve_uuid:
model_data['uuid'] = str(uuid.uuid4())
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.EmbeddingModel).values(**model_data)
)
embedding_model = await self.get_embedding_model(model_data['uuid'])
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
await self.ap.model_mgr.load_embedding_model(embedding_model)
runtime_embedding_model = await self.ap.model_mgr.load_embedding_model_with_provider(
persistence_model.EmbeddingModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.embedding_models.append(runtime_embedding_model)
return model_data['uuid']
async def get_embedding_model(self, model_uuid: str) -> dict | None:
"""Get a single embedding model with provider info"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.EmbeddingModel).where(
persistence_model.EmbeddingModel.uuid == model_uuid
)
)
model = result.first()
if model is None:
return None
return self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model)
model_dict = self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model)
provider_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.uuid == model.provider_uuid
)
)
provider = provider_result.first()
if provider:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
model_dict['provider'] = _parse_provider_api_keys(provider_dict)
return model_dict
async def update_embedding_model(self, model_uuid: str, model_data: dict) -> None:
"""Update an existing embedding model"""
if 'uuid' in model_data:
del model_data['uuid']
if 'provider' in model_data:
provider_data = model_data.pop('provider')
if provider_data.get('uuid'):
model_data['provider_uuid'] = provider_data['uuid']
else:
provider_uuid = await self.ap.provider_service.find_or_create_provider(
requester=provider_data.get('requester', ''),
base_url=provider_data.get('base_url', ''),
api_keys=provider_data.get('api_keys', []),
)
model_data['provider_uuid'] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_model.EmbeddingModel)
.where(persistence_model.EmbeddingModel.uuid == model_uuid)
@@ -171,20 +321,27 @@ class EmbeddingModelsService:
await self.ap.model_mgr.remove_embedding_model(model_uuid)
embedding_model = await self.get_embedding_model(model_uuid)
runtime_provider = self.ap.model_mgr.provider_dict.get(model_data['provider_uuid'])
if runtime_provider is None:
raise Exception('provider not found')
await self.ap.model_mgr.load_embedding_model(embedding_model)
runtime_embedding_model = await self.ap.model_mgr.load_embedding_model_with_provider(
persistence_model.EmbeddingModel(**model_data),
runtime_provider,
)
self.ap.model_mgr.embedding_models.append(runtime_embedding_model)
async def delete_embedding_model(self, model_uuid: str) -> None:
"""Delete an embedding model"""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_model.EmbeddingModel).where(
persistence_model.EmbeddingModel.uuid == model_uuid
)
)
await self.ap.model_mgr.remove_embedding_model(model_uuid)
async def test_embedding_model(self, model_uuid: str, model_data: dict) -> None:
"""Test an embedding model"""
runtime_embedding_model: model_requester.RuntimeEmbeddingModel | None = None
if model_uuid != '_':
@@ -192,14 +349,12 @@ class EmbeddingModelsService:
if model.model_entity.uuid == model_uuid:
runtime_embedding_model = model
break
if runtime_embedding_model is None:
raise Exception('model not found')
else:
runtime_embedding_model = await self.ap.model_mgr.init_runtime_embedding_model(model_data)
runtime_embedding_model = await self.ap.model_mgr.init_temporary_runtime_embedding_model(model_data)
await runtime_embedding_model.requester.invoke_embedding(
await runtime_embedding_model.provider.invoke_embedding(
model=runtime_embedding_model,
input_text=['Hello, world!'],
extra_args={},

View File

@@ -0,0 +1,796 @@
from __future__ import annotations
import uuid
import datetime
import sqlalchemy
from ....core import app
from ....entity.persistence import monitoring as persistence_monitoring
class MonitoringService:
"""Monitoring service"""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
# ========== Recording Methods ==========
async def record_message(
self,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
message_content: str,
session_id: str,
status: str = 'success',
level: str = 'info',
platform: str | None = None,
user_id: str | None = None,
runner_name: str | None = None,
variables: str | None = None,
) -> str:
"""Record a message"""
message_id = str(uuid.uuid4())
message_data = {
'id': message_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'bot_id': bot_id,
'bot_name': bot_name,
'pipeline_id': pipeline_id,
'pipeline_name': pipeline_name,
'message_content': message_content,
'session_id': session_id,
'status': status,
'level': level,
'platform': platform,
'user_id': user_id,
'runner_name': runner_name,
'variables': variables,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringMessage).values(message_data)
)
return message_id
async def record_llm_call(
self,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
session_id: str,
model_name: str,
input_tokens: int,
output_tokens: int,
duration: int,
status: str = 'success',
cost: float | None = None,
error_message: str | None = None,
message_id: str | None = None,
) -> str:
"""Record an LLM call"""
call_id = str(uuid.uuid4())
call_data = {
'id': call_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'model_name': model_name,
'input_tokens': input_tokens,
'output_tokens': output_tokens,
'total_tokens': input_tokens + output_tokens,
'duration': duration,
'cost': cost,
'status': status,
'bot_id': bot_id,
'bot_name': bot_name,
'pipeline_id': pipeline_id,
'pipeline_name': pipeline_name,
'session_id': session_id,
'error_message': error_message,
'message_id': message_id,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringLLMCall).values(call_data)
)
return call_id
async def record_embedding_call(
self,
model_name: str,
prompt_tokens: int,
total_tokens: int,
duration: int,
input_count: int,
status: str = 'success',
error_message: str | None = None,
knowledge_base_id: str | None = None,
query_text: str | None = None,
session_id: str | None = None,
message_id: str | None = None,
call_type: str | None = None,
) -> str:
"""Record an embedding call"""
call_id = str(uuid.uuid4())
call_data = {
'id': call_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'model_name': model_name,
'prompt_tokens': prompt_tokens,
'total_tokens': total_tokens,
'duration': duration,
'input_count': input_count,
'status': status,
'error_message': error_message,
'knowledge_base_id': knowledge_base_id,
'query_text': query_text,
'session_id': session_id,
'message_id': message_id,
'call_type': call_type,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringEmbeddingCall).values(call_data)
)
return call_id
async def record_session_start(
self,
session_id: str,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
platform: str | None = None,
user_id: str | None = None,
) -> None:
"""Record a new session"""
session_data = {
'session_id': session_id,
'bot_id': bot_id,
'bot_name': bot_name,
'pipeline_id': pipeline_id,
'pipeline_name': pipeline_name,
'message_count': 0,
'start_time': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'last_activity': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'is_active': True,
'platform': platform,
'user_id': user_id,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringSession).values(session_data)
)
async def update_session_activity(
self,
session_id: str,
pipeline_id: str | None = None,
pipeline_name: str | None = None,
) -> bool:
"""Update session last activity time and increment message count.
Also updates pipeline info if the bot's pipeline has changed.
Returns:
True if session was found and updated, False if session doesn't exist.
"""
update_values = {
'last_activity': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'message_count': persistence_monitoring.MonitoringSession.message_count + 1,
}
# Update pipeline info if provided (handles pipeline switch)
if pipeline_id is not None:
update_values['pipeline_id'] = pipeline_id
if pipeline_name is not None:
update_values['pipeline_name'] = pipeline_name
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_monitoring.MonitoringSession)
.where(persistence_monitoring.MonitoringSession.session_id == session_id)
.values(update_values)
)
# Check if any rows were updated
return result.rowcount > 0
async def record_error(
self,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
error_type: str,
error_message: str,
session_id: str | None = None,
stack_trace: str | None = None,
message_id: str | None = None,
) -> str:
"""Record an error"""
error_id = str(uuid.uuid4())
error_data = {
'id': error_id,
'timestamp': datetime.datetime.now(datetime.timezone.utc).replace(tzinfo=None),
'error_type': error_type,
'error_message': error_message,
'bot_id': bot_id,
'bot_name': bot_name,
'pipeline_id': pipeline_id,
'pipeline_name': pipeline_name,
'session_id': session_id,
'stack_trace': stack_trace,
'message_id': message_id,
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_monitoring.MonitoringError).values(error_data)
)
return error_id
async def update_message_status(
self,
message_id: str,
status: str,
level: str | None = None,
variables: str | None = None,
) -> None:
"""Update message status and optionally variables"""
update_values = {'status': status}
if level is not None:
update_values['level'] = level
if variables is not None:
update_values['variables'] = variables
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_monitoring.MonitoringMessage)
.where(persistence_monitoring.MonitoringMessage.id == message_id)
.values(update_values)
)
# ========== Query Methods ==========
async def get_overview_metrics(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
) -> dict:
"""Get overview metrics"""
# Build base query conditions
message_conditions = []
llm_conditions = []
embedding_conditions = []
session_conditions = []
if bot_ids:
message_conditions.append(persistence_monitoring.MonitoringMessage.bot_id.in_(bot_ids))
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.bot_id.in_(bot_ids))
session_conditions.append(persistence_monitoring.MonitoringSession.bot_id.in_(bot_ids))
if pipeline_ids:
message_conditions.append(persistence_monitoring.MonitoringMessage.pipeline_id.in_(pipeline_ids))
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.pipeline_id.in_(pipeline_ids))
session_conditions.append(persistence_monitoring.MonitoringSession.pipeline_id.in_(pipeline_ids))
if start_time:
message_conditions.append(persistence_monitoring.MonitoringMessage.timestamp >= start_time)
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp >= start_time)
embedding_conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp >= start_time)
session_conditions.append(persistence_monitoring.MonitoringSession.start_time >= start_time)
if end_time:
message_conditions.append(persistence_monitoring.MonitoringMessage.timestamp <= end_time)
llm_conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp <= end_time)
embedding_conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp <= end_time)
session_conditions.append(persistence_monitoring.MonitoringSession.start_time <= end_time)
# Total messages
message_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringMessage.id))
if message_conditions:
message_query = message_query.where(sqlalchemy.and_(*message_conditions))
total_messages_result = await self.ap.persistence_mgr.execute_async(message_query)
total_messages = total_messages_result.scalar() or 0
# Total LLM calls
llm_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringLLMCall.id))
if llm_conditions:
llm_query = llm_query.where(sqlalchemy.and_(*llm_conditions))
llm_calls_result = await self.ap.persistence_mgr.execute_async(llm_query)
llm_calls = llm_calls_result.scalar() or 0
# Total Embedding calls
embedding_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringEmbeddingCall.id))
if embedding_conditions:
embedding_query = embedding_query.where(sqlalchemy.and_(*embedding_conditions))
embedding_calls_result = await self.ap.persistence_mgr.execute_async(embedding_query)
embedding_calls = embedding_calls_result.scalar() or 0
# Total model calls (LLM + Embedding)
model_calls = llm_calls + embedding_calls
# Success rate (based on messages)
success_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringMessage.id)).where(
persistence_monitoring.MonitoringMessage.status == 'success'
)
if message_conditions:
success_query = success_query.where(sqlalchemy.and_(*message_conditions))
success_result = await self.ap.persistence_mgr.execute_async(success_query)
success_count = success_result.scalar() or 0
success_rate = (success_count / total_messages * 100) if total_messages > 0 else 100
# Active sessions
active_session_query = sqlalchemy.select(
sqlalchemy.func.count(persistence_monitoring.MonitoringSession.session_id)
).where(persistence_monitoring.MonitoringSession.is_active == True)
if session_conditions:
active_session_query = active_session_query.where(sqlalchemy.and_(*session_conditions))
active_sessions_result = await self.ap.persistence_mgr.execute_async(active_session_query)
active_sessions = active_sessions_result.scalar() or 0
return {
'total_messages': total_messages,
'llm_calls': llm_calls,
'embedding_calls': embedding_calls,
'model_calls': model_calls,
'success_rate': round(success_rate, 2),
'active_sessions': active_sessions,
}
async def get_messages(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get messages with filters"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringMessage.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringMessage.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringMessage.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringMessage.timestamp <= end_time)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringMessage.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get messages
query = sqlalchemy.select(persistence_monitoring.MonitoringMessage).order_by(
persistence_monitoring.MonitoringMessage.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
messages_rows = result.all()
serialized = []
for row in messages_rows:
# Extract model instance from Row (SQLAlchemy returns Row objects)
msg = row[0] if isinstance(row, tuple) else row
serialized_msg = self.ap.persistence_mgr.serialize_model(persistence_monitoring.MonitoringMessage, msg)
serialized.append(serialized_msg)
return (serialized, total)
async def get_llm_calls(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get LLM calls with filters"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringLLMCall.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringLLMCall.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringLLMCall.timestamp <= end_time)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringLLMCall.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get LLM calls
query = sqlalchemy.select(persistence_monitoring.MonitoringLLMCall).order_by(
persistence_monitoring.MonitoringLLMCall.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
llm_calls_rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringLLMCall, row[0] if isinstance(row, tuple) else row
)
for row in llm_calls_rows
],
total,
)
async def get_embedding_calls(
self,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
knowledge_base_id: str | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get embedding calls with filters"""
conditions = []
if start_time:
conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringEmbeddingCall.timestamp <= end_time)
if knowledge_base_id:
conditions.append(persistence_monitoring.MonitoringEmbeddingCall.knowledge_base_id == knowledge_base_id)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringEmbeddingCall.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get embedding calls
query = sqlalchemy.select(persistence_monitoring.MonitoringEmbeddingCall).order_by(
persistence_monitoring.MonitoringEmbeddingCall.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
embedding_calls_rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringEmbeddingCall, row[0] if isinstance(row, tuple) else row
)
for row in embedding_calls_rows
],
total,
)
async def get_sessions(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
is_active: bool | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get sessions with filters"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringSession.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringSession.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringSession.start_time >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringSession.start_time <= end_time)
if is_active is not None:
conditions.append(persistence_monitoring.MonitoringSession.is_active == is_active)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringSession.session_id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get sessions
query = sqlalchemy.select(persistence_monitoring.MonitoringSession).order_by(
persistence_monitoring.MonitoringSession.last_activity.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
sessions_rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringSession, row[0] if isinstance(row, tuple) else row
)
for row in sessions_rows
],
total,
)
async def get_errors(
self,
bot_ids: list[str] | None = None,
pipeline_ids: list[str] | None = None,
start_time: datetime.datetime | None = None,
end_time: datetime.datetime | None = None,
limit: int = 100,
offset: int = 0,
) -> tuple[list[dict], int]:
"""Get errors with filters"""
conditions = []
if bot_ids:
conditions.append(persistence_monitoring.MonitoringError.bot_id.in_(bot_ids))
if pipeline_ids:
conditions.append(persistence_monitoring.MonitoringError.pipeline_id.in_(pipeline_ids))
if start_time:
conditions.append(persistence_monitoring.MonitoringError.timestamp >= start_time)
if end_time:
conditions.append(persistence_monitoring.MonitoringError.timestamp <= end_time)
# Get total count
count_query = sqlalchemy.select(sqlalchemy.func.count(persistence_monitoring.MonitoringError.id))
if conditions:
count_query = count_query.where(sqlalchemy.and_(*conditions))
count_result = await self.ap.persistence_mgr.execute_async(count_query)
total = count_result.scalar() or 0
# Get errors
query = sqlalchemy.select(persistence_monitoring.MonitoringError).order_by(
persistence_monitoring.MonitoringError.timestamp.desc()
)
if conditions:
query = query.where(sqlalchemy.and_(*conditions))
query = query.limit(limit).offset(offset)
result = await self.ap.persistence_mgr.execute_async(query)
errors_rows = result.all()
return (
[
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringError, row[0] if isinstance(row, tuple) else row
)
for row in errors_rows
],
total,
)
async def get_session_analysis(
self,
session_id: str,
) -> dict:
"""Get detailed analysis for a specific session"""
# Get session info
session_query = sqlalchemy.select(persistence_monitoring.MonitoringSession).where(
persistence_monitoring.MonitoringSession.session_id == session_id
)
session_result = await self.ap.persistence_mgr.execute_async(session_query)
session_row = session_result.first()
if not session_row:
return {
'session_id': session_id,
'found': False,
}
session = session_row[0] if isinstance(session_row, tuple) else session_row
# Get messages for this session
messages_query = (
sqlalchemy.select(persistence_monitoring.MonitoringMessage)
.where(persistence_monitoring.MonitoringMessage.session_id == session_id)
.order_by(persistence_monitoring.MonitoringMessage.timestamp.asc())
)
messages_result = await self.ap.persistence_mgr.execute_async(messages_query)
messages_rows = messages_result.all()
# Count messages by status
success_messages = 0
error_messages = 0
pending_messages = 0
for row in messages_rows:
msg = row[0] if isinstance(row, tuple) else row
if msg.status == 'success':
success_messages += 1
elif msg.status == 'error':
error_messages += 1
elif msg.status == 'pending':
pending_messages += 1
# Get LLM calls for this session
llm_query = sqlalchemy.select(persistence_monitoring.MonitoringLLMCall).where(
persistence_monitoring.MonitoringLLMCall.session_id == session_id
)
llm_result = await self.ap.persistence_mgr.execute_async(llm_query)
llm_rows = llm_result.all()
# Calculate LLM statistics
total_llm_calls = len(llm_rows)
total_input_tokens = 0
total_output_tokens = 0
total_tokens = 0
total_duration = 0
success_llm_calls = 0
error_llm_calls = 0
for row in llm_rows:
llm_call = row[0] if isinstance(row, tuple) else row
total_input_tokens += llm_call.input_tokens
total_output_tokens += llm_call.output_tokens
total_tokens += llm_call.total_tokens
total_duration += llm_call.duration
if llm_call.status == 'success':
success_llm_calls += 1
else:
error_llm_calls += 1
# Get errors for this session
error_query = (
sqlalchemy.select(persistence_monitoring.MonitoringError)
.where(persistence_monitoring.MonitoringError.session_id == session_id)
.order_by(persistence_monitoring.MonitoringError.timestamp.desc())
)
error_result = await self.ap.persistence_mgr.execute_async(error_query)
error_rows = error_result.all()
errors = [
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringError, row[0] if isinstance(row, tuple) else row
)
for row in error_rows
]
# Calculate session duration
if messages_rows:
first_msg = messages_rows[0][0] if isinstance(messages_rows[0], tuple) else messages_rows[0]
last_msg = messages_rows[-1][0] if isinstance(messages_rows[-1], tuple) else messages_rows[-1]
session_duration_seconds = int((last_msg.timestamp - first_msg.timestamp).total_seconds())
else:
session_duration_seconds = 0
return {
'session_id': session_id,
'found': True,
'session': self.ap.persistence_mgr.serialize_model(persistence_monitoring.MonitoringSession, session),
'message_stats': {
'total': len(messages_rows),
'success': success_messages,
'error': error_messages,
'pending': pending_messages,
},
'llm_stats': {
'total_calls': total_llm_calls,
'success_calls': success_llm_calls,
'error_calls': error_llm_calls,
'total_input_tokens': total_input_tokens,
'total_output_tokens': total_output_tokens,
'total_tokens': total_tokens,
'average_duration_ms': int(total_duration / total_llm_calls) if total_llm_calls > 0 else 0,
},
'errors': errors,
'session_duration_seconds': session_duration_seconds,
}
async def get_message_details(
self,
message_id: str,
) -> dict:
"""Get detailed information for a specific message including associated LLM calls and errors"""
# Get message info
message_query = sqlalchemy.select(persistence_monitoring.MonitoringMessage).where(
persistence_monitoring.MonitoringMessage.id == message_id
)
message_result = await self.ap.persistence_mgr.execute_async(message_query)
message_row = message_result.first()
if not message_row:
return {
'message_id': message_id,
'found': False,
}
message = message_row[0] if isinstance(message_row, tuple) else message_row
# Get LLM calls for this message
llm_query = (
sqlalchemy.select(persistence_monitoring.MonitoringLLMCall)
.where(persistence_monitoring.MonitoringLLMCall.message_id == message_id)
.order_by(persistence_monitoring.MonitoringLLMCall.timestamp.asc())
)
llm_result = await self.ap.persistence_mgr.execute_async(llm_query)
llm_rows = llm_result.all()
llm_calls = [
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringLLMCall, row[0] if isinstance(row, tuple) else row
)
for row in llm_rows
]
# Calculate LLM statistics
total_input_tokens = sum(call.input_tokens for call in llm_rows)
total_output_tokens = sum(call.output_tokens for call in llm_rows)
total_tokens = sum(call.total_tokens for call in llm_rows)
total_duration = sum(call.duration for call in llm_rows)
# Get errors for this message
error_query = (
sqlalchemy.select(persistence_monitoring.MonitoringError)
.where(persistence_monitoring.MonitoringError.message_id == message_id)
.order_by(persistence_monitoring.MonitoringError.timestamp.asc())
)
error_result = await self.ap.persistence_mgr.execute_async(error_query)
error_rows = error_result.all()
errors = [
self.ap.persistence_mgr.serialize_model(
persistence_monitoring.MonitoringError, row[0] if isinstance(row, tuple) else row
)
for row in error_rows
]
return {
'message_id': message_id,
'found': True,
'message': self.ap.persistence_mgr.serialize_model(persistence_monitoring.MonitoringMessage, message),
'llm_calls': llm_calls,
'llm_stats': {
'total_calls': len(llm_rows),
'total_input_tokens': total_input_tokens,
'total_output_tokens': total_output_tokens,
'total_tokens': total_tokens,
'total_duration_ms': total_duration,
'average_duration_ms': int(total_duration / len(llm_rows)) if len(llm_rows) > 0 else 0,
},
'errors': errors,
}

View File

@@ -85,6 +85,15 @@ class PipelineService:
with open(template_path, 'r', encoding='utf-8') as f:
pipeline_data['config'] = json.load(f)
# Ensure extensions_preferences is set with enable_all_plugins and enable_all_mcp_servers=True by default
if 'extensions_preferences' not in pipeline_data:
pipeline_data['extensions_preferences'] = {
'enable_all_plugins': True,
'enable_all_mcp_servers': True,
'plugins': [],
'mcp_servers': [],
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_pipeline.LegacyPipeline).values(**pipeline_data)
)
@@ -142,8 +151,59 @@ class PipelineService:
)
await self.ap.pipeline_mgr.remove_pipeline(pipeline_uuid)
async def copy_pipeline(self, pipeline_uuid: str) -> str:
"""Copy a pipeline with all its configurations"""
# Get the original pipeline
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_pipeline.LegacyPipeline).where(
persistence_pipeline.LegacyPipeline.uuid == pipeline_uuid
)
)
original_pipeline = result.first()
if original_pipeline is None:
raise ValueError(f'Pipeline {pipeline_uuid} not found')
# Create new pipeline data
new_uuid = str(uuid.uuid4())
new_pipeline_data = {
'uuid': new_uuid,
'name': f'{original_pipeline.name} (Copy)',
'description': original_pipeline.description,
'for_version': self.ap.ver_mgr.get_current_version(),
'stages': original_pipeline.stages.copy() if original_pipeline.stages else default_stage_order.copy(),
'config': original_pipeline.config.copy() if original_pipeline.config else {},
'is_default': False,
'extensions_preferences': (
original_pipeline.extensions_preferences.copy()
if original_pipeline.extensions_preferences
else {
'enable_all_plugins': True,
'enable_all_mcp_servers': True,
'plugins': [],
'mcp_servers': [],
}
),
}
# Insert the new pipeline
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_pipeline.LegacyPipeline).values(**new_pipeline_data)
)
# Load the new pipeline
pipeline = await self.get_pipeline(new_uuid)
await self.ap.pipeline_mgr.load_pipeline(pipeline)
return new_uuid
async def update_pipeline_extensions(
self, pipeline_uuid: str, bound_plugins: list[dict], bound_mcp_servers: list[str] = None
self,
pipeline_uuid: str,
bound_plugins: list[dict],
bound_mcp_servers: list[str] = None,
enable_all_plugins: bool = True,
enable_all_mcp_servers: bool = True,
) -> None:
"""Update the bound plugins and MCP servers for a pipeline"""
# Get current pipeline
@@ -159,6 +219,8 @@ class PipelineService:
# Update extensions_preferences
extensions_preferences = pipeline.extensions_preferences or {}
extensions_preferences['enable_all_plugins'] = enable_all_plugins
extensions_preferences['enable_all_mcp_servers'] = enable_all_mcp_servers
extensions_preferences['plugins'] = bound_plugins
if bound_mcp_servers is not None:
extensions_preferences['mcp_servers'] = bound_mcp_servers

View File

@@ -0,0 +1,166 @@
from __future__ import annotations
import uuid
import sqlalchemy
from ....core import app
from ....entity.persistence import model as persistence_model
class ModelProviderService:
"""Service for managing model providers"""
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
async def get_providers(self) -> list[dict]:
"""Get all providers"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.ModelProvider))
providers = result.all()
providers_list = []
for p in providers:
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, p)
# Parse api_keys if it's a JSON string
if isinstance(provider_dict.get('api_keys'), str):
import json
try:
provider_dict['api_keys'] = json.loads(provider_dict['api_keys'])
except Exception:
provider_dict['api_keys'] = []
providers_list.append(provider_dict)
return providers_list
async def get_provider(self, provider_uuid: str) -> dict | None:
"""Get a single provider by UUID"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.uuid == provider_uuid
)
)
provider = result.first()
if provider is None:
return None
provider_dict = self.ap.persistence_mgr.serialize_model(persistence_model.ModelProvider, provider)
# Parse api_keys if it's a JSON string
if isinstance(provider_dict.get('api_keys'), str):
import json
try:
provider_dict['api_keys'] = json.loads(provider_dict['api_keys'])
except Exception:
provider_dict['api_keys'] = []
return provider_dict
async def create_provider(self, provider_data: dict) -> str:
"""Create a new provider"""
provider_data['uuid'] = str(uuid.uuid4())
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(persistence_model.ModelProvider).values(**provider_data)
)
# load to runtime
runtime_provider = await self.ap.model_mgr.load_provider(provider_data)
self.ap.model_mgr.provider_dict[runtime_provider.provider_entity.uuid] = runtime_provider
return provider_data['uuid']
async def update_provider(self, provider_uuid: str, provider_data: dict) -> None:
"""Update an existing provider"""
if 'uuid' in provider_data:
del provider_data['uuid']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_model.ModelProvider)
.where(persistence_model.ModelProvider.uuid == provider_uuid)
.values(**provider_data)
)
await self.ap.model_mgr.reload_provider(provider_uuid)
async def delete_provider(self, provider_uuid: str) -> None:
"""Delete a provider (only if no models reference it)"""
# Check if any models use this provider
llm_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.LLMModel).where(
persistence_model.LLMModel.provider_uuid == provider_uuid
)
)
if llm_result.first() is not None:
raise ValueError('Cannot delete provider: LLM models still reference it')
embedding_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.EmbeddingModel).where(
persistence_model.EmbeddingModel.provider_uuid == provider_uuid
)
)
if embedding_result.first() is not None:
raise ValueError('Cannot delete provider: Embedding models still reference it')
await self.ap.persistence_mgr.execute_async(
sqlalchemy.delete(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.uuid == provider_uuid
)
)
await self.ap.model_mgr.remove_provider(provider_uuid)
async def get_provider_model_counts(self, provider_uuid: str) -> dict:
"""Get count of models using this provider"""
llm_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(sqlalchemy.func.count())
.select_from(persistence_model.LLMModel)
.where(persistence_model.LLMModel.provider_uuid == provider_uuid)
)
llm_count = llm_result.scalar() or 0
embedding_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(sqlalchemy.func.count())
.select_from(persistence_model.EmbeddingModel)
.where(persistence_model.EmbeddingModel.provider_uuid == provider_uuid)
)
embedding_count = embedding_result.scalar() or 0
return {'llm_count': llm_count, 'embedding_count': embedding_count}
async def find_or_create_provider(self, requester: str, base_url: str, api_keys: list) -> str:
"""Find existing provider or create new one"""
# Try to find existing provider with same config
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.requester == requester,
persistence_model.ModelProvider.base_url == base_url,
)
)
for provider in result.all():
if sorted(provider.api_keys or []) == sorted(api_keys or []):
return provider.uuid
# Create new provider
provider_name = requester
if base_url:
try:
from urllib.parse import urlparse
parsed = urlparse(base_url)
provider_name = parsed.netloc or requester
except Exception:
pass
return await self.create_provider(
{
'name': provider_name,
'requester': requester,
'base_url': base_url,
'api_keys': api_keys or [],
}
)
async def update_space_model_provider_api_keys(self, api_key: str) -> None:
"""Update Space model provider API keys"""
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_model.ModelProvider)
.where(persistence_model.ModelProvider.uuid == '00000000-0000-0000-0000-000000000000')
.values(api_keys=[api_key])
)
await self.ap.model_mgr.reload_provider('00000000-0000-0000-0000-000000000000')

View File

@@ -0,0 +1,189 @@
from __future__ import annotations
import aiohttp
import typing
import datetime
import time
import sqlalchemy
from ....core import app
from ....entity.persistence import user
from ....entity.dto.space_model import SpaceModel
class SpaceService:
"""Service for interacting with LangBot Space API"""
ap: app.Application
_credits_cache: typing.Dict[str, typing.Tuple[int, float]] # {user_email: (credits, timestamp)}
def __init__(self, ap: app.Application) -> None:
self.ap = ap
self._credits_cache = {}
def _get_space_config(self) -> typing.Dict[str, str]:
"""Get Space configuration from config file"""
space_config = self.ap.instance_config.data.get('space', {})
return {
'url': space_config.get('url', 'https://space.langbot.app'),
'oauth_authorize_url': space_config.get('oauth_authorize_url', 'https://space.langbot.app/auth/authorize'),
}
async def _get_user_by_email(self, user_email: str) -> user.User | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(user.User).where(user.User.user == user_email)
)
result_list = result.all()
return result_list[0] if result_list else None
async def _ensure_valid_token(self, user_email: str) -> str | None:
"""Ensure access token is valid, refresh if expired. Returns valid access_token or None."""
user_obj = await self._get_user_by_email(user_email)
if not user_obj or user_obj.account_type != 'space':
return None
if not user_obj.space_access_token:
return None
# Check if token is expired (with 60s buffer)
if user_obj.space_access_token_expires_at:
if datetime.datetime.now() >= user_obj.space_access_token_expires_at - datetime.timedelta(seconds=60):
# Token expired, try to refresh
if user_obj.space_refresh_token:
try:
new_token = await self._refresh_and_save_token(user_obj)
return new_token
except Exception:
return None
return None
return user_obj.space_access_token
async def _refresh_and_save_token(self, user_obj: user.User) -> str:
"""Refresh token and save to database"""
token_data = await self.refresh_token(user_obj.space_refresh_token)
access_token = token_data.get('access_token')
expires_in = token_data.get('expires_in', 0)
if not access_token:
raise ValueError('Failed to refresh token')
expires_at = datetime.datetime.now() + datetime.timedelta(seconds=expires_in) if expires_in > 0 else None
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User)
.where(user.User.user == user_obj.user)
.values(
space_access_token=access_token,
space_access_token_expires_at=expires_at,
)
)
return access_token
# === Raw API calls (no token validation) ===
def get_oauth_authorize_url(self, redirect_uri: str, state: str = '') -> str:
"""Get the Space OAuth authorization URL for redirect"""
space_config = self._get_space_config()
authorize_url = space_config['oauth_authorize_url']
params = f'redirect_uri={redirect_uri}'
if state:
params += f'&state={state}'
return f'{authorize_url}?{params}'
async def exchange_oauth_code(self, code: str) -> typing.Dict:
"""Exchange OAuth authorization code for tokens"""
from langbot.pkg.utils import constants
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.post(
f'{space_url}/api/v1/accounts/oauth/token',
json={'code': code, 'instance_id': constants.instance_id},
) as response:
if response.status != 200:
raise ValueError(f'Failed to exchange OAuth code: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to exchange OAuth code: {data.get("msg")}')
return data.get('data', {})
async def refresh_token(self, refresh_token: str) -> typing.Dict:
"""Refresh Space access token"""
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.post(
f'{space_url}/api/v1/accounts/token/refresh', json={'refresh_token': refresh_token}
) as response:
if response.status != 200:
raise ValueError(f'Failed to refresh token: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to refresh token: {data.get("msg")}')
return data.get('data', {})
async def get_user_info_raw(self, access_token: str) -> typing.Dict:
"""Get user info from Space using access token (no validation)"""
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.get(
f'{space_url}/api/v1/accounts/me', headers={'Authorization': f'Bearer {access_token}'}
) as response:
if response.status != 200:
raise ValueError(f'Failed to get user info: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get user info: {data.get("msg")}')
return data.get('data', {})
# === API calls with token validation ===
async def get_user_info(self, user_email: str) -> typing.Dict | None:
"""Get user info from Space (with token validation)"""
access_token = await self._ensure_valid_token(user_email)
if not access_token:
return None
return await self.get_user_info_raw(access_token)
async def get_credits(self, user_email: str, force_refresh: bool = False) -> int | None:
"""Get Space credits for user with caching (60s TTL)"""
cache_ttl = 60
if not force_refresh and user_email in self._credits_cache:
credits, ts = self._credits_cache[user_email]
if time.time() - ts < cache_ttl:
return credits
try:
info = await self.get_user_info(user_email)
if info is None:
return None
credits = info.get('credits')
if credits is not None:
self._credits_cache[user_email] = (credits, time.time())
return credits
except Exception:
return self._credits_cache.get(user_email, (None, 0))[0]
async def get_models(self) -> typing.List[SpaceModel]:
"""Get models from Space"""
space_config = self._get_space_config()
space_url = space_config['url']
async with aiohttp.ClientSession() as session:
async with session.get(f'{space_url}/api/v1/models') as response:
if response.status != 200:
raise ValueError(f'Failed to get models: {await response.text()}')
data = await response.json()
if data.get('code') != 0:
raise ValueError(f'Failed to get models: {data.get("msg")}')
models_data = data.get('data', {}).get('models', [])
return [SpaceModel.model_validate(model_dict) for model_dict in models_data]

View File

@@ -4,17 +4,22 @@ import sqlalchemy
import argon2
import jwt
import datetime
import typing
import asyncio
from ....core import app
from ....entity.persistence import user
from ....utils import constants
from ....entity.errors import account as account_errors
class UserService:
ap: app.Application
_create_user_lock: asyncio.Lock
def __init__(self, ap: app.Application) -> None:
self.ap = ap
self._create_user_lock = asyncio.Lock()
async def is_initialized(self) -> bool:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(user.User).limit(1))
@@ -28,7 +33,7 @@ class UserService:
hashed_password = ph.hash(password)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(user.User).values(user=user_email, password=hashed_password)
sqlalchemy.insert(user.User).values(user=user_email, password=hashed_password, account_type='local')
)
async def get_user_by_email(self, user_email: str) -> user.User | None:
@@ -39,6 +44,15 @@ class UserService:
result_list = result.all()
return result_list[0] if result_list is not None and len(result_list) > 0 else None
async def get_user_by_space_account_uuid(self, space_account_uuid: str) -> user.User | None:
"""Get user by Space account UUID"""
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(user.User).where(user.User.space_account_uuid == space_account_uuid)
)
result_list = result.all()
return result_list[0] if result_list is not None and len(result_list) > 0 else None
async def authenticate(self, user_email: str, password: str) -> str | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(user.User).where(user.User.user == user_email)
@@ -51,6 +65,10 @@ class UserService:
user_obj = result_list[0]
# Check if this is a Space account
if user_obj.account_type == 'space':
raise ValueError('请使用 Space 账户登录')
ph = argon2.PasswordHasher()
ph.verify(user_obj.password, password)
@@ -90,6 +108,10 @@ class UserService:
if user_obj is None:
raise ValueError('User not found')
# Space accounts cannot change password locally
if user_obj.account_type == 'space':
raise ValueError('Space account cannot change password locally')
ph.verify(user_obj.password, current_password)
hashed_password = ph.hash(new_password)
@@ -97,3 +119,183 @@ class UserService:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User).where(user.User.user == user_email).values(password=hashed_password)
)
# Space user management
async def create_or_update_space_user(
self,
space_account_uuid: str,
email: str,
access_token: str,
refresh_token: str,
api_key: str,
expires_in: int = 0,
) -> user.User:
"""Create or update a Space user account (only if system not initialized or user exists)"""
expires_at = datetime.datetime.now() + datetime.timedelta(seconds=expires_in) if expires_in > 0 else None
async with self._create_user_lock:
# Check if user with this Space UUID already exists
existing_user = await self.get_user_by_space_account_uuid(space_account_uuid)
if existing_user:
# Update existing user's tokens
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User)
.where(user.User.space_account_uuid == space_account_uuid)
.values(
space_access_token=access_token,
space_refresh_token=refresh_token,
space_api_key=api_key,
space_access_token_expires_at=expires_at,
)
)
await self.ap.provider_service.update_space_model_provider_api_keys(api_key)
return await self.get_user_by_space_account_uuid(space_account_uuid)
# Check if user with same email exists
existing_email_user = await self.get_user_by_email(email)
if existing_email_user:
# Update existing user to link with Space account
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User)
.where(user.User.user == email)
.values(
account_type='space',
space_account_uuid=space_account_uuid,
space_access_token=access_token,
space_refresh_token=refresh_token,
space_api_key=api_key,
space_access_token_expires_at=expires_at,
)
)
await self.ap.provider_service.update_space_model_provider_api_keys(api_key)
return await self.get_user_by_email(email)
# Check if system is already initialized
is_initialized = await self.is_initialized()
if is_initialized:
raise account_errors.AccountEmailMismatchError()
# Create new Space user (first time initialization)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(user.User).values(
user=email,
password='', # Space users don't have local password
account_type='space',
space_account_uuid=space_account_uuid,
space_access_token=access_token,
space_refresh_token=refresh_token,
space_api_key=api_key,
space_access_token_expires_at=expires_at,
)
)
await self.ap.provider_service.update_space_model_provider_api_keys(api_key)
return await self.get_user_by_space_account_uuid(space_account_uuid)
async def authenticate_space_user(
self, access_token: str, refresh_token: str, expires_in: int = 0
) -> typing.Tuple[str, user.User]:
"""Authenticate with Space and return JWT token"""
# Get user info from Space using raw API (token just obtained, no need to validate)
user_info = await self.ap.space_service.get_user_info_raw(access_token)
account = user_info.get('account', {})
api_key = user_info.get('api_key', '')
space_account_uuid = account.get('uuid')
email = account.get('email')
if not space_account_uuid or not email:
raise ValueError('Invalid Space user info')
# Create or update Space user in local database
user_obj = await self.create_or_update_space_user(
space_account_uuid=space_account_uuid,
email=email,
access_token=access_token,
refresh_token=refresh_token,
api_key=api_key,
expires_in=expires_in,
)
# Generate JWT token
jwt_token = await self.generate_jwt_token(email)
return jwt_token, user_obj
async def get_first_user(self) -> user.User | None:
"""Get the first user (for single-user mode)"""
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(user.User).limit(1))
result_list = result.all()
return result_list[0] if result_list else None
async def set_password(self, user_email: str, new_password: str, current_password: str | None = None) -> None:
"""Set or change password for a user"""
ph = argon2.PasswordHasher()
user_obj = await self.get_user_by_email(user_email)
if user_obj is None:
raise ValueError('User not found')
# If user already has a password, verify current password
has_password = bool(user_obj.password and user_obj.password.strip())
if has_password:
if not current_password:
raise ValueError('Current password is required')
ph.verify(user_obj.password, current_password)
hashed_password = ph.hash(new_password)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User).where(user.User.user == user_email).values(password=hashed_password)
)
async def bind_space_account(self, user_email: str, code: str) -> user.User:
"""Bind Space account to existing local account"""
# Exchange code for tokens
token_data = await self.ap.space_service.exchange_oauth_code(code)
access_token = token_data.get('access_token')
refresh_token = token_data.get('refresh_token')
expires_in = token_data.get('expires_in', 0)
if not access_token:
raise ValueError('Failed to get access token from Space')
expires_at = datetime.datetime.now() + datetime.timedelta(seconds=expires_in) if expires_in > 0 else None
# Get Space user info (token just obtained, use raw API)
user_info = await self.ap.space_service.get_user_info_raw(access_token)
account = user_info.get('account', {})
api_key = user_info.get('api_key', '')
space_account_uuid = account.get('uuid')
space_email = account.get('email')
if not space_account_uuid or not space_email:
raise ValueError('Invalid Space user info')
# Check if this Space account is already bound to another user
existing_space_user = await self.get_user_by_space_account_uuid(space_account_uuid)
if existing_space_user and existing_space_user.user != user_email:
raise ValueError('This Space account is already bound to another user')
# Update local account to Space account
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(user.User)
.where(user.User.user == user_email)
.values(
user=space_email, # Update email to Space email
account_type='space',
space_account_uuid=space_account_uuid,
space_access_token=access_token,
space_refresh_token=refresh_token,
space_api_key=api_key,
space_access_token_expires_at=expires_at,
)
)
# Update Space model provider API keys
await self.ap.provider_service.update_space_model_provider_api_keys(api_key)
return await self.get_user_by_email(space_email)

View File

@@ -19,13 +19,17 @@ from ..utils import version as version_mgr, proxy as proxy_mgr
from ..persistence import mgr as persistencemgr
from ..api.http.controller import main as http_controller
from ..api.http.service import user as user_service
from ..api.http.service import space as space_service
from ..api.http.service import model as model_service
from ..api.http.service import provider as provider_service
from ..api.http.service import pipeline as pipeline_service
from ..api.http.service import bot as bot_service
from ..api.http.service import knowledge as knowledge_service
from ..api.http.service import mcp as mcp_service
from ..api.http.service import apikey as apikey_service
from ..api.http.service import webhook as webhook_service
from ..api.http.service import external_kb as external_kb_service
from ..api.http.service import monitoring as monitoring_service
from ..discover import engine as discover_engine
from ..storage import mgr as storagemgr
from ..utils import logcache
@@ -33,6 +37,7 @@ from . import taskmgr
from . import entities as core_entities
from ..rag.knowledge import kbmgr as rag_mgr
from ..vector import mgr as vectordb_mgr
from ..telemetry import telemetry as telemetry_module
class Application:
@@ -74,6 +79,8 @@ class Application:
instance_config: config_mgr.ConfigManager = None
instance_id: config_mgr.ConfigManager = None # used to identify the instance
# ======= Metadata config manager =======
sensitive_meta: config_mgr.ConfigManager = None
@@ -113,22 +120,32 @@ class Application:
user_service: user_service.UserService = None
space_service: space_service.SpaceService = None
llm_model_service: model_service.LLMModelsService = None
embedding_models_service: model_service.EmbeddingModelsService = None
provider_service: provider_service.ModelProviderService = None
pipeline_service: pipeline_service.PipelineService = None
bot_service: bot_service.BotService = None
knowledge_service: knowledge_service.KnowledgeService = None
external_kb_service: external_kb_service.ExternalKBService = None
mcp_service: mcp_service.MCPService = None
apikey_service: apikey_service.ApiKeyService = None
webhook_service: webhook_service.WebhookService = None
telemetry: telemetry_module.TelemetryManager = None
monitoring_service: monitoring_service.MonitoringService = None
def __init__(self):
pass

View File

@@ -1,4 +1,5 @@
import logging
import logging.handlers
import sys
import time
@@ -15,6 +16,10 @@ log_colors_config = {
'CRITICAL': 'cyan',
}
# Log rotation configuration to prevent unbounded log file growth
LOG_FILE_MAX_BYTES = 10 * 1024 * 1024 # 10MB per file
LOG_FILE_BACKUP_COUNT = 5 # Keep 5 backup files (total ~50MB max)
async def init_logging(extra_handlers: list[logging.Handler] = None) -> logging.Logger:
# Remove all existing loggers
@@ -43,9 +48,17 @@ async def init_logging(extra_handlers: list[logging.Handler] = None) -> logging.
# stream_handler.setFormatter(color_formatter)
stream_handler.stream = open(sys.stdout.fileno(), mode='w', encoding='utf-8', buffering=1)
# Use RotatingFileHandler to prevent unbounded log file growth
rotating_file_handler = logging.handlers.RotatingFileHandler(
log_file_name,
encoding='utf-8',
maxBytes=LOG_FILE_MAX_BYTES,
backupCount=LOG_FILE_BACKUP_COUNT,
)
log_handlers: list[logging.Handler] = [
stream_handler,
logging.FileHandler(log_file_name, encoding='utf-8'),
rotating_file_handler,
]
log_handlers += extra_handlers if extra_handlers is not None else []

View File

@@ -16,18 +16,23 @@ from ...platform.webhook_pusher import WebhookPusher
from ...persistence import mgr as persistencemgr
from ...api.http.controller import main as http_controller
from ...api.http.service import user as user_service
from ...api.http.service import space as space_service
from ...api.http.service import model as model_service
from ...api.http.service import provider as provider_service
from ...api.http.service import pipeline as pipeline_service
from ...api.http.service import bot as bot_service
from ...api.http.service import knowledge as knowledge_service
from ...api.http.service import mcp as mcp_service
from ...api.http.service import apikey as apikey_service
from ...api.http.service import webhook as webhook_service
from ...api.http.service import external_kb as external_kb_service
from ...api.http.service import monitoring as monitoring_service
from ...discover import engine as discover_engine
from ...storage import mgr as storagemgr
from ...utils import logcache
from ...vector import mgr as vectordb_mgr
from .. import taskmgr
from ...telemetry import telemetry as telemetry_module
@stage.stage_class('BuildAppStage')
@@ -42,6 +47,42 @@ class BuildAppStage(stage.BootingStage):
discover.discover_blueprint('templates/components.yaml')
ap.discover = discover
user_service_inst = user_service.UserService(ap)
ap.user_service = user_service_inst
space_service_inst = space_service.SpaceService(ap)
ap.space_service = space_service_inst
llm_model_service_inst = model_service.LLMModelsService(ap)
ap.llm_model_service = llm_model_service_inst
embedding_models_service_inst = model_service.EmbeddingModelsService(ap)
ap.embedding_models_service = embedding_models_service_inst
provider_service_inst = provider_service.ModelProviderService(ap)
ap.provider_service = provider_service_inst
pipeline_service_inst = pipeline_service.PipelineService(ap)
ap.pipeline_service = pipeline_service_inst
bot_service_inst = bot_service.BotService(ap)
ap.bot_service = bot_service_inst
knowledge_service_inst = knowledge_service.KnowledgeService(ap)
ap.knowledge_service = knowledge_service_inst
external_kb_service_inst = external_kb_service.ExternalKBService(ap)
ap.external_kb_service = external_kb_service_inst
mcp_service_inst = mcp_service.MCPService(ap)
ap.mcp_service = mcp_service_inst
apikey_service_inst = apikey_service.ApiKeyService(ap)
ap.apikey_service = apikey_service_inst
webhook_service_inst = webhook_service.WebhookService(ap)
ap.webhook_service = webhook_service_inst
proxy_mgr = proxy.ProxyManager(ap)
await proxy_mgr.initialize()
ap.proxy_mgr = proxy_mgr
@@ -63,21 +104,18 @@ class BuildAppStage(stage.BootingStage):
ap.persistence_mgr = persistence_mgr_inst
await persistence_mgr_inst.initialize()
async def runtime_disconnect_callback(connector: plugin_connector.PluginRuntimeConnector) -> None:
await asyncio.sleep(3)
await plugin_connector_inst.initialize()
plugin_connector_inst = plugin_connector.PluginRuntimeConnector(ap, runtime_disconnect_callback)
await plugin_connector_inst.initialize()
ap.plugin_connector = plugin_connector_inst
# Telemetry manager: attach to app so other components can call via self.ap.telemetry
telemetry_inst = telemetry_module.TelemetryManager(ap)
await telemetry_inst.initialize()
ap.telemetry = telemetry_inst
cmd_mgr_inst = cmdmgr.CommandManager(ap)
await cmd_mgr_inst.initialize()
ap.cmd_mgr = cmd_mgr_inst
llm_model_mgr_inst = llm_model_mgr.ModelManager(ap)
await llm_model_mgr_inst.initialize()
ap.model_mgr = llm_model_mgr_inst
await llm_model_mgr_inst.initialize()
llm_session_mgr_inst = llm_session_mgr.SessionManager(ap)
await llm_session_mgr_inst.initialize()
@@ -112,32 +150,16 @@ class BuildAppStage(stage.BootingStage):
await http_ctrl.initialize()
ap.http_ctrl = http_ctrl
user_service_inst = user_service.UserService(ap)
ap.user_service = user_service_inst
monitoring_service_inst = monitoring_service.MonitoringService(ap)
ap.monitoring_service = monitoring_service_inst
llm_model_service_inst = model_service.LLMModelsService(ap)
ap.llm_model_service = llm_model_service_inst
async def runtime_disconnect_callback(connector: plugin_connector.PluginRuntimeConnector) -> None:
await asyncio.sleep(3)
await plugin_connector_inst.initialize()
embedding_models_service_inst = model_service.EmbeddingModelsService(ap)
ap.embedding_models_service = embedding_models_service_inst
pipeline_service_inst = pipeline_service.PipelineService(ap)
ap.pipeline_service = pipeline_service_inst
bot_service_inst = bot_service.BotService(ap)
ap.bot_service = bot_service_inst
knowledge_service_inst = knowledge_service.KnowledgeService(ap)
ap.knowledge_service = knowledge_service_inst
mcp_service_inst = mcp_service.MCPService(ap)
ap.mcp_service = mcp_service_inst
apikey_service_inst = apikey_service.ApiKeyService(ap)
ap.apikey_service = apikey_service_inst
webhook_service_inst = webhook_service.WebhookService(ap)
ap.webhook_service = webhook_service_inst
plugin_connector_inst = plugin_connector.PluginRuntimeConnector(ap, runtime_disconnect_callback)
await plugin_connector_inst.initialize()
ap.plugin_connector = plugin_connector_inst
ctrl = controller.Controller(ap)
ap.ctrl = ctrl

View File

@@ -2,8 +2,11 @@ from __future__ import annotations
import os
from typing import Any
from langbot.pkg.utils import constants
import yaml
import importlib.resources as resources
import uuid
import time
from .. import stage, app
from ..bootutils import config
@@ -142,6 +145,22 @@ class LoadConfigStage(stage.BootingStage):
await ap.instance_config.dump_config()
# load or generate instance id
ap.instance_id = await config.load_json_config(
'data/labels/instance_id.json',
template_data={
'instance_id': f'instance_{str(uuid.uuid4())}',
'instance_create_ts': int(time.time()),
},
completion=False,
)
constants.instance_id = ap.instance_id.data['instance_id']
print(f'LangBot instance id: {constants.instance_id}')
await ap.instance_id.dump_config()
ap.sensitive_meta = await config.load_json_config(
'data/metadata/sensitive-words.json',
'metadata/sensitive-words.json',

View File

@@ -0,0 +1,49 @@
# [
# {
# "uuid": "7652ebdb-54dc-412c-a830-e9268ac88471",
# "model_id": "claude-opus-4-5-20251101",
# "display_name": {
# "en_US": "claude-opus-4-5-20251101",
# "zh_Hans": "claude-opus-4-5-20251101"
# },
# "description": {},
# "provider": "anthropic",
# "category": "chat",
# "icon_url": "Claude.Color",
# "tags": {},
# "is_featured": true,
# "featured_order": 999,
# "model_ratio": 2.5,
# "completion_ratio": 5,
# "quota_type": 0,
# "model_price": 0,
# "input_credits": 500,
# "output_credits": 2500,
# "vendor_id": 1,
# "vendor_name": "Anthropic",
# "vendor_icon": "Claude.Color",
# "supported_endpoints": [
# "anthropic",
# "openai"
# ],
# "status": "active",
# "metadata": null,
# "created_at": "2025-12-30T22:23:38.337207+08:00",
# "updated_at": "2025-12-30T22:23:38.337207+08:00"
# }
# ]
import pydantic
class SpaceModel(pydantic.BaseModel):
uuid: str
model_id: str
provider: str
category: str # chat / embedding
llm_abilities: list[str] | None = None
is_featured: bool = False
featured_order: int = 0
status: str
created_at: str | None = None
updated_at: str | None = None

View File

@@ -0,0 +1,6 @@
from __future__ import annotations
class AccountEmailMismatchError(Exception):
def __str__(self):
return 'Account email mismatch'

View File

@@ -7,3 +7,11 @@ class RequesterNotFoundError(Exception):
def __str__(self):
return f'Requester {self.requester_name} not found'
class ProviderNotFoundError(Exception):
def __init__(self, provider_name: str):
self.provider_name = provider_name
def __str__(self):
return f'Provider {self.provider_name} not found'

View File

@@ -9,7 +9,7 @@ class MCPServer(Base):
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
enable = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False)
mode = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) # stdio, sse
mode = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) # stdio, sse, http
extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(

View File

@@ -3,6 +3,25 @@ import sqlalchemy
from .base import Base
class ModelProvider(Base):
"""Model provider"""
__tablename__ = 'model_providers'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
requester = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
base_url = sqlalchemy.Column(sqlalchemy.String(512), nullable=False)
api_keys = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=[])
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,
nullable=False,
server_default=sqlalchemy.func.now(),
onupdate=sqlalchemy.func.now(),
)
class LLMModel(Base):
"""LLM model"""
@@ -10,12 +29,10 @@ class LLMModel(Base):
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
description = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
requester = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
requester_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
api_keys = sqlalchemy.Column(sqlalchemy.JSON, nullable=False)
provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
abilities = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default=[])
extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
prefered_ranking = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,
@@ -26,17 +43,15 @@ class LLMModel(Base):
class EmbeddingModel(Base):
"""Embedding 模型"""
"""Embedding model"""
__tablename__ = 'embedding_models'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
description = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
requester = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
requester_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
api_keys = sqlalchemy.Column(sqlalchemy.JSON, nullable=False)
provider_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
prefered_ranking = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,

View File

@@ -0,0 +1,105 @@
import sqlalchemy
from .base import Base
class MonitoringMessage(Base):
"""Monitoring message records"""
__tablename__ = 'monitoring_messages'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
message_content = sqlalchemy.Column(sqlalchemy.Text, nullable=False)
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
status = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # success, error, pending
level = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # info, warning, error, debug
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
runner_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) # Runner name for this query
variables = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # Query variables as JSON string
class MonitoringLLMCall(Base):
"""LLM call records"""
__tablename__ = 'monitoring_llm_calls'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
model_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
input_tokens = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
output_tokens = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
total_tokens = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
duration = sqlalchemy.Column(sqlalchemy.Integer, nullable=False) # milliseconds
cost = sqlalchemy.Column(sqlalchemy.Float, nullable=True)
status = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # success, error
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
error_message = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True) # Associated message ID
class MonitoringSession(Base):
"""Session tracking records"""
__tablename__ = 'monitoring_sessions'
session_id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
message_count = sqlalchemy.Column(sqlalchemy.Integer, nullable=False, default=0)
start_time = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
last_activity = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
is_active = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=True, index=True)
platform = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
user_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
class MonitoringError(Base):
"""Error log records"""
__tablename__ = 'monitoring_errors'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
error_type = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
error_message = sqlalchemy.Column(sqlalchemy.Text, nullable=False)
bot_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
bot_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
pipeline_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=False, index=True)
pipeline_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
stack_trace = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True) # Associated message ID
class MonitoringEmbeddingCall(Base):
"""Embedding call records"""
__tablename__ = 'monitoring_embedding_calls'
id = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True)
timestamp = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, index=True)
model_name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
prompt_tokens = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
total_tokens = sqlalchemy.Column(sqlalchemy.Integer, nullable=False)
duration = sqlalchemy.Column(sqlalchemy.Integer, nullable=False) # milliseconds
input_count = sqlalchemy.Column(sqlalchemy.Integer, nullable=False) # Number of input texts
status = sqlalchemy.Column(sqlalchemy.String(50), nullable=False) # success, error
error_message = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
# Optional context fields
knowledge_base_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
query_text = sqlalchemy.Column(sqlalchemy.Text, nullable=True) # For retrieval calls
session_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
message_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True, index=True)
call_type = sqlalchemy.Column(sqlalchemy.String(50), nullable=True) # embedding, retrieve

View File

@@ -11,6 +11,7 @@ class LegacyPipeline(Base):
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
description = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='⚙️')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,
@@ -22,7 +23,11 @@ class LegacyPipeline(Base):
is_default = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False)
stages = sqlalchemy.Column(sqlalchemy.JSON, nullable=False)
config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False)
extensions_preferences = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
extensions_preferences = sqlalchemy.Column(
sqlalchemy.JSON,
nullable=False,
default={'enable_all_plugins': True, 'enable_all_mcp_servers': True, 'plugins': [], 'mcp_servers': []},
)
class PipelineRunRecord(Base):

View File

@@ -1,27 +1,15 @@
import sqlalchemy
from .base import Base
# Base = declarative_base()
# DATABASE_URL = os.getenv('DATABASE_URL', 'sqlite:///./rag_knowledge.db')
# print("Using database URL:", DATABASE_URL)
# engine = create_engine(DATABASE_URL, connect_args={'check_same_thread': False})
# SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
# def create_db_and_tables():
# """Creates all database tables defined in the Base."""
# Base.metadata.create_all(bind=engine)
# print('Database tables created or already exist.')
class KnowledgeBase(Base):
__tablename__ = 'knowledge_bases'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String, index=True)
description = sqlalchemy.Column(sqlalchemy.Text)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='📚')
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now(), onupdate=sqlalchemy.func.now())
embedding_model_uuid = sqlalchemy.Column(sqlalchemy.String, default='')
top_k = sqlalchemy.Column(sqlalchemy.Integer, default=5)
@@ -43,8 +31,14 @@ class Chunk(Base):
text = sqlalchemy.Column(sqlalchemy.Text)
# class Vector(Base):
# __tablename__ = 'knowledge_base_vectors'
# uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
# chunk_id = sqlalchemy.Column(sqlalchemy.String, nullable=True)
# embedding = sqlalchemy.Column(sqlalchemy.LargeBinary)
class ExternalKnowledgeBase(Base):
__tablename__ = 'external_knowledge_bases'
uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True)
name = sqlalchemy.Column(sqlalchemy.String, index=True)
description = sqlalchemy.Column(sqlalchemy.Text)
emoji = sqlalchemy.Column(sqlalchemy.String(10), nullable=True, default='🔗')
plugin_author = sqlalchemy.Column(sqlalchemy.String, nullable=False)
plugin_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_name = sqlalchemy.Column(sqlalchemy.String, nullable=False)
retriever_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={})
created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now())

View File

@@ -9,6 +9,17 @@ class User(Base):
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True)
user = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
password = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
# Account type: 'local' (default) or 'space'
account_type = sqlalchemy.Column(sqlalchemy.String(32), nullable=False, server_default='local')
# Space account fields (nullable, only used when account_type='space')
space_account_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
space_access_token = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
space_refresh_token = sqlalchemy.Column(sqlalchemy.Text, nullable=True)
space_access_token_expires_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=True)
space_api_key = sqlalchemy.Column(sqlalchemy.String(255), nullable=True)
created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now())
updated_at = sqlalchemy.Column(
sqlalchemy.DateTime,

View File

@@ -1,13 +0,0 @@
from __future__ import annotations
import pydantic
from typing import Any
class RetrieveResultEntry(pydantic.BaseModel):
id: str
metadata: dict[str, Any]
distance: float

View File

@@ -9,7 +9,7 @@ import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
import sqlalchemy
from . import database, migration
from ..entity.persistence import base, pipeline, metadata
from ..entity.persistence import base, pipeline, metadata, model as persistence_model
from ..entity import persistence
from ..core import app
from ..utils import constants, importutil
@@ -79,6 +79,7 @@ class PersistenceManager:
self.ap.logger.info(f'Successfully upgraded database to version {last_migration_number}.')
await self.write_default_pipeline()
await self.write_space_model_providers()
async def create_tables(self):
# create tables
@@ -123,7 +124,42 @@ class PersistenceManager:
await self.execute_async(sqlalchemy.insert(pipeline.LegacyPipeline).values(pipeline_data))
# =================================
async def write_space_model_providers(self):
space_models_gateway_api_url = self.ap.instance_config.data.get('space', {}).get(
'models_gateway_api_url', 'https://api.langbot.cloud/v1'
)
# write space model providers
result = await self.execute_async(
sqlalchemy.select(persistence_model.ModelProvider).where(
persistence_model.ModelProvider.requester == 'space-chat-completions'
)
)
exists_space_chat_completions_model_provider = result.first()
# api keys will be set/updated when the oauth callback
if exists_space_chat_completions_model_provider is None:
self.ap.logger.info('Creating space model providers...')
space_chat_completions_model_provider = {
'uuid': '00000000-0000-0000-0000-000000000000',
'name': 'LangBot Models',
'requester': 'space-chat-completions',
'base_url': space_models_gateway_api_url,
'api_keys': [],
}
await self.execute_async(
sqlalchemy.insert(persistence_model.ModelProvider).values(space_chat_completions_model_provider)
)
else:
if exists_space_chat_completions_model_provider.base_url != space_models_gateway_api_url:
await self.execute_async(
sqlalchemy.update(persistence_model.ModelProvider)
.where(persistence_model.ModelProvider.uuid == exists_space_chat_completions_model_provider.uuid)
.values({'base_url': space_models_gateway_api_url})
)
# =================================
async def execute_async(self, *args, **kwargs) -> sqlalchemy.engine.cursor.CursorResult:
async with self.get_db_engine().connect() as conn:

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(2)
@@ -11,30 +10,45 @@ class DBMigrateCombineQuoteMsgConfig(migration.DBMigration):
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'trigger' exists
if 'trigger' not in config:
config['trigger'] = {}
# Ensure 'misc' exists in 'trigger'
if 'misc' not in config['trigger']:
config['trigger']['misc'] = {}
# Add 'combine-quote-message' if not exists
if 'combine-quote-message' not in config['trigger']['misc']:
config['trigger']['misc']['combine-quote-message'] = False
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""Downgrade"""

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(3)
@@ -11,14 +10,23 @@ class DBMigrateN8nConfig(migration.DBMigration):
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'ai' exists
if 'ai' not in config:
config['ai'] = {}
# Add 'n8n-service-api' if not exists
if 'n8n-service-api' not in config['ai']:
config['ai']['n8n-service-api'] = {
'webhook-url': 'http://your-n8n-webhook-url',
@@ -33,16 +41,21 @@ class DBMigrateN8nConfig(migration.DBMigration):
'output-key': 'response',
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""Downgrade"""

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(4)
@@ -11,27 +10,43 @@ class DBMigrateRAGKBUUID(migration.DBMigration):
async def upgrade(self):
"""升级"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'ai' not in config:
config['ai'] = {}
if 'local-agent' not in config['ai']:
config['ai']['local-agent'] = {}
# Add 'knowledge-base' if not exists
if 'knowledge-base' not in config['ai']['local-agent']:
config['ai']['local-agent']['knowledge-base'] = ''
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""降级"""

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(5)
@@ -11,27 +10,43 @@ class DBMigratePipelineRemoveCotConfig(migration.DBMigration):
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'output' not in config:
config['output'] = {}
if 'misc' not in config['output']:
config['output']['misc'] = {}
# Add 'remove-think' if not exists
if 'remove-think' not in config['output']['misc']:
config['output']['misc']['remove-think'] = False
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""Downgrade"""

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(6)
@@ -11,14 +10,23 @@ class DBMigrateLangflowApiConfig(migration.DBMigration):
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure 'ai' exists
if 'ai' not in config:
config['ai'] = {}
# Add 'langflow-api' if not exists
if 'langflow-api' not in config['ai']:
config['ai']['langflow-api'] = {
'base-url': 'http://localhost:7860',
@@ -29,16 +37,21 @@ class DBMigrateLangflowApiConfig(migration.DBMigration):
'tweaks': '{}',
}
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""Downgrade"""

View File

@@ -1,8 +1,7 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(10)
@@ -11,16 +10,20 @@ class DBMigratePipelineMultiKnowledgeBase(migration.DBMigration):
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Convert knowledge-base from string to array
if 'local-agent' in config['ai']:
if 'ai' in config and 'local-agent' in config['ai']:
current_kb = config['ai']['local-agent'].get('knowledge-base', '')
# If it's already a list, skip
@@ -37,29 +40,38 @@ class DBMigratePipelineMultiKnowledgeBase(migration.DBMigration):
if 'knowledge-base' in config['ai']['local-agent']:
del config['ai']['local-agent']['knowledge-base']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
)
)
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
async def downgrade(self):
"""Downgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Convert knowledge-bases from array back to string
if 'local-agent' in config['ai']:
if 'ai' in config and 'local-agent' in config['ai']:
current_kbs = config['ai']['local-agent'].get('knowledge-bases', [])
# If it's already a string, skip
@@ -76,13 +88,18 @@ class DBMigratePipelineMultiKnowledgeBase(migration.DBMigration):
if 'knowledge-bases' in config['ai']['local-agent']:
del config['ai']['local-agent']['knowledge-bases']
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
)
)
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)

View File

@@ -1,39 +1,54 @@
from .. import migration
import sqlalchemy
from ...entity.persistence import pipeline as persistence_pipeline
import json
@migration.migration_class(11)
class DBMigrateDifyApiConfig(migration.DBMigration):
"""Langflow API config"""
"""Dify base prompt config"""
async def upgrade(self):
"""Upgrade"""
# read all pipelines
pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline))
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, config FROM legacy_pipelines')
)
pipelines = result.fetchall()
for pipeline in pipelines:
serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline)
current_version = self.ap.ver_mgr.get_current_version()
config = serialized_pipeline['config']
for pipeline_row in pipelines:
uuid = pipeline_row[0]
config = json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
# Ensure nested structure exists
if 'ai' not in config:
config['ai'] = {}
if 'dify-service-api' not in config['ai']:
config['ai']['dify-service-api'] = {}
# Add 'base-prompt' if not exists
if 'base-prompt' not in config['ai']['dify-service-api']:
config['ai']['dify-service-api']['base-prompt'] = (
'When the file content is readable, please read the content of this file. When the file is an image, describe the content of this image.',
)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_pipeline.LegacyPipeline)
.where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid'])
.values(
{
'config': config,
'for_version': self.ap.ver_mgr.get_current_version(),
}
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET config = :config, for_version = :for_version WHERE uuid = :uuid'
),
{'config': json.dumps(config), 'for_version': current_version, 'uuid': uuid},
)
)
async def downgrade(self):
"""Downgrade"""

View File

@@ -0,0 +1,73 @@
from .. import migration
import sqlalchemy
import json
@migration.migration_class(12)
class DBMigratePipelineExtensionsEnableAll(migration.DBMigration):
"""Pipeline extensions enable all"""
async def upgrade(self):
"""Upgrade"""
# Read all pipelines using raw SQL
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, extensions_preferences FROM legacy_pipelines')
)
pipelines = result.fetchall()
current_version = self.ap.ver_mgr.get_current_version()
for pipeline_row in pipelines:
uuid = pipeline_row[0]
extensions_preferences = (
json.loads(pipeline_row[1]) if isinstance(pipeline_row[1], str) else pipeline_row[1]
)
# Ensure extensions_preferences is a dict
if extensions_preferences is None:
extensions_preferences = {}
# Add 'enable_all_plugins' if not exists
if 'enable_all_plugins' not in extensions_preferences:
if 'plugins' in extensions_preferences:
extensions_preferences['enable_all_plugins'] = False
else:
extensions_preferences['enable_all_plugins'] = True
extensions_preferences['plugins'] = []
# Add 'enable_all_mcp_servers' if not exists
if 'enable_all_mcp_servers' not in extensions_preferences:
if 'mcp_servers' in extensions_preferences:
extensions_preferences['enable_all_mcp_servers'] = False
else:
extensions_preferences['enable_all_mcp_servers'] = True
extensions_preferences['mcp_servers'] = []
# Update using raw SQL with compatibility for both SQLite and PostgreSQL
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET extensions_preferences = :extensions_preferences::jsonb, for_version = :for_version WHERE uuid = :uuid'
),
{
'extensions_preferences': json.dumps(extensions_preferences),
'for_version': current_version,
'uuid': uuid,
},
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'UPDATE legacy_pipelines SET extensions_preferences = :extensions_preferences, for_version = :for_version WHERE uuid = :uuid'
),
{
'extensions_preferences': json.dumps(extensions_preferences),
'for_version': current_version,
'uuid': uuid,
},
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,49 @@
import sqlalchemy
from .. import migration
@migration.migration_class(13)
class DBMigrateKnowledgeBaseUpdatedAt(migration.DBMigration):
"""Add updated_at field to knowledge_bases table"""
async def upgrade(self):
"""Upgrade"""
# Get all column names from the table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
"SELECT column_name FROM information_schema.columns WHERE table_name = 'knowledge_bases';"
)
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('PRAGMA table_info(knowledge_bases);'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Check and add updated_at column
if 'updated_at' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
'ALTER TABLE knowledge_bases ADD COLUMN updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP'
)
)
else:
# SQLite doesn't support DEFAULT CURRENT_TIMESTAMP in ALTER TABLE
# Add column without default first
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE knowledge_bases ADD COLUMN updated_at DATETIME')
)
# Set initial updated_at values to created_at for existing records
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE knowledge_bases SET updated_at = created_at WHERE updated_at IS NULL')
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,94 @@
import sqlalchemy
from .. import migration
@migration.migration_class(14)
class DBMigrateSpaceAccountSupport(migration.DBMigration):
"""Add Space account support fields to users table"""
async def upgrade(self):
"""Upgrade"""
# Get all column names from the users table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("SELECT column_name FROM information_schema.columns WHERE table_name = 'users';")
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text('PRAGMA table_info(users);'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Add account_type column
if 'account_type' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE users ADD COLUMN account_type VARCHAR(32) DEFAULT 'local' NOT NULL")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("ALTER TABLE users ADD COLUMN account_type VARCHAR(32) DEFAULT 'local' NOT NULL")
)
# Add space_account_uuid column
if 'space_account_uuid' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_account_uuid VARCHAR(255)')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_account_uuid VARCHAR(255)')
)
# Add space_access_token column
if 'space_access_token' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token TEXT')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token TEXT')
)
# Add space_refresh_token column
if 'space_refresh_token' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_refresh_token TEXT')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_refresh_token TEXT')
)
# Add space_access_token_expires_at column
if 'space_access_token_expires_at' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token_expires_at TIMESTAMP')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_access_token_expires_at DATETIME')
)
# Add space_api_key column
if 'space_api_key' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_api_key VARCHAR(255)')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE users ADD COLUMN space_api_key VARCHAR(255)')
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,15 @@
from .. import migration
# this is a deprecated migration
@migration.migration_class(15)
class DBMigrateModelSourceTracking(migration.DBMigration):
"""Add source tracking fields to models tables for Space integration"""
async def upgrade(self):
"""Upgrade"""
pass
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,305 @@
import uuid as uuid_lib
import sqlalchemy
from .. import migration
@migration.migration_class(16)
class DBMigrateModelProviderRefactor(migration.DBMigration):
"""Refactor model structure: create providers from existing models and update references"""
async def upgrade(self):
"""Upgrade"""
# Step 1: Create model_providers table if not exists
await self._create_providers_table()
# Step 2: Migrate existing models to use providers
await self._migrate_llm_models()
await self._migrate_embedding_models()
# Step 3: Remove deprecated columns
await self._cleanup_columns()
async def _create_providers_table(self):
"""Create model_providers table"""
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS model_providers (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
requester VARCHAR(255) NOT NULL,
base_url VARCHAR(512) NOT NULL,
api_keys JSONB NOT NULL DEFAULT '[]',
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
)
""")
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
CREATE TABLE IF NOT EXISTS model_providers (
uuid VARCHAR(255) PRIMARY KEY,
name VARCHAR(255) NOT NULL,
requester VARCHAR(255) NOT NULL,
base_url VARCHAR(512) NOT NULL,
api_keys JSON NOT NULL DEFAULT '[]',
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
)
""")
)
async def _migrate_llm_models(self):
"""Migrate LLM models to use providers"""
llm_columns = await self._get_columns('llm_models')
# Add provider_uuid column if not exists
if 'provider_uuid' not in llm_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN provider_uuid VARCHAR(255)')
)
# Add prefered_ranking column if not exists
if 'prefered_ranking' not in llm_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE llm_models ADD COLUMN prefered_ranking INTEGER NOT NULL DEFAULT 0')
)
# Only migrate if old columns exist
if 'requester' not in llm_columns:
return
# Get all LLM models with old structure
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, name, requester, requester_config, api_keys FROM llm_models')
)
models = result.fetchall()
# Create providers and update models
provider_cache = {} # (requester, base_url, api_keys_str) -> provider_uuid
for model in models:
model_uuid, model_name, requester, requester_config, api_keys = model
# Extract base_url from requester_config
base_url = ''
if requester_config:
if isinstance(requester_config, str):
import json
requester_config = json.loads(requester_config)
base_url = requester_config.get('base_url', '') or requester_config.get('base-url', '')
# Parse api_keys if it's a string
if isinstance(api_keys, str):
import json
try:
api_keys = json.loads(api_keys)
except Exception:
api_keys = []
if not api_keys:
api_keys = []
# Create cache key
api_keys_str = str(sorted(api_keys)) if api_keys else '[]'
cache_key = (requester, base_url, api_keys_str)
if cache_key in provider_cache:
provider_uuid = provider_cache[cache_key]
else:
# Create new provider
provider_uuid = str(uuid_lib.uuid4())
provider_name = f'{requester}'
if base_url:
# Extract domain for name
try:
from urllib.parse import urlparse
parsed = urlparse(base_url)
provider_name = parsed.netloc or requester
except Exception:
pass
import json
api_keys_json = json.dumps(api_keys) if api_keys else '[]'
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
INSERT INTO model_providers (uuid, name, requester, base_url, api_keys)
VALUES (:uuid, :name, :requester, :base_url, :api_keys)
"""),
{
'uuid': provider_uuid,
'name': provider_name,
'requester': requester,
'base_url': base_url,
'api_keys': api_keys_json,
},
)
provider_cache[cache_key] = provider_uuid
# Update model with provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE llm_models SET provider_uuid = :provider_uuid WHERE uuid = :uuid'),
{'provider_uuid': provider_uuid, 'uuid': model_uuid},
)
async def _migrate_embedding_models(self):
"""Migrate embedding models to use providers"""
embedding_columns = await self._get_columns('embedding_models')
# Add provider_uuid column if not exists
if 'provider_uuid' not in embedding_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE embedding_models ADD COLUMN provider_uuid VARCHAR(255)')
)
# Add prefered_ranking column if not exists
if 'prefered_ranking' not in embedding_columns:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('ALTER TABLE embedding_models ADD COLUMN prefered_ranking INTEGER NOT NULL DEFAULT 0')
)
# Only migrate if old columns exist
if 'requester' not in embedding_columns:
return
# Get all embedding models with old structure
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, name, requester, requester_config, api_keys FROM embedding_models')
)
models = result.fetchall()
# Get existing providers
provider_result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('SELECT uuid, requester, base_url, api_keys FROM model_providers')
)
existing_providers = provider_result.fetchall()
provider_cache = {}
for p in existing_providers:
p_uuid, p_requester, p_base_url, p_api_keys = p
api_keys_str = str(sorted(p_api_keys)) if p_api_keys else '[]'
provider_cache[(p_requester, p_base_url, api_keys_str)] = p_uuid
for model in models:
model_uuid, model_name, requester, requester_config, api_keys = model
base_url = ''
if requester_config:
if isinstance(requester_config, str):
import json
requester_config = json.loads(requester_config)
base_url = requester_config.get('base_url', '') or requester_config.get('base-url', '')
# Parse api_keys if it's a string
if isinstance(api_keys, str):
import json
try:
api_keys = json.loads(api_keys)
except Exception:
api_keys = []
if not api_keys:
api_keys = []
api_keys_str = str(sorted(api_keys)) if api_keys else '[]'
cache_key = (requester, base_url, api_keys_str)
if cache_key in provider_cache:
provider_uuid = provider_cache[cache_key]
else:
provider_uuid = str(uuid_lib.uuid4())
provider_name = f'{requester}'
if base_url:
try:
from urllib.parse import urlparse
parsed = urlparse(base_url)
provider_name = parsed.netloc or requester
except Exception:
pass
import json
api_keys_json = json.dumps(api_keys) if api_keys else '[]'
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text("""
INSERT INTO model_providers (uuid, name, requester, base_url, api_keys)
VALUES (:uuid, :name, :requester, :base_url, :api_keys)
"""),
{
'uuid': provider_uuid,
'name': provider_name,
'requester': requester,
'base_url': base_url,
'api_keys': api_keys_json,
},
)
provider_cache[cache_key] = provider_uuid
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text('UPDATE embedding_models SET provider_uuid = :provider_uuid WHERE uuid = :uuid'),
{'provider_uuid': provider_uuid, 'uuid': model_uuid},
)
async def _cleanup_columns(self):
"""Remove deprecated columns from model tables"""
llm_columns = await self._get_columns('llm_models')
deprecated_llm_cols = ['requester', 'requester_config', 'api_keys', 'description', 'source', 'space_model_id']
for col in deprecated_llm_cols:
if col in llm_columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE llm_models DROP COLUMN IF EXISTS {col}')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE llm_models DROP COLUMN {col}')
)
embedding_columns = await self._get_columns('embedding_models')
deprecated_embedding_cols = [
'requester',
'requester_config',
'api_keys',
'description',
'source',
'space_model_id',
]
for col in deprecated_embedding_cols:
if col in embedding_columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE embedding_models DROP COLUMN IF EXISTS {col}')
)
else:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE embedding_models DROP COLUMN {col}')
)
async def _get_columns(self, table_name: str) -> list:
"""Get column names for a table"""
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
f"SELECT column_name FROM information_schema.columns WHERE table_name = '{table_name}';"
)
)
all_result = result.fetchall()
return [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
all_result = result.fetchall()
return [row[1] for row in all_result]
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -0,0 +1,25 @@
from .. import migration
@migration.migration_class(17)
class MoveCloudServiceUrl(migration.DBMigration):
"""迁移云服务 URL 配置"""
async def upgrade(self):
"""升级"""
if 'space' not in self.ap.instance_config.data:
self.ap.instance_config.data['space'] = {
'url': 'https://space.langbot.app',
'models_gateway_api_url': 'https://api.langbot.cloud/v1',
'oauth_authorize_url': 'https://space.langbot.app/auth/authorize',
'disable_models_service': False,
}
if 'plugin' in self.ap.instance_config.data:
self.ap.instance_config.data['plugin'].pop('cloud_service_url', None)
await self.ap.instance_config.dump_config()
async def downgrade(self):
"""降级"""
pass

View File

@@ -0,0 +1,58 @@
import sqlalchemy
from .. import migration
@migration.migration_class(18)
class DBMigrateAddEmojiSupport(migration.DBMigration):
"""Add emoji field to knowledge_bases, external_knowledge_bases and legacy_pipelines tables"""
async def upgrade(self):
"""Upgrade"""
# Add emoji field to knowledge_bases
await self._add_emoji_to_table('knowledge_bases', '📚')
# Add emoji field to external_knowledge_bases
await self._add_emoji_to_table('external_knowledge_bases', '🔗')
# Add emoji field to legacy_pipelines
await self._add_emoji_to_table('legacy_pipelines', '⚙️')
async def _add_emoji_to_table(self, table_name: str, default_emoji: str):
"""Add emoji column to specified table if it doesn't exist"""
# Get all column names from the table
columns = []
if self.ap.persistence_mgr.db.name == 'postgresql':
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(
f"SELECT column_name FROM information_schema.columns WHERE table_name = '{table_name}';"
)
)
all_result = result.fetchall()
columns = [row[0] for row in all_result]
else:
result = await self.ap.persistence_mgr.execute_async(sqlalchemy.text(f'PRAGMA table_info({table_name});'))
all_result = result.fetchall()
columns = [row[1] for row in all_result]
# Check and add emoji column
if 'emoji' not in columns:
if self.ap.persistence_mgr.db.name == 'postgresql':
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f"ALTER TABLE {table_name} ADD COLUMN emoji VARCHAR(10) DEFAULT '{default_emoji}'")
)
else:
# SQLite doesn't support DEFAULT with emoji directly in ALTER TABLE
# Add column without default first
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f'ALTER TABLE {table_name} ADD COLUMN emoji VARCHAR(10)')
)
# Set default emoji value for existing records
await self.ap.persistence_mgr.execute_async(
sqlalchemy.text(f"UPDATE {table_name} SET emoji = '{default_emoji}' WHERE emoji IS NULL")
)
async def downgrade(self):
"""Downgrade"""
pass

View File

@@ -33,11 +33,14 @@ class Controller:
for query in queries:
session = await self.ap.sess_mgr.get_session(query)
self.ap.logger.debug(f'Checking query {query} session {session}')
# Debug logging removed from tight loop to prevent excessive log generation
# that can cause memory overflow in high-traffic scenarios
if not session._semaphore.locked():
selected_query = query
await session._semaphore.acquire()
# Only log when actually selecting a query
self.ap.logger.debug(f'Selected query {query.query_id} for processing')
break

View File

@@ -0,0 +1,270 @@
"""
Monitoring helper for recording events during pipeline execution.
This module provides convenient methods to record monitoring data
without cluttering the main pipeline code.
"""
from __future__ import annotations
import traceback
import typing
import time
import json
if typing.TYPE_CHECKING:
from ..core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
class MonitoringHelper:
"""Helper class for monitoring operations"""
@staticmethod
async def record_query_start(
ap: app.Application,
query: pipeline_query.Query,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
runner_name: str | None = None,
) -> str:
"""Record the start of query processing, returns message_id"""
try:
# Check if session exists, if not, record session start
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Try to record message
# Use JSON serialization to preserve message chain structure (including image URLs, etc.)
if hasattr(query, 'message_chain') and hasattr(query.message_chain, 'model_dump'):
message_content = json.dumps(query.message_chain.model_dump(), ensure_ascii=False)
else:
message_content = str(query)
# Variables will be updated in record_query_success after preproc stage sets them
# Here we just record None, the full variables will be set when query completes
message_id = await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
message_content=message_content,
session_id=session_id,
status='pending',
level='info',
platform=query.launcher_type.value
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
runner_name=runner_name,
variables=None, # Will be updated in record_query_success
)
# Update session activity or create new session if it doesn't exist
# Always pass pipeline info to handle pipeline switches
session_updated = await ap.monitoring_service.update_session_activity(
session_id,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
)
if not session_updated:
# Session doesn't exist, create it
await ap.monitoring_service.record_session_start(
session_id=session_id,
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
platform=query.launcher_type.value
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
)
return message_id
except Exception as e:
ap.logger.error(f'Failed to record query start: {e}')
return ''
@staticmethod
async def record_query_success(
ap: app.Application,
message_id: str,
query: pipeline_query.Query | None = None,
):
"""Record successful query processing by updating message status and variables"""
try:
if message_id:
# Serialize query.variables (filtering out internal variables)
query_variables_str = None
if query and hasattr(query, 'variables') and query.variables:
filtered_vars = {k: v for k, v in query.variables.items() if not k.startswith('_')}
if filtered_vars:
try:
query_variables_str = json.dumps(filtered_vars, ensure_ascii=False, default=str)
except Exception:
pass
await ap.monitoring_service.update_message_status(
message_id=message_id,
status='success',
variables=query_variables_str,
)
except Exception as e:
ap.logger.error(f'Failed to record query success: {e}')
@staticmethod
async def record_query_error(
ap: app.Application,
query: pipeline_query.Query,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
error: Exception,
runner_name: str | None = None,
) -> str:
"""Record query processing error, returns message_id"""
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Record error message
message_id = await ap.monitoring_service.record_message(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
message_content=f'Error: {str(error)}',
session_id=session_id,
status='error',
level='error',
platform=query.launcher_type.value
if hasattr(query.launcher_type, 'value')
else str(query.launcher_type),
user_id=query.sender_id,
runner_name=runner_name,
)
# Record error log
await ap.monitoring_service.record_error(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
error_type=type(error).__name__,
error_message=str(error),
session_id=session_id,
stack_trace=traceback.format_exc(),
message_id=message_id,
)
return message_id
except Exception as e:
ap.logger.error(f'Failed to record query error: {e}')
return ''
@staticmethod
async def record_llm_call(
ap: app.Application,
query: pipeline_query.Query,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
model_name: str,
input_tokens: int,
output_tokens: int,
duration_ms: int,
status: str = 'success',
cost: float | None = None,
error_message: str | None = None,
message_id: str | None = None,
):
"""Record LLM call"""
try:
session_id = f'{query.launcher_type}_{query.launcher_id}'
await ap.monitoring_service.record_llm_call(
bot_id=bot_id,
bot_name=bot_name,
pipeline_id=pipeline_id,
pipeline_name=pipeline_name,
session_id=session_id,
model_name=model_name,
input_tokens=input_tokens,
output_tokens=output_tokens,
duration=duration_ms,
status=status,
cost=cost,
error_message=error_message,
message_id=message_id,
)
except Exception as e:
ap.logger.error(f'Failed to record LLM call: {e}')
class LLMCallMonitor:
"""Context manager for monitoring LLM calls"""
def __init__(
self,
ap: app.Application,
query: pipeline_query.Query,
bot_id: str,
bot_name: str,
pipeline_id: str,
pipeline_name: str,
model_name: str,
):
self.ap = ap
self.query = query
self.bot_id = bot_id
self.bot_name = bot_name
self.pipeline_id = pipeline_id
self.pipeline_name = pipeline_name
self.model_name = model_name
self.start_time = None
self.input_tokens = 0
self.output_tokens = 0
async def __aenter__(self):
self.start_time = time.time()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
duration_ms = int((time.time() - self.start_time) * 1000)
if exc_type is not None:
# Error occurred
await MonitoringHelper.record_llm_call(
ap=self.ap,
query=self.query,
bot_id=self.bot_id,
bot_name=self.bot_name,
pipeline_id=self.pipeline_id,
pipeline_name=self.pipeline_name,
model_name=self.model_name,
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
duration_ms=duration_ms,
status='error',
error_message=str(exc_val) if exc_val else None,
)
else:
# Success
await MonitoringHelper.record_llm_call(
ap=self.ap,
query=self.query,
bot_id=self.bot_id,
bot_name=self.bot_name,
pipeline_id=self.pipeline_id,
pipeline_name=self.pipeline_name,
model_name=self.model_name,
input_tokens=self.input_tokens,
output_tokens=self.output_tokens,
duration_ms=duration_ms,
status='success',
)
return False # Don't suppress exceptions

View File

@@ -69,11 +69,17 @@ class RuntimePipeline:
stage_containers: list[StageInstContainer]
"""阶段实例容器"""
bound_plugins: list[str]
"""绑定到此流水线的插件列表格式author/plugin_name"""
bound_plugins: list[str] | None
"""绑定到此流水线的插件列表格式author/plugin_nameNone表示启用所有"""
bound_mcp_servers: list[str]
"""绑定到此流水线的MCP服务器列表格式uuid"""
bound_mcp_servers: list[str] | None
"""绑定到此流水线的MCP服务器列表格式uuidNone表示启用所有"""
enable_all_plugins: bool
"""是否启用所有插件"""
enable_all_mcp_servers: bool
"""是否启用所有MCP服务器"""
def __init__(
self,
@@ -87,17 +93,47 @@ class RuntimePipeline:
# Extract bound plugins and MCP servers from extensions_preferences
extensions_prefs = pipeline_entity.extensions_preferences or {}
plugin_list = extensions_prefs.get('plugins', [])
self.bound_plugins = [f'{p["author"]}/{p["name"]}' for p in plugin_list] if plugin_list else []
self.enable_all_plugins = extensions_prefs.get('enable_all_plugins', True)
self.enable_all_mcp_servers = extensions_prefs.get('enable_all_mcp_servers', True)
mcp_server_list = extensions_prefs.get('mcp_servers', [])
self.bound_mcp_servers = mcp_server_list if mcp_server_list else []
if self.enable_all_plugins:
# None indicates to use all available plugins
self.bound_plugins = None
else:
plugin_list = extensions_prefs.get('plugins', [])
self.bound_plugins = [f'{p["author"]}/{p["name"]}' for p in plugin_list] if plugin_list else []
if self.enable_all_mcp_servers:
# None indicates to use all available MCP servers
self.bound_mcp_servers = None
else:
mcp_server_list = extensions_prefs.get('mcp_servers', [])
self.bound_mcp_servers = mcp_server_list if mcp_server_list else []
async def run(self, query: pipeline_query.Query):
query.pipeline_config = self.pipeline_entity.config
# Store bound plugins and MCP servers in query for filtering
query.variables['_pipeline_bound_plugins'] = self.bound_plugins
query.variables['_pipeline_bound_mcp_servers'] = self.bound_mcp_servers
# Record query start for monitoring
try:
# Get bot name from bot_uuid
bot_name = 'WebChat'
if query.bot_uuid:
try:
bot = await self.ap.bot_service.get_bot(query.bot_uuid, include_secret=False)
if bot:
bot_name = bot.get('name', 'Unknown')
except Exception:
pass
# Store for later use in process_query
query.variables['_monitoring_bot_name'] = bot_name
query.variables['_monitoring_pipeline_name'] = self.pipeline_entity.name
except Exception as e:
self.ap.logger.error(f'Failed to prepare monitoring data: {e}')
await self.process_query(query)
async def _check_output(self, query: pipeline_query.Query, result: pipeline_entities.StageProcessResult):
@@ -114,7 +150,7 @@ class RuntimePipeline:
query.message_event, platform_events.GroupMessage
):
result.user_notice.insert(0, platform_message.At(target=query.message_event.sender.id))
if await query.adapter.is_stream_output_supported():
if await query.adapter.is_stream_output_supported() and query.resp_messages:
await query.adapter.reply_message_chunk(
message_source=query.message_event,
bot_message=query.resp_messages[-1],
@@ -134,6 +170,37 @@ class RuntimePipeline:
self.ap.logger.info(result.console_notice)
if result.error_notice:
self.ap.logger.error(result.error_notice)
# Mark query as having error
query.variables['_monitoring_has_error'] = True
# Record error to monitoring system
try:
bot_name = query.variables.get('_monitoring_bot_name', 'Unknown')
pipeline_name = query.variables.get('_monitoring_pipeline_name', 'Unknown')
message_id = query.variables.get('_monitoring_message_id', '')
session_id = f'{query.launcher_type}_{query.launcher_id}'
# Update message status to error
if message_id:
await self.ap.monitoring_service.update_message_status(
message_id=message_id,
status='error',
level='error',
)
# Record error log
await self.ap.monitoring_service.record_error(
bot_id=query.bot_uuid or 'unknown',
bot_name=bot_name,
pipeline_id=self.pipeline_entity.uuid,
pipeline_name=pipeline_name,
error_type='PipelineError',
error_message=result.error_notice,
session_id=session_id,
stack_trace=result.debug_notice if result.debug_notice else None,
message_id=message_id,
)
except Exception as e:
self.ap.logger.error(f'Failed to record error to monitoring: {e}')
async def _execute_from_stage(
self,
@@ -204,6 +271,34 @@ class RuntimePipeline:
async def process_query(self, query: pipeline_query.Query):
"""处理请求"""
# Get monitoring metadata
bot_name = query.variables.get('_monitoring_bot_name', 'Unknown')
pipeline_name = query.variables.get('_monitoring_pipeline_name', 'Unknown')
# Get runner name from pipeline config
runner_name = None
if query.pipeline_config and 'ai' in query.pipeline_config and 'runner' in query.pipeline_config['ai']:
runner_name = query.pipeline_config['ai']['runner'].get('runner')
# Record query start and store message_id
message_id = ''
try:
from . import monitoring_helper
message_id = await monitoring_helper.MonitoringHelper.record_query_start(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
bot_name=bot_name,
pipeline_id=self.pipeline_entity.uuid,
pipeline_name=pipeline_name,
runner_name=runner_name,
)
# Store message_id in query variables for LLM call monitoring
query.variables['_monitoring_message_id'] = message_id
except Exception as e:
self.ap.logger.error(f'Failed to record query start: {e}')
try:
# Get bound plugins for this pipeline
bound_plugins = query.variables.get('_pipeline_bound_plugins', None)
@@ -220,6 +315,7 @@ class RuntimePipeline:
launcher_type=query.launcher_type.value,
launcher_id=query.launcher_id,
sender_id=query.sender_id,
message_event=query.message_event,
message_chain=query.message_chain,
)
@@ -231,10 +327,40 @@ class RuntimePipeline:
self.ap.logger.debug(f'Processing query {query.query_id}')
await self._execute_from_stage(0, query)
# Record query success only if no error occurred during processing
if not query.variables.get('_monitoring_has_error', False):
try:
await monitoring_helper.MonitoringHelper.record_query_success(
ap=self.ap,
message_id=message_id,
query=query,
)
except Exception as e:
self.ap.logger.error(f'Failed to record query success: {e}')
except Exception as e:
inst_name = query.current_stage_name if query.current_stage_name else 'unknown'
self.ap.logger.error(f'Error processing query {query.query_id} stage={inst_name} : {e}')
self.ap.logger.error(f'Traceback: {traceback.format_exc()}')
# Record query error
try:
from . import monitoring_helper
await monitoring_helper.MonitoringHelper.record_query_error(
ap=self.ap,
query=query,
bot_id=query.bot_uuid or 'unknown',
bot_name=bot_name,
pipeline_id=self.pipeline_entity.uuid,
pipeline_name=pipeline_name,
error=e,
runner_name=runner_name,
)
except Exception as me:
self.ap.logger.error(f'Failed to record query error: {me}')
finally:
self.ap.logger.debug(f'Query {query.query_id} processed')
del self.ap.query_pool.cached_queries[query.query_id]

View File

@@ -7,6 +7,7 @@ from langbot_plugin.api.entities.builtin.provider import message as provider_mes
import langbot_plugin.api.entities.events as events
import langbot_plugin.api.entities.builtin.platform.message as platform_message
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.platform.events as platform_events
@stage.stage_class('PreProcessor')
@@ -74,12 +75,26 @@ class PreProcessor(stage.PipelineStage):
self.ap.logger.debug(f'Bound MCP servers: {bound_mcp_servers}')
self.ap.logger.debug(f'Use funcs: {query.use_funcs}')
sender_name = ''
if isinstance(query.message_event, platform_events.GroupMessage):
sender_name = query.message_event.sender.member_name
elif isinstance(query.message_event, platform_events.FriendMessage):
sender_name = query.message_event.sender.nickname
variables = {
'launcher_type': query.session.launcher_type.value,
'launcher_id': query.session.launcher_id,
'sender_id': query.sender_id,
'session_id': f'{query.session.launcher_type.value}_{query.session.launcher_id}',
'conversation_id': conversation.uuid,
'msg_create_time': (
int(query.message_event.time) if query.message_event.time else int(datetime.datetime.now().timestamp())
),
'group_name': query.message_event.group.name
if isinstance(query.message_event, platform_events.GroupMessage)
else '',
'sender_name': sender_name,
}
query.variables.update(variables)
@@ -99,7 +114,7 @@ class PreProcessor(stage.PipelineStage):
content_list: list[provider_message.ContentElement] = []
plain_text = ''
qoute_msg = query.pipeline_config['trigger'].get('misc', '').get('combine-quote-message')
quote_msg = query.pipeline_config['trigger'].get('misc', '').get('combine-quote-message')
for me in query.message_chain:
if isinstance(me, platform_message.Plain):
@@ -111,10 +126,16 @@ class PreProcessor(stage.PipelineStage):
):
if me.base64 is not None:
content_list.append(provider_message.ContentElement.from_image_base64(me.base64))
elif isinstance(me, platform_message.Voice):
# 转成文件链接,让下游 runner 上传到目标模型
if me.base64:
content_list.append(provider_message.ContentElement.from_file_base64(me.base64, 'voice.silk'))
elif me.url:
content_list.append(provider_message.ContentElement.from_file_url(me.url, 'voice'))
elif isinstance(me, platform_message.File):
# if me.url is not None:
content_list.append(provider_message.ContentElement.from_file_url(me.url, me.name))
elif isinstance(me, platform_message.Quote) and qoute_msg:
elif isinstance(me, platform_message.Quote) and quote_msg:
for msg in me.origin:
if isinstance(msg, platform_message.Plain):
content_list.append(provider_message.ContentElement.from_text(msg.text))

View File

@@ -3,6 +3,8 @@ from __future__ import annotations
import uuid
import typing
import traceback
import time
from datetime import datetime
from .. import handler
@@ -10,10 +12,11 @@ from ... import entities
from ....provider import runner as runner_module
import langbot_plugin.api.entities.events as events
from ....utils import importutil
from ....utils import importutil, constants
from ....provider import runners
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
importutil.import_modules_in_pkg(runners)
@@ -40,6 +43,8 @@ class ChatMessageHandler(handler.MessageHandler):
launcher_id=query.launcher_id,
sender_id=query.sender_id,
text_message=str(query.message_chain),
message_event=query.message_event,
message_chain=query.message_chain,
query=query,
)
@@ -59,8 +64,14 @@ class ChatMessageHandler(handler.MessageHandler):
yield entities.StageProcessResult(result_type=entities.ResultType.INTERRUPT, new_query=query)
else:
if event_ctx.event.user_message_alter is not None:
# if isinstance(event_ctx.event, str): # 现在暂时不考虑多模态alter
query.user_message.content = event_ctx.event.user_message_alter
if isinstance(event_ctx.event.user_message_alter, list):
query.user_message.content = event_ctx.event.user_message_alter
elif isinstance(event_ctx.event.user_message_alter, str):
query.user_message.content = [
provider_message.ContentElement.from_text(event_ctx.event.user_message_alter)
]
elif isinstance(event_ctx.event.user_message_alter, provider_message.ContentElement):
query.user_message.content = [event_ctx.event.user_message_alter]
text_length = 0
try:
@@ -74,9 +85,13 @@ class ChatMessageHandler(handler.MessageHandler):
runner = r(self.ap, query.pipeline_config)
break
else:
raise ValueError(f'未找到请求运行器: {query.pipeline_config["ai"]["runner"]["runner"]}')
raise ValueError(f'Request Runner not found: {query.pipeline_config["ai"]["runner"]["runner"]}')
# Mark start time for telemetry
start_ts = time.time()
if is_stream:
resp_message_id = uuid.uuid4()
chunk_count = 0 # Track streaming chunks to reduce excessive logging
async for result in runner.run(query):
result.resp_message_id = str(resp_message_id)
@@ -89,18 +104,37 @@ class ChatMessageHandler(handler.MessageHandler):
await query.adapter.create_message_card(str(resp_message_id), query.message_event)
is_create_card = True
query.resp_messages.append(result)
self.ap.logger.info(f'对话({query.query_id})流式响应: {self.cut_str(result.readable_str())}')
chunk_count += 1
# Only log every 10th chunk to reduce excessive logging during streaming
# This prevents memory overflow from thousands of log entries per conversation
# First chunk uses INFO level to confirm connection establishment
if chunk_count == 1:
self.ap.logger.info(
f'Conversation({query.query_id}) Streaming started: {self.cut_str(result.readable_str())}'
)
elif chunk_count % 10 == 0:
self.ap.logger.debug(
f'Conversation({query.query_id}) Streaming chunk {chunk_count}: {self.cut_str(result.readable_str())}'
)
if result.content is not None:
text_length += len(result.content)
yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
# Log final summary after streaming completes
self.ap.logger.info(
f'Conversation({query.query_id}) Streaming completed: {chunk_count} chunks, {text_length} chars'
)
else:
async for result in runner.run(query):
query.resp_messages.append(result)
self.ap.logger.info(f'对话({query.query_id})响应: {self.cut_str(result.readable_str())}')
self.ap.logger.info(
f'Conversation({query.query_id}) Response: {self.cut_str(result.readable_str())}'
)
if result.content is not None:
text_length += len(result.content)
@@ -111,7 +145,8 @@ class ChatMessageHandler(handler.MessageHandler):
query.session.using_conversation.messages.extend(query.resp_messages)
except Exception as e:
self.ap.logger.error(f'对话({query.query_id})请求失败: {type(e).__name__} {str(e)}')
error_info = f'{traceback.format_exc()}'
self.ap.logger.error(f'Conversation({query.query_id}) Request Failed: {error_info}')
traceback.print_exc()
hide_exception_info = query.pipeline_config['output']['misc']['hide-exception']
@@ -124,5 +159,47 @@ class ChatMessageHandler(handler.MessageHandler):
debug_notice=traceback.format_exc(),
)
finally:
# TODO statistics
pass
# Telemetry reporting: collect minimal per-query execution info and send asynchronously
try:
end_ts = time.time()
duration_ms = None
if 'start_ts' in locals():
duration_ms = int((end_ts - start_ts) * 1000)
adapter_name = query.adapter.__class__.__name__ if hasattr(query, 'adapter') else None
runner_name = (
query.pipeline_config.get('ai', {}).get('runner', {}).get('runner')
if query.pipeline_config
else None
)
# Model name if using localagent
model_name = None
try:
if runner_name == 'local-agent' and getattr(query, 'use_llm_model_uuid', None):
m = await self.ap.model_mgr.get_model_by_uuid(query.use_llm_model_uuid)
if m and getattr(m, 'model_entity', None):
model_name = getattr(m.model_entity, 'name', None)
except Exception:
model_name = None
pipeline_plugins = query.variables.get('_pipeline_bound_plugins', None)
payload = {
'query_id': query.query_id,
'adapter': adapter_name,
'runner': runner_name,
'duration_ms': duration_ms,
'model_name': model_name,
'version': constants.semantic_version,
'instance_id': constants.instance_id,
'pipeline_plugins': pipeline_plugins,
'error': locals().get('error_info', None),
'timestamp': datetime.utcnow().isoformat(),
}
# Send telemetry asynchronously and do not block pipeline via app's telemetry manager
await self.ap.telemetry.start_send_task(payload)
except Exception as ex:
# Ensure telemetry issues do not affect normal flow
self.ap.logger.warning(f'Failed to send telemetry: {ex}')

View File

@@ -31,4 +31,8 @@ class AtBotRule(rule_model.GroupRespondRule):
remove_at(message_chain)
remove_at(message_chain) # 回复消息时会at两次检查并删除重复的
should_respond_at = rule_dict.get('at', None)
if should_respond_at is not None:
return entities.RuleJudgeResult(matching=found and bool(should_respond_at), replacement=message_chain)
return entities.RuleJudgeResult(matching=found, replacement=message_chain)

View File

@@ -96,7 +96,7 @@ class ResponseWrapper(stage.PipelineStage):
if result.tool_calls is not None and len(result.tool_calls) > 0: # 有函数调用
function_names = [tc.function.name for tc in result.tool_calls]
reply_text = f'调用函数 {".".join(function_names)}...'
reply_text = f'Call {".".join(function_names)}...'
query.resp_message_chain.append(
platform_message.MessageChain([platform_message.Plain(text=reply_text)])

View File

@@ -66,22 +66,34 @@ class RuntimeBot:
message_session_id=f'person_{event.sender.id}',
)
# Push to webhooks
# Push to webhooks and check if pipeline should be skipped
skip_pipeline = False
if hasattr(self.ap, 'webhook_pusher') and self.ap.webhook_pusher:
asyncio.create_task(
self.ap.webhook_pusher.push_person_message(event, self.bot_entity.uuid, adapter.__class__.__name__)
skip_pipeline = await self.ap.webhook_pusher.push_person_message(
event, self.bot_entity.uuid, adapter.__class__.__name__
)
await self.ap.query_pool.add_query(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.PERSON,
launcher_id=event.sender.id,
sender_id=event.sender.id,
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
)
# Only add to query pool if no webhook requested to skip pipeline
if not skip_pipeline:
launcher_id = event.sender.id
if hasattr(adapter, 'get_launcher_id'):
custom_launcher_id = adapter.get_launcher_id(event)
if custom_launcher_id:
launcher_id = custom_launcher_id
await self.ap.query_pool.add_query(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.PERSON,
launcher_id=launcher_id,
sender_id=event.sender.id,
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
)
else:
await self.logger.info('Pipeline skipped for person message due to webhook response')
async def on_group_message(
event: platform_events.GroupMessage,
@@ -97,22 +109,34 @@ class RuntimeBot:
message_session_id=f'group_{event.group.id}',
)
# Push to webhooks
# Push to webhooks and check if pipeline should be skipped
skip_pipeline = False
if hasattr(self.ap, 'webhook_pusher') and self.ap.webhook_pusher:
asyncio.create_task(
self.ap.webhook_pusher.push_group_message(event, self.bot_entity.uuid, adapter.__class__.__name__)
skip_pipeline = await self.ap.webhook_pusher.push_group_message(
event, self.bot_entity.uuid, adapter.__class__.__name__
)
await self.ap.query_pool.add_query(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.GROUP,
launcher_id=event.group.id,
sender_id=event.sender.id,
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
)
# Only add to query pool if no webhook requested to skip pipeline
if not skip_pipeline:
launcher_id = event.group.id
if hasattr(adapter, 'get_launcher_id'):
custom_launcher_id = adapter.get_launcher_id(event)
if custom_launcher_id:
launcher_id = custom_launcher_id
await self.ap.query_pool.add_query(
bot_uuid=self.bot_entity.uuid,
launcher_type=provider_session.LauncherTypes.GROUP,
launcher_id=launcher_id,
sender_id=event.sender.id,
message_event=event,
message_chain=event.message_chain,
adapter=adapter,
pipeline_uuid=self.bot_entity.use_pipeline_uuid,
)
else:
await self.logger.info('Pipeline skipped for group message due to webhook response')
self.adapter.register_listener(platform_events.FriendMessage, on_friend_message)
self.adapter.register_listener(platform_events.GroupMessage, on_group_message)
@@ -156,7 +180,7 @@ class PlatformManager:
bots: list[RuntimeBot]
webchat_proxy_bot: RuntimeBot
websocket_proxy_bot: RuntimeBot
adapter_components: list[engine.Component]
@@ -178,31 +202,29 @@ class PlatformManager:
adapter_dict[component.metadata.name] = component.get_python_component_class()
self.adapter_dict = adapter_dict
webchat_adapter_class = self.adapter_dict['webchat']
# initialize webchat adapter
webchat_logger = EventLogger(name='webchat-adapter', ap=self.ap)
webchat_adapter_inst = webchat_adapter_class(
# initialize websocket adapter
websocket_adapter_class = self.adapter_dict['websocket']
websocket_logger = EventLogger(name='websocket-adapter', ap=self.ap)
websocket_adapter_inst = websocket_adapter_class(
{},
webchat_logger,
websocket_logger,
ap=self.ap,
is_stream=False,
)
self.webchat_proxy_bot = RuntimeBot(
self.websocket_proxy_bot = RuntimeBot(
ap=self.ap,
bot_entity=persistence_bot.Bot(
uuid='webchat-proxy-bot',
name='WebChat',
uuid='websocket-proxy-bot',
name='WebSocket',
description='',
adapter='webchat',
adapter='websocket',
adapter_config={},
enable=True,
),
adapter=webchat_adapter_inst,
logger=webchat_logger,
adapter=websocket_adapter_inst,
logger=websocket_logger,
)
await self.webchat_proxy_bot.initialize()
await self.websocket_proxy_bot.initialize()
await self.load_bots_from_db()
@@ -247,6 +269,10 @@ class PlatformManager:
logger,
)
# 如果 adapter 支持 set_bot_uuid 方法,设置 bot_uuid用于统一 webhook
if hasattr(adapter_inst, 'set_bot_uuid'):
adapter_inst.set_bot_uuid(bot_entity.uuid)
runtime_bot = RuntimeBot(ap=self.ap, bot_entity=bot_entity, adapter=adapter_inst, logger=logger)
await runtime_bot.initialize()
@@ -271,7 +297,7 @@ class PlatformManager:
def get_available_adapters_info(self) -> list[dict]:
return [
component.to_plain_dict() for component in self.adapter_components if component.metadata.name != 'webchat'
component.to_plain_dict() for component in self.adapter_components if component.metadata.name != 'websocket'
]
def get_available_adapter_info_by_name(self, name: str) -> dict | None:
@@ -288,7 +314,7 @@ class PlatformManager:
async def run(self):
# This method will only be called when the application launching
await self.webchat_proxy_bot.run()
await self.websocket_proxy_bot.run()
for bot in self.bots:
if bot.enable:

View File

@@ -209,7 +209,7 @@ class AiocqhttpMessageConverter(abstract_platform_adapter.AbstractMessageConvert
elif msg_data['type'] == 'text':
reply_list.append(platform_message.Plain(text=msg_data['data']['text']))
elif msg_data['type'] == 'forward': # 这里来应该传入转发消息组,暂时传入qoute
elif msg_data['type'] == 'forward': # 这里来应该传入转发消息组,暂时传入Quote
for forward_msg_datas in msg_data['data']['content']:
for forward_msg_data in forward_msg_datas['message']:
await process_message_data(forward_msg_data, reply_list)
@@ -259,7 +259,7 @@ class AiocqhttpMessageConverter(abstract_platform_adapter.AbstractMessageConvert
# await process_message_data(msg_data, yiri_msg_list)
pass
elif msg.type == 'reply': # 此处处理引用消息传入Qoute
elif msg.type == 'reply': # 此处处理引用消息传入Quote
msg_datas = await bot.get_msg(message_id=msg.data['id'])
for msg_data in msg_datas['message']:
@@ -327,9 +327,6 @@ class AiocqhttpEventConverter(abstract_platform_adapter.AbstractEventConverter):
permission=platform_entities.Permission.Member,
),
special_title=event.sender['title'] if 'title' in event.sender else '',
join_timestamp=0,
last_speak_timestamp=0,
mute_time_remaining=0,
),
message_chain=yiri_chain,
time=event.time,

View File

@@ -12,17 +12,41 @@ from langbot.pkg.platform.logger import EventLogger
class DingTalkMessageConverter(abstract_platform_adapter.AbstractMessageConverter):
@staticmethod
async def yiri2target(message_chain: platform_message.MessageChain):
def _format_image_as_markdown(msg: platform_message.Image) -> str:
"""Convert an Image message to Markdown format for DingTalk."""
if msg.url:
return f'\n![image]({msg.url})\n'
elif msg.base64:
# For base64 images, try to include them as data URIs
# DingTalk may have limited support for base64 in markdown
if msg.base64.startswith('data:'):
return f'\n![image]({msg.base64})\n'
else:
return f'\n![image](data:image/png;base64,{msg.base64})\n'
return ''
@staticmethod
async def yiri2target(message_chain: platform_message.MessageChain, markdown_enabled: bool = True):
content = ''
at = False
for msg in message_chain:
if type(msg) is platform_message.At:
at = True
if type(msg) is platform_message.Plain:
elif type(msg) is platform_message.Plain:
content += msg.text
if type(msg) is platform_message.Forward:
elif type(msg) is platform_message.Image:
# DingTalk supports markdown images when markdown_card is enabled
# When markdown is disabled, images cannot be rendered in plain text mode
if markdown_enabled:
content += DingTalkMessageConverter._format_image_as_markdown(msg)
# Note: When markdown_enabled is False, images are not included
# as DingTalk plain text messages don't support image embedding
elif type(msg) is platform_message.Forward:
for node in msg.node_list:
content += (await DingTalkMessageConverter.yiri2target(node.message_chain))[0]
forwarded_content, _ = await DingTalkMessageConverter.yiri2target(
node.message_chain, markdown_enabled
)
content += forwarded_content
return content, at
@staticmethod
@@ -95,9 +119,6 @@ class DingTalkEventConverter(abstract_platform_adapter.AbstractEventConverter):
permission=platform_entities.Permission.Member,
),
special_title='',
join_timestamp=0,
last_speak_timestamp=0,
mute_time_remaining=0,
)
time = event.incoming_message.create_at
return platform_events.GroupMessage(
@@ -157,7 +178,8 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
)
incoming_message = event.incoming_message
content, at = await DingTalkMessageConverter.yiri2target(message)
markdown_enabled = self.config.get('markdown_card', False)
content, at = await DingTalkMessageConverter.yiri2target(message, markdown_enabled)
await self.bot.send_message(content, incoming_message, at)
async def reply_message_chunk(
@@ -178,7 +200,8 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
msg_seq = bot_message.msg_sequence
if (msg_seq - 1) % 8 == 0 or is_final:
content, at = await DingTalkMessageConverter.yiri2target(message)
markdown_enabled = self.config.get('markdown_card', False)
content, at = await DingTalkMessageConverter.yiri2target(message, markdown_enabled)
card_instance, card_instance_id = self.card_instance_id_dict[message_id]
if not content and bot_message.content:
@@ -191,7 +214,8 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
self.card_instance_id_dict.pop(message_id) # 消息回复结束之后删除卡片实例id
async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain):
content = await DingTalkMessageConverter.yiri2target(message)
markdown_enabled = self.config.get('markdown_card', False)
content, _ = await DingTalkMessageConverter.yiri2target(message, markdown_enabled)
if target_type == 'person':
await self.bot.send_proactive_message_to_one(target_id, content)
if target_type == 'group':
@@ -236,7 +260,8 @@ class DingTalkAdapter(abstract_platform_adapter.AbstractMessagePlatformAdapter):
await self.bot.start()
async def kill(self) -> bool:
return False
await self.bot.stop()
return True
async def is_muted(self) -> bool:
return False

Some files were not shown because too many files have changed in this diff Show More