diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index f3389c25..71ef28fc 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -9,7 +9,7 @@ *请在方括号间写`x`以打勾 / Please tick the box with `x`* -- [ ] 阅读仓库[贡献指引](https://github.com/RockChinQ/LangBot/blob/master/CONTRIBUTING.md)了吗? / Have you read the [contribution guide](https://github.com/RockChinQ/LangBot/blob/master/CONTRIBUTING.md)? +- [ ] 阅读仓库[贡献指引](https://github.com/langbot-app/LangBot/blob/master/CONTRIBUTING.md)了吗? / Have you read the [contribution guide](https://github.com/langbot-app/LangBot/blob/master/CONTRIBUTING.md)? - [ ] 与项目所有者沟通过了吗? / Have you communicated with the project maintainer? - [ ] 我确定已自行测试所作的更改,确保功能符合预期。 / I have tested the changes and ensured they work as expected. diff --git a/.gitignore b/.gitignore index 2869b7cc..db62bdca 100644 --- a/.gitignore +++ b/.gitignore @@ -42,4 +42,5 @@ botpy.log* test.py /web_ui .venv/ -uv.lock \ No newline at end of file +uv.lock +/test \ No newline at end of file diff --git a/README.md b/README.md index ba6bbf90..dd518656 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,38 @@

-LangBot +LangBot

-RockChinQ%2FLangBot | Trendshift +[English](README_EN.md) / 简体中文 / [繁體中文](README_TW.md) / [日本語](README_JP.md) / (PR for your language) + +[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) +[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-966235608-blue)](https://qm.qq.com/q/JLi38whHum) +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot) +[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest) +python +[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot) 项目主页部署文档插件介绍 | -提交插件 +提交插件 -
-😎高稳定、🧩支持扩展、🦄多模态 - 大模型原生即时通信机器人平台🤖 -
- -
- -[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) -[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-966235608-blue)](https://qm.qq.com/q/JLi38whHum) -[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/RockChinQ/LangBot) -[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RockChinQ/LangBot)](https://github.com/RockChinQ/LangBot/releases/latest) -python -[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot) - -简体中文 / [English](README_EN.md) / [日本語](README_JP.md) / (PR for your language)

-## ✨ 特性 - -- 💬 大模型对话、Agent:支持多种大模型,适配群聊和私聊;具有多轮对话、工具调用、多模态能力,并深度适配 [Dify](https://dify.ai)。目前支持 QQ、QQ频道、企业微信、个人微信、飞书、Discord、Telegram 等平台。 -- 🛠️ 高稳定性、功能完备:原生支持访问控制、限速、敏感词过滤等机制;配置简单,支持多种部署方式。支持多流水线配置,不同机器人用于不同应用场景。 -- 🧩 插件扩展、活跃社区:支持事件驱动、组件扩展等插件机制;适配 Anthropic [MCP 协议](https://modelcontextprotocol.io/);目前已有数百个插件。 -- 😻 Web 管理面板:支持通过浏览器管理 LangBot 实例,不再需要手动编写配置文件。 +LangBot 是一个开源的大语言模型原生即时通信机器人开发平台,旨在提供开箱即用的 IM 机器人开发体验,具有 Agent、RAG、MCP 等多种 LLM 应用功能,适配全球主流即时通信平台,并提供丰富的 API 接口,支持自定义开发。 ## 📦 开始使用 #### Docker Compose 部署 ```bash -git clone https://github.com/RockChinQ/LangBot +git clone https://github.com/langbot-app/LangBot cd LangBot docker compose up -d ``` @@ -71,23 +59,25 @@ docker compose up -d 直接使用发行版运行,查看文档[手动部署](https://docs.langbot.app/zh/deploy/langbot/manual.html)。 -## 📸 效果展示 +## 😎 保持更新 -bots +点击仓库右上角 Star 和 Watch 按钮,获取最新动态。 -bots +![star gif](https://docs.langbot.app/star.gif) -bots +## ✨ 特性 -bots +- 💬 大模型对话、Agent:支持多种大模型,适配群聊和私聊;具有多轮对话、工具调用、多模态能力,自带 RAG(知识库)实现,并深度适配 [Dify](https://dify.ai)。 +- 🤖 多平台支持:目前支持 QQ、QQ频道、企业微信、个人微信、飞书、Discord、Telegram 等平台。 +- 🛠️ 高稳定性、功能完备:原生支持访问控制、限速、敏感词过滤等机制;配置简单,支持多种部署方式。支持多流水线配置,不同机器人用于不同应用场景。 +- 🧩 插件扩展、活跃社区:支持事件驱动、组件扩展等插件机制;适配 Anthropic [MCP 协议](https://modelcontextprotocol.io/);目前已有数百个插件。 +- 😻 Web 管理面板:支持通过浏览器管理 LangBot 实例,不再需要手动编写配置文件。 -回复效果(带有联网插件) +详细规格特性请访问[文档](https://docs.langbot.app/zh/insight/features.html)。 -- WebUI Demo: https://demo.langbot.dev/ - - 登录信息:邮箱:`demo@langbot.app` 密码:`langbot123456` - - 注意:仅展示webui效果,公开环境,请不要在其中填入您的任何敏感信息。 - -## 🔌 组件兼容性 +或访问 demo 环境:https://demo.langbot.dev/ + - 登录信息:邮箱:`demo@langbot.app` 密码:`langbot123456` + - 注意:仅展示 WebUI 效果,公开环境,请不要在其中填入您的任何敏感信息。 ### 消息平台 @@ -95,19 +85,14 @@ docker compose up -d | --- | --- | --- | | QQ 个人号 | ✅ | QQ 个人号私聊、群聊 | | QQ 官方机器人 | ✅ | QQ 官方机器人,支持频道、私聊、群聊 | -| 企业微信 | ✅ | | +| 微信 | ✅ | | | 企微对外客服 | ✅ | | -| 个人微信 | ✅ | | | 微信公众号 | ✅ | | | 飞书 | ✅ | | | 钉钉 | ✅ | | | Discord | ✅ | | | Telegram | ✅ | | | Slack | ✅ | | -| LINE | 🚧 | | -| WhatsApp | 🚧 | | - -🚧: 正在开发中 ### 大模型能力 @@ -119,8 +104,9 @@ docker compose up -d | [Anthropic](https://www.anthropic.com/) | ✅ | | | [xAI](https://x.ai/) | ✅ | | | [智谱AI](https://open.bigmodel.cn/) | ✅ | | +| [优云智算](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | 大模型和 GPU 资源平台 | | [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | 大模型和 GPU 资源平台 | -| [302 AI](https://share.302.ai/SuTG99) | ✅ | 大模型聚合平台 | +| [302.AI](https://share.302.ai/SuTG99) | ✅ | 大模型聚合平台 | | [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | | | [Dify](https://dify.ai) | ✅ | LLMOps 平台 | | [Ollama](https://ollama.com/) | ✅ | 本地大模型运行平台 | @@ -148,14 +134,8 @@ docker compose up -d ## 😘 社区贡献 -感谢以下[代码贡献者](https://github.com/RockChinQ/LangBot/graphs/contributors)和社区里其他成员对 LangBot 的贡献: +感谢以下[代码贡献者](https://github.com/langbot-app/LangBot/graphs/contributors)和社区里其他成员对 LangBot 的贡献: - - + + - -## 😎 保持更新 - -点击仓库右上角 Star 和 Watch 按钮,获取最新动态。 - -![star gif](https://docs.langbot.app/star.gif) diff --git a/README_EN.md b/README_EN.md index 0542f16d..b900a51c 100644 --- a/README_EN.md +++ b/README_EN.md @@ -1,48 +1,34 @@

-LangBot +LangBot

-RockChinQ%2FLangBot | Trendshift +English / [简体中文](README.md) / [繁體中文](README_TW.md) / [日本語](README_JP.md) / (PR for your language) + +[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot) +[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest) +python HomeDeploymentPlugin | -Submit Plugin - -
-😎High Stability, 🧩Extension Supported, 🦄Multi-modal - LLM Native Instant Messaging Bot Platform🤖 -
- -
- - -[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) -[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/RockChinQ/LangBot) -[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RockChinQ/LangBot)](https://github.com/RockChinQ/LangBot/releases/latest) -python - -[简体中文](README.md) / English / [日本語](README_JP.md) / (PR for your language) +Submit Plugin

-## ✨ Features - -- 💬 Chat with LLM / Agent: Supports multiple LLMs, adapt to group chats and private chats; Supports multi-round conversations, tool calls, and multi-modal capabilities. Deeply integrates with [Dify](https://dify.ai). Currently supports QQ, QQ Channel, WeCom, personal WeChat, Lark, DingTalk, Discord, Telegram, etc. -- 🛠️ High Stability, Feature-rich: Native access control, rate limiting, sensitive word filtering, etc. mechanisms; Easy to use, supports multiple deployment methods. Supports multiple pipeline configurations, different bots can be used for different scenarios. -- 🧩 Plugin Extension, Active Community: Support event-driven, component extension, etc. plugin mechanisms; Integrate Anthropic [MCP protocol](https://modelcontextprotocol.io/); Currently has hundreds of plugins. -- 😻 [New] Web UI: Support management LangBot instance through the browser. No need to manually write configuration files. +LangBot is an open-source LLM native instant messaging robot development platform, aiming to provide out-of-the-box IM robot development experience, with Agent, RAG, MCP and other LLM application functions, adapting to global instant messaging platforms, and providing rich API interfaces, supporting custom development. ## 📦 Getting Started #### Docker Compose Deployment ```bash -git clone https://github.com/RockChinQ/LangBot +git clone https://github.com/langbot-app/LangBot cd LangBot docker compose up -d ``` @@ -69,23 +55,25 @@ Community contributed Zeabur template. Directly use the released version to run, see the [Manual Deployment](https://docs.langbot.app/en/deploy/langbot/manual.html) documentation. -## 📸 Demo +## 😎 Stay Ahead -bots +Click the Star and Watch button in the upper right corner of the repository to get the latest updates. -bots +![star gif](https://docs.langbot.app/star.gif) -bots +## ✨ Features -bots +- 💬 Chat with LLM / Agent: Supports multiple LLMs, adapt to group chats and private chats; Supports multi-round conversations, tool calls, and multi-modal capabilities. Built-in RAG (knowledge base) implementation, and deeply integrates with [Dify](https://dify.ai). +- 🤖 Multi-platform Support: Currently supports QQ, QQ Channel, WeCom, personal WeChat, Lark, DingTalk, Discord, Telegram, etc. +- 🛠️ High Stability, Feature-rich: Native access control, rate limiting, sensitive word filtering, etc. mechanisms; Easy to use, supports multiple deployment methods. Supports multiple pipeline configurations, different bots can be used for different scenarios. +- 🧩 Plugin Extension, Active Community: Support event-driven, component extension, etc. plugin mechanisms; Integrate Anthropic [MCP protocol](https://modelcontextprotocol.io/); Currently has hundreds of plugins. +- 😻 Web UI: Support management LangBot instance through the browser. No need to manually write configuration files. -Reply Effect (with Internet Plugin) +For more detailed specifications, please refer to the [documentation](https://docs.langbot.app/en/insight/features.html). -- WebUI Demo: https://demo.langbot.dev/ - - Login information: Email: `demo@langbot.app` Password: `langbot123456` - - Note: Only the WebUI effect is shown, please do not fill in any sensitive information in the public environment. - -## 🔌 Component Compatibility +Or visit the demo environment: https://demo.langbot.dev/ + - Login information: Email: `demo@langbot.app` Password: `langbot123456` + - Note: For WebUI demo only, please do not fill in any sensitive information in the public environment. ### Message Platform @@ -101,10 +89,6 @@ Directly use the released version to run, see the [Manual Deployment](https://do | Discord | ✅ | | | Telegram | ✅ | | | Slack | ✅ | | -| LINE | 🚧 | | -| WhatsApp | 🚧 | | - -🚧: In development ### LLMs @@ -116,9 +100,10 @@ Directly use the released version to run, see the [Manual Deployment](https://do | [Anthropic](https://www.anthropic.com/) | ✅ | | | [xAI](https://x.ai/) | ✅ | | | [Zhipu AI](https://open.bigmodel.cn/) | ✅ | | +| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | LLM and GPU resource platform | | [Dify](https://dify.ai) | ✅ | LLMOps platform | | [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | LLM and GPU resource platform | -| [302 AI](https://share.302.ai/SuTG99) | ✅ | LLM gateway(MaaS) | +| [302.AI](https://share.302.ai/SuTG99) | ✅ | LLM gateway(MaaS) | | [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | | | [Ollama](https://ollama.com/) | ✅ | Local LLM running platform | | [LMStudio](https://lmstudio.ai/) | ✅ | Local LLM running platform | @@ -131,14 +116,8 @@ Directly use the released version to run, see the [Manual Deployment](https://do ## 🤝 Community Contribution -Thank you for the following [code contributors](https://github.com/RockChinQ/LangBot/graphs/contributors) and other members in the community for their contributions to LangBot: +Thank you for the following [code contributors](https://github.com/langbot-app/LangBot/graphs/contributors) and other members in the community for their contributions to LangBot: - - + + - -## 😎 Stay Ahead - -Click the Star and Watch button in the upper right corner of the repository to get the latest updates. - -![star gif](https://docs.langbot.app/star.gif) \ No newline at end of file diff --git a/README_JP.md b/README_JP.md index 7a3a16dd..d712d8e7 100644 --- a/README_JP.md +++ b/README_JP.md @@ -1,47 +1,34 @@

-LangBot +LangBot

-RockChinQ%2FLangBot | Trendshift +[English](README_EN.md) / [简体中文](README.md) / [繁體中文](README_TW.md) / 日本語 / (PR for your language) + +[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot) +[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest) +python ホームデプロイプラグイン | -プラグインの提出 - -
-😎高い安定性、🧩拡張サポート、🦄マルチモーダル - LLMネイティブインスタントメッセージングボットプラットフォーム🤖 -
- -
- -[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) -[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/RockChinQ/LangBot) -[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RockChinQ/LangBot)](https://github.com/RockChinQ/LangBot/releases/latest) -python - -[简体中文](README_CN.md) / [English](README.md) / [日本語](README_JP.md) / (PR for your language) +プラグインの提出

-## ✨ 機能 - -- 💬 LLM / エージェントとのチャット: 複数のLLMをサポートし、グループチャットとプライベートチャットに対応。マルチラウンドの会話、ツールの呼び出し、マルチモーダル機能をサポート。 [Dify](https://dify.ai) と深く統合。現在、QQ、QQ チャンネル、WeChat、個人 WeChat、Lark、DingTalk、Discord、Telegram など、複数のプラットフォームをサポートしています。 -- 🛠️ 高い安定性、豊富な機能: ネイティブのアクセス制御、レート制限、敏感な単語のフィルタリングなどのメカニズムをサポート。使いやすく、複数のデプロイ方法をサポート。複数のパイプライン設定をサポートし、異なるボットを異なる用途に使用できます。 -- 🧩 プラグイン拡張、活発なコミュニティ: イベント駆動、コンポーネント拡張などのプラグインメカニズムをサポート。適配 Anthropic [MCP プロトコル](https://modelcontextprotocol.io/);豊富なエコシステム、現在数百のプラグインが存在。 -- 😻 Web UI: ブラウザを通じてLangBotインスタンスを管理することをサポート。 +LangBot は、エージェント、RAG、MCP などの LLM アプリケーション機能を備えた、オープンソースの LLM ネイティブのインスタントメッセージングロボット開発プラットフォームです。世界中のインスタントメッセージングプラットフォームに適応し、豊富な API インターフェースを提供し、カスタム開発をサポートします。 ## 📦 始め方 #### Docker Compose デプロイ ```bash -git clone https://github.com/RockChinQ/LangBot +git clone https://github.com/langbot-app/LangBot cd LangBot docker compose up -d ``` @@ -50,7 +37,7 @@ http://localhost:5300 にアクセスして使用を開始します。 詳細なドキュメントは[Dockerデプロイ](https://docs.langbot.app/en/deploy/langbot/docker.html)を参照してください。 -#### BTPanelでのワンクリックデプロイ +#### Panelでのワンクリックデプロイ LangBotはBTPanelにリストされています。BTPanelをインストールしている場合は、[ドキュメント](https://docs.langbot.app/en/deploy/langbot/one-click/bt.html)を使用して使用できます。 @@ -68,23 +55,25 @@ LangBotはBTPanelにリストされています。BTPanelをインストール リリースバージョンを直接使用して実行します。[手動デプロイ](https://docs.langbot.app/en/deploy/langbot/manual.html)のドキュメントを参照してください。 -## 📸 デモ +## 😎 最新情報を入手 -bots +リポジトリの右上にある Star と Watch ボタンをクリックして、最新の更新を取得してください。 -bots +![star gif](https://docs.langbot.app/star.gif) -bots +## ✨ 機能 -bots +- 💬 LLM / エージェントとのチャット: 複数のLLMをサポートし、グループチャットとプライベートチャットに対応。マルチラウンドの会話、ツールの呼び出し、マルチモーダル機能をサポート、RAG(知識ベース)を組み込み、[Dify](https://dify.ai) と深く統合。 +- 🤖 多プラットフォーム対応: 現在、QQ、QQ チャンネル、WeChat、個人 WeChat、Lark、DingTalk、Discord、Telegram など、複数のプラットフォームをサポートしています。 +- 🛠️ 高い安定性、豊富な機能: ネイティブのアクセス制御、レート制限、敏感な単語のフィルタリングなどのメカニズムをサポート。使いやすく、複数のデプロイ方法をサポート。複数のパイプライン設定をサポートし、異なるボットを異なる用途に使用できます。 +- 🧩 プラグイン拡張、活発なコミュニティ: イベント駆動、コンポーネント拡張などのプラグインメカニズムをサポート。適配 Anthropic [MCP プロトコル](https://modelcontextprotocol.io/);豊富なエコシステム、現在数百のプラグインが存在。 +- 😻 Web UI: ブラウザを通じてLangBotインスタンスを管理することをサポート。 -返信効果(インターネットプラグイン付き) +詳細な仕様については、[ドキュメント](https://docs.langbot.app/en/insight/features.html)を参照してください。 -- WebUIデモ: https://demo.langbot.dev/ - - ログイン情報: メール: `demo@langbot.app` パスワード: `langbot123456` - - 注意: WebUIの効果のみを示しています。公開環境では、機密情報を入力しないでください。 - -## 🔌 コンポーネントの互換性 +または、デモ環境にアクセスしてください: https://demo.langbot.dev/ + - ログイン情報: メール: `demo@langbot.app` パスワード: `langbot123456` + - 注意: WebUI のデモンストレーションのみの場合、公開環境では機密情報を入力しないでください。 ### メッセージプラットフォーム @@ -100,10 +89,6 @@ LangBotはBTPanelにリストされています。BTPanelをインストール | Discord | ✅ | | | Telegram | ✅ | | | Slack | ✅ | | -| LINE | 🚧 | | -| WhatsApp | 🚧 | | - -🚧: 開発中 ### LLMs @@ -115,8 +100,9 @@ LangBotはBTPanelにリストされています。BTPanelをインストール | [Anthropic](https://www.anthropic.com/) | ✅ | | | [xAI](https://x.ai/) | ✅ | | | [Zhipu AI](https://open.bigmodel.cn/) | ✅ | | +| [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | 大模型とGPUリソースプラットフォーム | | [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | 大模型とGPUリソースプラットフォーム | -| [302 AI](https://share.302.ai/SuTG99) | ✅ | LLMゲートウェイ(MaaS) | +| [302.AI](https://share.302.ai/SuTG99) | ✅ | LLMゲートウェイ(MaaS) | | [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | | | [Dify](https://dify.ai) | ✅ | LLMOpsプラットフォーム | | [Ollama](https://ollama.com/) | ✅ | ローカルLLM実行プラットフォーム | @@ -130,14 +116,8 @@ LangBotはBTPanelにリストされています。BTPanelをインストール ## 🤝 コミュニティ貢献 -LangBot への貢献に対して、以下の [コード貢献者](https://github.com/RockChinQ/LangBot/graphs/contributors) とコミュニティの他のメンバーに感謝します。 +LangBot への貢献に対して、以下の [コード貢献者](https://github.com/langbot-app/LangBot/graphs/contributors) とコミュニティの他のメンバーに感謝します。 - - + + - -## 😎 最新情報を入手 - -リポジトリの右上にある Star と Watch ボタンをクリックして、最新の更新を取得してください。 - -![star gif](https://docs.langbot.app/star.gif) \ No newline at end of file diff --git a/README_TW.md b/README_TW.md new file mode 100644 index 00000000..27bf5e14 --- /dev/null +++ b/README_TW.md @@ -0,0 +1,139 @@ +

+ +LangBot + + +

+ +[English](README_EN.md) / [简体中文](README.md) / 繁體中文 / [日本語](README_JP.md) / (PR for your language) + +[![Discord](https://img.shields.io/discord/1335141740050649118?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb)](https://discord.gg/wdNEHETs87) +[![QQ Group](https://img.shields.io/badge/%E7%A4%BE%E5%8C%BAQQ%E7%BE%A4-966235608-blue)](https://qm.qq.com/q/JLi38whHum) +[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/langbot-app/LangBot) +[![GitHub release (latest by date)](https://img.shields.io/github/v/release/langbot-app/LangBot)](https://github.com/langbot-app/LangBot/releases/latest) +python +[![star](https://gitcode.com/RockChinQ/LangBot/star/badge.svg)](https://gitcode.com/RockChinQ/LangBot) + +主頁 | +部署文件 | +外掛介紹 | +提交外掛 + +
+ +

+ +LangBot 是一個開源的大語言模型原生即時通訊機器人開發平台,旨在提供開箱即用的 IM 機器人開發體驗,具有 Agent、RAG、MCP 等多種 LLM 應用功能,適配全球主流即時通訊平台,並提供豐富的 API 介面,支援自定義開發。 + +## 📦 開始使用 + +#### Docker Compose 部署 + +```bash +git clone https://github.com/langbot-app/LangBot +cd LangBot +docker compose up -d +``` + +訪問 http://localhost:5300 即可開始使用。 + +詳細文件[Docker 部署](https://docs.langbot.app/zh/deploy/langbot/docker.html)。 + +#### 寶塔面板部署 + +已上架寶塔面板,若您已安裝寶塔面板,可以根據[文件](https://docs.langbot.app/zh/deploy/langbot/one-click/bt.html)使用。 + +#### Zeabur 雲端部署 + +社群貢獻的 Zeabur 模板。 + +[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/zh-CN/templates/ZKTBDH) + +#### Railway 雲端部署 + +[![Deploy on Railway](https://railway.com/button.svg)](https://railway.app/template/yRrAyL?referralCode=vogKPF) + +#### 手動部署 + +直接使用發行版運行,查看文件[手動部署](https://docs.langbot.app/zh/deploy/langbot/manual.html)。 + +## 😎 保持更新 + +點擊倉庫右上角 Star 和 Watch 按鈕,獲取最新動態。 + +![star gif](https://docs.langbot.app/star.gif) + +## ✨ 特性 + +- 💬 大模型對話、Agent:支援多種大模型,適配群聊和私聊;具有多輪對話、工具調用、多模態能力,自帶 RAG(知識庫)實現,並深度適配 [Dify](https://dify.ai)。 +- 🤖 多平台支援:目前支援 QQ、QQ頻道、企業微信、個人微信、飛書、Discord、Telegram 等平台。 +- 🛠️ 高穩定性、功能完備:原生支援訪問控制、限速、敏感詞過濾等機制;配置簡單,支援多種部署方式。支援多流水線配置,不同機器人用於不同應用場景。 +- 🧩 外掛擴展、活躍社群:支援事件驅動、組件擴展等外掛機制;適配 Anthropic [MCP 協議](https://modelcontextprotocol.io/);目前已有數百個外掛。 +- 😻 Web 管理面板:支援通過瀏覽器管理 LangBot 實例,不再需要手動編寫配置文件。 + +詳細規格特性請訪問[文件](https://docs.langbot.app/zh/insight/features.html)。 + +或訪問 demo 環境:https://demo.langbot.dev/ + - 登入資訊:郵箱:`demo@langbot.app` 密碼:`langbot123456` + - 注意:僅展示 WebUI 效果,公開環境,請不要在其中填入您的任何敏感資訊。 + +### 訊息平台 + +| 平台 | 狀態 | 備註 | +| --- | --- | --- | +| QQ 個人號 | ✅ | QQ 個人號私聊、群聊 | +| QQ 官方機器人 | ✅ | QQ 官方機器人,支援頻道、私聊、群聊 | +| 微信 | ✅ | | +| 企微對外客服 | ✅ | | +| 微信公眾號 | ✅ | | +| Lark | ✅ | | +| DingTalk | ✅ | | +| Discord | ✅ | | +| Telegram | ✅ | | +| Slack | ✅ | | + +### 大模型能力 + +| 模型 | 狀態 | 備註 | +| --- | --- | --- | +| [OpenAI](https://platform.openai.com/) | ✅ | 可接入任何 OpenAI 介面格式模型 | +| [DeepSeek](https://www.deepseek.com/) | ✅ | | +| [Moonshot](https://www.moonshot.cn/) | ✅ | | +| [Anthropic](https://www.anthropic.com/) | ✅ | | +| [xAI](https://x.ai/) | ✅ | | +| [智譜AI](https://open.bigmodel.cn/) | ✅ | | +| [優雲智算](https://www.compshare.cn/?ytag=GPU_YY-gh_langbot) | ✅ | 大模型和 GPU 資源平台 | +| [PPIO](https://ppinfra.com/user/register?invited_by=QJKFYD&utm_source=github_langbot) | ✅ | 大模型和 GPU 資源平台 | +| [302.AI](https://share.302.ai/SuTG99) | ✅ | 大模型聚合平台 | +| [Google Gemini](https://aistudio.google.com/prompts/new_chat) | ✅ | | +| [Dify](https://dify.ai) | ✅ | LLMOps 平台 | +| [Ollama](https://ollama.com/) | ✅ | 本地大模型運行平台 | +| [LMStudio](https://lmstudio.ai/) | ✅ | 本地大模型運行平台 | +| [GiteeAI](https://ai.gitee.com/) | ✅ | 大模型介面聚合平台 | +| [SiliconFlow](https://siliconflow.cn/) | ✅ | 大模型聚合平台 | +| [阿里雲百煉](https://bailian.console.aliyun.com/) | ✅ | 大模型聚合平台, LLMOps 平台 | +| [火山方舟](https://console.volcengine.com/ark/region:ark+cn-beijing/model?vendor=Bytedance&view=LIST_VIEW) | ✅ | 大模型聚合平台, LLMOps 平台 | +| [ModelScope](https://modelscope.cn/docs/model-service/API-Inference/intro) | ✅ | 大模型聚合平台 | +| [MCP](https://modelcontextprotocol.io/) | ✅ | 支援通過 MCP 協議獲取工具 | + +### TTS + +| 平台/模型 | 備註 | +| --- | --- | +| [FishAudio](https://fish.audio/zh-CN/discovery/) | [外掛](https://github.com/the-lazy-me/NewChatVoice) | +| [海豚 AI](https://www.ttson.cn/?source=thelazy) | [外掛](https://github.com/the-lazy-me/NewChatVoice) | +| [AzureTTS](https://portal.azure.com/) | [外掛](https://github.com/Ingnaryk/LangBot_AzureTTS) | + +### 文生圖 + +| 平台/模型 | 備註 | +| --- | --- | +| 阿里雲百煉 | [外掛](https://github.com/Thetail001/LangBot_BailianTextToImagePlugin) + +## 😘 社群貢獻 + +感謝以下[程式碼貢獻者](https://github.com/langbot-app/LangBot/graphs/contributors)和社群裡其他成員對 LangBot 的貢獻: + + + + \ No newline at end of file diff --git a/main.py b/main.py index 19cb32d6..1909e343 100644 --- a/main.py +++ b/main.py @@ -11,7 +11,7 @@ asciiart = r""" |____\__,_|_||_\__, |___/\___/\__| |___/ -⭐️ Open Source 开源地址: https://github.com/RockChinQ/LangBot +⭐️ Open Source 开源地址: https://github.com/langbot-app/LangBot 📖 Documentation 文档地址: https://docs.langbot.app """ diff --git a/pkg/api/http/controller/group.py b/pkg/api/http/controller/group.py index ce366539..16fa1df1 100644 --- a/pkg/api/http/controller/group.py +++ b/pkg/api/http/controller/group.py @@ -11,10 +11,10 @@ from ....core import app preregistered_groups: list[type[RouterGroup]] = [] -"""RouterGroup 的预注册列表""" +"""Pre-registered list of RouterGroup""" -def group_class(name: str, path: str) -> None: +def group_class(name: str, path: str) -> typing.Callable[[typing.Type[RouterGroup]], typing.Type[RouterGroup]]: """注册一个 RouterGroup""" def decorator(cls: typing.Type[RouterGroup]) -> typing.Type[RouterGroup]: @@ -27,7 +27,7 @@ def group_class(name: str, path: str) -> None: class AuthType(enum.Enum): - """认证类型""" + """Authentication type""" NONE = 'none' USER_TOKEN = 'user-token' @@ -56,7 +56,7 @@ class RouterGroup(abc.ABC): auth_type: AuthType = AuthType.USER_TOKEN, **options: typing.Any, ) -> typing.Callable[[RouteCallable], RouteCallable]: # decorator - """注册一个路由""" + """Register a route""" def decorator(f: RouteCallable) -> RouteCallable: nonlocal rule @@ -64,11 +64,11 @@ class RouterGroup(abc.ABC): async def handler_error(*args, **kwargs): if auth_type == AuthType.USER_TOKEN: - # 从Authorization头中获取token + # get token from Authorization header token = quart.request.headers.get('Authorization', '').replace('Bearer ', '') if not token: - return self.http_status(401, -1, '未提供有效的用户令牌') + return self.http_status(401, -1, 'No valid user token provided') try: user_email = await self.ap.user_service.verify_jwt_token(token) @@ -76,9 +76,9 @@ class RouterGroup(abc.ABC): # check if this account exists user = await self.ap.user_service.get_user_by_email(user_email) if not user: - return self.http_status(401, -1, '用户不存在') + return self.http_status(401, -1, 'User not found') - # 检查f是否接受user_email参数 + # check if f accepts user_email parameter if 'user_email' in f.__code__.co_varnames: kwargs['user_email'] = user_email except Exception as e: @@ -86,10 +86,11 @@ class RouterGroup(abc.ABC): try: return await f(*args, **kwargs) - except Exception: # 自动 500 + + except Exception as e: # 自动 500 traceback.print_exc() # return self.http_status(500, -2, str(e)) - return self.http_status(500, -2, 'internal server error') + return self.http_status(500, -2, str(e)) new_f = handler_error new_f.__name__ = (self.name + rule).replace('/', '__') @@ -101,7 +102,7 @@ class RouterGroup(abc.ABC): return decorator def success(self, data: typing.Any = None) -> quart.Response: - """返回一个 200 响应""" + """Return a 200 response""" return quart.jsonify( { 'code': 0, @@ -111,7 +112,7 @@ class RouterGroup(abc.ABC): ) def fail(self, code: int, msg: str) -> quart.Response: - """返回一个异常响应""" + """Return an error response""" return quart.jsonify( { @@ -120,6 +121,6 @@ class RouterGroup(abc.ABC): } ) - def http_status(self, status: int, code: int, msg: str) -> quart.Response: + def http_status(self, status: int, code: int, msg: str) -> typing.Tuple[quart.Response, int]: """返回一个指定状态码的响应""" - return self.fail(code, msg), status + return (self.fail(code, msg), status) \ No newline at end of file diff --git a/pkg/api/http/controller/groups/files.py b/pkg/api/http/controller/groups/files.py index 0a8b2210..b3c1a3f1 100644 --- a/pkg/api/http/controller/groups/files.py +++ b/pkg/api/http/controller/groups/files.py @@ -2,6 +2,10 @@ from __future__ import annotations import quart import mimetypes +import uuid +import asyncio + +import quart.datastructures from .. import group @@ -20,3 +24,23 @@ class FilesRouterGroup(group.RouterGroup): mime_type = 'image/jpeg' return quart.Response(image_bytes, mimetype=mime_type) + + @self.route('/documents', methods=['POST'], auth_type=group.AuthType.USER_TOKEN) + async def _() -> quart.Response: + request = quart.request + # get file bytes from 'file' + file = (await request.files)['file'] + assert isinstance(file, quart.datastructures.FileStorage) + + file_bytes = await asyncio.to_thread(file.stream.read) + extension = file.filename.split('.')[-1] + file_name = file.filename.split('.')[0] + + file_key = file_name + '_' + str(uuid.uuid4())[:8] + '.' + extension + # save file to storage + await self.ap.storage_mgr.storage_provider.save(file_key, file_bytes) + return self.success( + data={ + 'file_id': file_key, + } + ) diff --git a/pkg/api/http/controller/groups/knowledge/__init__.py b/pkg/api/http/controller/groups/knowledge/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/pkg/api/http/controller/groups/knowledge/base.py b/pkg/api/http/controller/groups/knowledge/base.py new file mode 100644 index 00000000..a5bed5df --- /dev/null +++ b/pkg/api/http/controller/groups/knowledge/base.py @@ -0,0 +1,90 @@ +import quart +from ... import group + + +@group.group_class('knowledge_base', '/api/v1/knowledge/bases') +class KnowledgeBaseRouterGroup(group.RouterGroup): + async def initialize(self) -> None: + @self.route('', methods=['POST', 'GET']) + async def handle_knowledge_bases() -> quart.Response: + if quart.request.method == 'GET': + knowledge_bases = await self.ap.knowledge_service.get_knowledge_bases() + return self.success(data={'bases': knowledge_bases}) + + elif quart.request.method == 'POST': + json_data = await quart.request.json + knowledge_base_uuid = await self.ap.knowledge_service.create_knowledge_base(json_data) + return self.success(data={'uuid': knowledge_base_uuid}) + + return self.http_status(405, -1, 'Method not allowed') + + @self.route( + '/', + methods=['GET', 'DELETE', 'PUT'], + ) + async def handle_specific_knowledge_base(knowledge_base_uuid: str) -> quart.Response: + if quart.request.method == 'GET': + knowledge_base = await self.ap.knowledge_service.get_knowledge_base(knowledge_base_uuid) + + if knowledge_base is None: + return self.http_status(404, -1, 'knowledge base not found') + + return self.success( + data={ + 'base': knowledge_base, + } + ) + + elif quart.request.method == 'PUT': + json_data = await quart.request.json + await self.ap.knowledge_service.update_knowledge_base(knowledge_base_uuid, json_data) + return self.success({}) + + elif quart.request.method == 'DELETE': + await self.ap.knowledge_service.delete_knowledge_base(knowledge_base_uuid) + return self.success({}) + + @self.route( + '//files', + methods=['GET', 'POST'], + ) + async def get_knowledge_base_files(knowledge_base_uuid: str) -> str: + if quart.request.method == 'GET': + files = await self.ap.knowledge_service.get_files_by_knowledge_base(knowledge_base_uuid) + return self.success( + data={ + 'files': files, + } + ) + + elif quart.request.method == 'POST': + json_data = await quart.request.json + file_id = json_data.get('file_id') + if not file_id: + return self.http_status(400, -1, 'File ID is required') + + # 调用服务层方法将文件与知识库关联 + task_id = await self.ap.knowledge_service.store_file(knowledge_base_uuid, file_id) + return self.success( + { + 'task_id': task_id, + } + ) + + @self.route( + '//files/', + methods=['DELETE'], + ) + async def delete_specific_file_in_kb(file_id: str, knowledge_base_uuid: str) -> str: + await self.ap.knowledge_service.delete_file(knowledge_base_uuid, file_id) + return self.success({}) + + @self.route( + '//retrieve', + methods=['POST'], + ) + async def retrieve_knowledge_base(knowledge_base_uuid: str) -> str: + json_data = await quart.request.json + query = json_data.get('query') + results = await self.ap.knowledge_service.retrieve_knowledge_base(knowledge_base_uuid, query) + return self.success(data={'results': results}) diff --git a/pkg/api/http/controller/groups/pipelines/pipelines.py b/pkg/api/http/controller/groups/pipelines/pipelines.py index 96ca239a..d056afb4 100644 --- a/pkg/api/http/controller/groups/pipelines/pipelines.py +++ b/pkg/api/http/controller/groups/pipelines/pipelines.py @@ -11,7 +11,9 @@ class PipelinesRouterGroup(group.RouterGroup): @self.route('', methods=['GET', 'POST']) async def _() -> str: if quart.request.method == 'GET': - return self.success(data={'pipelines': await self.ap.pipeline_service.get_pipelines()}) + sort_by = quart.request.args.get('sort_by', 'created_at') + sort_order = quart.request.args.get('sort_order', 'DESC') + return self.success(data={'pipelines': await self.ap.pipeline_service.get_pipelines(sort_by, sort_order)}) elif quart.request.method == 'POST': json_data = await quart.request.json diff --git a/pkg/api/http/controller/groups/pipelines/webchat.py b/pkg/api/http/controller/groups/pipelines/webchat.py index 62e5da3f..c094731b 100644 --- a/pkg/api/http/controller/groups/pipelines/webchat.py +++ b/pkg/api/http/controller/groups/pipelines/webchat.py @@ -10,7 +10,7 @@ class WebChatDebugRouterGroup(group.RouterGroup): async def initialize(self) -> None: @self.route('/send', methods=['POST']) async def send_message(pipeline_uuid: str) -> str: - """发送调试消息到流水线""" + """Send a message to the pipeline for debugging""" async def stream_generator(generator): async for message in generator: @@ -62,7 +62,7 @@ class WebChatDebugRouterGroup(group.RouterGroup): @self.route('/messages/', methods=['GET']) async def get_messages(pipeline_uuid: str, session_type: str) -> str: - """获取调试消息历史""" + """Get the message history of the pipeline for debugging""" try: if session_type not in ['person', 'group']: return self.http_status(400, -1, 'session_type must be person or group') @@ -81,7 +81,7 @@ class WebChatDebugRouterGroup(group.RouterGroup): @self.route('/reset/', methods=['POST']) async def reset_session(session_type: str) -> str: - """重置调试会话""" + """Reset the debug session""" try: if session_type not in ['person', 'group']: return self.http_status(400, -1, 'session_type must be person or group') diff --git a/pkg/api/http/controller/groups/plugins.py b/pkg/api/http/controller/groups/plugins.py index daf6ea7d..b7e0a5e9 100644 --- a/pkg/api/http/controller/groups/plugins.py +++ b/pkg/api/http/controller/groups/plugins.py @@ -40,7 +40,7 @@ class PluginsRouterGroup(group.RouterGroup): self.ap.plugin_mgr.update_plugin(plugin_name, task_context=ctx), kind='plugin-operation', name=f'plugin-update-{plugin_name}', - label=f'更新插件 {plugin_name}', + label=f'Updating plugin {plugin_name}', context=ctx, ) return self.success(data={'task_id': wrapper.id}) @@ -62,7 +62,7 @@ class PluginsRouterGroup(group.RouterGroup): self.ap.plugin_mgr.uninstall_plugin(plugin_name, task_context=ctx), kind='plugin-operation', name=f'plugin-remove-{plugin_name}', - label=f'删除插件 {plugin_name}', + label=f'Removing plugin {plugin_name}', context=ctx, ) @@ -102,7 +102,7 @@ class PluginsRouterGroup(group.RouterGroup): self.ap.plugin_mgr.install_plugin(data['source'], task_context=ctx), kind='plugin-operation', name='plugin-install-github', - label=f'安装插件 ...{short_source_str}', + label=f'Installing plugin ...{short_source_str}', context=ctx, ) diff --git a/pkg/api/http/controller/groups/provider/models.py b/pkg/api/http/controller/groups/provider/models.py index bb77986c..0de0c922 100644 --- a/pkg/api/http/controller/groups/provider/models.py +++ b/pkg/api/http/controller/groups/provider/models.py @@ -9,18 +9,18 @@ class LLMModelsRouterGroup(group.RouterGroup): @self.route('', methods=['GET', 'POST']) async def _() -> str: if quart.request.method == 'GET': - return self.success(data={'models': await self.ap.model_service.get_llm_models()}) + return self.success(data={'models': await self.ap.llm_model_service.get_llm_models()}) elif quart.request.method == 'POST': json_data = await quart.request.json - model_uuid = await self.ap.model_service.create_llm_model(json_data) + model_uuid = await self.ap.llm_model_service.create_llm_model(json_data) return self.success(data={'uuid': model_uuid}) @self.route('/', methods=['GET', 'PUT', 'DELETE']) async def _(model_uuid: str) -> str: if quart.request.method == 'GET': - model = await self.ap.model_service.get_llm_model(model_uuid) + model = await self.ap.llm_model_service.get_llm_model(model_uuid) if model is None: return self.http_status(404, -1, 'model not found') @@ -29,11 +29,11 @@ class LLMModelsRouterGroup(group.RouterGroup): elif quart.request.method == 'PUT': json_data = await quart.request.json - await self.ap.model_service.update_llm_model(model_uuid, json_data) + await self.ap.llm_model_service.update_llm_model(model_uuid, json_data) return self.success() elif quart.request.method == 'DELETE': - await self.ap.model_service.delete_llm_model(model_uuid) + await self.ap.llm_model_service.delete_llm_model(model_uuid) return self.success() @@ -41,6 +41,49 @@ class LLMModelsRouterGroup(group.RouterGroup): async def _(model_uuid: str) -> str: json_data = await quart.request.json - await self.ap.model_service.test_llm_model(model_uuid, json_data) + await self.ap.llm_model_service.test_llm_model(model_uuid, json_data) + + return self.success() + + +@group.group_class('models/embedding', '/api/v1/provider/models/embedding') +class EmbeddingModelsRouterGroup(group.RouterGroup): + async def initialize(self) -> None: + @self.route('', methods=['GET', 'POST']) + async def _() -> str: + if quart.request.method == 'GET': + return self.success(data={'models': await self.ap.embedding_models_service.get_embedding_models()}) + elif quart.request.method == 'POST': + json_data = await quart.request.json + + model_uuid = await self.ap.embedding_models_service.create_embedding_model(json_data) + + return self.success(data={'uuid': model_uuid}) + + @self.route('/', methods=['GET', 'PUT', 'DELETE']) + async def _(model_uuid: str) -> str: + if quart.request.method == 'GET': + model = await self.ap.embedding_models_service.get_embedding_model(model_uuid) + + if model is None: + return self.http_status(404, -1, 'model not found') + + return self.success(data={'model': model}) + elif quart.request.method == 'PUT': + json_data = await quart.request.json + + await self.ap.embedding_models_service.update_embedding_model(model_uuid, json_data) + + return self.success() + elif quart.request.method == 'DELETE': + await self.ap.embedding_models_service.delete_embedding_model(model_uuid) + + return self.success() + + @self.route('//test', methods=['POST']) + async def _(model_uuid: str) -> str: + json_data = await quart.request.json + + await self.ap.embedding_models_service.test_embedding_model(model_uuid, json_data) return self.success() diff --git a/pkg/api/http/controller/groups/provider/requesters.py b/pkg/api/http/controller/groups/provider/requesters.py index 0f999288..af9e1540 100644 --- a/pkg/api/http/controller/groups/provider/requesters.py +++ b/pkg/api/http/controller/groups/provider/requesters.py @@ -8,7 +8,8 @@ class RequestersRouterGroup(group.RouterGroup): async def initialize(self) -> None: @self.route('', methods=['GET']) async def _() -> quart.Response: - return self.success(data={'requesters': self.ap.model_mgr.get_available_requesters_info()}) + model_type = quart.request.args.get('type', '') + return self.success(data={'requesters': self.ap.model_mgr.get_available_requesters_info(model_type)}) @self.route('/', methods=['GET']) async def _(requester_name: str) -> quart.Response: diff --git a/pkg/api/http/controller/groups/user.py b/pkg/api/http/controller/groups/user.py index 498efaa4..d8024107 100644 --- a/pkg/api/http/controller/groups/user.py +++ b/pkg/api/http/controller/groups/user.py @@ -1,5 +1,6 @@ import quart import argon2 +import asyncio from .. import group @@ -13,7 +14,7 @@ class UserRouterGroup(group.RouterGroup): return self.success(data={'initialized': await self.ap.user_service.is_initialized()}) if await self.ap.user_service.is_initialized(): - return self.fail(1, '系统已初始化') + return self.fail(1, 'System already initialized') json_data = await quart.request.json @@ -31,7 +32,7 @@ class UserRouterGroup(group.RouterGroup): try: token = await self.ap.user_service.authenticate(json_data['user'], json_data['password']) except argon2.exceptions.VerifyMismatchError: - return self.fail(1, '用户名或密码错误') + return self.fail(1, 'Invalid username or password') return self.success(data={'token': token}) @@ -40,3 +41,29 @@ class UserRouterGroup(group.RouterGroup): token = await self.ap.user_service.generate_jwt_token(user_email) return self.success(data={'token': token}) + + @self.route('/reset-password', methods=['POST'], auth_type=group.AuthType.NONE) + async def _() -> str: + json_data = await quart.request.json + + user_email = json_data['user'] + recovery_key = json_data['recovery_key'] + new_password = json_data['new_password'] + + # hard sleep 3s for security + await asyncio.sleep(3) + + if not await self.ap.user_service.is_initialized(): + return self.http_status(400, -1, 'System not initialized') + + user_obj = await self.ap.user_service.get_user_by_email(user_email) + + if user_obj is None: + return self.http_status(400, -1, 'User not found') + + if recovery_key != self.ap.instance_config.data['system']['recovery_key']: + return self.http_status(403, -1, 'Invalid recovery key') + + await self.ap.user_service.reset_password(user_email, new_password) + + return self.success(data={'user': user_email}) diff --git a/pkg/api/http/controller/main.py b/pkg/api/http/controller/main.py index eb434d88..e45b461d 100644 --- a/pkg/api/http/controller/main.py +++ b/pkg/api/http/controller/main.py @@ -14,11 +14,13 @@ from . import group from .groups import provider as groups_provider from .groups import platform as groups_platform from .groups import pipelines as groups_pipelines +from .groups import knowledge as groups_knowledge importutil.import_modules_in_pkg(groups) importutil.import_modules_in_pkg(groups_provider) importutil.import_modules_in_pkg(groups_platform) importutil.import_modules_in_pkg(groups_pipelines) +importutil.import_modules_in_pkg(groups_knowledge) class HTTPController: @@ -45,7 +47,7 @@ class HTTPController: try: await self.quart_app.run_task(*args, **kwargs) except Exception as e: - self.ap.logger.error(f'启动 HTTP 服务失败: {e}') + self.ap.logger.error(f'Failed to start HTTP service: {e}') self.ap.task_mgr.create_task( exception_handler( diff --git a/pkg/api/http/service/bot.py b/pkg/api/http/service/bot.py index e5010007..adf19d03 100644 --- a/pkg/api/http/service/bot.py +++ b/pkg/api/http/service/bot.py @@ -10,7 +10,7 @@ from ....entity.persistence import pipeline as persistence_pipeline class BotService: - """机器人服务""" + """Bot service""" ap: app.Application @@ -18,7 +18,7 @@ class BotService: self.ap = ap async def get_bots(self) -> list[dict]: - """获取所有机器人""" + """Get all bots""" result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_bot.Bot)) bots = result.all() @@ -26,7 +26,7 @@ class BotService: return [self.ap.persistence_mgr.serialize_model(persistence_bot.Bot, bot) for bot in bots] async def get_bot(self, bot_uuid: str) -> dict | None: - """获取机器人""" + """Get bot""" result = await self.ap.persistence_mgr.execute_async( sqlalchemy.select(persistence_bot.Bot).where(persistence_bot.Bot.uuid == bot_uuid) ) @@ -39,7 +39,7 @@ class BotService: return self.ap.persistence_mgr.serialize_model(persistence_bot.Bot, bot) async def create_bot(self, bot_data: dict) -> str: - """创建机器人""" + """Create bot""" # TODO: 检查配置信息格式 bot_data['uuid'] = str(uuid.uuid4()) @@ -63,7 +63,7 @@ class BotService: return bot_data['uuid'] async def update_bot(self, bot_uuid: str, bot_data: dict) -> None: - """更新机器人""" + """Update bot""" if 'uuid' in bot_data: del bot_data['uuid'] @@ -99,7 +99,7 @@ class BotService: session.using_conversation = None async def delete_bot(self, bot_uuid: str) -> None: - """删除机器人""" + """Delete bot""" await self.ap.platform_mgr.remove_bot(bot_uuid) await self.ap.persistence_mgr.execute_async( sqlalchemy.delete(persistence_bot.Bot).where(persistence_bot.Bot.uuid == bot_uuid) diff --git a/pkg/api/http/service/knowledge.py b/pkg/api/http/service/knowledge.py new file mode 100644 index 00000000..27506ec9 --- /dev/null +++ b/pkg/api/http/service/knowledge.py @@ -0,0 +1,118 @@ +from __future__ import annotations + +import uuid +import sqlalchemy + +from ....core import app +from ....entity.persistence import rag as persistence_rag + + +class KnowledgeService: + """知识库服务""" + + ap: app.Application + + def __init__(self, ap: app.Application) -> None: + self.ap = ap + + async def get_knowledge_bases(self) -> list[dict]: + """获取所有知识库""" + result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase)) + knowledge_bases = result.all() + return [ + self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base) + for knowledge_base in knowledge_bases + ] + + async def get_knowledge_base(self, kb_uuid: str) -> dict | None: + """获取知识库""" + result = await self.ap.persistence_mgr.execute_async( + sqlalchemy.select(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid) + ) + knowledge_base = result.first() + if knowledge_base is None: + return None + return self.ap.persistence_mgr.serialize_model(persistence_rag.KnowledgeBase, knowledge_base) + + async def create_knowledge_base(self, kb_data: dict) -> str: + """创建知识库""" + kb_data['uuid'] = str(uuid.uuid4()) + await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.KnowledgeBase).values(kb_data)) + + kb = await self.get_knowledge_base(kb_data['uuid']) + + await self.ap.rag_mgr.load_knowledge_base(kb) + + return kb_data['uuid'] + + async def update_knowledge_base(self, kb_uuid: str, kb_data: dict) -> None: + """更新知识库""" + if 'uuid' in kb_data: + del kb_data['uuid'] + + if 'embedding_model_uuid' in kb_data: + del kb_data['embedding_model_uuid'] + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_rag.KnowledgeBase) + .values(kb_data) + .where(persistence_rag.KnowledgeBase.uuid == kb_uuid) + ) + await self.ap.rag_mgr.remove_knowledge_base_from_runtime(kb_uuid) + + kb = await self.get_knowledge_base(kb_uuid) + + await self.ap.rag_mgr.load_knowledge_base(kb) + + async def store_file(self, kb_uuid: str, file_id: str) -> int: + """存储文件""" + # await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.File).values(kb_id=kb_uuid, file_id=file_id)) + # await self.ap.rag_mgr.store_file(file_id) + runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid) + if runtime_kb is None: + raise Exception('Knowledge base not found') + return await runtime_kb.store_file(file_id) + + async def retrieve_knowledge_base(self, kb_uuid: str, query: str) -> list[dict]: + """检索知识库""" + runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid) + if runtime_kb is None: + raise Exception('Knowledge base not found') + return [result.model_dump() for result in await runtime_kb.retrieve(query)] + + async def get_files_by_knowledge_base(self, kb_uuid: str) -> list[dict]: + """获取知识库文件""" + result = await self.ap.persistence_mgr.execute_async( + sqlalchemy.select(persistence_rag.File).where(persistence_rag.File.kb_id == kb_uuid) + ) + files = result.all() + return [self.ap.persistence_mgr.serialize_model(persistence_rag.File, file) for file in files] + + async def delete_file(self, kb_uuid: str, file_id: str) -> None: + """删除文件""" + runtime_kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid) + if runtime_kb is None: + raise Exception('Knowledge base not found') + await runtime_kb.delete_file(file_id) + + async def delete_knowledge_base(self, kb_uuid: str) -> None: + """删除知识库""" + await self.ap.rag_mgr.delete_knowledge_base(kb_uuid) + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_rag.KnowledgeBase).where(persistence_rag.KnowledgeBase.uuid == kb_uuid) + ) + + # delete files + files = await self.ap.persistence_mgr.execute_async( + sqlalchemy.select(persistence_rag.File).where(persistence_rag.File.kb_id == kb_uuid) + ) + for file in files: + # delete chunks + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_rag.Chunk).where(persistence_rag.Chunk.file_id == file.uuid) + ) + # delete file + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file.uuid) + ) diff --git a/pkg/api/http/service/model.py b/pkg/api/http/service/model.py index 74fb4e02..d8457da3 100644 --- a/pkg/api/http/service/model.py +++ b/pkg/api/http/service/model.py @@ -10,7 +10,7 @@ from ....provider.modelmgr import requester as model_requester from ....provider import entities as llm_entities -class ModelsService: +class LLMModelsService: ap: app.Application def __init__(self, ap: app.Application) -> None: @@ -103,3 +103,89 @@ class ModelsService: funcs=[], extra_args={}, ) + + +class EmbeddingModelsService: + ap: app.Application + + def __init__(self, ap: app.Application) -> None: + self.ap = ap + + async def get_embedding_models(self) -> list[dict]: + result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.EmbeddingModel)) + + models = result.all() + return [self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model) for model in models] + + async def create_embedding_model(self, model_data: dict) -> str: + model_data['uuid'] = str(uuid.uuid4()) + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.insert(persistence_model.EmbeddingModel).values(**model_data) + ) + + embedding_model = await self.get_embedding_model(model_data['uuid']) + + await self.ap.model_mgr.load_embedding_model(embedding_model) + + return model_data['uuid'] + + async def get_embedding_model(self, model_uuid: str) -> dict | None: + result = await self.ap.persistence_mgr.execute_async( + sqlalchemy.select(persistence_model.EmbeddingModel).where( + persistence_model.EmbeddingModel.uuid == model_uuid + ) + ) + + model = result.first() + + if model is None: + return None + + return self.ap.persistence_mgr.serialize_model(persistence_model.EmbeddingModel, model) + + async def update_embedding_model(self, model_uuid: str, model_data: dict) -> None: + if 'uuid' in model_data: + del model_data['uuid'] + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_model.EmbeddingModel) + .where(persistence_model.EmbeddingModel.uuid == model_uuid) + .values(**model_data) + ) + + await self.ap.model_mgr.remove_embedding_model(model_uuid) + + embedding_model = await self.get_embedding_model(model_uuid) + + await self.ap.model_mgr.load_embedding_model(embedding_model) + + async def delete_embedding_model(self, model_uuid: str) -> None: + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_model.EmbeddingModel).where( + persistence_model.EmbeddingModel.uuid == model_uuid + ) + ) + + await self.ap.model_mgr.remove_embedding_model(model_uuid) + + async def test_embedding_model(self, model_uuid: str, model_data: dict) -> None: + runtime_embedding_model: model_requester.RuntimeEmbeddingModel | None = None + + if model_uuid != '_': + for model in self.ap.model_mgr.embedding_models: + if model.model_entity.uuid == model_uuid: + runtime_embedding_model = model + break + + if runtime_embedding_model is None: + raise Exception('model not found') + + else: + runtime_embedding_model = await self.ap.model_mgr.init_runtime_embedding_model(model_data) + + await runtime_embedding_model.requester.invoke_embedding( + model=runtime_embedding_model, + input_text=['Hello, world!'], + extra_args={}, + ) diff --git a/pkg/api/http/service/pipeline.py b/pkg/api/http/service/pipeline.py index f0f6c083..96504d61 100644 --- a/pkg/api/http/service/pipeline.py +++ b/pkg/api/http/service/pipeline.py @@ -38,9 +38,21 @@ class PipelineService: self.ap.pipeline_config_meta_output.data, ] - async def get_pipelines(self) -> list[dict]: - result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline)) - + async def get_pipelines(self, sort_by: str = 'created_at', sort_order: str = 'DESC') -> list[dict]: + query = sqlalchemy.select(persistence_pipeline.LegacyPipeline) + + if sort_by == 'created_at': + if sort_order == 'DESC': + query = query.order_by(persistence_pipeline.LegacyPipeline.created_at.desc()) + else: + query = query.order_by(persistence_pipeline.LegacyPipeline.created_at.asc()) + elif sort_by == 'updated_at': + if sort_order == 'DESC': + query = query.order_by(persistence_pipeline.LegacyPipeline.updated_at.desc()) + else: + query = query.order_by(persistence_pipeline.LegacyPipeline.updated_at.asc()) + + result = await self.ap.persistence_mgr.execute_async(query) pipelines = result.all() return [ self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline) diff --git a/pkg/api/http/service/user.py b/pkg/api/http/service/user.py index 782aad75..c724bfcf 100644 --- a/pkg/api/http/service/user.py +++ b/pkg/api/http/service/user.py @@ -73,3 +73,12 @@ class UserService: jwt_secret = self.ap.instance_config.data['system']['jwt']['secret'] return jwt.decode(token, jwt_secret, algorithms=['HS256'])['user'] + + async def reset_password(self, user_email: str, new_password: str) -> None: + ph = argon2.PasswordHasher() + + hashed_password = ph.hash(new_password) + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(user.User).where(user.User.user == user_email).values(password=hashed_password) + ) diff --git a/pkg/config/impls/json.py b/pkg/config/impls/json.py index 07fc533c..44b4843c 100644 --- a/pkg/config/impls/json.py +++ b/pkg/config/impls/json.py @@ -6,7 +6,7 @@ from .. import model as file_model class JSONConfigFile(file_model.ConfigFile): - """JSON配置文件""" + """JSON config file""" def __init__( self, @@ -42,7 +42,7 @@ class JSONConfigFile(file_model.ConfigFile): try: cfg = json.load(f) except json.JSONDecodeError as e: - raise Exception(f'配置文件 {self.config_file_name} 语法错误: {e}') + raise Exception(f'Syntax error in config file {self.config_file_name}: {e}') if completion: for key in self.template_data: diff --git a/pkg/config/impls/pymodule.py b/pkg/config/impls/pymodule.py index 2311992e..c3d04bc8 100644 --- a/pkg/config/impls/pymodule.py +++ b/pkg/config/impls/pymodule.py @@ -7,13 +7,13 @@ from .. import model as file_model class PythonModuleConfigFile(file_model.ConfigFile): - """Python模块配置文件""" + """Python module config file""" config_file_name: str = None - """配置文件名""" + """Config file name""" template_file_name: str = None - """模板文件名""" + """Template file name""" def __init__(self, config_file_name: str, template_file_name: str) -> None: self.config_file_name = config_file_name @@ -42,7 +42,7 @@ class PythonModuleConfigFile(file_model.ConfigFile): cfg[key] = getattr(module, key) - # 从模板模块文件中进行补全 + # complete from template module file if completion: module_name = os.path.splitext(os.path.basename(self.template_file_name))[0] module = importlib.import_module(module_name) @@ -60,7 +60,7 @@ class PythonModuleConfigFile(file_model.ConfigFile): return cfg async def save(self, data: dict): - logging.warning('Python模块配置文件不支持保存') + logging.warning('Python module config file does not support saving') def save_sync(self, data: dict): - logging.warning('Python模块配置文件不支持保存') + logging.warning('Python module config file does not support saving') diff --git a/pkg/config/impls/yaml.py b/pkg/config/impls/yaml.py index 55045186..0d69ef9e 100644 --- a/pkg/config/impls/yaml.py +++ b/pkg/config/impls/yaml.py @@ -6,7 +6,7 @@ from .. import model as file_model class YAMLConfigFile(file_model.ConfigFile): - """YAML配置文件""" + """YAML config file""" def __init__( self, @@ -42,7 +42,7 @@ class YAMLConfigFile(file_model.ConfigFile): try: cfg = yaml.load(f, Loader=yaml.FullLoader) except yaml.YAMLError as e: - raise Exception(f'配置文件 {self.config_file_name} 语法错误: {e}') + raise Exception(f'Syntax error in config file {self.config_file_name}: {e}') if completion: for key in self.template_data: diff --git a/pkg/config/manager.py b/pkg/config/manager.py index c2e6bdf4..d552b038 100644 --- a/pkg/config/manager.py +++ b/pkg/config/manager.py @@ -5,27 +5,27 @@ from .impls import pymodule, json as json_file, yaml as yaml_file class ConfigManager: - """配置文件管理器""" + """Config file manager""" name: str = None - """配置管理器名""" + """Config manager name""" description: str = None - """配置管理器描述""" + """Config manager description""" schema: dict = None - """配置文件 schema - 需要符合 JSON Schema Draft 7 规范 + """Config file schema + Must conform to JSON Schema Draft 7 specification """ file: file_model.ConfigFile = None - """配置文件实例""" + """Config file instance""" data: dict = None - """配置数据""" + """Config data""" doc_link: str = None - """配置文件文档链接""" + """Config file documentation link""" def __init__(self, cfg_file: file_model.ConfigFile) -> None: self.file = cfg_file @@ -42,15 +42,15 @@ class ConfigManager: async def load_python_module_config(config_name: str, template_name: str, completion: bool = True) -> ConfigManager: - """加载Python模块配置文件 + """Load Python module config file Args: - config_name (str): 配置文件名 - template_name (str): 模板文件名 - completion (bool): 是否自动补全内存中的配置文件 + config_name (str): Config file name + template_name (str): Template file name + completion (bool): Whether to automatically complete the config file in memory Returns: - ConfigManager: 配置文件管理器 + ConfigManager: Config file manager """ cfg_inst = pymodule.PythonModuleConfigFile(config_name, template_name) @@ -66,13 +66,13 @@ async def load_json_config( template_data: dict = None, completion: bool = True, ) -> ConfigManager: - """加载JSON配置文件 + """Load JSON config file Args: - config_name (str): 配置文件名 - template_name (str): 模板文件名 - template_data (dict): 模板数据 - completion (bool): 是否自动补全内存中的配置文件 + config_name (str): Config file name + template_name (str): Template file name + template_data (dict): Template data + completion (bool): Whether to automatically complete the config file in memory """ cfg_inst = json_file.JSONConfigFile(config_name, template_name, template_data) @@ -88,16 +88,16 @@ async def load_yaml_config( template_data: dict = None, completion: bool = True, ) -> ConfigManager: - """加载YAML配置文件 + """Load YAML config file Args: - config_name (str): 配置文件名 - template_name (str): 模板文件名 - template_data (dict): 模板数据 - completion (bool): 是否自动补全内存中的配置文件 + config_name (str): Config file name + template_name (str): Template file name + template_data (dict): Template data + completion (bool): Whether to automatically complete the config file in memory Returns: - ConfigManager: 配置文件管理器 + ConfigManager: Config file manager """ cfg_inst = yaml_file.YAMLConfigFile(config_name, template_name, template_data) diff --git a/pkg/config/model.py b/pkg/config/model.py index f3536804..8b040f05 100644 --- a/pkg/config/model.py +++ b/pkg/config/model.py @@ -2,16 +2,16 @@ import abc class ConfigFile(metaclass=abc.ABCMeta): - """配置文件抽象类""" + """Config file abstract class""" config_file_name: str = None - """配置文件名""" + """Config file name""" template_file_name: str = None - """模板文件名""" + """Template file name""" template_data: dict = None - """模板数据""" + """Template data""" @abc.abstractmethod def exists(self) -> bool: diff --git a/pkg/core/app.py b/pkg/core/app.py index 911acd3d..21816cfc 100644 --- a/pkg/core/app.py +++ b/pkg/core/app.py @@ -22,15 +22,18 @@ from ..api.http.service import user as user_service from ..api.http.service import model as model_service from ..api.http.service import pipeline as pipeline_service from ..api.http.service import bot as bot_service +from ..api.http.service import knowledge as knowledge_service from ..discover import engine as discover_engine from ..storage import mgr as storagemgr from ..utils import logcache from . import taskmgr from . import entities as core_entities +from ..rag.knowledge import kbmgr as rag_mgr +from ..vector import mgr as vectordb_mgr class Application: - """运行时应用对象和上下文""" + """Runtime application object and context""" event_loop: asyncio.AbstractEventLoop = None @@ -47,10 +50,12 @@ class Application: model_mgr: llm_model_mgr.ModelManager = None - # TODO 移动到 pipeline 里 + rag_mgr: rag_mgr.RAGManager = None + + # TODO move to pipeline tool_mgr: llm_tool_mgr.ToolManager = None - # ======= 配置管理器 ======= + # ======= Config manager ======= command_cfg: config_mgr.ConfigManager = None # deprecated @@ -64,7 +69,7 @@ class Application: instance_config: config_mgr.ConfigManager = None - # ======= 元数据配置管理器 ======= + # ======= Metadata config manager ======= sensitive_meta: config_mgr.ConfigManager = None @@ -93,6 +98,8 @@ class Application: persistence_mgr: persistencemgr.PersistenceManager = None + vector_db_mgr: vectordb_mgr.VectorDBManager = None + http_ctrl: http_controller.HTTPController = None log_cache: logcache.LogCache = None @@ -103,12 +110,16 @@ class Application: user_service: user_service.UserService = None - model_service: model_service.ModelsService = None + llm_model_service: model_service.LLMModelsService = None + + embedding_models_service: model_service.EmbeddingModelsService = None pipeline_service: pipeline_service.PipelineService = None bot_service: bot_service.BotService = None + knowledge_service: knowledge_service.KnowledgeService = None + def __init__(self): pass @@ -143,6 +154,7 @@ class Application: name='http-api-controller', scopes=[core_entities.LifecycleControlScope.APPLICATION], ) + self.task_mgr.create_task( never_ending(), name='never-ending-task', @@ -154,11 +166,11 @@ class Application: except asyncio.CancelledError: pass except Exception as e: - self.logger.error(f'应用运行致命异常: {e}') + self.logger.error(f'Application runtime fatal exception: {e}') self.logger.debug(f'Traceback: {traceback.format_exc()}') async def print_web_access_info(self): - """打印访问 webui 的提示""" + """Print access webui tips""" if not os.path.exists(os.path.join('.', 'web/out')): self.logger.warning('WebUI 文件缺失,请根据文档部署:https://docs.langbot.app/zh') @@ -190,7 +202,7 @@ class Application: ): match scope: case core_entities.LifecycleControlScope.PLATFORM.value: - self.logger.info('执行热重载 scope=' + scope) + self.logger.info('Hot reload scope=' + scope) await self.platform_mgr.shutdown() self.platform_mgr = im_mgr.PlatformManager(self) @@ -206,7 +218,7 @@ class Application: ], ) case core_entities.LifecycleControlScope.PLUGIN.value: - self.logger.info('执行热重载 scope=' + scope) + self.logger.info('Hot reload scope=' + scope) await self.plugin_mgr.destroy_plugins() # 删除 sys.module 中所有的 plugins/* 下的模块 @@ -222,7 +234,7 @@ class Application: await self.plugin_mgr.load_plugins() await self.plugin_mgr.initialize_plugins() case core_entities.LifecycleControlScope.PROVIDER.value: - self.logger.info('执行热重载 scope=' + scope) + self.logger.info('Hot reload scope=' + scope) await self.tool_mgr.shutdown() diff --git a/pkg/core/boot.py b/pkg/core/boot.py index aff117e6..b8243d4a 100644 --- a/pkg/core/boot.py +++ b/pkg/core/boot.py @@ -8,7 +8,7 @@ from . import app from . import stage from ..utils import constants, importutil -# 引入启动阶段实现以便注册 +# Import startup stage implementation to register from . import stages importutil.import_modules_in_pkg(stages) @@ -25,7 +25,7 @@ stage_order = [ async def make_app(loop: asyncio.AbstractEventLoop) -> app.Application: - # 确定是否为调试模式 + # Determine if it is debug mode if 'DEBUG' in os.environ and os.environ['DEBUG'] in ['true', '1']: constants.debug_mode = True @@ -33,7 +33,7 @@ async def make_app(loop: asyncio.AbstractEventLoop) -> app.Application: ap.event_loop = loop - # 执行启动阶段 + # Execute startup stage for stage_name in stage_order: stage_cls = stage.preregistered_stages[stage_name] stage_inst = stage_cls() @@ -47,11 +47,11 @@ async def make_app(loop: asyncio.AbstractEventLoop) -> app.Application: async def main(loop: asyncio.AbstractEventLoop): try: - # 挂系统信号处理 + # Hang system signal processing import signal def signal_handler(sig, frame): - print('[Signal] 程序退出.') + print('[Signal] Program exit.') # ap.shutdown() os._exit(0) diff --git a/pkg/core/bootutils/deps.py b/pkg/core/bootutils/deps.py index b403bf8d..1a439af8 100644 --- a/pkg/core/bootutils/deps.py +++ b/pkg/core/bootutils/deps.py @@ -2,8 +2,8 @@ import pip import os from ...utils import pkgmgr -# 检查依赖,防止用户未安装 -# 左边为引入名称,右边为依赖名称 +# Check dependencies to prevent users from not installing +# Left is the import name, right is the dependency name required_deps = { 'requests': 'requests', 'openai': 'openai', @@ -65,7 +65,7 @@ async def install_deps(deps: list[str]): async def precheck_plugin_deps(): print('[Startup] Prechecking plugin dependencies...') - # 只有在plugins目录存在时才执行插件依赖安装 + # Only execute plugin dependency installation when the plugins directory exists if os.path.exists('plugins'): for dir in os.listdir('plugins'): subdir = os.path.join('plugins', dir) diff --git a/pkg/core/bootutils/log.py b/pkg/core/bootutils/log.py index eb6806fa..631b05e2 100644 --- a/pkg/core/bootutils/log.py +++ b/pkg/core/bootutils/log.py @@ -17,7 +17,7 @@ log_colors_config = { async def init_logging(extra_handlers: list[logging.Handler] = None) -> logging.Logger: - # 删除所有现有的logger + # Remove all existing loggers for handler in logging.root.handlers[:]: logging.root.removeHandler(handler) @@ -54,13 +54,13 @@ async def init_logging(extra_handlers: list[logging.Handler] = None) -> logging. handler.setFormatter(color_formatter) qcg_logger.addHandler(handler) - qcg_logger.debug('日志初始化完成,日志级别:%s' % level) + qcg_logger.debug('Logging initialized, log level: %s' % level) logging.basicConfig( - level=logging.CRITICAL, # 设置日志输出格式 + level=logging.CRITICAL, # Set log output format format='[DEPR][%(asctime)s.%(msecs)03d] %(pathname)s (%(lineno)d) - [%(levelname)s] :\n%(message)s', - # 日志输出的格式 - # -8表示占位符,让输出左对齐,输出长度都为8位 - datefmt='%Y-%m-%d %H:%M:%S', # 时间输出的格式 + # Log output format + # -8 is a placeholder, left-align the output, and output length is 8 + datefmt='%Y-%m-%d %H:%M:%S', # Time output format handlers=[logging.NullHandler()], ) diff --git a/pkg/core/entities.py b/pkg/core/entities.py index 1efee3fc..5f357d78 100644 --- a/pkg/core/entities.py +++ b/pkg/core/entities.py @@ -19,7 +19,7 @@ class LifecycleControlScope(enum.Enum): APPLICATION = 'application' PLATFORM = 'platform' PLUGIN = 'plugin' - PROVIDER = 'provider' + PROVIDER = 'provider' class LauncherTypes(enum.Enum): diff --git a/pkg/core/migration.py b/pkg/core/migration.py index e97c0cf3..a921e6c7 100644 --- a/pkg/core/migration.py +++ b/pkg/core/migration.py @@ -7,11 +7,11 @@ from . import app preregistered_migrations: list[typing.Type[Migration]] = [] -"""当前阶段暂不支持扩展""" +"""Currently not supported for extension""" def migration_class(name: str, number: int): - """注册一个迁移""" + """Register a migration""" def decorator(cls: typing.Type[Migration]) -> typing.Type[Migration]: cls.name = name @@ -23,7 +23,7 @@ def migration_class(name: str, number: int): class Migration(abc.ABC): - """一个版本的迁移""" + """A version migration""" name: str @@ -36,10 +36,10 @@ class Migration(abc.ABC): @abc.abstractmethod async def need_migrate(self) -> bool: - """判断当前环境是否需要运行此迁移""" + """Determine if the current environment needs to run this migration""" pass @abc.abstractmethod async def run(self): - """执行迁移""" + """Run migration""" pass diff --git a/pkg/core/note.py b/pkg/core/note.py index 07171581..b4c37ce1 100644 --- a/pkg/core/note.py +++ b/pkg/core/note.py @@ -9,7 +9,7 @@ preregistered_notes: list[typing.Type[LaunchNote]] = [] def note_class(name: str, number: int): - """注册一个启动信息""" + """Register a launch information""" def decorator(cls: typing.Type[LaunchNote]) -> typing.Type[LaunchNote]: cls.name = name @@ -21,7 +21,7 @@ def note_class(name: str, number: int): class LaunchNote(abc.ABC): - """启动信息""" + """Launch information""" name: str @@ -34,10 +34,10 @@ class LaunchNote(abc.ABC): @abc.abstractmethod async def need_show(self) -> bool: - """判断当前环境是否需要显示此启动信息""" + """Determine if the current environment needs to display this launch information""" pass @abc.abstractmethod async def yield_note(self) -> typing.AsyncGenerator[typing.Tuple[str, int], None]: - """生成启动信息""" + """Generate launch information""" pass diff --git a/pkg/core/notes/n001_classic_msgs.py b/pkg/core/notes/n001_classic_msgs.py index 3f3bd8e0..265ddbe9 100644 --- a/pkg/core/notes/n001_classic_msgs.py +++ b/pkg/core/notes/n001_classic_msgs.py @@ -7,7 +7,7 @@ from .. import note @note.note_class('ClassicNotes', 1) class ClassicNotes(note.LaunchNote): - """经典启动信息""" + """Classic launch information""" async def need_show(self) -> bool: return True diff --git a/pkg/core/notes/n002_selection_mode_on_windows.py b/pkg/core/notes/n002_selection_mode_on_windows.py index 23bff24a..16028de1 100644 --- a/pkg/core/notes/n002_selection_mode_on_windows.py +++ b/pkg/core/notes/n002_selection_mode_on_windows.py @@ -9,7 +9,7 @@ from .. import note @note.note_class('SelectionModeOnWindows', 2) class SelectionModeOnWindows(note.LaunchNote): - """Windows 上的选择模式提示信息""" + """Selection mode prompt information on Windows""" async def need_show(self) -> bool: return os.name == 'nt' @@ -19,3 +19,8 @@ class SelectionModeOnWindows(note.LaunchNote): """您正在使用 Windows 系统,若窗口左上角显示处于”选择“模式,程序将被暂停运行,此时请右键窗口中空白区域退出选择模式。""", logging.INFO, ) + + yield ( + """You are using Windows system, if the top left corner of the window displays "Selection" mode, the program will be paused running, please right-click on the blank area in the window to exit the selection mode.""", + logging.INFO, + ) diff --git a/pkg/core/stage.py b/pkg/core/stage.py index 220c474d..1483e23a 100644 --- a/pkg/core/stage.py +++ b/pkg/core/stage.py @@ -7,9 +7,9 @@ from . import app preregistered_stages: dict[str, typing.Type[BootingStage]] = {} -"""预注册的请求处理阶段。在初始化时,所有请求处理阶段类会被注册到此字典中。 +"""Pre-registered request processing stages. All request processing stage classes are registered in this dictionary during initialization. -当前阶段暂不支持扩展 +Currently not supported for extension """ @@ -22,11 +22,11 @@ def stage_class(name: str): class BootingStage(abc.ABC): - """启动阶段""" + """Booting stage""" name: str = None @abc.abstractmethod async def run(self, ap: app.Application): - """启动""" + """Run""" pass diff --git a/pkg/core/stages/build_app.py b/pkg/core/stages/build_app.py index 6ee35610..0f28f0c8 100644 --- a/pkg/core/stages/build_app.py +++ b/pkg/core/stages/build_app.py @@ -9,6 +9,7 @@ from ...command import cmdmgr from ...provider.session import sessionmgr as llm_session_mgr from ...provider.modelmgr import modelmgr as llm_model_mgr from ...provider.tools import toolmgr as llm_tool_mgr +from ...rag.knowledge import kbmgr as rag_mgr from ...platform import botmgr as im_mgr from ...persistence import mgr as persistencemgr from ...api.http.controller import main as http_controller @@ -16,18 +17,20 @@ from ...api.http.service import user as user_service from ...api.http.service import model as model_service from ...api.http.service import pipeline as pipeline_service from ...api.http.service import bot as bot_service +from ...api.http.service import knowledge as knowledge_service from ...discover import engine as discover_engine from ...storage import mgr as storagemgr from ...utils import logcache +from ...vector import mgr as vectordb_mgr from .. import taskmgr @stage.stage_class('BuildAppStage') class BuildAppStage(stage.BootingStage): - """构建应用阶段""" + """Build LangBot application""" async def run(self, ap: app.Application): - """构建app对象的各个组件对象并初始化""" + """Build LangBot application""" ap.task_mgr = taskmgr.AsyncTaskManager(ap) discover = discover_engine.ComponentDiscoveryEngine(ap) @@ -42,7 +45,7 @@ class BuildAppStage(stage.BootingStage): await ver_mgr.initialize() ap.ver_mgr = ver_mgr - # 发送公告 + # Send announcement ann_mgr = announce.AnnouncementManager(ap) ap.ann_mgr = ann_mgr @@ -88,6 +91,15 @@ class BuildAppStage(stage.BootingStage): await pipeline_mgr.initialize() ap.pipeline_mgr = pipeline_mgr + rag_mgr_inst = rag_mgr.RAGManager(ap) + await rag_mgr_inst.initialize() + ap.rag_mgr = rag_mgr_inst + + # 初始化向量数据库管理器 + vectordb_mgr_inst = vectordb_mgr.VectorDBManager(ap) + await vectordb_mgr_inst.initialize() + ap.vector_db_mgr = vectordb_mgr_inst + http_ctrl = http_controller.HTTPController(ap) await http_ctrl.initialize() ap.http_ctrl = http_ctrl @@ -95,8 +107,11 @@ class BuildAppStage(stage.BootingStage): user_service_inst = user_service.UserService(ap) ap.user_service = user_service_inst - model_service_inst = model_service.ModelsService(ap) - ap.model_service = model_service_inst + llm_model_service_inst = model_service.LLMModelsService(ap) + ap.llm_model_service = llm_model_service_inst + + embedding_models_service_inst = model_service.EmbeddingModelsService(ap) + ap.embedding_models_service = embedding_models_service_inst pipeline_service_inst = pipeline_service.PipelineService(ap) ap.pipeline_service = pipeline_service_inst @@ -104,5 +119,8 @@ class BuildAppStage(stage.BootingStage): bot_service_inst = bot_service.BotService(ap) ap.bot_service = bot_service_inst + knowledge_service_inst = knowledge_service.KnowledgeService(ap) + ap.knowledge_service = knowledge_service_inst + ctrl = controller.Controller(ap) ap.ctrl = ctrl diff --git a/pkg/core/stages/genkeys.py b/pkg/core/stages/genkeys.py index c24ebd70..f0412b9d 100644 --- a/pkg/core/stages/genkeys.py +++ b/pkg/core/stages/genkeys.py @@ -7,11 +7,18 @@ from .. import stage, app @stage.stage_class('GenKeysStage') class GenKeysStage(stage.BootingStage): - """生成密钥阶段""" + """Generate keys stage""" async def run(self, ap: app.Application): - """启动""" + """Generate keys""" if not ap.instance_config.data['system']['jwt']['secret']: ap.instance_config.data['system']['jwt']['secret'] = secrets.token_hex(16) await ap.instance_config.dump_config() + + if 'recovery_key' not in ap.instance_config.data['system']: + ap.instance_config.data['system']['recovery_key'] = '' + + if not ap.instance_config.data['system']['recovery_key']: + ap.instance_config.data['system']['recovery_key'] = secrets.token_hex(3).upper() + await ap.instance_config.dump_config() diff --git a/pkg/core/stages/load_config.py b/pkg/core/stages/load_config.py index ef5f611b..0474b33a 100644 --- a/pkg/core/stages/load_config.py +++ b/pkg/core/stages/load_config.py @@ -8,10 +8,10 @@ from ..bootutils import config @stage.stage_class('LoadConfigStage') class LoadConfigStage(stage.BootingStage): - """加载配置文件阶段""" + """Load config file stage""" async def run(self, ap: app.Application): - """启动""" + """Load config file""" # ======= deprecated ======= if os.path.exists('data/config/command.json'): diff --git a/pkg/core/stages/migrate.py b/pkg/core/stages/migrate.py index 02b03256..229e0060 100644 --- a/pkg/core/stages/migrate.py +++ b/pkg/core/stages/migrate.py @@ -11,10 +11,13 @@ importutil.import_modules_in_pkg(migrations) @stage.stage_class('MigrationStage') class MigrationStage(stage.BootingStage): - """迁移阶段""" + """Migration stage + + These migrations are legacy, only performed in version 3.x + """ async def run(self, ap: app.Application): - """启动""" + """Run migration""" if any( [ @@ -29,7 +32,7 @@ class MigrationStage(stage.BootingStage): migrations = migration.preregistered_migrations - # 按照迁移号排序 + # Sort by migration number migrations.sort(key=lambda x: x.number) for migration_cls in migrations: @@ -37,4 +40,4 @@ class MigrationStage(stage.BootingStage): if await migration_instance.need_migrate(): await migration_instance.run() - print(f'已执行迁移 {migration_instance.name}') + print(f'Migration {migration_instance.name} executed') diff --git a/pkg/core/stages/setup_logger.py b/pkg/core/stages/setup_logger.py index 0c630175..1f7c81ac 100644 --- a/pkg/core/stages/setup_logger.py +++ b/pkg/core/stages/setup_logger.py @@ -8,7 +8,7 @@ from ..bootutils import log class PersistenceHandler(logging.Handler, object): """ - 保存日志到数据库 + Save logs to database """ ap: app.Application @@ -19,9 +19,9 @@ class PersistenceHandler(logging.Handler, object): def emit(self, record): """ - emit函数为自定义handler类时必重写的函数,这里可以根据需要对日志消息做一些处理,比如发送日志到服务器 + emit function is a required function for custom handler classes, here you can process the log messages as needed, such as sending logs to the server - 发出记录(Emit a record) + Emit a record """ try: msg = self.format(record) @@ -34,10 +34,10 @@ class PersistenceHandler(logging.Handler, object): @stage.stage_class('SetupLoggerStage') class SetupLoggerStage(stage.BootingStage): - """设置日志器阶段""" + """Setup logger stage""" async def run(self, ap: app.Application): - """启动""" + """Setup logger""" persistence_handler = PersistenceHandler('LoggerHandler', ap) extra_handlers = [] diff --git a/pkg/core/stages/show_notes.py b/pkg/core/stages/show_notes.py index 5fa7ff08..d0f861ba 100644 --- a/pkg/core/stages/show_notes.py +++ b/pkg/core/stages/show_notes.py @@ -12,10 +12,10 @@ importutil.import_modules_in_pkg(notes) @stage.stage_class('ShowNotesStage') class ShowNotesStage(stage.BootingStage): - """显示启动信息阶段""" + """Show notes stage""" async def run(self, ap: app.Application): - # 排序 + # Sort note.preregistered_notes.sort(key=lambda x: x.number) for note_cls in note.preregistered_notes: diff --git a/pkg/core/taskmgr.py b/pkg/core/taskmgr.py index 0f756118..ca6eb029 100644 --- a/pkg/core/taskmgr.py +++ b/pkg/core/taskmgr.py @@ -9,13 +9,13 @@ from . import entities as core_entities class TaskContext: - """任务跟踪上下文""" + """Task tracking context""" current_action: str - """当前正在执行的动作""" + """Current action being executed""" log: str - """记录日志""" + """Log""" def __init__(self): self.current_action = 'default' @@ -58,40 +58,40 @@ placeholder_context: TaskContext | None = None class TaskWrapper: - """任务包装器""" + """Task wrapper""" _id_index: int = 0 - """任务ID索引""" + """Task ID index""" id: int - """任务ID""" + """Task ID""" - task_type: str = 'system' # 任务类型: system 或 user - """任务类型""" + task_type: str = 'system' # Task type: system or user + """Task type""" - kind: str = 'system_task' # 由发起者确定任务种类,通常同质化的任务种类相同 - """任务种类""" + kind: str = 'system_task' # Task type determined by the initiator, usually the same task type + """Task type""" name: str = '' - """任务唯一名称""" + """Task unique name""" label: str = '' - """任务显示名称""" + """Task display name""" task_context: TaskContext - """任务上下文""" + """Task context""" task: asyncio.Task - """任务""" + """Task""" task_stack: list = None - """任务堆栈""" + """Task stack""" ap: app.Application - """应用实例""" + """Application instance""" scopes: list[core_entities.LifecycleControlScope] - """任务所属生命周期控制范围""" + """Task scope""" def __init__( self, @@ -165,13 +165,13 @@ class TaskWrapper: class AsyncTaskManager: - """保存app中的所有异步任务 - 包含系统级的和用户级(插件安装、更新等由用户直接发起的)的""" + """Save all asynchronous tasks in the app + Include system-level and user-level (plugin installation, update, etc. initiated by users directly)""" ap: app.Application tasks: list[TaskWrapper] - """所有任务""" + """All tasks""" def __init__(self, ap: app.Application): self.ap = ap diff --git a/pkg/entity/persistence/bot.py b/pkg/entity/persistence/bot.py index 3c08f4ec..08eda478 100644 --- a/pkg/entity/persistence/bot.py +++ b/pkg/entity/persistence/bot.py @@ -4,7 +4,7 @@ from .base import Base class Bot(Base): - """机器人""" + """Bot""" __tablename__ = 'bots' diff --git a/pkg/entity/persistence/metadata.py b/pkg/entity/persistence/metadata.py index d9e03663..4db732b9 100644 --- a/pkg/entity/persistence/metadata.py +++ b/pkg/entity/persistence/metadata.py @@ -12,7 +12,7 @@ initial_metadata = [ class Metadata(Base): - """数据库元数据""" + """Database metadata""" __tablename__ = 'metadata' diff --git a/pkg/entity/persistence/model.py b/pkg/entity/persistence/model.py index 9eb2ccef..e9a104c4 100644 --- a/pkg/entity/persistence/model.py +++ b/pkg/entity/persistence/model.py @@ -4,7 +4,7 @@ from .base import Base class LLMModel(Base): - """LLM 模型""" + """LLM model""" __tablename__ = 'llm_models' @@ -23,3 +23,24 @@ class LLMModel(Base): server_default=sqlalchemy.func.now(), onupdate=sqlalchemy.func.now(), ) + + +class EmbeddingModel(Base): + """Embedding 模型""" + + __tablename__ = 'embedding_models' + + uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True) + name = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) + description = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) + requester = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) + requester_config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={}) + api_keys = sqlalchemy.Column(sqlalchemy.JSON, nullable=False) + extra_args = sqlalchemy.Column(sqlalchemy.JSON, nullable=False, default={}) + created_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False, server_default=sqlalchemy.func.now()) + updated_at = sqlalchemy.Column( + sqlalchemy.DateTime, + nullable=False, + server_default=sqlalchemy.func.now(), + onupdate=sqlalchemy.func.now(), + ) diff --git a/pkg/entity/persistence/pipeline.py b/pkg/entity/persistence/pipeline.py index 56e2cae9..3a21dbf2 100644 --- a/pkg/entity/persistence/pipeline.py +++ b/pkg/entity/persistence/pipeline.py @@ -4,7 +4,7 @@ from .base import Base class LegacyPipeline(Base): - """旧版流水线""" + """Legacy pipeline""" __tablename__ = 'legacy_pipelines' @@ -20,13 +20,12 @@ class LegacyPipeline(Base): ) for_version = sqlalchemy.Column(sqlalchemy.String(255), nullable=False) is_default = sqlalchemy.Column(sqlalchemy.Boolean, nullable=False, default=False) - stages = sqlalchemy.Column(sqlalchemy.JSON, nullable=False) config = sqlalchemy.Column(sqlalchemy.JSON, nullable=False) class PipelineRunRecord(Base): - """流水线运行记录""" + """Pipeline run record""" __tablename__ = 'pipeline_run_records' @@ -43,3 +42,4 @@ class PipelineRunRecord(Base): started_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False) finished_at = sqlalchemy.Column(sqlalchemy.DateTime, nullable=False) result = sqlalchemy.Column(sqlalchemy.JSON, nullable=False) + knowledge_base_uuid = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) diff --git a/pkg/entity/persistence/plugin.py b/pkg/entity/persistence/plugin.py index 30db6bd6..e777441f 100644 --- a/pkg/entity/persistence/plugin.py +++ b/pkg/entity/persistence/plugin.py @@ -4,7 +4,7 @@ from .base import Base class PluginSetting(Base): - """插件配置""" + """Plugin setting""" __tablename__ = 'plugin_settings' diff --git a/pkg/entity/persistence/rag.py b/pkg/entity/persistence/rag.py new file mode 100644 index 00000000..0ff93d28 --- /dev/null +++ b/pkg/entity/persistence/rag.py @@ -0,0 +1,50 @@ +import sqlalchemy +from .base import Base + +# Base = declarative_base() +# DATABASE_URL = os.getenv('DATABASE_URL', 'sqlite:///./rag_knowledge.db') +# print("Using database URL:", DATABASE_URL) + + +# engine = create_engine(DATABASE_URL, connect_args={'check_same_thread': False}) + +# SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) + +# def create_db_and_tables(): +# """Creates all database tables defined in the Base.""" +# Base.metadata.create_all(bind=engine) +# print('Database tables created or already exist.') + + +class KnowledgeBase(Base): + __tablename__ = 'knowledge_bases' + uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True) + name = sqlalchemy.Column(sqlalchemy.String, index=True) + description = sqlalchemy.Column(sqlalchemy.Text) + created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now()) + embedding_model_uuid = sqlalchemy.Column(sqlalchemy.String, default='') + top_k = sqlalchemy.Column(sqlalchemy.Integer, default=5) + + +class File(Base): + __tablename__ = 'knowledge_base_files' + uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True) + kb_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) + file_name = sqlalchemy.Column(sqlalchemy.String) + extension = sqlalchemy.Column(sqlalchemy.String) + created_at = sqlalchemy.Column(sqlalchemy.DateTime, default=sqlalchemy.func.now()) + status = sqlalchemy.Column(sqlalchemy.String, default='pending') # pending, processing, completed, failed + + +class Chunk(Base): + __tablename__ = 'knowledge_base_chunks' + uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True) + file_id = sqlalchemy.Column(sqlalchemy.String(255), nullable=True) + text = sqlalchemy.Column(sqlalchemy.Text) + + +# class Vector(Base): +# __tablename__ = 'knowledge_base_vectors' +# uuid = sqlalchemy.Column(sqlalchemy.String(255), primary_key=True, unique=True) +# chunk_id = sqlalchemy.Column(sqlalchemy.String, nullable=True) +# embedding = sqlalchemy.Column(sqlalchemy.LargeBinary) diff --git a/pkg/entity/persistence/vector.py b/pkg/entity/persistence/vector.py new file mode 100644 index 00000000..465125f5 --- /dev/null +++ b/pkg/entity/persistence/vector.py @@ -0,0 +1,13 @@ +from sqlalchemy import Column, Integer, ForeignKey, LargeBinary +from sqlalchemy.orm import declarative_base, relationship + +Base = declarative_base() + + +class Vector(Base): + __tablename__ = 'vectors' + id = Column(Integer, primary_key=True, index=True) + chunk_id = Column(Integer, ForeignKey('chunks.id'), unique=True) + embedding = Column(LargeBinary) # Store embeddings as binary + + chunk = relationship('Chunk', back_populates='vector') diff --git a/pkg/entity/rag/__init__.py b/pkg/entity/rag/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/pkg/entity/rag/retriever.py b/pkg/entity/rag/retriever.py new file mode 100644 index 00000000..becaf8db --- /dev/null +++ b/pkg/entity/rag/retriever.py @@ -0,0 +1,13 @@ +from __future__ import annotations + +import pydantic + +from typing import Any + + +class RetrieveResultEntry(pydantic.BaseModel): + id: str + + metadata: dict[str, Any] + + distance: float diff --git a/pkg/persistence/database.py b/pkg/persistence/database.py index 528c6a34..4debb03d 100644 --- a/pkg/persistence/database.py +++ b/pkg/persistence/database.py @@ -11,7 +11,7 @@ preregistered_managers: list[type[BaseDatabaseManager]] = [] def manager_class(name: str) -> None: - """注册一个数据库管理类""" + """Register a database manager class""" def decorator(cls: type[BaseDatabaseManager]) -> type[BaseDatabaseManager]: cls.name = name @@ -22,7 +22,7 @@ def manager_class(name: str) -> None: class BaseDatabaseManager(abc.ABC): - """基础数据库管理类""" + """Base database manager class""" name: str diff --git a/pkg/persistence/databases/sqlite.py b/pkg/persistence/databases/sqlite.py index 7b095e61..c1337459 100644 --- a/pkg/persistence/databases/sqlite.py +++ b/pkg/persistence/databases/sqlite.py @@ -7,7 +7,7 @@ from .. import database @database.manager_class('sqlite') class SQLiteDatabaseManager(database.BaseDatabaseManager): - """SQLite 数据库管理类""" + """SQLite database manager""" async def initialize(self) -> None: sqlite_path = 'data/langbot.db' diff --git a/pkg/persistence/mgr.py b/pkg/persistence/mgr.py index 606aa9fd..3aa21ad2 100644 --- a/pkg/persistence/mgr.py +++ b/pkg/persistence/mgr.py @@ -22,12 +22,12 @@ importutil.import_modules_in_pkg(persistence) class PersistenceManager: - """持久化模块管理器""" + """Persistence module manager""" ap: app.Application db: database.BaseDatabaseManager - """数据库管理器""" + """Database manager""" meta: sqlalchemy.MetaData @@ -79,7 +79,7 @@ class PersistenceManager: 'stages': pipeline_service.default_stage_order, 'is_default': True, 'name': 'ChatPipeline', - 'description': '默认提供的流水线,您配置的机器人、第一个模型将自动绑定到此流水线', + 'description': 'Default pipeline, new bots will be bound to this pipeline | 默认提供的流水线,您配置的机器人将自动绑定到此流水线', 'config': pipeline_config, } diff --git a/pkg/persistence/migration.py b/pkg/persistence/migration.py index c191b686..294e30ca 100644 --- a/pkg/persistence/migration.py +++ b/pkg/persistence/migration.py @@ -10,7 +10,7 @@ preregistered_db_migrations: list[typing.Type[DBMigration]] = [] def migration_class(number: int): - """迁移类装饰器""" + """Migration class decorator""" def wrapper(cls: typing.Type[DBMigration]) -> typing.Type[DBMigration]: cls.number = number @@ -21,20 +21,20 @@ def migration_class(number: int): class DBMigration(abc.ABC): - """数据库迁移""" + """Database migration""" number: int - """迁移号""" + """Migration number""" def __init__(self, ap: app.Application): self.ap = ap @abc.abstractmethod async def upgrade(self): - """升级""" + """Upgrade""" pass @abc.abstractmethod async def downgrade(self): - """降级""" + """Downgrade""" pass diff --git a/pkg/persistence/migrations/dbm001_migrate_v3_config.py b/pkg/persistence/migrations/dbm001_migrate_v3_config.py index a1145527..58f05e04 100644 --- a/pkg/persistence/migrations/dbm001_migrate_v3_config.py +++ b/pkg/persistence/migrations/dbm001_migrate_v3_config.py @@ -15,21 +15,21 @@ from ...entity.persistence import ( @migration.migration_class(1) class DBMigrateV3Config(migration.DBMigration): - """从 v3 的配置迁移到 v4 的数据库""" + """Migrate v3 config to v4 database""" async def upgrade(self): - """升级""" + """Upgrade""" """ - 将 data/config 下的所有配置文件进行迁移。 - 迁移后,之前的配置文件都保存到 data/legacy/config 下。 - 迁移后,data/metadata/ 下的所有配置文件都保存到 data/legacy/metadata 下。 + Migrate all config files under data/config. + After migration, all previous config files are saved under data/legacy/config. + After migration, all config files under data/metadata/ are saved under data/legacy/metadata. """ if self.ap.provider_cfg is None: return - # ======= 迁移模型 ======= - # 只迁移当前选中的模型 + # ======= Migrate model ======= + # Only migrate the currently selected model model_name = self.ap.provider_cfg.data.get('model', 'gpt-4o') model_requester = 'openai-chat-completions' @@ -91,8 +91,8 @@ class DBMigrateV3Config(migration.DBMigration): sqlalchemy.insert(persistence_model.LLMModel).values(**llm_model_data) ) - # ======= 迁移流水线配置 ======= - # 修改到默认流水线 + # ======= Migrate pipeline config ======= + # Modify to default pipeline default_pipeline = [ self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline) for pipeline in ( @@ -184,8 +184,8 @@ class DBMigrateV3Config(migration.DBMigration): .where(persistence_pipeline.LegacyPipeline.uuid == default_pipeline['uuid']) ) - # ======= 迁移机器人 ======= - # 只迁移启用的机器人 + # ======= Migrate bot ======= + # Only migrate enabled bots for adapter in self.ap.platform_cfg.data.get('platform-adapters', []): if not adapter.get('enable'): continue @@ -207,7 +207,7 @@ class DBMigrateV3Config(migration.DBMigration): await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_bot.Bot).values(**bot_data)) - # ======= 迁移系统设置 ======= + # ======= Migrate system settings ======= self.ap.instance_config.data['admins'] = self.ap.system_cfg.data['admin-sessions'] self.ap.instance_config.data['api']['port'] = self.ap.system_cfg.data['http-api']['port'] self.ap.instance_config.data['command'] = { @@ -223,7 +223,7 @@ class DBMigrateV3Config(migration.DBMigration): await self.ap.instance_config.dump_config() # ======= move files ======= - # 迁移 data/config 下的所有配置文件 + # Migrate all config files under data/config all_legacy_dir_name = [ 'config', # 'metadata', @@ -246,4 +246,4 @@ class DBMigrateV3Config(migration.DBMigration): move_legacy_files(dir_name) async def downgrade(self): - """降级""" + """Downgrade""" diff --git a/pkg/persistence/migrations/dbm002_combine_quote_msg_config.py b/pkg/persistence/migrations/dbm002_combine_quote_msg_config.py index cebf403b..349bb0c2 100644 --- a/pkg/persistence/migrations/dbm002_combine_quote_msg_config.py +++ b/pkg/persistence/migrations/dbm002_combine_quote_msg_config.py @@ -7,10 +7,10 @@ from ...entity.persistence import pipeline as persistence_pipeline @migration.migration_class(2) class DBMigrateCombineQuoteMsgConfig(migration.DBMigration): - """引用消息合并配置""" + """Combine quote message config""" async def upgrade(self): - """升级""" + """Upgrade""" # read all pipelines pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline)) @@ -37,5 +37,5 @@ class DBMigrateCombineQuoteMsgConfig(migration.DBMigration): ) async def downgrade(self): - """降级""" + """Downgrade""" pass diff --git a/pkg/persistence/migrations/dbm003_n8n_config.py b/pkg/persistence/migrations/dbm003_n8n_config.py index 8705040b..15484f22 100644 --- a/pkg/persistence/migrations/dbm003_n8n_config.py +++ b/pkg/persistence/migrations/dbm003_n8n_config.py @@ -7,10 +7,10 @@ from ...entity.persistence import pipeline as persistence_pipeline @migration.migration_class(3) class DBMigrateN8nConfig(migration.DBMigration): - """N8n配置""" + """N8n config""" async def upgrade(self): - """升级""" + """Upgrade""" # read all pipelines pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline)) @@ -45,5 +45,5 @@ class DBMigrateN8nConfig(migration.DBMigration): ) async def downgrade(self): - """降级""" + """Downgrade""" pass diff --git a/pkg/persistence/migrations/dbm004_rag_kb_uuid.py b/pkg/persistence/migrations/dbm004_rag_kb_uuid.py new file mode 100644 index 00000000..b45cfa78 --- /dev/null +++ b/pkg/persistence/migrations/dbm004_rag_kb_uuid.py @@ -0,0 +1,38 @@ +from .. import migration + +import sqlalchemy + +from ...entity.persistence import pipeline as persistence_pipeline + + +@migration.migration_class(4) +class DBMigrateRAGKBUUID(migration.DBMigration): + """RAG知识库UUID""" + + async def upgrade(self): + """升级""" + # read all pipelines + pipelines = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_pipeline.LegacyPipeline)) + + for pipeline in pipelines: + serialized_pipeline = self.ap.persistence_mgr.serialize_model(persistence_pipeline.LegacyPipeline, pipeline) + + config = serialized_pipeline['config'] + + if 'knowledge-base' not in config['ai']['local-agent']: + config['ai']['local-agent']['knowledge-base'] = '' + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_pipeline.LegacyPipeline) + .where(persistence_pipeline.LegacyPipeline.uuid == serialized_pipeline['uuid']) + .values( + { + 'config': config, + 'for_version': self.ap.ver_mgr.get_current_version(), + } + ) + ) + + async def downgrade(self): + """降级""" + pass diff --git a/pkg/pipeline/bansess/bansess.py b/pkg/pipeline/bansess/bansess.py index 3b927a55..c88a1aa2 100644 --- a/pkg/pipeline/bansess/bansess.py +++ b/pkg/pipeline/bansess/bansess.py @@ -6,9 +6,9 @@ from ...core import entities as core_entities @stage.stage_class('BanSessionCheckStage') class BanSessionCheckStage(stage.PipelineStage): - """访问控制处理阶段 + """Access control processing stage - 仅检查query中群号或个人号是否在访问控制列表中。 + Only check if the group or personal number in the query is in the access control list. """ async def initialize(self, pipeline_config: dict): @@ -41,5 +41,7 @@ class BanSessionCheckStage(stage.PipelineStage): return entities.StageProcessResult( result_type=entities.ResultType.CONTINUE if ctn else entities.ResultType.INTERRUPT, new_query=query, - console_notice=f'根据访问控制忽略消息: {query.launcher_type.value}_{query.launcher_id}' if not ctn else '', + console_notice=f'Ignore message according to access control: {query.launcher_type.value}_{query.launcher_id}' + if not ctn + else '', ) diff --git a/pkg/pipeline/cntfilter/filter.py b/pkg/pipeline/cntfilter/filter.py index 0a3ceaae..36d8a7f4 100644 --- a/pkg/pipeline/cntfilter/filter.py +++ b/pkg/pipeline/cntfilter/filter.py @@ -13,13 +13,13 @@ preregistered_filters: list[typing.Type[ContentFilter]] = [] def filter_class( name: str, ) -> typing.Callable[[typing.Type[ContentFilter]], typing.Type[ContentFilter]]: - """内容过滤器类装饰器 + """Content filter class decorator Args: - name (str): 过滤器名称 + name (str): Filter name Returns: - typing.Callable[[typing.Type[ContentFilter]], typing.Type[ContentFilter]]: 装饰器 + typing.Callable[[typing.Type[ContentFilter]], typing.Type[ContentFilter]]: Decorator """ def decorator(cls: typing.Type[ContentFilter]) -> typing.Type[ContentFilter]: @@ -35,7 +35,7 @@ def filter_class( class ContentFilter(metaclass=abc.ABCMeta): - """内容过滤器抽象类""" + """Content filter abstract class""" name: str @@ -46,31 +46,31 @@ class ContentFilter(metaclass=abc.ABCMeta): @property def enable_stages(self): - """启用的阶段 + """Enabled stages - 默认为消息请求AI前后的两个阶段。 + Default is the two stages before and after the message request to AI. - entity.EnableStage.PRE: 消息请求AI前,此时需要检查的内容是用户的输入消息。 - entity.EnableStage.POST: 消息请求AI后,此时需要检查的内容是AI的回复消息。 + entity.EnableStage.PRE: Before message request to AI, the content to check is the user's input message. + entity.EnableStage.POST: After message request to AI, the content to check is the AI's reply message. """ return [entities.EnableStage.PRE, entities.EnableStage.POST] async def initialize(self): - """初始化过滤器""" + """Initialize filter""" pass @abc.abstractmethod async def process(self, query: core_entities.Query, message: str = None, image_url=None) -> entities.FilterResult: - """处理消息 + """Process message - 分为前后阶段,具体取决于 enable_stages 的值。 - 对于内容过滤器来说,不需要考虑消息所处的阶段,只需要检查消息内容即可。 + It is divided into two stages, depending on the value of enable_stages. + For content filters, you do not need to consider the stage of the message, you only need to check the message content. Args: - message (str): 需要检查的内容 - image_url (str): 要检查的图片的 URL + message (str): Content to check + image_url (str): URL of the image to check Returns: - entities.FilterResult: 过滤结果,具体内容请查看 entities.FilterResult 类的文档 + entities.FilterResult: Filter result, please refer to the documentation of entities.FilterResult class """ raise NotImplementedError diff --git a/pkg/pipeline/cntfilter/filters/banwords.py b/pkg/pipeline/cntfilter/filters/banwords.py index 916a1bc1..b03e79a9 100644 --- a/pkg/pipeline/cntfilter/filters/banwords.py +++ b/pkg/pipeline/cntfilter/filters/banwords.py @@ -8,7 +8,7 @@ from ....core import entities as core_entities @filter_model.filter_class('ban-word-filter') class BanWordFilter(filter_model.ContentFilter): - """根据内容过滤""" + """Filter content""" async def initialize(self): pass diff --git a/pkg/pipeline/cntfilter/filters/cntignore.py b/pkg/pipeline/cntfilter/filters/cntignore.py index 5e410e31..b80d90eb 100644 --- a/pkg/pipeline/cntfilter/filters/cntignore.py +++ b/pkg/pipeline/cntfilter/filters/cntignore.py @@ -8,7 +8,7 @@ from ....core import entities as core_entities @filter_model.filter_class('content-ignore') class ContentIgnore(filter_model.ContentFilter): - """根据内容忽略消息""" + """Ignore message according to content""" @property def enable_stages(self): @@ -24,7 +24,7 @@ class ContentIgnore(filter_model.ContentFilter): level=entities.ResultLevel.BLOCK, replacement='', user_notice='', - console_notice='根据 ignore_rules 中的 prefix 规则,忽略消息', + console_notice='Ignore message according to prefix rule in ignore_rules', ) if 'regexp' in query.pipeline_config['trigger']['ignore-rules']: @@ -34,7 +34,7 @@ class ContentIgnore(filter_model.ContentFilter): level=entities.ResultLevel.BLOCK, replacement='', user_notice='', - console_notice='根据 ignore_rules 中的 regexp 规则,忽略消息', + console_notice='Ignore message according to regexp rule in ignore_rules', ) return entities.FilterResult( diff --git a/pkg/pipeline/longtext/longtext.py b/pkg/pipeline/longtext/longtext.py index 5be20650..03457212 100644 --- a/pkg/pipeline/longtext/longtext.py +++ b/pkg/pipeline/longtext/longtext.py @@ -16,9 +16,9 @@ importutil.import_modules_in_pkg(strategies) @stage.stage_class('LongTextProcessStage') class LongTextProcessStage(stage.PipelineStage): - """长消息处理阶段 + """Long message processing stage - 改写: + Rewrite: - resp_message_chain """ @@ -36,22 +36,22 @@ class LongTextProcessStage(stage.PipelineStage): use_font = 'C:/Windows/Fonts/msyh.ttc' if not os.path.exists(use_font): self.ap.logger.warn( - '未找到字体文件,且无法使用Windows自带字体,更换为转发消息组件以发送长消息,您可以在配置文件中调整相关设置。' + 'Font file not found, and Windows system font cannot be used, switch to forward message component to send long messages, you can adjust the related settings in the configuration file.' ) config['blob_message_strategy'] = 'forward' else: - self.ap.logger.info('使用Windows自带字体:' + use_font) + self.ap.logger.info('Using Windows system font: ' + use_font) config['font-path'] = use_font else: self.ap.logger.warn( - '未找到字体文件,且无法使用系统自带字体,更换为转发消息组件以发送长消息,您可以在配置文件中调整相关设置。' + 'Font file not found, and system font cannot be used, switch to forward message component to send long messages, you can adjust the related settings in the configuration file.' ) pipeline_config['output']['long-text-processing']['strategy'] = 'forward' except Exception: traceback.print_exc() self.ap.logger.error( - '加载字体文件失败({}),更换为转发消息组件以发送长消息,您可以在配置文件中调整相关设置。'.format( + 'Failed to load font file ({}), switch to forward message component to send long messages, you can adjust the related settings in the configuration file.'.format( use_font ) ) @@ -63,12 +63,12 @@ class LongTextProcessStage(stage.PipelineStage): self.strategy_impl = strategy_cls(self.ap) break else: - raise ValueError(f'未找到名为 {config["strategy"]} 的长消息处理策略') + raise ValueError(f'Long message processing strategy not found: {config["strategy"]}') await self.strategy_impl.initialize() async def process(self, query: core_entities.Query, stage_inst_name: str) -> entities.StageProcessResult: - # 检查是否包含非 Plain 组件 + # Check if it contains non-Plain components contains_non_plain = False for msg in query.resp_message_chain[-1]: @@ -77,7 +77,7 @@ class LongTextProcessStage(stage.PipelineStage): break if contains_non_plain: - self.ap.logger.debug('消息中包含非 Plain 组件,跳过长消息处理。') + self.ap.logger.debug('Message contains non-Plain components, skip long message processing.') elif ( len(str(query.resp_message_chain[-1])) > query.pipeline_config['output']['long-text-processing']['threshold'] diff --git a/pkg/pipeline/longtext/strategies/forward.py b/pkg/pipeline/longtext/strategies/forward.py index 6228d580..cb772339 100644 --- a/pkg/pipeline/longtext/strategies/forward.py +++ b/pkg/pipeline/longtext/strategies/forward.py @@ -15,17 +15,17 @@ Forward = platform_message.Forward class ForwardComponentStrategy(strategy_model.LongTextStrategy): async def process(self, message: str, query: core_entities.Query) -> list[platform_message.MessageComponent]: display = ForwardMessageDiaplay( - title='群聊的聊天记录', - brief='[聊天记录]', - source='聊天记录', - preview=['QQ用户: ' + message], - summary='查看1条转发消息', + title='Group chat history', + brief='[Chat history]', + source='Chat history', + preview=['User: ' + message], + summary='View 1 forwarded message', ) node_list = [ platform_message.ForwardMessageNode( sender_id=query.adapter.bot_account_id, - sender_name='QQ用户', + sender_name='User', message_chain=platform_message.MessageChain([message]), ) ] diff --git a/pkg/pipeline/longtext/strategy.py b/pkg/pipeline/longtext/strategy.py index 0ddec0c6..5b521067 100644 --- a/pkg/pipeline/longtext/strategy.py +++ b/pkg/pipeline/longtext/strategy.py @@ -14,13 +14,13 @@ preregistered_strategies: list[typing.Type[LongTextStrategy]] = [] def strategy_class( name: str, ) -> typing.Callable[[typing.Type[LongTextStrategy]], typing.Type[LongTextStrategy]]: - """长文本处理策略类装饰器 + """Long text processing strategy class decorator Args: - name (str): 策略名称 + name (str): Strategy name Returns: - typing.Callable[[typing.Type[LongTextStrategy]], typing.Type[LongTextStrategy]]: 装饰器 + typing.Callable[[typing.Type[LongTextStrategy]], typing.Type[LongTextStrategy]]: Decorator """ def decorator(cls: typing.Type[LongTextStrategy]) -> typing.Type[LongTextStrategy]: @@ -36,7 +36,7 @@ def strategy_class( class LongTextStrategy(metaclass=abc.ABCMeta): - """长文本处理策略抽象类""" + """Long text processing strategy abstract class""" name: str @@ -50,15 +50,15 @@ class LongTextStrategy(metaclass=abc.ABCMeta): @abc.abstractmethod async def process(self, message: str, query: core_entities.Query) -> list[platform_message.MessageComponent]: - """处理长文本 + """Process long text - 在 platform.json 中配置 long-text-process 字段,只要 文本长度超过了 threshold 就会调用此方法 + If the text length exceeds the threshold, this method will be called. Args: - message (str): 消息 - query (core_entities.Query): 此次请求的上下文对象 + message (str): Message + query (core_entities.Query): Query object Returns: - list[platform_message.MessageComponent]: 转换后的 平台 消息组件列表 + list[platform_message.MessageComponent]: Converted platform message components """ return [] diff --git a/pkg/pipeline/msgtrun/msgtrun.py b/pkg/pipeline/msgtrun/msgtrun.py index c64f67fc..1c5ee17d 100644 --- a/pkg/pipeline/msgtrun/msgtrun.py +++ b/pkg/pipeline/msgtrun/msgtrun.py @@ -12,9 +12,9 @@ importutil.import_modules_in_pkg(truncators) @stage.stage_class('ConversationMessageTruncator') class ConversationMessageTruncator(stage.PipelineStage): - """会话消息截断器 + """Conversation message truncator - 用于截断会话消息链,以适应平台消息长度限制。 + Used to truncate the conversation message chain to adapt to the LLM message length limit. """ trun: truncator.Truncator @@ -27,10 +27,10 @@ class ConversationMessageTruncator(stage.PipelineStage): self.trun = trun(self.ap) break else: - raise ValueError(f'未知的截断器: {use_method}') + raise ValueError(f'Unknown truncator: {use_method}') async def process(self, query: core_entities.Query, stage_inst_name: str) -> entities.StageProcessResult: - """处理""" + """Process""" query = await self.trun.truncate(query) return entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query) diff --git a/pkg/pipeline/msgtrun/truncators/round.py b/pkg/pipeline/msgtrun/truncators/round.py index fa72a0e1..2acb1d8c 100644 --- a/pkg/pipeline/msgtrun/truncators/round.py +++ b/pkg/pipeline/msgtrun/truncators/round.py @@ -6,17 +6,17 @@ from ....core import entities as core_entities @truncator.truncator_class('round') class RoundTruncator(truncator.Truncator): - """前文回合数阶段器""" + """Truncate the conversation message chain to adapt to the LLM message length limit.""" async def truncate(self, query: core_entities.Query) -> core_entities.Query: - """截断""" + """Truncate""" max_round = query.pipeline_config['ai']['local-agent']['max-round'] temp_messages = [] current_round = 0 - # 从后往前遍历 + # Traverse from back to front for msg in query.messages[::-1]: if current_round < max_round: temp_messages.append(msg) diff --git a/pkg/pipeline/pipelinemgr.py b/pkg/pipeline/pipelinemgr.py index b61e34ad..77df09dc 100644 --- a/pkg/pipeline/pipelinemgr.py +++ b/pkg/pipeline/pipelinemgr.py @@ -144,23 +144,27 @@ class RuntimePipeline: result = await result if isinstance(result, pipeline_entities.StageProcessResult): # 直接返回结果 - self.ap.logger.debug(f'Stage {stage_container.inst_name} processed query {query} res {result}') + self.ap.logger.debug( + f'Stage {stage_container.inst_name} processed query {query.query_id} res {result.result_type}' + ) await self._check_output(query, result) if result.result_type == pipeline_entities.ResultType.INTERRUPT: - self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query}') + self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query.query_id}') break elif result.result_type == pipeline_entities.ResultType.CONTINUE: query = result.new_query elif isinstance(result, typing.AsyncGenerator): # 生成器 - self.ap.logger.debug(f'Stage {stage_container.inst_name} processed query {query} gen') + self.ap.logger.debug(f'Stage {stage_container.inst_name} processed query {query.query_id} gen') async for sub_result in result: - self.ap.logger.debug(f'Stage {stage_container.inst_name} processed query {query} res {sub_result}') + self.ap.logger.debug( + f'Stage {stage_container.inst_name} processed query {query.query_id} res {sub_result.result_type}' + ) await self._check_output(query, sub_result) if sub_result.result_type == pipeline_entities.ResultType.INTERRUPT: - self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query}') + self.ap.logger.debug(f'Stage {stage_container.inst_name} interrupted query {query.query_id}') break elif sub_result.result_type == pipeline_entities.ResultType.CONTINUE: query = sub_result.new_query @@ -192,7 +196,7 @@ class RuntimePipeline: if event_ctx.is_prevented_default(): return - self.ap.logger.debug(f'Processing query {query}') + self.ap.logger.debug(f'Processing query {query.query_id}') await self._execute_from_stage(0, query) except Exception as e: @@ -200,7 +204,7 @@ class RuntimePipeline: self.ap.logger.error(f'处理请求时出错 query_id={query.query_id} stage={inst_name} : {e}') self.ap.logger.error(f'Traceback: {traceback.format_exc()}') finally: - self.ap.logger.debug(f'Query {query} processed') + self.ap.logger.debug(f'Query {query.query_id} processed') class PipelineManager: diff --git a/pkg/pipeline/preproc/preproc.py b/pkg/pipeline/preproc/preproc.py index 19478200..1aada6b3 100644 --- a/pkg/pipeline/preproc/preproc.py +++ b/pkg/pipeline/preproc/preproc.py @@ -11,11 +11,11 @@ from ...platform.types import message as platform_message @stage.stage_class('PreProcessor') class PreProcessor(stage.PipelineStage): - """请求预处理阶段 + """Request pre-processing stage - 签出会话、prompt、上文、模型、内容函数。 + Check out session, prompt, context, model, and content functions. - 改写: + Rewrite: - session - prompt - messages @@ -29,12 +29,12 @@ class PreProcessor(stage.PipelineStage): query: core_entities.Query, stage_inst_name: str, ) -> entities.StageProcessResult: - """处理""" + """Process""" selected_runner = query.pipeline_config['ai']['runner']['runner'] session = await self.ap.sess_mgr.get_session(query) - # 非 local-agent 时,llm_model 为 None + # When not local-agent, llm_model is None llm_model = ( await self.ap.model_mgr.get_model_by_uuid(query.pipeline_config['ai']['local-agent']['model']) if selected_runner == 'local-agent' @@ -51,7 +51,7 @@ class PreProcessor(stage.PipelineStage): conversation.use_llm_model = llm_model - # 设置query + # Set query query.session = session query.prompt = conversation.prompt.copy() query.messages = conversation.messages.copy() @@ -80,14 +80,15 @@ class PreProcessor(stage.PipelineStage): if me.type == 'image_url': msg.content.remove(me) - content_list = [] + content_list: list[llm_entities.ContentElement] = [] plain_text = '' qoute_msg = query.pipeline_config['trigger'].get('misc', '').get('combine-quote-message') + # tidy the content_list + # combine all text content into one, and put it in the first position for me in query.message_chain: if isinstance(me, platform_message.Plain): - content_list.append(llm_entities.ContentElement.from_text(me.text)) plain_text += me.text elif isinstance(me, platform_message.Image): if selected_runner != 'local-agent' or query.use_llm_model.model_entity.abilities.__contains__( @@ -106,10 +107,12 @@ class PreProcessor(stage.PipelineStage): if msg.base64 is not None: content_list.append(llm_entities.ContentElement.from_image_base64(msg.base64)) + content_list.insert(0, llm_entities.ContentElement.from_text(plain_text)) + query.variables['user_message_text'] = plain_text query.user_message = llm_entities.Message(role='user', content=content_list) - # =========== 触发事件 PromptPreProcessing + # =========== Trigger event PromptPreProcessing event_ctx = await self.ap.plugin_mgr.emit_event( event=events.PromptPreProcessing( diff --git a/pkg/pipeline/process/handler.py b/pkg/pipeline/process/handler.py index 8a32bcfb..837b72e2 100644 --- a/pkg/pipeline/process/handler.py +++ b/pkg/pipeline/process/handler.py @@ -25,7 +25,7 @@ class MessageHandler(metaclass=abc.ABCMeta): def cut_str(self, s: str) -> str: """ - 取字符串第一行,最多20个字符,若有多行,或超过20个字符,则加省略号 + Take the first line of the string, up to 20 characters, if there are multiple lines, or more than 20 characters, add an ellipsis """ s0 = s.split('\n')[0] if len(s0) > 20 or '\n' in s: diff --git a/pkg/pipeline/process/handlers/chat.py b/pkg/pipeline/process/handlers/chat.py index a1928703..6c428473 100644 --- a/pkg/pipeline/process/handlers/chat.py +++ b/pkg/pipeline/process/handlers/chat.py @@ -87,8 +87,6 @@ class ChatMessageHandler(handler.MessageHandler): text_length += len(result.content) yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query) - # else: - # yield entities.StageProcessResult(result_type=entities.ResultType.INTERRUPT, new_query=query) else: async for result in runner.run(query): @@ -119,4 +117,4 @@ class ChatMessageHandler(handler.MessageHandler): ) finally: # TODO statistics - pass + pass \ No newline at end of file diff --git a/pkg/pipeline/process/handlers/command.py b/pkg/pipeline/process/handlers/command.py index cc0e9314..7348d6b8 100644 --- a/pkg/pipeline/process/handlers/command.py +++ b/pkg/pipeline/process/handlers/command.py @@ -15,7 +15,7 @@ class CommandHandler(handler.MessageHandler): self, query: core_entities.Query, ) -> typing.AsyncGenerator[entities.StageProcessResult, None]: - """处理""" + """Process""" command_text = str(query.message_chain).strip()[1:] @@ -70,7 +70,7 @@ class CommandHandler(handler.MessageHandler): ) ) - self.ap.logger.info(f'命令({query.query_id})报错: {self.cut_str(str(ret.error))}') + self.ap.logger.info(f'Command({query.query_id}) error: {self.cut_str(str(ret.error))}') yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query) elif ret.text is not None or ret.image_url is not None: @@ -89,7 +89,7 @@ class CommandHandler(handler.MessageHandler): ) ) - self.ap.logger.info(f'命令返回: {self.cut_str(str(content[0]))}') + self.ap.logger.info(f'Command returned: {self.cut_str(str(content[0]))}') yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query) else: diff --git a/pkg/pipeline/process/process.py b/pkg/pipeline/process/process.py index 64903552..db66135c 100644 --- a/pkg/pipeline/process/process.py +++ b/pkg/pipeline/process/process.py @@ -33,11 +33,11 @@ class Processor(stage.PipelineStage): query: core_entities.Query, stage_inst_name: str, ) -> entities.StageProcessResult: - """处理""" + """Process""" message_text = str(query.message_chain).strip() self.ap.logger.info( - f'处理 {query.launcher_type.value}_{query.launcher_id} 的请求({query.query_id}): {message_text}' + f'Processing request from {query.launcher_type.value}_{query.launcher_id} ({query.query_id}): {message_text}' ) async def generator(): diff --git a/pkg/platform/logger.py b/pkg/platform/logger.py index 340baa07..a2ea2e25 100644 --- a/pkg/platform/logger.py +++ b/pkg/platform/logger.py @@ -119,7 +119,7 @@ class EventLogger: async def _truncate_logs(self): if len(self.logs) > MAX_LOG_COUNT: for i in range(DELETE_COUNT_PER_TIME): - for image_key in self.logs[i].images: + for image_key in self.logs[i].images: # type: ignore await self.ap.storage_mgr.storage_provider.delete(image_key) self.logs = self.logs[DELETE_COUNT_PER_TIME:] diff --git a/pkg/platform/sources/aiocqhttp.py b/pkg/platform/sources/aiocqhttp.py index c75d2c77..357eb48a 100644 --- a/pkg/platform/sources/aiocqhttp.py +++ b/pkg/platform/sources/aiocqhttp.py @@ -76,8 +76,7 @@ class AiocqhttpMessageConverter(adapter.MessageConverter): return msg_list, msg_id, msg_time @staticmethod - async def target2yiri(message: str, message_id: int = -1, bot=None): - print(message) + async def target2yiri(message: str, message_id: int = -1, bot: aiocqhttp.CQHttp = None): message = aiocqhttp.Message(message) def get_face_name(face_id): @@ -271,15 +270,16 @@ class AiocqhttpMessageConverter(adapter.MessageConverter): ) yiri_msg_list.append(reply_msg) - elif msg.type == 'file': - # file_name = msg.data['file'] - file_id = msg.data['file_id'] - file_data = await bot.get_file(file_id=file_id) - file_name = file_data.get('file_name') - # file_path = file_data.get('file') - file_url = file_data.get('file_url') - file_size = file_data.get('file_size') - yiri_msg_list.append(platform_message.File(id=file_id, name=file_name, url=file_url, size=file_size)) + # 这里下载所有文件会导致下载文件过多,暂时不下载 + # elif msg.type == 'file': + # # file_name = msg.data['file'] + # file_id = msg.data['file_id'] + # file_data = await bot.get_file(file_id=file_id) + # file_name = file_data.get('file_name') + # file_path = file_data.get('file') + # file_url = file_data.get('file_url') + # file_size = file_data.get('file_size') + # yiri_msg_list.append(platform_message.File(id=file_id, name=file_name,url=file_url,size=file_size)) elif msg.type == 'face': face_id = msg.data['id'] face_name = msg.data['raw']['faceText'] diff --git a/pkg/platform/sources/dingtalk.py b/pkg/platform/sources/dingtalk.py index 8bd6e187..eacc2a23 100644 --- a/pkg/platform/sources/dingtalk.py +++ b/pkg/platform/sources/dingtalk.py @@ -1,3 +1,4 @@ +from re import S import traceback import typing from libs.dingtalk_api.dingtalkevent import DingTalkEvent @@ -99,13 +100,15 @@ class DingTalkAdapter(adapter.MessagePlatformAdapter): message_converter: DingTalkMessageConverter = DingTalkMessageConverter() event_converter: DingTalkEventConverter = DingTalkEventConverter() config: dict - card_instance_id_dict: dict + card_instance_id_dict: dict # 回复卡片消息字典,key为消息id,value为回复卡片实例id,用于在流式消息时判断是否发送到指定卡片 + seq: int # 消息顺序,直接以seq作为标识 def __init__(self, config: dict, ap: app.Application, logger: EventLogger): self.config = config self.ap = ap self.logger = logger self.card_instance_id_dict = {} + self.seq = 1 required_keys = [ 'client_id', 'client_secret', @@ -155,14 +158,16 @@ class DingTalkAdapter(adapter.MessagePlatformAdapter): # incoming_message = event.incoming_message # msg_id = incoming_message.message_id + self.seq += 1 + if (self.seq - 1) % 8 == 0 or is_final: + content, at = await DingTalkMessageConverter.yiri2target(message) - content, at = await DingTalkMessageConverter.yiri2target(message) - - card_instance, card_instance_id = self.card_instance_id_dict[message_id] - # print(card_instance_id) - await self.bot.send_card_message(card_instance, card_instance_id, content, is_final) - if is_final: - self.card_instance_id_dict.pop(message_id) + card_instance, card_instance_id = self.card_instance_id_dict[message_id] + # print(card_instance_id) + await self.bot.send_card_message(card_instance, card_instance_id, content, is_final) + if is_final: + self.seq = 1 # 消息回复结束之后重置seq + self.card_instance_id_dict.pop(message_id) # 消息回复结束之后删除卡片实例id async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain): content = await DingTalkMessageConverter.yiri2target(message) diff --git a/pkg/platform/sources/discord.py b/pkg/platform/sources/discord.py index 6cc09a72..da32c7ac 100644 --- a/pkg/platform/sources/discord.py +++ b/pkg/platform/sources/discord.py @@ -8,15 +8,591 @@ import base64 import uuid import os import datetime +import asyncio +from enum import Enum import aiohttp from .. import adapter from ...core import app +from ..logger import EventLogger from ..types import message as platform_message from ..types import events as platform_events from ..types import entities as platform_entities -from ..logger import EventLogger + +# 语音功能相关异常定义 +class VoiceConnectionError(Exception): + """语音连接基础异常""" + def __init__(self, message: str, error_code: str = None, guild_id: int = None): + super().__init__(message) + self.error_code = error_code + self.guild_id = guild_id + self.timestamp = datetime.datetime.now() + + +class VoicePermissionError(VoiceConnectionError): + """语音权限异常""" + def __init__(self, message: str, missing_permissions: list = None, user_id: int = None, channel_id: int = None): + super().__init__(message, "PERMISSION_ERROR") + self.missing_permissions = missing_permissions or [] + self.user_id = user_id + self.channel_id = channel_id + + +class VoiceNetworkError(VoiceConnectionError): + """语音网络异常""" + def __init__(self, message: str, retry_count: int = 0): + super().__init__(message, "NETWORK_ERROR") + self.retry_count = retry_count + self.last_attempt = datetime.datetime.now() + + +class VoiceConnectionStatus(Enum): + """语音连接状态枚举""" + IDLE = "idle" + CONNECTING = "connecting" + CONNECTED = "connected" + PLAYING = "playing" + RECONNECTING = "reconnecting" + FAILED = "failed" + + +class VoiceConnectionInfo: + """ + 语音连接信息类 + + 用于存储和管理单个语音连接的详细信息,包括连接状态、时间戳、 + 频道信息等。提供连接信息的标准化数据结构。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + """ + + def __init__(self, guild_id: int, channel_id: int, channel_name: str = None): + """ + 初始化语音连接信息 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + channel_id (int): 语音频道ID + channel_name (str, optional): 语音频道名称 + """ + self.guild_id = guild_id + self.channel_id = channel_id + self.channel_name = channel_name or f"Channel-{channel_id}" + self.connected = False + self.connection_time: datetime.datetime = None + self.last_activity = datetime.datetime.now() + self.status = VoiceConnectionStatus.IDLE + self.user_count = 0 + self.latency = 0.0 + self.connection_health = "unknown" + self.voice_client = None + + def update_status(self, status: VoiceConnectionStatus): + """ + 更新连接状态 + + @author: @ydzat + + Args: + status (VoiceConnectionStatus): 新的连接状态 + """ + self.status = status + self.last_activity = datetime.datetime.now() + + if status == VoiceConnectionStatus.CONNECTED: + self.connected = True + if self.connection_time is None: + self.connection_time = datetime.datetime.now() + elif status in [VoiceConnectionStatus.IDLE, VoiceConnectionStatus.FAILED]: + self.connected = False + self.connection_time = None + self.voice_client = None + + def to_dict(self) -> dict: + """ + 转换为字典格式 + + @author: @ydzat + + Returns: + dict: 连接信息的字典表示 + """ + return { + "guild_id": self.guild_id, + "channel_id": self.channel_id, + "channel_name": self.channel_name, + "connected": self.connected, + "connection_time": self.connection_time.isoformat() if self.connection_time else None, + "last_activity": self.last_activity.isoformat(), + "status": self.status.value, + "user_count": self.user_count, + "latency": self.latency, + "connection_health": self.connection_health + } + + +class VoiceConnectionManager: + """ + 语音连接管理器 + + 负责管理多个服务器的语音连接,提供连接建立、断开、状态查询等功能。 + 采用单例模式确保全局只有一个连接管理器实例。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + """ + + def __init__(self, bot: discord.Client, logger: EventLogger): + """ + 初始化语音连接管理器 + + @author: @ydzat + + Args: + bot (discord.Client): Discord 客户端实例 + logger (EventLogger): 事件日志记录器 + """ + self.bot = bot + self.logger = logger + self.connections: typing.Dict[int, VoiceConnectionInfo] = {} + self._connection_lock = asyncio.Lock() + self._cleanup_task = None + self._monitoring_enabled = True + + async def join_voice_channel(self, guild_id: int, channel_id: int, + user_id: int = None) -> discord.VoiceClient: + """ + 加入语音频道 + + 验证用户权限和频道状态后,建立到指定语音频道的连接。 + 支持连接复用和自动重连机制。 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + channel_id (int): 语音频道ID + user_id (int, optional): 请求用户ID,用于权限验证 + + Returns: + discord.VoiceClient: 语音客户端实例 + + Raises: + VoicePermissionError: 权限不足时抛出 + VoiceNetworkError: 网络连接失败时抛出 + VoiceConnectionError: 其他连接错误时抛出 + """ + async with self._connection_lock: + try: + # 获取服务器和频道对象 + guild = self.bot.get_guild(guild_id) + if not guild: + raise VoiceConnectionError( + f"无法找到服务器 {guild_id}", + "GUILD_NOT_FOUND", + guild_id + ) + + channel = guild.get_channel(channel_id) + if not channel or not isinstance(channel, discord.VoiceChannel): + raise VoiceConnectionError( + f"无法找到语音频道 {channel_id}", + "CHANNEL_NOT_FOUND", + guild_id + ) + + # 验证用户是否在语音频道中(如果提供了用户ID) + if user_id: + await self._validate_user_in_channel(guild, channel, user_id) + + # 验证机器人权限 + await self._validate_bot_permissions(channel) + + # 检查是否已有连接 + if guild_id in self.connections: + existing_conn = self.connections[guild_id] + if existing_conn.connected and existing_conn.voice_client: + if existing_conn.channel_id == channel_id: + # 已连接到相同频道,返回现有连接 + await self.logger.info(f"复用现有语音连接: {guild.name} -> {channel.name}") + return existing_conn.voice_client + else: + # 连接到不同频道,先断开旧连接 + await self._disconnect_internal(guild_id) + + # 建立新连接 + voice_client = await channel.connect() + + # 更新连接信息 + conn_info = VoiceConnectionInfo(guild_id, channel_id, channel.name) + conn_info.voice_client = voice_client + conn_info.update_status(VoiceConnectionStatus.CONNECTED) + conn_info.user_count = len(channel.members) + self.connections[guild_id] = conn_info + + await self.logger.info(f"成功连接到语音频道: {guild.name} -> {channel.name}") + return voice_client + + except discord.ClientException as e: + raise VoiceNetworkError(f"Discord 客户端错误: {str(e)}") + except discord.opus.OpusNotLoaded as e: + raise VoiceConnectionError(f"Opus 编码器未加载: {str(e)}", "OPUS_NOT_LOADED", guild_id) + except Exception as e: + await self.logger.error(f"连接语音频道时发生未知错误: {str(e)}") + raise VoiceConnectionError(f"连接失败: {str(e)}", "UNKNOWN_ERROR", guild_id) + + async def leave_voice_channel(self, guild_id: int) -> bool: + """ + 离开语音频道 + + 断开指定服务器的语音连接,清理相关资源和状态信息。 + 确保音频播放停止后再断开连接。 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + + Returns: + bool: 断开是否成功 + """ + async with self._connection_lock: + return await self._disconnect_internal(guild_id) + + async def _disconnect_internal(self, guild_id: int) -> bool: + """ + 内部断开连接方法 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + + Returns: + bool: 断开是否成功 + """ + if guild_id not in self.connections: + return True + + conn_info = self.connections[guild_id] + + try: + if conn_info.voice_client and conn_info.voice_client.is_connected(): + # 停止当前播放 + if conn_info.voice_client.is_playing(): + conn_info.voice_client.stop() + + # 等待播放完全停止 + await asyncio.sleep(0.1) + + # 断开连接 + await conn_info.voice_client.disconnect() + + conn_info.update_status(VoiceConnectionStatus.IDLE) + del self.connections[guild_id] + + await self.logger.info(f"已断开语音连接: Guild {guild_id}") + return True + + except Exception as e: + await self.logger.error(f"断开语音连接时发生错误: {str(e)}") + # 即使出错也要清理连接记录 + conn_info.update_status(VoiceConnectionStatus.FAILED) + if guild_id in self.connections: + del self.connections[guild_id] + return False + + async def get_voice_client(self, guild_id: int) -> typing.Optional[discord.VoiceClient]: + """ + 获取语音客户端 + + 返回指定服务器的语音客户端实例,如果未连接则返回 None。 + 会验证连接的有效性,自动清理无效连接。 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + + Returns: + Optional[discord.VoiceClient]: 语音客户端实例或 None + """ + if guild_id not in self.connections: + return None + + conn_info = self.connections[guild_id] + + # 验证连接是否仍然有效 + if conn_info.voice_client and not conn_info.voice_client.is_connected(): + # 连接已失效,清理状态 + await self._disconnect_internal(guild_id) + return None + + return conn_info.voice_client if conn_info.connected else None + + async def is_connected_to_voice(self, guild_id: int) -> bool: + """ + 检查是否连接到语音频道 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + + Returns: + bool: 是否已连接 + """ + if guild_id not in self.connections: + return False + + conn_info = self.connections[guild_id] + + # 检查实际连接状态 + if conn_info.voice_client and not conn_info.voice_client.is_connected(): + # 连接已失效,清理状态 + await self._disconnect_internal(guild_id) + return False + + return conn_info.connected + + async def get_connection_status(self, guild_id: int) -> typing.Optional[dict]: + """ + 获取连接状态信息 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + + Returns: + Optional[dict]: 连接状态信息字典或 None + """ + if guild_id not in self.connections: + return None + + conn_info = self.connections[guild_id] + + # 更新实时信息 + if conn_info.voice_client and conn_info.voice_client.is_connected(): + conn_info.latency = conn_info.voice_client.latency * 1000 # 转换为毫秒 + conn_info.connection_health = "good" if conn_info.latency < 100 else "poor" + + # 更新频道用户数 + guild = self.bot.get_guild(guild_id) + if guild: + channel = guild.get_channel(conn_info.channel_id) + if channel and isinstance(channel, discord.VoiceChannel): + conn_info.user_count = len(channel.members) + + return conn_info.to_dict() + + async def list_active_connections(self) -> typing.List[dict]: + """ + 列出所有活跃连接 + + @author: @ydzat + + Returns: + List[dict]: 活跃连接列表 + """ + active_connections = [] + + for guild_id, conn_info in self.connections.items(): + if conn_info.connected: + status = await self.get_connection_status(guild_id) + if status: + active_connections.append(status) + + return active_connections + + async def get_voice_channel_info(self, guild_id: int, channel_id: int) -> typing.Optional[dict]: + """ + 获取语音频道信息 + + @author: @ydzat + + Args: + guild_id (int): 服务器ID + channel_id (int): 频道ID + + Returns: + Optional[dict]: 频道信息字典或 None + """ + guild = self.bot.get_guild(guild_id) + if not guild: + return None + + channel = guild.get_channel(channel_id) + if not channel or not isinstance(channel, discord.VoiceChannel): + return None + + # 获取用户信息 + users = [] + for member in channel.members: + users.append({ + "id": member.id, + "name": member.display_name, + "status": str(member.status), + "is_bot": member.bot + }) + + # 获取权限信息 + bot_member = guild.me + permissions = channel.permissions_for(bot_member) + + return { + "channel_id": channel_id, + "channel_name": channel.name, + "guild_id": guild_id, + "guild_name": guild.name, + "user_limit": channel.user_limit, + "current_users": users, + "user_count": len(users), + "bitrate": channel.bitrate, + "permissions": { + "connect": permissions.connect, + "speak": permissions.speak, + "use_voice_activation": permissions.use_voice_activation, + "priority_speaker": permissions.priority_speaker + } + } + + async def _validate_user_in_channel(self, guild: discord.Guild, + channel: discord.VoiceChannel, user_id: int): + """ + 验证用户是否在语音频道中 + + @author: @ydzat + + Args: + guild: Discord 服务器对象 + channel: 语音频道对象 + user_id: 用户ID + + Raises: + VoicePermissionError: 用户不在频道中时抛出 + """ + member = guild.get_member(user_id) + if not member: + raise VoicePermissionError( + f"无法找到用户 {user_id}", + ["member_not_found"], + user_id, + channel.id + ) + + if not member.voice or member.voice.channel != channel: + raise VoicePermissionError( + f"用户 {member.display_name} 不在语音频道 {channel.name} 中", + ["user_not_in_channel"], + user_id, + channel.id + ) + + async def _validate_bot_permissions(self, channel: discord.VoiceChannel): + """ + 验证机器人权限 + + @author: @ydzat + + Args: + channel: 语音频道对象 + + Raises: + VoicePermissionError: 权限不足时抛出 + """ + bot_member = channel.guild.me + permissions = channel.permissions_for(bot_member) + + missing_permissions = [] + + if not permissions.connect: + missing_permissions.append("connect") + if not permissions.speak: + missing_permissions.append("speak") + + if missing_permissions: + raise VoicePermissionError( + f"机器人在频道 {channel.name} 中缺少权限: {', '.join(missing_permissions)}", + missing_permissions, + channel_id=channel.id + ) + + async def cleanup_inactive_connections(self): + """ + 清理无效连接 + + 定期检查并清理已断开或无效的语音连接,释放资源。 + + @author: @ydzat + """ + cleanup_guilds = [] + + for guild_id, conn_info in self.connections.items(): + if not conn_info.voice_client or not conn_info.voice_client.is_connected(): + cleanup_guilds.append(guild_id) + + for guild_id in cleanup_guilds: + await self._disconnect_internal(guild_id) + + if cleanup_guilds: + await self.logger.info(f"清理了 {len(cleanup_guilds)} 个无效的语音连接") + + async def start_monitoring(self): + """ + 开始连接监控 + + @author: @ydzat + """ + if self._cleanup_task is None and self._monitoring_enabled: + self._cleanup_task = asyncio.create_task(self._monitoring_loop()) + + async def stop_monitoring(self): + """ + 停止连接监控 + + @author: @ydzat + """ + self._monitoring_enabled = False + if self._cleanup_task: + self._cleanup_task.cancel() + try: + await self._cleanup_task + except asyncio.CancelledError: + pass + self._cleanup_task = None + + async def _monitoring_loop(self): + """ + 监控循环 + + @author: @ydzat + """ + try: + while self._monitoring_enabled: + await asyncio.sleep(60) # 每分钟检查一次 + await self.cleanup_inactive_connections() + except asyncio.CancelledError: + pass + + async def disconnect_all(self): + """ + 断开所有连接 + + @author: @ydzat + """ + async with self._connection_lock: + guild_ids = list(self.connections.keys()) + for guild_id in guild_ids: + await self._disconnect_internal(guild_id) + + await self.stop_monitoring() class DiscordMessageConverter(adapter.MessageConverter): @@ -238,6 +814,9 @@ class DiscordAdapter(adapter.MessagePlatformAdapter): self.logger = logger self.bot_account_id = self.config['client_id'] + + # 初始化语音连接管理器 + self.voice_manager: VoiceConnectionManager = None adapter_self = self @@ -258,6 +837,169 @@ class DiscordAdapter(adapter.MessagePlatformAdapter): args['proxy'] = os.getenv('http_proxy') self.bot = MyClient(intents=intents, **args) + + # Voice functionality methods + async def join_voice_channel(self, guild_id: int, channel_id: int, + user_id: int = None) -> discord.VoiceClient: + """ + 加入语音频道 + + 为指定服务器的语音频道建立连接,支持用户权限验证和连接复用。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + channel_id (int): 语音频道ID + user_id (int, optional): 请求用户ID,用于权限验证 + + Returns: + discord.VoiceClient: 语音客户端实例 + + Raises: + VoicePermissionError: 权限不足 + VoiceNetworkError: 网络连接失败 + VoiceConnectionError: 其他连接错误 + """ + if not self.voice_manager: + raise VoiceConnectionError("语音管理器未初始化", "MANAGER_NOT_READY") + + return await self.voice_manager.join_voice_channel(guild_id, channel_id, user_id) + + async def leave_voice_channel(self, guild_id: int) -> bool: + """ + 离开语音频道 + + 断开指定服务器的语音连接,清理相关资源。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + + Returns: + bool: 是否成功断开连接 + """ + if not self.voice_manager: + return False + + return await self.voice_manager.leave_voice_channel(guild_id) + + async def get_voice_client(self, guild_id: int) -> typing.Optional[discord.VoiceClient]: + """ + 获取语音客户端 + + 返回指定服务器的语音客户端实例,用于音频播放控制。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + + Returns: + Optional[discord.VoiceClient]: 语音客户端实例或 None + """ + if not self.voice_manager: + return None + + return await self.voice_manager.get_voice_client(guild_id) + + async def is_connected_to_voice(self, guild_id: int) -> bool: + """ + 检查语音连接状态 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + + Returns: + bool: 是否已连接到语音频道 + """ + if not self.voice_manager: + return False + + return await self.voice_manager.is_connected_to_voice(guild_id) + + async def get_voice_connection_status(self, guild_id: int) -> typing.Optional[dict]: + """ + 获取语音连接详细状态 + + 返回包含连接时间、延迟、用户数等详细信息的状态字典。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + + Returns: + Optional[dict]: 连接状态信息或 None + """ + if not self.voice_manager: + return None + + return await self.voice_manager.get_connection_status(guild_id) + + async def list_active_voice_connections(self) -> typing.List[dict]: + """ + 列出所有活跃的语音连接 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Returns: + List[dict]: 活跃语音连接列表 + """ + if not self.voice_manager: + return [] + + return await self.voice_manager.list_active_connections() + + async def get_voice_channel_info(self, guild_id: int, channel_id: int) -> typing.Optional[dict]: + """ + 获取语音频道详细信息 + + 包括频道名称、用户列表、权限信息等。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + + Args: + guild_id (int): Discord 服务器ID + channel_id (int): 语音频道ID + + Returns: + Optional[dict]: 频道信息字典或 None + """ + if not self.voice_manager: + return None + + return await self.voice_manager.get_voice_channel_info(guild_id, channel_id) + + async def cleanup_voice_connections(self): + """ + 清理无效的语音连接 + + 手动触发语音连接清理,移除已断开或无效的连接。 + + @author: @ydzat + @version: 1.0 + @since: 2025-07-04 + """ + if self.voice_manager: + await self.voice_manager.cleanup_inactive_connections() async def send_message(self, target_type: str, target_id: str, message: platform_message.MessageChain): msg_to_send, image_files = await self.message_converter.yiri2target(message) @@ -324,9 +1066,32 @@ class DiscordAdapter(adapter.MessagePlatformAdapter): self.listeners.pop(event_type) async def run_async(self): + """ + 启动 Discord 适配器 + + 初始化语音管理器并启动 Discord 客户端连接。 + + @author: @ydzat (修改) + """ async with self.bot: + # 初始化语音管理器 + self.voice_manager = VoiceConnectionManager(self.bot, self.logger) + await self.voice_manager.start_monitoring() + + await self.logger.info("Discord 适配器语音功能已启用") await self.bot.start(self.config['token'], reconnect=True) async def kill(self) -> bool: + """ + 关闭 Discord 适配器 + + 清理语音连接并关闭 Discord 客户端。 + + @author: @ydzat (修改) + """ + if self.voice_manager: + await self.voice_manager.disconnect_all() + await self.bot.close() return True + diff --git a/pkg/platform/sources/lark.py b/pkg/platform/sources/lark.py index 5369be00..0d7fc0fb 100644 --- a/pkg/platform/sources/lark.py +++ b/pkg/platform/sources/lark.py @@ -344,11 +344,10 @@ class LarkAdapter(adapter.MessagePlatformAdapter): quart_app: quart.Quart ap: app.Application - message_id_to_card_id: typing.Dict[str, typing.Tuple[str, int]] - card_id_dict: dict[str, str] + card_id_dict: dict[str, str] # 消息id到卡片id的映射,便于创建卡片后的发送消息到指定卡片 - seq: int + seq: int # 用于在发送卡片消息中识别消息顺序,直接以seq作为标识 def __init__(self, config: dict, ap: app.Application, logger: EventLogger): self.config = config @@ -356,10 +355,9 @@ class LarkAdapter(adapter.MessagePlatformAdapter): self.logger = logger self.quart_app = quart.Quart(__name__) self.listeners = {} - self.message_id_to_card_id = {} self.card_id_dict = {} self.seq = 1 - self.card_id_time = {} + @self.quart_app.route('/lark/callback', methods=['POST']) async def lark_callback(): @@ -432,36 +430,34 @@ class LarkAdapter(adapter.MessagePlatformAdapter): async def create_card_id(self, message_id): try: - is_stream = await self.is_stream_output_supported() - if is_stream: - self.ap.logger.debug('飞书支持stream输出,创建卡片......') + self.ap.logger.debug('飞书支持stream输出,创建卡片......') - card_data = { - 'schema': '2.0', - 'header': {'title': {'content': 'bot', 'tag': 'plain_text'}}, - 'body': {'elements': [{'tag': 'markdown', 'content': '[思考中.....]', 'element_id': 'markdown_1'}]}, - 'config': {'streaming_mode': True, 'streaming_config': {'print_strategy': 'delay'}}, - } # delay / fast + card_data = { + 'schema': '2.0', + 'header': {'title': {'content': 'bot', 'tag': 'plain_text'}}, + 'body': {'elements': [{'tag': 'markdown', 'content': '[思考中.....]', 'element_id': 'markdown_1'}]}, + 'config': {'streaming_mode': True, 'streaming_config': {'print_strategy': 'delay'}}, + } # delay / fast 创建卡片模板,delay 延迟打印,fast 实时打印,可以自定义更好看的消息模板 - request: CreateCardRequest = ( - CreateCardRequest.builder() - .request_body(CreateCardRequestBody.builder().type('card_json').data(json.dumps(card_data)).build()) - .build() + request: CreateCardRequest = ( + CreateCardRequest.builder() + .request_body(CreateCardRequestBody.builder().type('card_json').data(json.dumps(card_data)).build()) + .build() + ) + + # 发起请求 + response: CreateCardResponse = self.api_client.cardkit.v1.card.create(request) + + # 处理失败返回 + if not response.success(): + raise Exception( + f'client.cardkit.v1.card.create failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}' ) - # 发起请求 - response: CreateCardResponse = self.api_client.cardkit.v1.card.create(request) + self.ap.logger.debug(f'飞书卡片创建成功,卡片ID: {response.data.card_id}') + self.card_id_dict[message_id] = response.data.card_id - # 处理失败返回 - if not response.success(): - raise Exception( - f'client.cardkit.v1.card.create failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}' - ) - - self.ap.logger.debug(f'飞书卡片创建成功,卡片ID: {response.data.card_id}') - self.card_id_dict[message_id] = response.data.card_id - - card_id = response.data.card_id + card_id = response.data.card_id return card_id except Exception as e: @@ -470,7 +466,7 @@ class LarkAdapter(adapter.MessagePlatformAdapter): async def create_message_card(self, message_id, event) -> str: """ 创建卡片消息。 - 使用卡片消息是因为普通消息更新次数有限制,而大模型流式返回结果可能很多而超过限制,而飞书卡片没有这个限制 + 使用卡片消息是因为普通消息更新次数有限制,而大模型流式返回结果可能很多而超过限制,而飞书卡片没有这个限制(api免费次数有限) """ # message_id = event.message_chain.message_id @@ -478,7 +474,7 @@ class LarkAdapter(adapter.MessagePlatformAdapter): content = { 'type': 'card', 'data': {'card_id': card_id, 'template_variable': {'content': 'Thinking...'}}, - } + } # 当收到消息时发送消息模板,可添加模板变量,详情查看飞书中接口文档 request: ReplyMessageRequest = ( ReplyMessageRequest.builder() .message_id(event.message_chain.message_id) @@ -547,48 +543,50 @@ class LarkAdapter(adapter.MessagePlatformAdapter): """ 回复消息变成更新卡片消息 """ - lark_message = await self.message_converter.yiri2target(message, self.api_client) - self.seq += 1 - text_message = '' - for ele in lark_message[0]: - if ele['tag'] == 'text': - text_message += ele['text'] - elif ele['tag'] == 'md': - text_message += ele['text'] + if (self.seq - 1) % 8 == 0 or is_final: + lark_message = await self.message_converter.yiri2target(message, self.api_client) - # content = { - # 'type': 'card_json', - # 'data': {'card_id': self.card_id_dict[message_id], 'elements': {'content': text_message}}, - # } - request: ContentCardElementRequest = ( - ContentCardElementRequest.builder() - .card_id(self.card_id_dict[message_id]) - .element_id('markdown_1') - .request_body( - ContentCardElementRequestBody.builder() - # .uuid("a0d69e20-1dd1-458b-k525-dfeca4015204") - .content(text_message) - .sequence(self.seq) + text_message = '' + for ele in lark_message[0]: + if ele['tag'] == 'text': + text_message += ele['text'] + elif ele['tag'] == 'md': + text_message += ele['text'] + + # content = { + # 'type': 'card_json', + # 'data': {'card_id': self.card_id_dict[message_id], 'elements': {'content': text_message}}, + # } + + request: ContentCardElementRequest = ( + ContentCardElementRequest.builder() + .card_id(self.card_id_dict[message_id]) + .element_id('markdown_1') + .request_body( + ContentCardElementRequestBody.builder() + # .uuid("a0d69e20-1dd1-458b-k525-dfeca4015204") + .content(text_message) + .sequence(self.seq) + .build() + ) .build() ) - .build() - ) - if is_final: - self.seq = 1 - self.card_id_dict.pop(message_id) - # 发起请求 - response: ContentCardElementResponse = self.api_client.cardkit.v1.card_element.content(request) + if is_final: + self.seq = 1 # 消息回复结束之后重置seq + self.card_id_dict.pop(message_id) # 清理已经使用过的卡片 + # 发起请求 + response: ContentCardElementResponse = self.api_client.cardkit.v1.card_element.content(request) - # 处理失败返回 - if not response.success(): - raise Exception( - f'client.im.v1.message.patch failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}' - ) - return + # 处理失败返回 + if not response.success(): + raise Exception( + f'client.im.v1.message.patch failed, code: {response.code}, msg: {response.msg}, log_id: {response.get_log_id()}, resp: \n{json.dumps(json.loads(response.raw.content), indent=4, ensure_ascii=False)}' + ) + return async def is_muted(self, group_id: int) -> bool: return False @@ -640,4 +638,4 @@ class LarkAdapter(adapter.MessagePlatformAdapter): # 所以要设置_auto_reconnect=False,让其不重连。 self.bot._auto_reconnect = False await self.bot._disconnect() - return False + return False \ No newline at end of file diff --git a/pkg/platform/sources/telegram.py b/pkg/platform/sources/telegram.py index d39bf23d..3c81fd6b 100644 --- a/pkg/platform/sources/telegram.py +++ b/pkg/platform/sources/telegram.py @@ -144,7 +144,9 @@ class TelegramAdapter(adapter.MessagePlatformAdapter): config: dict ap: app.Application - msg_stream_id: dict + msg_stream_id: dict # 流式消息id字典,key为流式消息id,value为首次消息源id,用于在流式消息时判断编辑那条消息 + + seq: int # 消息中识别消息顺序,直接以seq作为标识 listeners: typing.Dict[ typing.Type[platform_events.Event], @@ -156,6 +158,7 @@ class TelegramAdapter(adapter.MessagePlatformAdapter): self.ap = ap self.logger = logger self.msg_stream_id = {} + self.seq = 1 async def telegram_callback(update: Update, context: ContextTypes.DEFAULT_TYPE): if update.message.from_user.is_bot: @@ -213,52 +216,56 @@ class TelegramAdapter(adapter.MessagePlatformAdapter): quote_origin: bool = False, is_final: bool = False, ): - assert isinstance(message_source.source_platform_object, Update) - components = await TelegramMessageConverter.yiri2target(message, self.bot) - args = {} - message_id = message_source.source_platform_object.message.id - if quote_origin: - args['reply_to_message_id'] = message_source.source_platform_object.message.id + self.seq += 1 + if (self.seq - 1) % 8 == 0 or is_final: - component = components[0] - if message_id not in self.msg_stream_id: - # time.sleep(0.6) - if component['type'] == 'text': - if self.config['markdown_card'] is True: - content = telegramify_markdown.markdownify( - content=component['text'], - ) - else: - content = component['text'] - args = { - 'chat_id': message_source.source_platform_object.effective_chat.id, - 'text': content, - } - if self.config['markdown_card'] is True: - args['parse_mode'] = 'MarkdownV2' + assert isinstance(message_source.source_platform_object, Update) + components = await TelegramMessageConverter.yiri2target(message, self.bot) + args = {} + message_id = message_source.source_platform_object.message.id + if quote_origin: + args['reply_to_message_id'] = message_source.source_platform_object.message.id - send_msg = await self.bot.send_message(**args) - send_msg_id = send_msg.message_id - self.msg_stream_id[message_id] = send_msg_id - else: - if component['type'] == 'text': - if self.config['markdown_card'] is True: - content = telegramify_markdown.markdownify( - content=component['text'], - ) - else: - content = component['text'] - args = { - 'message_id': self.msg_stream_id[message_id], - 'chat_id': message_source.source_platform_object.effective_chat.id, - 'text': content, - } - if self.config['markdown_card'] is True: - args['parse_mode'] = 'MarkdownV2' + component = components[0] + if message_id not in self.msg_stream_id: # 当消息回复第一次时,发送新消息 + # time.sleep(0.6) + if component['type'] == 'text': + if self.config['markdown_card'] is True: + content = telegramify_markdown.markdownify( + content=component['text'], + ) + else: + content = component['text'] + args = { + 'chat_id': message_source.source_platform_object.effective_chat.id, + 'text': content, + } + if self.config['markdown_card'] is True: + args['parse_mode'] = 'MarkdownV2' - await self.bot.edit_message_text(**args) - if is_final: - self.msg_stream_id.pop(message_id) + send_msg = await self.bot.send_message(**args) + send_msg_id = send_msg.message_id + self.msg_stream_id[message_id] = send_msg_id + else: # 存在消息的时候直接编辑消息1 + if component['type'] == 'text': + if self.config['markdown_card'] is True: + content = telegramify_markdown.markdownify( + content=component['text'], + ) + else: + content = component['text'] + args = { + 'message_id': self.msg_stream_id[message_id], + 'chat_id': message_source.source_platform_object.effective_chat.id, + 'text': content, + } + if self.config['markdown_card'] is True: + args['parse_mode'] = 'MarkdownV2' + + await self.bot.edit_message_text(**args) + if is_final: + self.seq = 1 # 消息回复结束之后重置seq + self.msg_stream_id.pop(message_id) # 消息回复结束之后删除流式消息id async def is_stream_output_supported(self) -> bool: is_stream = False diff --git a/pkg/platform/sources/wechatpad.py b/pkg/platform/sources/wechatpad.py index 5d8ec75d..895e77fb 100644 --- a/pkg/platform/sources/wechatpad.py +++ b/pkg/platform/sources/wechatpad.py @@ -29,19 +29,21 @@ import logging class WeChatPadMessageConverter(adapter.MessageConverter): - def __init__(self, config: dict): + + def __init__(self, config: dict, logger: logging.Logger): self.config = config - self.bot = WeChatPadClient(self.config['wechatpad_url'], self.config['token']) - self.logger = logging.getLogger('WeChatPadMessageConverter') + self.bot = WeChatPadClient(self.config["wechatpad_url"],self.config["token"]) + self.logger = logger @staticmethod async def yiri2target(message_chain: platform_message.MessageChain) -> list[dict]: content_list = [] - # current_file_path = os.path.abspath(__file__) for component in message_chain: - if isinstance(component, platform_message.At): - content_list.append({'type': 'at', 'target': component.target}) + if isinstance(component, platform_message.AtAll): + content_list.append({"type": "at", "target": "all"}) + elif isinstance(component, platform_message.At): + content_list.append({"type": "at", "target": component.target}) elif isinstance(component, platform_message.Plain): content_list.append({'type': 'text', 'content': component.text}) elif isinstance(component, platform_message.Image): @@ -74,20 +76,34 @@ class WeChatPadMessageConverter(adapter.MessageConverter): return content_list - async def target2yiri(self, message: dict, bot_account_id: str) -> platform_message.MessageChain: + async def target2yiri( + self, + message: dict, + bot_account_id: str, + ) -> platform_message.MessageChain: """外部消息转平台消息""" # 数据预处理 message_list = [] + bot_wxid = self.config['wxid'] ats_bot = False # 是否被@ content = message['content']['str'] content_no_preifx = content # 群消息则去掉前缀 is_group_message = self._is_group_message(message) if is_group_message: ats_bot = self._ats_bot(message, bot_account_id) - if '@所有人' in content: + + self.logger.info(f"ats_bot: {ats_bot}; bot_account_id: {bot_account_id}; bot_wxid: {bot_wxid}") + if "@所有人" in content: message_list.append(platform_message.AtAll()) - elif ats_bot: + if ats_bot: message_list.append(platform_message.At(target=bot_account_id)) + + # 解析@信息并生成At组件 + at_targets = self._extract_at_targets(message) + for target_id in at_targets: + if target_id != bot_wxid: # 避免重复添加机器人的At + message_list.append(platform_message.At(target=target_id)) + content_no_preifx, _ = self._extract_content_and_sender(content) msg_type = message['msg_type'] @@ -393,6 +409,23 @@ class WeChatPadMessageConverter(adapter.MessageConverter): finally: return ats_bot + # 提取一下at的wxid列表 + def _extract_at_targets(self, message: dict) -> list[str]: + """从消息中提取被@用户的ID列表""" + at_targets = [] + try: + # 从msg_source中解析atuserlist + msg_source = message.get('msg_source', '') or '' + if len(msg_source) > 0: + msg_source_data = ET.fromstring(msg_source) + at_user_list = msg_source_data.findtext("atuserlist") or "" + if at_user_list: + # atuserlist格式通常是逗号分隔的用户ID列表 + at_targets = [user_id.strip() for user_id in at_user_list.split(',') if user_id.strip()] + except Exception as e: + self.logger.error(f"_extract_at_targets got except: {e}") + return at_targets + # 提取一下content前面的sender_id, 和去掉前缀的内容 def _extract_content_and_sender(self, raw_content: str) -> Tuple[str, Optional[str]]: try: @@ -416,16 +449,22 @@ class WeChatPadMessageConverter(adapter.MessageConverter): class WeChatPadEventConverter(adapter.EventConverter): - def __init__(self, config: dict): - self.config = config - self.message_converter = WeChatPadMessageConverter(config) - self.logger = logging.getLogger('WeChatPadEventConverter') + def __init__(self, config: dict, logger: logging.Logger): + self.config = config + self.message_converter = WeChatPadMessageConverter(config, logger) + self.logger = logger + @staticmethod async def yiri2target(event: platform_events.MessageEvent) -> dict: pass - async def target2yiri(self, event: dict, bot_account_id: str) -> platform_events.MessageEvent: + async def target2yiri( + self, + event: dict, + bot_account_id: str, + ) -> platform_events.MessageEvent: + # 排除公众号以及微信团队消息 if ( event['from_user_name']['str'].startswith('gh_') @@ -501,8 +540,8 @@ class WeChatPadAdapter(adapter.MessagePlatformAdapter): self.logger = logger self.quart_app = quart.Quart(__name__) - self.message_converter = WeChatPadMessageConverter(config) - self.event_converter = WeChatPadEventConverter(config) + self.message_converter = WeChatPadMessageConverter(config, ap.logger) + self.event_converter = WeChatPadEventConverter(config, ap.logger) async def ws_message(self, data): """处理接收到的消息""" @@ -537,19 +576,26 @@ class WeChatPadAdapter(adapter.MessagePlatformAdapter): for msg in content_list: # 文本消息处理@ if msg['type'] == 'text' and at_targets: - at_nick_name_list = [] - for member in member_info: - if member['user_name'] in at_targets: - at_nick_name_list.append(f'@{member["nick_name"]}') - msg['content'] = f'{" ".join(at_nick_name_list)} {msg["content"]}' + if "all" in at_targets: + msg['content'] = f'@所有人 {msg["content"]}' + else: + at_nick_name_list = [] + for member in member_info: + if member["user_name"] in at_targets: + at_nick_name_list.append(f'@{member["nick_name"]}') + msg['content'] = f'{" ".join(at_nick_name_list)} {msg["content"]}' # 统一消息派发 handler_map = { 'text': lambda msg: self.bot.send_text_message( - to_wxid=target_id, message=msg['content'], ats=at_targets + to_wxid=target_id, + message=msg['content'], + ats= ["notify@all"] if "all" in at_targets else at_targets ), 'image': lambda msg: self.bot.send_image_message( - to_wxid=target_id, img_url=msg['image'], ats=at_targets + to_wxid=target_id, + img_url=msg["image"], + ats = ["notify@all"] if "all" in at_targets else at_targets ), 'WeChatEmoji': lambda msg: self.bot.send_emoji_message( to_wxid=target_id, emoji_md5=msg['emoji_md5'], emoji_size=msg['emoji_size'] diff --git a/pkg/provider/modelmgr/entities.py b/pkg/provider/modelmgr/entities.py index cf856894..7bc02a32 100644 --- a/pkg/provider/modelmgr/entities.py +++ b/pkg/provider/modelmgr/entities.py @@ -17,7 +17,7 @@ class LLMModelInfo(pydantic.BaseModel): token_mgr: token.TokenManager - requester: requester.LLMAPIRequester + requester: requester.ProviderAPIRequester tool_call_supported: typing.Optional[bool] = False diff --git a/pkg/provider/modelmgr/modelmgr.py b/pkg/provider/modelmgr/modelmgr.py index b15e53a9..2c92eacc 100644 --- a/pkg/provider/modelmgr/modelmgr.py +++ b/pkg/provider/modelmgr/modelmgr.py @@ -18,7 +18,7 @@ class ModelManager: model_list: list[entities.LLMModelInfo] # deprecated - requesters: dict[str, requester.LLMAPIRequester] # deprecated + requesters: dict[str, requester.ProviderAPIRequester] # deprecated token_mgrs: dict[str, token.TokenManager] # deprecated @@ -28,9 +28,11 @@ class ModelManager: llm_models: list[requester.RuntimeLLMModel] + embedding_models: list[requester.RuntimeEmbeddingModel] + requester_components: list[engine.Component] - requester_dict: dict[str, type[requester.LLMAPIRequester]] # cache + requester_dict: dict[str, type[requester.ProviderAPIRequester]] # cache def __init__(self, ap: app.Application): self.ap = ap @@ -38,6 +40,7 @@ class ModelManager: self.requesters = {} self.token_mgrs = {} self.llm_models = [] + self.embedding_models = [] self.requester_components = [] self.requester_dict = {} @@ -45,7 +48,7 @@ class ModelManager: self.requester_components = self.ap.discover.get_components_by_kind('LLMAPIRequester') # forge requester class dict - requester_dict: dict[str, type[requester.LLMAPIRequester]] = {} + requester_dict: dict[str, type[requester.ProviderAPIRequester]] = {} for component in self.requester_components: requester_dict[component.metadata.name] = component.get_python_component_class() @@ -58,13 +61,11 @@ class ModelManager: self.ap.logger.info('Loading models from db...') self.llm_models = [] + self.embedding_models = [] # llm models result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.LLMModel)) - llm_models = result.all() - - # load models for llm_model in llm_models: try: await self.load_llm_model(llm_model) @@ -73,11 +74,17 @@ class ModelManager: except Exception as e: self.ap.logger.error(f'Failed to load model {llm_model.uuid}: {e}\n{traceback.format_exc()}') + # embedding models + result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_model.EmbeddingModel)) + embedding_models = result.all() + for embedding_model in embedding_models: + await self.load_embedding_model(embedding_model) + async def init_runtime_llm_model( self, model_info: persistence_model.LLMModel | sqlalchemy.Row[persistence_model.LLMModel] | dict, ): - """初始化运行时模型""" + """初始化运行时 LLM 模型""" if isinstance(model_info, sqlalchemy.Row): model_info = persistence_model.LLMModel(**model_info._mapping) elif isinstance(model_info, dict): @@ -101,14 +108,47 @@ class ModelManager: return runtime_llm_model + async def init_runtime_embedding_model( + self, + model_info: persistence_model.EmbeddingModel | sqlalchemy.Row[persistence_model.EmbeddingModel] | dict, + ): + """初始化运行时 Embedding 模型""" + if isinstance(model_info, sqlalchemy.Row): + model_info = persistence_model.EmbeddingModel(**model_info._mapping) + elif isinstance(model_info, dict): + model_info = persistence_model.EmbeddingModel(**model_info) + + requester_inst = self.requester_dict[model_info.requester](ap=self.ap, config=model_info.requester_config) + + await requester_inst.initialize() + + runtime_embedding_model = requester.RuntimeEmbeddingModel( + model_entity=model_info, + token_mgr=token.TokenManager( + name=model_info.uuid, + tokens=model_info.api_keys, + ), + requester=requester_inst, + ) + + return runtime_embedding_model + async def load_llm_model( self, model_info: persistence_model.LLMModel | sqlalchemy.Row[persistence_model.LLMModel] | dict, ): - """加载模型""" + """加载 LLM 模型""" runtime_llm_model = await self.init_runtime_llm_model(model_info) self.llm_models.append(runtime_llm_model) + async def load_embedding_model( + self, + model_info: persistence_model.EmbeddingModel | sqlalchemy.Row[persistence_model.EmbeddingModel] | dict, + ): + """加载 Embedding 模型""" + runtime_embedding_model = await self.init_runtime_embedding_model(model_info) + self.embedding_models.append(runtime_embedding_model) + async def get_model_by_name(self, name: str) -> entities.LLMModelInfo: # deprecated """通过名称获取模型""" for model in self.model_list: @@ -116,23 +156,44 @@ class ModelManager: return model raise ValueError(f'无法确定模型 {name} 的信息') - async def get_model_by_uuid(self, uuid: str) -> entities.LLMModelInfo: - """通过uuid获取模型""" + async def get_model_by_uuid(self, uuid: str) -> requester.RuntimeLLMModel: + """通过uuid获取 LLM 模型""" for model in self.llm_models: if model.model_entity.uuid == uuid: return model - raise ValueError(f'model {uuid} not found') + raise ValueError(f'LLM model {uuid} not found') + + async def get_embedding_model_by_uuid(self, uuid: str) -> requester.RuntimeEmbeddingModel: + """通过uuid获取 Embedding 模型""" + for model in self.embedding_models: + if model.model_entity.uuid == uuid: + return model + raise ValueError(f'Embedding model {uuid} not found') async def remove_llm_model(self, model_uuid: str): - """移除模型""" + """移除 LLM 模型""" for model in self.llm_models: if model.model_entity.uuid == model_uuid: self.llm_models.remove(model) return - def get_available_requesters_info(self) -> list[dict]: + async def remove_embedding_model(self, model_uuid: str): + """移除 Embedding 模型""" + for model in self.embedding_models: + if model.model_entity.uuid == model_uuid: + self.embedding_models.remove(model) + return + + def get_available_requesters_info(self, model_type: str) -> list[dict]: """获取所有可用的请求器""" - return [component.to_plain_dict() for component in self.requester_components] + if model_type != '': + return [ + component.to_plain_dict() + for component in self.requester_components + if model_type in component.spec['support_type'] + ] + else: + return [component.to_plain_dict() for component in self.requester_components] def get_available_requester_info_by_name(self, name: str) -> dict | None: """通过名称获取请求器信息""" diff --git a/pkg/provider/modelmgr/requester.py b/pkg/provider/modelmgr/requester.py index 5701d846..6af8ba70 100644 --- a/pkg/provider/modelmgr/requester.py +++ b/pkg/provider/modelmgr/requester.py @@ -20,22 +20,45 @@ class RuntimeLLMModel: token_mgr: token.TokenManager """api key管理器""" - requester: LLMAPIRequester + requester: ProviderAPIRequester """请求器实例""" def __init__( self, model_entity: persistence_model.LLMModel, token_mgr: token.TokenManager, - requester: LLMAPIRequester, + requester: ProviderAPIRequester, ): self.model_entity = model_entity self.token_mgr = token_mgr self.requester = requester -class LLMAPIRequester(metaclass=abc.ABCMeta): - """LLM API请求器""" +class RuntimeEmbeddingModel: + """运行时 Embedding 模型""" + + model_entity: persistence_model.EmbeddingModel + """模型数据""" + + token_mgr: token.TokenManager + """api key管理器""" + + requester: ProviderAPIRequester + """请求器实例""" + + def __init__( + self, + model_entity: persistence_model.EmbeddingModel, + token_mgr: token.TokenManager, + requester: ProviderAPIRequester, + ): + self.model_entity = model_entity + self.token_mgr = token_mgr + self.requester = requester + + +class ProviderAPIRequester(metaclass=abc.ABCMeta): + """Provider API请求器""" name: str = None @@ -61,6 +84,7 @@ class LLMAPIRequester(metaclass=abc.ABCMeta): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: """调用API @@ -69,6 +93,7 @@ class LLMAPIRequester(metaclass=abc.ABCMeta): messages (typing.List[llm_entities.Message]): 消息对象列表 funcs (typing.List[tools_entities.LLMFunction], optional): 使用的工具函数列表. Defaults to None. extra_args (dict[str, typing.Any], optional): 额外的参数. Defaults to {}. + remove_think (bool, optional): 是否移思考中的消息. Defaults to False. Returns: llm_entities.Message: 返回消息对象 @@ -82,6 +107,7 @@ class LLMAPIRequester(metaclass=abc.ABCMeta): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.MessageChunk: """调用API @@ -90,8 +116,28 @@ class LLMAPIRequester(metaclass=abc.ABCMeta): messages (typing.List[llm_entities.Message]): 消息对象列表 funcs (typing.List[tools_entities.LLMFunction], optional): 使用的工具函数列表. Defaults to None. extra_args (dict[str, typing.Any], optional): 额外的参数. Defaults to {}. + remove_think (bool, optional): 是否移除思考中的消息. Defaults to False. Returns: typing.AsyncGenerator[llm_entities.MessageChunk]: 返回消息对象 """ pass + + async def invoke_embedding( + self, + model: RuntimeEmbeddingModel, + input_text: list[str], + extra_args: dict[str, typing.Any] = {}, + ) -> list[list[float]]: + """调用 Embedding API + + Args: + query (core_entities.Query): 请求上下文 + model (RuntimeEmbeddingModel): 使用的模型信息 + input_text (list[str]): 输入文本 + extra_args (dict[str, typing.Any], optional): 额外的参数. Defaults to {}. + + Returns: + list[list[float]]: 返回的 embedding 向量 + """ + pass diff --git a/pkg/provider/modelmgr/requesters/302aichatcmpl.py b/pkg/provider/modelmgr/requesters/302aichatcmpl.py index bd9aaccd..40a41718 100644 --- a/pkg/provider/modelmgr/requesters/302aichatcmpl.py +++ b/pkg/provider/modelmgr/requesters/302aichatcmpl.py @@ -7,7 +7,7 @@ from . import chatcmpl class AI302ChatCompletions(chatcmpl.OpenAIChatCompletions): - """302 AI ChatCompletion API 请求器""" + """302.AI ChatCompletion API 请求器""" client: openai.AsyncClient diff --git a/pkg/provider/modelmgr/requesters/302aichatcmpl.yaml b/pkg/provider/modelmgr/requesters/302aichatcmpl.yaml index 9d8ce9ea..5128f61d 100644 --- a/pkg/provider/modelmgr/requesters/302aichatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/302aichatcmpl.yaml @@ -3,8 +3,8 @@ kind: LLMAPIRequester metadata: name: 302-ai-chat-completions label: - en_US: 302 AI - zh_Hans: 302 AI + en_US: 302.AI + zh_Hans: 302.AI icon: 302ai.png spec: config: @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./302aichatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/anthropicmsgs.py b/pkg/provider/modelmgr/requesters/anthropicmsgs.py index 38573854..75f2bf7e 100644 --- a/pkg/provider/modelmgr/requesters/anthropicmsgs.py +++ b/pkg/provider/modelmgr/requesters/anthropicmsgs.py @@ -15,7 +15,7 @@ from ...tools import entities as tools_entities from ....utils import image -class AnthropicMessages(requester.LLMAPIRequester): +class AnthropicMessages(requester.ProviderAPIRequester): """Anthropic Messages API 请求器""" client: anthropic.AsyncAnthropic @@ -53,6 +53,7 @@ class AnthropicMessages(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: self.client.api_key = model.token_mgr.get_token() @@ -151,7 +152,7 @@ class AnthropicMessages(requester.LLMAPIRequester): assert type(resp) is anthropic.types.message.Message for block in resp.content: - if block.type == 'thinking': + if not remove_think and block.type == 'thinking': args['content'] = '' + block.thinking + '\n' + args['content'] elif block.type == 'text': args['content'] += block.text diff --git a/pkg/provider/modelmgr/requesters/anthropicmsgs.yaml b/pkg/provider/modelmgr/requesters/anthropicmsgs.yaml index c124fed9..7dbcf3ed 100644 --- a/pkg/provider/modelmgr/requesters/anthropicmsgs.yaml +++ b/pkg/provider/modelmgr/requesters/anthropicmsgs.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./anthropicmsgs.py diff --git a/pkg/provider/modelmgr/requesters/bailianchatcmpl.yaml b/pkg/provider/modelmgr/requesters/bailianchatcmpl.yaml index 24beb915..10aae30f 100644 --- a/pkg/provider/modelmgr/requesters/bailianchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/bailianchatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./bailianchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/chatcmpl.py b/pkg/provider/modelmgr/requesters/chatcmpl.py index aa783bee..cf557755 100644 --- a/pkg/provider/modelmgr/requesters/chatcmpl.py +++ b/pkg/provider/modelmgr/requesters/chatcmpl.py @@ -13,7 +13,7 @@ from ... import entities as llm_entities from ...tools import entities as tools_entities -class OpenAIChatCompletions(requester.LLMAPIRequester): +class OpenAIChatCompletions(requester.ProviderAPIRequester): """OpenAI ChatCompletion API 请求器""" client: openai.AsyncClient @@ -51,11 +51,10 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): async def _make_msg( self, chat_completion: chat_completion.ChatCompletion, - pipeline_config: dict[str, typing.Any] = {'trigger': {'misc': {'remove_think': False}}}, + remove_think: bool = False, ) -> llm_entities.Message: chatcmpl_message = chat_completion.choices[0].message.model_dump() # print(chatcmpl_message.keys(),chatcmpl_message.values()) - # 确保 role 字段存在且不为 None if 'role' not in chatcmpl_message or chatcmpl_message['role'] is None: chatcmpl_message['role'] = 'assistant' @@ -63,7 +62,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): reasoning_content = chatcmpl_message['reasoning_content'] if 'reasoning_content' in chatcmpl_message else None # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: pass else: if reasoning_content is not None: @@ -77,7 +76,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): async def _make_msg_chunk( self, - pipeline_config: dict[str, typing.Any], + remove_think: bool, chat_completion: chat_completion.ChatCompletion, idx: int, ) -> llm_entities.MessageChunk: @@ -102,7 +101,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): # print(reasoning_content) # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: if reasoning_content is not None: pass else: @@ -130,6 +129,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.MessageChunk: self.client.api_key = use_model.token_mgr.get_token() @@ -161,10 +161,9 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): chunk_idx = 0 self.is_content = False tool_calls_map: dict[str, llm_entities.ToolCall] = {} - pipeline_config = query.pipeline_config async for chunk in self._req_stream(args, extra_body=extra_args): # 处理流式消息 - delta_message = await self._make_msg_chunk(pipeline_config, chunk, chunk_idx) + delta_message = await self._make_msg_chunk(remove_think, chunk, chunk_idx) if delta_message.content: current_content += delta_message.content delta_message.content = current_content @@ -199,6 +198,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: self.client.api_key = use_model.token_mgr.get_token() @@ -229,8 +229,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): resp = await self._req(args, extra_body=extra_args) # 处理请求结果 - pipeline_config = query.pipeline_config - message = await self._make_msg(resp, pipeline_config) + message = await self._make_msg(resp, remove_think) return message @@ -241,6 +240,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: req_messages = [] # req_messages 仅用于类内,外部同步由 query.messages 进行 for m in messages: @@ -260,6 +260,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): use_model=model, use_funcs=funcs, extra_args=extra_args, + remove_think=remove_think, ) return msg except asyncio.TimeoutError: @@ -278,6 +279,34 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): except openai.APIError as e: raise errors.RequesterError(f'请求错误: {e.message}') + async def invoke_embedding( + self, + model: requester.RuntimeEmbeddingModel, + input_text: list[str], + extra_args: dict[str, typing.Any] = {}, + ) -> list[list[float]]: + """调用 Embedding API""" + self.client.api_key = model.token_mgr.get_token() + + args = { + 'model': model.model_entity.name, + 'input': input_text, + } + + if model.model_entity.extra_args: + args.update(model.model_entity.extra_args) + + args.update(extra_args) + + try: + resp = await self.client.embeddings.create(**args) + + return [d.embedding for d in resp.data] + except asyncio.TimeoutError: + raise errors.RequesterError('请求超时') + except openai.BadRequestError as e: + raise errors.RequesterError(f'请求参数错误: {e.message}') + async def invoke_llm_stream( self, query: core_entities.Query, @@ -285,6 +314,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.MessageChunk: req_messages = [] # req_messages 仅用于类内,外部同步由 query.messages 进行 for m in messages: @@ -304,6 +334,7 @@ class OpenAIChatCompletions(requester.LLMAPIRequester): use_model=model, use_funcs=funcs, extra_args=extra_args, + remove_think=remove_think, ): yield item diff --git a/pkg/provider/modelmgr/requesters/chatcmpl.yaml b/pkg/provider/modelmgr/requesters/chatcmpl.yaml index 908b30ac..ff0de6f9 100644 --- a/pkg/provider/modelmgr/requesters/chatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/chatcmpl.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./chatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/compshare.png b/pkg/provider/modelmgr/requesters/compshare.png new file mode 100644 index 00000000..3ef1709c Binary files /dev/null and b/pkg/provider/modelmgr/requesters/compshare.png differ diff --git a/pkg/provider/modelmgr/requesters/compsharechatcmpl.py b/pkg/provider/modelmgr/requesters/compsharechatcmpl.py new file mode 100644 index 00000000..d272e721 --- /dev/null +++ b/pkg/provider/modelmgr/requesters/compsharechatcmpl.py @@ -0,0 +1,17 @@ +from __future__ import annotations + +import typing +import openai + +from . import chatcmpl + + +class CompShareChatCompletions(chatcmpl.OpenAIChatCompletions): + """CompShare ChatCompletion API 请求器""" + + client: openai.AsyncClient + + default_config: dict[str, typing.Any] = { + 'base_url': 'https://api.modelverse.cn/v1', + 'timeout': 120, + } diff --git a/pkg/provider/modelmgr/requesters/compsharechatcmpl.yaml b/pkg/provider/modelmgr/requesters/compsharechatcmpl.yaml new file mode 100644 index 00000000..2b7f9a70 --- /dev/null +++ b/pkg/provider/modelmgr/requesters/compsharechatcmpl.yaml @@ -0,0 +1,30 @@ +apiVersion: v1 +kind: LLMAPIRequester +metadata: + name: compshare-chat-completions + label: + en_US: CompShare + zh_Hans: 优云智算 + icon: compshare.png +spec: + config: + - name: base_url + label: + en_US: Base URL + zh_Hans: 基础 URL + type: string + required: true + default: "https://api.modelverse.cn/v1" + - name: timeout + label: + en_US: Timeout + zh_Hans: 超时时间 + type: integer + required: true + default: 120 + support_type: + - llm +execution: + python: + path: ./compsharechatcmpl.py + attr: CompShareChatCompletions \ No newline at end of file diff --git a/pkg/provider/modelmgr/requesters/deepseekchatcmpl.py b/pkg/provider/modelmgr/requesters/deepseekchatcmpl.py index d75d0fb6..4866caf4 100644 --- a/pkg/provider/modelmgr/requesters/deepseekchatcmpl.py +++ b/pkg/provider/modelmgr/requesters/deepseekchatcmpl.py @@ -24,6 +24,7 @@ class DeepseekChatCompletions(chatcmpl.OpenAIChatCompletions): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: self.client.api_key = use_model.token_mgr.get_token() @@ -53,8 +54,7 @@ class DeepseekChatCompletions(chatcmpl.OpenAIChatCompletions): if resp is None: raise errors.RequesterError('接口返回为空,请确定模型提供商服务是否正常') - pipeline_config = query.pipeline_config # 处理请求结果 - message = await self._make_msg(resp, pipeline_config) + message = await self._make_msg(resp, remove_think) return message diff --git a/pkg/provider/modelmgr/requesters/deepseekchatcmpl.yaml b/pkg/provider/modelmgr/requesters/deepseekchatcmpl.yaml index ea2c7eea..6f320e66 100644 --- a/pkg/provider/modelmgr/requesters/deepseekchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/deepseekchatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./deepseekchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/geminichatcmpl.yaml b/pkg/provider/modelmgr/requesters/geminichatcmpl.yaml index 6bfc085e..73fca19c 100644 --- a/pkg/provider/modelmgr/requesters/geminichatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/geminichatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./geminichatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/giteeaichatcmpl.py b/pkg/provider/modelmgr/requesters/giteeaichatcmpl.py index 7ac9fa1a..a8d6eb16 100644 --- a/pkg/provider/modelmgr/requesters/giteeaichatcmpl.py +++ b/pkg/provider/modelmgr/requesters/giteeaichatcmpl.py @@ -28,6 +28,7 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: self.client.api_key = use_model.token_mgr.get_token() @@ -49,16 +50,15 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): resp = await self._req(args, extra_body=extra_args) - pipeline_config = query.pipeline_config - message = await self._make_msg(resp, pipeline_config) + message = await self._make_msg(resp, remove_think) return message async def _make_msg( self, chat_completion: chat_completion.ChatCompletion, - pipeline_config: dict[str, typing.Any] = {'trigger': {'misc': {'remove_think': False}}}, + remove_think: bool, ) -> llm_entities.Message: chatcmpl_message = chat_completion.choices[0].message.model_dump() # print(chatcmpl_message.keys(), chatcmpl_message.values()) @@ -70,7 +70,7 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): reasoning_content = chatcmpl_message['reasoning_content'] if 'reasoning_content' in chatcmpl_message else None # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: chatcmpl_message['content'] = re.sub( r'.*?', '', chatcmpl_message['content'], flags=re.DOTALL ) @@ -86,7 +86,7 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): async def _make_msg_chunk( self, - pipeline_config: dict[str, typing.Any], + remove_think: bool, chat_completion: chat_completion.ChatCompletion, idx: int, ) -> llm_entities.MessageChunk: @@ -110,7 +110,7 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): # print(reasoning_content) # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: if delta['content'] == '': self.is_think = True delta['content'] = '' @@ -136,6 +136,7 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message | typing.AsyncGenerator[llm_entities.MessageChunk, None]: self.client.api_key = use_model.token_mgr.get_token() @@ -167,10 +168,9 @@ class GiteeAIChatCompletions(chatcmpl.OpenAIChatCompletions): chunk_idx = 0 self.is_content = False tool_calls_map: dict[str, llm_entities.ToolCall] = {} - pipeline_config = query.pipeline_config async for chunk in self._req_stream(args, extra_body=extra_args): # 处理流式消息 - delta_message = await self._make_msg_chunk(pipeline_config, chunk, chunk_idx) + delta_message = await self._make_msg_chunk(remove_think, chunk, chunk_idx) if delta_message.content: current_content += delta_message.content delta_message.content = current_content diff --git a/pkg/provider/modelmgr/requesters/giteeaichatcmpl.yaml b/pkg/provider/modelmgr/requesters/giteeaichatcmpl.yaml index a18675a1..d1aec26b 100644 --- a/pkg/provider/modelmgr/requesters/giteeaichatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/giteeaichatcmpl.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./giteeaichatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.yaml b/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.yaml index 893235b2..8c44ab39 100644 --- a/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/lmstudiochatcmpl.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./lmstudiochatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/modelscopechatcmpl.py b/pkg/provider/modelmgr/requesters/modelscopechatcmpl.py index 9a4e723e..7895a87e 100644 --- a/pkg/provider/modelmgr/requesters/modelscopechatcmpl.py +++ b/pkg/provider/modelmgr/requesters/modelscopechatcmpl.py @@ -14,7 +14,7 @@ from ... import entities as llm_entities from ...tools import entities as tools_entities -class ModelScopeChatCompletions(requester.LLMAPIRequester): +class ModelScopeChatCompletions(requester.ProviderAPIRequester): """ModelScope ChatCompletion API 请求器""" client: openai.AsyncClient @@ -174,7 +174,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): async def _make_msg_chunk( self, - pipeline_config: dict[str, typing.Any], + remove_think: bool, chat_completion: chat_completion.ChatCompletion, idx: int, ) -> llm_entities.MessageChunk: @@ -199,7 +199,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): # print(reasoning_content) # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: if reasoning_content is not None: pass else: @@ -227,6 +227,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message | typing.AsyncGenerator[llm_entities.MessageChunk, None]: self.client.api_key = use_model.token_mgr.get_token() @@ -258,10 +259,9 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): chunk_idx = 0 self.is_content = False tool_calls_map: dict[str, llm_entities.ToolCall] = {} - pipeline_config = query.pipeline_config async for chunk in self._req_stream(args, extra_body=extra_args): # 处理流式消息 - delta_message = await self._make_msg_chunk(pipeline_config, chunk, chunk_idx) + delta_message = await self._make_msg_chunk(remove_think, chunk, chunk_idx) if delta_message.content: current_content += delta_message.content delta_message.content = current_content @@ -296,6 +296,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: req_messages = [] # req_messages 仅用于类内,外部同步由 query.messages 进行 for m in messages: @@ -335,6 +336,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.MessageChunk: req_messages = [] # req_messages 仅用于类内,外部同步由 query.messages 进行 for m in messages: @@ -354,6 +356,7 @@ class ModelScopeChatCompletions(requester.LLMAPIRequester): use_model=model, use_funcs=funcs, extra_args=extra_args, + remove_think=remove_think, ): yield item diff --git a/pkg/provider/modelmgr/requesters/modelscopechatcmpl.yaml b/pkg/provider/modelmgr/requesters/modelscopechatcmpl.yaml index a641a672..a926d889 100644 --- a/pkg/provider/modelmgr/requesters/modelscopechatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/modelscopechatcmpl.yaml @@ -29,6 +29,8 @@ spec: type: int required: true default: 120 + support_type: + - llm execution: python: path: ./modelscopechatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/moonshotchatcmpl.py b/pkg/provider/modelmgr/requesters/moonshotchatcmpl.py index f3621a09..494b2b0f 100644 --- a/pkg/provider/modelmgr/requesters/moonshotchatcmpl.py +++ b/pkg/provider/modelmgr/requesters/moonshotchatcmpl.py @@ -25,6 +25,7 @@ class MoonshotChatCompletions(chatcmpl.OpenAIChatCompletions): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: self.client.api_key = use_model.token_mgr.get_token() @@ -54,6 +55,6 @@ class MoonshotChatCompletions(chatcmpl.OpenAIChatCompletions): resp = await self._req(args, extra_body=extra_args) # 处理请求结果 - message = await self._make_msg(resp) + message = await self._make_msg(resp, remove_think) return message diff --git a/pkg/provider/modelmgr/requesters/moonshotchatcmpl.yaml b/pkg/provider/modelmgr/requesters/moonshotchatcmpl.yaml index f3ae73c8..e51fdfa5 100644 --- a/pkg/provider/modelmgr/requesters/moonshotchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/moonshotchatcmpl.yaml @@ -14,7 +14,7 @@ spec: zh_Hans: 基础 URL type: string required: true - default: "https://api.moonshot.com/v1" + default: "https://api.moonshot.ai/v1" - name: timeout label: en_US: Timeout @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./moonshotchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/newapi.png b/pkg/provider/modelmgr/requesters/newapi.png new file mode 100644 index 00000000..851556f6 Binary files /dev/null and b/pkg/provider/modelmgr/requesters/newapi.png differ diff --git a/pkg/provider/modelmgr/requesters/newapichatcmpl.py b/pkg/provider/modelmgr/requesters/newapichatcmpl.py new file mode 100644 index 00000000..3c2bd3fb --- /dev/null +++ b/pkg/provider/modelmgr/requesters/newapichatcmpl.py @@ -0,0 +1,17 @@ +from __future__ import annotations + +import typing +import openai + +from . import chatcmpl + + +class NewAPIChatCompletions(chatcmpl.OpenAIChatCompletions): + """New API ChatCompletion API 请求器""" + + client: openai.AsyncClient + + default_config: dict[str, typing.Any] = { + 'base_url': 'http://localhost:3000/v1', + 'timeout': 120, + } diff --git a/pkg/provider/modelmgr/requesters/newapichatcmpl.yaml b/pkg/provider/modelmgr/requesters/newapichatcmpl.yaml new file mode 100644 index 00000000..33573df5 --- /dev/null +++ b/pkg/provider/modelmgr/requesters/newapichatcmpl.yaml @@ -0,0 +1,31 @@ +apiVersion: v1 +kind: LLMAPIRequester +metadata: + name: new-api-chat-completions + label: + en_US: New API + zh_Hans: New API + icon: newapi.png +spec: + config: + - name: base_url + label: + en_US: Base URL + zh_Hans: 基础 URL + type: string + required: true + default: "http://localhost:3000/v1" + - name: timeout + label: + en_US: Timeout + zh_Hans: 超时时间 + type: integer + required: true + default: 120 + support_type: + - llm + - text-embedding +execution: + python: + path: ./newapichatcmpl.py + attr: NewAPIChatCompletions \ No newline at end of file diff --git a/pkg/provider/modelmgr/requesters/ollamachat.py b/pkg/provider/modelmgr/requesters/ollamachat.py index 2ea4bb7d..0a8943c0 100644 --- a/pkg/provider/modelmgr/requesters/ollamachat.py +++ b/pkg/provider/modelmgr/requesters/ollamachat.py @@ -17,7 +17,7 @@ from ....core import entities as core_entities REQUESTER_NAME: str = 'ollama-chat' -class OllamaChatCompletions(requester.LLMAPIRequester): +class OllamaChatCompletions(requester.ProviderAPIRequester): """Ollama平台 ChatCompletion API请求器""" client: ollama.AsyncClient @@ -110,6 +110,7 @@ class OllamaChatCompletions(requester.LLMAPIRequester): messages: typing.List[llm_entities.Message], funcs: typing.List[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message: req_messages: list = [] for m in messages: @@ -126,6 +127,19 @@ class OllamaChatCompletions(requester.LLMAPIRequester): use_model=model, use_funcs=funcs, extra_args=extra_args, + remove_think=remove_think, ) except asyncio.TimeoutError: raise errors.RequesterError('请求超时') + + async def invoke_embedding( + self, + model: requester.RuntimeEmbeddingModel, + input_text: list[str], + extra_args: dict[str, typing.Any] = {}, + ) -> list[list[float]]: + return await self.client.embed( + model=model.model_entity.name, + input=input_text, + **extra_args, + ) diff --git a/pkg/provider/modelmgr/requesters/ollamachat.yaml b/pkg/provider/modelmgr/requesters/ollamachat.yaml index 01435775..f7cdeeba 100644 --- a/pkg/provider/modelmgr/requesters/ollamachat.yaml +++ b/pkg/provider/modelmgr/requesters/ollamachat.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./ollamachat.py diff --git a/pkg/provider/modelmgr/requesters/openrouterchatcmpl.yaml b/pkg/provider/modelmgr/requesters/openrouterchatcmpl.yaml index 2ecee6cc..8c957dba 100644 --- a/pkg/provider/modelmgr/requesters/openrouterchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/openrouterchatcmpl.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./openrouterchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/ppiochatcmpl.py b/pkg/provider/modelmgr/requesters/ppiochatcmpl.py index 46da6e01..ca49df10 100644 --- a/pkg/provider/modelmgr/requesters/ppiochatcmpl.py +++ b/pkg/provider/modelmgr/requesters/ppiochatcmpl.py @@ -27,7 +27,7 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): async def _make_msg( self, chat_completion: chat_completion.ChatCompletion, - pipeline_config: dict[str, typing.Any] = {'trigger': {'misc': {'remove_think': False}}}, + remove_think: bool, ) -> llm_entities.Message: chatcmpl_message = chat_completion.choices[0].message.model_dump() # print(chatcmpl_message.keys(), chatcmpl_message.values()) @@ -39,7 +39,7 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): reasoning_content = chatcmpl_message['reasoning_content'] if 'reasoning_content' in chatcmpl_message else None # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: chatcmpl_message['content'] = re.sub( r'.*?', '', chatcmpl_message['content'], flags=re.DOTALL ) @@ -55,7 +55,7 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): async def _make_msg_chunk( self, - pipeline_config: dict[str, typing.Any], + remove_think: bool, chat_completion: chat_completion.ChatCompletion, idx: int, ) -> llm_entities.MessageChunk: @@ -79,7 +79,7 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): # print(reasoning_content) # deepseek的reasoner模型 - if pipeline_config['trigger'].get('misc', '').get('remove_think'): + if remove_think: if '' in delta['content']: self.is_think = True delta['content'] = '' @@ -105,6 +105,7 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): use_model: requester.RuntimeLLMModel, use_funcs: list[tools_entities.LLMFunction] = None, extra_args: dict[str, typing.Any] = {}, + remove_think: bool = False, ) -> llm_entities.Message | typing.AsyncGenerator[llm_entities.MessageChunk, None]: self.client.api_key = use_model.token_mgr.get_token() @@ -136,10 +137,9 @@ class PPIOChatCompletions(chatcmpl.OpenAIChatCompletions): chunk_idx = 0 self.is_content = False tool_calls_map: dict[str, llm_entities.ToolCall] = {} - pipeline_config = query.pipeline_config async for chunk in self._req_stream(args, extra_body=extra_args): # 处理流式消息 - delta_message = await self._make_msg_chunk(pipeline_config, chunk, chunk_idx) + delta_message = await self._make_msg_chunk(remove_think, chunk, chunk_idx) if delta_message.content: current_content += delta_message.content delta_message.content = current_content diff --git a/pkg/provider/modelmgr/requesters/ppiochatcmpl.yaml b/pkg/provider/modelmgr/requesters/ppiochatcmpl.yaml index 9f201aa9..90a81614 100644 --- a/pkg/provider/modelmgr/requesters/ppiochatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/ppiochatcmpl.yaml @@ -29,6 +29,9 @@ spec: type: int required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./ppiochatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/qhaigc.png b/pkg/provider/modelmgr/requesters/qhaigc.png new file mode 100644 index 00000000..53400713 Binary files /dev/null and b/pkg/provider/modelmgr/requesters/qhaigc.png differ diff --git a/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.py b/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.py new file mode 100644 index 00000000..a68b6896 --- /dev/null +++ b/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.py @@ -0,0 +1,17 @@ +from __future__ import annotations + +import openai +import typing + +from . import chatcmpl + + +class QHAIGCChatCompletions(chatcmpl.OpenAIChatCompletions): + """启航 AI ChatCompletion API 请求器""" + + client: openai.AsyncClient + + default_config: dict[str, typing.Any] = { + 'base_url': 'https://api.qhaigc.com/v1', + 'timeout': 120, + } diff --git a/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.yaml b/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.yaml new file mode 100644 index 00000000..2cd777d0 --- /dev/null +++ b/pkg/provider/modelmgr/requesters/qhaigcchatcmpl.yaml @@ -0,0 +1,38 @@ +apiVersion: v1 +kind: LLMAPIRequester +metadata: + name: qhaigc-chat-completions + label: + en_US: QH AI + zh_Hans: 启航 AI + icon: qhaigc.png +spec: + config: + - name: base_url + label: + en_US: Base URL + zh_Hans: 基础 URL + type: string + required: true + default: "https://api.qhaigc.net/v1" + - name: args + label: + en_US: Args + zh_Hans: 附加参数 + type: object + required: true + default: {} + - name: timeout + label: + en_US: Timeout + zh_Hans: 超时时间 + type: int + required: true + default: 120 + support_type: + - llm + - text-embedding +execution: + python: + path: ./qhaigcchatcmpl.py + attr: QHAIGCChatCompletions \ No newline at end of file diff --git a/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.yaml b/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.yaml index 19b3dcc3..25a20653 100644 --- a/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/siliconflowchatcmpl.yaml @@ -22,6 +22,9 @@ spec: type: integer required: true default: 120 + support_type: + - llm + - text-embedding execution: python: path: ./siliconflowchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/volcarkchatcmpl.yaml b/pkg/provider/modelmgr/requesters/volcarkchatcmpl.yaml index 402f04e7..c711ef2d 100644 --- a/pkg/provider/modelmgr/requesters/volcarkchatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/volcarkchatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./volcarkchatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/xaichatcmpl.yaml b/pkg/provider/modelmgr/requesters/xaichatcmpl.yaml index 29db4eb3..2769a402 100644 --- a/pkg/provider/modelmgr/requesters/xaichatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/xaichatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./xaichatcmpl.py diff --git a/pkg/provider/modelmgr/requesters/zhipuaichatcmpl.yaml b/pkg/provider/modelmgr/requesters/zhipuaichatcmpl.yaml index a05184ef..34539d95 100644 --- a/pkg/provider/modelmgr/requesters/zhipuaichatcmpl.yaml +++ b/pkg/provider/modelmgr/requesters/zhipuaichatcmpl.yaml @@ -22,6 +22,8 @@ spec: type: integer required: true default: 120 + support_type: + - llm execution: python: path: ./zhipuaichatcmpl.py diff --git a/pkg/provider/runners/dashscopeapi.py b/pkg/provider/runners/dashscopeapi.py index 9bb5824c..7f66e6f0 100644 --- a/pkg/provider/runners/dashscopeapi.py +++ b/pkg/provider/runners/dashscopeapi.py @@ -148,7 +148,7 @@ class DashScopeAPIRunner(runner.RequestRunner): # 将参考资料替换到文本中 pending_content = self._replace_references(pending_content, references_dict) - if idx_chunk % 64 == 0 or is_final: + if idx_chunk % 8 == 0 or is_final: yield llm_entities.MessageChunk( role='assistant', content=pending_content, diff --git a/pkg/provider/runners/difysvapi.py b/pkg/provider/runners/difysvapi.py index 40a3140c..8c1307a5 100644 --- a/pkg/provider/runners/difysvapi.py +++ b/pkg/provider/runners/difysvapi.py @@ -115,7 +115,7 @@ class DifyServiceAPIRunner(runner.RequestRunner): stream_output_pending_chunk = '' - batch_pending_max_size = 64 # 积累一定量的消息更新消息一次 + batch_pending_max_size = 8 # 积累一定量的消息更新消息一次 batch_pending_index = 0 diff --git a/pkg/provider/runners/localagent.py b/pkg/provider/runners/localagent.py index 35a4ca17..918ec42c 100644 --- a/pkg/provider/runners/localagent.py +++ b/pkg/provider/runners/localagent.py @@ -1,13 +1,28 @@ from __future__ import annotations import json +import copy import typing - from .. import runner from ...core import entities as core_entities from .. import entities as llm_entities +rag_combined_prompt_template = """ +The following are relevant context entries retrieved from the knowledge base. +Please use them to answer the user's message. +Respond in the same language as the user's input. + + +{rag_context} + + + +{user_message} + +""" + + @runner.runner_class('local-agent') class LocalAgentRunner(runner.RequestRunner): """本地Agent请求运行器""" @@ -25,13 +40,62 @@ class LocalAgentRunner(runner.RequestRunner): """运行请求""" pending_tool_calls = [] - req_messages = query.prompt.messages.copy() + query.messages.copy() + [query.user_message] + kb_uuid = query.pipeline_config['ai']['local-agent']['knowledge-base'] + + if kb_uuid == '__none__': + kb_uuid = None + + user_message = copy.deepcopy(query.user_message) + + user_message_text = '' + + if isinstance(user_message.content, str): + user_message_text = user_message.content + elif isinstance(user_message.content, list): + for ce in user_message.content: + if ce.type == 'text': + user_message_text += ce.text + break + + if kb_uuid and user_message_text: + # only support text for now + kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid) + + if not kb: + self.ap.logger.warning(f'Knowledge base {kb_uuid} not found') + raise ValueError(f'Knowledge base {kb_uuid} not found') + + result = await kb.retrieve(user_message_text) + + final_user_message_text = '' + + if result: + rag_context = '\n\n'.join( + f'[{i + 1}] {entry.metadata.get("text", "")}' for i, entry in enumerate(result) + ) + final_user_message_text = rag_combined_prompt_template.format( + rag_context=rag_context, user_message=user_message_text + ) + + else: + final_user_message_text = user_message_text + + self.ap.logger.debug(f'Final user message text: {final_user_message_text}') + + for ce in user_message.content: + if ce.type == 'text': + ce.text = final_user_message_text + break + + req_messages = query.prompt.messages.copy() + query.messages.copy() + [user_message] + try: is_stream = await query.adapter.is_stream_output_supported() - except AttributeError: is_stream = False + remove_think = self.pipeline_config['output'].get('misc', '').get('remove-think') + if not is_stream: # 非流式输出,直接请求 @@ -41,6 +105,7 @@ class LocalAgentRunner(runner.RequestRunner): req_messages, query.use_funcs, extra_args=query.use_llm_model.model_entity.extra_args, + remove_think=remove_think, ) yield msg final_msg = msg @@ -54,6 +119,7 @@ class LocalAgentRunner(runner.RequestRunner): req_messages, query.use_funcs, extra_args=query.use_llm_model.model_entity.extra_args, + remove_think=remove_think, ): msg_idx = msg_idx + 1 if msg_idx % 8 == 0 or msg.is_final: @@ -116,6 +182,7 @@ class LocalAgentRunner(runner.RequestRunner): req_messages, query.use_funcs, extra_args=query.use_llm_model.model_entity.extra_args, + remove_think=remove_think, ): yield msg if msg.tool_calls: @@ -144,6 +211,7 @@ class LocalAgentRunner(runner.RequestRunner): req_messages, query.use_funcs, extra_args=query.use_llm_model.model_entity.extra_args, + remove_think=remove_think, ) yield msg diff --git a/pkg/rag/knowledge/kbmgr.py b/pkg/rag/knowledge/kbmgr.py new file mode 100644 index 00000000..a9e7e57a --- /dev/null +++ b/pkg/rag/knowledge/kbmgr.py @@ -0,0 +1,212 @@ +from __future__ import annotations +import traceback +import uuid +from .services import parser, chunker +from pkg.core import app +from pkg.rag.knowledge.services.embedder import Embedder +from pkg.rag.knowledge.services.retriever import Retriever +import sqlalchemy +from ...entity.persistence import rag as persistence_rag +from pkg.core import taskmgr +from ...entity.rag import retriever as retriever_entities + + +class RuntimeKnowledgeBase: + ap: app.Application + + knowledge_base_entity: persistence_rag.KnowledgeBase + + parser: parser.FileParser + + chunker: chunker.Chunker + + embedder: Embedder + + retriever: Retriever + + def __init__(self, ap: app.Application, knowledge_base_entity: persistence_rag.KnowledgeBase): + self.ap = ap + self.knowledge_base_entity = knowledge_base_entity + self.parser = parser.FileParser(ap=self.ap) + self.chunker = chunker.Chunker(ap=self.ap) + self.embedder = Embedder(ap=self.ap) + self.retriever = Retriever(ap=self.ap) + # 传递kb_id给retriever + self.retriever.kb_id = knowledge_base_entity.uuid + + async def initialize(self): + pass + + async def _store_file_task(self, file: persistence_rag.File, task_context: taskmgr.TaskContext): + try: + # set file status to processing + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_rag.File) + .where(persistence_rag.File.uuid == file.uuid) + .values(status='processing') + ) + + task_context.set_current_action('Parsing file') + # parse file + text = await self.parser.parse(file.file_name, file.extension) + if not text: + raise Exception(f'No text extracted from file {file.file_name}') + + task_context.set_current_action('Chunking file') + # chunk file + chunks_texts = await self.chunker.chunk(text) + if not chunks_texts: + raise Exception(f'No chunks extracted from file {file.file_name}') + + task_context.set_current_action('Embedding chunks') + + embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid( + self.knowledge_base_entity.embedding_model_uuid + ) + # embed chunks + await self.embedder.embed_and_store( + kb_id=self.knowledge_base_entity.uuid, + file_id=file.uuid, + chunks=chunks_texts, + embedding_model=embedding_model, + ) + + # set file status to completed + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_rag.File) + .where(persistence_rag.File.uuid == file.uuid) + .values(status='completed') + ) + + except Exception as e: + self.ap.logger.error(f'Error storing file {file.uuid}: {e}') + traceback.print_exc() + # set file status to failed + await self.ap.persistence_mgr.execute_async( + sqlalchemy.update(persistence_rag.File) + .where(persistence_rag.File.uuid == file.uuid) + .values(status='failed') + ) + + raise + + async def store_file(self, file_id: str) -> str: + # pre checking + if not await self.ap.storage_mgr.storage_provider.exists(file_id): + raise Exception(f'File {file_id} not found') + + file_uuid = str(uuid.uuid4()) + kb_id = self.knowledge_base_entity.uuid + file_name = file_id + extension = file_name.split('.')[-1] + + file_obj_data = { + 'uuid': file_uuid, + 'kb_id': kb_id, + 'file_name': file_name, + 'extension': extension, + 'status': 'pending', + } + + file_obj = persistence_rag.File(**file_obj_data) + + await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.File).values(file_obj_data)) + + # run background task asynchronously + ctx = taskmgr.TaskContext.new() + wrapper = self.ap.task_mgr.create_user_task( + self._store_file_task(file_obj, task_context=ctx), + kind='knowledge-operation', + name=f'knowledge-store-file-{file_id}', + label=f'Store file {file_id}', + context=ctx, + ) + return wrapper.id + + async def retrieve(self, query: str) -> list[retriever_entities.RetrieveResultEntry]: + embedding_model = await self.ap.model_mgr.get_embedding_model_by_uuid( + self.knowledge_base_entity.embedding_model_uuid + ) + return await self.retriever.retrieve(self.knowledge_base_entity.uuid, query, embedding_model) + + async def delete_file(self, file_id: str): + # delete vector + await self.ap.vector_db_mgr.vector_db.delete_by_file_id(self.knowledge_base_entity.uuid, file_id) + + # delete chunk + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_rag.Chunk).where(persistence_rag.Chunk.file_id == file_id) + ) + + await self.ap.persistence_mgr.execute_async( + sqlalchemy.delete(persistence_rag.File).where(persistence_rag.File.uuid == file_id) + ) + + async def dispose(self): + await self.ap.vector_db_mgr.vector_db.delete_collection(self.knowledge_base_entity.uuid) + + +class RAGManager: + ap: app.Application + + knowledge_bases: list[RuntimeKnowledgeBase] + + def __init__(self, ap: app.Application): + self.ap = ap + self.knowledge_bases = [] + + async def initialize(self): + await self.load_knowledge_bases_from_db() + + async def load_knowledge_bases_from_db(self): + self.ap.logger.info('Loading knowledge bases from db...') + + self.knowledge_bases = [] + + result = await self.ap.persistence_mgr.execute_async(sqlalchemy.select(persistence_rag.KnowledgeBase)) + + knowledge_bases = result.all() + + for knowledge_base in knowledge_bases: + try: + await self.load_knowledge_base(knowledge_base) + except Exception as e: + self.ap.logger.error( + f'Error loading knowledge base {knowledge_base.uuid}: {e}\n{traceback.format_exc()}' + ) + + async def load_knowledge_base( + self, + knowledge_base_entity: persistence_rag.KnowledgeBase | sqlalchemy.Row | dict, + ) -> RuntimeKnowledgeBase: + if isinstance(knowledge_base_entity, sqlalchemy.Row): + knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity._mapping) + elif isinstance(knowledge_base_entity, dict): + knowledge_base_entity = persistence_rag.KnowledgeBase(**knowledge_base_entity) + + runtime_knowledge_base = RuntimeKnowledgeBase(ap=self.ap, knowledge_base_entity=knowledge_base_entity) + + await runtime_knowledge_base.initialize() + + self.knowledge_bases.append(runtime_knowledge_base) + + return runtime_knowledge_base + + async def get_knowledge_base_by_uuid(self, kb_uuid: str) -> RuntimeKnowledgeBase | None: + for kb in self.knowledge_bases: + if kb.knowledge_base_entity.uuid == kb_uuid: + return kb + return None + + async def remove_knowledge_base_from_runtime(self, kb_uuid: str): + for kb in self.knowledge_bases: + if kb.knowledge_base_entity.uuid == kb_uuid: + self.knowledge_bases.remove(kb) + return + + async def delete_knowledge_base(self, kb_uuid: str): + for kb in self.knowledge_bases: + if kb.knowledge_base_entity.uuid == kb_uuid: + await kb.dispose() + self.knowledge_bases.remove(kb) + return diff --git a/pkg/rag/knowledge/services/__init__.py b/pkg/rag/knowledge/services/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/pkg/rag/knowledge/services/base_service.py b/pkg/rag/knowledge/services/base_service.py new file mode 100644 index 00000000..0f71a508 --- /dev/null +++ b/pkg/rag/knowledge/services/base_service.py @@ -0,0 +1,15 @@ +# 封装异步操作 +import asyncio + + +class BaseService: + def __init__(self): + pass + + async def _run_sync(self, func, *args, **kwargs): + """ + 在单独的线程中运行同步函数。 + 如果第一个参数是 session,则在 to_thread 中获取新的 session。 + """ + + return await asyncio.to_thread(func, *args, **kwargs) diff --git a/pkg/rag/knowledge/services/chunker.py b/pkg/rag/knowledge/services/chunker.py new file mode 100644 index 00000000..f169d5f1 --- /dev/null +++ b/pkg/rag/knowledge/services/chunker.py @@ -0,0 +1,63 @@ +from __future__ import annotations + +import json +from typing import List +from pkg.rag.knowledge.services import base_service +from pkg.core import app + + +class Chunker(base_service.BaseService): + """ + A class for splitting long texts into smaller, overlapping chunks. + """ + + def __init__(self, ap: app.Application, chunk_size: int = 500, chunk_overlap: int = 50): + self.ap = ap + self.chunk_size = chunk_size + self.chunk_overlap = chunk_overlap + if self.chunk_overlap >= self.chunk_size: + self.ap.logger.warning( + 'Chunk overlap is greater than or equal to chunk size. This may lead to empty or malformed chunks.' + ) + + def _split_text_sync(self, text: str) -> List[str]: + """ + Synchronously splits a long text into chunks with specified overlap. + This is a CPU-bound operation, intended to be run in a separate thread. + """ + if not text: + return [] + # words = text.split() + # chunks = [] + # current_chunk = [] + + # for word in words: + # current_chunk.append(word) + # if len(current_chunk) > self.chunk_size: + # chunks.append(" ".join(current_chunk[:self.chunk_size])) + # current_chunk = current_chunk[self.chunk_size - self.chunk_overlap:] + + # if current_chunk: + # chunks.append(" ".join(current_chunk)) + + # A more robust chunking strategy (e.g., using recursive character text splitter) + from langchain.text_splitter import RecursiveCharacterTextSplitter + + text_splitter = RecursiveCharacterTextSplitter( + chunk_size=self.chunk_size, + chunk_overlap=self.chunk_overlap, + length_function=len, + is_separator_regex=False, + ) + return text_splitter.split_text(text) + + async def chunk(self, text: str) -> List[str]: + """ + Asynchronously chunks a given text into smaller pieces. + """ + self.ap.logger.info(f'Chunking text (length: {len(text)})...') + # Run the synchronous splitting logic in a separate thread + chunks = await self._run_sync(self._split_text_sync, text) + self.ap.logger.info(f'Text chunked into {len(chunks)} pieces.') + self.ap.logger.debug(f'Chunks: {json.dumps(chunks, indent=4, ensure_ascii=False)}') + return chunks diff --git a/pkg/rag/knowledge/services/embedder.py b/pkg/rag/knowledge/services/embedder.py new file mode 100644 index 00000000..a0ae3d49 --- /dev/null +++ b/pkg/rag/knowledge/services/embedder.py @@ -0,0 +1,47 @@ +from __future__ import annotations +import uuid +from typing import List +from pkg.rag.knowledge.services.base_service import BaseService +from ....entity.persistence import rag as persistence_rag +from ....core import app +from ....provider.modelmgr.requester import RuntimeEmbeddingModel +import sqlalchemy + + +class Embedder(BaseService): + def __init__(self, ap: app.Application) -> None: + super().__init__() + self.ap = ap + + async def embed_and_store( + self, kb_id: str, file_id: str, chunks: List[str], embedding_model: RuntimeEmbeddingModel + ) -> list[persistence_rag.Chunk]: + # save chunk to db + chunk_entities: list[persistence_rag.Chunk] = [] + chunk_ids: list[str] = [] + + for chunk_text in chunks: + chunk_uuid = str(uuid.uuid4()) + chunk_ids.append(chunk_uuid) + chunk_entity = persistence_rag.Chunk(uuid=chunk_uuid, file_id=file_id, text=chunk_text) + chunk_entities.append(chunk_entity) + + chunk_dicts = [ + self.ap.persistence_mgr.serialize_model(persistence_rag.Chunk, chunk) for chunk in chunk_entities + ] + + await self.ap.persistence_mgr.execute_async(sqlalchemy.insert(persistence_rag.Chunk).values(chunk_dicts)) + + # get embeddings + embeddings_list: list[list[float]] = await embedding_model.requester.invoke_embedding( + model=embedding_model, + input_text=chunks, + extra_args={}, # TODO: add extra args + ) + + # save embeddings to vdb + await self.ap.vector_db_mgr.vector_db.add_embeddings(kb_id, chunk_ids, embeddings_list, chunk_dicts) + + self.ap.logger.info(f'Successfully saved {len(chunk_entities)} embeddings to Knowledge Base.') + + return chunk_entities diff --git a/pkg/rag/knowledge/services/parser.py b/pkg/rag/knowledge/services/parser.py new file mode 100644 index 00000000..004dbdaa --- /dev/null +++ b/pkg/rag/knowledge/services/parser.py @@ -0,0 +1,291 @@ +from __future__ import annotations + +import PyPDF2 +import io +from docx import Document +import chardet +from typing import Union, Callable, Any +import markdown +from bs4 import BeautifulSoup +import re +import asyncio # Import asyncio for async operations +from pkg.core import app + + +class FileParser: + """ + A robust file parser class to extract text content from various document formats. + It supports TXT, PDF, DOCX, XLSX, CSV, Markdown, HTML, and EPUB files. + All core file reading operations are designed to be run synchronously in a thread pool + to avoid blocking the asyncio event loop. + """ + + def __init__(self, ap: app.Application): + self.ap = ap + + async def _run_sync(self, sync_func: Callable, *args: Any, **kwargs: Any) -> Any: + """ + Runs a synchronous function in a separate thread to prevent blocking the event loop. + This is a general utility method for wrapping blocking I/O operations. + """ + try: + return await asyncio.to_thread(sync_func, *args, **kwargs) + except Exception as e: + self.ap.logger.error(f'Error running synchronous function {sync_func.__name__}: {e}') + raise + + async def parse(self, file_name: str, extension: str) -> Union[str, None]: + """ + Parses the file based on its extension and returns the extracted text content. + This is the main asynchronous entry point for parsing. + + Args: + file_name (str): The name of the file to be parsed, get from ap.storage_mgr + + Returns: + Union[str, None]: The extracted text content as a single string, or None if parsing fails. + """ + + file_extension = extension.lower() + parser_method = getattr(self, f'_parse_{file_extension}', None) + + if parser_method is None: + self.ap.logger.error(f'Unsupported file format: {file_extension} for file {file_name}') + return None + + try: + # Pass file_path to the specific parser methods + return await parser_method(file_name) + except Exception as e: + self.ap.logger.error(f'Failed to parse {file_extension} file {file_name}: {e}') + return None + + # --- Helper for reading files with encoding detection --- + async def _read_file_content(self, file_name: str) -> Union[str, bytes]: + """ + Reads a file with automatic encoding detection, ensuring the synchronous + file read operation runs in a separate thread. + """ + + # def _read_sync(): + # with open(file_path, 'rb') as file: + # raw_data = file.read() + # detected = chardet.detect(raw_data) + # encoding = detected['encoding'] or 'utf-8' + + # if mode == 'r': + # return raw_data.decode(encoding, errors='ignore') + # return raw_data # For binary mode + + # return await self._run_sync(_read_sync) + file_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + detected = chardet.detect(file_bytes) + encoding = detected['encoding'] or 'utf-8' + + return file_bytes.decode(encoding, errors='ignore') + + # --- Specific Parser Methods --- + + async def _parse_txt(self, file_name: str) -> str: + """Parses a TXT file and returns its content.""" + self.ap.logger.info(f'Parsing TXT file: {file_name}') + return await self._read_file_content(file_name) + + async def _parse_pdf(self, file_name: str) -> str: + """Parses a PDF file and returns its text content.""" + self.ap.logger.info(f'Parsing PDF file: {file_name}') + + # def _parse_pdf_sync(): + # text_content = [] + # with open(file_name, 'rb') as file: + # pdf_reader = PyPDF2.PdfReader(file) + # for page in pdf_reader.pages: + # text = page.extract_text() + # if text: + # text_content.append(text) + # return '\n'.join(text_content) + + # return await self._run_sync(_parse_pdf_sync) + + pdf_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + def _parse_pdf_sync(): + pdf_reader = PyPDF2.PdfReader(io.BytesIO(pdf_bytes)) + text_content = [] + for page in pdf_reader.pages: + text = page.extract_text() + if text: + text_content.append(text) + return '\n'.join(text_content) + + return await self._run_sync(_parse_pdf_sync) + + async def _parse_docx(self, file_name: str) -> str: + """Parses a DOCX file and returns its text content.""" + self.ap.logger.info(f'Parsing DOCX file: {file_name}') + + docx_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + def _parse_docx_sync(): + doc = Document(io.BytesIO(docx_bytes)) + text_content = [paragraph.text for paragraph in doc.paragraphs if paragraph.text.strip()] + return '\n'.join(text_content) + + return await self._run_sync(_parse_docx_sync) + + async def _parse_doc(self, file_name: str) -> str: + """Handles .doc files, explicitly stating lack of direct support.""" + self.ap.logger.warning(f'Direct .doc parsing is not supported for {file_name}. Please convert to .docx first.') + raise NotImplementedError('Direct .doc parsing not supported. Please convert to .docx first.') + + # async def _parse_xlsx(self, file_name: str) -> str: + # """Parses an XLSX file, returning text from all sheets.""" + # self.ap.logger.info(f'Parsing XLSX file: {file_name}') + + # xlsx_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + # def _parse_xlsx_sync(): + # excel_file = pd.ExcelFile(io.BytesIO(xlsx_bytes)) + # all_sheet_content = [] + # for sheet_name in excel_file.sheet_names: + # df = pd.read_excel(io.BytesIO(xlsx_bytes), sheet_name=sheet_name) + # sheet_text = f'--- Sheet: {sheet_name} ---\n{df.to_string(index=False)}\n' + # all_sheet_content.append(sheet_text) + # return '\n'.join(all_sheet_content) + + # return await self._run_sync(_parse_xlsx_sync) + + # async def _parse_csv(self, file_name: str) -> str: + # """Parses a CSV file and returns its content as a string.""" + # self.ap.logger.info(f'Parsing CSV file: {file_name}') + + # csv_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + # def _parse_csv_sync(): + # # pd.read_csv can often detect encoding, but explicit detection is safer + # # raw_data = self._read_file_content( + # # file_name, mode='rb' + # # ) # Note: this will need to be await outside this sync function + # # _ = raw_data + # # For simplicity, we'll let pandas handle encoding internally after a raw read. + # # A more robust solution might pass encoding directly to pd.read_csv after detection. + # detected = chardet.detect(io.BytesIO(csv_bytes)) + # encoding = detected['encoding'] or 'utf-8' + # df = pd.read_csv(io.BytesIO(csv_bytes), encoding=encoding) + # return df.to_string(index=False) + + # return await self._run_sync(_parse_csv_sync) + + async def _parse_md(self, file_name: str) -> str: + """Parses a Markdown file, converting it to structured plain text.""" + self.ap.logger.info(f'Parsing Markdown file: {file_name}') + + md_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + def _parse_markdown_sync(): + md_content = io.BytesIO(md_bytes).read().decode('utf-8', errors='ignore') + html_content = markdown.markdown( + md_content, extensions=['extra', 'codehilite', 'tables', 'toc', 'fenced_code'] + ) + soup = BeautifulSoup(html_content, 'html.parser') + text_parts = [] + for element in soup.children: + if element.name in ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']: + level = int(element.name[1]) + text_parts.append('#' * level + ' ' + element.get_text().strip()) + elif element.name == 'p': + text = element.get_text().strip() + if text: + text_parts.append(text) + elif element.name in ['ul', 'ol']: + for li in element.find_all('li'): + text_parts.append(f'* {li.get_text().strip()}') + elif element.name == 'pre': + code_block = element.get_text().strip() + if code_block: + text_parts.append(f'```\n{code_block}\n```') + elif element.name == 'table': + table_str = self._extract_table_to_markdown_sync(element) # Call sync helper + if table_str: + text_parts.append(table_str) + elif element.name: + text = element.get_text(separator=' ', strip=True) + if text: + text_parts.append(text) + cleaned_text = re.sub(r'\n\s*\n', '\n\n', '\n'.join(text_parts)) + return cleaned_text.strip() + + return await self._run_sync(_parse_markdown_sync) + + async def _parse_html(self, file_name: str) -> str: + """Parses an HTML file, extracting structured plain text.""" + self.ap.logger.info(f'Parsing HTML file: {file_name}') + + html_bytes = await self.ap.storage_mgr.storage_provider.load(file_name) + + def _parse_html_sync(): + html_content = io.BytesIO(html_bytes).read().decode('utf-8', errors='ignore') + soup = BeautifulSoup(html_content, 'html.parser') + for script_or_style in soup(['script', 'style']): + script_or_style.decompose() + text_parts = [] + for element in soup.body.children if soup.body else soup.children: + if element.name in ['h1', 'h2', 'h3', 'h4', 'h5', 'h6']: + level = int(element.name[1]) + text_parts.append('#' * level + ' ' + element.get_text().strip()) + elif element.name == 'p': + text = element.get_text().strip() + if text: + text_parts.append(text) + elif element.name in ['ul', 'ol']: + for li in element.find_all('li'): + text = li.get_text().strip() + if text: + text_parts.append(f'* {text}') + elif element.name == 'table': + table_str = self._extract_table_to_markdown_sync(element) # Call sync helper + if table_str: + text_parts.append(table_str) + elif element.name: + text = element.get_text(separator=' ', strip=True) + if text: + text_parts.append(text) + cleaned_text = re.sub(r'\n\s*\n', '\n\n', '\n'.join(text_parts)) + return cleaned_text.strip() + + return await self._run_sync(_parse_html_sync) + + def _add_toc_items_sync(self, toc_list: list, text_content: list, level: int): + """Recursively adds TOC items to text_content (synchronous helper).""" + indent = ' ' * level + for item in toc_list: + if isinstance(item, tuple): + chapter, subchapters = item + text_content.append(f'{indent}- {chapter.title}') + self._add_toc_items_sync(subchapters, text_content, level + 1) + else: + text_content.append(f'{indent}- {item.title}') + + def _extract_table_to_markdown_sync(self, table_element: BeautifulSoup) -> str: + """Helper to convert a BeautifulSoup table element into a Markdown table string (synchronous).""" + headers = [th.get_text().strip() for th in table_element.find_all('th')] + rows = [] + for tr in table_element.find_all('tr'): + cells = [td.get_text().strip() for td in tr.find_all('td')] + if cells: + rows.append(cells) + + if not headers and not rows: + return '' + + table_lines = [] + if headers: + table_lines.append(' | '.join(headers)) + table_lines.append(' | '.join(['---'] * len(headers))) + + for row_cells in rows: + padded_cells = row_cells + [''] * (len(headers) - len(row_cells)) if headers else row_cells + table_lines.append(' | '.join(padded_cells)) + + return '\n'.join(table_lines) diff --git a/pkg/rag/knowledge/services/retriever.py b/pkg/rag/knowledge/services/retriever.py new file mode 100644 index 00000000..73c7edaa --- /dev/null +++ b/pkg/rag/knowledge/services/retriever.py @@ -0,0 +1,48 @@ +from __future__ import annotations + +from . import base_service +from ....core import app +from ....provider.modelmgr.requester import RuntimeEmbeddingModel +from ....entity.rag import retriever as retriever_entities + + +class Retriever(base_service.BaseService): + def __init__(self, ap: app.Application): + super().__init__() + self.ap = ap + + async def retrieve( + self, kb_id: str, query: str, embedding_model: RuntimeEmbeddingModel, k: int = 5 + ) -> list[retriever_entities.RetrieveResultEntry]: + self.ap.logger.info( + f"Retrieving for query: '{query[:10]}' with k={k} using {embedding_model.model_entity.uuid}" + ) + + query_embedding: list[float] = await embedding_model.requester.invoke_embedding( + model=embedding_model, + input_text=[query], + extra_args={}, # TODO: add extra args + ) + + chroma_results = await self.ap.vector_db_mgr.vector_db.search(kb_id, query_embedding[0], k) + + # 'ids' is always returned by ChromaDB, even if not explicitly in 'include' + matched_chroma_ids = chroma_results.get('ids', [[]])[0] + distances = chroma_results.get('distances', [[]])[0] + chroma_metadatas = chroma_results.get('metadatas', [[]])[0] + + if not matched_chroma_ids: + self.ap.logger.info('No relevant chunks found in Chroma.') + return [] + + result: list[retriever_entities.RetrieveResultEntry] = [] + + for i, id in enumerate(matched_chroma_ids): + entry = retriever_entities.RetrieveResultEntry( + id=id, + metadata=chroma_metadatas[i], + distance=distances[i], + ) + result.append(entry) + + return result diff --git a/pkg/utils/announce.py b/pkg/utils/announce.py index 7108a08c..a6b8539a 100644 --- a/pkg/utils/announce.py +++ b/pkg/utils/announce.py @@ -46,7 +46,7 @@ class AnnouncementManager: async def fetch_all(self) -> list[Announcement]: """获取所有公告""" resp = requests.get( - url='https://api.github.com/repos/RockChinQ/LangBot/contents/res/announcement.json', + url='https://api.github.com/repos/langbot-app/LangBot/contents/res/announcement.json', proxies=self.ap.proxy_mgr.get_forward_proxies(), timeout=5, ) diff --git a/pkg/utils/constants.py b/pkg/utils/constants.py index 450a2e16..e13958d9 100644 --- a/pkg/utils/constants.py +++ b/pkg/utils/constants.py @@ -1,7 +1,7 @@ -semantic_version = 'v4.0.7' +semantic_version = 'v4.1.2' -required_database_version = 3 -"""标记本版本所需要的数据库结构版本,用于判断数据库迁移""" +required_database_version = 4 +"""Tag the version of the database schema, used to check if the database needs to be migrated""" debug_mode = False diff --git a/pkg/utils/version.py b/pkg/utils/version.py index ec0683c3..b26b1e33 100644 --- a/pkg/utils/version.py +++ b/pkg/utils/version.py @@ -29,7 +29,7 @@ class VersionManager: async def get_release_list(self) -> list: """获取发行列表""" rls_list_resp = requests.get( - url='https://api.github.com/repos/RockChinQ/LangBot/releases', + url='https://api.github.com/repos/langbot-app/LangBot/releases', proxies=self.ap.proxy_mgr.get_forward_proxies(), timeout=5, ) diff --git a/pkg/vector/__init__.py b/pkg/vector/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/pkg/vector/mgr.py b/pkg/vector/mgr.py new file mode 100644 index 00000000..ea198ac2 --- /dev/null +++ b/pkg/vector/mgr.py @@ -0,0 +1,18 @@ +from __future__ import annotations + +from ..core import app +from .vdb import VectorDatabase +from .vdbs.chroma import ChromaVectorDatabase + + +class VectorDBManager: + ap: app.Application + vector_db: VectorDatabase = None + + def __init__(self, ap: app.Application): + self.ap = ap + + async def initialize(self): + # 初始化 Chroma 向量数据库(可扩展为多种实现) + if self.vector_db is None: + self.vector_db = ChromaVectorDatabase(self.ap) diff --git a/pkg/vector/vdb.py b/pkg/vector/vdb.py new file mode 100644 index 00000000..73a3cc0e --- /dev/null +++ b/pkg/vector/vdb.py @@ -0,0 +1,37 @@ +from __future__ import annotations +import abc +from typing import Any, Dict +import numpy as np + + +class VectorDatabase(abc.ABC): + @abc.abstractmethod + async def add_embeddings( + self, + collection: str, + ids: list[str], + embeddings_list: list[list[float]], + metadatas: list[dict[str, Any]], + documents: list[str], + ) -> None: + """向指定 collection 添加向量数据。""" + pass + + @abc.abstractmethod + async def search(self, collection: str, query_embedding: np.ndarray, k: int = 5) -> Dict[str, Any]: + """在指定 collection 中检索最相似的向量。""" + pass + + @abc.abstractmethod + async def delete_by_file_id(self, collection: str, file_id: str) -> None: + """根据 file_id 删除指定 collection 中的向量。""" + pass + + @abc.abstractmethod + async def get_or_create_collection(self, collection: str): + """获取或创建 collection。""" + pass + + @abc.abstractmethod + async def delete_collection(self, collection: str): + pass diff --git a/pkg/vector/vdbs/__init__.py b/pkg/vector/vdbs/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/pkg/vector/vdbs/chroma.py b/pkg/vector/vdbs/chroma.py new file mode 100644 index 00000000..41ab7d36 --- /dev/null +++ b/pkg/vector/vdbs/chroma.py @@ -0,0 +1,61 @@ +from __future__ import annotations +import asyncio +from typing import Any +from chromadb import PersistentClient +from pkg.vector.vdb import VectorDatabase +from pkg.core import app +import chromadb +import chromadb.errors + + +class ChromaVectorDatabase(VectorDatabase): + def __init__(self, ap: app.Application, base_path: str = './data/chroma'): + self.ap = ap + self.client = PersistentClient(path=base_path) + self._collections = {} + + async def get_or_create_collection(self, collection: str) -> chromadb.Collection: + if collection not in self._collections: + self._collections[collection] = await asyncio.to_thread( + self.client.get_or_create_collection, name=collection + ) + self.ap.logger.info(f"Chroma collection '{collection}' accessed/created.") + return self._collections[collection] + + async def add_embeddings( + self, + collection: str, + ids: list[str], + embeddings_list: list[list[float]], + metadatas: list[dict[str, Any]], + ) -> None: + col = await self.get_or_create_collection(collection) + await asyncio.to_thread(col.add, embeddings=embeddings_list, ids=ids, metadatas=metadatas) + self.ap.logger.info(f"Added {len(ids)} embeddings to Chroma collection '{collection}'.") + + async def search(self, collection: str, query_embedding: list[float], k: int = 5) -> dict[str, Any]: + col = await self.get_or_create_collection(collection) + results = await asyncio.to_thread( + col.query, + query_embeddings=query_embedding, + n_results=k, + include=['metadatas', 'distances', 'documents'], + ) + self.ap.logger.info(f"Chroma search in '{collection}' returned {len(results.get('ids', [[]])[0])} results.") + return results + + async def delete_by_file_id(self, collection: str, file_id: str) -> None: + col = await self.get_or_create_collection(collection) + await asyncio.to_thread(col.delete, where={'file_id': file_id}) + self.ap.logger.info(f"Deleted embeddings from Chroma collection '{collection}' with file_id: {file_id}") + + async def delete_collection(self, collection: str): + if collection in self._collections: + del self._collections[collection] + + try: + await asyncio.to_thread(self.client.delete_collection, name=collection) + except chromadb.errors.NotFoundError: + self.ap.logger.warning(f"Chroma collection '{collection}' not found.") + return + self.ap.logger.info(f"Chroma collection '{collection}' deleted.") diff --git a/pyproject.toml b/pyproject.toml index 5e85bfb0..b12db3e1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "langbot" -version = "4.0.7" +version = "4.1.0" description = "高稳定、支持扩展、多模态 - 大模型原生即时通信机器人平台" readme = "README.md" requires-python = ">=3.10.1" @@ -19,6 +19,7 @@ dependencies = [ "dashscope>=1.23.2", "dingtalk-stream>=0.24.0", "discord-py>=2.5.2", + "pynacl>=1.5.0", # Required for Discord voice support "gewechat-client>=0.1.5", "lark-oapi>=1.4.15", "mcp>=1.8.1", @@ -49,6 +50,16 @@ dependencies = [ "ruff>=0.11.9", "pre-commit>=4.2.0", "uv>=0.7.11", + "PyPDF2>=3.0.1", + "python-docx>=1.1.0", + "pandas>=2.2.2", + "chardet>=5.2.0", + "markdown>=3.6", + "beautifulsoup4>=4.12.3", + "ebooklib>=0.18", + "html2text>=2024.2.26", + "langchain>=0.2.0", + "chromadb>=0.4.24", ] keywords = [ "bot", @@ -79,11 +90,13 @@ classifiers = [ [project.urls] Homepage = "https://langbot.app" Documentation = "https://docs.langbot.app" -Repository = "https://github.com/RockChinQ/langbot" +Repository = "https://github.com/langbot-app/LangBot" [dependency-groups] dev = [ "pre-commit>=4.2.0", + "pytest>=8.4.1", + "pytest-asyncio>=1.0.0", "ruff>=0.11.9", ] diff --git a/templates/config.yaml b/templates/config.yaml index 109cd8d7..d347af77 100644 --- a/templates/config.yaml +++ b/templates/config.yaml @@ -15,6 +15,7 @@ proxy: http: '' https: '' system: + recovery_key: '' jwt: expire: 604800 secret: '' diff --git a/templates/default-pipeline-config.json b/templates/default-pipeline-config.json index 796c6356..855e2ac6 100644 --- a/templates/default-pipeline-config.json +++ b/templates/default-pipeline-config.json @@ -44,7 +44,8 @@ "role": "system", "content": "You are a helpful assistant." } - ] + ], + "knowledge-base": "" }, "dify-service-api": { "base-url": "https://api.dify.ai/v1", @@ -86,7 +87,8 @@ "hide-exception": true, "at-sender": true, "quote-origin": true, - "track-function-calls": false + "track-function-calls": false, + "remove-think": true } } } \ No newline at end of file diff --git a/templates/metadata/pipeline/ai.yaml b/templates/metadata/pipeline/ai.yaml index ca739ce1..63c56a8a 100644 --- a/templates/metadata/pipeline/ai.yaml +++ b/templates/metadata/pipeline/ai.yaml @@ -68,6 +68,16 @@ stages: zh_Hans: 除非您了解消息结构,否则请只使用 system 单提示词 type: prompt-editor required: true + - name: knowledge-base + label: + en_US: Knowledge Base + zh_Hans: 知识库 + description: + en_US: Configure the knowledge base to use for the agent, if not selected, the agent will directly use the LLM to reply + zh_Hans: 配置用于提升回复质量的知识库,若不选择,则直接使用大模型回复 + type: knowledge-base-selector + required: false + default: '' - name: dify-service-api label: en_US: Dify Service API @@ -300,3 +310,4 @@ stages: type: string required: false default: 'response' + diff --git a/templates/metadata/pipeline/output.yaml b/templates/metadata/pipeline/output.yaml index 9fe0cd25..66bb312c 100644 --- a/templates/metadata/pipeline/output.yaml +++ b/templates/metadata/pipeline/output.yaml @@ -105,3 +105,13 @@ stages: type: boolean required: true default: false + - name: remove-think + label: + en_US: Remove CoT + zh_Hans: 删除思维链 + description: + en_US: If enabled, LangBot will remove the LLM thought content in response + zh_Hans: 如果启用,将自动删除大模型回复中的模型思考内容 + type: boolean + required: true + default: true diff --git a/web/.env.example b/web/.env.example new file mode 100644 index 00000000..963ad1c9 --- /dev/null +++ b/web/.env.example @@ -0,0 +1 @@ +NEXT_PUBLIC_API_BASE_URL=http://localhost:5300 diff --git a/web/.gitignore b/web/.gitignore index c77c37c5..9713d3c6 100644 --- a/web/.gitignore +++ b/web/.gitignore @@ -32,6 +32,7 @@ yarn-error.log* # env files (can opt-in for committing if needed) .env* +!.env.example # vercel .vercel diff --git a/web/README.md b/web/README.md index e215bc4c..ac69bf33 100644 --- a/web/README.md +++ b/web/README.md @@ -1,36 +1,3 @@ -This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app). +# Debug LangBot Frontend -## Getting Started - -First, run the development server: - -```bash -npm run dev -# or -yarn dev -# or -pnpm dev -# or -bun dev -``` - -Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. - -You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file. - -This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel. - -## Learn More - -To learn more about Next.js, take a look at the following resources: - -- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. -- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. - -You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome! - -## Deploy on Vercel - -The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. - -Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details. +Please refer to the [Development Guide](https://docs.langbot.app/en/develop/dev-config.html) for more information. diff --git a/web/package-lock.json b/web/package-lock.json index ee9b5767..cb2b05d8 100644 --- a/web/package-lock.json +++ b/web/package-lock.json @@ -12,23 +12,28 @@ "@dnd-kit/sortable": "^10.0.0", "@hookform/resolvers": "^5.0.1", "@radix-ui/react-checkbox": "^1.3.1", - "@radix-ui/react-dialog": "^1.1.13", + "@radix-ui/react-dialog": "^1.1.14", + "@radix-ui/react-dropdown-menu": "^2.1.15", "@radix-ui/react-hover-card": "^1.1.13", "@radix-ui/react-label": "^2.1.6", "@radix-ui/react-popover": "^1.1.14", "@radix-ui/react-scroll-area": "^1.2.9", "@radix-ui/react-select": "^2.2.4", - "@radix-ui/react-slot": "^1.2.2", + "@radix-ui/react-separator": "^1.1.7", + "@radix-ui/react-slot": "^1.2.3", "@radix-ui/react-switch": "^1.2.4", "@radix-ui/react-tabs": "^1.1.11", "@radix-ui/react-toggle": "^1.1.8", "@radix-ui/react-toggle-group": "^1.1.9", + "@radix-ui/react-tooltip": "^1.2.7", "@tailwindcss/postcss": "^4.1.5", + "@tanstack/react-table": "^8.21.3", "axios": "^1.8.4", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "i18next": "^25.1.2", "i18next-browser-languagedetector": "^8.1.0", + "input-otp": "^1.4.2", "lodash": "^4.17.21", "lucide-react": "^0.507.0", "next": "15.2.4", @@ -1037,6 +1042,24 @@ } } }, + "node_modules/@radix-ui/react-collection/node_modules/@radix-ui/react-slot": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.2.tgz", + "integrity": "sha512-y7TBO4xN4Y94FvcWIOIh18fM4R1A8S4q1jhoz4PNzOoHsFcN8pogcFmZrTYAm4F9VRUrWP/Mw7xSKybIeRI+CQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-compose-refs": "1.1.2" + }, + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-compose-refs": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/@radix-ui/react-compose-refs/-/react-compose-refs-1.1.2.tgz", @@ -1068,22 +1091,22 @@ } }, "node_modules/@radix-ui/react-dialog": { - "version": "1.1.13", - "resolved": "https://registry.npmjs.org/@radix-ui/react-dialog/-/react-dialog-1.1.13.tgz", - "integrity": "sha512-ARFmqUyhIVS3+riWzwGTe7JLjqwqgnODBUZdqpWar/z1WFs9z76fuOs/2BOWCR+YboRn4/WN9aoaGVwqNRr8VA==", + "version": "1.1.14", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dialog/-/react-dialog-1.1.14.tgz", + "integrity": "sha512-+CpweKjqpzTmwRwcYECQcNYbI8V9VSQt0SNFKeEBLgfucbsLssU6Ppq7wUdNXEGb573bMjFhVjKVll8rmV6zMw==", "license": "MIT", "dependencies": { "@radix-ui/primitive": "1.1.2", "@radix-ui/react-compose-refs": "1.1.2", "@radix-ui/react-context": "1.1.2", - "@radix-ui/react-dismissable-layer": "1.1.9", + "@radix-ui/react-dismissable-layer": "1.1.10", "@radix-ui/react-focus-guards": "1.1.2", - "@radix-ui/react-focus-scope": "1.1.6", + "@radix-ui/react-focus-scope": "1.1.7", "@radix-ui/react-id": "1.1.1", - "@radix-ui/react-portal": "1.1.8", + "@radix-ui/react-portal": "1.1.9", "@radix-ui/react-presence": "1.1.4", - "@radix-ui/react-primitive": "2.1.2", - "@radix-ui/react-slot": "1.2.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-slot": "1.2.3", "@radix-ui/react-use-controllable-state": "1.2.2", "aria-hidden": "^1.2.4", "react-remove-scroll": "^2.6.3" @@ -1103,6 +1126,80 @@ } } }, + "node_modules/@radix-ui/react-dialog/node_modules/@radix-ui/react-dismissable-layer": { + "version": "1.1.10", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dismissable-layer/-/react-dismissable-layer-1.1.10.tgz", + "integrity": "sha512-IM1zzRV4W3HtVgftdQiiOmA0AdJlCtMLe00FXaHwgt3rAnNsIyDqshvkIW3hj/iu5hu8ERP7KIYki6NkqDxAwQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-escape-keydown": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-dialog/node_modules/@radix-ui/react-portal": { + "version": "1.1.9", + "resolved": "https://registry.npmjs.org/@radix-ui/react-portal/-/react-portal-1.1.9.tgz", + "integrity": "sha512-bpIxvq03if6UNwXZ+HTK71JLh4APvnXntDc6XOX8UVq4XQOVl7lwok0AvIl+b8zgCw3fSaVTZMpAPPagXbKmHQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-layout-effect": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-dialog/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-direction": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/@radix-ui/react-direction/-/react-direction-1.1.1.tgz", @@ -1145,6 +1242,58 @@ } } }, + "node_modules/@radix-ui/react-dropdown-menu": { + "version": "2.1.15", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dropdown-menu/-/react-dropdown-menu-2.1.15.tgz", + "integrity": "sha512-mIBnOjgwo9AH3FyKaSWoSu/dYj6VdhJ7frEPiGTeXCdUFHjl9h3mFh2wwhEtINOmYXWhdpf1rY2minFsmaNgVQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-id": "1.1.1", + "@radix-ui/react-menu": "2.1.15", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-controllable-state": "1.2.2" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-dropdown-menu/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-focus-guards": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-guards/-/react-focus-guards-1.1.2.tgz", @@ -1161,13 +1310,13 @@ } }, "node_modules/@radix-ui/react-focus-scope": { - "version": "1.1.6", - "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-scope/-/react-focus-scope-1.1.6.tgz", - "integrity": "sha512-r9zpYNUQY+2jWHWZGyddQLL9YHkM/XvSFHVcWs7bdVuxMAnCwTAuy6Pf47Z4nw7dYcUou1vg/VgjjrrH03VeBw==", + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-scope/-/react-focus-scope-1.1.7.tgz", + "integrity": "sha512-t2ODlkXBQyn7jkl6TNaw/MtVEVvIGelJDCG41Okq/KwUsJBwQ4XVZsHAVUkK4mBv3ewiAS3PGuUWuY2BoK4ZUw==", "license": "MIT", "dependencies": { "@radix-ui/react-compose-refs": "1.1.2", - "@radix-ui/react-primitive": "2.1.2", + "@radix-ui/react-primitive": "2.1.3", "@radix-ui/react-use-callback-ref": "1.1.1" }, "peerDependencies": { @@ -1185,6 +1334,29 @@ } } }, + "node_modules/@radix-ui/react-focus-scope/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-hover-card": { "version": "1.1.13", "resolved": "https://registry.npmjs.org/@radix-ui/react-hover-card/-/react-hover-card-1.1.13.tgz", @@ -1257,6 +1429,232 @@ } } }, + "node_modules/@radix-ui/react-menu": { + "version": "2.1.15", + "resolved": "https://registry.npmjs.org/@radix-ui/react-menu/-/react-menu-2.1.15.tgz", + "integrity": "sha512-tVlmA3Vb9n8SZSd+YSbuFR66l87Wiy4du+YE+0hzKQEANA+7cWKH1WgqcEX4pXqxUFQKrWQGHdvEfw00TjFiew==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-collection": "1.1.7", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-direction": "1.1.1", + "@radix-ui/react-dismissable-layer": "1.1.10", + "@radix-ui/react-focus-guards": "1.1.2", + "@radix-ui/react-focus-scope": "1.1.7", + "@radix-ui/react-id": "1.1.1", + "@radix-ui/react-popper": "1.2.7", + "@radix-ui/react-portal": "1.1.9", + "@radix-ui/react-presence": "1.1.4", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-roving-focus": "1.1.10", + "@radix-ui/react-slot": "1.2.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "aria-hidden": "^1.2.4", + "react-remove-scroll": "^2.6.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-arrow": { + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-arrow/-/react-arrow-1.1.7.tgz", + "integrity": "sha512-F+M1tLhO+mlQaOWspE8Wstg+z6PwxwRd8oQ8IXceWz92kfAmalTRf0EjrouQeo7QssEPfCn05B4Ihs1K9WQ/7w==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-collection": { + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-collection/-/react-collection-1.1.7.tgz", + "integrity": "sha512-Fh9rGN0MoI4ZFUNyfFVNU4y9LUz93u9/0K+yLgA2bwRojxM8JU1DyvvMBabnZPBgMWREAJvU2jjVzq+LrFUglw==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-dismissable-layer": { + "version": "1.1.10", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dismissable-layer/-/react-dismissable-layer-1.1.10.tgz", + "integrity": "sha512-IM1zzRV4W3HtVgftdQiiOmA0AdJlCtMLe00FXaHwgt3rAnNsIyDqshvkIW3hj/iu5hu8ERP7KIYki6NkqDxAwQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-escape-keydown": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-popper": { + "version": "1.2.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-popper/-/react-popper-1.2.7.tgz", + "integrity": "sha512-IUFAccz1JyKcf/RjB552PlWwxjeCJB8/4KxT7EhBHOJM+mN7LdW+B3kacJXILm32xawcMMjb2i0cIZpo+f9kiQ==", + "license": "MIT", + "dependencies": { + "@floating-ui/react-dom": "^2.0.0", + "@radix-ui/react-arrow": "1.1.7", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-layout-effect": "1.1.1", + "@radix-ui/react-use-rect": "1.1.1", + "@radix-ui/react-use-size": "1.1.1", + "@radix-ui/rect": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-portal": { + "version": "1.1.9", + "resolved": "https://registry.npmjs.org/@radix-ui/react-portal/-/react-portal-1.1.9.tgz", + "integrity": "sha512-bpIxvq03if6UNwXZ+HTK71JLh4APvnXntDc6XOX8UVq4XQOVl7lwok0AvIl+b8zgCw3fSaVTZMpAPPagXbKmHQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-layout-effect": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-menu/node_modules/@radix-ui/react-roving-focus": { + "version": "1.1.10", + "resolved": "https://registry.npmjs.org/@radix-ui/react-roving-focus/-/react-roving-focus-1.1.10.tgz", + "integrity": "sha512-dT9aOXUen9JSsxnMPv/0VqySQf5eDQ6LCk5Sw28kamz8wSOW2bJdlX2Bg5VUIIcV+6XlHpWTIuTPCf/UNIyq8Q==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-collection": "1.1.7", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-direction": "1.1.1", + "@radix-ui/react-id": "1.1.1", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-controllable-state": "1.2.2" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-popover": { "version": "1.1.14", "resolved": "https://registry.npmjs.org/@radix-ui/react-popover/-/react-popover-1.1.14.tgz", @@ -1344,31 +1742,6 @@ } } }, - "node_modules/@radix-ui/react-popover/node_modules/@radix-ui/react-focus-scope": { - "version": "1.1.7", - "resolved": "https://registry.npmjs.org/@radix-ui/react-focus-scope/-/react-focus-scope-1.1.7.tgz", - "integrity": "sha512-t2ODlkXBQyn7jkl6TNaw/MtVEVvIGelJDCG41Okq/KwUsJBwQ4XVZsHAVUkK4mBv3ewiAS3PGuUWuY2BoK4ZUw==", - "license": "MIT", - "dependencies": { - "@radix-ui/react-compose-refs": "1.1.2", - "@radix-ui/react-primitive": "2.1.3", - "@radix-ui/react-use-callback-ref": "1.1.1" - }, - "peerDependencies": { - "@types/react": "*", - "@types/react-dom": "*", - "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", - "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" - }, - "peerDependenciesMeta": { - "@types/react": { - "optional": true - }, - "@types/react-dom": { - "optional": true - } - } - }, "node_modules/@radix-ui/react-popover/node_modules/@radix-ui/react-popper": { "version": "1.2.7", "resolved": "https://registry.npmjs.org/@radix-ui/react-popper/-/react-popper-1.2.7.tgz", @@ -1448,24 +1821,6 @@ } } }, - "node_modules/@radix-ui/react-popover/node_modules/@radix-ui/react-slot": { - "version": "1.2.3", - "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.3.tgz", - "integrity": "sha512-aeNmHnBxbi2St0au6VBVC7JXFlhLlOnvIIlePNniyUNAClzmtAUEY8/pBiK3iHjufOlwA+c20/8jngo7xcrg8A==", - "license": "MIT", - "dependencies": { - "@radix-ui/react-compose-refs": "1.1.2" - }, - "peerDependencies": { - "@types/react": "*", - "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" - }, - "peerDependenciesMeta": { - "@types/react": { - "optional": true - } - } - }, "node_modules/@radix-ui/react-popper": { "version": "1.2.6", "resolved": "https://registry.npmjs.org/@radix-ui/react-popper/-/react-popper-1.2.6.tgz", @@ -1569,6 +1924,24 @@ } } }, + "node_modules/@radix-ui/react-primitive/node_modules/@radix-ui/react-slot": { + "version": "1.2.2", + "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.2.tgz", + "integrity": "sha512-y7TBO4xN4Y94FvcWIOIh18fM4R1A8S4q1jhoz4PNzOoHsFcN8pogcFmZrTYAm4F9VRUrWP/Mw7xSKybIeRI+CQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-compose-refs": "1.1.2" + }, + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-roving-focus": { "version": "1.1.9", "resolved": "https://registry.npmjs.org/@radix-ui/react-roving-focus/-/react-roving-focus-1.1.9.tgz", @@ -1654,24 +2027,6 @@ } } }, - "node_modules/@radix-ui/react-scroll-area/node_modules/@radix-ui/react-slot": { - "version": "1.2.3", - "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.3.tgz", - "integrity": "sha512-aeNmHnBxbi2St0au6VBVC7JXFlhLlOnvIIlePNniyUNAClzmtAUEY8/pBiK3iHjufOlwA+c20/8jngo7xcrg8A==", - "license": "MIT", - "dependencies": { - "@radix-ui/react-compose-refs": "1.1.2" - }, - "peerDependencies": { - "@types/react": "*", - "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" - }, - "peerDependenciesMeta": { - "@types/react": { - "optional": true - } - } - }, "node_modules/@radix-ui/react-select": { "version": "2.2.4", "resolved": "https://registry.npmjs.org/@radix-ui/react-select/-/react-select-2.2.4.tgz", @@ -1715,7 +2070,7 @@ } } }, - "node_modules/@radix-ui/react-slot": { + "node_modules/@radix-ui/react-select/node_modules/@radix-ui/react-slot": { "version": "1.2.2", "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.2.tgz", "integrity": "sha512-y7TBO4xN4Y94FvcWIOIh18fM4R1A8S4q1jhoz4PNzOoHsFcN8pogcFmZrTYAm4F9VRUrWP/Mw7xSKybIeRI+CQ==", @@ -1733,6 +2088,70 @@ } } }, + "node_modules/@radix-ui/react-separator": { + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-separator/-/react-separator-1.1.7.tgz", + "integrity": "sha512-0HEb8R9E8A+jZjvmFCy/J4xhbXy3TV+9XSnGJ3KvTtjlIUy/YQ/p6UYZvi7YbeoeXdyU9+Y3scizK6hkY37baA==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-separator/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-slot": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-slot/-/react-slot-1.2.3.tgz", + "integrity": "sha512-aeNmHnBxbi2St0au6VBVC7JXFlhLlOnvIIlePNniyUNAClzmtAUEY8/pBiK3iHjufOlwA+c20/8jngo7xcrg8A==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-compose-refs": "1.1.2" + }, + "peerDependencies": { + "@types/react": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-switch": { "version": "1.2.4", "resolved": "https://registry.npmjs.org/@radix-ui/react-switch/-/react-switch-1.2.4.tgz", @@ -1846,6 +2265,192 @@ } } }, + "node_modules/@radix-ui/react-tooltip": { + "version": "1.2.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-tooltip/-/react-tooltip-1.2.7.tgz", + "integrity": "sha512-Ap+fNYwKTYJ9pzqW+Xe2HtMRbQ/EeWkj2qykZ6SuEV4iS/o1bZI5ssJbk4D2r8XuDuOBVz/tIx2JObtuqU+5Zw==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-dismissable-layer": "1.1.10", + "@radix-ui/react-id": "1.1.1", + "@radix-ui/react-popper": "1.2.7", + "@radix-ui/react-portal": "1.1.9", + "@radix-ui/react-presence": "1.1.4", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-slot": "1.2.3", + "@radix-ui/react-use-controllable-state": "1.2.2", + "@radix-ui/react-visually-hidden": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-arrow": { + "version": "1.1.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-arrow/-/react-arrow-1.1.7.tgz", + "integrity": "sha512-F+M1tLhO+mlQaOWspE8Wstg+z6PwxwRd8oQ8IXceWz92kfAmalTRf0EjrouQeo7QssEPfCn05B4Ihs1K9WQ/7w==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-dismissable-layer": { + "version": "1.1.10", + "resolved": "https://registry.npmjs.org/@radix-ui/react-dismissable-layer/-/react-dismissable-layer-1.1.10.tgz", + "integrity": "sha512-IM1zzRV4W3HtVgftdQiiOmA0AdJlCtMLe00FXaHwgt3rAnNsIyDqshvkIW3hj/iu5hu8ERP7KIYki6NkqDxAwQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/primitive": "1.1.2", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-escape-keydown": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-popper": { + "version": "1.2.7", + "resolved": "https://registry.npmjs.org/@radix-ui/react-popper/-/react-popper-1.2.7.tgz", + "integrity": "sha512-IUFAccz1JyKcf/RjB552PlWwxjeCJB8/4KxT7EhBHOJM+mN7LdW+B3kacJXILm32xawcMMjb2i0cIZpo+f9kiQ==", + "license": "MIT", + "dependencies": { + "@floating-ui/react-dom": "^2.0.0", + "@radix-ui/react-arrow": "1.1.7", + "@radix-ui/react-compose-refs": "1.1.2", + "@radix-ui/react-context": "1.1.2", + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-callback-ref": "1.1.1", + "@radix-ui/react-use-layout-effect": "1.1.1", + "@radix-ui/react-use-rect": "1.1.1", + "@radix-ui/react-use-size": "1.1.1", + "@radix-ui/rect": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-portal": { + "version": "1.1.9", + "resolved": "https://registry.npmjs.org/@radix-ui/react-portal/-/react-portal-1.1.9.tgz", + "integrity": "sha512-bpIxvq03if6UNwXZ+HTK71JLh4APvnXntDc6XOX8UVq4XQOVl7lwok0AvIl+b8zgCw3fSaVTZMpAPPagXbKmHQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3", + "@radix-ui/react-use-layout-effect": "1.1.1" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-primitive": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-primitive/-/react-primitive-2.1.3.tgz", + "integrity": "sha512-m9gTwRkhy2lvCPe6QJp4d3G1TYEUHn/FzJUtq9MjH46an1wJU+GdoGC5VLof8RX8Ft/DlpshApkhswDLZzHIcQ==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-slot": "1.2.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, + "node_modules/@radix-ui/react-tooltip/node_modules/@radix-ui/react-visually-hidden": { + "version": "1.2.3", + "resolved": "https://registry.npmjs.org/@radix-ui/react-visually-hidden/-/react-visually-hidden-1.2.3.tgz", + "integrity": "sha512-pzJq12tEaaIhqjbzpCuv/OypJY/BPavOofm+dbab+MHLajy277+1lLm6JFcGgF5eskJ6mquGirhXY2GD/8u8Ug==", + "license": "MIT", + "dependencies": { + "@radix-ui/react-primitive": "2.1.3" + }, + "peerDependencies": { + "@types/react": "*", + "@types/react-dom": "*", + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0 || ^19.0.0-rc" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "@types/react-dom": { + "optional": true + } + } + }, "node_modules/@radix-ui/react-use-callback-ref": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/@radix-ui/react-use-callback-ref/-/react-use-callback-ref-1.1.1.tgz", @@ -2295,6 +2900,39 @@ "tailwindcss": "4.1.5" } }, + "node_modules/@tanstack/react-table": { + "version": "8.21.3", + "resolved": "https://registry.npmjs.org/@tanstack/react-table/-/react-table-8.21.3.tgz", + "integrity": "sha512-5nNMTSETP4ykGegmVkhjcS8tTLW6Vl4axfEGQN3v0zdHYbK4UfoqfPChclTrJ4EoK9QynqAu9oUf8VEmrpZ5Ww==", + "license": "MIT", + "dependencies": { + "@tanstack/table-core": "8.21.3" + }, + "engines": { + "node": ">=12" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/tannerlinsley" + }, + "peerDependencies": { + "react": ">=16.8", + "react-dom": ">=16.8" + } + }, + "node_modules/@tanstack/table-core": { + "version": "8.21.3", + "resolved": "https://registry.npmjs.org/@tanstack/table-core/-/table-core-8.21.3.tgz", + "integrity": "sha512-ldZXEhOBb8Is7xLs01fR3YEc3DERiz5silj8tnGkFZytt1abEvl/GhUmCE0PMLaMPTa3Jk4HbKmRlHmu+gCftg==", + "license": "MIT", + "engines": { + "node": ">=12" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/tannerlinsley" + } + }, "node_modules/@tybys/wasm-util": { "version": "0.9.0", "resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.9.0.tgz", @@ -4763,6 +5401,16 @@ "node": ">=0.8.19" } }, + "node_modules/input-otp": { + "version": "1.4.2", + "resolved": "https://registry.npmjs.org/input-otp/-/input-otp-1.4.2.tgz", + "integrity": "sha512-l3jWwYNvrEa6NTCt7BECfCm48GvwuZzkoeG3gBL2w4CHeOXW3eKFmf9UNYkNfYc3mxMrthMnxjIE07MT0zLBQA==", + "license": "MIT", + "peerDependencies": { + "react": "^16.8 || ^17.0 || ^18.0 || ^19.0.0 || ^19.0.0-rc", + "react-dom": "^16.8 || ^17.0 || ^18.0 || ^19.0.0 || ^19.0.0-rc" + } + }, "node_modules/internal-slot": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/internal-slot/-/internal-slot-1.1.0.tgz", diff --git a/web/package.json b/web/package.json index 17516ac4..255ae452 100644 --- a/web/package.json +++ b/web/package.json @@ -4,7 +4,6 @@ "private": true, "scripts": { "dev": "next dev --turbopack", - "dev:local": "NEXT_PUBLIC_API_BASE_URL=http://localhost:5300 next dev --turbopack", "build": "next build", "start": "next start", "lint": "next lint", @@ -16,12 +15,16 @@ "prettier --write" ] }, + "overrides": { + "@radix-ui/react-focus-scope": "1.1.7" + }, "dependencies": { "@dnd-kit/core": "^6.3.1", "@dnd-kit/sortable": "^10.0.0", "@hookform/resolvers": "^5.0.1", "@radix-ui/react-checkbox": "^1.3.1", "@radix-ui/react-dialog": "^1.1.14", + "@radix-ui/react-dropdown-menu": "^2.1.15", "@radix-ui/react-hover-card": "^1.1.13", "@radix-ui/react-label": "^2.1.6", "@radix-ui/react-popover": "^1.1.14", @@ -35,11 +38,13 @@ "@radix-ui/react-toggle-group": "^1.1.9", "@radix-ui/react-tooltip": "^1.2.7", "@tailwindcss/postcss": "^4.1.5", + "@tanstack/react-table": "^8.21.3", "axios": "^1.8.4", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "i18next": "^25.1.2", "i18next-browser-languagedetector": "^8.1.0", + "input-otp": "^1.4.2", "lodash": "^4.17.21", "lucide-react": "^0.507.0", "next": "15.2.4", diff --git a/web/src/app/home/bots/BotDetailDialog.tsx b/web/src/app/home/bots/BotDetailDialog.tsx index 1c4a2403..db19e1d4 100644 --- a/web/src/app/home/bots/BotDetailDialog.tsx +++ b/web/src/app/home/bots/BotDetailDialog.tsx @@ -127,10 +127,8 @@ export default function BotDetailDialog({ @@ -199,10 +197,8 @@ export default function BotDetailDialog({ )} {activeMenu === 'logs' && botId && ( diff --git a/web/src/app/home/bots/components/bot-form/BotForm.tsx b/web/src/app/home/bots/components/bot-form/BotForm.tsx index 40a902c2..bd757ae0 100644 --- a/web/src/app/home/bots/components/bot-form/BotForm.tsx +++ b/web/src/app/home/bots/components/bot-form/BotForm.tsx @@ -64,17 +64,13 @@ const getFormSchema = (t: (key: string) => string) => export default function BotForm({ initBotId, onFormSubmit, - onFormCancel, onBotDeleted, onNewBotCreated, - hideButtons = false, }: { initBotId?: string; onFormSubmit: (value: z.infer>) => void; - onFormCancel: () => void; onBotDeleted: () => void; onNewBotCreated: (botId: string) => void; - hideButtons?: boolean; }) { const { t } = useTranslation(); const formSchema = getFormSchema(t); @@ -214,6 +210,7 @@ export default function BotForm({ }); setAdapterNameToDynamicConfigMap(adapterNameToDynamicConfigMap); } + async function getBotConfig( botId: string, ): Promise> { @@ -527,45 +524,6 @@ export default function BotForm({ )} - - {!hideButtons && ( -
-
- {!initBotId && ( - - )} - {initBotId && ( - <> - - - - )} - -
-
- )} diff --git a/web/src/app/home/bots/page.tsx b/web/src/app/home/bots/page.tsx index d4305898..ad130fae 100644 --- a/web/src/app/home/bots/page.tsx +++ b/web/src/app/home/bots/page.tsx @@ -92,7 +92,7 @@ export default function BotConfigPage() { } return ( -
+
; }) { const [llmModels, setLlmModels] = useState([]); + const [knowledgeBases, setKnowledgeBases] = useState([]); const { t } = useTranslation(); useEffect(() => { @@ -50,6 +52,19 @@ export default function DynamicFormItemComponent({ } }, [config.type]); + useEffect(() => { + if (config.type === DynamicFormItemType.KNOWLEDGE_BASE_SELECTOR) { + httpClient + .getKnowledgeBases() + .then((resp) => { + setKnowledgeBases(resp.bases); + }) + .catch((err) => { + toast.error('获取知识库列表失败:' + err.message); + }); + } + }, [config.type]); + switch (config.type) { case DynamicFormItemType.INT: case DynamicFormItemType.FLOAT: @@ -249,6 +264,25 @@ export default function DynamicFormItemComponent({ ); + case DynamicFormItemType.KNOWLEDGE_BASE_SELECTOR: + return ( + + ); + case DynamicFormItemType.PROMPT_EDITOR: return (
diff --git a/web/src/app/home/components/home-sidebar/HomeSidebar.tsx b/web/src/app/home/components/home-sidebar/HomeSidebar.tsx index 876baf33..fa435cf7 100644 --- a/web/src/app/home/components/home-sidebar/HomeSidebar.tsx +++ b/web/src/app/home/components/home-sidebar/HomeSidebar.tsx @@ -144,6 +144,11 @@ export default function HomeSidebar({ 'https://docs.langbot.app/zh/insight/guide.html', '_blank', ); + } else if (language === 'zh-Hant') { + window.open( + 'https://docs.langbot.app/zh/insight/guide.html', + '_blank', + ); } else { window.open( 'https://docs.langbot.app/en/insight/guide.html', diff --git a/web/src/app/home/components/home-sidebar/sidbarConfigList.tsx b/web/src/app/home/components/home-sidebar/sidbarConfigList.tsx index e21317d6..b3edb98a 100644 --- a/web/src/app/home/components/home-sidebar/sidbarConfigList.tsx +++ b/web/src/app/home/components/home-sidebar/sidbarConfigList.tsx @@ -47,6 +47,7 @@ export const sidebarConfigList = [ zh_Hans: 'https://docs.langbot.app/zh/deploy/models/readme.html', }, }), + new SidebarChildVO({ id: 'pipelines', name: t('pipelines.title'), @@ -67,6 +68,25 @@ export const sidebarConfigList = [ zh_Hans: 'https://docs.langbot.app/zh/deploy/pipelines/readme.html', }, }), + new SidebarChildVO({ + id: 'knowledge', + name: t('knowledge.title'), + icon: ( + + + + ), + route: '/home/knowledge', + description: t('knowledge.description'), + helpLink: { + en_US: 'https://docs.langbot.app/en/deploy/knowledge/readme.html', + zh_Hans: 'https://docs.langbot.app/zh/deploy/knowledge/readme.html', + }, + }), new SidebarChildVO({ id: 'plugins', name: t('plugins.title'), diff --git a/web/src/app/home/knowledge/KBDetailDialog.tsx b/web/src/app/home/knowledge/KBDetailDialog.tsx new file mode 100644 index 00000000..7ad8d4a4 --- /dev/null +++ b/web/src/app/home/knowledge/KBDetailDialog.tsx @@ -0,0 +1,253 @@ +'use client'; + +import { useEffect, useState } from 'react'; +import { + Dialog, + DialogContent, + DialogHeader, + DialogTitle, + DialogFooter, +} from '@/components/ui/dialog'; +import { + Sidebar, + SidebarContent, + SidebarGroup, + SidebarGroupContent, + SidebarMenu, + SidebarMenuButton, + SidebarMenuItem, + SidebarProvider, +} from '@/components/ui/sidebar'; +import { Button } from '@/components/ui/button'; +import { useTranslation } from 'react-i18next'; +import { httpClient } from '@/app/infra/http/HttpClient'; +// import { KnowledgeBase } from '@/app/infra/entities/api'; +import KBForm from '@/app/home/knowledge/components/kb-form/KBForm'; +import KBDoc from '@/app/home/knowledge/components/kb-docs/KBDoc'; +import KBRetrieve from '@/app/home/knowledge/components/kb-retrieve/KBRetrieve'; + +interface KBDetailDialogProps { + open: boolean; + onOpenChange: (open: boolean) => void; + kbId?: string; + onFormCancel: () => void; + onKbDeleted: () => void; + onNewKbCreated: (kbId: string) => void; + onKbUpdated: (kbId: string) => void; +} + +export default function KBDetailDialog({ + open, + onOpenChange, + kbId: propKbId, + onFormCancel, + onKbDeleted, + onNewKbCreated, + onKbUpdated, +}: KBDetailDialogProps) { + const { t } = useTranslation(); + const [kbId, setKbId] = useState(propKbId); + const [activeMenu, setActiveMenu] = useState('metadata'); + const [showDeleteConfirm, setShowDeleteConfirm] = useState(false); + + useEffect(() => { + setKbId(propKbId); + setActiveMenu('metadata'); + }, [propKbId, open]); + + const menu = [ + { + key: 'metadata', + label: t('knowledge.metadata'), + icon: ( + + + + ), + }, + { + key: 'documents', + label: t('knowledge.documents'), + icon: ( + + + + ), + }, + { + key: 'retrieve', + label: t('knowledge.retrieve'), + icon: ( + + + + ), + }, + ]; + + const confirmDelete = () => { + httpClient.deleteKnowledgeBase(kbId ?? '').then(() => { + onKbDeleted(); + }); + setShowDeleteConfirm(false); + }; + + if (!kbId) { + // new kb + return ( + + +
+ + {t('knowledge.createKnowledgeBase')} + +
+ {activeMenu === 'metadata' && ( + + )} + {activeMenu === 'documents' &&
documents
} +
+ {activeMenu === 'metadata' && ( + +
+ + +
+
+ )} +
+
+
+ ); + } + + return ( + <> + + + + + + + + + {menu.map((item) => ( + + setActiveMenu(item.key)} + > + + {item.icon} + {item.label} + + + + ))} + + + + + +
+ + + {activeMenu === 'metadata' + ? t('knowledge.editKnowledgeBase') + : activeMenu === 'documents' + ? t('knowledge.editDocument') + : t('knowledge.retrieveTest')} + + +
+ {activeMenu === 'metadata' && ( + + )} + {activeMenu === 'documents' && } + {activeMenu === 'retrieve' && } +
+ {activeMenu === 'metadata' && ( + +
+ + + +
+
+ )} +
+
+
+
+ + {/* 删除确认对话框 */} + + + + {t('common.confirmDelete')} + +
+ {t('knowledge.deleteKnowledgeBaseConfirmation')} +
+ + + + +
+
+ + ); +} diff --git a/web/src/app/home/knowledge/components/kb-card/KBCard.module.css b/web/src/app/home/knowledge/components/kb-card/KBCard.module.css new file mode 100644 index 00000000..2ecbd44a --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-card/KBCard.module.css @@ -0,0 +1,107 @@ +.cardContainer { + width: 100%; + height: 10rem; + background-color: #fff; + border-radius: 10px; + box-shadow: 0px 2px 2px 0 rgba(0, 0, 0, 0.2); + padding: 1.2rem; + cursor: pointer; + display: flex; + flex-direction: row; + justify-content: space-between; + gap: 0.5rem; +} + +.cardContainer:hover { + box-shadow: 0px 2px 8px 0 rgba(0, 0, 0, 0.1); +} + +.basicInfoContainer { + width: 100%; + height: 100%; + display: flex; + flex-direction: column; + justify-content: space-between; + gap: 0.4rem; + min-width: 0; +} + +.basicInfoNameContainer { + display: flex; + flex-direction: column; + gap: 0.2rem; +} + +.basicInfoNameText { + font-size: 1.4rem; + font-weight: 500; +} + +.basicInfoDescriptionText { + font-size: 0.9rem; + font-weight: 400; + display: -webkit-box; + -webkit-line-clamp: 3; + -webkit-box-orient: vertical; + overflow: hidden; + text-overflow: ellipsis; + color: #b1b1b1; +} + +.basicInfoLastUpdatedTimeContainer { + display: flex; + flex-direction: row; + align-items: center; + gap: 0.5rem; +} + +.basicInfoUpdateTimeIcon { + width: 1.2rem; + height: 1.2rem; +} + +.basicInfoUpdateTimeText { + font-size: 1rem; + font-weight: 400; +} + +.operationContainer { + display: flex; + flex-direction: column; + align-items: flex-end; + justify-content: space-between; + gap: 0.5rem; + width: 8rem; +} + +.operationDefaultBadge { + display: flex; + flex-direction: row; + gap: 0.5rem; +} + +.operationDefaultBadgeIcon { + width: 1.2rem; + height: 1.2rem; + color: #ffcd27; +} + +.operationDefaultBadgeText { + font-size: 1rem; + font-weight: 400; + color: #ffcd27; +} + +.bigText { + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; + font-size: 1.4rem; + font-weight: bold; + max-width: 100%; +} + +.debugButtonIcon { + width: 1.2rem; + height: 1.2rem; +} diff --git a/web/src/app/home/knowledge/components/kb-card/KBCard.tsx b/web/src/app/home/knowledge/components/kb-card/KBCard.tsx new file mode 100644 index 00000000..560b0497 --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-card/KBCard.tsx @@ -0,0 +1,36 @@ +import { KnowledgeBaseVO } from '@/app/home/knowledge/components/kb-card/KBCardVO'; +import { useTranslation } from 'react-i18next'; +import styles from './KBCard.module.css'; + +export default function KBCard({ kbCardVO }: { kbCardVO: KnowledgeBaseVO }) { + const { t } = useTranslation(); + return ( +
+
+
+
+ {kbCardVO.name} +
+
+ {kbCardVO.description} +
+
+ +
+ + + +
+ {t('knowledge.updateTime')} + {kbCardVO.lastUpdatedTimeAgo} +
+
+
+
+ ); +} diff --git a/web/src/app/home/knowledge/components/kb-card/KBCardVO.ts b/web/src/app/home/knowledge/components/kb-card/KBCardVO.ts new file mode 100644 index 00000000..bfbc2adb --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-card/KBCardVO.ts @@ -0,0 +1,23 @@ +export interface IKnowledgeBaseVO { + id: string; + name: string; + description: string; + embeddingModelUUID: string; + lastUpdatedTimeAgo: string; +} + +export class KnowledgeBaseVO implements IKnowledgeBaseVO { + id: string; + name: string; + description: string; + embeddingModelUUID: string; + lastUpdatedTimeAgo: string; + + constructor(props: IKnowledgeBaseVO) { + this.id = props.id; + this.name = props.name; + this.description = props.description; + this.embeddingModelUUID = props.embeddingModelUUID; + this.lastUpdatedTimeAgo = props.lastUpdatedTimeAgo; + } +} diff --git a/web/src/app/home/knowledge/components/kb-docs/FileUploadZone.tsx b/web/src/app/home/knowledge/components/kb-docs/FileUploadZone.tsx new file mode 100644 index 00000000..3b4123ec --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-docs/FileUploadZone.tsx @@ -0,0 +1,145 @@ +import React, { useCallback, useState } from 'react'; +import { Card, CardContent } from '@/components/ui/card'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { toast } from 'sonner'; +import { useTranslation } from 'react-i18next'; + +interface FileUploadZoneProps { + kbId: string; + onUploadSuccess: () => void; + onUploadError: (error: string) => void; +} + +export default function FileUploadZone({ + kbId, + onUploadSuccess, + onUploadError, +}: FileUploadZoneProps) { + const { t } = useTranslation(); + const [isDragOver, setIsDragOver] = useState(false); + const [isUploading, setIsUploading] = useState(false); + + const handleUpload = useCallback( + async (file: File) => { + if (isUploading) return; + + setIsUploading(true); + const toastId = toast.loading(t('knowledge.documentsTab.uploadingFile')); + + try { + // Step 1: Upload file to server + const uploadResult = await httpClient.uploadDocumentFile(file); + + // Step 2: Associate file with knowledge base + await httpClient.uploadKnowledgeBaseFile(kbId, uploadResult.file_id); + + toast.success(t('knowledge.documentsTab.uploadSuccess'), { + id: toastId, + }); + onUploadSuccess(); + } catch (error) { + console.error('File upload failed:', error); + const errorMessage = t('knowledge.documentsTab.uploadError'); + toast.error(errorMessage, { id: toastId }); + onUploadError(errorMessage); + } finally { + setIsUploading(false); + } + }, + [kbId, isUploading, onUploadSuccess, onUploadError], + ); + + const handleDragOver = useCallback((e: React.DragEvent) => { + e.preventDefault(); + setIsDragOver(true); + }, []); + + const handleDragLeave = useCallback((e: React.DragEvent) => { + e.preventDefault(); + setIsDragOver(false); + }, []); + + const handleDrop = useCallback( + (e: React.DragEvent) => { + e.preventDefault(); + setIsDragOver(false); + + const files = Array.from(e.dataTransfer.files); + if (files.length > 0) { + handleUpload(files[0]); + } + }, + [handleUpload], + ); + + const handleFileSelect = useCallback( + (e: React.ChangeEvent) => { + const files = e.target.files; + if (files && files.length > 0) { + handleUpload(files[0]); + } + }, + [handleUpload], + ); + + return ( + + +
+ + + +
+
+
+ ); +} diff --git a/web/src/app/home/knowledge/components/kb-docs/KBDoc.tsx b/web/src/app/home/knowledge/components/kb-docs/KBDoc.tsx new file mode 100644 index 00000000..fb94dace --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-docs/KBDoc.tsx @@ -0,0 +1,72 @@ +import { useEffect, useState } from 'react'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { KnowledgeBaseFile } from '@/app/infra/entities/api'; +import { columns, DocumentFile } from './documents/columns'; +import { DataTable } from './documents/data-table'; +import FileUploadZone from './FileUploadZone'; +import { toast } from 'sonner'; +import { useTranslation } from 'react-i18next'; + +export default function KBDoc({ kbId }: { kbId: string }) { + const [documentsList, setDocumentsList] = useState([]); + const { t } = useTranslation(); + + useEffect(() => { + getDocumentsList(); + + const intervalId = setInterval(() => { + getDocumentsList(); + }, 5000); + + return () => { + clearInterval(intervalId); + }; + }, [kbId]); + + async function getDocumentsList() { + const resp = await httpClient.getKnowledgeBaseFiles(kbId); + setDocumentsList( + resp.files.map((file: KnowledgeBaseFile) => { + return { + uuid: file.uuid, + name: file.file_name, + status: file.status, + }; + }), + ); + } + + const handleUploadSuccess = () => { + // Refresh document list after successful upload + getDocumentsList(); + }; + + const handleUploadError = (error: string) => { + // Error messages are already handled by toast in FileUploadZone component + console.error('Upload failed:', error); + }; + + const handleDelete = (id: string) => { + httpClient + .deleteKnowledgeBaseFile(kbId, id) + .then(() => { + getDocumentsList(); + toast.success(t('knowledge.documentsTab.fileDeleteSuccess')); + }) + .catch((error) => { + console.error('Delete failed:', error); + toast.error(t('knowledge.documentsTab.fileDeleteFailed')); + }); + }; + + return ( +
+ + +
+ ); +} diff --git a/web/src/app/home/knowledge/components/kb-docs/documents/columns.tsx b/web/src/app/home/knowledge/components/kb-docs/documents/columns.tsx new file mode 100644 index 00000000..6142cfc4 --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-docs/documents/columns.tsx @@ -0,0 +1,94 @@ +'use client'; + +import { ColumnDef } from '@tanstack/react-table'; +import { MoreHorizontal } from 'lucide-react'; +import { Button } from '@/components/ui/button'; +import { + DropdownMenu, + DropdownMenuContent, + DropdownMenuItem, + DropdownMenuLabel, + DropdownMenuTrigger, +} from '@/components/ui/dropdown-menu'; +import { Badge } from '@/components/ui/badge'; +import { TFunction } from 'i18next'; + +export type DocumentFile = { + uuid: string; + name: string; + status: string; +}; + +export const columns = ( + onDelete: (id: string) => void, + t: TFunction, +): ColumnDef[] => { + return [ + { + accessorKey: 'name', + header: t('knowledge.documentsTab.name'), + }, + { + accessorKey: 'status', + header: t('knowledge.documentsTab.status'), + cell: ({ row }) => { + const document = row.original; + + switch (document.status) { + case 'processing': + return ( + + {t('knowledge.documentsTab.processing')} + + ); + case 'completed': + return ( + + {t('knowledge.documentsTab.completed')} + + ); + case 'failed': + return ( + + {t('knowledge.documentsTab.failed')} + + ); + default: + return ( + + {document.status} + + ); + } + }, + }, + { + id: 'actions', + cell: ({ row }) => { + const document = row.original; + + return ( + + + + + + + {t('knowledge.documentsTab.actions')} + + + onDelete(document.uuid)}> + {t('knowledge.documentsTab.delete')} + + + + ); + }, + }, + ]; +}; diff --git a/web/src/app/home/knowledge/components/kb-docs/documents/data-table.tsx b/web/src/app/home/knowledge/components/kb-docs/documents/data-table.tsx new file mode 100644 index 00000000..178ccad9 --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-docs/documents/data-table.tsx @@ -0,0 +1,81 @@ +'use client'; + +import { + ColumnDef, + flexRender, + getCoreRowModel, + useReactTable, +} from '@tanstack/react-table'; +import { + Table, + TableBody, + TableCell, + TableHead, + TableHeader, + TableRow, +} from '@/components/ui/table'; +import { useTranslation } from 'react-i18next'; + +interface DataTableProps { + columns: ColumnDef[]; + data: TData[]; +} + +export function DataTable({ + columns, + data, +}: DataTableProps) { + const { t } = useTranslation(); + const table = useReactTable({ + data, + columns, + getCoreRowModel: getCoreRowModel(), + }); + + return ( +
+ + + {table.getHeaderGroups().map((headerGroup) => ( + + {headerGroup.headers.map((header) => { + return ( + + {header.isPlaceholder + ? null + : flexRender( + header.column.columnDef.header, + header.getContext(), + )} + + ); + })} + + ))} + + + {table.getRowModel().rows?.length ? ( + table.getRowModel().rows.map((row) => ( + + {row.getVisibleCells().map((cell) => ( + + {flexRender(cell.column.columnDef.cell, cell.getContext())} + + ))} + + )) + ) : ( + + + {t('knowledge.documentsTab.noResults')} + + + )} + +
+
+ ); +} diff --git a/web/src/app/home/knowledge/components/kb-form/ChooseEntity.ts b/web/src/app/home/knowledge/components/kb-form/ChooseEntity.ts new file mode 100644 index 00000000..54f983e4 --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-form/ChooseEntity.ts @@ -0,0 +1,4 @@ +export interface IEmbeddingModelEntity { + label: string; + value: string; +} diff --git a/web/src/app/home/knowledge/components/kb-form/KBForm.tsx b/web/src/app/home/knowledge/components/kb-form/KBForm.tsx new file mode 100644 index 00000000..54d5d6e4 --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-form/KBForm.tsx @@ -0,0 +1,234 @@ +import { useEffect, useState } from 'react'; +import { useForm } from 'react-hook-form'; +import { zodResolver } from '@hookform/resolvers/zod'; +import { z } from 'zod'; +import { useTranslation } from 'react-i18next'; +import { Input } from '@/components/ui/input'; +import { + Form, + FormControl, + FormField, + FormItem, + FormLabel, + FormMessage, + FormDescription, +} from '@/components/ui/form'; +import { IEmbeddingModelEntity } from './ChooseEntity'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { + Select, + SelectContent, + SelectGroup, + SelectItem, + SelectTrigger, + SelectValue, +} from '@/components/ui/select'; +import { KnowledgeBase } from '@/app/infra/entities/api'; +import { toast } from 'sonner'; + +const getFormSchema = (t: (key: string) => string) => + z.object({ + name: z.string().min(1, { message: t('knowledge.kbNameRequired') }), + description: z + .string() + .min(1, { message: t('knowledge.kbDescriptionRequired') }), + embeddingModelUUID: z + .string() + .min(1, { message: t('knowledge.embeddingModelUUIDRequired') }), + }); + +export default function KBForm({ + initKbId, + onNewKbCreated, + onKbUpdated, +}: { + initKbId?: string; + onNewKbCreated: (kbId: string) => void; + onKbUpdated: (kbId: string) => void; +}) { + const { t } = useTranslation(); + const formSchema = getFormSchema(t); + + const form = useForm>({ + resolver: zodResolver(formSchema), + defaultValues: { + name: '', + description: t('knowledge.defaultDescription'), + embeddingModelUUID: '', + }, + }); + + const [embeddingModelNameList, setEmbeddingModelNameList] = useState< + IEmbeddingModelEntity[] + >([]); + + useEffect(() => { + getEmbeddingModelNameList().then(() => { + if (initKbId) { + getKbConfig(initKbId).then((val) => { + form.setValue('name', val.name); + form.setValue('description', val.description); + form.setValue('embeddingModelUUID', val.embeddingModelUUID); + }); + } + }); + }, []); + + const getKbConfig = async ( + kbId: string, + ): Promise> => { + return new Promise((resolve) => { + httpClient.getKnowledgeBase(kbId).then((res) => { + resolve({ + name: res.base.name, + description: res.base.description, + embeddingModelUUID: res.base.embedding_model_uuid, + }); + }); + }); + }; + + const getEmbeddingModelNameList = async () => { + const resp = await httpClient.getProviderEmbeddingModels(); + setEmbeddingModelNameList( + resp.models.map((item) => { + return { + label: item.name, + value: item.uuid, + }; + }), + ); + }; + + const onSubmit = (data: z.infer) => { + console.log('data', data); + + if (initKbId) { + // update knowledge base + const updateKb: KnowledgeBase = { + name: data.name, + description: data.description, + embedding_model_uuid: data.embeddingModelUUID, + }; + httpClient + .updateKnowledgeBase(initKbId, updateKb) + .then((res) => { + console.log('update knowledge base success', res); + onKbUpdated(res.uuid); + toast.success(t('knowledge.updateKnowledgeBaseSuccess')); + }) + .catch((err) => { + console.error('update knowledge base failed', err); + toast.error(t('knowledge.updateKnowledgeBaseFailed')); + }); + } else { + // create knowledge base + const newKb: KnowledgeBase = { + name: data.name, + description: data.description, + embedding_model_uuid: data.embeddingModelUUID, + }; + httpClient + .createKnowledgeBase(newKb) + .then((res) => { + console.log('create knowledge base success', res); + onNewKbCreated(res.uuid); + }) + .catch((err) => { + console.error('create knowledge base failed', err); + }); + } + }; + + return ( + <> +
+ +
+ ( + + + {t('knowledge.kbName')} + * + + + + + + + )} + /> + ( + + + {t('knowledge.kbDescription')} + * + + + + + + + )} + /> + ( + + + {t('knowledge.embeddingModelUUID')} + * + + +
+ +
+
+ + {initKbId + ? t('knowledge.cannotChangeEmbeddingModel') + : t('knowledge.embeddingModelDescription')} + + +
+ )} + /> +
+
+ + + ); +} diff --git a/web/src/app/home/knowledge/components/kb-retrieve/KBRetrieve.tsx b/web/src/app/home/knowledge/components/kb-retrieve/KBRetrieve.tsx new file mode 100644 index 00000000..1db7d11d --- /dev/null +++ b/web/src/app/home/knowledge/components/kb-retrieve/KBRetrieve.tsx @@ -0,0 +1,99 @@ +'use client'; + +import React, { useState, useEffect } from 'react'; +import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; +import { Button } from '@/components/ui/button'; +import { Input } from '@/components/ui/input'; +import { useTranslation } from 'react-i18next'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { RetrieveResult, KnowledgeBaseFile } from '@/app/infra/entities/api'; +import { toast } from 'sonner'; + +interface KBRetrieveProps { + kbId: string; +} + +export default function KBRetrieve({ kbId }: KBRetrieveProps) { + const { t } = useTranslation(); + const [query, setQuery] = useState(''); + const [results, setResults] = useState([]); + const [files, setFiles] = useState([]); + const [loading, setLoading] = useState(false); + + useEffect(() => { + const loadFiles = async () => { + try { + const response = await httpClient.getKnowledgeBaseFiles(kbId); + setFiles(response.files); + } catch (error) { + console.error('Failed to load files:', error); + } + }; + loadFiles(); + }, [kbId]); + + const handleRetrieve = async () => { + if (!query.trim()) return; + + setLoading(true); + try { + setResults([]); + const response = await httpClient.retrieveKnowledgeBase(kbId, query); + setResults(response.results); + } catch (error) { + console.error('Retrieve failed:', error); + toast.error(t('knowledge.retrieveError')); + } finally { + setLoading(false); + } + }; + + const getFileName = (fileId: string) => { + const file = files.find((f) => f.uuid === fileId); + return file?.file_name || fileId; + }; + + return ( +
+
+ setQuery(e.target.value)} + placeholder={t('knowledge.queryPlaceholder')} + onKeyPress={(e) => e.key === 'Enter' && handleRetrieve()} + /> + +
+ +
+ {results.length === 0 && !loading && ( +

{t('knowledge.noResults')}

+ )} + + {loading ? ( +

{t('common.loading')}

+ ) : ( + results.map((result) => ( + + + + {getFileName(result.metadata.file_id)} + + {t('knowledge.distance')}: {result.distance.toFixed(4)} + + + + +

+ {result.metadata.text} +

+
+
+ )) + )} +
+
+ ); +} diff --git a/web/src/app/home/knowledge/knowledgeBase.module.css b/web/src/app/home/knowledge/knowledgeBase.module.css new file mode 100644 index 00000000..e811b521 --- /dev/null +++ b/web/src/app/home/knowledge/knowledgeBase.module.css @@ -0,0 +1,15 @@ +.configPageContainer { + width: 100%; + height: 100%; +} + +.knowledgeListContainer { + width: 100%; + padding-left: 0.8rem; + padding-right: 0.8rem; + display: grid; + grid-template-columns: repeat(auto-fill, minmax(24rem, 1fr)); + gap: 2rem; + justify-items: stretch; + align-items: start; +} diff --git a/web/src/app/home/knowledge/page.tsx b/web/src/app/home/knowledge/page.tsx new file mode 100644 index 00000000..0a8cc2eb --- /dev/null +++ b/web/src/app/home/knowledge/page.tsx @@ -0,0 +1,114 @@ +'use client'; + +import CreateCardComponent from '@/app/infra/basic-component/create-card-component/CreateCardComponent'; +import styles from './knowledgeBase.module.css'; +import { useTranslation } from 'react-i18next'; +import { useEffect, useState } from 'react'; +import { KnowledgeBaseVO } from '@/app/home/knowledge/components/kb-card/KBCardVO'; +import KBCard from '@/app/home/knowledge/components/kb-card/KBCard'; +import KBDetailDialog from '@/app/home/knowledge/KBDetailDialog'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { KnowledgeBase } from '@/app/infra/entities/api'; + +export default function KnowledgePage() { + const { t } = useTranslation(); + const [knowledgeBaseList, setKnowledgeBaseList] = useState( + [], + ); + const [selectedKbId, setSelectedKbId] = useState(''); + const [detailDialogOpen, setDetailDialogOpen] = useState(false); + + useEffect(() => { + getKnowledgeBaseList(); + }, []); + + async function getKnowledgeBaseList() { + const resp = await httpClient.getKnowledgeBases(); + setKnowledgeBaseList( + resp.bases.map((kb: KnowledgeBase) => { + const currentTime = new Date(); + const lastUpdatedTimeAgo = Math.floor( + (currentTime.getTime() - + new Date(kb.updated_at ?? currentTime.getTime()).getTime()) / + 1000 / + 60 / + 60 / + 24, + ); + + const lastUpdatedTimeAgoText = + lastUpdatedTimeAgo > 0 + ? ` ${lastUpdatedTimeAgo} ${t('knowledge.daysAgo')}` + : t('knowledge.today'); + + return new KnowledgeBaseVO({ + id: kb.uuid || '', + name: kb.name, + description: kb.description, + embeddingModelUUID: kb.embedding_model_uuid, + lastUpdatedTimeAgo: lastUpdatedTimeAgoText, + }); + }), + ); + } + + const handleKBCardClick = (kbId: string) => { + setSelectedKbId(kbId); + setDetailDialogOpen(true); + }; + + const handleCreateKBClick = () => { + setSelectedKbId(''); + setDetailDialogOpen(true); + }; + + const handleFormCancel = () => { + setDetailDialogOpen(false); + }; + + const handleKbDeleted = () => { + getKnowledgeBaseList(); + setDetailDialogOpen(false); + }; + + const handleNewKbCreated = (newKbId: string) => { + getKnowledgeBaseList(); + setSelectedKbId(newKbId); + setDetailDialogOpen(true); + }; + + const handleKbUpdated = () => { + getKnowledgeBaseList(); + }; + + return ( +
+ + +
+ + + {knowledgeBaseList.map((kb) => { + return ( +
handleKBCardClick(kb.id)}> + +
+ ); + })} +
+
+ ); +} diff --git a/web/src/app/home/models/component/llm-form/ChooseRequesterEntity.ts b/web/src/app/home/models/component/ChooseRequesterEntity.ts similarity index 100% rename from web/src/app/home/models/component/llm-form/ChooseRequesterEntity.ts rename to web/src/app/home/models/component/ChooseRequesterEntity.ts diff --git a/web/src/app/home/models/component/ICreateEmbeddingField.ts b/web/src/app/home/models/component/ICreateEmbeddingField.ts new file mode 100644 index 00000000..ea198f3f --- /dev/null +++ b/web/src/app/home/models/component/ICreateEmbeddingField.ts @@ -0,0 +1,7 @@ +export interface ICreateEmbeddingField { + name: string; + model_provider: string; + url: string; + api_key: string; + extra_args?: string[]; +} diff --git a/web/src/app/home/models/ICreateLLMField.ts b/web/src/app/home/models/component/ICreateLLMField.ts similarity index 100% rename from web/src/app/home/models/ICreateLLMField.ts rename to web/src/app/home/models/component/ICreateLLMField.ts diff --git a/web/src/app/home/models/component/embedding-card/EmbeddingCard.module.css b/web/src/app/home/models/component/embedding-card/EmbeddingCard.module.css new file mode 100644 index 00000000..9c6c54f7 --- /dev/null +++ b/web/src/app/home/models/component/embedding-card/EmbeddingCard.module.css @@ -0,0 +1,97 @@ +.cardContainer { + width: 100%; + height: 10rem; + background-color: #fff; + border-radius: 10px; + box-shadow: 0px 2px 2px 0 rgba(0, 0, 0, 0.2); + padding: 1.2rem; + cursor: pointer; +} + +.cardContainer:hover { + box-shadow: 0px 2px 8px 0 rgba(0, 0, 0, 0.1); +} + +.iconBasicInfoContainer { + width: 100%; + height: 100%; + display: flex; + flex-direction: row; + gap: 0.8rem; + user-select: none; +} + +.iconImage { + width: 3.8rem; + height: 3.8rem; + margin: 0.2rem; + border-radius: 50%; +} + +.basicInfoContainer { + display: flex; + flex-direction: column; + gap: 0.2rem; + min-width: 0; + width: 100%; +} + +.basicInfoText { + font-size: 1.4rem; + font-weight: bold; +} + +.providerContainer { + display: flex; + flex-direction: row; + align-items: center; + justify-content: flex-start; + gap: 0.2rem; +} + +.providerIcon { + width: 1.2rem; + height: 1.2rem; + margin-top: 0.2rem; + color: #626262; +} + +.providerLabel { + font-size: 1.2rem; + font-weight: 600; + color: #626262; +} + +.baseURLContainer { + display: flex; + flex-direction: row; + align-items: center; + justify-content: flex-start; + gap: 0.2rem; + width: calc(100% - 3rem); +} + +.baseURLIcon { + width: 1.2rem; + height: 1.2rem; + color: #626262; +} + +.baseURLText { + font-size: 1rem; + width: 100%; + color: #626262; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + max-width: 100%; +} + +.bigText { + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; + font-size: 1.4rem; + font-weight: bold; + max-width: 100%; +} diff --git a/web/src/app/home/models/component/embedding-card/EmbeddingCard.tsx b/web/src/app/home/models/component/embedding-card/EmbeddingCard.tsx new file mode 100644 index 00000000..e3dfaf80 --- /dev/null +++ b/web/src/app/home/models/component/embedding-card/EmbeddingCard.tsx @@ -0,0 +1,53 @@ +import styles from './EmbeddingCard.module.css'; +import { EmbeddingCardVO } from '@/app/home/models/component/embedding-card/EmbeddingCardVO'; + +export default function EmbeddingCard({ cardVO }: { cardVO: EmbeddingCardVO }) { + return ( +
+
+ icon + +
+ {/* 名称 */} +
+ {cardVO.name} +
+ {/* 厂商 */} +
+ + + + + {cardVO.providerLabel} + +
+ {/* baseURL */} +
+ + + + {cardVO.baseURL} +
+
+
+
+ ); +} diff --git a/web/src/app/home/models/component/embedding-card/EmbeddingCardVO.ts b/web/src/app/home/models/component/embedding-card/EmbeddingCardVO.ts new file mode 100644 index 00000000..f6d960f6 --- /dev/null +++ b/web/src/app/home/models/component/embedding-card/EmbeddingCardVO.ts @@ -0,0 +1,23 @@ +export interface IEmbeddingCardVO { + id: string; + iconURL: string; + name: string; + providerLabel: string; + baseURL: string; +} + +export class EmbeddingCardVO implements IEmbeddingCardVO { + id: string; + iconURL: string; + providerLabel: string; + name: string; + baseURL: string; + + constructor(props: IEmbeddingCardVO) { + this.id = props.id; + this.iconURL = props.iconURL; + this.providerLabel = props.providerLabel; + this.name = props.name; + this.baseURL = props.baseURL; + } +} diff --git a/web/src/app/home/models/component/embedding-form/EmbeddingForm.tsx b/web/src/app/home/models/component/embedding-form/EmbeddingForm.tsx new file mode 100644 index 00000000..d50885ae --- /dev/null +++ b/web/src/app/home/models/component/embedding-form/EmbeddingForm.tsx @@ -0,0 +1,566 @@ +import { ICreateEmbeddingField } from '@/app/home/models/component/ICreateEmbeddingField'; +import { useEffect, useState } from 'react'; +import { IChooseRequesterEntity } from '@/app/home/models/component/ChooseRequesterEntity'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { EmbeddingModel } from '@/app/infra/entities/api'; +import { UUID } from 'uuidjs'; + +import { zodResolver } from '@hookform/resolvers/zod'; +import { useForm } from 'react-hook-form'; +import { z } from 'zod'; +import { useTranslation } from 'react-i18next'; + +import { + Dialog, + DialogContent, + DialogDescription, + DialogHeader, + DialogTitle, + DialogFooter, +} from '@/components/ui/dialog'; +import { Button } from '@/components/ui/button'; +import { + Form, + FormControl, + FormDescription, + FormField, + FormItem, + FormLabel, + FormMessage, +} from '@/components/ui/form'; +import { Input } from '@/components/ui/input'; +import { + Select, + SelectContent, + SelectGroup, + SelectItem, + SelectTrigger, + SelectValue, +} from '@/components/ui/select'; +import { toast } from 'sonner'; +import { i18nObj } from '@/i18n/I18nProvider'; + +const getExtraArgSchema = (t: (key: string) => string) => + z + .object({ + key: z.string().min(1, { message: t('models.keyNameRequired') }), + type: z.enum(['string', 'number', 'boolean']), + value: z.string(), + }) + .superRefine((data, ctx) => { + if (data.type === 'number' && isNaN(Number(data.value))) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: t('models.mustBeValidNumber'), + path: ['value'], + }); + } + if ( + data.type === 'boolean' && + data.value !== 'true' && + data.value !== 'false' + ) { + ctx.addIssue({ + code: z.ZodIssueCode.custom, + message: t('models.mustBeTrueOrFalse'), + path: ['value'], + }); + } + }); + +const getFormSchema = (t: (key: string) => string) => + z.object({ + name: z.string().min(1, { message: t('models.modelNameRequired') }), + model_provider: z + .string() + .min(1, { message: t('models.modelProviderRequired') }), + url: z.string().min(1, { message: t('models.requestURLRequired') }), + api_key: z.string().min(1, { message: t('models.apiKeyRequired') }), + extra_args: z.array(getExtraArgSchema(t)).optional(), + }); + +export default function EmbeddingForm({ + editMode, + initEmbeddingId, + onFormSubmit, + onFormCancel, + onEmbeddingDeleted, +}: { + editMode: boolean; + initEmbeddingId?: string; + onFormSubmit: () => void; + onFormCancel: () => void; + onEmbeddingDeleted: () => void; +}) { + const { t } = useTranslation(); + const formSchema = getFormSchema(t); + + const form = useForm>({ + resolver: zodResolver(formSchema), + defaultValues: { + name: '', + model_provider: '', + url: '', + api_key: 'sk-xxxxx', + extra_args: [], + }, + }); + + const [extraArgs, setExtraArgs] = useState< + { key: string; type: 'string' | 'number' | 'boolean'; value: string }[] + >([]); + + const [showDeleteConfirmModal, setShowDeleteConfirmModal] = useState(false); + const [requesterNameList, setRequesterNameList] = useState< + IChooseRequesterEntity[] + >([]); + const [requesterDefaultURLList, setRequesterDefaultURLList] = useState< + string[] + >([]); + const [modelTesting, setModelTesting] = useState(false); + const [currentModelProvider, setCurrentModelProvider] = useState(''); + + useEffect(() => { + initEmbeddingModelFormComponent().then(() => { + if (editMode && initEmbeddingId) { + getEmbeddingConfig(initEmbeddingId).then((val) => { + form.setValue('name', val.name); + form.setValue('model_provider', val.model_provider); + setCurrentModelProvider(val.model_provider); + form.setValue('url', val.url); + form.setValue('api_key', val.api_key); + if (val.extra_args) { + const args = val.extra_args.map((arg) => { + const [key, value] = arg.split(':'); + let type: 'string' | 'number' | 'boolean' = 'string'; + if (!isNaN(Number(value))) { + type = 'number'; + } else if (value === 'true' || value === 'false') { + type = 'boolean'; + } + return { + key, + type, + value, + }; + }); + setExtraArgs(args); + form.setValue('extra_args', args); + } + }); + } else { + form.reset(); + } + }); + }, []); + + const addExtraArg = () => { + setExtraArgs([...extraArgs, { key: '', type: 'string', value: '' }]); + }; + + const updateExtraArg = ( + index: number, + field: 'key' | 'type' | 'value', + value: string, + ) => { + const newArgs = [...extraArgs]; + newArgs[index] = { + ...newArgs[index], + [field]: value, + }; + setExtraArgs(newArgs); + form.setValue('extra_args', newArgs); + }; + + const removeExtraArg = (index: number) => { + const newArgs = extraArgs.filter((_, i) => i !== index); + setExtraArgs(newArgs); + form.setValue('extra_args', newArgs); + }; + + async function initEmbeddingModelFormComponent() { + const requesterNameList = + await httpClient.getProviderRequesters('text-embedding'); + setRequesterNameList( + requesterNameList.requesters.map((item) => { + return { + label: i18nObj(item.label), + value: item.name, + }; + }), + ); + setRequesterDefaultURLList( + requesterNameList.requesters.map((item) => { + const config = item.spec.config; + for (let i = 0; i < config.length; i++) { + if (config[i].name == 'base_url') { + return config[i].default?.toString() || ''; + } + } + return ''; + }), + ); + } + + async function getEmbeddingConfig( + id: string, + ): Promise { + const embeddingModel = await httpClient.getProviderEmbeddingModel(id); + + const fakeExtraArgs = []; + const extraArgs = embeddingModel.model.extra_args as Record; + for (const key in extraArgs) { + fakeExtraArgs.push(`${key}:${extraArgs[key]}`); + } + return { + name: embeddingModel.model.name, + model_provider: embeddingModel.model.requester, + url: embeddingModel.model.requester_config?.base_url, + api_key: embeddingModel.model.api_keys[0], + extra_args: fakeExtraArgs, + }; + } + + function handleFormSubmit(value: z.infer) { + const extraArgsObj: Record = {}; + value.extra_args?.forEach( + (arg: { key: string; type: string; value: string }) => { + if (arg.type === 'number') { + extraArgsObj[arg.key] = Number(arg.value); + } else if (arg.type === 'boolean') { + extraArgsObj[arg.key] = arg.value === 'true'; + } else { + extraArgsObj[arg.key] = arg.value; + } + }, + ); + + const embeddingModel: EmbeddingModel = { + uuid: editMode ? initEmbeddingId || '' : UUID.generate(), + name: value.name, + description: '', + requester: value.model_provider, + requester_config: { + base_url: value.url, + timeout: 120, + }, + extra_args: extraArgsObj, + api_keys: [value.api_key], + }; + + if (editMode) { + onSaveEdit(embeddingModel).then(() => { + form.reset(); + }); + } else { + onCreateEmbedding(embeddingModel).then(() => { + form.reset(); + }); + } + } + + async function onCreateEmbedding(embeddingModel: EmbeddingModel) { + try { + await httpClient.createProviderEmbeddingModel(embeddingModel); + onFormSubmit(); + toast.success(t('models.createSuccess')); + } catch (err) { + toast.error(t('models.createError') + (err as Error).message); + } + } + + async function onSaveEdit(embeddingModel: EmbeddingModel) { + try { + await httpClient.updateProviderEmbeddingModel( + initEmbeddingId || '', + embeddingModel, + ); + onFormSubmit(); + toast.success(t('models.saveSuccess')); + } catch (err) { + toast.error(t('models.saveError') + (err as Error).message); + } + } + + function deleteModel() { + if (initEmbeddingId) { + httpClient + .deleteProviderEmbeddingModel(initEmbeddingId) + .then(() => { + onEmbeddingDeleted(); + toast.success(t('models.deleteSuccess')); + }) + .catch((err) => { + toast.error(t('models.deleteError') + err.message); + }); + } + } + + function testEmbeddingModelInForm() { + setModelTesting(true); + httpClient + .testEmbeddingModel('_', { + uuid: '', + name: form.getValues('name'), + description: '', + requester: form.getValues('model_provider'), + requester_config: { + base_url: form.getValues('url'), + timeout: 120, + }, + api_keys: [form.getValues('api_key')], + }) + .then((res) => { + console.log(res); + toast.success(t('models.testSuccess')); + }) + .catch(() => { + toast.error(t('models.testError')); + }) + .finally(() => { + setModelTesting(false); + }); + } + + return ( +
+ + + + {t('common.confirmDelete')} + + + {t('models.deleteConfirmation')} + + + + + + + + +
+ +
+ ( + + + {t('models.modelName')} + * + + + + + + + {t('models.modelProviderDescription')} + + + )} + /> + + ( + + + {t('models.modelProvider')} + * + + + + + + + )} + /> + + ( + + + {t('models.requestURL')} + * + + + + + + + )} + /> + + {!['ollama-chat'].includes(currentModelProvider) && ( + ( + + + {t('models.apiKey')} + * + + + + + + + )} + /> + )} + + + {t('models.extraParameters')} +
+ {extraArgs.map((arg, index) => ( +
+ + updateExtraArg(index, 'key', e.target.value) + } + /> + + + updateExtraArg(index, 'value', e.target.value) + } + /> + +
+ ))} + +
+ + {t('embedding.extraParametersDescription')} + + +
+
+ + {editMode && ( + + )} + + + + + + + +
+ +
+ ); +} diff --git a/web/src/app/home/models/component/llm-form/LLMForm.tsx b/web/src/app/home/models/component/llm-form/LLMForm.tsx index f483f183..73cc32fe 100644 --- a/web/src/app/home/models/component/llm-form/LLMForm.tsx +++ b/web/src/app/home/models/component/llm-form/LLMForm.tsx @@ -1,6 +1,6 @@ -import { ICreateLLMField } from '@/app/home/models/ICreateLLMField'; +import { ICreateLLMField } from '@/app/home/models/component/ICreateLLMField'; import { useEffect, useState } from 'react'; -import { IChooseRequesterEntity } from '@/app/home/models/component/llm-form/ChooseRequesterEntity'; +import { IChooseRequesterEntity } from '@/app/home/models/component/ChooseRequesterEntity'; import { httpClient } from '@/app/infra/http/HttpClient'; import { LLMModel } from '@/app/infra/entities/api'; import { UUID } from 'uuidjs'; @@ -197,7 +197,7 @@ export default function LLMForm({ }; async function initLLMModelFormComponent() { - const requesterNameList = await httpClient.getProviderRequesters(); + const requesterNameList = await httpClient.getProviderRequesters('llm'); setRequesterNameList( requesterNameList.requesters.map((item) => { return { @@ -596,7 +596,7 @@ export default function LLMForm({
- {t('models.extraParametersDescription')} + {t('llm.extraParametersDescription')} diff --git a/web/src/app/home/models/page.tsx b/web/src/app/home/models/page.tsx index 3ccec486..2f936753 100644 --- a/web/src/app/home/models/page.tsx +++ b/web/src/app/home/models/page.tsx @@ -8,6 +8,7 @@ import LLMForm from '@/app/home/models/component/llm-form/LLMForm'; import CreateCardComponent from '@/app/infra/basic-component/create-card-component/CreateCardComponent'; import { httpClient } from '@/app/infra/http/HttpClient'; import { LLMModel } from '@/app/infra/entities/api'; +import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; import { Dialog, DialogContent, @@ -17,6 +18,9 @@ import { import { toast } from 'sonner'; import { useTranslation } from 'react-i18next'; import { i18nObj } from '@/i18n/I18nProvider'; +import { EmbeddingCardVO } from '@/app/home/models/component/embedding-card/EmbeddingCardVO'; +import EmbeddingCard from '@/app/home/models/component/embedding-card/EmbeddingCard'; +import EmbeddingForm from '@/app/home/models/component/embedding-form/EmbeddingForm'; export default function LLMConfigPage() { const { t } = useTranslation(); @@ -24,13 +28,21 @@ export default function LLMConfigPage() { const [modalOpen, setModalOpen] = useState(false); const [isEditForm, setIsEditForm] = useState(false); const [nowSelectedLLM, setNowSelectedLLM] = useState(null); + const [embeddingCardList, setEmbeddingCardList] = useState( + [], + ); + const [embeddingModalOpen, setEmbeddingModalOpen] = useState(false); + const [isEditEmbeddingForm, setIsEditEmbeddingForm] = useState(false); + const [nowSelectedEmbedding, setNowSelectedEmbedding] = + useState(null); useEffect(() => { getLLMModelList(); + getEmbeddingModelList(); }, []); async function getLLMModelList() { - const requesterNameListResp = await httpClient.getProviderRequesters(); + const requesterNameListResp = await httpClient.getProviderRequesters('llm'); const requesterNameList = requesterNameListResp.requesters.map((item) => { return { label: i18nObj(item.label), @@ -74,6 +86,55 @@ export default function LLMConfigPage() { setNowSelectedLLM(null); setModalOpen(true); } + function selectEmbedding(cardVO: EmbeddingCardVO) { + setIsEditEmbeddingForm(true); + setNowSelectedEmbedding(cardVO); + setEmbeddingModalOpen(true); + } + + function handleCreateEmbeddingModelClick() { + setIsEditEmbeddingForm(false); + setNowSelectedEmbedding(null); + setEmbeddingModalOpen(true); + } + async function getEmbeddingModelList() { + const requesterNameListResp = + await httpClient.getProviderRequesters('text-embedding'); + const requesterNameList = requesterNameListResp.requesters.map((item) => { + return { + label: i18nObj(item.label), + value: item.name, + }; + }); + + httpClient + .getProviderEmbeddingModels() + .then((resp) => { + const embeddingModelList: EmbeddingCardVO[] = resp.models.map( + (model: { + uuid: string; + requester: string; + name: string; + requester_config?: { base_url?: string }; + }) => { + return new EmbeddingCardVO({ + id: model.uuid, + iconURL: httpClient.getProviderRequesterIconURL(model.requester), + name: model.name, + providerLabel: + requesterNameList.find((item) => item.value === model.requester) + ?.label || model.requester.substring(0, 10), + baseURL: model.requester_config?.base_url || '', + }); + }, + ); + setEmbeddingCardList(embeddingModelList); + }) + .catch((err) => { + console.error('get Embedding model list error', err); + toast.error(t('embedding.getModelListError') + err.message); + }); + } return (
@@ -101,26 +162,108 @@ export default function LLMConfigPage() { /> -
- - {cardList.map((cardVO) => { - return ( -
{ - selectLLM(cardVO); - }} - > - + + + + + {isEditEmbeddingForm + ? t('embedding.editModel') + : t('embedding.createModel')} + + + { + setEmbeddingModalOpen(false); + getEmbeddingModelList(); + }} + onFormCancel={() => { + setEmbeddingModalOpen(false); + }} + onEmbeddingDeleted={() => { + setEmbeddingModalOpen(false); + getEmbeddingModelList(); + }} + /> + + + + +
+
+ + + {t('llm.llmModels')} + + + {t('embedding.embeddingModels')} + + +
+ +
+

{t('llm.description')}

- ); - })} -
+ + +
+

+ {t('embedding.description')} +

+
+
+
+ + +
+ + {cardList.map((cardVO) => { + return ( +
{ + selectLLM(cardVO); + }} + > + +
+ ); + })} +
+
+ + +
+ + {embeddingCardList.map((cardVO) => { + return ( +
{ + selectEmbedding(cardVO); + }} + > + +
+ ); + })} +
+
+
); } diff --git a/web/src/app/home/pipelines/PipelineDetailDialog.tsx b/web/src/app/home/pipelines/PipelineDetailDialog.tsx index 72b4ac76..53cf9eaa 100644 --- a/web/src/app/home/pipelines/PipelineDetailDialog.tsx +++ b/web/src/app/home/pipelines/PipelineDetailDialog.tsx @@ -18,7 +18,6 @@ import { } from '@/components/ui/sidebar'; import PipelineFormComponent from './components/pipeline-form/PipelineFormComponent'; import DebugDialog from './components/debug-dialog/DebugDialog'; -import { PipelineFormEntity } from '@/app/infra/entities/pipeline'; interface PipelineDialogProps { open: boolean; @@ -26,7 +25,6 @@ interface PipelineDialogProps { pipelineId?: string; isEditMode?: boolean; isDefaultPipeline?: boolean; - initValues?: PipelineFormEntity; onFinish: () => void; onNewPipelineCreated?: (pipelineId: string) => void; onDeletePipeline: () => void; @@ -41,7 +39,6 @@ export default function PipelineDialog({ pipelineId: propPipelineId, isEditMode = false, isDefaultPipeline = false, - initValues, onFinish, onNewPipelineCreated, onDeletePipeline, @@ -119,7 +116,6 @@ export default function PipelineDialog({
{currentMode === 'config' && ( void; onNewPipelineCreated: (pipelineId: string) => void; onDeletePipeline: () => void; @@ -132,13 +128,26 @@ export default function PipelineFormComponent({ } } }); + + if (isEditMode) { + httpClient + .getPipeline(pipelineId || '') + .then((resp: GetPipelineResponseData) => { + form.reset({ + basic: { + name: resp.pipeline.name, + description: resp.pipeline.description, + }, + ai: resp.pipeline.config.ai, + trigger: resp.pipeline.config.trigger, + safety: resp.pipeline.config.safety, + output: resp.pipeline.config.output, + }); + }); + } }, []); useEffect(() => { - if (initValues) { - form.reset(initValues); - } - if (!isEditMode) { form.reset({ basic: { @@ -147,7 +156,7 @@ export default function PipelineFormComponent({ }, }); } - }, [initValues, form, isEditMode]); + }, [form, isEditMode]); function handleFormSubmit(values: FormValues) { console.log('handleFormSubmit', values); diff --git a/web/src/app/home/pipelines/page.tsx b/web/src/app/home/pipelines/page.tsx index 40875f6e..bec13e54 100644 --- a/web/src/app/home/pipelines/page.tsx +++ b/web/src/app/home/pipelines/page.tsx @@ -4,11 +4,17 @@ import CreateCardComponent from '@/app/infra/basic-component/create-card-compone import { httpClient } from '@/app/infra/http/HttpClient'; import { PipelineCardVO } from '@/app/home/pipelines/components/pipeline-card/PipelineCardVO'; import PipelineCard from '@/app/home/pipelines/components/pipeline-card/PipelineCard'; -import { PipelineFormEntity } from '@/app/infra/entities/pipeline'; import styles from './pipelineConfig.module.css'; import { toast } from 'sonner'; import { useTranslation } from 'react-i18next'; import PipelineDialog from './PipelineDetailDialog'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '@/components/ui/select'; export default function PluginConfigPage() { const { t } = useTranslation(); @@ -16,24 +22,22 @@ export default function PluginConfigPage() { const [isEditForm, setIsEditForm] = useState(false); const [pipelineList, setPipelineList] = useState([]); const [selectedPipelineId, setSelectedPipelineId] = useState(''); - const [selectedPipelineFormValue, setSelectedPipelineFormValue] = - useState({ - basic: {}, - ai: {}, - trigger: {}, - safety: {}, - output: {}, - }); + const [selectedPipelineIsDefault, setSelectedPipelineIsDefault] = useState(false); + const [sortByValue, setSortByValue] = useState('created_at'); + const [sortOrderValue, setSortOrderValue] = useState('DESC'); useEffect(() => { getPipelines(); }, []); - function getPipelines() { + function getPipelines( + sortBy: string = sortByValue, + sortOrder: string = sortOrderValue, + ) { httpClient - .getPipelines() + .getPipelines(sortBy, sortOrder) .then((value) => { const currentTime = new Date(); const pipelineList = value.pipelines.map((pipeline) => { @@ -69,43 +73,27 @@ export default function PluginConfigPage() { }); } - function getSelectedPipelineForm(id?: string) { - httpClient.getPipeline(id ?? selectedPipelineId).then((value) => { - setSelectedPipelineFormValue({ - ai: value.pipeline.config.ai, - basic: { - description: value.pipeline.description, - name: value.pipeline.name, - }, - output: value.pipeline.config.output, - safety: value.pipeline.config.safety, - trigger: value.pipeline.config.trigger, - }); - setSelectedPipelineIsDefault(value.pipeline.is_default ?? false); - }); - } - const handlePipelineClick = (pipelineId: string) => { setSelectedPipelineId(pipelineId); setIsEditForm(true); setDialogOpen(true); - getSelectedPipelineForm(pipelineId); }; const handleCreateNew = () => { setIsEditForm(false); setSelectedPipelineId(''); - setSelectedPipelineFormValue({ - basic: {}, - ai: {}, - trigger: {}, - safety: {}, - output: {}, - }); + setSelectedPipelineIsDefault(false); setDialogOpen(true); }; + function handleSortChange(value: string) { + const [newSortBy, newSortOrder] = value.split(',').map((s) => s.trim()); + setSortByValue(newSortBy); + setSortOrderValue(newSortOrder); + getPipelines(newSortBy, newSortOrder); + } + return (
{ getPipelines(); }} @@ -123,7 +110,6 @@ export default function PluginConfigPage() { setSelectedPipelineId(pipelineId); setIsEditForm(true); setDialogOpen(true); - getSelectedPipelineForm(pipelineId); }} onDeletePipeline={() => { getPipelines(); @@ -134,6 +120,27 @@ export default function PluginConfigPage() { }} /> +
+ +
-
- {/* */} -
+ {cardVO.repository && + cardVO.repository.trim() && + cardVO.repository.startsWith('http') && ( +
+ { + e.stopPropagation(); // 阻止事件冒泡 + if ( + cardVO.repository && + cardVO.repository.trim() && + cardVO.repository.startsWith('http') + ) { + window.open(cardVO.repository, '_blank'); + } + }} + > + + +
+ )}
diff --git a/web/src/app/home/plugins/plugin-market/PluginMarketComponent.tsx b/web/src/app/home/plugins/plugin-market/PluginMarketComponent.tsx index 3db0b290..ac4a3297 100644 --- a/web/src/app/home/plugins/plugin-market/PluginMarketComponent.tsx +++ b/web/src/app/home/plugins/plugin-market/PluginMarketComponent.tsx @@ -39,7 +39,7 @@ export default function PluginMarketComponent({ const [sortByValue, setSortByValue] = useState('pushed_at'); const [sortOrderValue, setSortOrderValue] = useState('DESC'); const searchTimeout = useRef(null); - const pageSize = 10; + const pageSize = 12; useEffect(() => { initData(); diff --git a/web/src/app/infra/entities/api/index.ts b/web/src/app/infra/entities/api/index.ts index d86a8be0..76d0a1f6 100644 --- a/web/src/app/infra/entities/api/index.ts +++ b/web/src/app/infra/entities/api/index.ts @@ -55,6 +55,38 @@ export interface LLMModel { // updated_at: string; } +export interface KnowledgeBase { + uuid?: string; + name: string; + description: string; + embedding_model_uuid: string; + created_at?: string; + top_k?: number; +} + +export interface ApiRespProviderEmbeddingModels { + models: EmbeddingModel[]; +} + +export interface ApiRespProviderEmbeddingModel { + model: EmbeddingModel; +} + +export interface EmbeddingModel { + name: string; + description: string; + uuid: string; + requester: string; + requester_config: { + base_url: string; + timeout: number; + }; + extra_args?: object; + api_keys: string[]; + // created_at: string; + // updated_at: string; +} + export interface ApiRespPipelines { pipelines: Pipeline[]; } @@ -110,6 +142,33 @@ export interface Bot { updated_at?: string; } +export interface ApiRespKnowledgeBases { + bases: KnowledgeBase[]; +} + +export interface ApiRespKnowledgeBase { + base: KnowledgeBase; +} + +export interface KnowledgeBase { + uuid?: string; + name: string; + description: string; + embedding_model_uuid: string; + created_at?: string; + updated_at?: string; +} + +export interface ApiRespKnowledgeBaseFiles { + files: KnowledgeBaseFile[]; +} + +export interface KnowledgeBaseFile { + uuid: string; + file_name: string; + status: string; +} + // plugins export interface ApiRespPlugins { plugins: Plugin[]; @@ -237,3 +296,18 @@ export interface ApiRespWebChatMessage { export interface ApiRespWebChatMessages { messages: Message[]; } + +export interface RetrieveResult { + id: string; + metadata: { + file_id: string; + text: string; + uuid: string; + [key: string]: unknown; + }; + distance: number; +} + +export interface ApiRespKnowledgeBaseRetrieve { + results: RetrieveResult[]; +} diff --git a/web/src/app/infra/entities/form/dynamic.ts b/web/src/app/infra/entities/form/dynamic.ts index 6a185c8b..6d6de096 100644 --- a/web/src/app/infra/entities/form/dynamic.ts +++ b/web/src/app/infra/entities/form/dynamic.ts @@ -21,6 +21,7 @@ export enum DynamicFormItemType { LLM_MODEL_SELECTOR = 'llm-model-selector', PROMPT_EDITOR = 'prompt-editor', UNKNOWN = 'unknown', + KNOWLEDGE_BASE_SELECTOR = 'knowledge-base-selector', } export interface IDynamicFormItemOption { diff --git a/web/src/app/infra/http/HttpClient.ts b/web/src/app/infra/http/HttpClient.ts index 0a8aa36c..f6ff6a50 100644 --- a/web/src/app/infra/http/HttpClient.ts +++ b/web/src/app/infra/http/HttpClient.ts @@ -10,6 +10,9 @@ import { ApiRespProviderLLMModels, ApiRespProviderLLMModel, LLMModel, + ApiRespProviderEmbeddingModels, + ApiRespProviderEmbeddingModel, + EmbeddingModel, ApiRespPipelines, Pipeline, ApiRespPlatformAdapters, @@ -31,6 +34,11 @@ import { AsyncTask, ApiRespWebChatMessage, ApiRespWebChatMessages, + ApiRespKnowledgeBases, + ApiRespKnowledgeBase, + KnowledgeBase, + ApiRespKnowledgeBaseFiles, + ApiRespKnowledgeBaseRetrieve, } from '@/app/infra/entities/api'; import { GetBotLogsRequest } from '@/app/infra/http/requestParam/bots/GetBotLogsRequest'; import { GetBotLogsResponse } from '@/app/infra/http/requestParam/bots/GetBotLogsResponse'; @@ -226,8 +234,10 @@ class HttpClient { // real api request implementation // ============ Provider API ============ - public getProviderRequesters(): Promise { - return this.get('/api/v1/provider/requesters'); + public getProviderRequesters( + model_type: string, + ): Promise { + return this.get('/api/v1/provider/requesters', { type: model_type }); } public getProviderRequester(name: string): Promise { @@ -275,14 +285,54 @@ class HttpClient { return this.post(`/api/v1/provider/models/llm/${uuid}/test`, model); } + // ============ Provider Model Embedding ============ + public getProviderEmbeddingModels(): Promise { + return this.get('/api/v1/provider/models/embedding'); + } + + public getProviderEmbeddingModel( + uuid: string, + ): Promise { + return this.get(`/api/v1/provider/models/embedding/${uuid}`); + } + + public createProviderEmbeddingModel(model: EmbeddingModel): Promise { + return this.post('/api/v1/provider/models/embedding', model); + } + + public deleteProviderEmbeddingModel(uuid: string): Promise { + return this.delete(`/api/v1/provider/models/embedding/${uuid}`); + } + + public updateProviderEmbeddingModel( + uuid: string, + model: EmbeddingModel, + ): Promise { + return this.put(`/api/v1/provider/models/embedding/${uuid}`, model); + } + + public testEmbeddingModel( + uuid: string, + model: EmbeddingModel, + ): Promise { + return this.post(`/api/v1/provider/models/embedding/${uuid}/test`, model); + } + // ============ Pipeline API ============ public getGeneralPipelineMetadata(): Promise { // as designed, this method will be deprecated, and only for developer to check the prefered config schema return this.get('/api/v1/pipelines/_/metadata'); } - public getPipelines(): Promise { - return this.get('/api/v1/pipelines'); + public getPipelines( + sortBy?: string, + sortOrder?: string, + ): Promise { + const params = new URLSearchParams(); + if (sortBy) params.append('sort_by', sortBy); + if (sortOrder) params.append('sort_order', sortOrder); + const queryString = params.toString(); + return this.get(`/api/v1/pipelines${queryString ? `?${queryString}` : ''}`); } public getPipeline(uuid: string): Promise { @@ -479,6 +529,74 @@ class HttpClient { return this.post(`/api/v1/platform/bots/${botId}/logs`, request); } + // ============ File management API ============ + public uploadDocumentFile(file: File): Promise<{ file_id: string }> { + const formData = new FormData(); + formData.append('file', file); + + return this.request<{ file_id: string }>({ + method: 'post', + url: '/api/v1/files/documents', + data: formData, + headers: { + 'Content-Type': 'multipart/form-data', + }, + }); + } + + // ============ Knowledge Base API ============ + public getKnowledgeBases(): Promise { + return this.get('/api/v1/knowledge/bases'); + } + + public getKnowledgeBase(uuid: string): Promise { + return this.get(`/api/v1/knowledge/bases/${uuid}`); + } + + public createKnowledgeBase(base: KnowledgeBase): Promise<{ uuid: string }> { + return this.post('/api/v1/knowledge/bases', base); + } + + public updateKnowledgeBase( + uuid: string, + base: KnowledgeBase, + ): Promise<{ uuid: string }> { + return this.put(`/api/v1/knowledge/bases/${uuid}`, base); + } + + public uploadKnowledgeBaseFile( + uuid: string, + file_id: string, + ): Promise { + return this.post(`/api/v1/knowledge/bases/${uuid}/files`, { + file_id, + }); + } + + public getKnowledgeBaseFiles( + uuid: string, + ): Promise { + return this.get(`/api/v1/knowledge/bases/${uuid}/files`); + } + + public deleteKnowledgeBaseFile( + uuid: string, + file_id: string, + ): Promise { + return this.delete(`/api/v1/knowledge/bases/${uuid}/files/${file_id}`); + } + + public deleteKnowledgeBase(uuid: string): Promise { + return this.delete(`/api/v1/knowledge/bases/${uuid}`); + } + + public retrieveKnowledgeBase( + uuid: string, + query: string, + ): Promise { + return this.post(`/api/v1/knowledge/bases/${uuid}/retrieve`, { query }); + } + // ============ Plugins API ============ public getPlugins(): Promise { return this.get('/api/v1/plugins'); @@ -582,6 +700,18 @@ class HttpClient { public checkUserToken(): Promise { return this.get('/api/v1/user/check-token'); } + + public resetPassword( + user: string, + recoveryKey: string, + newPassword: string, + ): Promise<{ user: string }> { + return this.post('/api/v1/user/reset-password', { + user, + recovery_key: recoveryKey, + new_password: newPassword, + }); + } } const getBaseURL = (): string => { diff --git a/web/src/app/login/page.tsx b/web/src/app/login/page.tsx index 9d4b3a17..3a5e22e3 100644 --- a/web/src/app/login/page.tsx +++ b/web/src/app/login/page.tsx @@ -34,6 +34,7 @@ import langbotIcon from '@/app/assets/langbot-logo.webp'; import { toast } from 'sonner'; import { useTranslation } from 'react-i18next'; import i18n from '@/i18n'; +import Link from 'next/link'; const formSchema = (t: (key: string) => string) => z.object({ @@ -64,6 +65,9 @@ export default function Login() { if (i18n.language === 'zh-CN' || i18n.language === 'zh-Hans') { setCurrentLanguage('zh-Hans'); localStorage.setItem('langbot_language', 'zh-Hans'); + } else if (i18n.language === 'zh-TW' || i18n.language === 'zh-Hant') { + setCurrentLanguage('zh-Hant'); + localStorage.setItem('langbot_language', 'zh-Hant'); } else if (i18n.language === 'ja' || i18n.language === 'ja-JP') { setCurrentLanguage('ja-JP'); localStorage.setItem('langbot_language', 'ja-JP'); @@ -83,6 +87,8 @@ export default function Login() { let lang = 'zh-Hans'; if (language === 'zh-CN') { lang = 'zh-Hans'; + } else if (language === 'zh-TW') { + lang = 'zh-Hant'; } else if (language === 'ja' || language === 'ja-JP') { lang = 'ja-JP'; } else { @@ -163,6 +169,7 @@ export default function Login() { 简体中文 + 繁體中文 English 日本語 @@ -209,7 +216,16 @@ export default function Login() { name="password" render={({ field }) => ( - {t('common.password')} +
+ {t('common.password')} + + {t('common.forgotPassword')} + +
+
diff --git a/web/src/app/register/page.tsx b/web/src/app/register/page.tsx index d885f3b0..56ae7e9c 100644 --- a/web/src/app/register/page.tsx +++ b/web/src/app/register/page.tsx @@ -63,6 +63,9 @@ export default function Register() { if (i18n.language === 'zh-CN' || i18n.language === 'zh-Hans') { setCurrentLanguage('zh-Hans'); localStorage.setItem('langbot_language', 'zh-Hans'); + } else if (i18n.language === 'zh-TW' || i18n.language === 'zh-Hant') { + setCurrentLanguage('zh-Hant'); + localStorage.setItem('langbot_language', 'zh-Hant'); } else if (i18n.language === 'ja' || i18n.language === 'ja-JP') { setCurrentLanguage('ja-JP'); localStorage.setItem('langbot_language', 'ja-JP'); @@ -82,6 +85,8 @@ export default function Register() { let lang = 'zh-Hans'; if (language === 'zh-CN') { lang = 'zh-Hans'; + } else if (language === 'zh-TW') { + lang = 'zh-Hant'; } else if (language === 'ja' || language === 'ja-JP') { lang = 'ja-JP'; } else { @@ -148,6 +153,7 @@ export default function Register() { 简体中文 + 繁體中文 English 日本語 diff --git a/web/src/app/reset-password/layout.tsx b/web/src/app/reset-password/layout.tsx new file mode 100644 index 00000000..5db7817e --- /dev/null +++ b/web/src/app/reset-password/layout.tsx @@ -0,0 +1,15 @@ +'use client'; + +import React from 'react'; + +export default function ResetPasswordLayout({ + children, +}: Readonly<{ + children: React.ReactNode; +}>) { + return ( +
+
{children}
+
+ ); +} diff --git a/web/src/app/reset-password/page.tsx b/web/src/app/reset-password/page.tsx new file mode 100644 index 00000000..30671595 --- /dev/null +++ b/web/src/app/reset-password/page.tsx @@ -0,0 +1,205 @@ +'use client'; +import { Button } from '@/components/ui/button'; +import { Input } from '@/components/ui/input'; +import { + Card, + CardContent, + CardHeader, + CardTitle, + CardDescription, +} from '@/components/ui/card'; +import { + InputOTP, + InputOTPGroup, + InputOTPSlot, + InputOTPSeparator, +} from '@/components/ui/input-otp'; +import { useForm } from 'react-hook-form'; +import { zodResolver } from '@hookform/resolvers/zod'; +import * as z from 'zod'; +import { + Form, + FormControl, + FormField, + FormItem, + FormLabel, + FormMessage, + FormDescription, +} from '@/components/ui/form'; +import { useState } from 'react'; +import { httpClient } from '@/app/infra/http/HttpClient'; +import { useRouter } from 'next/navigation'; +import { Mail, Lock, ArrowLeft } from 'lucide-react'; +import { toast } from 'sonner'; +import { useTranslation } from 'react-i18next'; +import Link from 'next/link'; + +const REGEXP_ONLY_DIGITS_AND_CHARS = /^[0-9a-zA-Z]+$/; + +const formSchema = (t: (key: string) => string) => + z.object({ + email: z.string().email(t('common.invalidEmail')), + recoveryKey: z.string().min(1, t('resetPassword.recoveryKeyRequired')), + newPassword: z.string().min(1, t('resetPassword.newPasswordRequired')), + }); + +export default function ResetPassword() { + const router = useRouter(); + const { t } = useTranslation(); + const [isResetting, setIsResetting] = useState(false); + + const form = useForm>>({ + resolver: zodResolver(formSchema(t)), + defaultValues: { + email: '', + recoveryKey: '', + newPassword: '', + }, + }); + + function onSubmit(values: z.infer>) { + handleResetPassword(values.email, values.recoveryKey, values.newPassword); + } + + function handleResetPassword( + email: string, + recoveryKey: string, + newPassword: string, + ) { + setIsResetting(true); + httpClient + .resetPassword(email, recoveryKey, newPassword) + .then((res) => { + console.log('reset password success: ', res); + toast.success(t('resetPassword.resetSuccess')); + router.push('/login'); + }) + .catch((err) => { + console.log('reset password error: ', err); + toast.error(t('resetPassword.resetFailed')); + }) + .finally(() => { + setIsResetting(false); + }); + } + + return ( +
+ + +
+ + + {t('resetPassword.backToLogin')} + +
+ + {t('resetPassword.title')} + + + {t('resetPassword.description')} + +
+ +
+ + ( + + {t('common.email')} + +
+ + +
+
+ +
+ )} + /> + + ( + + {t('resetPassword.recoveryKey')} + + {t('resetPassword.recoveryKeyDescription')} + + + { + // 将输入的值转换为大写 + const upperValue = value.toUpperCase(); + field.onChange(upperValue); + }} + > + + + + + + + + + + + + + + + + )} + /> + + ( + + {t('resetPassword.newPassword')} + +
+ + +
+
+ +
+ )} + /> + + + + +
+
+
+ ); +} diff --git a/web/src/components/ui/dropdown-menu.tsx b/web/src/components/ui/dropdown-menu.tsx new file mode 100644 index 00000000..26027549 --- /dev/null +++ b/web/src/components/ui/dropdown-menu.tsx @@ -0,0 +1,257 @@ +'use client'; + +import * as React from 'react'; +import * as DropdownMenuPrimitive from '@radix-ui/react-dropdown-menu'; +import { CheckIcon, ChevronRightIcon, CircleIcon } from 'lucide-react'; + +import { cn } from '@/lib/utils'; + +function DropdownMenu({ + ...props +}: React.ComponentProps) { + return ; +} + +function DropdownMenuPortal({ + ...props +}: React.ComponentProps) { + return ( + + ); +} + +function DropdownMenuTrigger({ + ...props +}: React.ComponentProps) { + return ( + + ); +} + +function DropdownMenuContent({ + className, + sideOffset = 4, + ...props +}: React.ComponentProps) { + return ( + + + + ); +} + +function DropdownMenuGroup({ + ...props +}: React.ComponentProps) { + return ( + + ); +} + +function DropdownMenuItem({ + className, + inset, + variant = 'default', + ...props +}: React.ComponentProps & { + inset?: boolean; + variant?: 'default' | 'destructive'; +}) { + return ( + + ); +} + +function DropdownMenuCheckboxItem({ + className, + children, + checked, + ...props +}: React.ComponentProps) { + return ( + + + + + + + {children} + + ); +} + +function DropdownMenuRadioGroup({ + ...props +}: React.ComponentProps) { + return ( + + ); +} + +function DropdownMenuRadioItem({ + className, + children, + ...props +}: React.ComponentProps) { + return ( + + + + + + + {children} + + ); +} + +function DropdownMenuLabel({ + className, + inset, + ...props +}: React.ComponentProps & { + inset?: boolean; +}) { + return ( + + ); +} + +function DropdownMenuSeparator({ + className, + ...props +}: React.ComponentProps) { + return ( + + ); +} + +function DropdownMenuShortcut({ + className, + ...props +}: React.ComponentProps<'span'>) { + return ( + + ); +} + +function DropdownMenuSub({ + ...props +}: React.ComponentProps) { + return ; +} + +function DropdownMenuSubTrigger({ + className, + inset, + children, + ...props +}: React.ComponentProps & { + inset?: boolean; +}) { + return ( + + {children} + + + ); +} + +function DropdownMenuSubContent({ + className, + ...props +}: React.ComponentProps) { + return ( + + ); +} + +export { + DropdownMenu, + DropdownMenuPortal, + DropdownMenuTrigger, + DropdownMenuContent, + DropdownMenuGroup, + DropdownMenuLabel, + DropdownMenuItem, + DropdownMenuCheckboxItem, + DropdownMenuRadioGroup, + DropdownMenuRadioItem, + DropdownMenuSeparator, + DropdownMenuShortcut, + DropdownMenuSub, + DropdownMenuSubTrigger, + DropdownMenuSubContent, +}; diff --git a/web/src/components/ui/input-otp.tsx b/web/src/components/ui/input-otp.tsx new file mode 100644 index 00000000..26c5f7af --- /dev/null +++ b/web/src/components/ui/input-otp.tsx @@ -0,0 +1,77 @@ +'use client'; + +import * as React from 'react'; +import { OTPInput, OTPInputContext } from 'input-otp'; +import { MinusIcon } from 'lucide-react'; + +import { cn } from '@/lib/utils'; + +function InputOTP({ + className, + containerClassName, + ...props +}: React.ComponentProps & { + containerClassName?: string; +}) { + return ( + + ); +} + +function InputOTPGroup({ className, ...props }: React.ComponentProps<'div'>) { + return ( +
+ ); +} + +function InputOTPSlot({ + index, + className, + ...props +}: React.ComponentProps<'div'> & { + index: number; +}) { + const inputOTPContext = React.useContext(OTPInputContext); + const { char, hasFakeCaret, isActive } = inputOTPContext?.slots[index] ?? {}; + + return ( +
+ {char} + {hasFakeCaret && ( +
+
+
+ )} +
+ ); +} + +function InputOTPSeparator({ ...props }: React.ComponentProps<'div'>) { + return ( +
+ +
+ ); +} + +export { InputOTP, InputOTPGroup, InputOTPSlot, InputOTPSeparator }; diff --git a/web/src/components/ui/table.tsx b/web/src/components/ui/table.tsx new file mode 100644 index 00000000..ebded8ed --- /dev/null +++ b/web/src/components/ui/table.tsx @@ -0,0 +1,116 @@ +'use client'; + +import * as React from 'react'; + +import { cn } from '@/lib/utils'; + +function Table({ className, ...props }: React.ComponentProps<'table'>) { + return ( +
+ + + ); +} + +function TableHeader({ className, ...props }: React.ComponentProps<'thead'>) { + return ( + + ); +} + +function TableBody({ className, ...props }: React.ComponentProps<'tbody'>) { + return ( + + ); +} + +function TableFooter({ className, ...props }: React.ComponentProps<'tfoot'>) { + return ( + tr]:last:border-b-0', + className, + )} + {...props} + /> + ); +} + +function TableRow({ className, ...props }: React.ComponentProps<'tr'>) { + return ( + + ); +} + +function TableHead({ className, ...props }: React.ComponentProps<'th'>) { + return ( +
[role=checkbox]]:translate-y-[2px]', + className, + )} + {...props} + /> + ); +} + +function TableCell({ className, ...props }: React.ComponentProps<'td'>) { + return ( + [role=checkbox]]:translate-y-[2px]', + className, + )} + {...props} + /> + ); +} + +function TableCaption({ + className, + ...props +}: React.ComponentProps<'caption'>) { + return ( +
+ ); +} + +export { + Table, + TableHeader, + TableBody, + TableFooter, + TableHead, + TableRow, + TableCell, + TableCaption, +}; diff --git a/web/src/i18n/index.ts b/web/src/i18n/index.ts index fd8f5c7c..74995e00 100644 --- a/web/src/i18n/index.ts +++ b/web/src/i18n/index.ts @@ -6,6 +6,7 @@ import LanguageDetector from 'i18next-browser-languagedetector'; import enUS from './locales/en-US'; import zhHans from './locales/zh-Hans'; +import zhHant from './locales/zh-Hant'; import jaJP from './locales/ja-JP'; i18n @@ -19,6 +20,9 @@ i18n 'zh-Hans': { translation: zhHans, }, + 'zh-Hant': { + translation: zhHant, + }, 'ja-JP': { translation: jaJP, }, diff --git a/web/src/i18n/locales/en-US.ts b/web/src/i18n/locales/en-US.ts index dd276519..2da16025 100644 --- a/web/src/i18n/locales/en-US.ts +++ b/web/src/i18n/locales/en-US.ts @@ -39,6 +39,8 @@ const enUS = { addRound: 'Add Round', copySuccess: 'Copy Successfully', test: 'Test', + forgotPassword: 'Forgot Password?', + loading: 'Loading...', }, notFound: { title: 'Page not found', @@ -85,14 +87,13 @@ const enUS = { string: 'String', number: 'Number', boolean: 'Boolean', - extraParametersDescription: - 'Will be attached to the request body, such as max_tokens, temperature, top_p, etc.', selectModelProvider: 'Select Model Provider', modelProviderDescription: 'Please fill in the model name provided by the supplier', selectModel: 'Select Model', testSuccess: 'Test successful', testError: 'Test failed, please check your model configuration', + llmModels: 'LLM Models', }, bots: { title: 'Bots', @@ -194,6 +195,10 @@ const enUS = { today: 'Today', updateTime: 'Updated ', defaultBadge: 'Default', + sortBy: 'Sort by', + newestCreated: 'Newest Created', + recentlyEdited: 'Recently Edited', + earliestEdited: 'Earliest Edited', basicInfo: 'Basic', aiCapabilities: 'AI', triggerConditions: 'Trigger', @@ -231,6 +236,64 @@ const enUS = { streaming: 'Streaming', }, }, + knowledge: { + title: 'Knowledge', + createKnowledgeBase: 'Create Knowledge Base', + editKnowledgeBase: 'Edit Knowledge Base', + selectKnowledgeBase: 'Select Knowledge Base', + empty: 'Empty', + editDocument: 'Documents', + description: 'Configuring knowledge bases for improved LLM responses', + metadata: 'Metadata', + documents: 'Documents', + kbNameRequired: 'Knowledge base name cannot be empty', + kbDescriptionRequired: 'Knowledge base description cannot be empty', + embeddingModelUUIDRequired: 'Embedding model cannot be empty', + daysAgo: 'days ago', + today: 'Today', + kbName: 'Knowledge Base Name', + kbDescription: 'Knowledge Base Description', + defaultDescription: 'A knowledge base', + embeddingModelUUID: 'Embedding Model', + selectEmbeddingModel: 'Select Embedding Model', + embeddingModelDescription: + 'Used to vectorize the text, you can configure it in the Models page', + updateTime: 'Updated ', + cannotChangeEmbeddingModel: + 'Knowledge base created cannot be modified embedding model', + updateKnowledgeBaseSuccess: 'Knowledge base updated successfully', + updateKnowledgeBaseFailed: 'Knowledge base update failed', + documentsTab: { + name: 'Name', + status: 'Status', + noResults: 'No documents', + dragAndDrop: 'Drag and drop files here or click to upload', + uploading: 'Uploading...', + supportedFormats: + 'Supports PDF, Word, TXT, Markdown and other document formats', + uploadSuccess: 'File uploaded successfully!', + uploadError: 'File upload failed, please try again', + uploadingFile: 'Uploading file...', + actions: 'Actions', + delete: 'Delete File', + fileDeleteSuccess: 'File deleted successfully', + fileDeleteFailed: 'File deletion failed', + processing: 'Processing', + completed: 'Completed', + failed: 'Failed', + }, + deleteKnowledgeBaseConfirmation: + 'Are you sure you want to delete this knowledge base? All documents in this knowledge base will be deleted.', + retrieve: 'Retrieve Test', + retrieveTest: 'Retrieve Test', + query: 'Query', + queryPlaceholder: 'Enter query text...', + distance: 'Distance', + content: 'Content', + fileName: 'File Name', + noResults: 'No results', + retrieveError: 'Retrieve failed', + }, register: { title: 'Initialize LangBot 👋', description: 'This is your first time starting LangBot', @@ -240,6 +303,40 @@ const enUS = { initSuccess: 'Initialization successful, please login', initFailed: 'Initialization failed: ', }, + resetPassword: { + title: 'Reset Password 🔐', + description: + 'Enter your recovery key and new password to reset your account password', + recoveryKey: 'Recovery Key', + recoveryKeyDescription: + 'Stored in `system.recovery_key` of config file `data/config.yaml`', + newPassword: 'New Password', + enterRecoveryKey: 'Enter recovery key', + enterNewPassword: 'Enter new password', + recoveryKeyRequired: 'Recovery key cannot be empty', + newPasswordRequired: 'New password cannot be empty', + resetPassword: 'Reset Password', + resetting: 'Resetting...', + resetSuccess: 'Password reset successfully, please login', + resetFailed: + 'Password reset failed, please check your email and recovery key', + backToLogin: 'Back to Login', + }, + embedding: { + description: 'Manage Embedding models for text vectorization', + createModel: 'Create Embedding Model', + editModel: 'Edit Embedding Model', + getModelListError: 'Failed to get Embedding model list: ', + embeddingModels: 'Embedding', + extraParametersDescription: + 'Will be attached to the request body, such as encoding_format, dimensions, etc.', + }, + llm: { + description: 'Manage LLM models for conversation generation', + llmModels: 'LLM', + extraParametersDescription: + 'Will be attached to the request body, such as max_tokens, temperature, top_p, etc.', + }, }; export default enUS; diff --git a/web/src/i18n/locales/ja-JP.ts b/web/src/i18n/locales/ja-JP.ts index 9a600d3a..03ec8398 100644 --- a/web/src/i18n/locales/ja-JP.ts +++ b/web/src/i18n/locales/ja-JP.ts @@ -40,6 +40,8 @@ const jaJP = { addRound: 'ラウンドを追加', copySuccess: 'コピーに成功しました', test: 'テスト', + forgotPassword: 'パスワードを忘れた?', + loading: '読み込み中...', }, notFound: { title: 'ページが見つかりません', @@ -194,6 +196,10 @@ const jaJP = { today: '今日', updateTime: '更新日時', defaultBadge: 'デフォルト', + sortBy: '並び順', + newestCreated: '最新作成', + recentlyEdited: '最近編集', + earliestEdited: '最古編集', basicInfo: '基本情報', aiCapabilities: 'AI機能', triggerConditions: 'トリガー条件', @@ -232,6 +238,65 @@ const jaJP = { streaming: 'ストリーミング', }, }, + knowledge: { + title: '知識ベース', + createKnowledgeBase: '知識ベースを作成', + editKnowledgeBase: '知識ベースを編集', + selectKnowledgeBase: '知識ベースを選択', + empty: 'なし', + editDocument: 'ドキュメント', + description: 'LLMの回答品質向上のための知識ベースを設定します', + metadata: 'メタデータ', + documents: 'ドキュメント', + kbNameRequired: '知識ベース名は必須です', + kbDescriptionRequired: '知識ベースの説明は必須です', + embeddingModelUUIDRequired: '埋め込みモデルは必須です', + daysAgo: '日前', + today: '今日', + kbName: '知識ベース名', + kbDescription: '知識ベースの説明', + defaultDescription: '知識ベース', + embeddingModelUUID: '埋め込みモデル', + selectEmbeddingModel: '埋め込みモデルを選択', + embeddingModelDescription: + 'テキストのベクトル化に使用する埋め込みモデルを管理します', + updateTime: '更新日時', + cannotChangeEmbeddingModel: + '知識ベース作成後は埋め込みモデルを変更できません', + updateKnowledgeBaseSuccess: '知識ベースの更新に成功しました', + updateKnowledgeBaseFailed: '知識ベースの更新に失敗しました', + documentsTab: { + name: '名前', + status: 'ステータス', + noResults: 'ドキュメントがありません', + dragAndDrop: + 'ファイルをここにドラッグ&ドロップするか、クリックしてアップロードしてください', + uploading: 'アップロード中...', + supportedFormats: + 'PDF、Word、TXT、Markdownなどのドキュメントファイルをサポートしています', + uploadSuccess: 'ファイルのアップロードに成功しました!', + uploadError: 'ファイルのアップロードに失敗しました。再度お試しください', + uploadingFile: 'ファイルをアップロード中...', + actions: 'アクション', + delete: 'ドキュメントを削除', + fileDeleteSuccess: 'ドキュメントの削除に成功しました', + fileDeleteFailed: 'ドキュメントの削除に失敗しました', + processing: '処理中', + completed: '完了', + failed: '失敗', + }, + deleteKnowledgeBaseConfirmation: + '本当にこの知識ベースを削除しますか?この知識ベースに紐付けられたドキュメントは削除されます。', + retrieve: '検索テスト', + retrieveTest: '検索テスト', + query: '検索', + queryPlaceholder: '検索内容を入力...', + distance: '距離', + content: '内容', + fileName: 'ファイル名', + noResults: '検索結果がありません', + retrieveError: '検索に失敗しました', + }, register: { title: 'LangBot を初期化 👋', description: 'これはLangBotの初回起動です', @@ -241,6 +306,40 @@ const jaJP = { initSuccess: '初期化に成功しました。ログインしてください', initFailed: '初期化に失敗しました:', }, + resetPassword: { + title: 'パスワードをリセット 🔐', + description: + '復旧キーと新しいパスワードを入力して、アカウントのパスワードをリセットします', + recoveryKey: '復旧キー', + recoveryKeyDescription: + '設定ファイル `data/config.yaml` の `system.recovery_key` に保存されています', + newPassword: '新しいパスワード', + enterRecoveryKey: '復旧キーを入力', + enterNewPassword: '新しいパスワードを入力', + recoveryKeyRequired: '復旧キーは必須です', + newPasswordRequired: '新しいパスワードは必須です', + resetPassword: 'パスワードをリセット', + resetting: 'リセット中...', + resetSuccess: 'パスワードのリセットに成功しました。ログインしてください', + resetFailed: + 'パスワードのリセットに失敗しました。メールアドレスと復旧キーを確認してください', + backToLogin: 'ログインに戻る', + }, + embedding: { + description: 'テキストのベクトル化に使用する埋め込みモデルを管理します', + createModel: '埋め込みモデルを作成', + editModel: '埋め込みモデルを編集', + getModelListError: '埋め込みモデルリストの取得に失敗しました:', + embeddingModels: '埋め込みモデル', + extraParametersDescription: + 'リクエストボディに追加されるパラメータ(encoding_format、dimensions など)', + }, + llm: { + description: 'チャットメッセージの生成に使用するLLMモデルを管理します', + llmModels: 'LLMモデル', + extraParametersDescription: + 'リクエストボディに追加されるパラメータ(max_tokens、temperature、top_p など)', + }, }; export default jaJP; diff --git a/web/src/i18n/locales/zh-Hans.ts b/web/src/i18n/locales/zh-Hans.ts index 62eb6563..cb156e46 100644 --- a/web/src/i18n/locales/zh-Hans.ts +++ b/web/src/i18n/locales/zh-Hans.ts @@ -39,6 +39,8 @@ const zhHans = { addRound: '添加回合', copySuccess: '复制成功', test: '测试', + forgotPassword: '忘记密码?', + loading: '加载中...', }, notFound: { title: '页面不存在', @@ -86,13 +88,12 @@ const zhHans = { string: '字符串', number: '数字', boolean: '布尔值', - extraParametersDescription: - '将在请求时附加到请求体中,如 max_tokens, temperature, top_p 等', selectModelProvider: '选择模型供应商', modelProviderDescription: '请填写供应商向您提供的模型名称', selectModel: '请选择模型', testSuccess: '测试成功', testError: '测试失败,请检查模型配置', + llmModels: '对话模型', }, bots: { title: '机器人', @@ -189,6 +190,10 @@ const zhHans = { today: '今天', updateTime: '更新于', defaultBadge: '默认', + sortBy: '排序方式', + newestCreated: '最新创建', + recentlyEdited: '最近编辑', + earliestEdited: '最早编辑', basicInfo: '基础信息', aiCapabilities: 'AI 能力', triggerConditions: '触发条件', @@ -226,6 +231,61 @@ const zhHans = { streaming: '流式传输', }, }, + knowledge: { + title: '知识库', + createKnowledgeBase: '创建知识库', + editKnowledgeBase: '编辑知识库', + selectKnowledgeBase: '选择知识库', + empty: '无', + editDocument: '文档', + description: '配置可用于提升模型回复质量的知识库', + metadata: '元数据', + documents: '文档', + kbNameRequired: '知识库名称不能为空', + kbDescriptionRequired: '知识库描述不能为空', + embeddingModelUUIDRequired: '嵌入模型不能为空', + daysAgo: '天前', + today: '今天', + kbName: '知识库名称', + kbDescription: '知识库描述', + defaultDescription: '一个知识库', + embeddingModelUUID: '嵌入模型', + selectEmbeddingModel: '选择嵌入模型', + embeddingModelDescription: '用于向量化文本,可在模型配置页面配置', + updateTime: '更新于', + cannotChangeEmbeddingModel: '知识库创建后不可修改嵌入模型', + updateKnowledgeBaseSuccess: '知识库更新成功', + updateKnowledgeBaseFailed: '知识库更新失败', + documentsTab: { + name: '名称', + status: '状态', + noResults: '暂无文档', + dragAndDrop: '拖拽文件到此处或点击上传', + uploading: '上传中...', + supportedFormats: '支持 PDF、Word、TXT、Markdown 等文档格式', + uploadSuccess: '文件上传成功!', + uploadError: '文件上传失败,请重试', + uploadingFile: '上传文件中...', + actions: '操作', + delete: '删除文件', + fileDeleteSuccess: '文件删除成功', + fileDeleteFailed: '文件删除失败', + processing: '处理中', + completed: '完成', + failed: '失败', + }, + deleteKnowledgeBaseConfirmation: + '你确定要删除这个知识库吗?此知识库下的所有文档将被删除。', + retrieve: '检索测试', + retrieveTest: '检索测试', + query: '查询', + queryPlaceholder: '输入查询内容...', + distance: '距离', + content: '内容', + fileName: '文件名', + noResults: '暂无结果', + retrieveError: '检索失败', + }, register: { title: '初始化 LangBot 👋', description: '这是您首次启动 LangBot', @@ -234,6 +294,38 @@ const zhHans = { initSuccess: '初始化成功 请登录', initFailed: '初始化失败:', }, + resetPassword: { + title: '重置密码 🔐', + description: '输入恢复密钥和新的密码来重置您的账户密码', + recoveryKey: '恢复密钥', + recoveryKeyDescription: + '存储在配置文件`data/config.yaml`的`system.recovery_key`中', + newPassword: '新密码', + enterRecoveryKey: '输入恢复密钥', + enterNewPassword: '输入新密码', + recoveryKeyRequired: '恢复密钥不能为空', + newPasswordRequired: '新密码不能为空', + resetPassword: '重置密码', + resetting: '重置中...', + resetSuccess: '密码重置成功,请登录', + resetFailed: '密码重置失败,请检查邮箱和恢复密钥是否正确', + backToLogin: '返回登录', + }, + embedding: { + description: '管理嵌入模型,用于向量化文本', + createModel: '创建嵌入模型', + editModel: '编辑嵌入模型', + getModelListError: '获取嵌入模型列表失败:', + embeddingModels: '嵌入模型', + extraParametersDescription: + '将在请求时附加到请求体中,如 encoding_format, dimensions 等', + }, + llm: { + llmModels: '对话模型', + description: '管理 LLM 模型,用于对话消息生成', + extraParametersDescription: + '将在请求时附加到请求体中,如 max_tokens, temperature, top_p 等', + }, }; export default zhHans; diff --git a/web/src/i18n/locales/zh-Hant.ts b/web/src/i18n/locales/zh-Hant.ts new file mode 100644 index 00000000..e71f0e53 --- /dev/null +++ b/web/src/i18n/locales/zh-Hant.ts @@ -0,0 +1,330 @@ +const zhHant = { + common: { + login: '登入', + logout: '登出', + email: '電子郵件', + password: '密碼', + welcome: '歡迎回到 LangBot 👋', + continueToLogin: '登入以繼續', + loginSuccess: '登入成功', + loginFailed: '登入失敗,請檢查電子郵件和密碼是否正確', + enterEmail: '輸入電子郵件地址', + enterPassword: '輸入密碼', + invalidEmail: '請輸入有效的電子郵件地址', + emptyPassword: '請輸入密碼', + language: '語言', + helpDocs: '輔助說明', + create: '建立', + edit: '編輯', + delete: '刪除', + add: '新增', + select: '請選擇', + cancel: '取消', + submit: '提交', + error: '錯誤', + success: '成功', + save: '儲存', + saving: '儲存中...', + confirm: '確認', + confirmDelete: '確認刪除', + deleteConfirmation: '您確定要刪除這個嗎?', + selectOption: '選擇一個選項', + required: '必填', + enable: '是否啟用', + name: '名稱', + description: '描述', + close: '關閉', + deleteSuccess: '刪除成功', + deleteError: '刪除失敗:', + addRound: '新增回合', + copySuccess: '複製成功', + test: '測試', + forgotPassword: '忘記密碼?', + loading: '載入中...', + }, + notFound: { + title: '頁面不存在', + description: + '您要查詢的頁面似乎不存在。請檢查您輸入的 URL 是否正確,或返回首頁。', + back: '上一級', + home: '返回主頁', + help: '查看說明文件', + }, + models: { + title: '模型設定', + description: '設定和管理可在流程線中使用的模型', + createModel: '建立模型', + editModel: '編輯模型', + getModelListError: '取得模型清單失敗:', + modelName: '模型名稱', + modelProvider: '模型供應商', + modelBaseURL: '基礎 URL', + modelAbilities: '模型能力', + saveSuccess: '儲存成功', + saveError: '儲存失敗:', + createSuccess: '建立成功', + createError: '建立失敗:', + deleteSuccess: '刪除成功', + deleteError: '刪除失敗:', + deleteConfirmation: '您確定要刪除這個模型嗎?', + modelNameRequired: '模型名稱不能為空', + modelProviderRequired: '模型供應商不能為空', + requestURLRequired: '請求URL不能為空', + apiKeyRequired: 'API Key不能為空', + keyNameRequired: '鍵名不能為空', + mustBeValidNumber: '必須是有效的數字', + mustBeTrueOrFalse: '必須是 true 或 false', + requestURL: '請求URL', + apiKey: 'API Key', + abilities: '能力', + selectModelAbilities: '選擇模型能力', + visionAbility: '視覺能力', + functionCallAbility: '函數呼叫', + extraParameters: '額外參數', + addParameter: '新增參數', + keyName: '鍵名', + type: '類型', + value: '值', + string: '字串', + number: '數字', + boolean: '布林值', + selectModelProvider: '選擇模型供應商', + modelProviderDescription: '請填寫供應商向您提供的模型名稱', + selectModel: '請選擇模型', + testSuccess: '測試成功', + testError: '測試失敗,請檢查模型設定', + llmModels: '對話模型', + }, + bots: { + title: '機器人', + description: '建立和管理機器人,這是 LangBot 與各個平台連接的入口', + createBot: '建立機器人', + editBot: '編輯機器人', + getBotListError: '取得機器人清單失敗:', + botName: '機器人名稱', + botDescription: '機器人描述', + botNameRequired: '機器人名稱不能為空', + botDescriptionRequired: '機器人描述不能為空', + adapterRequired: '適配器不能為空', + defaultDescription: '一個機器人', + getBotConfigError: '取得機器人設定失敗:', + saveSuccess: '儲存成功', + saveError: '儲存失敗:', + createSuccess: '建立成功 請啟用或修改綁定流程線', + createError: '建立失敗:', + deleteSuccess: '刪除成功', + deleteError: '刪除失敗:', + deleteConfirmation: '您確定要刪除這個機器人嗎?', + platformAdapter: '平台/適配器選擇', + selectAdapter: '選擇適配器', + adapterConfig: '適配器設定', + bindPipeline: '綁定流程線', + selectPipeline: '選擇流程線', + botLogTitle: '機器人日誌', + enableAutoRefresh: '開啟自動重新整理', + session: '對話', + yesterday: '昨天', + earlier: '更久之前', + dateFormat: '{{month}}月{{day}}日', + setBotEnableError: '設定機器人啟用狀態失敗', + log: '日誌', + configuration: '設定', + logs: '日誌', + }, + plugins: { + title: '外掛管理', + description: '安裝和設定用於擴展 LangBot 功能的外掛', + createPlugin: '建立外掛', + editPlugin: '編輯外掛', + installed: '已安裝', + marketplace: 'Marketplace', + arrange: '編排', + install: '安裝', + installFromGithub: '從 GitHub 安裝外掛', + onlySupportGithub: '目前僅支援從 GitHub 安裝', + enterGithubLink: '請輸入外掛的Github連結', + installing: '正在安裝外掛...', + installSuccess: '外掛安裝成功', + installFailed: '外掛安裝失敗:', + searchPlugin: '搜尋外掛', + sortBy: '排序方式', + mostStars: '最多星標', + recentlyAdded: '最近新增', + recentlyUpdated: '最近更新', + noMatchingPlugins: '沒有找到符合的外掛', + loading: '載入中...', + getPluginListError: '取得外掛清單失敗:', + pluginConfig: '外掛設定', + noPluginInstalled: '暫未安裝任何外掛', + pluginSort: '外掛排序', + pluginSortDescription: + '外掛順序會影響同一事件內的處理順序,請拖曳外掛卡片排序', + pluginSortSuccess: '外掛排序成功', + pluginSortError: '外掛排序失敗:', + pluginNoConfig: '外掛沒有設定項目。', + deleting: '刪除中...', + deletePlugin: '刪除外掛', + cancel: '取消', + saveConfig: '儲存設定', + saving: '儲存中...', + confirmDeletePlugin: '您確定要刪除外掛({{author}}/{{name}})嗎?', + confirmDelete: '確認刪除', + deleteError: '刪除失敗:', + close: '關閉', + deleteConfirm: '刪除確認', + modifyFailed: '修改失敗:', + eventCount: '事件:{{count}}', + toolCount: '工具:{{count}}', + starCount: '星標:{{count}}', + }, + pipelines: { + title: '流程線', + description: '流程線定義了對訊息事件的處理流程,用於綁定到機器人', + createPipeline: '建立流程線', + editPipeline: '編輯流程線', + chat: '對話', + configuration: '設定', + debugChat: '對話除錯', + getPipelineListError: '取得流程線清單失敗:', + daysAgo: '天前', + today: '今天', + updateTime: '更新於', + defaultBadge: '預設', + sortBy: '排序方式', + newestCreated: '最新建立', + recentlyEdited: '最近編輯', + earliestEdited: '最早編輯', + basicInfo: '基本資訊', + aiCapabilities: 'AI 能力', + triggerConditions: '觸發條件', + safetyControls: '安全控制', + outputProcessing: '輸出處理', + nameRequired: '名稱不能為空', + descriptionRequired: '描述不能為空', + createSuccess: '建立成功 請編輯流程線詳細參數', + createError: '建立失敗:', + saveSuccess: '儲存成功', + saveError: '儲存失敗:', + deleteConfirmation: + '您確定要刪除這個流程線嗎?已綁定此流程線的機器人將無法使用。', + defaultPipelineCannotDelete: '預設流程線不可刪除', + deleteSuccess: '刪除成功', + deleteError: '刪除失敗:', + debugDialog: { + title: '流程線對話', + selectPipeline: '選擇流程線', + sessionType: '對話類型', + privateChat: '私聊', + groupChat: '群聊', + send: '傳送', + reset: '重設對話', + inputPlaceholder: '傳送 {{type}} 訊息...', + noMessages: '暫無訊息', + userMessage: '使用者', + botMessage: '機器人', + sendFailed: '傳送失敗', + resetSuccess: '對話已重設', + resetFailed: '重設失敗', + loadMessagesFailed: '載入訊息失敗', + loadPipelinesFailed: '載入流程線失敗', + atTips: '提及機器人', + }, + }, + knowledge: { + title: '知識庫', + createKnowledgeBase: '建立知識庫', + editKnowledgeBase: '編輯知識庫', + selectKnowledgeBase: '選擇知識庫', + empty: '無', + editDocument: '文件', + description: '設定可用於提升模型回覆品質的知識庫', + metadata: '中繼資料', + documents: '文件', + kbNameRequired: '知識庫名稱不能為空', + kbDescriptionRequired: '知識庫描述不能為空', + embeddingModelUUIDRequired: '嵌入模型不能為空', + daysAgo: '天前', + today: '今天', + kbName: '知識庫名稱', + kbDescription: '知識庫描述', + defaultDescription: '一個知識庫', + embeddingModelUUID: '嵌入模型', + selectEmbeddingModel: '選擇嵌入模型', + embeddingModelDescription: '用於向量化文字,可在模型設定頁面設定', + updateTime: '更新於', + cannotChangeEmbeddingModel: '知識庫建立後不可修改嵌入模型', + updateKnowledgeBaseSuccess: '知識庫更新成功', + updateKnowledgeBaseFailed: '知識庫更新失敗', + documentsTab: { + name: '名稱', + status: '狀態', + noResults: '暫無文件', + dragAndDrop: '拖曳檔案到此處或點擊上傳', + uploading: '上傳中...', + supportedFormats: '支援 PDF、Word、TXT、Markdown 等文件格式', + uploadSuccess: '檔案上傳成功!', + uploadError: '檔案上傳失敗,請重試', + uploadingFile: '上傳檔案中...', + actions: '操作', + delete: '刪除檔案', + fileDeleteSuccess: '檔案刪除成功', + fileDeleteFailed: '檔案刪除失敗', + processing: '處理中', + completed: '完成', + failed: '失敗', + }, + deleteKnowledgeBaseConfirmation: + '您確定要刪除這個知識庫嗎?此知識庫下的所有文件將被刪除。', + retrieve: '檢索測試', + retrieveTest: '檢索測試', + query: '查詢', + queryPlaceholder: '輸入查詢內容...', + distance: '距離', + content: '內容', + fileName: '檔案名稱', + noResults: '暫無結果', + retrieveError: '檢索失敗', + }, + register: { + title: '初始化 LangBot 👋', + description: '這是您首次啟動 LangBot', + adminAccountNote: '您填寫的電子郵件和密碼將作為初始管理員帳號', + register: '註冊', + initSuccess: '初始化成功 請登入', + initFailed: '初始化失敗:', + }, + resetPassword: { + title: '重設密碼 🔐', + description: '輸入恢復金鑰和新的密碼來重設您的帳戶密碼', + recoveryKey: '恢復金鑰', + recoveryKeyDescription: + '儲存在設定檔案`data/config.yaml`的`system.recovery_key`中', + newPassword: '新密碼', + enterRecoveryKey: '輸入恢復金鑰', + enterNewPassword: '輸入新密碼', + recoveryKeyRequired: '恢復金鑰不能為空', + newPasswordRequired: '新密碼不能為空', + resetPassword: '重設密碼', + resetting: '重設中...', + resetSuccess: '密碼重設成功,請登入', + resetFailed: '密碼重設失敗,請檢查電子郵件和恢復金鑰是否正確', + backToLogin: '返回登入', + }, + embedding: { + description: '管理嵌入模型,用於向量化文字', + createModel: '建立嵌入模型', + editModel: '編輯嵌入模型', + getModelListError: '取得嵌入模型清單失敗:', + embeddingModels: '嵌入模型', + extraParametersDescription: + '將在請求時附加到請求體中,如 encoding_format, dimensions 等', + }, + llm: { + llmModels: '對話模型', + description: '管理 LLM 模型,用於對話訊息產生', + extraParametersDescription: + '將在請求時附加到請求體中,如 max_tokens, temperature, top_p 等', + }, +}; + +export default zhHant;