Compare commits

...

121 Commits

Author SHA1 Message Date
glay
cb8144e168 Merge branch 'ChatGPTNextWeb:main' into main 2025-02-25 10:39:14 +08:00
RiverRay
f5f3ce94f6 Update README.md
Some checks failed
Run Tests / test (push) Has been cancelled
2025-02-21 08:56:43 +08:00
RiverRay
2b5f600308 Update README.md 2025-02-21 08:55:40 +08:00
RiverRay
b966107117 Merge pull request #6235 from DBCDK/danish-locale
Some checks failed
Run Tests / test (push) Has been cancelled
Translation to danish
2025-02-17 22:58:01 +08:00
river
377480b448 Merge branch 'main' of https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web
Some checks failed
Run Tests / test (push) Has been cancelled
2025-02-16 10:50:07 +08:00
river
8bd0d6a1a7 chore: Update NextChatAI domain from nextchat.dev to nextchat.club 2025-02-16 10:48:54 +08:00
glay
00816304df Merge branch 'main' into main 2025-02-16 08:42:41 +08:00
Rasmus Erik Voel Jensen
90827fc593 danish rewording / improved button label 2025-02-15 13:08:58 +01:00
Rasmus Erik Voel Jensen
008e339b6d danish locale 2025-02-15 12:52:44 +01:00
RiverRay
12863f5213 Merge pull request #6204 from bestsanmao/ali_bytedance_reasoning_content
Some checks failed
Run Tests / test (push) Has been cancelled
add 3 type of reasoning_content support (+deepseek-r1@OpenAI @Alibaba @ByteDance), parse <think></think> from SSE
2025-02-13 14:53:47 +08:00
suruiqiang
cf140d4228 Merge branch 'main' of https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web into ali_bytedance_reasoning_content 2025-02-12 17:54:50 +08:00
suruiqiang
476d946f96 fix bug (trim eats space or \n mistakenly), optimize timeout by model 2025-02-12 17:49:54 +08:00
suruiqiang
9714258322 support deepseek-r1@OpenAI's reasoning_content, parse <think></think> from stream 2025-02-11 18:57:16 +08:00
RiverRay
48cd4b11b5 Merge pull request #6190 from siliconflow/refine-emoji-siliconflow
Some checks failed
Run Tests / test (push) Has been cancelled
Fix model icon on SiliconFlow
2025-02-11 18:37:47 +08:00
RiverRay
77c78b230a Merge pull request #6193 from siliconflow/get-models-siliconflow
Model listing of SiliconFlow
2025-02-11 18:37:22 +08:00
RiverRay
b44686b887 Merge pull request #6189 from bestsanmao/bug_fix
fix avatar for export message preview and saved image
2025-02-11 18:36:50 +08:00
RiverRay
34bdd4b945 Merge pull request #6194 from siliconflow/vl-support-on-sf
Support VLM on SiliconFlow
2025-02-11 18:35:02 +08:00
suruiqiang
b0758cccde optimization 2025-02-11 16:08:30 +08:00
suruiqiang
98a11e56d2 support alibaba and bytedance's reasoning_content 2025-02-11 12:46:46 +08:00
glay
645ecbcad8 Merge branch 'main' into main 2025-02-11 09:06:33 +08:00
Shenghang Tsai
86f86962fb Support VLM on SiliconFlow 2025-02-10 13:39:06 +08:00
Shenghang Tsai
2137aa65bf Model listing of SiliconFlow 2025-02-10 11:03:49 +08:00
Shenghang Tsai
18fa2cc30d fix model icon on siliconflow 2025-02-09 18:49:26 +08:00
Shenghang Tsai
0bfc648085 fix model icon on siliconflow 2025-02-09 18:47:57 +08:00
suruiqiang
9f91c2d05c fix avatar for export message preview and saved image 2025-02-09 16:52:46 +08:00
RiverRay
a029b4330b Merge pull request #6188 from ChatGPTNextWeb/Leizhenpeng-patch-4
Some checks failed
Run Tests / test (push) Has been cancelled
Update LICENSE
2025-02-09 11:05:43 +08:00
glay
65dc0e26d2 Merge branch 'main' into main 2025-02-06 12:25:47 +08:00
glay
81b14b7b8d Merge branch 'main' into main 2025-01-31 13:17:00 +08:00
glay
40c00374e7 Merge branch 'main' into main 2025-01-22 14:28:55 +08:00
glay
e94566d258 Merge branch 'ChatGPTNextWeb:main' into main 2025-01-14 08:02:24 +08:00
glay
6d72a04854 Merge branch 'ChatGPTNextWeb:main' into main 2025-01-01 15:53:15 +08:00
glay
b0f78e9d1c Merge branch 'main' into main 2024-12-30 09:50:42 +08:00
glay
29b9a20acf feat:add meta.llama3-3-70b-instruct model 2024-12-25 22:47:53 +08:00
glay
89b1774996 Merge branch 'ChatGPTNextWeb:main' into main 2024-12-25 09:54:30 +08:00
glay
26f79aa6e6 feat:add nova VISION_MODEL_REGEXES 2024-12-22 23:38:45 +08:00
glay
92615dae32 Merge branch 'main' into main 2024-12-22 23:28:24 +08:00
glay
9643adcf15 Merge branch 'main' into main 2024-12-21 11:20:20 +08:00
glay
e839940a26 feat:add amazon.nova model tool use support. 2024-12-14 11:01:10 +08:00
glay
0ec1ae6276 Enhance encryption security with additional safeguards. 2024-12-10 11:55:14 +08:00
glay
cb0422b8f2 Enhance processChunks by attempting to recover by processing the next chunk. 2024-12-10 11:53:49 +08:00
glay
e455840ab3 Update app/utils/aws.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-10 11:47:30 +08:00
glay
19437c7aa1 Enhance encryption security with additional safeguards. 2024-12-10 11:35:26 +08:00
glay
12d38aa4b2 Update app/utils/aws.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-09 14:43:52 +08:00
glay
2a9f7d72fc Update app/client/platforms/bedrock.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-09 14:26:37 +08:00
glay
372a327522 Update app/utils/aws.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-09 10:22:08 +08:00
glay
50a241b715 Update app/utils/aws.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-09 10:18:15 +08:00
glay
44a1cf6d6e Update app/client/platforms/bedrock.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-09 10:03:06 +08:00
glay
a0886875e1 Enhance encryption security with additional safeguards. 2024-12-09 00:03:10 +08:00
glay
93337b2b92 Update app/client/platforms/bedrock.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-08 23:58:05 +08:00
glay
4b2f447474 Enhance encryption security with additional safeguards. 2024-12-08 23:52:31 +08:00
glay
7830b37a90 Enhance encryption security with additional safeguards. 2024-12-08 23:49:59 +08:00
glay
fb3437ca01 Update app/client/platforms/bedrock.ts
增加友好提示

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-12-08 23:47:16 +08:00
glay
f5ae086d3c Enhance encryption security with additional safeguards. 2024-12-08 23:28:59 +08:00
glay
26b9fa97cd 优化代码 2024-12-08 08:47:46 +08:00
glay
603415f9e1 Enhance log security 2024-12-08 08:33:45 +08:00
glay
5ac651ad8e Enhance encryption security 2024-12-07 20:34:05 +08:00
glay
ad49cd0454 优化bedrock工具类,以更安全的处理流式消息的边界值 2024-12-07 17:54:12 +08:00
glay
57dc44a54f 增加bedrock最新nova模型,优化后台代码 2024-12-07 16:49:26 +08:00
glay
4254fd34f9 增加bedrock最新nova模型,包括image解析的支持 2024-12-07 14:20:59 +08:00
glay
0c55850641 优化bedrock流式消息处理的缓冲机制,简化app和后台api调用逻辑判断和处理 2024-12-07 12:18:15 +08:00
glay
a75b9f7fbe Merge branch 'ChatGPTNextWeb:main' into main 2024-12-04 08:22:36 +08:00
glay
d9d2a27d8f modify the BEDROCK_BASE_URL to use the region from the access store 2024-11-26 16:19:54 +08:00
glay
471b17831a modify the BEDROCK_BASE_URL to use the region from the access stor 2024-11-26 16:18:46 +08:00
glay
8ce2cf5c3b Remove detailed error error message. 2024-11-26 11:57:49 +08:00
glay
9c648e5b64 Remove detailed error error message. 2024-11-26 11:46:16 +08:00
glay
b39b3f7a5e Remove detailed error logging in decryption function. 2024-11-26 11:41:56 +08:00
glay
448babd27f 完善mistral tool use功能 2024-11-26 10:10:34 +08:00
glay
e6633753a4 完善mistral tool use功能 和llama3消息格式问题 2024-11-25 20:08:21 +08:00
glay
15d0600642 Merge branch 'ChatGPTNextWeb:main' into main 2024-11-25 19:02:50 +08:00
glay
9a473048b1 去掉Debug日志打印 2024-11-25 09:51:46 +08:00
glay
2fe848ecf3 去掉Debug日志打印 2024-11-25 09:51:02 +08:00
glay
0abfd279e4 完善bedrock中文翻译 2024-11-25 09:20:22 +08:00
glay
5bd7e28f82 去掉Debug日志打印 2024-11-25 09:12:30 +08:00
glay
6f7a635030 完善llama和mistral模型的推理功能 2024-11-24 23:54:04 +08:00
glay
2ccdd1706a 优化前后端代码,将公共方法抽取到util类,修改客户端加密方式 2024-11-24 15:37:49 +08:00
glay
a19ba6933a Merge branch 'ChatGPTNextWeb:main' into main 2024-11-24 11:52:37 +08:00
glay
513cf1b206 完善llama和mistral模型的推理功能 2024-11-23 18:23:20 +08:00
glay
238eb70986 完善mistral模型的推理结果 2024-11-23 16:27:19 +08:00
glay
a6337e9f23 完善总结功能的代码逻辑 2024-11-23 15:13:52 +08:00
glay
ff88421904 修改密钥加密逻辑 2024-11-23 13:52:14 +08:00
glay
a85db21e1f 优化代码,修改方法命名错误 2024-11-23 12:09:45 +08:00
glay
b0c1ccd0a0 优化和重构代码,增加前端可以设置加密配置数据的密钥 2024-11-22 22:03:42 +08:00
glay
bd68df1d9b 修改: app/api/bedrock.ts
修改:     app/client/platforms/bedrock.ts
	修改:     app/constant.ts
2024-11-22 06:33:39 +08:00
glay
f60c237b16 去掉sdk的引入,客户端也能直连 2024-11-20 15:25:36 +08:00
glay
9d3f1d2529 remove document function,only keep the bedrock service provider 2024-11-18 16:12:03 +08:00
glay
dfeb9e7f27 Merge branch 'ChatGPTNextWeb:main' into main 2024-11-17 08:53:59 +08:00
glay
b2d5e0e309 Merge branch 'ChatGPTNextWeb:main' into main 2024-11-13 19:42:15 +08:00
glay
225ad30898 修改: app/api/bedrock.ts
修改:     app/components/ui-lib.tsx
2024-11-13 18:37:47 +08:00
glay
6bc1612720 修改: app/locales/cn.ts
修改:     app/locales/en.ts
	修改:     yarn.lock
2024-11-13 12:33:59 +08:00
glay
24261d20b3 Consider adding more Bedrock-specific configurations 2024-11-13 08:18:20 +08:00
glay
1b5a81c7ad Add AWS secret key validation. 2024-11-13 08:10:10 +08:00
glay
70f066c15f Add AWS region validation and improve code style. 2024-11-13 08:08:01 +08:00
glay
bfa433919f Add AWS access key validation. 2024-11-13 07:57:51 +08:00
glay
afb0752d5d Merge branch 'ChatGPTNextWeb:main' into main 2024-11-13 07:52:30 +08:00
glay
f12058463b Merge branch 'main' into main 2024-11-12 08:45:28 +08:00
glay
09e4f95272 Merge branch 'main' into main 2024-11-11 15:55:20 +08:00
glay
0e09697274 Merge branch 'ChatGPTNextWeb:main' into main 2024-11-08 20:48:27 +08:00
glay
82a368a3aa 修改: .env.template 2024-11-07 08:13:24 +08:00
glay
9e04198bdc Merge branch 'main' into main 2024-11-06 23:39:54 +08:00
glay
4204890d90 Merge branch 'ChatGPTNextWeb:main' into main 2024-11-06 18:19:14 +08:00
glay
5d5456c1c5 修改: app/api/bedrock.ts
修改:     app/client/platforms/bedrock.ts
2024-11-06 17:23:53 +08:00
glay
f0c23cc6aa 修改: app/store/access.ts 2024-11-06 13:10:20 +08:00
glay
3bf55d3530 Merge branch 'main' into main 2024-11-06 10:52:34 +08:00
glay
952d8835a3 修改: app/components/settings.tsx
修改:     app/components/ui-lib.tsx
	修改:     app/constant.ts
	修改:     app/utils/encryption.ts
2024-11-06 09:44:36 +08:00
glay
c55cea5853 修改: app/store/access.ts 2024-11-06 09:09:49 +08:00
glay
ca17e90c52 修改: app/utils/encryption.ts 2024-11-06 00:43:03 +08:00
glay
cae20af24d 修改: app/api/bedrock.ts
修改:     app/client/api.ts
	修改:     app/store/access.ts
	新文件:   app/utils/encryption.ts
	修改:     package.json
2024-11-06 00:21:30 +08:00
glay
1f66d3779c 修改: app/api/bedrock.ts 2024-11-05 23:32:24 +08:00
glay
045adc3567 修改: app/api/bedrock.ts 2024-11-05 23:28:07 +08:00
glay
1998cf5ced Merge feature/update-bedrock-api into main 2024-11-05 20:20:20 +08:00
glay
1164e1bdf6 Merge feature/update-bedrock-api into main 2024-11-05 20:04:36 +08:00
glay
d55c752e1e 修改: app/api/bedrock.ts
修改:     app/utils.ts
2024-11-05 19:13:32 +08:00
glay
e3c18bb123 修改: app/api/bedrock.ts 2024-11-05 18:53:12 +08:00
glay
f532731e2a 修改: app/client/platforms/bedrock.ts 2024-11-05 17:32:52 +08:00
glay
58837f6dec 修改: app/api/bedrock.ts
修改:     app/client/platforms/bedrock.ts
2024-11-05 17:28:19 +08:00
glay
afbf5eb541 修改: .env.template
修改:     app/api/auth.ts
	修改:     app/api/bedrock.ts
	修改:     app/client/api.ts
	修改:     app/client/platforms/bedrock.ts
	修改:     app/components/settings.tsx
	修改:     app/config/server.ts
	修改:     app/constant.t
2024-11-05 14:27:52 +08:00
glay
0f276f59bb 修改: app/client/platforms/bedrock.ts 2024-11-05 10:34:33 +08:00
glay
fc391168e9 修改: app/api/bedrock.ts
删除:     app/api/bedrock/models.ts
	删除:     app/api/bedrock/utils.ts
	修改:     app/client/platforms/bedrock.ts
	删除:     app/components/chat-actions.tsx
	修改:     app/components/chat.tsx
	修改:     app/constant.ts
	删除:     app/icons/document.svg
2024-11-04 16:24:10 +08:00
glay
dca4a0e48f 修改: app/api/bedrock.ts
修改:     app/api/bedrock/models.ts
	修改:     app/api/bedrock/utils.ts
	修改:     app/client/api.ts
	修改:     app/client/platforms/bedrock.ts
	新文件:   app/components/chat-actions.tsx
	修改:     app/components/chat.module.scss
	修改:     app/components/chat.tsx
	修改:     app/constant.ts
	新文件:   app/icons/document.svg
	修改:     app/locales/cn.ts
	修改:     app/locales/en.ts
2024-10-31 14:23:38 +08:00
glay
722c28839f Merge branch 'ChatGPTNextWeb:main' into main 2024-10-31 11:51:41 +08:00
glay
ff356f0c8c 修改: app/api/[provider]/[...path]/route.ts
修改:     app/api/auth.ts
	新文件:   app/api/bedrock.ts
	新文件:   app/api/bedrock/models.ts
	新文件:   app/api/bedrock/utils.ts
	修改:     app/client/api.ts
	新文件:   app/client/platforms/bedrock.ts
	修改:     app/components/settings.tsx
	修改:     app/config/server.ts
	修改:     app/constant.ts
	修改:     app/locales/cn.ts
	修改:     app/locales/en.ts
	修改:     app/store/access.ts
	修改:     app/utils.ts
	修改:     package.json
2024-10-29 22:20:26 +08:00
38 changed files with 5629 additions and 1489 deletions

View File

@@ -76,6 +76,12 @@ ANTHROPIC_URL=
### (optional)
WHITE_WEBDAV_ENDPOINTS=
### bedrock (optional)
AWS_REGION=
AWS_ACCESS_KEY=AKIA
AWS_SECRET_KEY=
### siliconflow Api key (optional)
SILICONFLOW_API_KEY=

View File

@@ -1,6 +1,6 @@
<div align="center">
<a href='https://nextchat.dev/chat'>
<a href='https://nextchat.club'>
<img src="https://github.com/user-attachments/assets/83bdcc07-ae5e-4954-a53a-ac151ba6ccf3" width="1000" alt="icon"/>
</a>
@@ -23,9 +23,10 @@ English / [简体中文](./README_CN.md)
[![Linux][Linux-image]][download-url]
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [Web App Demo](https://app.nextchat.dev) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases)
[NextChatAI](https://nextchat.club?utm_source=readme) / [Web App Demo](https://app.nextchat.dev) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases) / [Discord](https://discord.gg/YCkeafCafC) / [Enterprise Edition](#enterprise-edition) / [Twitter](https://twitter.com/NextChatDev)
[saas-url]: https://nextchat.dev/chat?utm_source=readme
[saas-url]: https://nextchat.club?utm_source=readme
[saas-image]: https://img.shields.io/badge/NextChat-Saas-green?logo=microsoftedge
[web-url]: https://app.nextchat.dev/
[download-url]: https://github.com/Yidadaa/ChatGPT-Next-Web/releases
@@ -40,6 +41,24 @@ English / [简体中文](./README_CN.md)
</div>
## 👋 Hey, NextChat is going to develop a native app!
> This week we are going to start working on iOS and Android APP, and we want to find some reliable friends to do it together!
✨ Several key points:
- Starting from 0, you are a veteran
- Completely open source, not hidden
- Native development, pursuing the ultimate experience
Will you come and do something together? 😎
https://github.com/ChatGPTNextWeb/NextChat/issues/6269
#Seeking for talents is thirsty #lack of people
## 🥳 Cheer for DeepSeek, China's AI star!
> Purpose-Built UI for DeepSeek Reasoner Model

View File

@@ -8,7 +8,7 @@
一键免费部署你的私人 ChatGPT 网页应用,支持 Claude, GPT4 & Gemini Pro 模型。
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [企业版](#%E4%BC%81%E4%B8%9A%E7%89%88) / [演示 Demo](https://chat-gpt-next-web.vercel.app/) / [反馈 Issues](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [加入 Discord](https://discord.gg/zrhvHCr79N)
[NextChatAI](https://nextchat.club?utm_source=readme) / [企业版](#%E4%BC%81%E4%B8%9A%E7%89%88) / [演示 Demo](https://chat-gpt-next-web.vercel.app/) / [反馈 Issues](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [加入 Discord](https://discord.gg/zrhvHCr79N)
[<img src="https://vercel.com/button" alt="Deploy on Zeabur" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="30">](https://zeabur.com/templates/ZBUEFA) [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Open in Gitpod" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)

View File

@@ -5,7 +5,7 @@
ワンクリックで無料であなた専用の ChatGPT ウェブアプリをデプロイ。GPT3、GPT4 & Gemini Pro モデルをサポート。
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [企業版](#企業版) / [デモ](https://chat-gpt-next-web.vercel.app/) / [フィードバック](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [Discordに参加](https://discord.gg/zrhvHCr79N)
[NextChatAI](https://nextchat.club?utm_source=readme) / [企業版](#企業版) / [デモ](https://chat-gpt-next-web.vercel.app/) / [フィードバック](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [Discordに参加](https://discord.gg/zrhvHCr79N)
[<img src="https://vercel.com/button" alt="Zeaburでデプロイ" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Zeaburでデプロイ" height="30">](https://zeabur.com/templates/ZBUEFA) [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Gitpodで開く" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)

View File

@@ -1,6 +1,7 @@
import { ApiPath } from "@/app/constant";
import { NextRequest } from "next/server";
import { handle as openaiHandler } from "../../openai";
import { handle as bedrockHandler } from "../../bedrock";
import { handle as azureHandler } from "../../azure";
import { handle as googleHandler } from "../../google";
import { handle as anthropicHandler } from "../../anthropic";
@@ -23,12 +24,15 @@ async function handle(
const apiPath = `/api/${params.provider}`;
console.log(`[${params.provider} Route] params `, params);
switch (apiPath) {
case ApiPath.Bedrock:
return bedrockHandler(req, { params });
case ApiPath.Azure:
return azureHandler(req, { params });
case ApiPath.Google:
return googleHandler(req, { params });
case ApiPath.Anthropic:
return anthropicHandler(req, { params });
case ApiPath.Baidu:
return baiduHandler(req, { params });
case ApiPath.ByteDance:

View File

@@ -52,7 +52,6 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
msg: "you are not allowed to access with your own api key",
};
}
// if user does not provide an api key, inject system api key
if (!apiKey) {
const serverConfig = getServerSideConfig();
@@ -101,6 +100,14 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
case ModelProvider.ChatGLM:
systemApiKey = serverConfig.chatglmApiKey;
break;
case ModelProvider.Bedrock:
systemApiKey =
serverConfig.awsRegion +
":" +
serverConfig.awsAccessKey +
":" +
serverConfig.awsSecretKey;
break;
case ModelProvider.SiliconFlow:
systemApiKey = serverConfig.siliconFlowApiKey;
break;

177
app/api/bedrock.ts Normal file
View File

@@ -0,0 +1,177 @@
import { NextRequest, NextResponse } from "next/server";
import { auth } from "./auth";
import {
sign,
decrypt,
getBedrockEndpoint,
BedrockCredentials,
} from "../utils/aws";
import { getServerSideConfig } from "../config/server";
import { ModelProvider } from "../constant";
import { prettyObject } from "../utils/format";
const ALLOWED_PATH = new Set(["chat", "models"]);
async function getBedrockCredentials(
req: NextRequest,
): Promise<BedrockCredentials> {
// Get AWS credentials from server config first
const config = getServerSideConfig();
let awsRegion = config.awsRegion;
let awsAccessKey = config.awsAccessKey;
let awsSecretKey = config.awsSecretKey;
// If server-side credentials are not available, parse from Authorization header
if (!awsRegion || !awsAccessKey || !awsSecretKey) {
const authHeader = req.headers.get("Authorization");
if (!authHeader || !authHeader.startsWith("Bearer ")) {
throw new Error("Missing or invalid Authorization header");
}
const [_, credentials] = authHeader.split("Bearer ");
const [encryptedRegion, encryptedAccessKey, encryptedSecretKey] =
credentials.split(":");
if (!encryptedRegion || !encryptedAccessKey || !encryptedSecretKey) {
throw new Error("Invalid Authorization header format");
}
const encryptionKey = req.headers.get("XEncryptionKey") || "";
// Decrypt the credentials
[awsRegion, awsAccessKey, awsSecretKey] = await Promise.all([
decrypt(encryptedRegion, encryptionKey),
decrypt(encryptedAccessKey, encryptionKey),
decrypt(encryptedSecretKey, encryptionKey),
]);
if (!awsRegion || !awsAccessKey || !awsSecretKey) {
throw new Error(
"Failed to decrypt AWS credentials. Please ensure ENCRYPTION_KEY is set correctly.",
);
}
}
return {
region: awsRegion,
accessKeyId: awsAccessKey,
secretAccessKey: awsSecretKey,
};
}
async function requestBedrock(req: NextRequest) {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 10 * 60 * 1000);
try {
// Get credentials and model info
const credentials = await getBedrockCredentials(req);
const modelId = req.headers.get("XModelID");
const shouldStream = req.headers.get("ShouldStream") !== "false";
if (!modelId) {
throw new Error("Missing model ID");
}
// Parse and validate request body
const bodyText = await req.clone().text();
if (!bodyText) {
throw new Error("Request body is empty");
}
let bodyJson;
try {
bodyJson = JSON.parse(bodyText);
} catch (e) {
throw new Error(`Invalid JSON in request body: ${e}`);
}
console.log("[Bedrock Request] Initiating request");
// Get endpoint and prepare request
const endpoint = getBedrockEndpoint(
credentials.region,
modelId,
shouldStream,
);
const requestBody: any = {
...bodyJson,
};
// Sign request
const headers = await sign({
method: "POST",
url: endpoint,
region: credentials.region,
accessKeyId: credentials.accessKeyId,
secretAccessKey: credentials.secretAccessKey,
body: JSON.stringify(requestBody),
service: "bedrock",
isStreaming: shouldStream,
});
// Make request to AWS Bedrock
// console.log(
// "[Bedrock Request] Final Body:",
// JSON.stringify(requestBody, null, 2),
// );
const res = await fetch(endpoint, {
method: "POST",
headers,
body: JSON.stringify(requestBody),
redirect: "manual",
// @ts-ignore
duplex: "half",
signal: controller.signal,
});
if (!res.ok) {
const error = await res.text();
console.error("[Bedrock Error] Request failed with status:", res.status);
try {
const errorJson = JSON.parse(error);
throw new Error(errorJson.message || error);
} catch {
throw new Error(
`Bedrock request failed with status ${res.status}: ${
error || "No error message"
}`,
);
}
}
if (!res.body) {
console.error("[Bedrock Error] Empty response body");
throw new Error(
"Empty response from Bedrock. Please check AWS credentials and permissions.",
);
}
return res;
} catch (e) {
console.error("[Bedrock Request Error]:", e);
throw e;
} finally {
clearTimeout(timeoutId);
}
}
export async function handle(
req: NextRequest,
{ params }: { params: { path: string[] } },
) {
const subpath = params.path.join("/");
if (!ALLOWED_PATH.has(subpath)) {
return NextResponse.json(
{ error: true, msg: "you are not allowed to request " + subpath },
{ status: 403 },
);
}
const authResult = auth(req, ModelProvider.Bedrock);
if (authResult.error) {
return NextResponse.json(authResult, {
status: 401,
});
}
try {
return await requestBedrock(req);
} catch (e) {
console.error("Handler error:", e);
return NextResponse.json(prettyObject(e));
}
}

View File

@@ -14,8 +14,12 @@ function getModels(remoteModelRes: OpenAIListModelResponse) {
if (config.disableGPT4) {
remoteModelRes.data = remoteModelRes.data.filter(
(m) =>
!(m.id.startsWith("gpt-4") || m.id.startsWith("chatgpt-4o") || m.id.startsWith("o1") || m.id.startsWith("o3")) ||
m.id.startsWith("gpt-4o-mini"),
!(
m.id.startsWith("gpt-4") ||
m.id.startsWith("chatgpt-4o") ||
m.id.startsWith("o1") ||
m.id.startsWith("o3")
) || m.id.startsWith("gpt-4o-mini"),
);
}

View File

@@ -23,8 +23,10 @@ import { SparkApi } from "./platforms/iflytek";
import { DeepSeekApi } from "./platforms/deepseek";
import { XAIApi } from "./platforms/xai";
import { ChatGLMApi } from "./platforms/glm";
import { BedrockApi } from "./platforms/bedrock";
import { SiliconflowApi } from "./platforms/siliconflow";
export const ROLES = ["system", "user", "assistant"] as const;
export type MessageRole = (typeof ROLES)[number];
@@ -132,6 +134,9 @@ export class ClientApi {
constructor(provider: ModelProvider = ModelProvider.GPT) {
switch (provider) {
case ModelProvider.Bedrock:
this.llm = new BedrockApi();
break;
case ModelProvider.GeminiPro:
this.llm = new GeminiProApi();
break;
@@ -247,6 +252,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
function getConfig() {
const modelConfig = chatStore.currentSession().mask.modelConfig;
const isBedrock = modelConfig.providerName === ServiceProvider.Bedrock;
const isGoogle = modelConfig.providerName === ServiceProvider.Google;
const isAzure = modelConfig.providerName === ServiceProvider.Azure;
const isAnthropic = modelConfig.providerName === ServiceProvider.Anthropic;
@@ -287,6 +293,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
: ""
: accessStore.openaiApiKey;
return {
isBedrock,
isGoogle,
isAzure,
isAnthropic,
@@ -315,6 +322,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
}
const {
isBedrock,
isGoogle,
isAzure,
isAnthropic,
@@ -335,17 +343,23 @@ export function getHeaders(ignoreHeaders: boolean = false) {
const authHeader = getAuthHeader();
const bearerToken = getBearerToken(
apiKey,
isAzure || isAnthropic || isGoogle,
);
if (bearerToken) {
headers[authHeader] = bearerToken;
} else if (isEnabledAccessControl && validString(accessStore.accessCode)) {
headers["Authorization"] = getBearerToken(
ACCESS_CODE_PREFIX + accessStore.accessCode,
if (isBedrock) {
if (apiKey) {
headers[authHeader] = getBearerToken(apiKey);
}
} else {
const bearerToken = getBearerToken(
apiKey,
isAzure || isAnthropic || isGoogle,
);
if (bearerToken) {
headers[authHeader] = bearerToken;
} else if (isEnabledAccessControl && validString(accessStore.accessCode)) {
headers["Authorization"] = getBearerToken(
ACCESS_CODE_PREFIX + accessStore.accessCode,
);
}
}
return headers;
@@ -353,6 +367,8 @@ export function getHeaders(ignoreHeaders: boolean = false) {
export function getClientApi(provider: ServiceProvider): ClientApi {
switch (provider) {
case ServiceProvider.Bedrock:
return new ClientApi(ModelProvider.Bedrock);
case ServiceProvider.Google:
return new ClientApi(ModelProvider.GeminiPro);
case ServiceProvider.Anthropic:

View File

@@ -1,12 +1,13 @@
"use client";
import { ApiPath, Alibaba, ALIBABA_BASE_URL } from "@/app/constant";
import {
ApiPath,
Alibaba,
ALIBABA_BASE_URL,
REQUEST_TIMEOUT_MS,
} from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
useAccessStore,
useAppConfig,
useChatStore,
ChatMessageTool,
usePluginStore,
} from "@/app/store";
import { streamWithThink } from "@/app/utils/chat";
import {
ChatOptions,
getHeaders,
@@ -15,14 +16,12 @@ import {
SpeechOptions,
MultimodalContent,
} from "../api";
import Locale from "../../locales";
import {
EventStreamContentType,
fetchEventSource,
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
import { getMessageTextContent } from "@/app/utils";
import {
getMessageTextContent,
getMessageTextContentWithoutThinking,
getTimeoutMSByModel,
} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -92,7 +91,10 @@ export class QwenApi implements LLMApi {
async chat(options: ChatOptions) {
const messages = options.messages.map((v) => ({
role: v.role,
content: getMessageTextContent(v),
content:
v.role === "assistant"
? getMessageTextContentWithoutThinking(v)
: getMessageTextContent(v),
}));
const modelConfig = {
@@ -122,134 +124,118 @@ export class QwenApi implements LLMApi {
options.onController?.(controller);
try {
const headers = {
...getHeaders(),
"X-DashScope-SSE": shouldStream ? "enable" : "disable",
};
const chatPath = this.path(Alibaba.ChatPath);
const chatPayload = {
method: "POST",
body: JSON.stringify(requestPayload),
signal: controller.signal,
headers: {
...getHeaders(),
"X-DashScope-SSE": shouldStream ? "enable" : "disable",
},
headers: headers,
};
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
let responseText = "";
let remainText = "";
let finished = false;
let responseRes: Response;
const [tools, funcs] = usePluginStore
.getState()
.getAsTools(
useChatStore.getState().currentSession().mask?.plugin || [],
);
return streamWithThink(
chatPath,
requestPayload,
headers,
tools as any,
funcs,
controller,
// parseSSE
(text: string, runTools: ChatMessageTool[]) => {
// console.log("parseSSE", text, runTools);
const json = JSON.parse(text);
const choices = json.output.choices as Array<{
message: {
content: string | null;
tool_calls: ChatMessageTool[];
reasoning_content: string | null;
};
}>;
// animate response to make it looks smooth
function animateResponseText() {
if (finished || controller.signal.aborted) {
responseText += remainText;
console.log("[Response Animation] finished");
if (responseText?.length === 0) {
options.onError?.(new Error("empty response from server"));
}
return;
}
if (!choices?.length) return { isThinking: false, content: "" };
if (remainText.length > 0) {
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
const fetchText = remainText.slice(0, fetchCount);
responseText += fetchText;
remainText = remainText.slice(fetchCount);
options.onUpdate?.(responseText, fetchText);
}
requestAnimationFrame(animateResponseText);
}
// start animaion
animateResponseText();
const finish = () => {
if (!finished) {
finished = true;
options.onFinish(responseText + remainText, responseRes);
}
};
controller.signal.onabort = finish;
fetchEventSource(chatPath, {
fetch: fetch as any,
...chatPayload,
async onopen(res) {
clearTimeout(requestTimeoutId);
const contentType = res.headers.get("content-type");
console.log(
"[Alibaba] request response content type: ",
contentType,
);
responseRes = res;
if (contentType?.startsWith("text/plain")) {
responseText = await res.clone().text();
return finish();
const tool_calls = choices[0]?.message?.tool_calls;
if (tool_calls?.length > 0) {
const index = tool_calls[0]?.index;
const id = tool_calls[0]?.id;
const args = tool_calls[0]?.function?.arguments;
if (id) {
runTools.push({
id,
type: tool_calls[0]?.type,
function: {
name: tool_calls[0]?.function?.name as string,
arguments: args,
},
});
} else {
// @ts-ignore
runTools[index]["function"]["arguments"] += args;
}
}
const reasoning = choices[0]?.message?.reasoning_content;
const content = choices[0]?.message?.content;
// Skip if both content and reasoning_content are empty or null
if (
!res.ok ||
!res.headers
.get("content-type")
?.startsWith(EventStreamContentType) ||
res.status !== 200
(!reasoning || reasoning.length === 0) &&
(!content || content.length === 0)
) {
const responseTexts = [responseText];
let extraInfo = await res.clone().text();
try {
const resJson = await res.clone().json();
extraInfo = prettyObject(resJson);
} catch {}
if (res.status === 401) {
responseTexts.push(Locale.Error.Unauthorized);
}
if (extraInfo) {
responseTexts.push(extraInfo);
}
responseText = responseTexts.join("\n\n");
return finish();
return {
isThinking: false,
content: "",
};
}
},
onmessage(msg) {
if (msg.data === "[DONE]" || finished) {
return finish();
}
const text = msg.data;
try {
const json = JSON.parse(text);
const choices = json.output.choices as Array<{
message: { content: string };
}>;
const delta = choices[0]?.message?.content;
if (delta) {
remainText += delta;
}
} catch (e) {
console.error("[Request] parse error", text, msg);
if (reasoning && reasoning.length > 0) {
return {
isThinking: true,
content: reasoning,
};
} else if (content && content.length > 0) {
return {
isThinking: false,
content: content,
};
}
return {
isThinking: false,
content: "",
};
},
onclose() {
finish();
// processToolMessage, include tool_calls message and tool call results
(
requestPayload: RequestPayload,
toolCallMessage: any,
toolCallResult: any[],
) => {
requestPayload?.input?.messages?.splice(
requestPayload?.input?.messages?.length,
0,
toolCallMessage,
...toolCallResult,
);
},
onerror(e) {
options.onError?.(e);
throw e;
},
openWhenHidden: true,
});
options,
);
} else {
const res = await fetch(chatPath, chatPayload);
clearTimeout(requestTimeoutId);

View File

@@ -1,10 +1,5 @@
"use client";
import {
ApiPath,
Baidu,
BAIDU_BASE_URL,
REQUEST_TIMEOUT_MS,
} from "@/app/constant";
import { ApiPath, Baidu, BAIDU_BASE_URL } from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
import { getAccessToken } from "@/app/utils/baidu";
@@ -23,7 +18,7 @@ import {
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
import { getMessageTextContent } from "@/app/utils";
import { getMessageTextContent, getTimeoutMSByModel } from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -155,7 +150,7 @@ export class ErnieApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {

View File

@@ -0,0 +1,859 @@
"use client";
import { ChatOptions, getHeaders, LLMApi, SpeechOptions } from "../api";
import {
useAppConfig,
usePluginStore,
useChatStore,
useAccessStore,
ChatMessageTool,
} from "@/app/store";
import { preProcessImageContent } from "@/app/utils/chat";
import { getMessageTextContent, isVisionModel } from "@/app/utils";
import { ApiPath, BEDROCK_BASE_URL, REQUEST_TIMEOUT_MS } from "@/app/constant";
import { getClientConfig } from "@/app/config/client";
import {
extractMessage,
processMessage,
processChunks,
parseEventData,
sign,
} from "@/app/utils/aws";
import { prettyObject } from "@/app/utils/format";
import Locale from "@/app/locales";
import { encrypt } from "@/app/utils/aws";
const ClaudeMapper = {
assistant: "assistant",
user: "user",
system: "user",
} as const;
const MistralMapper = {
system: "system",
user: "user",
assistant: "assistant",
} as const;
type MistralRole = keyof typeof MistralMapper;
interface Tool {
function?: {
name?: string;
description?: string;
parameters?: any;
};
}
const isApp = !!getClientConfig()?.isApp;
// const isApp = true;
async function getBedrockHeaders(
modelId: string,
chatPath: string,
finalRequestBody: any,
shouldStream: boolean,
): Promise<Record<string, string>> {
const accessStore = useAccessStore.getState();
const bedrockHeaders = isApp
? await sign({
method: "POST",
url: chatPath,
region: accessStore.awsRegion,
accessKeyId: accessStore.awsAccessKey,
secretAccessKey: accessStore.awsSecretKey,
body: finalRequestBody,
service: "bedrock",
headers: {},
isStreaming: shouldStream,
})
: getHeaders();
if (!isApp) {
const { awsRegion, awsAccessKey, awsSecretKey, encryptionKey } =
accessStore;
const bedrockHeadersConfig = {
XModelID: modelId,
XEncryptionKey: encryptionKey,
ShouldStream: String(shouldStream),
Authorization: await createAuthHeader(
awsRegion,
awsAccessKey,
awsSecretKey,
encryptionKey,
),
};
Object.assign(bedrockHeaders, bedrockHeadersConfig);
}
return bedrockHeaders;
}
// Helper function to create Authorization header
async function createAuthHeader(
region: string,
accessKey: string,
secretKey: string,
encryptionKey: string,
): Promise<string> {
const encryptedValues = await Promise.all([
encrypt(region, encryptionKey),
encrypt(accessKey, encryptionKey),
encrypt(secretKey, encryptionKey),
]);
return `Bearer ${encryptedValues.join(":")}`;
}
export class BedrockApi implements LLMApi {
speech(options: SpeechOptions): Promise<ArrayBuffer> {
throw new Error("Speech not implemented for Bedrock.");
}
formatRequestBody(messages: ChatOptions["messages"], modelConfig: any) {
const model = modelConfig.model;
const visionModel = isVisionModel(modelConfig.model);
// Get tools if available
const [tools] = usePluginStore
.getState()
.getAsTools(useChatStore.getState().currentSession().mask?.plugin || []);
const toolsArray = (tools as Tool[]) || [];
// Handle Nova models
if (model.includes("amazon.nova")) {
// Extract system message if present
const systemMessage = messages.find((m) => m.role === "system");
const conversationMessages = messages.filter((m) => m.role !== "system");
const requestBody: any = {
schemaVersion: "messages-v1",
messages: conversationMessages.map((message) => {
const content = Array.isArray(message.content)
? message.content
: [{ text: getMessageTextContent(message) }];
return {
role: message.role,
content: content.map((item: any) => {
// Handle text content
if (item.text || typeof item === "string") {
return { text: item.text || item };
}
// Handle image content
if (item.image_url?.url) {
const { url = "" } = item.image_url;
const colonIndex = url.indexOf(":");
const semicolonIndex = url.indexOf(";");
const comma = url.indexOf(",");
// Extract format from mime type
const mimeType = url.slice(colonIndex + 1, semicolonIndex);
const format = mimeType.split("/")[1];
const data = url.slice(comma + 1);
return {
image: {
format,
source: {
bytes: data,
},
},
};
}
return item;
}),
};
}),
inferenceConfig: {
temperature: modelConfig.temperature || 0.7,
top_p: modelConfig.top_p || 0.9,
top_k: modelConfig.top_k || 50,
max_new_tokens: modelConfig.max_tokens || 1000,
stopSequences: modelConfig.stop || [],
},
};
// Add system message if present
if (systemMessage) {
requestBody.system = [
{
text: getMessageTextContent(systemMessage),
},
];
}
// Add tools if available - exact Nova format
if (toolsArray.length > 0) {
requestBody.toolConfig = {
tools: toolsArray.map((tool) => ({
toolSpec: {
name: tool?.function?.name || "",
description: tool?.function?.description || "",
inputSchema: {
json: {
type: "object",
properties: tool?.function?.parameters?.properties || {},
required: tool?.function?.parameters?.required || [],
},
},
},
})),
toolChoice: { auto: {} },
};
}
return requestBody;
}
// Handle Titan models
if (model.startsWith("amazon.titan")) {
const inputText = messages
.map((message) => {
return `${message.role}: ${getMessageTextContent(message)}`;
})
.join("\n\n");
return {
inputText,
textGenerationConfig: {
maxTokenCount: modelConfig.max_tokens,
temperature: modelConfig.temperature,
stopSequences: [],
},
};
}
// Handle LLaMA models
if (model.includes("meta.llama")) {
let prompt = "<|begin_of_text|>";
// Extract system message if present
const systemMessage = messages.find((m) => m.role === "system");
if (systemMessage) {
prompt += `<|start_header_id|>system<|end_header_id|>\n${getMessageTextContent(
systemMessage,
)}<|eot_id|>`;
}
// Format the conversation
const conversationMessages = messages.filter((m) => m.role !== "system");
for (const message of conversationMessages) {
const role = message.role === "assistant" ? "assistant" : "user";
const content = getMessageTextContent(message);
prompt += `<|start_header_id|>${role}<|end_header_id|>\n${content}<|eot_id|>`;
}
// Add the final assistant header to prompt completion
prompt += "<|start_header_id|>assistant<|end_header_id|>";
return {
prompt,
max_gen_len: modelConfig.max_tokens || 512,
temperature: modelConfig.temperature || 0.7,
top_p: modelConfig.top_p || 0.9,
};
}
// Handle Mistral models
if (model.includes("mistral.mistral")) {
const formattedMessages = messages.map((message) => ({
role: MistralMapper[message.role as MistralRole] || "user",
content: getMessageTextContent(message),
}));
const requestBody: any = {
messages: formattedMessages,
max_tokens: modelConfig.max_tokens || 4096,
temperature: modelConfig.temperature || 0.7,
top_p: modelConfig.top_p || 0.9,
};
// Add tools if available
if (toolsArray.length > 0) {
requestBody.tool_choice = "auto";
requestBody.tools = toolsArray.map((tool) => ({
type: "function",
function: {
name: tool?.function?.name,
description: tool?.function?.description,
parameters: tool?.function?.parameters,
},
}));
}
return requestBody;
}
// Handle Claude models
const keys = ["system", "user"];
// roles must alternate between "user" and "assistant" in claude, so add a fake assistant message between two user messages
for (let i = 0; i < messages.length - 1; i++) {
const message = messages[i];
const nextMessage = messages[i + 1];
if (keys.includes(message.role) && keys.includes(nextMessage.role)) {
messages[i] = [
message,
{
role: "assistant",
content: ";",
},
] as any;
}
}
const prompt = messages
.flat()
.filter((v) => {
if (!v.content) return false;
if (typeof v.content === "string" && !v.content.trim()) return false;
return true;
})
.map((v) => {
const { role, content } = v;
const insideRole = ClaudeMapper[role] ?? "user";
if (!visionModel || typeof content === "string") {
return {
role: insideRole,
content: getMessageTextContent(v),
};
}
return {
role: insideRole,
content: content
.filter((v) => v.image_url || v.text)
.map(({ type, text, image_url }) => {
if (type === "text") {
return {
type,
text: text!,
};
}
const { url = "" } = image_url || {};
const colonIndex = url.indexOf(":");
const semicolonIndex = url.indexOf(";");
const comma = url.indexOf(",");
const mimeType = url.slice(colonIndex + 1, semicolonIndex);
const encodeType = url.slice(semicolonIndex + 1, comma);
const data = url.slice(comma + 1);
return {
type: "image" as const,
source: {
type: encodeType,
media_type: mimeType,
data,
},
};
}),
};
});
if (prompt[0]?.role === "assistant") {
prompt.unshift({
role: "user",
content: ";",
});
}
const requestBody: any = {
anthropic_version: useAccessStore.getState().bedrockAnthropicVersion,
max_tokens: modelConfig.max_tokens,
messages: prompt,
temperature: modelConfig.temperature,
top_p: modelConfig.top_p || 0.9,
top_k: modelConfig.top_k || 5,
};
// Add tools if available for Claude models
if (toolsArray.length > 0 && model.includes("anthropic.claude")) {
requestBody.tools = toolsArray.map((tool) => ({
name: tool?.function?.name || "",
description: tool?.function?.description || "",
input_schema: tool?.function?.parameters || {},
}));
}
return requestBody;
}
async chat(options: ChatOptions) {
const accessStore = useAccessStore.getState();
const shouldStream = !!options.config.stream;
const modelConfig = {
...useAppConfig.getState().modelConfig,
...useChatStore.getState().currentSession().mask.modelConfig,
...{
model: options.config.model,
},
};
// try get base64image from local cache image_url
const messages: ChatOptions["messages"] = [];
for (const v of options.messages) {
const content = await preProcessImageContent(v.content);
messages.push({ role: v.role, content });
}
const controller = new AbortController();
options.onController?.(controller);
let finalRequestBody = this.formatRequestBody(messages, modelConfig);
try {
const bedrockAPIPath = `${BEDROCK_BASE_URL}/model/${
modelConfig.model
}/invoke${shouldStream ? "-with-response-stream" : ""}`;
const chatPath = isApp ? bedrockAPIPath : ApiPath.Bedrock + "/chat";
if (process.env.NODE_ENV !== "production") {
console.debug("[Bedrock Client] Request:", {
path: chatPath,
model: modelConfig.model,
messages: messages.length,
stream: shouldStream,
});
}
if (shouldStream) {
const [tools, funcs] = usePluginStore
.getState()
.getAsTools(
useChatStore.getState().currentSession().mask?.plugin || [],
);
return bedrockStream(
modelConfig.model,
chatPath,
finalRequestBody,
funcs,
controller,
// processToolMessage, include tool_calls message and tool call results
(
requestPayload: any[],
toolCallMessage: any,
toolCallResult: any[],
) => {
const modelId = modelConfig.model;
const isMistral = modelId.includes("mistral.mistral");
const isClaude = modelId.includes("anthropic.claude");
const isNova = modelId.includes("amazon.nova");
if (isClaude) {
// Format for Claude
// @ts-ignore
requestPayload?.messages?.splice(
// @ts-ignore
requestPayload?.messages?.length,
0,
{
role: "assistant",
content: toolCallMessage.tool_calls.map(
(tool: ChatMessageTool) => ({
type: "tool_use",
id: tool.id,
name: tool?.function?.name,
input: tool?.function?.arguments
? JSON.parse(tool?.function?.arguments)
: {},
}),
),
},
// @ts-ignore
...toolCallResult.map((result) => ({
role: "user",
content: [
{
type: "tool_result",
tool_use_id: result.tool_call_id,
content: result.content,
},
],
})),
);
} else if (isMistral) {
// Format for Mistral
// @ts-ignore
requestPayload?.messages?.splice(
// @ts-ignore
requestPayload?.messages?.length,
0,
{
role: "assistant",
content: "",
// @ts-ignore
tool_calls: toolCallMessage.tool_calls.map(
(tool: ChatMessageTool) => ({
id: tool.id,
function: {
name: tool?.function?.name,
arguments: tool?.function?.arguments || "{}",
},
}),
),
},
...toolCallResult.map((result) => ({
role: "tool",
tool_call_id: result.tool_call_id,
content: result.content,
})),
);
} else if (isNova) {
// Format for Nova - Updated format
// @ts-ignore
requestPayload?.messages?.splice(
// @ts-ignore
requestPayload?.messages?.length,
0,
{
role: "assistant",
content: [
{
toolUse: {
toolUseId: toolCallMessage.tool_calls[0].id,
name: toolCallMessage.tool_calls[0]?.function?.name,
input:
typeof toolCallMessage.tool_calls[0]?.function
?.arguments === "string"
? JSON.parse(
toolCallMessage.tool_calls[0]?.function
?.arguments,
)
: toolCallMessage.tool_calls[0]?.function
?.arguments || {},
},
},
],
},
{
role: "user",
content: [
{
toolResult: {
toolUseId: toolCallResult[0].tool_call_id,
content: [
{
json: {
content: toolCallResult[0].content,
},
},
],
},
},
],
},
);
} else {
console.warn(
`[Bedrock Client] Unhandled model type for tool calls: ${modelId}`,
);
}
},
options,
);
} else {
try {
controller.signal.onabort = () =>
options.onFinish("", new Response(null, { status: 400 }));
const newHeaders = await getBedrockHeaders(
modelConfig.model,
chatPath,
JSON.stringify(finalRequestBody),
shouldStream,
);
const res = await fetch(chatPath, {
method: "POST",
headers: newHeaders,
body: JSON.stringify(finalRequestBody),
});
const contentType = res.headers.get("content-type");
console.log(
"[Bedrock Not Stream Request] response content type: ",
contentType,
);
const resJson = await res.json();
const message = extractMessage(resJson);
options.onFinish(message, res);
} catch (e) {
const error =
e instanceof Error ? e : new Error("Unknown error occurred");
console.error("[Bedrock Client] Chat failed:", error.message);
options.onError?.(error);
}
}
} catch (e) {
console.error("[Bedrock Client] Chat error:", e);
options.onError?.(e as Error);
}
}
async usage() {
return { used: 0, total: 0 };
}
async models() {
return [];
}
}
function bedrockStream(
modelId: string,
chatPath: string,
requestPayload: any,
funcs: Record<string, Function>,
controller: AbortController,
processToolMessage: (
requestPayload: any,
toolCallMessage: any,
toolCallResult: any[],
) => void,
options: any,
) {
let responseText = "";
let remainText = "";
let finished = false;
let running = false;
let runTools: any[] = [];
let responseRes: Response;
let index = -1;
let chunks: Uint8Array[] = [];
let pendingChunk: Uint8Array | null = null;
function animateResponseText() {
if (finished || controller.signal.aborted) {
responseText += remainText;
console.log("[Response Animation] finished");
if (responseText?.length === 0) {
options.onError?.(new Error("empty response from server"));
}
return;
}
if (remainText.length > 0) {
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
const fetchText = remainText.slice(0, fetchCount);
responseText += fetchText;
remainText = remainText.slice(fetchCount);
options.onUpdate?.(responseText, fetchText);
}
requestAnimationFrame(animateResponseText);
}
animateResponseText();
const finish = () => {
if (!finished) {
if (!running && runTools.length > 0) {
const toolCallMessage = {
role: "assistant",
tool_calls: [...runTools],
};
running = true;
runTools.splice(0, runTools.length);
return Promise.all(
toolCallMessage.tool_calls.map((tool) => {
options?.onBeforeTool?.(tool);
const funcName = tool?.function?.name || tool?.name;
if (!funcName || !funcs[funcName]) {
console.error(`Function ${funcName} not found in funcs:`, funcs);
return Promise.reject(`Function ${funcName} not found`);
}
return Promise.resolve(
funcs[funcName](
tool?.function?.arguments
? JSON.parse(tool?.function?.arguments)
: {},
),
)
.then((res) => {
let content = res.data || res?.statusText;
content =
typeof content === "string"
? content
: JSON.stringify(content);
if (res.status >= 300) {
return Promise.reject(content);
}
return content;
})
.then((content) => {
options?.onAfterTool?.({
...tool,
content,
isError: false,
});
return content;
})
.catch((e) => {
options?.onAfterTool?.({
...tool,
isError: true,
errorMsg: e.toString(),
});
return e.toString();
})
.then((content) => ({
name: funcName,
role: "tool",
content,
tool_call_id: tool.id,
}));
}),
).then((toolCallResult) => {
processToolMessage(requestPayload, toolCallMessage, toolCallResult);
setTimeout(() => {
console.debug("[BedrockAPI for toolCallResult] restart");
running = false;
bedrockChatApi(modelId, chatPath, requestPayload, true);
}, 60);
});
}
if (running) {
return;
}
console.debug("[BedrockAPI] end");
finished = true;
options.onFinish(responseText + remainText, responseRes);
}
};
controller.signal.onabort = finish;
async function bedrockChatApi(
modelId: string,
chatPath: string,
requestPayload: any,
shouldStream: boolean,
) {
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
);
const newHeaders = await getBedrockHeaders(
modelId,
chatPath,
JSON.stringify(requestPayload),
shouldStream,
);
try {
const res = await fetch(chatPath, {
method: "POST",
headers: newHeaders,
body: JSON.stringify(requestPayload),
redirect: "manual",
// @ts-ignore
duplex: "half",
signal: controller.signal,
});
clearTimeout(requestTimeoutId);
responseRes = res;
const contentType = res.headers.get("content-type");
// console.log(
// "[Bedrock Stream Request] response content type: ",
// contentType,
// );
if (contentType?.startsWith("text/plain")) {
responseText = await res.text();
return finish();
}
if (
!res.ok ||
res.status !== 200 ||
!contentType?.startsWith("application/vnd.amazon.eventstream")
) {
const responseTexts = [responseText];
let extraInfo = await res.text();
try {
const resJson = await res.clone().json();
extraInfo = prettyObject(resJson);
} catch {}
if (res.status === 401) {
responseTexts.push(Locale.Error.Unauthorized);
}
if (extraInfo) {
responseTexts.push(extraInfo);
}
responseText = responseTexts.join("\n\n");
return finish();
}
const reader = res.body?.getReader();
if (!reader) {
throw new Error("No response body reader available");
}
try {
while (true) {
const { done, value } = await reader.read();
if (done) {
if (pendingChunk) {
try {
const parsed = parseEventData(pendingChunk);
if (parsed) {
const result = processMessage(
parsed,
remainText,
runTools,
index,
);
remainText = result.remainText;
index = result.index;
}
} catch (e) {
console.error("[Final Chunk Process Error]:", e);
}
}
break;
}
chunks.push(value);
const result = processChunks(
chunks,
pendingChunk,
remainText,
runTools,
index,
);
chunks = result.chunks;
pendingChunk = result.pendingChunk;
remainText = result.remainText;
index = result.index;
}
} catch (err) {
console.error(
"[Bedrock Stream]:",
err instanceof Error ? err.message : "Stream processing failed",
);
throw new Error("Failed to process stream response");
} finally {
reader.releaseLock();
finish();
}
} catch (e) {
if (e instanceof Error && e.name === "AbortError") {
console.log("[Bedrock Client] Aborted by user");
return;
}
console.error(
"[Bedrock Request] Failed:",
e instanceof Error ? e.message : "Request failed",
);
options.onError?.(e);
throw new Error("Request processing failed");
}
}
console.debug("[BedrockAPI] start");
bedrockChatApi(modelId, chatPath, requestPayload, true);
}

View File

@@ -1,11 +1,12 @@
"use client";
import { ApiPath, ByteDance, BYTEDANCE_BASE_URL } from "@/app/constant";
import {
ApiPath,
ByteDance,
BYTEDANCE_BASE_URL,
REQUEST_TIMEOUT_MS,
} from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
useAccessStore,
useAppConfig,
useChatStore,
ChatMessageTool,
usePluginStore,
} from "@/app/store";
import {
ChatOptions,
@@ -15,14 +16,14 @@ import {
MultimodalContent,
SpeechOptions,
} from "../api";
import Locale from "../../locales";
import {
EventStreamContentType,
fetchEventSource,
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { streamWithThink } from "@/app/utils/chat";
import { getClientConfig } from "@/app/config/client";
import { preProcessImageContent } from "@/app/utils/chat";
import {
getMessageTextContentWithoutThinking,
getTimeoutMSByModel,
} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
export interface OpenAIListModelResponse {
@@ -34,7 +35,7 @@ export interface OpenAIListModelResponse {
}>;
}
interface RequestPayload {
interface RequestPayloadForByteDance {
messages: {
role: "system" | "user" | "assistant";
content: string | MultimodalContent[];
@@ -86,7 +87,10 @@ export class DoubaoApi implements LLMApi {
async chat(options: ChatOptions) {
const messages: ChatOptions["messages"] = [];
for (const v of options.messages) {
const content = await preProcessImageContent(v.content);
const content =
v.role === "assistant"
? getMessageTextContentWithoutThinking(v)
: await preProcessImageContent(v.content);
messages.push({ role: v.role, content });
}
@@ -99,7 +103,7 @@ export class DoubaoApi implements LLMApi {
};
const shouldStream = !!options.config.stream;
const requestPayload: RequestPayload = {
const requestPayload: RequestPayloadForByteDance = {
messages,
stream: shouldStream,
model: modelConfig.model,
@@ -124,119 +128,101 @@ export class DoubaoApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
let responseText = "";
let remainText = "";
let finished = false;
let responseRes: Response;
const [tools, funcs] = usePluginStore
.getState()
.getAsTools(
useChatStore.getState().currentSession().mask?.plugin || [],
);
return streamWithThink(
chatPath,
requestPayload,
getHeaders(),
tools as any,
funcs,
controller,
// parseSSE
(text: string, runTools: ChatMessageTool[]) => {
// console.log("parseSSE", text, runTools);
const json = JSON.parse(text);
const choices = json.choices as Array<{
delta: {
content: string | null;
tool_calls: ChatMessageTool[];
reasoning_content: string | null;
};
}>;
// animate response to make it looks smooth
function animateResponseText() {
if (finished || controller.signal.aborted) {
responseText += remainText;
console.log("[Response Animation] finished");
if (responseText?.length === 0) {
options.onError?.(new Error("empty response from server"));
}
return;
}
if (remainText.length > 0) {
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
const fetchText = remainText.slice(0, fetchCount);
responseText += fetchText;
remainText = remainText.slice(fetchCount);
options.onUpdate?.(responseText, fetchText);
}
requestAnimationFrame(animateResponseText);
}
// start animaion
animateResponseText();
const finish = () => {
if (!finished) {
finished = true;
options.onFinish(responseText + remainText, responseRes);
}
};
controller.signal.onabort = finish;
fetchEventSource(chatPath, {
fetch: fetch as any,
...chatPayload,
async onopen(res) {
clearTimeout(requestTimeoutId);
const contentType = res.headers.get("content-type");
console.log(
"[ByteDance] request response content type: ",
contentType,
);
responseRes = res;
if (contentType?.startsWith("text/plain")) {
responseText = await res.clone().text();
return finish();
if (!choices?.length) return { isThinking: false, content: "" };
const tool_calls = choices[0]?.delta?.tool_calls;
if (tool_calls?.length > 0) {
const index = tool_calls[0]?.index;
const id = tool_calls[0]?.id;
const args = tool_calls[0]?.function?.arguments;
if (id) {
runTools.push({
id,
type: tool_calls[0]?.type,
function: {
name: tool_calls[0]?.function?.name as string,
arguments: args,
},
});
} else {
// @ts-ignore
runTools[index]["function"]["arguments"] += args;
}
}
const reasoning = choices[0]?.delta?.reasoning_content;
const content = choices[0]?.delta?.content;
// Skip if both content and reasoning_content are empty or null
if (
!res.ok ||
!res.headers
.get("content-type")
?.startsWith(EventStreamContentType) ||
res.status !== 200
(!reasoning || reasoning.length === 0) &&
(!content || content.length === 0)
) {
const responseTexts = [responseText];
let extraInfo = await res.clone().text();
try {
const resJson = await res.clone().json();
extraInfo = prettyObject(resJson);
} catch {}
if (res.status === 401) {
responseTexts.push(Locale.Error.Unauthorized);
}
if (extraInfo) {
responseTexts.push(extraInfo);
}
responseText = responseTexts.join("\n\n");
return finish();
return {
isThinking: false,
content: "",
};
}
},
onmessage(msg) {
if (msg.data === "[DONE]" || finished) {
return finish();
}
const text = msg.data;
try {
const json = JSON.parse(text);
const choices = json.choices as Array<{
delta: { content: string };
}>;
const delta = choices[0]?.delta?.content;
if (delta) {
remainText += delta;
}
} catch (e) {
console.error("[Request] parse error", text, msg);
if (reasoning && reasoning.length > 0) {
return {
isThinking: true,
content: reasoning,
};
} else if (content && content.length > 0) {
return {
isThinking: false,
content: content,
};
}
return {
isThinking: false,
content: "",
};
},
onclose() {
finish();
// processToolMessage, include tool_calls message and tool call results
(
requestPayload: RequestPayloadForByteDance,
toolCallMessage: any,
toolCallResult: any[],
) => {
requestPayload?.messages?.splice(
requestPayload?.messages?.length,
0,
toolCallMessage,
...toolCallResult,
);
},
onerror(e) {
options.onError?.(e);
throw e;
},
openWhenHidden: true,
});
options,
);
} else {
const res = await fetch(chatPath, chatPayload);
clearTimeout(requestTimeoutId);

View File

@@ -1,12 +1,6 @@
"use client";
// azure and openai, using same models. so using same LLMApi.
import {
ApiPath,
DEEPSEEK_BASE_URL,
DeepSeek,
REQUEST_TIMEOUT_MS,
REQUEST_TIMEOUT_MS_FOR_THINKING,
} from "@/app/constant";
import { ApiPath, DEEPSEEK_BASE_URL, DeepSeek } from "@/app/constant";
import {
useAccessStore,
useAppConfig,
@@ -26,6 +20,7 @@ import { getClientConfig } from "@/app/config/client";
import {
getMessageTextContent,
getMessageTextContentWithoutThinking,
getTimeoutMSByModel,
} from "@/app/utils";
import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
@@ -116,16 +111,10 @@ export class DeepSeekApi implements LLMApi {
headers: getHeaders(),
};
// console.log(chatPayload);
const isR1 =
options.config.model.endsWith("-reasoner") ||
options.config.model.endsWith("-r1");
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
isR1 ? REQUEST_TIMEOUT_MS_FOR_THINKING : REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
@@ -176,8 +165,8 @@ export class DeepSeekApi implements LLMApi {
// Skip if both content and reasoning_content are empty or null
if (
(!reasoning || reasoning.trim().length === 0) &&
(!content || content.trim().length === 0)
(!reasoning || reasoning.length === 0) &&
(!content || content.length === 0)
) {
return {
isThinking: false,
@@ -185,12 +174,12 @@ export class DeepSeekApi implements LLMApi {
};
}
if (reasoning && reasoning.trim().length > 0) {
if (reasoning && reasoning.length > 0) {
return {
isThinking: true,
content: reasoning,
};
} else if (content && content.trim().length > 0) {
} else if (content && content.length > 0) {
return {
isThinking: false,
content: content,

View File

@@ -1,10 +1,5 @@
"use client";
import {
ApiPath,
CHATGLM_BASE_URL,
ChatGLM,
REQUEST_TIMEOUT_MS,
} from "@/app/constant";
import { ApiPath, CHATGLM_BASE_URL, ChatGLM } from "@/app/constant";
import {
useAccessStore,
useAppConfig,
@@ -21,7 +16,11 @@ import {
SpeechOptions,
} from "../api";
import { getClientConfig } from "@/app/config/client";
import { getMessageTextContent, isVisionModel } from "@/app/utils";
import {
getMessageTextContent,
isVisionModel,
getTimeoutMSByModel,
} from "@/app/utils";
import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
import { preProcessImageContent } from "@/app/utils/chat";
@@ -191,7 +190,7 @@ export class ChatGLMApi implements LLMApi {
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (modelType === "image" || modelType === "video") {

View File

@@ -1,9 +1,4 @@
import {
ApiPath,
Google,
REQUEST_TIMEOUT_MS,
REQUEST_TIMEOUT_MS_FOR_THINKING,
} from "@/app/constant";
import { ApiPath, Google } from "@/app/constant";
import {
ChatOptions,
getHeaders,
@@ -27,6 +22,7 @@ import {
getMessageTextContent,
getMessageImages,
isVisionModel,
getTimeoutMSByModel,
} from "@/app/utils";
import { preProcessImageContent } from "@/app/utils/chat";
import { nanoid } from "nanoid";
@@ -206,7 +202,7 @@ export class GeminiProApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
isThinking ? REQUEST_TIMEOUT_MS_FOR_THINKING : REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {

View File

@@ -8,7 +8,6 @@ import {
Azure,
REQUEST_TIMEOUT_MS,
ServiceProvider,
REQUEST_TIMEOUT_MS_FOR_THINKING,
} from "@/app/constant";
import {
ChatMessageTool,
@@ -22,7 +21,7 @@ import {
preProcessImageContent,
uploadImage,
base64Image2Blob,
stream,
streamWithThink,
} from "@/app/utils/chat";
import { cloudflareAIGatewayUrl } from "@/app/utils/cloudflare";
import { ModelSize, DalleQuality, DalleStyle } from "@/app/typing";
@@ -42,6 +41,7 @@ import {
getMessageTextContent,
isVisionModel,
isDalle3 as _isDalle3,
getTimeoutMSByModel,
} from "@/app/utils";
import { fetch } from "@/app/utils/stream";
@@ -294,7 +294,7 @@ export class ChatGPTApi implements LLMApi {
useChatStore.getState().currentSession().mask?.plugin || [],
);
// console.log("getAsTools", tools, funcs);
stream(
streamWithThink(
chatPath,
requestPayload,
getHeaders(),
@@ -309,8 +309,12 @@ export class ChatGPTApi implements LLMApi {
delta: {
content: string;
tool_calls: ChatMessageTool[];
reasoning_content: string | null;
};
}>;
if (!choices?.length) return { isThinking: false, content: "" };
const tool_calls = choices[0]?.delta?.tool_calls;
if (tool_calls?.length > 0) {
const id = tool_calls[0]?.id;
@@ -330,7 +334,37 @@ export class ChatGPTApi implements LLMApi {
runTools[index]["function"]["arguments"] += args;
}
}
return choices[0]?.delta?.content;
const reasoning = choices[0]?.delta?.reasoning_content;
const content = choices[0]?.delta?.content;
// Skip if both content and reasoning_content are empty or null
if (
(!reasoning || reasoning.length === 0) &&
(!content || content.length === 0)
) {
return {
isThinking: false,
content: "",
};
}
if (reasoning && reasoning.length > 0) {
return {
isThinking: true,
content: reasoning,
};
} else if (content && content.length > 0) {
return {
isThinking: false,
content: content,
};
}
return {
isThinking: false,
content: "",
};
},
// processToolMessage, include tool_calls message and tool call results
(
@@ -362,9 +396,7 @@ export class ChatGPTApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
isDalle3 || isO1OrO3
? REQUEST_TIMEOUT_MS_FOR_THINKING
: REQUEST_TIMEOUT_MS, // dalle3 using b64_json is slow.
getTimeoutMSByModel(options.config.model),
);
const res = await fetch(chatPath, chatPayload);

View File

@@ -4,7 +4,7 @@ import {
ApiPath,
SILICONFLOW_BASE_URL,
SiliconFlow,
REQUEST_TIMEOUT_MS_FOR_THINKING,
DEFAULT_MODELS,
} from "@/app/constant";
import {
useAccessStore,
@@ -13,7 +13,7 @@ import {
ChatMessageTool,
usePluginStore,
} from "@/app/store";
import { streamWithThink } from "@/app/utils/chat";
import { preProcessImageContent, streamWithThink } from "@/app/utils/chat";
import {
ChatOptions,
getHeaders,
@@ -25,12 +25,23 @@ import { getClientConfig } from "@/app/config/client";
import {
getMessageTextContent,
getMessageTextContentWithoutThinking,
isVisionModel,
getTimeoutMSByModel,
} from "@/app/utils";
import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
export interface SiliconFlowListModelResponse {
object: string;
data: Array<{
id: string;
object: string;
root: string;
}>;
}
export class SiliconflowApi implements LLMApi {
private disableListModels = true;
private disableListModels = false;
path(path: string): string {
const accessStore = useAccessStore.getState();
@@ -71,13 +82,16 @@ export class SiliconflowApi implements LLMApi {
}
async chat(options: ChatOptions) {
const visionModel = isVisionModel(options.config.model);
const messages: ChatOptions["messages"] = [];
for (const v of options.messages) {
if (v.role === "assistant") {
const content = getMessageTextContentWithoutThinking(v);
messages.push({ role: v.role, content });
} else {
const content = getMessageTextContent(v);
const content = visionModel
? await preProcessImageContent(v.content)
: getMessageTextContent(v);
messages.push({ role: v.role, content });
}
}
@@ -123,7 +137,7 @@ export class SiliconflowApi implements LLMApi {
// Use extended timeout for thinking models as they typically require more processing time
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS_FOR_THINKING,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {
@@ -238,6 +252,36 @@ export class SiliconflowApi implements LLMApi {
}
async models(): Promise<LLMModel[]> {
return [];
if (this.disableListModels) {
return DEFAULT_MODELS.slice();
}
const res = await fetch(this.path(SiliconFlow.ListModelPath), {
method: "GET",
headers: {
...getHeaders(),
},
});
const resJson = (await res.json()) as SiliconFlowListModelResponse;
const chatModels = resJson.data;
console.log("[Models]", chatModels);
if (!chatModels) {
return [];
}
let seq = 1000; //同 Constant.ts 中的排序保持一致
return chatModels.map((m) => ({
name: m.id,
available: true,
sorted: seq++,
provider: {
id: "siliconflow",
providerName: "SiliconFlow",
providerType: "siliconflow",
sorted: 14,
},
}));
}
}

View File

@@ -1,5 +1,5 @@
"use client";
import { ApiPath, TENCENT_BASE_URL, REQUEST_TIMEOUT_MS } from "@/app/constant";
import { ApiPath, TENCENT_BASE_URL } from "@/app/constant";
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
import {
@@ -17,7 +17,11 @@ import {
} from "@fortaine/fetch-event-source";
import { prettyObject } from "@/app/utils/format";
import { getClientConfig } from "@/app/config/client";
import { getMessageTextContent, isVisionModel } from "@/app/utils";
import {
getMessageTextContent,
isVisionModel,
getTimeoutMSByModel,
} from "@/app/utils";
import mapKeys from "lodash-es/mapKeys";
import mapValues from "lodash-es/mapValues";
import isArray from "lodash-es/isArray";
@@ -135,7 +139,7 @@ export class HunyuanApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {

View File

@@ -1,6 +1,6 @@
"use client";
// azure and openai, using same models. so using same LLMApi.
import { ApiPath, XAI_BASE_URL, XAI, REQUEST_TIMEOUT_MS } from "@/app/constant";
import { ApiPath, XAI_BASE_URL, XAI } from "@/app/constant";
import {
useAccessStore,
useAppConfig,
@@ -17,6 +17,7 @@ import {
SpeechOptions,
} from "../api";
import { getClientConfig } from "@/app/config/client";
import { getTimeoutMSByModel } from "@/app/utils";
import { preProcessImageContent } from "@/app/utils/chat";
import { RequestPayload } from "./openai";
import { fetch } from "@/app/utils/stream";
@@ -103,7 +104,7 @@ export class XAIApi implements LLMApi {
// make a fetch request
const requestTimeoutId = setTimeout(
() => controller.abort(),
REQUEST_TIMEOUT_MS,
getTimeoutMSByModel(options.config.model),
);
if (shouldStream) {

View File

@@ -750,4 +750,4 @@
transform: translateX(0);
}
}
}
}

View File

@@ -66,11 +66,11 @@ export function Avatar(props: { model?: ModelType; avatar?: string }) {
LlmIcon = BotIconGemma;
} else if (modelName.startsWith("claude")) {
LlmIcon = BotIconClaude;
} else if (modelName.startsWith("llama")) {
} else if (modelName.toLowerCase().includes("llama")) {
LlmIcon = BotIconMeta;
} else if (modelName.startsWith("mixtral")) {
LlmIcon = BotIconMistral;
} else if (modelName.startsWith("deepseek")) {
} else if (modelName.toLowerCase().includes("deepseek")) {
LlmIcon = BotIconDeepseek;
} else if (modelName.startsWith("moonshot")) {
LlmIcon = BotIconMoonshot;
@@ -85,7 +85,7 @@ export function Avatar(props: { model?: ModelType; avatar?: string }) {
} else if (modelName.startsWith("doubao") || modelName.startsWith("ep-")) {
LlmIcon = BotIconDoubao;
} else if (
modelName.startsWith("glm") ||
modelName.toLowerCase().includes("glm") ||
modelName.startsWith("cogview-") ||
modelName.startsWith("cogvideox-")
) {

View File

@@ -23,7 +23,6 @@ import CopyIcon from "../icons/copy.svg";
import LoadingIcon from "../icons/three-dots.svg";
import ChatGptIcon from "../icons/chatgpt.png";
import ShareIcon from "../icons/share.svg";
import BotIcon from "../icons/bot.png";
import DownloadIcon from "../icons/download.svg";
import { useEffect, useMemo, useRef, useState } from "react";
@@ -33,13 +32,13 @@ import dynamic from "next/dynamic";
import NextImage from "next/image";
import { toBlob, toPng } from "html-to-image";
import { DEFAULT_MASK_AVATAR } from "../store/mask";
import { prettyObject } from "../utils/format";
import { EXPORT_MESSAGE_CLASS_NAME } from "../constant";
import { getClientConfig } from "../config/client";
import { type ClientApi, getClientApi } from "../client/api";
import { getMessageTextContent } from "../utils";
import { MaskAvatar } from "./mask";
import clsx from "clsx";
const Markdown = dynamic(async () => (await import("./markdown")).Markdown, {
@@ -407,22 +406,6 @@ export function PreviewActions(props: {
);
}
function ExportAvatar(props: { avatar: string }) {
if (props.avatar === DEFAULT_MASK_AVATAR) {
return (
<img
src={BotIcon.src}
width={30}
height={30}
alt="bot"
className="user-avatar"
/>
);
}
return <Avatar avatar={props.avatar} />;
}
export function ImagePreviewer(props: {
messages: ChatMessage[];
topic: string;
@@ -546,9 +529,12 @@ export function ImagePreviewer(props: {
github.com/ChatGPTNextWeb/ChatGPT-Next-Web
</div>
<div className={styles["icons"]}>
<ExportAvatar avatar={config.avatar} />
<MaskAvatar avatar={config.avatar} />
<span className={styles["icon-space"]}>&</span>
<ExportAvatar avatar={mask.avatar} />
<MaskAvatar
avatar={mask.avatar}
model={session.mask.modelConfig.model}
/>
</div>
</div>
<div>
@@ -576,9 +562,14 @@ export function ImagePreviewer(props: {
key={i}
>
<div className={styles["avatar"]}>
<ExportAvatar
avatar={m.role === "user" ? config.avatar : mask.avatar}
/>
{m.role === "user" ? (
<Avatar avatar={config.avatar}></Avatar>
) : (
<MaskAvatar
avatar={session.mask.avatar}
model={m.model || session.mask.modelConfig.model}
/>
)}
</div>
<div className={styles["body"]}>

View File

@@ -1,5 +1,4 @@
import { useState, useEffect, useMemo } from "react";
import styles from "./settings.module.scss";
import ResetIcon from "../icons/reload.svg";
@@ -967,7 +966,89 @@ export function Settings() {
</ListItem>
</>
);
const bedrockConfigComponent = accessStore.provider ===
ServiceProvider.Bedrock && (
<>
<ListItem
title={Locale.Settings.Access.Bedrock.Region.Title}
subTitle={Locale.Settings.Access.Bedrock.Region.SubTitle}
>
<input
aria-label={Locale.Settings.Access.Bedrock.Region.Title}
type="text"
value={accessStore.awsRegion}
placeholder="us-west-2"
onChange={(e) =>
accessStore.update((access) => {
const region = e.currentTarget.value;
access.awsRegion = region;
})
}
/>
</ListItem>
<ListItem
title={Locale.Settings.Access.Bedrock.AccessKey.Title}
subTitle={Locale.Settings.Access.Bedrock.AccessKey.SubTitle}
>
<PasswordInput
aria-label={Locale.Settings.Access.Bedrock.AccessKey.Title}
value={accessStore.awsAccessKey}
type="text"
placeholder={Locale.Settings.Access.Bedrock.AccessKey.Placeholder}
onChange={(e) => {
accessStore.update((access) => {
const accessKey = e.currentTarget.value;
access.awsAccessKey = accessKey;
});
}}
maskWhenShow={true}
/>
</ListItem>
<ListItem
title={Locale.Settings.Access.Bedrock.SecretKey.Title}
subTitle={Locale.Settings.Access.Bedrock.SecretKey.SubTitle}
>
<PasswordInput
aria-label={Locale.Settings.Access.Bedrock.SecretKey.Title}
value={accessStore.awsSecretKey}
type="text"
placeholder={Locale.Settings.Access.Bedrock.SecretKey.Placeholder}
onChange={(e) => {
accessStore.update((access) => {
const secretKey = e.currentTarget.value;
access.awsSecretKey = secretKey;
});
}}
maskWhenShow={true}
/>
</ListItem>
<ListItem
title={Locale.Settings.Access.Bedrock.EncryptionKey.Title}
subTitle={Locale.Settings.Access.Bedrock.EncryptionKey.SubTitle}
>
<PasswordInput
aria-label={Locale.Settings.Access.Bedrock.EncryptionKey.Title}
value={accessStore.encryptionKey}
type="text"
placeholder={Locale.Settings.Access.Bedrock.EncryptionKey.Placeholder}
onChange={(e) => {
accessStore.update(
(access) => (access.encryptionKey = e.currentTarget.value),
);
}}
onBlur={(e) => {
const value = e.currentTarget.value;
if (!value || value.length < 8) {
showToast(Locale.Settings.Access.Bedrock.EncryptionKey.Invalid);
accessStore.update((access) => (access.encryptionKey = ""));
return;
}
}}
maskWhenShow={true}
/>
</ListItem>
</>
);
const baiduConfigComponent = accessStore.provider ===
ServiceProvider.Baidu && (
<>
@@ -1808,6 +1889,7 @@ export function Settings() {
</ListItem>
{openAIConfigComponent}
{bedrockConfigComponent}
{azureConfigComponent}
{googleConfigComponent}
{anthropicConfigComponent}

View File

@@ -11,6 +11,7 @@ import MaxIcon from "../icons/max.svg";
import MinIcon from "../icons/min.svg";
import Locale from "../locales";
import { maskSensitiveValue } from "../utils/aws";
import { createRoot } from "react-dom/client";
import React, {
@@ -270,13 +271,25 @@ export function Input(props: InputProps) {
}
export function PasswordInput(
props: HTMLProps<HTMLInputElement> & { aria?: string },
props: HTMLProps<HTMLInputElement> & {
aria?: string;
maskWhenShow?: boolean;
},
) {
const [visible, setVisible] = useState(false);
const [isEditing, setIsEditing] = useState(false);
const { maskWhenShow, onChange, value, ...inputProps } = props;
function changeVisibility() {
setVisible(!visible);
}
// Get display value - use masked value only when showing and maskWhenShow is true and not editing
const displayValue =
maskWhenShow && visible && value && !isEditing
? maskSensitiveValue(value as string)
: value;
return (
<div className={"password-input-container"}>
<IconButton
@@ -286,7 +299,11 @@ export function PasswordInput(
className={"password-eye"}
/>
<input
{...props}
{...inputProps}
value={displayValue}
onChange={onChange}
onFocus={() => setIsEditing(true)}
onBlur={() => setIsEditing(false)}
type={visible ? "text" : "password"}
className={"password-input"}
/>
@@ -552,6 +569,7 @@ export function Selector<T>(props: {
</div>
);
}
export function FullScreen(props: any) {
const { children, right = 10, top = 10, ...rest } = props;
const ref = useRef<HTMLDivElement>();

View File

@@ -13,6 +13,12 @@ declare global {
BASE_URL?: string;
OPENAI_ORG_ID?: string; // openai only
// bedrock only
AWS_REGION?: string;
AWS_ACCESS_KEY?: string;
AWS_SECRET_KEY?: string;
ENCRYPTION_KEY?: string;
VERCEL?: string;
BUILD_MODE?: "standalone" | "export";
BUILD_APP?: string; // is building desktop app
@@ -148,7 +154,10 @@ export const getServerSideConfig = () => {
}
const isStability = !!process.env.STABILITY_API_KEY;
const isBedrock =
!!process.env.AWS_REGION &&
!!process.env.AWS_ACCESS_KEY &&
!!process.env.AWS_SECRET_KEY;
const isAzure = !!process.env.AZURE_URL;
const isGoogle = !!process.env.GOOGLE_API_KEY;
const isAnthropic = !!process.env.ANTHROPIC_API_KEY;
@@ -180,6 +189,12 @@ export const getServerSideConfig = () => {
apiKey: getApiKey(process.env.OPENAI_API_KEY),
openaiOrgId: process.env.OPENAI_ORG_ID,
isBedrock,
awsRegion: process.env.AWS_REGION,
awsAccessKey: process.env.AWS_ACCESS_KEY,
awsSecretKey: process.env.AWS_SECRET_KEY,
encryptionKey: process.env.ENCRYPTION_KEY,
isStability,
stabilityUrl: process.env.STABILITY_URL,
stabilityApiKey: getApiKey(process.env.STABILITY_API_KEY),

View File

@@ -56,6 +56,7 @@ export enum Path {
export enum ApiPath {
Cors = "",
Bedrock = "/api/bedrock",
Azure = "/api/azure",
OpenAI = "/api/openai",
Anthropic = "/api/anthropic",
@@ -129,6 +130,7 @@ export enum ServiceProvider {
XAI = "XAI",
ChatGLM = "ChatGLM",
DeepSeek = "DeepSeek",
Bedrock = "Bedrock",
SiliconFlow = "SiliconFlow",
}
@@ -155,6 +157,7 @@ export enum ModelProvider {
XAI = "XAI",
ChatGLM = "ChatGLM",
DeepSeek = "DeepSeek",
Bedrock = "Bedrock",
SiliconFlow = "SiliconFlow",
}
@@ -255,9 +258,19 @@ export const ChatGLM = {
VideoPath: "api/paas/v4/videos/generations",
};
export const Bedrock = {
ChatPath: "model", // Simplified path since we'll append the full path in bedrock.ts
ApiVersion: "2023-11-01",
getEndpoint: (region: string = "us-west-2") =>
`https://bedrock-runtime.${region}.amazonaws.com`,
};
// Get the region from access store for BEDROCK_BASE_URL
export const BEDROCK_BASE_URL = Bedrock.getEndpoint();
export const SiliconFlow = {
ExampleEndpoint: SILICONFLOW_BASE_URL,
ChatPath: "v1/chat/completions",
ListModelPath: "v1/models?&sub_type=chat",
};
export const DEFAULT_INPUT_TEMPLATE = `{{input}}`; // input / time / model / lang
@@ -462,6 +475,9 @@ export const VISION_MODEL_REGEXES = [
/gpt-4-turbo(?!.*preview)/, // Matches "gpt-4-turbo" but not "gpt-4-turbo-preview"
/^dall-e-3$/, // Matches exactly "dall-e-3"
/glm-4v/,
/nova-lite/,
/nova-pro/,
/vl/i,
];
export const EXCLUDE_VISION_MODEL_REGEXES = [/claude-3-5-haiku-20241022/];
@@ -494,6 +510,28 @@ const openaiModels = [
"o3-mini",
];
const bedrockModels = [
// Amazon nova Models
"us.amazon.nova-micro-v1:0",
"us.amazon.nova-lite-v1:0",
"us.amazon.nova-pro-v1:0",
// Claude Models
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-opus-20240229-v1:0",
// Meta Llama Models
"us.meta.llama3-1-8b-instruct-v1:0",
"us.meta.llama3-1-70b-instruct-v1:0",
"us.meta.llama3-2-11b-instruct-v1:0",
"us.meta.llama3-2-90b-instruct-v1:0",
"us.meta.llama3-3-70b-instruct-v1:0",
// Mistral Models
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-large-2407-v1:0",
];
const googleModels = [
"gemini-1.0-pro", // Deprecated on 2/15/2025
"gemini-1.5-pro-latest",
@@ -761,6 +799,7 @@ export const DEFAULT_MODELS = [
sorted: 11,
},
})),
...chatglmModels.map((name) => ({
name,
available: true,
@@ -794,6 +833,17 @@ export const DEFAULT_MODELS = [
sorted: 14,
},
})),
...bedrockModels.map((name) => ({
name,
available: true,
sorted: seq++,
provider: {
id: "bedrock",
providerName: "Bedrock",
providerType: "bedrock",
sorted: 15,
},
})),
] as const;
export const CHAT_PAGE_SIZE = 15;
@@ -814,5 +864,5 @@ export const internalAllowedWebDavEndpoints = [
export const DEFAULT_GA_ID = "G-89WN60ZK2E";
export const SAAS_CHAT_URL = "https://nextchat.dev/chat";
export const SAAS_CHAT_UTM_URL = "https://nextchat.dev/chat?utm=github";
export const SAAS_CHAT_URL = "https://nextchat.club";
export const SAAS_CHAT_UTM_URL = "https://nextchat.club?utm=github";

View File

@@ -343,6 +343,32 @@ const cn = {
SubTitle: "除默认地址外,必须包含 http(s)://",
},
},
Bedrock: {
Region: {
Title: "AWS 区域",
SubTitle: "Bedrock 服务所在的 AWS 区域",
Placeholder: "us-west-2",
Invalid: "无效的 AWS 区域格式。示例us-west-2",
},
AccessKey: {
Title: "AWS 访问密钥 ID",
SubTitle: "用于 Bedrock 服务的 AWS 访问密钥 ID",
Placeholder: "AKIA...",
Invalid: "无效的 AWS Access Key 格式。必须为20个字符。",
},
SecretKey: {
Title: "AWS 私有访问密钥",
SubTitle: "用于 Bedrock 服务的 AWS 私有访问密钥",
Placeholder: "****",
Invalid: "无效的 AWS Secret Key 格式。必须为40个字符。",
},
EncryptionKey: {
Title: "加密密钥",
SubTitle: "用于配置数据的加密密钥",
Placeholder: "输入加密密钥",
Invalid: "无效的加密密钥。必须至少包含8个字符",
},
},
Azure: {
ApiKey: {
Title: "接口密钥",

832
app/locales/da.ts Normal file
View File

@@ -0,0 +1,832 @@
import { getClientConfig } from "../config/client";
import { SubmitKey } from "../store/config";
import { SAAS_CHAT_UTM_URL } from "@/app/constant";
import { PartialLocaleType } from "./index";
const isApp = !!getClientConfig()?.isApp;
const da: PartialLocaleType = {
WIP: "Der kommer snart mere...",
Error: {
Unauthorized: isApp
? `Hov, der skete en fejl. Sådan kan du komme videre:
\\ 1⃣ Er du ny her? [Tryk for at starte nu 🚀](${SAAS_CHAT_UTM_URL})
\\ 2⃣ Vil du bruge dine egne OpenAI-nøgler? [Tryk her](/#/settings) for at ændre indstillinger ⚙️`
: `Hov, der skete en fejl. Lad os løse det:
\\ 1⃣ Er du ny her? [Tryk for at starte nu 🚀](${SAAS_CHAT_UTM_URL})
\\ 2⃣ Bruger du en privat opsætning? [Tryk her](/#/auth) for at taste din nøgle 🔑
\\ 3⃣ Vil du bruge dine egne OpenAI-nøgler? [Tryk her](/#/settings) for at ændre indstillinger ⚙️
`,
},
Auth: {
Return: "Tilbage",
Title: "Adgangskode",
Tips: "Skriv venligst koden herunder",
SubTips: "Eller brug din egen OpenAI- eller Google-nøgle",
Input: "Adgangskode",
Confirm: "OK",
Later: "Senere",
SaasTips: "Hvis det er for svært, kan du starte nu",
},
ChatItem: {
ChatItemCount: (count: number) => `${count} beskeder`,
},
Chat: {
SubTitle: (count: number) => `${count} beskeder`,
EditMessage: {
Title: "Rediger beskeder",
Topic: {
Title: "Emne",
SubTitle: "Skift emne for denne chat",
},
},
Actions: {
ChatList: "Gå til chatliste",
CompressedHistory: "Komprimeret historie",
Export: "Eksporter alle beskeder som Markdown",
Copy: "Kopiér",
Stop: "Stop",
Retry: "Prøv igen",
Pin: "Fastgør",
PinToastContent: "1 besked er nu fastgjort",
PinToastAction: "Se",
Delete: "Slet",
Edit: "Rediger",
FullScreen: "Fuld skærm",
RefreshTitle: "Opdatér titel",
RefreshToast: "Anmodning om ny titel sendt",
Speech: "Afspil",
StopSpeech: "Stop",
},
Commands: {
new: "Ny chat",
newm: "Ny chat med persona",
next: "Næste chat",
prev: "Forrige chat",
clear: "Ryd alt før",
fork: "Kopiér chat",
del: "Slet chat",
},
InputActions: {
Stop: "Stop",
ToBottom: "Ned til nyeste",
Theme: {
auto: "Automatisk",
light: "Lyst tema",
dark: "Mørkt tema",
},
Prompt: "Prompts",
Masks: "Personaer",
Clear: "Ryd kontekst",
Settings: "Indstillinger",
UploadImage: "Upload billeder",
},
Rename: "Omdøb chat",
Typing: "Skriver…",
Input: (submitKey: string) => {
let inputHints = `${submitKey} for at sende`;
if (submitKey === String(SubmitKey.Enter)) {
inputHints += ", Shift + Enter for ny linje";
}
return (
inputHints + ", / for at søge i prompts, : for at bruge kommandoer"
);
},
Send: "Send",
StartSpeak: "Start oplæsning",
StopSpeak: "Stop oplæsning",
Config: {
Reset: "Nulstil til standard",
SaveAs: "Gem som persona",
},
IsContext: "Ekstra prompt til baggrund",
ShortcutKey: {
Title: "Hurtigtaster",
newChat: "Åbn ny chat",
focusInput: "Fokus på tekstfeltet",
copyLastMessage: "Kopiér sidste svar",
copyLastCode: "Kopiér sidste kodeblok",
showShortcutKey: "Vis hurtigtaster",
clearContext: "Ryd kontekst",
},
},
Export: {
Title: "Eksportér beskeder",
Copy: "Kopiér alt",
Download: "Download",
MessageFromYou: "Fra dig",
MessageFromChatGPT: "Fra ChatGPT",
Share: "Del til ShareGPT",
Format: {
Title: "Filformat",
SubTitle: "Vælg enten Markdown eller PNG-billede",
},
IncludeContext: {
Title: "Tag baggrund med",
SubTitle: "Skal ekstra baggrund (persona) med i eksporten?",
},
Steps: {
Select: "Vælg",
Preview: "Forhåndsvis",
},
Image: {
Toast: "Laver billede...",
Modal: "Tryk længe eller højreklik for at gemme",
},
Artifacts: {
Title: "Del side",
Error: "Fejl ved deling",
},
},
Select: {
Search: "Søg",
All: "Vælg alle",
Latest: "Vælg nyeste",
Clear: "Ryd alt",
},
Memory: {
Title: "Huskesætning",
EmptyContent: "Ingenting lige nu.",
Send: "Send huskesætning",
Copy: "Kopiér huskesætning",
Reset: "Nulstil chat",
ResetConfirm:
"Dette sletter nuværende samtale og hukommelse. Er du sikker?",
},
Home: {
NewChat: "Ny Chat",
DeleteChat: "Vil du slette den valgte chat?",
DeleteToast: "Chat slettet",
Revert: "Fortryd",
},
Settings: {
Title: "Indstillinger",
SubTitle: "Alle indstillinger",
ShowPassword: "Vis kodeord",
Danger: {
Reset: {
Title: "Nulstil alle indstillinger",
SubTitle: "Gendan alt til standard",
Action: "Nulstil",
Confirm: "Vil du virkelig nulstille alt?",
},
Clear: {
Title: "Slet alle data",
SubTitle: "Sletter alt om beskeder og indstillinger",
Action: "Slet",
Confirm: "Er du sikker på, at du vil slette alt?",
},
},
Lang: {
Name: "Language",
All: "Alle sprog",
},
Avatar: "Avatar",
FontSize: {
Title: "Skriftstørrelse",
SubTitle: "Vælg, hvor stor teksten skal være",
},
FontFamily: {
Title: "Skrifttype",
SubTitle: "Hvis tom, bruger den standard skrifttype",
Placeholder: "Skrifttype-navn",
},
InjectSystemPrompts: {
Title: "Tilføj system-prompt",
SubTitle: "Læg altid en ekstra prompt først i anmodninger",
},
InputTemplate: {
Title: "Tekstskabelon",
SubTitle: "Den seneste besked placeres i denne skabelon",
},
Update: {
Version: (x: string) => `Version: ${x}`,
IsLatest: "Du har nyeste version",
CheckUpdate: "Tjek efter opdatering",
IsChecking: "Tjekker...",
FoundUpdate: (x: string) => `Ny version fundet: ${x}`,
GoToUpdate: "Opdatér",
Success: "Opdatering lykkedes.",
Failed: "Opdatering mislykkedes.",
},
SendKey: "Tast for send",
Theme: "Tema",
TightBorder: "Stram kant",
SendPreviewBubble: {
Title: "Forhåndsvisnings-boble",
SubTitle: "Vis tekst, før den sendes",
},
AutoGenerateTitle: {
Title: "Lav titel automatisk",
SubTitle: "Foreslå en titel ud fra chatten",
},
Sync: {
CloudState: "Seneste opdatering",
NotSyncYet: "Endnu ikke synkroniseret",
Success: "Synkronisering lykkedes",
Fail: "Synkronisering mislykkedes",
Config: {
Modal: {
Title: "Indstil synk",
Check: "Tjek forbindelse",
},
SyncType: {
Title: "Synk-type",
SubTitle: "Vælg en synk-tjeneste",
},
Proxy: {
Title: "Aktivér proxy",
SubTitle: "Brug proxy for at undgå netværksproblemer",
},
ProxyUrl: {
Title: "Proxy-adresse",
SubTitle: "Bruges kun til projektets egen proxy",
},
WebDav: {
Endpoint: "WebDAV-adresse",
UserName: "Brugernavn",
Password: "Kodeord",
},
UpStash: {
Endpoint: "UpStash Redis REST URL",
UserName: "Backup-navn",
Password: "UpStash Redis REST Token",
},
},
LocalState: "Lokale data",
Overview: (overview: any) =>
`${overview.chat} chats, ${overview.message} beskeder, ${overview.prompt} prompts, ${overview.mask} personaer`,
ImportFailed: "Import mislykkedes",
},
Mask: {
Splash: {
Title: "Persona-forside",
SubTitle: "Vis denne side, når du opretter ny chat",
},
Builtin: {
Title: "Skjul indbyggede personaer",
SubTitle: "Vis ikke de indbyggede personaer i listen",
},
},
Prompt: {
Disable: {
Title: "Slå auto-forslag fra",
SubTitle: "Tast / for at få forslag",
},
List: "Prompt-liste",
ListCount: (builtin: number, custom: number) =>
`${builtin} indbygget, ${custom} brugerdefineret`,
Edit: "Rediger",
Modal: {
Title: "Prompt-liste",
Add: "Tilføj",
Search: "Søg prompts",
},
EditModal: {
Title: "Rediger prompt",
},
},
HistoryCount: {
Title: "Antal beskeder, der følger med",
SubTitle: "Hvor mange af de tidligere beskeder, der sendes hver gang",
},
CompressThreshold: {
Title: "Komprimeringsgrænse",
SubTitle:
"Hvis chatten bliver for lang, vil den komprimeres efter dette antal tegn",
},
Usage: {
Title: "Brug og saldo",
SubTitle(used: any, total: any) {
return `Du har brugt $${used} i denne måned, og din grænse er $${total}.`;
},
IsChecking: "Tjekker...",
Check: "Tjek igen",
NoAccess: "Indtast API-nøgle for at se forbrug",
},
Access: {
AccessCode: {
Title: "Adgangskode",
SubTitle: "Adgangskontrol er slået til",
Placeholder: "Skriv kode her",
},
CustomEndpoint: {
Title: "Brugerdefineret adresse",
SubTitle: "Brug Azure eller OpenAI fra egen server",
},
Provider: {
Title: "Model-udbyder",
SubTitle: "Vælg Azure eller OpenAI",
},
OpenAI: {
ApiKey: {
Title: "OpenAI API-nøgle",
SubTitle: "Brug din egen nøgle",
Placeholder: "sk-xxx",
},
Endpoint: {
Title: "OpenAI Endpoint",
SubTitle: "Skal starte med http(s):// eller /api/openai som standard",
},
},
Azure: {
ApiKey: {
Title: "Azure Api Key",
SubTitle: "Hent din nøgle fra Azure-portalen",
Placeholder: "Azure Api Key",
},
Endpoint: {
Title: "Azure Endpoint",
SubTitle: "F.eks.: ",
},
ApiVerion: {
Title: "Azure Api Version",
SubTitle: "Hentet fra Azure-portalen",
},
},
Anthropic: {
ApiKey: {
Title: "Anthropic API-nøgle",
SubTitle: "Brug din egen Anthropic-nøgle",
Placeholder: "Anthropic API Key",
},
Endpoint: {
Title: "Endpoint-adresse",
SubTitle: "F.eks.: ",
},
ApiVerion: {
Title: "API-version (Claude)",
SubTitle: "Vælg den ønskede version",
},
},
Baidu: {
ApiKey: {
Title: "Baidu-nøgle",
SubTitle: "Din egen Baidu-nøgle",
Placeholder: "Baidu API Key",
},
SecretKey: {
Title: "Baidu hemmelig nøgle",
SubTitle: "Din egen hemmelige nøgle fra Baidu",
Placeholder: "Baidu Secret Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "Kan ikke ændres, se .env",
},
},
Tencent: {
ApiKey: {
Title: "Tencent-nøgle",
SubTitle: "Din egen nøgle fra Tencent",
Placeholder: "Tencent API Key",
},
SecretKey: {
Title: "Tencent hemmelig nøgle",
SubTitle: "Din egen hemmelige nøgle fra Tencent",
Placeholder: "Tencent Secret Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "Kan ikke ændres, se .env",
},
},
ByteDance: {
ApiKey: {
Title: "ByteDance-nøgle",
SubTitle: "Din egen nøgle til ByteDance",
Placeholder: "ByteDance API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
Alibaba: {
ApiKey: {
Title: "Alibaba-nøgle",
SubTitle: "Din egen Alibaba Cloud-nøgle",
Placeholder: "Alibaba Cloud API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
Moonshot: {
ApiKey: {
Title: "Moonshot-nøgle",
SubTitle: "Din egen Moonshot-nøgle",
Placeholder: "Moonshot API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
DeepSeek: {
ApiKey: {
Title: "DeepSeek-nøgle",
SubTitle: "Din egen DeepSeek-nøgle",
Placeholder: "DeepSeek API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
XAI: {
ApiKey: {
Title: "XAI-nøgle",
SubTitle: "Din egen XAI-nøgle",
Placeholder: "XAI API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
ChatGLM: {
ApiKey: {
Title: "ChatGLM-nøgle",
SubTitle: "Din egen ChatGLM-nøgle",
Placeholder: "ChatGLM API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
SiliconFlow: {
ApiKey: {
Title: "SiliconFlow-nøgle",
SubTitle: "Din egen SiliconFlow-nøgle",
Placeholder: "SiliconFlow API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
Stability: {
ApiKey: {
Title: "Stability-nøgle",
SubTitle: "Din egen Stability-nøgle",
Placeholder: "Stability API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
Iflytek: {
ApiKey: {
Title: "Iflytek API Key",
SubTitle: "Nøgle fra Iflytek",
Placeholder: "Iflytek API Key",
},
ApiSecret: {
Title: "Iflytek hemmelig nøgle",
SubTitle: "Hentet fra Iflytek",
Placeholder: "Iflytek API Secret",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
},
CustomModel: {
Title: "Egne modelnavne",
SubTitle: "Skriv komma-adskilte navne",
},
Google: {
ApiKey: {
Title: "Google-nøgle",
SubTitle: "Få din nøgle hos Google AI",
Placeholder: "Google AI API Key",
},
Endpoint: {
Title: "Adresse",
SubTitle: "F.eks.: ",
},
ApiVersion: {
Title: "API-version (til gemini-pro)",
SubTitle: "Vælg en bestemt version",
},
GoogleSafetySettings: {
Title: "Google sikkerhedsindstillinger",
SubTitle: "Vælg et niveau for indholdskontrol",
},
},
},
Model: "Model",
CompressModel: {
Title: "Opsummeringsmodel",
SubTitle: "Bruges til at korte historik ned og lave titel",
},
Temperature: {
Title: "Temperatur",
SubTitle: "Jo højere tal, jo mere kreativt svar",
},
TopP: {
Title: "Top P",
SubTitle: "Skal ikke ændres sammen med temperatur",
},
MaxTokens: {
Title: "Maks. længde",
SubTitle: "Hvor mange tokens (ord/stykker tekst) der kan bruges",
},
PresencePenalty: {
Title: "Nye emner",
SubTitle: "Jo højere tal, jo mere nyt indhold",
},
FrequencyPenalty: {
Title: "Gentagelsesstraf",
SubTitle: "Jo højere tal, jo mindre gentagelse",
},
TTS: {
Enable: {
Title: "Tænd for oplæsning (TTS)",
SubTitle: "Slå tekst-til-tale til",
},
Autoplay: {
Title: "Automatisk oplæsning",
SubTitle: "Laver lyd automatisk, hvis TTS er slået til",
},
Model: "Model",
Voice: {
Title: "Stemme",
SubTitle: "Hvilken stemme der bruges til lyd",
},
Speed: {
Title: "Hastighed",
SubTitle: "Hvor hurtigt der oplæses",
},
Engine: "TTS-motor",
},
Realtime: {
Enable: {
Title: "Live-chat",
SubTitle: "Slå live-svar til",
},
Provider: {
Title: "Modeludbyder",
SubTitle: "Vælg forskellig udbyder",
},
Model: {
Title: "Model",
SubTitle: "Vælg en model",
},
ApiKey: {
Title: "API-nøgle",
SubTitle: "Din nøgle",
Placeholder: "API-nøgle",
},
Azure: {
Endpoint: {
Title: "Adresse",
SubTitle: "Endpoint til Azure",
},
Deployment: {
Title: "Udrulningsnavn",
SubTitle: "Navn for dit Azure-setup",
},
},
Temperature: {
Title: "Temperatur",
SubTitle: "Højere tal = mere varierede svar",
},
},
},
Store: {
DefaultTopic: "Ny samtale",
BotHello: "Hej! Hvordan kan jeg hjælpe dig i dag?",
Error: "Noget gik galt. Prøv igen senere.",
Prompt: {
History: (content: string) =>
"Her er et kort resume af, hvad vi har snakket om: " + content,
Topic:
"Find en kort overskrift med 4-5 ord om emnet. Ingen tegnsætning eller anførselstegn.",
Summarize:
"Skriv et kort resumé (under 200 ord) af vores samtale til senere brug.",
},
},
Copy: {
Success: "Kopieret",
Failed: "Kunne ikke kopiere. Giv adgang til udklipsholder.",
},
Download: {
Success: "Filen er downloadet.",
Failed: "Download fejlede.",
},
Context: {
Toast: (x: any) => `Inkluderer ${x} ekstra prompts`,
Edit: "Chatindstillinger",
Add: "Tilføj prompt",
Clear: "Kontekst ryddet",
Revert: "Fortryd",
},
Discovery: {
Name: "Søgning og plugins",
},
Mcp: {
Name: "MCP",
},
FineTuned: {
Sysmessage: "Du er en hjælper, der skal...",
},
SearchChat: {
Name: "Søg",
Page: {
Title: "Søg i tidligere chats",
Search: "Skriv her for at søge",
NoResult: "Ingen resultater",
NoData: "Ingen data",
Loading: "Henter...",
SubTitle: (count: number) => `Fandt ${count} resultater`,
},
Item: {
View: "Vis",
},
},
Plugin: {
Name: "Plugin",
Page: {
Title: "Plugins",
SubTitle: (count: number) => `${count} plugins`,
Search: "Søg plugin",
Create: "Opret nyt",
Find: "Du kan finde flere plugins på GitHub: ",
},
Item: {
Info: (count: number) => `${count} metode`,
View: "Vis",
Edit: "Rediger",
Delete: "Slet",
DeleteConfirm: "Vil du slette?",
},
Auth: {
None: "Ingen",
Basic: "Basic",
Bearer: "Bearer",
Custom: "Tilpasset",
CustomHeader: "Parameternavn",
Token: "Token",
Proxy: "Brug Proxy",
ProxyDescription: "Løs CORS-problemer med Proxy",
Location: "Sted",
LocationHeader: "Header",
LocationQuery: "Query",
LocationBody: "Body",
},
EditModal: {
Title: (readonly: boolean) =>
`Rediger Plugin ${readonly ? "(skrivebeskyttet)" : ""}`,
Download: "Download",
Auth: "Godkendelsestype",
Content: "OpenAPI Schema",
Load: "Hent fra URL",
Method: "Metode",
Error: "Fejl i OpenAPI Schema",
},
},
Mask: {
Name: "Persona",
Page: {
Title: "Prompts som personaer",
SubTitle: (count: number) => `${count} skabeloner`,
Search: "Søg skabeloner",
Create: "Opret ny",
},
Item: {
Info: (count: number) => `${count} prompts`,
Chat: "Chat",
View: "Vis",
Edit: "Rediger",
Delete: "Slet",
DeleteConfirm: "Vil du slette?",
},
EditModal: {
Title: (readonly: boolean) =>
`Rediger skabelon ${readonly ? "(skrivebeskyttet)" : ""}`,
Download: "Download",
Clone: "Klon",
},
Config: {
Avatar: "Chat-avatar",
Name: "Chat-navn",
Sync: {
Title: "Brug globale indstillinger",
SubTitle: "Gældende for denne chat",
Confirm: "Erstat nuværende indstillinger med globale?",
},
HideContext: {
Title: "Skjul ekstra prompts",
SubTitle: "Vis dem ikke på chat-skærmen",
},
Artifacts: {
Title: "Brug Artefakter",
SubTitle: "Gør det muligt at vise HTML-sider",
},
CodeFold: {
Title: "Fold kode sammen",
SubTitle: "Luk/åbn lange kodestykker automatisk",
},
Share: {
Title: "Del denne persona",
SubTitle: "Få et link til denne skabelon",
Action: "Kopiér link",
},
},
},
NewChat: {
Return: "Tilbage",
Skip: "Start straks",
Title: "Vælg en persona",
SubTitle: "Chat med den persona, du vælger",
More: "Se flere",
NotShow: "Vis ikke igen",
ConfirmNoShow:
"Er du sikker på, at du ikke vil se det igen? Du kan altid slå det til under indstillinger.",
},
UI: {
Confirm: "OK",
Cancel: "Fortryd",
Close: "Luk",
Create: "Opret",
Edit: "Rediger",
Export: "Eksporter",
Import: "Importér",
Sync: "Synk",
Config: "Konfigurer",
},
Exporter: {
Description: {
Title: "Kun beskeder efter sidste rydning vises",
},
Model: "Model",
Messages: "Beskeder",
Topic: "Emne",
Time: "Tid",
},
URLCommand: {
Code: "Så ud til, at der var en kode i linket. Vil du bruge den?",
Settings: "Så ud til, at der var indstillinger i linket. Vil du bruge dem?",
},
SdPanel: {
Prompt: "Prompt",
NegativePrompt: "Negativ prompt",
PleaseInput: (name: string) => `Indtast: ${name}`,
AspectRatio: "Billedformat",
ImageStyle: "Stil",
OutFormat: "Uddataformat",
AIModel: "AI-model",
ModelVersion: "Version",
Submit: "Send",
ParamIsRequired: (name: string) => `${name} er krævet`,
Styles: {
D3Model: "3d-model",
AnalogFilm: "analog-film",
Anime: "anime",
Cinematic: "cinematisk",
ComicBook: "tegneserie",
DigitalArt: "digital-art",
Enhance: "enhance",
FantasyArt: "fantasy-art",
Isometric: "isometric",
LineArt: "line-art",
LowPoly: "low-poly",
ModelingCompound: "modeling-compound",
NeonPunk: "neon-punk",
Origami: "origami",
Photographic: "fotografisk",
PixelArt: "pixel-art",
TileTexture: "tile-texture",
},
},
Sd: {
SubTitle: (count: number) => `${count} billeder`,
Actions: {
Params: "Se indstillinger",
Copy: "Kopiér prompt",
Delete: "Slet",
Retry: "Prøv igen",
ReturnHome: "Til forsiden",
History: "Historik",
},
EmptyRecord: "Ingen billeder endnu",
Status: {
Name: "Status",
Success: "Ok",
Error: "Fejl",
Wait: "Venter",
Running: "I gang",
},
Danger: {
Delete: "Vil du slette?",
},
GenerateParams: "Genereringsvalg",
Detail: "Detaljer",
},
};
export default da;

View File

@@ -347,6 +347,33 @@ const en: LocaleType = {
SubTitle: "Must start with http(s):// or use /api/openai as default",
},
},
Bedrock: {
Region: {
Title: "AWS Region",
SubTitle: "The AWS region where Bedrock service is located",
Placeholder: "us-west-2",
Invalid: "Invalid AWS region format. Example: us-west-2",
},
AccessKey: {
Title: "AWS Access Key ID",
SubTitle: "Your AWS access key ID for Bedrock service",
Placeholder: "AKIA...",
Invalid: "Invalid AWS access key format. Must be 20 characters long.",
},
SecretKey: {
Title: "AWS Secret Access Key",
SubTitle: "Your AWS secret access key for Bedrock service",
Placeholder: "****",
Invalid: "Invalid AWS secret key format. Must be 40 characters long.",
},
EncryptionKey: {
Title: "Encryption Key",
SubTitle: "Your encryption key for configuration data",
Placeholder: "Enter encryption key",
Invalid:
"Invalid encryption key format. Must no less than 8 characters long!",
},
},
Azure: {
ApiKey: {
Title: "Azure Api Key",

View File

@@ -2,6 +2,7 @@ import cn from "./cn";
import en from "./en";
import pt from "./pt";
import tw from "./tw";
import da from "./da";
import id from "./id";
import fr from "./fr";
import es from "./es";
@@ -30,6 +31,7 @@ const ALL_LANGS = {
en,
tw,
pt,
da,
jp,
ko,
id,
@@ -56,6 +58,7 @@ export const ALL_LANG_OPTIONS: Record<Lang, string> = {
en: "English",
pt: "Português",
tw: "繁體中文",
da: "Dansk",
jp: "日本語",
ko: "한국어",
id: "Indonesia",
@@ -141,6 +144,7 @@ export const STT_LANG_MAP: Record<Lang, string> = {
en: "en-US",
pt: "pt-BR",
tw: "zh-TW",
da: "da-DK",
jp: "ja-JP",
ko: "ko-KR",
id: "id-ID",

View File

@@ -16,6 +16,7 @@ import {
DEEPSEEK_BASE_URL,
XAI_BASE_URL,
CHATGLM_BASE_URL,
BEDROCK_BASE_URL,
SILICONFLOW_BASE_URL,
} from "../constant";
import { getHeaders } from "../client/api";
@@ -24,36 +25,26 @@ import { createPersistStore } from "../utils/store";
import { ensure } from "../utils/clone";
import { DEFAULT_CONFIG } from "./config";
import { getModelProvider } from "../utils/model";
import { encrypt, decrypt } from "../utils/aws";
let fetchState = 0; // 0 not fetch, 1 fetching, 2 done
const isApp = getClientConfig()?.buildMode === "export";
const DEFAULT_OPENAI_URL = isApp ? OPENAI_BASE_URL : ApiPath.OpenAI;
const DEFAULT_GOOGLE_URL = isApp ? GEMINI_BASE_URL : ApiPath.Google;
const DEFAULT_ANTHROPIC_URL = isApp ? ANTHROPIC_BASE_URL : ApiPath.Anthropic;
const DEFAULT_BAIDU_URL = isApp ? BAIDU_BASE_URL : ApiPath.Baidu;
const DEFAULT_BYTEDANCE_URL = isApp ? BYTEDANCE_BASE_URL : ApiPath.ByteDance;
const DEFAULT_ALIBABA_URL = isApp ? ALIBABA_BASE_URL : ApiPath.Alibaba;
const DEFAULT_TENCENT_URL = isApp ? TENCENT_BASE_URL : ApiPath.Tencent;
const DEFAULT_MOONSHOT_URL = isApp ? MOONSHOT_BASE_URL : ApiPath.Moonshot;
const DEFAULT_STABILITY_URL = isApp ? STABILITY_BASE_URL : ApiPath.Stability;
const DEFAULT_IFLYTEK_URL = isApp ? IFLYTEK_BASE_URL : ApiPath.Iflytek;
const DEFAULT_DEEPSEEK_URL = isApp ? DEEPSEEK_BASE_URL : ApiPath.DeepSeek;
const DEFAULT_XAI_URL = isApp ? XAI_BASE_URL : ApiPath.XAI;
const DEFAULT_CHATGLM_URL = isApp ? CHATGLM_BASE_URL : ApiPath.ChatGLM;
const DEFAULT_BEDROCK_URL = isApp ? BEDROCK_BASE_URL : ApiPath.Bedrock;
const DEFAULT_SILICONFLOW_URL = isApp
? SILICONFLOW_BASE_URL
@@ -128,10 +119,19 @@ const DEFAULT_ACCESS_STATE = {
chatglmUrl: DEFAULT_CHATGLM_URL,
chatglmApiKey: "",
// aws bedrock
bedrockUrl: DEFAULT_BEDROCK_URL,
awsRegion: "",
awsAccessKey: "",
awsSecretKey: "",
encryptionKey: "",
bedrockAnthropicVersion: "bedrock-2023-05-31",
// siliconflow
siliconflowUrl: DEFAULT_SILICONFLOW_URL,
siliconflowApiKey: "",
// server config
needCode: true,
hideUserApiKey: false,
@@ -148,11 +148,9 @@ const DEFAULT_ACCESS_STATE = {
export const useAccessStore = createPersistStore(
{ ...DEFAULT_ACCESS_STATE },
(set, get) => ({
enabledAccessControl() {
this.fetch();
return get().needCode;
},
getVisionModels() {
@@ -161,7 +159,6 @@ export const useAccessStore = createPersistStore(
},
edgeVoiceName() {
this.fetch();
return get().edgeTTSVoiceName;
},
@@ -200,6 +197,7 @@ export const useAccessStore = createPersistStore(
isValidMoonshot() {
return ensure(get(), ["moonshotApiKey"]);
},
isValidIflytek() {
return ensure(get(), ["iflytekApiKey"]);
},
@@ -215,8 +213,19 @@ export const useAccessStore = createPersistStore(
return ensure(get(), ["chatglmApiKey"]);
},
isValidBedrock() {
return ensure(get(), [
"awsRegion",
"awsAccessKey",
"awsSecretKey",
"encryptionKey",
]);
},
isValidSiliconFlow() {
return ensure(get(), ["siliconflowApiKey"]);
},
isAuthorized() {
@@ -237,11 +246,13 @@ export const useAccessStore = createPersistStore(
this.isValidDeepSeek() ||
this.isValidXAI() ||
this.isValidChatGLM() ||
this.isValidBedrock() ||
this.isValidSiliconFlow() ||
!this.enabledAccessControl() ||
(this.enabledAccessControl() && ensure(get(), ["accessCode"]))
);
},
fetch() {
if (fetchState > 0 || getClientConfig()?.buildMode === "export") return;
fetchState = 1;
@@ -260,7 +271,6 @@ export const useAccessStore = createPersistStore(
DEFAULT_CONFIG.modelConfig.model = model;
DEFAULT_CONFIG.modelConfig.providerName = providerName as any;
}
return res;
})
.then((res: DangerConfig) => {
@@ -274,6 +284,43 @@ export const useAccessStore = createPersistStore(
fetchState = 2;
});
},
// Override the set method to encrypt AWS credentials before storage
set: (partial: { [key: string]: any }) => {
if (partial.awsAccessKey) {
partial.awsAccessKey = encrypt(
partial.awsAccessKey,
partial.encryptionKey,
);
}
if (partial.awsSecretKey) {
partial.awsSecretKey = encrypt(
partial.awsSecretKey,
partial.encryptionKey,
);
}
if (partial.awsRegion) {
partial.awsRegion = encrypt(partial.awsRegion, partial.encryptionKey);
}
set(partial);
},
// Add getter to decrypt AWS credentials when needed
get: () => {
const state = get();
return {
...state,
awsRegion: state.awsRegion
? decrypt(state.awsRegion, state.encryptionKey)
: "",
awsAccessKey: state.awsAccessKey
? decrypt(state.awsAccessKey, state.encryptionKey)
: "",
awsSecretKey: state.awsSecretKey
? decrypt(state.awsSecretKey, state.encryptionKey)
: "",
};
},
}),
{
name: StoreKey.Access,

View File

@@ -2,7 +2,11 @@ import { useEffect, useState } from "react";
import { showToast } from "./components/ui-lib";
import Locale from "./locales";
import { RequestMessage } from "./client/api";
import { ServiceProvider } from "./constant";
import {
REQUEST_TIMEOUT_MS,
REQUEST_TIMEOUT_MS_FOR_THINKING,
ServiceProvider,
} from "./constant";
// import { fetch as tauriFetch, ResponseType } from "@tauri-apps/api/http";
import { fetch as tauriStreamFetch } from "./utils/stream";
import { VISION_MODEL_REGEXES, EXCLUDE_VISION_MODEL_REGEXES } from "./constant";
@@ -292,6 +296,20 @@ export function isDalle3(model: string) {
return "dall-e-3" === model;
}
export function getTimeoutMSByModel(model: string) {
model = model.toLowerCase();
if (
model.startsWith("dall-e") ||
model.startsWith("dalle") ||
model.startsWith("o1") ||
model.startsWith("o3") ||
model.includes("deepseek-r") ||
model.includes("-thinking")
)
return REQUEST_TIMEOUT_MS_FOR_THINKING;
return REQUEST_TIMEOUT_MS;
}
export function getModelSizes(model: string): ModelSize[] {
if (isDalle3(model)) {
return ["1024x1024", "1792x1024", "1024x1792"];
@@ -326,6 +344,13 @@ export function showPlugins(provider: ServiceProvider, model: string) {
if (provider == ServiceProvider.Anthropic && !model.includes("claude-2")) {
return true;
}
if (
(provider == ServiceProvider.Bedrock && model.includes("claude-3")) ||
model.includes("mistral-large") ||
model.includes("amazon.nova")
) {
return true;
}
if (provider == ServiceProvider.Google && !model.includes("vision")) {
return true;
}

696
app/utils/aws.ts Normal file
View File

@@ -0,0 +1,696 @@
// Types and Interfaces
export interface BedrockCredentials {
region: string;
accessKeyId: string;
secretAccessKey: string;
}
// Type definitions for better type safety
type ParsedEvent = Record<string, any>;
type EventResult = ParsedEvent[];
// Using a dot as separator since it's not used in Base64
const SEPARATOR = "~";
// Unified crypto utilities for both frontend and backend
async function generateKey(
password: string,
salt: Uint8Array,
): Promise<CryptoKey> {
const enc = new TextEncoder();
const keyMaterial = await crypto.subtle.importKey(
"raw",
enc.encode(password),
{ name: "PBKDF2" },
false,
["deriveBits", "deriveKey"],
);
return crypto.subtle.deriveKey(
{
name: "PBKDF2",
salt,
iterations: 100000,
hash: "SHA-256",
},
keyMaterial,
{ name: "AES-GCM", length: 256 },
false,
["encrypt", "decrypt"],
);
}
function arrayBufferToBase64(buffer: ArrayBuffer | Uint8Array): string {
const bytes = buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
return btoa(String.fromCharCode(...bytes));
}
function base64ToArrayBuffer(base64: string): Uint8Array {
const binaryString = atob(base64);
const bytes = new Uint8Array(binaryString.length);
for (let i = 0; i < binaryString.length; i++) {
bytes[i] = binaryString.charCodeAt(i);
}
return bytes;
}
export async function encrypt(
data: string,
encryptionKey: string,
): Promise<string> {
if (!data) return "";
if (!encryptionKey) {
throw new Error("Encryption key is required for AWS credential encryption");
}
try {
const enc = new TextEncoder();
const salt = crypto.getRandomValues(new Uint8Array(16));
const iv = crypto.getRandomValues(new Uint8Array(12));
const key = await generateKey(encryptionKey, salt);
const encrypted = await crypto.subtle.encrypt(
{
name: "AES-GCM",
iv,
},
key,
enc.encode(data),
);
// Convert to base64 strings
const encryptedBase64 = arrayBufferToBase64(encrypted);
const saltBase64 = arrayBufferToBase64(salt);
const ivBase64 = arrayBufferToBase64(iv);
return [saltBase64, ivBase64, encryptedBase64].join(SEPARATOR);
} catch (error) {
// console.error("[Encryption Error]:", error);
throw new Error("Failed to encrypt AWS credentials");
}
}
export async function decrypt(
encryptedData: string,
encryptionKey: string,
): Promise<string> {
if (!encryptedData) return "";
if (!encryptionKey) {
throw new Error("Encryption key is required for AWS credential decryption");
}
try {
const [saltBase64, ivBase64, cipherBase64] = encryptedData.split(SEPARATOR);
// Convert base64 strings back to Uint8Arrays
const salt = base64ToArrayBuffer(saltBase64);
const iv = base64ToArrayBuffer(ivBase64);
const cipherData = base64ToArrayBuffer(cipherBase64);
const key = await generateKey(encryptionKey, salt);
const decrypted = await crypto.subtle.decrypt(
{
name: "AES-GCM",
iv,
},
key,
cipherData,
);
const dec = new TextDecoder();
return dec.decode(decrypted);
} catch (error) {
throw new Error("Failed to decrypt AWS credentials");
}
}
export function maskSensitiveValue(value: string): string {
if (!value) return "";
if (value.length <= 6) return value;
const masked = "*".repeat(value.length - 6);
return value.slice(0, 3) + masked + value.slice(-3);
}
// AWS Signing
export interface SignParams {
method: string;
url: string;
region: string;
accessKeyId: string;
secretAccessKey: string;
body: string | object;
service: string;
headers?: Record<string, string>;
isStreaming?: boolean;
}
async function createHmac(
key: ArrayBuffer | Uint8Array,
data: string,
): Promise<ArrayBuffer> {
const encoder = new TextEncoder();
const keyData = key instanceof Uint8Array ? key : new Uint8Array(key);
const keyObject = await crypto.subtle.importKey(
"raw",
keyData,
{ name: "HMAC", hash: "SHA-256" },
false,
["sign"],
);
return crypto.subtle.sign("HMAC", keyObject, encoder.encode(data));
}
async function getSigningKey(
secretKey: string,
dateStamp: string,
region: string,
service: string,
): Promise<ArrayBuffer> {
const encoder = new TextEncoder();
const kDate = await createHmac(encoder.encode("AWS4" + secretKey), dateStamp);
const kRegion = await createHmac(kDate, region);
const kService = await createHmac(kRegion, service);
const kSigning = await createHmac(kService, "aws4_request");
return kSigning;
}
function normalizeHeaderValue(value: string): string {
return value.replace(/\s+/g, " ").trim();
}
function encodeRFC3986(str: string): string {
return encodeURIComponent(str)
.replace(
/[!'()*]/g,
(c) => "%" + c.charCodeAt(0).toString(16).toUpperCase(),
)
.replace(/[-_.~]/g, (c) => c);
}
function getCanonicalUri(path: string): string {
if (!path || path === "/") return "/";
return (
"/" +
path
.split("/")
.map((segment) => {
if (!segment) return "";
if (segment === "invoke-with-response-stream") return segment;
if (segment.includes("model/")) {
return segment
.split(/(model\/)/)
.map((part) => {
if (part === "model/") return part;
return part
.split(/([.:])/g)
.map((subpart, i) =>
i % 2 === 1 ? subpart : encodeRFC3986(subpart),
)
.join("");
})
.join("");
}
return encodeRFC3986(segment);
})
.join("/")
);
}
export async function sign({
method,
url,
region,
accessKeyId,
secretAccessKey,
body,
service,
headers: customHeaders = {},
isStreaming = true,
}: SignParams): Promise<Record<string, string>> {
try {
const endpoint = new URL(url);
const canonicalUri = getCanonicalUri(endpoint.pathname.slice(1));
const canonicalQueryString = endpoint.search.slice(1);
const now = new Date();
const amzDate = now.toISOString().replace(/[:-]|\.\d{3}/g, "");
const dateStamp = amzDate.slice(0, 8);
const bodyString = typeof body === "string" ? body : JSON.stringify(body);
const encoder = new TextEncoder();
const payloadBuffer = await crypto.subtle.digest(
"SHA-256",
encoder.encode(bodyString),
);
const payloadHash = Array.from(new Uint8Array(payloadBuffer))
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
const headers: Record<string, string> = {
accept: isStreaming
? "application/vnd.amazon.eventstream"
: "application/json",
"content-type": "application/json",
host: endpoint.host,
"x-amz-content-sha256": payloadHash,
"x-amz-date": amzDate,
...customHeaders,
};
// Add x-amzn-bedrock-accept header for streaming requests
if (isStreaming) {
headers["x-amzn-bedrock-accept"] = "*/*";
}
const sortedHeaderKeys = Object.keys(headers).sort((a, b) =>
a.toLowerCase().localeCompare(b.toLowerCase()),
);
const canonicalHeaders = sortedHeaderKeys
.map(
(key) => `${key.toLowerCase()}:${normalizeHeaderValue(headers[key])}\n`,
)
.join("");
const signedHeaders = sortedHeaderKeys
.map((key) => key.toLowerCase())
.join(";");
const canonicalRequest = [
method.toUpperCase(),
canonicalUri,
canonicalQueryString,
canonicalHeaders,
signedHeaders,
payloadHash,
].join("\n");
const algorithm = "AWS4-HMAC-SHA256";
const credentialScope = `${dateStamp}/${region}/${service}/aws4_request`;
const canonicalRequestHash = Array.from(
new Uint8Array(
await crypto.subtle.digest("SHA-256", encoder.encode(canonicalRequest)),
),
)
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
const stringToSign = [
algorithm,
amzDate,
credentialScope,
canonicalRequestHash,
].join("\n");
const signingKey = await getSigningKey(
secretAccessKey,
dateStamp,
region,
service,
);
const signature = Array.from(
new Uint8Array(await createHmac(signingKey, stringToSign)),
)
.map((b) => b.toString(16).padStart(2, "0"))
.join("");
const authorization = [
`${algorithm} Credential=${accessKeyId}/${credentialScope}`,
`SignedHeaders=${signedHeaders}`,
`Signature=${signature}`,
].join(", ");
return {
...headers,
Authorization: authorization,
};
} catch (error) {
console.error("[AWS Signing Error]: Failed to sign request");
throw new Error("Failed to sign AWS request");
}
}
// Bedrock utilities
function decodeBase64(base64String: string): string {
try {
const bytes = Buffer.from(base64String, "base64");
const decoder = new TextDecoder("utf-8");
return decoder.decode(bytes);
} catch (e) {
console.error("[Base64 Decode Error]:", e);
return "";
}
}
export function parseEventData(chunk: Uint8Array): EventResult {
const decoder = new TextDecoder("utf-8");
const text = decoder.decode(chunk);
const results: EventResult = [];
try {
// First try to parse as regular JSON
const parsed = JSON.parse(text);
if (parsed.bytes) {
const decoded = decodeBase64(parsed.bytes);
try {
const decodedJson = JSON.parse(decoded);
results.push(decodedJson);
} catch (e) {
results.push({ output: decoded });
}
return results;
}
if (typeof parsed.body === "string") {
try {
const parsedBody = JSON.parse(parsed.body);
results.push(parsedBody);
} catch (e) {
results.push({ output: parsed.body });
}
return results;
}
results.push(parsed.body || parsed);
return results;
} catch (e) {
// If regular JSON parse fails, try to extract event content
const eventRegex = /:event-type[^\{]+(\{[^\}]+\})/g;
let match;
while ((match = eventRegex.exec(text)) !== null) {
try {
const eventData = match[1];
const parsed = JSON.parse(eventData);
if (parsed.bytes) {
const decoded = decodeBase64(parsed.bytes);
try {
const decodedJson = JSON.parse(decoded);
if (decodedJson.choices?.[0]?.message?.content) {
results.push({ output: decodedJson.choices[0].message.content });
} else {
results.push(decodedJson);
}
} catch (e) {
results.push({ output: decoded });
}
} else {
results.push(parsed);
}
} catch (e) {
console.debug("[Event Parse Warning]:", e);
}
}
// If no events were found, try to extract clean text
if (results.length === 0) {
// Remove event metadata markers and clean the text
const cleanText = text
.replace(/\{KG[^:]+:event-type[^}]+\}/g, "") // Remove event markers
.replace(/[\x00-\x1F\x7F-\x9F\uFEFF]/g, "") // Remove control characters
.trim();
if (cleanText) {
results.push({ output: cleanText });
}
}
}
return results;
}
export function processMessage(
data: ParsedEvent,
remainText: string,
runTools: any[],
index: number,
): { remainText: string; index: number } {
if (!data) return { remainText, index };
try {
// Handle Nova's tool calls with exact schema match
// console.log("processMessage data=========================",data);
if (data.contentBlockStart?.start?.toolUse) {
const toolUse = data.contentBlockStart.start.toolUse;
index += 1;
runTools.push({
id: toolUse.toolUseId,
type: "function",
function: {
name: toolUse.name || "", // Ensure name is always present
arguments: "{}", // Initialize empty arguments
},
});
return { remainText, index };
}
// Handle Nova's tool input in contentBlockDelta
if (data.contentBlockDelta?.delta?.toolUse?.input) {
if (runTools[index]) {
runTools[index].function.arguments =
data.contentBlockDelta.delta.toolUse.input;
}
return { remainText, index };
}
// Handle Nova's text content
if (data.output?.message?.content?.[0]?.text) {
remainText += data.output.message.content[0].text;
return { remainText, index };
}
// Handle Nova's messageStart event
if (data.messageStart) {
return { remainText, index };
}
// Handle Nova's text delta
if (data.contentBlockDelta?.delta?.text) {
remainText += data.contentBlockDelta.delta.text;
return { remainText, index };
}
// Handle Nova's contentBlockStop event
if (data.contentBlockStop) {
return { remainText, index };
}
// Handle Nova's messageStop event
if (data.messageStop) {
return { remainText, index };
}
// Handle message_start event (for other models)
if (data.type === "message_start") {
return { remainText, index };
}
// Handle content_block_start event (for other models)
if (data.type === "content_block_start") {
if (data.content_block?.type === "tool_use") {
index += 1;
runTools.push({
id: data.content_block.id,
type: "function",
function: {
name: data.content_block.name || "", // Ensure name is always present
arguments: "",
},
});
}
return { remainText, index };
}
// Handle content_block_delta event (for other models)
if (data.type === "content_block_delta") {
if (data.delta?.type === "input_json_delta" && runTools[index]) {
runTools[index].function.arguments += data.delta.partial_json;
} else if (data.delta?.type === "text_delta") {
const newText = data.delta.text || "";
remainText += newText;
}
return { remainText, index };
}
// Handle tool calls for other models
if (data.choices?.[0]?.message?.tool_calls) {
for (const toolCall of data.choices[0].message.tool_calls) {
index += 1;
runTools.push({
id: toolCall.id || `tool-${Date.now()}`,
type: "function",
function: {
name: toolCall.function?.name || "", // Ensure name is always present
arguments: toolCall.function?.arguments || "",
},
});
}
return { remainText, index };
}
// Handle various response formats
let newText = "";
if (data.delta?.text) {
newText = data.delta.text;
} else if (data.choices?.[0]?.message?.content) {
newText = data.choices[0].message.content;
} else if (data.content?.[0]?.text) {
newText = data.content[0].text;
} else if (data.generation) {
newText = data.generation;
} else if (data.outputText) {
newText = data.outputText;
} else if (data.response) {
newText = data.response;
} else if (data.output) {
newText = data.output;
}
// Only append if we have new text
if (newText) {
remainText += newText;
}
} catch (e) {
console.warn("Failed to process Bedrock message:");
}
return { remainText, index };
}
export function processChunks(
chunks: Uint8Array[],
pendingChunk: Uint8Array | null,
remainText: string,
runTools: any[],
index: number,
): {
chunks: Uint8Array[];
pendingChunk: Uint8Array | null;
remainText: string;
index: number;
} {
let currentText = remainText;
let currentIndex = index;
while (chunks.length > 0) {
const chunk = chunks[0];
try {
// If there's a pending chunk, try to merge it with the current chunk
let chunkToProcess = chunk;
if (pendingChunk) {
const mergedChunk = new Uint8Array(pendingChunk.length + chunk.length);
mergedChunk.set(pendingChunk);
mergedChunk.set(chunk, pendingChunk.length);
chunkToProcess = mergedChunk;
pendingChunk = null;
}
// Try to process the chunk
const parsedEvents = parseEventData(chunkToProcess);
if (parsedEvents.length > 0) {
// Process each event in the chunk
for (const parsed of parsedEvents) {
const result = processMessage(
parsed,
currentText,
runTools,
currentIndex,
);
currentText = result.remainText;
currentIndex = result.index;
}
chunks.shift(); // Remove processed chunk
} else {
// If parsing fails, it might be an incomplete chunk
pendingChunk = chunkToProcess;
chunks.shift();
}
} catch (e) {
// console.error("[Chunk Process Error]:", e);
// chunks.shift(); // Remove error chunk
// pendingChunk = null; // Reset pending chunk on error
console.warn("Failed to process chunk, attempting recovery");
// Attempt to recover by processing the next chunk
if (chunks.length > 1) {
chunks.shift();
pendingChunk = null;
} else {
// If this is the last chunk, throw to prevent data loss
throw new Error("Failed to process final chunk");
}
}
}
return {
chunks,
pendingChunk,
remainText: currentText,
index: currentIndex,
};
}
export function getBedrockEndpoint(
region: string,
modelId: string,
shouldStream: boolean,
): string {
if (!region || !modelId) {
throw new Error("Region and model ID are required for Bedrock endpoint");
}
const baseEndpoint = `https://bedrock-runtime.${region}.amazonaws.com`;
const endpoint =
shouldStream === false
? `${baseEndpoint}/model/${modelId}/invoke`
: `${baseEndpoint}/model/${modelId}/invoke-with-response-stream`;
return endpoint;
}
export function extractMessage(res: any, modelId: string = ""): string {
if (!res) {
throw new Error("Empty response received");
}
let message = "";
// Handle Nova model response format
if (modelId.toLowerCase().includes("nova")) {
if (res.output?.message?.content?.[0]?.text) {
message = res.output.message.content[0].text;
} else {
message = res.output || "";
}
}
// Handle Mistral model response format
else if (modelId.toLowerCase().includes("mistral")) {
if (res.choices?.[0]?.message?.content) {
message = res.choices[0].message.content;
} else {
message = res.output || "";
}
}
// Handle Llama model response format
else if (modelId.toLowerCase().includes("llama")) {
message = res?.generation || "";
}
// Handle Titan model response format
else if (modelId.toLowerCase().includes("titan")) {
message = res?.outputText || "";
}
// Handle Claude and other models
else if (res.content?.[0]?.text) {
message = res.content[0].text;
}
// Handle other response formats
else {
message = res.output || res.response || res.message || "";
}
return message;
}

View File

@@ -400,6 +400,7 @@ export function streamWithThink(
let responseRes: Response;
let isInThinkingMode = false;
let lastIsThinking = false;
let lastIsThinkingTagged = false; //between <think> and </think> tags
// animate response to make it looks smooth
function animateResponseText() {
@@ -579,6 +580,23 @@ export function streamWithThink(
if (!chunk?.content || chunk.content.length === 0) {
return;
}
// deal with <think> and </think> tags start
if (!chunk.isThinking) {
if (chunk.content.startsWith("<think>")) {
chunk.isThinking = true;
chunk.content = chunk.content.slice(7).trim();
lastIsThinkingTagged = true;
} else if (chunk.content.endsWith("</think>")) {
chunk.isThinking = false;
chunk.content = chunk.content.slice(0, -8).trim();
lastIsThinkingTagged = false;
} else if (lastIsThinkingTagged) {
chunk.isThinking = true;
}
}
// deal with <think> and </think> tags start
// Check if thinking mode changed
const isThinkingChanged = lastIsThinking !== chunk.isThinking;
lastIsThinking = chunk.isThinking;

View File

@@ -0,0 +1,258 @@
# Understanding Bedrock Response Format
The AWS Bedrock streaming response format consists of multiple Server-Sent Events (SSE) chunks. Each chunk follows this structure:
```
:event-type chunk
:content-type application/json
:message-type event
{"bytes":"base64_encoded_data","p":"signature"}
```
## Model-Specific Response Formats
### Claude 3 Format
When using Claude 3 models (e.g., claude-3-haiku-20240307), the decoded messages include:
1. **message_start**
```json
{
"type": "message_start",
"message": {
"id": "msg_bdrk_01A6sahWac4XVTR9sX3rgvsZ",
"type": "message",
"role": "assistant",
"model": "claude-3-haiku-20240307",
"content": [],
"stop_reason": null,
"stop_sequence": null,
"usage": {
"input_tokens": 8,
"output_tokens": 1
}
}
}
```
2. **content_block_start**
```json
{
"type": "content_block_start",
"index": 0,
"content_block": {
"type": "text",
"text": ""
}
}
```
3. **content_block_delta**
```json
{
"type": "content_block_delta",
"index": 0,
"delta": {
"type": "text_delta",
"text": "Hello"
}
}
```
### Mistral Format
When using Mistral models (e.g., mistral-large-2407), the decoded messages have a different structure:
```json
{
"id": "b0098812-0ad9-42da-9f17-a5e2f554eb6b",
"object": "chat.completion.chunk",
"created": 1732582566,
"model": "mistral-large-2407",
"choices": [{
"index": 0,
"logprobs": null,
"context_logits": null,
"generation_logits": null,
"message": {
"role": null,
"content": "Hello",
"tool_calls": null,
"index": null,
"tool_call_id": null
},
"stop_reason": null
}],
"usage": null,
"p": null
}
```
### Llama Format
When using Llama models (3.1 or 3.2), the decoded messages use a simpler structure focused on generation tokens:
```json
{
"generation": "Hello",
"prompt_token_count": null,
"generation_token_count": 2,
"stop_reason": null
}
```
Each chunk contains:
- generation: The generated text piece
- prompt_token_count: Token count of the input (only present in first chunk)
- generation_token_count: Running count of generated tokens
- stop_reason: Indicates completion (null until final chunk)
First chunk example (includes prompt_token_count):
```json
{
"generation": "\n\n",
"prompt_token_count": 10,
"generation_token_count": 1,
"stop_reason": null
}
```
### Titan Text Format
When using Amazon's Titan models (text or TG1), the response comes as a single chunk with complete text and metrics:
```json
{
"outputText": "\nBot: Hello! How can I help you today?",
"index": 0,
"totalOutputTextTokenCount": 13,
"completionReason": "FINISH",
"inputTextTokenCount": 3,
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 3,
"outputTokenCount": 13,
"invocationLatency": 833,
"firstByteLatency": 833
}
}
```
Both Titan text and Titan TG1 use the same response format, with only minor differences in token counts and latency values. For example, here's a TG1 response:
```json
{
"outputText": "\nBot: Hello! How can I help you?",
"index": 0,
"totalOutputTextTokenCount": 12,
"completionReason": "FINISH",
"inputTextTokenCount": 3,
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 3,
"outputTokenCount": 12,
"invocationLatency": 845,
"firstByteLatency": 845
}
}
```
Key fields:
- outputText: The complete generated response
- totalOutputTextTokenCount: Total tokens in the response
- completionReason: Reason for completion (e.g., "FINISH")
- inputTextTokenCount: Number of input tokens
- amazon-bedrock-invocationMetrics: Detailed performance metrics
## Model-Specific Completion Metrics
### Mistral
```json
{
"usage": {
"prompt_tokens": 5,
"total_tokens": 29,
"completion_tokens": 24
},
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 5,
"outputTokenCount": 24,
"invocationLatency": 719,
"firstByteLatency": 148
}
}
```
### Claude 3
Included in the message_delta with stop_reason.
### Llama
Included in the final chunk with stop_reason "stop":
```json
{
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 10,
"outputTokenCount": 11,
"invocationLatency": 873,
"firstByteLatency": 550
}
}
```
### Titan
Both Titan text and TG1 include metrics in the single response chunk:
```json
{
"amazon-bedrock-invocationMetrics": {
"inputTokenCount": 3,
"outputTokenCount": 12,
"invocationLatency": 845,
"firstByteLatency": 845
}
}
```
## How the Response is Processed
1. The raw response is first split into chunks based on SSE format
2. For each chunk:
- The base64 encoded data is decoded
- The JSON is parsed to extract the message content
- Based on the model type and message type, different processing is applied:
### Claude 3 Processing
- message_start: Initializes a new message with model info and usage stats
- content_block_start: Starts a new content block (text, tool use, etc.)
- content_block_delta: Adds incremental content to the current block
- message_delta: Updates message metadata
### Mistral Processing
- Each chunk contains a complete message object with choices array
- The content is streamed through the message.content field
- Final chunk includes token usage and invocation metrics
### Llama Processing
- Each chunk contains a generation field with the text piece
- First chunk includes prompt_token_count
- Tracks generation progress through generation_token_count
- Simple streaming format focused on text generation
- Final chunk includes complete metrics
### Titan Processing
- Single chunk response with complete text
- No streaming - returns full response at once
- Includes comprehensive metrics in the same chunk
## Handling in Code
The response is processed by the `transformBedrockStream` function in `app/utils/aws.ts`, which:
1. Reads the stream chunks
2. Parses each chunk using `parseEventData`
3. Handles model-specific formats:
- For Claude: Processes message_start, content_block_start, content_block_delta
- For Mistral: Extracts content from choices[0].message.content
- For Llama: Uses the generation field directly
- For Titan: Uses the outputText field from the single response
4. Transforms the parsed data into a consistent format for the client
5. Yields the transformed data as SSE events
This allows for real-time streaming of the model's response while maintaining a consistent format for the client application, regardless of which model is being used.

View File

@@ -94,4 +94,4 @@
"lint-staged/yaml": "^2.2.2"
},
"packageManager": "yarn@1.22.19"
}
}

3177
yarn.lock

File diff suppressed because it is too large Load Diff