mirror of
https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web.git
synced 2026-02-19 21:04:25 +08:00
Compare commits
121 Commits
Leizhenpen
...
cb8144e168
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cb8144e168 | ||
|
|
f5f3ce94f6 | ||
|
|
2b5f600308 | ||
|
|
b966107117 | ||
|
|
377480b448 | ||
|
|
8bd0d6a1a7 | ||
|
|
00816304df | ||
|
|
90827fc593 | ||
|
|
008e339b6d | ||
|
|
12863f5213 | ||
|
|
cf140d4228 | ||
|
|
476d946f96 | ||
|
|
9714258322 | ||
|
|
48cd4b11b5 | ||
|
|
77c78b230a | ||
|
|
b44686b887 | ||
|
|
34bdd4b945 | ||
|
|
b0758cccde | ||
|
|
98a11e56d2 | ||
|
|
645ecbcad8 | ||
|
|
86f86962fb | ||
|
|
2137aa65bf | ||
|
|
18fa2cc30d | ||
|
|
0bfc648085 | ||
|
|
9f91c2d05c | ||
|
|
a029b4330b | ||
|
|
65dc0e26d2 | ||
|
|
81b14b7b8d | ||
|
|
40c00374e7 | ||
|
|
e94566d258 | ||
|
|
6d72a04854 | ||
|
|
b0f78e9d1c | ||
|
|
29b9a20acf | ||
|
|
89b1774996 | ||
|
|
26f79aa6e6 | ||
|
|
92615dae32 | ||
|
|
9643adcf15 | ||
|
|
e839940a26 | ||
|
|
0ec1ae6276 | ||
|
|
cb0422b8f2 | ||
|
|
e455840ab3 | ||
|
|
19437c7aa1 | ||
|
|
12d38aa4b2 | ||
|
|
2a9f7d72fc | ||
|
|
372a327522 | ||
|
|
50a241b715 | ||
|
|
44a1cf6d6e | ||
|
|
a0886875e1 | ||
|
|
93337b2b92 | ||
|
|
4b2f447474 | ||
|
|
7830b37a90 | ||
|
|
fb3437ca01 | ||
|
|
f5ae086d3c | ||
|
|
26b9fa97cd | ||
|
|
603415f9e1 | ||
|
|
5ac651ad8e | ||
|
|
ad49cd0454 | ||
|
|
57dc44a54f | ||
|
|
4254fd34f9 | ||
|
|
0c55850641 | ||
|
|
a75b9f7fbe | ||
|
|
d9d2a27d8f | ||
|
|
471b17831a | ||
|
|
8ce2cf5c3b | ||
|
|
9c648e5b64 | ||
|
|
b39b3f7a5e | ||
|
|
448babd27f | ||
|
|
e6633753a4 | ||
|
|
15d0600642 | ||
|
|
9a473048b1 | ||
|
|
2fe848ecf3 | ||
|
|
0abfd279e4 | ||
|
|
5bd7e28f82 | ||
|
|
6f7a635030 | ||
|
|
2ccdd1706a | ||
|
|
a19ba6933a | ||
|
|
513cf1b206 | ||
|
|
238eb70986 | ||
|
|
a6337e9f23 | ||
|
|
ff88421904 | ||
|
|
a85db21e1f | ||
|
|
b0c1ccd0a0 | ||
|
|
bd68df1d9b | ||
|
|
f60c237b16 | ||
|
|
9d3f1d2529 | ||
|
|
dfeb9e7f27 | ||
|
|
b2d5e0e309 | ||
|
|
225ad30898 | ||
|
|
6bc1612720 | ||
|
|
24261d20b3 | ||
|
|
1b5a81c7ad | ||
|
|
70f066c15f | ||
|
|
bfa433919f | ||
|
|
afb0752d5d | ||
|
|
f12058463b | ||
|
|
09e4f95272 | ||
|
|
0e09697274 | ||
|
|
82a368a3aa | ||
|
|
9e04198bdc | ||
|
|
4204890d90 | ||
|
|
5d5456c1c5 | ||
|
|
f0c23cc6aa | ||
|
|
3bf55d3530 | ||
|
|
952d8835a3 | ||
|
|
c55cea5853 | ||
|
|
ca17e90c52 | ||
|
|
cae20af24d | ||
|
|
1f66d3779c | ||
|
|
045adc3567 | ||
|
|
1998cf5ced | ||
|
|
1164e1bdf6 | ||
|
|
d55c752e1e | ||
|
|
e3c18bb123 | ||
|
|
f532731e2a | ||
|
|
58837f6dec | ||
|
|
afbf5eb541 | ||
|
|
0f276f59bb | ||
|
|
fc391168e9 | ||
|
|
dca4a0e48f | ||
|
|
722c28839f | ||
|
|
ff356f0c8c |
@@ -76,6 +76,12 @@ ANTHROPIC_URL=
|
||||
### (optional)
|
||||
WHITE_WEBDAV_ENDPOINTS=
|
||||
|
||||
|
||||
### bedrock (optional)
|
||||
AWS_REGION=
|
||||
AWS_ACCESS_KEY=AKIA
|
||||
AWS_SECRET_KEY=
|
||||
|
||||
### siliconflow Api key (optional)
|
||||
SILICONFLOW_API_KEY=
|
||||
|
||||
|
||||
23
README.md
23
README.md
@@ -1,6 +1,6 @@
|
||||
<div align="center">
|
||||
|
||||
<a href='https://nextchat.dev/chat'>
|
||||
<a href='https://nextchat.club'>
|
||||
<img src="https://github.com/user-attachments/assets/83bdcc07-ae5e-4954-a53a-ac151ba6ccf3" width="1000" alt="icon"/>
|
||||
</a>
|
||||
|
||||
@@ -23,9 +23,10 @@ English / [简体中文](./README_CN.md)
|
||||
[![Linux][Linux-image]][download-url]
|
||||
|
||||
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [Web App Demo](https://app.nextchat.dev) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases)
|
||||
[NextChatAI](https://nextchat.club?utm_source=readme) / [Web App Demo](https://app.nextchat.dev) / [Desktop App](https://github.com/Yidadaa/ChatGPT-Next-Web/releases) / [Discord](https://discord.gg/YCkeafCafC) / [Enterprise Edition](#enterprise-edition) / [Twitter](https://twitter.com/NextChatDev)
|
||||
|
||||
|
||||
[saas-url]: https://nextchat.dev/chat?utm_source=readme
|
||||
[saas-url]: https://nextchat.club?utm_source=readme
|
||||
[saas-image]: https://img.shields.io/badge/NextChat-Saas-green?logo=microsoftedge
|
||||
[web-url]: https://app.nextchat.dev/
|
||||
[download-url]: https://github.com/Yidadaa/ChatGPT-Next-Web/releases
|
||||
@@ -40,6 +41,24 @@ English / [简体中文](./README_CN.md)
|
||||
|
||||
</div>
|
||||
|
||||
## 👋 Hey, NextChat is going to develop a native app!
|
||||
|
||||
> This week we are going to start working on iOS and Android APP, and we want to find some reliable friends to do it together!
|
||||
|
||||
|
||||
✨ Several key points:
|
||||
|
||||
- Starting from 0, you are a veteran
|
||||
- Completely open source, not hidden
|
||||
- Native development, pursuing the ultimate experience
|
||||
|
||||
Will you come and do something together? 😎
|
||||
|
||||
https://github.com/ChatGPTNextWeb/NextChat/issues/6269
|
||||
|
||||
#Seeking for talents is thirsty #lack of people
|
||||
|
||||
|
||||
## 🥳 Cheer for DeepSeek, China's AI star!
|
||||
> Purpose-Built UI for DeepSeek Reasoner Model
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
|
||||
一键免费部署你的私人 ChatGPT 网页应用,支持 Claude, GPT4 & Gemini Pro 模型。
|
||||
|
||||
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [企业版](#%E4%BC%81%E4%B8%9A%E7%89%88) / [演示 Demo](https://chat-gpt-next-web.vercel.app/) / [反馈 Issues](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [加入 Discord](https://discord.gg/zrhvHCr79N)
|
||||
[NextChatAI](https://nextchat.club?utm_source=readme) / [企业版](#%E4%BC%81%E4%B8%9A%E7%89%88) / [演示 Demo](https://chat-gpt-next-web.vercel.app/) / [反馈 Issues](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [加入 Discord](https://discord.gg/zrhvHCr79N)
|
||||
|
||||
[<img src="https://vercel.com/button" alt="Deploy on Zeabur" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Deploy on Zeabur" height="30">](https://zeabur.com/templates/ZBUEFA) [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Open in Gitpod" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
ワンクリックで無料であなた専用の ChatGPT ウェブアプリをデプロイ。GPT3、GPT4 & Gemini Pro モデルをサポート。
|
||||
|
||||
[NextChatAI](https://nextchat.dev/chat?utm_source=readme) / [企業版](#企業版) / [デモ](https://chat-gpt-next-web.vercel.app/) / [フィードバック](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [Discordに参加](https://discord.gg/zrhvHCr79N)
|
||||
[NextChatAI](https://nextchat.club?utm_source=readme) / [企業版](#企業版) / [デモ](https://chat-gpt-next-web.vercel.app/) / [フィードバック](https://github.com/Yidadaa/ChatGPT-Next-Web/issues) / [Discordに参加](https://discord.gg/zrhvHCr79N)
|
||||
|
||||
[<img src="https://vercel.com/button" alt="Zeaburでデプロイ" height="30">](https://vercel.com/new/clone?repository-url=https%3A%2F%2Fgithub.com%2FChatGPTNextWeb%2FChatGPT-Next-Web&env=OPENAI_API_KEY&env=CODE&project-name=nextchat&repository-name=NextChat) [<img src="https://zeabur.com/button.svg" alt="Zeaburでデプロイ" height="30">](https://zeabur.com/templates/ZBUEFA) [<img src="https://gitpod.io/button/open-in-gitpod.svg" alt="Gitpodで開く" height="30">](https://gitpod.io/#https://github.com/Yidadaa/ChatGPT-Next-Web)
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { ApiPath } from "@/app/constant";
|
||||
import { NextRequest } from "next/server";
|
||||
import { handle as openaiHandler } from "../../openai";
|
||||
import { handle as bedrockHandler } from "../../bedrock";
|
||||
import { handle as azureHandler } from "../../azure";
|
||||
import { handle as googleHandler } from "../../google";
|
||||
import { handle as anthropicHandler } from "../../anthropic";
|
||||
@@ -23,12 +24,15 @@ async function handle(
|
||||
const apiPath = `/api/${params.provider}`;
|
||||
console.log(`[${params.provider} Route] params `, params);
|
||||
switch (apiPath) {
|
||||
case ApiPath.Bedrock:
|
||||
return bedrockHandler(req, { params });
|
||||
case ApiPath.Azure:
|
||||
return azureHandler(req, { params });
|
||||
case ApiPath.Google:
|
||||
return googleHandler(req, { params });
|
||||
case ApiPath.Anthropic:
|
||||
return anthropicHandler(req, { params });
|
||||
|
||||
case ApiPath.Baidu:
|
||||
return baiduHandler(req, { params });
|
||||
case ApiPath.ByteDance:
|
||||
|
||||
@@ -52,7 +52,6 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
|
||||
msg: "you are not allowed to access with your own api key",
|
||||
};
|
||||
}
|
||||
|
||||
// if user does not provide an api key, inject system api key
|
||||
if (!apiKey) {
|
||||
const serverConfig = getServerSideConfig();
|
||||
@@ -101,6 +100,14 @@ export function auth(req: NextRequest, modelProvider: ModelProvider) {
|
||||
case ModelProvider.ChatGLM:
|
||||
systemApiKey = serverConfig.chatglmApiKey;
|
||||
break;
|
||||
case ModelProvider.Bedrock:
|
||||
systemApiKey =
|
||||
serverConfig.awsRegion +
|
||||
":" +
|
||||
serverConfig.awsAccessKey +
|
||||
":" +
|
||||
serverConfig.awsSecretKey;
|
||||
break;
|
||||
case ModelProvider.SiliconFlow:
|
||||
systemApiKey = serverConfig.siliconFlowApiKey;
|
||||
break;
|
||||
|
||||
177
app/api/bedrock.ts
Normal file
177
app/api/bedrock.ts
Normal file
@@ -0,0 +1,177 @@
|
||||
import { NextRequest, NextResponse } from "next/server";
|
||||
import { auth } from "./auth";
|
||||
import {
|
||||
sign,
|
||||
decrypt,
|
||||
getBedrockEndpoint,
|
||||
BedrockCredentials,
|
||||
} from "../utils/aws";
|
||||
import { getServerSideConfig } from "../config/server";
|
||||
import { ModelProvider } from "../constant";
|
||||
import { prettyObject } from "../utils/format";
|
||||
|
||||
const ALLOWED_PATH = new Set(["chat", "models"]);
|
||||
|
||||
async function getBedrockCredentials(
|
||||
req: NextRequest,
|
||||
): Promise<BedrockCredentials> {
|
||||
// Get AWS credentials from server config first
|
||||
const config = getServerSideConfig();
|
||||
let awsRegion = config.awsRegion;
|
||||
let awsAccessKey = config.awsAccessKey;
|
||||
let awsSecretKey = config.awsSecretKey;
|
||||
|
||||
// If server-side credentials are not available, parse from Authorization header
|
||||
if (!awsRegion || !awsAccessKey || !awsSecretKey) {
|
||||
const authHeader = req.headers.get("Authorization");
|
||||
if (!authHeader || !authHeader.startsWith("Bearer ")) {
|
||||
throw new Error("Missing or invalid Authorization header");
|
||||
}
|
||||
|
||||
const [_, credentials] = authHeader.split("Bearer ");
|
||||
const [encryptedRegion, encryptedAccessKey, encryptedSecretKey] =
|
||||
credentials.split(":");
|
||||
|
||||
if (!encryptedRegion || !encryptedAccessKey || !encryptedSecretKey) {
|
||||
throw new Error("Invalid Authorization header format");
|
||||
}
|
||||
const encryptionKey = req.headers.get("XEncryptionKey") || "";
|
||||
// Decrypt the credentials
|
||||
[awsRegion, awsAccessKey, awsSecretKey] = await Promise.all([
|
||||
decrypt(encryptedRegion, encryptionKey),
|
||||
decrypt(encryptedAccessKey, encryptionKey),
|
||||
decrypt(encryptedSecretKey, encryptionKey),
|
||||
]);
|
||||
|
||||
if (!awsRegion || !awsAccessKey || !awsSecretKey) {
|
||||
throw new Error(
|
||||
"Failed to decrypt AWS credentials. Please ensure ENCRYPTION_KEY is set correctly.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
region: awsRegion,
|
||||
accessKeyId: awsAccessKey,
|
||||
secretAccessKey: awsSecretKey,
|
||||
};
|
||||
}
|
||||
|
||||
async function requestBedrock(req: NextRequest) {
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 10 * 60 * 1000);
|
||||
|
||||
try {
|
||||
// Get credentials and model info
|
||||
const credentials = await getBedrockCredentials(req);
|
||||
const modelId = req.headers.get("XModelID");
|
||||
const shouldStream = req.headers.get("ShouldStream") !== "false";
|
||||
|
||||
if (!modelId) {
|
||||
throw new Error("Missing model ID");
|
||||
}
|
||||
|
||||
// Parse and validate request body
|
||||
const bodyText = await req.clone().text();
|
||||
if (!bodyText) {
|
||||
throw new Error("Request body is empty");
|
||||
}
|
||||
let bodyJson;
|
||||
try {
|
||||
bodyJson = JSON.parse(bodyText);
|
||||
} catch (e) {
|
||||
throw new Error(`Invalid JSON in request body: ${e}`);
|
||||
}
|
||||
console.log("[Bedrock Request] Initiating request");
|
||||
// Get endpoint and prepare request
|
||||
const endpoint = getBedrockEndpoint(
|
||||
credentials.region,
|
||||
modelId,
|
||||
shouldStream,
|
||||
);
|
||||
const requestBody: any = {
|
||||
...bodyJson,
|
||||
};
|
||||
// Sign request
|
||||
const headers = await sign({
|
||||
method: "POST",
|
||||
url: endpoint,
|
||||
region: credentials.region,
|
||||
accessKeyId: credentials.accessKeyId,
|
||||
secretAccessKey: credentials.secretAccessKey,
|
||||
body: JSON.stringify(requestBody),
|
||||
service: "bedrock",
|
||||
isStreaming: shouldStream,
|
||||
});
|
||||
|
||||
// Make request to AWS Bedrock
|
||||
// console.log(
|
||||
// "[Bedrock Request] Final Body:",
|
||||
// JSON.stringify(requestBody, null, 2),
|
||||
// );
|
||||
const res = await fetch(endpoint, {
|
||||
method: "POST",
|
||||
headers,
|
||||
body: JSON.stringify(requestBody),
|
||||
redirect: "manual",
|
||||
// @ts-ignore
|
||||
duplex: "half",
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
const error = await res.text();
|
||||
console.error("[Bedrock Error] Request failed with status:", res.status);
|
||||
try {
|
||||
const errorJson = JSON.parse(error);
|
||||
throw new Error(errorJson.message || error);
|
||||
} catch {
|
||||
throw new Error(
|
||||
`Bedrock request failed with status ${res.status}: ${
|
||||
error || "No error message"
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (!res.body) {
|
||||
console.error("[Bedrock Error] Empty response body");
|
||||
throw new Error(
|
||||
"Empty response from Bedrock. Please check AWS credentials and permissions.",
|
||||
);
|
||||
}
|
||||
return res;
|
||||
} catch (e) {
|
||||
console.error("[Bedrock Request Error]:", e);
|
||||
throw e;
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
export async function handle(
|
||||
req: NextRequest,
|
||||
{ params }: { params: { path: string[] } },
|
||||
) {
|
||||
const subpath = params.path.join("/");
|
||||
if (!ALLOWED_PATH.has(subpath)) {
|
||||
return NextResponse.json(
|
||||
{ error: true, msg: "you are not allowed to request " + subpath },
|
||||
{ status: 403 },
|
||||
);
|
||||
}
|
||||
|
||||
const authResult = auth(req, ModelProvider.Bedrock);
|
||||
if (authResult.error) {
|
||||
return NextResponse.json(authResult, {
|
||||
status: 401,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
return await requestBedrock(req);
|
||||
} catch (e) {
|
||||
console.error("Handler error:", e);
|
||||
return NextResponse.json(prettyObject(e));
|
||||
}
|
||||
}
|
||||
@@ -14,8 +14,12 @@ function getModels(remoteModelRes: OpenAIListModelResponse) {
|
||||
if (config.disableGPT4) {
|
||||
remoteModelRes.data = remoteModelRes.data.filter(
|
||||
(m) =>
|
||||
!(m.id.startsWith("gpt-4") || m.id.startsWith("chatgpt-4o") || m.id.startsWith("o1") || m.id.startsWith("o3")) ||
|
||||
m.id.startsWith("gpt-4o-mini"),
|
||||
!(
|
||||
m.id.startsWith("gpt-4") ||
|
||||
m.id.startsWith("chatgpt-4o") ||
|
||||
m.id.startsWith("o1") ||
|
||||
m.id.startsWith("o3")
|
||||
) || m.id.startsWith("gpt-4o-mini"),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -23,8 +23,10 @@ import { SparkApi } from "./platforms/iflytek";
|
||||
import { DeepSeekApi } from "./platforms/deepseek";
|
||||
import { XAIApi } from "./platforms/xai";
|
||||
import { ChatGLMApi } from "./platforms/glm";
|
||||
import { BedrockApi } from "./platforms/bedrock";
|
||||
import { SiliconflowApi } from "./platforms/siliconflow";
|
||||
|
||||
|
||||
export const ROLES = ["system", "user", "assistant"] as const;
|
||||
export type MessageRole = (typeof ROLES)[number];
|
||||
|
||||
@@ -132,6 +134,9 @@ export class ClientApi {
|
||||
|
||||
constructor(provider: ModelProvider = ModelProvider.GPT) {
|
||||
switch (provider) {
|
||||
case ModelProvider.Bedrock:
|
||||
this.llm = new BedrockApi();
|
||||
break;
|
||||
case ModelProvider.GeminiPro:
|
||||
this.llm = new GeminiProApi();
|
||||
break;
|
||||
@@ -247,6 +252,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
|
||||
|
||||
function getConfig() {
|
||||
const modelConfig = chatStore.currentSession().mask.modelConfig;
|
||||
const isBedrock = modelConfig.providerName === ServiceProvider.Bedrock;
|
||||
const isGoogle = modelConfig.providerName === ServiceProvider.Google;
|
||||
const isAzure = modelConfig.providerName === ServiceProvider.Azure;
|
||||
const isAnthropic = modelConfig.providerName === ServiceProvider.Anthropic;
|
||||
@@ -287,6 +293,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
|
||||
: ""
|
||||
: accessStore.openaiApiKey;
|
||||
return {
|
||||
isBedrock,
|
||||
isGoogle,
|
||||
isAzure,
|
||||
isAnthropic,
|
||||
@@ -315,6 +322,7 @@ export function getHeaders(ignoreHeaders: boolean = false) {
|
||||
}
|
||||
|
||||
const {
|
||||
isBedrock,
|
||||
isGoogle,
|
||||
isAzure,
|
||||
isAnthropic,
|
||||
@@ -335,17 +343,23 @@ export function getHeaders(ignoreHeaders: boolean = false) {
|
||||
|
||||
const authHeader = getAuthHeader();
|
||||
|
||||
const bearerToken = getBearerToken(
|
||||
apiKey,
|
||||
isAzure || isAnthropic || isGoogle,
|
||||
);
|
||||
|
||||
if (bearerToken) {
|
||||
headers[authHeader] = bearerToken;
|
||||
} else if (isEnabledAccessControl && validString(accessStore.accessCode)) {
|
||||
headers["Authorization"] = getBearerToken(
|
||||
ACCESS_CODE_PREFIX + accessStore.accessCode,
|
||||
if (isBedrock) {
|
||||
if (apiKey) {
|
||||
headers[authHeader] = getBearerToken(apiKey);
|
||||
}
|
||||
} else {
|
||||
const bearerToken = getBearerToken(
|
||||
apiKey,
|
||||
isAzure || isAnthropic || isGoogle,
|
||||
);
|
||||
|
||||
if (bearerToken) {
|
||||
headers[authHeader] = bearerToken;
|
||||
} else if (isEnabledAccessControl && validString(accessStore.accessCode)) {
|
||||
headers["Authorization"] = getBearerToken(
|
||||
ACCESS_CODE_PREFIX + accessStore.accessCode,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return headers;
|
||||
@@ -353,6 +367,8 @@ export function getHeaders(ignoreHeaders: boolean = false) {
|
||||
|
||||
export function getClientApi(provider: ServiceProvider): ClientApi {
|
||||
switch (provider) {
|
||||
case ServiceProvider.Bedrock:
|
||||
return new ClientApi(ModelProvider.Bedrock);
|
||||
case ServiceProvider.Google:
|
||||
return new ClientApi(ModelProvider.GeminiPro);
|
||||
case ServiceProvider.Anthropic:
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
"use client";
|
||||
import { ApiPath, Alibaba, ALIBABA_BASE_URL } from "@/app/constant";
|
||||
import {
|
||||
ApiPath,
|
||||
Alibaba,
|
||||
ALIBABA_BASE_URL,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
} from "@/app/constant";
|
||||
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
|
||||
|
||||
useAccessStore,
|
||||
useAppConfig,
|
||||
useChatStore,
|
||||
ChatMessageTool,
|
||||
usePluginStore,
|
||||
} from "@/app/store";
|
||||
import { streamWithThink } from "@/app/utils/chat";
|
||||
import {
|
||||
ChatOptions,
|
||||
getHeaders,
|
||||
@@ -15,14 +16,12 @@ import {
|
||||
SpeechOptions,
|
||||
MultimodalContent,
|
||||
} from "../api";
|
||||
import Locale from "../../locales";
|
||||
import {
|
||||
EventStreamContentType,
|
||||
fetchEventSource,
|
||||
} from "@fortaine/fetch-event-source";
|
||||
import { prettyObject } from "@/app/utils/format";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { getMessageTextContent } from "@/app/utils";
|
||||
import {
|
||||
getMessageTextContent,
|
||||
getMessageTextContentWithoutThinking,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
|
||||
export interface OpenAIListModelResponse {
|
||||
@@ -92,7 +91,10 @@ export class QwenApi implements LLMApi {
|
||||
async chat(options: ChatOptions) {
|
||||
const messages = options.messages.map((v) => ({
|
||||
role: v.role,
|
||||
content: getMessageTextContent(v),
|
||||
content:
|
||||
v.role === "assistant"
|
||||
? getMessageTextContentWithoutThinking(v)
|
||||
: getMessageTextContent(v),
|
||||
}));
|
||||
|
||||
const modelConfig = {
|
||||
@@ -122,134 +124,118 @@ export class QwenApi implements LLMApi {
|
||||
options.onController?.(controller);
|
||||
|
||||
try {
|
||||
const headers = {
|
||||
...getHeaders(),
|
||||
"X-DashScope-SSE": shouldStream ? "enable" : "disable",
|
||||
};
|
||||
|
||||
const chatPath = this.path(Alibaba.ChatPath);
|
||||
const chatPayload = {
|
||||
method: "POST",
|
||||
body: JSON.stringify(requestPayload),
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
...getHeaders(),
|
||||
"X-DashScope-SSE": shouldStream ? "enable" : "disable",
|
||||
},
|
||||
headers: headers,
|
||||
};
|
||||
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
let responseText = "";
|
||||
let remainText = "";
|
||||
let finished = false;
|
||||
let responseRes: Response;
|
||||
const [tools, funcs] = usePluginStore
|
||||
.getState()
|
||||
.getAsTools(
|
||||
useChatStore.getState().currentSession().mask?.plugin || [],
|
||||
);
|
||||
return streamWithThink(
|
||||
chatPath,
|
||||
requestPayload,
|
||||
headers,
|
||||
tools as any,
|
||||
funcs,
|
||||
controller,
|
||||
// parseSSE
|
||||
(text: string, runTools: ChatMessageTool[]) => {
|
||||
// console.log("parseSSE", text, runTools);
|
||||
const json = JSON.parse(text);
|
||||
const choices = json.output.choices as Array<{
|
||||
message: {
|
||||
content: string | null;
|
||||
tool_calls: ChatMessageTool[];
|
||||
reasoning_content: string | null;
|
||||
};
|
||||
}>;
|
||||
|
||||
// animate response to make it looks smooth
|
||||
function animateResponseText() {
|
||||
if (finished || controller.signal.aborted) {
|
||||
responseText += remainText;
|
||||
console.log("[Response Animation] finished");
|
||||
if (responseText?.length === 0) {
|
||||
options.onError?.(new Error("empty response from server"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (!choices?.length) return { isThinking: false, content: "" };
|
||||
|
||||
if (remainText.length > 0) {
|
||||
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
|
||||
const fetchText = remainText.slice(0, fetchCount);
|
||||
responseText += fetchText;
|
||||
remainText = remainText.slice(fetchCount);
|
||||
options.onUpdate?.(responseText, fetchText);
|
||||
}
|
||||
|
||||
requestAnimationFrame(animateResponseText);
|
||||
}
|
||||
|
||||
// start animaion
|
||||
animateResponseText();
|
||||
|
||||
const finish = () => {
|
||||
if (!finished) {
|
||||
finished = true;
|
||||
options.onFinish(responseText + remainText, responseRes);
|
||||
}
|
||||
};
|
||||
|
||||
controller.signal.onabort = finish;
|
||||
|
||||
fetchEventSource(chatPath, {
|
||||
fetch: fetch as any,
|
||||
...chatPayload,
|
||||
async onopen(res) {
|
||||
clearTimeout(requestTimeoutId);
|
||||
const contentType = res.headers.get("content-type");
|
||||
console.log(
|
||||
"[Alibaba] request response content type: ",
|
||||
contentType,
|
||||
);
|
||||
responseRes = res;
|
||||
|
||||
if (contentType?.startsWith("text/plain")) {
|
||||
responseText = await res.clone().text();
|
||||
return finish();
|
||||
const tool_calls = choices[0]?.message?.tool_calls;
|
||||
if (tool_calls?.length > 0) {
|
||||
const index = tool_calls[0]?.index;
|
||||
const id = tool_calls[0]?.id;
|
||||
const args = tool_calls[0]?.function?.arguments;
|
||||
if (id) {
|
||||
runTools.push({
|
||||
id,
|
||||
type: tool_calls[0]?.type,
|
||||
function: {
|
||||
name: tool_calls[0]?.function?.name as string,
|
||||
arguments: args,
|
||||
},
|
||||
});
|
||||
} else {
|
||||
// @ts-ignore
|
||||
runTools[index]["function"]["arguments"] += args;
|
||||
}
|
||||
}
|
||||
|
||||
const reasoning = choices[0]?.message?.reasoning_content;
|
||||
const content = choices[0]?.message?.content;
|
||||
|
||||
// Skip if both content and reasoning_content are empty or null
|
||||
if (
|
||||
!res.ok ||
|
||||
!res.headers
|
||||
.get("content-type")
|
||||
?.startsWith(EventStreamContentType) ||
|
||||
res.status !== 200
|
||||
(!reasoning || reasoning.length === 0) &&
|
||||
(!content || content.length === 0)
|
||||
) {
|
||||
const responseTexts = [responseText];
|
||||
let extraInfo = await res.clone().text();
|
||||
try {
|
||||
const resJson = await res.clone().json();
|
||||
extraInfo = prettyObject(resJson);
|
||||
} catch {}
|
||||
|
||||
if (res.status === 401) {
|
||||
responseTexts.push(Locale.Error.Unauthorized);
|
||||
}
|
||||
|
||||
if (extraInfo) {
|
||||
responseTexts.push(extraInfo);
|
||||
}
|
||||
|
||||
responseText = responseTexts.join("\n\n");
|
||||
|
||||
return finish();
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
}
|
||||
},
|
||||
onmessage(msg) {
|
||||
if (msg.data === "[DONE]" || finished) {
|
||||
return finish();
|
||||
}
|
||||
const text = msg.data;
|
||||
try {
|
||||
const json = JSON.parse(text);
|
||||
const choices = json.output.choices as Array<{
|
||||
message: { content: string };
|
||||
}>;
|
||||
const delta = choices[0]?.message?.content;
|
||||
if (delta) {
|
||||
remainText += delta;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("[Request] parse error", text, msg);
|
||||
|
||||
if (reasoning && reasoning.length > 0) {
|
||||
return {
|
||||
isThinking: true,
|
||||
content: reasoning,
|
||||
};
|
||||
} else if (content && content.length > 0) {
|
||||
return {
|
||||
isThinking: false,
|
||||
content: content,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
},
|
||||
onclose() {
|
||||
finish();
|
||||
// processToolMessage, include tool_calls message and tool call results
|
||||
(
|
||||
requestPayload: RequestPayload,
|
||||
toolCallMessage: any,
|
||||
toolCallResult: any[],
|
||||
) => {
|
||||
requestPayload?.input?.messages?.splice(
|
||||
requestPayload?.input?.messages?.length,
|
||||
0,
|
||||
toolCallMessage,
|
||||
...toolCallResult,
|
||||
);
|
||||
},
|
||||
onerror(e) {
|
||||
options.onError?.(e);
|
||||
throw e;
|
||||
},
|
||||
openWhenHidden: true,
|
||||
});
|
||||
options,
|
||||
);
|
||||
} else {
|
||||
const res = await fetch(chatPath, chatPayload);
|
||||
clearTimeout(requestTimeoutId);
|
||||
|
||||
@@ -1,10 +1,5 @@
|
||||
"use client";
|
||||
import {
|
||||
ApiPath,
|
||||
Baidu,
|
||||
BAIDU_BASE_URL,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
} from "@/app/constant";
|
||||
import { ApiPath, Baidu, BAIDU_BASE_URL } from "@/app/constant";
|
||||
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
|
||||
import { getAccessToken } from "@/app/utils/baidu";
|
||||
|
||||
@@ -23,7 +18,7 @@ import {
|
||||
} from "@fortaine/fetch-event-source";
|
||||
import { prettyObject } from "@/app/utils/format";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { getMessageTextContent } from "@/app/utils";
|
||||
import { getMessageTextContent, getTimeoutMSByModel } from "@/app/utils";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
|
||||
export interface OpenAIListModelResponse {
|
||||
@@ -155,7 +150,7 @@ export class ErnieApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
|
||||
859
app/client/platforms/bedrock.ts
Normal file
859
app/client/platforms/bedrock.ts
Normal file
@@ -0,0 +1,859 @@
|
||||
"use client";
|
||||
import { ChatOptions, getHeaders, LLMApi, SpeechOptions } from "../api";
|
||||
import {
|
||||
useAppConfig,
|
||||
usePluginStore,
|
||||
useChatStore,
|
||||
useAccessStore,
|
||||
ChatMessageTool,
|
||||
} from "@/app/store";
|
||||
import { preProcessImageContent } from "@/app/utils/chat";
|
||||
import { getMessageTextContent, isVisionModel } from "@/app/utils";
|
||||
import { ApiPath, BEDROCK_BASE_URL, REQUEST_TIMEOUT_MS } from "@/app/constant";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import {
|
||||
extractMessage,
|
||||
processMessage,
|
||||
processChunks,
|
||||
parseEventData,
|
||||
sign,
|
||||
} from "@/app/utils/aws";
|
||||
import { prettyObject } from "@/app/utils/format";
|
||||
import Locale from "@/app/locales";
|
||||
import { encrypt } from "@/app/utils/aws";
|
||||
|
||||
const ClaudeMapper = {
|
||||
assistant: "assistant",
|
||||
user: "user",
|
||||
system: "user",
|
||||
} as const;
|
||||
|
||||
const MistralMapper = {
|
||||
system: "system",
|
||||
user: "user",
|
||||
assistant: "assistant",
|
||||
} as const;
|
||||
type MistralRole = keyof typeof MistralMapper;
|
||||
|
||||
interface Tool {
|
||||
function?: {
|
||||
name?: string;
|
||||
description?: string;
|
||||
parameters?: any;
|
||||
};
|
||||
}
|
||||
const isApp = !!getClientConfig()?.isApp;
|
||||
// const isApp = true;
|
||||
async function getBedrockHeaders(
|
||||
modelId: string,
|
||||
chatPath: string,
|
||||
finalRequestBody: any,
|
||||
shouldStream: boolean,
|
||||
): Promise<Record<string, string>> {
|
||||
const accessStore = useAccessStore.getState();
|
||||
const bedrockHeaders = isApp
|
||||
? await sign({
|
||||
method: "POST",
|
||||
url: chatPath,
|
||||
region: accessStore.awsRegion,
|
||||
accessKeyId: accessStore.awsAccessKey,
|
||||
secretAccessKey: accessStore.awsSecretKey,
|
||||
body: finalRequestBody,
|
||||
service: "bedrock",
|
||||
headers: {},
|
||||
isStreaming: shouldStream,
|
||||
})
|
||||
: getHeaders();
|
||||
|
||||
if (!isApp) {
|
||||
const { awsRegion, awsAccessKey, awsSecretKey, encryptionKey } =
|
||||
accessStore;
|
||||
|
||||
const bedrockHeadersConfig = {
|
||||
XModelID: modelId,
|
||||
XEncryptionKey: encryptionKey,
|
||||
ShouldStream: String(shouldStream),
|
||||
Authorization: await createAuthHeader(
|
||||
awsRegion,
|
||||
awsAccessKey,
|
||||
awsSecretKey,
|
||||
encryptionKey,
|
||||
),
|
||||
};
|
||||
|
||||
Object.assign(bedrockHeaders, bedrockHeadersConfig);
|
||||
}
|
||||
|
||||
return bedrockHeaders;
|
||||
}
|
||||
|
||||
// Helper function to create Authorization header
|
||||
async function createAuthHeader(
|
||||
region: string,
|
||||
accessKey: string,
|
||||
secretKey: string,
|
||||
encryptionKey: string,
|
||||
): Promise<string> {
|
||||
const encryptedValues = await Promise.all([
|
||||
encrypt(region, encryptionKey),
|
||||
encrypt(accessKey, encryptionKey),
|
||||
encrypt(secretKey, encryptionKey),
|
||||
]);
|
||||
|
||||
return `Bearer ${encryptedValues.join(":")}`;
|
||||
}
|
||||
|
||||
export class BedrockApi implements LLMApi {
|
||||
speech(options: SpeechOptions): Promise<ArrayBuffer> {
|
||||
throw new Error("Speech not implemented for Bedrock.");
|
||||
}
|
||||
|
||||
formatRequestBody(messages: ChatOptions["messages"], modelConfig: any) {
|
||||
const model = modelConfig.model;
|
||||
const visionModel = isVisionModel(modelConfig.model);
|
||||
|
||||
// Get tools if available
|
||||
const [tools] = usePluginStore
|
||||
.getState()
|
||||
.getAsTools(useChatStore.getState().currentSession().mask?.plugin || []);
|
||||
|
||||
const toolsArray = (tools as Tool[]) || [];
|
||||
|
||||
// Handle Nova models
|
||||
if (model.includes("amazon.nova")) {
|
||||
// Extract system message if present
|
||||
const systemMessage = messages.find((m) => m.role === "system");
|
||||
const conversationMessages = messages.filter((m) => m.role !== "system");
|
||||
|
||||
const requestBody: any = {
|
||||
schemaVersion: "messages-v1",
|
||||
messages: conversationMessages.map((message) => {
|
||||
const content = Array.isArray(message.content)
|
||||
? message.content
|
||||
: [{ text: getMessageTextContent(message) }];
|
||||
|
||||
return {
|
||||
role: message.role,
|
||||
content: content.map((item: any) => {
|
||||
// Handle text content
|
||||
if (item.text || typeof item === "string") {
|
||||
return { text: item.text || item };
|
||||
}
|
||||
// Handle image content
|
||||
if (item.image_url?.url) {
|
||||
const { url = "" } = item.image_url;
|
||||
const colonIndex = url.indexOf(":");
|
||||
const semicolonIndex = url.indexOf(";");
|
||||
const comma = url.indexOf(",");
|
||||
|
||||
// Extract format from mime type
|
||||
const mimeType = url.slice(colonIndex + 1, semicolonIndex);
|
||||
const format = mimeType.split("/")[1];
|
||||
const data = url.slice(comma + 1);
|
||||
|
||||
return {
|
||||
image: {
|
||||
format,
|
||||
source: {
|
||||
bytes: data,
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
return item;
|
||||
}),
|
||||
};
|
||||
}),
|
||||
inferenceConfig: {
|
||||
temperature: modelConfig.temperature || 0.7,
|
||||
top_p: modelConfig.top_p || 0.9,
|
||||
top_k: modelConfig.top_k || 50,
|
||||
max_new_tokens: modelConfig.max_tokens || 1000,
|
||||
stopSequences: modelConfig.stop || [],
|
||||
},
|
||||
};
|
||||
|
||||
// Add system message if present
|
||||
if (systemMessage) {
|
||||
requestBody.system = [
|
||||
{
|
||||
text: getMessageTextContent(systemMessage),
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
// Add tools if available - exact Nova format
|
||||
if (toolsArray.length > 0) {
|
||||
requestBody.toolConfig = {
|
||||
tools: toolsArray.map((tool) => ({
|
||||
toolSpec: {
|
||||
name: tool?.function?.name || "",
|
||||
description: tool?.function?.description || "",
|
||||
inputSchema: {
|
||||
json: {
|
||||
type: "object",
|
||||
properties: tool?.function?.parameters?.properties || {},
|
||||
required: tool?.function?.parameters?.required || [],
|
||||
},
|
||||
},
|
||||
},
|
||||
})),
|
||||
toolChoice: { auto: {} },
|
||||
};
|
||||
}
|
||||
|
||||
return requestBody;
|
||||
}
|
||||
|
||||
// Handle Titan models
|
||||
if (model.startsWith("amazon.titan")) {
|
||||
const inputText = messages
|
||||
.map((message) => {
|
||||
return `${message.role}: ${getMessageTextContent(message)}`;
|
||||
})
|
||||
.join("\n\n");
|
||||
|
||||
return {
|
||||
inputText,
|
||||
textGenerationConfig: {
|
||||
maxTokenCount: modelConfig.max_tokens,
|
||||
temperature: modelConfig.temperature,
|
||||
stopSequences: [],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Handle LLaMA models
|
||||
if (model.includes("meta.llama")) {
|
||||
let prompt = "<|begin_of_text|>";
|
||||
|
||||
// Extract system message if present
|
||||
const systemMessage = messages.find((m) => m.role === "system");
|
||||
if (systemMessage) {
|
||||
prompt += `<|start_header_id|>system<|end_header_id|>\n${getMessageTextContent(
|
||||
systemMessage,
|
||||
)}<|eot_id|>`;
|
||||
}
|
||||
|
||||
// Format the conversation
|
||||
const conversationMessages = messages.filter((m) => m.role !== "system");
|
||||
for (const message of conversationMessages) {
|
||||
const role = message.role === "assistant" ? "assistant" : "user";
|
||||
const content = getMessageTextContent(message);
|
||||
prompt += `<|start_header_id|>${role}<|end_header_id|>\n${content}<|eot_id|>`;
|
||||
}
|
||||
|
||||
// Add the final assistant header to prompt completion
|
||||
prompt += "<|start_header_id|>assistant<|end_header_id|>";
|
||||
|
||||
return {
|
||||
prompt,
|
||||
max_gen_len: modelConfig.max_tokens || 512,
|
||||
temperature: modelConfig.temperature || 0.7,
|
||||
top_p: modelConfig.top_p || 0.9,
|
||||
};
|
||||
}
|
||||
|
||||
// Handle Mistral models
|
||||
if (model.includes("mistral.mistral")) {
|
||||
const formattedMessages = messages.map((message) => ({
|
||||
role: MistralMapper[message.role as MistralRole] || "user",
|
||||
content: getMessageTextContent(message),
|
||||
}));
|
||||
|
||||
const requestBody: any = {
|
||||
messages: formattedMessages,
|
||||
max_tokens: modelConfig.max_tokens || 4096,
|
||||
temperature: modelConfig.temperature || 0.7,
|
||||
top_p: modelConfig.top_p || 0.9,
|
||||
};
|
||||
|
||||
// Add tools if available
|
||||
if (toolsArray.length > 0) {
|
||||
requestBody.tool_choice = "auto";
|
||||
requestBody.tools = toolsArray.map((tool) => ({
|
||||
type: "function",
|
||||
function: {
|
||||
name: tool?.function?.name,
|
||||
description: tool?.function?.description,
|
||||
parameters: tool?.function?.parameters,
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
return requestBody;
|
||||
}
|
||||
|
||||
// Handle Claude models
|
||||
const keys = ["system", "user"];
|
||||
// roles must alternate between "user" and "assistant" in claude, so add a fake assistant message between two user messages
|
||||
for (let i = 0; i < messages.length - 1; i++) {
|
||||
const message = messages[i];
|
||||
const nextMessage = messages[i + 1];
|
||||
|
||||
if (keys.includes(message.role) && keys.includes(nextMessage.role)) {
|
||||
messages[i] = [
|
||||
message,
|
||||
{
|
||||
role: "assistant",
|
||||
content: ";",
|
||||
},
|
||||
] as any;
|
||||
}
|
||||
}
|
||||
const prompt = messages
|
||||
.flat()
|
||||
.filter((v) => {
|
||||
if (!v.content) return false;
|
||||
if (typeof v.content === "string" && !v.content.trim()) return false;
|
||||
return true;
|
||||
})
|
||||
.map((v) => {
|
||||
const { role, content } = v;
|
||||
const insideRole = ClaudeMapper[role] ?? "user";
|
||||
|
||||
if (!visionModel || typeof content === "string") {
|
||||
return {
|
||||
role: insideRole,
|
||||
content: getMessageTextContent(v),
|
||||
};
|
||||
}
|
||||
return {
|
||||
role: insideRole,
|
||||
content: content
|
||||
.filter((v) => v.image_url || v.text)
|
||||
.map(({ type, text, image_url }) => {
|
||||
if (type === "text") {
|
||||
return {
|
||||
type,
|
||||
text: text!,
|
||||
};
|
||||
}
|
||||
const { url = "" } = image_url || {};
|
||||
const colonIndex = url.indexOf(":");
|
||||
const semicolonIndex = url.indexOf(";");
|
||||
const comma = url.indexOf(",");
|
||||
|
||||
const mimeType = url.slice(colonIndex + 1, semicolonIndex);
|
||||
const encodeType = url.slice(semicolonIndex + 1, comma);
|
||||
const data = url.slice(comma + 1);
|
||||
|
||||
return {
|
||||
type: "image" as const,
|
||||
source: {
|
||||
type: encodeType,
|
||||
media_type: mimeType,
|
||||
data,
|
||||
},
|
||||
};
|
||||
}),
|
||||
};
|
||||
});
|
||||
|
||||
if (prompt[0]?.role === "assistant") {
|
||||
prompt.unshift({
|
||||
role: "user",
|
||||
content: ";",
|
||||
});
|
||||
}
|
||||
const requestBody: any = {
|
||||
anthropic_version: useAccessStore.getState().bedrockAnthropicVersion,
|
||||
max_tokens: modelConfig.max_tokens,
|
||||
messages: prompt,
|
||||
temperature: modelConfig.temperature,
|
||||
top_p: modelConfig.top_p || 0.9,
|
||||
top_k: modelConfig.top_k || 5,
|
||||
};
|
||||
|
||||
// Add tools if available for Claude models
|
||||
if (toolsArray.length > 0 && model.includes("anthropic.claude")) {
|
||||
requestBody.tools = toolsArray.map((tool) => ({
|
||||
name: tool?.function?.name || "",
|
||||
description: tool?.function?.description || "",
|
||||
input_schema: tool?.function?.parameters || {},
|
||||
}));
|
||||
}
|
||||
|
||||
return requestBody;
|
||||
}
|
||||
|
||||
async chat(options: ChatOptions) {
|
||||
const accessStore = useAccessStore.getState();
|
||||
|
||||
const shouldStream = !!options.config.stream;
|
||||
|
||||
const modelConfig = {
|
||||
...useAppConfig.getState().modelConfig,
|
||||
...useChatStore.getState().currentSession().mask.modelConfig,
|
||||
...{
|
||||
model: options.config.model,
|
||||
},
|
||||
};
|
||||
|
||||
// try get base64image from local cache image_url
|
||||
const messages: ChatOptions["messages"] = [];
|
||||
for (const v of options.messages) {
|
||||
const content = await preProcessImageContent(v.content);
|
||||
messages.push({ role: v.role, content });
|
||||
}
|
||||
|
||||
const controller = new AbortController();
|
||||
options.onController?.(controller);
|
||||
|
||||
let finalRequestBody = this.formatRequestBody(messages, modelConfig);
|
||||
|
||||
try {
|
||||
const bedrockAPIPath = `${BEDROCK_BASE_URL}/model/${
|
||||
modelConfig.model
|
||||
}/invoke${shouldStream ? "-with-response-stream" : ""}`;
|
||||
const chatPath = isApp ? bedrockAPIPath : ApiPath.Bedrock + "/chat";
|
||||
|
||||
if (process.env.NODE_ENV !== "production") {
|
||||
console.debug("[Bedrock Client] Request:", {
|
||||
path: chatPath,
|
||||
model: modelConfig.model,
|
||||
messages: messages.length,
|
||||
stream: shouldStream,
|
||||
});
|
||||
}
|
||||
|
||||
if (shouldStream) {
|
||||
const [tools, funcs] = usePluginStore
|
||||
.getState()
|
||||
.getAsTools(
|
||||
useChatStore.getState().currentSession().mask?.plugin || [],
|
||||
);
|
||||
return bedrockStream(
|
||||
modelConfig.model,
|
||||
chatPath,
|
||||
finalRequestBody,
|
||||
funcs,
|
||||
controller,
|
||||
// processToolMessage, include tool_calls message and tool call results
|
||||
(
|
||||
requestPayload: any[],
|
||||
toolCallMessage: any,
|
||||
toolCallResult: any[],
|
||||
) => {
|
||||
const modelId = modelConfig.model;
|
||||
const isMistral = modelId.includes("mistral.mistral");
|
||||
const isClaude = modelId.includes("anthropic.claude");
|
||||
const isNova = modelId.includes("amazon.nova");
|
||||
|
||||
if (isClaude) {
|
||||
// Format for Claude
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.splice(
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.length,
|
||||
0,
|
||||
{
|
||||
role: "assistant",
|
||||
content: toolCallMessage.tool_calls.map(
|
||||
(tool: ChatMessageTool) => ({
|
||||
type: "tool_use",
|
||||
id: tool.id,
|
||||
name: tool?.function?.name,
|
||||
input: tool?.function?.arguments
|
||||
? JSON.parse(tool?.function?.arguments)
|
||||
: {},
|
||||
}),
|
||||
),
|
||||
},
|
||||
// @ts-ignore
|
||||
...toolCallResult.map((result) => ({
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
type: "tool_result",
|
||||
tool_use_id: result.tool_call_id,
|
||||
content: result.content,
|
||||
},
|
||||
],
|
||||
})),
|
||||
);
|
||||
} else if (isMistral) {
|
||||
// Format for Mistral
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.splice(
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.length,
|
||||
0,
|
||||
{
|
||||
role: "assistant",
|
||||
content: "",
|
||||
// @ts-ignore
|
||||
tool_calls: toolCallMessage.tool_calls.map(
|
||||
(tool: ChatMessageTool) => ({
|
||||
id: tool.id,
|
||||
function: {
|
||||
name: tool?.function?.name,
|
||||
arguments: tool?.function?.arguments || "{}",
|
||||
},
|
||||
}),
|
||||
),
|
||||
},
|
||||
...toolCallResult.map((result) => ({
|
||||
role: "tool",
|
||||
tool_call_id: result.tool_call_id,
|
||||
content: result.content,
|
||||
})),
|
||||
);
|
||||
} else if (isNova) {
|
||||
// Format for Nova - Updated format
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.splice(
|
||||
// @ts-ignore
|
||||
requestPayload?.messages?.length,
|
||||
0,
|
||||
{
|
||||
role: "assistant",
|
||||
content: [
|
||||
{
|
||||
toolUse: {
|
||||
toolUseId: toolCallMessage.tool_calls[0].id,
|
||||
name: toolCallMessage.tool_calls[0]?.function?.name,
|
||||
input:
|
||||
typeof toolCallMessage.tool_calls[0]?.function
|
||||
?.arguments === "string"
|
||||
? JSON.parse(
|
||||
toolCallMessage.tool_calls[0]?.function
|
||||
?.arguments,
|
||||
)
|
||||
: toolCallMessage.tool_calls[0]?.function
|
||||
?.arguments || {},
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
role: "user",
|
||||
content: [
|
||||
{
|
||||
toolResult: {
|
||||
toolUseId: toolCallResult[0].tool_call_id,
|
||||
content: [
|
||||
{
|
||||
json: {
|
||||
content: toolCallResult[0].content,
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
} else {
|
||||
console.warn(
|
||||
`[Bedrock Client] Unhandled model type for tool calls: ${modelId}`,
|
||||
);
|
||||
}
|
||||
},
|
||||
options,
|
||||
);
|
||||
} else {
|
||||
try {
|
||||
controller.signal.onabort = () =>
|
||||
options.onFinish("", new Response(null, { status: 400 }));
|
||||
const newHeaders = await getBedrockHeaders(
|
||||
modelConfig.model,
|
||||
chatPath,
|
||||
JSON.stringify(finalRequestBody),
|
||||
shouldStream,
|
||||
);
|
||||
const res = await fetch(chatPath, {
|
||||
method: "POST",
|
||||
headers: newHeaders,
|
||||
body: JSON.stringify(finalRequestBody),
|
||||
});
|
||||
const contentType = res.headers.get("content-type");
|
||||
console.log(
|
||||
"[Bedrock Not Stream Request] response content type: ",
|
||||
contentType,
|
||||
);
|
||||
const resJson = await res.json();
|
||||
const message = extractMessage(resJson);
|
||||
options.onFinish(message, res);
|
||||
} catch (e) {
|
||||
const error =
|
||||
e instanceof Error ? e : new Error("Unknown error occurred");
|
||||
console.error("[Bedrock Client] Chat failed:", error.message);
|
||||
options.onError?.(error);
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("[Bedrock Client] Chat error:", e);
|
||||
options.onError?.(e as Error);
|
||||
}
|
||||
}
|
||||
|
||||
async usage() {
|
||||
return { used: 0, total: 0 };
|
||||
}
|
||||
|
||||
async models() {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function bedrockStream(
|
||||
modelId: string,
|
||||
chatPath: string,
|
||||
requestPayload: any,
|
||||
funcs: Record<string, Function>,
|
||||
controller: AbortController,
|
||||
processToolMessage: (
|
||||
requestPayload: any,
|
||||
toolCallMessage: any,
|
||||
toolCallResult: any[],
|
||||
) => void,
|
||||
options: any,
|
||||
) {
|
||||
let responseText = "";
|
||||
let remainText = "";
|
||||
let finished = false;
|
||||
let running = false;
|
||||
let runTools: any[] = [];
|
||||
let responseRes: Response;
|
||||
let index = -1;
|
||||
let chunks: Uint8Array[] = [];
|
||||
let pendingChunk: Uint8Array | null = null;
|
||||
|
||||
function animateResponseText() {
|
||||
if (finished || controller.signal.aborted) {
|
||||
responseText += remainText;
|
||||
console.log("[Response Animation] finished");
|
||||
if (responseText?.length === 0) {
|
||||
options.onError?.(new Error("empty response from server"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (remainText.length > 0) {
|
||||
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
|
||||
const fetchText = remainText.slice(0, fetchCount);
|
||||
responseText += fetchText;
|
||||
remainText = remainText.slice(fetchCount);
|
||||
options.onUpdate?.(responseText, fetchText);
|
||||
}
|
||||
|
||||
requestAnimationFrame(animateResponseText);
|
||||
}
|
||||
|
||||
animateResponseText();
|
||||
|
||||
const finish = () => {
|
||||
if (!finished) {
|
||||
if (!running && runTools.length > 0) {
|
||||
const toolCallMessage = {
|
||||
role: "assistant",
|
||||
tool_calls: [...runTools],
|
||||
};
|
||||
running = true;
|
||||
runTools.splice(0, runTools.length);
|
||||
return Promise.all(
|
||||
toolCallMessage.tool_calls.map((tool) => {
|
||||
options?.onBeforeTool?.(tool);
|
||||
const funcName = tool?.function?.name || tool?.name;
|
||||
if (!funcName || !funcs[funcName]) {
|
||||
console.error(`Function ${funcName} not found in funcs:`, funcs);
|
||||
return Promise.reject(`Function ${funcName} not found`);
|
||||
}
|
||||
return Promise.resolve(
|
||||
funcs[funcName](
|
||||
tool?.function?.arguments
|
||||
? JSON.parse(tool?.function?.arguments)
|
||||
: {},
|
||||
),
|
||||
)
|
||||
.then((res) => {
|
||||
let content = res.data || res?.statusText;
|
||||
content =
|
||||
typeof content === "string"
|
||||
? content
|
||||
: JSON.stringify(content);
|
||||
if (res.status >= 300) {
|
||||
return Promise.reject(content);
|
||||
}
|
||||
return content;
|
||||
})
|
||||
.then((content) => {
|
||||
options?.onAfterTool?.({
|
||||
...tool,
|
||||
content,
|
||||
isError: false,
|
||||
});
|
||||
return content;
|
||||
})
|
||||
.catch((e) => {
|
||||
options?.onAfterTool?.({
|
||||
...tool,
|
||||
isError: true,
|
||||
errorMsg: e.toString(),
|
||||
});
|
||||
return e.toString();
|
||||
})
|
||||
.then((content) => ({
|
||||
name: funcName,
|
||||
role: "tool",
|
||||
content,
|
||||
tool_call_id: tool.id,
|
||||
}));
|
||||
}),
|
||||
).then((toolCallResult) => {
|
||||
processToolMessage(requestPayload, toolCallMessage, toolCallResult);
|
||||
setTimeout(() => {
|
||||
console.debug("[BedrockAPI for toolCallResult] restart");
|
||||
running = false;
|
||||
bedrockChatApi(modelId, chatPath, requestPayload, true);
|
||||
}, 60);
|
||||
});
|
||||
}
|
||||
if (running) {
|
||||
return;
|
||||
}
|
||||
console.debug("[BedrockAPI] end");
|
||||
finished = true;
|
||||
options.onFinish(responseText + remainText, responseRes);
|
||||
}
|
||||
};
|
||||
|
||||
controller.signal.onabort = finish;
|
||||
|
||||
async function bedrockChatApi(
|
||||
modelId: string,
|
||||
chatPath: string,
|
||||
requestPayload: any,
|
||||
shouldStream: boolean,
|
||||
) {
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
);
|
||||
|
||||
const newHeaders = await getBedrockHeaders(
|
||||
modelId,
|
||||
chatPath,
|
||||
JSON.stringify(requestPayload),
|
||||
shouldStream,
|
||||
);
|
||||
try {
|
||||
const res = await fetch(chatPath, {
|
||||
method: "POST",
|
||||
headers: newHeaders,
|
||||
body: JSON.stringify(requestPayload),
|
||||
redirect: "manual",
|
||||
// @ts-ignore
|
||||
duplex: "half",
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
clearTimeout(requestTimeoutId);
|
||||
responseRes = res;
|
||||
|
||||
const contentType = res.headers.get("content-type");
|
||||
// console.log(
|
||||
// "[Bedrock Stream Request] response content type: ",
|
||||
// contentType,
|
||||
// );
|
||||
|
||||
if (contentType?.startsWith("text/plain")) {
|
||||
responseText = await res.text();
|
||||
return finish();
|
||||
}
|
||||
|
||||
if (
|
||||
!res.ok ||
|
||||
res.status !== 200 ||
|
||||
!contentType?.startsWith("application/vnd.amazon.eventstream")
|
||||
) {
|
||||
const responseTexts = [responseText];
|
||||
let extraInfo = await res.text();
|
||||
try {
|
||||
const resJson = await res.clone().json();
|
||||
extraInfo = prettyObject(resJson);
|
||||
} catch {}
|
||||
|
||||
if (res.status === 401) {
|
||||
responseTexts.push(Locale.Error.Unauthorized);
|
||||
}
|
||||
|
||||
if (extraInfo) {
|
||||
responseTexts.push(extraInfo);
|
||||
}
|
||||
|
||||
responseText = responseTexts.join("\n\n");
|
||||
return finish();
|
||||
}
|
||||
|
||||
const reader = res.body?.getReader();
|
||||
if (!reader) {
|
||||
throw new Error("No response body reader available");
|
||||
}
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) {
|
||||
if (pendingChunk) {
|
||||
try {
|
||||
const parsed = parseEventData(pendingChunk);
|
||||
if (parsed) {
|
||||
const result = processMessage(
|
||||
parsed,
|
||||
remainText,
|
||||
runTools,
|
||||
index,
|
||||
);
|
||||
remainText = result.remainText;
|
||||
index = result.index;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("[Final Chunk Process Error]:", e);
|
||||
}
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
chunks.push(value);
|
||||
|
||||
const result = processChunks(
|
||||
chunks,
|
||||
pendingChunk,
|
||||
remainText,
|
||||
runTools,
|
||||
index,
|
||||
);
|
||||
chunks = result.chunks;
|
||||
pendingChunk = result.pendingChunk;
|
||||
remainText = result.remainText;
|
||||
index = result.index;
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(
|
||||
"[Bedrock Stream]:",
|
||||
err instanceof Error ? err.message : "Stream processing failed",
|
||||
);
|
||||
throw new Error("Failed to process stream response");
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
finish();
|
||||
}
|
||||
} catch (e) {
|
||||
if (e instanceof Error && e.name === "AbortError") {
|
||||
console.log("[Bedrock Client] Aborted by user");
|
||||
return;
|
||||
}
|
||||
console.error(
|
||||
"[Bedrock Request] Failed:",
|
||||
e instanceof Error ? e.message : "Request failed",
|
||||
);
|
||||
options.onError?.(e);
|
||||
throw new Error("Request processing failed");
|
||||
}
|
||||
}
|
||||
|
||||
console.debug("[BedrockAPI] start");
|
||||
bedrockChatApi(modelId, chatPath, requestPayload, true);
|
||||
}
|
||||
@@ -1,11 +1,12 @@
|
||||
"use client";
|
||||
import { ApiPath, ByteDance, BYTEDANCE_BASE_URL } from "@/app/constant";
|
||||
import {
|
||||
ApiPath,
|
||||
ByteDance,
|
||||
BYTEDANCE_BASE_URL,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
} from "@/app/constant";
|
||||
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
|
||||
useAccessStore,
|
||||
useAppConfig,
|
||||
useChatStore,
|
||||
ChatMessageTool,
|
||||
usePluginStore,
|
||||
} from "@/app/store";
|
||||
|
||||
import {
|
||||
ChatOptions,
|
||||
@@ -15,14 +16,14 @@ import {
|
||||
MultimodalContent,
|
||||
SpeechOptions,
|
||||
} from "../api";
|
||||
import Locale from "../../locales";
|
||||
import {
|
||||
EventStreamContentType,
|
||||
fetchEventSource,
|
||||
} from "@fortaine/fetch-event-source";
|
||||
import { prettyObject } from "@/app/utils/format";
|
||||
|
||||
import { streamWithThink } from "@/app/utils/chat";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { preProcessImageContent } from "@/app/utils/chat";
|
||||
import {
|
||||
getMessageTextContentWithoutThinking,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
|
||||
export interface OpenAIListModelResponse {
|
||||
@@ -34,7 +35,7 @@ export interface OpenAIListModelResponse {
|
||||
}>;
|
||||
}
|
||||
|
||||
interface RequestPayload {
|
||||
interface RequestPayloadForByteDance {
|
||||
messages: {
|
||||
role: "system" | "user" | "assistant";
|
||||
content: string | MultimodalContent[];
|
||||
@@ -86,7 +87,10 @@ export class DoubaoApi implements LLMApi {
|
||||
async chat(options: ChatOptions) {
|
||||
const messages: ChatOptions["messages"] = [];
|
||||
for (const v of options.messages) {
|
||||
const content = await preProcessImageContent(v.content);
|
||||
const content =
|
||||
v.role === "assistant"
|
||||
? getMessageTextContentWithoutThinking(v)
|
||||
: await preProcessImageContent(v.content);
|
||||
messages.push({ role: v.role, content });
|
||||
}
|
||||
|
||||
@@ -99,7 +103,7 @@ export class DoubaoApi implements LLMApi {
|
||||
};
|
||||
|
||||
const shouldStream = !!options.config.stream;
|
||||
const requestPayload: RequestPayload = {
|
||||
const requestPayload: RequestPayloadForByteDance = {
|
||||
messages,
|
||||
stream: shouldStream,
|
||||
model: modelConfig.model,
|
||||
@@ -124,119 +128,101 @@ export class DoubaoApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
let responseText = "";
|
||||
let remainText = "";
|
||||
let finished = false;
|
||||
let responseRes: Response;
|
||||
const [tools, funcs] = usePluginStore
|
||||
.getState()
|
||||
.getAsTools(
|
||||
useChatStore.getState().currentSession().mask?.plugin || [],
|
||||
);
|
||||
return streamWithThink(
|
||||
chatPath,
|
||||
requestPayload,
|
||||
getHeaders(),
|
||||
tools as any,
|
||||
funcs,
|
||||
controller,
|
||||
// parseSSE
|
||||
(text: string, runTools: ChatMessageTool[]) => {
|
||||
// console.log("parseSSE", text, runTools);
|
||||
const json = JSON.parse(text);
|
||||
const choices = json.choices as Array<{
|
||||
delta: {
|
||||
content: string | null;
|
||||
tool_calls: ChatMessageTool[];
|
||||
reasoning_content: string | null;
|
||||
};
|
||||
}>;
|
||||
|
||||
// animate response to make it looks smooth
|
||||
function animateResponseText() {
|
||||
if (finished || controller.signal.aborted) {
|
||||
responseText += remainText;
|
||||
console.log("[Response Animation] finished");
|
||||
if (responseText?.length === 0) {
|
||||
options.onError?.(new Error("empty response from server"));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (remainText.length > 0) {
|
||||
const fetchCount = Math.max(1, Math.round(remainText.length / 60));
|
||||
const fetchText = remainText.slice(0, fetchCount);
|
||||
responseText += fetchText;
|
||||
remainText = remainText.slice(fetchCount);
|
||||
options.onUpdate?.(responseText, fetchText);
|
||||
}
|
||||
|
||||
requestAnimationFrame(animateResponseText);
|
||||
}
|
||||
|
||||
// start animaion
|
||||
animateResponseText();
|
||||
|
||||
const finish = () => {
|
||||
if (!finished) {
|
||||
finished = true;
|
||||
options.onFinish(responseText + remainText, responseRes);
|
||||
}
|
||||
};
|
||||
|
||||
controller.signal.onabort = finish;
|
||||
|
||||
fetchEventSource(chatPath, {
|
||||
fetch: fetch as any,
|
||||
...chatPayload,
|
||||
async onopen(res) {
|
||||
clearTimeout(requestTimeoutId);
|
||||
const contentType = res.headers.get("content-type");
|
||||
console.log(
|
||||
"[ByteDance] request response content type: ",
|
||||
contentType,
|
||||
);
|
||||
responseRes = res;
|
||||
if (contentType?.startsWith("text/plain")) {
|
||||
responseText = await res.clone().text();
|
||||
return finish();
|
||||
if (!choices?.length) return { isThinking: false, content: "" };
|
||||
|
||||
const tool_calls = choices[0]?.delta?.tool_calls;
|
||||
if (tool_calls?.length > 0) {
|
||||
const index = tool_calls[0]?.index;
|
||||
const id = tool_calls[0]?.id;
|
||||
const args = tool_calls[0]?.function?.arguments;
|
||||
if (id) {
|
||||
runTools.push({
|
||||
id,
|
||||
type: tool_calls[0]?.type,
|
||||
function: {
|
||||
name: tool_calls[0]?.function?.name as string,
|
||||
arguments: args,
|
||||
},
|
||||
});
|
||||
} else {
|
||||
// @ts-ignore
|
||||
runTools[index]["function"]["arguments"] += args;
|
||||
}
|
||||
}
|
||||
const reasoning = choices[0]?.delta?.reasoning_content;
|
||||
const content = choices[0]?.delta?.content;
|
||||
|
||||
// Skip if both content and reasoning_content are empty or null
|
||||
if (
|
||||
!res.ok ||
|
||||
!res.headers
|
||||
.get("content-type")
|
||||
?.startsWith(EventStreamContentType) ||
|
||||
res.status !== 200
|
||||
(!reasoning || reasoning.length === 0) &&
|
||||
(!content || content.length === 0)
|
||||
) {
|
||||
const responseTexts = [responseText];
|
||||
let extraInfo = await res.clone().text();
|
||||
try {
|
||||
const resJson = await res.clone().json();
|
||||
extraInfo = prettyObject(resJson);
|
||||
} catch {}
|
||||
|
||||
if (res.status === 401) {
|
||||
responseTexts.push(Locale.Error.Unauthorized);
|
||||
}
|
||||
|
||||
if (extraInfo) {
|
||||
responseTexts.push(extraInfo);
|
||||
}
|
||||
|
||||
responseText = responseTexts.join("\n\n");
|
||||
|
||||
return finish();
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
}
|
||||
},
|
||||
onmessage(msg) {
|
||||
if (msg.data === "[DONE]" || finished) {
|
||||
return finish();
|
||||
}
|
||||
const text = msg.data;
|
||||
try {
|
||||
const json = JSON.parse(text);
|
||||
const choices = json.choices as Array<{
|
||||
delta: { content: string };
|
||||
}>;
|
||||
const delta = choices[0]?.delta?.content;
|
||||
if (delta) {
|
||||
remainText += delta;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("[Request] parse error", text, msg);
|
||||
|
||||
if (reasoning && reasoning.length > 0) {
|
||||
return {
|
||||
isThinking: true,
|
||||
content: reasoning,
|
||||
};
|
||||
} else if (content && content.length > 0) {
|
||||
return {
|
||||
isThinking: false,
|
||||
content: content,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
},
|
||||
onclose() {
|
||||
finish();
|
||||
// processToolMessage, include tool_calls message and tool call results
|
||||
(
|
||||
requestPayload: RequestPayloadForByteDance,
|
||||
toolCallMessage: any,
|
||||
toolCallResult: any[],
|
||||
) => {
|
||||
requestPayload?.messages?.splice(
|
||||
requestPayload?.messages?.length,
|
||||
0,
|
||||
toolCallMessage,
|
||||
...toolCallResult,
|
||||
);
|
||||
},
|
||||
onerror(e) {
|
||||
options.onError?.(e);
|
||||
throw e;
|
||||
},
|
||||
openWhenHidden: true,
|
||||
});
|
||||
options,
|
||||
);
|
||||
} else {
|
||||
const res = await fetch(chatPath, chatPayload);
|
||||
clearTimeout(requestTimeoutId);
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
"use client";
|
||||
// azure and openai, using same models. so using same LLMApi.
|
||||
import {
|
||||
ApiPath,
|
||||
DEEPSEEK_BASE_URL,
|
||||
DeepSeek,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
} from "@/app/constant";
|
||||
import { ApiPath, DEEPSEEK_BASE_URL, DeepSeek } from "@/app/constant";
|
||||
import {
|
||||
useAccessStore,
|
||||
useAppConfig,
|
||||
@@ -26,6 +20,7 @@ import { getClientConfig } from "@/app/config/client";
|
||||
import {
|
||||
getMessageTextContent,
|
||||
getMessageTextContentWithoutThinking,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { RequestPayload } from "./openai";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
@@ -116,16 +111,10 @@ export class DeepSeekApi implements LLMApi {
|
||||
headers: getHeaders(),
|
||||
};
|
||||
|
||||
// console.log(chatPayload);
|
||||
|
||||
const isR1 =
|
||||
options.config.model.endsWith("-reasoner") ||
|
||||
options.config.model.endsWith("-r1");
|
||||
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
isR1 ? REQUEST_TIMEOUT_MS_FOR_THINKING : REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
@@ -176,8 +165,8 @@ export class DeepSeekApi implements LLMApi {
|
||||
|
||||
// Skip if both content and reasoning_content are empty or null
|
||||
if (
|
||||
(!reasoning || reasoning.trim().length === 0) &&
|
||||
(!content || content.trim().length === 0)
|
||||
(!reasoning || reasoning.length === 0) &&
|
||||
(!content || content.length === 0)
|
||||
) {
|
||||
return {
|
||||
isThinking: false,
|
||||
@@ -185,12 +174,12 @@ export class DeepSeekApi implements LLMApi {
|
||||
};
|
||||
}
|
||||
|
||||
if (reasoning && reasoning.trim().length > 0) {
|
||||
if (reasoning && reasoning.length > 0) {
|
||||
return {
|
||||
isThinking: true,
|
||||
content: reasoning,
|
||||
};
|
||||
} else if (content && content.trim().length > 0) {
|
||||
} else if (content && content.length > 0) {
|
||||
return {
|
||||
isThinking: false,
|
||||
content: content,
|
||||
|
||||
@@ -1,10 +1,5 @@
|
||||
"use client";
|
||||
import {
|
||||
ApiPath,
|
||||
CHATGLM_BASE_URL,
|
||||
ChatGLM,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
} from "@/app/constant";
|
||||
import { ApiPath, CHATGLM_BASE_URL, ChatGLM } from "@/app/constant";
|
||||
import {
|
||||
useAccessStore,
|
||||
useAppConfig,
|
||||
@@ -21,7 +16,11 @@ import {
|
||||
SpeechOptions,
|
||||
} from "../api";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { getMessageTextContent, isVisionModel } from "@/app/utils";
|
||||
import {
|
||||
getMessageTextContent,
|
||||
isVisionModel,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { RequestPayload } from "./openai";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
import { preProcessImageContent } from "@/app/utils/chat";
|
||||
@@ -191,7 +190,7 @@ export class ChatGLMApi implements LLMApi {
|
||||
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (modelType === "image" || modelType === "video") {
|
||||
|
||||
@@ -1,9 +1,4 @@
|
||||
import {
|
||||
ApiPath,
|
||||
Google,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
} from "@/app/constant";
|
||||
import { ApiPath, Google } from "@/app/constant";
|
||||
import {
|
||||
ChatOptions,
|
||||
getHeaders,
|
||||
@@ -27,6 +22,7 @@ import {
|
||||
getMessageTextContent,
|
||||
getMessageImages,
|
||||
isVisionModel,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { preProcessImageContent } from "@/app/utils/chat";
|
||||
import { nanoid } from "nanoid";
|
||||
@@ -206,7 +202,7 @@ export class GeminiProApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
isThinking ? REQUEST_TIMEOUT_MS_FOR_THINKING : REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
|
||||
@@ -8,7 +8,6 @@ import {
|
||||
Azure,
|
||||
REQUEST_TIMEOUT_MS,
|
||||
ServiceProvider,
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
} from "@/app/constant";
|
||||
import {
|
||||
ChatMessageTool,
|
||||
@@ -22,7 +21,7 @@ import {
|
||||
preProcessImageContent,
|
||||
uploadImage,
|
||||
base64Image2Blob,
|
||||
stream,
|
||||
streamWithThink,
|
||||
} from "@/app/utils/chat";
|
||||
import { cloudflareAIGatewayUrl } from "@/app/utils/cloudflare";
|
||||
import { ModelSize, DalleQuality, DalleStyle } from "@/app/typing";
|
||||
@@ -42,6 +41,7 @@ import {
|
||||
getMessageTextContent,
|
||||
isVisionModel,
|
||||
isDalle3 as _isDalle3,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
|
||||
@@ -294,7 +294,7 @@ export class ChatGPTApi implements LLMApi {
|
||||
useChatStore.getState().currentSession().mask?.plugin || [],
|
||||
);
|
||||
// console.log("getAsTools", tools, funcs);
|
||||
stream(
|
||||
streamWithThink(
|
||||
chatPath,
|
||||
requestPayload,
|
||||
getHeaders(),
|
||||
@@ -309,8 +309,12 @@ export class ChatGPTApi implements LLMApi {
|
||||
delta: {
|
||||
content: string;
|
||||
tool_calls: ChatMessageTool[];
|
||||
reasoning_content: string | null;
|
||||
};
|
||||
}>;
|
||||
|
||||
if (!choices?.length) return { isThinking: false, content: "" };
|
||||
|
||||
const tool_calls = choices[0]?.delta?.tool_calls;
|
||||
if (tool_calls?.length > 0) {
|
||||
const id = tool_calls[0]?.id;
|
||||
@@ -330,7 +334,37 @@ export class ChatGPTApi implements LLMApi {
|
||||
runTools[index]["function"]["arguments"] += args;
|
||||
}
|
||||
}
|
||||
return choices[0]?.delta?.content;
|
||||
|
||||
const reasoning = choices[0]?.delta?.reasoning_content;
|
||||
const content = choices[0]?.delta?.content;
|
||||
|
||||
// Skip if both content and reasoning_content are empty or null
|
||||
if (
|
||||
(!reasoning || reasoning.length === 0) &&
|
||||
(!content || content.length === 0)
|
||||
) {
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
}
|
||||
|
||||
if (reasoning && reasoning.length > 0) {
|
||||
return {
|
||||
isThinking: true,
|
||||
content: reasoning,
|
||||
};
|
||||
} else if (content && content.length > 0) {
|
||||
return {
|
||||
isThinking: false,
|
||||
content: content,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
isThinking: false,
|
||||
content: "",
|
||||
};
|
||||
},
|
||||
// processToolMessage, include tool_calls message and tool call results
|
||||
(
|
||||
@@ -362,9 +396,7 @@ export class ChatGPTApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
isDalle3 || isO1OrO3
|
||||
? REQUEST_TIMEOUT_MS_FOR_THINKING
|
||||
: REQUEST_TIMEOUT_MS, // dalle3 using b64_json is slow.
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
const res = await fetch(chatPath, chatPayload);
|
||||
|
||||
@@ -4,7 +4,7 @@ import {
|
||||
ApiPath,
|
||||
SILICONFLOW_BASE_URL,
|
||||
SiliconFlow,
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
DEFAULT_MODELS,
|
||||
} from "@/app/constant";
|
||||
import {
|
||||
useAccessStore,
|
||||
@@ -13,7 +13,7 @@ import {
|
||||
ChatMessageTool,
|
||||
usePluginStore,
|
||||
} from "@/app/store";
|
||||
import { streamWithThink } from "@/app/utils/chat";
|
||||
import { preProcessImageContent, streamWithThink } from "@/app/utils/chat";
|
||||
import {
|
||||
ChatOptions,
|
||||
getHeaders,
|
||||
@@ -25,12 +25,23 @@ import { getClientConfig } from "@/app/config/client";
|
||||
import {
|
||||
getMessageTextContent,
|
||||
getMessageTextContentWithoutThinking,
|
||||
isVisionModel,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import { RequestPayload } from "./openai";
|
||||
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
export interface SiliconFlowListModelResponse {
|
||||
object: string;
|
||||
data: Array<{
|
||||
id: string;
|
||||
object: string;
|
||||
root: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
export class SiliconflowApi implements LLMApi {
|
||||
private disableListModels = true;
|
||||
private disableListModels = false;
|
||||
|
||||
path(path: string): string {
|
||||
const accessStore = useAccessStore.getState();
|
||||
@@ -71,13 +82,16 @@ export class SiliconflowApi implements LLMApi {
|
||||
}
|
||||
|
||||
async chat(options: ChatOptions) {
|
||||
const visionModel = isVisionModel(options.config.model);
|
||||
const messages: ChatOptions["messages"] = [];
|
||||
for (const v of options.messages) {
|
||||
if (v.role === "assistant") {
|
||||
const content = getMessageTextContentWithoutThinking(v);
|
||||
messages.push({ role: v.role, content });
|
||||
} else {
|
||||
const content = getMessageTextContent(v);
|
||||
const content = visionModel
|
||||
? await preProcessImageContent(v.content)
|
||||
: getMessageTextContent(v);
|
||||
messages.push({ role: v.role, content });
|
||||
}
|
||||
}
|
||||
@@ -123,7 +137,7 @@ export class SiliconflowApi implements LLMApi {
|
||||
// Use extended timeout for thinking models as they typically require more processing time
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
@@ -238,6 +252,36 @@ export class SiliconflowApi implements LLMApi {
|
||||
}
|
||||
|
||||
async models(): Promise<LLMModel[]> {
|
||||
return [];
|
||||
if (this.disableListModels) {
|
||||
return DEFAULT_MODELS.slice();
|
||||
}
|
||||
|
||||
const res = await fetch(this.path(SiliconFlow.ListModelPath), {
|
||||
method: "GET",
|
||||
headers: {
|
||||
...getHeaders(),
|
||||
},
|
||||
});
|
||||
|
||||
const resJson = (await res.json()) as SiliconFlowListModelResponse;
|
||||
const chatModels = resJson.data;
|
||||
console.log("[Models]", chatModels);
|
||||
|
||||
if (!chatModels) {
|
||||
return [];
|
||||
}
|
||||
|
||||
let seq = 1000; //同 Constant.ts 中的排序保持一致
|
||||
return chatModels.map((m) => ({
|
||||
name: m.id,
|
||||
available: true,
|
||||
sorted: seq++,
|
||||
provider: {
|
||||
id: "siliconflow",
|
||||
providerName: "SiliconFlow",
|
||||
providerType: "siliconflow",
|
||||
sorted: 14,
|
||||
},
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
"use client";
|
||||
import { ApiPath, TENCENT_BASE_URL, REQUEST_TIMEOUT_MS } from "@/app/constant";
|
||||
import { ApiPath, TENCENT_BASE_URL } from "@/app/constant";
|
||||
import { useAccessStore, useAppConfig, useChatStore } from "@/app/store";
|
||||
|
||||
import {
|
||||
@@ -17,7 +17,11 @@ import {
|
||||
} from "@fortaine/fetch-event-source";
|
||||
import { prettyObject } from "@/app/utils/format";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { getMessageTextContent, isVisionModel } from "@/app/utils";
|
||||
import {
|
||||
getMessageTextContent,
|
||||
isVisionModel,
|
||||
getTimeoutMSByModel,
|
||||
} from "@/app/utils";
|
||||
import mapKeys from "lodash-es/mapKeys";
|
||||
import mapValues from "lodash-es/mapValues";
|
||||
import isArray from "lodash-es/isArray";
|
||||
@@ -135,7 +139,7 @@ export class HunyuanApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client";
|
||||
// azure and openai, using same models. so using same LLMApi.
|
||||
import { ApiPath, XAI_BASE_URL, XAI, REQUEST_TIMEOUT_MS } from "@/app/constant";
|
||||
import { ApiPath, XAI_BASE_URL, XAI } from "@/app/constant";
|
||||
import {
|
||||
useAccessStore,
|
||||
useAppConfig,
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
SpeechOptions,
|
||||
} from "../api";
|
||||
import { getClientConfig } from "@/app/config/client";
|
||||
import { getTimeoutMSByModel } from "@/app/utils";
|
||||
import { preProcessImageContent } from "@/app/utils/chat";
|
||||
import { RequestPayload } from "./openai";
|
||||
import { fetch } from "@/app/utils/stream";
|
||||
@@ -103,7 +104,7 @@ export class XAIApi implements LLMApi {
|
||||
// make a fetch request
|
||||
const requestTimeoutId = setTimeout(
|
||||
() => controller.abort(),
|
||||
REQUEST_TIMEOUT_MS,
|
||||
getTimeoutMSByModel(options.config.model),
|
||||
);
|
||||
|
||||
if (shouldStream) {
|
||||
|
||||
@@ -750,4 +750,4 @@
|
||||
transform: translateX(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -66,11 +66,11 @@ export function Avatar(props: { model?: ModelType; avatar?: string }) {
|
||||
LlmIcon = BotIconGemma;
|
||||
} else if (modelName.startsWith("claude")) {
|
||||
LlmIcon = BotIconClaude;
|
||||
} else if (modelName.startsWith("llama")) {
|
||||
} else if (modelName.toLowerCase().includes("llama")) {
|
||||
LlmIcon = BotIconMeta;
|
||||
} else if (modelName.startsWith("mixtral")) {
|
||||
LlmIcon = BotIconMistral;
|
||||
} else if (modelName.startsWith("deepseek")) {
|
||||
} else if (modelName.toLowerCase().includes("deepseek")) {
|
||||
LlmIcon = BotIconDeepseek;
|
||||
} else if (modelName.startsWith("moonshot")) {
|
||||
LlmIcon = BotIconMoonshot;
|
||||
@@ -85,7 +85,7 @@ export function Avatar(props: { model?: ModelType; avatar?: string }) {
|
||||
} else if (modelName.startsWith("doubao") || modelName.startsWith("ep-")) {
|
||||
LlmIcon = BotIconDoubao;
|
||||
} else if (
|
||||
modelName.startsWith("glm") ||
|
||||
modelName.toLowerCase().includes("glm") ||
|
||||
modelName.startsWith("cogview-") ||
|
||||
modelName.startsWith("cogvideox-")
|
||||
) {
|
||||
|
||||
@@ -23,7 +23,6 @@ import CopyIcon from "../icons/copy.svg";
|
||||
import LoadingIcon from "../icons/three-dots.svg";
|
||||
import ChatGptIcon from "../icons/chatgpt.png";
|
||||
import ShareIcon from "../icons/share.svg";
|
||||
import BotIcon from "../icons/bot.png";
|
||||
|
||||
import DownloadIcon from "../icons/download.svg";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
@@ -33,13 +32,13 @@ import dynamic from "next/dynamic";
|
||||
import NextImage from "next/image";
|
||||
|
||||
import { toBlob, toPng } from "html-to-image";
|
||||
import { DEFAULT_MASK_AVATAR } from "../store/mask";
|
||||
|
||||
import { prettyObject } from "../utils/format";
|
||||
import { EXPORT_MESSAGE_CLASS_NAME } from "../constant";
|
||||
import { getClientConfig } from "../config/client";
|
||||
import { type ClientApi, getClientApi } from "../client/api";
|
||||
import { getMessageTextContent } from "../utils";
|
||||
import { MaskAvatar } from "./mask";
|
||||
import clsx from "clsx";
|
||||
|
||||
const Markdown = dynamic(async () => (await import("./markdown")).Markdown, {
|
||||
@@ -407,22 +406,6 @@ export function PreviewActions(props: {
|
||||
);
|
||||
}
|
||||
|
||||
function ExportAvatar(props: { avatar: string }) {
|
||||
if (props.avatar === DEFAULT_MASK_AVATAR) {
|
||||
return (
|
||||
<img
|
||||
src={BotIcon.src}
|
||||
width={30}
|
||||
height={30}
|
||||
alt="bot"
|
||||
className="user-avatar"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
return <Avatar avatar={props.avatar} />;
|
||||
}
|
||||
|
||||
export function ImagePreviewer(props: {
|
||||
messages: ChatMessage[];
|
||||
topic: string;
|
||||
@@ -546,9 +529,12 @@ export function ImagePreviewer(props: {
|
||||
github.com/ChatGPTNextWeb/ChatGPT-Next-Web
|
||||
</div>
|
||||
<div className={styles["icons"]}>
|
||||
<ExportAvatar avatar={config.avatar} />
|
||||
<MaskAvatar avatar={config.avatar} />
|
||||
<span className={styles["icon-space"]}>&</span>
|
||||
<ExportAvatar avatar={mask.avatar} />
|
||||
<MaskAvatar
|
||||
avatar={mask.avatar}
|
||||
model={session.mask.modelConfig.model}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
@@ -576,9 +562,14 @@ export function ImagePreviewer(props: {
|
||||
key={i}
|
||||
>
|
||||
<div className={styles["avatar"]}>
|
||||
<ExportAvatar
|
||||
avatar={m.role === "user" ? config.avatar : mask.avatar}
|
||||
/>
|
||||
{m.role === "user" ? (
|
||||
<Avatar avatar={config.avatar}></Avatar>
|
||||
) : (
|
||||
<MaskAvatar
|
||||
avatar={session.mask.avatar}
|
||||
model={m.model || session.mask.modelConfig.model}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className={styles["body"]}>
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { useState, useEffect, useMemo } from "react";
|
||||
|
||||
import styles from "./settings.module.scss";
|
||||
|
||||
import ResetIcon from "../icons/reload.svg";
|
||||
@@ -967,7 +966,89 @@ export function Settings() {
|
||||
</ListItem>
|
||||
</>
|
||||
);
|
||||
|
||||
const bedrockConfigComponent = accessStore.provider ===
|
||||
ServiceProvider.Bedrock && (
|
||||
<>
|
||||
<ListItem
|
||||
title={Locale.Settings.Access.Bedrock.Region.Title}
|
||||
subTitle={Locale.Settings.Access.Bedrock.Region.SubTitle}
|
||||
>
|
||||
<input
|
||||
aria-label={Locale.Settings.Access.Bedrock.Region.Title}
|
||||
type="text"
|
||||
value={accessStore.awsRegion}
|
||||
placeholder="us-west-2"
|
||||
onChange={(e) =>
|
||||
accessStore.update((access) => {
|
||||
const region = e.currentTarget.value;
|
||||
access.awsRegion = region;
|
||||
})
|
||||
}
|
||||
/>
|
||||
</ListItem>
|
||||
<ListItem
|
||||
title={Locale.Settings.Access.Bedrock.AccessKey.Title}
|
||||
subTitle={Locale.Settings.Access.Bedrock.AccessKey.SubTitle}
|
||||
>
|
||||
<PasswordInput
|
||||
aria-label={Locale.Settings.Access.Bedrock.AccessKey.Title}
|
||||
value={accessStore.awsAccessKey}
|
||||
type="text"
|
||||
placeholder={Locale.Settings.Access.Bedrock.AccessKey.Placeholder}
|
||||
onChange={(e) => {
|
||||
accessStore.update((access) => {
|
||||
const accessKey = e.currentTarget.value;
|
||||
access.awsAccessKey = accessKey;
|
||||
});
|
||||
}}
|
||||
maskWhenShow={true}
|
||||
/>
|
||||
</ListItem>
|
||||
<ListItem
|
||||
title={Locale.Settings.Access.Bedrock.SecretKey.Title}
|
||||
subTitle={Locale.Settings.Access.Bedrock.SecretKey.SubTitle}
|
||||
>
|
||||
<PasswordInput
|
||||
aria-label={Locale.Settings.Access.Bedrock.SecretKey.Title}
|
||||
value={accessStore.awsSecretKey}
|
||||
type="text"
|
||||
placeholder={Locale.Settings.Access.Bedrock.SecretKey.Placeholder}
|
||||
onChange={(e) => {
|
||||
accessStore.update((access) => {
|
||||
const secretKey = e.currentTarget.value;
|
||||
access.awsSecretKey = secretKey;
|
||||
});
|
||||
}}
|
||||
maskWhenShow={true}
|
||||
/>
|
||||
</ListItem>
|
||||
<ListItem
|
||||
title={Locale.Settings.Access.Bedrock.EncryptionKey.Title}
|
||||
subTitle={Locale.Settings.Access.Bedrock.EncryptionKey.SubTitle}
|
||||
>
|
||||
<PasswordInput
|
||||
aria-label={Locale.Settings.Access.Bedrock.EncryptionKey.Title}
|
||||
value={accessStore.encryptionKey}
|
||||
type="text"
|
||||
placeholder={Locale.Settings.Access.Bedrock.EncryptionKey.Placeholder}
|
||||
onChange={(e) => {
|
||||
accessStore.update(
|
||||
(access) => (access.encryptionKey = e.currentTarget.value),
|
||||
);
|
||||
}}
|
||||
onBlur={(e) => {
|
||||
const value = e.currentTarget.value;
|
||||
if (!value || value.length < 8) {
|
||||
showToast(Locale.Settings.Access.Bedrock.EncryptionKey.Invalid);
|
||||
accessStore.update((access) => (access.encryptionKey = ""));
|
||||
return;
|
||||
}
|
||||
}}
|
||||
maskWhenShow={true}
|
||||
/>
|
||||
</ListItem>
|
||||
</>
|
||||
);
|
||||
const baiduConfigComponent = accessStore.provider ===
|
||||
ServiceProvider.Baidu && (
|
||||
<>
|
||||
@@ -1808,6 +1889,7 @@ export function Settings() {
|
||||
</ListItem>
|
||||
|
||||
{openAIConfigComponent}
|
||||
{bedrockConfigComponent}
|
||||
{azureConfigComponent}
|
||||
{googleConfigComponent}
|
||||
{anthropicConfigComponent}
|
||||
|
||||
@@ -11,6 +11,7 @@ import MaxIcon from "../icons/max.svg";
|
||||
import MinIcon from "../icons/min.svg";
|
||||
|
||||
import Locale from "../locales";
|
||||
import { maskSensitiveValue } from "../utils/aws";
|
||||
|
||||
import { createRoot } from "react-dom/client";
|
||||
import React, {
|
||||
@@ -270,13 +271,25 @@ export function Input(props: InputProps) {
|
||||
}
|
||||
|
||||
export function PasswordInput(
|
||||
props: HTMLProps<HTMLInputElement> & { aria?: string },
|
||||
props: HTMLProps<HTMLInputElement> & {
|
||||
aria?: string;
|
||||
maskWhenShow?: boolean;
|
||||
},
|
||||
) {
|
||||
const [visible, setVisible] = useState(false);
|
||||
const [isEditing, setIsEditing] = useState(false);
|
||||
const { maskWhenShow, onChange, value, ...inputProps } = props;
|
||||
|
||||
function changeVisibility() {
|
||||
setVisible(!visible);
|
||||
}
|
||||
|
||||
// Get display value - use masked value only when showing and maskWhenShow is true and not editing
|
||||
const displayValue =
|
||||
maskWhenShow && visible && value && !isEditing
|
||||
? maskSensitiveValue(value as string)
|
||||
: value;
|
||||
|
||||
return (
|
||||
<div className={"password-input-container"}>
|
||||
<IconButton
|
||||
@@ -286,7 +299,11 @@ export function PasswordInput(
|
||||
className={"password-eye"}
|
||||
/>
|
||||
<input
|
||||
{...props}
|
||||
{...inputProps}
|
||||
value={displayValue}
|
||||
onChange={onChange}
|
||||
onFocus={() => setIsEditing(true)}
|
||||
onBlur={() => setIsEditing(false)}
|
||||
type={visible ? "text" : "password"}
|
||||
className={"password-input"}
|
||||
/>
|
||||
@@ -552,6 +569,7 @@ export function Selector<T>(props: {
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function FullScreen(props: any) {
|
||||
const { children, right = 10, top = 10, ...rest } = props;
|
||||
const ref = useRef<HTMLDivElement>();
|
||||
|
||||
@@ -13,6 +13,12 @@ declare global {
|
||||
BASE_URL?: string;
|
||||
OPENAI_ORG_ID?: string; // openai only
|
||||
|
||||
// bedrock only
|
||||
AWS_REGION?: string;
|
||||
AWS_ACCESS_KEY?: string;
|
||||
AWS_SECRET_KEY?: string;
|
||||
ENCRYPTION_KEY?: string;
|
||||
|
||||
VERCEL?: string;
|
||||
BUILD_MODE?: "standalone" | "export";
|
||||
BUILD_APP?: string; // is building desktop app
|
||||
@@ -148,7 +154,10 @@ export const getServerSideConfig = () => {
|
||||
}
|
||||
|
||||
const isStability = !!process.env.STABILITY_API_KEY;
|
||||
|
||||
const isBedrock =
|
||||
!!process.env.AWS_REGION &&
|
||||
!!process.env.AWS_ACCESS_KEY &&
|
||||
!!process.env.AWS_SECRET_KEY;
|
||||
const isAzure = !!process.env.AZURE_URL;
|
||||
const isGoogle = !!process.env.GOOGLE_API_KEY;
|
||||
const isAnthropic = !!process.env.ANTHROPIC_API_KEY;
|
||||
@@ -180,6 +189,12 @@ export const getServerSideConfig = () => {
|
||||
apiKey: getApiKey(process.env.OPENAI_API_KEY),
|
||||
openaiOrgId: process.env.OPENAI_ORG_ID,
|
||||
|
||||
isBedrock,
|
||||
awsRegion: process.env.AWS_REGION,
|
||||
awsAccessKey: process.env.AWS_ACCESS_KEY,
|
||||
awsSecretKey: process.env.AWS_SECRET_KEY,
|
||||
encryptionKey: process.env.ENCRYPTION_KEY,
|
||||
|
||||
isStability,
|
||||
stabilityUrl: process.env.STABILITY_URL,
|
||||
stabilityApiKey: getApiKey(process.env.STABILITY_API_KEY),
|
||||
|
||||
@@ -56,6 +56,7 @@ export enum Path {
|
||||
|
||||
export enum ApiPath {
|
||||
Cors = "",
|
||||
Bedrock = "/api/bedrock",
|
||||
Azure = "/api/azure",
|
||||
OpenAI = "/api/openai",
|
||||
Anthropic = "/api/anthropic",
|
||||
@@ -129,6 +130,7 @@ export enum ServiceProvider {
|
||||
XAI = "XAI",
|
||||
ChatGLM = "ChatGLM",
|
||||
DeepSeek = "DeepSeek",
|
||||
Bedrock = "Bedrock",
|
||||
SiliconFlow = "SiliconFlow",
|
||||
}
|
||||
|
||||
@@ -155,6 +157,7 @@ export enum ModelProvider {
|
||||
XAI = "XAI",
|
||||
ChatGLM = "ChatGLM",
|
||||
DeepSeek = "DeepSeek",
|
||||
Bedrock = "Bedrock",
|
||||
SiliconFlow = "SiliconFlow",
|
||||
}
|
||||
|
||||
@@ -255,9 +258,19 @@ export const ChatGLM = {
|
||||
VideoPath: "api/paas/v4/videos/generations",
|
||||
};
|
||||
|
||||
export const Bedrock = {
|
||||
ChatPath: "model", // Simplified path since we'll append the full path in bedrock.ts
|
||||
ApiVersion: "2023-11-01",
|
||||
getEndpoint: (region: string = "us-west-2") =>
|
||||
`https://bedrock-runtime.${region}.amazonaws.com`,
|
||||
};
|
||||
// Get the region from access store for BEDROCK_BASE_URL
|
||||
export const BEDROCK_BASE_URL = Bedrock.getEndpoint();
|
||||
|
||||
export const SiliconFlow = {
|
||||
ExampleEndpoint: SILICONFLOW_BASE_URL,
|
||||
ChatPath: "v1/chat/completions",
|
||||
ListModelPath: "v1/models?&sub_type=chat",
|
||||
};
|
||||
|
||||
export const DEFAULT_INPUT_TEMPLATE = `{{input}}`; // input / time / model / lang
|
||||
@@ -462,6 +475,9 @@ export const VISION_MODEL_REGEXES = [
|
||||
/gpt-4-turbo(?!.*preview)/, // Matches "gpt-4-turbo" but not "gpt-4-turbo-preview"
|
||||
/^dall-e-3$/, // Matches exactly "dall-e-3"
|
||||
/glm-4v/,
|
||||
/nova-lite/,
|
||||
/nova-pro/,
|
||||
/vl/i,
|
||||
];
|
||||
|
||||
export const EXCLUDE_VISION_MODEL_REGEXES = [/claude-3-5-haiku-20241022/];
|
||||
@@ -494,6 +510,28 @@ const openaiModels = [
|
||||
"o3-mini",
|
||||
];
|
||||
|
||||
const bedrockModels = [
|
||||
// Amazon nova Models
|
||||
"us.amazon.nova-micro-v1:0",
|
||||
"us.amazon.nova-lite-v1:0",
|
||||
"us.amazon.nova-pro-v1:0",
|
||||
// Claude Models
|
||||
"anthropic.claude-3-haiku-20240307-v1:0",
|
||||
"anthropic.claude-3-5-haiku-20241022-v1:0",
|
||||
"anthropic.claude-3-sonnet-20240229-v1:0",
|
||||
"anthropic.claude-3-5-sonnet-20241022-v2:0",
|
||||
"anthropic.claude-3-opus-20240229-v1:0",
|
||||
// Meta Llama Models
|
||||
"us.meta.llama3-1-8b-instruct-v1:0",
|
||||
"us.meta.llama3-1-70b-instruct-v1:0",
|
||||
"us.meta.llama3-2-11b-instruct-v1:0",
|
||||
"us.meta.llama3-2-90b-instruct-v1:0",
|
||||
"us.meta.llama3-3-70b-instruct-v1:0",
|
||||
// Mistral Models
|
||||
"mistral.mistral-large-2402-v1:0",
|
||||
"mistral.mistral-large-2407-v1:0",
|
||||
];
|
||||
|
||||
const googleModels = [
|
||||
"gemini-1.0-pro", // Deprecated on 2/15/2025
|
||||
"gemini-1.5-pro-latest",
|
||||
@@ -761,6 +799,7 @@ export const DEFAULT_MODELS = [
|
||||
sorted: 11,
|
||||
},
|
||||
})),
|
||||
|
||||
...chatglmModels.map((name) => ({
|
||||
name,
|
||||
available: true,
|
||||
@@ -794,6 +833,17 @@ export const DEFAULT_MODELS = [
|
||||
sorted: 14,
|
||||
},
|
||||
})),
|
||||
...bedrockModels.map((name) => ({
|
||||
name,
|
||||
available: true,
|
||||
sorted: seq++,
|
||||
provider: {
|
||||
id: "bedrock",
|
||||
providerName: "Bedrock",
|
||||
providerType: "bedrock",
|
||||
sorted: 15,
|
||||
},
|
||||
})),
|
||||
] as const;
|
||||
|
||||
export const CHAT_PAGE_SIZE = 15;
|
||||
@@ -814,5 +864,5 @@ export const internalAllowedWebDavEndpoints = [
|
||||
|
||||
export const DEFAULT_GA_ID = "G-89WN60ZK2E";
|
||||
|
||||
export const SAAS_CHAT_URL = "https://nextchat.dev/chat";
|
||||
export const SAAS_CHAT_UTM_URL = "https://nextchat.dev/chat?utm=github";
|
||||
export const SAAS_CHAT_URL = "https://nextchat.club";
|
||||
export const SAAS_CHAT_UTM_URL = "https://nextchat.club?utm=github";
|
||||
|
||||
@@ -343,6 +343,32 @@ const cn = {
|
||||
SubTitle: "除默认地址外,必须包含 http(s)://",
|
||||
},
|
||||
},
|
||||
Bedrock: {
|
||||
Region: {
|
||||
Title: "AWS 区域",
|
||||
SubTitle: "Bedrock 服务所在的 AWS 区域",
|
||||
Placeholder: "us-west-2",
|
||||
Invalid: "无效的 AWS 区域格式。示例:us-west-2",
|
||||
},
|
||||
AccessKey: {
|
||||
Title: "AWS 访问密钥 ID",
|
||||
SubTitle: "用于 Bedrock 服务的 AWS 访问密钥 ID",
|
||||
Placeholder: "AKIA...",
|
||||
Invalid: "无效的 AWS Access Key 格式。必须为20个字符。",
|
||||
},
|
||||
SecretKey: {
|
||||
Title: "AWS 私有访问密钥",
|
||||
SubTitle: "用于 Bedrock 服务的 AWS 私有访问密钥",
|
||||
Placeholder: "****",
|
||||
Invalid: "无效的 AWS Secret Key 格式。必须为40个字符。",
|
||||
},
|
||||
EncryptionKey: {
|
||||
Title: "加密密钥",
|
||||
SubTitle: "用于配置数据的加密密钥",
|
||||
Placeholder: "输入加密密钥",
|
||||
Invalid: "无效的加密密钥。必须至少包含8个字符!",
|
||||
},
|
||||
},
|
||||
Azure: {
|
||||
ApiKey: {
|
||||
Title: "接口密钥",
|
||||
|
||||
832
app/locales/da.ts
Normal file
832
app/locales/da.ts
Normal file
@@ -0,0 +1,832 @@
|
||||
import { getClientConfig } from "../config/client";
|
||||
import { SubmitKey } from "../store/config";
|
||||
import { SAAS_CHAT_UTM_URL } from "@/app/constant";
|
||||
import { PartialLocaleType } from "./index";
|
||||
|
||||
const isApp = !!getClientConfig()?.isApp;
|
||||
const da: PartialLocaleType = {
|
||||
WIP: "Der kommer snart mere...",
|
||||
Error: {
|
||||
Unauthorized: isApp
|
||||
? `Hov, der skete en fejl. Sådan kan du komme videre:
|
||||
\\ 1️⃣ Er du ny her? [Tryk for at starte nu 🚀](${SAAS_CHAT_UTM_URL})
|
||||
\\ 2️⃣ Vil du bruge dine egne OpenAI-nøgler? [Tryk her](/#/settings) for at ændre indstillinger ⚙️`
|
||||
: `Hov, der skete en fejl. Lad os løse det:
|
||||
\\ 1️⃣ Er du ny her? [Tryk for at starte nu 🚀](${SAAS_CHAT_UTM_URL})
|
||||
\\ 2️⃣ Bruger du en privat opsætning? [Tryk her](/#/auth) for at taste din nøgle 🔑
|
||||
\\ 3️⃣ Vil du bruge dine egne OpenAI-nøgler? [Tryk her](/#/settings) for at ændre indstillinger ⚙️
|
||||
`,
|
||||
},
|
||||
Auth: {
|
||||
Return: "Tilbage",
|
||||
Title: "Adgangskode",
|
||||
Tips: "Skriv venligst koden herunder",
|
||||
SubTips: "Eller brug din egen OpenAI- eller Google-nøgle",
|
||||
Input: "Adgangskode",
|
||||
Confirm: "OK",
|
||||
Later: "Senere",
|
||||
SaasTips: "Hvis det er for svært, kan du starte nu",
|
||||
},
|
||||
ChatItem: {
|
||||
ChatItemCount: (count: number) => `${count} beskeder`,
|
||||
},
|
||||
Chat: {
|
||||
SubTitle: (count: number) => `${count} beskeder`,
|
||||
EditMessage: {
|
||||
Title: "Rediger beskeder",
|
||||
Topic: {
|
||||
Title: "Emne",
|
||||
SubTitle: "Skift emne for denne chat",
|
||||
},
|
||||
},
|
||||
Actions: {
|
||||
ChatList: "Gå til chatliste",
|
||||
CompressedHistory: "Komprimeret historie",
|
||||
Export: "Eksporter alle beskeder som Markdown",
|
||||
Copy: "Kopiér",
|
||||
Stop: "Stop",
|
||||
Retry: "Prøv igen",
|
||||
Pin: "Fastgør",
|
||||
PinToastContent: "1 besked er nu fastgjort",
|
||||
PinToastAction: "Se",
|
||||
Delete: "Slet",
|
||||
Edit: "Rediger",
|
||||
FullScreen: "Fuld skærm",
|
||||
RefreshTitle: "Opdatér titel",
|
||||
RefreshToast: "Anmodning om ny titel sendt",
|
||||
Speech: "Afspil",
|
||||
StopSpeech: "Stop",
|
||||
},
|
||||
Commands: {
|
||||
new: "Ny chat",
|
||||
newm: "Ny chat med persona",
|
||||
next: "Næste chat",
|
||||
prev: "Forrige chat",
|
||||
clear: "Ryd alt før",
|
||||
fork: "Kopiér chat",
|
||||
del: "Slet chat",
|
||||
},
|
||||
InputActions: {
|
||||
Stop: "Stop",
|
||||
ToBottom: "Ned til nyeste",
|
||||
Theme: {
|
||||
auto: "Automatisk",
|
||||
light: "Lyst tema",
|
||||
dark: "Mørkt tema",
|
||||
},
|
||||
Prompt: "Prompts",
|
||||
Masks: "Personaer",
|
||||
Clear: "Ryd kontekst",
|
||||
Settings: "Indstillinger",
|
||||
UploadImage: "Upload billeder",
|
||||
},
|
||||
Rename: "Omdøb chat",
|
||||
Typing: "Skriver…",
|
||||
Input: (submitKey: string) => {
|
||||
let inputHints = `${submitKey} for at sende`;
|
||||
if (submitKey === String(SubmitKey.Enter)) {
|
||||
inputHints += ", Shift + Enter for ny linje";
|
||||
}
|
||||
return (
|
||||
inputHints + ", / for at søge i prompts, : for at bruge kommandoer"
|
||||
);
|
||||
},
|
||||
Send: "Send",
|
||||
StartSpeak: "Start oplæsning",
|
||||
StopSpeak: "Stop oplæsning",
|
||||
Config: {
|
||||
Reset: "Nulstil til standard",
|
||||
SaveAs: "Gem som persona",
|
||||
},
|
||||
IsContext: "Ekstra prompt til baggrund",
|
||||
ShortcutKey: {
|
||||
Title: "Hurtigtaster",
|
||||
newChat: "Åbn ny chat",
|
||||
focusInput: "Fokus på tekstfeltet",
|
||||
copyLastMessage: "Kopiér sidste svar",
|
||||
copyLastCode: "Kopiér sidste kodeblok",
|
||||
showShortcutKey: "Vis hurtigtaster",
|
||||
clearContext: "Ryd kontekst",
|
||||
},
|
||||
},
|
||||
Export: {
|
||||
Title: "Eksportér beskeder",
|
||||
Copy: "Kopiér alt",
|
||||
Download: "Download",
|
||||
MessageFromYou: "Fra dig",
|
||||
MessageFromChatGPT: "Fra ChatGPT",
|
||||
Share: "Del til ShareGPT",
|
||||
Format: {
|
||||
Title: "Filformat",
|
||||
SubTitle: "Vælg enten Markdown eller PNG-billede",
|
||||
},
|
||||
IncludeContext: {
|
||||
Title: "Tag baggrund med",
|
||||
SubTitle: "Skal ekstra baggrund (persona) med i eksporten?",
|
||||
},
|
||||
Steps: {
|
||||
Select: "Vælg",
|
||||
Preview: "Forhåndsvis",
|
||||
},
|
||||
Image: {
|
||||
Toast: "Laver billede...",
|
||||
Modal: "Tryk længe eller højreklik for at gemme",
|
||||
},
|
||||
Artifacts: {
|
||||
Title: "Del side",
|
||||
Error: "Fejl ved deling",
|
||||
},
|
||||
},
|
||||
Select: {
|
||||
Search: "Søg",
|
||||
All: "Vælg alle",
|
||||
Latest: "Vælg nyeste",
|
||||
Clear: "Ryd alt",
|
||||
},
|
||||
Memory: {
|
||||
Title: "Huskesætning",
|
||||
EmptyContent: "Ingenting lige nu.",
|
||||
Send: "Send huskesætning",
|
||||
Copy: "Kopiér huskesætning",
|
||||
Reset: "Nulstil chat",
|
||||
ResetConfirm:
|
||||
"Dette sletter nuværende samtale og hukommelse. Er du sikker?",
|
||||
},
|
||||
Home: {
|
||||
NewChat: "Ny Chat",
|
||||
DeleteChat: "Vil du slette den valgte chat?",
|
||||
DeleteToast: "Chat slettet",
|
||||
Revert: "Fortryd",
|
||||
},
|
||||
Settings: {
|
||||
Title: "Indstillinger",
|
||||
SubTitle: "Alle indstillinger",
|
||||
ShowPassword: "Vis kodeord",
|
||||
Danger: {
|
||||
Reset: {
|
||||
Title: "Nulstil alle indstillinger",
|
||||
SubTitle: "Gendan alt til standard",
|
||||
Action: "Nulstil",
|
||||
Confirm: "Vil du virkelig nulstille alt?",
|
||||
},
|
||||
Clear: {
|
||||
Title: "Slet alle data",
|
||||
SubTitle: "Sletter alt om beskeder og indstillinger",
|
||||
Action: "Slet",
|
||||
Confirm: "Er du sikker på, at du vil slette alt?",
|
||||
},
|
||||
},
|
||||
Lang: {
|
||||
Name: "Language",
|
||||
All: "Alle sprog",
|
||||
},
|
||||
Avatar: "Avatar",
|
||||
FontSize: {
|
||||
Title: "Skriftstørrelse",
|
||||
SubTitle: "Vælg, hvor stor teksten skal være",
|
||||
},
|
||||
FontFamily: {
|
||||
Title: "Skrifttype",
|
||||
SubTitle: "Hvis tom, bruger den standard skrifttype",
|
||||
Placeholder: "Skrifttype-navn",
|
||||
},
|
||||
InjectSystemPrompts: {
|
||||
Title: "Tilføj system-prompt",
|
||||
SubTitle: "Læg altid en ekstra prompt først i anmodninger",
|
||||
},
|
||||
InputTemplate: {
|
||||
Title: "Tekstskabelon",
|
||||
SubTitle: "Den seneste besked placeres i denne skabelon",
|
||||
},
|
||||
Update: {
|
||||
Version: (x: string) => `Version: ${x}`,
|
||||
IsLatest: "Du har nyeste version",
|
||||
CheckUpdate: "Tjek efter opdatering",
|
||||
IsChecking: "Tjekker...",
|
||||
FoundUpdate: (x: string) => `Ny version fundet: ${x}`,
|
||||
GoToUpdate: "Opdatér",
|
||||
Success: "Opdatering lykkedes.",
|
||||
Failed: "Opdatering mislykkedes.",
|
||||
},
|
||||
SendKey: "Tast for send",
|
||||
Theme: "Tema",
|
||||
TightBorder: "Stram kant",
|
||||
SendPreviewBubble: {
|
||||
Title: "Forhåndsvisnings-boble",
|
||||
SubTitle: "Vis tekst, før den sendes",
|
||||
},
|
||||
AutoGenerateTitle: {
|
||||
Title: "Lav titel automatisk",
|
||||
SubTitle: "Foreslå en titel ud fra chatten",
|
||||
},
|
||||
Sync: {
|
||||
CloudState: "Seneste opdatering",
|
||||
NotSyncYet: "Endnu ikke synkroniseret",
|
||||
Success: "Synkronisering lykkedes",
|
||||
Fail: "Synkronisering mislykkedes",
|
||||
Config: {
|
||||
Modal: {
|
||||
Title: "Indstil synk",
|
||||
Check: "Tjek forbindelse",
|
||||
},
|
||||
SyncType: {
|
||||
Title: "Synk-type",
|
||||
SubTitle: "Vælg en synk-tjeneste",
|
||||
},
|
||||
Proxy: {
|
||||
Title: "Aktivér proxy",
|
||||
SubTitle: "Brug proxy for at undgå netværksproblemer",
|
||||
},
|
||||
ProxyUrl: {
|
||||
Title: "Proxy-adresse",
|
||||
SubTitle: "Bruges kun til projektets egen proxy",
|
||||
},
|
||||
WebDav: {
|
||||
Endpoint: "WebDAV-adresse",
|
||||
UserName: "Brugernavn",
|
||||
Password: "Kodeord",
|
||||
},
|
||||
UpStash: {
|
||||
Endpoint: "UpStash Redis REST URL",
|
||||
UserName: "Backup-navn",
|
||||
Password: "UpStash Redis REST Token",
|
||||
},
|
||||
},
|
||||
LocalState: "Lokale data",
|
||||
Overview: (overview: any) =>
|
||||
`${overview.chat} chats, ${overview.message} beskeder, ${overview.prompt} prompts, ${overview.mask} personaer`,
|
||||
ImportFailed: "Import mislykkedes",
|
||||
},
|
||||
Mask: {
|
||||
Splash: {
|
||||
Title: "Persona-forside",
|
||||
SubTitle: "Vis denne side, når du opretter ny chat",
|
||||
},
|
||||
Builtin: {
|
||||
Title: "Skjul indbyggede personaer",
|
||||
SubTitle: "Vis ikke de indbyggede personaer i listen",
|
||||
},
|
||||
},
|
||||
Prompt: {
|
||||
Disable: {
|
||||
Title: "Slå auto-forslag fra",
|
||||
SubTitle: "Tast / for at få forslag",
|
||||
},
|
||||
List: "Prompt-liste",
|
||||
ListCount: (builtin: number, custom: number) =>
|
||||
`${builtin} indbygget, ${custom} brugerdefineret`,
|
||||
Edit: "Rediger",
|
||||
Modal: {
|
||||
Title: "Prompt-liste",
|
||||
Add: "Tilføj",
|
||||
Search: "Søg prompts",
|
||||
},
|
||||
EditModal: {
|
||||
Title: "Rediger prompt",
|
||||
},
|
||||
},
|
||||
HistoryCount: {
|
||||
Title: "Antal beskeder, der følger med",
|
||||
SubTitle: "Hvor mange af de tidligere beskeder, der sendes hver gang",
|
||||
},
|
||||
CompressThreshold: {
|
||||
Title: "Komprimeringsgrænse",
|
||||
SubTitle:
|
||||
"Hvis chatten bliver for lang, vil den komprimeres efter dette antal tegn",
|
||||
},
|
||||
Usage: {
|
||||
Title: "Brug og saldo",
|
||||
SubTitle(used: any, total: any) {
|
||||
return `Du har brugt $${used} i denne måned, og din grænse er $${total}.`;
|
||||
},
|
||||
IsChecking: "Tjekker...",
|
||||
Check: "Tjek igen",
|
||||
NoAccess: "Indtast API-nøgle for at se forbrug",
|
||||
},
|
||||
Access: {
|
||||
AccessCode: {
|
||||
Title: "Adgangskode",
|
||||
SubTitle: "Adgangskontrol er slået til",
|
||||
Placeholder: "Skriv kode her",
|
||||
},
|
||||
CustomEndpoint: {
|
||||
Title: "Brugerdefineret adresse",
|
||||
SubTitle: "Brug Azure eller OpenAI fra egen server",
|
||||
},
|
||||
Provider: {
|
||||
Title: "Model-udbyder",
|
||||
SubTitle: "Vælg Azure eller OpenAI",
|
||||
},
|
||||
OpenAI: {
|
||||
ApiKey: {
|
||||
Title: "OpenAI API-nøgle",
|
||||
SubTitle: "Brug din egen nøgle",
|
||||
Placeholder: "sk-xxx",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "OpenAI Endpoint",
|
||||
SubTitle: "Skal starte med http(s):// eller /api/openai som standard",
|
||||
},
|
||||
},
|
||||
Azure: {
|
||||
ApiKey: {
|
||||
Title: "Azure Api Key",
|
||||
SubTitle: "Hent din nøgle fra Azure-portalen",
|
||||
Placeholder: "Azure Api Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Azure Endpoint",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
ApiVerion: {
|
||||
Title: "Azure Api Version",
|
||||
SubTitle: "Hentet fra Azure-portalen",
|
||||
},
|
||||
},
|
||||
Anthropic: {
|
||||
ApiKey: {
|
||||
Title: "Anthropic API-nøgle",
|
||||
SubTitle: "Brug din egen Anthropic-nøgle",
|
||||
Placeholder: "Anthropic API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Endpoint-adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
ApiVerion: {
|
||||
Title: "API-version (Claude)",
|
||||
SubTitle: "Vælg den ønskede version",
|
||||
},
|
||||
},
|
||||
Baidu: {
|
||||
ApiKey: {
|
||||
Title: "Baidu-nøgle",
|
||||
SubTitle: "Din egen Baidu-nøgle",
|
||||
Placeholder: "Baidu API Key",
|
||||
},
|
||||
SecretKey: {
|
||||
Title: "Baidu hemmelig nøgle",
|
||||
SubTitle: "Din egen hemmelige nøgle fra Baidu",
|
||||
Placeholder: "Baidu Secret Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "Kan ikke ændres, se .env",
|
||||
},
|
||||
},
|
||||
Tencent: {
|
||||
ApiKey: {
|
||||
Title: "Tencent-nøgle",
|
||||
SubTitle: "Din egen nøgle fra Tencent",
|
||||
Placeholder: "Tencent API Key",
|
||||
},
|
||||
SecretKey: {
|
||||
Title: "Tencent hemmelig nøgle",
|
||||
SubTitle: "Din egen hemmelige nøgle fra Tencent",
|
||||
Placeholder: "Tencent Secret Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "Kan ikke ændres, se .env",
|
||||
},
|
||||
},
|
||||
ByteDance: {
|
||||
ApiKey: {
|
||||
Title: "ByteDance-nøgle",
|
||||
SubTitle: "Din egen nøgle til ByteDance",
|
||||
Placeholder: "ByteDance API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
Alibaba: {
|
||||
ApiKey: {
|
||||
Title: "Alibaba-nøgle",
|
||||
SubTitle: "Din egen Alibaba Cloud-nøgle",
|
||||
Placeholder: "Alibaba Cloud API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
Moonshot: {
|
||||
ApiKey: {
|
||||
Title: "Moonshot-nøgle",
|
||||
SubTitle: "Din egen Moonshot-nøgle",
|
||||
Placeholder: "Moonshot API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
DeepSeek: {
|
||||
ApiKey: {
|
||||
Title: "DeepSeek-nøgle",
|
||||
SubTitle: "Din egen DeepSeek-nøgle",
|
||||
Placeholder: "DeepSeek API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
XAI: {
|
||||
ApiKey: {
|
||||
Title: "XAI-nøgle",
|
||||
SubTitle: "Din egen XAI-nøgle",
|
||||
Placeholder: "XAI API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
ChatGLM: {
|
||||
ApiKey: {
|
||||
Title: "ChatGLM-nøgle",
|
||||
SubTitle: "Din egen ChatGLM-nøgle",
|
||||
Placeholder: "ChatGLM API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
SiliconFlow: {
|
||||
ApiKey: {
|
||||
Title: "SiliconFlow-nøgle",
|
||||
SubTitle: "Din egen SiliconFlow-nøgle",
|
||||
Placeholder: "SiliconFlow API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
Stability: {
|
||||
ApiKey: {
|
||||
Title: "Stability-nøgle",
|
||||
SubTitle: "Din egen Stability-nøgle",
|
||||
Placeholder: "Stability API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
Iflytek: {
|
||||
ApiKey: {
|
||||
Title: "Iflytek API Key",
|
||||
SubTitle: "Nøgle fra Iflytek",
|
||||
Placeholder: "Iflytek API Key",
|
||||
},
|
||||
ApiSecret: {
|
||||
Title: "Iflytek hemmelig nøgle",
|
||||
SubTitle: "Hentet fra Iflytek",
|
||||
Placeholder: "Iflytek API Secret",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
},
|
||||
CustomModel: {
|
||||
Title: "Egne modelnavne",
|
||||
SubTitle: "Skriv komma-adskilte navne",
|
||||
},
|
||||
Google: {
|
||||
ApiKey: {
|
||||
Title: "Google-nøgle",
|
||||
SubTitle: "Få din nøgle hos Google AI",
|
||||
Placeholder: "Google AI API Key",
|
||||
},
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "F.eks.: ",
|
||||
},
|
||||
ApiVersion: {
|
||||
Title: "API-version (til gemini-pro)",
|
||||
SubTitle: "Vælg en bestemt version",
|
||||
},
|
||||
GoogleSafetySettings: {
|
||||
Title: "Google sikkerhedsindstillinger",
|
||||
SubTitle: "Vælg et niveau for indholdskontrol",
|
||||
},
|
||||
},
|
||||
},
|
||||
Model: "Model",
|
||||
CompressModel: {
|
||||
Title: "Opsummeringsmodel",
|
||||
SubTitle: "Bruges til at korte historik ned og lave titel",
|
||||
},
|
||||
Temperature: {
|
||||
Title: "Temperatur",
|
||||
SubTitle: "Jo højere tal, jo mere kreativt svar",
|
||||
},
|
||||
TopP: {
|
||||
Title: "Top P",
|
||||
SubTitle: "Skal ikke ændres sammen med temperatur",
|
||||
},
|
||||
MaxTokens: {
|
||||
Title: "Maks. længde",
|
||||
SubTitle: "Hvor mange tokens (ord/stykker tekst) der kan bruges",
|
||||
},
|
||||
PresencePenalty: {
|
||||
Title: "Nye emner",
|
||||
SubTitle: "Jo højere tal, jo mere nyt indhold",
|
||||
},
|
||||
FrequencyPenalty: {
|
||||
Title: "Gentagelsesstraf",
|
||||
SubTitle: "Jo højere tal, jo mindre gentagelse",
|
||||
},
|
||||
TTS: {
|
||||
Enable: {
|
||||
Title: "Tænd for oplæsning (TTS)",
|
||||
SubTitle: "Slå tekst-til-tale til",
|
||||
},
|
||||
Autoplay: {
|
||||
Title: "Automatisk oplæsning",
|
||||
SubTitle: "Laver lyd automatisk, hvis TTS er slået til",
|
||||
},
|
||||
Model: "Model",
|
||||
Voice: {
|
||||
Title: "Stemme",
|
||||
SubTitle: "Hvilken stemme der bruges til lyd",
|
||||
},
|
||||
Speed: {
|
||||
Title: "Hastighed",
|
||||
SubTitle: "Hvor hurtigt der oplæses",
|
||||
},
|
||||
Engine: "TTS-motor",
|
||||
},
|
||||
Realtime: {
|
||||
Enable: {
|
||||
Title: "Live-chat",
|
||||
SubTitle: "Slå live-svar til",
|
||||
},
|
||||
Provider: {
|
||||
Title: "Modeludbyder",
|
||||
SubTitle: "Vælg forskellig udbyder",
|
||||
},
|
||||
Model: {
|
||||
Title: "Model",
|
||||
SubTitle: "Vælg en model",
|
||||
},
|
||||
ApiKey: {
|
||||
Title: "API-nøgle",
|
||||
SubTitle: "Din nøgle",
|
||||
Placeholder: "API-nøgle",
|
||||
},
|
||||
Azure: {
|
||||
Endpoint: {
|
||||
Title: "Adresse",
|
||||
SubTitle: "Endpoint til Azure",
|
||||
},
|
||||
Deployment: {
|
||||
Title: "Udrulningsnavn",
|
||||
SubTitle: "Navn for dit Azure-setup",
|
||||
},
|
||||
},
|
||||
Temperature: {
|
||||
Title: "Temperatur",
|
||||
SubTitle: "Højere tal = mere varierede svar",
|
||||
},
|
||||
},
|
||||
},
|
||||
Store: {
|
||||
DefaultTopic: "Ny samtale",
|
||||
BotHello: "Hej! Hvordan kan jeg hjælpe dig i dag?",
|
||||
Error: "Noget gik galt. Prøv igen senere.",
|
||||
Prompt: {
|
||||
History: (content: string) =>
|
||||
"Her er et kort resume af, hvad vi har snakket om: " + content,
|
||||
Topic:
|
||||
"Find en kort overskrift med 4-5 ord om emnet. Ingen tegnsætning eller anførselstegn.",
|
||||
Summarize:
|
||||
"Skriv et kort resumé (under 200 ord) af vores samtale til senere brug.",
|
||||
},
|
||||
},
|
||||
Copy: {
|
||||
Success: "Kopieret",
|
||||
Failed: "Kunne ikke kopiere. Giv adgang til udklipsholder.",
|
||||
},
|
||||
Download: {
|
||||
Success: "Filen er downloadet.",
|
||||
Failed: "Download fejlede.",
|
||||
},
|
||||
Context: {
|
||||
Toast: (x: any) => `Inkluderer ${x} ekstra prompts`,
|
||||
Edit: "Chatindstillinger",
|
||||
Add: "Tilføj prompt",
|
||||
Clear: "Kontekst ryddet",
|
||||
Revert: "Fortryd",
|
||||
},
|
||||
Discovery: {
|
||||
Name: "Søgning og plugins",
|
||||
},
|
||||
Mcp: {
|
||||
Name: "MCP",
|
||||
},
|
||||
FineTuned: {
|
||||
Sysmessage: "Du er en hjælper, der skal...",
|
||||
},
|
||||
SearchChat: {
|
||||
Name: "Søg",
|
||||
Page: {
|
||||
Title: "Søg i tidligere chats",
|
||||
Search: "Skriv her for at søge",
|
||||
NoResult: "Ingen resultater",
|
||||
NoData: "Ingen data",
|
||||
Loading: "Henter...",
|
||||
SubTitle: (count: number) => `Fandt ${count} resultater`,
|
||||
},
|
||||
Item: {
|
||||
View: "Vis",
|
||||
},
|
||||
},
|
||||
Plugin: {
|
||||
Name: "Plugin",
|
||||
Page: {
|
||||
Title: "Plugins",
|
||||
SubTitle: (count: number) => `${count} plugins`,
|
||||
Search: "Søg plugin",
|
||||
Create: "Opret nyt",
|
||||
Find: "Du kan finde flere plugins på GitHub: ",
|
||||
},
|
||||
Item: {
|
||||
Info: (count: number) => `${count} metode`,
|
||||
View: "Vis",
|
||||
Edit: "Rediger",
|
||||
Delete: "Slet",
|
||||
DeleteConfirm: "Vil du slette?",
|
||||
},
|
||||
Auth: {
|
||||
None: "Ingen",
|
||||
Basic: "Basic",
|
||||
Bearer: "Bearer",
|
||||
Custom: "Tilpasset",
|
||||
CustomHeader: "Parameternavn",
|
||||
Token: "Token",
|
||||
Proxy: "Brug Proxy",
|
||||
ProxyDescription: "Løs CORS-problemer med Proxy",
|
||||
Location: "Sted",
|
||||
LocationHeader: "Header",
|
||||
LocationQuery: "Query",
|
||||
LocationBody: "Body",
|
||||
},
|
||||
EditModal: {
|
||||
Title: (readonly: boolean) =>
|
||||
`Rediger Plugin ${readonly ? "(skrivebeskyttet)" : ""}`,
|
||||
Download: "Download",
|
||||
Auth: "Godkendelsestype",
|
||||
Content: "OpenAPI Schema",
|
||||
Load: "Hent fra URL",
|
||||
Method: "Metode",
|
||||
Error: "Fejl i OpenAPI Schema",
|
||||
},
|
||||
},
|
||||
Mask: {
|
||||
Name: "Persona",
|
||||
Page: {
|
||||
Title: "Prompts som personaer",
|
||||
SubTitle: (count: number) => `${count} skabeloner`,
|
||||
Search: "Søg skabeloner",
|
||||
Create: "Opret ny",
|
||||
},
|
||||
Item: {
|
||||
Info: (count: number) => `${count} prompts`,
|
||||
Chat: "Chat",
|
||||
View: "Vis",
|
||||
Edit: "Rediger",
|
||||
Delete: "Slet",
|
||||
DeleteConfirm: "Vil du slette?",
|
||||
},
|
||||
EditModal: {
|
||||
Title: (readonly: boolean) =>
|
||||
`Rediger skabelon ${readonly ? "(skrivebeskyttet)" : ""}`,
|
||||
Download: "Download",
|
||||
Clone: "Klon",
|
||||
},
|
||||
Config: {
|
||||
Avatar: "Chat-avatar",
|
||||
Name: "Chat-navn",
|
||||
Sync: {
|
||||
Title: "Brug globale indstillinger",
|
||||
SubTitle: "Gældende for denne chat",
|
||||
Confirm: "Erstat nuværende indstillinger med globale?",
|
||||
},
|
||||
HideContext: {
|
||||
Title: "Skjul ekstra prompts",
|
||||
SubTitle: "Vis dem ikke på chat-skærmen",
|
||||
},
|
||||
Artifacts: {
|
||||
Title: "Brug Artefakter",
|
||||
SubTitle: "Gør det muligt at vise HTML-sider",
|
||||
},
|
||||
CodeFold: {
|
||||
Title: "Fold kode sammen",
|
||||
SubTitle: "Luk/åbn lange kodestykker automatisk",
|
||||
},
|
||||
Share: {
|
||||
Title: "Del denne persona",
|
||||
SubTitle: "Få et link til denne skabelon",
|
||||
Action: "Kopiér link",
|
||||
},
|
||||
},
|
||||
},
|
||||
NewChat: {
|
||||
Return: "Tilbage",
|
||||
Skip: "Start straks",
|
||||
Title: "Vælg en persona",
|
||||
SubTitle: "Chat med den persona, du vælger",
|
||||
More: "Se flere",
|
||||
NotShow: "Vis ikke igen",
|
||||
ConfirmNoShow:
|
||||
"Er du sikker på, at du ikke vil se det igen? Du kan altid slå det til under indstillinger.",
|
||||
},
|
||||
UI: {
|
||||
Confirm: "OK",
|
||||
Cancel: "Fortryd",
|
||||
Close: "Luk",
|
||||
Create: "Opret",
|
||||
Edit: "Rediger",
|
||||
Export: "Eksporter",
|
||||
Import: "Importér",
|
||||
Sync: "Synk",
|
||||
Config: "Konfigurer",
|
||||
},
|
||||
Exporter: {
|
||||
Description: {
|
||||
Title: "Kun beskeder efter sidste rydning vises",
|
||||
},
|
||||
Model: "Model",
|
||||
Messages: "Beskeder",
|
||||
Topic: "Emne",
|
||||
Time: "Tid",
|
||||
},
|
||||
URLCommand: {
|
||||
Code: "Så ud til, at der var en kode i linket. Vil du bruge den?",
|
||||
Settings: "Så ud til, at der var indstillinger i linket. Vil du bruge dem?",
|
||||
},
|
||||
SdPanel: {
|
||||
Prompt: "Prompt",
|
||||
NegativePrompt: "Negativ prompt",
|
||||
PleaseInput: (name: string) => `Indtast: ${name}`,
|
||||
AspectRatio: "Billedformat",
|
||||
ImageStyle: "Stil",
|
||||
OutFormat: "Uddataformat",
|
||||
AIModel: "AI-model",
|
||||
ModelVersion: "Version",
|
||||
Submit: "Send",
|
||||
ParamIsRequired: (name: string) => `${name} er krævet`,
|
||||
Styles: {
|
||||
D3Model: "3d-model",
|
||||
AnalogFilm: "analog-film",
|
||||
Anime: "anime",
|
||||
Cinematic: "cinematisk",
|
||||
ComicBook: "tegneserie",
|
||||
DigitalArt: "digital-art",
|
||||
Enhance: "enhance",
|
||||
FantasyArt: "fantasy-art",
|
||||
Isometric: "isometric",
|
||||
LineArt: "line-art",
|
||||
LowPoly: "low-poly",
|
||||
ModelingCompound: "modeling-compound",
|
||||
NeonPunk: "neon-punk",
|
||||
Origami: "origami",
|
||||
Photographic: "fotografisk",
|
||||
PixelArt: "pixel-art",
|
||||
TileTexture: "tile-texture",
|
||||
},
|
||||
},
|
||||
Sd: {
|
||||
SubTitle: (count: number) => `${count} billeder`,
|
||||
Actions: {
|
||||
Params: "Se indstillinger",
|
||||
Copy: "Kopiér prompt",
|
||||
Delete: "Slet",
|
||||
Retry: "Prøv igen",
|
||||
ReturnHome: "Til forsiden",
|
||||
History: "Historik",
|
||||
},
|
||||
EmptyRecord: "Ingen billeder endnu",
|
||||
Status: {
|
||||
Name: "Status",
|
||||
Success: "Ok",
|
||||
Error: "Fejl",
|
||||
Wait: "Venter",
|
||||
Running: "I gang",
|
||||
},
|
||||
Danger: {
|
||||
Delete: "Vil du slette?",
|
||||
},
|
||||
GenerateParams: "Genereringsvalg",
|
||||
Detail: "Detaljer",
|
||||
},
|
||||
};
|
||||
|
||||
export default da;
|
||||
@@ -347,6 +347,33 @@ const en: LocaleType = {
|
||||
SubTitle: "Must start with http(s):// or use /api/openai as default",
|
||||
},
|
||||
},
|
||||
Bedrock: {
|
||||
Region: {
|
||||
Title: "AWS Region",
|
||||
SubTitle: "The AWS region where Bedrock service is located",
|
||||
Placeholder: "us-west-2",
|
||||
Invalid: "Invalid AWS region format. Example: us-west-2",
|
||||
},
|
||||
AccessKey: {
|
||||
Title: "AWS Access Key ID",
|
||||
SubTitle: "Your AWS access key ID for Bedrock service",
|
||||
Placeholder: "AKIA...",
|
||||
Invalid: "Invalid AWS access key format. Must be 20 characters long.",
|
||||
},
|
||||
SecretKey: {
|
||||
Title: "AWS Secret Access Key",
|
||||
SubTitle: "Your AWS secret access key for Bedrock service",
|
||||
Placeholder: "****",
|
||||
Invalid: "Invalid AWS secret key format. Must be 40 characters long.",
|
||||
},
|
||||
EncryptionKey: {
|
||||
Title: "Encryption Key",
|
||||
SubTitle: "Your encryption key for configuration data",
|
||||
Placeholder: "Enter encryption key",
|
||||
Invalid:
|
||||
"Invalid encryption key format. Must no less than 8 characters long!",
|
||||
},
|
||||
},
|
||||
Azure: {
|
||||
ApiKey: {
|
||||
Title: "Azure Api Key",
|
||||
|
||||
@@ -2,6 +2,7 @@ import cn from "./cn";
|
||||
import en from "./en";
|
||||
import pt from "./pt";
|
||||
import tw from "./tw";
|
||||
import da from "./da";
|
||||
import id from "./id";
|
||||
import fr from "./fr";
|
||||
import es from "./es";
|
||||
@@ -30,6 +31,7 @@ const ALL_LANGS = {
|
||||
en,
|
||||
tw,
|
||||
pt,
|
||||
da,
|
||||
jp,
|
||||
ko,
|
||||
id,
|
||||
@@ -56,6 +58,7 @@ export const ALL_LANG_OPTIONS: Record<Lang, string> = {
|
||||
en: "English",
|
||||
pt: "Português",
|
||||
tw: "繁體中文",
|
||||
da: "Dansk",
|
||||
jp: "日本語",
|
||||
ko: "한국어",
|
||||
id: "Indonesia",
|
||||
@@ -141,6 +144,7 @@ export const STT_LANG_MAP: Record<Lang, string> = {
|
||||
en: "en-US",
|
||||
pt: "pt-BR",
|
||||
tw: "zh-TW",
|
||||
da: "da-DK",
|
||||
jp: "ja-JP",
|
||||
ko: "ko-KR",
|
||||
id: "id-ID",
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
DEEPSEEK_BASE_URL,
|
||||
XAI_BASE_URL,
|
||||
CHATGLM_BASE_URL,
|
||||
BEDROCK_BASE_URL,
|
||||
SILICONFLOW_BASE_URL,
|
||||
} from "../constant";
|
||||
import { getHeaders } from "../client/api";
|
||||
@@ -24,36 +25,26 @@ import { createPersistStore } from "../utils/store";
|
||||
import { ensure } from "../utils/clone";
|
||||
import { DEFAULT_CONFIG } from "./config";
|
||||
import { getModelProvider } from "../utils/model";
|
||||
import { encrypt, decrypt } from "../utils/aws";
|
||||
|
||||
let fetchState = 0; // 0 not fetch, 1 fetching, 2 done
|
||||
|
||||
const isApp = getClientConfig()?.buildMode === "export";
|
||||
|
||||
const DEFAULT_OPENAI_URL = isApp ? OPENAI_BASE_URL : ApiPath.OpenAI;
|
||||
|
||||
const DEFAULT_GOOGLE_URL = isApp ? GEMINI_BASE_URL : ApiPath.Google;
|
||||
|
||||
const DEFAULT_ANTHROPIC_URL = isApp ? ANTHROPIC_BASE_URL : ApiPath.Anthropic;
|
||||
|
||||
const DEFAULT_BAIDU_URL = isApp ? BAIDU_BASE_URL : ApiPath.Baidu;
|
||||
|
||||
const DEFAULT_BYTEDANCE_URL = isApp ? BYTEDANCE_BASE_URL : ApiPath.ByteDance;
|
||||
|
||||
const DEFAULT_ALIBABA_URL = isApp ? ALIBABA_BASE_URL : ApiPath.Alibaba;
|
||||
|
||||
const DEFAULT_TENCENT_URL = isApp ? TENCENT_BASE_URL : ApiPath.Tencent;
|
||||
|
||||
const DEFAULT_MOONSHOT_URL = isApp ? MOONSHOT_BASE_URL : ApiPath.Moonshot;
|
||||
|
||||
const DEFAULT_STABILITY_URL = isApp ? STABILITY_BASE_URL : ApiPath.Stability;
|
||||
|
||||
const DEFAULT_IFLYTEK_URL = isApp ? IFLYTEK_BASE_URL : ApiPath.Iflytek;
|
||||
|
||||
const DEFAULT_DEEPSEEK_URL = isApp ? DEEPSEEK_BASE_URL : ApiPath.DeepSeek;
|
||||
|
||||
const DEFAULT_XAI_URL = isApp ? XAI_BASE_URL : ApiPath.XAI;
|
||||
|
||||
const DEFAULT_CHATGLM_URL = isApp ? CHATGLM_BASE_URL : ApiPath.ChatGLM;
|
||||
const DEFAULT_BEDROCK_URL = isApp ? BEDROCK_BASE_URL : ApiPath.Bedrock;
|
||||
|
||||
const DEFAULT_SILICONFLOW_URL = isApp
|
||||
? SILICONFLOW_BASE_URL
|
||||
@@ -128,10 +119,19 @@ const DEFAULT_ACCESS_STATE = {
|
||||
chatglmUrl: DEFAULT_CHATGLM_URL,
|
||||
chatglmApiKey: "",
|
||||
|
||||
// aws bedrock
|
||||
bedrockUrl: DEFAULT_BEDROCK_URL,
|
||||
awsRegion: "",
|
||||
awsAccessKey: "",
|
||||
awsSecretKey: "",
|
||||
encryptionKey: "",
|
||||
bedrockAnthropicVersion: "bedrock-2023-05-31",
|
||||
|
||||
// siliconflow
|
||||
siliconflowUrl: DEFAULT_SILICONFLOW_URL,
|
||||
siliconflowApiKey: "",
|
||||
|
||||
|
||||
// server config
|
||||
needCode: true,
|
||||
hideUserApiKey: false,
|
||||
@@ -148,11 +148,9 @@ const DEFAULT_ACCESS_STATE = {
|
||||
|
||||
export const useAccessStore = createPersistStore(
|
||||
{ ...DEFAULT_ACCESS_STATE },
|
||||
|
||||
(set, get) => ({
|
||||
enabledAccessControl() {
|
||||
this.fetch();
|
||||
|
||||
return get().needCode;
|
||||
},
|
||||
getVisionModels() {
|
||||
@@ -161,7 +159,6 @@ export const useAccessStore = createPersistStore(
|
||||
},
|
||||
edgeVoiceName() {
|
||||
this.fetch();
|
||||
|
||||
return get().edgeTTSVoiceName;
|
||||
},
|
||||
|
||||
@@ -200,6 +197,7 @@ export const useAccessStore = createPersistStore(
|
||||
isValidMoonshot() {
|
||||
return ensure(get(), ["moonshotApiKey"]);
|
||||
},
|
||||
|
||||
isValidIflytek() {
|
||||
return ensure(get(), ["iflytekApiKey"]);
|
||||
},
|
||||
@@ -215,8 +213,19 @@ export const useAccessStore = createPersistStore(
|
||||
return ensure(get(), ["chatglmApiKey"]);
|
||||
},
|
||||
|
||||
|
||||
isValidBedrock() {
|
||||
return ensure(get(), [
|
||||
"awsRegion",
|
||||
"awsAccessKey",
|
||||
"awsSecretKey",
|
||||
"encryptionKey",
|
||||
]);
|
||||
},
|
||||
|
||||
isValidSiliconFlow() {
|
||||
return ensure(get(), ["siliconflowApiKey"]);
|
||||
|
||||
},
|
||||
|
||||
isAuthorized() {
|
||||
@@ -237,11 +246,13 @@ export const useAccessStore = createPersistStore(
|
||||
this.isValidDeepSeek() ||
|
||||
this.isValidXAI() ||
|
||||
this.isValidChatGLM() ||
|
||||
this.isValidBedrock() ||
|
||||
this.isValidSiliconFlow() ||
|
||||
!this.enabledAccessControl() ||
|
||||
(this.enabledAccessControl() && ensure(get(), ["accessCode"]))
|
||||
);
|
||||
},
|
||||
|
||||
fetch() {
|
||||
if (fetchState > 0 || getClientConfig()?.buildMode === "export") return;
|
||||
fetchState = 1;
|
||||
@@ -260,7 +271,6 @@ export const useAccessStore = createPersistStore(
|
||||
DEFAULT_CONFIG.modelConfig.model = model;
|
||||
DEFAULT_CONFIG.modelConfig.providerName = providerName as any;
|
||||
}
|
||||
|
||||
return res;
|
||||
})
|
||||
.then((res: DangerConfig) => {
|
||||
@@ -274,6 +284,43 @@ export const useAccessStore = createPersistStore(
|
||||
fetchState = 2;
|
||||
});
|
||||
},
|
||||
|
||||
// Override the set method to encrypt AWS credentials before storage
|
||||
set: (partial: { [key: string]: any }) => {
|
||||
if (partial.awsAccessKey) {
|
||||
partial.awsAccessKey = encrypt(
|
||||
partial.awsAccessKey,
|
||||
partial.encryptionKey,
|
||||
);
|
||||
}
|
||||
if (partial.awsSecretKey) {
|
||||
partial.awsSecretKey = encrypt(
|
||||
partial.awsSecretKey,
|
||||
partial.encryptionKey,
|
||||
);
|
||||
}
|
||||
if (partial.awsRegion) {
|
||||
partial.awsRegion = encrypt(partial.awsRegion, partial.encryptionKey);
|
||||
}
|
||||
set(partial);
|
||||
},
|
||||
|
||||
// Add getter to decrypt AWS credentials when needed
|
||||
get: () => {
|
||||
const state = get();
|
||||
return {
|
||||
...state,
|
||||
awsRegion: state.awsRegion
|
||||
? decrypt(state.awsRegion, state.encryptionKey)
|
||||
: "",
|
||||
awsAccessKey: state.awsAccessKey
|
||||
? decrypt(state.awsAccessKey, state.encryptionKey)
|
||||
: "",
|
||||
awsSecretKey: state.awsSecretKey
|
||||
? decrypt(state.awsSecretKey, state.encryptionKey)
|
||||
: "",
|
||||
};
|
||||
},
|
||||
}),
|
||||
{
|
||||
name: StoreKey.Access,
|
||||
|
||||
27
app/utils.ts
27
app/utils.ts
@@ -2,7 +2,11 @@ import { useEffect, useState } from "react";
|
||||
import { showToast } from "./components/ui-lib";
|
||||
import Locale from "./locales";
|
||||
import { RequestMessage } from "./client/api";
|
||||
import { ServiceProvider } from "./constant";
|
||||
import {
|
||||
REQUEST_TIMEOUT_MS,
|
||||
REQUEST_TIMEOUT_MS_FOR_THINKING,
|
||||
ServiceProvider,
|
||||
} from "./constant";
|
||||
// import { fetch as tauriFetch, ResponseType } from "@tauri-apps/api/http";
|
||||
import { fetch as tauriStreamFetch } from "./utils/stream";
|
||||
import { VISION_MODEL_REGEXES, EXCLUDE_VISION_MODEL_REGEXES } from "./constant";
|
||||
@@ -292,6 +296,20 @@ export function isDalle3(model: string) {
|
||||
return "dall-e-3" === model;
|
||||
}
|
||||
|
||||
export function getTimeoutMSByModel(model: string) {
|
||||
model = model.toLowerCase();
|
||||
if (
|
||||
model.startsWith("dall-e") ||
|
||||
model.startsWith("dalle") ||
|
||||
model.startsWith("o1") ||
|
||||
model.startsWith("o3") ||
|
||||
model.includes("deepseek-r") ||
|
||||
model.includes("-thinking")
|
||||
)
|
||||
return REQUEST_TIMEOUT_MS_FOR_THINKING;
|
||||
return REQUEST_TIMEOUT_MS;
|
||||
}
|
||||
|
||||
export function getModelSizes(model: string): ModelSize[] {
|
||||
if (isDalle3(model)) {
|
||||
return ["1024x1024", "1792x1024", "1024x1792"];
|
||||
@@ -326,6 +344,13 @@ export function showPlugins(provider: ServiceProvider, model: string) {
|
||||
if (provider == ServiceProvider.Anthropic && !model.includes("claude-2")) {
|
||||
return true;
|
||||
}
|
||||
if (
|
||||
(provider == ServiceProvider.Bedrock && model.includes("claude-3")) ||
|
||||
model.includes("mistral-large") ||
|
||||
model.includes("amazon.nova")
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
if (provider == ServiceProvider.Google && !model.includes("vision")) {
|
||||
return true;
|
||||
}
|
||||
|
||||
696
app/utils/aws.ts
Normal file
696
app/utils/aws.ts
Normal file
@@ -0,0 +1,696 @@
|
||||
// Types and Interfaces
|
||||
export interface BedrockCredentials {
|
||||
region: string;
|
||||
accessKeyId: string;
|
||||
secretAccessKey: string;
|
||||
}
|
||||
|
||||
// Type definitions for better type safety
|
||||
type ParsedEvent = Record<string, any>;
|
||||
type EventResult = ParsedEvent[];
|
||||
|
||||
// Using a dot as separator since it's not used in Base64
|
||||
const SEPARATOR = "~";
|
||||
|
||||
// Unified crypto utilities for both frontend and backend
|
||||
async function generateKey(
|
||||
password: string,
|
||||
salt: Uint8Array,
|
||||
): Promise<CryptoKey> {
|
||||
const enc = new TextEncoder();
|
||||
const keyMaterial = await crypto.subtle.importKey(
|
||||
"raw",
|
||||
enc.encode(password),
|
||||
{ name: "PBKDF2" },
|
||||
false,
|
||||
["deriveBits", "deriveKey"],
|
||||
);
|
||||
|
||||
return crypto.subtle.deriveKey(
|
||||
{
|
||||
name: "PBKDF2",
|
||||
salt,
|
||||
iterations: 100000,
|
||||
hash: "SHA-256",
|
||||
},
|
||||
keyMaterial,
|
||||
{ name: "AES-GCM", length: 256 },
|
||||
false,
|
||||
["encrypt", "decrypt"],
|
||||
);
|
||||
}
|
||||
|
||||
function arrayBufferToBase64(buffer: ArrayBuffer | Uint8Array): string {
|
||||
const bytes = buffer instanceof Uint8Array ? buffer : new Uint8Array(buffer);
|
||||
return btoa(String.fromCharCode(...bytes));
|
||||
}
|
||||
|
||||
function base64ToArrayBuffer(base64: string): Uint8Array {
|
||||
const binaryString = atob(base64);
|
||||
const bytes = new Uint8Array(binaryString.length);
|
||||
for (let i = 0; i < binaryString.length; i++) {
|
||||
bytes[i] = binaryString.charCodeAt(i);
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
export async function encrypt(
|
||||
data: string,
|
||||
encryptionKey: string,
|
||||
): Promise<string> {
|
||||
if (!data) return "";
|
||||
if (!encryptionKey) {
|
||||
throw new Error("Encryption key is required for AWS credential encryption");
|
||||
}
|
||||
|
||||
try {
|
||||
const enc = new TextEncoder();
|
||||
const salt = crypto.getRandomValues(new Uint8Array(16));
|
||||
const iv = crypto.getRandomValues(new Uint8Array(12));
|
||||
const key = await generateKey(encryptionKey, salt);
|
||||
|
||||
const encrypted = await crypto.subtle.encrypt(
|
||||
{
|
||||
name: "AES-GCM",
|
||||
iv,
|
||||
},
|
||||
key,
|
||||
enc.encode(data),
|
||||
);
|
||||
|
||||
// Convert to base64 strings
|
||||
const encryptedBase64 = arrayBufferToBase64(encrypted);
|
||||
const saltBase64 = arrayBufferToBase64(salt);
|
||||
const ivBase64 = arrayBufferToBase64(iv);
|
||||
|
||||
return [saltBase64, ivBase64, encryptedBase64].join(SEPARATOR);
|
||||
} catch (error) {
|
||||
// console.error("[Encryption Error]:", error);
|
||||
throw new Error("Failed to encrypt AWS credentials");
|
||||
}
|
||||
}
|
||||
|
||||
export async function decrypt(
|
||||
encryptedData: string,
|
||||
encryptionKey: string,
|
||||
): Promise<string> {
|
||||
if (!encryptedData) return "";
|
||||
if (!encryptionKey) {
|
||||
throw new Error("Encryption key is required for AWS credential decryption");
|
||||
}
|
||||
|
||||
try {
|
||||
const [saltBase64, ivBase64, cipherBase64] = encryptedData.split(SEPARATOR);
|
||||
|
||||
// Convert base64 strings back to Uint8Arrays
|
||||
const salt = base64ToArrayBuffer(saltBase64);
|
||||
const iv = base64ToArrayBuffer(ivBase64);
|
||||
const cipherData = base64ToArrayBuffer(cipherBase64);
|
||||
|
||||
const key = await generateKey(encryptionKey, salt);
|
||||
|
||||
const decrypted = await crypto.subtle.decrypt(
|
||||
{
|
||||
name: "AES-GCM",
|
||||
iv,
|
||||
},
|
||||
key,
|
||||
cipherData,
|
||||
);
|
||||
|
||||
const dec = new TextDecoder();
|
||||
return dec.decode(decrypted);
|
||||
} catch (error) {
|
||||
throw new Error("Failed to decrypt AWS credentials");
|
||||
}
|
||||
}
|
||||
|
||||
export function maskSensitiveValue(value: string): string {
|
||||
if (!value) return "";
|
||||
if (value.length <= 6) return value;
|
||||
const masked = "*".repeat(value.length - 6);
|
||||
return value.slice(0, 3) + masked + value.slice(-3);
|
||||
}
|
||||
|
||||
// AWS Signing
|
||||
export interface SignParams {
|
||||
method: string;
|
||||
url: string;
|
||||
region: string;
|
||||
accessKeyId: string;
|
||||
secretAccessKey: string;
|
||||
body: string | object;
|
||||
service: string;
|
||||
headers?: Record<string, string>;
|
||||
isStreaming?: boolean;
|
||||
}
|
||||
|
||||
async function createHmac(
|
||||
key: ArrayBuffer | Uint8Array,
|
||||
data: string,
|
||||
): Promise<ArrayBuffer> {
|
||||
const encoder = new TextEncoder();
|
||||
const keyData = key instanceof Uint8Array ? key : new Uint8Array(key);
|
||||
const keyObject = await crypto.subtle.importKey(
|
||||
"raw",
|
||||
keyData,
|
||||
{ name: "HMAC", hash: "SHA-256" },
|
||||
false,
|
||||
["sign"],
|
||||
);
|
||||
return crypto.subtle.sign("HMAC", keyObject, encoder.encode(data));
|
||||
}
|
||||
|
||||
async function getSigningKey(
|
||||
secretKey: string,
|
||||
dateStamp: string,
|
||||
region: string,
|
||||
service: string,
|
||||
): Promise<ArrayBuffer> {
|
||||
const encoder = new TextEncoder();
|
||||
const kDate = await createHmac(encoder.encode("AWS4" + secretKey), dateStamp);
|
||||
const kRegion = await createHmac(kDate, region);
|
||||
const kService = await createHmac(kRegion, service);
|
||||
const kSigning = await createHmac(kService, "aws4_request");
|
||||
return kSigning;
|
||||
}
|
||||
|
||||
function normalizeHeaderValue(value: string): string {
|
||||
return value.replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
function encodeRFC3986(str: string): string {
|
||||
return encodeURIComponent(str)
|
||||
.replace(
|
||||
/[!'()*]/g,
|
||||
(c) => "%" + c.charCodeAt(0).toString(16).toUpperCase(),
|
||||
)
|
||||
.replace(/[-_.~]/g, (c) => c);
|
||||
}
|
||||
|
||||
function getCanonicalUri(path: string): string {
|
||||
if (!path || path === "/") return "/";
|
||||
|
||||
return (
|
||||
"/" +
|
||||
path
|
||||
.split("/")
|
||||
.map((segment) => {
|
||||
if (!segment) return "";
|
||||
if (segment === "invoke-with-response-stream") return segment;
|
||||
|
||||
if (segment.includes("model/")) {
|
||||
return segment
|
||||
.split(/(model\/)/)
|
||||
.map((part) => {
|
||||
if (part === "model/") return part;
|
||||
return part
|
||||
.split(/([.:])/g)
|
||||
.map((subpart, i) =>
|
||||
i % 2 === 1 ? subpart : encodeRFC3986(subpart),
|
||||
)
|
||||
.join("");
|
||||
})
|
||||
.join("");
|
||||
}
|
||||
|
||||
return encodeRFC3986(segment);
|
||||
})
|
||||
.join("/")
|
||||
);
|
||||
}
|
||||
|
||||
export async function sign({
|
||||
method,
|
||||
url,
|
||||
region,
|
||||
accessKeyId,
|
||||
secretAccessKey,
|
||||
body,
|
||||
service,
|
||||
headers: customHeaders = {},
|
||||
isStreaming = true,
|
||||
}: SignParams): Promise<Record<string, string>> {
|
||||
try {
|
||||
const endpoint = new URL(url);
|
||||
const canonicalUri = getCanonicalUri(endpoint.pathname.slice(1));
|
||||
const canonicalQueryString = endpoint.search.slice(1);
|
||||
|
||||
const now = new Date();
|
||||
const amzDate = now.toISOString().replace(/[:-]|\.\d{3}/g, "");
|
||||
const dateStamp = amzDate.slice(0, 8);
|
||||
|
||||
const bodyString = typeof body === "string" ? body : JSON.stringify(body);
|
||||
const encoder = new TextEncoder();
|
||||
const payloadBuffer = await crypto.subtle.digest(
|
||||
"SHA-256",
|
||||
encoder.encode(bodyString),
|
||||
);
|
||||
const payloadHash = Array.from(new Uint8Array(payloadBuffer))
|
||||
.map((b) => b.toString(16).padStart(2, "0"))
|
||||
.join("");
|
||||
|
||||
const headers: Record<string, string> = {
|
||||
accept: isStreaming
|
||||
? "application/vnd.amazon.eventstream"
|
||||
: "application/json",
|
||||
"content-type": "application/json",
|
||||
host: endpoint.host,
|
||||
"x-amz-content-sha256": payloadHash,
|
||||
"x-amz-date": amzDate,
|
||||
...customHeaders,
|
||||
};
|
||||
|
||||
// Add x-amzn-bedrock-accept header for streaming requests
|
||||
if (isStreaming) {
|
||||
headers["x-amzn-bedrock-accept"] = "*/*";
|
||||
}
|
||||
|
||||
const sortedHeaderKeys = Object.keys(headers).sort((a, b) =>
|
||||
a.toLowerCase().localeCompare(b.toLowerCase()),
|
||||
);
|
||||
|
||||
const canonicalHeaders = sortedHeaderKeys
|
||||
.map(
|
||||
(key) => `${key.toLowerCase()}:${normalizeHeaderValue(headers[key])}\n`,
|
||||
)
|
||||
.join("");
|
||||
|
||||
const signedHeaders = sortedHeaderKeys
|
||||
.map((key) => key.toLowerCase())
|
||||
.join(";");
|
||||
|
||||
const canonicalRequest = [
|
||||
method.toUpperCase(),
|
||||
canonicalUri,
|
||||
canonicalQueryString,
|
||||
canonicalHeaders,
|
||||
signedHeaders,
|
||||
payloadHash,
|
||||
].join("\n");
|
||||
|
||||
const algorithm = "AWS4-HMAC-SHA256";
|
||||
const credentialScope = `${dateStamp}/${region}/${service}/aws4_request`;
|
||||
|
||||
const canonicalRequestHash = Array.from(
|
||||
new Uint8Array(
|
||||
await crypto.subtle.digest("SHA-256", encoder.encode(canonicalRequest)),
|
||||
),
|
||||
)
|
||||
.map((b) => b.toString(16).padStart(2, "0"))
|
||||
.join("");
|
||||
|
||||
const stringToSign = [
|
||||
algorithm,
|
||||
amzDate,
|
||||
credentialScope,
|
||||
canonicalRequestHash,
|
||||
].join("\n");
|
||||
|
||||
const signingKey = await getSigningKey(
|
||||
secretAccessKey,
|
||||
dateStamp,
|
||||
region,
|
||||
service,
|
||||
);
|
||||
|
||||
const signature = Array.from(
|
||||
new Uint8Array(await createHmac(signingKey, stringToSign)),
|
||||
)
|
||||
.map((b) => b.toString(16).padStart(2, "0"))
|
||||
.join("");
|
||||
|
||||
const authorization = [
|
||||
`${algorithm} Credential=${accessKeyId}/${credentialScope}`,
|
||||
`SignedHeaders=${signedHeaders}`,
|
||||
`Signature=${signature}`,
|
||||
].join(", ");
|
||||
|
||||
return {
|
||||
...headers,
|
||||
Authorization: authorization,
|
||||
};
|
||||
} catch (error) {
|
||||
console.error("[AWS Signing Error]: Failed to sign request");
|
||||
throw new Error("Failed to sign AWS request");
|
||||
}
|
||||
}
|
||||
|
||||
// Bedrock utilities
|
||||
function decodeBase64(base64String: string): string {
|
||||
try {
|
||||
const bytes = Buffer.from(base64String, "base64");
|
||||
const decoder = new TextDecoder("utf-8");
|
||||
return decoder.decode(bytes);
|
||||
} catch (e) {
|
||||
console.error("[Base64 Decode Error]:", e);
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
export function parseEventData(chunk: Uint8Array): EventResult {
|
||||
const decoder = new TextDecoder("utf-8");
|
||||
const text = decoder.decode(chunk);
|
||||
const results: EventResult = [];
|
||||
|
||||
try {
|
||||
// First try to parse as regular JSON
|
||||
const parsed = JSON.parse(text);
|
||||
if (parsed.bytes) {
|
||||
const decoded = decodeBase64(parsed.bytes);
|
||||
try {
|
||||
const decodedJson = JSON.parse(decoded);
|
||||
results.push(decodedJson);
|
||||
} catch (e) {
|
||||
results.push({ output: decoded });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
if (typeof parsed.body === "string") {
|
||||
try {
|
||||
const parsedBody = JSON.parse(parsed.body);
|
||||
results.push(parsedBody);
|
||||
} catch (e) {
|
||||
results.push({ output: parsed.body });
|
||||
}
|
||||
return results;
|
||||
}
|
||||
|
||||
results.push(parsed.body || parsed);
|
||||
return results;
|
||||
} catch (e) {
|
||||
// If regular JSON parse fails, try to extract event content
|
||||
const eventRegex = /:event-type[^\{]+(\{[^\}]+\})/g;
|
||||
let match;
|
||||
|
||||
while ((match = eventRegex.exec(text)) !== null) {
|
||||
try {
|
||||
const eventData = match[1];
|
||||
const parsed = JSON.parse(eventData);
|
||||
|
||||
if (parsed.bytes) {
|
||||
const decoded = decodeBase64(parsed.bytes);
|
||||
try {
|
||||
const decodedJson = JSON.parse(decoded);
|
||||
if (decodedJson.choices?.[0]?.message?.content) {
|
||||
results.push({ output: decodedJson.choices[0].message.content });
|
||||
} else {
|
||||
results.push(decodedJson);
|
||||
}
|
||||
} catch (e) {
|
||||
results.push({ output: decoded });
|
||||
}
|
||||
} else {
|
||||
results.push(parsed);
|
||||
}
|
||||
} catch (e) {
|
||||
console.debug("[Event Parse Warning]:", e);
|
||||
}
|
||||
}
|
||||
|
||||
// If no events were found, try to extract clean text
|
||||
if (results.length === 0) {
|
||||
// Remove event metadata markers and clean the text
|
||||
const cleanText = text
|
||||
.replace(/\{KG[^:]+:event-type[^}]+\}/g, "") // Remove event markers
|
||||
.replace(/[\x00-\x1F\x7F-\x9F\uFEFF]/g, "") // Remove control characters
|
||||
.trim();
|
||||
|
||||
if (cleanText) {
|
||||
results.push({ output: cleanText });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
export function processMessage(
|
||||
data: ParsedEvent,
|
||||
remainText: string,
|
||||
runTools: any[],
|
||||
index: number,
|
||||
): { remainText: string; index: number } {
|
||||
if (!data) return { remainText, index };
|
||||
|
||||
try {
|
||||
// Handle Nova's tool calls with exact schema match
|
||||
// console.log("processMessage data=========================",data);
|
||||
if (data.contentBlockStart?.start?.toolUse) {
|
||||
const toolUse = data.contentBlockStart.start.toolUse;
|
||||
index += 1;
|
||||
runTools.push({
|
||||
id: toolUse.toolUseId,
|
||||
type: "function",
|
||||
function: {
|
||||
name: toolUse.name || "", // Ensure name is always present
|
||||
arguments: "{}", // Initialize empty arguments
|
||||
},
|
||||
});
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's tool input in contentBlockDelta
|
||||
if (data.contentBlockDelta?.delta?.toolUse?.input) {
|
||||
if (runTools[index]) {
|
||||
runTools[index].function.arguments =
|
||||
data.contentBlockDelta.delta.toolUse.input;
|
||||
}
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's text content
|
||||
if (data.output?.message?.content?.[0]?.text) {
|
||||
remainText += data.output.message.content[0].text;
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's messageStart event
|
||||
if (data.messageStart) {
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's text delta
|
||||
if (data.contentBlockDelta?.delta?.text) {
|
||||
remainText += data.contentBlockDelta.delta.text;
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's contentBlockStop event
|
||||
if (data.contentBlockStop) {
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle Nova's messageStop event
|
||||
if (data.messageStop) {
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle message_start event (for other models)
|
||||
if (data.type === "message_start") {
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle content_block_start event (for other models)
|
||||
if (data.type === "content_block_start") {
|
||||
if (data.content_block?.type === "tool_use") {
|
||||
index += 1;
|
||||
runTools.push({
|
||||
id: data.content_block.id,
|
||||
type: "function",
|
||||
function: {
|
||||
name: data.content_block.name || "", // Ensure name is always present
|
||||
arguments: "",
|
||||
},
|
||||
});
|
||||
}
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle content_block_delta event (for other models)
|
||||
if (data.type === "content_block_delta") {
|
||||
if (data.delta?.type === "input_json_delta" && runTools[index]) {
|
||||
runTools[index].function.arguments += data.delta.partial_json;
|
||||
} else if (data.delta?.type === "text_delta") {
|
||||
const newText = data.delta.text || "";
|
||||
remainText += newText;
|
||||
}
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle tool calls for other models
|
||||
if (data.choices?.[0]?.message?.tool_calls) {
|
||||
for (const toolCall of data.choices[0].message.tool_calls) {
|
||||
index += 1;
|
||||
runTools.push({
|
||||
id: toolCall.id || `tool-${Date.now()}`,
|
||||
type: "function",
|
||||
function: {
|
||||
name: toolCall.function?.name || "", // Ensure name is always present
|
||||
arguments: toolCall.function?.arguments || "",
|
||||
},
|
||||
});
|
||||
}
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
// Handle various response formats
|
||||
let newText = "";
|
||||
if (data.delta?.text) {
|
||||
newText = data.delta.text;
|
||||
} else if (data.choices?.[0]?.message?.content) {
|
||||
newText = data.choices[0].message.content;
|
||||
} else if (data.content?.[0]?.text) {
|
||||
newText = data.content[0].text;
|
||||
} else if (data.generation) {
|
||||
newText = data.generation;
|
||||
} else if (data.outputText) {
|
||||
newText = data.outputText;
|
||||
} else if (data.response) {
|
||||
newText = data.response;
|
||||
} else if (data.output) {
|
||||
newText = data.output;
|
||||
}
|
||||
|
||||
// Only append if we have new text
|
||||
if (newText) {
|
||||
remainText += newText;
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn("Failed to process Bedrock message:");
|
||||
}
|
||||
|
||||
return { remainText, index };
|
||||
}
|
||||
|
||||
export function processChunks(
|
||||
chunks: Uint8Array[],
|
||||
pendingChunk: Uint8Array | null,
|
||||
remainText: string,
|
||||
runTools: any[],
|
||||
index: number,
|
||||
): {
|
||||
chunks: Uint8Array[];
|
||||
pendingChunk: Uint8Array | null;
|
||||
remainText: string;
|
||||
index: number;
|
||||
} {
|
||||
let currentText = remainText;
|
||||
let currentIndex = index;
|
||||
|
||||
while (chunks.length > 0) {
|
||||
const chunk = chunks[0];
|
||||
try {
|
||||
// If there's a pending chunk, try to merge it with the current chunk
|
||||
let chunkToProcess = chunk;
|
||||
if (pendingChunk) {
|
||||
const mergedChunk = new Uint8Array(pendingChunk.length + chunk.length);
|
||||
mergedChunk.set(pendingChunk);
|
||||
mergedChunk.set(chunk, pendingChunk.length);
|
||||
chunkToProcess = mergedChunk;
|
||||
pendingChunk = null;
|
||||
}
|
||||
|
||||
// Try to process the chunk
|
||||
const parsedEvents = parseEventData(chunkToProcess);
|
||||
if (parsedEvents.length > 0) {
|
||||
// Process each event in the chunk
|
||||
for (const parsed of parsedEvents) {
|
||||
const result = processMessage(
|
||||
parsed,
|
||||
currentText,
|
||||
runTools,
|
||||
currentIndex,
|
||||
);
|
||||
currentText = result.remainText;
|
||||
currentIndex = result.index;
|
||||
}
|
||||
chunks.shift(); // Remove processed chunk
|
||||
} else {
|
||||
// If parsing fails, it might be an incomplete chunk
|
||||
pendingChunk = chunkToProcess;
|
||||
chunks.shift();
|
||||
}
|
||||
} catch (e) {
|
||||
// console.error("[Chunk Process Error]:", e);
|
||||
// chunks.shift(); // Remove error chunk
|
||||
// pendingChunk = null; // Reset pending chunk on error
|
||||
console.warn("Failed to process chunk, attempting recovery");
|
||||
// Attempt to recover by processing the next chunk
|
||||
if (chunks.length > 1) {
|
||||
chunks.shift();
|
||||
pendingChunk = null;
|
||||
} else {
|
||||
// If this is the last chunk, throw to prevent data loss
|
||||
throw new Error("Failed to process final chunk");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
chunks,
|
||||
pendingChunk,
|
||||
remainText: currentText,
|
||||
index: currentIndex,
|
||||
};
|
||||
}
|
||||
|
||||
export function getBedrockEndpoint(
|
||||
region: string,
|
||||
modelId: string,
|
||||
shouldStream: boolean,
|
||||
): string {
|
||||
if (!region || !modelId) {
|
||||
throw new Error("Region and model ID are required for Bedrock endpoint");
|
||||
}
|
||||
const baseEndpoint = `https://bedrock-runtime.${region}.amazonaws.com`;
|
||||
const endpoint =
|
||||
shouldStream === false
|
||||
? `${baseEndpoint}/model/${modelId}/invoke`
|
||||
: `${baseEndpoint}/model/${modelId}/invoke-with-response-stream`;
|
||||
return endpoint;
|
||||
}
|
||||
|
||||
export function extractMessage(res: any, modelId: string = ""): string {
|
||||
if (!res) {
|
||||
throw new Error("Empty response received");
|
||||
}
|
||||
|
||||
let message = "";
|
||||
|
||||
// Handle Nova model response format
|
||||
if (modelId.toLowerCase().includes("nova")) {
|
||||
if (res.output?.message?.content?.[0]?.text) {
|
||||
message = res.output.message.content[0].text;
|
||||
} else {
|
||||
message = res.output || "";
|
||||
}
|
||||
}
|
||||
// Handle Mistral model response format
|
||||
else if (modelId.toLowerCase().includes("mistral")) {
|
||||
if (res.choices?.[0]?.message?.content) {
|
||||
message = res.choices[0].message.content;
|
||||
} else {
|
||||
message = res.output || "";
|
||||
}
|
||||
}
|
||||
// Handle Llama model response format
|
||||
else if (modelId.toLowerCase().includes("llama")) {
|
||||
message = res?.generation || "";
|
||||
}
|
||||
// Handle Titan model response format
|
||||
else if (modelId.toLowerCase().includes("titan")) {
|
||||
message = res?.outputText || "";
|
||||
}
|
||||
// Handle Claude and other models
|
||||
else if (res.content?.[0]?.text) {
|
||||
message = res.content[0].text;
|
||||
}
|
||||
// Handle other response formats
|
||||
else {
|
||||
message = res.output || res.response || res.message || "";
|
||||
}
|
||||
|
||||
return message;
|
||||
}
|
||||
@@ -400,6 +400,7 @@ export function streamWithThink(
|
||||
let responseRes: Response;
|
||||
let isInThinkingMode = false;
|
||||
let lastIsThinking = false;
|
||||
let lastIsThinkingTagged = false; //between <think> and </think> tags
|
||||
|
||||
// animate response to make it looks smooth
|
||||
function animateResponseText() {
|
||||
@@ -579,6 +580,23 @@ export function streamWithThink(
|
||||
if (!chunk?.content || chunk.content.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
// deal with <think> and </think> tags start
|
||||
if (!chunk.isThinking) {
|
||||
if (chunk.content.startsWith("<think>")) {
|
||||
chunk.isThinking = true;
|
||||
chunk.content = chunk.content.slice(7).trim();
|
||||
lastIsThinkingTagged = true;
|
||||
} else if (chunk.content.endsWith("</think>")) {
|
||||
chunk.isThinking = false;
|
||||
chunk.content = chunk.content.slice(0, -8).trim();
|
||||
lastIsThinkingTagged = false;
|
||||
} else if (lastIsThinkingTagged) {
|
||||
chunk.isThinking = true;
|
||||
}
|
||||
}
|
||||
// deal with <think> and </think> tags start
|
||||
|
||||
// Check if thinking mode changed
|
||||
const isThinkingChanged = lastIsThinking !== chunk.isThinking;
|
||||
lastIsThinking = chunk.isThinking;
|
||||
|
||||
258
docs/bedrock-response-format.md
Normal file
258
docs/bedrock-response-format.md
Normal file
@@ -0,0 +1,258 @@
|
||||
# Understanding Bedrock Response Format
|
||||
|
||||
The AWS Bedrock streaming response format consists of multiple Server-Sent Events (SSE) chunks. Each chunk follows this structure:
|
||||
|
||||
```
|
||||
:event-type chunk
|
||||
:content-type application/json
|
||||
:message-type event
|
||||
{"bytes":"base64_encoded_data","p":"signature"}
|
||||
```
|
||||
|
||||
## Model-Specific Response Formats
|
||||
|
||||
### Claude 3 Format
|
||||
|
||||
When using Claude 3 models (e.g., claude-3-haiku-20240307), the decoded messages include:
|
||||
|
||||
1. **message_start**
|
||||
```json
|
||||
{
|
||||
"type": "message_start",
|
||||
"message": {
|
||||
"id": "msg_bdrk_01A6sahWac4XVTR9sX3rgvsZ",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"model": "claude-3-haiku-20240307",
|
||||
"content": [],
|
||||
"stop_reason": null,
|
||||
"stop_sequence": null,
|
||||
"usage": {
|
||||
"input_tokens": 8,
|
||||
"output_tokens": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **content_block_start**
|
||||
```json
|
||||
{
|
||||
"type": "content_block_start",
|
||||
"index": 0,
|
||||
"content_block": {
|
||||
"type": "text",
|
||||
"text": ""
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **content_block_delta**
|
||||
```json
|
||||
{
|
||||
"type": "content_block_delta",
|
||||
"index": 0,
|
||||
"delta": {
|
||||
"type": "text_delta",
|
||||
"text": "Hello"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Mistral Format
|
||||
|
||||
When using Mistral models (e.g., mistral-large-2407), the decoded messages have a different structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "b0098812-0ad9-42da-9f17-a5e2f554eb6b",
|
||||
"object": "chat.completion.chunk",
|
||||
"created": 1732582566,
|
||||
"model": "mistral-large-2407",
|
||||
"choices": [{
|
||||
"index": 0,
|
||||
"logprobs": null,
|
||||
"context_logits": null,
|
||||
"generation_logits": null,
|
||||
"message": {
|
||||
"role": null,
|
||||
"content": "Hello",
|
||||
"tool_calls": null,
|
||||
"index": null,
|
||||
"tool_call_id": null
|
||||
},
|
||||
"stop_reason": null
|
||||
}],
|
||||
"usage": null,
|
||||
"p": null
|
||||
}
|
||||
```
|
||||
|
||||
### Llama Format
|
||||
|
||||
When using Llama models (3.1 or 3.2), the decoded messages use a simpler structure focused on generation tokens:
|
||||
|
||||
```json
|
||||
{
|
||||
"generation": "Hello",
|
||||
"prompt_token_count": null,
|
||||
"generation_token_count": 2,
|
||||
"stop_reason": null
|
||||
}
|
||||
```
|
||||
|
||||
Each chunk contains:
|
||||
- generation: The generated text piece
|
||||
- prompt_token_count: Token count of the input (only present in first chunk)
|
||||
- generation_token_count: Running count of generated tokens
|
||||
- stop_reason: Indicates completion (null until final chunk)
|
||||
|
||||
First chunk example (includes prompt_token_count):
|
||||
```json
|
||||
{
|
||||
"generation": "\n\n",
|
||||
"prompt_token_count": 10,
|
||||
"generation_token_count": 1,
|
||||
"stop_reason": null
|
||||
}
|
||||
```
|
||||
|
||||
### Titan Text Format
|
||||
|
||||
When using Amazon's Titan models (text or TG1), the response comes as a single chunk with complete text and metrics:
|
||||
|
||||
```json
|
||||
{
|
||||
"outputText": "\nBot: Hello! How can I help you today?",
|
||||
"index": 0,
|
||||
"totalOutputTextTokenCount": 13,
|
||||
"completionReason": "FINISH",
|
||||
"inputTextTokenCount": 3,
|
||||
"amazon-bedrock-invocationMetrics": {
|
||||
"inputTokenCount": 3,
|
||||
"outputTokenCount": 13,
|
||||
"invocationLatency": 833,
|
||||
"firstByteLatency": 833
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Both Titan text and Titan TG1 use the same response format, with only minor differences in token counts and latency values. For example, here's a TG1 response:
|
||||
|
||||
```json
|
||||
{
|
||||
"outputText": "\nBot: Hello! How can I help you?",
|
||||
"index": 0,
|
||||
"totalOutputTextTokenCount": 12,
|
||||
"completionReason": "FINISH",
|
||||
"inputTextTokenCount": 3,
|
||||
"amazon-bedrock-invocationMetrics": {
|
||||
"inputTokenCount": 3,
|
||||
"outputTokenCount": 12,
|
||||
"invocationLatency": 845,
|
||||
"firstByteLatency": 845
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key fields:
|
||||
- outputText: The complete generated response
|
||||
- totalOutputTextTokenCount: Total tokens in the response
|
||||
- completionReason: Reason for completion (e.g., "FINISH")
|
||||
- inputTextTokenCount: Number of input tokens
|
||||
- amazon-bedrock-invocationMetrics: Detailed performance metrics
|
||||
|
||||
## Model-Specific Completion Metrics
|
||||
|
||||
### Mistral
|
||||
```json
|
||||
{
|
||||
"usage": {
|
||||
"prompt_tokens": 5,
|
||||
"total_tokens": 29,
|
||||
"completion_tokens": 24
|
||||
},
|
||||
"amazon-bedrock-invocationMetrics": {
|
||||
"inputTokenCount": 5,
|
||||
"outputTokenCount": 24,
|
||||
"invocationLatency": 719,
|
||||
"firstByteLatency": 148
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Claude 3
|
||||
Included in the message_delta with stop_reason.
|
||||
|
||||
### Llama
|
||||
Included in the final chunk with stop_reason "stop":
|
||||
```json
|
||||
{
|
||||
"amazon-bedrock-invocationMetrics": {
|
||||
"inputTokenCount": 10,
|
||||
"outputTokenCount": 11,
|
||||
"invocationLatency": 873,
|
||||
"firstByteLatency": 550
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Titan
|
||||
Both Titan text and TG1 include metrics in the single response chunk:
|
||||
```json
|
||||
{
|
||||
"amazon-bedrock-invocationMetrics": {
|
||||
"inputTokenCount": 3,
|
||||
"outputTokenCount": 12,
|
||||
"invocationLatency": 845,
|
||||
"firstByteLatency": 845
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## How the Response is Processed
|
||||
|
||||
1. The raw response is first split into chunks based on SSE format
|
||||
2. For each chunk:
|
||||
- The base64 encoded data is decoded
|
||||
- The JSON is parsed to extract the message content
|
||||
- Based on the model type and message type, different processing is applied:
|
||||
|
||||
### Claude 3 Processing
|
||||
- message_start: Initializes a new message with model info and usage stats
|
||||
- content_block_start: Starts a new content block (text, tool use, etc.)
|
||||
- content_block_delta: Adds incremental content to the current block
|
||||
- message_delta: Updates message metadata
|
||||
|
||||
### Mistral Processing
|
||||
- Each chunk contains a complete message object with choices array
|
||||
- The content is streamed through the message.content field
|
||||
- Final chunk includes token usage and invocation metrics
|
||||
|
||||
### Llama Processing
|
||||
- Each chunk contains a generation field with the text piece
|
||||
- First chunk includes prompt_token_count
|
||||
- Tracks generation progress through generation_token_count
|
||||
- Simple streaming format focused on text generation
|
||||
- Final chunk includes complete metrics
|
||||
|
||||
### Titan Processing
|
||||
- Single chunk response with complete text
|
||||
- No streaming - returns full response at once
|
||||
- Includes comprehensive metrics in the same chunk
|
||||
|
||||
## Handling in Code
|
||||
|
||||
The response is processed by the `transformBedrockStream` function in `app/utils/aws.ts`, which:
|
||||
|
||||
1. Reads the stream chunks
|
||||
2. Parses each chunk using `parseEventData`
|
||||
3. Handles model-specific formats:
|
||||
- For Claude: Processes message_start, content_block_start, content_block_delta
|
||||
- For Mistral: Extracts content from choices[0].message.content
|
||||
- For Llama: Uses the generation field directly
|
||||
- For Titan: Uses the outputText field from the single response
|
||||
4. Transforms the parsed data into a consistent format for the client
|
||||
5. Yields the transformed data as SSE events
|
||||
|
||||
This allows for real-time streaming of the model's response while maintaining a consistent format for the client application, regardless of which model is being used.
|
||||
@@ -94,4 +94,4 @@
|
||||
"lint-staged/yaml": "^2.2.2"
|
||||
},
|
||||
"packageManager": "yarn@1.22.19"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user