xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-06-14 01:36:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(provider): use LiteLLM as unified LLM requester backend (#2150)

Browse Source 
* refactor(provider): use LiteLLM as unified LLM requester backend

  - Replace 23+ individual requester implementations with unified litellmchat.py
  - Add litellm_provider field to 27 YAML manifests for provider routing
  - Delete redundant requester subclasses
  - Add unit tests for LiteLLMRequester (29 tests)
  - Fix num_retries parameter name (was max_retries)
  - Fix exception handling order for subclass exceptions

  LiteLLM provides unified API for 100+ providers, eliminating need for
  provider-specific requesters.

* fix: ruff format provider.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(provider): simplify LiteLLM requester usage handling

  - Remove unused Anthropic-specific tool schema generation
  - Share completion argument construction between normal and streaming calls
  - Use LiteLLM/OpenAI native usage fields for monitoring
  - Collect stream token usage from LiteLLM stream_options
  - Update LiteLLM requester tests for unified usage fields

* restore: restore deleted provider requester files

Restore individual provider requester implementations that were
removed in de61b5d3. These files coexist with the unified
litellmchat.py backend.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: update requesters and improve provider selection UI

- Added `litellm_provider` field to various requesters' YAML configurations.
- Removed obsolete Python requester files for OpenRouter, PPIO, QHAIGC, ShengSuanYun, SiliconFlow, Space, TokenPony, VolcArk, and Xai.
- Introduced new requesters for Tencent and Together AI with corresponding YAML configurations and SVG icons.
- Enhanced the ProviderForm component to include a searchable dropdown for selecting providers, improving user experience.
- Updated localization files to include search provider text for both English and Chinese.

* fix(provider): align litellm rebase with master

* fix(provider): capture streaming token usage; add token observability

The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).

- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
  endpoint: summary, per-model breakdown, token timeseries, and a
  zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
  per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization

Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.

* refactor(provider): simplify litellm capabilities

* style: simplify wrapped expressions

* feat(models): persist context metadata

* fix(provider): handle dict embeddings and openai-compatible rerank in LiteLLMRequester

- invoke_embedding: support both object- and dict-shaped response.data
  entries (OpenAI-compatible gateways like new-api return dicts)
- invoke_rerank: litellm.arerank rejects the 'openai' provider, so for
  openai-compatible (or unspecified) providers call the standard
  Jina/Cohere-style POST /v1/rerank endpoint directly over HTTP
- accept both 'relevance_score' and 'score' fields in rerank results
- add unit tests for the openai-compatible HTTP rerank path

* feat(provider): enforce requester support_type when adding models

- frontend: AddModelPopover only shows model-type tabs (llm/embedding/
  rerank) that the provider's requester declares in its manifest
  support_type; ModelsDialog fetches requester manifests and maps
  requester -> support_type, passed down through ProviderCard
- backend: add _validate_provider_supports guard in create_llm_model /
  create_embedding_model / create_rerank_model so a model cannot be
  attached to a provider whose requester does not support that type,
  even if the frontend restriction is bypassed (manifests without
  support_type are allowed for backward compatibility)
- manifests: correct support_type for providers that do not offer all
  three model types:
  - llm only: anthropic, deepseek, groq, moonshot, openrouter, xai
  - llm + text-embedding: openai, gemini, mistral
  - add rerank to new-api (verified working via /v1/rerank)
  - set llm + text-embedding + rerank for aggregator/unknown gateways

* feat(provider): add searchable alias to requester manifests

- add a free-text 'alias' field to every requester manifest spec,
  containing the vendor's English/Chinese names, pinyin, common
  nicknames and flagship model-series names (e.g. moonshot -> kimi,
  月之暗面; zhipu -> glm, 智谱清言)
- frontend: ProviderForm requester search now also matches against
  alias (substring/contains), so searching 'kimi' surfaces Moonshot,
  '硅基' surfaces SiliconFlow, etc.
- also fix support_type: openrouter (relay) supports embedding+rerank;
  LangBot Space gains rerank (coming soon)

* fix(provider): make support_type guard defensive against incomplete model_mgr

- _validate_provider_supports now uses getattr to gracefully skip when
  model_mgr / provider_dict / manifest lookup is unavailable, instead of
  raising AttributeError (fixes unit tests that mock ap.model_mgr as a
  bare SimpleNamespace)
- add TestValidateProviderSupports covering: allow supported type,
  reject unsupported type, allow when support_type missing, allow when
  provider unknown, degrade safely when model_mgr is incomplete

* fix(persistence): guard 0004 migration against missing llm_models table

The 0004_add_llm_model_context_length migration called
inspector.get_columns('llm_models') unconditionally, raising
NoSuchTableError when the table does not exist (e.g. migrating a
fresh/empty DB, as exercised by the integration tests where
create_all() registers no tables because the ORM models are not
imported). Every other migration guards with a table-existence check
first; add the same guard here for both upgrade and downgrade.

Also restore the test head assertion to 0004 (it had been lowered to
0003 to mask this failure).

* Merge branch 'master' into feat/litellm

Resolve conflicts:
- uv.lock: regenerated via 'uv lock' to reconcile litellm/fastuuid
  (ours) with openai bump (master).
- Alembic migrations: master added 0004_add_mcp_readme while this
  branch added 0004_add_llm_model_context_length, both as children of
  0003 (would create multiple heads). Re-chain the litellm migration as
  0005_add_llm_model_context_length with down_revision=0004_add_mcp_readme
  for a single linear head. Update test head assertion accordingly.

* fix(persistence): shorten migration revision id to fit varchar(32)

PostgreSQL stores alembic_version.version_num as varchar(32).
'0005_add_llm_model_context_length' (33 chars) overflowed it, raising
StringDataRightTruncationError in the PG migration tests. Rename the
revision (and file) to '0005_add_llm_context_length' (27 chars) and
update the head assertions in both SQLite and PostgreSQL migration
tests.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: fdc310 <2213070223@qq.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
This commit is contained in:
huanghuoguoguo
2026-06-13 16:59:48 +08:00
committed by GitHub
parent 7965d333ac
commit 9ecb587ac0
123 changed files with 4098 additions and 4513 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
web/src/app/home/components/models-dialog/ModelsDialog.tsx
View File

@@ -64,6 +64,17 @@ function convertExtraArgsToObject(
return obj;
}
function parseContextLength(
value: number | null | undefined,
invalidMessage: string,
): number | null {
if (value === undefined || value === null) return null;
if (!Number.isInteger(value) || value <= 0) {
throw new Error(invalidMessage);
}
return value;
}
export default function ModelsDialog({
open,
onOpenChange,
@@ -91,6 +102,12 @@ export default function ModelsDialog({
null,
);
// Map of requester name -> support_type[] (from requester manifests),
// used to restrict which model-type tabs are shown when adding models.
const [requesterSupportTypes, setRequesterSupportTypes] = useState<
Record<string, string[]>
>({});
// Popover states
const [addModelPopoverOpen, setAddModelPopoverOpen] = useState<string | null>(
null,
@@ -122,6 +139,7 @@ export default function ModelsDialog({
if (open) {
loadUserInfo();
loadProviders();
loadRequesterSupportTypes();
}
}, [open]);
@@ -161,6 +179,19 @@ export default function ModelsDialog({
}
}
async function loadRequesterSupportTypes() {
try {
const resp = await httpClient.getProviderRequesters();
const map: Record<string, string[]> = {};
for (const r of resp.requesters) {
map[r.name] = r.spec?.support_type ?? [];
}
setRequesterSupportTypes(map);
} catch (err) {
console.error('Failed to load requester support types', err);
}
}
async function loadProviderModels(providerUuid: string, silent = false) {
if (loadingProviders.has(providerUuid)) return;
@@ -254,6 +285,7 @@ export default function ModelsDialog({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) {
if (!name.trim()) {
toast.error(t('models.modelNameRequired'));
@@ -268,6 +300,10 @@ export default function ModelsDialog({
name,
provider_uuid: providerUuid,
abilities,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
),
extra_args: extraArgsObj,
} as never);
} else if (modelType === 'embedding') {
@@ -325,6 +361,7 @@ export default function ModelsDialog({
name: item.model.name,
provider_uuid: providerUuid,
abilities: item.abilities,
context_length: item.model.context_length ?? null,
extra_args: {},
} as never);
} else if (effectiveType === 'embedding') {
@@ -361,6 +398,7 @@ export default function ModelsDialog({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) {
if (!name.trim()) {
toast.error(t('models.modelNameRequired'));
@@ -375,6 +413,10 @@ export default function ModelsDialog({
name,
provider_uuid: providerUuid,
abilities,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
),
extra_args: extraArgsObj,
} as never);
} else if (modelType === 'embedding') {
@@ -495,6 +537,7 @@ export default function ModelsDialog({
key={provider.uuid}
provider={provider}
isLangBotModels={isLangBotModels}
supportTypes={requesterSupportTypes[provider.requester]}
isExpanded={expandedProviders.has(provider.uuid)}
isLoading={loadingProviders.has(provider.uuid)}
models={providerModels[provider.uuid]}
@@ -509,8 +552,15 @@ export default function ModelsDialog({
onSpaceLogin={handleSpaceLogin}
onOpenAddModel={() => setAddModelPopoverOpen(provider.uuid)}
onCloseAddModel={() => setAddModelPopoverOpen(null)}
onAddModel={(modelType, name, abilities, extraArgs) =>
handleAddModel(provider.uuid, modelType, name, abilities, extraArgs)
onAddModel={(modelType, name, abilities, extraArgs, contextLength) =>
handleAddModel(
provider.uuid,
modelType,
name,
abilities,
extraArgs,
contextLength,
)
}
onScanModels={(modelType) => handleScanModels(provider.uuid, modelType)}
onAddScannedModels={(modelType, models) =>
@@ -518,7 +568,14 @@ export default function ModelsDialog({
}
onOpenEditModel={(modelId) => setEditModelPopoverOpen(modelId)}
onCloseEditModel={() => setEditModelPopoverOpen(null)}
onUpdateModel={(modelId, modelType, name, abilities, extraArgs) =>
onUpdateModel={(
modelId,
modelType,
name,
abilities,
extraArgs,
contextLength,
) =>
handleUpdateModel(
provider.uuid,
modelId,
@@ -526,6 +583,7 @@ export default function ModelsDialog({
name,
abilities,
extraArgs,
contextLength,
)
}
onOpenDeleteConfirm={(modelId) => setDeleteConfirmOpen(modelId)}

311
web/src/app/home/components/models-dialog/component/provider-form/ProviderForm.tsx
View File

@@ -1,4 +1,4 @@
import { useEffect, useState } from 'react';
import { useEffect, useState, useRef, useCallback } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient';
import { zodResolver } from '@hookform/resolvers/zod';
@@ -16,19 +16,12 @@ import {
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import {
Select,
SelectContent,
SelectGroup,
SelectItem,
SelectLabel,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { DialogFooter } from '@/components/ui/dialog';
import { toast } from 'sonner';
import { extractI18nObject } from '@/i18n/I18nProvider';
import { CustomApiError } from '@/app/infra/entities/common';
import { cn } from '@/lib/utils';
import { Check, ChevronDown, Search } from 'lucide-react';
const getFormSchema = (t: (key: string) => string) =>
z.object({
@@ -61,6 +54,7 @@ export default function ProviderForm({
api_key: '',
},
});
const { setValue } = form;
const [requesterList, setRequesterList] = useState<
{
@@ -69,20 +63,15 @@ export default function ProviderForm({
category: string;
defaultUrl: string;
description: string;
alias: string;
}[]
>([]);
const [searchQuery, setSearchQuery] = useState('');
const [isOpen, setIsOpen] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
const searchInputRef = useRef<HTMLInputElement>(null);
useEffect(() => {
async function init() {
await loadRequesters();
if (providerId) {
await loadProvider(providerId);
}
}
init();
}, [providerId]);
async function loadRequesters() {
const loadRequesters = useCallback(async () => {
const resp = await httpClient.getProviderRequesters();
setRequesterList(
resp.requesters
@@ -96,19 +85,82 @@ export default function ProviderForm({
.find((c) => c.name === 'base_url')
?.default?.toString() || '',
description: extractI18nObject(item.description),
alias: item.spec.alias || '',
})),
);
}
}, []);
async function loadProvider(id: string) {
const resp = await httpClient.getModelProvider(id);
const provider = resp.provider;
const loadProvider = useCallback(
async (id: string) => {
const resp = await httpClient.getModelProvider(id);
const provider = resp.provider;
form.setValue('name', provider.name);
form.setValue('requester', provider.requester);
form.setValue('base_url', provider.base_url);
form.setValue('api_key', provider.api_keys?.[0] || '');
}
setValue('name', provider.name);
setValue('requester', provider.requester);
setValue('base_url', provider.base_url);
setValue('api_key', provider.api_keys?.[0] || '');
},
[setValue],
);
useEffect(() => {
async function init() {
await loadRequesters();
if (providerId) {
await loadProvider(providerId);
}
}
init();
}, [providerId, loadProvider, loadRequesters]);
// Close dropdown when clicking outside
useEffect(() => {
function handleClickOutside(event: MouseEvent) {
if (
dropdownRef.current &&
!dropdownRef.current.contains(event.target as Node)
) {
setIsOpen(false);
setSearchQuery('');
}
}
document.addEventListener('mousedown', handleClickOutside);
return () => document.removeEventListener('mousedown', handleClickOutside);
}, []);
// Focus search input when dropdown opens
useEffect(() => {
if (isOpen && searchInputRef.current) {
searchInputRef.current.focus();
}
}, [isOpen]);
// Filter requesters based on search query
const filteredRequesters = requesterList.filter(
(r) =>
r.label.toLowerCase().includes(searchQuery.toLowerCase()) ||
r.value.toLowerCase().includes(searchQuery.toLowerCase()) ||
r.alias.toLowerCase().includes(searchQuery.toLowerCase()),
);
// Group filtered requesters by category
const groupedRequesters = {
builtin: filteredRequesters.filter((r) => r.category === 'builtin'),
manufacturer: filteredRequesters.filter(
(r) => r.category === 'manufacturer',
),
maas: filteredRequesters.filter((r) => r.category === 'maas'),
'self-hosted': filteredRequesters.filter(
(r) => r.category === 'self-hosted',
),
};
const categoryLabels: Record<string, string> = {
builtin: t('models.builtin'),
manufacturer: t('models.modelManufacturer'),
maas: t('models.aggregationPlatform'),
'self-hosted': t('models.selfDeployed'),
};
async function handleFormSubmit(values: z.infer<typeof formSchema>) {
const data = {
@@ -168,17 +220,16 @@ export default function ProviderForm({
{t('models.requester')}
<span className="text-red-500">*</span>
</FormLabel>
<Select
onValueChange={(v) => {
field.onChange(v);
const req = requesterList.find((r) => r.value === v);
if (req && (!providerId || !form.getValues('base_url'))) {
form.setValue('base_url', req.defaultUrl);
}
}}
value={field.value}
>
<SelectTrigger className="bg-background">
<div ref={dropdownRef} className="relative">
{/* Trigger button */}
<button
type="button"
onClick={() => setIsOpen(!isOpen)}
className={cn(
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
isOpen && 'ring-2 ring-ring ring-offset-2',
)}
>
{selectedRequester ? (
<div className="flex items-center gap-2">
<img
@@ -191,90 +242,102 @@ export default function ProviderForm({
<span>{selectedRequester.label}</span>
</div>
) : (
<SelectValue placeholder={t('models.selectRequester')} />
<span className="text-muted-foreground">
{t('models.selectRequester')}
</span>
)}
</SelectTrigger>
<SelectContent>
<SelectGroup>
<SelectLabel>{t('models.builtin')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'builtin')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>{t('models.modelManufacturer')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'manufacturer')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>
{t('models.aggregationPlatform')}
</SelectLabel>
{requesterList
.filter((r) => r.category === 'maas')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>{t('models.selfDeployed')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'self-hosted')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
</SelectContent>
</Select>
<ChevronDown
className={cn(
'h-4 w-4 opacity-50 transition-transform',
isOpen && 'rotate-180',
)}
/>
</button>
{/* Dropdown */}
{isOpen && (
<div className="absolute z-50 mt-1 w-full rounded-md border bg-popover text-popover-foreground shadow-md animate-in fade-in-0 zoom-in-95">
{/* Search input */}
<div className="flex items-center border-b px-3">
<Search className="mr-2 h-4 w-4 shrink-0 opacity-50" />
<input
ref={searchInputRef}
type="text"
placeholder={
t('models.searchProviders') || 'Search providers...'
}
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="flex h-10 w-full rounded-md bg-transparent py-3 text-sm outline-none placeholder:text-muted-foreground"
/>
</div>
{/* Options list */}
<div className="max-h-[300px] overflow-y-auto p-1">
{Object.entries(groupedRequesters).map(
([category, items]) => {
if (items.length === 0) return null;
return (
<div key={category}>
<div className="py-1.5 px-2 text-xs font-semibold text-muted-foreground">
{categoryLabels[category]}
</div>
{items.map((r) => (
<button
key={r.value}
type="button"
onClick={() => {
field.onChange(r.value);
const req = requesterList.find(
(req) => req.value === r.value,
);
if (
req &&
(!providerId ||
!form.getValues('base_url'))
) {
form.setValue(
'base_url',
req.defaultUrl,
);
}
setIsOpen(false);
setSearchQuery('');
}}
className={cn(
'flex w-full items-center gap-2 rounded-sm px-2 py-1.5 text-sm outline-none hover:bg-accent hover:text-accent-foreground cursor-pointer',
field.value === r.value &&
'bg-accent text-accent-foreground',
)}
>
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span className="flex-1 text-left">
{r.label}
</span>
{field.value === r.value && (
<Check className="h-4 w-4" />
)}
</button>
))}
</div>
);
},
)}
{filteredRequesters.length === 0 && (
<div className="py-6 text-center text-sm text-muted-foreground">
No results found.
</div>
)}
</div>
</div>
)}
</div>
<FormMessage />
{selectedRequester?.description && (
<p className="text-sm text-muted-foreground">

113
web/src/app/home/components/models-dialog/components/AddModelPopover.tsx
View File

@@ -34,6 +34,7 @@ interface AddModelPopoverProps {
isOpen: boolean;
initialMode?: 'manual' | 'scan';
trigger?: React.ReactNode;
supportTypes?: string[];
onOpen: () => void;
onClose: () => void;
onAddModel: (
@@ -41,6 +42,7 @@ interface AddModelPopoverProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -63,6 +65,7 @@ export default function AddModelPopover({
isOpen,
initialMode = 'manual',
trigger,
supportTypes,
onOpen,
onClose,
onAddModel,
@@ -77,10 +80,26 @@ export default function AddModelPopover({
const { t } = useTranslation();
const prevIsOpenRef = useRef(false);
const [tab, setTab] = useState<ModelType>('llm');
// Map manifest support_type values to UI tab values.
// Manifest uses 'text-embedding'; the UI tab uses 'embedding'.
const tabSupport: Record<ModelType, string> = {
llm: 'llm',
embedding: 'text-embedding',
rerank: 'rerank',
};
const allTabs: ModelType[] = ['llm', 'embedding', 'rerank'];
// When supportTypes is undefined (unknown requester), show all tabs for
// backward compatibility. Otherwise only show supported tabs.
const visibleTabs: ModelType[] = supportTypes
? allTabs.filter((tabKey) => supportTypes.includes(tabSupport[tabKey]))
: allTabs;
const defaultTab: ModelType = visibleTabs[0] ?? 'llm';
const [tab, setTab] = useState<ModelType>(defaultTab);
const [mode, setMode] = useState<'manual' | 'scan'>('manual');
const [name, setName] = useState('');
const [abilities, setAbilities] = useState<string[]>([]);
const [contextLength, setContextLength] = useState('');
const [extraArgs, setExtraArgs] = useState<ExtraArg[]>([]);
const [scanLoading, setScanLoading] = useState(false);
const [scannedModels, setScannedModels] = useState<ScannedProviderModel[]>(
@@ -94,10 +113,11 @@ export default function AddModelPopover({
useEffect(() => {
const wasOpen = prevIsOpenRef.current;
if (isOpen && !wasOpen) {
setTab('llm');
setTab(defaultTab);
setMode(initialMode);
setName('');
setAbilities([]);
setContextLength('');
setExtraArgs([]);
setScanLoading(false);
setScannedModels([]);
@@ -119,7 +139,11 @@ export default function AddModelPopover({
}, [tab, mode]);
const handleAdd = async () => {
await onAddModel(tab, name, abilities, extraArgs);
const parsedContextLength =
tab === 'llm' && contextLength.trim()
? Number(contextLength.trim())
: null;
await onAddModel(tab, name, abilities, extraArgs, parsedContextLength);
};
const handleTest = async () => {
@@ -130,32 +154,6 @@ export default function AddModelPopover({
setScanLoading(true);
try {
const result = await onScanModels(trigger ? undefined : tab);
const debugData = (
result.debug?.response as { data?: Record<string, unknown>[] }
)?.data;
if (Array.isArray(debugData)) {
const debugMap = new Map<string, Record<string, unknown>>();
for (const item of debugData) {
if (typeof item?.id === 'string') {
debugMap.set(item.id, item);
}
}
for (const model of result.models) {
const debugItem = debugMap.get(model.id);
if (!debugItem) continue;
const features = debugItem.features as
| Record<string, unknown>
| undefined;
const tools = features?.tools as Record<string, unknown> | undefined;
if (tools?.function_calling === true) {
const nextAbilities = new Set(model.abilities || []);
nextAbilities.add('func_call');
model.abilities = [...nextAbilities];
}
}
}
setScannedModels(result.models);
setSelectedScannedModels({});
} finally {
@@ -279,20 +277,31 @@ export default function AddModelPopover({
className="flex flex-col min-h-0 flex-1"
>
<div className="flex-shrink-0">
{!(trigger && initialMode === 'scan') && (
<TabsList className="grid w-full grid-cols-3">
<TabsTrigger value="llm">
<MessageSquareText className="h-4 w-4 mr-1" />
{t('models.chat')}
</TabsTrigger>
<TabsTrigger value="embedding">
<Cpu className="h-4 w-4 mr-1" />
{t('models.embedding')}
</TabsTrigger>
<TabsTrigger value="rerank">
<ArrowUpDown className="h-4 w-4 mr-1" />
{t('models.rerank')}
</TabsTrigger>
{!(trigger && initialMode === 'scan') && visibleTabs.length > 1 && (
<TabsList
className="grid w-full"
style={{
gridTemplateColumns: `repeat(${visibleTabs.length}, minmax(0, 1fr))`,
}}
>
{visibleTabs.includes('llm') && (
<TabsTrigger value="llm">
<MessageSquareText className="h-4 w-4 mr-1" />
{t('models.chat')}
</TabsTrigger>
)}
{visibleTabs.includes('embedding') && (
<TabsTrigger value="embedding">
<Cpu className="h-4 w-4 mr-1" />
{t('models.embedding')}
</TabsTrigger>
)}
{visibleTabs.includes('rerank') && (
<TabsTrigger value="rerank">
<ArrowUpDown className="h-4 w-4 mr-1" />
{t('models.rerank')}
</TabsTrigger>
)}
</TabsList>
)}
</div>
@@ -344,6 +353,24 @@ export default function AddModelPopover({
</div>
)}
{tab === 'llm' && (
<div className="space-y-2">
<Label htmlFor="add-context-length">
{t('models.contextLength')}
</Label>
<Input
id="add-context-length"
type="number"
min={1}
step={1}
inputMode="numeric"
placeholder={t('models.contextLengthPlaceholder')}
value={contextLength}
onChange={(e) => setContextLength(e.target.value)}
/>
</div>
)}
<ExtraArgsEditor
args={extraArgs}
onChange={setExtraArgs}

41
web/src/app/home/components/models-dialog/components/ModelItem.tsx
View File

@@ -31,6 +31,7 @@ interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onTestModel: (
name: string,
@@ -92,6 +93,11 @@ export default function ModelItem({
const [editAbilities, setEditAbilities] = useState<string[]>(
modelType === 'llm' ? (model as LLMModel).abilities || [] : [],
);
const [editContextLength, setEditContextLength] = useState(
modelType === 'llm' && (model as LLMModel).context_length
? String((model as LLMModel).context_length)
: '',
);
const [editExtraArgs, setEditExtraArgs] = useState<ExtraArg[]>(
convertExtraArgsToArray(model.extra_args),
);
@@ -106,13 +112,27 @@ export default function ModelItem({
setEditAbilities(
modelType === 'llm' ? (model as LLMModel).abilities || [] : [],
);
setEditContextLength(
modelType === 'llm' && (model as LLMModel).context_length
? String((model as LLMModel).context_length)
: '',
);
setEditExtraArgs(convertExtraArgsToArray(model.extra_args));
onResetTestResult();
}
}, [isEditOpen]);
const handleSave = async () => {
await onUpdateModel(editName, editAbilities, editExtraArgs);
const parsedContextLength =
modelType === 'llm' && editContextLength.trim()
? Number(editContextLength.trim())
: null;
await onUpdateModel(
editName,
editAbilities,
editExtraArgs,
parsedContextLength,
);
};
const handleTest = async () => {
@@ -287,6 +307,25 @@ export default function ModelItem({
</div>
)}
{modelType === 'llm' && (
<div className="space-y-2">
<Label htmlFor={`edit-context-length-${model.uuid}`}>
{t('models.contextLength')}
</Label>
<Input
id={`edit-context-length-${model.uuid}`}
type="number"
min={1}
step={1}
inputMode="numeric"
placeholder={t('models.contextLengthPlaceholder')}
value={editContextLength}
disabled={isLangBotModels}
onChange={(e) => setEditContextLength(e.target.value)}
/>
</div>
)}
<ExtraArgsEditor
args={editExtraArgs}
onChange={setEditExtraArgs}

14
web/src/app/home/components/models-dialog/components/ProviderCard.tsx
View File

@@ -39,6 +39,7 @@ import AddModelPopover from './AddModelPopover';
interface ProviderCardProps {
provider: ModelProvider;
isLangBotModels?: boolean;
supportTypes?: string[];
isExpanded: boolean;
isLoading: boolean;
models?: ProviderModels;
@@ -60,6 +61,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -74,6 +76,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
onCloseDeleteConfirm: () => void;
@@ -99,6 +102,7 @@ function maskApiKey(key: string): string {
export default function ProviderCard({
provider,
isLangBotModels = false,
supportTypes,
isExpanded,
isLoading,
models,
@@ -319,6 +323,7 @@ export default function ProviderCard({
addModelMode === 'manual'
}
initialMode="manual"
supportTypes={supportTypes}
trigger={
<Button
variant="ghost"
@@ -353,6 +358,7 @@ export default function ProviderCard({
addModelMode === 'scan'
}
initialMode="scan"
supportTypes={supportTypes}
trigger={
<Button
variant="ghost"
@@ -405,13 +411,19 @@ export default function ProviderCard({
onOpenDeleteConfirm={onOpenDeleteConfirm}
onCloseDeleteConfirm={onCloseDeleteConfirm}
onDeleteModel={() => onDeleteModel(model.uuid, 'llm')}
onUpdateModel={(name, abilities, extraArgs) =>
onUpdateModel={(
name,
abilities,
extraArgs,
contextLength,
) =>
onUpdateModel(
model.uuid,
'llm',
name,
abilities,
extraArgs,
contextLength,
)
}
onTestModel={(name, abilities, extraArgs) =>

3
web/src/app/home/components/models-dialog/types.ts
View File

@@ -53,6 +53,7 @@ export interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onTest: (
name: string,
@@ -89,6 +90,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -103,6 +105,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
onCloseDeleteConfirm: () => void;

462
web/src/app/home/monitoring/components/TokenMonitoring.tsx Normal file
View File

@@ -0,0 +1,462 @@
import React, { useEffect, useMemo, useState, useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import {
ComposedChart,
Area,
Bar,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
ResponsiveContainer,
Legend,
} from 'recharts';
import {
Coins,
ArrowDownToLine,
ArrowUpFromLine,
Gauge,
AlertTriangle,
TrendingUp,
} from 'lucide-react';
import { httpClient } from '@/app/infra/http/HttpClient';
interface TokenSummary {
total_calls: number;
success_calls: number;
error_calls: number;
total_input_tokens: number;
total_output_tokens: number;
total_tokens: number;
total_cost: number;
avg_tokens_per_call: number;
avg_duration_ms: number;
avg_tokens_per_second: number;
zero_token_success_calls: number;
}
interface TokenByModel {
model_name: string;
calls: number;
error_calls: number;
input_tokens: number;
output_tokens: number;
total_tokens: number;
cost: number;
avg_tokens_per_call: number;
avg_duration_ms: number;
}
interface TokenTimeseriesPoint {
bucket: string;
input_tokens: number;
output_tokens: number;
total_tokens: number;
calls: number;
}
interface TokenStatistics {
summary: TokenSummary;
by_model: TokenByModel[];
timeseries: TokenTimeseriesPoint[];
bucket: string;
}
interface TokenMonitoringProps {
botIds?: string[];
pipelineIds?: string[];
startTime?: string;
endTime?: string;
/** Bumped by the parent to trigger a refetch on manual refresh. */
refreshKey?: number;
}
function formatNumber(n: number): string {
if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(2)}M`;
if (n >= 1_000) return `${(n / 1_000).toFixed(1)}K`;
return n.toLocaleString();
}
const TOOLTIP_STYLE: React.CSSProperties = {
backgroundColor: 'var(--card)',
border: '1px solid var(--border)',
borderRadius: '12px',
boxShadow:
'0 10px 15px -3px rgb(0 0 0 / 0.1), 0 4px 6px -4px rgb(0 0 0 / 0.1)',
fontSize: '13px',
padding: '12px',
color: 'var(--foreground)',
};
function MetricTile({
icon,
label,
value,
sub,
accent,
}: {
icon: React.ReactNode;
label: string;
value: string;
sub?: string;
accent?: string;
}) {
return (
<div className="bg-card rounded-xl border p-4 flex flex-col gap-2">
<div className="flex items-center gap-2 text-muted-foreground text-sm">
<span
className="flex items-center justify-center h-7 w-7 rounded-lg"
style={{
backgroundColor: accent ? `${accent}1a` : 'var(--muted)',
color: accent || 'var(--foreground)',
}}
>
{icon}
</span>
{label}
</div>
<div className="text-2xl font-semibold text-foreground tabular-nums">
{value}
</div>
{sub && <div className="text-xs text-muted-foreground">{sub}</div>}
</div>
);
}
export default function TokenMonitoring({
botIds,
pipelineIds,
startTime,
endTime,
refreshKey,
}: TokenMonitoringProps) {
const { t } = useTranslation();
const [bucket, setBucket] = useState<'hour' | 'day'>('hour');
const [stats, setStats] = useState<TokenStatistics | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const botIdsKey = JSON.stringify(botIds);
const pipelineIdsKey = JSON.stringify(pipelineIds);
const fetchStats = useCallback(async () => {
setLoading(true);
setError(null);
try {
const result = await httpClient.getTokenStatistics({
botId: botIds,
pipelineId: pipelineIds,
startTime,
endTime,
bucket,
});
setStats(result);
} catch (e) {
setError(e instanceof Error ? e.message : String(e));
} finally {
setLoading(false);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [botIdsKey, pipelineIdsKey, startTime, endTime, bucket, refreshKey]);
useEffect(() => {
fetchStats();
}, [fetchStats]);
const chartData = useMemo(() => {
if (!stats) return [];
return stats.timeseries.map((p) => ({
bucket: p.bucket,
input: p.input_tokens,
output: p.output_tokens,
total: p.total_tokens,
}));
}, [stats]);
if (loading) {
return (
<div className="space-y-4">
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-6 gap-4">
{Array.from({ length: 6 }).map((_, i) => (
<div
key={i}
className="bg-card rounded-xl border p-4 h-24 animate-pulse"
/>
))}
</div>
<div className="bg-card rounded-xl border p-6 h-[320px] animate-pulse" />
</div>
);
}
if (error) {
return (
<div className="bg-card rounded-xl border p-6 text-sm text-destructive flex items-center gap-2">
<AlertTriangle className="h-4 w-4" />
{t('monitoring.tokens.loadError', { error })}
</div>
);
}
if (!stats || stats.summary.total_calls === 0) {
return (
<div className="bg-card rounded-xl border p-6">
<div className="h-[260px] flex flex-col items-center justify-center text-muted-foreground gap-2">
<Coins className="h-[3rem] w-[3rem]" />
<div className="text-sm">{t('monitoring.tokens.noData')}</div>
</div>
</div>
);
}
const { summary, by_model } = stats;
return (
<div className="space-y-6">
{/* Data-quality warning: streamed calls that recorded 0 tokens */}
{summary.zero_token_success_calls > 0 && (
<div className="bg-amber-500/10 border border-amber-500/30 text-amber-700 dark:text-amber-400 rounded-xl p-4 text-sm flex items-start gap-2">
<AlertTriangle className="h-4 w-4 mt-0.5 shrink-0" />
<span>
{t('monitoring.tokens.zeroTokenWarning', {
count: summary.zero_token_success_calls,
})}
</span>
</div>
)}
{/* Summary tiles */}
<div className="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-6 gap-4">
<MetricTile
icon={<Coins className="h-4 w-4" />}
label={t('monitoring.tokens.totalTokens')}
value={formatNumber(summary.total_tokens)}
sub={t('monitoring.tokens.acrossCalls', {
count: summary.total_calls,
})}
accent="#8b5cf6"
/>
<MetricTile
icon={<ArrowDownToLine className="h-4 w-4" />}
label={t('monitoring.tokens.inputTokens')}
value={formatNumber(summary.total_input_tokens)}
accent="#3b82f6"
/>
<MetricTile
icon={<ArrowUpFromLine className="h-4 w-4" />}
label={t('monitoring.tokens.outputTokens')}
value={formatNumber(summary.total_output_tokens)}
accent="#10b981"
/>
<MetricTile
icon={<TrendingUp className="h-4 w-4" />}
label={t('monitoring.tokens.avgPerCall')}
value={formatNumber(summary.avg_tokens_per_call)}
accent="#f59e0b"
/>
<MetricTile
icon={<Gauge className="h-4 w-4" />}
label={t('monitoring.tokens.throughput')}
value={`${summary.avg_tokens_per_second}`}
sub={t('monitoring.tokens.tokensPerSec')}
accent="#06b6d4"
/>
<MetricTile
icon={<AlertTriangle className="h-4 w-4" />}
label={t('monitoring.tokens.errorCalls')}
value={`${summary.error_calls}`}
sub={t('monitoring.tokens.ofTotal', { count: summary.total_calls })}
accent="#ef4444"
/>
</div>
{/* Token usage over time */}
<div className="bg-card rounded-xl border p-6">
<div className="flex items-center justify-between mb-6">
<h3 className="text-base font-semibold text-foreground">
{t('monitoring.tokens.usageOverTime')}
</h3>
<div className="inline-flex rounded-lg border p-0.5 text-sm">
{(['hour', 'day'] as const).map((b) => (
<button
key={b}
onClick={() => setBucket(b)}
className={`px-3 py-1 rounded-md transition-colors ${
bucket === b
? 'bg-primary text-primary-foreground'
: 'text-muted-foreground hover:text-foreground'
}`}
>
{t(`monitoring.tokens.bucket.${b}`)}
</button>
))}
</div>
</div>
<div className="h-[320px]">
<ResponsiveContainer width="100%" height="100%">
<ComposedChart
data={chartData}
margin={{ top: 10, right: 20, left: 0, bottom: 0 }}
>
<defs>
<linearGradient id="tokTotal" x1="0" y1="0" x2="0" y2="1">
<stop offset="5%" stopColor="#8b5cf6" stopOpacity={0.35} />
<stop offset="95%" stopColor="#8b5cf6" stopOpacity={0.03} />
</linearGradient>
</defs>
<CartesianGrid
strokeDasharray="3 3"
stroke="var(--border)"
vertical={false}
/>
<XAxis
dataKey="bucket"
tick={{ fontSize: 12, fill: 'var(--muted-foreground)' }}
tickLine={false}
axisLine={{ stroke: 'var(--border)' }}
dy={10}
/>
<YAxis
tick={{ fontSize: 12, fill: 'var(--muted-foreground)' }}
tickLine={false}
axisLine={{ stroke: 'var(--border)' }}
width={48}
tickFormatter={(v) => formatNumber(Number(v))}
/>
<Tooltip
contentStyle={TOOLTIP_STYLE}
labelStyle={{
fontWeight: 600,
marginBottom: '8px',
color: 'var(--foreground)',
}}
formatter={(value: number) => formatNumber(Number(value))}
/>
<Legend
wrapperStyle={{
fontSize: '13px',
paddingTop: '16px',
fontWeight: 500,
}}
iconType="circle"
iconSize={10}
/>
<Bar
dataKey="input"
name={t('monitoring.tokens.inputTokens')}
stackId="io"
fill="#3b82f6"
radius={[0, 0, 0, 0]}
barSize={18}
/>
<Bar
dataKey="output"
name={t('monitoring.tokens.outputTokens')}
stackId="io"
fill="#10b981"
radius={[4, 4, 0, 0]}
barSize={18}
/>
<Area
type="monotone"
dataKey="total"
name={t('monitoring.tokens.totalTokens')}
stroke="#8b5cf6"
strokeWidth={2.5}
fill="url(#tokTotal)"
dot={false}
activeDot={{ r: 5, strokeWidth: 2 }}
/>
</ComposedChart>
</ResponsiveContainer>
</div>
</div>
{/* Per-model breakdown */}
<div className="bg-card rounded-xl border p-6">
<h3 className="text-base font-semibold text-foreground mb-4">
{t('monitoring.tokens.byModel')}
</h3>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="text-left text-muted-foreground border-b">
<th className="py-2 pr-4 font-medium">
{t('monitoring.tokens.model')}
</th>
<th className="py-2 px-4 font-medium text-right">
{t('monitoring.tokens.calls')}
</th>
<th className="py-2 px-4 font-medium text-right">
{t('monitoring.tokens.inputTokens')}
</th>
<th className="py-2 px-4 font-medium text-right">
{t('monitoring.tokens.outputTokens')}
</th>
<th className="py-2 px-4 font-medium text-right">
{t('monitoring.tokens.totalTokens')}
</th>
<th className="py-2 px-4 font-medium text-right">
{t('monitoring.tokens.avgPerCall')}
</th>
<th className="py-2 pl-4 font-medium text-right">
{t('monitoring.tokens.avgLatency')}
</th>
</tr>
</thead>
<tbody>
{by_model.map((m) => {
const share =
summary.total_tokens > 0
? (m.total_tokens / summary.total_tokens) * 100
: 0;
return (
<tr
key={m.model_name}
className="border-b last:border-0 hover:bg-muted/40 transition-colors"
>
<td className="py-2.5 pr-4">
<div className="font-medium text-foreground">
{m.model_name}
</div>
<div className="mt-1 h-1.5 w-32 rounded-full bg-muted overflow-hidden">
<div
className="h-full rounded-full bg-violet-500"
style={{ width: `${share}%` }}
/>
</div>
</td>
<td className="py-2.5 px-4 text-right tabular-nums">
{m.calls}
{m.error_calls > 0 && (
<span className="text-destructive">
{' '}
({m.error_calls}✕)
</span>
)}
</td>
<td className="py-2.5 px-4 text-right tabular-nums">
{formatNumber(m.input_tokens)}
</td>
<td className="py-2.5 px-4 text-right tabular-nums">
{formatNumber(m.output_tokens)}
</td>
<td className="py-2.5 px-4 text-right tabular-nums font-medium">
{formatNumber(m.total_tokens)}
</td>
<td className="py-2.5 px-4 text-right tabular-nums">
{formatNumber(m.avg_tokens_per_call)}
</td>
<td className="py-2.5 pl-4 text-right tabular-nums">
{m.avg_duration_ms}ms
</td>
</tr>
);
})}
</tbody>
</table>
</div>
</div>
</div>
);
}

22
web/src/app/home/monitoring/page.tsx
View File

@@ -13,6 +13,7 @@ import {
} from 'lucide-react';
import OverviewCards from './components/overview-cards/OverviewCards';
import MonitoringFilters from './components/filters/MonitoringFilters';
import TokenMonitoring from './components/TokenMonitoring';
import { ExportDropdown } from './components/ExportDropdown';
import { useMonitoringFilters } from './hooks/useMonitoringFilters';
import { useMonitoringData } from './hooks/useMonitoringData';
@@ -319,6 +320,9 @@ function MonitoringPageContent() {
<TabsTrigger value="modelCalls" className="px-6 py-2">
{t('monitoring.tabs.modelCalls')}
</TabsTrigger>
<TabsTrigger value="tokens" className="px-6 py-2">
{t('monitoring.tabs.tokens')}
</TabsTrigger>
<TabsTrigger value="feedback" className="px-6 py-2">
{t('monitoring.tabs.feedback')}
</TabsTrigger>
@@ -668,6 +672,24 @@ function MonitoringPageContent() {
</div>
</TabsContent>
<TabsContent value="tokens" className="p-6 m-0">
<TokenMonitoring
botIds={
filterState.selectedBots.length > 0
? filterState.selectedBots
: undefined
}
pipelineIds={
filterState.selectedPipelines.length > 0
? filterState.selectedPipelines
: undefined
}
startTime={feedbackTimeRange.startTime}
endTime={feedbackTimeRange.endTime}
refreshKey={feedbackRefreshKey}
/>
</TabsContent>
<TabsContent value="feedback" className="p-6 m-0">
<div>
{loading && (

3
web/src/app/infra/entities/api/index.ts
View File

@@ -30,6 +30,8 @@ export interface Requester {
spec: {
config: IDynamicFormItemSchema[];
provider_category: string;
support_type?: string[];
alias?: string;
};
}
@@ -96,6 +98,7 @@ export interface LLMModel {
provider_uuid: string;
provider?: ModelProvider;
abilities?: string[];
context_length?: number | null;
extra_args?: object;
}

62
web/src/app/infra/http/BackendClient.ts
View File

@@ -1224,6 +1224,68 @@ export class BackendClient extends BaseHttpClient {
return this.get(`/api/v1/monitoring/overview?${queryParams.toString()}`);
}
public getTokenStatistics(params: {
botId?: string[];
pipelineId?: string[];
startTime?: string;
endTime?: string;
bucket?: 'hour' | 'day';
}): Promise<{
summary: {
total_calls: number;
success_calls: number;
error_calls: number;
total_input_tokens: number;
total_output_tokens: number;
total_tokens: number;
total_cost: number;
avg_tokens_per_call: number;
avg_duration_ms: number;
avg_tokens_per_second: number;
zero_token_success_calls: number;
};
by_model: Array<{
model_name: string;
calls: number;
error_calls: number;
input_tokens: number;
output_tokens: number;
total_tokens: number;
cost: number;
avg_tokens_per_call: number;
avg_duration_ms: number;
}>;
timeseries: Array<{
bucket: string;
input_tokens: number;
output_tokens: number;
total_tokens: number;
calls: number;
}>;
bucket: string;
}> {
const queryParams = new URLSearchParams();
if (params.botId) {
params.botId.forEach((id) => queryParams.append('botId', id));
}
if (params.pipelineId) {
params.pipelineId.forEach((id) => queryParams.append('pipelineId', id));
}
if (params.startTime) {
queryParams.append('startTime', params.startTime);
}
if (params.endTime) {
queryParams.append('endTime', params.endTime);
}
if (params.bucket) {
queryParams.append('bucket', params.bucket);
}
return this.get(
`/api/v1/monitoring/token-statistics?${queryParams.toString()}`,
);
}
// ============ Survey API ============
public getSurveyPending(): Promise<{
survey: {

29
web/src/i18n/locales/en-US.ts
View File

@@ -201,6 +201,9 @@ const enUS = {
selectModelAbilities: 'Select model abilities',
visionAbility: 'Vision Ability',
functionCallAbility: 'Function Call',
contextLength: 'Context Window',
contextLengthPlaceholder: 'Unknown',
contextLengthInvalid: 'Context window must be a positive integer',
extraParameters: 'Extra Parameters',
addParameter: 'Add Parameter',
keyName: 'Key Name',
@@ -258,6 +261,7 @@ const enUS = {
selectProvider: 'Select Provider',
requester: 'Provider Type',
selectRequester: 'Select Provider Type',
searchProviders: 'Search providers...',
langbotModelsDescription: 'Cloud models powered by LangBot Space',
credits: 'Credits',
loginWithSpace: 'Login with Space',
@@ -1201,6 +1205,7 @@ const enUS = {
llmCalls: 'LLM Calls',
embeddingCalls: 'Embedding Calls',
modelCalls: 'Model Calls',
tokens: 'Token Monitoring',
feedback: 'User Feedback',
sessions: 'Session Analysis',
errors: 'Error Logs',
@@ -1239,6 +1244,30 @@ const enUS = {
avgDuration: 'Avg Duration',
calls: 'Calls',
},
tokens: {
totalTokens: 'Total Tokens',
inputTokens: 'Input Tokens',
outputTokens: 'Output Tokens',
avgPerCall: 'Avg / Call',
throughput: 'Throughput',
tokensPerSec: 'tokens/sec',
errorCalls: 'Failed Calls',
acrossCalls: 'across {{count}} calls',
ofTotal: 'of {{count}} total',
usageOverTime: 'Token Usage Over Time',
byModel: 'By Model',
model: 'Model',
calls: 'Calls',
avgLatency: 'Avg Latency',
noData: 'No token usage in the selected time range',
loadError: 'Failed to load token statistics: {{error}}',
zeroTokenWarning:
'{{count}} successful call(s) reported zero token usage. This usually means the upstream provider did not return usage info — check the model provider configuration.',
bucket: {
hour: 'Hourly',
day: 'Daily',
},
},
embeddingCalls: {
title: 'Embedding Calls',
model: 'Model',

3
web/src/i18n/locales/es-ES.ts
View File

@@ -206,6 +206,9 @@ const esES = {
selectModelAbilities: 'Seleccionar capacidades del modelo',
visionAbility: 'Capacidad de visión',
functionCallAbility: 'Llamada a funciones',
contextLength: 'Ventana de contexto',
contextLengthPlaceholder: 'Desconocido',
contextLengthInvalid: 'La ventana de contexto debe ser un entero positivo',
extraParameters: 'Parámetros adicionales',
addParameter: 'Añadir parámetro',
keyName: 'Nombre de la clave',

4
web/src/i18n/locales/ja-JP.ts
View File

@@ -204,6 +204,10 @@ const jaJP = {
selectModelAbilities: 'モデル機能を選択',
visionAbility: '視覚機能',
functionCallAbility: '関数呼び出し',
contextLength: 'コンテキストウィンドウ',
contextLengthPlaceholder: '不明',
contextLengthInvalid:
'コンテキストウィンドウは正の整数である必要があります',
extraParameters: '追加パラメータ',
addParameter: 'パラメータを追加',
keyName: 'キー名',

4
web/src/i18n/locales/ru-RU.ts
View File

@@ -203,6 +203,10 @@ const ruRU = {
selectModelAbilities: 'Выберите возможности модели',
visionAbility: 'Распознавание изображений',
functionCallAbility: 'Вызов функций',
contextLength: 'Контекстное окно',
contextLengthPlaceholder: 'Неизвестно',
contextLengthInvalid:
'Контекстное окно должно быть положительным целым числом',
extraParameters: 'Дополнительные параметры',
addParameter: 'Добавить параметр',
keyName: 'Имя ключа',

3
web/src/i18n/locales/th-TH.ts
View File

@@ -199,6 +199,9 @@ const thTH = {
selectModelAbilities: 'เลือกความสามารถของโมเดล',
visionAbility: 'ความสามารถด้านภาพ',
functionCallAbility: 'การเรียกฟังก์ชัน',
contextLength: 'หน้าต่างบริบท',
contextLengthPlaceholder: 'ไม่ทราบ',
contextLengthInvalid: 'หน้าต่างบริบทต้องเป็นจำนวนเต็มบวก',
extraParameters: 'พารามิเตอร์เพิ่มเติม',
addParameter: 'เพิ่มพารามิเตอร์',
keyName: 'ชื่อคีย์',

3
web/src/i18n/locales/vi-VN.ts
View File

@@ -203,6 +203,9 @@ const viVN = {
selectModelAbilities: 'Chọn khả năng mô hình',
visionAbility: 'Khả năng thị giác',
functionCallAbility: 'Gọi hàm',
contextLength: 'Cửa sổ ngữ cảnh',
contextLengthPlaceholder: 'Không rõ',
contextLengthInvalid: 'Cửa sổ ngữ cảnh phải là số nguyên dương',
extraParameters: 'Tham số bổ sung',
addParameter: 'Thêm tham số',
keyName: 'Tên khóa',

29
web/src/i18n/locales/zh-Hans.ts
View File

@@ -193,6 +193,9 @@ const zhHans = {
selectModelAbilities: '选择模型能力',
visionAbility: '视觉能力',
functionCallAbility: '函数调用',
contextLength: '上下文窗口',
contextLengthPlaceholder: '未知',
contextLengthInvalid: '上下文窗口必须是正整数',
extraParameters: '额外参数',
addParameter: '添加参数',
keyName: '键名',
@@ -248,6 +251,7 @@ const zhHans = {
selectProvider: '选择供应商',
requester: '供应商类型',
selectRequester: '选择供应商类型',
searchProviders: '搜索供应商...',
langbotModelsDescription: 'LangBot Space 提供的云端模型',
credits: '积分',
loginWithSpace: '通过 Space 登录',
@@ -1144,6 +1148,7 @@ const zhHans = {
llmCalls: 'LLM调用',
embeddingCalls: 'Embedding调用',
modelCalls: '模型调用',
tokens: 'Token 监控',
feedback: '用户反馈',
sessions: '会话分析',
errors: '错误日志',
@@ -1182,6 +1187,30 @@ const zhHans = {
avgDuration: '平均耗时',
calls: '调用次数',
},
tokens: {
totalTokens: '总 Token 数',
inputTokens: '输入 Token',
outputTokens: '输出 Token',
avgPerCall: '平均每次调用',
throughput: '吞吐量',
tokensPerSec: 'Token/秒',
errorCalls: '失败调用',
acrossCalls: '共 {{count}} 次调用',
ofTotal: '共 {{count}} 次',
usageOverTime: 'Token 用量趋势',
byModel: '按模型统计',
model: '模型',
calls: '调用次数',
avgLatency: '平均延迟',
noData: '所选时间范围内暂无 Token 用量数据',
loadError: '加载 Token 统计失败：{{error}}',
zeroTokenWarning:
'检测到 {{count}} 次成功调用未上报 Token 用量（记为 0）。这通常表示上游未返回 usage 信息，请检查模型供应商配置。',
bucket: {
hour: '按小时',
day: '按天',
},
},
embeddingCalls: {
title: 'Embedding调用',
model: '模型',

3
web/src/i18n/locales/zh-Hant.ts
View File

@@ -193,6 +193,9 @@ const zhHant = {
selectModelAbilities: '選擇模型能力',
visionAbility: '視覺能力',
functionCallAbility: '函數呼叫',
contextLength: '上下文視窗',
contextLengthPlaceholder: '未知',
contextLengthInvalid: '上下文視窗必須是正整數',
extraParameters: '額外參數',
addParameter: '新增參數',
keyName: '鍵名',