Files
LangBot/web/src/app/home/components/models-dialog/components/AddModelPopover.tsx
T
huanghuoguoguo 9ecb587ac0 refactor(provider): use LiteLLM as unified LLM requester backend (#2150)
* refactor(provider): use LiteLLM as unified LLM requester backend

  - Replace 23+ individual requester implementations with unified litellmchat.py
  - Add litellm_provider field to 27 YAML manifests for provider routing
  - Delete redundant requester subclasses
  - Add unit tests for LiteLLMRequester (29 tests)
  - Fix num_retries parameter name (was max_retries)
  - Fix exception handling order for subclass exceptions

  LiteLLM provides unified API for 100+ providers, eliminating need for
  provider-specific requesters.

* fix: ruff format provider.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(provider): simplify LiteLLM requester usage handling

  - Remove unused Anthropic-specific tool schema generation
  - Share completion argument construction between normal and streaming calls
  - Use LiteLLM/OpenAI native usage fields for monitoring
  - Collect stream token usage from LiteLLM stream_options
  - Update LiteLLM requester tests for unified usage fields

* restore: restore deleted provider requester files

Restore individual provider requester implementations that were
removed in de61b5d3. These files coexist with the unified
litellmchat.py backend.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: update requesters and improve provider selection UI

- Added `litellm_provider` field to various requesters' YAML configurations.
- Removed obsolete Python requester files for OpenRouter, PPIO, QHAIGC, ShengSuanYun, SiliconFlow, Space, TokenPony, VolcArk, and Xai.
- Introduced new requesters for Tencent and Together AI with corresponding YAML configurations and SVG icons.
- Enhanced the ProviderForm component to include a searchable dropdown for selecting providers, improving user experience.
- Updated localization files to include search provider text for both English and Chinese.

* fix(provider): align litellm rebase with master

* fix(provider): capture streaming token usage; add token observability

The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).

- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
  endpoint: summary, per-model breakdown, token timeseries, and a
  zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
  per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization

Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.

* refactor(provider): simplify litellm capabilities

* style: simplify wrapped expressions

* feat(models): persist context metadata

* fix(provider): handle dict embeddings and openai-compatible rerank in LiteLLMRequester

- invoke_embedding: support both object- and dict-shaped response.data
  entries (OpenAI-compatible gateways like new-api return dicts)
- invoke_rerank: litellm.arerank rejects the 'openai' provider, so for
  openai-compatible (or unspecified) providers call the standard
  Jina/Cohere-style POST /v1/rerank endpoint directly over HTTP
- accept both 'relevance_score' and 'score' fields in rerank results
- add unit tests for the openai-compatible HTTP rerank path

* feat(provider): enforce requester support_type when adding models

- frontend: AddModelPopover only shows model-type tabs (llm/embedding/
  rerank) that the provider's requester declares in its manifest
  support_type; ModelsDialog fetches requester manifests and maps
  requester -> support_type, passed down through ProviderCard
- backend: add _validate_provider_supports guard in create_llm_model /
  create_embedding_model / create_rerank_model so a model cannot be
  attached to a provider whose requester does not support that type,
  even if the frontend restriction is bypassed (manifests without
  support_type are allowed for backward compatibility)
- manifests: correct support_type for providers that do not offer all
  three model types:
  - llm only: anthropic, deepseek, groq, moonshot, openrouter, xai
  - llm + text-embedding: openai, gemini, mistral
  - add rerank to new-api (verified working via /v1/rerank)
  - set llm + text-embedding + rerank for aggregator/unknown gateways

* feat(provider): add searchable alias to requester manifests

- add a free-text 'alias' field to every requester manifest spec,
  containing the vendor's English/Chinese names, pinyin, common
  nicknames and flagship model-series names (e.g. moonshot -> kimi,
  月之暗面; zhipu -> glm, 智谱清言)
- frontend: ProviderForm requester search now also matches against
  alias (substring/contains), so searching 'kimi' surfaces Moonshot,
  '硅基' surfaces SiliconFlow, etc.
- also fix support_type: openrouter (relay) supports embedding+rerank;
  LangBot Space gains rerank (coming soon)

* fix(provider): make support_type guard defensive against incomplete model_mgr

- _validate_provider_supports now uses getattr to gracefully skip when
  model_mgr / provider_dict / manifest lookup is unavailable, instead of
  raising AttributeError (fixes unit tests that mock ap.model_mgr as a
  bare SimpleNamespace)
- add TestValidateProviderSupports covering: allow supported type,
  reject unsupported type, allow when support_type missing, allow when
  provider unknown, degrade safely when model_mgr is incomplete

* fix(persistence): guard 0004 migration against missing llm_models table

The 0004_add_llm_model_context_length migration called
inspector.get_columns('llm_models') unconditionally, raising
NoSuchTableError when the table does not exist (e.g. migrating a
fresh/empty DB, as exercised by the integration tests where
create_all() registers no tables because the ORM models are not
imported). Every other migration guards with a table-existence check
first; add the same guard here for both upgrade and downgrade.

Also restore the test head assertion to 0004 (it had been lowered to
0003 to mask this failure).

* Merge branch 'master' into feat/litellm

Resolve conflicts:
- uv.lock: regenerated via 'uv lock' to reconcile litellm/fastuuid
  (ours) with openai bump (master).
- Alembic migrations: master added 0004_add_mcp_readme while this
  branch added 0004_add_llm_model_context_length, both as children of
  0003 (would create multiple heads). Re-chain the litellm migration as
  0005_add_llm_model_context_length with down_revision=0004_add_mcp_readme
  for a single linear head. Update test head assertion accordingly.

* fix(persistence): shorten migration revision id to fit varchar(32)

PostgreSQL stores alembic_version.version_num as varchar(32).
'0005_add_llm_model_context_length' (33 chars) overflowed it, raising
StringDataRightTruncationError in the PG migration tests. Rename the
revision (and file) to '0005_add_llm_context_length' (27 chars) and
update the head assertions in both SQLite and PostgreSQL migration
tests.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: fdc310 <2213070223@qq.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
2026-06-13 16:59:48 +08:00

591 lines
21 KiB
TypeScript

import { useState, useEffect, useRef } from 'react';
import {
Plus,
MessageSquareText,
Cpu,
ArrowUpDown,
Eye,
Wrench,
Check,
RefreshCw,
} from 'lucide-react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Checkbox } from '@/components/ui/checkbox';
import {
Popover,
PopoverContent,
PopoverTrigger,
} from '@/components/ui/popover';
import { Tabs, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { useTranslation } from 'react-i18next';
import { ScannedProviderModel } from '@/app/infra/entities/api';
import {
ExtraArg,
ModelType,
ScanModelsResult,
SelectedScannedModel,
TestResult,
} from '../types';
import ExtraArgsEditor from './ExtraArgsEditor';
interface AddModelPopoverProps {
isOpen: boolean;
initialMode?: 'manual' | 'scan';
trigger?: React.ReactNode;
supportTypes?: string[];
onOpen: () => void;
onClose: () => void;
onAddModel: (
modelType: ModelType,
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
modelType: ModelType,
models: SelectedScannedModel[],
) => Promise<void>;
onTestModel: (
name: string,
modelType: ModelType,
abilities: string[],
extraArgs: ExtraArg[],
) => Promise<void>;
isSubmitting: boolean;
isTesting: boolean;
testResult: TestResult | null;
onResetTestResult: () => void;
}
export default function AddModelPopover({
isOpen,
initialMode = 'manual',
trigger,
supportTypes,
onOpen,
onClose,
onAddModel,
onScanModels,
onAddScannedModels,
onTestModel,
isSubmitting,
isTesting,
testResult,
onResetTestResult,
}: AddModelPopoverProps) {
const { t } = useTranslation();
const prevIsOpenRef = useRef(false);
// Map manifest support_type values to UI tab values.
// Manifest uses 'text-embedding'; the UI tab uses 'embedding'.
const tabSupport: Record<ModelType, string> = {
llm: 'llm',
embedding: 'text-embedding',
rerank: 'rerank',
};
const allTabs: ModelType[] = ['llm', 'embedding', 'rerank'];
// When supportTypes is undefined (unknown requester), show all tabs for
// backward compatibility. Otherwise only show supported tabs.
const visibleTabs: ModelType[] = supportTypes
? allTabs.filter((tabKey) => supportTypes.includes(tabSupport[tabKey]))
: allTabs;
const defaultTab: ModelType = visibleTabs[0] ?? 'llm';
const [tab, setTab] = useState<ModelType>(defaultTab);
const [mode, setMode] = useState<'manual' | 'scan'>('manual');
const [name, setName] = useState('');
const [abilities, setAbilities] = useState<string[]>([]);
const [contextLength, setContextLength] = useState('');
const [extraArgs, setExtraArgs] = useState<ExtraArg[]>([]);
const [scanLoading, setScanLoading] = useState(false);
const [scannedModels, setScannedModels] = useState<ScannedProviderModel[]>(
[],
);
const [selectedScannedModels, setSelectedScannedModels] = useState<
Record<string, SelectedScannedModel>
>({});
const [scanQuery, setScanQuery] = useState('');
useEffect(() => {
const wasOpen = prevIsOpenRef.current;
if (isOpen && !wasOpen) {
setTab(defaultTab);
setMode(initialMode);
setName('');
setAbilities([]);
setContextLength('');
setExtraArgs([]);
setScanLoading(false);
setScannedModels([]);
setSelectedScannedModels({});
setScanQuery('');
onResetTestResult();
if (initialMode === 'scan') {
handleScan();
}
}
prevIsOpenRef.current = isOpen;
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [isOpen, onResetTestResult]);
useEffect(() => {
setScannedModels([]);
setSelectedScannedModels({});
setScanQuery('');
}, [tab, mode]);
const handleAdd = async () => {
const parsedContextLength =
tab === 'llm' && contextLength.trim()
? Number(contextLength.trim())
: null;
await onAddModel(tab, name, abilities, extraArgs, parsedContextLength);
};
const handleTest = async () => {
await onTestModel(name, tab, tab === 'llm' ? abilities : [], extraArgs);
};
const handleScan = async () => {
setScanLoading(true);
try {
const result = await onScanModels(trigger ? undefined : tab);
setScannedModels(result.models);
setSelectedScannedModels({});
} finally {
setScanLoading(false);
}
};
const handleAddScanned = async () => {
const selectedModels = Object.values(selectedScannedModels);
if (selectedModels.length === 0) return;
await onAddScannedModels(tab, selectedModels);
};
const toggleAbility = (ability: string, checked: boolean) => {
if (checked) {
setAbilities([...abilities, ability]);
} else {
setAbilities(abilities.filter((a) => a !== ability));
}
};
const toggleScannedModel = (
model: ScannedProviderModel,
checked: boolean,
) => {
setSelectedScannedModels((prev) => {
const next = { ...prev };
if (checked) {
next[model.id] = {
model,
abilities:
model.type === 'llm'
? prev[model.id]?.abilities || model.abilities || []
: [],
};
} else {
delete next[model.id];
}
return next;
});
};
const toggleScannedModelAbility = (
modelId: string,
ability: string,
checked: boolean,
) => {
setSelectedScannedModels((prev) => {
const current = prev[modelId];
if (!current) return prev;
const nextAbilities = checked
? [...current.abilities, ability]
: current.abilities.filter((item) => item !== ability);
return {
...prev,
[modelId]: {
...current,
abilities: nextAbilities,
},
};
});
};
const filteredScannedModels = scannedModels.filter((model) =>
model.name.toLowerCase().includes(scanQuery.trim().toLowerCase()),
);
const selectableModels = filteredScannedModels.filter(
(m) => !m.already_added,
);
const allSelected =
selectableModels.length > 0 &&
selectableModels.every((m) => Boolean(selectedScannedModels[m.id]));
const toggleSelectAll = () => {
if (allSelected) {
setSelectedScannedModels({});
} else {
const next: Record<string, SelectedScannedModel> = {};
for (const model of selectableModels) {
next[model.id] = {
model,
abilities: model.type === 'llm' ? model.abilities || [] : [],
};
}
setSelectedScannedModels(next);
}
};
return (
<Popover
open={isOpen}
onOpenChange={(open) => (open ? onOpen() : onClose())}
>
<PopoverTrigger asChild>
{trigger || (
<Button
variant="ghost"
size="sm"
className="h-6 text-xs"
onClick={(e) => e.stopPropagation()}
>
<Plus className="h-3 w-3 mr-1" />
{t('models.addModel')}
</Button>
)}
</PopoverTrigger>
<PopoverContent
className="w-[min(24rem,calc(100vw-2rem))] max-h-[calc(100vh-8rem)] flex flex-col overflow-hidden"
align="end"
side="bottom"
sideOffset={8}
collisionPadding={16}
onClick={(e) => e.stopPropagation()}
>
<Tabs
value={tab}
onValueChange={(v) => setTab(v as ModelType)}
className="flex flex-col min-h-0 flex-1"
>
<div className="flex-shrink-0">
{!(trigger && initialMode === 'scan') && visibleTabs.length > 1 && (
<TabsList
className="grid w-full"
style={{
gridTemplateColumns: `repeat(${visibleTabs.length}, minmax(0, 1fr))`,
}}
>
{visibleTabs.includes('llm') && (
<TabsTrigger value="llm">
<MessageSquareText className="h-4 w-4 mr-1" />
{t('models.chat')}
</TabsTrigger>
)}
{visibleTabs.includes('embedding') && (
<TabsTrigger value="embedding">
<Cpu className="h-4 w-4 mr-1" />
{t('models.embedding')}
</TabsTrigger>
)}
{visibleTabs.includes('rerank') && (
<TabsTrigger value="rerank">
<ArrowUpDown className="h-4 w-4 mr-1" />
{t('models.rerank')}
</TabsTrigger>
)}
</TabsList>
)}
</div>
<div className="overflow-y-auto flex-1 min-h-0">
{mode === 'manual' ? (
<div className="mt-3">
<div className="space-y-3">
<div className="space-y-2">
<Label>{t('models.modelName')}</Label>
<Input
placeholder={t('models.modelName')}
value={name}
onChange={(e) => setName(e.target.value)}
/>
</div>
{tab === 'llm' && (
<div className="space-y-2">
<Label>{t('models.abilities')}</Label>
<div className="flex gap-4">
<div className="flex items-center gap-2">
<Checkbox
id="add-vision"
checked={abilities.includes('vision')}
onCheckedChange={(checked) =>
toggleAbility('vision', checked as boolean)
}
/>
<Label htmlFor="add-vision" className="text-sm">
<Eye className="h-3 w-3 inline mr-1" />
{t('models.visionAbility')}
</Label>
</div>
<div className="flex items-center gap-2">
<Checkbox
id="add-func-call"
checked={abilities.includes('func_call')}
onCheckedChange={(checked) =>
toggleAbility('func_call', checked as boolean)
}
/>
<Label htmlFor="add-func-call" className="text-sm">
<Wrench className="h-3 w-3 inline mr-1" />
{t('models.functionCallAbility')}
</Label>
</div>
</div>
</div>
)}
{tab === 'llm' && (
<div className="space-y-2">
<Label htmlFor="add-context-length">
{t('models.contextLength')}
</Label>
<Input
id="add-context-length"
type="number"
min={1}
step={1}
inputMode="numeric"
placeholder={t('models.contextLengthPlaceholder')}
value={contextLength}
onChange={(e) => setContextLength(e.target.value)}
/>
</div>
)}
<ExtraArgsEditor
args={extraArgs}
onChange={setExtraArgs}
modelType={tab}
/>
<div className="flex gap-2">
<Button
className="flex-1"
size="sm"
onClick={handleAdd}
disabled={isSubmitting || isTesting}
>
{isSubmitting ? t('common.saving') : t('common.add')}
</Button>
<Button
className="flex-1"
size="sm"
variant="outline"
onClick={handleTest}
disabled={isSubmitting || isTesting}
>
{isTesting ? (
t('common.loading')
) : testResult?.success ? (
<>
<Check className="h-4 w-4 mr-1 text-green-500" />
{(testResult.duration / 1000).toFixed(1)}s
</>
) : (
t('common.test')
)}
</Button>
</div>
</div>
</div>
) : (
<div className="space-y-2 mt-3">
{scanLoading ? (
<div className="flex items-center justify-center py-4">
<RefreshCw className="h-4 w-4 mr-2 animate-spin text-muted-foreground" />
<span className="text-sm text-muted-foreground">
{t('models.scanModels')}...
</span>
</div>
) : (
<>
<div className="space-y-2">
<Input
placeholder={t('models.searchScannedModels')}
value={scanQuery}
onChange={(e) => setScanQuery(e.target.value)}
disabled={scannedModels.length === 0}
/>
{selectableModels.length > 0 && (
<div className="flex items-center gap-2 pt-1">
<Checkbox
id="scan-select-all"
checked={allSelected}
onCheckedChange={toggleSelectAll}
/>
<Label
htmlFor="scan-select-all"
className="text-sm font-medium"
>
{t('models.selectAll')}
<span className="text-muted-foreground ml-1">
({Object.keys(selectedScannedModels).length}/
{selectableModels.length})
</span>
</Label>
</div>
)}
</div>
<div
className="h-64 overflow-y-auto overscroll-contain rounded-md border"
onWheel={(e) => e.stopPropagation()}
>
<div className="p-3 space-y-2">
{filteredScannedModels.length === 0 ? (
<p className="text-sm text-muted-foreground">
{scannedModels.length === 0
? t('models.noScannedModels')
: t('models.noScannedModelsMatch')}
</p>
) : (
filteredScannedModels.map((model) => {
const isSelected = Boolean(
selectedScannedModels[model.id],
);
const selectedAbilities =
selectedScannedModels[model.id]?.abilities || [];
return (
<div
key={model.id}
className="rounded-md border p-3 space-y-2"
>
<div className="flex items-start gap-3">
<Checkbox
checked={isSelected || model.already_added}
disabled={model.already_added}
onCheckedChange={(checked) =>
toggleScannedModel(
model,
checked as boolean,
)
}
/>
<div className="min-w-0 flex-1">
<div className="text-sm font-medium break-all">
{model.name}
</div>
<div className="text-xs text-muted-foreground">
{model.already_added
? t('models.alreadyAdded')
: model.type === 'llm'
? t('models.chat')
: model.type === 'embedding'
? t('models.embedding')
: t('models.rerank')}
</div>
</div>
</div>
{model.type === 'llm' &&
isSelected &&
!model.already_added && (
<div className="flex gap-4 pl-7">
<div className="flex items-center gap-2">
<Checkbox
id={`scan-vision-${model.id}`}
checked={selectedAbilities.includes(
'vision',
)}
onCheckedChange={(checked) =>
toggleScannedModelAbility(
model.id,
'vision',
checked as boolean,
)
}
/>
<Label
htmlFor={`scan-vision-${model.id}`}
className="text-sm"
>
<Eye className="h-3 w-3 inline mr-1" />
{t('models.visionAbility')}
</Label>
</div>
<div className="flex items-center gap-2">
<Checkbox
id={`scan-func-${model.id}`}
checked={selectedAbilities.includes(
'func_call',
)}
onCheckedChange={(checked) =>
toggleScannedModelAbility(
model.id,
'func_call',
checked as boolean,
)
}
/>
<Label
htmlFor={`scan-func-${model.id}`}
className="text-sm"
>
<Wrench className="h-3 w-3 inline mr-1" />
{t('models.functionCallAbility')}
</Label>
</div>
</div>
)}
</div>
);
})
)}
</div>
</div>
</>
)}
<div className="flex gap-2">
<Button
className="flex-1"
size="sm"
onClick={handleAddScanned}
disabled={
isSubmitting ||
scanLoading ||
Object.keys(selectedScannedModels).length === 0
}
>
{isSubmitting
? t('common.saving')
: t('models.addSelectedModels')}
</Button>
<Button
variant="outline"
size="sm"
onClick={handleScan}
disabled={scanLoading || isSubmitting}
>
<RefreshCw
className={`h-3.5 w-3.5 ${scanLoading ? 'animate-spin' : ''}`}
/>
</Button>
</div>
</div>
)}
</div>
</Tabs>
</PopoverContent>
</Popover>
);
}