xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-06-27 07:54:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(provider): use LiteLLM as unified LLM requester backend (#2150)

Browse Source 
* refactor(provider): use LiteLLM as unified LLM requester backend

  - Replace 23+ individual requester implementations with unified litellmchat.py
  - Add litellm_provider field to 27 YAML manifests for provider routing
  - Delete redundant requester subclasses
  - Add unit tests for LiteLLMRequester (29 tests)
  - Fix num_retries parameter name (was max_retries)
  - Fix exception handling order for subclass exceptions

  LiteLLM provides unified API for 100+ providers, eliminating need for
  provider-specific requesters.

* fix: ruff format provider.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(provider): simplify LiteLLM requester usage handling

  - Remove unused Anthropic-specific tool schema generation
  - Share completion argument construction between normal and streaming calls
  - Use LiteLLM/OpenAI native usage fields for monitoring
  - Collect stream token usage from LiteLLM stream_options
  - Update LiteLLM requester tests for unified usage fields

* restore: restore deleted provider requester files

Restore individual provider requester implementations that were
removed in de61b5d3. These files coexist with the unified
litellmchat.py backend.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: update requesters and improve provider selection UI

- Added `litellm_provider` field to various requesters' YAML configurations.
- Removed obsolete Python requester files for OpenRouter, PPIO, QHAIGC, ShengSuanYun, SiliconFlow, Space, TokenPony, VolcArk, and Xai.
- Introduced new requesters for Tencent and Together AI with corresponding YAML configurations and SVG icons.
- Enhanced the ProviderForm component to include a searchable dropdown for selecting providers, improving user experience.
- Updated localization files to include search provider text for both English and Chinese.

* fix(provider): align litellm rebase with master

* fix(provider): capture streaming token usage; add token observability

The LiteLLM streaming requester only captured usage when a chunk had an
empty `choices` list. Many OpenAI-compatible gateways (e.g. new-api) and
providers send the final usage payload in a chunk that still carries an
empty-delta choice, so streamed calls always recorded 0 tokens in the
monitoring logs/dashboard (non-streaming worked).

- Capture stream usage whenever a chunk carries it, regardless of choices
- Add robust _normalize_usage (dict/obj shapes, derive missing total_tokens)
- Register litellm in bootutils/deps.py (was in pyproject only)
- Add MonitoringService.get_token_statistics + /monitoring/token-statistics
  endpoint: summary, per-model breakdown, token timeseries, and a
  zero-token-success data-quality signal
- Add TokenMonitoring dashboard tab (summary tiles, stacked token chart,
  per-model table) + i18n (en/zh)
- Regression tests for stream usage capture and usage normalization

Verified end-to-end against a real OpenAI-compatible endpoint with
gpt-5.5 and claude-opus-4-8: tokens now recorded non-zero for both
streaming and non-streaming paths.

* refactor(provider): simplify litellm capabilities

* style: simplify wrapped expressions

* feat(models): persist context metadata

* fix(provider): handle dict embeddings and openai-compatible rerank in LiteLLMRequester

- invoke_embedding: support both object- and dict-shaped response.data
  entries (OpenAI-compatible gateways like new-api return dicts)
- invoke_rerank: litellm.arerank rejects the 'openai' provider, so for
  openai-compatible (or unspecified) providers call the standard
  Jina/Cohere-style POST /v1/rerank endpoint directly over HTTP
- accept both 'relevance_score' and 'score' fields in rerank results
- add unit tests for the openai-compatible HTTP rerank path

* feat(provider): enforce requester support_type when adding models

- frontend: AddModelPopover only shows model-type tabs (llm/embedding/
  rerank) that the provider's requester declares in its manifest
  support_type; ModelsDialog fetches requester manifests and maps
  requester -> support_type, passed down through ProviderCard
- backend: add _validate_provider_supports guard in create_llm_model /
  create_embedding_model / create_rerank_model so a model cannot be
  attached to a provider whose requester does not support that type,
  even if the frontend restriction is bypassed (manifests without
  support_type are allowed for backward compatibility)
- manifests: correct support_type for providers that do not offer all
  three model types:
  - llm only: anthropic, deepseek, groq, moonshot, openrouter, xai
  - llm + text-embedding: openai, gemini, mistral
  - add rerank to new-api (verified working via /v1/rerank)
  - set llm + text-embedding + rerank for aggregator/unknown gateways

* feat(provider): add searchable alias to requester manifests

- add a free-text 'alias' field to every requester manifest spec,
  containing the vendor's English/Chinese names, pinyin, common
  nicknames and flagship model-series names (e.g. moonshot -> kimi,
  月之暗面; zhipu -> glm, 智谱清言)
- frontend: ProviderForm requester search now also matches against
  alias (substring/contains), so searching 'kimi' surfaces Moonshot,
  '硅基' surfaces SiliconFlow, etc.
- also fix support_type: openrouter (relay) supports embedding+rerank;
  LangBot Space gains rerank (coming soon)

* fix(provider): make support_type guard defensive against incomplete model_mgr

- _validate_provider_supports now uses getattr to gracefully skip when
  model_mgr / provider_dict / manifest lookup is unavailable, instead of
  raising AttributeError (fixes unit tests that mock ap.model_mgr as a
  bare SimpleNamespace)
- add TestValidateProviderSupports covering: allow supported type,
  reject unsupported type, allow when support_type missing, allow when
  provider unknown, degrade safely when model_mgr is incomplete

* fix(persistence): guard 0004 migration against missing llm_models table

The 0004_add_llm_model_context_length migration called
inspector.get_columns('llm_models') unconditionally, raising
NoSuchTableError when the table does not exist (e.g. migrating a
fresh/empty DB, as exercised by the integration tests where
create_all() registers no tables because the ORM models are not
imported). Every other migration guards with a table-existence check
first; add the same guard here for both upgrade and downgrade.

Also restore the test head assertion to 0004 (it had been lowered to
0003 to mask this failure).

* Merge branch 'master' into feat/litellm

Resolve conflicts:
- uv.lock: regenerated via 'uv lock' to reconcile litellm/fastuuid
  (ours) with openai bump (master).
- Alembic migrations: master added 0004_add_mcp_readme while this
  branch added 0004_add_llm_model_context_length, both as children of
  0003 (would create multiple heads). Re-chain the litellm migration as
  0005_add_llm_model_context_length with down_revision=0004_add_mcp_readme
  for a single linear head. Update test head assertion accordingly.

* fix(persistence): shorten migration revision id to fit varchar(32)

PostgreSQL stores alembic_version.version_num as varchar(32).
'0005_add_llm_model_context_length' (33 chars) overflowed it, raising
StringDataRightTruncationError in the PG migration tests. Rename the
revision (and file) to '0005_add_llm_context_length' (27 chars) and
update the head assertions in both SQLite and PostgreSQL migration
tests.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: fdc310 <2213070223@qq.com>
Co-authored-by: RockChinQ <rockchinq@gmail.com>
This commit is contained in:
huanghuoguoguo
2026-06-13 16:59:48 +08:00
committed by GitHub
parent 7965d333ac
commit 9ecb587ac0
123 changed files with 4098 additions and 4513 deletions
Download Patch File Download Diff File Expand all files Collapse all files
web/src/app/home/components/models-dialog/ModelsDialog.tsx
+61 -3
View File
@@ -64,6 +64,17 @@ function convertExtraArgsToObject(
return obj;
}
function parseContextLength(
value: number | null | undefined,
invalidMessage: string,
): number | null {
if (value === undefined || value === null) return null;
if (!Number.isInteger(value) || value <= 0) {
throw new Error(invalidMessage);
}
return value;
}
export default function ModelsDialog({
open,
onOpenChange,
@@ -91,6 +102,12 @@ export default function ModelsDialog({
null,
);
// Map of requester name -> support_type[] (from requester manifests),
// used to restrict which model-type tabs are shown when adding models.
const [requesterSupportTypes, setRequesterSupportTypes] = useState<
Record<string, string[]>
>({});
// Popover states
const [addModelPopoverOpen, setAddModelPopoverOpen] = useState<string | null>(
null,
@@ -122,6 +139,7 @@ export default function ModelsDialog({
if (open) {
loadUserInfo();
loadProviders();
loadRequesterSupportTypes();
}
}, [open]);
@@ -161,6 +179,19 @@ export default function ModelsDialog({
}
}
async function loadRequesterSupportTypes() {
try {
const resp = await httpClient.getProviderRequesters();
const map: Record<string, string[]> = {};
for (const r of resp.requesters) {
map[r.name] = r.spec?.support_type ?? [];
}
setRequesterSupportTypes(map);
} catch (err) {
console.error('Failed to load requester support types', err);
}
}
async function loadProviderModels(providerUuid: string, silent = false) {
if (loadingProviders.has(providerUuid)) return;
@@ -254,6 +285,7 @@ export default function ModelsDialog({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) {
if (!name.trim()) {
toast.error(t('models.modelNameRequired'));
@@ -268,6 +300,10 @@ export default function ModelsDialog({
name,
provider_uuid: providerUuid,
abilities,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
),
extra_args: extraArgsObj,
} as never);
} else if (modelType === 'embedding') {
@@ -325,6 +361,7 @@ export default function ModelsDialog({
name: item.model.name,
provider_uuid: providerUuid,
abilities: item.abilities,
context_length: item.model.context_length ?? null,
extra_args: {},
} as never);
} else if (effectiveType === 'embedding') {
@@ -361,6 +398,7 @@ export default function ModelsDialog({
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) {
if (!name.trim()) {
toast.error(t('models.modelNameRequired'));
@@ -375,6 +413,10 @@ export default function ModelsDialog({
name,
provider_uuid: providerUuid,
abilities,
context_length: parseContextLength(
contextLength,
t('models.contextLengthInvalid'),
),
extra_args: extraArgsObj,
} as never);
} else if (modelType === 'embedding') {
@@ -495,6 +537,7 @@ export default function ModelsDialog({
key={provider.uuid}
provider={provider}
isLangBotModels={isLangBotModels}
supportTypes={requesterSupportTypes[provider.requester]}
isExpanded={expandedProviders.has(provider.uuid)}
isLoading={loadingProviders.has(provider.uuid)}
models={providerModels[provider.uuid]}
@@ -509,8 +552,15 @@ export default function ModelsDialog({
onSpaceLogin={handleSpaceLogin}
onOpenAddModel={() => setAddModelPopoverOpen(provider.uuid)}
onCloseAddModel={() => setAddModelPopoverOpen(null)}
onAddModel={(modelType, name, abilities, extraArgs) =>
handleAddModel(provider.uuid, modelType, name, abilities, extraArgs)
onAddModel={(modelType, name, abilities, extraArgs, contextLength) =>
handleAddModel(
provider.uuid,
modelType,
name,
abilities,
extraArgs,
contextLength,
)
}
onScanModels={(modelType) => handleScanModels(provider.uuid, modelType)}
onAddScannedModels={(modelType, models) =>
@@ -518,7 +568,14 @@ export default function ModelsDialog({
}
onOpenEditModel={(modelId) => setEditModelPopoverOpen(modelId)}
onCloseEditModel={() => setEditModelPopoverOpen(null)}
onUpdateModel={(modelId, modelType, name, abilities, extraArgs) =>
onUpdateModel={(
modelId,
modelType,
name,
abilities,
extraArgs,
contextLength,
) =>
handleUpdateModel(
provider.uuid,
modelId,
@@ -526,6 +583,7 @@ export default function ModelsDialog({
name,
abilities,
extraArgs,
contextLength,
)
}
onOpenDeleteConfirm={(modelId) => setDeleteConfirmOpen(modelId)}
web/src/app/home/components/models-dialog/component/provider-form/ProviderForm.tsx
+187 -124
View File
@@ -1,4 +1,4 @@
import { useEffect, useState } from 'react';
import { useEffect, useState, useRef, useCallback } from 'react';
import { httpClient } from '@/app/infra/http/HttpClient';
import { zodResolver } from '@hookform/resolvers/zod';
@@ -16,19 +16,12 @@ import {
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import {
Select,
SelectContent,
SelectGroup,
SelectItem,
SelectLabel,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
import { DialogFooter } from '@/components/ui/dialog';
import { toast } from 'sonner';
import { extractI18nObject } from '@/i18n/I18nProvider';
import { CustomApiError } from '@/app/infra/entities/common';
import { cn } from '@/lib/utils';
import { Check, ChevronDown, Search } from 'lucide-react';
const getFormSchema = (t: (key: string) => string) =>
z.object({
@@ -61,6 +54,7 @@ export default function ProviderForm({
api_key: '',
},
});
const { setValue } = form;
const [requesterList, setRequesterList] = useState<
{
@@ -69,20 +63,15 @@ export default function ProviderForm({
category: string;
defaultUrl: string;
description: string;
alias: string;
}[]
>([]);
const [searchQuery, setSearchQuery] = useState('');
const [isOpen, setIsOpen] = useState(false);
const dropdownRef = useRef<HTMLDivElement>(null);
const searchInputRef = useRef<HTMLInputElement>(null);
useEffect(() => {
async function init() {
await loadRequesters();
if (providerId) {
await loadProvider(providerId);
}
}
init();
}, [providerId]);
async function loadRequesters() {
const loadRequesters = useCallback(async () => {
const resp = await httpClient.getProviderRequesters();
setRequesterList(
resp.requesters
@@ -96,19 +85,82 @@ export default function ProviderForm({
.find((c) => c.name === 'base_url')
?.default?.toString() || '',
description: extractI18nObject(item.description),
alias: item.spec.alias || '',
})),
);
}
}, []);
async function loadProvider(id: string) {
const resp = await httpClient.getModelProvider(id);
const provider = resp.provider;
const loadProvider = useCallback(
async (id: string) => {
const resp = await httpClient.getModelProvider(id);
const provider = resp.provider;
form.setValue('name', provider.name);
form.setValue('requester', provider.requester);
form.setValue('base_url', provider.base_url);
form.setValue('api_key', provider.api_keys?.[0] || '');
}
setValue('name', provider.name);
setValue('requester', provider.requester);
setValue('base_url', provider.base_url);
setValue('api_key', provider.api_keys?.[0] || '');
},
[setValue],
);
useEffect(() => {
async function init() {
await loadRequesters();
if (providerId) {
await loadProvider(providerId);
}
}
init();
}, [providerId, loadProvider, loadRequesters]);
// Close dropdown when clicking outside
useEffect(() => {
function handleClickOutside(event: MouseEvent) {
if (
dropdownRef.current &&
!dropdownRef.current.contains(event.target as Node)
) {
setIsOpen(false);
setSearchQuery('');
}
}
document.addEventListener('mousedown', handleClickOutside);
return () => document.removeEventListener('mousedown', handleClickOutside);
}, []);
// Focus search input when dropdown opens
useEffect(() => {
if (isOpen && searchInputRef.current) {
searchInputRef.current.focus();
}
}, [isOpen]);
// Filter requesters based on search query
const filteredRequesters = requesterList.filter(
(r) =>
r.label.toLowerCase().includes(searchQuery.toLowerCase()) ||
r.value.toLowerCase().includes(searchQuery.toLowerCase()) ||
r.alias.toLowerCase().includes(searchQuery.toLowerCase()),
);
// Group filtered requesters by category
const groupedRequesters = {
builtin: filteredRequesters.filter((r) => r.category === 'builtin'),
manufacturer: filteredRequesters.filter(
(r) => r.category === 'manufacturer',
),
maas: filteredRequesters.filter((r) => r.category === 'maas'),
'self-hosted': filteredRequesters.filter(
(r) => r.category === 'self-hosted',
),
};
const categoryLabels: Record<string, string> = {
builtin: t('models.builtin'),
manufacturer: t('models.modelManufacturer'),
maas: t('models.aggregationPlatform'),
'self-hosted': t('models.selfDeployed'),
};
async function handleFormSubmit(values: z.infer<typeof formSchema>) {
const data = {
@@ -168,17 +220,16 @@ export default function ProviderForm({
{t('models.requester')}
<span className="text-red-500">*</span>
</FormLabel>
<Select
onValueChange={(v) => {
field.onChange(v);
const req = requesterList.find((r) => r.value === v);
if (req && (!providerId || !form.getValues('base_url'))) {
form.setValue('base_url', req.defaultUrl);
}
}}
value={field.value}
>
<SelectTrigger className="bg-background">
<div ref={dropdownRef} className="relative">
{/* Trigger button */}
<button
type="button"
onClick={() => setIsOpen(!isOpen)}
className={cn(
'flex h-10 w-full items-center justify-between rounded-md border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
isOpen && 'ring-2 ring-ring ring-offset-2',
)}
>
{selectedRequester ? (
<div className="flex items-center gap-2">
<img
@@ -191,90 +242,102 @@ export default function ProviderForm({
<span>{selectedRequester.label}</span>
</div>
) : (
<SelectValue placeholder={t('models.selectRequester')} />
<span className="text-muted-foreground">
{t('models.selectRequester')}
</span>
)}
</SelectTrigger>
<SelectContent>
<SelectGroup>
<SelectLabel>{t('models.builtin')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'builtin')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>{t('models.modelManufacturer')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'manufacturer')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>
{t('models.aggregationPlatform')}
</SelectLabel>
{requesterList
.filter((r) => r.category === 'maas')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
<SelectGroup>
<SelectLabel>{t('models.selfDeployed')}</SelectLabel>
{requesterList
.filter((r) => r.category === 'self-hosted')
.map((r) => (
<SelectItem key={r.value} value={r.value}>
<div className="flex items-center gap-2">
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span>{r.label}</span>
</div>
</SelectItem>
))}
</SelectGroup>
</SelectContent>
</Select>
<ChevronDown
className={cn(
'h-4 w-4 opacity-50 transition-transform',
isOpen && 'rotate-180',
)}
/>
</button>
{/* Dropdown */}
{isOpen && (
<div className="absolute z-50 mt-1 w-full rounded-md border bg-popover text-popover-foreground shadow-md animate-in fade-in-0 zoom-in-95">
{/* Search input */}
<div className="flex items-center border-b px-3">
<Search className="mr-2 h-4 w-4 shrink-0 opacity-50" />
<input
ref={searchInputRef}
type="text"
placeholder={
t('models.searchProviders') || 'Search providers...'
}
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="flex h-10 w-full rounded-md bg-transparent py-3 text-sm outline-none placeholder:text-muted-foreground"
/>
</div>
{/* Options list */}
<div className="max-h-[300px] overflow-y-auto p-1">
{Object.entries(groupedRequesters).map(
([category, items]) => {
if (items.length === 0) return null;
return (
<div key={category}>
<div className="py-1.5 px-2 text-xs font-semibold text-muted-foreground">
{categoryLabels[category]}
</div>
{items.map((r) => (
<button
key={r.value}
type="button"
onClick={() => {
field.onChange(r.value);
const req = requesterList.find(
(req) => req.value === r.value,
);
if (
req &&
(!providerId ||
!form.getValues('base_url'))
) {
form.setValue(
'base_url',
req.defaultUrl,
);
}
setIsOpen(false);
setSearchQuery('');
}}
className={cn(
'flex w-full items-center gap-2 rounded-sm px-2 py-1.5 text-sm outline-none hover:bg-accent hover:text-accent-foreground cursor-pointer',
field.value === r.value &&
'bg-accent text-accent-foreground',
)}
>
<img
src={httpClient.getProviderRequesterIconURL(
r.value,
)}
alt={r.label}
className="h-5 w-5 rounded"
/>
<span className="flex-1 text-left">
{r.label}
</span>
{field.value === r.value && (
<Check className="h-4 w-4" />
)}
</button>
))}
</div>
);
},
)}
{filteredRequesters.length === 0 && (
<div className="py-6 text-center text-sm text-muted-foreground">
No results found.
</div>
)}
</div>
</div>
)}
</div>
<FormMessage />
{selectedRequester?.description && (
<p className="text-sm text-muted-foreground">
web/src/app/home/components/models-dialog/components/AddModelPopover.tsx
+70 -43
View File
@@ -34,6 +34,7 @@ interface AddModelPopoverProps {
isOpen: boolean;
initialMode?: 'manual' | 'scan';
trigger?: React.ReactNode;
supportTypes?: string[];
onOpen: () => void;
onClose: () => void;
onAddModel: (
@@ -41,6 +42,7 @@ interface AddModelPopoverProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -63,6 +65,7 @@ export default function AddModelPopover({
isOpen,
initialMode = 'manual',
trigger,
supportTypes,
onOpen,
onClose,
onAddModel,
@@ -77,10 +80,26 @@ export default function AddModelPopover({
const { t } = useTranslation();
const prevIsOpenRef = useRef(false);
const [tab, setTab] = useState<ModelType>('llm');
// Map manifest support_type values to UI tab values.
// Manifest uses 'text-embedding'; the UI tab uses 'embedding'.
const tabSupport: Record<ModelType, string> = {
llm: 'llm',
embedding: 'text-embedding',
rerank: 'rerank',
};
const allTabs: ModelType[] = ['llm', 'embedding', 'rerank'];
// When supportTypes is undefined (unknown requester), show all tabs for
// backward compatibility. Otherwise only show supported tabs.
const visibleTabs: ModelType[] = supportTypes
? allTabs.filter((tabKey) => supportTypes.includes(tabSupport[tabKey]))
: allTabs;
const defaultTab: ModelType = visibleTabs[0] ?? 'llm';
const [tab, setTab] = useState<ModelType>(defaultTab);
const [mode, setMode] = useState<'manual' | 'scan'>('manual');
const [name, setName] = useState('');
const [abilities, setAbilities] = useState<string[]>([]);
const [contextLength, setContextLength] = useState('');
const [extraArgs, setExtraArgs] = useState<ExtraArg[]>([]);
const [scanLoading, setScanLoading] = useState(false);
const [scannedModels, setScannedModels] = useState<ScannedProviderModel[]>(
@@ -94,10 +113,11 @@ export default function AddModelPopover({
useEffect(() => {
const wasOpen = prevIsOpenRef.current;
if (isOpen && !wasOpen) {
setTab('llm');
setTab(defaultTab);
setMode(initialMode);
setName('');
setAbilities([]);
setContextLength('');
setExtraArgs([]);
setScanLoading(false);
setScannedModels([]);
@@ -119,7 +139,11 @@ export default function AddModelPopover({
}, [tab, mode]);
const handleAdd = async () => {
await onAddModel(tab, name, abilities, extraArgs);
const parsedContextLength =
tab === 'llm' && contextLength.trim()
? Number(contextLength.trim())
: null;
await onAddModel(tab, name, abilities, extraArgs, parsedContextLength);
};
const handleTest = async () => {
@@ -130,32 +154,6 @@ export default function AddModelPopover({
setScanLoading(true);
try {
const result = await onScanModels(trigger ? undefined : tab);
const debugData = (
result.debug?.response as { data?: Record<string, unknown>[] }
)?.data;
if (Array.isArray(debugData)) {
const debugMap = new Map<string, Record<string, unknown>>();
for (const item of debugData) {
if (typeof item?.id === 'string') {
debugMap.set(item.id, item);
}
}
for (const model of result.models) {
const debugItem = debugMap.get(model.id);
if (!debugItem) continue;
const features = debugItem.features as
| Record<string, unknown>
| undefined;
const tools = features?.tools as Record<string, unknown> | undefined;
if (tools?.function_calling === true) {
const nextAbilities = new Set(model.abilities || []);
nextAbilities.add('func_call');
model.abilities = [...nextAbilities];
}
}
}
setScannedModels(result.models);
setSelectedScannedModels({});
} finally {
@@ -279,20 +277,31 @@ export default function AddModelPopover({
className="flex flex-col min-h-0 flex-1"
>
<div className="flex-shrink-0">
{!(trigger && initialMode === 'scan') && (
<TabsList className="grid w-full grid-cols-3">
<TabsTrigger value="llm">
<MessageSquareText className="h-4 w-4 mr-1" />
{t('models.chat')}
</TabsTrigger>
<TabsTrigger value="embedding">
<Cpu className="h-4 w-4 mr-1" />
{t('models.embedding')}
</TabsTrigger>
<TabsTrigger value="rerank">
<ArrowUpDown className="h-4 w-4 mr-1" />
{t('models.rerank')}
</TabsTrigger>
{!(trigger && initialMode === 'scan') && visibleTabs.length > 1 && (
<TabsList
className="grid w-full"
style={{
gridTemplateColumns: `repeat(${visibleTabs.length}, minmax(0, 1fr))`,
}}
>
{visibleTabs.includes('llm') && (
<TabsTrigger value="llm">
<MessageSquareText className="h-4 w-4 mr-1" />
{t('models.chat')}
</TabsTrigger>
)}
{visibleTabs.includes('embedding') && (
<TabsTrigger value="embedding">
<Cpu className="h-4 w-4 mr-1" />
{t('models.embedding')}
</TabsTrigger>
)}
{visibleTabs.includes('rerank') && (
<TabsTrigger value="rerank">
<ArrowUpDown className="h-4 w-4 mr-1" />
{t('models.rerank')}
</TabsTrigger>
)}
</TabsList>
)}
</div>
@@ -344,6 +353,24 @@ export default function AddModelPopover({
</div>
)}
{tab === 'llm' && (
<div className="space-y-2">
<Label htmlFor="add-context-length">
{t('models.contextLength')}
</Label>
<Input
id="add-context-length"
type="number"
min={1}
step={1}
inputMode="numeric"
placeholder={t('models.contextLengthPlaceholder')}
value={contextLength}
onChange={(e) => setContextLength(e.target.value)}
/>
</div>
)}
<ExtraArgsEditor
args={extraArgs}
onChange={setExtraArgs}
web/src/app/home/components/models-dialog/components/ModelItem.tsx
+40 -1
View File
@@ -31,6 +31,7 @@ interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onTestModel: (
name: string,
@@ -92,6 +93,11 @@ export default function ModelItem({
const [editAbilities, setEditAbilities] = useState<string[]>(
modelType === 'llm' ? (model as LLMModel).abilities || [] : [],
);
const [editContextLength, setEditContextLength] = useState(
modelType === 'llm' && (model as LLMModel).context_length
? String((model as LLMModel).context_length)
: '',
);
const [editExtraArgs, setEditExtraArgs] = useState<ExtraArg[]>(
convertExtraArgsToArray(model.extra_args),
);
@@ -106,13 +112,27 @@ export default function ModelItem({
setEditAbilities(
modelType === 'llm' ? (model as LLMModel).abilities || [] : [],
);
setEditContextLength(
modelType === 'llm' && (model as LLMModel).context_length
? String((model as LLMModel).context_length)
: '',
);
setEditExtraArgs(convertExtraArgsToArray(model.extra_args));
onResetTestResult();
}
}, [isEditOpen]);
const handleSave = async () => {
await onUpdateModel(editName, editAbilities, editExtraArgs);
const parsedContextLength =
modelType === 'llm' && editContextLength.trim()
? Number(editContextLength.trim())
: null;
await onUpdateModel(
editName,
editAbilities,
editExtraArgs,
parsedContextLength,
);
};
const handleTest = async () => {
@@ -287,6 +307,25 @@ export default function ModelItem({
</div>
)}
{modelType === 'llm' && (
<div className="space-y-2">
<Label htmlFor={`edit-context-length-${model.uuid}`}>
{t('models.contextLength')}
</Label>
<Input
id={`edit-context-length-${model.uuid}`}
type="number"
min={1}
step={1}
inputMode="numeric"
placeholder={t('models.contextLengthPlaceholder')}
value={editContextLength}
disabled={isLangBotModels}
onChange={(e) => setEditContextLength(e.target.value)}
/>
</div>
)}
<ExtraArgsEditor
args={editExtraArgs}
onChange={setEditExtraArgs}
web/src/app/home/components/models-dialog/components/ProviderCard.tsx
+13 -1
View File
@@ -39,6 +39,7 @@ import AddModelPopover from './AddModelPopover';
interface ProviderCardProps {
provider: ModelProvider;
isLangBotModels?: boolean;
supportTypes?: string[];
isExpanded: boolean;
isLoading: boolean;
models?: ProviderModels;
@@ -60,6 +61,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -74,6 +76,7 @@ interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
onCloseDeleteConfirm: () => void;
@@ -99,6 +102,7 @@ function maskApiKey(key: string): string {
export default function ProviderCard({
provider,
isLangBotModels = false,
supportTypes,
isExpanded,
isLoading,
models,
@@ -319,6 +323,7 @@ export default function ProviderCard({
addModelMode === 'manual'
}
initialMode="manual"
supportTypes={supportTypes}
trigger={
<Button
variant="ghost"
@@ -353,6 +358,7 @@ export default function ProviderCard({
addModelMode === 'scan'
}
initialMode="scan"
supportTypes={supportTypes}
trigger={
<Button
variant="ghost"
@@ -405,13 +411,19 @@ export default function ProviderCard({
onOpenDeleteConfirm={onOpenDeleteConfirm}
onCloseDeleteConfirm={onCloseDeleteConfirm}
onDeleteModel={() => onDeleteModel(model.uuid, 'llm')}
onUpdateModel={(name, abilities, extraArgs) =>
onUpdateModel={(
name,
abilities,
extraArgs,
contextLength,
) =>
onUpdateModel(
model.uuid,
'llm',
name,
abilities,
extraArgs,
contextLength,
)
}
onTestModel={(name, abilities, extraArgs) =>
web/src/app/home/components/models-dialog/types.ts
+3
View File
@@ -53,6 +53,7 @@ export interface ModelItemProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onTest: (
name: string,
@@ -89,6 +90,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onScanModels: (modelType?: ModelType) => Promise<ScanModelsResult>;
onAddScannedModels: (
@@ -103,6 +105,7 @@ export interface ProviderCardProps {
name: string,
abilities: string[],
extraArgs: ExtraArg[],
contextLength?: number | null,
) => Promise<void>;
onOpenDeleteConfirm: (modelId: string) => void;
onCloseDeleteConfirm: () => void;