Files
LangBot/tests/unit_tests/api/service/test_model_service.py
huanghuoguoguo 17bbc8bf10 Feat/test build (#2174)
* fix(ci): update unit-test workflow paths to match current source layout

Replace stale pkg/** filter with src/langbot/** and add uv.lock.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(tests): update README to reflect current test layout

- Fix stale paths: tests/pipeline → tests/unit_tests/pipeline
- Update CI Python versions: 3.11, 3.12, 3.13
- Add test directory structure for box, config, platform, plugin, provider, storage
- Document pytest markers and uv commands
- Mention planned E2E tests

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add shared test factories package

Create tests/factories/ with reusable test factories:
- FakeApp: mock application with all dependencies
- Message chains: text_chain, mention_chain, image_chain
- Query factories: text_query, group_text_query, command_query, etc.

No test changes - maintains backward compatibility.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add fake provider factory

Add tests/factories/provider.py with:
- FakeProvider: deterministic fake LLM provider
- Error simulation: timeout, auth, rate-limit, malformed
- Request capture for assertions
- fake_model: mock model with attached provider

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add fake platform factory

Add tests/factories/platform.py with:
- FakePlatform: simulated platform adapter
- Inbound message construction: friend/group/image
- Mention-bot flag simulation
- Outbound message capture for assertions
- Streaming output support simulation
- Send failure simulation

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add comprehensive message/query factories

Extend tests/factories/message.py with:
- file_query: file attachment query
- unsupported_query: unknown message segment
- voice_query: audio/voice query
- at_all_query: group @All mention
- query_with_session: query with session object
- query_with_config: query with custom pipeline config

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add fake message flow smoke test

Create tests/smoke/test_fake_message_flow.py:
- TestFakeMessageFlow: factory verification tests
- TestMessageFlowIntegration: minimal flow smoke test
- Tests FakeApp, FakeProvider, FakePlatform, query factories
- Verifies LANGBOT_FAKE_PONG marker response
- Captures outbound messages for assertions

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add developer test-quick command

Add scripts/test-quick.sh and Makefile with:
- test-quick: runs ruff check + unit tests + smoke tests
- No real provider keys or platform accounts required
- Suitable for local branch self-test

Update tests/README.md:
- Document test-quick command
- Document test factories package
- Add smoke tests and factories directory structure

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(test): make test-quick reliable as developer gate

Fixes for D-001验收问题:
1. test-quick.sh: use set -euo pipefail, uv run ruff, no tail pipe
2. Remove unused imports in factories (app.py, platform.py, provider.py)
3. Fix unused variable in smoke test
4. Add noqa: E402 to test_n8nsvapi.py lazy imports
5. Update smoke test docs: "minimal fake flow" not full pipeline

Now test-quick is a reliable gate: lint failures exit 1, test failures propagate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(unit): add preproc and taskmgr unit tests

U-001: Pipeline Preprocessor tests
- Normal text message processing
- Empty message handling
- Image segment with/without vision model
- Model selection and fallback
- Variable extraction

U-004: Core Task Manager tests (pattern-based)
- Task creation and tracking patterns
- Task cancellation patterns
- Scope-based cancellation
- Task type filtering
- Pruning completed tasks
- Wait all tasks

Taskmgr tests use pattern-based approach to avoid circular import
in source code (taskmgr → app → http_controller → migration → taskmgr).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(unit): add config loader unit tests

U-005: Config Loader tests
- Valid YAML config loading
- Valid JSON config loading
- Invalid YAML/JSON error behavior
- Missing config file creation from template
- Template completion for missing keys
- ConfigManager load/dump operations
- Exists check for both YAML and JSON

All tests use tmp_path fixture, no real project config.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(unit): add chat and command handler pattern tests

U-002: Chat Handler tests (pattern-based)
- Normal message event emission pattern
- prevent_default handling
- User message alteration pattern
- Runner selection pattern
- Streaming/non-streaming response patterns
- Exception handling modes (show-error, show-hint, hide)
- Message history update pattern
- Telemetry payload pattern

U-003: Command Handler tests (pattern-based)
- Command parsing and text extraction
- Event creation pattern
- Privilege/admin check pattern
- Command result handling (text, error, image)
- prevent_default handling
- String truncation helper

Uses pattern-based testing to avoid circular import issues in source code.
Direct imports of handler modules trigger circular import chain.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* style: fix unused imports after ruff auto-fix

Remove unused imports in test files:
- test_config_loader.py: remove unused os
- test_taskmgr.py: remove unused Mock
- test_preproc.py: remove unused unsupported_query, image_chain

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(unit): improve taskmgr tests to test real classes

U-004 improved: Tests now import and test actual classes:
- TaskContext: new(), trace(), to_dict(), placeholder()
- TaskWrapper: task creation, context, exception/result capture, cancel, to_dict
- AsyncTaskManager: create_task, create_user_task, cancel_task, cancel_by_scope
- Task pruning behavior

Uses pre-mocking technique:
- Mock langbot.pkg.core.app before import (breaks circular chain)
- Mock langbot.pkg.core.entities with proper Enum

All 24 tests now test real class behavior, not patterns.
taskmgr.py coverage should improve significantly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(test): consolidate FakeApp and add sys.modules isolation utility

- Extract tests/utils/import_isolation.py with isolated_sys_modules context manager
- Extend tests/factories/app.py FakeApp with handler-specific attributes
- Refactor test_chat_handler.py to use centralized FakeApp and cached imports
- Refactor test_command_handler.py with mock_execute_factory fixture
- Refactor test_smoke.py to move import-time sys.modules manipulation into fixture
- Add SQLite migration integration tests (G-002)
- Add HTTP API smoke integration tests (G-005)
- Update CI workflow to call pytest for SQLite migrations (G-004)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add developer quality gate consolidation (G-007)

- Add scripts/test-integration-fast.sh for fast integration tests
- Add scripts/test-coverage.sh with 12% baseline threshold
- Update Makefile with test-integration-fast, test-coverage, test-all-local
- Update CI workflow with integration and coverage jobs
- Add smoke marker to pytest.ini
- Update tests/README.md with quality gate layers documentation
- Add tests/integration/pipeline/ for pipeline stage-chain tests

Quality gate layers:
- Quick: ruff + unit + smoke (~2 min)
- Fast Integration: SQLite/API/Pipeline (~3 min)
- Coverage: 12% threshold gate (~8 min)
- Full Local: all three combined

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): add PostgreSQL migration slow integration tests (G-003)

- Add tests/integration/persistence/test_migrations_postgres.py
- All tests marked with @pytest.mark.slow
- Tests skip when TEST_POSTGRES_URL is not set (no local PostgreSQL)
- Database isolation via clean_tables and clean_alembic_version fixtures
- Update CI workflow to use pytest instead of inline Python script
- Remove TODO(G-003) comment
- Update tests/README.md with PostgreSQL test documentation

Covered scenarios:
- Baseline stamp sets revision
- Upgrade from baseline to head
- Upgrade idempotent
- Get current on unstamped DB returns None

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(test): Phase 1.5 coverage expansion - COV-001 to COV-013

Coverage baseline raised from 13.65% to 26% (+12.35%)
Gate raised from 12% to 18%

Tasks completed:
- COV-001: Command system unit tests (100% coverage)
- COV-002: API service unit tests batch 1 (user/apikey/model/provider)
- COV-003: Provider model manager unit tests
- COV-004: Pipeline remaining stage tests (aggregator/cntfilter/longtext/msgtrun)
- COV-005: Storage and utils coverage pass
- COV-006: Gate ratchet 12%→15%
- COV-007: Gate ratchet 15%→18%
- COV-008: API service batch 2 (bot/pipeline/webhook/space/maintenance/mcp)
- COV-009: Blocked - API controller circular import issue documented
- COV-010: Plugin runtime unit tests (+0.08%)
- COV-011: RAG and vector unit tests (+0.68%)
- COV-012: Core boot and migration unit tests
- COV-013: Provider requester logic unit tests (+0.62%)

Key additions:
- tests/utils/import_isolation.py: sys.modules isolation for circular imports
- Provider requester mock tests: proved HTTP-dependent code can be tested locally
- Vector filter utilities: 100% coverage on pure functions
- API services: fake persistence pattern for unit testing

Blocked issue COV-009 documented in langbot-test-plan/1.5/issues/

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(phase1): add unit tests for telemetry, plugin, rag, persistence

Add initial unit tests for Phase 1 of test coverage improvement:
- telemetry: test initialization, payload sanitization, early returns (14.3% → 62.9%)
- plugin: test _parse_plugin_id static method
- rag: test _to_i18n_name static method
- persistence: test serialize_model with datetime handling

Overall core coverage: 41.9% → 42.2%

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(phase2): add unit tests for core, persistence, plugin, utils

- Add test_handler_helpers.py for plugin handler helpers (7 tests)
- Add test_mgr_methods.py for persistence manager (5 tests)
- Add test_app_config_validation.py for core app config (12 tests)
- Add test_knowledge_service.py for API knowledge service (22 tests)
- Add test_kbmgr.py for RAG knowledge base manager (39 tests)
- Add test_survey_manager.py for survey manager (22 tests)
- Add test_connector_methods.py for plugin connector (24 tests)
- Add test_funcschema.py for utils function schema (9 tests)
- Add test_platform.py for utils platform detection (7 tests)
- Add test_extract_deps.py for plugin deps extraction (7 tests)
- Add test_database_decorator.py for persistence decorator (7 tests)
- Add test_load_config.py for core config loading (19 tests)
- Add COVERAGE_EXCLUSIONS.md documenting external adapter exclusions
- Fix test_chat_session_limit.py path for portability

Coverage: core 28% → 30%, persistence 24% → 24.4%, plugin 27% → 28%
Total: 1082 tests passed, core module coverage 45.5%

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(integration): add API controller integration tests

- Add test_pipelines.py (10 tests) covering pipelines CRUD operations
  - GET/POST/PUT/DELETE on /api/v1/pipelines
  - Extensions endpoint
  - Metadata endpoint
  - Coverage: pipelines controller 27% → 80%

- Add test_providers.py (10 tests) covering provider/model management
  - Provider CRUD with model counts
  - LLM model CRUD
  - Coverage: providers controller 23% → 81%, models 29% → 45%

Tests use Quart TestClient with mocked services for real HTTP behavior
without external dependencies.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(integration): add knowledge, bots, and model endpoints tests

- Add test_knowledge.py (10 tests) covering knowledge base management
  - CRUD operations on /api/v1/knowledge/bases
  - Files management endpoints
  - Retrieve endpoint with validation
  - Coverage: knowledge/base.py 26% → 91%

- Add test_bots.py (9 tests) covering bot management
  - CRUD operations on /api/v1/platform/bots
  - Logs endpoint
  - Send message endpoint with validation
  - Coverage: platform/bots.py 24% → 87%

- Extend test_providers.py (+4 tests) for embedding/rerank models
  - Embedding models CRUD
  - Rerank models CRUD
  - Coverage: provider/models.py 29% → 60%

Total integration tests: 53 (smoke 12 + pipelines 10 + providers 14 + knowledge 10 + bots 9)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(integration): add embed and monitoring endpoint tests

Add integration tests for embed widget and monitoring API endpoints:
- test_embed.py: 15 tests for widget.js, logo, turnstile, messages, reset, feedback
- test_monitoring.py: 15 tests for overview, messages, llm-calls, sessions, errors, export

Coverage improvements:
- embed.py: 17% → 56%
- monitoring.py: 17% → 93%

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(e2e): add minimal startup E2E tests

Add E2E tests for LangBot startup flow:
- tests/e2e/utils/config_factory.py: minimal config generation
- tests/e2e/utils/process_manager.py: LangBot subprocess management
- tests/e2e/conftest.py: E2E fixtures (session-scoped process)
- tests/e2e/test_startup.py: 12 tests for startup verification

Tests verify:
- boot.py + stages execution
- database initialization (SQLite)
- API availability
- migrations applied

Uses embedded databases (SQLite, Chroma) - no external dependencies.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(quality): fix fake tests and add missing coverage

P0 fixes:
- telemetry: rewrite fake tests with real behavior verification (25 tests)
- config: delete copied-source tests, use proper imports (2 deleted)
- persistence: fix try-except pass to verify specific errors

P1 fixes:
- pipeline: add real FixedWindowAlgo tests instead of mocks (12 tests)
- provider: add SessionManager and ToolManager tests (25 tests)
- storage: add S3StorageProvider tests with moto mock (16 tests)
- plugin: add handler action tests for setting inheritance (15 tests)
- rag: add file storage and ZIP processing tests (21 tests)
- vector: add VDB filter conversion tests (30 tests)

P2 fixes:
- pipeline/msgtrun: strengthen assertions for exact message count
- api: add response structure validation in integration tests

New test files:
- provider/test_session_manager.py
- provider/test_tool_manager.py
- storage/test_s3storage.py
- plugin/test_handler_actions.py
- rag/test_file_storage.py
- vector/test_vdb_filter_conversion.py

Source code bugs documented:
- provider: TokenManager.next_token() ZeroDivisionError
- telemetry: send_tasks class variable shared state
- command: empty command IndexError, unused parameters
- utils: funcschema KeyError
- entity: vector.py independent declarative_base

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs(test): update coverage stats and test structure

- Update coverage from 22% to 30%
- Add new test files to structure:
  - provider: session_manager, tool_manager
  - storage: s3storage
  - plugin: handler_actions
  - rag: file_storage
  - vector: vdb_filter_conversion
  - telemetry: rewritten tests
- Update module coverage percentages

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: add 105 new unit tests for untested core functionality

Add comprehensive tests for B-class issues (core functionality untested):

Pipeline:
- test_pool.py: QueryPool ID generation, caching, async context (12 tests)
- test_ratelimit.py: Fixed timing-sensitive test tolerance
- test_pipelinemgr.py: Use real Pydantic StageProcessResult instead of Mock

Utils:
- test_version.py: Version comparison functions (20 tests)
- test_logcache.py: Log page management and retrieval (18 tests)
- test_httpclient.py: HTTP session pool management (10 tests)
- test_proxy.py: Proxy configuration from env and config (10 tests)
- test_image.py: URL parsing and base64 extraction (12 tests)
- test_pkgmgr.py: Pip command generation (8 tests)

Discover:
- test_engine.py: I18nString, Metadata, Component manifest (15 tests)

Test count: 1193 → 1298 (+105 tests)

Note: Some B-class issues cannot be tested due to circular import bugs
filed as GitHub issues #2175 (pipeline) and #2176 (persistence).

* test: tighten phase 1 coverage contracts

* test: align ci integration isolation

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 12:05:54 +08:00

964 lines
32 KiB
Python

"""
Unit tests for LLMModelsService, EmbeddingModelsService, and RerankModelsService.
Tests model management operations including:
- Model CRUD operations
- Model with provider info
- Provider auto-creation on model create/update
- Runtime model loading/unloading
- Model deletion
Source: src/langbot/pkg/api/http/service/model.py
"""
from __future__ import annotations
import pytest
from unittest.mock import AsyncMock, Mock
from types import SimpleNamespace
from langbot.pkg.api.http.service.model import (
LLMModelsService,
EmbeddingModelsService,
RerankModelsService,
_parse_provider_api_keys,
_runtime_model_data,
)
from langbot.pkg.entity.persistence.model import LLMModel, EmbeddingModel, RerankModel, ModelProvider
pytestmark = pytest.mark.asyncio
def _create_mock_llm_model(
model_uuid: str = 'llm-uuid',
name: str = 'Test LLM',
provider_uuid: str = 'provider-uuid',
abilities: list = None,
extra_args: dict = None,
) -> Mock:
"""Helper to create mock LLMModel entity."""
model = Mock(spec=LLMModel)
model.uuid = model_uuid
model.name = name
model.provider_uuid = provider_uuid
model.abilities = abilities or []
model.extra_args = extra_args or {}
return model
def _create_mock_embedding_model(
model_uuid: str = 'embedding-uuid',
name: str = 'Test Embedding',
provider_uuid: str = 'provider-uuid',
) -> Mock:
"""Helper to create mock EmbeddingModel entity."""
model = Mock(spec=EmbeddingModel)
model.uuid = model_uuid
model.name = name
model.provider_uuid = provider_uuid
model.extra_args = {}
return model
def _create_mock_rerank_model(
model_uuid: str = 'rerank-uuid',
name: str = 'Test Rerank',
provider_uuid: str = 'provider-uuid',
) -> Mock:
"""Helper to create mock RerankModel entity."""
model = Mock(spec=RerankModel)
model.uuid = model_uuid
model.name = name
model.provider_uuid = provider_uuid
model.extra_args = {}
return model
def _create_mock_provider(
provider_uuid: str = 'provider-uuid',
name: str = 'Test Provider',
api_keys: list = None,
) -> Mock:
"""Helper to create mock ModelProvider entity."""
provider = Mock(spec=ModelProvider)
provider.uuid = provider_uuid
provider.name = name
provider.requester = 'openai'
provider.base_url = 'https://api.openai.com'
provider.api_keys = api_keys or ['key']
return provider
def _create_mock_result(items: list = None, first_item=None):
"""Create mock result object for persistence queries."""
result = Mock()
result.all = Mock(return_value=items or [])
result.first = Mock(return_value=first_item)
return result
class TestParseProviderApiKeys:
"""Tests for _parse_provider_api_keys helper function."""
def test_parse_valid_json_string(self):
"""Parses valid JSON string to list."""
provider_dict = {'api_keys': '["key1", "key2"]'}
result = _parse_provider_api_keys(provider_dict)
assert result['api_keys'] == ['key1', 'key2']
def test_parse_invalid_json_returns_empty(self):
"""Returns empty list for invalid JSON."""
provider_dict = {'api_keys': 'invalid json'}
result = _parse_provider_api_keys(provider_dict)
assert result['api_keys'] == []
def test_parse_already_list(self):
"""Returns unchanged if already a list."""
provider_dict = {'api_keys': ['key1', 'key2']}
result = _parse_provider_api_keys(provider_dict)
assert result['api_keys'] == ['key1', 'key2']
def test_parse_missing_key(self):
"""Handles missing api_keys key."""
provider_dict = {'name': 'Provider'}
result = _parse_provider_api_keys(provider_dict)
assert 'api_keys' not in result
class TestRuntimeModelData:
"""Tests for _runtime_model_data helper function."""
def test_runtime_data_preserves_uuid(self):
"""Preserves UUID in runtime data."""
update_payload = {'name': 'Updated', 'provider_uuid': 'provider'}
result = _runtime_model_data('model-uuid', update_payload)
assert result['uuid'] == 'model-uuid'
assert result['name'] == 'Updated'
def test_runtime_data_copies_all_fields(self):
"""Copies all fields from payload."""
update_payload = {
'name': 'Model',
'provider_uuid': 'provider',
'abilities': ['vision'],
'extra_args': {'temp': 0.7},
}
result = _runtime_model_data('uuid', update_payload)
assert result['abilities'] == ['vision']
assert result['extra_args'] == {'temp': 0.7}
class TestLLMModelsServiceGetLLMModels:
"""Tests for LLMModelsService.get_llm_models method."""
async def test_get_llm_models_empty_list(self):
"""Returns empty list when no models exist."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([])
mock_provider_result = _create_mock_result([])
call_count = 0
async def mock_execute(query):
return mock_result if call_count == 0 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
ap.persistence_mgr.serialize_model = Mock(
side_effect=lambda model_cls, entity: {
'uuid': entity.uuid,
'name': entity.name,
'provider_uuid': entity.provider_uuid,
}
)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_models()
# Verify
assert result == []
async def test_get_llm_models_with_provider_info(self):
"""Returns models with provider info."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_llm_model()
provider = _create_mock_provider()
mock_model_result = _create_mock_result([model])
mock_provider_result = _create_mock_result([provider])
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
side_effect=lambda model_cls, entity: {
'uuid': entity.uuid,
'name': entity.name,
'provider_uuid': entity.provider_uuid if hasattr(entity, 'provider_uuid') else None,
'api_keys': entity.api_keys if hasattr(entity, 'api_keys') else None,
}
)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_models()
# Verify
assert len(result) == 1
assert result[0]['name'] == 'Test LLM'
async def test_get_llm_models_hide_secret_keys(self):
"""Hides secret API keys when include_secret=False."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_llm_model()
provider = _create_mock_provider(api_keys=['secret-key-1', 'secret-key-2'])
mock_model_result = _create_mock_result([model])
mock_provider_result = _create_mock_result([provider])
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
side_effect=lambda model_cls, entity: {
'uuid': entity.uuid,
'name': entity.name,
'provider_uuid': entity.provider_uuid if hasattr(entity, 'provider_uuid') else None,
'api_keys': entity.api_keys if hasattr(entity, 'api_keys') else None,
}
)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_models(include_secret=False)
# Verify - keys should be masked
assert result[0]['provider']['api_keys'] == ['***', '***']
class TestLLMModelsServiceGetLLMModel:
"""Tests for LLMModelsService.get_llm_model method."""
async def test_get_llm_model_found(self):
"""Returns model when found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_llm_model(model_uuid='found-uuid')
provider = _create_mock_provider()
mock_model_result = _create_mock_result([], first_item=model)
mock_provider_result = _create_mock_result([], first_item=provider)
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
return_value={
'uuid': 'found-uuid',
'name': 'Test LLM',
'provider_uuid': 'provider-uuid',
'provider': {'uuid': 'provider-uuid', 'api_keys': ['key']},
}
)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_model('found-uuid')
# Verify
assert result is not None
assert result['uuid'] == 'found-uuid'
async def test_get_llm_model_not_found(self):
"""Returns None when model not found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([], first_item=None)
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_model('nonexistent-uuid')
# Verify
assert result is None
class TestLLMModelsServiceGetLLMModelsByProvider:
"""Tests for LLMModelsService.get_llm_models_by_provider method."""
async def test_get_models_by_provider_uuid(self):
"""Returns models for specific provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model1 = _create_mock_llm_model(model_uuid='model-1', provider_uuid='target-provider')
model2 = _create_mock_llm_model(model_uuid='model-2', provider_uuid='target-provider')
mock_result = _create_mock_result([model1, model2])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
ap.persistence_mgr.serialize_model = Mock(
return_value={'uuid': 'model-1', 'name': 'Model 1'}
)
service = LLMModelsService(ap)
# Execute
result = await service.get_llm_models_by_provider('target-provider')
# Verify
assert len(result) == 2
class TestLLMModelsServiceCreateLLMModel:
"""Tests for LLMModelsService.create_llm_model method."""
async def test_create_llm_model_generates_uuid(self):
"""Creates LLM model with generated UUID."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {'provider-uuid': Mock()}
ap.model_mgr.llm_models = []
ap.model_mgr.load_llm_model_with_provider = AsyncMock(return_value=Mock())
ap.pipeline_service = SimpleNamespace()
ap.pipeline_service.update_pipeline = AsyncMock()
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = LLMModelsService(ap)
# Execute
model_uuid = await service.create_llm_model({
'name': 'New LLM',
'provider_uuid': 'provider-uuid',
'abilities': [],
'extra_args': {},
})
# Verify
assert model_uuid is not None
assert len(model_uuid) == 36 # UUID format
async def test_create_llm_model_preserve_uuid(self):
"""Creates LLM model preserving provided UUID."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {'provider-uuid': Mock()}
ap.model_mgr.llm_models = []
ap.model_mgr.load_llm_model_with_provider = AsyncMock(return_value=Mock())
ap.pipeline_service = SimpleNamespace()
ap.pipeline_service.update_pipeline = AsyncMock()
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = LLMModelsService(ap)
# Execute
model_uuid = await service.create_llm_model({
'uuid': 'preserved-uuid',
'name': 'Preserved UUID Model',
'provider_uuid': 'provider-uuid',
'abilities': [],
'extra_args': {},
}, preserve_uuid=True)
# Verify
assert model_uuid == 'preserved-uuid'
async def test_create_llm_model_provider_not_found_raises_error(self):
"""Raises Exception when provider not found in runtime."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {} # Empty - no provider
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = LLMModelsService(ap)
# Execute & Verify
with pytest.raises(Exception, match='provider not found'):
await service.create_llm_model({
'name': 'No Provider Model',
'provider_uuid': 'nonexistent-provider',
'abilities': [],
'extra_args': {},
})
async def test_create_llm_model_with_provider_data(self):
"""Creates provider when provider data provided."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {}
ap.model_mgr.llm_models = []
ap.model_mgr.load_llm_model_with_provider = AsyncMock(return_value=Mock())
ap.provider_service = SimpleNamespace()
ap.provider_service.find_or_create_provider = AsyncMock(return_value='new-provider-uuid')
ap.pipeline_service = SimpleNamespace()
ap.pipeline_service.update_pipeline = AsyncMock()
# Create runtime provider
runtime_provider = Mock()
ap.model_mgr.provider_dict['new-provider-uuid'] = runtime_provider
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = LLMModelsService(ap)
# Execute - with provider data (no UUID)
result_uuid = await service.create_llm_model({
'name': 'Model with New Provider',
'provider': {
'requester': 'openai',
'base_url': 'https://api.openai.com',
'api_keys': ['key'],
},
'abilities': [],
'extra_args': {},
})
# Verify - provider_service was called and UUID generated
ap.provider_service.find_or_create_provider.assert_called_once()
assert result_uuid is not None
class TestLLMModelsServiceUpdateLLMModel:
"""Tests for LLMModelsService.update_llm_model method."""
async def test_update_llm_model_removes_uuid_from_data(self):
"""Removes uuid from update data before persisting."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {'provider-uuid': Mock()}
ap.model_mgr.llm_models = []
ap.model_mgr.remove_llm_model = AsyncMock()
ap.model_mgr.load_llm_model_with_provider = AsyncMock(return_value=Mock())
ap.persistence_mgr.execute_async = AsyncMock()
service = LLMModelsService(ap)
# Execute
await service.update_llm_model('existing-uuid', {
'uuid': 'should-be-removed',
'name': 'Updated Name',
'provider_uuid': 'provider-uuid',
})
# Verify - remove and load called
ap.model_mgr.remove_llm_model.assert_called_once_with('existing-uuid')
async def test_update_llm_model_provider_not_found_raises_error(self):
"""Raises Exception when provider not found after update."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {} # Empty
ap.model_mgr.remove_llm_model = AsyncMock()
ap.persistence_mgr.execute_async = AsyncMock()
service = LLMModelsService(ap)
# Execute & Verify
with pytest.raises(Exception, match='provider not found'):
await service.update_llm_model('model-uuid', {
'name': 'Update',
'provider_uuid': 'nonexistent-provider',
})
class TestLLMModelsServiceDeleteLLMModel:
"""Tests for LLMModelsService.delete_llm_model method."""
async def test_delete_llm_model_success(self):
"""Deletes LLM model successfully."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.remove_llm_model = AsyncMock()
ap.persistence_mgr.execute_async = AsyncMock()
service = LLMModelsService(ap)
# Execute
await service.delete_llm_model('delete-uuid')
# Verify
ap.persistence_mgr.execute_async.assert_called_once()
ap.model_mgr.remove_llm_model.assert_called_once_with('delete-uuid')
class TestEmbeddingModelsServiceGetEmbeddingModels:
"""Tests for EmbeddingModelsService.get_embedding_models method."""
async def test_get_embedding_models_empty_list(self):
"""Returns empty list when no models exist."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
ap.persistence_mgr.serialize_model = Mock(
return_value={'uuid': 'embedding-uuid', 'name': 'Test'}
)
service = EmbeddingModelsService(ap)
# Execute
result = await service.get_embedding_models()
# Verify
assert result == []
async def test_get_embedding_models_with_provider(self):
"""Returns embedding models with provider info."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_embedding_model()
provider = _create_mock_provider()
mock_model_result = _create_mock_result([model])
mock_provider_result = _create_mock_result([provider])
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
side_effect=lambda model_cls, entity: {
'uuid': entity.uuid,
'name': entity.name,
'provider_uuid': getattr(entity, 'provider_uuid', None),
'api_keys': getattr(entity, 'api_keys', ['key']),
}
)
service = EmbeddingModelsService(ap)
# Execute
result = await service.get_embedding_models()
# Verify
assert len(result) == 1
class TestEmbeddingModelsServiceGetEmbeddingModel:
"""Tests for EmbeddingModelsService.get_embedding_model method."""
async def test_get_embedding_model_found(self):
"""Returns embedding model when found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_embedding_model(model_uuid='found-embedding')
provider = _create_mock_provider()
mock_model_result = _create_mock_result([], first_item=model)
mock_provider_result = _create_mock_result([], first_item=provider)
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
return_value={
'uuid': 'found-embedding',
'name': 'Found Embedding',
'provider': {'uuid': 'provider-uuid'},
}
)
service = EmbeddingModelsService(ap)
# Execute
result = await service.get_embedding_model('found-embedding')
# Verify
assert result is not None
async def test_get_embedding_model_not_found(self):
"""Returns None when model not found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([], first_item=None)
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = EmbeddingModelsService(ap)
# Execute
result = await service.get_embedding_model('nonexistent-embedding')
# Verify
assert result is None
class TestEmbeddingModelsServiceCreateEmbeddingModel:
"""Tests for EmbeddingModelsService.create_embedding_model method."""
async def test_create_embedding_model_success(self):
"""Creates embedding model successfully."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {'provider-uuid': Mock()}
ap.model_mgr.embedding_models = []
ap.model_mgr.load_embedding_model_with_provider = AsyncMock(return_value=Mock())
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = EmbeddingModelsService(ap)
# Execute
model_uuid = await service.create_embedding_model({
'name': 'New Embedding',
'provider_uuid': 'provider-uuid',
'extra_args': {},
})
# Verify
assert model_uuid is not None
assert len(model_uuid) == 36
async def test_create_embedding_model_provider_not_found_raises(self):
"""Raises Exception when provider not found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {} # Empty
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = EmbeddingModelsService(ap)
# Execute & Verify
with pytest.raises(Exception, match='provider not found'):
await service.create_embedding_model({
'name': 'No Provider Embedding',
'provider_uuid': 'nonexistent',
'extra_args': {},
})
class TestEmbeddingModelsServiceDeleteEmbeddingModel:
"""Tests for EmbeddingModelsService.delete_embedding_model method."""
async def test_delete_embedding_model_success(self):
"""Deletes embedding model successfully."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.remove_embedding_model = AsyncMock()
ap.persistence_mgr.execute_async = AsyncMock()
service = EmbeddingModelsService(ap)
# Execute
await service.delete_embedding_model('delete-embedding-uuid')
# Verify
ap.model_mgr.remove_embedding_model.assert_called_once()
class TestRerankModelsServiceGetRerankModels:
"""Tests for RerankModelsService.get_rerank_models method."""
async def test_get_rerank_models_empty_list(self):
"""Returns empty list when no models exist."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = RerankModelsService(ap)
# Execute
result = await service.get_rerank_models()
# Verify
assert result == []
async def test_get_rerank_models_with_provider(self):
"""Returns rerank models with provider info."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_rerank_model()
provider = _create_mock_provider()
mock_model_result = _create_mock_result([model])
mock_provider_result = _create_mock_result([provider])
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
side_effect=lambda model_cls, entity: {
'uuid': entity.uuid,
'name': entity.name,
'provider_uuid': getattr(entity, 'provider_uuid', None),
'api_keys': getattr(entity, 'api_keys', ['key']),
}
)
service = RerankModelsService(ap)
# Execute
result = await service.get_rerank_models()
# Verify
assert len(result) == 1
class TestRerankModelsServiceGetRerankModel:
"""Tests for RerankModelsService.get_rerank_model method."""
async def test_get_rerank_model_found(self):
"""Returns rerank model when found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model = _create_mock_rerank_model(model_uuid='found-rerank')
provider = _create_mock_provider()
mock_model_result = _create_mock_result([], first_item=model)
mock_provider_result = _create_mock_result([], first_item=provider)
call_count = 0
async def mock_execute(query):
nonlocal call_count
call_count += 1
return mock_model_result if call_count == 1 else mock_provider_result
ap.persistence_mgr.execute_async = AsyncMock(side_effect=mock_execute)
ap.persistence_mgr.serialize_model = Mock(
return_value={
'uuid': 'found-rerank',
'name': 'Found Rerank',
'provider': {'uuid': 'provider-uuid'},
}
)
service = RerankModelsService(ap)
# Execute
result = await service.get_rerank_model('found-rerank')
# Verify
assert result is not None
async def test_get_rerank_model_not_found(self):
"""Returns None when model not found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
mock_result = _create_mock_result([], first_item=None)
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = RerankModelsService(ap)
# Execute
result = await service.get_rerank_model('nonexistent-rerank')
# Verify
assert result is None
class TestRerankModelsServiceCreateRerankModel:
"""Tests for RerankModelsService.create_rerank_model method."""
async def test_create_rerank_model_success(self):
"""Creates rerank model successfully."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {'provider-uuid': Mock()}
ap.model_mgr.rerank_models = []
ap.model_mgr.load_rerank_model_with_provider = AsyncMock(return_value=Mock())
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = RerankModelsService(ap)
# Execute
model_uuid = await service.create_rerank_model({
'name': 'New Rerank',
'provider_uuid': 'provider-uuid',
'extra_args': {},
})
# Verify
assert model_uuid is not None
async def test_create_rerank_model_provider_not_found_raises(self):
"""Raises Exception when provider not found."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.provider_dict = {}
mock_result = _create_mock_result([])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
service = RerankModelsService(ap)
# Execute & Verify
with pytest.raises(Exception, match='provider not found'):
await service.create_rerank_model({
'name': 'No Provider Rerank',
'provider_uuid': 'nonexistent',
'extra_args': {},
})
class TestRerankModelsServiceDeleteRerankModel:
"""Tests for RerankModelsService.delete_rerank_model method."""
async def test_delete_rerank_model_success(self):
"""Deletes rerank model successfully."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
ap.model_mgr = SimpleNamespace()
ap.model_mgr.remove_rerank_model = AsyncMock()
ap.persistence_mgr.execute_async = AsyncMock()
service = RerankModelsService(ap)
# Execute
await service.delete_rerank_model('delete-rerank-uuid')
# Verify
ap.model_mgr.remove_rerank_model.assert_called_once()
class TestEmbeddingModelsServiceGetEmbeddingModelsByProvider:
"""Tests for EmbeddingModelsService.get_embedding_models_by_provider method."""
async def test_get_embedding_models_by_provider_uuid(self):
"""Returns embedding models for specific provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model1 = _create_mock_embedding_model(model_uuid='emb-1', provider_uuid='provider-uuid')
model2 = _create_mock_embedding_model(model_uuid='emb-2', provider_uuid='provider-uuid')
mock_result = _create_mock_result([model1, model2])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
ap.persistence_mgr.serialize_model = Mock(
return_value={'uuid': 'emb-1', 'name': 'Embedding 1'}
)
service = EmbeddingModelsService(ap)
# Execute
result = await service.get_embedding_models_by_provider('provider-uuid')
# Verify
assert len(result) == 2
class TestRerankModelsServiceGetRerankModelsByProvider:
"""Tests for RerankModelsService.get_rerank_models_by_provider method."""
async def test_get_rerank_models_by_provider_uuid(self):
"""Returns rerank models for specific provider."""
# Setup
ap = SimpleNamespace()
ap.persistence_mgr = SimpleNamespace()
model1 = _create_mock_rerank_model(model_uuid='rerank-1', provider_uuid='provider-uuid')
model2 = _create_mock_rerank_model(model_uuid='rerank-2', provider_uuid='provider-uuid')
mock_result = _create_mock_result([model1, model2])
ap.persistence_mgr.execute_async = AsyncMock(return_value=mock_result)
ap.persistence_mgr.serialize_model = Mock(
return_value={'uuid': 'rerank-1', 'name': 'Rerank 1'}
)
service = RerankModelsService(ap)
# Execute
result = await service.get_rerank_models_by_provider('provider-uuid')
# Verify
assert len(result) == 2