mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-07-14 08:26:07 +00:00
0c405901d2
* feat(vector): add Valkey Search vector database backend Add a new opt-in VectorDatabase backend backed by the Valkey Search module (valkey/valkey-bundle), accessed via the official valkey-glide client's native ft command namespace. - Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search, all 8 metadata filter operators, and pagination with exact totals. - HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is accepted for interface parity but NOT honored (docstring + one-time warning + docs caveat). - Lazy connect so a down Valkey never blocks boot; mandatory client_name=langbot_vector_client; optional auth + TLS (never logged). - Registered via a single elif branch in vector/mgr.py; disabled by default (vdb.use stays chroma) for toC compatibility. - Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM change so no Alembic migration. - Unit tests (fast lane, no server) + slow-gated integration tests (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc. * fix(vector): paginate Valkey Search deletes and guard delete_by_filter Address self-review follow-ups for the Valkey Search VDB backend: - _search_keys now paginates through the full result set in batches of _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key page, so delete_by_file_id / delete_by_filter fully remove files and filters that match more than one page of chunks (no orphaned vectors). - Add unit regression tests for the delete_by_filter mass-deletion guard: a filter referencing only non-indexed fields must skip and return 0 (never fall back to match-all), and a supported filter still deletes matching keys. * refactor(vector): harden Valkey Search backend and add adversarial tests Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend: - Guard the username-without-password credential edge (skip auth + warn instead of building ServerCredentials(password=None, ...), which glide rejects). - Add an async close() teardown that closes the glide client and resets cached state (re-init is safe via the existing None guard). - Hoist 'import json' to module top (was imported inside three methods). - Document the FT TAG literal-brace limitation in _escape_tag (fails closed, never widens). Tests: - Add an adversarial-input integration test proving crafted file_id / query_text cannot break out of or widen a query (fail-closed on braces). - Add unit tests for close() and the credential-build guard. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * fix(vector): make Valkey Search file_id TAG support arbitrary characters Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when backslash-escaped, so a file_id containing those characters previously produced an unparseable query (it failed closed / raised). Percent-encode exactly those FT-unsafe characters (plus '%' for reversibility) in the file_id TAG value, applied identically at write time and query time, so an arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and the stored value is unchanged; the original file_id is always preserved verbatim in metadata_json. Strengthen the adversarial integration test to assert a brace/star-bearing file_id matches and deletes exactly its own row (no widening, no raise), and add unit tests for _encode_file_id and the filter encoding. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * refactor(vector): address Valkey Search review feedback - Add configurable request_timeout (default 5000ms; glide default 250ms is too low for KNN); expose in config.yaml + docs table - Validate embedding dimension consistency in add_embeddings (fail fast on mixed lengths to avoid silent KNN corruption) - Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in the query hot path; also closes the check-then-create TOCTOU window - Pipeline HSETs via a non-atomic Batch instead of N sequential awaits - Extract shared _iter_reply_docs to deduplicate reply parsing between _reply_to_chroma and list_by_filter - Parenthesize multi-condition pre-filters before the => KNN clause - Fail closed when a username is configured without a password - Catch only RequestError on ft.dropindex (let connection/auth errors surface) - Bound the delete_collection SCAN loop with a safety cap - Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown() - Simplify _MATCH_ALL literal; normalize typing to builtin generics * fix(vector/valkey_search): address round-2 review feedback - Serialize lazy client creation with an asyncio.Lock (double-checked) so concurrent first-use callers don't construct and leak duplicate clients. - Make the filter operator chain exhaustive: raise on an unhandled op rather than silently dropping the condition (which could widen delete_by_filter). - Cast numeric range (///) values to float, failing closed on non-numeric input and pre-empting a future NUMERIC-field injection surface. * refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method. Signed-off-by: Daria Korenieva <daric2612@gmail.com> * docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests. Signed-off-by: Daria Korenieva <daric2612@gmail.com> --------- Signed-off-by: Daria Korenieva <daric2612@gmail.com>
340 lines
12 KiB
Python
340 lines
12 KiB
Python
"""Tests for VectorDBManager provider selection logic.
|
|
|
|
Tests the initialization logic that selects the appropriate VDB backend
|
|
based on configuration, without actually creating real VDB instances.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from unittest.mock import MagicMock
|
|
|
|
from tests.utils.import_isolation import isolated_sys_modules
|
|
|
|
|
|
class TestVectorDBManagerInitialization:
|
|
"""Tests for VectorDBManager.initialize provider selection."""
|
|
|
|
def _create_mock_app(self, vdb_config: dict | None):
|
|
"""Create mock app with vdb configuration."""
|
|
mock_app = MagicMock()
|
|
mock_app.instance_config = MagicMock()
|
|
mock_app.instance_config.data = MagicMock()
|
|
mock_app.instance_config.data.get = MagicMock(return_value=vdb_config)
|
|
mock_app.logger = MagicMock()
|
|
mock_app.logger.info = MagicMock()
|
|
mock_app.logger.warning = MagicMock()
|
|
return mock_app
|
|
|
|
def _make_vector_import_mocks(self):
|
|
"""Create mocks for VDB backends to prevent real imports."""
|
|
mocks = {}
|
|
|
|
# Mock core.app to break circular import
|
|
mocks['langbot.pkg.core.app'] = MagicMock()
|
|
|
|
# Mock all VDB backend implementations
|
|
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db', 'valkey_search']:
|
|
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
|
|
|
|
return mocks
|
|
|
|
def test_initialize_no_config_defaults_to_chroma(self):
|
|
"""No vdb config defaults to Chroma."""
|
|
mock_app = self._create_mock_app(None)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
# Create mock Chroma class
|
|
mock_chroma_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
# Import after mocking
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
# Run initialize synchronously for test
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
# Chroma should be instantiated
|
|
mock_chroma_class.assert_called_once_with(mock_app)
|
|
mock_app.logger.warning.assert_called()
|
|
|
|
def test_initialize_chroma_backend(self):
|
|
"""Explicit chroma config uses Chroma backend."""
|
|
vdb_config = {'use': 'chroma'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_chroma_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_chroma_class.assert_called_once_with(mock_app)
|
|
mock_app.logger.info.assert_called()
|
|
|
|
def test_initialize_qdrant_backend(self):
|
|
"""Qdrant config uses Qdrant backend."""
|
|
vdb_config = {'use': 'qdrant'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_qdrant_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.qdrant'].QdrantVectorDatabase = mock_qdrant_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_qdrant_class.assert_called_once_with(mock_app)
|
|
|
|
def test_initialize_seekdb_backend(self):
|
|
"""SeekDB config uses SeekDB backend."""
|
|
vdb_config = {'use': 'seekdb'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_seekdb_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.seekdb'].SeekDBVectorDatabase = mock_seekdb_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_seekdb_class.assert_called_once_with(mock_app)
|
|
|
|
def test_initialize_valkey_search_backend(self):
|
|
"""Valkey Search config uses ValkeySearchVectorDatabase backend."""
|
|
vdb_config = {'use': 'valkey_search'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_valkey_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.valkey_search'].ValkeySearchVectorDatabase = mock_valkey_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_valkey_class.assert_called_once_with(mock_app)
|
|
|
|
def test_initialize_milvus_backend_with_uri(self):
|
|
"""Milvus config with custom URI."""
|
|
vdb_config = {
|
|
'use': 'milvus',
|
|
'milvus': {'uri': 'http://localhost:19530', 'token': 'root:Milvus', 'db_name': 'langbot_db'},
|
|
}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_milvus_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.milvus'].MilvusVectorDatabase = mock_milvus_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_milvus_class.assert_called_once_with(
|
|
mock_app, uri='http://localhost:19530', token='root:Milvus', db_name='langbot_db'
|
|
)
|
|
|
|
def test_initialize_milvus_backend_defaults(self):
|
|
"""Milvus defaults when config not fully specified."""
|
|
vdb_config = {'use': 'milvus'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_milvus_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.milvus'].MilvusVectorDatabase = mock_milvus_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
# Should use default values
|
|
mock_milvus_class.assert_called_once_with(mock_app, uri='./data/milvus.db', token=None, db_name='default')
|
|
|
|
def test_initialize_pgvector_with_connection_string(self):
|
|
"""pgvector with connection string."""
|
|
vdb_config = {'use': 'pgvector', 'pgvector': {'connection_string': 'postgresql://user:pass@host:5432/langbot'}}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_pgvector_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_pgvector_class.assert_called_once_with(
|
|
mock_app, connection_string='postgresql://user:pass@host:5432/langbot'
|
|
)
|
|
|
|
def test_initialize_pgvector_with_individual_params(self):
|
|
"""pgvector with individual connection parameters."""
|
|
vdb_config = {
|
|
'use': 'pgvector',
|
|
'pgvector': {
|
|
'host': 'db.example.com',
|
|
'port': 5433,
|
|
'database': 'vectordb',
|
|
'user': 'admin',
|
|
'password': 'secret',
|
|
},
|
|
}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_pgvector_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_pgvector_class.assert_called_once_with(
|
|
mock_app, host='db.example.com', port=5433, database='vectordb', user='admin', password='secret'
|
|
)
|
|
|
|
def test_initialize_pgvector_defaults(self):
|
|
"""pgvector defaults when no config params."""
|
|
vdb_config = {'use': 'pgvector'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_pgvector_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_pgvector_class.assert_called_once_with(
|
|
mock_app, host='localhost', port=5432, database='langbot', user='postgres', password='postgres'
|
|
)
|
|
|
|
def test_initialize_unknown_backend_defaults_to_chroma(self):
|
|
"""Unknown vdb type defaults to Chroma with warning."""
|
|
vdb_config = {'use': 'unknown_backend'}
|
|
mock_app = self._create_mock_app(vdb_config)
|
|
|
|
mocks = self._make_vector_import_mocks()
|
|
mock_chroma_class = MagicMock()
|
|
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
|
|
import asyncio
|
|
|
|
asyncio.get_event_loop().run_until_complete(mgr.initialize())
|
|
|
|
mock_chroma_class.assert_called_once_with(mock_app)
|
|
mock_app.logger.warning.assert_called()
|
|
# Should warn about no valid backend
|
|
warning_msg = mock_app.logger.warning.call_args[0][0]
|
|
assert 'No valid' in warning_msg or 'defaulting' in warning_msg
|
|
|
|
|
|
class TestVectorDBManagerProxies:
|
|
"""Tests for VectorDBManager proxy methods."""
|
|
|
|
def test_get_supported_search_types_no_vector_db(self):
|
|
"""get_supported_search_types returns vector when no vector_db."""
|
|
mock_app = MagicMock()
|
|
mock_app.instance_config = MagicMock()
|
|
mock_app.instance_config.data = MagicMock()
|
|
mock_app.instance_config.data.get = MagicMock(return_value=None)
|
|
mock_app.logger = MagicMock()
|
|
|
|
mocks = {'langbot.pkg.core.app': MagicMock()}
|
|
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db']:
|
|
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
mgr.vector_db = None # Explicitly None
|
|
|
|
result = mgr.get_supported_search_types()
|
|
assert result == ['vector']
|
|
|
|
def test_get_supported_search_types_with_vector_db(self):
|
|
"""get_supported_search_types delegates to vector_db."""
|
|
mock_app = MagicMock()
|
|
|
|
# Create mock vector_db with supported_search_types
|
|
mock_vector_db = MagicMock()
|
|
mock_vector_db.supported_search_types = MagicMock(
|
|
return_value=[
|
|
MagicMock(value='vector'),
|
|
MagicMock(value='full_text'),
|
|
]
|
|
)
|
|
|
|
mocks = {'langbot.pkg.core.app': MagicMock()}
|
|
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db']:
|
|
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
|
|
|
|
with isolated_sys_modules(mocks):
|
|
from langbot.pkg.vector.mgr import VectorDBManager
|
|
|
|
mgr = VectorDBManager(mock_app)
|
|
mgr.vector_db = mock_vector_db
|
|
|
|
result = mgr.get_supported_search_types()
|
|
assert result == ['vector', 'full_text']
|