Files
LangBot/tests/unit_tests/vector/test_mgr.py
T
Daria Korenieva 0c405901d2 feat(vector): add Valkey Search vector database backend (#2276)
* feat(vector): add Valkey Search vector database backend

Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.

- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
  all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
  accepted for interface parity but NOT honored (docstring + one-time warning +
  docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
  client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
  (vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
  change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
  (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.

* fix(vector): paginate Valkey Search deletes and guard delete_by_filter

Address self-review follow-ups for the Valkey Search VDB backend:

- _search_keys now paginates through the full result set in batches of
  _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
  page, so delete_by_file_id / delete_by_filter fully remove files and
  filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
  a filter referencing only non-indexed fields must skip and return 0
  (never fall back to match-all), and a supported filter still deletes
  matching keys.

* refactor(vector): harden Valkey Search backend and add adversarial tests

Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
  instead of building ServerCredentials(password=None, ...), which glide
  rejects).
- Add an async close() teardown that closes the glide client and resets
  cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
  never widens).

Tests:
- Add an adversarial-input integration test proving crafted file_id /
  query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* fix(vector): make Valkey Search file_id TAG support arbitrary characters

Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.

Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* refactor(vector): address Valkey Search review feedback

- Add configurable request_timeout (default 5000ms; glide default 250ms is
  too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
  mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
  the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
  _reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics

* fix(vector/valkey_search): address round-2 review feedback

- Serialize lazy client creation with an asyncio.Lock (double-checked) so
  concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
  than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
  non-numeric input and pre-empting a future NUMERIC-field injection surface.

* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-07-08 06:59:16 +08:00

340 lines
12 KiB
Python

"""Tests for VectorDBManager provider selection logic.
Tests the initialization logic that selects the appropriate VDB backend
based on configuration, without actually creating real VDB instances.
"""
from __future__ import annotations
from unittest.mock import MagicMock
from tests.utils.import_isolation import isolated_sys_modules
class TestVectorDBManagerInitialization:
"""Tests for VectorDBManager.initialize provider selection."""
def _create_mock_app(self, vdb_config: dict | None):
"""Create mock app with vdb configuration."""
mock_app = MagicMock()
mock_app.instance_config = MagicMock()
mock_app.instance_config.data = MagicMock()
mock_app.instance_config.data.get = MagicMock(return_value=vdb_config)
mock_app.logger = MagicMock()
mock_app.logger.info = MagicMock()
mock_app.logger.warning = MagicMock()
return mock_app
def _make_vector_import_mocks(self):
"""Create mocks for VDB backends to prevent real imports."""
mocks = {}
# Mock core.app to break circular import
mocks['langbot.pkg.core.app'] = MagicMock()
# Mock all VDB backend implementations
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db', 'valkey_search']:
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
return mocks
def test_initialize_no_config_defaults_to_chroma(self):
"""No vdb config defaults to Chroma."""
mock_app = self._create_mock_app(None)
mocks = self._make_vector_import_mocks()
# Create mock Chroma class
mock_chroma_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
with isolated_sys_modules(mocks):
# Import after mocking
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
# Run initialize synchronously for test
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
# Chroma should be instantiated
mock_chroma_class.assert_called_once_with(mock_app)
mock_app.logger.warning.assert_called()
def test_initialize_chroma_backend(self):
"""Explicit chroma config uses Chroma backend."""
vdb_config = {'use': 'chroma'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_chroma_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_chroma_class.assert_called_once_with(mock_app)
mock_app.logger.info.assert_called()
def test_initialize_qdrant_backend(self):
"""Qdrant config uses Qdrant backend."""
vdb_config = {'use': 'qdrant'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_qdrant_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.qdrant'].QdrantVectorDatabase = mock_qdrant_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_qdrant_class.assert_called_once_with(mock_app)
def test_initialize_seekdb_backend(self):
"""SeekDB config uses SeekDB backend."""
vdb_config = {'use': 'seekdb'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_seekdb_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.seekdb'].SeekDBVectorDatabase = mock_seekdb_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_seekdb_class.assert_called_once_with(mock_app)
def test_initialize_valkey_search_backend(self):
"""Valkey Search config uses ValkeySearchVectorDatabase backend."""
vdb_config = {'use': 'valkey_search'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_valkey_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.valkey_search'].ValkeySearchVectorDatabase = mock_valkey_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_valkey_class.assert_called_once_with(mock_app)
def test_initialize_milvus_backend_with_uri(self):
"""Milvus config with custom URI."""
vdb_config = {
'use': 'milvus',
'milvus': {'uri': 'http://localhost:19530', 'token': 'root:Milvus', 'db_name': 'langbot_db'},
}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_milvus_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.milvus'].MilvusVectorDatabase = mock_milvus_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_milvus_class.assert_called_once_with(
mock_app, uri='http://localhost:19530', token='root:Milvus', db_name='langbot_db'
)
def test_initialize_milvus_backend_defaults(self):
"""Milvus defaults when config not fully specified."""
vdb_config = {'use': 'milvus'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_milvus_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.milvus'].MilvusVectorDatabase = mock_milvus_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
# Should use default values
mock_milvus_class.assert_called_once_with(mock_app, uri='./data/milvus.db', token=None, db_name='default')
def test_initialize_pgvector_with_connection_string(self):
"""pgvector with connection string."""
vdb_config = {'use': 'pgvector', 'pgvector': {'connection_string': 'postgresql://user:pass@host:5432/langbot'}}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_pgvector_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_pgvector_class.assert_called_once_with(
mock_app, connection_string='postgresql://user:pass@host:5432/langbot'
)
def test_initialize_pgvector_with_individual_params(self):
"""pgvector with individual connection parameters."""
vdb_config = {
'use': 'pgvector',
'pgvector': {
'host': 'db.example.com',
'port': 5433,
'database': 'vectordb',
'user': 'admin',
'password': 'secret',
},
}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_pgvector_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_pgvector_class.assert_called_once_with(
mock_app, host='db.example.com', port=5433, database='vectordb', user='admin', password='secret'
)
def test_initialize_pgvector_defaults(self):
"""pgvector defaults when no config params."""
vdb_config = {'use': 'pgvector'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_pgvector_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.pgvector_db'].PgVectorDatabase = mock_pgvector_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_pgvector_class.assert_called_once_with(
mock_app, host='localhost', port=5432, database='langbot', user='postgres', password='postgres'
)
def test_initialize_unknown_backend_defaults_to_chroma(self):
"""Unknown vdb type defaults to Chroma with warning."""
vdb_config = {'use': 'unknown_backend'}
mock_app = self._create_mock_app(vdb_config)
mocks = self._make_vector_import_mocks()
mock_chroma_class = MagicMock()
mocks['langbot.pkg.vector.vdbs.chroma'].ChromaVectorDatabase = mock_chroma_class
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
import asyncio
asyncio.get_event_loop().run_until_complete(mgr.initialize())
mock_chroma_class.assert_called_once_with(mock_app)
mock_app.logger.warning.assert_called()
# Should warn about no valid backend
warning_msg = mock_app.logger.warning.call_args[0][0]
assert 'No valid' in warning_msg or 'defaulting' in warning_msg
class TestVectorDBManagerProxies:
"""Tests for VectorDBManager proxy methods."""
def test_get_supported_search_types_no_vector_db(self):
"""get_supported_search_types returns vector when no vector_db."""
mock_app = MagicMock()
mock_app.instance_config = MagicMock()
mock_app.instance_config.data = MagicMock()
mock_app.instance_config.data.get = MagicMock(return_value=None)
mock_app.logger = MagicMock()
mocks = {'langbot.pkg.core.app': MagicMock()}
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db']:
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
mgr.vector_db = None # Explicitly None
result = mgr.get_supported_search_types()
assert result == ['vector']
def test_get_supported_search_types_with_vector_db(self):
"""get_supported_search_types delegates to vector_db."""
mock_app = MagicMock()
# Create mock vector_db with supported_search_types
mock_vector_db = MagicMock()
mock_vector_db.supported_search_types = MagicMock(
return_value=[
MagicMock(value='vector'),
MagicMock(value='full_text'),
]
)
mocks = {'langbot.pkg.core.app': MagicMock()}
for backend in ['chroma', 'qdrant', 'seekdb', 'milvus', 'pgvector_db']:
mocks[f'langbot.pkg.vector.vdbs.{backend}'] = MagicMock()
with isolated_sys_modules(mocks):
from langbot.pkg.vector.mgr import VectorDBManager
mgr = VectorDBManager(mock_app)
mgr.vector_db = mock_vector_db
result = mgr.get_supported_search_types()
assert result == ['vector', 'full_text']