Files
LangBot/tests/e2e/utils/config_factory.py
T
Daria Korenieva 0c405901d2 feat(vector): add Valkey Search vector database backend (#2276)
* feat(vector): add Valkey Search vector database backend

Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.

- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
  all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
  accepted for interface parity but NOT honored (docstring + one-time warning +
  docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
  client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
  (vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
  change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
  (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.

* fix(vector): paginate Valkey Search deletes and guard delete_by_filter

Address self-review follow-ups for the Valkey Search VDB backend:

- _search_keys now paginates through the full result set in batches of
  _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
  page, so delete_by_file_id / delete_by_filter fully remove files and
  filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
  a filter referencing only non-indexed fields must skip and return 0
  (never fall back to match-all), and a supported filter still deletes
  matching keys.

* refactor(vector): harden Valkey Search backend and add adversarial tests

Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
  instead of building ServerCredentials(password=None, ...), which glide
  rejects).
- Add an async close() teardown that closes the glide client and resets
  cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
  never widens).

Tests:
- Add an adversarial-input integration test proving crafted file_id /
  query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* fix(vector): make Valkey Search file_id TAG support arbitrary characters

Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.

Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* refactor(vector): address Valkey Search review feedback

- Add configurable request_timeout (default 5000ms; glide default 250ms is
  too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
  mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
  the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
  _reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics

* fix(vector/valkey_search): address round-2 review feedback

- Serialize lazy client creation with an asyncio.Lock (double-checked) so
  concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
  than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
  non-numeric input and pre-empting a future NUMERIC-field injection surface.

* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-07-08 06:59:16 +08:00

191 lines
5.8 KiB
Python

"""E2E test configuration factory.
Generates minimal config.yaml for testing LangBot startup without external dependencies.
"""
from __future__ import annotations
import yaml
from pathlib import Path
def create_minimal_config(tmpdir: Path, port: int = 15300) -> Path:
"""Create minimal config.yaml for E2E testing.
Uses embedded databases (SQLite, Chroma) to avoid external dependencies.
Config is created at tmpdir/data/config.yaml (LangBot expects this location).
"""
# LangBot expects config at data/config.yaml
data_dir = tmpdir / 'data'
data_dir.mkdir(parents=True, exist_ok=True)
config = {
'admins': [],
'api': {
'port': port,
'webhook_prefix': f'http://127.0.0.1:{port}',
'extra_webhook_prefix': '',
},
'command': {
'enable': True,
'prefix': ['!', '!'],
'privilege': {},
},
'concurrency': {
'pipeline': 20,
'session': 1,
},
'proxy': {
'http': '',
'https': '',
},
'system': {
'instance_id': '',
'edition': 'community',
'recovery_key': '',
'allow_modify_login_info': True,
'disabled_adapters': [],
'limitation': {
'max_bots': -1,
'max_pipelines': -1,
'max_extensions': -1,
},
'task_retention': {
'completed_limit': 200,
},
'jwt': {
'expire': 604800,
'secret': 'e2e-test-secret-key',
},
},
'database': {
'use': 'sqlite',
'sqlite': {
'path': str(tmpdir / 'data' / 'langbot.db'),
},
'postgresql': {
'host': '127.0.0.1',
'port': 5432,
'user': 'postgres',
'password': 'postgres',
'database': 'postgres',
},
},
'vdb': {
'use': 'chroma', # Chroma is embedded, no external dependency
'chroma': {
'path': str(tmpdir / 'chroma'),
},
'qdrant': {
'url': '',
'host': 'localhost',
'port': 6333,
'api_key': '',
},
'seekdb': {
'mode': 'embedded',
'path': str(tmpdir / 'seekdb'),
'database': 'langbot',
'host': 'localhost',
'port': 2881,
'user': 'root',
'password': '',
'tenant': '',
},
'milvus': {
'uri': 'http://127.0.0.1:19530',
'token': '',
'db_name': '',
},
'pgvector': {
'host': '127.0.0.1',
'port': 5433,
'database': 'langbot',
'user': 'postgres',
'password': 'postgres',
},
'valkey_search': {
'host': 'localhost',
'port': 6379,
'db': 0,
'password': '',
'username': '',
'tls': False,
'index_algorithm': 'HNSW',
'distance_metric': 'COSINE',
'request_timeout': 5000,
},
},
'storage': {
'use': 'local',
'cleanup': {
'enabled': False, # Disable cleanup for tests
'check_interval_hours': 1,
'uploaded_file_retention_days': 7,
'log_retention_days': 3,
},
'local': {
'path': str(tmpdir / 'storage'),
},
's3': {
'endpoint_url': '',
'access_key_id': '',
'secret_access_key': '',
'region': 'us-east-1',
'bucket': 'langbot-storage',
},
},
'plugin': {
'enable': False, # Disable plugin system for minimal startup
'runtime_ws_url': '',
'enable_marketplace': False,
'display_plugin_debug_url': '',
'binary_storage': {
'max_value_bytes': 10485760,
},
},
'monitoring': {
'auto_cleanup': {
'enabled': False, # Disable cleanup for tests
'retention_days': 30,
'check_interval_hours': 1,
'delete_batch_size': 1000,
},
},
'space': {
'url': 'https://space.langbot.app',
'models_gateway_api_url': 'https://api.langbot.cloud/v1',
'oauth_authorize_url': 'https://space.langbot.app/auth/authorize',
'disable_models_service': True, # Disable external services
'disable_telemetry': True, # Disable telemetry for tests
},
'provider': {}, # Empty providers - minimal startup
'llm': [], # Empty LLM models
}
# Ensure data directory exists (LangBot expects config at data/config.yaml)
data_dir = tmpdir / 'data'
data_dir.mkdir(parents=True, exist_ok=True)
# Write config to data/config.yaml (LangBot's expected location)
config_path = data_dir / 'config.yaml'
with open(config_path, 'w', encoding='utf-8') as f:
yaml.dump(config, f, default_flow_style=False)
return config_path
def create_test_directories(tmpdir: Path) -> dict[str, Path]:
"""Create necessary directories for LangBot testing."""
directories = {
'data': tmpdir / 'data',
'logs': tmpdir / 'logs',
'storage': tmpdir / 'storage',
'chroma': tmpdir / 'chroma',
}
for path in directories.values():
path.mkdir(parents=True, exist_ok=True)
return directories