xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-07-14 08:26:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
0c405901d27ae7ad593030745f9dcd50a4711aeb
LangBot/tests/unit_tests/COVERAGE_EXCLUSIONS.md
T
Daria Korenieva 0c405901d2 feat(vector): add Valkey Search vector database backend (#2276) 
* feat(vector): add Valkey Search vector database backend

Add a new opt-in VectorDatabase backend backed by the Valkey Search module
(valkey/valkey-bundle), accessed via the official valkey-glide client's native
ft command namespace.

- Implements the full VectorDatabase ABC: VECTOR, FULL_TEXT and HYBRID search,
  all 8 metadata filter operators, and pagination with exact totals.
- HYBRID uses filter-then-KNN (no app-side weighted fusion); vector_weight is
  accepted for interface parity but NOT honored (docstring + one-time warning +
  docs caveat).
- Lazy connect so a down Valkey never blocks boot; mandatory
  client_name=langbot_vector_client; optional auth + TLS (never logged).
- Registered via a single elif branch in vector/mgr.py; disabled by default
  (vdb.use stays chroma) for toC compatibility.
- Adds valkey-glide>=2.4.1,<3.0.0; no protobuf/pydantic downgrade; no ORM
  change so no Alembic migration.
- Unit tests (fast lane, no server) + slow-gated integration tests
  (TEST_VALKEY_URL, valkey/valkey-bundle:9.1.0) + integration doc.

* fix(vector): paginate Valkey Search deletes and guard delete_by_filter

Address self-review follow-ups for the Valkey Search VDB backend:

- _search_keys now paginates through the full result set in batches of
  _DELETE_SCAN_BATCH instead of capping at a single hard-coded 10000-key
  page, so delete_by_file_id / delete_by_filter fully remove files and
  filters that match more than one page of chunks (no orphaned vectors).
- Add unit regression tests for the delete_by_filter mass-deletion guard:
  a filter referencing only non-indexed fields must skip and return 0
  (never fall back to match-all), and a supported filter still deletes
  matching keys.

* refactor(vector): harden Valkey Search backend and add adversarial tests

Address the self-review NICE-TO-HAVE items for the Valkey Search VDB backend:
- Guard the username-without-password credential edge (skip auth + warn
  instead of building ServerCredentials(password=None, ...), which glide
  rejects).
- Add an async close() teardown that closes the glide client and resets
  cached state (re-init is safe via the existing None guard).
- Hoist 'import json' to module top (was imported inside three methods).
- Document the FT TAG literal-brace limitation in _escape_tag (fails closed,
  never widens).

Tests:
- Add an adversarial-input integration test proving crafted file_id /
  query_text cannot break out of or widen a query (fail-closed on braces).
- Add unit tests for close() and the credential-build guard.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* fix(vector): make Valkey Search file_id TAG support arbitrary characters

Valkey Search's FT TAG query parser cannot handle '{', '}' or '*' even when
backslash-escaped, so a file_id containing those characters previously
produced an unparseable query (it failed closed / raised). Percent-encode
exactly those FT-unsafe characters (plus '%' for reversibility) in the
file_id TAG value, applied identically at write time and query time, so an
arbitrary file_id round-trips. For normal UUID/hash ids this is a no-op and
the stored value is unchanged; the original file_id is always preserved
verbatim in metadata_json.

Strengthen the adversarial integration test to assert a brace/star-bearing
file_id matches and deletes exactly its own row (no widening, no raise), and
add unit tests for _encode_file_id and the filter encoding.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* refactor(vector): address Valkey Search review feedback

- Add configurable request_timeout (default 5000ms; glide default 250ms is
  too low for KNN); expose in config.yaml + docs table
- Validate embedding dimension consistency in add_embeddings (fail fast on
  mixed lengths to avoid silent KNN corruption)
- Use ft.info (O(1)) instead of ft.list (O(n)) for index existence checks in
  the query hot path; also closes the check-then-create TOCTOU window
- Pipeline HSETs via a non-atomic Batch instead of N sequential awaits
- Extract shared _iter_reply_docs to deduplicate reply parsing between
  _reply_to_chroma and list_by_filter
- Parenthesize multi-condition pre-filters before the => KNN clause
- Fail closed when a username is configured without a password
- Catch only RequestError on ft.dropindex (let connection/auth errors surface)
- Bound the delete_collection SCAN loop with a safety cap
- Add VectorDatabase.close() (no-op default) + VectorDBManager.shutdown()
- Simplify _MATCH_ALL literal; normalize typing to builtin generics

* fix(vector/valkey_search): address round-2 review feedback

- Serialize lazy client creation with an asyncio.Lock (double-checked) so
  concurrent first-use callers don't construct and leak duplicate clients.
- Make the filter operator chain exhaustive: raise on an unhandled op rather
  than silently dropping the condition (which could widen delete_by_filter).
- Cast numeric range (///) values to float, failing closed on
  non-numeric input and pre-empting a future NUMERIC-field injection surface.

* refactor(vector): remove shutdown/close from base ABC per maintainer feedback Per maintainer request, interface changes to VectorDatabase ABC and VectorDBManager should be in a separate PR with implementation across all backends. The ValkeySearchVectorDatabase.close() method remains but does not override an ABC method.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* docs(test): list valkey_search in vdb coverage exclusions Add valkey_search to the documented vector/vdbs/ coverage-exclusion list, matching the existing chroma/milvus/pgvector/qdrant/seekdb entries. These adapters require a live database instance and are covered by env-gated integration tests instead of unit tests.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-07-08 06:59:16 +08:00

179 lines
6.8 KiB
Markdown
Raw Blame History

# 单元测试覆盖率排除说明
## 排除范围
以下外部适配器模块不纳入测试覆盖目标，因为它们需要实际外部环境才能测试：
### 1. 消息平台适配器 (`platform/sources/`)
- **路径**: `src/langbot/pkg/platform/sources/`
- **模块**: aiocqhttp, dingtalk, discord, feishu, gestep, kook, lark, slack, telegram, wecom, wechatpv, wechatmp, qqbot
- **排除原因**: 需要真实消息平台账号和 webhook 连接，无法纯单元测试
- **测试方式**: 需要 mock 平台 API 或集成测试环境
- **状态**: 后续可补充 mock 测试
### 2. LLM Requester (`provider/modelmgr/requesters/`)
- **路径**: `src/langbot/pkg/provider/modelmgr/requesters/`
- **模块**: deepseek, openai, anthropic, gemini, moonshot, ollama, zhipuai 等 20+ 个 requester
- **排除原因**: 需要真实 LLM API 密钥和网络请求，涉及付费 API 调用
- **测试方式**: 需要 mock HTTP 响应或使用 fake LLM server
- **状态**: 后续可补充 mock HTTP 测试
### 3. Agent Runner (`provider/runners/`)
- **路径**: `src/langbot/pkg/provider/runners/`
- **模块**: cozeapi, difysvapi, n8nsvapi, langflowapi, dashscopeapi, localagent, tboxapi
- **排除原因**: 需要真实 Agent 平台（Coze、Dify、n8n 等）的 API 连接
- **测试方式**: 需要 mock Agent 平台响应
- **状态**: 后续可补充 mock 测试
### 4. 向量数据库 (`vector/vdbs/`)
- **路径**: `src/langbot/pkg/vector/vdbs/`
- **模块**: chroma, milvus, pgvector, qdrant, seekdb, valkey_search
- **排除原因**: 需要真实向量数据库实例运行
- **测试方式**: 需要 Docker 启动测试数据库或 mock
- **状态**: 后续可补充 mock 测试
---
## 覆盖率计算（排除外部适配器）
### 统计方法
```bash
# 排除外部适配器后计算覆盖率
pytest tests/unit_tests/ --cov=langbot.pkg \
--cov-fail-under=0 \
-o "cov_exclude_patterns=platform/sources/*,provider/modelmgr/requesters/*,provider/runners/*,vector/vdbs/*"
```
### 当前覆盖率（排除后）
| 模块 | 覆盖率 | 状态 |
|------|--------|------|
| `command` | **99%** | ✅ 完成 |
| `entity` | **99%** | ✅ 完成 |
| `vector` | **76%** | ✅ 完成 |
| `survey` | **84%** | ✅ 完成 |
| `pipeline` | **72%** | ✅ 核心流程 |
| `rag` | **66%** | ✅ 完成 |
| `telemetry` | **87%** | ✅ 完成 |
| `storage` | **80%** | ✅ 完成 |
| `provider` | **83%** | ✅ 完成 |
| `discover` | **61%** | ✅ 完成 |
| `config` | **70%** | ✅ 完成 |
| `utils` | **48%** | 🔄 部分完成 |
| `api` | **34%** | 🔄 需补充 controller |
| `platform` | **35%** | 🔄 需补充 adapter base |
| `plugin` | **27%** | 🔄 需补充 handler |
| `core` | **28%** | 🔄 需补充 app 启动 |
| `persistence` | **24%** | 🔄 需补充 mgr |
---
## 后续计划
### 可补充的 Mock 测试（优先级排序）
1. **`provider/modelmgr/requesters/`** (优先级：中)
- 使用 `httpx` mock 测试 API 响应解析
- 测试重试逻辑、错误处理
2. **`provider/runners/`** (优先级：中)
- Mock Agent 平台响应
- 测试 session 管理、错误处理
3. **`platform/sources/`** (优先级：低)
- Mock 平台 webhook 事件
- 测试消息解析、事件处理
4. **`vector/vdbs/`** (优先级：低)
- Mock 向量数据库操作
- 测试 CRUD、查询逻辑
---
## 测试文件结构
```
tests/unit_tests/
├── api/
│ └── service/
│ ├── test_knowledge_service.py # 22 tests ✅
│ └── ...
├── core/
│ ├── test_taskmgr.py # 21 tests ✅
│ ├── test_load_config.py # 21 tests ✅ (含env override)
│ └── ...
├── plugin/
│ ├── test_connector_static.py # 8 tests ✅
│ ├── test_connector_pure.py # 7 tests ✅
│ ├── test_connector_methods.py # 24 tests ✅
│ ├── test_extract_deps.py # 7 tests ✅
│ ├── test_handler_actions.py # 15 tests ✅ (新增)
│ └── ...
├── provider/
│ ├── test_session_manager.py # 11 tests ✅ (新增)
│ ├── test_tool_manager.py # 14 tests ✅ (新增)
│ └── ...
├── rag/
│ ├── test_i18n_conversion.py # 8 tests ✅
│ ├── test_kbmgr.py # 39 tests ✅
│ ├── test_file_storage.py # 21 tests ✅ (新增)
│ └── ...
├── storage/
│ ├── test_s3storage.py # 16 tests ✅ (新增)
│ ├── test_localstorage_path_traversal.py # 11 tests ✅
│ └── ...
├── survey/
│ └── test_survey_manager.py # 22 tests ✅
├── telemetry/
│ └── test_telemetry.py # 25 tests ✅ (重写)
├── vector/
│ ├── test_filter_utils.py # 21 tests ✅
│ ├── test_vdb_filter_conversion.py # 30 tests ✅ (新增)
│ └── ...
├── utils/
│ ├── test_platform.py # 7 tests ✅
│ ├── test_funcschema.py # 9 tests ✅
│ └── ...
├── pipeline/
│ ├── test_ratelimit.py # 12 tests ✅ (新增真实算法)
│ ├── test_msgtrun.py # 9 tests ✅ (强化断言)
│ └── ...
└── persistence/
├── test_serialize_model.py # 6 tests ✅
├── test_database_decorator.py # 7 tests ✅
└── ...
```
---
## 总结
- **总测试数**: 1193 passed
- **总体覆盖率**: 30%
- **核心模块覆盖率**: **51.2%** (6549/12825 语句) - 排除外部适配器
- **外部适配器覆盖率**: 5.6% (535/9483 语句) - 不纳入目标
### 核心模块覆盖率详情
| 模块 | 覆盖率 | 语句数 | 说明 |
|------|--------|--------|------|
| `command` | **99%** | 93 | ✅ 完成 |
| `entity` | **99%** | 335 | ✅ 完成 |
| `vector` | **76%** | 139 | ✅ 完成 (新增filter转换测试) |
| `survey` | **84%** | 95 | ✅ 完成 |
| `pipeline` | **72%** | 1761 | ✅ 核心流程 (新增算法测试) |
| `rag` | **66%** | 347 | ✅ 完成 (新增ZIP处理测试) |
| `telemetry` | **87%** | 70 | ✅ 完成 (重写假测试) |
| `storage` | **80%** | 170 | ✅ 完成 (新增S3测试) |
| `provider` | **83%** | 854 | ✅ 完成 (新增Session/Tool测试) |
| `discover` | **61%** | 188 | ✅ 完成 |
| `config` | **70%** | 198 | ✅ 完成 |
| `utils` | **48%** | 478 | 🔄 部分完成 |
| `api` | **34%** | 4061 | 🔄 需补充 controller |
| `platform` | **35%** | 433 | 🔄 需补充 adapter base |
| `plugin` | **27%** | 815 | 🔄 需补充 handler (新增action测试) |
| `core` | **28%** | 1289 | 🔄 需补充 app 启动 |
| `persistence` | **24%** | 1099 | 🔄 需补充 mgr |
外部适配器测试需要 mock 环境或集成测试，不属于纯单元测试范畴。
Reference in New Issue View Git Blame Copy Permalink