mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-08-13 22:20:58 +00:00
huanghuoguoguo
5b2826fa49
* Add performance and reliability QA gates * test(skills): prepare user path performance gate * test(skills): add debug chat load gate * test(skills): extend fake provider load profiles * test(skills): add debug chat timing and isolation probes * test(skills): clarify manual QA perf gates
45 lines
3.9 KiB
Markdown
45 lines
3.9 KiB
Markdown
---
|
|
name: langbot-testing
|
|
description: Test LangBot WebUI and core product flows with an automated browser and backend logs. Use when validating the configured LangBot frontend, pipeline Debug Chat, model provider setup and test buttons, bot and knowledge-base UI flows, or troubleshooting failed LangBot end-to-end tests.
|
|
---
|
|
|
|
# LangBot Testing
|
|
|
|
Use this skill when an agent needs to verify LangBot behavior through the WebUI instead of only reading code.
|
|
|
|
## Routing
|
|
|
|
- **General WebUI testing**: read `references/web-ui-testing.md`.
|
|
- **Pipeline Debug Chat**: read `references/pipeline-debug-chat.md`.
|
|
- **Dify AgentRunner**: read `references/dify-agent-runner.md`.
|
|
- **Model provider setup or test button**: read `references/model-provider-testing.md`.
|
|
- **Plugin install/runtime/tool/page smoke**: read `references/plugin-e2e-smoke.md`.
|
|
- **Local Agent Runner**: read `references/local-agent-runner.md`.
|
|
- **Local Agent Runner path coverage**: read `references/local-agent-runner-coverage.md`.
|
|
- **Diff-aware AgentRunner QA after code changes**: read `references/agent-runner-qa-workflow.md`.
|
|
- **Agent Runner release gate**: read `references/agent-runner-release-gate.md`.
|
|
- **Sandbox-backed skill authoring**: read `references/sandbox-skill-authoring.md`.
|
|
- **LangRAG knowledge bases**: read `references/langrag-knowledge-base.md`.
|
|
- **MCP stdio tool testing**: read `references/mcp-stdio-testing.md`.
|
|
- **Performance, reliability, or chaos probes**: read `references/performance-reliability-testing.md`.
|
|
- **Drive a live instance over MCP (not raw HTTP)**: use the `langbot-mcp-ops` skill — the instance exposes an MCP server at `http://<host>:5300/mcp` (reuses API keys). Useful for setting up bots/pipelines/models as test fixtures programmatically.
|
|
- **Known failures and fixes**: read `references/troubleshooting.md`.
|
|
- **Reusable test groups**: run `bin/lbs suite list` and `bin/lbs suite plan <suite-id>` before manually assembling a case set.
|
|
|
|
## Rules
|
|
|
|
- Read `../.env` first and use `LANGBOT_FRONTEND_URL` and `LANGBOT_BACKEND_URL` instead of hardcoded ports.
|
|
- If a standalone frontend dev server is running, `LANGBOT_FRONTEND_URL` may point to `LANGBOT_DEV_FRONTEND_URL`; otherwise it may point to the backend WebUI.
|
|
- Confirm the backend and frontend are actually running before testing.
|
|
- Run `bin/lbs fixture check` before fixture-heavy MCP, RAG, multimodal, or plugin smoke tests.
|
|
- For runner externalization release checks, run `bin/lbs test run agent-runner-release-preflight` before the full `agent-runner-release-gate` suite so configuration blockers are separated from product failures.
|
|
- Read `Manual Readiness` in `bin/lbs test plan <case-id>`; `manual_check` means the declared preconditions or setup still need operator confirmation for this run.
|
|
- Use an authenticated browser profile prepared by `langbot-env-setup`.
|
|
- Do not expose API keys, OAuth secrets, tokens, or localStorage token values in output.
|
|
- A WebUI test is not complete until the visible UI result is checked against backend logs or network behavior.
|
|
- A performance result is not complete without `metrics` evidence and a clear split between LangBot overhead and external provider/tool/network time.
|
|
- A chaos or reliability result is not complete until the fault scope, cleanup, and recovery checks are recorded.
|
|
- For a suite, use `bin/lbs suite start <suite-id>` to create the suite evidence root, per-case directories, and `suite-start.json`/`suite-start.md` handoff files; use `bin/lbs test result <case-id>` to write final per-case `result.json`, then run `bin/lbs suite report <suite-id> --evidence-dir <dir>`.
|
|
- Do not mark a case `pass` until `test result --evidence` covers every value in the case's `evidence_required`.
|
|
- For runner-specific Debug Chat cases, use the case-specific pipeline env declared by `automation_pipeline_url_env` / `automation_pipeline_name_env`; do not silently reuse a generic `LANGBOT_PIPELINE_URL`.
|