LangBot/docs/agent-runner-pluginization/EVENT_DEBUG_FULL_QA_2026-09-05.md

# Event debug full QA and release verification — 2026-09-05

## Scope and inventory

This follow-up investigates repeated welcome actions, verifies the debug surface with standalone Box/Plugin Runtime, and checks the changes before committing/pushing Core, SDK, and LocalAgent. Real messaging-platform effects must remain isolated from debug runs.

| Area | Required checks | Evidence |
| --- | --- | --- |
| Root cause | Follow-up messages/tool IDs preserved; direct model replay before/after mock guidance; original welcome prompt | Sanitized local traces, browser tool count |
| Events | All 15 built-in event types plus custom; nested/Unicode data and identity/target mapping | Browser results and parameterized service tests |
| Platform actions | Every catalog tool; frozen and explicit targets; permission/event/API intersection | Parameterized host boundary tests, browser reply/query/moderation/request cases |
| Mock scenarios | Success, error, query fixture, unsupported API; malformed options rejected | Browser and service tests |
| Input | Empty message/name, invalid/non-object JSON, unsupported event, actor/subject validation | Browser and request tests |
| Trace | Thinking/text/tool ordering, final snapshot deduplication, status/error correctness, no-call summary | Browser and reducer tests |
| Lifecycle | Cancel retains partial record; rerun; tabs/Agent switching; provider failure/fallback; reconnect | Browser and transport/runner tests |
| Native tools | Exec/file operations, nonzero exit, timeout and path escape | Browser and native-tool tests |
| Layout | Long output/parameters, automatic bottom scroll, narrow view, KB card spacing | DOM geometry and screenshots |
| Release | Python/SDK suites, frontend build/lint, whitespace checks, change review, commit and push | Commands, counts and remote SHAs |

## Investigation

The real follow-up request contained the assistant tool call and successful tool result with matching IDs. No loss occurred at the LocalAgent → SDK → Host model boundary. Direct replay to the configured provider, bypassing the runner, reproduced `event_reply` after success. Adding explicit mock completion semantics stopped further calls in two direct replays. The previous LocalAgent system context omitted the debug/mock semantics although the tool result said no real platform operation occurred.

The configured upstream `claude-opus-4-8` also returned an unsolicited CLI identity statement in direct replay. This originates in the provider response, not the debug renderer. It is separate from preserving tool results; it must not be presented as normal LangBot-generated status.

## Final results

### Browser event matrix

Executed through the real Edge WebUI, against Core `:5300`, Vite `:3000`, standalone Plugin Runtime `:5400/:5401`, and Docker-backed standalone Box `:5410`. The unbound `E2E Debug 0905` Agent uses `deepseek-v4-flash`; the user's original Agent and welcome prompt remain unchanged and use `claude-opus-4-8`.

| Event | Observed result in this pass |
| --- | --- |
| `bot.invited_to_group` | Actor/group queries and `event_respond_group_invite(approve=true)` succeeded; request ID frozen to `debug-group-request`. |
| `bot.muted`, `bot.removed_from_group` | One `event_get_group` call each, correct group target. |
| `bot.unmuted` | One actor query; correct user target. |
| `feedback.received` | One mock reply, `FEEDBACK_MOCK_OK`, person target. |
| `friend.added` | One mock reply, `FRIEND_ADDED_OK`, person target. |
| `friend.request_received` | Acceptance and separate explicit rejection exercised; rejection preserved `approve=false`, Unicode remark and frozen request ID. |
| `group.member_banned` | One member lookup, SDK-compatible nested `user` and `group_id`. |
| `group.member_left` | One group lookup, SDK-compatible `id`/`name`. |
| `group.member_joined` | Welcome reply, configured failure, query fixture, unsupported API, and mute/unmute/kick scenarios. Original-model repeat caveat below. |
| `message.deleted` | One actor lookup, SDK-compatible `id`/`nickname`. |
| `message.edited` | One simulated deletion; frozen group, chat and message IDs. |
| `message.reaction` | One reply, `REACTION_OK`, and correct `👍` event data. |
| `message.received` | Real six-tool file/exec chain, plain-text recovery, cancellation, draft retention, nonzero exit and timeout. |
| `platform.specific` | Preserved nested arrays, booleans, null and `测试🙂`; no tools called. |
| `custom.event`, `custom.e2e` | Empty object and named custom event with `CUSTOM_中文🙂`; no tools called. |

The first invitation/default friend-request runs submitted the preset text; they are recorded as default-behavior cases, not as explicit one-call/rejection tests. The rejection was separately rerun with the actual submitted instruction verified. Browser automation reads the controlled input after filling before treating a scenario as submitted.

### Mock and execution behavior

- Platform Mock runs the actual model/tool selection, authorization, parameter validation and frozen-target resolution; only the adapter boundary is simulated. Tests assert that mock execution never accesses the real bot manager. Native tools retain real Box behavior.
- The platform catalog contains 24 tools. Parameterized tests exercise success and configured failure for all 24, plus permission/event/API filtering, explicit/frozen targets, SDK fixture shapes and invalid options. This is full catalog contract coverage, not 24 separate browser clicks or proof of live adapter support.
- `errors.event_reply = "E2E permission denied"` produced one failed call and the **模拟执行失败 · Mock** status. The model reported the actual error without retrying in this scenario.
- `results.event_get_actor` returned the configured `fixture-user-77` / `测试用户🙂`; the model used these values instead of the original event identity. Default query results now serialize SDK platform models; default lists contain one synthetic entry and can be overridden with `[]`.
- Disabling `send_message` removed `event_reply` from the model's available tools. It returned `REPLY_UNAVAILABLE`, with the explicit no-tool-call summary. Clearing Mock options restored action availability; the three moderation calls succeeded with the correct group/member IDs.
- Mock `[]`, unknown tool names, whitespace-only message/custom name, malformed event JSON and event JSON `[]` were rejected. Browser transcript counts did not increase for client-side validation failures. Backend tests additionally cover falsey/non-object actor, subject and data values, conflicting outcomes and invalid API names.
- Thinking returned by the provider, intermediate text, tool parameters, results and final text remain distinct and ordered. Missing thinking is not fabricated. Cumulative final snapshots and prior tool-turn prefixes are deduplicated without hiding actual repeated tool calls.

### Lifecycle, native tools and layout

- A browser cancellation regression exposed a stuck **停止调试** button: the `finally` block skipped resetting state for aborted requests. It now clears the matching controller and resets running state even after user cancellation. Retest retained partial thinking/tool arguments, marked the unfinished call **未返回结果**, showed the cancellation notice and restored **运行测试**.
- Two subsequent recovery requests encountered the bounded-stream error below. A later request in the same tab/session returned `42` successfully; further real tool runs also completed. Cancellation and recovery are therefore verified, but immediate model success after cancellation is not guaranteed.
- Editing `NEXT_DRAFT_FINAL_0905` while a `sleep 5` run was still awaiting its final response preserved the draft after completion.
- Real `write → read → edit → glob → grep → exec` succeeded. `final.txt` changed from `FINAL_WRITE` to `FINAL_EDIT`; grep without `include` found one match; exec returned `FINAL_EXEC` with exit code 0. Writes stayed under `/workspace/e2e-debug-0905`.
- `printf E2E_ERROR >&2; exit 7` preserved stderr and exit code 7 and displayed failure. `sleep 3` with `timeout_sec=1` returned `timed_out` in 1025 ms and displayed failure. Neither was retried.
- Separate tabs retained separate transcripts during concurrent runs. Agent switching reset the debug surface. File workspace sharing remains intentional; transcript isolation does not mean separate Box filesystems.
- The final transcript measured 674 px high with 16,630 px scroll height, distance from bottom 0, and equal client/scroll widths of 468 px. Visual inspection confirmed contained parameters/results and accessible input controls. The earlier pass also verified 1280 × 800; the final attempted viewport override did not change this tab, so that attempt is not counted as an additional narrow-screen result. Temporary overrides were reset.
- Knowledge-base card geometry: retrieval card bottom 1413 px, danger card top 1437 px, giving the expected **24 px** gap. Both adjacent card gaps measured 24 px.
- Core health check returned `ok`; Plugin Runtime was connected and all development service ports remained listening. Previous restart/reconnect evidence is in the earlier report.

### Automated checks

| Suite | Result |
| --- | --- |
| Core agent unit directory plus debug service/controller, native/skill tools and model conversion | 659 passed, 24 Windows-only skips |
| Final changed platform/debug service/controller checks | 133 passed |
| Linux Docker: native tools, skill tools and all platform tools | 150 passed, 0 skipped; includes the 24 POSIX cases skipped on Windows |
| SDK API suite plus runtime I/O handler | 411 passed |
| LocalAgent full suite (`PYTHONUTF8=1`) | 187 passed |
| Frontend unit suite | 69 passed |
| Skills CLI suite | 122 passed |
| Frontend TypeScript/Vite build and changed-file ESLint | Passed; existing large-bundle advisory remains |
| Changed Python Ruff; Git whitespace checks | Passed |
| Skills index generation, validate and index consistency | Passed |

Counts overlap where a final focused/Linux run repeats an earlier suite; do not add them as unique tests. Existing Pydantic deprecation warnings remain. This is the relevant subsystem regression set, not the entire repository/integration matrix. Provider fallback has deterministic LocalAgent tests; live fallback after a partially emitted response is deliberately unsupported and was not claimed as a successful browser fallback.

The full regression also found Windows portability problems in the skills tooling: LF-only frontmatter parsing, `/cases/` detection and native path separators in generated references. These are fixed and covered by the CLI suite. Two frontend source-contract expectations were stale after the existing three-tab workbench/layout changes; they were updated to the actual product structure.

### Remaining limits and reproduction

1. **Original provider is still nondeterministic.** With the latest LocalAgent guidance, one rerun made exactly one successful `event_reply`; another made repeated successful replies before stopping. The trace faithfully records each call. Earlier invalid `_call` arguments were rejected. Direct provider replay proved that repeated calls can originate upstream even with a valid matching tool result. Prompt guidance reduces ambiguity but does not establish exactly-once actions. No silent deduplication or fabricated success was added.
2. **Bounded stream pressure can fail long responses.** Two recovery runs emitted thinking and then `Streaming action consumer is too slow; response buffer full`; later recovery and six-tool runs succeeded. The error originates in the SDK's existing 128-frame response queue and is preserved in the UI. It is not a missing-tool-result or 30-second HTTP timeout. Full end-to-end flow control under sustained overload remains open; queue limits were not removed to conceal the failure.
3. Mock results verify debug behavior, not real platform credentials, permissions, delivery, or every adapter/event Cartesian combination. Already launched external tool termination timing was not independently measured. A development-branch push does not mean these remaining production release gates passed.

For reproduction, use the retained test Agent, the Mock examples in [AGENT_RUNNER_QA_GUIDE.md](./AGENT_RUNNER_QA_GUIDE.md), and the original welcome Agent. Local diagnostic traces and fixture artifacts are excluded from Git; they contain model responses and local runtime state. This report contains outcomes rather than raw provider reasoning or secrets.