xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-07-16 17:36:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(agent-runner): aggregate non-streaming replies

Browse Source
This commit is contained in:
huanghuoguoguo
2026-07-15 08:48:14 +08:00
parent 730d52bcd0
commit 0060edaf3e
6 changed files with 86 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/agent-runner-pluginization/STATUS.md
+3 -3
View File
@@ -15,7 +15,7 @@
| Result payload validation | Done | Wire 保持 `{type, data}`；Host 对投递/副作用类 payload 严格校验，tool-call telemetry 宽松，未知 type 忽略并 warning。 |
| Old built-in runners | Done | 旧 `src/langbot/pkg/provider/runners/*` 与 `RequestRunner` 路径已从本分支删除。 |
| Official runner manifests | Done | `local-agent`、ACP / Claude Code / Codex 外部 harness runner、外部服务 runner 已重新声明真实生效的 LangBot resource permissions。 |
| Skill 链路 | Unit-pass; WebUI E2E pass | 已按 **skill 全 tool 化** 收敛：发现走 `list_skills` / `langbot_list_assets` 和 skill resources；`activate` / `register_skill` 走统一 tool 授权；`skill_authoring` capability 降级为便捷开关。`activate` 会 best-effort 写入 conversation-scope `host.activated_skills`，后续 run 通过当前 pipeline-visible skill cache 恢复。新注册 Skill 在当前 Query 内立即获得临时可见性；Docker `exec` 产生的宿主侧不可写文件由 `write` / `edit` 回退到 Box 执行。2026-07-15 真实 LocalAgent Debug Chat 已完成创建、注册、同 Query 激活、编辑和执行闭环。 |
| Skill 链路 | Unit-pass; WebUI E2E pass | 已按 **skill 全 tool 化** 收敛：发现走 `list_skills` / `langbot_list_assets` 和 skill resources；`activate` / `register_skill` 走统一 tool 授权；`skill_authoring` capability 降级为便捷开关。`activate` 会 best-effort 写入 conversation-scope `host.activated_skills`，后续 run 通过当前 pipeline-visible skill cache 恢复。新注册 Skill 在当前 Query 内立即获得临时可见性；Docker `exec` 产生的宿主侧不可写文件由 `write` / `edit` 回退到 Box 执行。2026-07-15 真实 LocalAgent Debug Chat 已完成创建、注册、同 Query 激活、编辑和执行闭环；非流式 runner turn 只向下游 Pipeline 产出一次，工具中间结果不再拆成额外 Bot 气泡。 |
| Runtime Control Plane v2 foundation | Partial | Host-owned `AgentRun` / `AgentRunEvent` ledger、orchestrator 自动建账、result event persistence、run get/list/event page/cancel/append/finalize actions 已落地；`agent_run:admin` / `runtime:admin` 控制权限、最小 runtime register/heartbeat/list/reconcile 和 run claim/renew/release 原语已落地。完整 Agent Platform 产品形态、daemon supervisor、任务唤醒/长轮询/WebSocket、分布式 runtime 管控仍未完成。 |
| Security boundary | Done | 当前口径降级为轻量边界：LangBot 保护自身持有资源；external harness 的 OS / process / network / workspace 风险由用户或部署环境承担；managed sandbox 不是当前承诺。 |
| Steering control path | Done | claim 异常不再逃逸 consumer loop；queue 有上限；未 pull 的 claimed 输入在 run 结束时写 `steering.dropped` 审计终态。 |
@@ -37,7 +37,7 @@
| Runner | 状态 | 最近证据 |
| --- | --- | --- |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 AgentRunner catalog 安装 `langbot-team/LocalAgent` 0.1.0，Host 注册 `plugin:langbot-team/LocalAgent/default`，Wizard 自动选中并解锁后续操作。2026-07-15 `local-agent-functional-20260715-skill-edit-final` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行；三阶段 UI、浏览器诊断和结构化文件系统检查全部通过，p95 18.8 秒、错误率 0。 |
| `plugin:langbot-team/LocalAgent/default` | Unit-pass; Marketplace UI pass; Debug Chat E2E pass | 2026-07-12 隔离 first-run 实例从真实 AgentRunner catalog 安装 `langbot-team/LocalAgent` 0.1.0，Host 注册 `plugin:langbot-team/LocalAgent/default`，Wizard 自动选中并解锁后续操作。2026-07-15 `2026-07-15-08-44-10-770-08-00-sandbox-skill-authoring-edit-existing-e2e` 使用真实 `gpt-5.5` 完成 Skill 创建、注册、同 Query 激活、已激活包编辑与脚本执行；三阶段 UI、浏览器诊断和结构化文件系统检查全部通过，每阶段恰好新增一个 Bot 气泡，p95 14.6 秒、错误率 0。 |
| `plugin:langbot-team/ACPAgentRunner/default` / `plugin:langbot-team/ClaudeCodeAgent/default` / `plugin:langbot-team/CodexAgent/default` | Unit-pass; E2E pending | 通过 runner 仓库单测覆盖 session、run_id 注入和 LangBot MCP gateway；真实 harness E2E 取决于对应运行环境、CLI/daemon 可用性和 provider 登录态。 |
| Dify / n8n / Coze / DashScope / Langflow / Tbox / DeerFlow / WeKnora | Unit-pass; credential smoke optional | 2026-06-13 plugin layout / parser tests 通过；真实服务凭据 smoke 非每轮必跑。 |
@@ -46,7 +46,7 @@
| 范围 | 状态 | 最近证据 |
| --- | --- | --- |
| LangBot Runtime Control Plane v2 foundation | Unit-pass; EBA release gate 5/5 pass | 2026-07-12 `eba-functional-20260712-release-gate-rerun` 通过 Quick Start 场景筛选、隔离实例 Runner Marketplace 安装、Runner 健康状态、事件路由 dry-run / 合成派发，以及真实 OneBot `group.member_joined` → Agent → `send_group_msg` 链路。 |
| Host Skill / native tool integration | Unit-pass; WebUI E2E pass | 2026-07-15 provider / native / Skill / monitoring 定向测试 67 项通过，Skills CLI 104 项通过；真实 Debug Chat 验证 `register_skill` 后同 Query `activate` 成功，监控工具调用不再把 SQL 行误取为字符串，结构化 JSON 文件检查不依赖格式空格。 |
| Host Skill / native tool integration | Unit-pass; WebUI E2E pass | 2026-07-15 provider / native / Skill / monitoring 定向测试 67 项通过，Pipeline / Chat / Wrapper 定向测试 61 项通过，Skills CLI 105 项通过；真实 Debug Chat 验证 `register_skill` 后同 Query `activate` 成功，监控工具调用不再把 SQL 行误取为字符串，结构化 JSON 文件检查不依赖格式空格，非流式多阶段 runner 结果只生成一个最终 Bot 气泡。 |
| SDK AgentRunner control entities / proxy | Unit-pass | 2026-06-23 SDK `tests/api/entities/builtin/agent_runner`、`tests/api/proxies`、`tests/api/test_agent_tools_mcp_bridge.py`、`tests/runtime/plugin/test_mgr_agent_runner.py`、`tests/runtime/test_pull_api_handlers.py`、`tests/runtime/io/handlers/test_plugin_handler.py`、EBA event entities 和 message tests 通过，覆盖 typed entities、AgentRunAPIProxy、MCP bridge、runtime manager 与 pull API handlers。 |
## 历史高价值记录
skills/scripts/e2e/lib/debug-chat.mjs
+39 -1
View File
@@ -91,6 +91,7 @@ export function classifyDebugChatResult({
beforeMessages = null,
afterMessages = null,
latestAssistantText = "",
maxNewAssistantMessages = null,
failureSignals = DEBUG_CHAT_FAILURE_SIGNALS,
}) {
const minExpectedCount = minExpectedOccurrences(beforeText, expectedText, prompt);
@@ -107,6 +108,20 @@ export function classifyDebugChatResult({
const assistantExpectedIncreased = hasMessageEvidence
? afterAssistantExpectedCount > beforeAssistantExpectedCount
: false;
const beforeAssistantMessageCount = hasMessageEvidence
? beforeMessages.filter((message) => message.role === "assistant").length
: null;
const afterAssistantMessageCount = hasMessageEvidence
? afterMessages.filter((message) => message.role === "assistant").length
: null;
const newAssistantMessageCount = hasMessageEvidence
? afterAssistantMessageCount - beforeAssistantMessageCount
: null;
const assistantMessageEvidence = {
before_assistant_message_count: beforeAssistantMessageCount,
after_assistant_message_count: afterAssistantMessageCount,
new_assistant_message_count: newAssistantMessageCount,
};
if (hasMessageEvidence) {
const latestAssistantFailure = findFailureSignalInText(latestAssistantText, failureSignals);
@@ -119,6 +134,18 @@ export function classifyDebugChatResult({
failure_signal: latestAssistantFailure,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
...assistantMessageEvidence,
};
}
if (maxNewAssistantMessages !== null && newAssistantMessageCount > maxNewAssistantMessages) {
return {
status: "fail",
reason: `Debug Chat created ${newAssistantMessageCount} assistant messages; expected at most ${maxNewAssistantMessages}.`,
min_expected_count: minExpectedCount,
final_count: finalCount,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
...assistantMessageEvidence,
};
}
if (assistantExpectedIncreased && String(latestAssistantText).includes(expectedText)) {
@@ -129,6 +156,7 @@ export function classifyDebugChatResult({
final_count: finalCount,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
...assistantMessageEvidence,
};
}
if (failureText) {
@@ -140,6 +168,7 @@ export function classifyDebugChatResult({
failure_signal: failureText,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
...assistantMessageEvidence,
};
}
return {
@@ -149,6 +178,7 @@ export function classifyDebugChatResult({
final_count: finalCount,
before_assistant_expected_count: beforeAssistantExpectedCount,
after_assistant_expected_count: afterAssistantExpectedCount,
...assistantMessageEvidence,
};
}
if (failureText) {
@@ -345,7 +375,14 @@ export async function sendDebugChatPrompt(page, prompt, imagePath = "") {
return true;
}
export async function runDebugChatPrompt(page, { prompt, expectedText, responseTimeoutMs, imagePath = "", failureSignals = DEBUG_CHAT_FAILURE_SIGNALS }) {
export async function runDebugChatPrompt(page, {
prompt,
expectedText,
responseTimeoutMs,
imagePath = "",
maxNewAssistantMessages = null,
failureSignals = DEBUG_CHAT_FAILURE_SIGNALS,
}) {
const beforeText = await bodyText(page);
const beforeMessages = await visibleDebugChatMessages(page);
const minExpectedCount = minExpectedOccurrences(beforeText, expectedText, prompt);
@@ -380,6 +417,7 @@ export async function runDebugChatPrompt(page, { prompt, expectedText, responseT
beforeMessages,
afterMessages,
latestAssistantText,
maxNewAssistantMessages,
failureSignals,
});
}
skills/scripts/e2e/pipeline-debug-chat.mjs
+4
View File
@@ -989,6 +989,7 @@ try {
expectedText: step.expectedText,
responseTimeoutMs: step.responseTimeoutMs,
imagePath: index === 0 ? imagePath : "",
maxNewAssistantMessages: streamOutput === false ? 1 : null,
failureSignals: failureSignals.length > 0 ? failureSignals : undefined,
});
const promptDurationMs = Date.now() - promptStartedAt;
@@ -1002,6 +1003,9 @@ try {
final_count: chatResult.final_count,
before_assistant_expected_count: chatResult.before_assistant_expected_count,
after_assistant_expected_count: chatResult.after_assistant_expected_count,
before_assistant_message_count: chatResult.before_assistant_message_count,
after_assistant_message_count: chatResult.after_assistant_message_count,
new_assistant_message_count: chatResult.new_assistant_message_count,
failure_signal: chatResult.failure_signal || "",
});
result.status = chatResult.status;
skills/test/lbs-cli.test.ts
+27
View File
@@ -2006,6 +2006,33 @@ test("debug chat classifier passes when expected text appears in a new assistant
assert.match(result.reason, /new assistant message/);
assert.equal(result.before_assistant_expected_count, 0);
assert.equal(result.after_assistant_expected_count, 1);
assert.equal(result.new_assistant_message_count, 1);
});
test("debug chat classifier rejects split non-streaming assistant messages", () => {
const expectedText = "E2E_OK:skill";
const result = classifyDebugChatResult({
beforeText: "",
afterText: `Bot: intermediate one\nBot: intermediate two\nBot: ${expectedText}`,
expectedText,
prompt: "Run a tool and return the result",
latestExpectedLeaf: expectedText,
latestFailureLeaf: "",
beforeMessages: [],
afterMessages: [
{ role: "assistant", text: "intermediate one" },
{ role: "assistant", text: "intermediate two" },
{ role: "assistant", text: expectedText },
],
latestAssistantText: expectedText,
maxNewAssistantMessages: 1,
});
assert.equal(result.status, "fail");
assert.match(result.reason, /created 3 assistant messages/);
assert.equal(result.before_assistant_message_count, 0);
assert.equal(result.after_assistant_message_count, 3);
assert.equal(result.new_assistant_message_count, 3);
});
test("debug chat classifier allows a recovered failure when latest assistant is successful", () => {
src/langbot/pkg/pipeline/process/handlers/chat.py
+10 -1
View File
@@ -113,10 +113,12 @@ class ChatMessageHandler(handler.MessageHandler):
# Create a single resp_message_id for the entire streaming response
resp_message_id = uuid.uuid4()
chunk_count = 0
has_result = False
# Use AgentRunOrchestrator to run the agent
# This replaces direct runner lookup and PluginAgentRunnerWrapper
async for result in self.ap.agent_run_orchestrator.run_from_query(query):
has_result = True
result.resp_message_id = str(resp_message_id)
# For streaming mode, pop previous response before adding new chunk
@@ -156,7 +158,8 @@ class ChatMessageHandler(handler.MessageHandler):
if result.content is not None:
text_length += len(result.content)
yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
if is_stream:
yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
# Log final summary after streaming completes
if is_stream:
@@ -170,6 +173,12 @@ class ChatMessageHandler(handler.MessageHandler):
# history source for this path.
await self._ensure_conversation_for_history(query)
# Non-streaming adapters should receive only the completed runner turn.
# Keep every normalized result on the query for diagnostics and history,
# while running downstream response stages once with the final result.
if not is_stream and has_result:
yield entities.StageProcessResult(result_type=entities.ResultType.CONTINUE, new_query=query)
except Exception as e:
# Import orchestrator errors for specific handling
from ....agent.runner.errors import (
tests/unit_tests/agent/test_chat_handler.py
+3 -2
View File
@@ -384,8 +384,8 @@ class TestChatHandlerAsyncBehavior:
assert query.resp_messages[0].content == 'Hello World!'
@pytest.mark.asyncio
async def test_non_streaming_no_pop(self):
"""Non-streaming mode: all chunks should remain."""
async def test_non_streaming_keeps_results_and_yields_once(self):
"""Non-streaming mode keeps runner results but runs downstream stages once."""
from langbot.pkg.pipeline.process.handlers.chat import ChatMessageHandler
from langbot.pkg.pipeline import entities
@@ -424,6 +424,7 @@ class TestChatHandlerAsyncBehavior:
assert len(query.resp_messages) == 2
assert query.resp_messages[0].content == 'Response 1'
assert query.resp_messages[1].content == 'Response 2'
assert len(results) == 1
@pytest.mark.asyncio
async def test_agent_turn_recreates_conversation_if_tool_resets_it(self):