From 4bdbcaf8e91ea8866b76b8c4dfb3b6411b420ad7 Mon Sep 17 00:00:00 2001 From: huanghuoguoguo <60681390+huanghuoguoguo@users.noreply.github.com> Date: Fri, 23 Oct 2026 18:37:46 +0800 Subject: [PATCH] docs(tools): clarify sustained comparison prerequisites --- tools/README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/README.md b/tools/README.md index 026630b79..1b3c00e82 100644 --- a/tools/README.md +++ b/tools/README.md @@ -430,7 +430,7 @@ This profile checks more than reply text: P95 limits. Reports retain planned/actual counts, scheduling delay and request drain time. The load is bounded by workers and serial group sessions; it is not an open arrival-rate capacity benchmark. -- Process RSS, FD, thread and process-count medians after cooldown are compared +- Process RSS, FD, thread and process-count medians from the final 30 seconds of cooldown are compared with a stable window after the first minute. Docker memory, PIDs and container counts are checked separately because sandbox processes are not children of the Box service. If a discovered container disappears before stats collection, @@ -460,7 +460,9 @@ uv run --no-sync python tools/regression_analysis.py \ The comparison rejects failed runs, missing dependency evidence, different enabled plugin versions, fixtures, expanded case/resource inputs, phase geometry, budgets, -or observer-host identity. Target URL origins and credentials may differ; URL +or observer-host identity. Both runs must use identical load/analysis/smoke oracle +files and mock scenarios; their hashes are recorded in the contract. Target URL +hosts, ports and credentials may differ; URL schemes and paths, model names and actual stimulus variables remain part of the contract. Both runs must use the same hardware allocation, model backend and background load; the contract does not establish those external conditions automatically.