Merge master into dev/4.11.x and preserve plugin runner architecture

Reconcile migration branches without rewriting published revisions; retain additive Codex, monitoring, provider and platform fixes. Keep dynamic runner schemas and Host ownership, restore compatibility regressions, and preserve safe model-test error handling.
2026-09-27 03:46:39 +08:00 · 2026-09-16 08:13:07 +00:00
parent 03854b5d33 9b7ba0d647
commit 273b1ea3cf
228 changed files with 22138 additions and 5057 deletions
@@ -1818,7 +1818,8 @@
            "local-agent",
            "tools",
            "e2b",
-            "nsjail"
+            "nsjail",
+            "host"
          ],
          "automation": "",
          "setup_automation": [],
@@ -48,7 +48,7 @@ tools, skill add/edit, and stdio MCP are disabled. Set `box.enabled: false`
 ## Kubernetes

 See `docker/kubernetes.yaml` and the deployment guide at
-https://docs.langbot.app. `docker/deploy-k8s-test.sh` is a test helper.
+https://langbot.app/docs. `docker/deploy-k8s-test.sh` is a test helper.

 ## config.yaml (generated at `data/config.yaml` on first run)

@@ -63,7 +63,7 @@ Key settings:
 | `api.global_api_key` | **Global API key** for the HTTP API + MCP server. Non-empty = accepted with no login/DB record; no `lbk_` prefix required. Empty = disabled. Plaintext — trusted/internal only, serve over HTTPS. |
 | `plugin.runtime_ws_url` | Standalone plugin runtime WS URL (e.g. `ws://langbot_plugin_runtime:5400/control/ws`) |
 | `box.enabled` | Master switch for the Box sandbox runtime |
-| `box.backend` | `local` (Docker/nsjail autopick) / `docker` / `nsjail` / `e2b`; env override `BOX__BACKEND` |
+| `box.backend` | `local` (Docker/nsjail autopick) / `docker` / `nsjail` / `e2b` / explicit unsafe `host`; env override `BOX__BACKEND` |
 | `box.runtime.endpoint` | External Box runtime URL (e.g. `ws://127.0.0.1:5410`); empty = local auto-managed |

 Many keys have `ENV__SUBKEY` overrides (e.g. `BOX__BACKEND`, `BOX__ENABLED`).
@@ -75,6 +75,10 @@ Many keys have `ENV__SUBKEY` overrides (e.g. `BOX__BACKEND`, `BOX__ENABLED`).
  with `--standalone-runtime`.
 - Box has a parallel `--standalone-box` flag; the Docker box host is
  `langbot_box:5410`.
+- `box.backend: host` runs commands directly as the Box Runtime system user.
+  It is never auto-selected, provides no sandbox isolation, and is only for
+  trusted local development. A WebSocket-controlled host backend requires
+  `LANGBOT_BOX_CONTROL_TOKEN`; local stdio control is allowed.

 ## Global API key — enabling for agents/automation

@@ -93,5 +97,7 @@ login session. See `langbot-mcp-ops` for using it, and `docs/API_KEY_AUTH.md`.
 - "No supported sandbox backend (Docker / nsjail / E2B)" with Docker running
  usually means the user isn't in the `docker` group →
  `sudo usermod -aG docker <user>` and restart in a new shell.
+- Do not use `box.backend: host` as a production fallback. It cannot enforce
+  image, filesystem, network, PID, CPU, memory, or storage isolation.
 - Box root host/container path mismatch breaks sandbox container creation.
 - Don't commit a non-empty `api.global_api_key` to version control.
@@ -43,6 +43,8 @@ Two kinds of key are accepted:
 Invalid, revoked, or expired keys get `401 Unauthorized`. A valid key whose
 scopes do not authorize a tool gets `403 Forbidden`.

+To inspect key identity and permissions, call `GET /api/v1/system/context` with the API key.
+
 ## Client configuration

 ```json
@@ -91,6 +93,38 @@ already have a default pipeline.
 4. Use `list_*` tools to discover, then `get_*` / `create_*` / `update_*` /
   `delete_*` as needed.

+## ChatGPT / Codex subscription providers
+
+`list_model_providers` can return the `openai-codex` requester. Its OAuth
+credentials are server-only and are not provider API keys. Never ask a user
+to paste ChatGPT access tokens, refresh tokens, or a Codex auth cache into an
+MCP tool or model configuration.
+
+A human connects or disconnects the subscription through **Models → provider
+settings** in the LangBot web UI. The provider-scoped `/codex/*` authentication
+routes deliberately require a browser-user session and are not exposed as MCP
+tools or authorized by a LangBot API key. Once connected, models are managed
+and selected through the normal provider/model workflow. A disconnected
+provider must be reauthorized; do not silently replace it with API-key billing.
+
+See [ChatGPT / Codex subscription](../../../docs/CODEX_SUBSCRIPTION.md) for setup,
+usage limits, and the personal-account versus shared-service boundary.
+
+## Provider deletion
+
+The curated MCP surface currently lists providers but has no provider-deletion
+tool. In the web UI, **Edit Provider → Delete** asks for confirmation before
+removing that provider and all its LLM, embedding, and rerank models. This is
+irreversible; never interpret a request to edit a provider as authorization to
+delete it.
+
+The equivalent HTTP operation is
+`DELETE /api/v1/provider/providers/{uuid}?cascade=true`, requiring
+`resource.manage` in the authenticated Workspace. Omitting `cascade` preserves
+the existing refusal to delete providers that still have models. Cloud-managed
+providers remain protected. Cascade deletion removes stored Codex authorization
+state as well; it is not the same operation as disconnecting an account.
+
 ## Implementation & maintenance (for LangBot developers)

 - Server: `src/langbot/pkg/api/mcp/server.py` (FastMCP). Tools call the service
@@ -25,7 +25,10 @@ CLI uses. Create one in your Space account (Profile → Personal Access Tokens),
 then send it as a Bearer token:

 ```
-Authorization: Bearer lbpat_...uests without a valid PAT get `401 Unauthorized`.
+Authorization: Bearer <your-pat>
+```
+
+Requests without a valid PAT get `401 Unauthorized`.

 ## Client configuration

@@ -66,6 +69,36 @@ All tools are read-only.
   state (available, unprobed, unavailable), then Space recommendation. Each
   item includes `availability.up`, `last_probed_at`, latency, and HTTP status.

+## Runner usage recommendations
+
+Use `search_plugins` with `runner_usage: "agent"` for Agent, pipeline, and
+setup-wizard recommendations, or `runner_usage: "event"` for event processors.
+The component kind remains `Runner`. Only these two exact values are accepted;
+omit the optional field to preserve unfiltered browsing.
+
+```json
+{"query":"", "runner_usage":"agent", "page":1, "page_size":100}
+```
+
+Plugin results include `latest_version` and `runner_usages: string[]`, the
+explicit union of usages in that latest installable version. Only recommend a
+plugin when this array explicitly contains the target usage. Missing, empty,
+malformed, or unknown usages must never mean agent-compatible. Event-only
+plugins must never enter Agent recommendations. Empty filtered results are
+valid while legacy packages await corrected releases; never remove the filter
+to fill a recommendation list.
+
+REST callers use `runner_usage` on both
+`POST /api/v1/marketplace/extensions/search` and the compatibility
+`POST /api/v1/marketplace/plugins/search`; preserve it during fallback. Add
+`"type_filter":"plugin", "component_filter":"Runner"` on the unified endpoint.
+Usage is ANDed with other filters before pagination and `total`; MCP/Skill items
+do not match. Invalid REST values return HTTP 400.
+
+Open the same filter in the webpage:
+`https://space.langbot.app/market?type=plugin&component=Runner&runner_usage=agent`
+(or `runner_usage=event`). Switch All / Agent / Event in the Runner usage row.
+
 ## Implementation & maintenance (for Space developers)

 - Server: `internal/controller/mcp/server.go` (official Go MCP SDK
@@ -13,6 +13,7 @@ tags:
  - tools
  - e2b
  - nsjail
+  - host
 skills:
  - langbot-env-setup
  - langbot-testing
@@ -23,7 +24,7 @@ env:
  - LANGBOT_LOCAL_AGENT_PIPELINE_NAME
 preconditions:
  - "LANGBOT_LOCAL_AGENT_PIPELINE_URL or LANGBOT_LOCAL_AGENT_PIPELINE_NAME points to the local-agent pipeline under test."
-  - "LangBot is started with the sandbox backend intended for this run, such as e2b or nsjail."
+  - "LangBot is started with the Box backend intended for this run, such as e2b, nsjail, or explicit host development mode."
  - "The selected model route supports tool/function calling strongly enough to invoke sandbox tools."
 steps:
  - "Start LangBot with the target sandbox backend and confirm the Box status UI or LANGBOT_BACKEND_URL /api/v1/box/status reports the expected backend."
@@ -33,7 +34,7 @@ steps:
 checks:
  - "UI: Debug Chat final assistant response contains E2E_OK:<skill-name>."
  - "Logs: The model called exec, register_skill, activate, then exec again from the activated skill path."
-  - "Logs: The selected backend name is the expected one, such as e2b or nsjail."
+  - "Logs: The selected backend name is the expected one, such as e2b, nsjail, or host."
  - "Skill store: The registered package and activated writeback match references/sandbox-skill-authoring.md."
  - "Box status: recent_error_count is 0 after the run."
 evidence_required:
@@ -4,7 +4,7 @@

 Verify that Local Agent can use sandbox tools to create, register, activate, and use a LangBot skill package through the same path a user would exercise in Debug Chat.

-This flow applies to Docker, nsjail, and E2B backends. API calls are useful diagnostics, but the primary pass/fail signal is the model-driven Debug Chat tool sequence.
+This flow applies to Docker, nsjail, E2B, and the explicit host development backend. Host runs commands directly as the Box Runtime user and must never be treated as sandbox-isolation coverage. API calls are useful diagnostics, but the primary pass/fail signal is the model-driven Debug Chat tool sequence.

 ## Preconditions

@@ -13,6 +13,7 @@ This flow applies to Docker, nsjail, and E2B backends. API calls are useful diag
   - `BOX_BACKEND=e2b` when validating E2B.
   - `BOX_BACKEND=nsjail` when validating nsjail.
   - `BOX_BACKEND=local` or `docker` when validating local container fallback.
+   - `BOX_BACKEND=host` only when validating explicit, trusted local direct execution.
 3. Confirm `/api/v1/box/status` reports `available: true` and the expected backend name.
 4. Confirm Debug Chat uses a model with function-calling ability.
 5. Confirm backend logs say native sandbox tools are available.
@@ -71,7 +72,7 @@ Backend logs should show:
 - `register_skill`
 - `activate`
 - a second `exec` whose workdir is `/workspace/.skills/<skill-name>`
- `backend=e2b`, `backend=nsjail`, or the expected local backend
+- `backend=e2b`, `backend=nsjail`, `backend=host`, or the expected local backend

 After the run, verify the skill store through the UI or API:

@@ -125,6 +126,8 @@ For E2B raw HTTP diagnostics, include a valid template id such as `base`; a miss
 - Session metadata should keep LangBot logical paths such as `/workspace`; storing provider-internal paths can make later requests look incompatible.
 - nsjail versions differ. Some expose only `--disable_clone_new*` flags and use `--bindmount` instead of `--rw_bind`.
 - On WSL, cgroup v2 may exist but not be writable. The backend should warn and fall back to rlimits rather than fail the sandbox.
+- The host backend does not honor sandbox image, network, rootfs, process, or
+  resource isolation. Use a disposable workspace and low-privilege account.
 - If `ALL_PROXY` uses a SOCKS URL and `socksio` is not installed, some Python HTTP clients can fail during startup. Prefer consistent HTTP proxy variables unless SOCKS support is installed.

 ## Related Troubleshooting
@@ -3,7 +3,7 @@ title: "Native sandbox tools are unavailable even though a backend is configured
 date: 2026-05-18
 symptoms:
  - "Backend logs show Native sandbox tools (exec/read/write/edit/glob/grep) are NOT available."
-  - "The Box runtime later reports that E2B, nsjail, or Docker is configured."
+  - "The Box runtime later reports that E2B, nsjail, Docker, or explicit host mode is configured."
  - "Debug Chat does not expose exec, register_skill, or activate as usable tools."
 patterns:
  - "Native sandbox tools ... are NOT available"
@@ -19,6 +19,7 @@ fix_steps:
  - "Ensure the Box runtime reselects a backend when get_backend_info is called and the cached backend is empty."
  - "For E2B, verify the key without printing it and confirm any required template setting."
  - "For nsjail, run nsjail --help and confirm the binary is on PATH for the LangBot process."
+  - "For trusted local development only, explicitly set box.backend=host; never use host as a production sandbox fallback."
 verification: "Run sandbox-skill-authoring-e2e. Logs should show Native sandbox tools are available and /api/v1/box/status should report available=true with the expected backend."
 related_cases:
  - sandbox-skill-authoring-e2e