From 0256670b51eda8aea7d5b5dff69a28db7b4da69e Mon Sep 17 00:00:00 2001 From: Junyan Qin Date: Mon, 20 Jul 2026 04:03:34 +0800 Subject: [PATCH] docs(tenancy): record final isolation verification --- docs/multi-tenant/verification-report.md | 244 ++++++++++++++--------- 1 file changed, 148 insertions(+), 96 deletions(-) diff --git a/docs/multi-tenant/verification-report.md b/docs/multi-tenant/verification-report.md index e3c6e0057..b22974010 100644 --- a/docs/multi-tenant/verification-report.md +++ b/docs/multi-tenant/verification-report.md @@ -2,10 +2,10 @@ Date: 2026-07-20 -Status: `CORE ISOLATION KERNEL VERIFIED — SAAS ACTIVATION REMAINS DISABLED` +Status: `CORE ISOLATION FOUNDATION VERIFIED — SAAS ACTIVATION REMAINS DISABLED` This report records the final implementation and verification evidence for the -shared Workspace isolation kernel. It does not claim that the closed SaaS +shared Workspace isolation foundation. It does not claim that the closed SaaS Control Plane, billing system, or greenfield Cloud v2 deployment is production-ready. @@ -13,77 +13,108 @@ production-ready. - LangBot branch: `feat/multi-tenants` - LangBot implementation commit: - `a47bfe8167e12d1853667b3d059cfb940ff0643e` + `90a977488212d3f3f50fb92d94e3fcae60f27eab` - LangBot base: `origin/master` at `6baeb032a7f76c65337b51c7b58593de4687a61c` - Plugin SDK branch: `feat/multi-tenants` - Plugin SDK commit and LangBot dependency pin: - `044536f3720a2a8424d92f78934b9623da9f7f1d` -- Space remained on `main` at - `5058b039c2629e3b01ebd7358369b50e7df62a1c`; this implementation made no - tracked Space change and does not reuse the legacy Space deployment model. + `95a1805af2038c745de2c018a00db1305089a32e` +- No tracked `langbot-space` change was made. Cloud v2 does not extend or + preserve the legacy per-account instance/pod deployment topology. The SDK protocol is versioned as 0.4.15 and is intentionally consumed from the -exact pushed Git commit on this feature branch. Publishing 0.4.15 and replacing -the Git pin with a registry pin remain merge-to-master release steps. +exact pushed Git commit above. Publishing the release and replacing the Git +pin with a registry pin remain merge-to-master release work; no registry +release was performed here. ## Implemented boundaries -- OSS bootstraps exactly one Workspace per instance while allowing multiple - Accounts, memberships, invitations, and role-based permissions inside it. - Local configuration or an edition flag cannot activate SaaS multi-Workspace - routing. -- The Cloud bootstrap remains a closed-policy injection point. The closed entry - point owns Manifest signature verification; Core accepts only a verified - deployment receipt and validates its instance, expiry, generation, - capability, entitlement-adapter, and runtime-configuration invariants. +### Workspace, identity, and authorization + +- Community bootstraps exactly one local Workspace per LangBot instance while + allowing multiple Accounts, memberships, invitations, and role-based + permissions inside it. A mutable edition setting cannot activate SaaS + multi-Workspace routing. +- Each Account registration resolves or creates its local singleton Workspace. + Invitation acceptance is one-time, email-bound, and survives the signed-out + login transition without exposing the invitation secret in URLs after the + initial fragment capture. +- The Account-token Workspace discovery route is intentionally separate from + Workspace-authorized routes. `X-Workspace-Id` becomes authority only after + membership resolution; API keys and public resources derive Workspace from + their trusted owner. +- Ordinary monitoring uses `resource.view`. Export remains `data.export`, + privileged runtime/system audit remains `audit.view`, and frontend controls + follow the same permission split. + +### Persistence and PostgreSQL + - PostgreSQL uses one shared business schema with application scope plus exact `ENABLE` and `FORCE ROW LEVEL SECURITY` policies. Tenant UoWs keep `SET LOCAL` - and SQL on one transaction; transaction-free scopes avoid holding pool - connections across model or network waits. + and business SQL on one owned root transaction and one task. +- The public tenant Session accepts only structured SQLAlchemy query/DML trees. + Raw/textual SQL, textual modifiers, custom AST/compiler nodes, unapproved + functions/operators/types/casts, hidden dialect values, foreign binds, + loader/execution options, Session event callbacks, ORM SQL-expression + attributes, transaction control, and live-result escape fail closed and make + the root transaction rollback-only. +- The SQL guard is a trusted-Core misuse boundary, not an in-process Python + sandbox. Mapped metadata and registered compilers are trusted boot-time code; + plugins remain out of process. SQLAlchemy dialect-private container fields + are pinned and covered by regression tests before dependency upgrades. +- Legacy local RAG restore remains structured and tenant-fenced while accepting + SQLite string-valued timestamps and both historical PostgreSQL `TEXT` and + fresh-schema `JSON` settings columns. - The one-shot release migrator owns DDL and grants a distinct runtime role only - business-table DML, `alembic_version` read access, required sequence access, - database `CONNECT`, and schema `USAGE`. Runtime startup reruns the complete + the required business-table DML, `alembic_version` read access, sequence + access, database `CONNECT`, and schema `USAGE`. Runtime startup reruns the role, ACL, schema, session, extension, routine, parameter, and RLS audit. - pgvector is stored in the same business database with Workspace-scoped keys, - checked dimensions, release-created partial ANN indexes, and RLS. Legacy - vector migration can temporarily suspend source-table RLS as a non-superuser, - non-`BYPASSRLS` table owner and restores each source table's prior state. -- One shared Plugin Runtime supervisor serves the logical instance. Every - enabled installation still owns one nsjail worker and one immutable - Workspace/installation/generation/revision/digest binding. Verified - same-digest code and dependency environments are read-only shared; processes - and writable state are never merged. + checked dimensions, release-created partial ANN indexes, and RLS. + +### Shared Plugin Runtime and Box + +- One instance-scoped Plugin Runtime supervisor serves multiple Workspaces. + Every enabled installation still owns one nsjail worker with an immutable + instance/Workspace/generation/installation/revision/artifact binding. +- Verified same-digest code and dependency environments may be mounted + read-only and reused. Plugin processes and writable installation state are + never merged, including for the same plugin and version. +- Runtime limits come only from instance configuration with native environment + overrides. Plugin manifests cannot raise limits. Shared mode rejects legacy + global plugin directories and legacy lifecycle control paths. - One shared Box control plane admits at most one persistent logical `global` - sandbox for an entitled Workspace. Cloud admission is entitlement-gated, + sandbox for an entitled Workspace. Admission is entitlement-gated, generation-fenced, shared-volume-challenged, quota-aware, and fixed to - nsjail. Plain nsjail correctly fails Cloud readiness without a hard storage - quota provider. -- stdio MCP has an independent policy gate and is forced off by the Cloud - bootstrap even when Box is available. + nsjail. Plain nsjail correctly fails Cloud readiness without a hard quota + provider. +- stdio MCP has an independent gate and is forced off by Cloud bootstrap even + when Box is available. ## Automated verification ### LangBot backend and static checks ```text -uv run pytest tests/unit_tests tests/integration -q - 2374 passed, 30 skipped, 148 warnings in 26.76s +uv run pytest -q --tb=short + 2527 passed, 32 skipped, 177 warnings in 73.37s + +focused tenant UoW and legacy RAG compatibility suite + 120 passed, 10 warnings in 1.30s uv run ruff check . passed -changed and untracked Python format check - 112 files already formatted +changed Python format check + 15 files already formatted git diff --check passed ``` -The fresh SQLite journey test covers first-Account registration, automatic -singleton Workspace creation, authenticated current-Workspace bootstrap, -wizard-state persistence, and a 403 `edition_limit` response when an OSS owner -attempts to create a second Workspace. +The full suite includes startup E2E, the fresh SQLite registration/Workspace +journey, API authorization, storage, RAG, Plugin, MCP, and 18 Docker-backed Box +integration cases. ### Real PostgreSQL 16 and pgvector @@ -91,16 +122,15 @@ attempts to create a second Workspace. test_migrations_postgres.py test_pgvector_postgres.py test_release_migration_postgres.py -test_plugin_identity_migration.py - 20 passed, 11 warnings in 13.51s + 21 passed, 11 warnings in 11.61s ``` The real-database suite covers fresh and upgraded RLS schemas, two-Workspace -isolation, pgvector CRUD and ANN indexes, plugin installation identity, -operator/runtime credential separation, advisory locking, and actual Cloud -runtime initialization as the least-privilege role. Negative tests inject and -reject role membership, grant options, persistent GUCs, extra schemas and -relations, column ACLs, explicit routine and parameter ACLs, runtime-owned and +isolation, pgvector CRUD and ANN indexes, least-privilege runtime bootstrap, +release locking, historical/fresh RAG settings column compatibility, and +transaction-scope escape rejection. Negative tests inject and reject role +membership, grant options, persistent GUCs, extra schemas and relations, +column ACLs, explicit routine and parameter ACLs, runtime-owned and `SECURITY DEFINER` routines, extra/runtime-owned extensions, and foreign data wrappers, servers, and user mappings. @@ -110,61 +140,75 @@ Alembic has exactly one head: `0013_tenant_pgvector`. ```text Plugin SDK full suite - 1154 passed, 18 warnings in 11.99s + 1160 passed, 18 warnings Plugin SDK Ruff and action consistency checks passed Docker-backed Box integration suite - 18 passed, 33 warnings in 13.89s + 18 passed, 33 warnings ``` -The installed LangBot environment resolves `langbot-plugin==0.4.15` from the -same exact SDK commit recorded above. +The LangBot environment resolves `langbot-plugin==0.4.15` from the same exact +SDK commit recorded above. The SDK local ref and remote +`refs/heads/feat/multi-tenants` were independently verified equal. ### Frontend ```text pnpm lint - 0 errors, 34 warnings + 0 errors, 34 existing warnings VITE_API_BASE_URL=/ pnpm build passed; 3168 modules transformed pnpm exec playwright test --project=chromium - 40 passed in 21.9s + 43 passed in 22.2s ``` +The frontend suite includes Viewer monitoring visibility, privileged control +hiding, stable terminal invitation states, email-mismatch login handoff, and a +temporary invitation-accept failure that retains the authenticated session and +succeeds on retry. + ### Packaging and deployment configuration - `uv lock --check` resolved 277 packages and passed. -- The configuration template, migration workflow, and Docker Compose file - passed YAML parsing. +- Changed TypeScript/TSX files passed Prettier verification. - `docker compose -f docker/docker-compose.yaml config --quiet` passed with the existing warning that the top-level `version` field is obsolete. -- Migration CI now runs the PostgreSQL RLS, pgvector, release-role, and plugin - identity suites against PostgreSQL 16 with pgvector. -- The staged high-signal credential scan found no private key or provider token. +- The staged high-signal credential scan found no private key or provider + token. ## Real browser E2E A production frontend and backend were run against an isolated temporary -SQLite data directory and operated in a real local Google Chrome Guest window: +SQLite data directory and operated through a real local Chrome session: -1. The first Account registered and received the automatically created default +1. The first Account registered and automatically received the default singleton Workspace as owner. -2. Skipping the setup wizard reached `/home/monitoring`; reload preserved the - dashboard and authenticated Workspace state. -3. Account Settings showed the OSS single-Workspace/multi-user model and owner - membership. The owner created an invitation for another Account. -4. After refresh, the pending invitation remained while its one-time raw secret - was no longer displayed. -5. The related Workspace, user, system, invitation, and monitoring requests - returned successful responses in the backend log. +2. Skipping the setup wizard reached `/home/monitoring`; reload and a new tab + recovered the authenticated Account, membership, and Workspace selection. +3. The owner invited a Viewer. After signing out, the second Account registered + through the invitation and joined the same Workspace. +4. Owner and Admin could read overview/export/audit/API-key surfaces and reach + resource-write validation. Runtime debug mutation remained disabled. +5. Developer retained ordinary resource management and monitoring, while + export, audit, and API-key management returned 403. +6. Operator and Viewer retained ordinary monitoring reads while resource + writes, export, audit, API-key management, and runtime mutation returned 403. +7. The UI matched the server matrix: Add/API Key/Export controls appeared only + with their permissions, while ordinary monitoring refresh remained visible + to read-only members. +8. A second Workspace creation attempt returned 403 `edition_limit`, proving + Community remains single-Workspace and multi-user. +9. Used, revoked, expired, and email-mismatched invitation paths produced + stable visible states. The mismatch survived login without a contradictory + generic success toast. -The server was stopped, port 15321 was verified free, and the two temporary -test directories were moved to the system Trash for recoverable cleanup. No -invitation secret is recorded in this report. +The server was stopped, port 15321 was verified free, and temporary artifacts +were moved to the system Trash for recoverable cleanup. No invitation secret, +password, token, or provider credential is recorded in this report. ## Deliberately incomplete SaaS activation gates @@ -178,28 +222,36 @@ independently verified: - ordinary tenant writes hold a generation-aware fence through commit, and a generation-stamped outbox or equivalent atomic publish fence protects external side effects; -- durable object references survive execution-generation cutovers without - stranding files, images, plugin artifacts, or knowledge-base content; -- Plugin Runtime deployment provides delegated cgroup v2 plus tenant-safe - network isolation and egress policy. The current shared network namespace is - not an acceptable public-SaaS isolation boundary; -- Box deployment supplies and proves hard byte and inode quotas for Workspace, - Skill, root, home, and temporary storage; stock nsjail intentionally fails - this readiness contract; -- production PostgreSQL uses a dedicated cluster/endpoint, or a tested - HBA/proxy policy proves that the cluster-wide runtime credential can connect - only to the target business database; +- durable object references survive generation cutovers without stranding + files, images, plugin artifacts, or knowledge-base content; +- every tenant-configurable outbound path has complete SSRF/egress policy, and + Plugin Runtime has production Linux namespace, cgroup v2, and network + isolation evidence; +- an unexpectedly exited enabled plugin worker is restored by a completion + callback with bounded backoff and cross-tenant restart-storm isolation; +- plugin installation data has an operator-owned hard disk quota, not a + directory-scan approximation; +- Box supplies and proves hard byte/inode quotas for Workspace, Skill, root, + home, and temporary storage, and the production shared volume passes the + authenticated marker challenge; +- production PostgreSQL uses a dedicated endpoint/cluster, or a tested + HBA/proxy policy proves the runtime credential can connect only to the target + business database; - the release migrator is deployed as a one-shot Job with credential issuance, - backup, rollback, and orchestration retry procedures. A future direct - migrator/pooler runtime endpoint split additionally requires an immutable - database-backed cluster identity; -- OAuth exchange and directory projection state use an atomic shared store for - horizontally scaled replicas; -- Workspace release, data export, deletion, and single-Workspace restore - semantics are decided and implemented; and -- the greenfield Cloud v2 deployment is validated without inheriting the old - Space deployment topology. + backup, rollback, failure/retry, and orchestration procedures; +- OAuth exchange, directory projection, leases, snapshots, event replay, and + outbox state use durable shared stores suitable for horizontally scaled + replicas; +- Workspace release, export, deletion, restore, and disaster-recovery semantics + are decided and implemented; and +- production Runtime restart, crash, partition, revocation, and fault-injection + acceptance is completed against the closed Control Plane and greenfield + Cloud v2 deployment. -These are explicit release gates, not hidden fallbacks. The current branch is a -verified isolation foundation on which the closed SaaS system can be built; it -is not a production Cloud activation. +The following product work is also deliberately deferred: WebUI configuration +for tenant-provided remote E2B, multi-replica lease storage and sharding rules, +artifact signing/garbage collection, custom roles, SSO, and SCIM. + +These are explicit release gates, not hidden fallbacks. The current branches +are a verified isolation foundation on which the closed SaaS system can be +built; they are not a production Cloud activation.