LangBot/docs/multi-tenant/implementation-checklist.md

Multi-tenant implementation checklist

This checklist turns the Workspace architecture into implementation and verification gates. Exact commands and observed results are recorded in the verification report.

Scope guard

• LangBot uses branch feat/multi-tenants.
• langbot-plugin-sdk uses branch feat/multi-tenants.
• langbot-space has no changes made by this implementation; Cloud v2 does not extend the legacy Space deployment scheme.
• Unrelated untracked files in either repository remain untouched.
• Open-source startup cannot enable SaaS multi-workspace through edition flags or unsigned configuration.

SaaS activation gates

These items intentionally remain incomplete. Some require additional Core transaction/cutover primitives and others require the closed Control Plane or deployment. The feature branch delivers the Core isolation kernel, not the closed SaaS product or a production Cloud v2 deployment. Checked implementation items later in this document do not supersede these gates.

• The closed Control Plane owns the global Account, Workspace, Membership, and Invitation directory.
• The closed placement service issues monotonic generations and leases for projected Workspaces.
• Core verifies a signed InstanceManifest before the closed bootstrap can inject CloudWorkspacePolicy.
• Tenant database writes hold a generation-aware shared transaction fence through commit, while placement cutovers take the exclusive fence.
• Business writes and non-transactional side effects use a generation-stamped outbox or equivalent publish fence.
• Durable object references survive a placement-generation change through stable published keys or an explicitly atomic key/reference migration.
• The SaaS runtime pools enforce tenant-safe egress and SSRF controls for Webhooks, providers, MCP servers, and every tenant-configurable outbound URL.
• Entitlement checks, usage aggregation, subscription lifecycle, and billing are implemented in the closed Control Plane.
• OAuth state and directory projection use an atomic shared store suitable for horizontally scaled SaaS services.
• A greenfield Cloud v2 deployment is designed and validated independently of the legacy Space deployment scheme.
• The Plugin Runtime deployment provides delegated cgroup v2 and tenant-safe egress; the shared profile refuses to run without hard CPU, memory, and PID limits.
• The Box deployment provides an operator-owned quota provider that proves hard byte and inode limits for Workspace, Skill, root, tmp, and home storage.
• Core and Box Runtime mount the same durable volume and pass the authenticated marker challenge during startup and reconnect.
• Production provisions distinct migrator/runtime credentials and runs the implemented same-host/port/database release command as a one-shot Job, with tested orchestration retry, backup, and rollback procedures.
• Production PostgreSQL uses a dedicated cluster/endpoint, or a tested HBA/proxy policy proves the cluster-wide runtime credential can connect only to the target business database.
• Any future direct-migrator/pooler-runtime endpoint split is admitted only by a migrator-owned, runtime-read-only database cluster identity that the runtime role cannot spoof.
• Legacy pgvector migration failure and retry integration paths prove exact source-table RLS/FORCE restoration; the non-superuser, non-BYPASSRLS success path is already covered below.
• Multi-workspace is enabled in SaaS only after all closed Control Plane, deployment, and security gates pass.

1. Persistence foundation

Account and directory

• User has a stable, unique account UUID and explicit status.
• Existing email and password behavior remains compatible during migration.
• Workspace table represents the instance-local tenant.
• WorkspaceMembership has a unique Workspace and Account pair.
• WorkspaceInvitation stores only a token hash and supports expiry, revoke, and one-time accept.
• WorkspaceExecutionState stores generation, state, source, and write fence.
• OSS initialization creates exactly one Workspace and one owner membership atomically.
• OSS refuses a second Workspace while allowing multiple members.

Migration

• Alembic migration upgrades SQLite.
• Alembic migration upgrades PostgreSQL.
• Existing first user becomes owner of the default Workspace.
• Existing tenant resources are backfilled with the default Workspace UUID.
• SQLite destructive boundaries create verified, revision-aware backups and atomically restore after failure.
• Migration can resume safely after interruption.
• New installs and upgraded installs produce the same tenancy-kernel schema.
• The first Cloud release pins migrator and runtime sessions to public with current_schemas(false) containing only that business schema; runtime-role/database search_path overrides are rejected.
• Cloud sessions require session_replication_role=origin, row_security=on, and lo_compat_privileges=off; every persistent pg_db_role_setting applicable to the runtime role or current business database is rejected.
• The release migrator grants the runtime role exact business-table DML, alembic_version read-only access, and business-sequence USAGE/SELECT, with no WITH GRANT OPTION or non-business object grants.
• Every Cloud runtime startup revalidates the login role, current user, schema, effective/direct ACLs, ownership, memberships in all directions, column ACLs, routines, extensions, foreign objects, parameter ACLs, and other-schema access before serving traffic.
• The business database requires vector, permits only plpgsql/vector extensions, forbids runtime extension ownership, and contains no foreign data wrapper, foreign server, or user mapping.
• The runtime role and PUBLIC have no explicit routine or parameter ACL; the runtime owns no routine and cannot effectively execute any SECURITY DEFINER routine, including extension-owned routines.
• PostgreSQL's default PUBLIC TEMP is documented and tested as a dedicated-business-database v1 compatibility exception; the migrator never grants TEMP directly to the runtime role.
• Legacy pgvector migration succeeds as a non-superuser, non-BYPASSRLS source-table owner and restores mixed source-table RLS/FORCE states exactly.
• Legacy pgvector migration still needs explicit failure-and-retry integration coverage before SaaS activation.

2. Authentication and authorization

Identity

• JWT sub uses account UUID, with a bounded compatibility path for legacy email tokens.
• Disabled or deleted accounts cannot authenticate.
• Local password and Space-linked account flows support more than one local Account.
• Public registration closes after initialization by default.
• Invitation registration works without requiring SMTP.
• An unknown Space OAuth subject cannot claim an existing Account by email; explicit account-bound binding is required.

Request context

• PrincipalContext identifies Account, API Key, or trusted runtime principal.
• WorkspaceContext contains Workspace, Membership, role, permissions, and revision.
• RequestContext contains instance UUID, Workspace context, auth type, request ID, and generation.
• ExecutionContext propagates Workspace and generation to runtime work.
• SaaS-style requests never fall back to the first or most recent Workspace.
• OSS may resolve the single Workspace when the selector is omitted.
• Account-token bootstrap can list only the authenticated Account's active memberships before a Workspace selector exists.

Fixed RBAC

• owner, admin, developer, operator, and viewer permissions match the architecture matrix.
• Invitation cannot grant owner.
• The last owner cannot be removed or demoted.
• Cross-Workspace resources return 404.
• Same-Workspace permission failures return 403.

3. Workspace and member APIs

• GET /api/v1/workspaces returns the OSS singleton Workspace.
• POST /api/v1/workspaces returns edition_limit in OSS.
• Current Workspace endpoint returns the authenticated Membership.
• Member list is permission scoped.
• Invitation create, revoke, inspect, and accept are atomic.
• Member role update and removal enforce owner rules.
• Invitation tokens travel in a request body and are redacted from logs.
• Relevant MCP tools and in-repo skills are updated with the same contract.

4. Tenant-scoped persistence and services

Each row type must have a non-null Workspace UUID, scoped indexes, scoped uniqueness, and scoped CRUD tests.

• Bots and bot admins.
• Legacy pipelines and pipeline run records.
• Model providers.
• LLM models.
• Embedding models.
• Rerank models.
• Plugin installations, settings, and configuration.
• MCP servers and resource preferences.
• Knowledge bases, files, and chunks.
• Vector collections and handles.
• Monitoring messages, calls, sessions, errors, embeddings, and feedback.
• API keys and scopes.
• Webhooks and public route resolution.
• Binary storage and Workspace storage.
• Workspace metadata, separated from system metadata.

Service and API rules

• Every tenant Service receives RequestContext or an explicit Workspace UUID.
• No tenant Service treats context None as global access.
• Every applicable get, list, create, update, delete, copy, export, and bulk operation is scoped.
• Parent-child references use the same Workspace.
• API Key authentication derives Workspace from the key, not a header.
• Webhook and Bot public routes derive Workspace from a trusted resource.
• Background jobs carry Workspace and generation explicitly.

5. Runtime isolation

Core runtime

• RuntimeBot carries Workspace UUID and placement generation.
• RuntimePipeline carries Workspace UUID and placement generation.
• Query and Event carry Workspace UUID without making it an authorization source.
• Session key includes Workspace UUID, Bot UUID, launcher type, and launcher ID.
• QueryPool and manager indexes cannot collide across Workspaces.
• Query and aggregation cache keys and locks include Workspace UUID.
• Runtime transports, cached results, object operations, and long-lived tasks revalidate WorkspaceExecutionState generation at side-effect boundaries.
• Ordinary tenant database writes hold the generation fence in the same transaction until commit; this remains a SaaS activation gate.

Plugin

• Plugin installation and configuration are Workspace scoped.
• Runtime control actions carry trusted Workspace binding and placement generation.
• The Plugin Runtime supervisor is instance-scoped and intentionally serves multiple Workspaces.
• Every plugin process is bound to exactly one Workspace, installation, generation, revision, and verified artifact digest.
• Same-digest plugin code may be cached once, while worker processes and writable data remain isolated.
• Same-digest plugin dependencies are prepared once in a Runtime-owned immutable environment and mounted read-only into each isolated worker; dependency failure is surfaced before launch and recorded per installation without blocking other desired-state recovery.
• Host API derives Workspace from the connection, installation, and trusted action context, not plugin input.
• Plugin get_bots, models, tools, vector, RAG, configuration, and messaging calls are scoped.
• Plugin Workspace storage no longer uses owner default.
• Plugin page APIs check Membership and installation ownership.
• Local plugin launches use short-lived, one-use registration capabilities bound to manifest identity.

MCP, RAG, and Box

• MCP runtime key contains instance UUID, Workspace UUID, placement generation, and server UUID.
• Same-named MCP servers in two Workspaces do not share sessions.
• Pipeline cannot reference another Workspace's MCP resource.
• RAG collection names and handles are server-derived and Workspace scoped.
• Legacy global vector migration is available only to the local OSS singleton Workspace.
• Object storage paths include instance, Workspace, and placement generation for the fixed-generation OSS runtime.
• Object storage revalidates generation before touching a provider or resolving an opaque key.
• Cloud cutover uses generation-scoped staging plus stable published object references, rather than making the staging generation the durable identity.
• Box persistent and ephemeral namespaces include the required instance, Workspace, and generation scope.
• Same-named Box sessions and processes cannot collide across Workspaces or placement generations.
• Box relay and process I/O reject or retire stale generations.
• External paths and privileged mounts cannot be supplied by an untrusted plugin.
• Cloud attachment host I/O uses query UUIDs and link-free dirfd operations with bounded inode traversal.
• Cloud Skill package paths are Runtime-owned, Workspace-scoped, read-only mounts; Python env/cache stays tenant-writable.
• Skill ZIP preview/install rejects path escape, links, non-regular files, duplicate entries, excessive compression ratio, entry count, per-file size, and total size.
• Cloud Box startup proves Core and Runtime see the same durable volume.
• Cloud Box readiness fails until hard Workspace, Skill, ephemeral-storage, and inode quota capabilities are available.

6. SDK and protocol

• Public Query, Event, Session, and context entities carry backward-compatible Workspace data.
• Action RPC request models carry trusted Workspace binding where required.
• Action enums and callers remain consistent.
• Old plugins continue to deserialize compatible events.
• Plugins cannot select an arbitrary Workspace through a Host API argument.
• Runtime storage uses the bound Workspace UUID.
• SDK API tests pass.
• Runtime tests pass.
• Action consistency script passes.

7. Frontend

• Every browser tenant API request carries the current Workspace selector after bootstrap.
• OSS automatically selects the singleton Workspace.
• OSS does not show Create Workspace or a misleading switcher.
• Workspace settings show current Workspace information.
• Members page lists roles and permissions.
• Invitation creation shows a one-time link when SMTP is unavailable.
• Invitation acceptance supports a signed-out user flow.
• Role controls are hidden or disabled consistently with backend permissions.
• Switching accounts clears stale Workspace query cache and local state.
• User-facing strings support en_US, zh_Hans, and ja_JP.

8. Automated verification

Persistence and authorization

• SQLite fresh install.
• SQLite upgrade from pre-tenant schema, including verified failure recovery.
• PostgreSQL fresh install.
• PostgreSQL upgrade from pre-tenant schema.
• All fixed roles have positive and negative permission-matrix tests.
• Concurrent invitation acceptance creates one Membership.
• Concurrent owner changes never leave zero owners.

Cross-tenant isolation

• Two Workspaces are created through a test-only policy.
• Applicable resource operations and parent-child references have cross-Workspace negative coverage.
• Resource UUID guessing cannot cross Workspace.
• API Key cannot cross Workspace.
• Plugin cannot enumerate or invoke another Workspace's resources.
• Sessions, caches, locks, MCP, RAG, Box, storage, and monitoring do not collide.
• Background jobs cannot execute without an explicit Workspace and placement generation.

Security and revocation

• Space login and binding use purpose-bound, one-time opaque OAuth state; caller-supplied state is rejected.
• OAuth redirects trust only server-configured WebUI or webhook origins, never request Host or Origin headers.
• Dashboard WebSockets revalidate authentication, Membership, resource, permission, and generation per message.
• Public embed WebSockets re-resolve Bot availability and execution binding per message.
• Runtime, storage, Plugin Runtime, MCP, RAG, and Box reject a stale placement generation.
• Unhandled API and webhook failures return a generic error plus request ID without exception text.
• URL user information and sensitive query parameters are redacted before configuration is serialized or logged.

Regression

• LangBot unit tests pass.
• LangBot integration tests pass.
• Frontend lint completes without errors and the production build passes.
• SDK focused and full relevant tests pass.
• LangBot is pinned to the exact pushed SDK commit and cross-repo tests pass against that revision.

9. Real browser E2E

• Start from a clean local data directory.
• First user initializes the singleton Workspace as owner.
• Owner creates an invitation link.
• A second signed-out browser identity accepts the invitation and registers.
• owner, admin, developer, operator, and viewer UI permissions match backend enforcement.
• Direct API calls cannot bypass hidden controls.
• Account switch does not expose prior account or Workspace data.
• Refresh and a new browser tab recover the correct Workspace safely.
• OSS rejects a second Workspace with edition_limit; same-name and same-identifier isolation is covered by the test-only multi-Workspace policy because OSS deliberately has no multi-Workspace browser surface.
• Explicit error states are visible for expired, revoked, reused, and email-mismatched invitations.

10. Completion evidence

• LangBot and SDK branch refs are recorded in the verification report.
• Space git diff is empty relative to the pre-work snapshot.
• Migration output is captured for SQLite and PostgreSQL.
• Test commands and results are recorded.
• Browser E2E actions and observed results are recorded.
• No remaining tenant table, global Service query, owner default, or unscoped runtime key is found by the final audit.