feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>

src/langbot/pkg/provider/tools/loaders/mcp_stdio.py

@@ -6,10 +6,12 @@ import os
 import shutil
 import shlex
 import threading
-from contextlib import suppress, AsyncExitStack
+import weakref
+from contextlib import suppress, AsyncExitStack, asynccontextmanager
 from typing import TYPE_CHECKING, Any
 
 import pydantic
+from ....utils import bounded_executor
 from mcp import ClientSession
 from mcp.client.websocket import websocket_client
 from ....box.workspace import (
@@ -27,7 +29,7 @@ if TYPE_CHECKING:
     from .mcp import RuntimeMCPSession
 
 
-_WORKSPACE_COPY_LOCKS: dict[str, threading.Lock] = {}
+_WORKSPACE_COPY_LOCKS: weakref.WeakValueDictionary[str, threading.Lock] = weakref.WeakValueDictionary()
 _WORKSPACE_COPY_LOCKS_GUARD = threading.Lock()
 
 
@@ -94,6 +96,60 @@ class MCPServerBoxConfig(pydantic.BaseModel):
 _HANDSHAKE_ATTEMPT_TIMEOUT_SEC = 10.0
 
 
+@asynccontextmanager
+async def authenticated_websocket_client(url: str, headers: dict[str, str]):
+    """MCP WebSocket transport with host-only Box relay headers.
+
+    The upstream MCP helper does not expose WebSocket handshake headers. This
+    mirrors that transport while keeping the Box control token out of the URL,
+    JSON-RPC payloads, and logs.
+    """
+
+    import json
+
+    import anyio
+    import mcp.types as mcp_types
+    from mcp.shared.message import SessionMessage
+    from pydantic import ValidationError
+    from websockets.asyncio.client import connect as ws_connect
+    from websockets.typing import Subprotocol
+
+    read_stream_writer, read_stream = anyio.create_memory_object_stream(0)
+    write_stream, write_stream_reader = anyio.create_memory_object_stream(0)
+
+    async with ws_connect(
+        url,
+        subprotocols=[Subprotocol('mcp')],
+        additional_headers=dict(headers),
+        proxy=None,
+    ) as websocket:
+
+        async def ws_reader():
+            async with read_stream_writer:
+                async for raw_text in websocket:
+                    try:
+                        message = mcp_types.JSONRPCMessage.model_validate_json(raw_text)
+                        await read_stream_writer.send(SessionMessage(message))
+                    except ValidationError as exc:  # pragma: no cover - upstream parity
+                        await read_stream_writer.send(exc)
+
+        async def ws_writer():
+            async with write_stream_reader:
+                async for session_message in write_stream_reader:
+                    payload = session_message.message.model_dump(
+                        by_alias=True,
+                        mode='json',
+                        exclude_none=True,
+                    )
+                    await websocket.send(json.dumps(payload))
+
+        async with anyio.create_task_group() as task_group:
+            task_group.start_soon(ws_reader)
+            task_group.start_soon(ws_writer)
+            yield read_stream, write_stream
+            task_group.cancel_scope.cancel()
+
+
 class _TransferredStack:
     """Adapts an already-populated AsyncExitStack into an async context manager
     so ownership of its resources can be transferred into another exit stack.
@@ -149,6 +205,7 @@ class BoxStdioSessionRuntime:
         resolved_host_path = self.resolve_host_path() if host_path is ... else host_path
         return BoxWorkspaceSession(
             self.ap.box_service,
+            self.owner.execution_context,
             self.owner._build_box_session_id(),
             host_path=resolved_host_path,
             host_path_mode=self.config.host_path_mode,
@@ -249,7 +306,11 @@ class BoxStdioSessionRuntime:
                 if install_cmd:
                     payload = self._wrap_process_payload_with_python_env(payload, process_cwd)
                 payload['process_id'] = self.process_id
-                await workspace.box_service.start_managed_process(workspace.session_id, payload)
+                await workspace.box_service.start_managed_process(
+                    workspace.execution_context,
+                    workspace.session_id,
+                    payload,
+                )
             except Exception:
                 self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
                 raise
@@ -259,7 +320,10 @@ class BoxStdioSessionRuntime:
                 f'process_id={self.process_id} (transport reconnect)'
             )
 
-        websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
+        (
+            websocket_url,
+            websocket_headers,
+        ) = await workspace.get_managed_process_websocket_connection(self.process_id)
 
         # Attach the WS transport + MCP session ONCE, on the owner's exit stack,
         # in the same task as the serve loop that follows. websocket_client and
@@ -277,7 +341,12 @@ class BoxStdioSessionRuntime:
         # attempt re-attaches to the same live process; once it has finished
         # cold start the handshake succeeds and stays healthy.
         try:
-            transport = await self.owner.exit_stack.enter_async_context(websocket_client(websocket_url))
+            transport_context = (
+                authenticated_websocket_client(websocket_url, websocket_headers)
+                if websocket_headers
+                else websocket_client(websocket_url)
+            )
+            transport = await self.owner.exit_stack.enter_async_context(transport_context)
             read_stream, write_stream = transport
             self.owner.session = await self.owner.exit_stack.enter_async_context(
                 ClientSession(read_stream, write_stream)
@@ -469,7 +538,11 @@ class BoxStdioSessionRuntime:
             return
         try:
             process_host_root = os.path.join(self._shared_workspace_host_path(), '.mcp', self.process_id)
-            await asyncio.to_thread(shutil.rmtree, process_host_root, True)
+            await bounded_executor.run_blocking_cleanup(
+                shutil.rmtree,
+                process_host_root,
+                True,
+            )
         except Exception as exc:
             self.ap.logger.warning(
                 f'MCP server {self.server_name}: failed to clean staged workspace '