feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>

tests/unit_tests/provider/test_mcp_box_integration.py

@@ -154,7 +154,17 @@ def mcp_module():
 def _make_ap():
     ap = Mock()
     ap.logger = Mock()
+    ap.instance_config = SimpleNamespace(data={'mcp': {'stdio': {'enabled': True}}})
+    ap.workspace_service = Mock()
+    ap.workspace_service.get_execution_binding = AsyncMock(
+        return_value=SimpleNamespace(
+            instance_uuid='instance-a',
+            workspace_uuid='workspace-a',
+            placement_generation=1,
+        )
+    )
     ap.box_service = Mock()
+    ap.box_service.get_managed_process_websocket_connection = AsyncMock(return_value=('ws://box.example/process', {}))
     return ap
 
 
@@ -166,6 +176,11 @@ def _make_session(mcp_module, server_config: dict, ap=None):
         server_config=server_config,
         enable=True,
         ap=ap,
+        execution_context=mcp_module.ExecutionContext(
+            instance_uuid='instance-a',
+            workspace_uuid='workspace-a',
+            placement_generation=1,
+        ),
     )
 
 
@@ -591,6 +606,26 @@ class TestGetRuntimeInfoDict:
         assert info['status'] == 'connecting'
         assert 'box_session_id' not in info
 
+    def test_runtime_error_detail_never_echoes_secret_config(self, mcp_module):
+        s = _make_session(
+            mcp_module,
+            {
+                'name': 'test',
+                'uuid': 'test-uuid',
+                'mode': 'invalid',
+                'headers': {'Authorization': 'Bearer TOPSECRET'},
+                'env': {'API_KEY': 'TOPSECRET'},
+            },
+        )
+        s.status = mcp_module.MCPSessionStatus.ERROR
+        s.error_message = f'Unknown MCP server mode: {s.server_config}'
+
+        info = s.get_runtime_info_dict()
+
+        assert info['error_message'] == 'MCP runtime failed'
+        assert info['error_code'] == 'runtime_error'
+        assert 'TOPSECRET' not in str(info)
+
     def test_runtime_tools_include_parameters(self, mcp_module):
         s = _make_session(
             mcp_module,
@@ -794,7 +829,7 @@ class TestGetRuntimeInfoDict:
         assert ap.box_service.available is True
 
     @pytest.mark.asyncio
-    async def test_enabled_box_timeout_does_not_exhaust_mcp_retry_budget(self, mcp_module, monkeypatch):
+    async def test_enabled_box_timeout_does_not_exhaust_mcp_retry_budget(self, mcp_module):
         ap = _make_ap()
         ap.box_service.available = False
         ap.box_service.enabled = True
@@ -819,14 +854,13 @@ class TestGetRuntimeInfoDict:
                 raise RuntimeError('Box runtime is not available after 1 seconds')
 
         session._lifecycle_loop = lifecycle
-        sleep = AsyncMock()
-        monkeypatch.setattr(mcp_module.asyncio, 'sleep', sleep)
+        session._sleep_with_execution_fence = AsyncMock()
 
         await session._lifecycle_loop_with_retry()
 
         assert attempts == 2
         assert session.retry_count == 0
-        sleep.assert_awaited_once_with(1)
+        session._sleep_with_execution_fence.assert_awaited_once_with(1)
 
     @pytest.mark.asyncio
     async def test_disabled_box_still_stops_mcp_retry_loop(self, mcp_module):
@@ -914,6 +948,31 @@ class TestBoxConfigParsing:
         assert s.box_config.host_path_mode == 'ro'
 
 
+@pytest.mark.asyncio
+async def test_stdio_instance_gate_runs_before_box_transport(mcp_module):
+    ap = _make_ap()
+    ap.instance_config.data['mcp']['stdio']['enabled'] = False
+    ap.box_service.available = True
+    session = _make_session(
+        mcp_module,
+        {
+            'name': 'blocked',
+            'uuid': 'blocked-uuid',
+            'mode': 'stdio',
+            'command': 'python',
+            'args': [],
+            'env': {},
+        },
+        ap=ap,
+    )
+    session._box_stdio_runtime.initialize = AsyncMock()
+
+    with pytest.raises(RuntimeError, match='disabled by instance policy'):
+        await session._init_stdio_python_server()
+
+    session._box_stdio_runtime.initialize.assert_not_awaited()
+
+
 @pytest.mark.asyncio
 async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_module, tmp_path):
     mcp_stdio_module = sys.modules['langbot.pkg.provider.tools.loaders.mcp_stdio']
@@ -931,12 +990,16 @@ async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_mo
         async def initialize(self):
             return None
 
+    captured_transport = {}
+
     @asynccontextmanager
-    async def fake_websocket_client(_url: str):
+    async def fake_authenticated_websocket_client(url: str, headers: dict[str, str]):
+        captured_transport['url'] = url
+        captured_transport['headers'] = headers
         yield ('read-stream', 'write-stream')
 
     mcp_stdio_module.ClientSession = FakeClientSession
-    mcp_stdio_module.websocket_client = fake_websocket_client
+    mcp_stdio_module.authenticated_websocket_client = fake_authenticated_websocket_client
 
     ap = _make_ap()
     ap.box_service.available = True
@@ -947,7 +1010,17 @@ async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_mo
         execute=AsyncMock(return_value=SimpleNamespace(ok=True, stderr='', exit_code=0))
     )
     ap.box_service.start_managed_process = AsyncMock(return_value={})
-    ap.box_service.get_managed_process_websocket_url = Mock(return_value='ws://box.example/process')
+    ap.box_service.get_managed_process_websocket_connection = AsyncMock(
+        return_value=(
+            'ws://box.example/process',
+            {
+                'X-LangBot-Box-Control-Token': 'secret-token',
+                'X-LangBot-Instance-Id': 'instance-a',
+                'X-LangBot-Workspace-Id': 'workspace-a',
+                'X-LangBot-Placement-Generation': '1',
+            },
+        )
+    )
 
     host_path = tmp_path / 'mcp-source'
     host_path.mkdir()
@@ -971,7 +1044,8 @@ async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_mo
     await session.exit_stack.aclose()
 
     assert ap.box_service.create_session.await_count == 1
-    session_payload = ap.box_service.create_session.await_args.args[0]
+    assert ap.box_service.create_session.await_args.args[0] == session.execution_context
+    session_payload = ap.box_service.create_session.await_args.args[1]
     assert session_payload['session_id'] == 'mcp-shared'
     assert 'host_path' not in session_payload
     assert ap.box_service.build_spec.call_count == 1
@@ -981,11 +1055,22 @@ async def test_init_box_stdio_server_stages_host_path_in_shared_workspace(mcp_mo
     staged_file = tmp_path / 'shared-box-workspace' / '.mcp' / 'u1' / 'workspace' / 'server.py'
     assert staged_file.read_text(encoding='utf-8') == 'print("hello")\n'
 
-    process_payload = ap.box_service.start_managed_process.await_args.args[1]
+    assert ap.box_service.start_managed_process.await_args.args[0] == session.execution_context
+    process_payload = ap.box_service.start_managed_process.await_args.args[2]
     assert process_payload['process_id'] == 'u1'
     assert process_payload['command'] == 'python'
     assert process_payload['args'] == ['/workspace/.mcp/u1/workspace/server.py']
     assert process_payload['cwd'] == '/workspace/.mcp/u1/workspace'
+    assert captured_transport == {
+        'url': 'ws://box.example/process',
+        'headers': {
+            'X-LangBot-Box-Control-Token': 'secret-token',
+            'X-LangBot-Instance-Id': 'instance-a',
+            'X-LangBot-Workspace-Id': 'workspace-a',
+            'X-LangBot-Placement-Generation': '1',
+        },
+    }
+    assert 'secret-token' not in captured_transport['url']
 
 
 @pytest.mark.asyncio
@@ -1024,7 +1109,7 @@ async def test_stdio_handshake_raises_coldstart_retry_while_process_alive(mcp_mo
     ap.box_service.available = True
     ap.box_service.create_session = AsyncMock(return_value={})
     ap.box_service.start_managed_process = AsyncMock(return_value={})
-    ap.box_service.get_managed_process_websocket_url = Mock(return_value='ws://box/p')
+    ap.box_service.get_managed_process_websocket_connection = AsyncMock(return_value=('ws://box/p', {}))
 
     session = _make_session(
         mcp_module,
@@ -1090,7 +1175,7 @@ async def test_stdio_handshake_raises_fatal_when_process_exited(mcp_module, tmp_
     ap.box_service.available = True
     ap.box_service.create_session = AsyncMock(return_value={})
     ap.box_service.start_managed_process = AsyncMock(return_value={})
-    ap.box_service.get_managed_process_websocket_url = Mock(return_value='ws://box/p')
+    ap.box_service.get_managed_process_websocket_connection = AsyncMock(return_value=('ws://box/p', {}))
 
     session = _make_session(
         mcp_module,