feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>

src/langbot/pkg/api/http/controller/groups/monitoring.py

@@ -3,6 +3,8 @@ from __future__ import annotations
 import datetime
 import quart
 
+from ...authz import Permission
+from ...context import RequestContext
 from .. import group
 
 
@@ -24,8 +26,8 @@ def parse_iso_datetime(datetime_str: str | None) -> datetime.datetime | None:
 @group.group_class('monitoring', '/api/v1/monitoring')
 class MonitoringRouterGroup(group.RouterGroup):
     async def initialize(self) -> None:
-        @self.route('/overview', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_overview() -> str:
+        @self.route('/overview', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_overview(request_context: RequestContext) -> str:
             """Get overview metrics"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -38,6 +40,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             metrics = await self.ap.monitoring_service.get_overview_metrics(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -46,8 +49,8 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             return self.success(data=metrics)
 
-        @self.route('/token-statistics', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_token_statistics() -> str:
+        @self.route('/token-statistics', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_token_statistics(request_context: RequestContext) -> str:
             """Get detailed token usage statistics (summary, per-model, timeseries)."""
             bot_ids = quart.request.args.getlist('botId')
             pipeline_ids = quart.request.args.getlist('pipelineId')
@@ -61,6 +64,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             stats = await self.ap.monitoring_service.get_token_statistics(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -70,8 +74,8 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             return self.success(data=stats)
 
-        @self.route('/messages', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_messages() -> str:
+        @self.route('/messages', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_messages(request_context: RequestContext) -> str:
             """Get message logs"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -87,6 +91,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             messages, total = await self.ap.monitoring_service.get_messages(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 session_ids=session_ids if session_ids else None,
@@ -105,8 +110,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/llm-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_llm_calls() -> str:
+        @self.route('/llm-calls', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_llm_calls(request_context: RequestContext) -> str:
             """Get LLM call records"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -121,6 +126,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             llm_calls, total = await self.ap.monitoring_service.get_llm_calls(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -138,8 +144,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/tool-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_tool_calls() -> str:
+        @self.route('/tool-calls', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_tool_calls(request_context: RequestContext) -> str:
             """Get tool call records"""
             bot_ids = quart.request.args.getlist('botId')
             pipeline_ids = quart.request.args.getlist('pipelineId')
@@ -153,6 +159,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             tool_calls, total = await self.ap.monitoring_service.get_tool_calls(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 session_ids=session_ids if session_ids else None,
@@ -171,8 +178,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/embedding-calls', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_embedding_calls() -> str:
+        @self.route('/embedding-calls', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_embedding_calls(request_context: RequestContext) -> str:
             """Get embedding call records"""
             # Parse query parameters
             start_time_str = quart.request.args.get('startTime')
@@ -186,6 +193,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             embedding_calls, total = await self.ap.monitoring_service.get_embedding_calls(
+                request_context,
                 start_time=start_time,
                 end_time=end_time,
                 knowledge_base_id=knowledge_base_id if knowledge_base_id else None,
@@ -202,8 +210,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/sessions', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_sessions() -> str:
+        @self.route('/sessions', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_sessions(request_context: RequestContext) -> str:
             """Get session information"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -224,6 +232,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 is_active = is_active_str.lower() == 'true'
 
             sessions, total = await self.ap.monitoring_service.get_sessions(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -242,8 +251,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/errors', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_errors() -> str:
+        @self.route('/errors', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_errors(request_context: RequestContext) -> str:
             """Get error logs"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -258,6 +267,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             errors, total = await self.ap.monitoring_service.get_errors(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -275,8 +285,8 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/data', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_all_data() -> str:
+        @self.route('/data', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_all_data(request_context: RequestContext) -> str:
             """Get all monitoring data in a single request"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -291,6 +301,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get overview metrics
             overview = await self.ap.monitoring_service.get_overview_metrics(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -299,6 +310,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get messages
             messages, messages_total = await self.ap.monitoring_service.get_messages(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -309,6 +321,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get LLM calls
             llm_calls, llm_calls_total = await self.ap.monitoring_service.get_llm_calls(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -319,6 +332,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get tool calls
             tool_calls, tool_calls_total = await self.ap.monitoring_service.get_tool_calls(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -329,6 +343,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get sessions
             sessions, sessions_total = await self.ap.monitoring_service.get_sessions(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -340,6 +355,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get errors
             errors, errors_total = await self.ap.monitoring_service.get_errors(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -350,6 +366,7 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             # Get embedding calls
             embedding_calls, embedding_calls_total = await self.ap.monitoring_service.get_embedding_calls(
+                request_context,
                 start_time=start_time,
                 end_time=end_time,
                 limit=limit,
@@ -376,27 +393,27 @@ class MonitoringRouterGroup(group.RouterGroup):
                 }
             )
 
-        @self.route('/sessions/<session_id>/analysis', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_session_analysis(session_id: str) -> str:
+        @self.route('/sessions/<session_id>/analysis', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_session_analysis(session_id: str, request_context: RequestContext) -> str:
             """Get detailed analysis for a specific session"""
-            analysis = await self.ap.monitoring_service.get_session_analysis(session_id)
+            analysis = await self.ap.monitoring_service.get_session_analysis(request_context, session_id)
 
             # Always return success with the analysis data
             # The frontend will handle the 'found: false' case
             return self.success(data=analysis)
 
-        @self.route('/messages/<message_id>/details', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_message_details(message_id: str) -> str:
+        @self.route('/messages/<message_id>/details', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_message_details(message_id: str, request_context: RequestContext) -> str:
             """Get detailed information for a specific message"""
-            details = await self.ap.monitoring_service.get_message_details(message_id)
+            details = await self.ap.monitoring_service.get_message_details(request_context, message_id)
 
             if not details.get('found'):
                 return self.error(message=f'Message {message_id} not found', code=404)
 
             return self.success(data=details)
 
-        @self.route('/export', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def export_data() -> tuple[str, int]:
+        @self.route('/export', methods=['GET'], permission=Permission.DATA_EXPORT)
+        async def export_data(request_context: RequestContext) -> tuple[str, int]:
             """Export monitoring data as CSV"""
             # Parse query parameters
             export_type = quart.request.args.get('type', 'messages')
@@ -413,6 +430,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             # Get data based on export type
             if export_type == 'messages':
                 data = await self.ap.monitoring_service.export_messages(
+                    request_context,
                     bot_ids=bot_ids if bot_ids else None,
                     pipeline_ids=pipeline_ids if pipeline_ids else None,
                     start_time=start_time,
@@ -437,6 +455,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 ]
             elif export_type == 'llm-calls':
                 data = await self.ap.monitoring_service.export_llm_calls(
+                    request_context,
                     bot_ids=bot_ids if bot_ids else None,
                     pipeline_ids=pipeline_ids if pipeline_ids else None,
                     start_time=start_time,
@@ -463,6 +482,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 ]
             elif export_type == 'embedding-calls':
                 data = await self.ap.monitoring_service.export_embedding_calls(
+                    request_context,
                     start_time=start_time,
                     end_time=end_time,
                     limit=limit,
@@ -485,6 +505,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 ]
             elif export_type == 'errors':
                 data = await self.ap.monitoring_service.export_errors(
+                    request_context,
                     bot_ids=bot_ids if bot_ids else None,
                     pipeline_ids=pipeline_ids if pipeline_ids else None,
                     start_time=start_time,
@@ -506,6 +527,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 ]
             elif export_type == 'sessions':
                 data = await self.ap.monitoring_service.export_sessions(
+                    request_context,
                     bot_ids=bot_ids if bot_ids else None,
                     pipeline_ids=pipeline_ids if pipeline_ids else None,
                     start_time=start_time,
@@ -527,6 +549,7 @@ class MonitoringRouterGroup(group.RouterGroup):
                 ]
             elif export_type == 'feedback':
                 data = await self.ap.monitoring_service.export_feedback(
+                    request_context,
                     bot_ids=bot_ids if bot_ids else None,
                     pipeline_ids=pipeline_ids if pipeline_ids else None,
                     start_time=start_time,
@@ -581,8 +604,8 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             return response, 200
 
-        @self.route('/feedback/stats', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_feedback_stats() -> str:
+        @self.route('/feedback/stats', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_feedback_stats(request_context: RequestContext) -> str:
             """Get feedback statistics"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -595,6 +618,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             end_time = parse_iso_datetime(end_time_str)
 
             stats = await self.ap.monitoring_service.get_feedback_stats(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 start_time=start_time,
@@ -603,8 +627,8 @@ class MonitoringRouterGroup(group.RouterGroup):
 
             return self.success(data=stats)
 
-        @self.route('/feedback', methods=['GET'], auth_type=group.AuthType.USER_TOKEN)
-        async def get_feedback() -> str:
+        @self.route('/feedback', methods=['GET'], permission=Permission.RESOURCE_VIEW)
+        async def get_feedback(request_context: RequestContext) -> str:
             """Get feedback list"""
             # Parse query parameters
             bot_ids = quart.request.args.getlist('botId')
@@ -623,6 +647,7 @@ class MonitoringRouterGroup(group.RouterGroup):
             feedback_type = int(feedback_type_str) if feedback_type_str else None
 
             feedback_list, total = await self.ap.monitoring_service.get_feedback_list(
+                request_context,
                 bot_ids=bot_ids if bot_ids else None,
                 pipeline_ids=pipeline_ids if pipeline_ids else None,
                 feedback_type=feedback_type,