feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture * Add OSS and commercial workspace boundaries * docs: redesign multi-tenant workspace architecture * feat(tenancy): implement workspace isolation * docs(tenancy): record verification evidence * docs(tenancy): revise single-instance SaaS topology * docs(tenancy): refine architecture options * docs: finalize cloud v2 multi-tenant decisions * feat(tenancy): establish cloud isolation foundations * feat(tenancy): harden shared cloud runtime boundaries * docs(tenancy): record final isolation verification * fix(tenancy): close isolation and permission gaps * docs(tenancy): record final isolation verification * feat(tenancy): connect cloud workspace control plane * fix(build): install git for pinned SDK * docs(cloud): update control plane verification * chore: update multi-tenant SDK pin * fix(cloud): skip legacy model sync during startup * test(cloud): preserve minimal model manager fixtures * fix(cloud): preserve authenticated account context * fix(cloud): reuse authenticated account for user info * feat(cloud): complete Workspace settings navigation * test(web): cover Workspace dropdown menu * feat(web): place workspace controls in sidebar * refactor(web): streamline workspace controls * style(web): format workspace layout test * fix(cloud): surface runtime and workspace plan status * fix(plugin): keep runtime identity stable across restarts * fix(ui): widen and center workspace switcher * fix(ui): hide roles from workspace switcher * fix(ui): align workspace switcher with sidebar entries * feat(workspace): add in-product collaboration and direct Cloud launch * style: format collaboration changes * fix(workspace): bind collaboration APIs to tenant UoW * fix(cloud): preserve Core-owned collaboration state * test(cloud): require Space identity for invite registration * feat(cloud): complete secure invitation experience * style(web): format invitation flows * fix(cloud): recover box runtime without unscoped skill reload * feat(oss): enforce invitation account and owner billing flows * style: format OSS account service * test(oss): cover invitation logout handoff * fix(oss): resolve workspace owner in scoped session * feat(cloud): harden multi-tenant runtime resources * fix(cloud): bound runtime restart storms * fix(cloud): eliminate periodic runtime CPU spikes * fix(cloud): enforce instance capacity ceilings * fix(cloud): scope public login capability discovery * fix(cloud): bound tenant maintenance and monitoring work * fix(runtime): bound tenant resource amplification * fix(deps): pin green multi-tenant plugin SDK * fix(cloud): handle unavailable skill capability * fix(security): require authentication for image file endpoint (H-2) - Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY - Added Permission.RESOURCE_VIEW requirement - Prevents unauthenticated cross-tenant file access via leaked keys - Fixes HIGH severity finding from multi-tenant security review docs: add comprehensive database migration guide - Complete migration steps for OSS → multi-tenant - Backup, execution, verification procedures - Rollback scenarios and recovery plans - Performance tuning recommendations * test: add comprehensive cross-tenant isolation tests Added 7 critical test scenarios for multi-tenant boundaries: - Cross-tenant bot access prevention - Viewer role read-only enforcement - Removed member immediate access revocation - Model provider credential isolation - WebSocket message isolation - Invitation token workspace scoping - Multi-workspace context validation These tests address P0-2 coverage gaps for: - workspaces.py (membership & invitation flows) - user.py (authentication & authorization) - websocket_chat.py (real-time isolation) - plugins.py (resource access control) docs: finalize database migration guide * fix(security): resolve M-1, M-2, M-3 security findings M-1: WebSocket authorization TOCTOU race (FIXED) - Changed _revalidate_websocket_authorization to return RequestContext - Ensures validated context is used immediately without race window - Prevents removed members from sending messages during revalidation gap M-2: Model Manager cache workspace isolation (VERIFIED) - Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource) - Cache is properly scoped per workspace, no cross-tenant leakage possible - No code change needed, documented as working correctly M-3: Invitation lock workspace scoping (FIXED) - Changed lock key from token_digest to workspace_uuid:token_digest - Prevents DoS where attacker locks token in Workspace A to block Workspace B - Locks now isolated per workspace All MEDIUM severity findings from security review now resolved. * fix(cloud): unblock tenant CI and enforce knowledge quotas * fix(tenancy): scope rerank model sync --------- Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
2026-08-09 04:40:57 +00:00 · 2026-07-30 21:43:35 +08:00
parent 463b120923
commit e1ac5e0fc8
468 changed files with 78320 additions and 13137 deletions
@@ -10,13 +10,16 @@ import re
 from typing import Any, Callable, Optional, Tuple
 from urllib.parse import unquote

-import httpx
 from Crypto.Cipher import AES
 from quart import Quart, request, Response, jsonify

 from langbot.libs.wecom_ai_bot_api import wecombotevent
 from langbot.libs.wecom_ai_bot_api.WXBizMsgCrypt3 import WXBizMsgCrypt
 from langbot.pkg.platform.logger import EventLogger
+from langbot.pkg.utils import httpclient
+
+_CLIENT_TRANSIENT_CACHE_MAX = 4096
+_MAX_STREAM_CONTENT_CHARS = 200000


@dataclass
@@ -56,7 +59,7 @@ class StreamSession:
    last_access: float = field(default_factory=time.time)

    # 将流水线增量结果缓存到队列，刷新请求逐条消费
-    queue: asyncio.Queue = field(default_factory=asyncio.Queue)
+    queue: asyncio.Queue = field(default_factory=lambda: asyncio.Queue(maxsize=1))

    # 是否已经完成（收到最终片段）
    finished: bool = False
@@ -85,6 +88,7 @@ class StreamSessionManager:
    # full like → cancel → dislike feedback flow. Must align with the adapter's
    # _stream_to_monitoring_msg TTL (wecombot.py).
    _FEEDBACK_SESSION_TTL = 600  # 10 minutes
+    _MAX_SESSIONS = 4096

    def __init__(self, logger: EventLogger, ttl: int = 60) -> None:
        self.logger = logger
@@ -165,6 +169,26 @@ class StreamSessionManager:
        if task_id:
            self._task_index.pop(task_id, None)

+    def clear(self) -> None:
+        """Release every retained stream and reverse index."""
+
+        self._sessions.clear()
+        self._msg_index.clear()
+        self._feedback_index.clear()
+        self._task_index.clear()
+
+    def _drop_session(self, stream_id: str) -> StreamSession | None:
+        session = self._sessions.pop(stream_id, None)
+        if session is None:
+            return None
+        if session.msg_id and self._msg_index.get(session.msg_id) == stream_id:
+            self._msg_index.pop(session.msg_id, None)
+        if session.feedback_id:
+            self._feedback_index.pop(session.feedback_id, None)
+        if session.pending_form_task_id:
+            self._task_index.pop(session.pending_form_task_id, None)
+        return session
+
    def create_or_get(self, msg_json: dict[str, Any]) -> tuple[StreamSession, bool]:
        """根据企业微信回调创建或获取会话。

@@ -185,6 +209,14 @@ class StreamSessionManager:
                session.last_access = time.time()
                return session, False

+        self.cleanup()
+        while len(self._sessions) >= self._MAX_SESSIONS:
+            oldest_stream_id = min(
+                self._sessions,
+                key=lambda candidate: self._sessions[candidate].last_access,
+            )
+            self._drop_session(oldest_stream_id)
+
        stream_id = str(uuid.uuid4())
        session = StreamSession(
            stream_id=stream_id,
@@ -221,8 +253,13 @@ class StreamSessionManager:
        try:
            session.queue.put_nowait(chunk)
        except asyncio.QueueFull:
-            # 默认无界队列，此处兜底防御
-            await session.queue.put(chunk)
+            # Each chunk is a complete snapshot. Coalesce a slow consumer to
+            # the newest value instead of retaining every intermediate body.
+            try:
+                session.queue.get_nowait()
+            except asyncio.QueueEmpty:
+                pass
+            session.queue.put_nowait(chunk)

        if chunk.is_final:
            session.finished = True
@@ -265,7 +302,7 @@ class StreamSessionManager:
            session.finished = True
            session.last_access = time.time()

-    def cleanup(self) -> None:
+    def cleanup(self) -> list[str]:
        """定期清理过期会话，防止队列与映射无上限累积。

        已注册 feedback_id 的会话使用更长的 TTL，确保用户在点赞/取消/点踩流程中
@@ -279,16 +316,14 @@ class StreamSessionManager:
            if now - session.last_access > effective_ttl:
                expired.append(stream_id)

+        removed_msg_ids: list[str] = []
        for stream_id in expired:
-            session = self._sessions.pop(stream_id, None)
+            session = self._drop_session(stream_id)
            if not session:
                continue
-            msg_id = session.msg_id
-            if msg_id and self._msg_index.get(msg_id) == stream_id:
-                self._msg_index.pop(msg_id, None)
-            # Clean up feedback index for expired sessions
-            if session.feedback_id:
-                self._feedback_index.pop(session.feedback_id, None)
+            if session.msg_id:
+                removed_msg_ids.append(session.msg_id)
+        return removed_msg_ids


 def _decrypt_file(encrypted_data: bytes, aes_key_str: str) -> bytes:
@@ -405,19 +440,19 @@ async def download_encrypted_file(

    filename: Optional[str] = None
    try:
-        async with httpx.AsyncClient(timeout=30.0) as client:
-            response = await client.get(download_url)
-            if response.status_code != 200:
-                await logger.error(f'Failed to download file (HTTP {response.status_code}): {response.text[:200]}')
+        client = httpclient.get_session()
+        async with client.get(download_url, timeout=30.0) as response:
+            if response.status != 200:
+                await logger.error(f'Failed to download file (HTTP {response.status})')
                return None, None
-            encrypted_bytes = response.content
+            encrypted_bytes = await httpclient.read_limited(response)
            filename = _extract_filename(response.headers.get('content-disposition', ''))
    except Exception:
        await logger.error(f'Failed to download file: {traceback.format_exc()}')
        return None, None

    try:
-        decrypted = _decrypt_file(encrypted_bytes, aes_key)
+        decrypted = await asyncio.to_thread(_decrypt_file, encrypted_bytes, aes_key)
        return decrypted, filename
    except Exception:
        await logger.error(f'Failed to decrypt file: {traceback.format_exc()}')
@@ -466,7 +501,7 @@ async def parse_wecom_bot_message(
        """Download, decrypt, and convert to data URI for backward compatibility."""
        data, _filename = await _safe_download(url, per_msg_aeskey)
        if data:
-            return _bytes_to_data_uri(data)
+            return await asyncio.to_thread(_bytes_to_data_uri, data)
        return None

    if msg_type == 'text':
@@ -579,7 +614,10 @@ async def parse_wecom_bot_message(
                if (file_data.get('filesize') or 0) <= max_inline_file_size:
                    file_bytes, dl_filename = await _safe_download(download_url, item_aeskey)
                    if file_bytes:
-                        file_data['base64'] = _bytes_to_data_uri(file_bytes)
+                        file_data['base64'] = await asyncio.to_thread(
+                            _bytes_to_data_uri,
+                            file_bytes,
+                        )
                        if dl_filename and not file_data.get('filename'):
                            file_data['filename'] = dl_filename
                files.append(file_data)
@@ -1567,6 +1605,8 @@ def build_multiple_interaction_update_card(


 class WecomBotClient:
+    _MAX_DISPATCH_TASKS = 100
+
    def __init__(
        self,
        Token: str,
@@ -1613,6 +1653,7 @@ class WecomBotClient:
        self._feedback_callback: Optional[Callable] = None
        self._card_action_callback: Optional[Callable] = None
        self._stream_last_content: dict[str, str] = {}
+        self._dispatch_tasks: set[asyncio.Task] = set()
        # Optional `source` block injected into every interactive template_card
        # the client builds. Set via `set_card_source` from the adapter after
        # reading config. Format: {icon_url, desc, desc_color}.
@@ -1695,7 +1736,12 @@ class WecomBotClient:
        """
        reply_plain_str = json.dumps(payload, ensure_ascii=False)
        reply_timestamp = str(int(time.time()))
-        ret, encrypt_text = self.wxcpt.EncryptMsg(reply_plain_str, nonce, reply_timestamp)
+        ret, encrypt_text = await asyncio.to_thread(
+            self.wxcpt.EncryptMsg,
+            reply_plain_str,
+            nonce,
+            reply_timestamp,
+        )
        if ret != 0:
            await self.logger.error(f'加密失败: {ret}')
            return jsonify({'error': 'encrypt_failed'}), 500
@@ -1718,6 +1764,41 @@ class WecomBotClient:
        except Exception:
            await self.logger.error(traceback.format_exc())

+    def _start_dispatch_task(self, event: wecombotevent.WecomBotEvent) -> bool:
+        """Start one bounded pipeline dispatch task."""
+
+        for task in tuple(self._dispatch_tasks):
+            if task.done():
+                self._dispatch_tasks.discard(task)
+        if len(self._dispatch_tasks) >= self._MAX_DISPATCH_TASKS:
+            return False
+
+        task = asyncio.create_task(self._dispatch_event(event))
+        self._dispatch_tasks.add(task)
+
+        def done(done_task: asyncio.Task) -> None:
+            self._dispatch_tasks.discard(done_task)
+            if not done_task.cancelled():
+                done_task.exception()
+
+        task.add_done_callback(done)
+        return True
+
+    async def close(self) -> None:
+        """Cancel callbacks and release retained webhook state."""
+
+        dispatch_tasks = list(self._dispatch_tasks)
+        for task in dispatch_tasks:
+            if not task.done():
+                task.cancel()
+        if dispatch_tasks:
+            await asyncio.gather(*dispatch_tasks, return_exceptions=True)
+        self._dispatch_tasks.clear()
+        self.generated_content.clear()
+        self.msg_id_map.clear()
+        self._stream_last_content.clear()
+        self.stream_sessions.clear()
+
    async def _handle_post_initial_response(self, msg_json: dict[str, Any], nonce: str) -> tuple[Response, int]:
        """处理企业微信首次推送的消息，返回 stream_id 并开启流水线。

@@ -1747,7 +1828,8 @@ class WecomBotClient:
                await self.logger.error(traceback.format_exc())
            else:
                if is_new:
-                    asyncio.create_task(self._dispatch_event(event))
+                    if not self._start_dispatch_task(event):
+                        await self.logger.warning('WeCom webhook dispatch capacity reached; dropping message')

        payload = self._build_stream_payload(session.stream_id, '', False, feedback_id)
        return await self._encrypt_and_reply(payload, nonce)
@@ -1870,7 +1952,10 @@ class WecomBotClient:
    async def _handle_post_callback(self, req) -> tuple[Response, int] | Response:
        """处理企业微信的 POST 回调请求。"""

-        self.stream_sessions.cleanup()
+        for expired_msg_id in self.stream_sessions.cleanup():
+            self.generated_content.pop(expired_msg_id, None)
+            self._stream_last_content.pop(expired_msg_id, None)
+            self.msg_id_map.pop(expired_msg_id, None)

        msg_signature = unquote(req.args.get('msg_signature', ''))
        timestamp = unquote(req.args.get('timestamp', ''))
@@ -1883,12 +1968,18 @@ class WecomBotClient:
            return Response('Bad Request', status=400)

        xml_post_data = f'<xml><Encrypt><![CDATA[{encrypted_msg}]]></Encrypt></xml>'
-        ret, decrypted_xml = self.wxcpt.DecryptMsg(xml_post_data, msg_signature, timestamp, nonce)
+        ret, decrypted_xml = await asyncio.to_thread(
+            self.wxcpt.DecryptMsg,
+            xml_post_data,
+            msg_signature,
+            timestamp,
+            nonce,
+        )
        if ret != 0:
            await self.logger.error('解密失败')
            return Response('解密失败', status=400)

-        msg_json = json.loads(decrypted_xml)
+        msg_json = await asyncio.to_thread(json.loads, decrypted_xml)

        event_type = extract_wecom_event_type(msg_json)

@@ -2014,6 +2105,8 @@ class WecomBotClient:
                self.msg_id_map[message_id] += 1
                return
            self.msg_id_map[message_id] = 1
+            while len(self.msg_id_map) > _CLIENT_TRANSIENT_CACHE_MAX:
+                self.msg_id_map.pop(next(iter(self.msg_id_map)), None)
            msg_type = event.type
            if msg_type in self._message_handlers:
                for handler in self._message_handlers[msg_type]:
@@ -2047,6 +2140,8 @@ class WecomBotClient:
            next_content = previous_content
        else:
            next_content = previous_content + content if previous_content else content
+        if len(next_content) > _MAX_STREAM_CONTENT_CHARS:
+            next_content = next_content[-_MAX_STREAM_CONTENT_CHARS:]

        if not is_final and next_content == previous_content:
            return True
@@ -2096,7 +2191,9 @@ class WecomBotClient:
        """
        handled = await self.push_stream_chunk(msg_id, content, is_final=True)
        if not handled:
-            self.generated_content[msg_id] = content
+            self.generated_content[msg_id] = content[-_MAX_STREAM_CONTENT_CHARS:]
+            while len(self.generated_content) > _CLIENT_TRANSIENT_CACHE_MAX:
+                self.generated_content.pop(next(iter(self.generated_content)), None)

    def on_message(self, msg_type: str):
        def decorator(func: Callable[[wecombotevent.WecomBotEvent], None]):
@@ -2119,7 +2216,7 @@ class WecomBotClient:
    async def download_url_to_base64(self, download_url, encoding_aes_key):
        data, _filename = await download_encrypted_file(download_url, encoding_aes_key, self.logger)
        if data:
-            return _bytes_to_data_uri(data)
+            return await asyncio.to_thread(_bytes_to_data_uri, data)
        return None

    async def run_task(self, host: str, port: int, *args, **kwargs):