xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-08-09 04:40:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(tenancy): add Workspace multi-tenant foundation (#2353)

Browse Source 
* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
This commit is contained in:
RockChinQ
2026-07-30 21:43:35 +08:00
committed by GitHub
parent 463b120923
commit e1ac5e0fc8
468 changed files with 78320 additions and 13137 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/langbot/pkg/pipeline/controller.py
+129 -39
View File
@@ -5,8 +5,10 @@ import traceback
from ..core import app
from ..core import entities as core_entities
from ..workspace.errors import WorkspaceError, WorkspaceInvariantError
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from .pool import get_query_execution_context
class Controller:
@@ -21,11 +23,95 @@ class Controller:
self.ap = ap
self.semaphore = asyncio.Semaphore(self.ap.instance_config.data['concurrency']['pipeline'])
async def _assert_query_execution_active(
self,
query: pipeline_query.Query,
):
"""Revalidate a queued query immediately before runtime work starts."""
execution_context = get_query_execution_context(query)
binding = await self.ap.workspace_service.get_execution_binding(
execution_context.workspace_uuid,
expected_generation=execution_context.placement_generation,
)
if binding.instance_uuid != execution_context.instance_uuid:
raise WorkspaceInvariantError('Queued query instance does not match the active Workspace binding')
return execution_context
async def _process_query(
self,
selected_query: pipeline_query.Query,
*,
selected_session=None,
global_slot_reserved: bool = False,
) -> None:
"""Run one selected query and always release its scheduling slot."""
try:
queued_context = get_query_execution_context(selected_query)
async def run_scoped_query() -> None:
execution_context = await self._assert_query_execution_active(selected_query)
pipeline_uuid = selected_query.pipeline_uuid
if pipeline_uuid:
pipeline = await self.ap.pipeline_mgr.get_pipeline_by_uuid(
execution_context,
pipeline_uuid,
)
if pipeline:
await pipeline.run(selected_query)
else:
self.ap.logger.warning(
f'Pipeline {pipeline_uuid} not found for query {selected_query.query_id}, query dropped'
)
else:
self.ap.logger.warning(f'No pipeline_uuid for query {selected_query.query_id}, query dropped')
tenant_scope = getattr(self.ap.persistence_mgr, 'tenant_scope', None)
cloud_runtime = getattr(getattr(self.ap.persistence_mgr, 'mode', None), 'value', None) == 'cloud_runtime'
if cloud_runtime:
if not callable(tenant_scope):
raise RuntimeError('Cloud query processing requires an explicit tenant scope')
async with tenant_scope(queued_context.workspace_uuid):
await run_scoped_query()
else:
await run_scoped_query()
except WorkspaceError as exc:
self.ap.logger.info(
f'Dropped query {selected_query.query_id} because its Workspace execution binding is stale: {exc}'
)
finally:
try:
try:
await self.ap.query_pool.remove_query(selected_query)
finally:
async with self.ap.query_pool:
session = selected_session or await self.ap.sess_mgr.get_session(selected_query)
try:
session._semaphore.release()
finally:
self.ap.query_pool.condition.notify_all()
finally:
if global_slot_reserved:
self.semaphore.release()
async def _drop_selected_query(self, selected_query, selected_session) -> None:
"""Undo scheduler ownership when work cannot be handed to a task."""
try:
await self.ap.query_pool.remove_query(selected_query)
finally:
async with self.ap.query_pool:
selected_session._semaphore.release()
self.ap.query_pool.condition.notify_all()
async def consumer(self):
"""事件处理循环"""
try:
while True:
while True:
try:
selected_query: pipeline_query.Query = None
selected_session = None
# 取请求
async with self.ap.query_pool:
@@ -38,7 +124,9 @@ class Controller:
if not session._semaphore.locked():
selected_query = query
selected_session = session
await session._semaphore.acquire()
self.ap.query_pool.mark_query_running_locked(query)
# Only log when actually selecting a query
self.ap.logger.debug(f'Selected query {query.query_id} for processing')
@@ -51,46 +139,48 @@ class Controller:
continue
if selected_query:
try:
# Reserve global capacity before creating the task.
# At most one selected query is held by this consumer
# while all pipeline slots are busy.
await self.semaphore.acquire()
except asyncio.CancelledError:
await self._drop_selected_query(selected_query, selected_session)
raise
async def _process_query(selected_query: pipeline_query.Query):
async with self.semaphore: # 总并发上限
# find pipeline
# Here firstly find the bot, then find the pipeline, in case the bot adapter's config is not the latest one.
# Like aiocqhttp, once a client is connected, even the adapter was updated and restarted, the existing client connection will not be affected.
pipeline_uuid = selected_query.pipeline_uuid
if pipeline_uuid:
pipeline = await self.ap.pipeline_mgr.get_pipeline_by_uuid(pipeline_uuid)
if pipeline:
await pipeline.run(selected_query)
else:
self.ap.logger.warning(
f'Pipeline {pipeline_uuid} not found for query {selected_query.query_id}, query dropped'
)
else:
self.ap.logger.warning(
f'No pipeline_uuid for query {selected_query.query_id}, query dropped'
)
async with self.ap.query_pool:
(await self.ap.sess_mgr.get_session(selected_query))._semaphore.release()
# 通知其他协程，有新的请求可以处理了
self.ap.query_pool.condition.notify_all()
self.ap.task_mgr.create_task(
_process_query(selected_query),
kind='query',
name=f'query-{selected_query.query_id}',
scopes=[
core_entities.LifecycleControlScope.APPLICATION,
core_entities.LifecycleControlScope.PLATFORM,
],
execution_context = get_query_execution_context(selected_query)
process_coro = self._process_query(
selected_query,
selected_session=selected_session,
global_slot_reserved=True,
)
try:
self.ap.task_mgr.create_task(
process_coro,
kind='query',
name=f'query-{selected_query.query_id}',
scopes=[
core_entities.LifecycleControlScope.APPLICATION,
core_entities.LifecycleControlScope.PLATFORM,
],
instance_uuid=execution_context.instance_uuid,
workspace_uuid=execution_context.workspace_uuid,
placement_generation=execution_context.placement_generation,
)
except Exception:
process_coro.close()
self.semaphore.release()
await self._drop_selected_query(selected_query, selected_session)
raise
except Exception as e:
# traceback.print_exc()
self.ap.logger.error(f'控制器循环出错: {e}')
self.ap.logger.error(f'Traceback: {traceback.format_exc()}')
except asyncio.CancelledError:
raise
except Exception as e:
self.ap.logger.error(f'控制器循环出错: {e}')
self.ap.logger.error(f'Traceback: {traceback.format_exc()}')
# A persistent external failure must not turn this recovery
# loop into a CPU spin.
await asyncio.sleep(1)
async def run(self):
"""运行控制器"""