xinyin025/LangBot
1
0
Fork 0
mirror of https://github.com/langbot-app/LangBot.git synced 2026-08-09 20:50:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(tenancy): add Workspace multi-tenant foundation (#2353)

Browse Source 
* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
This commit is contained in:
RockChinQ
2026-07-30 21:43:35 +08:00
committed by GitHub
parent 463b120923
commit e1ac5e0fc8
468 changed files with 78320 additions and 13137 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/langbot/pkg/vector/mgr.py
+343 -21
View File
@@ -1,6 +1,15 @@
from __future__ import annotations
import uuid
import sqlalchemy
from ..api.http.authz import WorkspaceRequiredError
from ..api.http.context import ExecutionContext
from ..core import app
from ..entity.persistence import rag as persistence_rag
from ..entity.persistence import workspace as persistence_workspace
from ..workspace.errors import WorkspaceNotFoundError
from .vdb import VectorDatabase, SearchType
@@ -55,9 +64,26 @@ class VectorDBManager:
# Get pgvector configuration
pgvector_config = kb_config.get('pgvector', {})
use_business_database = pgvector_config.get('use_business_database', False)
allowed_dimensions = pgvector_config.get(
'allowed_dimensions',
[384, 512, 768, 1024, 1536],
)
common_options = {
'use_business_database': use_business_database,
'allowed_dimensions': allowed_dimensions,
}
if use_business_database:
self.vector_db = PgVectorDatabase(self.ap, **common_options)
self.ap.logger.info('Initialized pgvector on the shared business PostgreSQL database.')
return
connection_string = pgvector_config.get('connection_string')
if connection_string:
self.vector_db = PgVectorDatabase(self.ap, connection_string=connection_string)
self.vector_db = PgVectorDatabase(
self.ap,
connection_string=connection_string,
**common_options,
)
else:
# Use individual parameters
host = pgvector_config.get('host', 'localhost')
@@ -66,7 +92,13 @@ class VectorDBManager:
user = pgvector_config.get('user', 'postgres')
password = pgvector_config.get('password', 'postgres')
self.vector_db = PgVectorDatabase(
self.ap, host=host, port=port, database=database, user=user, password=password
self.ap,
host=host,
port=port,
database=database,
user=user,
password=password,
**common_options,
)
self.ap.logger.info('Initialized pgvector database backend.')
@@ -81,32 +113,227 @@ class VectorDBManager:
self.vector_db = ChromaVectorDatabase(self.ap)
self.ap.logger.warning('No vector database backend configured, defaulting to Chroma.')
async def shutdown(self) -> None:
"""Release the active vector backend deterministically."""
vector_db = self.vector_db
self.vector_db = None
if vector_db is not None:
await vector_db.close()
def get_supported_search_types(self) -> list[str]:
"""Return the search types supported by the current VDB backend."""
if self.vector_db is None:
return [SearchType.VECTOR.value]
return [st.value for st in self.vector_db.supported_search_types()]
@staticmethod
def physical_collection_name(
execution_context: ExecutionContext,
knowledge_base_uuid: str,
) -> str:
"""Derive an opaque physical collection from trusted tenant identity.
Vector backends have different collection-name constraints, so the
instance, Workspace and knowledge-base identifiers are encoded through
UUIDv5 instead of being concatenated into a client-visible handle.
Placement generation is deliberately not part of the name: generation
fencing rejects stale work while preserving data across placements.
"""
if not isinstance(execution_context, ExecutionContext):
raise WorkspaceRequiredError('ExecutionContext is required for vector access')
instance_uuid = execution_context.instance_uuid.strip()
workspace_uuid = execution_context.workspace_uuid.strip()
kb_uuid = knowledge_base_uuid.strip() if isinstance(knowledge_base_uuid, str) else ''
if not instance_uuid or not workspace_uuid or not kb_uuid:
raise WorkspaceRequiredError('Instance, Workspace and knowledge-base context are required')
if execution_context.placement_generation <= 0:
raise WorkspaceRequiredError('A positive placement generation is required')
collection_uuid = uuid.uuid5(
uuid.NAMESPACE_URL,
f'langbot:knowledge-vector:{instance_uuid}:{workspace_uuid}:{kb_uuid}',
)
return f'lb_{collection_uuid.hex}'
async def _validate_execution_context(self, execution_context: ExecutionContext) -> None:
"""Validate the active placement before touching a vector backend."""
# Also performs structural validation before accessing app services.
self.physical_collection_name(execution_context, 'context-validation')
workspace_service = getattr(self.ap, 'workspace_service', None)
if workspace_service is None:
raise WorkspaceRequiredError('Workspace execution service is unavailable')
binding = await workspace_service.get_execution_binding(
execution_context.workspace_uuid,
expected_generation=execution_context.placement_generation,
)
if binding.instance_uuid != execution_context.instance_uuid:
raise WorkspaceRequiredError('ExecutionContext belongs to another LangBot instance')
async def _resolve_physical_collection_name(
self,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
) -> str:
"""Resolve a scoped collection or an explicitly migrated OSS handle.
Legacy handles are server-owned migration state, not caller input.
They are honored only for the one local Workspace under the OSS
single-Workspace policy. A projected/cloud Workspace always gets the
opaque tenant-derived collection, even if its database row was
incorrectly marked as legacy.
"""
await self._validate_execution_context(execution_context)
async with self.ap.persistence_mgr.tenant_uow(execution_context.workspace_uuid):
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(
persistence_rag.KnowledgeBase.collection_id,
persistence_rag.KnowledgeBase.legacy_vector_collection,
persistence_workspace.Workspace.source,
)
.join(
persistence_workspace.Workspace,
persistence_workspace.Workspace.uuid == persistence_rag.KnowledgeBase.workspace_uuid,
)
.where(
persistence_rag.KnowledgeBase.workspace_uuid == execution_context.workspace_uuid,
persistence_rag.KnowledgeBase.uuid == knowledge_base_uuid,
persistence_workspace.Workspace.instance_uuid == execution_context.instance_uuid,
)
.limit(1)
)
row = result.first()
if row is None:
raise WorkspaceNotFoundError('Knowledge base not found')
collection_id, legacy_vector_collection, workspace_source = row
if legacy_vector_collection:
policy = getattr(self.ap, 'workspace_policy', None)
is_single_workspace = policy is not None and not getattr(
policy,
'multi_workspace_enabled',
True,
)
is_local_workspace = workspace_source == persistence_workspace.WorkspaceSource.LOCAL.value
if is_single_workspace and is_local_workspace and isinstance(collection_id, str) and collection_id.strip():
return collection_id
self.ap.logger.warning(
'Ignored a legacy vector collection marker outside the local single-Workspace compatibility boundary.'
)
return self.physical_collection_name(execution_context, knowledge_base_uuid)
def _pgvector_database(self):
from .vdbs.pgvector_db import PgVectorDatabase
return self.vector_db if isinstance(self.vector_db, PgVectorDatabase) else None
async def _resolve_pgvector_scope(
self,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
*,
expected_dimension: int | None,
initialize_dimension: bool,
):
"""Bind and verify the server-owned knowledge-base vector dimension."""
from .vdbs.pgvector_db import PgVectorScope
pgvector = self._pgvector_database()
if pgvector is None: # pragma: no cover - private call invariant
raise RuntimeError('pgvector scope requested for another vector backend')
if expected_dimension is not None and expected_dimension not in pgvector.allowed_dimensions:
raise ValueError(f'Embedding dimension {expected_dimension} is not enabled for this deployment')
async with self.ap.persistence_mgr.tenant_uow(execution_context.workspace_uuid):
query = sqlalchemy.select(persistence_rag.KnowledgeBase.embedding_dimension).where(
persistence_rag.KnowledgeBase.workspace_uuid == execution_context.workspace_uuid,
persistence_rag.KnowledgeBase.uuid == knowledge_base_uuid,
)
current_dimension = (await self.ap.persistence_mgr.execute_async(query)).scalar_one_or_none()
if current_dimension is None and expected_dimension is not None and initialize_dimension:
await self.ap.persistence_mgr.execute_async(
sqlalchemy.update(persistence_rag.KnowledgeBase)
.where(
persistence_rag.KnowledgeBase.workspace_uuid == execution_context.workspace_uuid,
persistence_rag.KnowledgeBase.uuid == knowledge_base_uuid,
persistence_rag.KnowledgeBase.embedding_dimension.is_(None),
)
.values(embedding_dimension=expected_dimension)
)
current_dimension = (await self.ap.persistence_mgr.execute_async(query)).scalar_one_or_none()
if expected_dimension is not None and current_dimension != expected_dimension:
if current_dimension is None:
raise ValueError('Knowledge base has no selected pgvector embedding dimension')
raise ValueError(f'Knowledge base embedding dimension is {current_dimension}, not {expected_dimension}')
return PgVectorScope(
workspace_uuid=execution_context.workspace_uuid,
knowledge_base_uuid=knowledge_base_uuid,
embedding_dimension=current_dimension,
)
async def upsert(
self,
collection_name: str,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
vectors: list[list[float]],
ids: list[str],
metadata: list[dict] | None = None,
documents: list[str] | None = None,
):
"""Proxy: Upsert vectors"""
"""Upsert vectors into a server-derived tenant collection."""
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
source_metadata = metadata or [{} for _ in vectors]
scoped_metadata = [
{
**item,
'_langbot_instance_uuid': execution_context.instance_uuid,
'_langbot_workspace_uuid': execution_context.workspace_uuid,
'_langbot_knowledge_base_uuid': knowledge_base_uuid,
}
for item in source_metadata
]
pgvector = self._pgvector_database()
if pgvector is not None:
if not vectors:
return
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=len(vectors[0]),
initialize_dimension=True,
)
await pgvector.add_embeddings(
collection=collection_name,
ids=ids,
embeddings_list=vectors,
metadatas=scoped_metadata,
documents=documents,
scope=scope,
)
return
await self.vector_db.add_embeddings(
collection=collection_name,
ids=ids,
embeddings_list=vectors,
metadatas=metadata or [{} for _ in vectors],
metadatas=scoped_metadata,
documents=documents,
)
async def search(
self,
collection_name: str,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
query_vector: list[float],
limit: int,
filter: dict | None = None,
@@ -120,15 +347,38 @@ class VectorDBManager:
The underlying VectorDatabase.search returns Chroma-style format:
{ 'ids': [['id1']], 'distances': [[0.1]], 'metadatas': [[{}]] }
"""
results = await self.vector_db.search(
collection=collection_name,
query_embedding=query_vector,
k=limit,
search_type=search_type,
query_text=query_text,
filter=filter,
vector_weight=vector_weight,
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
pgvector = self._pgvector_database()
if pgvector is not None:
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=len(query_vector),
initialize_dimension=False,
)
results = await pgvector.search(
collection=collection_name,
query_embedding=query_vector,
k=limit,
search_type=search_type,
query_text=query_text,
filter=filter,
vector_weight=vector_weight,
scope=scope,
)
else:
results = await self.vector_db.search(
collection=collection_name,
query_embedding=query_vector,
k=limit,
search_type=search_type,
query_text=query_text,
filter=filter,
vector_weight=vector_weight,
)
if not results or 'ids' not in results or not results['ids']:
return []
@@ -154,30 +404,89 @@ class VectorDBManager:
return parsed_results
async def delete_by_file_id(self, collection_name: str, file_ids: list[str]):
async def delete_by_file_id(
self,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
file_ids: list[str],
):
"""Proxy: Delete vectors by file_id (metadata-level identifier).
This delegates to VectorDatabase.delete_by_file_id which removes
all vectors associated with the given file IDs.
"""
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
pgvector = self._pgvector_database()
scope = None
if pgvector is not None:
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=None,
initialize_dimension=False,
)
for file_id in file_ids:
await self.vector_db.delete_by_file_id(collection_name, file_id)
if pgvector is not None:
await pgvector.delete_by_file_id(collection_name, file_id, scope=scope)
else:
await self.vector_db.delete_by_file_id(collection_name, file_id)
async def delete_collection(self, collection_name: str):
"""Proxy: Delete an entire collection."""
await self.vector_db.delete_collection(collection_name)
async def delete_collection(
self,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
):
"""Delete one server-derived tenant collection."""
async def delete_by_filter(self, collection_name: str, filter: dict) -> int:
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
pgvector = self._pgvector_database()
if pgvector is not None:
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=None,
initialize_dimension=False,
)
await pgvector.delete_collection(collection_name, scope=scope)
else:
await self.vector_db.delete_collection(collection_name)
async def delete_by_filter(
self,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
filter: dict,
) -> int:
"""Proxy: Delete vectors by metadata filter.
Returns:
Number of deleted vectors (best-effort; some backends return 0).
"""
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
pgvector = self._pgvector_database()
if pgvector is not None:
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=None,
initialize_dimension=False,
)
return await pgvector.delete_by_filter(collection_name, filter, scope=scope)
return await self.vector_db.delete_by_filter(collection_name, filter)
async def list_by_filter(
self,
collection_name: str,
execution_context: ExecutionContext,
knowledge_base_uuid: str,
filter: dict | None = None,
limit: int = 20,
offset: int = 0,
@@ -187,4 +496,17 @@ class VectorDBManager:
Returns:
Tuple of (items, total).
"""
collection_name = await self._resolve_physical_collection_name(
execution_context,
knowledge_base_uuid,
)
pgvector = self._pgvector_database()
if pgvector is not None:
scope = await self._resolve_pgvector_scope(
execution_context,
knowledge_base_uuid,
expected_dimension=None,
initialize_dimension=False,
)
return await pgvector.list_by_filter(collection_name, filter, limit, offset, scope=scope)
return await self.vector_db.list_by_filter(collection_name, filter, limit, offset)