mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-08-09 12:40:59 +00:00
feat(tenancy): add Workspace multi-tenant foundation (#2353)
* Document multi-tenant workspace architecture * Add OSS and commercial workspace boundaries * docs: redesign multi-tenant workspace architecture * feat(tenancy): implement workspace isolation * docs(tenancy): record verification evidence * docs(tenancy): revise single-instance SaaS topology * docs(tenancy): refine architecture options * docs: finalize cloud v2 multi-tenant decisions * feat(tenancy): establish cloud isolation foundations * feat(tenancy): harden shared cloud runtime boundaries * docs(tenancy): record final isolation verification * fix(tenancy): close isolation and permission gaps * docs(tenancy): record final isolation verification * feat(tenancy): connect cloud workspace control plane * fix(build): install git for pinned SDK * docs(cloud): update control plane verification * chore: update multi-tenant SDK pin * fix(cloud): skip legacy model sync during startup * test(cloud): preserve minimal model manager fixtures * fix(cloud): preserve authenticated account context * fix(cloud): reuse authenticated account for user info * feat(cloud): complete Workspace settings navigation * test(web): cover Workspace dropdown menu * feat(web): place workspace controls in sidebar * refactor(web): streamline workspace controls * style(web): format workspace layout test * fix(cloud): surface runtime and workspace plan status * fix(plugin): keep runtime identity stable across restarts * fix(ui): widen and center workspace switcher * fix(ui): hide roles from workspace switcher * fix(ui): align workspace switcher with sidebar entries * feat(workspace): add in-product collaboration and direct Cloud launch * style: format collaboration changes * fix(workspace): bind collaboration APIs to tenant UoW * fix(cloud): preserve Core-owned collaboration state * test(cloud): require Space identity for invite registration * feat(cloud): complete secure invitation experience * style(web): format invitation flows * fix(cloud): recover box runtime without unscoped skill reload * feat(oss): enforce invitation account and owner billing flows * style: format OSS account service * test(oss): cover invitation logout handoff * fix(oss): resolve workspace owner in scoped session * feat(cloud): harden multi-tenant runtime resources * fix(cloud): bound runtime restart storms * fix(cloud): eliminate periodic runtime CPU spikes * fix(cloud): enforce instance capacity ceilings * fix(cloud): scope public login capability discovery * fix(cloud): bound tenant maintenance and monitoring work * fix(runtime): bound tenant resource amplification * fix(deps): pin green multi-tenant plugin SDK * fix(cloud): handle unavailable skill capability * fix(security): require authentication for image file endpoint (H-2) - Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY - Added Permission.RESOURCE_VIEW requirement - Prevents unauthenticated cross-tenant file access via leaked keys - Fixes HIGH severity finding from multi-tenant security review docs: add comprehensive database migration guide - Complete migration steps for OSS → multi-tenant - Backup, execution, verification procedures - Rollback scenarios and recovery plans - Performance tuning recommendations * test: add comprehensive cross-tenant isolation tests Added 7 critical test scenarios for multi-tenant boundaries: - Cross-tenant bot access prevention - Viewer role read-only enforcement - Removed member immediate access revocation - Model provider credential isolation - WebSocket message isolation - Invitation token workspace scoping - Multi-workspace context validation These tests address P0-2 coverage gaps for: - workspaces.py (membership & invitation flows) - user.py (authentication & authorization) - websocket_chat.py (real-time isolation) - plugins.py (resource access control) docs: finalize database migration guide * fix(security): resolve M-1, M-2, M-3 security findings M-1: WebSocket authorization TOCTOU race (FIXED) - Changed _revalidate_websocket_authorization to return RequestContext - Ensures validated context is used immediately without race window - Prevents removed members from sending messages during revalidation gap M-2: Model Manager cache workspace isolation (VERIFIED) - Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource) - Cache is properly scoped per workspace, no cross-tenant leakage possible - No code change needed, documented as working correctly M-3: Invitation lock workspace scoping (FIXED) - Changed lock key from token_digest to workspace_uuid:token_digest - Prevents DoS where attacker locks token in Workspace A to block Workspace B - Locks now isolated per workspace All MEDIUM severity findings from security review now resolved. * fix(cloud): unblock tenant CI and enforce knowledge quotas * fix(tenancy): scope rerank model sync --------- Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
This commit is contained in:
@@ -0,0 +1,218 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import datetime as dt
|
||||
import time
|
||||
import weakref
|
||||
from collections.abc import Callable
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from langbot_plugin.box.errors import BoxAdmissionError, BoxRuntimeUnavailableError
|
||||
from langbot_plugin.box.models import (
|
||||
SandboxAdmissionGrant,
|
||||
SandboxAdmissionPolicy,
|
||||
SandboxAdmissionRevocation,
|
||||
)
|
||||
|
||||
from ..api.http.context import ExecutionContext
|
||||
from ..cloud.entitlements import EntitlementSnapshot, EntitlementUnavailableError
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from langbot_plugin.box.client import BoxRuntimeClient
|
||||
|
||||
from ..core.app import Application
|
||||
|
||||
|
||||
_UTC = dt.timezone.utc
|
||||
_MANAGED_SANDBOX_FEATURE = 'managed_sandbox'
|
||||
_MANAGED_SANDBOX_SESSION_LIMIT = 'managed_sandbox_sessions'
|
||||
_MAX_GRANT_TTL_SEC = 300
|
||||
|
||||
|
||||
class SandboxAdmissionController:
|
||||
"""Project Cloud entitlements into short-lived Box Runtime grants.
|
||||
|
||||
Product and plan names intentionally never cross this boundary. The
|
||||
closed Control Plane supplies a versioned generic entitlement, while Core
|
||||
installs only the numeric authority understood by the shared Box Runtime.
|
||||
|
||||
No state is allocated for a Workspace until it attempts to use the
|
||||
managed sandbox. Per-Workspace locks serialize renewal/revocation so a
|
||||
concurrent first use cannot install conflicting grants.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
ap: Application,
|
||||
client: BoxRuntimeClient,
|
||||
*,
|
||||
policy: SandboxAdmissionPolicy,
|
||||
wall_time: Callable[[], float] = time.time,
|
||||
) -> None:
|
||||
self.ap = ap
|
||||
self.client = client
|
||||
self.policy = policy
|
||||
self._wall_time = wall_time
|
||||
self._locks: weakref.WeakValueDictionary[str, asyncio.Lock] = weakref.WeakValueDictionary()
|
||||
self._highest_revisions: dict[str, int] = {}
|
||||
|
||||
def _workspace_lock(self, workspace_uuid: str) -> asyncio.Lock:
|
||||
lock = self._locks.get(workspace_uuid)
|
||||
if lock is None:
|
||||
lock = asyncio.Lock()
|
||||
self._locks[workspace_uuid] = lock
|
||||
return lock
|
||||
|
||||
@staticmethod
|
||||
def _context_revision(context: ExecutionContext) -> int:
|
||||
revision = getattr(context, 'entitlement_revision', 0)
|
||||
if isinstance(revision, bool) or not isinstance(revision, int):
|
||||
return 0
|
||||
return max(revision, 0)
|
||||
|
||||
def _revocation_revision(self, context: ExecutionContext, candidate_revision: int = 0) -> int:
|
||||
return max(
|
||||
1,
|
||||
self._highest_revisions.get(context.workspace_uuid, 0),
|
||||
self._context_revision(context),
|
||||
candidate_revision,
|
||||
)
|
||||
|
||||
async def _revoke_locked(
|
||||
self,
|
||||
context: ExecutionContext,
|
||||
*,
|
||||
candidate_revision: int = 0,
|
||||
) -> None:
|
||||
revision = self._revocation_revision(context, candidate_revision)
|
||||
revocation = SandboxAdmissionRevocation(
|
||||
instance_uuid=context.instance_uuid,
|
||||
workspace_uuid=context.workspace_uuid,
|
||||
entitlement_revision=revision,
|
||||
)
|
||||
try:
|
||||
result = await self.client.revoke_sandbox_admission_grant(revocation)
|
||||
if (
|
||||
not isinstance(result, dict)
|
||||
or result.get('revoked') is not True
|
||||
or result.get('workspace_uuid') != context.workspace_uuid
|
||||
or result.get('entitlement_revision') != revision
|
||||
):
|
||||
raise BoxRuntimeUnavailableError('Box Runtime returned an invalid sandbox revocation receipt')
|
||||
except Exception as exc:
|
||||
# The caller still fails closed even if the control connection is
|
||||
# unavailable. A previously installed grant expires independently
|
||||
# in at most five minutes inside the Runtime.
|
||||
self.ap.logger.warning(
|
||||
'Failed to install Box sandbox admission revocation: '
|
||||
f'workspace_uuid={context.workspace_uuid} revision={revision} error={exc}'
|
||||
)
|
||||
self._highest_revisions[context.workspace_uuid] = revision
|
||||
|
||||
@staticmethod
|
||||
def _require_managed_sandbox(snapshot: EntitlementSnapshot) -> None:
|
||||
snapshot.require_feature(_MANAGED_SANDBOX_FEATURE)
|
||||
sessions = snapshot.limit(_MANAGED_SANDBOX_SESSION_LIMIT)
|
||||
if sessions != 1:
|
||||
raise EntitlementUnavailableError('Workspace entitlement must grant exactly one managed sandbox session')
|
||||
|
||||
def _grant_expiry(self, snapshot: EntitlementSnapshot) -> dt.datetime:
|
||||
now_epoch = int(self._wall_time())
|
||||
ttl_sec = min(self.policy.max_grant_ttl_sec, _MAX_GRANT_TTL_SEC)
|
||||
expires_epoch = min(snapshot.expires_at, now_epoch + ttl_sec)
|
||||
if expires_epoch <= now_epoch:
|
||||
raise EntitlementUnavailableError('Workspace entitlement expired before sandbox admission')
|
||||
return dt.datetime.fromtimestamp(expires_epoch, tz=_UTC)
|
||||
|
||||
async def require(self, context: ExecutionContext) -> SandboxAdmissionGrant:
|
||||
"""Validate entitlement freshness and install/renew one Runtime grant."""
|
||||
|
||||
resolver = getattr(self.ap, 'entitlement_resolver', None)
|
||||
if resolver is None:
|
||||
raise EntitlementUnavailableError('Workspace entitlement resolver is unavailable')
|
||||
if context.instance_uuid != resolver.instance_uuid:
|
||||
raise EntitlementUnavailableError('Workspace entitlement targets another LangBot instance')
|
||||
|
||||
lock = self._workspace_lock(context.workspace_uuid)
|
||||
async with lock:
|
||||
try:
|
||||
snapshot = await resolver.resolve(
|
||||
context.workspace_uuid,
|
||||
minimum_revision=self._context_revision(context),
|
||||
now=int(self._wall_time()),
|
||||
)
|
||||
except EntitlementUnavailableError as exc:
|
||||
# Only a verified, scoped snapshot can authoritatively revoke
|
||||
# a revision. Provider timeouts, malformed responses, and
|
||||
# rollback/equivocation errors fail this request closed but do
|
||||
# not tombstone a still-valid revision forever.
|
||||
authoritative_revision = exc.entitlement_revision
|
||||
if authoritative_revision is not None:
|
||||
await self._revoke_locked(
|
||||
context,
|
||||
candidate_revision=authoritative_revision,
|
||||
)
|
||||
raise
|
||||
|
||||
try:
|
||||
self._require_managed_sandbox(snapshot)
|
||||
except EntitlementUnavailableError:
|
||||
await self._revoke_locked(
|
||||
context,
|
||||
candidate_revision=snapshot.entitlement_revision,
|
||||
)
|
||||
raise
|
||||
|
||||
grant = SandboxAdmissionGrant(
|
||||
instance_uuid=context.instance_uuid,
|
||||
workspace_uuid=context.workspace_uuid,
|
||||
execution_generation=context.placement_generation,
|
||||
entitlement_revision=snapshot.entitlement_revision,
|
||||
expires_at=self._grant_expiry(snapshot),
|
||||
max_sessions=1,
|
||||
max_managed_processes=0,
|
||||
)
|
||||
result = await self.client.upsert_sandbox_admission_grant(grant)
|
||||
if (
|
||||
not isinstance(result, dict)
|
||||
or result.get('installed') is not True
|
||||
or result.get('workspace_uuid') != context.workspace_uuid
|
||||
or result.get('execution_generation') != context.placement_generation
|
||||
or result.get('entitlement_revision') != snapshot.entitlement_revision
|
||||
or result.get('max_sessions') != 1
|
||||
or result.get('max_managed_processes') != 0
|
||||
):
|
||||
raise BoxRuntimeUnavailableError('Box Runtime returned an invalid sandbox admission receipt')
|
||||
self._highest_revisions[context.workspace_uuid] = max(
|
||||
self._highest_revisions.get(context.workspace_uuid, 0),
|
||||
snapshot.entitlement_revision,
|
||||
)
|
||||
return grant
|
||||
|
||||
async def revoke(self, context: ExecutionContext, *, entitlement_revision: int = 0) -> None:
|
||||
"""Explicitly revoke a Workspace grant using a monotonic tombstone."""
|
||||
|
||||
async with self._workspace_lock(context.workspace_uuid):
|
||||
await self._revoke_locked(context, candidate_revision=entitlement_revision)
|
||||
|
||||
|
||||
def require_cloud_admission_policy(raw_policy: object) -> SandboxAdmissionPolicy:
|
||||
"""Parse the Cloud Box policy without permitting an OSS downgrade."""
|
||||
|
||||
try:
|
||||
policy = SandboxAdmissionPolicy.model_validate(raw_policy)
|
||||
except Exception as exc:
|
||||
raise BoxAdmissionError('Cloud Box sandbox admission policy is invalid') from exc
|
||||
if not policy.required:
|
||||
raise BoxAdmissionError('Cloud Box sandbox admission must be required')
|
||||
if policy.logical_session_id != 'global':
|
||||
raise BoxAdmissionError('Cloud Box sandbox session ID must be global')
|
||||
if policy.required_backend != 'nsjail':
|
||||
raise BoxAdmissionError('Cloud Box sandbox backend must be nsjail')
|
||||
if policy.max_sessions != 1 or policy.max_managed_processes != 0:
|
||||
raise BoxAdmissionError('Cloud Box sandbox policy must allow one session and zero managed processes')
|
||||
if policy.max_grant_ttl_sec > _MAX_GRANT_TTL_SEC:
|
||||
raise BoxAdmissionError('Cloud Box sandbox admission grant TTL must not exceed 300 seconds')
|
||||
if policy.workspace_quota_mb <= 0:
|
||||
raise BoxAdmissionError('Cloud Box sandbox workspace quota must be a positive integer')
|
||||
return policy
|
||||
@@ -4,6 +4,7 @@ import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import sys
|
||||
import typing
|
||||
from typing import TYPE_CHECKING
|
||||
@@ -16,6 +17,17 @@ from langbot_plugin.runtime.io.connection import Connection
|
||||
from langbot_plugin.box.client import ActionRPCBoxClient
|
||||
from langbot_plugin.box.errors import BoxRuntimeUnavailableError
|
||||
from langbot_plugin.box.actions import LangBotToBoxAction
|
||||
from langbot_plugin.box.security import (
|
||||
BOX_CONTROL_TOKEN_ENV,
|
||||
BOX_CONTROL_TOKEN_HEADER,
|
||||
BOX_INSTANCE_HEADER,
|
||||
BOX_PLACEMENT_GENERATION_HEADER,
|
||||
BOX_TRUSTED_INSTANCE_ENV,
|
||||
BOX_WORKSPACE_HEADER,
|
||||
normalize_instance_uuid,
|
||||
validate_control_token,
|
||||
)
|
||||
from langbot_plugin.entities.io.context import ActionContext
|
||||
|
||||
from ..utils import platform
|
||||
from ..utils.managed_runtime import ManagedRuntimeConnector
|
||||
@@ -123,6 +135,8 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
self._relay_host = parsed.hostname or '127.0.0.1'
|
||||
self._relay_port = parsed.port or _DEFAULT_PORT
|
||||
self._filtered_box_config = _filter_config_for_runtime(_get_box_config(ap))
|
||||
self._trusted_instance_uuid = normalize_instance_uuid(self.ap.workspace_service.instance_uuid)
|
||||
self._control_token = str(os.environ.get(BOX_CONTROL_TOKEN_ENV) or '').strip()
|
||||
|
||||
def uses_websocket(self) -> bool:
|
||||
"""Whether the connector should use WebSocket to reach the Box runtime.
|
||||
@@ -223,8 +237,11 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
from langbot_plugin.runtime.io.controllers.stdio.client import StdioClientController
|
||||
|
||||
self.ap.logger.info('Use stdio to connect to box runtime')
|
||||
self._ensure_control_token(allow_generate=True)
|
||||
python_path = sys.executable
|
||||
env = os.environ.copy()
|
||||
env[BOX_CONTROL_TOKEN_ENV] = self._control_token
|
||||
env[BOX_TRUSTED_INSTANCE_ENV] = self._trusted_instance_uuid
|
||||
if self._filtered_box_config:
|
||||
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
|
||||
|
||||
@@ -259,7 +276,10 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
"""Launch box server as detached subprocess, then connect via WS (Windows)."""
|
||||
self.ap.logger.info('(windows) Use cmd to launch box runtime and communicate via ws')
|
||||
|
||||
self._ensure_control_token(allow_generate=True)
|
||||
env = os.environ.copy()
|
||||
env[BOX_CONTROL_TOKEN_ENV] = self._control_token
|
||||
env[BOX_TRUSTED_INSTANCE_ENV] = self._trusted_instance_uuid
|
||||
if self._filtered_box_config:
|
||||
env['LANGBOT_BOX_CONFIG'] = json.dumps(self._filtered_box_config)
|
||||
|
||||
@@ -282,6 +302,7 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
|
||||
async def _connect_remote_ws(self) -> None:
|
||||
"""Connect to a remote (or Docker) box server via WebSocket."""
|
||||
self._ensure_control_token(allow_generate=False)
|
||||
ws_url = self._resolve_rpc_ws_url()
|
||||
self.ap.logger.info(f'Use WebSocket to connect to box runtime ({ws_url})')
|
||||
await self._connect_ws(ws_url, 'WebSocket')
|
||||
@@ -325,7 +346,11 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
if self.runtime_disconnect_callback is not None:
|
||||
await self.runtime_disconnect_callback(self)
|
||||
|
||||
ctrl = WebSocketClientController(ws_url=ws_url, make_connection_failed_callback=on_connect_failed)
|
||||
ctrl = WebSocketClientController(
|
||||
ws_url=ws_url,
|
||||
make_connection_failed_callback=on_connect_failed,
|
||||
additional_headers=self.get_control_headers(),
|
||||
)
|
||||
self._ctrl = ctrl
|
||||
self._ctrl_task = asyncio.create_task(
|
||||
ctrl.run(self._make_connection_callback(transport_name, connected, connect_error, self._generation))
|
||||
@@ -339,6 +364,41 @@ class BoxRuntimeConnector(ManagedRuntimeConnector):
|
||||
if connect_error:
|
||||
raise BoxRuntimeUnavailableError(f'box runtime connection failed: {connect_error[0]}')
|
||||
|
||||
def _ensure_control_token(self, *, allow_generate: bool) -> str:
|
||||
if not self._control_token and allow_generate:
|
||||
self._control_token = secrets.token_urlsafe(48)
|
||||
try:
|
||||
self._control_token = validate_control_token(self._control_token)
|
||||
except ValueError as exc:
|
||||
raise BoxRuntimeUnavailableError(
|
||||
f'{BOX_CONTROL_TOKEN_ENV} must be configured with a strong shared secret for an external Box runtime'
|
||||
) from exc
|
||||
return self._control_token
|
||||
|
||||
def get_control_headers(self) -> dict[str, str]:
|
||||
"""Headers for the instance-authenticated RPC control handshake."""
|
||||
|
||||
self._ensure_control_token(allow_generate=False)
|
||||
return {
|
||||
BOX_CONTROL_TOKEN_HEADER: self._control_token,
|
||||
BOX_INSTANCE_HEADER: self._trusted_instance_uuid,
|
||||
}
|
||||
|
||||
def get_relay_headers(
|
||||
self,
|
||||
action_context: ActionContext,
|
||||
) -> dict[str, str]:
|
||||
"""Return authenticated, placement-scoped relay handshake headers."""
|
||||
|
||||
context = ActionContext.model_validate(action_context).without_installation()
|
||||
if context.instance_uuid != self._trusted_instance_uuid:
|
||||
raise BoxRuntimeUnavailableError('Box relay context belongs to another LangBot instance')
|
||||
return {
|
||||
**self.get_control_headers(),
|
||||
BOX_WORKSPACE_HEADER: context.workspace_uuid,
|
||||
BOX_PLACEMENT_GENERATION_HEADER: str(context.placement_generation),
|
||||
}
|
||||
|
||||
def _make_connection_callback(
|
||||
self,
|
||||
transport_name: str,
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import errno
|
||||
import os
|
||||
import stat
|
||||
from collections.abc import Iterable
|
||||
|
||||
|
||||
class UnsafeWorkspacePathError(OSError):
|
||||
"""A tenant-controlled path could not be opened without following links."""
|
||||
|
||||
|
||||
_DIRECTORY_FLAGS = (
|
||||
os.O_RDONLY | getattr(os, 'O_DIRECTORY', 0) | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_CLOEXEC', 0)
|
||||
)
|
||||
_FILE_READ_FLAGS = os.O_RDONLY | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_CLOEXEC', 0)
|
||||
_FILE_WRITE_FLAGS = os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_CLOEXEC', 0)
|
||||
_MAX_REMOVAL_ENTRIES = 4096
|
||||
_MAX_REMOVAL_DEPTH = 16
|
||||
|
||||
|
||||
def _component(value: str) -> str:
|
||||
normalized = str(value or '').strip()
|
||||
if (
|
||||
not normalized
|
||||
or normalized in {'.', '..'}
|
||||
or '/' in normalized
|
||||
or '\\' in normalized
|
||||
or '\x00' in normalized
|
||||
or len(os.fsencode(normalized)) > 240
|
||||
):
|
||||
raise UnsafeWorkspacePathError('Unsafe Workspace path component')
|
||||
return normalized
|
||||
|
||||
|
||||
def _unsafe(path: str, exc: BaseException | None = None) -> UnsafeWorkspacePathError:
|
||||
error = UnsafeWorkspacePathError(f'Workspace path is not a link-free directory: {path}')
|
||||
if exc is not None:
|
||||
error.__cause__ = exc
|
||||
return error
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _root_fd(root: str):
|
||||
try:
|
||||
fd = os.open(root, _DIRECTORY_FLAGS)
|
||||
except OSError as exc:
|
||||
raise _unsafe(root, exc)
|
||||
try:
|
||||
if not stat.S_ISDIR(os.fstat(fd).st_mode):
|
||||
raise _unsafe(root)
|
||||
yield fd
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def _open_dir_at(parent_fd: int, name: str, *, create: bool) -> int:
|
||||
name = _component(name)
|
||||
if create:
|
||||
try:
|
||||
os.mkdir(name, mode=0o700, dir_fd=parent_fd)
|
||||
except FileExistsError:
|
||||
pass
|
||||
try:
|
||||
fd = os.open(name, _DIRECTORY_FLAGS, dir_fd=parent_fd)
|
||||
except OSError as exc:
|
||||
raise _unsafe(name, exc)
|
||||
if not stat.S_ISDIR(os.fstat(fd).st_mode):
|
||||
os.close(fd)
|
||||
raise _unsafe(name)
|
||||
return fd
|
||||
|
||||
|
||||
def _remove_entry(
|
||||
parent_fd: int,
|
||||
name: str,
|
||||
*,
|
||||
budget: list[int] | None = None,
|
||||
depth: int = 0,
|
||||
) -> None:
|
||||
"""Remove an entry recursively without following a symlink at any depth."""
|
||||
|
||||
name = _component(name)
|
||||
budget = budget if budget is not None else [_MAX_REMOVAL_ENTRIES]
|
||||
if depth > _MAX_REMOVAL_DEPTH or budget[0] <= 0:
|
||||
raise UnsafeWorkspacePathError('Workspace cleanup exceeded its inode budget')
|
||||
budget[0] -= 1
|
||||
for _ in range(4):
|
||||
try:
|
||||
child_fd = os.open(name, _DIRECTORY_FLAGS, dir_fd=parent_fd)
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as exc:
|
||||
if exc.errno not in {errno.ELOOP, errno.ENOTDIR, errno.EACCES}:
|
||||
raise
|
||||
try:
|
||||
os.unlink(name, dir_fd=parent_fd)
|
||||
return
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except IsADirectoryError:
|
||||
continue
|
||||
else:
|
||||
try:
|
||||
_clear_dir(child_fd, budget=budget, depth=depth + 1)
|
||||
finally:
|
||||
os.close(child_fd)
|
||||
try:
|
||||
os.rmdir(name, dir_fd=parent_fd)
|
||||
return
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except NotADirectoryError:
|
||||
continue
|
||||
raise UnsafeWorkspacePathError('Workspace entry changed while it was being removed')
|
||||
|
||||
|
||||
def _clear_dir(directory_fd: int, *, budget: list[int], depth: int) -> None:
|
||||
# ``scandir(fd)`` enumerates the already-open directory. Names are then
|
||||
# resolved relative to the same fd, so a tenant cannot redirect the walk by
|
||||
# swapping an ancestor symlink between validation and use.
|
||||
# Do not materialize the whole directory: an attacker-controlled outbox
|
||||
# may contain an inode bomb even when its byte size is tiny. Removal is
|
||||
# deliberately budgeted and fails closed once the per-operation cap is
|
||||
# reached; hard filesystem/inode quota remains a Cloud readiness gate.
|
||||
with os.scandir(directory_fd) as iterator:
|
||||
for entry in iterator:
|
||||
_remove_entry(directory_fd, entry.name, budget=budget, depth=depth)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _query_fd(root: str, subdir: str, query_key: str, *, create: bool, reset: bool = False):
|
||||
subdir = _component(subdir)
|
||||
query_key = _component(query_key)
|
||||
with _root_fd(root) as root_fd:
|
||||
subdir_fd = _open_dir_at(root_fd, subdir, create=create)
|
||||
try:
|
||||
if reset:
|
||||
_remove_entry(subdir_fd, query_key)
|
||||
query_fd = _open_dir_at(subdir_fd, query_key, create=create)
|
||||
try:
|
||||
yield query_fd
|
||||
finally:
|
||||
os.close(query_fd)
|
||||
finally:
|
||||
os.close(subdir_fd)
|
||||
|
||||
|
||||
def write_files(
|
||||
root: str,
|
||||
subdir: str,
|
||||
query_key: str,
|
||||
files: Iterable[tuple[str, bytes]],
|
||||
) -> None:
|
||||
"""Atomically recreate one query directory and write regular files only."""
|
||||
|
||||
with _query_fd(root, subdir, query_key, create=True, reset=True) as query_fd:
|
||||
for raw_name, data in files:
|
||||
name = _component(raw_name)
|
||||
try:
|
||||
file_fd = os.open(name, _FILE_WRITE_FLAGS, 0o600, dir_fd=query_fd)
|
||||
except OSError as exc:
|
||||
raise UnsafeWorkspacePathError(f'Could not create a link-free Workspace file: {name}') from exc
|
||||
with os.fdopen(file_fd, 'wb') as file_obj:
|
||||
file_obj.write(data)
|
||||
|
||||
|
||||
def _read_directory(
|
||||
directory_fd: int,
|
||||
*,
|
||||
prefix: str,
|
||||
max_file_bytes: int,
|
||||
max_files: int,
|
||||
max_total_bytes: int,
|
||||
output: list[tuple[str, bytes]],
|
||||
total: list[int],
|
||||
remaining_entries: list[int],
|
||||
remaining_directories: list[int],
|
||||
depth: int,
|
||||
) -> None:
|
||||
if depth > 8:
|
||||
return
|
||||
with os.scandir(directory_fd) as iterator:
|
||||
for entry in iterator:
|
||||
if len(output) >= max_files or total[0] >= max_total_bytes:
|
||||
return
|
||||
if remaining_entries[0] <= 0:
|
||||
raise UnsafeWorkspacePathError('Sandbox outbox exceeds the directory-entry limit')
|
||||
remaining_entries[0] -= 1
|
||||
name = _component(entry.name)
|
||||
relative = f'{prefix}/{name}' if prefix else name
|
||||
if entry.is_symlink():
|
||||
continue
|
||||
if entry.is_dir(follow_symlinks=False):
|
||||
if remaining_directories[0] <= 0:
|
||||
raise UnsafeWorkspacePathError('Sandbox outbox exceeds the directory limit')
|
||||
remaining_directories[0] -= 1
|
||||
try:
|
||||
child_fd = os.open(name, _DIRECTORY_FLAGS, dir_fd=directory_fd)
|
||||
except OSError:
|
||||
continue
|
||||
try:
|
||||
_read_directory(
|
||||
child_fd,
|
||||
prefix=relative,
|
||||
max_file_bytes=max_file_bytes,
|
||||
max_files=max_files,
|
||||
max_total_bytes=max_total_bytes,
|
||||
output=output,
|
||||
total=total,
|
||||
remaining_entries=remaining_entries,
|
||||
remaining_directories=remaining_directories,
|
||||
depth=depth + 1,
|
||||
)
|
||||
finally:
|
||||
os.close(child_fd)
|
||||
continue
|
||||
try:
|
||||
file_fd = os.open(name, _FILE_READ_FLAGS, dir_fd=directory_fd)
|
||||
except OSError:
|
||||
continue
|
||||
try:
|
||||
metadata = os.fstat(file_fd)
|
||||
if not stat.S_ISREG(metadata.st_mode) or metadata.st_size > max_file_bytes:
|
||||
continue
|
||||
remaining = max_total_bytes - total[0]
|
||||
if metadata.st_size > remaining:
|
||||
continue
|
||||
with os.fdopen(file_fd, 'rb', closefd=False) as file_obj:
|
||||
data = file_obj.read(max_file_bytes + 1)
|
||||
if len(data) > max_file_bytes or len(data) > remaining:
|
||||
continue
|
||||
output.append((relative, data))
|
||||
total[0] += len(data)
|
||||
finally:
|
||||
os.close(file_fd)
|
||||
|
||||
|
||||
def read_regular_files(
|
||||
root: str,
|
||||
subdir: str,
|
||||
query_key: str,
|
||||
*,
|
||||
max_file_bytes: int,
|
||||
max_files: int,
|
||||
max_total_bytes: int,
|
||||
max_entries: int = 512,
|
||||
max_directories: int = 64,
|
||||
) -> list[tuple[str, bytes]]:
|
||||
"""Read bounded regular files without following tenant-created links."""
|
||||
|
||||
output: list[tuple[str, bytes]] = []
|
||||
try:
|
||||
with _query_fd(root, subdir, query_key, create=False) as query_fd:
|
||||
_read_directory(
|
||||
query_fd,
|
||||
prefix='',
|
||||
max_file_bytes=max_file_bytes,
|
||||
max_files=max_files,
|
||||
max_total_bytes=max_total_bytes,
|
||||
output=output,
|
||||
total=[0],
|
||||
remaining_entries=[max_entries],
|
||||
remaining_directories=[max_directories],
|
||||
depth=0,
|
||||
)
|
||||
except (FileNotFoundError, UnsafeWorkspacePathError):
|
||||
# A missing directory is an empty outbox. An unsafe existing path is
|
||||
# deliberately surfaced to the caller rather than followed.
|
||||
if os.path.lexists(os.path.join(root, subdir, query_key)):
|
||||
raise
|
||||
return output
|
||||
|
||||
|
||||
def reset_directory(root: str, subdir: str, query_key: str) -> None:
|
||||
with _query_fd(root, subdir, query_key, create=True, reset=True):
|
||||
return
|
||||
|
||||
|
||||
def purge_subdirectory(root: str, subdir: str) -> None:
|
||||
"""Remove one known subtree without following a hostile replacement link."""
|
||||
|
||||
with _root_fd(root) as root_fd:
|
||||
_remove_entry(root_fd, _component(subdir))
|
||||
+871
-147
File diff suppressed because it is too large
Load Diff
@@ -126,24 +126,30 @@ def should_prepare_python_env(host_path: str | None) -> bool:
|
||||
return bool(list_python_manifest_files(normalized_root))
|
||||
|
||||
|
||||
def wrap_python_command_with_env(command: str, *, mount_path: str = '/workspace') -> str:
|
||||
def wrap_python_command_with_env(
|
||||
command: str,
|
||||
*,
|
||||
mount_path: str = '/workspace',
|
||||
state_path: str | None = None,
|
||||
) -> str:
|
||||
"""Wrap a command with a reusable sandbox-local Python env bootstrap.
|
||||
|
||||
This is the generic "workspace is a Python project" path used by mutable
|
||||
workspaces such as skills. Read-only installation strategies stay in the
|
||||
higher-level caller because they are application policy, not workspace
|
||||
semantics.
|
||||
``mount_path`` is always the source tree used for manifest hashing and
|
||||
installation. ``state_path`` may point at a separate writable directory
|
||||
for read-only source mounts; when omitted, legacy mutable-workspace behavior
|
||||
stores the environment beside the source.
|
||||
"""
|
||||
writable_state_path = state_path or mount_path
|
||||
bootstrap = textwrap.dedent(
|
||||
f"""
|
||||
set -e
|
||||
|
||||
_LB_VENV_DIR="{mount_path}/.venv"
|
||||
_LB_META_DIR="{mount_path}/.langbot"
|
||||
_LB_VENV_DIR="{writable_state_path}/.venv"
|
||||
_LB_META_DIR="{writable_state_path}/.langbot"
|
||||
_LB_META_FILE="$_LB_META_DIR/python-env.json"
|
||||
_LB_LOCK_DIR="$_LB_META_DIR/python-env.lock"
|
||||
_LB_TMP_DIR="{mount_path}/.tmp"
|
||||
_LB_PIP_CACHE_DIR="{mount_path}/.cache/pip"
|
||||
_LB_TMP_DIR="{writable_state_path}/.tmp"
|
||||
_LB_PIP_CACHE_DIR="{writable_state_path}/.cache/pip"
|
||||
|
||||
mkdir -p "$_LB_META_DIR" "$_LB_TMP_DIR" "$_LB_PIP_CACHE_DIR"
|
||||
_LB_SYSTEM_PYTHON="$(command -v python3 || command -v python || true)"
|
||||
@@ -165,17 +171,23 @@ def wrap_python_command_with_env(command: str, *, mount_path: str = '/workspace'
|
||||
import sys
|
||||
|
||||
root = "{mount_path}"
|
||||
max_manifest_bytes = 10 * 1024 * 1024
|
||||
digest = hashlib.sha256()
|
||||
manifest_files = []
|
||||
for rel in ("requirements.txt", "pyproject.toml", "setup.py", "setup.cfg"):
|
||||
path = os.path.join(root, rel)
|
||||
if not os.path.isfile(path):
|
||||
continue
|
||||
if os.path.getsize(path) > max_manifest_bytes:
|
||||
raise RuntimeError(
|
||||
f"Python project manifest exceeds {{max_manifest_bytes}} bytes: {{rel}}"
|
||||
)
|
||||
manifest_files.append(rel)
|
||||
with open(path, "rb") as handle:
|
||||
digest.update(rel.encode("utf-8"))
|
||||
digest.update(b"\\0")
|
||||
digest.update(handle.read())
|
||||
while chunk := handle.read(1024 * 1024):
|
||||
digest.update(chunk)
|
||||
digest.update(b"\\0")
|
||||
|
||||
print(
|
||||
@@ -274,6 +286,7 @@ class BoxWorkspaceSession:
|
||||
def __init__(
|
||||
self,
|
||||
box_service,
|
||||
execution_context,
|
||||
session_id: str,
|
||||
*,
|
||||
host_path: str | None = None,
|
||||
@@ -290,6 +303,7 @@ class BoxWorkspaceSession:
|
||||
persistent: bool = False,
|
||||
):
|
||||
self.box_service = box_service
|
||||
self.execution_context = execution_context
|
||||
self.session_id = session_id
|
||||
self.host_path = host_path
|
||||
self.host_path_mode = host_path_mode
|
||||
@@ -363,7 +377,7 @@ class BoxWorkspaceSession:
|
||||
timeout_sec: int | None = None,
|
||||
):
|
||||
payload = self.build_exec_payload(cmd, workdir=workdir, env=env, timeout_sec=timeout_sec)
|
||||
return await self.box_service.client.execute(self.box_service.build_spec(payload))
|
||||
return await self.box_service.execute_in_context(self.execution_context, payload)
|
||||
|
||||
async def execute_for_query(
|
||||
self,
|
||||
@@ -378,7 +392,7 @@ class BoxWorkspaceSession:
|
||||
return await self.box_service.execute_spec_payload(payload, query)
|
||||
|
||||
async def create_session(self):
|
||||
return await self.box_service.create_session(self.build_session_payload())
|
||||
return await self.box_service.create_session(self.execution_context, self.build_session_payload())
|
||||
|
||||
def build_process_payload(
|
||||
self,
|
||||
@@ -415,16 +429,26 @@ class BoxWorkspaceSession:
|
||||
):
|
||||
payload = self.build_process_payload(command, args, env=env, cwd=cwd)
|
||||
payload['process_id'] = process_id
|
||||
return await self.box_service.start_managed_process(self.session_id, payload)
|
||||
return await self.box_service.start_managed_process(self.execution_context, self.session_id, payload)
|
||||
|
||||
async def get_managed_process(self, process_id: str = 'default'):
|
||||
return await self.box_service.get_managed_process(self.session_id, process_id)
|
||||
return await self.box_service.get_managed_process(self.execution_context, self.session_id, process_id)
|
||||
|
||||
async def stop_managed_process(self, process_id: str = 'default') -> None:
|
||||
await self.box_service.stop_managed_process(self.session_id, process_id)
|
||||
await self.box_service.stop_managed_process(self.execution_context, self.session_id, process_id)
|
||||
|
||||
def get_managed_process_websocket_url(self, process_id: str = 'default') -> str:
|
||||
return self.box_service.get_managed_process_websocket_url(self.session_id, process_id)
|
||||
async def get_managed_process_websocket_connection(
|
||||
self,
|
||||
process_id: str = 'default',
|
||||
) -> tuple[str, dict[str, str]]:
|
||||
return await self.box_service.get_managed_process_websocket_connection(
|
||||
self.execution_context,
|
||||
self.session_id,
|
||||
process_id,
|
||||
)
|
||||
|
||||
async def cleanup(self) -> None:
|
||||
await self.box_service.client.delete_session(self.session_id)
|
||||
await self.box_service.client.delete_session(
|
||||
self.session_id,
|
||||
action_context=self.box_service._action_context(self.execution_context),
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user