feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
This commit is contained in:
RockChinQ
2026-07-30 21:43:35 +08:00
committed by GitHub
parent 463b120923
commit e1ac5e0fc8
468 changed files with 78320 additions and 13137 deletions
+199 -48
View File
@@ -7,6 +7,37 @@ from unittest.mock import AsyncMock, Mock
import pytest
from langbot.pkg.api.http.context import ExecutionContext
_CONTEXT = ExecutionContext(
instance_uuid='instance-a',
workspace_uuid='workspace-a',
placement_generation=1,
query_uuid='query-a',
)
def _make_query(*, variables=None, **kwargs):
return SimpleNamespace(
query_id=kwargs.pop('query_id', 'query-a'),
query_uuid=kwargs.pop('query_uuid', 'query-a'),
instance_uuid=kwargs.pop('instance_uuid', _CONTEXT.instance_uuid),
workspace_uuid=kwargs.pop('workspace_uuid', _CONTEXT.workspace_uuid),
placement_generation=kwargs.pop('placement_generation', _CONTEXT.placement_generation),
variables={} if variables is None else variables,
**kwargs,
)
def _make_skill_manager(skills: dict[str, dict], **kwargs):
return SimpleNamespace(
skills=skills,
get_skills=Mock(return_value=skills),
get_skill_by_name=Mock(side_effect=lambda _context, name: skills.get(name)),
**kwargs,
)
def _make_ap(logger=None):
ap = SimpleNamespace()
@@ -51,14 +82,14 @@ class TestSkillManagerCache:
mgr = SkillManager(ap)
# Empty cache → returns False
assert mgr.refresh_skill_from_disk('test-skill') is False
assert mgr.refresh_skill_from_disk(_CONTEXT, 'test-skill') is False
# Cache populated → returns True; method does NOT mutate the cache
cached = _make_skill_data(name='test-skill', instructions='Cached')
mgr.skills['test-skill'] = cached
assert mgr.refresh_skill_from_disk('test-skill') is True
assert mgr.skills['test-skill'] is cached
assert mgr.refresh_skill_from_disk('') is False
mgr._skills_by_scope[mgr._scope_key(_CONTEXT)] = {'test-skill': cached}
assert mgr.refresh_skill_from_disk(_CONTEXT, 'test-skill') is True
assert mgr.get_skills(_CONTEXT)['test-skill'] is cached
assert mgr.refresh_skill_from_disk(_CONTEXT, '') is False
@pytest.mark.asyncio
async def test_reload_skills_drops_box_skills_with_missing_package_root(self):
@@ -85,9 +116,9 @@ class TestSkillManagerCache:
ap.box_service = box_service
mgr = SkillManager(ap)
await mgr.reload_skills()
await mgr.reload_skills(_CONTEXT)
assert list(mgr.skills) == ['alive']
assert list(mgr.get_skills(_CONTEXT)) == ['alive']
# Warning fired with the dropped skill name so operators can see it.
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert any('ghost' in msg and 'package_root missing' in msg for msg in warning_messages)
@@ -116,9 +147,9 @@ class TestSkillManagerCache:
ap.box_service = box_service
mgr = SkillManager(ap)
await mgr.reload_skills()
await mgr.reload_skills(_CONTEXT)
assert sorted(mgr.skills) == ['alpha', 'beta']
assert sorted(mgr.get_skills(_CONTEXT)) == ['alpha', 'beta']
# No skill dropped → no "package_root missing" warning.
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert not any('package_root missing' in msg for msg in warning_messages)
@@ -141,12 +172,12 @@ class TestSkillActivationHelper:
ap = _make_ap()
mgr = SkillManager(ap)
mgr.skills = {
mgr._skills_by_scope[mgr._scope_key(_CONTEXT)] = {
'primary': _make_skill_data(name='primary', instructions='Primary instructions'),
}
ap.skill_mgr = mgr
query = SimpleNamespace(variables={})
query = _make_query()
assert register_activated_skill(ap, query, 'primary') is True
assert set(query.variables[ACTIVATED_SKILLS_KEY].keys()) == {'primary'}
@@ -159,10 +190,10 @@ class TestSkillActivationHelper:
ap = _make_ap()
mgr = SkillManager(ap)
mgr.skills = {'primary': _make_skill_data(name='primary')}
mgr._skills_by_scope[mgr._scope_key(_CONTEXT)] = {'primary': _make_skill_data(name='primary')}
ap.skill_mgr = mgr
query = SimpleNamespace(variables={})
query = _make_query()
assert register_activated_skill(ap, query, 'missing') is False
assert ACTIVATED_SKILLS_KEY not in query.variables
@@ -171,7 +202,7 @@ class TestSkillActivationHelper:
from langbot.pkg.skill.activation import register_activated_skill
ap = _make_ap() # no skill_mgr attribute
query = SimpleNamespace(variables={})
query = _make_query()
assert register_activated_skill(ap, query, 'primary') is False
@@ -181,13 +212,13 @@ class TestSkillPathHelpers:
from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY, get_visible_skills
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(
skills={
ap.skill_mgr = _make_skill_manager(
{
'visible': _make_skill_data(name='visible'),
'hidden': _make_skill_data(name='hidden'),
}
)
query = SimpleNamespace(variables={PIPELINE_BOUND_SKILLS_KEY: ['visible']})
query = _make_query(variables={PIPELINE_BOUND_SKILLS_KEY: ['visible']})
result = get_visible_skills(ap, query)
@@ -202,13 +233,13 @@ class TestSkillPathHelpers:
)
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(
skills={
ap.skill_mgr = _make_skill_manager(
{
'visible': _make_skill_data(name='visible'),
'hidden': _make_skill_data(name='hidden'),
}
)
query = SimpleNamespace(variables={PIPELINE_BOUND_SKILLS_KEY: ['visible']})
query = _make_query(variables={PIPELINE_BOUND_SKILLS_KEY: ['visible']})
restored = restore_activated_skills(ap, query, ['visible', 'hidden', 'visible', ''])
@@ -223,8 +254,8 @@ class TestSkillPathHelpers:
)
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo')})
query = SimpleNamespace(variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']})
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo')})
query = _make_query(variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']})
skill, rewritten = resolve_virtual_skill_path(
ap,
@@ -273,6 +304,22 @@ class TestSkillPathHelpers:
assert 'export VIRTUAL_ENV="$_LB_VENV_DIR"' in command
assert command.rstrip().endswith('python scripts/run.py')
def test_wrap_skill_python_env_keeps_state_outside_read_only_source(self):
from langbot.pkg.provider.tools.loaders.skill import wrap_skill_command_with_python_env
command = wrap_skill_command_with_python_env(
'python scripts/run.py',
mount_path='/workspace/.skills/demo',
state_path='/workspace/.skill-envs/demo',
)
assert '_LB_VENV_DIR="/workspace/.skill-envs/demo/.venv"' in command
assert '_LB_META_DIR="/workspace/.skill-envs/demo/.langbot"' in command
assert '_LB_TMP_DIR="/workspace/.skill-envs/demo/.tmp"' in command
assert '_LB_PIP_CACHE_DIR="/workspace/.skill-envs/demo/.cache/pip"' in command
assert 'root = "/workspace/.skills/demo"' in command
assert 'pip install "/workspace/.skills/demo"' in command
class TestSkillToolLoader:
"""The skill tool surface is now just ``activate`` + ``register_skill``.
@@ -292,13 +339,10 @@ class TestSkillToolLoader:
skill = _make_skill_data(name='demo', package_root='/data/skills/demo', instructions='Step 1')
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(
skills={'demo': skill},
get_skill_by_name=lambda name: skill if name == 'demo' else None,
)
ap.skill_mgr = _make_skill_manager({'demo': skill})
loader = SkillToolLoader(ap)
query = SimpleNamespace(variables={})
query = _make_query()
result = await loader.invoke_tool(ACTIVATE_SKILL_TOOL_NAME, {'skill_name': 'demo'}, query)
@@ -317,10 +361,7 @@ class TestSkillToolLoader:
)
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(
skills={'demo': _make_skill_data(name='demo')},
get_skill_by_name=lambda name: None,
)
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo')})
loader = SkillToolLoader(ap)
@@ -328,7 +369,7 @@ class TestSkillToolLoader:
await loader.invoke_tool(
ACTIVATE_SKILL_TOOL_NAME,
{'skill_name': 'ghost'},
SimpleNamespace(variables={}),
_make_query(),
)
@pytest.mark.asyncio
@@ -362,18 +403,19 @@ class TestSkillToolLoader:
result = await loader.invoke_tool(
REGISTER_SKILL_TOOL_NAME,
{'path': '/workspace/repo'},
SimpleNamespace(),
_make_query(),
)
ap.skill_service.scan_directory_async.assert_awaited_once_with(os.path.realpath(repo_dir))
ap.skill_service.scan_directory_async.assert_awaited_once_with(_CONTEXT, os.path.realpath(repo_dir))
ap.skill_service.create_skill.assert_awaited_once_with(
_CONTEXT,
{
'name': 'cloned-skill',
'display_name': 'Cloned Skill',
'description': 'Imported from clone',
'instructions': 'Do work',
'package_root': os.path.realpath(repo_dir),
}
},
)
assert result['registered'] is True
assert result['skill_name'] == 'cloned-skill'
@@ -397,7 +439,7 @@ class TestSkillToolLoader:
await loader.invoke_tool(
REGISTER_SKILL_TOOL_NAME,
{'path': '/workspace/../../etc'},
SimpleNamespace(),
_make_query(),
)
@pytest.mark.asyncio
@@ -417,7 +459,7 @@ class TestSkillToolLoader:
await loader.invoke_tool(
REGISTER_SKILL_TOOL_NAME,
{'path': '/workspace/foo'},
SimpleNamespace(),
_make_query(),
)
@pytest.mark.asyncio
@@ -428,7 +470,7 @@ class TestSkillToolLoader:
ap.skill_mgr = SimpleNamespace(skills={})
ap.box_service = SimpleNamespace(
available=True,
get_status=AsyncMock(return_value={'backend': {'available': False}}),
get_backend_status=AsyncMock(return_value={'backend': {'available': False}}),
)
loader = SkillToolLoader(ap)
@@ -443,10 +485,10 @@ class TestSkillToolLoader:
from langbot.pkg.provider.tools.loaders.skill_authoring import SkillToolLoader
ap = _make_ap()
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo')})
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo')})
ap.box_service = SimpleNamespace(
available=True,
get_status=AsyncMock(return_value={'backend': {'available': True}}),
get_backend_status=AsyncMock(return_value={'backend': {'available': True}}),
)
loader = SkillToolLoader(ap)
@@ -466,7 +508,7 @@ class TestSkillToolLoader:
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo')})
ap.box_service = SimpleNamespace(
available=False,
get_status=AsyncMock(return_value={'backend': {'available': True}}),
get_backend_status=AsyncMock(return_value={'backend': {'available': True}}),
)
loader = SkillToolLoader(ap)
@@ -490,20 +532,88 @@ class TestNativeToolLoaderSkillPaths:
f.write('demo instructions')
ap = _make_ap()
ap.box_service = SimpleNamespace(available=True, default_workspace=tmpdir)
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo', package_root=tmpdir)})
ap.box_service = SimpleNamespace(
available=True,
default_workspace=tmpdir,
shares_filesystem_with_box=True,
)
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)})
loader = NativeToolLoader(ap)
result = await loader.invoke_tool(
'read',
{'path': '/workspace/.skills/demo/SKILL.md'},
SimpleNamespace(query_id='q1', variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']}),
_make_query(query_id='q1', variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']}),
)
assert result['ok'] is True
assert result['content'] == 'demo instructions'
assert result['truncated'] is False
@pytest.mark.asyncio
async def test_external_runtime_read_never_interprets_package_root_on_core_host(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY
with tempfile.TemporaryDirectory() as tmpdir:
with open(os.path.join(tmpdir, 'SKILL.md'), 'w', encoding='utf-8') as file_obj:
file_obj.write('core-host-secret')
ap = _make_ap()
ap.box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=False,
read_skill_file=AsyncMock(return_value={'content': 'runtime-owned-content'}),
)
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)})
loader = NativeToolLoader(ap)
query = _make_query(
query_id='q-external-read',
variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']},
)
result = await loader.invoke_tool(
'read',
{'path': '/workspace/.skills/demo/SKILL.md'},
query,
)
assert result['ok'] is True
assert result['content'] == 'runtime-owned-content'
assert 'core-host-secret' not in repr(result)
ap.box_service.read_skill_file.assert_awaited_once_with(_CONTEXT, 'demo', 'SKILL.md')
@pytest.mark.asyncio
async def test_external_runtime_rejects_skill_host_fallback_without_protocol_capability(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import PIPELINE_BOUND_SKILLS_KEY
with tempfile.TemporaryDirectory() as tmpdir:
with open(os.path.join(tmpdir, 'secret.txt'), 'w', encoding='utf-8') as file_obj:
file_obj.write('core-host-secret')
ap = _make_ap()
ap.box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=False,
)
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)})
loader = NativeToolLoader(ap)
query = _make_query(
query_id='q-external-no-protocol',
variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']},
)
with pytest.raises(ValueError, match='owned by the Box Runtime'):
await loader.invoke_tool(
'grep',
{
'path': '/workspace/.skills/demo',
'pattern': 'core-host-secret',
},
query,
)
@pytest.mark.asyncio
async def test_exec_in_activated_skill_mount_rewrites_command_and_refreshes(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
@@ -519,7 +629,7 @@ class TestNativeToolLoaderSkillPaths:
ap.skill_mgr = SimpleNamespace(refresh_skill_from_disk=Mock())
loader = NativeToolLoader(ap)
query = SimpleNamespace(query_id='q1', launcher_type='person', launcher_id='123', variables={})
query = _make_query(query_id='q1', launcher_type='person', launcher_id='123')
register_activated_skill(query, _make_skill_data(name='demo', package_root=tmpdir))
result = await loader.invoke_tool(
@@ -535,7 +645,48 @@ class TestNativeToolLoaderSkillPaths:
tool_parameters = ap.box_service.execute_tool.await_args.args[0]
assert tool_parameters['command'] == 'python /workspace/.skills/demo/scripts/run.py'
assert tool_parameters['workdir'] == '/workspace/.skills/demo'
ap.skill_mgr.refresh_skill_from_disk.assert_called_once_with('demo')
assert ap.box_service.execute_tool.await_args.kwargs['skill_name'] == 'demo'
ap.skill_mgr.refresh_skill_from_disk.assert_called_once_with(_CONTEXT, 'demo')
@pytest.mark.asyncio
async def test_external_runtime_python_skill_uses_trusted_metadata_and_writable_env(self):
from langbot.pkg.provider.tools.loaders.native import NativeToolLoader
from langbot.pkg.provider.tools.loaders.skill import register_activated_skill
ap = _make_ap()
ap.box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=False,
execute_tool=AsyncMock(return_value={'ok': True}),
)
ap.skill_mgr = SimpleNamespace(refresh_skill_from_disk=Mock())
loader = NativeToolLoader(ap)
query = _make_query(query_id='q-external', launcher_type='person', launcher_id='123')
register_activated_skill(
query,
_make_skill_data(
name='demo',
package_root='/box-runtime/skills/tenants/workspace/demo',
python_project=True,
),
)
result = await loader.invoke_tool(
'exec',
{
'command': 'python /workspace/.skills/demo/scripts/run.py',
'workdir': '/workspace/.skills/demo',
},
query,
)
assert result['ok'] is True
tool_parameters = ap.box_service.execute_tool.await_args.args[0]
wrapped = tool_parameters['command']
assert '_LB_VENV_DIR="/workspace/.skill-envs/demo/.venv"' in wrapped
assert 'root = "/workspace/.skills/demo"' in wrapped
assert '/box-runtime/skills/tenants/workspace/demo' not in wrapped
assert ap.box_service.execute_tool.await_args.kwargs['skill_name'] == 'demo'
@pytest.mark.asyncio
async def test_write_requires_skill_activation(self):
@@ -545,10 +696,10 @@ class TestNativeToolLoaderSkillPaths:
with tempfile.TemporaryDirectory() as tmpdir:
ap = _make_ap()
ap.box_service = SimpleNamespace(available=True, default_workspace=tmpdir)
ap.skill_mgr = SimpleNamespace(skills={'demo': _make_skill_data(name='demo', package_root=tmpdir)})
ap.skill_mgr = _make_skill_manager({'demo': _make_skill_data(name='demo', package_root=tmpdir)})
loader = NativeToolLoader(ap)
query = SimpleNamespace(query_id='q1', variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']})
query = _make_query(query_id='q1', variables={PIPELINE_BOUND_SKILLS_KEY: ['demo']})
with pytest.raises(ValueError, match='Skill "demo" is not available at this path'):
await loader.invoke_tool(