Feat/saas sandbox adaptation (#2234)

* fix(box): trust Box-reported skill paths when filesystem is not shared

In separated deployments (Docker Compose, k8s sidecar, --standalone-box,
remote runtime.endpoint) the Box runtime owns its own filesystem, so the
skill package_root it reports via list_skills is not resolvable on the
LangBot side. LangBot's reload_skills and build_skill_extra_mounts
validated those paths with os.path.isdir() against its own filesystem,
which silently dropped every skill in such deployments — breaking the
sandbox skill feature for the nsjail/SaaS backend.

Add BoxService.shares_filesystem_with_box, derived from the connector
transport (stdio = shared, WebSocket = separated), with an explicit
override seam for tests/embedders. Gate both isdir() guards on it: keep
local validation in shared-fs stdio mode, trust Box-reported paths
otherwise. The Box runtime only reports skills found on its own
filesystem, so those paths are valid there by construction.

Adds topology-derivation tests (real connector, no mocks) and
skill-retention tests for both shared and separated filesystems.

* build(docker): ship a self-contained nsjail sandbox backend in the image

Compile nsjail 3.6 from source in a dedicated multi-stage build and carry
only the binary plus its runtime libs (libprotobuf32, libnl-route-3-200)
into the final image. This lets the Box runtime isolate sandboxed code via
nsjail user/mount/pid/net namespaces without a host Docker socket — the
prerequisite for running Box on LangBot Cloud (k8s), where mounting
docker.sock would grant node root and is not acceptable for multi-tenant.

The build toolchain (build-essential/bison/flex/protobuf-dev/libnl-dev)
stays in the nsjail-build stage and is not present in the shipped image.

Verified: image builds (583MB), nsjail --help exits 0, libraries resolve,
and the real NsjailBackend executes an isolated command end-to-end on a
v6.1/cgroup2 host matching LangBot Cloud prod (rlimit fallback path, since
container /sys/fs/cgroup is read-only; PID-namespace isolation confirmed).

* feat(box): SaaS guard to force a single global sandbox scope

Add system.limitation.force_box_session_id_template: when non-empty it
overrides every pipeline's box-session-id-template at resolve time, pinning
all queries to one shared sandbox (e.g. {global}). This is the authoritative,
unbypassable guard — it runs on every exec call, so editing the pipeline
config via API cannot escape it. The web UI locks the Sandbox Scope selector
via a combined box_scope_editable flag (box available AND not forced).

* build(deps): pin langbot-plugin==0.4.2b1 (nsjail cgroup container-safety beta)

* fix(web): show forced sandbox scope + make disabled tooltip tap-friendly

When a SaaS deployment pins every pipeline to a fixed sandbox scope via
system.limitation.force_box_session_id_template, the Sandbox Scope selector was
correctly locked but still displayed the pipeline's stored value (e.g. the
per-chat default), misrepresenting the scope that the runtime actually enforces
on every exec. Coerce the displayed/saved value to the forced template so the
locked selector truthfully shows the active scope (e.g. Global).

Also fix the disabled_tooltip being invisible on touch devices: hover-only Radix
tooltips never open without a pointer, so the explanation of why the field is
locked could not be read on mobile. Wrap the info icon so a tap toggles the
tooltip while desktop hover still works.

* feat(web): hide sidebar new-version prompt for edition=cloud

Cloud instances are upgraded centrally by the operator, so surfacing a GitHub
'new version available' badge to tenants is misleading and actionable only by
the operator. Skip the release check entirely when edition=cloud.

* style(web): prettier formatting for DisabledTooltipIcon ternary

* chore(deps): bump langbot-plugin to 0.4.2b2

Picks up the SDK fix that creates a read-write host_path before the
nsjail bind-mount, fixing the SaaS MCP shared-workspace sandbox failure
(exec exit 255 with empty output when host_path didn't exist).

* chore(deps): bump langbot-plugin to 0.4.2b3

Picks up the nsjail /dev-node fix so stdio MCP servers (uvx-launched) can
start under force_global_sandbox instead of failing with 'Connection closed
/ please check URL'.

* fix(web): show real MCP runtime status on installed extensions list

The installed-extensions list badge keyed solely off the enable flag, so a
server that was still CONNECTING (or in ERROR) was shown as 'Connected'.
Reflect the actual runtime_info.status (connecting/connected/error/disabled)
with matching colors, and poll quietly every 3s while any MCP server is
connecting so the badge transitions without a manual refresh.

* chore(deps): bump langbot-plugin to 0.4.2b4

Picks up the 30s start_managed_process timeout so cold uvx MCP bootstraps
don't get torn down mid-install.

* style(web): satisfy prettier — parenthesize nullish-coalescing in ternary

* fix(mcp): isolate transient test sessions from the shared Box session

A config-page 'test' (server_name='_', no persisted UUID) ran in the same
shared 'mcp-shared' Box session as live MCP servers. A failing test (e.g.
empty args) churned that shared session and tore down healthy, already-
connected servers — leaving them stuck after exhausting their retries.

Mark UUID-less sessions as transient, give them their own isolated Box
session ('mcp-test-<uuid>'), and fully delete that session on cleanup so
tests can never disturb live servers and don't leak sessions.

* fix(mcp): tear down transient test session after test completes

A successful config-page test left its isolated 'mcp-test-<uuid>' Box
session running (the lifecycle task blocks until shutdown). Wrap the
transient test coroutine so it always shuts the session down afterward,
preventing isolated test sessions from leaking.
This commit is contained in:
Junyan Chin
2026-06-09 19:30:17 +08:00
committed by GitHub
parent 47fe9bde03
commit 8e558ad3a1
20 changed files with 579 additions and 87 deletions
+169 -3
View File
@@ -153,6 +153,7 @@ def make_app(
host_root: str = '',
workspace_quota_mb: int | None = None,
enabled: bool = True,
force_box_session_id_template: str = '',
):
box_config = {
'enabled': enabled,
@@ -171,7 +172,12 @@ def make_app(
return SimpleNamespace(
logger=logger,
instance_config=SimpleNamespace(data={'box': box_config}),
instance_config=SimpleNamespace(
data={
'box': box_config,
'system': {'limitation': {'force_box_session_id_template': force_box_session_id_template}},
}
),
)
@@ -190,6 +196,66 @@ async def test_box_service_without_explicit_client_initializes_internal_connecto
connector.initialize.assert_awaited_once()
class TestSharesFilesystemWithBox:
"""``shares_filesystem_with_box`` must reflect the real LangBot<->Box
filesystem topology, which is derived from the connector transport:
- stdio (local child process) → shared filesystem → True
- WebSocket (Docker / sidecar / --standalone-box / remote) → separated → False
This drives whether LangBot validates Box-reported skill paths locally.
Getting it wrong silently drops every skill in separated deployments.
"""
def test_true_for_stdio_connector(self, monkeypatch: pytest.MonkeyPatch):
# Non-Docker Unix, no endpoint, not standalone → stdio transport.
monkeypatch.setattr('langbot.pkg.utils.platform.get_platform', lambda: 'linux')
monkeypatch.setattr('langbot.pkg.utils.platform.standalone_box', False)
service = BoxService(make_app(Mock()))
assert service._runtime_connector is not None
assert service._runtime_connector.uses_websocket() is False
assert service.shares_filesystem_with_box is True
def test_false_for_websocket_connector_via_endpoint(self, monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr('langbot.pkg.utils.platform.get_platform', lambda: 'linux')
monkeypatch.setattr('langbot.pkg.utils.platform.standalone_box', False)
app = make_app(Mock())
app.instance_config.data['box']['runtime']['endpoint'] = 'ws://pod-x-box:5410'
service = BoxService(app)
assert service._runtime_connector is not None
assert service._runtime_connector.uses_websocket() is True
assert service.shares_filesystem_with_box is False
def test_false_for_websocket_connector_in_docker(self, monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr('langbot.pkg.utils.platform.get_platform', lambda: 'docker')
monkeypatch.setattr('langbot.pkg.utils.platform.standalone_box', False)
service = BoxService(make_app(Mock()))
assert service.shares_filesystem_with_box is False
def test_false_when_client_injected_without_connector(self):
# Injected client (no connector) → unknown topology → conservative False
# so LangBot never wrongly drops Box-reported skills.
service = BoxService(make_app(Mock()), client=Mock(spec=BoxRuntimeClient))
assert service._runtime_connector is None
assert service.shares_filesystem_with_box is False
def test_explicit_override_wins(self):
service = BoxService(make_app(Mock()), client=Mock(spec=BoxRuntimeClient))
service._shares_filesystem_with_box_override = True
assert service.shares_filesystem_with_box is True
service._shares_filesystem_with_box_override = False
assert service.shares_filesystem_with_box is False
@pytest.mark.asyncio
async def test_box_service_get_sessions_delegates_to_client():
client = Mock()
@@ -302,6 +368,69 @@ async def test_box_service_session_id_falls_back_to_query_id_for_synthetic_queri
assert backend.start_calls == ['query_7']
@pytest.mark.asyncio
async def test_box_service_forced_global_scope_overrides_pipeline_template():
"""SaaS guard: a non-empty ``force_box_session_id_template`` pins every
query to one shared sandbox regardless of the pipeline's own scope."""
logger = Mock()
backend = FakeBackend(logger)
runtime = BoxRuntime(logger=logger, backends=[backend], session_ttl_sec=300)
service = BoxService(
make_app(logger, force_box_session_id_template='{global}'),
client=_InProcessBoxRuntimeClient(logger, runtime),
)
await service.initialize()
# Two distinct callers that would otherwise get separate sandboxes.
q1 = pipeline_query.Query.model_construct(query_id=1, launcher_type='group', launcher_id='room-1')
q2 = pipeline_query.Query.model_construct(query_id=2, launcher_type='person', launcher_id='alice')
r1 = await service.execute_tool({'command': 'pwd'}, q1)
r2 = await service.execute_tool({'command': 'pwd'}, q2)
assert r1['session_id'] == 'global'
assert r2['session_id'] == 'global'
# Only one sandbox was ever started — the shared global one.
assert backend.start_calls == ['global']
def test_box_service_forced_template_ignores_pipeline_config():
"""The forced template wins even when the pipeline explicitly sets a
per-user scope — proving the override is not bypassable via pipeline config."""
logger = Mock()
service = BoxService(
make_app(logger, force_box_session_id_template='{global}'),
client=Mock(spec=BoxRuntimeClient),
)
query = pipeline_query.Query.model_construct(
query_id=7,
launcher_type='person',
launcher_id='test_user',
sender_id='test_user',
pipeline_config={'ai': {'local-agent': {'box-session-id-template': '{launcher_type}_{launcher_id}_{sender_id}'}}},
)
assert service.resolve_box_session_id(query) == 'global'
def test_box_service_empty_forced_template_respects_pipeline_config():
"""An empty/whitespace forced template is a no-op: the pipeline's own
scope template is honoured (default non-SaaS behaviour)."""
logger = Mock()
service = BoxService(
make_app(logger, force_box_session_id_template=' '),
client=Mock(spec=BoxRuntimeClient),
)
query = pipeline_query.Query.model_construct(
query_id=7,
launcher_type='group',
launcher_id='room-1',
pipeline_config={'ai': {'local-agent': {'box-session-id-template': '{launcher_type}_{launcher_id}'}}},
)
assert service.resolve_box_session_id(query) == 'group_room-1'
@pytest.mark.asyncio
async def test_box_service_fails_closed_when_backend_unavailable():
logger = Mock()
@@ -1342,11 +1471,16 @@ class TestBuildSkillExtraMounts:
the backend never sees a bad mount.
"""
def _make_service(self, logger, skills):
def _make_service(self, logger, skills, *, shares_filesystem=True):
app = make_app(logger)
app.skill_mgr = SimpleNamespace(skills=skills)
client = Mock(spec=BoxRuntimeClient)
return BoxService(app, client=client)
service = BoxService(app, client=client)
# Tests construct BoxService with an injected client (no connector), so
# set the topology explicitly. Most cases exercise the shared-fs (local
# stdio) path where local package_root validation applies.
service._shares_filesystem_with_box_override = shares_filesystem
return service
def test_skips_skill_with_missing_package_root(self):
logger = Mock()
@@ -1373,6 +1507,30 @@ class TestBuildSkillExtraMounts:
for call in logger.warning.call_args_list
)
def test_trusts_box_paths_when_filesystem_not_shared(self):
"""In separated deployments (Docker Compose, k8s sidecar,
--standalone-box, remote endpoint) the Box runtime owns its own
filesystem. package_root values it reports are NOT resolvable on the
LangBot side, so LangBot must trust them rather than dropping every
skill via a local isdir() check."""
logger = Mock()
skills = {
'a': {'name': 'a', 'package_root': '/box/skills/a'},
'b': {'name': 'b', 'package_root': '/box/skills/b'},
}
service = self._make_service(logger, skills, shares_filesystem=False)
mounts = service.build_skill_extra_mounts(make_query())
assert mounts == [
{'host_path': '/box/skills/a', 'mount_path': '/workspace/.skills/a', 'mode': 'rw'},
{'host_path': '/box/skills/b', 'mount_path': '/workspace/.skills/b', 'mode': 'rw'},
]
# No skill is dropped, so no "missing" warning should be logged.
assert not any(
'package_root missing' in str(call.args[0]) for call in logger.warning.call_args_list
)
def test_skips_skill_with_empty_package_root(self):
logger = Mock()
skills = {
@@ -1383,6 +1541,14 @@ class TestBuildSkillExtraMounts:
assert service.build_skill_extra_mounts(make_query()) == []
def test_empty_package_root_skipped_even_when_not_shared(self):
"""An empty package_root is always invalid regardless of topology."""
logger = Mock()
skills = {'no_root': {'name': 'no_root', 'package_root': ''}}
service = self._make_service(logger, skills, shares_filesystem=False)
assert service.build_skill_extra_mounts(make_query()) == []
def test_returns_empty_when_no_skill_manager(self):
logger = Mock()
app = make_app(logger)
@@ -561,6 +561,42 @@ class TestGetRuntimeInfoDict:
assert info['box_session_id'] == 'mcp-shared'
assert info['box_enabled'] is True
def test_transient_test_session_is_isolated_from_shared(self, mcp_module):
"""A transient test session (config-page "test", no persisted UUID)
must NOT share the live "mcp-shared" Box session. Regression: a failing
test churned the shared session and tore down healthy live servers."""
ap = _make_ap()
ap.box_service.available = True
transient = _make_session(
mcp_module,
{
'name': 'test',
'uuid': 'gen-uuid-123',
'mode': 'stdio',
'command': 'uvx',
'args': ['mcp-server-time'],
'_transient': True,
},
ap=ap,
)
live = _make_session(
mcp_module,
{
'name': 'time',
'uuid': 'real-uuid',
'mode': 'stdio',
'command': 'uvx',
'args': ['mcp-server-time'],
},
ap=ap,
)
assert transient.is_transient is True
assert live.is_transient is False
# Isolated session id for the test, shared for the live server.
assert transient._build_box_session_id() == 'mcp-test-gen-uuid-123'
assert live._build_box_session_id() == 'mcp-shared'
assert transient._build_box_session_id() != live._build_box_session_id()
def test_stdio_session_refuses_when_box_unavailable(self, mcp_module):
"""Policy: when Box is configured but unavailable (disabled in config
OR connection failed), stdio MCP servers are NOT treated as box-stdio.
+36 -3
View File
@@ -62,15 +62,17 @@ class TestSkillManagerCache:
@pytest.mark.asyncio
async def test_reload_skills_drops_box_skills_with_missing_package_root(self):
"""When Box reports a skill whose package_root is gone from the
LangBot-visible filesystem, the cache must drop it instead of
keeping a stale entry that would later produce a bad mount."""
"""When LangBot shares a filesystem with Box (local stdio mode) and Box
reports a skill whose package_root is gone from that shared filesystem,
the cache must drop it instead of keeping a stale entry that would later
produce a bad mount."""
from langbot.pkg.skill.manager import SkillManager
with tempfile.TemporaryDirectory() as live_dir:
ghost_dir = os.path.join(live_dir, '_does_not_exist')
box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=True,
list_skills=AsyncMock(
return_value=[
_make_skill_data(name='alive', package_root=live_dir),
@@ -90,6 +92,37 @@ class TestSkillManagerCache:
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert any('ghost' in msg and 'package_root missing' in msg for msg in warning_messages)
@pytest.mark.asyncio
async def test_reload_skills_trusts_box_paths_when_filesystem_not_shared(self):
"""In separated deployments (Docker Compose, k8s sidecar,
--standalone-box, remote endpoint) the package_root reported by Box
lives on the Box runtime's filesystem and is not resolvable on the
LangBot side. The cache must keep every Box-reported skill rather than
dropping them all via a local isdir() check."""
from langbot.pkg.skill.manager import SkillManager
box_service = SimpleNamespace(
available=True,
shares_filesystem_with_box=False,
list_skills=AsyncMock(
return_value=[
_make_skill_data(name='alpha', package_root='/box/skills/alpha'),
_make_skill_data(name='beta', package_root='/box/skills/beta'),
]
),
)
ap = _make_ap()
ap.box_service = box_service
mgr = SkillManager(ap)
await mgr.reload_skills()
assert sorted(mgr.skills) == ['alpha', 'beta']
# No skill dropped → no "package_root missing" warning.
warning_messages = [str(call.args[0]) for call in ap.logger.warning.call_args_list]
assert not any('package_root missing' in msg for msg in warning_messages)
class TestSkillActivationHelper:
"""Skill activation is now Tool-Call based.