diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 02b59f1ea..87fe5c9b0 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -258,6 +258,10 @@ marked for removal in the next major version. Start LangBot with ## HTTP API, Web UI, and MCP Server +The [Application API foundation](dev-docs/architecture/application-api.md) provides +versioned Workspace and execution-context operations for future consumer adoption. +Existing HTTP/MCP/assistant entrypoints retain their current service paths. + `pkg/api/http/controller/main.py` builds a Quart app, registers route groups, serves the built SPA, and wraps the ASGI app with the MCP dispatcher. - HTTP route groups live under `pkg/api/http/controller/groups/`. diff --git a/dev-docs/architecture/application-api.md b/dev-docs/architecture/application-api.md new file mode 100644 index 000000000..3734e5ea5 --- /dev/null +++ b/dev-docs/architecture/application-api.md @@ -0,0 +1,117 @@ +# Application API foundation + +The application package provides a callable business boundary inside Core. A +consumer supplies a trusted identity and operation arguments, then receives a +validated result or a stable application error. HTTP, MCP, the management +assistant and `lbctl` retain their existing entrypoints in this change; their +adoption of this boundary belongs in separate changes. + +## Workspace operations + +`ap.application_api.operation_catalog()` returns the explicit operation allowlist, +with version, permission, principal constraints, side effects, execution mode, +and JSON schemas for inputs and results. The catalog covers the existing +Workspace management capabilities: + +| Area | Capabilities | +| --- | --- | +| Pipelines and agents | CRUD, metadata, extension configuration, migration preview/execute, diagnostic runs, run history and events | +| Models and providers | LLM/embedding/rerank CRUD and tests, defaults, discovery, provider credential authorization | +| Bots and platforms | CRUD, routes and dry runs, messages, administrators, adapter metadata and credential setup sessions | +| Knowledge | Bases, ingestion tasks, files, retrieval, engine schemas and parser discovery | +| Extensions | Plugin installation/configuration/assets/logs, MCP servers/resources, skills and files, tools | +| Workspace | Current workspace, language, membership, invitations, API keys and webhooks | +| Operations | Monitoring and exports, governance and audit, task status, Box visibility, setup wizard and storage statistics | +| Conversations | Management-assistant conversations/runs and Pipeline chat history, reset, connections and broadcast | + +Service-backed operations reuse existing business signatures to build strict +Pydantic inputs. Only methods explicitly listed in `service_operations.py` are +exposed; adding a public service method does not publish an operation. Domain +configuration objects remain dictionaries because their schemas are owned by +providers, plugins or the existing resource services. Those services retain +validation, quotas, secret placeholder restoration and runtime side effects. + +```python +# context is established by a trusted authenticator, never from request JSON. +result = await ap.application_api.execute( + 'pipeline.get', context, {'pipeline_uuid': pipeline_uuid} +) + +async for frame in ap.application_api.stream( + 'agent.debug.stream', context, + {'agent_uuid': agent_uuid, 'payload': {'text': 'hello'}}, +): + consume(frame) +``` + +The Workspace comes only from `RequestContext`. Each invocation checks the +operation permission and the current execution binding/generation, then carries +that tenant and its message language through the business call. Additional +identity constraints remain where required by the business service: membership, +interactive provider authorization, assistant conversations and Pipeline +migration require an account. API-key grants are not replaced by the creator's +role. Authentication, key revocation/expiry and current account membership must +be resolved by the trusted caller before constructing the context. + +Secret reads require the corresponding secret-management permission. Mutating +operations preserve the domain's existing authorization and resource ownership +checks. Audit recording uses the existing Workspace configuration and bounded +writer, with operation identity and outcome; raw arguments are not copied into +the generic trace. Queued tasks retain their Workspace/generation and revalidate +runtime admission before accessing plugins. + +`ApplicationAPIException.error` carries `code`, `message`, `status_code` and +optional structured `details`. Protocol adapters map that object to their own +response shape. Invalid inputs and outputs fail explicitly. Cancellation is not +converted into success, and writes are never automatically retried. Pipeline +run timeouts retain the existing `unknown` outcome: a timeout does not establish +that execution failed or was rolled back. + +Streaming scopes live in isolated producer tasks with bounded queues. Closing +an Agent debug stream cancels its execution. Management-assistant streaming +retains its existing detached-run behavior; use the explicit stop operation to +cancel an assistant run. Consumers should close partially consumed iterators +with `contextlib.aclosing`. + +## Execution-context operations + +`ExecutionAPI` publishes the existing Core business actions available to an +installed plugin or a running processor. Its catalog describes each action's +input/output schema, streaming mode and required plugin/run scope. It includes +model and RAG access, tools, event replies and platform actions, state/history, +run control and interactions, and Box/file operations. + +```python +from langbot.pkg.api.application.execution import ExecutionAPI, ExecutionAPIContext + +api = ExecutionAPI(ap) +catalog = api.catalog() +context = ExecutionAPIContext(binding=trusted_installation_binding, run_id=run_id) +result = await api.invoke(operation_id, context, arguments) +``` + +The caller must supply the trusted installation binding established by Core. +The foundation invokes the existing secured action dispatcher, preserving +installation revision/grant checks, active placement and frozen run resources. +A run-bound context also verifies that the run belongs to that installation's +Workspace and plugin. Arguments cannot override the run ID or fall back to the +legacy plugin path by omitting it. Plugin-only operations remain explicit and +cannot be called with a run-bound context. Stream isolation also prevents one +consumer's context from escaping into another consumer's task. + +## Scope and adoption + +This is the full **existing Workspace and execution business capability** +foundation. Instance administration, account login/bootstrap, public ingress, +OAuth callbacks, transport assets and internal Runtime handshake/chunk/ping +messages remain with their owning boundaries. Composite HTTP response layouts +(such as a combined monitoring page) are consumer projections of these +operations. This does not add new platform APIs or new instance-level powers. + +Existing routes do not call the new application dispatcher. No SDK protocol or `lbctl` command is +changed by this foundation. + +Consumer adoption should verify the same operation through each selected +transport, including denied access, missing resources, invalid inputs, +cancellation and secret handling. The existing Core smoke exercises current +consumers; it does not by itself prove a new consumer integration. diff --git a/src/langbot/pkg/api/application/__init__.py b/src/langbot/pkg/api/application/__init__.py new file mode 100644 index 000000000..459c7c9de --- /dev/null +++ b/src/langbot/pkg/api/application/__init__.py @@ -0,0 +1,6 @@ +"""Public foundation for Workspace and execution-context operations.""" + +from .api import ApplicationAPI +from .contracts import ApplicationAPIError, ApplicationAPIException, Input, Operation + +__all__ = ['ApplicationAPI', 'ApplicationAPIError', 'ApplicationAPIException', 'Input', 'Operation'] diff --git a/src/langbot/pkg/api/application/api.py b/src/langbot/pkg/api/application/api.py new file mode 100644 index 000000000..37f81253f --- /dev/null +++ b/src/langbot/pkg/api/application/api.py @@ -0,0 +1,283 @@ +"""Workspace application boundary; protocol consumers opt in separately.""" + +from __future__ import annotations + +from collections.abc import Iterable +from contextlib import asynccontextmanager, aclosing +import time +import asyncio +import logging + +from pydantic import TypeAdapter, ValidationError + +from ..authz import AuthenticationDeniedError, require_permission +from ..context import PrincipalType, RequestContext +from ...i18n import language_scope +from ...i18n.workspace import get_workspace_language +from ...utils.bounded_executor import blocking_work_scope +from .contracts import ApplicationAPIError, ApplicationAPIException, Input, Operation +from .streaming import isolated_stream + + +class ApplicationAPI: + def __init__(self, ap, *, operations: Iterable[Operation] | None = None): + self.ap = ap + if operations is None: + from .service_operations import operations as service_operations + from .auxiliary import operations as auxiliary_operations + from .plugin_operations import operations as plugin_operations + from .platform_setup import operations as platform_operations + from .connection_operations import operations as connection_operations + from .workspace_streams import operations as workspace_streams + + operations = [ + *service_operations(), + *auxiliary_operations(), + *plugin_operations(), + *platform_operations(), + *connection_operations(), + *workspace_streams(), + ] + self._operations: dict[str, Operation] = {} + for spec in operations: + if spec.operation_id in self._operations: + raise ValueError(f'Duplicate application operation: {spec.operation_id}') + self._operations[spec.operation_id] = spec + + def operation_catalog(self) -> list[dict]: + return [spec.as_dict() for spec in self._operations.values()] + + def _operation(self, operation_id, streaming): + spec = self._operations.get(operation_id) + if spec is None: + raise ApplicationAPIException( + ApplicationAPIError('unknown_operation', 'Unknown application operation', 404) + ) + if spec.streaming != streaming: + raise ApplicationAPIException( + ApplicationAPIError('invalid_execution_mode', 'Use the operation execution mode', 400) + ) + return spec + + @asynccontextmanager + async def _scope(self, spec, context, input): + if not isinstance(context, RequestContext) or not context.workspace_uuid.strip(): + raise AuthenticationDeniedError('An authenticated Workspace context is required') + if context.principal.principal_type not in { + PrincipalType.ACCOUNT, + PrincipalType.API_KEY, + PrincipalType.SUPPORT_ADMIN, + }: + raise AuthenticationDeniedError('This principal cannot invoke Workspace management operations') + if spec.principals is not None and context.principal.principal_type not in spec.principals: + raise AuthenticationDeniedError('This operation requires a different principal') + require_permission(context, spec.permission) + try: + payload = spec.input_type.model_validate(input) + except ValidationError as exc: + # Validation errors may contain credentials; never include their input values. + fields = ', '.join('.'.join(map(str, e['loc'])) for e in exc.errors(include_input=False)) + raise ApplicationAPIException( + ApplicationAPIError('invalid_input', f'Invalid fields: {fields}', 400) + ) from exc + binding = await self.ap.workspace_service.get_execution_binding( + context.workspace_uuid, + expected_generation=context.placement_generation, + ) + if binding.instance_uuid != context.instance_uuid: + raise AuthenticationDeniedError('Workspace belongs to a different instance') + async with self.ap.persistence_mgr.tenant_scope(context.workspace_uuid): + with blocking_work_scope(context.workspace_uuid): + with language_scope(await get_workspace_language(self.ap, context.workspace_uuid)): + yield payload + + @staticmethod + def _validate_result(spec, result): + try: + return TypeAdapter(spec.output_type).validate_python(result, strict=True) + except ValidationError as exc: + raise ApplicationAPIException( + ApplicationAPIError('invalid_output', 'Application operation returned an invalid result', 500) + ) from exc + + async def execute(self, operation_id: str, context: RequestContext, input: dict | Input): + spec = self._operation(operation_id, streaming=False) + observed_at = time.monotonic() + payload = None + started = False + outcome, status = 'ok', 200 + try: + async with self._scope(spec, context, input) as payload: + started = True + result = await spec.handler(self.ap, context, payload) + return self._validate_result(spec, result) + except asyncio.CancelledError: + outcome, status = ('unknown' if started and spec.side_effect else 'error'), 499 + raise + except Exception as exc: + from .errors import error_from_exception + + error = error_from_exception(exc, operation_id=operation_id) + outcome, status = self._failure_outcome(spec, error, started), error.status_code + if isinstance(exc, ApplicationAPIException): + raise + raise ApplicationAPIException(error) from exc + finally: + await self._record(spec, context, observed_at, payload, outcome=outcome, status=status, started=started) + + async def stream(self, operation_id: str, context: RequestContext, input: dict | Input): + spec = self._operation(operation_id, streaming=True) + observed_at = time.monotonic() + + async def produce(): + payload = None + started = False + outcome, status = 'ok', 200 + try: + async with self._scope(spec, context, input) as payload: + started = True + source = await spec.handler(self.ap, context, payload) + async with aclosing(source): + async for item in source: + yield self._validate_result(spec, item) + except (asyncio.CancelledError, GeneratorExit): + outcome, status = ('unknown' if started and spec.side_effect else 'error'), 499 + raise + except Exception as exc: + from .errors import error_from_exception + + error = error_from_exception(exc, operation_id=operation_id) + outcome, status = self._failure_outcome(spec, error, started), error.status_code + if isinstance(exc, ApplicationAPIException): + raise + raise ApplicationAPIException(error) from exc + finally: + await self._record(spec, context, observed_at, payload, outcome=outcome, status=status, started=started) + + async with aclosing(isolated_stream(produce())) as responses: + async for response in responses: + yield response + + @staticmethod + def _failure_outcome(spec, error, started): + if error.status_code in {401, 403}: + return 'denied' + if started and spec.side_effect and error.code in {'invalid_output', 'application_error', 'bot_apply_failed'}: + return 'unknown' + return 'error' + + async def _record(self, spec, context, observed_at, payload=None, *, outcome='ok', status=200, started=False): + from ..context import PrincipalContext, WorkspaceContext + + if not ( + isinstance(context, RequestContext) + and isinstance(context.principal, PrincipalContext) + and isinstance(context.workspace, WorkspaceContext) + and isinstance(context.workspace_uuid, str) + and context.workspace_uuid.strip() + and context.instance_uuid + and type(context.placement_generation) is int + and context.placement_generation > 0 + and context.principal.principal_type + in { + PrincipalType.ACCOUNT, + PrincipalType.API_KEY, + PrincipalType.SUPPORT_ADMIN, + } + ): + return + principal_id = ( + context.principal.api_key_uuid + if context.principal.principal_type == PrincipalType.API_KEY + else context.principal.actor_account_uuid + if context.principal.principal_type == PrincipalType.SUPPORT_ADMIN + else context.account_uuid + ) + if not isinstance(principal_id, str) or not principal_id.strip(): + return + # These successful handlers already emit their richer domain records. + if outcome == 'ok' and spec.operation_id in { + 'workspace.member.update', + 'workspace.member.remove', + 'workspace.invitation.create', + 'settings.governance.update', + }: + return + service = getattr(self.ap, 'workspace_settings_service', None) + if service is None: + return + from ...operation_trace.service import ActionRule, bucket_allows + + try: + if not started: + # Authorization can fail before the normal execution fence. Never + # publish a trace into an unknown or foreign Workspace. + binding = await self.ap.workspace_service.get_execution_binding( + context.workspace_uuid, + expected_generation=context.placement_generation, + ) + if binding.instance_uuid != context.instance_uuid: + return + level = await service.effective_level(context.workspace_uuid, context.workspace.role) + bucket = 'write' if spec.side_effect else 'read' + if not bucket_allows(bucket, level): + return + parts = spec.operation_id.split('.') + if not spec.side_effect: + action = 'export' if 'export' in parts else 'view' + elif any(part in {'delete', 'remove', 'revoke', 'uninstall'} for part in parts): + action = 'delete' + elif any(part in {'create', 'install', 'upload', 'copy', 'add', 'import'} for part in parts): + action = 'create' + else: + action = 'update' + # Only named resource identifiers from validated DTOs enter the trace. + # Config objects, file bytes, URLs, credentials and request text never do. + identity = {} + if isinstance(payload, Input): + for name in type(payload).model_fields: + if name.endswith(('_uuid', '_id')) or name in { + 'author', + 'plugin_author', + 'plugin_name', + 'skill_name', + 'server_name', + 'owner', + 'repo', + }: + value = getattr(payload, name) + if type(value) in {str, int} and value != '': + identity[name] = str(value)[:256] + resource_id = None + for names in (('author', 'plugin_name'), ('plugin_author', 'plugin_name'), ('owner', 'repo')): + if all(name in identity for name in names): + resource_id = '/'.join(identity[name] for name in names) + break + if resource_id is None and identity: + resource_id = next(iter(identity.values())) + await service.record( + context.workspace_uuid, + rule=ActionRule(action, 'resource', bucket, parts[0]), + level=level, + actor_account_uuid=context.principal.actor_account_uuid or context.account_uuid, + actor_role=context.workspace.role, + principal_type=context.principal.principal_type.value, + api_key_uuid=context.principal.api_key_uuid, + auth_type=context.auth_type, + request_id=context.request_id, + route=spec.operation_id, + resource_id=resource_id, + outcome=outcome, + status_code=status, + detail={'operation': spec.operation_id, 'version': spec.version}, + duration_ms=int((time.monotonic() - observed_at) * 1000), + ) + except Exception: + # Audit failures must preserve the original operation result or error. + try: + getattr(self.ap, 'logger', logging.getLogger(__name__)).exception('Application operation trace failed') + except Exception: + pass + + async def invoke(self, operation_id: str, context: RequestContext, input: dict | Input): + return await self.execute(operation_id, context, input) diff --git a/src/langbot/pkg/api/application/auxiliary.py b/src/langbot/pkg/api/application/auxiliary.py new file mode 100644 index 000000000..d5228f1cd --- /dev/null +++ b/src/langbot/pkg/api/application/auxiliary.py @@ -0,0 +1,694 @@ +"""Workspace governance and resource operations without transport dependencies.""" + +from __future__ import annotations + +import datetime +import json +import mimetypes +import uuid +from contextlib import asynccontextmanager +from typing import Literal + +import sqlalchemy +from pydantic import Field + +from .contracts import Input, Operation +from ..authz import Permission, PermissionDeniedError +from ..context import ExecutionContext, PrincipalType +from ..secrets import redact_secrets +from ...entity.persistence.metadata import WorkspaceMetadata +from ...i18n import SUPPORTED_LANGUAGES +from ...i18n.workspace import get_workspace_language, set_workspace_language +from ...operation_trace import service as tracing +from ...pipeline.extension_preferences import normalize_extension_preferences +from ...utils import constants +from ...workspace.invitation_delivery import InvitationDeliveryService + + +class Empty(Input): + pass + + +class Language(Input): + language: str + + +class Member(Input): + account_uuid: str + + +class MemberRole(Member): + role: Literal['admin', 'developer', 'operator', 'viewer'] + + +class Invite(Input): + email: str + role: Literal['admin', 'developer', 'operator', 'viewer'] = 'viewer' + + +class Invitation(Input): + invitation_uuid: str + + +class Governance(Input): + level: int | None = None + retention_days: int | None = None + max_rows: int | None = None + dedupe_window_seconds: int | None = None + + +class LogFilters(Input): + action: str | None = None + resource_type: str | None = None + actor_account_uuid: str | None = None + level: int | None = None + since: datetime.datetime | None = None + until: datetime.datetime | None = None + + +class LogPage(LogFilters): + limit: int = Field(default=100, ge=1) + offset: int = Field(default=0, ge=0) + integrity: str | None = None + search: str | None = None + + +class TaskList(Input): + task_type: str | None = None + task_kind: str | None = None + + +class TaskGet(Input): + task_id: int = Field(ge=0) + + +class Upload(Input): + filename: str = Field(min_length=1) + content: bytes = Field(max_length=10 * 1024 * 1024) + + +class ImageGet(Input): + file_key: str + + +class ToolList(Input): + pipeline_uuid: str | None = None + + +class ToolGet(ToolList): + tool_name: str + + +class RequesterList(Input): + model_type: str = '' + + +class RequesterGet(Input): + requester_name: str + + +class AdapterGet(Input): + adapter_name: str + + +class WizardStatus(Input): + status: Literal['completed', 'skipped'] = 'completed' + + +class WizardProgress(Input): + step: int = Field(ge=0) + selected_scenario: str | None = None + selected_adapter: str | None = None + created_bot_uuid: str | None = None + bot_saved: bool = False + selected_runner: str | None = None + + +def _workspace(row): + return {key: getattr(row, key) for key in ('uuid', 'instance_uuid', 'name', 'slug', 'type', 'status', 'source')} + + +def _membership(row, *, display_name='', email=''): + value = {key: getattr(row, key) for key in ('uuid', 'workspace_uuid', 'account_uuid', 'role', 'status')} + value.update(display_name=display_name, email=email) + for key in ('joined_at', 'created_at'): + timestamp = getattr(row, key) + value[key] = timestamp.isoformat() if timestamp else None + return value + + +def _invitation(row): + value = {key: getattr(row, key) for key in ('uuid', 'workspace_uuid', 'normalized_email', 'role', 'status')} + for key in ('expires_at', 'created_at'): + timestamp = getattr(row, key) + value[key] = timestamp.isoformat() if timestamp else None + return value + + +@asynccontextmanager +async def _collaboration(ap, context): + # Collaboration mutators require a tenant transaction in cloud runtime. + if getattr(getattr(ap.persistence_mgr, 'mode', None), 'value', None) == 'cloud_runtime': + async with ap.persistence_mgr.tenant_uow(context.workspace_uuid): + yield + else: + yield + + +async def _actor(ap, context): + if context.principal.principal_type != PrincipalType.ACCOUNT or not context.account_uuid: + raise PermissionDeniedError('account_membership_required') + access = await ap.workspace_collaboration_service.resolve_account_workspace( + context.account_uuid, context.workspace_uuid + ) + if access.membership.uuid != context.workspace.membership_uuid: + raise PermissionDeniedError('account_membership_required') + return access.membership + + +async def workspace_get(ap, context, payload): + return _workspace(await ap.workspace_service.get_workspace(context.workspace_uuid)) + + +async def workspace_current(ap, context, payload): + workspace = await ap.workspace_service.get_workspace(context.workspace_uuid) + result = { + 'workspace': _workspace(workspace), + 'permissions': sorted(context.workspace.permissions), + 'placement_generation': context.placement_generation, + 'membership': None, + 'plan_name': None, + 'current_plan': None, + 'limitation': None, + } + if context.principal.principal_type == PrincipalType.ACCOUNT: + member = await _actor(ap, context) + account = await ap.user_service.get_user_by_uuid(context.account_uuid) + result['membership'] = _membership(member, display_name=account.user, email=account.normalized_email) + resolver = getattr(ap, 'entitlement_resolver', None) + if workspace.source == 'cloud_projection' and resolver is not None: + from ..http.service.workspace_limits import LIMIT_KEYS + + entitlement = await resolver.resolve(workspace.uuid, minimum_revision=context.entitlement_revision) + result['plan_name'] = entitlement.plan_name + result['limitation'] = {key: entitlement.limit(value) for key, value in LIMIT_KEYS.items()} + if entitlement.plan_revision is not None and entitlement.monthly_bonus_credits is not None: + result['current_plan'] = { + 'name': entitlement.plan_name, + 'subscription_started_at': entitlement.subscription_started_at, + 'revision': entitlement.plan_revision, + 'monthly_bonus_credits': entitlement.monthly_bonus_credits, + 'limits': entitlement.limits, + 'entitlement_features': entitlement.features, + } + return result + + +async def language_get(ap, context, payload): + return { + 'language': await get_workspace_language(ap, context.workspace_uuid, refresh=True), + 'supported_languages': list(SUPPORTED_LANGUAGES), + } + + +async def language_update(ap, context, payload): + return {'language': await set_workspace_language(ap, context.workspace_uuid, payload.language)} + + +async def members_list(ap, context, payload): + async with _collaboration(ap, context): + rows = await ap.workspace_collaboration_service.list_members(context.workspace_uuid, await _actor(ap, context)) + return [_membership(row.membership, display_name=row.display_name, email=row.email) for row in rows] + + +async def _record_change(ap, context, action, changes, resource_id=None): + service = ap.workspace_settings_service + rule = tracing.ACTION_RULES_BY_ACTION[action] + await service.record( + context.workspace_uuid, + rule=rule, + level=max( + await service.effective_level(context.workspace_uuid, context.workspace.role), + tracing.OPERATION_LEVEL_MUTATION, + ) + if action == 'settings_update' + else await service.effective_level(context.workspace_uuid, context.workspace.role), + actor_account_uuid=context.account_uuid, + actor_role=context.workspace.role, + principal_type=context.principal.principal_type.value, + api_key_uuid=context.principal.api_key_uuid, + auth_type=context.auth_type, + request_id=context.request_id, + resource_id=resource_id, + outcome='ok', + summary=tracing.build_summary(rule, changes), + changes=changes, + ) + + +async def member_update(ap, context, payload): + async with _collaboration(ap, context): + actor = await _actor(ap, context) + members = await ap.workspace_collaboration_service.list_members(context.workspace_uuid, actor) + before = next( + (row.membership.role for row in members if row.membership.account_uuid == payload.account_uuid), None + ) + member = await ap.workspace_collaboration_service.update_member_role( + context.workspace_uuid, payload.account_uuid, payload.role, actor + ) + account = await ap.user_service.get_user_by_uuid(member.account_uuid) + result = _membership( + member, display_name=account.user if account else '', email=account.normalized_email if account else '' + ) + await _record_change( + ap, + context, + 'member_role_update', + [{'field': 'role', 'before': before, 'after': member.role}], + payload.account_uuid, + ) + return result + + +async def member_remove(ap, context, payload): + async with _collaboration(ap, context): + actor = await _actor(ap, context) + member = await ap.workspace_collaboration_service.remove_member( + context.workspace_uuid, payload.account_uuid, actor + ) + await _record_change( + ap, + context, + 'member_remove', + [{'field': 'member', 'before': f'{member.account_uuid}:{member.role}', 'after': 'removed'}], + member.account_uuid, + ) + return {'account_uuid': member.account_uuid} + + +async def invitations_list(ap, context, payload): + async with _collaboration(ap, context): + rows = await ap.workspace_collaboration_service.list_invitations( + context.workspace_uuid, await _actor(ap, context) + ) + return [_invitation(row) for row in rows] + + +async def invitation_create(ap, context, payload): + async with _collaboration(ap, context): + created = await ap.workspace_collaboration_service.create_invitation( + context.workspace_uuid, await _actor(ap, context), payload.email, payload.role + ) + await _record_change( + ap, + context, + 'member_invite', + [ + { + 'field': 'invitation', + 'before': None, + 'after': f'{created.invitation.normalized_email}:{created.invitation.role}', + } + ], + created.invitation.uuid, + ) + delivery_service = getattr(ap, 'invitation_delivery_service', None) or InvitationDeliveryService(ap) + link = delivery_service.build_invitation_link(created.token) + workspace = await ap.workspace_service.get_workspace(context.workspace_uuid) + delivery = await delivery_service.deliver_invitation( + recipient_email=created.invitation.normalized_email, workspace_name=workspace.name, invitation_link=link + ) + return { + 'invitation': _invitation(created.invitation), + 'token': created.token, + 'link': link, + 'delivery': delivery.to_public_dict(), + } + + +async def invitation_revoke(ap, context, payload): + async with _collaboration(ap, context): + row = await ap.workspace_collaboration_service.revoke_invitation( + context.workspace_uuid, payload.invitation_uuid, await _actor(ap, context) + ) + return _invitation(row) + + +async def governance_get(ap, context, payload): + return await ap.workspace_settings_service.describe_governance(context.workspace_uuid) + + +async def governance_update(ap, context, payload): + if not tracing.role_may_configure(context.workspace.role): + raise PermissionDeniedError('workspace_owner_or_admin_required') + service = ap.workspace_settings_service + before = await service.describe_governance(context.workspace_uuid) + options = payload.model_dump(exclude={'level'}) + result = await service.set_operation_level( + context.workspace_uuid, payload.level if payload.level is not None else before['configured_level'], **options + ) + keys = ('configured_level', 'retention_days', 'max_rows', 'dedupe_window_seconds') + changes = tracing.changed_fields({key: before[key] for key in keys}, {key: result[key] for key in keys}) + if changes: + await _record_change(ap, context, 'settings_update', changes) + return result + + +def _log_filters(payload): + values = payload.model_dump() + for key in ('since', 'until'): + if values[key] is not None and values[key].tzinfo is not None: + values[key] = values[key].astimezone(datetime.UTC).replace(tzinfo=None) + return values + + +async def logs_list(ap, context, payload): + return await ap.workspace_settings_service.query_logs(context.workspace_uuid, **_log_filters(payload)) + + +async def logs_export(ap, context, payload): + return await ap.workspace_settings_service.export_logs(context.workspace_uuid, **_log_filters(payload)) + + +async def logs_filters(ap, context, payload): + options = await ap.workspace_settings_service.list_filter_options(context.workspace_uuid) + options['governance'] = await ap.workspace_settings_service.describe_governance(context.workspace_uuid) + return options + + +def _task_scope(context): + return { + 'instance_uuid': context.instance_uuid, + 'workspace_uuid': context.workspace_uuid, + 'placement_generation': context.placement_generation, + } + + +async def tasks_list(ap, context, payload): + return ap.task_mgr.get_tasks_dict( + payload.task_type or None, + payload.task_kind or None, + **_task_scope(context), + public=context.principal.principal_type == PrincipalType.API_KEY, + ) + + +async def task_get(ap, context, payload): + task = ap.task_mgr.get_task_by_id(payload.task_id, **_task_scope(context)) + if task is None: + raise FileNotFoundError('Task not found') + return task.to_public_dict() if context.principal.principal_type == PrincipalType.API_KEY else task.to_dict() + + +async def _upload(ap, context, payload, *, image): + if '/' in payload.filename or '\\' in payload.filename or '\x00' in payload.filename: + raise ValueError('File name contains invalid characters') + extension = payload.filename.rsplit('.', 1)[1] if '.' in payload.filename else '' + if image: + extension = extension.lower() + if extension not in {'jpg', 'jpeg', 'png', 'gif', 'webp'}: + raise ValueError('Invalid image format') + owner = ( + f'account:{context.account_uuid}' + if context.account_uuid + else f'api-key:{context.principal.api_key_uuid}' + if context.principal.api_key_uuid + else f'principal:{context.principal.principal_type.value}' + ) + key = str(uuid.uuid4()) + (f'.{extension}' if extension else '') + return await ap.storage_mgr.save_scoped( + context, owner_type='upload_image' if image else 'upload_document', owner=owner, key=key, value=payload.content + ) + + +async def image_upload(ap, context, payload): + return {'file_key': await _upload(ap, context, payload, image=True)} + + +async def document_upload(ap, context, payload): + return {'file_id': await _upload(ap, context, payload, image=False)} + + +async def image_get(ap, context, payload): + # The foundation reads only this Workspace, including keys otherwise publicly addressable. + storage = ap.storage_mgr + if storage.is_scoped_object_key(payload.file_key, expected_owner_type='upload_image'): + owner_type = 'upload_image' + elif storage.is_scoped_object_key(payload.file_key, expected_owner_type='bot_log'): + owner_type = 'bot_log' + else: + raise FileNotFoundError('Image not found') + content = await storage.load_scoped_object_key(context, payload.file_key, expected_owner_type=owner_type) + return {'content': content, 'media_type': mimetypes.guess_type(payload.file_key)[0] or 'image/jpeg'} + + +async def box_status(ap, context, payload): + status = await ap.box_service.get_status(context) + return {**status, 'hidden': constants.edition == 'cloud' and status.get('enabled') is False} + + +async def box_sessions(ap, context, payload): + return await ap.box_service.get_sessions(context) + + +async def box_errors(ap, context, payload): + if ap.box_service.managed_admission_required: + await ap.box_service.require_workspace_sandbox(context) + return ap.box_service.get_recent_errors(context) + + +async def tools_list(ap, context, payload): + bound_plugins = bound_mcp_servers = None + if payload.pipeline_uuid: + pipeline = await ap.pipeline_service.get_pipeline(context, payload.pipeline_uuid) + if pipeline is None: + raise FileNotFoundError('Pipeline not found') + prefs = normalize_extension_preferences(pipeline.get('extensions_preferences')) + if not prefs['enable_all_plugins']: + bound_plugins = [f'{plugin["author"]}/{plugin["name"]}' for plugin in prefs['plugins']] + if not prefs['enable_all_mcp_servers']: + bound_mcp_servers = prefs['mcp_servers'] + return await ap.tool_mgr.get_resolved_tool_catalog( + context, bound_plugins, bound_mcp_servers, include_skill_authoring=True + ) + + +async def tool_get(ap, context, payload): + for tool in await tools_list(ap, context, payload): + if tool.get('name') == payload.tool_name: + return tool + raise FileNotFoundError('Tool not found') + + +async def extensions_list(ap, context, payload): + if ap.plugin_connector.is_enable_plugin: + await ap.plugin_connector.require_workspace_context(context) + # Keep failures visible to programmatic callers instead of returning a partial catalog. + plugins = await ap.plugin_connector.list_plugins() + servers = await ap.mcp_service.get_mcp_servers(context, contain_runtime_info=True) + skills = await ap.skill_service.list_skills(context) + extensions = [{'type': 'plugin', 'plugin': redact_secrets(plugin)} for plugin in plugins] + extensions += [{'type': 'mcp', 'server': server} for server in servers] + extensions += [{'type': 'skill', 'skill': skill} for skill in skills] + return extensions + + +async def basic_stats(ap, context, payload): + execution = ExecutionContext.from_request(context) + sessions = [ + session + for session in ap.sess_mgr.session_list + if all(getattr(session, key, None) == value for key, value in _task_scope(context).items()) + ] + return { + 'active_session_count': len(sessions), + 'conversation_count': sum(len(session.conversations or []) for session in sessions), + 'query_count': ap.query_pool.get_query_count(execution), + } + + +async def _metadata_set(ap, context, key, value): + condition = (WorkspaceMetadata.workspace_uuid == context.workspace_uuid, WorkspaceMetadata.key == key) + result = await ap.persistence_mgr.execute_async(sqlalchemy.select(WorkspaceMetadata).where(*condition)) + statement = ( + sqlalchemy.update(WorkspaceMetadata).where(*condition).values(value=value) + if result.first() + else sqlalchemy.insert(WorkspaceMetadata).values(workspace_uuid=context.workspace_uuid, key=key, value=value) + ) + await ap.persistence_mgr.execute_async(statement) + + +async def wizard_complete(ap, context, payload): + await _metadata_set(ap, context, 'wizard_status', payload.status) + await ap.persistence_mgr.execute_async( + sqlalchemy.delete(WorkspaceMetadata).where( + WorkspaceMetadata.workspace_uuid == context.workspace_uuid, WorkspaceMetadata.key == 'wizard_progress' + ) + ) + return None + + +async def wizard_progress(ap, context, payload): + await _metadata_set(ap, context, 'wizard_progress', json.dumps(payload.model_dump(), ensure_ascii=False)) + return None + + +async def recommended_model(ap, context, payload): + return await ap.space_service.get_recommended_chat_model(context) + + +async def model_availability(ap, context, payload): + return [model.model_dump(mode='json') for model in await ap.space_service.get_model_selection()] + + +async def requester_list(ap, context, payload): + return ap.model_mgr.get_available_requesters_info(payload.model_type) + + +async def requester_get(ap, context, payload): + result = ap.model_mgr.get_available_requester_info_by_name(payload.requester_name) + if result is None: + raise FileNotFoundError('Requester not found') + return result + + +async def adapter_list(ap, context, payload): + from ...cloud.plan_policy import workspace_entitlement, platform_feature_names + + snapshot = await workspace_entitlement(ap, context.workspace_uuid) + adapters = ap.platform_mgr.get_available_adapters_info() + if snapshot is not None: + adapters = [ + adapter + for adapter in adapters + if not any(snapshot.features.get(feature) is False for feature in platform_feature_names(adapter['name'])) + ] + return adapters + + +async def adapter_get(ap, context, payload): + from ...cloud.plan_policy import require_workspace_platform + + await require_workspace_platform(ap, context.workspace_uuid, payload.adapter_name) + result = ap.platform_mgr.get_available_adapter_info_by_name(payload.adapter_name) + if result is None: + raise FileNotFoundError('Adapter not found') + return result + + +async def storage_analysis(ap, context, payload): + return await ap.maintenance_service.get_storage_analysis(context) + + +async def wizard_get(ap, context, payload): + result = await ap.persistence_mgr.execute_async( + sqlalchemy.select(WorkspaceMetadata).where( + WorkspaceMetadata.workspace_uuid == context.workspace_uuid, + WorkspaceMetadata.key.in_(['wizard_status', 'wizard_progress']), + ) + ) + values = {row.key: row.value for row in result.all()} + return { + 'status': values.get('wizard_status'), + 'progress': json.loads(values['wizard_progress']) if values.get('wizard_progress') else None, + } + + +async def plugin_status(ap, context, payload): + await ap.plugin_connector.require_workspace_context(context) + try: + await ap.plugin_connector.ping_plugin_runtime() + except Exception as exc: + return { + 'is_enable': ap.plugin_connector.is_enable_plugin, + 'is_connected': False, + 'plugin_connector_error': str(exc), + } + return {'is_enable': ap.plugin_connector.is_enable_plugin, 'is_connected': True, 'plugin_connector_error': 'ok'} + + +def operations() -> list[Operation]: + view, manage = Permission.RESOURCE_VIEW, Permission.RESOURCE_MANAGE + account = frozenset({PrincipalType.ACCOUNT}) + return [ + Operation('workspace.get', Permission.WORKSPACE_VIEW, workspace_get, Empty, dict), + Operation('workspace.current', Permission.WORKSPACE_VIEW, workspace_current, Empty, dict), + Operation('workspace.language.get', Permission.WORKSPACE_VIEW, language_get, Empty, dict), + Operation('workspace.language.update', Permission.WORKSPACE_UPDATE, language_update, Language, dict, True), + Operation('workspace.member.list', Permission.MEMBER_VIEW, members_list, Empty, list[dict], principals=account), + Operation( + 'workspace.member.update', + Permission.MEMBER_UPDATE_ROLE, + member_update, + MemberRole, + dict, + True, + principals=account, + ), + Operation( + 'workspace.member.remove', Permission.MEMBER_REMOVE, member_remove, Member, dict, True, principals=account + ), + Operation( + 'workspace.invitation.list', + Permission.MEMBER_INVITE, + invitations_list, + Empty, + list[dict], + principals=account, + ), + Operation( + 'workspace.invitation.create', + Permission.MEMBER_INVITE, + invitation_create, + Invite, + dict, + True, + principals=account, + ), + Operation( + 'workspace.invitation.revoke', + Permission.MEMBER_INVITE, + invitation_revoke, + Invitation, + dict, + True, + principals=account, + ), + Operation('settings.governance.get', Permission.AUDIT_VIEW, governance_get, Empty, dict), + Operation('settings.governance.update', Permission.AUDIT_VIEW, governance_update, Governance, dict, True), + Operation('settings.operation_logs.list', Permission.AUDIT_VIEW, logs_list, LogPage, dict), + Operation('settings.operation_logs.filters', Permission.AUDIT_VIEW, logs_filters, Empty, dict), + Operation('settings.operation_logs.export', Permission.AUDIT_VIEW, logs_export, LogFilters, dict), + Operation('task.list', view, tasks_list, TaskList, dict), + Operation('task.get', view, task_get, TaskGet, dict), + Operation('file.image.upload', manage, image_upload, Upload, dict[str, str], True), + Operation('file.document.upload', manage, document_upload, Upload, dict[str, str], True), + Operation('file.image.get', view, image_get, ImageGet, dict[str, bytes | str]), + Operation('box.status', view, box_status, Empty, dict), + Operation('box.sessions.list', Permission.AUDIT_VIEW, box_sessions, Empty, list[dict]), + Operation('box.errors.list', Permission.AUDIT_VIEW, box_errors, Empty, list[dict]), + Operation('tool.list', view, tools_list, ToolList, list[dict]), + Operation('tool.get', view, tool_get, ToolGet, dict), + Operation('extension.list', view, extensions_list, Empty, list[dict]), + Operation('requester.list', view, requester_list, RequesterList, list[dict]), + Operation('requester.get', view, requester_get, RequesterGet, dict), + Operation('adapter.list', view, adapter_list, Empty, list[dict]), + Operation('adapter.get', view, adapter_get, AdapterGet, dict), + Operation('stats.basic', view, basic_stats, Empty, dict[str, int]), + Operation('system.wizard.get', Permission.WORKSPACE_VIEW, wizard_get, Empty, dict), + Operation('system.storage.analysis', Permission.AUDIT_VIEW, storage_analysis, Empty, dict), + Operation( + 'system.wizard.complete', Permission.WORKSPACE_UPDATE, wizard_complete, WizardStatus, type(None), True + ), + Operation( + 'system.wizard.progress.update', + Permission.WORKSPACE_UPDATE, + wizard_progress, + WizardProgress, + type(None), + True, + ), + Operation('system.model.recommended', manage, recommended_model, Empty, dict), + Operation('system.model.availability', view, model_availability, Empty, list[dict]), + Operation('system.plugin.status', view, plugin_status, Empty, dict), + ] diff --git a/src/langbot/pkg/api/application/connection_operations.py b/src/langbot/pkg/api/application/connection_operations.py new file mode 100644 index 000000000..896882326 --- /dev/null +++ b/src/langbot/pkg/api/application/connection_operations.py @@ -0,0 +1,82 @@ +"""Workspace management of dashboard chat history and active connections.""" + +from __future__ import annotations + +import datetime +from typing import Literal + +from pydantic import Field + +from .contracts import Input, Operation +from ..authz import Permission + + +class Pipeline(Input): + pipeline_uuid: str = Field(min_length=1) + + +class Session(Pipeline): + session_type: Literal['person', 'group'] + + +class Broadcast(Pipeline): + message: str = Field(min_length=1, max_length=100_000) + + +async def _require_pipeline(ap, context, pipeline_uuid): + if await ap.pipeline_service.get_pipeline(context, pipeline_uuid) is None: + raise FileNotFoundError('Pipeline not found') + + +async def history(ap, context, payload): + await _require_pipeline(ap, context, payload.pipeline_uuid) + proxy = await ap.platform_mgr.get_websocket_proxy_bot(context) + return proxy.adapter.get_websocket_messages(payload.pipeline_uuid, payload.session_type) + + +async def reset(ap, context, payload): + await _require_pipeline(ap, context, payload.pipeline_uuid) + proxy = await ap.platform_mgr.get_websocket_proxy_bot(context) + proxy.adapter.reset_session(payload.pipeline_uuid, payload.session_type) + + +async def connections(ap, context, payload): + from ...platform.sources.websocket_manager import WebSocketScope, ws_connection_manager + + await _require_pipeline(ap, context, payload.pipeline_uuid) + scope = WebSocketScope.from_context(context) + rows = await ws_connection_manager.get_connections_by_pipeline(payload.pipeline_uuid, scope=scope) + return { + 'stats': ws_connection_manager.get_stats(scope=scope), + 'connections': [ + { + 'connection_id': row.connection_id, + 'session_type': row.session_type, + 'created_at': row.created_at.isoformat(), + 'last_active': row.last_active.isoformat(), + 'is_active': row.is_active, + } + for row in rows + ], + } + + +async def broadcast(ap, context, payload): + from ...platform.sources.websocket_manager import WebSocketScope, ws_connection_manager + + await _require_pipeline(ap, context, payload.pipeline_uuid) + await ws_connection_manager.broadcast_to_pipeline( + payload.pipeline_uuid, + {'type': 'broadcast', 'message': payload.message, 'timestamp': datetime.datetime.now().isoformat()}, + scope=WebSocketScope.from_context(context), + ) + + +def operations() -> list[Operation]: + permission = Permission.RUNTIME_OPERATE + return [ + Operation('pipeline.chat.history', permission, history, Session, list[dict]), + Operation('pipeline.chat.reset', permission, reset, Session, type(None), True), + Operation('pipeline.chat.connections', permission, connections, Pipeline, dict), + Operation('pipeline.chat.broadcast', permission, broadcast, Broadcast, type(None), True), + ] diff --git a/src/langbot/pkg/api/application/contracts.py b/src/langbot/pkg/api/application/contracts.py new file mode 100644 index 000000000..811d31fa8 --- /dev/null +++ b/src/langbot/pkg/api/application/contracts.py @@ -0,0 +1,56 @@ +"""Versioned, transport-independent application operation contracts.""" + +from __future__ import annotations + +from collections.abc import Awaitable, Callable +from dataclasses import dataclass +from typing import Any + +from pydantic import BaseModel, ConfigDict, TypeAdapter + +from ..authz import Permission +from ..context import PrincipalType, RequestContext + + +class Input(BaseModel): + model_config = ConfigDict(strict=True, extra='forbid', revalidate_instances='always') + + +@dataclass(frozen=True, slots=True) +class ApplicationAPIError: + code: str + message: str + status_code: int = 400 + details: dict | None = None + + +class ApplicationAPIException(Exception): + def __init__(self, error: ApplicationAPIError): + self.error = error + super().__init__(error.message) + + +@dataclass(frozen=True, slots=True) +class Operation: + operation_id: str + permission: Permission + handler: Callable[[Any, RequestContext, Input], Awaitable[Any]] + input_type: type[Input] + output_type: Any + side_effect: bool = False + version: str = '1' + principals: frozenset[PrincipalType] | None = None + streaming: bool = False + + def as_dict(self) -> dict: + return { + 'id': self.operation_id, + 'version': self.version, + 'scope': 'workspace', + 'permission': self.permission.value, + 'side_effect': self.side_effect, + 'streaming': self.streaming, + 'principals': sorted(self.principals) if self.principals is not None else None, + 'input_schema': self.input_type.model_json_schema(), + 'output_schema': TypeAdapter(self.output_type).json_schema(), + } diff --git a/src/langbot/pkg/api/application/errors.py b/src/langbot/pkg/api/application/errors.py new file mode 100644 index 000000000..a45d3bdb6 --- /dev/null +++ b/src/langbot/pkg/api/application/errors.py @@ -0,0 +1,70 @@ +"""Translate business failures into stable application errors without HTTP imports.""" + +from __future__ import annotations + +from langbot_plugin.box.errors import BoxAdmissionError +from pydantic import ValidationError + + +from ...cloud.entitlements import EntitlementUnavailableError +from ...core.errors import TaskCapacityError +from ...utils.bounded_executor import BlockingWorkCapacityError +from ...workspace.collaboration import MembershipPermissionError, WorkspaceCollaborationError +from ...workspace.errors import WorkspaceExecutionUnavailableError, WorkspaceNotFoundError +from ..authz import AuthorizationError +from ..http.service.bot_errors import BotApplyError +from .contracts import ApplicationAPIError, ApplicationAPIException + + +def error_from_exception(exc: Exception, *, operation_id: str = '') -> ApplicationAPIError: + from ...agent.runner.errors import ( + RunnerError, + RunnerExecutionError, + RunnerNotAuthorizedError, + RunnerNotFoundError, + RunnerProtocolError, + ) + + if isinstance(exc, ApplicationAPIException): + return exc.error + if isinstance(exc, AuthorizationError): + return ApplicationAPIError(exc.error_code, str(exc), exc.status_code) + if isinstance(exc, (WorkspaceNotFoundError, FileNotFoundError)): + return ApplicationAPIError('resource_not_found', 'Resource not found', 404) + if isinstance(exc, WorkspaceExecutionUnavailableError): + return ApplicationAPIError('workspace_unavailable', 'Workspace execution is unavailable', 409) + if isinstance(exc, MembershipPermissionError): + return ApplicationAPIError(exc.code, str(exc), 403) + if isinstance(exc, WorkspaceCollaborationError): + return ApplicationAPIError(exc.code, str(exc), 400) + if isinstance(exc, BoxAdmissionError): + return ApplicationAPIError('managed_sandbox_unavailable', str(exc), 403) + if isinstance(exc, EntitlementUnavailableError): + code = 'managed_sandbox_unavailable' if operation_id.startswith('box.') else 'entitlement_unavailable' + return ApplicationAPIError(code, str(exc), 403) + if isinstance(exc, TaskCapacityError): + return ApplicationAPIError('task_capacity_exceeded', str(exc), 429) + if isinstance(exc, BlockingWorkCapacityError): + return ApplicationAPIError('blocking_work_capacity_exceeded', str(exc), 429) + if isinstance(exc, BotApplyError): + return ApplicationAPIError('bot_apply_failed', str(exc), 400, details={'uuid': exc.bot_uuid}) + if isinstance(exc, RunnerExecutionError): + return ApplicationAPIError(exc.error_code or 'runner_execution_failed', exc.message, 422) + if isinstance(exc, RunnerNotFoundError): + return ApplicationAPIError('runner_not_found', 'The configured Agent runner is unavailable', 409) + if isinstance(exc, RunnerNotAuthorizedError): + return ApplicationAPIError('runner_not_authorized', 'The configured Agent runner is not authorized', 403) + if isinstance(exc, RunnerProtocolError): + return ApplicationAPIError('runner_protocol_error', 'The Agent runner returned an invalid response', 502) + if isinstance(exc, RunnerError): + return ApplicationAPIError('runner_error', 'The Agent runner could not complete this test', 502) + code = getattr(exc, 'error_code', None) or getattr(exc, 'code', None) + status = getattr(exc, 'status_code', None) or getattr(exc, 'status', None) + if isinstance(code, str) and type(status) is int and 400 <= status <= 599: + return ApplicationAPIError(code, str(exc), status) + if isinstance(exc, ValidationError): + # Pydantic's normal text includes raw provider/plugin configuration values. + return ApplicationAPIError('validation_error', 'Invalid operation data', 400) + if isinstance(exc, ValueError): + return ApplicationAPIError('validation_error', str(exc), 400) + return ApplicationAPIError('application_error', 'Application operation failed', 500) diff --git a/src/langbot/pkg/api/application/execution.py b/src/langbot/pkg/api/application/execution.py new file mode 100644 index 000000000..d2e77f2c1 --- /dev/null +++ b/src/langbot/pkg/api/application/execution.py @@ -0,0 +1,759 @@ +"""Transport-neutral plugin and run operations over the existing secured Host actions. + +Installation contexts are capabilities supplied by trusted Host code, never by a +request body. A run context cannot fall back to legacy plugin-wide privileges. +The existing runtime consumers are deliberately not routed through this facade. +""" + +from __future__ import annotations + +import contextlib +from dataclasses import dataclass +from typing import Annotated, Any, Literal + +from langbot_plugin.entities.io.context import InstallationBinding +from langbot_plugin.entities.io.resp import ActionResponse +from pydantic import BaseModel, ConfigDict, Field, JsonValue, RootModel, TypeAdapter, ValidationError, create_model + +from .contracts import ApplicationAPIError, ApplicationAPIException, Input +from .streaming import isolated_stream + +Text = Annotated[str, Field(min_length=1)] +Positive = Annotated[int, Field(gt=0)] +Nonnegative = Annotated[int, Field(ge=0)] +Object = dict[str, JsonValue] +QueryID = str | int + + +class JSONOutput(RootModel[Object]): + model_config = ConfigDict(strict=True, allow_inf_nan=False) + + +class Output(BaseModel): + model_config = ConfigDict(extra='allow', strict=True, allow_inf_nan=False) + __pydantic_extra__: dict[str, JsonValue] = Field(init=False) + + +def _input(name: str, **fields: Any) -> type[Input]: + return create_model(name, __base__=Input, **fields) + + +def _output(name: str, **fields: Any) -> type[Output]: + return create_model(name, __base__=Output, **fields) + + +@dataclass(frozen=True, slots=True) +class ExecutionAPIContext: + binding: InstallationBinding + run_id: str | None = None + + +@dataclass(frozen=True, slots=True) +class ExecutionOperation: + action: str + input_type: type[Input] + output_type: Any + scope: Literal['plugin', 'run', 'plugin_or_run', 'runner_admin'] = 'plugin' + side_effect: bool = False + streaming: bool = False + + @property + def operation_id(self) -> str: + return f'execution.{self.action}' + + def as_dict(self) -> dict[str, Any]: + return { + 'id': self.operation_id, + 'version': '1', + 'scope': self.scope, + 'authorization': 'installation_binding_and_existing_host_capabilities', + 'side_effect': self.side_effect, + 'streaming': self.streaming, + 'input_schema': self.input_type.model_json_schema(), + 'output_schema': TypeAdapter(self.output_type).json_schema(), + } + + +# Only reviewed business actions are public; runtime setup/chunks/ping stay internal. +Empty = _input('ExecutionEmptyInput') +Query = _input('ExecutionQueryInput', query_id=(QueryID, ...)) +QueryKey = _input('ExecutionQueryKeyInput', query_id=(QueryID, ...), key=(Text, ...)) +ModelCall = _input( + 'ExecutionModelInput', + llm_model_uuid=(Text, ...), + messages=(list[Object], ...), + funcs=(list[Object], []), + extra_args=(Object, {}), + reasoning_level=(str | None, None), + remove_think=(bool, False), + query_id=(QueryID | None, None), +) +Retrieval = _input( + 'ExecutionRetrievalInput', + kb_id=(Text, ...), + query_text=(str, ...), + top_k=(Positive, 5), + filters=(Object | None, None), +) +Storage = _input('ExecutionStorageInput', owner_type=(Literal['plugin', 'workspace'], ...), key=(Text, ...)) +State = _input('ExecutionStateInput', scope=(Text, ...), key=(Text, ...)) +TargetRun = _input('ExecutionTargetRunInput', target_run_id=(Text | None, None)) +Page = _input( + 'ExecutionPageInput', + conversation_id=(Text | None, None), + before_cursor=(str | None, None), + after_cursor=(str | None, None), + limit=(Positive, 50), + direction=(Literal['forward', 'backward'], 'backward'), + include_attachments=(bool, False), +) +TargetPage = _input( + 'ExecutionTargetPageInput', + target_run_id=(Text | None, None), + before_cursor=(str | None, None), + after_cursor=(str | None, None), + limit=(Positive, 50), + direction=(Literal['forward', 'backward'], 'forward'), +) +ClaimFields = {'runtime_id': (Text, ...), 'claim_token': (Text, ...), 'target_run_id': (Text | None, None)} +RuntimeFields = { + 'runtime_id': (Text, ...), + 'status': (Text | None, None), + 'capabilities': (Object | None, None), + 'labels': (Object | None, None), + 'metadata': (Object | None, None), + 'heartbeat_deadline_at': (float | int | None, None), + 'heartbeat_ttl_seconds': (Positive, 60), +} + +EmptyOutput = _output('ExecutionEmptyOutput') +ValueOutput = _output('ExecutionValueOutput', value=(JsonValue, ...)) +ResultOutput = _output('ExecutionResultOutput', result=(JsonValue, ...)) +ItemsOutput = _output('ExecutionItemsOutput', items=(list[JsonValue], ...)) +ResultsOutput = _output('ExecutionResultsOutput', results=(list[JsonValue], ...)) +KeysOutput = _output('ExecutionKeysOutput', keys=(list[str], ...)) +PageOutput = _output( + 'ExecutionPageOutput', + items=(list[JsonValue], ...), + next_cursor=(str | None, ...), + prev_cursor=(str | None, ...), + has_more=(bool, ...), +) +RunOutput = _output('ExecutionRunOutput', run_id=(str, ...), status=(str, ...)) +RuntimeOutput = _output('ExecutionRuntimeOutput', runtime_id=(str, ...)) +EventOutput = _output('ExecutionEventOutput', event_id=(str, ...)) +SuccessOutput = _output('ExecutionSuccessOutput', success=(bool, ...)) +ToolOutput = _output('ExecutionToolResultOutput', tool_response=(JsonValue, ...)) | ResultOutput +StreamOutput = _output('ExecutionModelChunkOutput', chunk=(Object, ...)) | _output( + 'ExecutionModelUsageOutput', usage=(Object, ...) +) + +EXECUTION_OPERATIONS = ( + ExecutionOperation( + 'reply_message', + _input( + 'ExecutionReplyInput', query_id=(QueryID, ...), message_chain=(list[Object], ...), quote_origin=(bool, ...) + ), + EmptyOutput, + side_effect=True, + ), + ExecutionOperation('get_bot_uuid', Query, _output('ExecutionBotIDOutput', bot_uuid=(str, ...))), + ExecutionOperation( + 'set_query_var', + _input('ExecutionSetQueryVariableInput', query_id=(QueryID, ...), key=(Text, ...), value=(JsonValue, ...)), + EmptyOutput, + side_effect=True, + ), + ExecutionOperation('get_query_var', QueryKey, ValueOutput), + ExecutionOperation('get_query_vars', Query, _output('ExecutionVariablesOutput', vars=(Object, ...))), + ExecutionOperation('create_new_conversation', Query, EmptyOutput, side_effect=True), + ExecutionOperation( + 'get_langbot_version', Empty, _output('ExecutionVersionOutput', version=(str, ...)), 'plugin_or_run' + ), + ExecutionOperation('get_bots', Empty, _output('ExecutionBotsOutput', bots=(list[JsonValue], ...))), + ExecutionOperation( + 'get_bot_info', + _input('ExecutionBotInput', bot_uuid=(Text, ...)), + _output('ExecutionBotOutput', bot=(Object, ...)), + ), + ExecutionOperation( + 'send_message', + _input( + 'ExecutionSendInput', + bot_uuid=(Text, ...), + target_type=(Literal['person', 'group'], ...), + target_id=(Text, ...), + message_chain=(list[Object], ...), + ), + ResultOutput, + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'call_platform_api', + _input( + 'ExecutionPlatformInput', + bot_uuid=(Text | None, None), + action=(Text, ...), + params=(Object, {}), + context_tool=(str | None, None), + file_ids=(list[str] | None, None), + ), + ResultOutput, + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'reply_stream', + _input( + 'ExecutionReplyStreamInput', + stream_id=(Text, ...), + operation=(Literal['update', 'finish', 'abort'], ...), + text=(str, ...), + ), + ResultOutput, + 'run', + True, + ), + ExecutionOperation('get_llm_models', Empty, _output('ExecutionModelsOutput', llm_models=(list[str], ...))), + ExecutionOperation('count_tokens', ModelCall, _output('ExecutionTokensOutput', tokens=(int, ...)), 'plugin_or_run'), + ExecutionOperation( + 'invoke_llm', ModelCall, _output('ExecutionModelOutput', message=(Object, ...)), 'plugin_or_run', True + ), + ExecutionOperation('invoke_llm_stream', ModelCall, StreamOutput, 'plugin_or_run', True, True), + ExecutionOperation( + 'call_tool', + _input( + 'ExecutionCallToolInput', + tool_name=(Text, ...), + parameters=(Object, {}), + tool_parameters=(Object | None, None), + ), + ToolOutput, + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'get_tool_detail', + _input('ExecutionToolInput', tool_name=(Text, ...)), + _output('ExecutionToolOutput', tool=(Object, ...)), + 'plugin_or_run', + ), + ExecutionOperation( + 'set_binary_storage', + _input( + 'ExecutionSetStorageInput', + owner_type=(Literal['plugin', 'workspace'], ...), + key=(Text, ...), + value_base64=(str, ...), + ), + EmptyOutput, + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'get_binary_storage', Storage, _output('ExecutionStorageOutput', value_base64=(str, ...)), 'plugin_or_run' + ), + ExecutionOperation('delete_binary_storage', Storage, EmptyOutput, 'plugin_or_run', True), + ExecutionOperation( + 'get_binary_storage_keys', + _input('ExecutionStorageKeysInput', owner_type=(Literal['plugin', 'workspace'], ...)), + KeysOutput, + 'plugin_or_run', + ), + ExecutionOperation( + 'get_config_file', + _input('ExecutionConfigFileInput', file_key=(Text, ...)), + _output('ExecutionConfigFileOutput', file_base64=(str, ...)), + ), + ExecutionOperation( + 'invoke_embedding', + _input('ExecutionEmbeddingInput', embedding_model_uuid=(Text, ...), texts=(list[str], ...)), + _output('ExecutionEmbeddingOutput', vectors=(list[list[float]], ...)), + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'invoke_rerank', + _input( + 'ExecutionRerankInput', + rerank_model_uuid=(Text, ...), + query=(str, ...), + documents=(list[str], ...), + top_k=(Positive | None, None), + extra_args=(Object, {}), + ), + ResultsOutput, + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'vector_upsert', + _input( + 'ExecutionVectorUpsertInput', + collection_id=(Text, ...), + ids=(list[str], ...), + vectors=(list[list[float]], ...), + documents=(list[str] | None, None), + metadata=(list[Object] | None, None), + ), + EmptyOutput, + side_effect=True, + ), + ExecutionOperation( + 'vector_search', + _input( + 'ExecutionVectorSearchInput', + collection_id=(Text, ...), + query_vector=(list[float], ...), + top_k=(Positive, ...), + filters=(Object | None, None), + query_text=(str, ''), + search_type=(str, 'vector'), + vector_weight=(float, 0.5), + ), + ResultsOutput, + ), + ExecutionOperation( + 'vector_delete', + _input( + 'ExecutionVectorDeleteInput', + collection_id=(Text, ...), + file_ids=(list[str] | None, None), + filters=(Object | None, None), + ), + _output('ExecutionVectorDeleteOutput', count=(int, ...)), + side_effect=True, + ), + ExecutionOperation( + 'vector_list', + _input( + 'ExecutionVectorListInput', + collection_id=(Text, ...), + filters=(Object | None, None), + limit=(Positive, 20), + offset=(Nonnegative, 0), + ), + _output('ExecutionVectorListOutput', items=(list[JsonValue], ...), total=(int, ...)), + ), + ExecutionOperation( + 'get_knowledge_file_stream', + _input('ExecutionKnowledgeFileInput', storage_path=(Text, ...)), + _output('ExecutionKnowledgeFileOutput', file_key=(str, ...)), + side_effect=True, + ), + ExecutionOperation( + 'list_parsers', + _input('ExecutionParsersInput', mime_type=(str | None, None)), + _output('ExecutionParsersOutput', parsers=(list[JsonValue], ...)), + ), + ExecutionOperation( + 'invoke_parser', + _input( + 'ExecutionParserInput', + plugin_author=(Text, ...), + plugin_name=(Text, ...), + storage_path=(Text, ...), + filename=(str, ''), + mime_type=(str, 'application/octet-stream'), + metadata=(Object, {}), + ), + JSONOutput, + side_effect=True, + ), + ExecutionOperation( + 'list_knowledge_bases', Empty, _output('ExecutionKnowledgeBasesOutput', knowledge_bases=(list[JsonValue], ...)) + ), + ExecutionOperation('retrieve_knowledge', Retrieval, ResultsOutput, 'plugin_or_run'), + ExecutionOperation( + 'list_pipeline_knowledge_bases', + Query, + _output('ExecutionPipelineKnowledgeBasesOutput', knowledge_bases=(list[JsonValue], ...)), + ), + ExecutionOperation( + 'retrieve_knowledge_base', + _input( + 'ExecutionPipelineRetrievalInput', + kb_id=(Text, ...), + query_text=(str, ...), + query_id=(QueryID | None, None), + top_k=(Positive, 5), + filters=(Object | None, None), + ), + ResultsOutput, + 'plugin_or_run', + ), + ExecutionOperation('get_prompt', Empty, _output('ExecutionPromptOutput', prompt=(list[Object], ...)), 'run'), + ExecutionOperation('history_page', Page, PageOutput, 'run'), + ExecutionOperation( + 'history_search', + _input('ExecutionHistorySearchInput', query=(str, ''), filters=(Object | None, None), top_k=(Positive, 10)), + _output('ExecutionHistorySearchOutput', items=(list[JsonValue], ...), total_count=(int, ...), query=(str, ...)), + 'run', + ), + ExecutionOperation('event_get', _input('ExecutionEventInput', event_id=(Text, ...)), EventOutput, 'run'), + ExecutionOperation( + 'event_page', + _input( + 'ExecutionEventPageInput', + conversation_id=(Text | None, None), + before_cursor=(str | None, None), + event_types=(list[str] | None, None), + limit=(Positive, 50), + ), + PageOutput, + 'run', + ), + ExecutionOperation( + 'steering_pull', + _input('ExecutionSteeringInput', mode=(str, 'all'), limit=(Positive | None, None)), + ItemsOutput, + 'run', + True, + ), + ExecutionOperation('state_get', State, ValueOutput, 'run'), + ExecutionOperation( + 'state_set', + _input('ExecutionSetStateInput', scope=(Text, ...), key=(Text, ...), value=(JsonValue, ...)), + SuccessOutput, + 'run', + True, + ), + ExecutionOperation('state_delete', State, SuccessOutput, 'run', True), + ExecutionOperation( + 'state_list', + _input('ExecutionStateListInput', scope=(Text, ...), prefix=(str | None, None), limit=(Positive, 100)), + _output('ExecutionStateKeysOutput', keys=(list[str], ...), has_more=(bool, ...)), + 'run', + ), + ExecutionOperation( + 'get_box_status', + Empty, + _output('ExecutionBoxStatusOutput', available=(bool, ...), enabled=(bool, ...)), + 'plugin_or_run', + ), + ExecutionOperation('list_boxes', Empty, ItemsOutput, 'plugin_or_run'), + ExecutionOperation( + 'acquire_box', + _input('ExecutionAcquireBoxInput', reuse_key=(Text, ...), options=(Object, {})), + _output('ExecutionBoxOutput', id=(str, ...), status=(str, ...)), + 'plugin_or_run', + True, + ), + ExecutionOperation( + 'bind_box', + _input('ExecutionBindBoxInput', box_id=(Text, ...)), + _output('ExecutionBoxBindingOutput', box_id=(str, ...), outbox=(str, ...)), + 'run', + True, + ), + ExecutionOperation( + 'import_box_attachments', + _input('ExecutionImportAttachmentsInput', attachment_ids=(list[str] | None, None)), + ItemsOutput, + 'run', + True, + ), + ExecutionOperation( + 'export_box_files', _input('ExecutionExportBoxInput', paths=(list[str] | None, None)), ItemsOutput, 'run', True + ), + ExecutionOperation('run_get', TargetRun, RunOutput, 'runner_admin'), + ExecutionOperation( + 'run_list', + _input( + 'ExecutionRunsInput', + conversation_id=(Text | None, None), + statuses=(list[str] | None, None), + before_cursor=(str | None, None), + limit=(Positive, 50), + ), + PageOutput, + 'runner_admin', + ), + ExecutionOperation('run_events_page', TargetPage, PageOutput, 'runner_admin'), + ExecutionOperation( + 'run_cancel', + _input( + 'ExecutionCancelRunInput', + target_run_id=(Text | None, None), + reason=(str | None, None), + status_reason=(str | None, None), + ), + RunOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'run_append_result', + _input( + 'ExecutionAppendResultInput', + target_run_id=(Text | None, None), + sequence=(Positive | None, None), + event_type=(str | None, None), + type=(str | None, None), + data=(Object | None, None), + result=(Object | None, None), + usage=(Object | None, None), + metadata=(Object | None, None), + source=(str | None, None), + runtime_id=(str | None, None), + claim_token=(str | None, None), + ), + JSONOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'run_finalize', + _input( + 'ExecutionFinalizeRunInput', + target_run_id=(Text | None, None), + status=(Text, ...), + reason=(str | None, None), + status_reason=(str | None, None), + usage=(Object | None, None), + cost=(Object | None, None), + metadata=(Object | None, None), + runtime_id=(str | None, None), + claim_token=(str | None, None), + ), + RunOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'runner_list', + _input('ExecutionRunnersInput', include_plugins=(list[str] | None, None), use_cache=(bool, True)), + ItemsOutput, + 'runner_admin', + ), + ExecutionOperation( + 'runtime_register', + _input( + 'ExecutionRegisterRuntimeInput', + **RuntimeFields, + display_name=(str | None, None), + endpoint=(str | None, None), + version=(str | None, None), + ), + RuntimeOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'runtime_heartbeat', + _input('ExecutionRuntimeHeartbeatInput', **RuntimeFields), + RuntimeOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'runtime_list', + _input( + 'ExecutionRuntimesInput', + statuses=(list[str] | None, None), + labels=(Object | None, None), + limit=(Positive, 50), + ), + PageOutput, + 'runner_admin', + ), + ExecutionOperation( + 'runtime_reconcile', + _input('ExecutionRuntimeReconcileInput', stale_after_seconds=(Positive, 60)), + JSONOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'run_stats', + _input( + 'ExecutionRunStatsInput', + runner_id=(str | None, None), + start_time=(int | float | None, None), + end_time=(int | float | None, None), + ), + JSONOutput, + 'runner_admin', + ), + ExecutionOperation('runtime_stats', Empty, JSONOutput, 'runner_admin'), + ExecutionOperation( + 'runner_stats', + _input( + 'ExecutionRunnerStatsInput', + start_time=(int | float | None, None), + end_time=(int | float | None, None), + limit=(Positive, 50), + ), + ItemsOutput, + 'runner_admin', + ), + ExecutionOperation( + 'run_claim', + _input( + 'ExecutionClaimRunInput', + runtime_id=(Text, ...), + lease_seconds=(Positive, 60), + queue_name=(str | None, None), + runner_ids=(list[str] | None, None), + ), + JSONOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'run_renew_claim', + _input('ExecutionRenewClaimInput', **ClaimFields, lease_seconds=(Positive, 60)), + RunOutput, + 'runner_admin', + True, + ), + ExecutionOperation( + 'run_release_claim', + _input( + 'ExecutionReleaseClaimInput', + **ClaimFields, + reason=(str | None, None), + status_reason=(str | None, None), + status=(Text, 'queued'), + ), + RunOutput, + 'runner_admin', + True, + ), +) +_BY_ID = {operation.operation_id: operation for operation in EXECUTION_OPERATIONS} + + +def _fail(code: str, message: str, status: int = 400) -> None: + raise ApplicationAPIException(ApplicationAPIError(code, message, status)) + + +class ExecutionAPI: + """Invoke Host APIs with trusted installation/run capabilities, without a transport.""" + + def __init__(self, ap: Any): + self.ap = ap + + def operations(self) -> tuple[ExecutionOperation, ...]: + return EXECUTION_OPERATIONS + + def catalog(self) -> list[dict[str, Any]]: + return [operation.as_dict() for operation in self.operations()] + + @contextlib.asynccontextmanager + async def _invocation(self, operation_id: str, context: ExecutionAPIContext, payload: dict, *, streaming: bool): + operation = _BY_ID.get(operation_id) + if operation is None: + _fail('unknown_operation', f'Unknown execution operation: {operation_id}', 404) + if operation.streaming != streaming: + _fail('invalid_input', 'Use stream for streaming operations and invoke for unary operations') + if not isinstance(context, ExecutionAPIContext) or not isinstance(context.binding, InstallationBinding): + _fail('invalid_authentication', 'A trusted installation binding is required', 401) + if context.run_id is not None and (not isinstance(context.run_id, str) or not context.run_id.strip()): + _fail('invalid_authentication', 'The trusted run ID is invalid', 401) + if context.run_id is None and operation.scope == 'run': + _fail('permission_denied', 'This operation requires a trusted run context', 403) + if context.run_id is not None and operation.scope == 'plugin': + _fail('permission_denied', 'A run cannot use legacy plugin-wide operations', 403) + try: + data = operation.input_type.model_validate(payload).model_dump(exclude_unset=True) + except ValidationError: + _fail('invalid_input', f'Input does not match {operation.input_type.__name__}') + if operation.action == 'call_platform_api' and context.run_id is None and not data.get('bot_uuid'): + _fail('invalid_input', 'bot_uuid is required outside a run') + if operation.action == 'call_tool': + if context.run_id is not None: + if 'tool_parameters' in data: + _fail('invalid_input', 'Run tool calls use parameters') + data.setdefault('parameters', {}) + elif 'parameters' in data: + if 'tool_parameters' in data: + _fail('invalid_input', 'Supply either parameters or tool_parameters') + data['tool_parameters'] = data.pop('parameters') + handler = getattr(getattr(self.ap, 'plugin_connector', None), 'handler', None) + if handler is None or operation.action not in handler.actions: + _fail('application_error', 'The execution Host does not provide this operation', 503) + try: + handler.validate_inbound_action_context(operation.action, context.binding) + except (TypeError, ValueError): + _fail('invalid_authentication', 'Invalid installation binding', 401) + token = handler._current_action_context.set(context.binding) + try: + async with handler._tenant_action_scope(context.binding): + try: + _, identity = await handler._require_plugin_action_context() + await handler._require_active_action_context(context.binding) + except (TypeError, ValueError): + _fail('permission_denied', 'Installation binding is invalid or stale', 403) + if context.run_id is not None: + if 'query_id' in data: + _fail('invalid_input', 'A run context supplies its own query identity') + from ...agent.runner.session_registry import get_session_registry + from ...plugin.agent_run_support import _load_persistent_agent_run_session + + session = await get_session_registry().get(context.run_id) + if session is None and operation.scope == 'runner_admin': + session = await _load_persistent_agent_run_session(context.run_id, self.ap, operation.action) + if ( + session is None + or session['authorization'].get('workspace_id') != context.binding.workspace_uuid + or session.get('plugin_identity') != f'{identity.plugin_author}/{identity.plugin_name}' + ): + _fail('permission_denied', 'Run does not belong to this installation Workspace', 403) + data['run_id'] = context.run_id + yield operation, handler.actions[operation.action], data + finally: + handler._current_action_context.reset(token) + + @staticmethod + def _result(operation: ExecutionOperation, response: Any) -> dict: + if not isinstance(response, ActionResponse): + _fail('invalid_output', 'Execution Host returned an invalid response', 500) + if response.code != 0: + # The Host exposes structured Box export failures as a public error contract. + from langbot_plugin.api.entities.builtin.runner.errors import AgentAPIError + + details = None + if operation.action == 'export_box_files' and 'error' in response.data: + try: + details = {'error': AgentAPIError.model_validate(response.data['error']).model_dump(mode='json')} + except ValidationError: + _fail('invalid_output', 'Invalid Box export error details', 500) + raise ApplicationAPIException(ApplicationAPIError('application_error', response.message, 400, details)) + try: + result = TypeAdapter(operation.output_type).validate_python(response.data, strict=True) + return result.model_dump(mode='json') + except ValidationError: + _fail('invalid_output', f'Output does not match {operation.action}', 500) + + async def invoke(self, operation_id: str, context: ExecutionAPIContext, payload: dict | None = None) -> dict: + try: + async with self._invocation( + operation_id, context, {} if payload is None else payload, streaming=False + ) as call: + operation, action, data = call + return self._result(operation, await action(data)) + except ApplicationAPIException: + raise + except Exception: + _fail('application_error', 'Execution operation failed', 500) + + async def stream(self, operation_id: str, context: ExecutionAPIContext, payload: dict | None = None): + async def produce(): + try: + async with self._invocation( + operation_id, context, {} if payload is None else payload, streaming=True + ) as call: + operation, action, data = call + async with contextlib.aclosing(action(data)) as responses: + async for response in responses: + yield self._result(operation, response) + except ApplicationAPIException: + raise + except Exception: + _fail('application_error', 'Execution stream failed', 500) + + async with contextlib.aclosing(isolated_stream(produce())) as responses: + async for response in responses: + yield response diff --git a/src/langbot/pkg/api/application/platform_setup.py b/src/langbot/pkg/api/application/platform_setup.py new file mode 100644 index 000000000..a7d77305f --- /dev/null +++ b/src/langbot/pkg/api/application/platform_setup.py @@ -0,0 +1,129 @@ +"""Application operations for principal-bound platform credential setup.""" + +from __future__ import annotations + +from typing import Any +from typing_extensions import NotRequired, TypedDict + +from pydantic import Field + +from ..authz import Permission +from ..http.service.slack_setup import SlackSetupError, get_slack_setup +from ...platform.credential_setup import CredentialSetupError, get_credential_setup +from .contracts import Input, Operation + + +class Empty(Input): + pass + + +class Session(Input): + session_id: str = Field(min_length=1) + + +class SlackStart(Input): + bot_uuid: str = Field(min_length=1) + access_token: str = Field(min_length=1) + refresh_token: str = Field(min_length=1) + webhook_url: str = '' + name: str = 'LangBot' + socket_mode: bool = False + redirect_url: str = '' + + +class SessionStarted(TypedDict): + session_id: str + + +class QRStarted(SessionStarted): + qr_url: str + expire_at: float | None + + +class WeixinStarted(SessionStarted): + qr_data_url: str + expire_at: float | None + + +class SessionStatus(TypedDict): + status: str + error: NotRequired[str | None] + app_id: NotRequired[str | None] + app_secret: NotRequired[str | None] + client_id: NotRequired[str | None] + client_secret: NotRequired[str | None] + botid: NotRequired[str | None] + secret: NotRequired[str | None] + appid: NotRequired[str | None] + user_openid: NotRequired[str | None] + qr_data_url: NotRequired[str | None] + expire_at: NotRequired[float | None] + token: NotRequired[str | None] + base_url: NotRequired[str | None] + account_id: NotRequired[str | None] + authorize_url: NotRequired[str | None] + config: NotRequired[dict[str, Any]] + + +def _credential_operation(method: str): + async def execute(ap, context, payload): + service = get_credential_setup(ap) + return await getattr(service, method)(request_context=context, **payload.model_dump()) + + return execute + + +async def slack_start(ap, context, payload: SlackStart): + try: + return await get_slack_setup(ap).start(context, **payload.model_dump()) + except SlackSetupError as exc: + raise CredentialSetupError(400, str(exc)) from exc + + +async def slack_status(ap, context, payload: Session): + try: + return get_slack_setup(ap).status(context, payload.session_id) + except SlackSetupError as exc: + raise CredentialSetupError(404, 'session_not_found') from exc + + +async def slack_cancel(ap, context, payload: Session): + service = get_slack_setup(ap) + try: + service.owned(context, payload.session_id) + except SlackSetupError as exc: + raise CredentialSetupError(404, 'session_not_found') from exc + service.discard(payload.session_id) + return {} + + +def operations() -> list[Operation]: + result = [] + for platform in ('lark', 'weixin', 'dingtalk', 'wecombot', 'qqofficial'): + for action in ('start', 'status', 'cancel'): + output = ( + (WeixinStarted if platform == 'weixin' else QRStarted) + if action == 'start' + else (SessionStatus if action == 'status' else dict) + ) + result.append( + Operation( + f'platform.setup.{platform}.{action}', + Permission.RESOURCE_MANAGE, + _credential_operation(f'{platform}_{action}'), + Empty if action == 'start' else Session, + output, + # Polling consumes successful credentials and completed error sessions. + side_effect=True, + ) + ) + result.extend( + ( + Operation( + 'platform.setup.slack.start', Permission.RESOURCE_MANAGE, slack_start, SlackStart, SessionStarted, True + ), + Operation('platform.setup.slack.status', Permission.RESOURCE_MANAGE, slack_status, Session, SessionStatus), + Operation('platform.setup.slack.cancel', Permission.RESOURCE_MANAGE, slack_cancel, Session, dict, True), + ) + ) + return result diff --git a/src/langbot/pkg/api/application/plugin_operations.py b/src/langbot/pkg/api/application/plugin_operations.py new file mode 100644 index 000000000..0777cbfb6 --- /dev/null +++ b/src/langbot/pkg/api/application/plugin_operations.py @@ -0,0 +1,521 @@ +"""Workspace plugin management operations independent of HTTP adapters.""" + +from __future__ import annotations + +import asyncio +import base64 +import os +import re +import uuid +import zipfile +from typing import Any +from typing_extensions import TypedDict +from urllib.parse import urlparse + +import httpx +import sqlalchemy +from langbot_plugin.runtime.plugin.mgr import PluginInstallSource +from pydantic import Field + +from ...core import taskmgr +from ...entity.persistence import plugin as persistence_plugin +from ...i18n import t +from ...plugin.archive import inspect_plugin_archive_metadata +from ...plugin.github import validate_github_plugin_install_info +from ...plugin.management import ( + count_plugin_components, + normalize_plugin_asset_path, + parse_github_repo_url, + redact_plugin_secrets, + restore_plugin_secret_placeholders, +) +from ...utils import httpclient +from ...workspace.errors import WorkspaceNotFoundError +from ..authz import Permission +from ..http.service.workspace_limits import workspace_limits +from .contracts import Input, Operation + + +class PluginsResult(TypedDict): + plugins: list[dict[str, Any]] + + +class PluginResult(TypedDict): + plugin: dict[str, Any] + + +class ConfigResult(TypedDict): + config: dict[str, Any] + + +class DebugResult(TypedDict): + debug_url: str + plugin_debug_key: str + expires_at: str + + +class TaskResult(TypedDict): + task_id: int + + +class BinaryResult(TypedDict): + content: bytes + mime_type: str + + +class ReadmeResult(TypedDict): + readme: str + + +class FileResult(TypedDict): + file_key: str + + +class DeletedResult(TypedDict): + deleted: bool + + +class PluginInput(Input): + author: str = Field(min_length=1) + plugin_name: str = Field(min_length=1) + + +class DeleteInput(PluginInput): + delete_data: bool = False + + +class ConfigInput(PluginInput): + config: dict[str, Any] + + +class ReadmeInput(PluginInput): + language: str = 'en' + + +class LogsInput(PluginInput): + limit: int = Field(default=200, ge=1) + level: str | None = None + + +class AssetInput(PluginInput): + filepath: str + + +class PageInput(PluginInput): + page_id: str + endpoint: str + method: str = 'POST' + body: Any = None + + +class GithubRepoInput(Input): + repo_url: str + + +class GithubAssetsInput(Input): + owner: str = Field(pattern=r'^[A-Za-z0-9][A-Za-z0-9-]{0,38}$') + repo: str = Field(pattern=r'^[A-Za-z0-9._-]{1,100}$') + release_id: int = Field(gt=0) + + +class GithubInstallInput(Input): + owner: str + repo: str + release_tag: str + asset_url: str | None = None + asset_id: int | None = None + release_id: int | None = None + + +class MarketplaceInput(Input): + plugin_author: str = Field(min_length=1) + plugin_name: str = Field(min_length=1) + plugin_version: str | None = None + + +class FileInput(Input): + content: bytes + filename: str = 'config.bin' + + +class LocalInstallInput(FileInput): + filename: str = 'local plugin' + administrator_force: bool = False + + +class FileKeyInput(Input): + file_key: str + + +async def _fence(ap, context): + return await ap.plugin_connector.require_workspace_context(context) + + +async def _get(ap, context, payload): + await _fence(ap, context) + plugin = await ap.plugin_connector.get_plugin_info(payload.author, payload.plugin_name) + if plugin is None: + raise WorkspaceNotFoundError('Plugin not found') + return plugin + + +async def _stored_config(ap, context, payload, plugin): + result = await ap.persistence_mgr.execute_async( + sqlalchemy.select(persistence_plugin.PluginSetting.config) + .where(persistence_plugin.PluginSetting.workspace_uuid == context.workspace_uuid) + .where(persistence_plugin.PluginSetting.plugin_author == payload.author) + .where(persistence_plugin.PluginSetting.plugin_name == payload.plugin_name) + ) + config = result.scalar_one_or_none() + return config if config is not None else plugin['plugin_config'] + + +async def list_plugins(ap, context, payload): + await _fence(ap, context) + return {'plugins': redact_plugin_secrets(await ap.plugin_connector.list_plugins())} + + +async def get_plugin(ap, context, payload): + return {'plugin': redact_plugin_secrets(await _get(ap, context, payload))} + + +async def debug_info(ap, context, payload): + execution = await _fence(ap, context) + info = await ap.plugin_connector.get_debug_info(execution) + url = ap.instance_config.data.get('plugin', {}).get( + 'display_plugin_debug_url', 'ws://localhost:5401/plugin/debug/ws' + ) + parsed = urlparse(url) + if parsed.scheme in {'http', 'https'}: + url = parsed._replace( + scheme='wss' if parsed.scheme == 'https' else 'ws', path=parsed.path or '/plugin/debug/ws' + ).geturl() + return { + 'debug_url': url, + 'plugin_debug_key': info.get('plugin_debug_key', ''), + 'expires_at': info.get('expires_at', ''), + } + + +async def get_config(ap, context, payload): + plugin = await _get(ap, context, payload) + return {'config': redact_plugin_secrets(await _stored_config(ap, context, payload, plugin))} + + +async def update_config(ap, context, payload): + plugin = await _get(ap, context, payload) + config = restore_plugin_secret_placeholders(payload.config, await _stored_config(ap, context, payload, plugin)) + await _fence(ap, context) + await ap.plugin_connector.set_plugin_config(payload.author, payload.plugin_name, config) + return {} + + +async def readme(ap, context, payload): + await _fence(ap, context) + return { + 'readme': await ap.plugin_connector.get_plugin_readme( + payload.author, payload.plugin_name, language=payload.language + ) + } + + +async def logs(ap, context, payload): + await _fence(ap, context) + return { + 'logs': await ap.plugin_connector.get_plugin_logs( + payload.author, payload.plugin_name, limit=payload.limit, level=payload.level + ) + } + + +async def icon(ap, context, payload): + await _fence(ap, context) + try: + result = await ap.plugin_connector.get_plugin_icon(payload.author, payload.plugin_name) + except ValueError as exc: + if 'is not installed in this workspace' in str(exc).lower(): + raise WorkspaceNotFoundError('Icon not found') from exc + raise + return { + 'content': await asyncio.to_thread(base64.b64decode, result['plugin_icon_base64']), + 'mime_type': result['mime_type'], + } + + +async def asset(ap, context, payload): + await _fence(ap, context) + path = normalize_plugin_asset_path(payload.filepath) + if path is None: + raise WorkspaceNotFoundError('Asset not found') + result = await ap.plugin_connector.get_plugin_assets(payload.author, payload.plugin_name, path) + if not result.get('asset_base64'): + raise WorkspaceNotFoundError('Asset not found') + return { + 'content': await asyncio.to_thread(base64.b64decode, result['asset_base64']), + 'mime_type': result['mime_type'], + } + + +async def page_action(ap, context, payload): + await _fence(ap, context) + if not payload.endpoint.startswith('/') or '..' in payload.endpoint: + raise ValueError('Invalid endpoint') + result = await ap.plugin_connector.handle_page_api( + payload.author, payload.plugin_name, payload.page_id, payload.endpoint, payload.method.upper(), payload.body + ) + if result.get('error'): + raise ValueError(result['error']) + return result.get('data') + + +async def _github_json(url): + async with httpx.AsyncClient( + trust_env=True, follow_redirects=True, timeout=10, event_hooks=httpclient.httpx_response_limit_hooks() + ) as client: + response = await client.get(url) + response.raise_for_status() + return await httpclient.parse_json_response(response) + + +async def github_releases(ap, context, payload): + await _fence(ap, context) + repo = parse_github_repo_url(payload.repo_url) + if not repo: + raise ValueError('Invalid GitHub repository URL') + owner, name, ref = repo['owner'], repo['repo'], repo['ref'] + # Repository parts become URL path segments, never a caller-selected host/path. + if ( + re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9-]{0,38}', owner) is None + or re.fullmatch(r'[A-Za-z0-9._-]{1,100}', name) is None + ): + raise ValueError('Invalid GitHub repository URL') + if ref: + releases = [ + { + 'id': 0, + 'tag_name': ref, + 'name': ref, + 'published_at': '', + 'prerelease': False, + 'draft': False, + 'source_type': 'branch', + 'archive_url': f'https://api.github.com/repos/{owner}/{name}/zipball/{ref}', + } + ] + else: + data = await _github_json(f'https://api.github.com/repos/{owner}/{name}/releases') + releases = [ + {key: release[key] for key in ('id', 'tag_name', 'name', 'published_at', 'prerelease', 'draft')} + for release in data + ] + return {'releases': releases, 'owner': owner, 'repo': name, 'source_subdir': repo['subdir']} + + +async def github_assets(ap, context, payload): + await _fence(ap, context) + release = await _github_json( + f'https://api.github.com/repos/{payload.owner}/{payload.repo}/releases/{payload.release_id}' + ) + return { + 'assets': [ + { + 'id': item['id'], + 'name': item['name'], + 'size': item['size'], + 'download_url': item['browser_download_url'], + 'content_type': item['content_type'], + } + for item in release.get('assets', []) + ] + } + + +async def _check_limit(ap, context): + await _fence(ap, context) + limits = await workspace_limits(ap, context.workspace_uuid) + limit = limits.get('max_extensions', -1) + if limit >= 0: + plugins = await ap.plugin_connector.list_plugins() + servers = await ap.mcp_service.get_mcp_servers(context) + if len(plugins) + len(servers) >= limit: + raise ValueError(t('Maximum number of {resource} ({limit}) reached', resource=t('extensions'), limit=limit)) + + +async def _run_fenced(ap, execution, operation): + async with ap.persistence_mgr.tenant_scope(execution.workspace_uuid): + await _fence(ap, execution) + return await operation() + + +async def _queue(ap, context, name, label, task, operation): + execution = await _fence(ap, context) + wrapper = ap.task_mgr.create_user_task( + _run_fenced(ap, execution, operation), + kind='plugin-operation', + name=name, + label=label, + context=task, + instance_uuid=context.instance_uuid, + workspace_uuid=context.workspace_uuid, + placement_generation=context.placement_generation, + ) + return {'task_id': wrapper.id} + + +async def upgrade(ap, context, payload): + task = taskmgr.TaskContext.new() + task.metadata.update( + { + 'plugin_name': f'{payload.author}/{payload.plugin_name}', + 'install_source': 'marketplace', + 'operation': 'upgrade', + 'progress_percent': 3, + } + ) + task.set_current_action('checking for latest version') + return await _queue( + ap, + context, + f'plugin-upgrade-{payload.plugin_name}', + f'Upgrading plugin {payload.plugin_name}', + task, + lambda: ap.plugin_connector.upgrade_plugin(payload.author, payload.plugin_name, task_context=task), + ) + + +async def delete(ap, context, payload): + task = taskmgr.TaskContext.new() + return await _queue( + ap, + context, + f'plugin-remove-{payload.plugin_name}', + f'Removing plugin {payload.plugin_name}', + task, + lambda: ap.plugin_connector.delete_plugin( + payload.author, payload.plugin_name, delete_data=payload.delete_data, task_context=task + ), + ) + + +async def _install(ap, context, source, data, identity, label): + task = taskmgr.TaskContext.new() + task.metadata.update({'plugin_name': identity, 'install_source': source.value}) + return await _queue( + ap, + context, + f'plugin-install-{source.value}', + label, + task, + lambda: ap.plugin_connector.install_plugin(source, data, task_context=task), + ) + + +async def install_github(ap, context, payload): + await _check_limit(ap, context) + data = validate_github_plugin_install_info(payload.model_dump()) + identity = f'{data["owner"]}/{data["repo"]}' + return await _install( + ap, + context, + PluginInstallSource.GITHUB, + data, + identity, + f'Installing plugin from GitHub {identity}@{data["release_tag"]}', + ) + + +async def install_marketplace(ap, context, payload): + await _check_limit(ap, context) + identity = f'{payload.plugin_author}/{payload.plugin_name}' + return await _install( + ap, + context, + PluginInstallSource.MARKETPLACE, + payload.model_dump(exclude_none=True), + identity, + f'Installing plugin from marketplace {identity}', + ) + + +async def install_local(ap, context, payload): + await _check_limit(ap, context) + return await _install( + ap, + context, + PluginInstallSource.LOCAL, + {'plugin_file': payload.content, 'administrator_force': payload.administrator_force}, + payload.filename, + f'Installing plugin from local {payload.filename}', + ) + + +async def preview_local(ap, context, payload): + await _fence(ap, context) + try: + manifest, requirements, names = await asyncio.to_thread(inspect_plugin_archive_metadata, payload.content) + except zipfile.BadZipFile as exc: + raise ValueError('Invalid .lbpkg file') from exc + counts = count_plugin_components((manifest.get('spec') or {}).get('components') or {}, names) + return { + 'filename': payload.filename, + 'size': len(payload.content), + 'manifest': manifest, + 'metadata': manifest.get('metadata') or {}, + 'component_types': list(counts), + 'component_counts': counts, + 'requirements': requirements, + 'file_count': len(names), + } + + +async def upload_config_file(ap, context, payload): + await _fence(ap, context) + if len(payload.content) > 10 * 1024 * 1024: + raise ValueError('File size exceeds 10MB limit') + _, ext = os.path.splitext(payload.filename) + key = await ap.storage_mgr.save_scoped( + context, + owner_type='plugin_config', + owner=context.workspace_uuid, + key=f'plugin_config_{uuid.uuid4().hex}{ext}', + value=payload.content, + ) + return {'file_key': key} + + +async def delete_config_file(ap, context, payload): + await _fence(ap, context) + if not ap.storage_mgr.is_scoped_object_key(payload.file_key, expected_owner_type='plugin_config'): + raise ValueError('Invalid file key') + await ap.storage_mgr.delete_scoped_object_key(context, payload.file_key, expected_owner_type='plugin_config') + return {'deleted': True} + + +def operations() -> list[Operation]: + view, manage = Permission.RESOURCE_VIEW, Permission.RESOURCE_MANAGE + return [ + Operation('plugin.list', view, list_plugins, Input, PluginsResult), + Operation('plugin.get', view, get_plugin, PluginInput, PluginResult), + Operation('plugin.debug_info', manage, debug_info, Input, DebugResult), + Operation('plugin.config.get', view, get_config, PluginInput, ConfigResult), + Operation('plugin.config.update', manage, update_config, ConfigInput, dict, side_effect=True), + Operation('plugin.readme', view, readme, ReadmeInput, ReadmeResult), + Operation('plugin.logs', Permission.AUDIT_VIEW, logs, LogsInput, dict), + Operation('plugin.icon', view, icon, PluginInput, BinaryResult), + Operation('plugin.asset', view, asset, AssetInput, BinaryResult), + Operation('plugin.page_action', manage, page_action, PageInput, Any, side_effect=True), + Operation('plugin.github.releases', view, github_releases, GithubRepoInput, dict), + Operation('plugin.github.assets', view, github_assets, GithubAssetsInput, dict), + Operation('plugin.upgrade', manage, upgrade, PluginInput, TaskResult, side_effect=True), + Operation('plugin.delete', manage, delete, DeleteInput, TaskResult, side_effect=True), + Operation('plugin.install.github', manage, install_github, GithubInstallInput, TaskResult, side_effect=True), + Operation( + 'plugin.install.marketplace', manage, install_marketplace, MarketplaceInput, TaskResult, side_effect=True + ), + Operation('plugin.install.local', manage, install_local, LocalInstallInput, TaskResult, side_effect=True), + Operation('plugin.install.local.preview', manage, preview_local, FileInput, dict), + Operation('plugin.config_file.upload', manage, upload_config_file, FileInput, FileResult, side_effect=True), + Operation( + 'plugin.config_file.delete', manage, delete_config_file, FileKeyInput, DeletedResult, side_effect=True + ), + ] diff --git a/src/langbot/pkg/api/application/service_operations.py b/src/langbot/pkg/api/application/service_operations.py new file mode 100644 index 000000000..d09147b9c --- /dev/null +++ b/src/langbot/pkg/api/application/service_operations.py @@ -0,0 +1,999 @@ +"""Explicit service-backed operations; schemas follow their business signatures.""" + +from __future__ import annotations + +import importlib +import inspect +import typing +from functools import lru_cache + +from pydantic import Field, create_model + +from ..authz import Permission, require_permission +from ..context import PrincipalType +from ...workspace.errors import WorkspaceNotFoundError +from .contracts import Input, Operation + +# Only these service methods are public. Adding a service method never exposes it. +_BINDINGS = ( + ( + 'monitoring.session.reset', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'reset_session_context', + 'RESOURCE_MANAGE', + True, + ), + ('pipeline.list', 'pipeline_service', 'pipeline', 'PipelineService', 'get_pipelines', 'RESOURCE_VIEW', False), + ('pipeline.get', 'pipeline_service', 'pipeline', 'PipelineService', 'get_pipeline', 'RESOURCE_VIEW', False), + ('pipeline.create', 'pipeline_service', 'pipeline', 'PipelineService', 'create_pipeline', 'RESOURCE_MANAGE', True), + ('pipeline.update', 'pipeline_service', 'pipeline', 'PipelineService', 'update_pipeline', 'RESOURCE_MANAGE', True), + ('pipeline.delete', 'pipeline_service', 'pipeline', 'PipelineService', 'delete_pipeline', 'RESOURCE_MANAGE', True), + ('pipeline.copy', 'pipeline_service', 'pipeline', 'PipelineService', 'copy_pipeline', 'RESOURCE_MANAGE', True), + ( + 'pipeline.metadata', + 'pipeline_service', + 'pipeline', + 'PipelineService', + 'get_pipeline_metadata', + 'RESOURCE_VIEW', + False, + ), + ( + 'pipeline.defaults', + 'pipeline_service', + 'pipeline', + 'PipelineService', + 'get_default_pipeline_config', + 'RESOURCE_VIEW', + False, + ), + ( + 'pipeline.extensions.get', + 'pipeline_service', + 'pipeline', + 'PipelineService', + 'get_pipeline_extensions', + 'RESOURCE_VIEW', + False, + ), + ( + 'pipeline.extensions.update', + 'pipeline_service', + 'pipeline', + 'PipelineService', + 'update_pipeline_extensions', + 'RESOURCE_MANAGE', + True, + ), + ('model.llm.list', 'llm_model_service', 'model', 'LLMModelsService', 'get_llm_models', 'RESOURCE_VIEW', False), + ( + 'model.llm.list_by_provider', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'get_llm_models_by_provider', + 'RESOURCE_VIEW', + False, + ), + ('model.llm.get', 'llm_model_service', 'model', 'LLMModelsService', 'get_llm_model', 'RESOURCE_VIEW', False), + ( + 'model.llm.create', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'create_llm_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.llm.update', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'update_llm_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ('model.llm.delete', 'llm_model_service', 'model', 'LLMModelsService', 'delete_llm_model', 'RESOURCE_MANAGE', True), + ( + 'model.llm.test', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'test_llm_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.llm.starred.get', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'get_starred_model', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.llm.starred.set', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'set_starred_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.llm.default.get', + 'llm_model_service', + 'model', + 'LLMModelsService', + 'get_default_model', + 'RESOURCE_MANAGE', + False, + ), + ( + 'model.embedding.list', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'get_embedding_models', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.embedding.list_by_provider', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'get_embedding_models_by_provider', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.embedding.get', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'get_embedding_model', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.embedding.create', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'create_embedding_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.embedding.update', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'update_embedding_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.embedding.delete', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'delete_embedding_model', + 'RESOURCE_MANAGE', + True, + ), + ( + 'model.embedding.test', + 'embedding_models_service', + 'model', + 'EmbeddingModelsService', + 'test_embedding_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.rerank.list', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'get_rerank_models', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.rerank.list_by_provider', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'get_rerank_models_by_provider', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.rerank.get', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'get_rerank_model', + 'RESOURCE_VIEW', + False, + ), + ( + 'model.rerank.create', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'create_rerank_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.rerank.update', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'update_rerank_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'model.rerank.delete', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'delete_rerank_model', + 'RESOURCE_MANAGE', + True, + ), + ( + 'model.rerank.test', + 'rerank_models_service', + 'model', + 'RerankModelsService', + 'test_rerank_model', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ('provider.list', 'provider_service', 'provider', 'ModelProviderService', 'get_providers', 'RESOURCE_VIEW', False), + ('provider.get', 'provider_service', 'provider', 'ModelProviderService', 'get_provider', 'RESOURCE_VIEW', False), + ( + 'provider.create', + 'provider_service', + 'provider', + 'ModelProviderService', + 'create_provider', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'provider.update', + 'provider_service', + 'provider', + 'ModelProviderService', + 'update_provider', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'provider.delete', + 'provider_service', + 'provider', + 'ModelProviderService', + 'delete_provider', + 'RESOURCE_MANAGE', + True, + ), + ( + 'provider.models.scan', + 'provider_service', + 'provider', + 'ModelProviderService', + 'scan_provider_models', + 'PROVIDER_SECRET_MANAGE', + True, + ), + ( + 'provider.models.count', + 'provider_service', + 'provider', + 'ModelProviderService', + 'get_provider_model_counts', + 'RESOURCE_VIEW', + False, + ), + ('agent.list', 'agent_service', 'agent', 'AgentService', 'get_agents', 'RESOURCE_VIEW', False), + ('agent.get', 'agent_service', 'agent', 'AgentService', 'get_agent', 'RESOURCE_VIEW', False), + ('agent.create', 'agent_service', 'agent', 'AgentService', 'create_agent', 'RESOURCE_MANAGE', True), + ('agent.update', 'agent_service', 'agent', 'AgentService', 'update_agent', 'RESOURCE_MANAGE', True), + ('agent.delete', 'agent_service', 'agent', 'AgentService', 'delete_agent', 'RESOURCE_MANAGE', True), + ('agent.metadata', 'agent_service', 'agent', 'AgentService', 'get_agent_metadata', 'RESOURCE_VIEW', False), + ('agent.debug', 'agent_service', 'agent', 'AgentService', 'debug_agent', 'RUNTIME_OPERATE', True), + ('agent.runs.list', 'agent_service', 'agent', 'AgentService', 'get_processor_runs', 'RESOURCE_VIEW', False), + ('agent.runs.events', 'agent_service', 'agent', 'AgentService', 'get_processor_run_events', 'RESOURCE_VIEW', False), + ('bot.list', 'bot_service', 'bot', 'BotService', 'get_bots', 'RESOURCE_VIEW', False), + ('bot.get', 'bot_service', 'bot', 'BotService', 'get_bot', 'RESOURCE_VIEW', False), + ('bot.runtime.get', 'bot_service', 'bot', 'BotService', 'get_runtime_bot_info', 'RESOURCE_VIEW', False), + ('bot.create', 'bot_service', 'bot', 'BotService', 'create_bot', 'RESOURCE_MANAGE', True), + ('bot.update', 'bot_service', 'bot', 'BotService', 'update_bot', 'RESOURCE_MANAGE', True), + ('bot.delete', 'bot_service', 'bot', 'BotService', 'delete_bot', 'RESOURCE_MANAGE', True), + ('bot.logs', 'bot_service', 'bot', 'BotService', 'list_event_logs', 'RESOURCE_VIEW', False), + ('bot.routes.status', 'bot_service', 'bot', 'BotService', 'list_event_route_statuses', 'RESOURCE_VIEW', False), + ( + 'bot.group_filter.preview', + 'bot_service', + 'bot', + 'BotService', + 'preview_group_filter_migration', + 'RESOURCE_MANAGE', + False, + ), + ( + 'bot.group_filter.apply', + 'bot_service', + 'bot', + 'BotService', + 'apply_group_filter_migration', + 'RESOURCE_MANAGE', + True, + ), + ('bot.message.send', 'bot_service', 'bot', 'BotService', 'send_message', 'RUNTIME_OPERATE', True), + ('bot.message.test', 'bot_service', 'bot', 'BotService', 'send_http_bot_test_message', 'RESOURCE_MANAGE', True), + ('bot.admins.list', 'bot_service', 'bot', 'BotService', 'get_bot_admins', 'RESOURCE_VIEW', False), + ('bot.admins.add', 'bot_service', 'bot', 'BotService', 'add_bot_admin', 'RESOURCE_MANAGE', True), + ('bot.admins.delete', 'bot_service', 'bot', 'BotService', 'delete_bot_admin', 'RESOURCE_MANAGE', True), + ( + 'knowledge.base.list', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'get_knowledge_bases', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.base.get', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'get_knowledge_base', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.base.create', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'create_knowledge_base', + 'RESOURCE_MANAGE', + True, + ), + ( + 'knowledge.base.update', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'update_knowledge_base', + 'RESOURCE_MANAGE', + True, + ), + ( + 'knowledge.base.delete', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'delete_knowledge_base', + 'RESOURCE_MANAGE', + True, + ), + ( + 'knowledge.base.retrieve', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'retrieve_knowledge_base', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.file.list', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'get_files_by_knowledge_base', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.file.store', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'store_file', + 'RESOURCE_MANAGE', + True, + ), + ( + 'knowledge.file.delete', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'delete_file', + 'RESOURCE_MANAGE', + True, + ), + ( + 'knowledge.engine.list', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'list_knowledge_engines', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.engine.creation_schema', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'get_engine_creation_schema', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.engine.retrieval_schema', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'get_engine_retrieval_schema', + 'RESOURCE_VIEW', + False, + ), + ( + 'knowledge.parser.list', + 'knowledge_service', + 'knowledge', + 'KnowledgeService', + 'list_parsers', + 'RESOURCE_VIEW', + False, + ), + ('mcp.server.list', 'mcp_service', 'mcp', 'MCPService', 'get_mcp_servers', 'RESOURCE_VIEW', False), + ('mcp.server.get', 'mcp_service', 'mcp', 'MCPService', 'get_mcp_server_by_name', 'RESOURCE_VIEW', False), + ('mcp.server.create', 'mcp_service', 'mcp', 'MCPService', 'create_mcp_server', 'RESOURCE_MANAGE', True), + ('mcp.server.update', 'mcp_service', 'mcp', 'MCPService', 'update_mcp_server', 'RESOURCE_MANAGE', True), + ('mcp.server.delete', 'mcp_service', 'mcp', 'MCPService', 'delete_mcp_server', 'RESOURCE_MANAGE', True), + ('mcp.server.test', 'mcp_service', 'mcp', 'MCPService', 'test_mcp_server', 'RESOURCE_MANAGE', True), + ('mcp.server.logs', 'mcp_service', 'mcp', 'MCPService', 'get_mcp_server_logs', 'AUDIT_VIEW', False), + ('mcp.server.runtime', 'mcp_service', 'mcp', 'MCPService', 'get_runtime_info', 'RESOURCE_VIEW', False), + ('mcp.resource.list', 'mcp_service', 'mcp', 'MCPService', 'get_mcp_server_resources', 'RESOURCE_VIEW', False), + ( + 'mcp.resource.templates', + 'mcp_service', + 'mcp', + 'MCPService', + 'get_mcp_server_resource_templates', + 'RESOURCE_VIEW', + False, + ), + ( + 'mcp.resource.read', + 'mcp_service', + 'mcp', + 'MCPService', + 'read_mcp_server_resource_envelope', + 'RESOURCE_VIEW', + False, + ), + ('skill.list', 'skill_service', 'skill', 'SkillService', 'list_skills', 'RESOURCE_VIEW', False), + ('skill.get', 'skill_service', 'skill', 'SkillService', 'get_skill', 'RESOURCE_VIEW', False), + ('skill.create', 'skill_service', 'skill', 'SkillService', 'create_skill', 'RESOURCE_MANAGE', True), + ('skill.update', 'skill_service', 'skill', 'SkillService', 'update_skill', 'RESOURCE_MANAGE', True), + ('skill.delete', 'skill_service', 'skill', 'SkillService', 'delete_skill', 'RESOURCE_MANAGE', True), + ('skill.file.list', 'skill_service', 'skill', 'SkillService', 'list_skill_files', 'RESOURCE_VIEW', False), + ('skill.file.read', 'skill_service', 'skill', 'SkillService', 'read_skill_file', 'RESOURCE_VIEW', False), + ('skill.file.write', 'skill_service', 'skill', 'SkillService', 'write_skill_file', 'RESOURCE_MANAGE', True), + ('skill.github.install', 'skill_service', 'skill', 'SkillService', 'install_from_github', 'RESOURCE_MANAGE', True), + ( + 'skill.github.preview', + 'skill_service', + 'skill', + 'SkillService', + 'preview_install_from_github', + 'RESOURCE_MANAGE', + False, + ), + ('skill.zip.install', 'skill_service', 'skill', 'SkillService', 'install_from_zip_upload', 'RESOURCE_MANAGE', True), + ( + 'skill.zip.preview', + 'skill_service', + 'skill', + 'SkillService', + 'preview_install_from_zip_upload', + 'RESOURCE_MANAGE', + False, + ), + ( + 'skill.directory.scan', + 'skill_service', + 'skill', + 'SkillService', + 'scan_directory_async', + 'RESOURCE_MANAGE', + False, + ), + ( + 'skill.directory.import', + 'skill_service', + 'skill', + 'SkillService', + 'import_skill_directory', + 'RESOURCE_MANAGE', + True, + ), + ('skill.reload', 'skill_service', 'skill', 'SkillService', 'reload_skills', 'RESOURCE_MANAGE', True), + ('api_key.list', 'apikey_service', 'apikey', 'ApiKeyService', 'get_api_keys', 'API_KEY_MANAGE', False), + ('api_key.get', 'apikey_service', 'apikey', 'ApiKeyService', 'get_api_key', 'API_KEY_MANAGE', False), + ('api_key.create', 'apikey_service', 'apikey', 'ApiKeyService', 'create_api_key', 'API_KEY_MANAGE', True), + ('api_key.update', 'apikey_service', 'apikey', 'ApiKeyService', 'update_api_key', 'API_KEY_MANAGE', True), + ('api_key.delete', 'apikey_service', 'apikey', 'ApiKeyService', 'delete_api_key', 'API_KEY_MANAGE', True), + ('webhook.list', 'webhook_service', 'webhook', 'WebhookService', 'get_webhooks', 'RESOURCE_VIEW', False), + ('webhook.get', 'webhook_service', 'webhook', 'WebhookService', 'get_webhook', 'RESOURCE_VIEW', False), + ('webhook.create', 'webhook_service', 'webhook', 'WebhookService', 'create_webhook', 'RESOURCE_MANAGE', True), + ('webhook.update', 'webhook_service', 'webhook', 'WebhookService', 'update_webhook', 'RESOURCE_MANAGE', True), + ('webhook.delete', 'webhook_service', 'webhook', 'WebhookService', 'delete_webhook', 'RESOURCE_MANAGE', True), + ( + 'monitoring.overview', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_overview_metrics', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.executions', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_executions', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.inflight', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_inflight_snapshot', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.tokens', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_token_statistics', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.messages', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_messages', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.llm_calls', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_llm_calls', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.tool_calls', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_tool_calls', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.embedding_calls', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_embedding_calls', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.sessions', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_sessions', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.errors', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_errors', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.session.analysis', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_session_analysis', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.message.detail', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_message_details', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.feedback.stats', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_feedback_stats', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.feedback.list', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'get_feedback_list', + 'RESOURCE_VIEW', + False, + ), + ( + 'monitoring.export.messages', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_messages', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.export.llm_calls', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_llm_calls', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.export.embedding_calls', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_embedding_calls', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.export.errors', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_errors', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.export.sessions', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_sessions', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.export.feedback', + 'monitoring_service', + 'monitoring', + 'MonitoringService', + 'export_feedback', + 'DATA_EXPORT', + False, + ), + ( + 'monitoring.execution.detail', + 'monitoring_service', + 'monitoring_execution_details', + 'ExecutionDetailsMixin', + 'get_execution_detail', + 'RESOURCE_VIEW', + False, + ), +) + +# These methods predate annotated service contracts. Specify their public types explicitly. +_FIELD_TYPES = { + 'processor_id': str, + 'run_id': str, + 'before_id': int | None, + 'after_sequence': int | None, + 'source': str, + 'execution_id': str, + 'section': str | None, + 'scopes': list[str] | None, +} +_OUTPUT_TYPES = { + 'get_processor_runs': dict, + 'get_processor_run_events': dict, + 'get_execution_detail': dict, + 'get_engine_creation_schema': dict | list[dict], + 'get_engine_retrieval_schema': dict | list[dict], +} +_INTERNAL_PARAMETERS = {'preserve_uuid', 'default', 'on_result', 'for_inflight'} + + +def _contract(operation_id, method): + signature = inspect.signature(method) + hints = typing.get_type_hints(method) + fields = {} + for name, parameter in signature.parameters.items(): + if name in {'self', 'context', 'ctx'} | _INTERNAL_PARAMETERS: + continue + annotation = _FIELD_TYPES.get(name, hints.get(name)) + default = parameter.default + if annotation is None: + if default is inspect.Parameter.empty or default is None: + raise TypeError(f'Missing application input type: {operation_id}.{name}') + annotation = type(default) + if default is None: + annotation = annotation | None + default = ... if default is inspect.Parameter.empty else default + constraints = {} + if name in {'limit', 'max_count', 'max_entries'}: + constraints = {'ge': 1, 'le': 100_000 if '.export.' in operation_id else 1000} + elif name in {'offset', 'from_index', 'after_sequence', 'before_id'}: + constraints = {'ge': 0} + elif annotation is str and (name.endswith(('_uuid', '_id')) or name in {'query', 'uri'}): + constraints = {'min_length': 1} + fields[name] = (annotation, Field(default=default, **constraints)) + model_name = ''.join(part.title() for part in operation_id.split('.')) + 'Input' + model = create_model(model_name, __base__=Input, **fields) + output = _OUTPUT_TYPES.get(method.__name__, hints.get('return', inspect.Parameter.empty)) + if output is inspect.Parameter.empty: + raise TypeError(f'Missing application output type: {operation_id}') + return model, output + + +def _operation(operation_id, service_name, method, permission, side_effect): + input_type, output_type = _contract(operation_id, method) + + async def invoke(ap, context, payload): + arguments = payload.model_dump() + if arguments.get('include_secret'): + secret_permission = ( + Permission.PROVIDER_SECRET_MANAGE + if operation_id.startswith(('model.', 'provider.')) + else Permission.RESOURCE_MANAGE + ) + require_permission(context, secret_permission) + service = getattr(ap, service_name) + result = await getattr(service, method.__name__)(context, **arguments) + if result is None and operation_id.endswith('.get') and output_type != type(None): + # Optional defaults (e.g. starred model) are intentionally nullable. + if operation_id not in {'model.llm.starred.get', 'model.llm.default.get'}: + raise WorkspaceNotFoundError('Resource not found') + return result + + return Operation(operation_id, Permission[permission], invoke, input_type, output_type, side_effect) + + +@lru_cache(maxsize=1) +def operations() -> tuple[Operation, ...]: + result = [] + for operation_id, service_name, module, cls, method_name, permission, side_effect in _BINDINGS: + service_module = importlib.import_module('langbot.pkg.api.http.service.' + module) + method = getattr(getattr(service_module, cls), method_name) + result.append(_operation(operation_id, service_name, method, permission, side_effect)) + result.extend(_special_operations()) + return tuple(result) + + +class PipelineRunInput(Input): + pipeline_uuid: str = Field(min_length=1) + text: str = Field(min_length=1, max_length=100_000) + timeout: float = Field(default=60.0, gt=0, le=300) + + +async def _pipeline_run(ap, context, payload): + from ..http.service.pipeline_run import run_pipeline + + return await run_pipeline(ap, context, **payload.model_dump()) + + +class CodexProviderInput(Input): + provider: str = Field(min_length=1) + + +class CodexAttemptInput(CodexProviderInput): + authorization_id: str = Field(min_length=1) + + +class ConversationInput(Input): + conversation_id: str = Field(min_length=1) + + +class StopInput(ConversationInput): + revision: int = Field(ge=0) + + +class TurnInput(StopInput): + text: str | None = Field(default=None, min_length=1, max_length=8000) + approved: bool | None = None + model_uuid: str | None = None + + +def _assistant_service(ap): + from ..http.service.assistant import AssistantService + + service = getattr(ap, '_application_assistant_service', None) + if service is None: + service = ap._application_assistant_service = AssistantService(ap) + return service + + +def _validate_turn(payload): + if (payload.text is None) == (payload.approved is None) or (payload.text is not None and not payload.text.strip()): + raise ValueError('Provide text or an approval decision') + + +async def _assistant(ap, context, payload, method): + service = _assistant_service(ap) + arguments = payload.model_dump() + if method == 'turn': + _validate_turn(payload) + result = await getattr(service, method)(context, **arguments) + return service.public_view(result) if method in {'create', 'turn'} else result + + +class RoutePreviewInput(Input): + bot_uuid: str = Field(min_length=1) + event_type: str = Field(min_length=1) + event_data: dict | None = None + event_context: dict | None = None + event_bindings: list[dict] | None = None + + +async def _route_preview(ap, context, payload): + return await ap.bot_service.dry_run_event_route( + context, + payload.bot_uuid, + payload.event_type, + event_data=payload.event_data, + context=payload.event_context, + event_bindings=payload.event_bindings, + ) + + +class MigrationInput(Input): + body: dict + + +def _migration_service(ap): + from ..http.service.pipeline_migration import PipelineMigrationService + + service = getattr(ap, '_application_pipeline_migration_service', None) + if service is None: + service = ap._application_pipeline_migration_service = PipelineMigrationService(ap) + return service + + +async def _migration_preview(ap, context, payload): + return await _migration_service(ap).preview(context) + + +async def _migration_execute(ap, context, payload): + return await _migration_service(ap).execute(context, payload.body) + + +async def _assistant_start(ap, context, payload): + _validate_turn(payload) + service = _assistant_service(ap) + await service.get(context, payload.conversation_id) + service.start_run(context, payload.conversation_id, payload) + return {'conversation_id': payload.conversation_id, 'revision': payload.revision + 1} + + +def _special_operations(): + account = frozenset({PrincipalType.ACCOUNT}) + result = [ + Operation('pipeline.run', Permission.RUNTIME_OPERATE, _pipeline_run, PipelineRunInput, dict, True), + Operation('bot.routes.preview', Permission.RESOURCE_MANAGE, _route_preview, RoutePreviewInput, dict), + Operation( + 'pipeline.migration.preview', Permission.RESOURCE_VIEW, _migration_preview, Input, dict, principals=account + ), + Operation( + 'pipeline.migration.execute', + Permission.RESOURCE_MANAGE, + _migration_execute, + MigrationInput, + dict, + True, + principals=account, + ), + Operation( + 'assistant.run.start', + Permission.RUNTIME_OPERATE, + _assistant_start, + TurnInput, + dict, + True, + principals=account, + ), + ] + for method in ('status', 'start', 'poll', 'disconnect', 'cancel'): + + async def invoke(ap, context, payload, method=method): + return await getattr(ap.provider_service.codex_auth, method)(context, **payload.model_dump()) + + result.append( + Operation( + 'provider.codex.' + method, + Permission.PROVIDER_SECRET_MANAGE, + invoke, + CodexAttemptInput if method in {'poll', 'cancel'} else CodexProviderInput, + type(None) if method in {'disconnect', 'cancel'} else dict, + method != 'status', + principals=frozenset({PrincipalType.ACCOUNT}), + ) + ) + for name, method, model, output, permission, write in ( + ('list', 'list_conversations', Input, list[dict], Permission.RESOURCE_VIEW, False), + ('get', 'progress', ConversationInput, dict, Permission.RESOURCE_VIEW, False), + ('create', 'create', Input, dict, Permission.RUNTIME_OPERATE, True), + ('stop', 'stop', StopInput, dict, Permission.RUNTIME_OPERATE, True), + ('turn', 'turn', TurnInput, dict, Permission.RUNTIME_OPERATE, True), + ('recommended_model', 'recommended_model', Input, dict, Permission.RUNTIME_OPERATE, False), + ): + + async def invoke(ap, context, payload, method=method): + return await _assistant(ap, context, payload, method) + + result.append( + Operation( + 'assistant.' + name, + permission, + invoke, + model, + output, + write, + principals=frozenset({PrincipalType.ACCOUNT}), + ) + ) + return result diff --git a/src/langbot/pkg/api/application/streaming.py b/src/langbot/pkg/api/application/streaming.py new file mode 100644 index 000000000..aef21c88b --- /dev/null +++ b/src/langbot/pkg/api/application/streaming.py @@ -0,0 +1,49 @@ +"""Bounded streaming that keeps producer authorization scopes out of the consumer.""" + +from __future__ import annotations + +import asyncio +from collections.abc import AsyncIterator +from contextlib import aclosing, suppress +from typing import TypeVar + +T = TypeVar('T') + + +async def isolated_stream(source: AsyncIterator[T]) -> AsyncIterator[T]: + queue: asyncio.Queue[tuple[bool, T | Exception | None]] = asyncio.Queue(maxsize=1) + + async def produce(): + try: + async with aclosing(source): + async for item in source: + await queue.put((True, item)) + except Exception as exc: + await queue.put((False, exc)) + else: + await queue.put((False, None)) + + producer = asyncio.create_task(produce()) + try: + while True: + receive = asyncio.create_task(queue.get()) + try: + done, _ = await asyncio.wait({receive, producer}, return_when=asyncio.FIRST_COMPLETED) + if receive not in done and queue.empty(): + # Cancellation inside a provider must also wake the consumer. + await producer + available, value = await receive + finally: + receive.cancel() + with suppress(asyncio.CancelledError): + await receive + if available: + yield value + elif value is None: + return + else: + raise value + finally: + producer.cancel() + with suppress(asyncio.CancelledError): + await producer diff --git a/src/langbot/pkg/api/application/workspace_streams.py b/src/langbot/pkg/api/application/workspace_streams.py new file mode 100644 index 000000000..cd8fc40c8 --- /dev/null +++ b/src/langbot/pkg/api/application/workspace_streams.py @@ -0,0 +1,105 @@ +"""Business streams with the lifetime rules of Agent debug and Assistant runs.""" + +from __future__ import annotations + +import asyncio +from contextlib import suppress + +from pydantic import Field + +from .contracts import Input, Operation +from .service_operations import TurnInput +from ..authz import Permission +from ..context import PrincipalType + + +class AgentDebugInput(Input): + agent_uuid: str = Field(min_length=1) + payload: dict + + +async def agent_debug(ap, context, payload): + async def frames(): + queue: asyncio.Queue[dict | Exception] = asyncio.Queue(maxsize=32) + + async def on_result(result): + await queue.put({'kind': 'result', 'data': result}) + + async def execute(): + try: + # The callback producer is a different task and needs its own DB scope. + async with ap.persistence_mgr.tenant_scope(context.workspace_uuid): + result = await ap.agent_service.debug_agent( + context, payload.agent_uuid, payload.payload, on_result=on_result + ) + await queue.put({'kind': 'completed', 'data': result}) + except Exception as exc: + await queue.put(exc) + + task = asyncio.create_task(execute()) + try: + while True: + receive = asyncio.create_task(queue.get()) + try: + done, _ = await asyncio.wait({receive, task}, return_when=asyncio.FIRST_COMPLETED) + if receive not in done and queue.empty(): + # A provider cancellation has no queued terminal frame. + await task + frame = await receive + finally: + receive.cancel() + with suppress(asyncio.CancelledError): + await receive + if isinstance(frame, Exception): + raise frame + yield frame + if frame['kind'] == 'completed': + return + finally: + task.cancel() + await asyncio.gather(task, return_exceptions=True) + + return frames() + + +async def assistant_turn(ap, context, payload): + from .service_operations import _assistant_service, _validate_turn + + _validate_turn(payload) + service = _assistant_service(ap) + await service.get(context, payload.conversation_id) + queue = service.start_run(context, payload.conversation_id, payload) + + async def frames(): + # start_run owns a bounded queue and the run survives a subscriber disconnect. + while True: + frame = await queue.get() + yield frame + if frame['kind'] in {'completed', 'error'}: + return + + return frames() + + +def operations() -> list[Operation]: + return [ + Operation( + 'agent.debug.stream', + Permission.RUNTIME_OPERATE, + agent_debug, + AgentDebugInput, + dict, + side_effect=True, + streaming=True, + ), + Operation( + 'assistant.stream', + Permission.RUNTIME_OPERATE, + assistant_turn, + TurnInput, + dict, + side_effect=True, + principals=frozenset({PrincipalType.ACCOUNT}), + streaming=True, + ), + ] diff --git a/src/langbot/pkg/api/authz.py b/src/langbot/pkg/api/authz.py new file mode 100644 index 000000000..336ceecbd --- /dev/null +++ b/src/langbot/pkg/api/authz.py @@ -0,0 +1,133 @@ +from __future__ import annotations + +import enum +import types +import typing + +from .context import RequestContext + + +class WorkspaceRole(enum.StrEnum): + OWNER = 'owner' + ADMIN = 'admin' + DEVELOPER = 'developer' + OPERATOR = 'operator' + VIEWER = 'viewer' + + +class Permission(enum.StrEnum): + WORKSPACE_VIEW = 'workspace.view' + WORKSPACE_UPDATE = 'workspace.update' + WORKSPACE_DELETE = 'workspace.delete' + MEMBER_VIEW = 'member.view' + MEMBER_INVITE = 'member.invite' + MEMBER_UPDATE_ROLE = 'member.update_role' + MEMBER_REMOVE = 'member.remove' + RESOURCE_VIEW = 'resource.view' + RESOURCE_MANAGE = 'resource.manage' + RUNTIME_OPERATE = 'runtime.operate' + PROVIDER_SECRET_MANAGE = 'provider_secret.manage' + API_KEY_MANAGE = 'api_key.manage' + AUDIT_VIEW = 'audit.view' + DATA_EXPORT = 'data.export' + BILLING_LINK_MANAGE = 'billing_link.manage' + + +_VIEW_PERMISSIONS = { + Permission.WORKSPACE_VIEW, + Permission.MEMBER_VIEW, + Permission.RESOURCE_VIEW, +} + +_ROLE_PERMISSIONS: typing.Final = types.MappingProxyType( + { + WorkspaceRole.OWNER: frozenset(Permission), + WorkspaceRole.ADMIN: frozenset( + permission + for permission in Permission + if permission + not in { + Permission.WORKSPACE_DELETE, + Permission.BILLING_LINK_MANAGE, + } + ), + WorkspaceRole.DEVELOPER: frozenset( + _VIEW_PERMISSIONS + | { + Permission.RESOURCE_MANAGE, + Permission.RUNTIME_OPERATE, + Permission.PROVIDER_SECRET_MANAGE, + } + ), + WorkspaceRole.OPERATOR: frozenset(_VIEW_PERMISSIONS | {Permission.RUNTIME_OPERATE}), + WorkspaceRole.VIEWER: frozenset(_VIEW_PERMISSIONS), + } +) + + +class AuthorizationError(Exception): + """Base class for errors that map to an HTTP authorization response.""" + + status_code = 403 + error_code = 'forbidden' + + +class AuthenticationDeniedError(AuthorizationError): + status_code = 401 + error_code = 'invalid_authentication' + + +class WorkspaceRequiredError(AuthorizationError): + status_code = 400 + error_code = 'workspace_required' + + +class PermissionDeniedError(AuthorizationError): + error_code = 'permission_denied' + + def __init__(self, permission: str) -> None: + super().__init__(f'Missing Workspace permission: {permission}') + self.permission = permission + + +class EditionLimitError(AuthorizationError): + error_code = 'edition_limit' + + +def permissions_for_role(role: str | WorkspaceRole) -> frozenset[str]: + """Return the canonical fixed permissions for a Workspace role.""" + + try: + parsed_role = WorkspaceRole(role) + except ValueError: + return frozenset() + return frozenset(permission.value for permission in _ROLE_PERMISSIONS[parsed_role]) + + +def has_permission(ctx: RequestContext, permission: str | Permission) -> bool: + """Return whether the context contains one effective permission.""" + + permission_value = permission.value if isinstance(permission, Permission) else permission + return permission_value in ctx.workspace.permissions + + +def require_permission(ctx: RequestContext, permission: str | Permission) -> None: + """Raise a stable authorization error when a permission is missing.""" + + permission_value = permission.value if isinstance(permission, Permission) else permission + if not has_permission(ctx, permission_value): + raise PermissionDeniedError(permission_value) + + +__all__ = [ + 'AuthenticationDeniedError', + 'AuthorizationError', + 'EditionLimitError', + 'Permission', + 'PermissionDeniedError', + 'WorkspaceRequiredError', + 'WorkspaceRole', + 'has_permission', + 'permissions_for_role', + 'require_permission', +] diff --git a/src/langbot/pkg/api/context.py b/src/langbot/pkg/api/context.py new file mode 100644 index 000000000..254e6ba97 --- /dev/null +++ b/src/langbot/pkg/api/context.py @@ -0,0 +1,113 @@ +"""Transport-neutral identity and Workspace execution context types. + +The application operation layer uses these immutable values as its trusted +authorization boundary. HTTP, MCP and other protocol adapters may construct +or carry them, but the types themselves do not depend on any transport. +""" + +from __future__ import annotations + +import dataclasses +import enum + + +class PrincipalType(enum.StrEnum): + """Kinds of authenticated principals accepted by LangBot.""" + + ACCOUNT = 'account' + API_KEY = 'api_key' + SUPPORT_ADMIN = 'support_admin' + SYSTEM = 'system' + PUBLIC_BOT = 'public_bot' + + +@dataclasses.dataclass(frozen=True, slots=True) +class PrincipalContext: + """Authenticated identity before Workspace authorization is applied.""" + + principal_type: PrincipalType + account_uuid: str | None = None + actor_account_uuid: str | None = None + api_key_uuid: str | None = None + support_session_id: str | None = None + + +@dataclasses.dataclass(frozen=True, slots=True) +class WorkspaceContext: + """Workspace membership and effective permissions for one request.""" + + workspace_uuid: str + membership_uuid: str | None + role: str | None + permissions: frozenset[str] + membership_revision: int = 0 + + +@dataclasses.dataclass(frozen=True, slots=True) +class RequestContext: + """Trusted authorization context passed to application operations.""" + + instance_uuid: str + placement_generation: int + request_id: str + auth_type: str + principal: PrincipalContext + workspace: WorkspaceContext + entitlement_revision: int = 0 + + @property + def workspace_uuid(self) -> str: + """Return the selected Workspace UUID.""" + + return self.workspace.workspace_uuid + + @property + def account_uuid(self) -> str | None: + """Return the Account UUID when the principal is an Account.""" + + return self.principal.account_uuid + + +@dataclasses.dataclass(frozen=True, slots=True) +class ExecutionContext: + """Workspace context propagated to asynchronous and runtime work.""" + + instance_uuid: str + workspace_uuid: str + placement_generation: int + bot_uuid: str | None = None + pipeline_uuid: str | None = None + query_uuid: str | None = None + trigger_principal: PrincipalContext | None = None + entitlement_revision: int = 0 + + @classmethod + def from_request( + cls, + ctx: RequestContext, + *, + bot_uuid: str | None = None, + pipeline_uuid: str | None = None, + query_uuid: str | None = None, + ) -> ExecutionContext: + """Create a runtime context without losing the tenant generation.""" + + return cls( + instance_uuid=ctx.instance_uuid, + workspace_uuid=ctx.workspace_uuid, + placement_generation=ctx.placement_generation, + bot_uuid=bot_uuid, + pipeline_uuid=pipeline_uuid, + query_uuid=query_uuid, + trigger_principal=ctx.principal, + entitlement_revision=ctx.entitlement_revision, + ) + + +__all__ = [ + 'ExecutionContext', + 'PrincipalContext', + 'PrincipalType', + 'RequestContext', + 'WorkspaceContext', +] diff --git a/src/langbot/pkg/api/http/authz.py b/src/langbot/pkg/api/http/authz.py index 7bfc8dbe3..43f17d3b2 100644 --- a/src/langbot/pkg/api/http/authz.py +++ b/src/langbot/pkg/api/http/authz.py @@ -1,119 +1,9 @@ -from __future__ import annotations +"""HTTP compatibility exports for transport-neutral API authorization. -import enum -import types -import typing +Authorization primitives live in :mod:`langbot.pkg.api.authz` so application +operations can be used by HTTP, MCP, assistant and CLI adapters without +importing a transport package. Keep this module as a compatibility re-export +for existing HTTP service/controller imports. +""" -from .context import RequestContext - - -class WorkspaceRole(enum.StrEnum): - OWNER = 'owner' - ADMIN = 'admin' - DEVELOPER = 'developer' - OPERATOR = 'operator' - VIEWER = 'viewer' - - -class Permission(enum.StrEnum): - WORKSPACE_VIEW = 'workspace.view' - WORKSPACE_UPDATE = 'workspace.update' - WORKSPACE_DELETE = 'workspace.delete' - MEMBER_VIEW = 'member.view' - MEMBER_INVITE = 'member.invite' - MEMBER_UPDATE_ROLE = 'member.update_role' - MEMBER_REMOVE = 'member.remove' - RESOURCE_VIEW = 'resource.view' - RESOURCE_MANAGE = 'resource.manage' - RUNTIME_OPERATE = 'runtime.operate' - PROVIDER_SECRET_MANAGE = 'provider_secret.manage' - API_KEY_MANAGE = 'api_key.manage' - AUDIT_VIEW = 'audit.view' - DATA_EXPORT = 'data.export' - BILLING_LINK_MANAGE = 'billing_link.manage' - - -_VIEW_PERMISSIONS = { - Permission.WORKSPACE_VIEW, - Permission.MEMBER_VIEW, - Permission.RESOURCE_VIEW, -} - -_ROLE_PERMISSIONS: typing.Final = types.MappingProxyType( - { - WorkspaceRole.OWNER: frozenset(Permission), - WorkspaceRole.ADMIN: frozenset( - permission - for permission in Permission - if permission - not in { - Permission.WORKSPACE_DELETE, - Permission.BILLING_LINK_MANAGE, - } - ), - WorkspaceRole.DEVELOPER: frozenset( - _VIEW_PERMISSIONS - | { - Permission.RESOURCE_MANAGE, - Permission.RUNTIME_OPERATE, - Permission.PROVIDER_SECRET_MANAGE, - } - ), - WorkspaceRole.OPERATOR: frozenset(_VIEW_PERMISSIONS | {Permission.RUNTIME_OPERATE}), - WorkspaceRole.VIEWER: frozenset(_VIEW_PERMISSIONS), - } -) - - -class AuthorizationError(Exception): - """Base class for errors that map to an HTTP authorization response.""" - - status_code = 403 - error_code = 'forbidden' - - -class AuthenticationDeniedError(AuthorizationError): - status_code = 401 - error_code = 'invalid_authentication' - - -class WorkspaceRequiredError(AuthorizationError): - status_code = 400 - error_code = 'workspace_required' - - -class PermissionDeniedError(AuthorizationError): - error_code = 'permission_denied' - - def __init__(self, permission: str) -> None: - super().__init__(f'Missing Workspace permission: {permission}') - self.permission = permission - - -class EditionLimitError(AuthorizationError): - error_code = 'edition_limit' - - -def permissions_for_role(role: str | WorkspaceRole) -> frozenset[str]: - """Return the canonical fixed permissions for a Workspace role.""" - - try: - parsed_role = WorkspaceRole(role) - except ValueError: - return frozenset() - return frozenset(permission.value for permission in _ROLE_PERMISSIONS[parsed_role]) - - -def has_permission(ctx: RequestContext, permission: str | Permission) -> bool: - """Return whether the context contains one effective permission.""" - - permission_value = permission.value if isinstance(permission, Permission) else permission - return permission_value in ctx.workspace.permissions - - -def require_permission(ctx: RequestContext, permission: str | Permission) -> None: - """Raise a stable authorization error when a permission is missing.""" - - permission_value = permission.value if isinstance(permission, Permission) else permission - if not has_permission(ctx, permission_value): - raise PermissionDeniedError(permission_value) +from ..authz import * # noqa: F401,F403 diff --git a/src/langbot/pkg/api/http/context.py b/src/langbot/pkg/api/http/context.py index f9c7b4258..6994483b3 100644 --- a/src/langbot/pkg/api/http/context.py +++ b/src/langbot/pkg/api/http/context.py @@ -1,97 +1,17 @@ -from __future__ import annotations +"""Backward-compatible HTTP import surface for transport-neutral contexts.""" -import dataclasses -import enum +from ..context import ( + ExecutionContext, + PrincipalContext, + PrincipalType, + RequestContext, + WorkspaceContext, +) - -class PrincipalType(enum.StrEnum): - """Kinds of authenticated principals accepted by LangBot.""" - - ACCOUNT = 'account' - API_KEY = 'api_key' - SUPPORT_ADMIN = 'support_admin' - SYSTEM = 'system' - PUBLIC_BOT = 'public_bot' - - -@dataclasses.dataclass(frozen=True, slots=True) -class PrincipalContext: - """Authenticated identity before Workspace authorization is applied.""" - - principal_type: PrincipalType - account_uuid: str | None = None - actor_account_uuid: str | None = None - api_key_uuid: str | None = None - support_session_id: str | None = None - - -@dataclasses.dataclass(frozen=True, slots=True) -class WorkspaceContext: - """Workspace membership and effective permissions for one request.""" - - workspace_uuid: str - membership_uuid: str | None - role: str | None - permissions: frozenset[str] - membership_revision: int = 0 - - -@dataclasses.dataclass(frozen=True, slots=True) -class RequestContext: - """Trusted authorization context passed to HTTP services.""" - - instance_uuid: str - placement_generation: int - request_id: str - auth_type: str - principal: PrincipalContext - workspace: WorkspaceContext - entitlement_revision: int = 0 - - @property - def workspace_uuid(self) -> str: - """Return the selected Workspace UUID.""" - - return self.workspace.workspace_uuid - - @property - def account_uuid(self) -> str | None: - """Return the Account UUID when the principal is an Account.""" - - return self.principal.account_uuid - - -@dataclasses.dataclass(frozen=True, slots=True) -class ExecutionContext: - """Workspace context propagated to asynchronous and runtime work.""" - - instance_uuid: str - workspace_uuid: str - placement_generation: int - bot_uuid: str | None = None - pipeline_uuid: str | None = None - query_uuid: str | None = None - trigger_principal: PrincipalContext | None = None - entitlement_revision: int = 0 - - @classmethod - def from_request( - cls, - ctx: RequestContext, - *, - bot_uuid: str | None = None, - pipeline_uuid: str | None = None, - query_uuid: str | None = None, - ) -> ExecutionContext: - """Create a runtime context without losing the tenant generation.""" - - return cls( - instance_uuid=ctx.instance_uuid, - workspace_uuid=ctx.workspace_uuid, - placement_generation=ctx.placement_generation, - bot_uuid=bot_uuid, - pipeline_uuid=pipeline_uuid, - query_uuid=query_uuid, - trigger_principal=ctx.principal, - entitlement_revision=ctx.entitlement_revision, - ) +__all__ = [ + 'ExecutionContext', + 'PrincipalContext', + 'PrincipalType', + 'RequestContext', + 'WorkspaceContext', +] diff --git a/src/langbot/pkg/api/http/controller/groups/platform/adapters.py b/src/langbot/pkg/api/http/controller/groups/platform/adapters.py index 6b7f41d16..590d82dbb 100644 --- a/src/langbot/pkg/api/http/controller/groups/platform/adapters.py +++ b/src/langbot/pkg/api/http/controller/groups/platform/adapters.py @@ -1,7 +1,5 @@ from langbot.pkg.i18n import t, DEFAULT_LANGUAGE from langbot.pkg.i18n.workspace import get_workspace_language -import asyncio -import dataclasses import mimetypes import quart @@ -9,186 +7,23 @@ import quart from langbot.pkg.api.http.authz import Permission from langbot.pkg.api.http.context import RequestContext from langbot.pkg.cloud.plan_policy import workspace_entitlement, platform_feature_names, require_workspace_platform -from langbot.pkg.core.errors import TaskCapacityError -from langbot.pkg.utils import httpclient, importutil +from langbot.pkg.utils import importutil from ... import group - -@dataclasses.dataclass(frozen=True, slots=True) -class _AdapterSessionScope: - """Immutable tenant and principal binding for a credential exchange.""" - - instance_uuid: str - workspace_uuid: str - placement_generation: int - principal_type: str - account_uuid: str | None - api_key_uuid: str | None - support_session_id: str | None - - @classmethod - def from_request_context(cls, request_context: RequestContext) -> '_AdapterSessionScope': - principal = request_context.principal - return cls( - instance_uuid=request_context.instance_uuid, - workspace_uuid=request_context.workspace_uuid, - placement_generation=request_context.placement_generation, - principal_type=principal.principal_type.value, - account_uuid=principal.account_uuid, - api_key_uuid=principal.api_key_uuid, - support_session_id=principal.support_session_id, - ) - - def matches(self, request_context: RequestContext) -> bool: - """Return whether a request is from the exact initiating tenant principal.""" - - return self == self.from_request_context(request_context) - - -def _bind_session_scope(session: dict, request_context: RequestContext) -> None: - session['scope'] = _AdapterSessionScope.from_request_context(request_context) - - -def _get_owned_session( - sessions: dict[str, dict], - session_id: str, - request_context: RequestContext, -) -> dict | None: - """Resolve a session without revealing sessions owned by another scope.""" - - session = sessions.get(session_id) - scope = session.get('scope') if session is not None else None - if not isinstance(scope, _AdapterSessionScope) or not scope.matches(request_context): - return None - return session - - -def _pop_owned_session( - sessions: dict[str, dict], - session_id: str, - request_context: RequestContext, -) -> dict | None: - """Remove an owned session without allowing cross-scope cancellation.""" - - session = _get_owned_session(sessions, session_id, request_context) - if session is None: - return None - return sessions.pop(session_id, None) - - -_MAX_ADAPTER_SESSIONS = 100 -# How often the credential-session sweeper enforces its TTL, independent of traffic. -_CREDENTIAL_SWEEP_INTERVAL_SECONDS = 60 -_MAX_ADAPTER_SESSIONS_PER_WORKSPACE = 10 - - -def _scrub_credential_fields(session: dict) -> None: - """Overwrite credential fields before a session dict is dropped.""" - for key in ('app_secret', 'client_secret', 'secret'): - if key in session: - session[key] = None - - -def _sweep_credential_sessions(stores: 'list[tuple[dict, float]]') -> int: - """Evict sessions past their TTL and scrub the secrets they held. - - Returns the number of evicted sessions. Each entry is ``(store, ttl)``; a - session is evicted when ``now - created_at > ttl``. - """ - import time - - now = time.time() - evicted = 0 - for store, ttl in stores: - expired = [sid for sid, session in store.items() if now - session.get('created_at', 0) > ttl] - for sid in expired: - session = store.pop(sid, None) - if not session: - continue - task = session.get('task') - if task is not None and not task.done(): - task.cancel() - _scrub_credential_fields(session) - evicted += 1 - return evicted - - -def _start_adapter_session_task( - ap, - coro, - *, - adapter: str, - session_id: str, - request_context: RequestContext, -) -> asyncio.Task | None: - """Attach one credential exchange to tenant admission and app shutdown.""" - - try: - wrapper = ap.task_mgr.create_user_task( - coro, - kind='platform-adapter-credential-exchange', - name=f'{adapter}-credential-{session_id}', - label=f'{adapter} credential exchange', - instance_uuid=request_context.instance_uuid, - workspace_uuid=request_context.workspace_uuid, - placement_generation=request_context.placement_generation, - ) - except TaskCapacityError: - coro.close() - return None - return wrapper.task - - -def _make_room_for_session( - sessions: dict[str, dict], - request_context: RequestContext, -) -> None: - """Bound credential-exchange sessions globally and per workspace.""" - - workspace_uuid = request_context.workspace_uuid - owned = [ - (session_id, session) - for session_id, session in sessions.items() - if getattr(session.get('scope'), 'workspace_uuid', None) == workspace_uuid - ] - evict_workspace_session = len(owned) >= _MAX_ADAPTER_SESSIONS_PER_WORKSPACE - evict_global_session = len(sessions) >= _MAX_ADAPTER_SESSIONS - if not evict_workspace_session and not evict_global_session: - return - - candidates = owned if evict_workspace_session else list(sessions.items()) - session_id, _ = min( - candidates, - key=lambda item: float(item[1].get('created_at', 0.0)), - ) - session = sessions.pop(session_id, None) - task = session.get('task') if session is not None else None - if task is not None and not task.done(): - task.cancel() - - -def _decrypt_qqofficial_secret(encrypted_b64: str, key: bytes) -> str: - """Decrypt the AppSecret returned by the QQ Official QR binding endpoint. - - The base64 payload is laid out as `nonce (12 B) | ciphertext | tag (16 B)`. - `key` is the 32-byte AES-256 key locally generated when the bind task - was created and submitted as `key` to `q.qq.com/lite/create_bind_task`. - """ - import base64 - from cryptography.hazmat.primitives.ciphers.aead import AESGCM - - try: - raw = base64.b64decode(encrypted_b64) - except Exception as exc: - raise ValueError('Malformed encrypted credential') from exc - if len(key) != 32 or len(raw) <= 28: - raise ValueError('Invalid encrypted credential layout') - nonce, ciphertext, tag = raw[:12], raw[12:-16], raw[-16:] - try: - return AESGCM(key).decrypt(nonce, ciphertext + tag, None).decode('utf-8') - except Exception as exc: - raise ValueError('Failed to decrypt credential') from exc +from langbot.pkg.platform.credential_setup import ( + _AdapterSessionScope as _AdapterSessionScope, + _bind_session_scope as _bind_session_scope, + _get_owned_session as _get_owned_session, + _pop_owned_session as _pop_owned_session, + _scrub_credential_fields as _scrub_credential_fields, + _sweep_credential_sessions as _sweep_credential_sessions, + _start_adapter_session_task as _start_adapter_session_task, + _make_room_for_session as _make_room_for_session, + _decrypt_qqofficial_secret as _decrypt_qqofficial_secret, + CredentialSetupError, + get_credential_setup, +) @group.group_class('adapters', '/api/v1/platform/adapters') @@ -328,980 +163,169 @@ class AdaptersRouterGroup(group.RouterGroup): return response # In-memory session store for active registrations - _create_app_sessions: dict = {} - _SESSION_TTL = 900 # 15 minutes - - def _cleanup_expired_sessions(): - """Remove sessions that have exceeded their TTL.""" - import time - - now = time.time() - expired = [sid for sid, s in _create_app_sessions.items() if now - s.get('created_at', 0) > _SESSION_TTL] - for sid in expired: - session = _create_app_sessions.pop(sid, None) - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() + credential_setup = get_credential_setup(self.ap) + self._sweep_credential_sessions = credential_setup._sweep_credential_sessions @self.route('/lark/create-app', methods=['POST'], permission=Permission.RESOURCE_MANAGE) - async def _(request_context: RequestContext) -> str: - """Start Feishu one-click app registration. Returns session_id + QR code URL.""" - import uuid - import time - import lark_oapi as lark - from lark_oapi.scene.registration.errors import AppAccessDeniedError, AppExpiredError - - _cleanup_expired_sessions() - - session_id = str(uuid.uuid4()) - loop = asyncio.get_running_loop() - - session = { - 'status': 'pending', - 'qr_url': None, - 'expire_at': None, - 'app_id': None, - 'app_secret': None, - 'error': None, - 'created_at': time.time(), - } - _bind_session_scope(session, request_context) - _make_room_for_session(_create_app_sessions, request_context) - _create_app_sessions[session_id] = session - - def on_qr_code(info): - # May be called from a background thread by the SDK; - # use call_soon_threadsafe to safely update session state. - def _update(): - session['qr_url'] = info['url'] - session['expire_at'] = time.time() + 600 # 10 minutes - session['status'] = 'waiting' - - loop.call_soon_threadsafe(_update) - - async def run_registration(): - try: - result = await lark.aregister_app( - on_qr_code=on_qr_code, - source='langbot', - ) - session['status'] = 'success' - session['app_id'] = result['client_id'] - session['app_secret'] = result['client_secret'] - except AppAccessDeniedError: - session['status'] = 'error' - session['error'] = 'User denied authorization' - except AppExpiredError: - session['status'] = 'error' - session['error'] = 'QR code expired' - except Exception as e: - session['status'] = 'error' - session['error'] = str(e) - - task = _start_adapter_session_task( - self.ap, - run_registration(), - adapter='lark', - session_id=session_id, - request_context=request_context, - ) - if task is None: - _create_app_sessions.pop(session_id, None) - return self.http_status(429, -1, 'Too many active credential exchanges') - session['task'] = task - - # Wait for QR code to be ready (max 10 seconds) - for _ in range(20): - if session['qr_url']: - break - await asyncio.sleep(0.5) - - if not session['qr_url']: - task.cancel() - session['status'] = 'error' - session['error'] = 'Timeout waiting for QR code' - return self.http_status(504, -1, 'Timeout waiting for QR code') - - return self.success( - data={ - 'session_id': session_id, - 'qr_url': session['qr_url'], - 'expire_at': session['expire_at'], - } - ) + async def lark_start(request_context: RequestContext) -> str: + try: + result = await credential_setup.lark_start(request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/lark/create-app/status/', methods=['GET'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Poll registration status.""" - _cleanup_expired_sessions() - session = _get_owned_session(_create_app_sessions, session_id, request_context) - if not session: - return self.http_status(404, -1, 'Session not found') - - data = {'status': session['status']} - - if session['status'] == 'success': - data['app_id'] = session['app_id'] - data['app_secret'] = session['app_secret'] - _create_app_sessions.pop(session_id, None) - elif session['status'] == 'error': - data['error'] = session['error'] - _create_app_sessions.pop(session_id, None) - - return self.success(data=data) + async def lark_status(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.lark_status(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/lark/create-app/', methods=['DELETE'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Cancel and clean up a registration session.""" - session = _pop_owned_session(_create_app_sessions, session_id, request_context) - if session is None: - return self.http_status(404, -1, 'Session not found') - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() - return self.success(data={}) - - # ----------------------------------------------------------------------- - # WeChat QR Code Login - # ----------------------------------------------------------------------- - - _weixin_login_sessions: dict = {} - _WEIXIN_SESSION_TTL = 600 # 10 minutes (3 retries × 3 min QR validity) - - def _cleanup_expired_weixin_sessions(): - import time - - now = time.time() - expired = [ - sid for sid, s in _weixin_login_sessions.items() if now - s.get('created_at', 0) > _WEIXIN_SESSION_TTL - ] - for sid in expired: - session = _weixin_login_sessions.pop(sid, None) - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() + async def lark_cancel(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.lark_cancel(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route('/weixin/login', methods=['POST'], permission=Permission.RESOURCE_MANAGE) - async def _(request_context: RequestContext) -> str: - """Start WeChat QR code login. Returns session_id + QR code data URL.""" - import uuid - import time - - from langbot.libs.openclaw_weixin_api.client import OpenClawWeixinClient, DEFAULT_BASE_URL - - _cleanup_expired_weixin_sessions() - - session_id = str(uuid.uuid4()) - loop = asyncio.get_running_loop() - - session = { - 'status': 'pending', - 'qr_data_url': None, - 'expire_at': None, - 'token': None, - 'base_url': None, - 'account_id': None, - 'error': None, - 'created_at': time.time(), - } - _bind_session_scope(session, request_context) - _make_room_for_session(_weixin_login_sessions, request_context) - _weixin_login_sessions[session_id] = session - - client = OpenClawWeixinClient( - base_url=DEFAULT_BASE_URL, - token='', - ) - - async def run_login(): - try: - - def on_qrcode(qr_data_url: str, _qr_url: str): - def _update(): - session['qr_data_url'] = qr_data_url - session['expire_at'] = time.time() + 180 - session['status'] = 'waiting' - - loop.call_soon_threadsafe(_update) - - result = await client.login( - max_retries=1, - poll_timeout_ms=180_000, - on_qrcode=on_qrcode, - ) - session['status'] = 'success' - session['token'] = result.token - session['base_url'] = result.base_url - session['account_id'] = result.account_id - except Exception as e: - error_message = str(e) - if 'expired' in error_message.lower() or 'max retries exceeded' in error_message.lower(): - session['status'] = 'expired' - session['error'] = 'QR code expired' - else: - session['status'] = 'error' - session['error'] = error_message - finally: - await client.close() - - task = _start_adapter_session_task( - self.ap, - run_login(), - adapter='weixin', - session_id=session_id, - request_context=request_context, - ) - if task is None: - _weixin_login_sessions.pop(session_id, None) - return self.http_status(429, -1, 'Too many active credential exchanges') - session['task'] = task - - # Wait for QR code to be ready (max 10 seconds) - for _ in range(20): - if session['qr_data_url']: - break - await asyncio.sleep(0.5) - - if not session['qr_data_url']: - task.cancel() - session['status'] = 'error' - session['error'] = 'Timeout waiting for QR code' - return self.http_status(504, -1, 'Timeout waiting for QR code') - - return self.success( - data={ - 'session_id': session_id, - 'qr_data_url': session['qr_data_url'], - 'expire_at': session['expire_at'], - } - ) + async def weixin_start(request_context: RequestContext) -> str: + try: + result = await credential_setup.weixin_start(request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/weixin/login/status/', methods=['GET'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Poll WeChat login status.""" - _cleanup_expired_weixin_sessions() - session = _get_owned_session(_weixin_login_sessions, session_id, request_context) - if not session: - return self.http_status(404, -1, 'Session not found') - - data = { - 'status': session['status'], - 'qr_data_url': session['qr_data_url'], - 'expire_at': session['expire_at'], - } - - if session['status'] == 'success': - data['token'] = session['token'] - data['base_url'] = session['base_url'] - data['account_id'] = session['account_id'] - _weixin_login_sessions.pop(session_id, None) - elif session['status'] == 'error': - data['error'] = session['error'] - _weixin_login_sessions.pop(session_id, None) - elif session['status'] == 'expired': - data['error'] = session['error'] - _weixin_login_sessions.pop(session_id, None) - - return self.success(data=data) + async def weixin_status(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.weixin_status(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/weixin/login/', methods=['DELETE'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Cancel and clean up a WeChat login session.""" - session = _pop_owned_session(_weixin_login_sessions, session_id, request_context) - if session is None: - return self.http_status(404, -1, 'Session not found') - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() - return self.success(data={}) - - # ----------------------------------------------------------------------- - # DingTalk Device Flow QR Code Login - # ----------------------------------------------------------------------- - - _dingtalk_sessions: dict = {} - _DINGTALK_SESSION_TTL = 600 # 10 minutes (QR code validity window) - - def _cleanup_expired_dingtalk_sessions(): - import time - - now = time.time() - expired = [ - sid for sid, s in _dingtalk_sessions.items() if now - s.get('created_at', 0) > _DINGTALK_SESSION_TTL - ] - for sid in expired: - session = _dingtalk_sessions.pop(sid, None) - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() + async def weixin_cancel(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.weixin_cancel(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route('/dingtalk/create-app', methods=['POST'], permission=Permission.RESOURCE_MANAGE) - async def _(request_context: RequestContext) -> str: - """Start DingTalk one-click app creation via Device Flow. Returns session_id + QR code URL.""" - import uuid - import time - import aiohttp - - DINGTALK_BASE_URL = 'https://oapi.dingtalk.com' - - _cleanup_expired_dingtalk_sessions() - - session_id = str(uuid.uuid4()) - - session = { - 'status': 'pending', - 'qr_url': None, - 'expire_at': None, - 'client_id': None, - 'client_secret': None, - 'error': None, - 'created_at': time.time(), - 'device_code': None, - 'interval': 5, - } - _bind_session_scope(session, request_context) - _make_room_for_session(_dingtalk_sessions, request_context) - _dingtalk_sessions[session_id] = session - - async def run_device_flow(): - try: - timeout = aiohttp.ClientTimeout(total=10) - async with aiohttp.ClientSession(timeout=timeout) as http: - # Step 1: Init — get nonce - async with http.post( - f'{DINGTALK_BASE_URL}/app/registration/init', - json={'source': 'langbot'}, - ) as resp: - try: - data = await httpclient.read_json_limited(resp) - except (aiohttp.ContentTypeError, ValueError): - session['status'] = 'error' - session['error'] = 'Invalid response from DingTalk service' - return - if data.get('errcode', -1) != 0: - session['status'] = 'error' - session['error'] = data.get('errmsg', 'Failed to init') - return - nonce = data['nonce'] - - # Step 2: Begin — get device_code + QR URL - async with http.post( - f'{DINGTALK_BASE_URL}/app/registration/begin', - json={'nonce': nonce}, - ) as resp: - try: - data = await httpclient.read_json_limited(resp) - except (aiohttp.ContentTypeError, ValueError): - session['status'] = 'error' - session['error'] = 'Invalid response from DingTalk service' - return - if data.get('errcode', -1) != 0: - session['status'] = 'error' - session['error'] = data.get('errmsg', 'Failed to begin authorization') - return - - device_code = data['device_code'] - verification_uri_complete = data.get('verification_uri_complete', '') - expires_in = data.get('expires_in', 7200) - interval = data.get('interval', 5) - - session['device_code'] = device_code - session['interval'] = interval - session['qr_url'] = verification_uri_complete - session['expire_at'] = time.time() + 600 # QR code valid for ~10 min - session['status'] = 'waiting' - - # Step 3: Poll for authorization result - deadline = time.time() + expires_in - while time.time() < deadline: - await asyncio.sleep(interval) - - async with http.post( - f'{DINGTALK_BASE_URL}/app/registration/poll', - json={'device_code': device_code}, - ) as poll_resp: - try: - poll_data = await httpclient.read_json_limited(poll_resp) - except (aiohttp.ContentTypeError, ValueError): - continue - - if poll_data.get('errcode', -1) != 0: - session['status'] = 'error' - session['error'] = poll_data.get('errmsg', 'Poll failed') - return - - status = poll_data.get('status', '') - - if status == 'SUCCESS': - session['status'] = 'success' - session['client_id'] = poll_data.get('client_id', '') - session['client_secret'] = poll_data.get('client_secret', '') - return - elif status == 'FAIL': - session['status'] = 'error' - session['error'] = poll_data.get('fail_reason', 'Authorization failed') - return - elif status == 'EXPIRED': - session['status'] = 'error' - session['error'] = 'QR code expired' - return - # status == 'WAITING': continue polling - - # Timeout - session['status'] = 'error' - session['error'] = 'QR code expired' - - except asyncio.CancelledError: - return - except Exception as e: - session['status'] = 'error' - session['error'] = str(e) - - task = _start_adapter_session_task( - self.ap, - run_device_flow(), - adapter='dingtalk', - session_id=session_id, - request_context=request_context, - ) - if task is None: - _dingtalk_sessions.pop(session_id, None) - return self.http_status(429, -1, 'Too many active credential exchanges') - session['task'] = task - - # Wait for QR code to be ready (max 10 seconds) - for _ in range(20): - if session['qr_url'] or session['error']: - break - await asyncio.sleep(0.5) - - if session['error']: - task.cancel() - return self.http_status(502, -1, session['error']) - - if not session['qr_url']: - task.cancel() - session['status'] = 'error' - session['error'] = 'Timeout waiting for QR code' - return self.http_status(504, -1, 'Timeout waiting for QR code') - - return self.success( - data={ - 'session_id': session_id, - 'qr_url': session['qr_url'], - 'expire_at': session['expire_at'], - } - ) + async def dingtalk_start(request_context: RequestContext) -> str: + try: + result = await credential_setup.dingtalk_start(request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/dingtalk/create-app/status/', methods=['GET'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Poll DingTalk Device Flow status.""" - _cleanup_expired_dingtalk_sessions() - session = _get_owned_session(_dingtalk_sessions, session_id, request_context) - if not session: - return self.http_status(404, -1, 'Session not found') - - data = {'status': session['status']} - - if session['status'] == 'success': - data['client_id'] = session['client_id'] - data['client_secret'] = session['client_secret'] - _dingtalk_sessions.pop(session_id, None) - elif session['status'] == 'error': - data['error'] = session['error'] - _dingtalk_sessions.pop(session_id, None) - - return self.success(data=data) + async def dingtalk_status(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.dingtalk_status(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/dingtalk/create-app/', methods=['DELETE'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Cancel and clean up a DingTalk Device Flow session.""" - session = _pop_owned_session(_dingtalk_sessions, session_id, request_context) - if session is None: - return self.http_status(404, -1, 'Session not found') - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() - return self.success(data={}) - - # ----------------------------------------------------------------------- - # WeComBot QR Code One-Click Create - # ----------------------------------------------------------------------- - - _wecombot_sessions: dict = {} - _WECOMBOT_SESSION_TTL = 300 # 5 minutes (WeCom QR validity window) - - def _cleanup_expired_wecombot_sessions(): - import time - - now = time.time() - expired = [ - sid for sid, s in _wecombot_sessions.items() if now - s.get('created_at', 0) > _WECOMBOT_SESSION_TTL - ] - for sid in expired: - session = _wecombot_sessions.pop(sid, None) - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() + async def dingtalk_cancel(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.dingtalk_cancel(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route('/wecombot/create-bot', methods=['POST'], permission=Permission.RESOURCE_MANAGE) - async def _(request_context: RequestContext) -> str: - """Start WeComBot one-click creation via QR code. Returns session_id + QR code URL.""" - import uuid - import time - import aiohttp - - WECOM_QC_GENERATE_URL = 'https://work.weixin.qq.com/ai/qc/generate' - WECOM_QC_QUERY_URL = 'https://work.weixin.qq.com/ai/qc/query_result' - - _cleanup_expired_wecombot_sessions() - - session_id = str(uuid.uuid4()) - - session = { - 'status': 'pending', - 'qr_url': None, - 'expire_at': None, - 'botid': None, - 'secret': None, - 'error': None, - 'created_at': time.time(), - 'scode': None, - 'task': None, - } - _bind_session_scope(session, request_context) - _make_room_for_session(_wecombot_sessions, request_context) - _wecombot_sessions[session_id] = session - - async def run_qr_flow(): - try: - timeout = aiohttp.ClientTimeout(total=10) - async with aiohttp.ClientSession(timeout=timeout) as http: - # Step 1: Generate QR code - async with http.get( - f'{WECOM_QC_GENERATE_URL}?source=langbot&plat=0', - ) as resp: - try: - data = await httpclient.read_json_limited(resp) - except (aiohttp.ContentTypeError, ValueError): - session['status'] = 'error' - session['error'] = 'Invalid response from WeCom service' - return - if not data.get('data', {}).get('scode') or not data.get('data', {}).get('auth_url'): - session['status'] = 'error' - session['error'] = data.get('errmsg', 'Failed to generate QR code') - return - - scode = data['data']['scode'] - auth_url = data['data']['auth_url'] - - session['scode'] = scode - session['qr_url'] = auth_url - session['expire_at'] = time.time() + _WECOMBOT_SESSION_TTL - session['status'] = 'waiting' - - # Step 2: Poll for scan result - deadline = time.time() + _WECOMBOT_SESSION_TTL - while time.time() < deadline: - await asyncio.sleep(3) - - async with http.get( - f'{WECOM_QC_QUERY_URL}?scode={scode}', - ) as poll_resp: - try: - poll_data = await httpclient.read_json_limited(poll_resp) - except (aiohttp.ContentTypeError, ValueError): - continue - - status = poll_data.get('data', {}).get('status', '') - if status == 'success': - bot_info = poll_data.get('data', {}).get('bot_info', {}) - if bot_info.get('botid') and bot_info.get('secret'): - session['status'] = 'success' - session['botid'] = bot_info['botid'] - session['secret'] = bot_info['secret'] - return - else: - session['status'] = 'error' - session['error'] = 'Scan succeeded but bot info is incomplete' - return - - # Timeout - session['status'] = 'error' - session['error'] = 'QR code expired' - - except asyncio.CancelledError: - return - except Exception as e: - session['status'] = 'error' - session['error'] = str(e) - - task = _start_adapter_session_task( - self.ap, - run_qr_flow(), - adapter='wecombot', - session_id=session_id, - request_context=request_context, - ) - if task is None: - _wecombot_sessions.pop(session_id, None) - return self.http_status(429, -1, 'Too many active credential exchanges') - session['task'] = task - - # Wait for QR code to be ready (max 10 seconds) - for _ in range(20): - if session['qr_url'] or session['error']: - break - await asyncio.sleep(0.5) - - if session['error']: - task.cancel() - return self.http_status(502, -1, session['error']) - - if not session['qr_url']: - task.cancel() - session['status'] = 'error' - session['error'] = 'Timeout waiting for QR code' - return self.http_status(504, -1, 'Timeout waiting for QR code') - - return self.success( - data={ - 'session_id': session_id, - 'qr_url': session['qr_url'], - 'expire_at': session['expire_at'], - } - ) + async def wecombot_start(request_context: RequestContext) -> str: + try: + result = await credential_setup.wecombot_start(request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/wecombot/create-bot/status/', methods=['GET'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Poll WeComBot creation status.""" - _cleanup_expired_wecombot_sessions() - session = _get_owned_session(_wecombot_sessions, session_id, request_context) - if not session: - return self.http_status(404, -1, 'Session not found') - - data = {'status': session['status']} - - if session['status'] == 'success': - data['botid'] = session['botid'] - data['secret'] = session['secret'] - _wecombot_sessions.pop(session_id, None) - elif session['status'] == 'error': - data['error'] = session['error'] - _wecombot_sessions.pop(session_id, None) - - return self.success(data=data) + async def wecombot_status(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.wecombot_status(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/wecombot/create-bot/', methods=['DELETE'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Cancel and clean up a WeComBot creation session.""" - session = _pop_owned_session(_wecombot_sessions, session_id, request_context) - if session is None: - return self.http_status(404, -1, 'Session not found') - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() - return self.success(data={}) - - # ----------------------------------------------------------------------- - # QQ Official QR Binding - # ----------------------------------------------------------------------- - - _qqofficial_sessions: dict = {} - _QQOFFICIAL_SESSION_TTL = 300 # 5 minutes (QQ bind QR validity window) - - def _cleanup_expired_qqofficial_sessions(): - import time - - now = time.time() - expired = [ - sid for sid, s in _qqofficial_sessions.items() if now - s.get('created_at', 0) > _QQOFFICIAL_SESSION_TTL - ] - for sid in expired: - session = _qqofficial_sessions.pop(sid, None) - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() + async def wecombot_cancel(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.wecombot_cancel(session_id=session_id, request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route('/qqofficial/bind', methods=['POST'], permission=Permission.RESOURCE_MANAGE) - async def _(request_context: RequestContext) -> str: - """Start QQ Official QR binding. Returns session_id + QR URL. - - Flow: generate a local AES-256 key, register it with - `q.qq.com/lite/create_bind_task`, then poll - `q.qq.com/lite/poll_bind_result` until the user authorizes the - bind inside the QQ Bot Assistant on mobile QQ. The encrypted - AppSecret returned by the poll endpoint is decrypted with the - same key. The key never leaves this process. - """ - import uuid - import time - import secrets - import base64 - import aiohttp - - QQ_BIND_BASE = 'https://q.qq.com' - _cleanup_expired_qqofficial_sessions() - - bind_key_bytes = secrets.token_bytes(32) - bind_key = base64.b64encode(bind_key_bytes).decode('ascii') - - session_id = str(uuid.uuid4()) - session = { - 'status': 'pending', - 'qr_url': None, - 'expire_at': None, - 'appid': None, - 'secret': None, - 'user_openid': None, - 'error': None, - 'created_at': time.time(), - 'task_id': None, - 'bind_key_bytes': bind_key_bytes, - 'interval': 2, - } - _bind_session_scope(session, request_context) - _make_room_for_session(_qqofficial_sessions, request_context) - _qqofficial_sessions[session_id] = session - - async def run_qr_binding(): - try: - timeout = aiohttp.ClientTimeout(total=10) - async with aiohttp.ClientSession(timeout=timeout) as http: - # Step 1: create_bind_task — register our AES key, get task_id - async with http.post( - f'{QQ_BIND_BASE}/lite/create_bind_task', - json={'key': bind_key}, - headers={'Accept': 'application/json'}, - ) as resp: - try: - data = await httpclient.read_json_limited(resp) - except (aiohttp.ContentTypeError, ValueError): - session['status'] = 'error' - session['error'] = 'Invalid response from QQ bind service' - return - if int(data.get('retcode', -1)) != 0: - session['status'] = 'error' - session['error'] = ( - data.get('msg') or data.get('message') or 'Failed to create bind task' - ) - return - task_id = str((data.get('data') or {}).get('task_id') or '').strip() - if not task_id: - session['status'] = 'error' - session['error'] = 'Missing task_id in QQ response' - return - - # The QR encodes a URL that mobile QQ opens inside the QQ Bot Assistant. - # `source=langbot` is a courtesy attribution parameter so Tencent - # can see LangBot adoption metrics, matching the convention used by - # other third-party integrations (e.g. hermes-agent uses `source=hermes`). - qr_url = f'{QQ_BIND_BASE}/qqbot/openclaw/connect.html?task_id={task_id}&_wv=2&source=langbot' - session['task_id'] = task_id - session['qr_url'] = qr_url - session['expire_at'] = time.time() + _QQOFFICIAL_SESSION_TTL - session['status'] = 'waiting' - - # Step 2: poll_bind_result until completed (status=2) or expired (3). - deadline = time.time() + _QQOFFICIAL_SESSION_TTL - while time.time() < deadline: - await asyncio.sleep(session['interval']) - - async with http.post( - f'{QQ_BIND_BASE}/lite/poll_bind_result', - json={'task_id': task_id}, - headers={'Accept': 'application/json'}, - ) as poll_resp: - try: - poll_data = await httpclient.read_json_limited(poll_resp) - except (aiohttp.ContentTypeError, ValueError): - continue - - if int(poll_data.get('retcode', -1)) != 0: - session['status'] = 'error' - session['error'] = poll_data.get('msg') or poll_data.get('message') or 'Poll failed' - return - - payload = poll_data.get('data') or {} - try: - raw_status = int(payload.get('status', 0)) - except (TypeError, ValueError): - raw_status = 0 - - if raw_status == 2: - appid = str(payload.get('bot_appid') or '').strip() - encrypted = str(payload.get('bot_encrypt_secret') or '').strip() - if not appid or not encrypted: - session['status'] = 'error' - session['error'] = 'Incomplete credential payload' - return - try: - session['secret'] = _decrypt_qqofficial_secret( - encrypted, - bind_key_bytes, - ) - except ValueError as exc: - session['status'] = 'error' - session['error'] = str(exc) - return - session['appid'] = appid - # The scanner's OpenID is returned alongside the credentials — - # surfaced to the dashboard for audit / "bound by" display. - session['user_openid'] = str(payload.get('user_openid') or '').strip() or None - session['status'] = 'success' - return - - if raw_status == 3: - session['status'] = 'expired' - session['error'] = 'QR code expired' - return - # status 0 / 1: still pending, continue polling - - session['status'] = 'expired' - session['error'] = 'QR code expired' - - except asyncio.CancelledError: - return - except Exception as e: - session['status'] = 'error' - session['error'] = str(e) - - task = _start_adapter_session_task( - self.ap, - run_qr_binding(), - adapter='qqofficial', - session_id=session_id, - request_context=request_context, - ) - if task is None: - _qqofficial_sessions.pop(session_id, None) - return self.http_status(429, -1, 'Too many active credential exchanges') - session['task'] = task - - # Wait up to 10s for the QR URL to be ready before responding. - for _ in range(20): - if session['qr_url'] or session['error']: - break - await asyncio.sleep(0.5) - - if session['error']: - task.cancel() - return self.http_status(502, -1, session['error']) - - if not session['qr_url']: - task.cancel() - session['status'] = 'error' - session['error'] = 'Timeout waiting for QR code' - return self.http_status(504, -1, 'Timeout waiting for QR code') - - return self.success( - data={ - 'session_id': session_id, - 'qr_url': session['qr_url'], - 'expire_at': session['expire_at'], - } - ) + async def qqofficial_start(request_context: RequestContext) -> str: + try: + result = await credential_setup.qqofficial_start(request_context=request_context) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/qqofficial/bind/status/', methods=['GET'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Poll QQ Official QR binding status.""" - _cleanup_expired_qqofficial_sessions() - session = _get_owned_session(_qqofficial_sessions, session_id, request_context) - if not session: - return self.http_status(404, -1, 'Session not found') - - data = {'status': session['status']} - - if session['status'] == 'success': - data['appid'] = session['appid'] - data['secret'] = session['secret'] - if session.get('user_openid'): - data['user_openid'] = session['user_openid'] - _qqofficial_sessions.pop(session_id, None) - elif session['status'] in ('error', 'expired'): - data['error'] = session['error'] - _qqofficial_sessions.pop(session_id, None) - - return self.success(data=data) + async def qqofficial_status(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.qqofficial_status( + session_id=session_id, request_context=request_context + ) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) @self.route( '/qqofficial/bind/', methods=['DELETE'], permission=Permission.RESOURCE_MANAGE, ) - async def _(session_id: str, request_context: RequestContext) -> str: - """Cancel and clean up a QQ Official QR binding session.""" - session = _pop_owned_session(_qqofficial_sessions, session_id, request_context) - if session is None: - return self.http_status(404, -1, 'Session not found') - if session and session.get('task') and not session['task'].done(): - session['task'].cancel() - return self.success(data={}) - - # ---- credential-session TTL enforcement ---- - # - # Every store above holds a short-lived secret (app_secret, - # client_secret, bot secret) between the platform handshake and the - # caller persisting it. The per-route cleanups only run when another - # request arrives, so an abandoned flow would keep its secret in memory - # long past its validity window. This sweeper enforces the TTL on a - # timer instead, and scrubs the secret fields as it evicts. - - def _credential_stores() -> list[tuple[dict, float]]: - return [ - (_create_app_sessions, _SESSION_TTL), - (_dingtalk_sessions, _DINGTALK_SESSION_TTL), - (_wecombot_sessions, _WECOMBOT_SESSION_TTL), - (_qqofficial_sessions, _QQOFFICIAL_SESSION_TTL), - ] - - def _sweep_now() -> int: - return _sweep_credential_sessions(_credential_stores()) - - async def _credential_session_sweeper() -> None: - """Enforce the session TTL on a timer, so it does not depend on traffic.""" - import asyncio as _asyncio - - while True: - await _asyncio.sleep(_CREDENTIAL_SWEEP_INTERVAL_SECONDS) - try: - _sweep_now() - except Exception: # pragma: no cover - a sweep must never kill the app - _stdout_logger.exception('credential session sweep failed') - - self._sweep_credential_sessions = _sweep_now - # Some embeddings (notably test harnesses) provide a task manager without - # create_task; the per-route cleanups still enforce the TTL there. - create_task = getattr(self.ap.task_mgr, 'create_task', None) - if callable(create_task): - create_task( - _credential_session_sweeper(), - name='platform-credential-session-sweeper', - ) + async def qqofficial_cancel(session_id: str, request_context: RequestContext) -> str: + try: + result = await credential_setup.qqofficial_cancel( + session_id=session_id, request_context=request_context + ) + return self.success(data=result) + except CredentialSetupError as exc: + return self.http_status(exc.status_code, -1, str(exc)) diff --git a/src/langbot/pkg/api/http/controller/groups/plugins.py b/src/langbot/pkg/api/http/controller/groups/plugins.py index 5b8fd5487..a0c2fa4dc 100644 --- a/src/langbot/pkg/api/http/controller/groups/plugins.py +++ b/src/langbot/pkg/api/http/controller/groups/plugins.py @@ -5,15 +5,12 @@ from .....i18n import t import asyncio import base64 import collections.abc -import copy import quart -import re import httpx import uuid import os import zipfile from urllib.parse import urlparse -import posixpath import sqlalchemy from .....core import taskmgr @@ -30,130 +27,13 @@ from .....utils import httpclient from langbot_plugin.runtime.plugin.mgr import PluginInstallSource -_SECRET_MASK = '***' -_MISSING_SECRET = object() -_SENSITIVE_CONFIG_NAMES = frozenset( - { - 'api_key', - 'apikey', - 'auth', - 'authorization', - 'cookie', - 'credentials', - 'database_url', - 'dsn', - 'key', - 'proxy_authorization', - 'set_cookie', - } +from .....plugin.management import ( + count_plugin_components, + parse_github_repo_url, + normalize_plugin_asset_path as _normalize_plugin_asset_path, + redact_plugin_secrets, + restore_plugin_secret_placeholders, ) -_SENSITIVE_CONFIG_TOKENS = frozenset( - { - 'credential', - 'credentials', - 'passwd', - 'password', - 'secret', - 'token', - } -) -_SENSITIVE_KEY_QUALIFIERS = frozenset( - { - 'access', - 'api', - 'auth', - 'bearer', - 'client', - 'debug', - 'encryption', - 'private', - 'signing', - } -) - - -def _normalize_config_key(key: object) -> str: - value = re.sub(r'([a-z0-9])([A-Z])', r'\1_\2', str(key or '')) - return re.sub(r'[^a-zA-Z0-9]+', '_', value).strip('_').lower() - - -def _is_sensitive_config_key(key: object) -> bool: - normalized = _normalize_config_key(key) - if normalized in _SENSITIVE_CONFIG_NAMES: - return True - tokens = frozenset(token for token in normalized.split('_') if token) - if tokens & _SENSITIVE_CONFIG_TOKENS: - return True - return 'key' in tokens and bool(tokens & _SENSITIVE_KEY_QUALIFIERS) - - -def _mask_secret_structure(value): - """Mask every non-empty leaf while preserving container structure.""" - - if isinstance(value, dict): - return {key: _mask_secret_structure(item) for key, item in value.items()} - if isinstance(value, list): - return [_mask_secret_structure(item) for item in value] - if isinstance(value, tuple): - return tuple(_mask_secret_structure(item) for item in value) - if value is None or value == '': - return value - return _SECRET_MASK - - -def redact_plugin_secrets(value): - """Return a recursively redacted copy of plugin-facing data.""" - - if isinstance(value, dict): - return { - key: (_mask_secret_structure(item) if _is_sensitive_config_key(key) else redact_plugin_secrets(item)) - for key, item in value.items() - } - if isinstance(value, list): - return [redact_plugin_secrets(item) for item in value] - if isinstance(value, tuple): - return tuple(redact_plugin_secrets(item) for item in value) - return value - - -def restore_plugin_secret_placeholders(value, current_value=_MISSING_SECRET, *, sensitive: bool = False): - """Restore masked leaves from the current config before a management write.""" - - if sensitive and value == _SECRET_MASK: - if current_value is _MISSING_SECRET: - raise ValueError('Masked plugin secret has no existing value') - return copy.deepcopy(current_value) - if isinstance(value, dict): - current_mapping = current_value if isinstance(current_value, dict) else {} - return { - key: restore_plugin_secret_placeholders( - item, - current_mapping.get(key, _MISSING_SECRET), - sensitive=sensitive or _is_sensitive_config_key(key), - ) - for key, item in value.items() - } - if isinstance(value, list): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return [ - restore_plugin_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - sensitive=sensitive, - ) - for index, item in enumerate(value) - ] - if isinstance(value, tuple): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return tuple( - restore_plugin_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - sensitive=sensitive, - ) - for index, item in enumerate(value) - ) - return value # Resolve the built-in page SDK JS from the langbot_plugin package @@ -178,21 +58,6 @@ def _is_plugin_not_installed(exc: BaseException) -> bool: return 'is not installed in this workspace' in str(exc).lower() -def _normalize_plugin_asset_path(filepath: str) -> str | None: - filepath = filepath.replace('\\', '/') - if filepath.startswith('/'): - return None - - normalized = posixpath.normpath(filepath) - if normalized == '.' or normalized.startswith('../') or normalized == '..': - return None - - if normalized.startswith('components/pages/'): - return normalized - - return f'assets/{normalized}' - - def _get_request_origin() -> str: """Return the public request origin, respecting reverse-proxy headers.""" forwarded_proto = quart.request.headers.get('X-Forwarded-Proto', '').split(',')[0].strip() @@ -205,96 +70,8 @@ def _get_request_origin() -> str: @group.group_class('plugins', '/api/v1/plugins') class PluginsRouterGroup(group.RouterGroup): - @staticmethod - def _normalize_archive_path(path: str) -> str: - normalized = str(path or '').replace('\\', '/').strip('/') - return posixpath.normpath(normalized) if normalized else '' - - @classmethod - def _component_source_path(cls, entry) -> str: - if isinstance(entry, dict): - return cls._normalize_archive_path(entry.get('path') or '') - return cls._normalize_archive_path(str(entry or '')) - - @classmethod - def _count_component_configs(cls, component_config, archive_names: list[str]) -> int: - normalized_names = [cls._normalize_archive_path(name) for name in archive_names] - component_files: set[str] = set() - - if isinstance(component_config, list): - return len(component_config) - if not isinstance(component_config, dict): - return 1 if component_config else 0 - - for entry in component_config.get('fromFiles') or []: - source_path = cls._component_source_path(entry) - if source_path and source_path in normalized_names: - component_files.add(source_path) - - for entry in component_config.get('fromDirs') or []: - source_dir = cls._component_source_path(entry).rstrip('/') - if not source_dir: - continue - prefix = f'{source_dir}/' - for archive_name in normalized_names: - if not archive_name.startswith(prefix): - continue - if archive_name.lower().endswith(('.yaml', '.yml')): - component_files.add(archive_name) - - if component_files: - return len(component_files) - - return 1 if any(key in component_config for key in ('path', 'name', 'kind')) else 0 - - @classmethod - def _count_plugin_components(cls, components, archive_names: list[str]) -> dict[str, int]: - if not isinstance(components, dict): - return {} - - component_counts: dict[str, int] = {} - for kind, component_config in components.items(): - count = cls._count_component_configs(component_config, archive_names) - if count > 0: - component_counts[str(kind)] = count - return component_counts - - @staticmethod - def _parse_github_repo_url(repo_url: str) -> dict | None: - raw_url = str(repo_url or '').strip() - if not raw_url: - return None - - if not re.match(r'^[a-zA-Z][a-zA-Z0-9+.-]*://', raw_url): - raw_url = f'https://{raw_url}' - - parsed = urlparse(raw_url) - if parsed.netloc.lower() not in ('github.com', 'www.github.com'): - return None - - parts = [part for part in parsed.path.strip('/').split('/') if part] - if len(parts) < 2: - return None - - owner = parts[0] - repo = parts[1] - if repo.endswith('.git'): - repo = repo[:-4] - if not owner or not repo: - return None - - ref = '' - subdir = '' - if len(parts) >= 4 and parts[2] in ('tree', 'blob'): - ref = parts[3] - subdir = '/'.join(parts[4:]).strip('/') - - return { - 'owner': owner, - 'repo': repo, - 'ref': ref, - 'subdir': subdir, - } + _count_plugin_components = staticmethod(count_plugin_components) + _parse_github_repo_url = staticmethod(parse_github_repo_url) async def _check_extensions_limit(self, request_context: RequestContext) -> str | None: """Check if extensions limit is reached. Returns error response if limit exceeded, None otherwise.""" diff --git a/src/langbot/pkg/api/http/service/secrets.py b/src/langbot/pkg/api/http/service/secrets.py index bd0b82ac7..250f6134c 100644 --- a/src/langbot/pkg/api/http/service/secrets.py +++ b/src/langbot/pkg/api/http/service/secrets.py @@ -1,336 +1,3 @@ -from __future__ import annotations +"""HTTP compatibility exports for transport-neutral secret helpers.""" -import copy -import re -from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit - - -SECRET_MASK = '***' -_MISSING_SECRET = object() - -_SENSITIVE_NAMES = frozenset( - { - 'api_key', - 'api_keys', - 'apikey', - 'apikeys', - 'auth', - 'authorization', - 'cookie', - 'credentials', - 'database_url', - 'dsn', - 'header_value', - 'key', - 'proxy_authorization', - 'set_cookie', - 'webhook_url', - } -) -_SENSITIVE_TOKENS = frozenset( - { - 'apikey', - 'credential', - 'credentials', - 'passwd', - 'password', - 'secret', - 'token', - } -) -_KEY_QUALIFIERS = frozenset( - { - 'access', - 'api', - 'auth', - 'bearer', - 'client', - 'debug', - 'encryption', - 'private', - 'signing', - } -) -_SENSITIVE_URL_QUERY_NAMES = frozenset( - { - 'code', - 'credential', - 'credentials', - 'password', - 'passwd', - 'sig', - 'signature', - } -) - - -def _normalize_key(key: object) -> str: - value = re.sub(r'([a-z0-9])([A-Z])', r'\1_\2', str(key or '')) - return re.sub(r'[^a-zA-Z0-9]+', '_', value).strip('_').lower() - - -def is_sensitive_key(key: object) -> bool: - """Return whether a configuration key conventionally carries a secret.""" - - normalized = _normalize_key(key) - if normalized in _SENSITIVE_NAMES: - return True - tokens = frozenset(token for token in normalized.split('_') if token) - if tokens & _SENSITIVE_TOKENS: - return True - return bool(tokens & {'key', 'keys'}) and bool(tokens & _KEY_QUALIFIERS) - - -def is_url_key(key: object) -> bool: - """Return whether a configuration field conventionally carries a URL.""" - - normalized = _normalize_key(key) - return normalized == 'url' or normalized.endswith('_url') - - -def _is_sensitive_url_query_key(key: object) -> bool: - normalized = _normalize_key(key) - return ( - is_sensitive_key(key) or normalized in _SENSITIVE_URL_QUERY_NAMES or normalized.endswith(('_sig', '_signature')) - ) - - -def _redact_url_string(value: str) -> str: - if not value: - return value - try: - parsed = urlsplit(value) - netloc = parsed.netloc - if '@' in netloc: - _, host = netloc.rsplit('@', 1) - netloc = f'{SECRET_MASK}@{host}' - query = urlencode( - [ - (key, SECRET_MASK if _is_sensitive_url_query_key(key) and item else item) - for key, item in parse_qsl(parsed.query, keep_blank_values=True) - ], - doseq=True, - safe='*', - ) - return urlunsplit((parsed.scheme, netloc, parsed.path, query, parsed.fragment)) - except (TypeError, ValueError): - # A malformed URL cannot be safely decomposed, so fail closed. - return SECRET_MASK - - -def redact_url_secrets(value): - """Redact URL userinfo and credential-like query values.""" - - if isinstance(value, str): - return _redact_url_string(value) - if isinstance(value, list): - return [redact_url_secrets(item) for item in value] - if isinstance(value, tuple): - return tuple(redact_url_secrets(item) for item in value) - return copy.deepcopy(value) - - -def _contains_url_secret_placeholder(value) -> bool: - if isinstance(value, str): - if value == SECRET_MASK: - return True - try: - parsed = urlsplit(value) - if '@' in parsed.netloc and SECRET_MASK in parsed.netloc.rsplit('@', 1)[0]: - return True - return any( - item == SECRET_MASK and _is_sensitive_url_query_key(key) - for key, item in parse_qsl(parsed.query, keep_blank_values=True) - ) - except (TypeError, ValueError): - return False - if isinstance(value, (list, tuple)): - return any(_contains_url_secret_placeholder(item) for item in value) - return False - - -def _restore_url_string(value: str, current_value) -> str: - if value == SECRET_MASK: - if current_value is _MISSING_SECRET: - raise ValueError('Masked URL secret has no existing value') - return copy.deepcopy(current_value) - - try: - submitted = urlsplit(value) - except (TypeError, ValueError): - return value - - current = None - if isinstance(current_value, str): - try: - current = urlsplit(current_value) - except (TypeError, ValueError): - current = None - - netloc = submitted.netloc - if '@' in netloc: - submitted_userinfo, host = netloc.rsplit('@', 1) - if SECRET_MASK in submitted_userinfo: - if current is None or '@' not in current.netloc: - raise ValueError('Masked URL userinfo has no existing value') - current_userinfo, _ = current.netloc.rsplit('@', 1) - netloc = f'{current_userinfo}@{host}' - - current_query: dict[str, list[str]] = {} - if current is not None: - for key, item in parse_qsl(current.query, keep_blank_values=True): - current_query.setdefault(_normalize_key(key), []).append(item) - consumed: dict[str, int] = {} - restored_query: list[tuple[str, str]] = [] - for key, item in parse_qsl(submitted.query, keep_blank_values=True): - normalized = _normalize_key(key) - if item == SECRET_MASK and _is_sensitive_url_query_key(key): - index = consumed.get(normalized, 0) - candidates = current_query.get(normalized, []) - if index >= len(candidates): - raise ValueError('Masked URL query secret has no existing value') - item = candidates[index] - consumed[normalized] = index + 1 - restored_query.append((key, item)) - - return urlunsplit( - ( - submitted.scheme, - netloc, - submitted.path, - urlencode(restored_query, doseq=True, safe='*'), - submitted.fragment, - ) - ) - - -def restore_url_secret_placeholders(value, current_value=_MISSING_SECRET): - """Restore URL placeholders from the corresponding persisted URL.""" - - if isinstance(value, str): - return _restore_url_string(value, current_value) - if isinstance(value, list): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return [ - restore_url_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - ) - for index, item in enumerate(value) - ] - if isinstance(value, tuple): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return tuple( - restore_url_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - ) - for index, item in enumerate(value) - ) - return copy.deepcopy(value) - - -def mask_secret_value(value): - """Return a shape-preserving copy whose non-empty leaves are masked.""" - - if isinstance(value, dict): - return {key: mask_secret_value(item) for key, item in value.items()} - if isinstance(value, list): - return [mask_secret_value(item) for item in value] - if isinstance(value, tuple): - return tuple(mask_secret_value(item) for item in value) - if value is None or value == '': - return value - return SECRET_MASK - - -def redact_secrets(value): - """Return a recursively redacted copy without mutating the source value.""" - - if isinstance(value, dict): - return { - key: ( - mask_secret_value(item) - if is_sensitive_key(key) - else redact_url_secrets(item) - if is_url_key(key) - else redact_secrets(item) - ) - for key, item in value.items() - } - if isinstance(value, list): - return [redact_secrets(item) for item in value] - if isinstance(value, tuple): - return tuple(redact_secrets(item) for item in value) - return copy.deepcopy(value) - - -def restore_secret_placeholders(value, current_value=_MISSING_SECRET, *, sensitive: bool = False): - """Restore masked leaves from existing data before a management write. - - ``***`` is a reserved placeholder only inside a sensitive field. A masked - leaf without an existing counterpart is rejected so it can never become a - persisted credential. Empty values and explicit replacements pass through. - """ - - if sensitive and value == SECRET_MASK: - if current_value is _MISSING_SECRET: - raise ValueError('Masked secret has no existing value') - return copy.deepcopy(current_value) - if isinstance(value, dict): - current_mapping = current_value if isinstance(current_value, dict) else {} - return { - key: ( - restore_url_secret_placeholders( - item, - current_mapping.get(key, _MISSING_SECRET), - ) - if not sensitive and not is_sensitive_key(key) and is_url_key(key) - else restore_secret_placeholders( - item, - current_mapping.get(key, _MISSING_SECRET), - sensitive=sensitive or is_sensitive_key(key), - ) - ) - for key, item in value.items() - } - if isinstance(value, list): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return [ - restore_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - sensitive=sensitive, - ) - for index, item in enumerate(value) - ] - if isinstance(value, tuple): - current_items = current_value if isinstance(current_value, (list, tuple)) else () - return tuple( - restore_secret_placeholders( - item, - current_items[index] if index < len(current_items) else _MISSING_SECRET, - sensitive=sensitive, - ) - for index, item in enumerate(value) - ) - return copy.deepcopy(value) - - -def contains_secret_placeholder(value, *, sensitive: bool = False) -> bool: - """Return whether ``value`` contains a meaningful masked secret leaf.""" - - if sensitive and value == SECRET_MASK: - return True - if isinstance(value, dict): - return any( - ( - _contains_url_secret_placeholder(item) - if not sensitive and not is_sensitive_key(key) and is_url_key(key) - else contains_secret_placeholder(item, sensitive=sensitive or is_sensitive_key(key)) - ) - for key, item in value.items() - ) - if isinstance(value, (list, tuple)): - return any(contains_secret_placeholder(item, sensitive=sensitive) for item in value) - return False +from ...secrets import * # noqa: F401,F403 diff --git a/src/langbot/pkg/api/secrets.py b/src/langbot/pkg/api/secrets.py new file mode 100644 index 000000000..f498892d2 --- /dev/null +++ b/src/langbot/pkg/api/secrets.py @@ -0,0 +1,349 @@ +from __future__ import annotations + +import copy +import re +from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit + + +SECRET_MASK = '***' +_MISSING_SECRET = object() + +_SENSITIVE_NAMES = frozenset( + { + 'api_key', + 'api_keys', + 'apikey', + 'apikeys', + 'auth', + 'authorization', + 'cookie', + 'credentials', + 'database_url', + 'dsn', + 'header_value', + 'key', + 'proxy_authorization', + 'set_cookie', + 'webhook_url', + } +) +_SENSITIVE_TOKENS = frozenset( + { + 'apikey', + 'credential', + 'credentials', + 'passwd', + 'password', + 'secret', + 'token', + } +) +_KEY_QUALIFIERS = frozenset( + { + 'access', + 'api', + 'auth', + 'bearer', + 'client', + 'debug', + 'encryption', + 'private', + 'signing', + } +) +_SENSITIVE_URL_QUERY_NAMES = frozenset( + { + 'code', + 'credential', + 'credentials', + 'password', + 'passwd', + 'sig', + 'signature', + } +) + + +def _normalize_key(key: object) -> str: + value = re.sub(r'([a-z0-9])([A-Z])', r'\1_\2', str(key or '')) + return re.sub(r'[^a-zA-Z0-9]+', '_', value).strip('_').lower() + + +def is_sensitive_key(key: object) -> bool: + """Return whether a configuration key conventionally carries a secret.""" + + normalized = _normalize_key(key) + if normalized in _SENSITIVE_NAMES: + return True + tokens = frozenset(token for token in normalized.split('_') if token) + if tokens & _SENSITIVE_TOKENS: + return True + return bool(tokens & {'key', 'keys'}) and bool(tokens & _KEY_QUALIFIERS) + + +def is_url_key(key: object) -> bool: + """Return whether a configuration field conventionally carries a URL.""" + + normalized = _normalize_key(key) + return normalized == 'url' or normalized.endswith('_url') + + +def _is_sensitive_url_query_key(key: object) -> bool: + normalized = _normalize_key(key) + return ( + is_sensitive_key(key) or normalized in _SENSITIVE_URL_QUERY_NAMES or normalized.endswith(('_sig', '_signature')) + ) + + +def _redact_url_string(value: str) -> str: + if not value: + return value + try: + parsed = urlsplit(value) + netloc = parsed.netloc + if '@' in netloc: + _, host = netloc.rsplit('@', 1) + netloc = f'{SECRET_MASK}@{host}' + query = urlencode( + [ + (key, SECRET_MASK if _is_sensitive_url_query_key(key) and item else item) + for key, item in parse_qsl(parsed.query, keep_blank_values=True) + ], + doseq=True, + safe='*', + ) + return urlunsplit((parsed.scheme, netloc, parsed.path, query, parsed.fragment)) + except (TypeError, ValueError): + # A malformed URL cannot be safely decomposed, so fail closed. + return SECRET_MASK + + +def redact_url_secrets(value): + """Redact URL userinfo and credential-like query values.""" + + if isinstance(value, str): + return _redact_url_string(value) + if isinstance(value, list): + return [redact_url_secrets(item) for item in value] + if isinstance(value, tuple): + return tuple(redact_url_secrets(item) for item in value) + return copy.deepcopy(value) + + +def _contains_url_secret_placeholder(value) -> bool: + if isinstance(value, str): + if value == SECRET_MASK: + return True + try: + parsed = urlsplit(value) + if '@' in parsed.netloc and SECRET_MASK in parsed.netloc.rsplit('@', 1)[0]: + return True + return any( + item == SECRET_MASK and _is_sensitive_url_query_key(key) + for key, item in parse_qsl(parsed.query, keep_blank_values=True) + ) + except (TypeError, ValueError): + return False + if isinstance(value, (list, tuple)): + return any(_contains_url_secret_placeholder(item) for item in value) + return False + + +def _restore_url_string(value: str, current_value) -> str: + if value == SECRET_MASK: + if current_value is _MISSING_SECRET: + raise ValueError('Masked URL secret has no existing value') + return copy.deepcopy(current_value) + + try: + submitted = urlsplit(value) + except (TypeError, ValueError): + return value + + current = None + if isinstance(current_value, str): + try: + current = urlsplit(current_value) + except (TypeError, ValueError): + current = None + + netloc = submitted.netloc + if '@' in netloc: + submitted_userinfo, host = netloc.rsplit('@', 1) + if SECRET_MASK in submitted_userinfo: + if current is None or '@' not in current.netloc: + raise ValueError('Masked URL userinfo has no existing value') + current_userinfo, _ = current.netloc.rsplit('@', 1) + netloc = f'{current_userinfo}@{host}' + + current_query: dict[str, list[str]] = {} + if current is not None: + for key, item in parse_qsl(current.query, keep_blank_values=True): + current_query.setdefault(_normalize_key(key), []).append(item) + consumed: dict[str, int] = {} + restored_query: list[tuple[str, str]] = [] + for key, item in parse_qsl(submitted.query, keep_blank_values=True): + normalized = _normalize_key(key) + if item == SECRET_MASK and _is_sensitive_url_query_key(key): + index = consumed.get(normalized, 0) + candidates = current_query.get(normalized, []) + if index >= len(candidates): + raise ValueError('Masked URL query secret has no existing value') + item = candidates[index] + consumed[normalized] = index + 1 + restored_query.append((key, item)) + + return urlunsplit( + ( + submitted.scheme, + netloc, + submitted.path, + urlencode(restored_query, doseq=True, safe='*'), + submitted.fragment, + ) + ) + + +def restore_url_secret_placeholders(value, current_value=_MISSING_SECRET): + """Restore URL placeholders from the corresponding persisted URL.""" + + if isinstance(value, str): + return _restore_url_string(value, current_value) + if isinstance(value, list): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return [ + restore_url_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + ) + for index, item in enumerate(value) + ] + if isinstance(value, tuple): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return tuple( + restore_url_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + ) + for index, item in enumerate(value) + ) + return copy.deepcopy(value) + + +def mask_secret_value(value): + """Return a shape-preserving copy whose non-empty leaves are masked.""" + + if isinstance(value, dict): + return {key: mask_secret_value(item) for key, item in value.items()} + if isinstance(value, list): + return [mask_secret_value(item) for item in value] + if isinstance(value, tuple): + return tuple(mask_secret_value(item) for item in value) + if value is None or value == '': + return value + return SECRET_MASK + + +def redact_secrets(value): + """Return a recursively redacted copy without mutating the source value.""" + + if isinstance(value, dict): + return { + key: ( + mask_secret_value(item) + if is_sensitive_key(key) + else redact_url_secrets(item) + if is_url_key(key) + else redact_secrets(item) + ) + for key, item in value.items() + } + if isinstance(value, list): + return [redact_secrets(item) for item in value] + if isinstance(value, tuple): + return tuple(redact_secrets(item) for item in value) + return copy.deepcopy(value) + + +def restore_secret_placeholders(value, current_value=_MISSING_SECRET, *, sensitive: bool = False): + """Restore masked leaves from existing data before a management write. + + ``***`` is a reserved placeholder only inside a sensitive field. A masked + leaf without an existing counterpart is rejected so it can never become a + persisted credential. Empty values and explicit replacements pass through. + """ + + if sensitive and value == SECRET_MASK: + if current_value is _MISSING_SECRET: + raise ValueError('Masked secret has no existing value') + return copy.deepcopy(current_value) + if isinstance(value, dict): + current_mapping = current_value if isinstance(current_value, dict) else {} + return { + key: ( + restore_url_secret_placeholders( + item, + current_mapping.get(key, _MISSING_SECRET), + ) + if not sensitive and not is_sensitive_key(key) and is_url_key(key) + else restore_secret_placeholders( + item, + current_mapping.get(key, _MISSING_SECRET), + sensitive=sensitive or is_sensitive_key(key), + ) + ) + for key, item in value.items() + } + if isinstance(value, list): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return [ + restore_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + sensitive=sensitive, + ) + for index, item in enumerate(value) + ] + if isinstance(value, tuple): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return tuple( + restore_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + sensitive=sensitive, + ) + for index, item in enumerate(value) + ) + return copy.deepcopy(value) + + +def contains_secret_placeholder(value, *, sensitive: bool = False) -> bool: + """Return whether ``value`` contains a meaningful masked secret leaf.""" + + if sensitive and value == SECRET_MASK: + return True + if isinstance(value, dict): + return any( + ( + _contains_url_secret_placeholder(item) + if not sensitive and not is_sensitive_key(key) and is_url_key(key) + else contains_secret_placeholder(item, sensitive=sensitive or is_sensitive_key(key)) + ) + for key, item in value.items() + ) + if isinstance(value, (list, tuple)): + return any(contains_secret_placeholder(item, sensitive=sensitive) for item in value) + return False + + +__all__ = [ + 'SECRET_MASK', + 'contains_secret_placeholder', + 'is_sensitive_key', + 'is_url_key', + 'mask_secret_value', + 'redact_secrets', + 'redact_url_secrets', + 'restore_secret_placeholders', + 'restore_url_secret_placeholders', +] diff --git a/src/langbot/pkg/core/app.py b/src/langbot/pkg/core/app.py index 9f84ee10f..3200fbe87 100644 --- a/src/langbot/pkg/core/app.py +++ b/src/langbot/pkg/core/app.py @@ -61,6 +61,7 @@ from ..cloud import directory_projection as cloud_directory_projection_module from ..cloud import entitlements as cloud_entitlements_module from ..cloud import model_catalog as cloud_model_catalog_module from ..api.http.context import ExecutionContext, PrincipalContext, PrincipalType +from ..api.application import ApplicationAPI if TYPE_CHECKING: from ..agent.runner import RunnerRegistry, AgentRunOrchestrator, RunnerDefaultConfigService @@ -183,6 +184,8 @@ class Application: pipeline_service: pipeline_service.PipelineService = None + application_api: ApplicationAPI = None + agent_service: agent_service.AgentService = None bot_service: bot_service.BotService = None diff --git a/src/langbot/pkg/core/stages/build_app.py b/src/langbot/pkg/core/stages/build_app.py index c42dd769e..beeef9117 100644 --- a/src/langbot/pkg/core/stages/build_app.py +++ b/src/langbot/pkg/core/stages/build_app.py @@ -53,6 +53,7 @@ from ...cloud.directory_projection import DirectoryProjectionService from ...cloud.entitlements import EntitlementResolver from ...cloud.model_catalog import CloudModelCatalogSyncService from ...api.http.context import ExecutionContext, PrincipalContext, PrincipalType +from ...api.application import ApplicationAPI from ...api.http.authz import WorkspaceRequiredError @@ -141,6 +142,7 @@ class BuildAppStage(stage.BootingStage): ap.skill_repository = skill_repository.SkillRepository(ap) skill_service_inst = skill_service.SkillService(ap) ap.skill_service = skill_service_inst + ap.application_api = ApplicationAPI(ap) proxy_mgr = proxy.ProxyManager(ap) await proxy_mgr.initialize() diff --git a/src/langbot/pkg/core/taskmgr.py b/src/langbot/pkg/core/taskmgr.py index b3ae3d1c7..4a56bd21b 100644 --- a/src/langbot/pkg/core/taskmgr.py +++ b/src/langbot/pkg/core/taskmgr.py @@ -8,11 +8,13 @@ import typing import datetime import time -from . import app from . import entities as core_entities from .errors import TaskCapacityError from .task_boundary import create_detached_task +if typing.TYPE_CHECKING: + from . import app + class TaskContext: """Task tracking context""" diff --git a/src/langbot/pkg/platform/credential_setup.py b/src/langbot/pkg/platform/credential_setup.py new file mode 100644 index 000000000..ec85e7d9c --- /dev/null +++ b/src/langbot/pkg/platform/credential_setup.py @@ -0,0 +1,1136 @@ +"""Principal-scoped platform credential exchanges shared by application entrypoints.""" + +from __future__ import annotations + +import asyncio +import dataclasses +import logging + +from langbot.pkg.api.context import RequestContext +from langbot.pkg.core.errors import TaskCapacityError +from langbot.pkg.utils import httpclient + + +@dataclasses.dataclass(frozen=True, slots=True) +class _AdapterSessionScope: + """Immutable tenant and principal binding for a credential exchange.""" + + instance_uuid: str + workspace_uuid: str + placement_generation: int + principal_type: str + account_uuid: str | None + api_key_uuid: str | None + support_session_id: str | None + + @classmethod + def from_request_context(cls, request_context: RequestContext) -> '_AdapterSessionScope': + principal = request_context.principal + return cls( + instance_uuid=request_context.instance_uuid, + workspace_uuid=request_context.workspace_uuid, + placement_generation=request_context.placement_generation, + principal_type=principal.principal_type.value, + account_uuid=principal.account_uuid, + api_key_uuid=principal.api_key_uuid, + support_session_id=principal.support_session_id, + ) + + def matches(self, request_context: RequestContext) -> bool: + """Return whether a request is from the exact initiating tenant principal.""" + + return self == self.from_request_context(request_context) + + +def _bind_session_scope(session: dict, request_context: RequestContext) -> None: + session['scope'] = _AdapterSessionScope.from_request_context(request_context) + + +def _get_owned_session( + sessions: dict[str, dict], + session_id: str, + request_context: RequestContext, +) -> dict | None: + """Resolve a session without revealing sessions owned by another scope.""" + + session = sessions.get(session_id) + scope = session.get('scope') if session is not None else None + if not isinstance(scope, _AdapterSessionScope) or not scope.matches(request_context): + return None + return session + + +def _pop_owned_session( + sessions: dict[str, dict], + session_id: str, + request_context: RequestContext, +) -> dict | None: + """Remove an owned session without allowing cross-scope cancellation.""" + + session = _get_owned_session(sessions, session_id, request_context) + if session is None: + return None + return sessions.pop(session_id, None) + + +_MAX_ADAPTER_SESSIONS = 100 +# How often the credential-session sweeper enforces its TTL, independent of traffic. +_CREDENTIAL_SWEEP_INTERVAL_SECONDS = 60 +_MAX_ADAPTER_SESSIONS_PER_WORKSPACE = 10 + + +def _scrub_credential_fields(session: dict) -> None: + """Overwrite credential fields before a session dict is dropped.""" + for key in ('app_secret', 'client_secret', 'secret'): + if key in session: + session[key] = None + + +def _sweep_credential_sessions(stores: 'list[tuple[dict, float]]') -> int: + """Evict sessions past their TTL and scrub the secrets they held. + + Returns the number of evicted sessions. Each entry is ``(store, ttl)``; a + session is evicted when ``now - created_at > ttl``. + """ + import time + + now = time.time() + evicted = 0 + for store, ttl in stores: + expired = [sid for sid, session in store.items() if now - session.get('created_at', 0) > ttl] + for sid in expired: + session = store.pop(sid, None) + if not session: + continue + task = session.get('task') + if task is not None and not task.done(): + task.cancel() + _scrub_credential_fields(session) + evicted += 1 + return evicted + + +def _start_adapter_session_task( + ap, + coro, + *, + adapter: str, + session_id: str, + request_context: RequestContext, +) -> asyncio.Task | None: + """Attach one credential exchange to tenant admission and app shutdown.""" + + try: + wrapper = ap.task_mgr.create_user_task( + coro, + kind='platform-adapter-credential-exchange', + name=f'{adapter}-credential-{session_id}', + label=f'{adapter} credential exchange', + instance_uuid=request_context.instance_uuid, + workspace_uuid=request_context.workspace_uuid, + placement_generation=request_context.placement_generation, + ) + except TaskCapacityError: + coro.close() + return None + return wrapper.task + + +def _make_room_for_session( + sessions: dict[str, dict], + request_context: RequestContext, +) -> None: + """Bound credential-exchange sessions globally and per workspace.""" + + workspace_uuid = request_context.workspace_uuid + owned = [ + (session_id, session) + for session_id, session in sessions.items() + if getattr(session.get('scope'), 'workspace_uuid', None) == workspace_uuid + ] + evict_workspace_session = len(owned) >= _MAX_ADAPTER_SESSIONS_PER_WORKSPACE + evict_global_session = len(sessions) >= _MAX_ADAPTER_SESSIONS + if not evict_workspace_session and not evict_global_session: + return + + candidates = owned if evict_workspace_session else list(sessions.items()) + session_id, _ = min( + candidates, + key=lambda item: float(item[1].get('created_at', 0.0)), + ) + session = sessions.pop(session_id, None) + task = session.get('task') if session is not None else None + if task is not None and not task.done(): + task.cancel() + + +def _decrypt_qqofficial_secret(encrypted_b64: str, key: bytes) -> str: + """Decrypt the AppSecret returned by the QQ Official QR binding endpoint. + + The base64 payload is laid out as `nonce (12 B) | ciphertext | tag (16 B)`. + `key` is the 32-byte AES-256 key locally generated when the bind task + was created and submitted as `key` to `q.qq.com/lite/create_bind_task`. + """ + import base64 + from cryptography.hazmat.primitives.ciphers.aead import AESGCM + + try: + raw = base64.b64decode(encrypted_b64) + except Exception as exc: + raise ValueError('Malformed encrypted credential') from exc + if len(key) != 32 or len(raw) <= 28: + raise ValueError('Invalid encrypted credential layout') + nonce, ciphertext, tag = raw[:12], raw[12:-16], raw[-16:] + try: + return AESGCM(key).decrypt(nonce, ciphertext + tag, None).decode('utf-8') + except Exception as exc: + raise ValueError('Failed to decrypt credential') from exc + + +class CredentialSetupError(RuntimeError): + error_code = 'credential_setup_failed' + + def __init__(self, status_code: int, message: str): + self.status_code = status_code + super().__init__(message) + + +class CredentialSetupService: + def __init__(self, ap): + self.ap = ap + _create_app_sessions: dict = {} + _SESSION_TTL = 900 # 15 minutes + + def _cleanup_expired_sessions(): + """Remove sessions that have exceeded their TTL.""" + import time + + now = time.time() + expired = [sid for sid, s in _create_app_sessions.items() if now - s.get('created_at', 0) > _SESSION_TTL] + for sid in expired: + session = _create_app_sessions.pop(sid, None) + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + + async def lark_start(request_context: RequestContext) -> dict: + """Start Feishu one-click app registration. Returns session_id + QR code URL.""" + import uuid + import time + import lark_oapi as lark + from lark_oapi.scene.registration.errors import AppAccessDeniedError, AppExpiredError + + _cleanup_expired_sessions() + + session_id = str(uuid.uuid4()) + loop = asyncio.get_running_loop() + + session = { + 'status': 'pending', + 'qr_url': None, + 'expire_at': None, + 'app_id': None, + 'app_secret': None, + 'error': None, + 'created_at': time.time(), + } + _bind_session_scope(session, request_context) + _make_room_for_session(_create_app_sessions, request_context) + _create_app_sessions[session_id] = session + + def on_qr_code(info): + # May be called from a background thread by the SDK; + # use call_soon_threadsafe to safely update session state. + def _update(): + session['qr_url'] = info['url'] + session['expire_at'] = time.time() + 600 # 10 minutes + session['status'] = 'waiting' + + loop.call_soon_threadsafe(_update) + + async def run_registration(): + try: + result = await lark.aregister_app( + on_qr_code=on_qr_code, + source='langbot', + ) + session['status'] = 'success' + session['app_id'] = result['client_id'] + session['app_secret'] = result['client_secret'] + except AppAccessDeniedError: + session['status'] = 'error' + session['error'] = 'User denied authorization' + except AppExpiredError: + session['status'] = 'error' + session['error'] = 'QR code expired' + except Exception as e: + session['status'] = 'error' + session['error'] = str(e) + + task = _start_adapter_session_task( + self.ap, + run_registration(), + adapter='lark', + session_id=session_id, + request_context=request_context, + ) + if task is None: + _create_app_sessions.pop(session_id, None) + raise CredentialSetupError(429, 'Too many active credential exchanges') + session['task'] = task + + # Wait for QR code to be ready (max 10 seconds) + for _ in range(20): + if session['qr_url']: + break + await asyncio.sleep(0.5) + + if not session['qr_url']: + task.cancel() + session['status'] = 'error' + session['error'] = 'Timeout waiting for QR code' + raise CredentialSetupError(504, 'Timeout waiting for QR code') + + return { + 'session_id': session_id, + 'qr_url': session['qr_url'], + 'expire_at': session['expire_at'], + } + + async def lark_status(session_id: str, request_context: RequestContext) -> dict: + """Poll registration status.""" + _cleanup_expired_sessions() + session = _get_owned_session(_create_app_sessions, session_id, request_context) + if not session: + raise CredentialSetupError(404, 'Session not found') + + data = {'status': session['status']} + + if session['status'] == 'success': + data['app_id'] = session['app_id'] + data['app_secret'] = session['app_secret'] + _create_app_sessions.pop(session_id, None) + elif session['status'] == 'error': + data['error'] = session['error'] + _create_app_sessions.pop(session_id, None) + + return data + + async def lark_cancel(session_id: str, request_context: RequestContext) -> dict: + """Cancel and clean up a registration session.""" + session = _pop_owned_session(_create_app_sessions, session_id, request_context) + if session is None: + raise CredentialSetupError(404, 'Session not found') + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + return {} + + # ----------------------------------------------------------------------- + # WeChat QR Code Login + # ----------------------------------------------------------------------- + + _weixin_login_sessions: dict = {} + _WEIXIN_SESSION_TTL = 600 # 10 minutes (3 retries × 3 min QR validity) + + def _cleanup_expired_weixin_sessions(): + import time + + now = time.time() + expired = [ + sid for sid, s in _weixin_login_sessions.items() if now - s.get('created_at', 0) > _WEIXIN_SESSION_TTL + ] + for sid in expired: + session = _weixin_login_sessions.pop(sid, None) + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + + async def weixin_start(request_context: RequestContext) -> dict: + """Start WeChat QR code login. Returns session_id + QR code data URL.""" + import uuid + import time + + from langbot.libs.openclaw_weixin_api.client import OpenClawWeixinClient, DEFAULT_BASE_URL + + _cleanup_expired_weixin_sessions() + + session_id = str(uuid.uuid4()) + loop = asyncio.get_running_loop() + + session = { + 'status': 'pending', + 'qr_data_url': None, + 'expire_at': None, + 'token': None, + 'base_url': None, + 'account_id': None, + 'error': None, + 'created_at': time.time(), + } + _bind_session_scope(session, request_context) + _make_room_for_session(_weixin_login_sessions, request_context) + _weixin_login_sessions[session_id] = session + + client = OpenClawWeixinClient( + base_url=DEFAULT_BASE_URL, + token='', + ) + + async def run_login(): + try: + + def on_qrcode(qr_data_url: str, _qr_url: str): + def _update(): + session['qr_data_url'] = qr_data_url + session['expire_at'] = time.time() + 180 + session['status'] = 'waiting' + + loop.call_soon_threadsafe(_update) + + result = await client.login( + max_retries=1, + poll_timeout_ms=180_000, + on_qrcode=on_qrcode, + ) + session['status'] = 'success' + session['token'] = result.token + session['base_url'] = result.base_url + session['account_id'] = result.account_id + except Exception as e: + error_message = str(e) + if 'expired' in error_message.lower() or 'max retries exceeded' in error_message.lower(): + session['status'] = 'expired' + session['error'] = 'QR code expired' + else: + session['status'] = 'error' + session['error'] = error_message + finally: + await client.close() + + task = _start_adapter_session_task( + self.ap, + run_login(), + adapter='weixin', + session_id=session_id, + request_context=request_context, + ) + if task is None: + _weixin_login_sessions.pop(session_id, None) + raise CredentialSetupError(429, 'Too many active credential exchanges') + session['task'] = task + + # Wait for QR code to be ready (max 10 seconds) + for _ in range(20): + if session['qr_data_url']: + break + await asyncio.sleep(0.5) + + if not session['qr_data_url']: + task.cancel() + session['status'] = 'error' + session['error'] = 'Timeout waiting for QR code' + raise CredentialSetupError(504, 'Timeout waiting for QR code') + + return { + 'session_id': session_id, + 'qr_data_url': session['qr_data_url'], + 'expire_at': session['expire_at'], + } + + async def weixin_status(session_id: str, request_context: RequestContext) -> dict: + """Poll WeChat login status.""" + _cleanup_expired_weixin_sessions() + session = _get_owned_session(_weixin_login_sessions, session_id, request_context) + if not session: + raise CredentialSetupError(404, 'Session not found') + + data = { + 'status': session['status'], + 'qr_data_url': session['qr_data_url'], + 'expire_at': session['expire_at'], + } + + if session['status'] == 'success': + data['token'] = session['token'] + data['base_url'] = session['base_url'] + data['account_id'] = session['account_id'] + _weixin_login_sessions.pop(session_id, None) + elif session['status'] == 'error': + data['error'] = session['error'] + _weixin_login_sessions.pop(session_id, None) + elif session['status'] == 'expired': + data['error'] = session['error'] + _weixin_login_sessions.pop(session_id, None) + + return data + + async def weixin_cancel(session_id: str, request_context: RequestContext) -> dict: + """Cancel and clean up a WeChat login session.""" + session = _pop_owned_session(_weixin_login_sessions, session_id, request_context) + if session is None: + raise CredentialSetupError(404, 'Session not found') + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + return {} + + # ----------------------------------------------------------------------- + # DingTalk Device Flow QR Code Login + # ----------------------------------------------------------------------- + + _dingtalk_sessions: dict = {} + _DINGTALK_SESSION_TTL = 600 # 10 minutes (QR code validity window) + + def _cleanup_expired_dingtalk_sessions(): + import time + + now = time.time() + expired = [ + sid for sid, s in _dingtalk_sessions.items() if now - s.get('created_at', 0) > _DINGTALK_SESSION_TTL + ] + for sid in expired: + session = _dingtalk_sessions.pop(sid, None) + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + + async def dingtalk_start(request_context: RequestContext) -> dict: + """Start DingTalk one-click app creation via Device Flow. Returns session_id + QR code URL.""" + import uuid + import time + import aiohttp + + DINGTALK_BASE_URL = 'https://oapi.dingtalk.com' + + _cleanup_expired_dingtalk_sessions() + + session_id = str(uuid.uuid4()) + + session = { + 'status': 'pending', + 'qr_url': None, + 'expire_at': None, + 'client_id': None, + 'client_secret': None, + 'error': None, + 'created_at': time.time(), + 'device_code': None, + 'interval': 5, + } + _bind_session_scope(session, request_context) + _make_room_for_session(_dingtalk_sessions, request_context) + _dingtalk_sessions[session_id] = session + + async def run_device_flow(): + try: + timeout = aiohttp.ClientTimeout(total=10) + async with aiohttp.ClientSession(timeout=timeout) as http: + # Step 1: Init — get nonce + async with http.post( + f'{DINGTALK_BASE_URL}/app/registration/init', + json={'source': 'langbot'}, + ) as resp: + try: + data = await httpclient.read_json_limited(resp) + except (aiohttp.ContentTypeError, ValueError): + session['status'] = 'error' + session['error'] = 'Invalid response from DingTalk service' + return + if data.get('errcode', -1) != 0: + session['status'] = 'error' + session['error'] = data.get('errmsg', 'Failed to init') + return + nonce = data['nonce'] + + # Step 2: Begin — get device_code + QR URL + async with http.post( + f'{DINGTALK_BASE_URL}/app/registration/begin', + json={'nonce': nonce}, + ) as resp: + try: + data = await httpclient.read_json_limited(resp) + except (aiohttp.ContentTypeError, ValueError): + session['status'] = 'error' + session['error'] = 'Invalid response from DingTalk service' + return + if data.get('errcode', -1) != 0: + session['status'] = 'error' + session['error'] = data.get('errmsg', 'Failed to begin authorization') + return + + device_code = data['device_code'] + verification_uri_complete = data.get('verification_uri_complete', '') + expires_in = data.get('expires_in', 7200) + interval = data.get('interval', 5) + + session['device_code'] = device_code + session['interval'] = interval + session['qr_url'] = verification_uri_complete + session['expire_at'] = time.time() + 600 # QR code valid for ~10 min + session['status'] = 'waiting' + + # Step 3: Poll for authorization result + deadline = time.time() + expires_in + while time.time() < deadline: + await asyncio.sleep(interval) + + async with http.post( + f'{DINGTALK_BASE_URL}/app/registration/poll', + json={'device_code': device_code}, + ) as poll_resp: + try: + poll_data = await httpclient.read_json_limited(poll_resp) + except (aiohttp.ContentTypeError, ValueError): + continue + + if poll_data.get('errcode', -1) != 0: + session['status'] = 'error' + session['error'] = poll_data.get('errmsg', 'Poll failed') + return + + status = poll_data.get('status', '') + + if status == 'SUCCESS': + session['status'] = 'success' + session['client_id'] = poll_data.get('client_id', '') + session['client_secret'] = poll_data.get('client_secret', '') + return + elif status == 'FAIL': + session['status'] = 'error' + session['error'] = poll_data.get('fail_reason', 'Authorization failed') + return + elif status == 'EXPIRED': + session['status'] = 'error' + session['error'] = 'QR code expired' + return + # status == 'WAITING': continue polling + + # Timeout + session['status'] = 'error' + session['error'] = 'QR code expired' + + except asyncio.CancelledError: + return + except Exception as e: + session['status'] = 'error' + session['error'] = str(e) + + task = _start_adapter_session_task( + self.ap, + run_device_flow(), + adapter='dingtalk', + session_id=session_id, + request_context=request_context, + ) + if task is None: + _dingtalk_sessions.pop(session_id, None) + raise CredentialSetupError(429, 'Too many active credential exchanges') + session['task'] = task + + # Wait for QR code to be ready (max 10 seconds) + for _ in range(20): + if session['qr_url'] or session['error']: + break + await asyncio.sleep(0.5) + + if session['error']: + task.cancel() + raise CredentialSetupError(502, session['error']) + + if not session['qr_url']: + task.cancel() + session['status'] = 'error' + session['error'] = 'Timeout waiting for QR code' + raise CredentialSetupError(504, 'Timeout waiting for QR code') + + return { + 'session_id': session_id, + 'qr_url': session['qr_url'], + 'expire_at': session['expire_at'], + } + + async def dingtalk_status(session_id: str, request_context: RequestContext) -> dict: + """Poll DingTalk Device Flow status.""" + _cleanup_expired_dingtalk_sessions() + session = _get_owned_session(_dingtalk_sessions, session_id, request_context) + if not session: + raise CredentialSetupError(404, 'Session not found') + + data = {'status': session['status']} + + if session['status'] == 'success': + data['client_id'] = session['client_id'] + data['client_secret'] = session['client_secret'] + _dingtalk_sessions.pop(session_id, None) + elif session['status'] == 'error': + data['error'] = session['error'] + _dingtalk_sessions.pop(session_id, None) + + return data + + async def dingtalk_cancel(session_id: str, request_context: RequestContext) -> dict: + """Cancel and clean up a DingTalk Device Flow session.""" + session = _pop_owned_session(_dingtalk_sessions, session_id, request_context) + if session is None: + raise CredentialSetupError(404, 'Session not found') + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + return {} + + # ----------------------------------------------------------------------- + # WeComBot QR Code One-Click Create + # ----------------------------------------------------------------------- + + _wecombot_sessions: dict = {} + _WECOMBOT_SESSION_TTL = 300 # 5 minutes (WeCom QR validity window) + + def _cleanup_expired_wecombot_sessions(): + import time + + now = time.time() + expired = [ + sid for sid, s in _wecombot_sessions.items() if now - s.get('created_at', 0) > _WECOMBOT_SESSION_TTL + ] + for sid in expired: + session = _wecombot_sessions.pop(sid, None) + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + + async def wecombot_start(request_context: RequestContext) -> dict: + """Start WeComBot one-click creation via QR code. Returns session_id + QR code URL.""" + import uuid + import time + import aiohttp + + WECOM_QC_GENERATE_URL = 'https://work.weixin.qq.com/ai/qc/generate' + WECOM_QC_QUERY_URL = 'https://work.weixin.qq.com/ai/qc/query_result' + + _cleanup_expired_wecombot_sessions() + + session_id = str(uuid.uuid4()) + + session = { + 'status': 'pending', + 'qr_url': None, + 'expire_at': None, + 'botid': None, + 'secret': None, + 'error': None, + 'created_at': time.time(), + 'scode': None, + 'task': None, + } + _bind_session_scope(session, request_context) + _make_room_for_session(_wecombot_sessions, request_context) + _wecombot_sessions[session_id] = session + + async def run_qr_flow(): + try: + timeout = aiohttp.ClientTimeout(total=10) + async with aiohttp.ClientSession(timeout=timeout) as http: + # Step 1: Generate QR code + async with http.get( + f'{WECOM_QC_GENERATE_URL}?source=langbot&plat=0', + ) as resp: + try: + data = await httpclient.read_json_limited(resp) + except (aiohttp.ContentTypeError, ValueError): + session['status'] = 'error' + session['error'] = 'Invalid response from WeCom service' + return + if not data.get('data', {}).get('scode') or not data.get('data', {}).get('auth_url'): + session['status'] = 'error' + session['error'] = data.get('errmsg', 'Failed to generate QR code') + return + + scode = data['data']['scode'] + auth_url = data['data']['auth_url'] + + session['scode'] = scode + session['qr_url'] = auth_url + session['expire_at'] = time.time() + _WECOMBOT_SESSION_TTL + session['status'] = 'waiting' + + # Step 2: Poll for scan result + deadline = time.time() + _WECOMBOT_SESSION_TTL + while time.time() < deadline: + await asyncio.sleep(3) + + async with http.get( + f'{WECOM_QC_QUERY_URL}?scode={scode}', + ) as poll_resp: + try: + poll_data = await httpclient.read_json_limited(poll_resp) + except (aiohttp.ContentTypeError, ValueError): + continue + + status = poll_data.get('data', {}).get('status', '') + if status == 'success': + bot_info = poll_data.get('data', {}).get('bot_info', {}) + if bot_info.get('botid') and bot_info.get('secret'): + session['status'] = 'success' + session['botid'] = bot_info['botid'] + session['secret'] = bot_info['secret'] + return + else: + session['status'] = 'error' + session['error'] = 'Scan succeeded but bot info is incomplete' + return + + # Timeout + session['status'] = 'error' + session['error'] = 'QR code expired' + + except asyncio.CancelledError: + return + except Exception as e: + session['status'] = 'error' + session['error'] = str(e) + + task = _start_adapter_session_task( + self.ap, + run_qr_flow(), + adapter='wecombot', + session_id=session_id, + request_context=request_context, + ) + if task is None: + _wecombot_sessions.pop(session_id, None) + raise CredentialSetupError(429, 'Too many active credential exchanges') + session['task'] = task + + # Wait for QR code to be ready (max 10 seconds) + for _ in range(20): + if session['qr_url'] or session['error']: + break + await asyncio.sleep(0.5) + + if session['error']: + task.cancel() + raise CredentialSetupError(502, session['error']) + + if not session['qr_url']: + task.cancel() + session['status'] = 'error' + session['error'] = 'Timeout waiting for QR code' + raise CredentialSetupError(504, 'Timeout waiting for QR code') + + return { + 'session_id': session_id, + 'qr_url': session['qr_url'], + 'expire_at': session['expire_at'], + } + + async def wecombot_status(session_id: str, request_context: RequestContext) -> dict: + """Poll WeComBot creation status.""" + _cleanup_expired_wecombot_sessions() + session = _get_owned_session(_wecombot_sessions, session_id, request_context) + if not session: + raise CredentialSetupError(404, 'Session not found') + + data = {'status': session['status']} + + if session['status'] == 'success': + data['botid'] = session['botid'] + data['secret'] = session['secret'] + _wecombot_sessions.pop(session_id, None) + elif session['status'] == 'error': + data['error'] = session['error'] + _wecombot_sessions.pop(session_id, None) + + return data + + async def wecombot_cancel(session_id: str, request_context: RequestContext) -> dict: + """Cancel and clean up a WeComBot creation session.""" + session = _pop_owned_session(_wecombot_sessions, session_id, request_context) + if session is None: + raise CredentialSetupError(404, 'Session not found') + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + return {} + + # ----------------------------------------------------------------------- + # QQ Official QR Binding + # ----------------------------------------------------------------------- + + _qqofficial_sessions: dict = {} + _QQOFFICIAL_SESSION_TTL = 300 # 5 minutes (QQ bind QR validity window) + + def _cleanup_expired_qqofficial_sessions(): + import time + + now = time.time() + expired = [ + sid for sid, s in _qqofficial_sessions.items() if now - s.get('created_at', 0) > _QQOFFICIAL_SESSION_TTL + ] + for sid in expired: + session = _qqofficial_sessions.pop(sid, None) + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + + async def qqofficial_start(request_context: RequestContext) -> dict: + """Start QQ Official QR binding. Returns session_id + QR URL. + + Flow: generate a local AES-256 key, register it with + `q.qq.com/lite/create_bind_task`, then poll + `q.qq.com/lite/poll_bind_result` until the user authorizes the + bind inside the QQ Bot Assistant on mobile QQ. The encrypted + AppSecret returned by the poll endpoint is decrypted with the + same key. The key never leaves this process. + """ + import uuid + import time + import secrets + import base64 + import aiohttp + + QQ_BIND_BASE = 'https://q.qq.com' + _cleanup_expired_qqofficial_sessions() + + bind_key_bytes = secrets.token_bytes(32) + bind_key = base64.b64encode(bind_key_bytes).decode('ascii') + + session_id = str(uuid.uuid4()) + session = { + 'status': 'pending', + 'qr_url': None, + 'expire_at': None, + 'appid': None, + 'secret': None, + 'user_openid': None, + 'error': None, + 'created_at': time.time(), + 'task_id': None, + 'bind_key_bytes': bind_key_bytes, + 'interval': 2, + } + _bind_session_scope(session, request_context) + _make_room_for_session(_qqofficial_sessions, request_context) + _qqofficial_sessions[session_id] = session + + async def run_qr_binding(): + try: + timeout = aiohttp.ClientTimeout(total=10) + async with aiohttp.ClientSession(timeout=timeout) as http: + # Step 1: create_bind_task — register our AES key, get task_id + async with http.post( + f'{QQ_BIND_BASE}/lite/create_bind_task', + json={'key': bind_key}, + headers={'Accept': 'application/json'}, + ) as resp: + try: + data = await httpclient.read_json_limited(resp) + except (aiohttp.ContentTypeError, ValueError): + session['status'] = 'error' + session['error'] = 'Invalid response from QQ bind service' + return + if int(data.get('retcode', -1)) != 0: + session['status'] = 'error' + session['error'] = ( + data.get('msg') or data.get('message') or 'Failed to create bind task' + ) + return + task_id = str((data.get('data') or {}).get('task_id') or '').strip() + if not task_id: + session['status'] = 'error' + session['error'] = 'Missing task_id in QQ response' + return + + # The QR encodes a URL that mobile QQ opens inside the QQ Bot Assistant. + # `source=langbot` is a courtesy attribution parameter so Tencent + # can see LangBot adoption metrics, matching the convention used by + # other third-party integrations (e.g. hermes-agent uses `source=hermes`). + qr_url = f'{QQ_BIND_BASE}/qqbot/openclaw/connect.html?task_id={task_id}&_wv=2&source=langbot' + session['task_id'] = task_id + session['qr_url'] = qr_url + session['expire_at'] = time.time() + _QQOFFICIAL_SESSION_TTL + session['status'] = 'waiting' + + # Step 2: poll_bind_result until completed (status=2) or expired (3). + deadline = time.time() + _QQOFFICIAL_SESSION_TTL + while time.time() < deadline: + await asyncio.sleep(session['interval']) + + async with http.post( + f'{QQ_BIND_BASE}/lite/poll_bind_result', + json={'task_id': task_id}, + headers={'Accept': 'application/json'}, + ) as poll_resp: + try: + poll_data = await httpclient.read_json_limited(poll_resp) + except (aiohttp.ContentTypeError, ValueError): + continue + + if int(poll_data.get('retcode', -1)) != 0: + session['status'] = 'error' + session['error'] = poll_data.get('msg') or poll_data.get('message') or 'Poll failed' + return + + payload = poll_data.get('data') or {} + try: + raw_status = int(payload.get('status', 0)) + except (TypeError, ValueError): + raw_status = 0 + + if raw_status == 2: + appid = str(payload.get('bot_appid') or '').strip() + encrypted = str(payload.get('bot_encrypt_secret') or '').strip() + if not appid or not encrypted: + session['status'] = 'error' + session['error'] = 'Incomplete credential payload' + return + try: + session['secret'] = _decrypt_qqofficial_secret( + encrypted, + bind_key_bytes, + ) + except ValueError as exc: + session['status'] = 'error' + session['error'] = str(exc) + return + session['appid'] = appid + # The scanner's OpenID is returned alongside the credentials — + # surfaced to the dashboard for audit / "bound by" display. + session['user_openid'] = str(payload.get('user_openid') or '').strip() or None + session['status'] = 'success' + return + + if raw_status == 3: + session['status'] = 'expired' + session['error'] = 'QR code expired' + return + # status 0 / 1: still pending, continue polling + + session['status'] = 'expired' + session['error'] = 'QR code expired' + + except asyncio.CancelledError: + return + except Exception as e: + session['status'] = 'error' + session['error'] = str(e) + + task = _start_adapter_session_task( + self.ap, + run_qr_binding(), + adapter='qqofficial', + session_id=session_id, + request_context=request_context, + ) + if task is None: + _qqofficial_sessions.pop(session_id, None) + raise CredentialSetupError(429, 'Too many active credential exchanges') + session['task'] = task + + # Wait up to 10s for the QR URL to be ready before responding. + for _ in range(20): + if session['qr_url'] or session['error']: + break + await asyncio.sleep(0.5) + + if session['error']: + task.cancel() + raise CredentialSetupError(502, session['error']) + + if not session['qr_url']: + task.cancel() + session['status'] = 'error' + session['error'] = 'Timeout waiting for QR code' + raise CredentialSetupError(504, 'Timeout waiting for QR code') + + return { + 'session_id': session_id, + 'qr_url': session['qr_url'], + 'expire_at': session['expire_at'], + } + + async def qqofficial_status(session_id: str, request_context: RequestContext) -> dict: + """Poll QQ Official QR binding status.""" + _cleanup_expired_qqofficial_sessions() + session = _get_owned_session(_qqofficial_sessions, session_id, request_context) + if not session: + raise CredentialSetupError(404, 'Session not found') + + data = {'status': session['status']} + + if session['status'] == 'success': + data['appid'] = session['appid'] + data['secret'] = session['secret'] + if session.get('user_openid'): + data['user_openid'] = session['user_openid'] + _qqofficial_sessions.pop(session_id, None) + elif session['status'] in ('error', 'expired'): + data['error'] = session['error'] + _qqofficial_sessions.pop(session_id, None) + + return data + + async def qqofficial_cancel(session_id: str, request_context: RequestContext) -> dict: + """Cancel and clean up a QQ Official QR binding session.""" + session = _pop_owned_session(_qqofficial_sessions, session_id, request_context) + if session is None: + raise CredentialSetupError(404, 'Session not found') + if session and session.get('task') and not session['task'].done(): + session['task'].cancel() + return {} + + # ---- credential-session TTL enforcement ---- + # + # Every store above holds a short-lived secret (app_secret, + # client_secret, bot secret) between the platform handshake and the + # caller persisting it. The per-route cleanups only run when another + # request arrives, so an abandoned flow would keep its secret in memory + # long past its validity window. This sweeper enforces the TTL on a + # timer instead, and scrubs the secret fields as it evicts. + + def _credential_stores() -> list[tuple[dict, float]]: + return [ + (_create_app_sessions, _SESSION_TTL), + (_dingtalk_sessions, _DINGTALK_SESSION_TTL), + (_wecombot_sessions, _WECOMBOT_SESSION_TTL), + (_qqofficial_sessions, _QQOFFICIAL_SESSION_TTL), + ] + + def _sweep_now() -> int: + return _sweep_credential_sessions(_credential_stores()) + + async def _credential_session_sweeper() -> None: + """Enforce the session TTL on a timer, so it does not depend on traffic.""" + import asyncio as _asyncio + + while True: + await _asyncio.sleep(_CREDENTIAL_SWEEP_INTERVAL_SECONDS) + try: + _sweep_now() + except Exception: # pragma: no cover - a sweep must never kill the app + logging.getLogger(__name__).exception('credential session sweep failed') + + self._sweep_credential_sessions = _sweep_now + # Some embeddings (notably test harnesses) provide a task manager without + # create_task; the per-route cleanups still enforce the TTL there. + create_task = getattr(self.ap.task_mgr, 'create_task', None) + if callable(create_task): + create_task( + _credential_session_sweeper(), + name='platform-credential-session-sweeper', + ) + + self.lark_start = lark_start + self.lark_status = lark_status + self.lark_cancel = lark_cancel + self.weixin_start = weixin_start + self.weixin_status = weixin_status + self.weixin_cancel = weixin_cancel + self.dingtalk_start = dingtalk_start + self.dingtalk_status = dingtalk_status + self.dingtalk_cancel = dingtalk_cancel + self.wecombot_start = wecombot_start + self.wecombot_status = wecombot_status + self.wecombot_cancel = wecombot_cancel + self.qqofficial_start = qqofficial_start + self.qqofficial_status = qqofficial_status + self.qqofficial_cancel = qqofficial_cancel + + +def get_credential_setup(ap) -> CredentialSetupService: + service = getattr(ap, 'credential_setup_service', None) + if service is None: + service = CredentialSetupService(ap) + ap.credential_setup_service = service + return service diff --git a/src/langbot/pkg/plugin/management.py b/src/langbot/pkg/plugin/management.py new file mode 100644 index 000000000..dbd1791e0 --- /dev/null +++ b/src/langbot/pkg/plugin/management.py @@ -0,0 +1,240 @@ +"""Shared plugin management validation and secret handling.""" + +from __future__ import annotations + +import copy +import posixpath +import re +from urllib.parse import urlparse + + +_SECRET_MASK = '***' +_MISSING_SECRET = object() +_SENSITIVE_CONFIG_NAMES = frozenset( + { + 'api_key', + 'apikey', + 'auth', + 'authorization', + 'cookie', + 'credentials', + 'database_url', + 'dsn', + 'key', + 'proxy_authorization', + 'set_cookie', + } +) +_SENSITIVE_CONFIG_TOKENS = frozenset( + { + 'credential', + 'credentials', + 'passwd', + 'password', + 'secret', + 'token', + } +) +_SENSITIVE_KEY_QUALIFIERS = frozenset( + { + 'access', + 'api', + 'auth', + 'bearer', + 'client', + 'debug', + 'encryption', + 'private', + 'signing', + } +) + + +def _normalize_config_key(key: object) -> str: + value = re.sub(r'([a-z0-9])([A-Z])', r'\1_\2', str(key or '')) + return re.sub(r'[^a-zA-Z0-9]+', '_', value).strip('_').lower() + + +def _is_sensitive_config_key(key: object) -> bool: + normalized = _normalize_config_key(key) + if normalized in _SENSITIVE_CONFIG_NAMES: + return True + tokens = frozenset(token for token in normalized.split('_') if token) + if tokens & _SENSITIVE_CONFIG_TOKENS: + return True + return 'key' in tokens and bool(tokens & _SENSITIVE_KEY_QUALIFIERS) + + +def _mask_secret_structure(value): + """Mask every non-empty leaf while preserving container structure.""" + + if isinstance(value, dict): + return {key: _mask_secret_structure(item) for key, item in value.items()} + if isinstance(value, list): + return [_mask_secret_structure(item) for item in value] + if isinstance(value, tuple): + return tuple(_mask_secret_structure(item) for item in value) + if value is None or value == '': + return value + return _SECRET_MASK + + +def redact_plugin_secrets(value): + """Return a recursively redacted copy of plugin-facing data.""" + + if isinstance(value, dict): + return { + key: (_mask_secret_structure(item) if _is_sensitive_config_key(key) else redact_plugin_secrets(item)) + for key, item in value.items() + } + if isinstance(value, list): + return [redact_plugin_secrets(item) for item in value] + if isinstance(value, tuple): + return tuple(redact_plugin_secrets(item) for item in value) + return value + + +def restore_plugin_secret_placeholders(value, current_value=_MISSING_SECRET, *, sensitive: bool = False): + """Restore masked leaves from the current config before a management write.""" + + if sensitive and value == _SECRET_MASK: + if current_value is _MISSING_SECRET: + raise ValueError('Masked plugin secret has no existing value') + return copy.deepcopy(current_value) + if isinstance(value, dict): + current_mapping = current_value if isinstance(current_value, dict) else {} + return { + key: restore_plugin_secret_placeholders( + item, + current_mapping.get(key, _MISSING_SECRET), + sensitive=sensitive or _is_sensitive_config_key(key), + ) + for key, item in value.items() + } + if isinstance(value, list): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return [ + restore_plugin_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + sensitive=sensitive, + ) + for index, item in enumerate(value) + ] + if isinstance(value, tuple): + current_items = current_value if isinstance(current_value, (list, tuple)) else () + return tuple( + restore_plugin_secret_placeholders( + item, + current_items[index] if index < len(current_items) else _MISSING_SECRET, + sensitive=sensitive, + ) + for index, item in enumerate(value) + ) + return value + + +def normalize_plugin_asset_path(filepath: str) -> str | None: + filepath = filepath.replace('\\', '/') + if filepath.startswith('/'): + return None + + normalized = posixpath.normpath(filepath) + if normalized == '.' or normalized.startswith('../') or normalized == '..': + return None + + if normalized.startswith('components/pages/'): + return normalized + + return f'assets/{normalized}' + + +def _normalize_archive_path(path: str) -> str: + normalized = str(path or '').replace('\\', '/').strip('/') + return posixpath.normpath(normalized) if normalized else '' + + +def _component_source_path(entry) -> str: + if isinstance(entry, dict): + return _normalize_archive_path(entry.get('path') or '') + return _normalize_archive_path(str(entry or '')) + + +def _count_component_configs(component_config, archive_names: list[str]) -> int: + normalized_names = [_normalize_archive_path(name) for name in archive_names] + component_files: set[str] = set() + + if isinstance(component_config, list): + return len(component_config) + if not isinstance(component_config, dict): + return 1 if component_config else 0 + + for entry in component_config.get('fromFiles') or []: + source_path = _component_source_path(entry) + if source_path and source_path in normalized_names: + component_files.add(source_path) + + for entry in component_config.get('fromDirs') or []: + source_dir = _component_source_path(entry).rstrip('/') + if not source_dir: + continue + prefix = f'{source_dir}/' + for archive_name in normalized_names: + if not archive_name.startswith(prefix): + continue + if archive_name.lower().endswith(('.yaml', '.yml')): + component_files.add(archive_name) + + if component_files: + return len(component_files) + + return 1 if any(key in component_config for key in ('path', 'name', 'kind')) else 0 + + +def count_plugin_components(components, archive_names: list[str]) -> dict[str, int]: + if not isinstance(components, dict): + return {} + + component_counts: dict[str, int] = {} + for kind, component_config in components.items(): + count = _count_component_configs(component_config, archive_names) + if count > 0: + component_counts[str(kind)] = count + return component_counts + + +def parse_github_repo_url(repo_url: str) -> dict | None: + raw_url = str(repo_url or '').strip() + if not raw_url: + return None + + if not re.match(r'^[a-zA-Z][a-zA-Z0-9+.-]*://', raw_url): + raw_url = f'https://{raw_url}' + + parsed = urlparse(raw_url) + if parsed.netloc.lower() not in ('github.com', 'www.github.com'): + return None + + parts = [part for part in parsed.path.strip('/').split('/') if part] + if len(parts) < 2: + return None + + owner = parts[0] + repo = parts[1] + if repo.endswith('.git'): + repo = repo[:-4] + if not owner or not repo: + return None + + ref = '' + subdir = '' + if len(parts) >= 4 and parts[2] in ('tree', 'blob'): + ref = parts[3] + subdir = '/'.join(parts[4:]).strip('/') + + return { + 'owner': owner, + 'repo': repo, + 'ref': ref, + 'subdir': subdir, + } diff --git a/tests/integration/api/test_application_foundation.py b/tests/integration/api/test_application_foundation.py new file mode 100644 index 000000000..c67217064 --- /dev/null +++ b/tests/integration/api/test_application_foundation.py @@ -0,0 +1,177 @@ +"""Exercise the foundation without HTTP, with independent SQLite readback.""" + +import dataclasses +import hashlib +import logging +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import sqlalchemy as sa +from sqlalchemy.ext.asyncio import create_async_engine + +from langbot.pkg.api.application import ApplicationAPI, ApplicationAPIException, Input, Operation +from langbot.pkg.api.authz import Permission +from langbot.pkg.api.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext +from langbot.pkg.api.http.service.apikey import ApiKeyService +from langbot.pkg.api.http.service.webhook import WebhookService +from langbot.pkg.entity.persistence.apikey import ApiKey +from langbot.pkg.entity.persistence.base import Base +from langbot.pkg.entity.persistence.webhook import Webhook +from langbot.pkg.entity.persistence.workspace import Workspace, WorkspaceExecutionState +from langbot.pkg.persistence.mgr import PersistenceManager +from langbot.pkg.workspace.service import WorkspaceService + + +def context(workspace='workspace-a', permissions=frozenset(Permission), generation=1): + return RequestContext( + instance_uuid='foundation-instance', + placement_generation=generation, + request_id='test-request', + auth_type='api-key', + principal=PrincipalContext(PrincipalType.API_KEY, api_key_uuid='test-key'), + workspace=WorkspaceContext(workspace, None, None, frozenset(permissions)), + ) + + +@pytest.fixture +async def foundation(tmp_path): + engine = create_async_engine(f'sqlite+aiosqlite:///{tmp_path / "foundation.db"}') + try: + async with engine.begin() as connection: + await connection.run_sync(Base.metadata.create_all) + await connection.execute( + sa.insert(Workspace), + [ + { + 'uuid': name, + 'instance_uuid': 'foundation-instance', + 'name': name, + 'slug': name, + 'source': 'cloud_projection', + } + for name in ('workspace-a', 'workspace-b') + ], + ) + await connection.execute( + sa.insert(WorkspaceExecutionState), + [ + { + 'workspace_uuid': name, + 'instance_uuid': 'foundation-instance', + 'active_generation': 1, + 'source': 'cloud', + } + for name in ('workspace-a', 'workspace-b') + ], + ) + ap = SimpleNamespace(instance_config=SimpleNamespace(data={}), logger=logging.getLogger('foundation')) + ap.persistence_mgr = PersistenceManager(ap) + ap.persistence_mgr.db = SimpleNamespace(get_engine=lambda: engine) + ap.workspace_service = WorkspaceService(ap, instance_uuid='foundation-instance') + ap.webhook_service = WebhookService(ap) + ap.apikey_service = ApiKeyService(ap) + api = ApplicationAPI(ap) + yield api, ap, engine + finally: + await engine.dispose() + + +@pytest.mark.asyncio +async def test_webhook_lifecycle_secrets_and_workspace_isolation(foundation): + api, _, engine = foundation + created = await api.execute( + 'webhook.create', context(), {'name': 'same-name', 'url': 'https://example.test/?token=private'} + ) + other = await api.execute( + 'webhook.create', context('workspace-b'), {'name': 'same-name', 'url': 'https://other.test/'} + ) + assert created['id'] != other['id'] + viewer = context(permissions={Permission.RESOURCE_VIEW}) + assert (await api.execute('webhook.list', viewer, {}))[0]['url'] == '***' + with pytest.raises(ApplicationAPIException) as denied: + await api.execute('webhook.get', viewer, {'webhook_id': created['id'], 'include_secret': True}) + assert denied.value.error.code == 'permission_denied' + with pytest.raises(ApplicationAPIException) as missing: + await api.execute('webhook.get', context('workspace-b'), {'webhook_id': created['id']}) + assert missing.value.error.code == 'resource_not_found' + await api.execute('webhook.update', context(), {'webhook_id': created['id'], 'name': 'renamed', 'url': '***'}) + async with engine.connect() as connection: + row = (await connection.execute(sa.select(Webhook).where(Webhook.id == created['id']))).mappings().one() + assert row['workspace_uuid'] == 'workspace-a' + assert row['name'] == 'renamed' + assert row['url'] == 'https://example.test/?token=private' + assert await api.execute('webhook.delete', context(), {'webhook_id': created['id']}) is True + assert await api.execute('webhook.list', context(), {}) == [] + assert len(await api.execute('webhook.list', context('workspace-b'), {})) == 1 + + +@pytest.mark.asyncio +async def test_key_grants_are_persisted_without_secret_or_escalation(foundation): + api, _, engine = foundation + limited = context(permissions={Permission.API_KEY_MANAGE, Permission.RESOURCE_VIEW}) + with pytest.raises(ApplicationAPIException) as error: + await api.execute('api_key.create', limited, {'name': 'escalation', 'scopes': ['resource.manage']}) + assert error.value.error.code == 'permission_denied' + created = await api.execute('api_key.create', limited, {'name': 'reader', 'scopes': ['resource.view']}) + empty = await api.execute('api_key.create', limited, {'name': 'empty', 'scopes': []}) + async with engine.connect() as connection: + rows = (await connection.execute(sa.select(ApiKey))).mappings().all() + assert len(rows) == 2 + assert rows[0]['key_hash'] == hashlib.sha256(created['key'].encode()).hexdigest() + assert rows[0]['scopes'] == ['resource.view'] + assert rows[1]['scopes'] == [] + listed = await api.execute('api_key.list', limited, {}) + assert all('key' not in row and 'key_hash' not in row for row in listed) + assert empty['key'] != created['key'] + + +@pytest.mark.asyncio +async def test_untrusted_fields_stale_placement_and_wrong_instance_fail_before_write(foundation): + api, _, engine = foundation + for payload in ( + {'name': 'bad', 'url': 'secret://private', 'workspace_uuid': 'workspace-b'}, + {'name': 'bad', 'url': 'secret://private', 'enabled': 'false'}, + ): + with pytest.raises(ApplicationAPIException) as error: + await api.execute('webhook.create', context(), payload) + assert error.value.error.code == 'invalid_input' + assert 'secret://private' not in str(error.value) + with pytest.raises(ApplicationAPIException) as stale: + await api.execute('webhook.create', context(generation=2), {'name': 'stale', 'url': 'https://example.test/'}) + assert stale.value.error.code == 'workspace_unavailable' + with pytest.raises(ApplicationAPIException) as wrong_instance: + await api.execute('webhook.list', dataclasses.replace(context(), instance_uuid='other-instance'), {}) + assert wrong_instance.value.error.code == 'invalid_authentication' + async with engine.connect() as connection: + assert (await connection.execute(sa.select(sa.func.count()).select_from(Webhook))).scalar_one() == 0 + + +@pytest.mark.asyncio +async def test_every_registered_operation_denies_missing_permission(): + api = ApplicationAPI(SimpleNamespace()) + for spec in api._operations.values(): + principal = PrincipalContext(PrincipalType.ACCOUNT, account_uuid='user') + denied_context = dataclasses.replace(context(permissions=set()), principal=principal) + with pytest.raises(ApplicationAPIException) as error: + if spec.streaming: + await anext(api.stream(spec.operation_id, denied_context, {})) + else: + await api.execute(spec.operation_id, denied_context, {}) + assert error.value.error.code == 'permission_denied', spec.operation_id + spec.as_dict() + + +@pytest.mark.asyncio +async def test_invalid_output_fails_and_cancellation_propagates(foundation): + import asyncio + + _, ap, _ = foundation + handler = AsyncMock(return_value='not-an-object') + api = ApplicationAPI(ap, operations=[Operation('example', Permission.RESOURCE_VIEW, handler, Input, dict)]) + with pytest.raises(ApplicationAPIException) as error: + await api.execute('example', context(), {}) + assert error.value.error.code == 'invalid_output' + handler.side_effect = asyncio.CancelledError() + with pytest.raises(asyncio.CancelledError): + await api.execute('example', context(), {}) diff --git a/tests/integration/api/test_application_resources.py b/tests/integration/api/test_application_resources.py new file mode 100644 index 000000000..580ef592b --- /dev/null +++ b/tests/integration/api/test_application_resources.py @@ -0,0 +1,257 @@ +"""Real resource services and SQLite; only external runtime caches are stubbed.""" + +from types import SimpleNamespace +from unittest.mock import AsyncMock + +import pytest +import sqlalchemy as sa + +from langbot.pkg.api.application import ApplicationAPIException +from langbot.pkg.api.authz import Permission +from langbot.pkg.api.http.service.knowledge import KnowledgeService +from langbot.pkg.api.http.service.model import EmbeddingModelsService, LLMModelsService, RerankModelsService +from langbot.pkg.api.http.service.pipeline import PipelineService +from langbot.pkg.api.http.service.provider import ModelProviderService +from langbot.pkg.entity.persistence.model import EmbeddingModel, LLMModel, ModelProvider, RerankModel +from langbot.pkg.entity.persistence.pipeline import LegacyPipeline +from langbot.pkg.entity.persistence.rag import KnowledgeBase +from langbot.pkg.rag.knowledge.kbmgr import RAGManager +from tests.integration.api.test_application_foundation import context, foundation # noqa: F401 + + +@pytest.fixture +async def resources(foundation): # noqa: F811 - pytest resolves the imported fixture by name. + api, ap, engine = foundation + runtime_provider = SimpleNamespace(provider_entity=SimpleNamespace(requester='test-requester')) + ap.model_mgr = SimpleNamespace( + load_provider=AsyncMock(return_value=runtime_provider), + cache_provider=AsyncMock(), + reload_provider=AsyncMock(), + get_provider_by_uuid=AsyncMock(return_value=runtime_provider), + get_available_requester_manifest_by_name=lambda name: SimpleNamespace( + spec={'support_type': ['llm', 'text-embedding', 'rerank']} + ), + llm_model_dict={}, + ) + for family in ('llm', 'embedding', 'rerank'): + setattr(ap.model_mgr, f'load_{family}_model_with_provider', AsyncMock()) + setattr(ap.model_mgr, f'cache_{family}_model', AsyncMock()) + setattr(ap.model_mgr, f'remove_{family}_model', AsyncMock()) + ap.provider_service = ModelProviderService(ap) + ap.llm_model_service = LLMModelsService(ap) + ap.embedding_models_service = EmbeddingModelsService(ap) + ap.rerank_models_service = RerankModelsService(ap) + ap.pipeline_service = PipelineService(ap) + ap.pipeline_mgr = SimpleNamespace(load_pipeline=AsyncMock(), remove_pipeline=AsyncMock()) + ap.ver_mgr = SimpleNamespace(get_current_version=lambda: 'test-foundation') + ap.sess_mgr = SimpleNamespace(session_list=[]) + ap.plugin_connector = SimpleNamespace(is_enable_plugin=False) + ap.rag_mgr = RAGManager(ap) + ap.knowledge_service = KnowledgeService(ap) + return api, ap, engine + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + ('family', 'table'), [('llm', LLMModel), ('embedding', EmbeddingModel), ('rerank', RerankModel)] +) +async def test_provider_models_sql_ownership_and_secret_round_trip(resources, family, table): + api, _, engine = resources + provider = await api.execute( + 'provider.create', + context(), + { + 'provider_data': { + 'name': 'provider', + 'requester': 'test-requester', + 'base_url': 'https://example.test/v1', + 'api_keys': ['secret-key'], + 'workspace_uuid': 'workspace-b', + } + }, + ) + model = await api.execute( + f'model.{family}.create', + context(), + { + 'model_data': { + 'name': 'model', + 'provider_uuid': provider, + 'extra_args': {'api_key': 'model-secret'}, + 'workspace_uuid': 'workspace-b', + } + }, + ) + reader = context(permissions={Permission.RESOURCE_VIEW}) + visible = await api.execute(f'model.{family}.get', reader, {'model_uuid': model}) + assert visible['extra_args']['api_key'] == '***' + assert visible['provider']['api_keys'] == ['***'] + with pytest.raises(ApplicationAPIException) as denied: + await api.execute(f'model.{family}.get', reader, {'model_uuid': model, 'include_secret': True}) + assert denied.value.error.code == 'permission_denied' + for operation, payload in ( + (f'model.{family}.get', {'model_uuid': model}), + (f'model.{family}.update', {'model_uuid': model, 'model_data': {'name': 'stolen'}}), + (f'model.{family}.create', {'model_data': {'name': 'foreign-provider', 'provider_uuid': provider}}), + ('provider.update', {'provider_uuid': provider, 'provider_data': {'name': 'stolen'}}), + ): + with pytest.raises(ApplicationAPIException) as missing: + await api.execute(operation, context('workspace-b'), payload) + assert missing.value.error.code == 'resource_not_found' + await api.execute( + 'provider.update', + context(), + {'provider_uuid': provider, 'provider_data': {'name': 'renamed', 'api_keys': ['***']}}, + ) + await api.execute( + f'model.{family}.update', + context(), + {'model_uuid': model, 'model_data': {'name': 'renamed-model', 'extra_args': {'api_key': '***'}}}, + ) + async with engine.connect() as connection: + stored_provider = ( + (await connection.execute(sa.select(ModelProvider).where(ModelProvider.uuid == provider))).mappings().one() + ) + stored_model = (await connection.execute(sa.select(table).where(table.uuid == model))).mappings().one() + assert stored_provider['workspace_uuid'] == stored_model['workspace_uuid'] == 'workspace-a' + assert stored_provider['api_keys'] == ['secret-key'] + assert stored_provider['name'] == 'renamed' + assert stored_model['extra_args']['api_key'] == 'model-secret' + assert stored_model['name'] == 'renamed-model' + assert (await connection.execute(sa.select(sa.func.count()).select_from(table))).scalar_one() == 1 + await api.execute(f'model.{family}.delete', context(), {'model_uuid': model}) + assert await api.execute(f'model.{family}.list', context(), {}) == [] + + +@pytest.mark.asyncio +async def test_pipeline_copy_update_and_isolation_use_real_persistence(resources): + api, _, engine = resources + config = {'ai': {'api_key': 'pipeline-secret'}} + pipeline = await api.execute( + 'pipeline.create', context(), {'pipeline_data': {'name': 'pipeline', 'config': config}} + ) + copied = await api.execute('pipeline.copy', context(), {'pipeline_uuid': pipeline}) + assert copied != pipeline + await api.execute( + 'pipeline.update', + context(), + { + 'pipeline_uuid': pipeline, + 'pipeline_data': { + 'name': 'updated', + 'workspace_uuid': 'workspace-b', + 'config': {'ai': {'api_key': '***'}}, + }, + }, + ) + viewer = context(permissions={Permission.RESOURCE_VIEW}) + assert (await api.execute('pipeline.get', viewer, {'pipeline_uuid': pipeline}))['config']['ai']['api_key'] == '***' + with pytest.raises(ApplicationAPIException): + await api.execute('pipeline.copy', context('workspace-b'), {'pipeline_uuid': pipeline}) + with pytest.raises(ApplicationAPIException): + await api.execute('pipeline.delete', context('workspace-b'), {'pipeline_uuid': pipeline}) + async with engine.connect() as connection: + rows = (await connection.execute(sa.select(LegacyPipeline))).mappings().all() + assert len(rows) == 2 + assert {row['workspace_uuid'] for row in rows} == {'workspace-a'} + assert {row['config']['ai']['api_key'] for row in rows} == {'pipeline-secret'} + assert {row['name'] for row in rows} == {'updated', 'pipeline (Copy)'} + await api.execute('pipeline.delete', context(), {'pipeline_uuid': copied}) + assert len(await api.execute('pipeline.list', context(), {})) == 1 + + +@pytest.mark.asyncio +async def test_knowledge_draft_crud_and_secret_projection_with_real_rag_manager(resources): + api, _, engine = resources + kb = await api.execute( + 'knowledge.base.create', + context(), + { + 'kb_data': { + 'name': 'draft', + 'knowledge_engine_plugin_id': 'test/engine', + 'defer_initialization': True, + 'creation_settings': {'api_key': 'knowledge-secret'}, + 'workspace_uuid': 'workspace-b', + } + }, + ) + visible = await api.execute('knowledge.base.get', context(permissions={Permission.RESOURCE_VIEW}), {'kb_uuid': kb}) + assert visible['creation_settings']['api_key'] == '***' + assert visible['initialized'] is False + await api.execute( + 'knowledge.base.update', + context(), + {'kb_uuid': kb, 'kb_data': {'name': 'updated', 'workspace_uuid': 'workspace-b'}}, + ) + for operation, payload in ( + ('knowledge.base.get', {'kb_uuid': kb}), + ('knowledge.base.update', {'kb_uuid': kb, 'kb_data': {'name': 'stolen'}}), + ('knowledge.base.delete', {'kb_uuid': kb}), + ): + with pytest.raises(ApplicationAPIException) as missing: + await api.execute(operation, context('workspace-b'), payload) + assert missing.value.error.code == 'resource_not_found' + async with engine.connect() as connection: + row = (await connection.execute(sa.select(KnowledgeBase).where(KnowledgeBase.uuid == kb))).mappings().one() + assert row['workspace_uuid'] == 'workspace-a' + assert row['name'] == 'updated' + assert row['creation_settings']['api_key'] == 'knowledge-secret' + await api.execute('knowledge.base.delete', context(), {'kb_uuid': kb}) + async with engine.connect() as connection: + assert (await connection.execute(sa.select(sa.func.count()).select_from(KnowledgeBase))).scalar_one() == 0 + + +@pytest.mark.asyncio +async def test_chat_management_uses_scoped_history_and_real_connection_queues(resources, monkeypatch): + from langbot.pkg.api.application import ApplicationAPI + from langbot.pkg.api.application.connection_operations import operations + from langbot.pkg.platform.sources import websocket_manager + from langbot.pkg.platform.sources.websocket_adapter import WebSocketAdapter, WebSocketMessage, WebSocketSession + + api, ap, _ = resources + pipeline = await api.execute( + 'pipeline.create', context(), {'pipeline_data': {'name': 'chat', 'config': {'ai': {}}}} + ) + api = ApplicationAPI(ap, operations=operations()) + manager = websocket_manager.WebSocketConnectionManager() + monkeypatch.setattr(websocket_manager, 'ws_connection_manager', manager) + owned = await manager.add_connection( + None, websocket_manager.WebSocketScope.from_context(context()), pipeline, 'person' + ) + foreign = await manager.add_connection( + None, websocket_manager.WebSocketScope.from_context(context('workspace-b')), pipeline, 'person' + ) + stale = await manager.add_connection( + None, websocket_manager.WebSocketScope('foundation-instance', 'workspace-a', 2), pipeline, 'person' + ) + visible = await api.execute('pipeline.chat.connections', context(), {'pipeline_uuid': pipeline}) + assert [row['connection_id'] for row in visible['connections']] == [owned.connection_id] + await api.execute('pipeline.chat.broadcast', context(), {'pipeline_uuid': pipeline, 'message': '你好'}) + assert owned.send_queue.get_nowait()['message'] == '你好' + assert foreign.send_queue.empty() + assert stale.send_queue.empty() + with pytest.raises(ApplicationAPIException): + await api.execute( + 'pipeline.chat.broadcast', context('workspace-b'), {'pipeline_uuid': pipeline, 'message': 'stolen'} + ) + assert owned.send_queue.empty() + + adapter = WebSocketAdapter.model_construct( + ap=ap, + websocket_person_session=WebSocketSession(id='person'), + websocket_group_session=WebSocketSession(id='group'), + ) + message = WebSocketMessage(id=1, role='user', content='history', message_chain=[], timestamp='2026-10-24') + adapter.websocket_person_session.append_message(pipeline, message) + adapter.websocket_group_session.append_message(pipeline, message) + ap.platform_mgr = SimpleNamespace(get_websocket_proxy_bot=AsyncMock(return_value=SimpleNamespace(adapter=adapter))) + assert ( + await api.execute('pipeline.chat.history', context(), {'pipeline_uuid': pipeline, 'session_type': 'person'}) + )[0]['content'] == 'history' + await api.execute('pipeline.chat.reset', context(), {'pipeline_uuid': pipeline, 'session_type': 'person'}) + assert ( + await api.execute('pipeline.chat.history', context(), {'pipeline_uuid': pipeline, 'session_type': 'person'}) + == [] + ) + assert len(adapter.websocket_group_session.message_lists[pipeline]) == 1 diff --git a/tests/unit_tests/api/test_application_audit.py b/tests/unit_tests/api/test_application_audit.py new file mode 100644 index 000000000..553e6ff33 --- /dev/null +++ b/tests/unit_tests/api/test_application_audit.py @@ -0,0 +1,176 @@ +"""Trace operation outcomes without recording input values or changing failures.""" + +import asyncio +from contextlib import asynccontextmanager +from dataclasses import replace +from types import SimpleNamespace +from unittest.mock import AsyncMock, Mock + +import pytest + +from langbot.pkg.api.application import ApplicationAPI, ApplicationAPIException, Input, Operation +from langbot.pkg.api.authz import Permission +from langbot.pkg.api.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext +from langbot.pkg.operation_trace.service import OPERATION_LEVEL_MUTATION, OPERATION_LEVEL_READ + + +class Payload(Input): + provider_uuid: str = 'provider-a' + config: dict = {} + + +def context(allowed=True): + return RequestContext( + 'instance-a', + 1, + 'request-a', + 'user-token', + PrincipalContext(PrincipalType.ACCOUNT, account_uuid='account-a'), + WorkspaceContext('workspace-a', 'member-a', 'developer', frozenset(Permission) if allowed else frozenset()), + ) + + +@pytest.fixture +def host(monkeypatch): + @asynccontextmanager + async def scope(workspace): + yield + + monkeypatch.setattr('langbot.pkg.api.application.api.get_workspace_language', AsyncMock(return_value='en_US')) + return SimpleNamespace( + persistence_mgr=SimpleNamespace(tenant_scope=scope), + workspace_service=SimpleNamespace( + get_execution_binding=AsyncMock(return_value=SimpleNamespace(instance_uuid='instance-a')) + ), + workspace_settings_service=SimpleNamespace( + effective_level=AsyncMock(return_value=OPERATION_LEVEL_READ), record=AsyncMock() + ), + logger=Mock(), + ) + + +def operation_api(host, handler, name='provider.update', *, write=True, streaming=False): + return ApplicationAPI( + host, + operations=[ + Operation(name, Permission.RESOURCE_MANAGE, handler, Payload, dict, side_effect=write, streaming=streaming) + ], + ) + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + ('name', 'write', 'action'), + [ + ('provider.create', True, 'create'), + ('provider.delete', True, 'delete'), + ('provider.get', False, 'view'), + ], +) +async def test_success_classifies_actions_and_excludes_configuration(host, name, write, action): + await operation_api(host, AsyncMock(return_value={}), name, write=write).execute( + name, + context(), + { + 'provider_uuid': 'provider-a', + 'config': {'api_key': 'private-secret', 'url': 'private-url'}, + }, + ) + call = host.workspace_settings_service.record.await_args + assert call.args == ('workspace-a',) + assert call.kwargs['rule'].action == action + assert call.kwargs['rule'].bucket == ('write' if write else 'read') + assert call.kwargs['outcome'] == 'ok' + assert call.kwargs['resource_id'] == 'provider-a' + assert 'private' not in repr(call) + host.workspace_settings_service.record.assert_awaited_once() + + +@pytest.mark.asyncio +async def test_read_capture_honors_mutation_only_level(host): + host.workspace_settings_service.effective_level.return_value = OPERATION_LEVEL_MUTATION + await operation_api(host, AsyncMock(return_value={}), 'provider.get', write=False).execute( + 'provider.get', context(), {} + ) + host.workspace_settings_service.record.assert_not_awaited() + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + ('error', 'outcome', 'status'), + [(ValueError('invalid secret'), 'error', 400), (RuntimeError('private error'), 'unknown', 500)], +) +async def test_failed_mutation_records_once_without_error_payload(host, error, outcome, status): + with pytest.raises(ApplicationAPIException): + await operation_api(host, AsyncMock(side_effect=error)).execute( + 'provider.update', context(), {'config': {'api_key': 'private'}} + ) + host.workspace_settings_service.record.assert_awaited_once() + data = host.workspace_settings_service.record.await_args.kwargs + assert (data['outcome'], data['status_code']) == (outcome, status) + assert 'private' not in repr(data) and 'invalid secret' not in repr(data) + + +@pytest.mark.asyncio +async def test_denied_records_once_but_unknown_workspace_and_malformed_context_do_not(host): + handler = AsyncMock(return_value={}) + api = operation_api(host, handler) + with pytest.raises(ApplicationAPIException) as denied: + await api.execute('provider.update', context(False), {}) + assert denied.value.error.code == 'permission_denied' + assert host.workspace_settings_service.record.await_args.kwargs['outcome'] == 'denied' + host.workspace_settings_service.record.assert_awaited_once() + handler.assert_not_awaited() + host.workspace_settings_service.record.reset_mock() + for invalid in (None, replace(context(), instance_uuid='foreign')): + with pytest.raises(ApplicationAPIException): + await api.execute('provider.update', invalid, {}) + host.workspace_settings_service.record.assert_not_awaited() + + +@pytest.mark.asyncio +async def test_invalid_result_and_cancellation_record_unknown_and_keep_original_failure(host): + api = operation_api(host, AsyncMock(return_value='invalid')) + with pytest.raises(ApplicationAPIException) as error: + await api.execute('provider.update', context(), {}) + assert error.value.error.code == 'invalid_output' + assert host.workspace_settings_service.record.await_args.kwargs['outcome'] == 'unknown' + api = operation_api(host, AsyncMock(side_effect=asyncio.CancelledError())) + with pytest.raises(asyncio.CancelledError): + await api.execute('provider.update', context(), {}) + assert host.workspace_settings_service.record.await_args.kwargs['status_code'] == 499 + host.workspace_settings_service.record.side_effect = RuntimeError('audit failed') + host.logger.exception.side_effect = RuntimeError('logger failed') + with pytest.raises(ApplicationAPIException) as original: + await operation_api(host, AsyncMock(side_effect=ValueError('original'))).execute( + 'provider.update', context(), {} + ) + assert original.value.error.message == 'original' + + +@pytest.mark.asyncio +async def test_late_stream_failure_is_recorded_once(host): + async def handler(ap, ctx, payload): + async def source(): + yield {} + raise RuntimeError('private stream error') + + return source() + + api = operation_api(host, handler, streaming=True) + with pytest.raises(ApplicationAPIException): + async for _ in api.stream('provider.update', context(), {}): + pass + host.workspace_settings_service.record.assert_awaited_once() + assert host.workspace_settings_service.record.await_args.kwargs['outcome'] == 'unknown' + + +@pytest.mark.asyncio +async def test_domain_audit_exclusion_applies_only_to_success(host): + name = 'workspace.member.update' + await operation_api(host, AsyncMock(return_value={}), name).execute(name, context(), {}) + host.workspace_settings_service.record.assert_not_awaited() + with pytest.raises(ApplicationAPIException): + await operation_api(host, AsyncMock(side_effect=ValueError('last owner')), name).execute(name, context(), {}) + host.workspace_settings_service.record.assert_awaited_once() + assert host.workspace_settings_service.record.await_args.kwargs['outcome'] == 'error' diff --git a/tests/unit_tests/api/test_application_auxiliary.py b/tests/unit_tests/api/test_application_auxiliary.py new file mode 100644 index 000000000..8dbdff0ec --- /dev/null +++ b/tests/unit_tests/api/test_application_auxiliary.py @@ -0,0 +1,212 @@ +"""Application operations exercised with SQLite, local files and real task ownership.""" + +from __future__ import annotations + +import asyncio +import contextvars +import dataclasses +import logging +from contextlib import asynccontextmanager +from types import SimpleNamespace + +import pytest +import sqlalchemy + +from langbot.pkg.api.application import ApplicationAPI, ApplicationAPIException +from langbot.pkg.api.application.auxiliary import operations +from langbot.pkg.api.authz import Permission, permissions_for_role +from langbot.pkg.api.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext +from langbot.pkg.core.taskmgr import AsyncTaskManager +from langbot.pkg.entity.persistence.user import User +from langbot.pkg.entity.persistence.workspace import WorkspaceInvitation, WorkspaceMembership +from langbot.pkg.operation_trace.service import WorkspaceSettingsService +from langbot.pkg.storage.mgr import StorageMgr +from langbot.pkg.storage.providers import localstorage + +pytest_plugins = ['tests.unit_tests.workspace.test_workspace_collaboration'] + + +@pytest.fixture +async def foundation(collaboration_context, tmp_path, monkeypatch): + collaboration, workspace_service, session_factory, owner, workspace, membership = collaboration_context + ap = collaboration.ap + ap.workspace_service = workspace_service + ap.workspace_collaboration_service = collaboration + ap.instance_config = SimpleNamespace(data={}) + ap.logger = logging.getLogger('application-auxiliary-test') + ap.event_loop = asyncio.get_running_loop() + manager = ap.persistence_mgr + current_session = contextvars.ContextVar('auxiliary_session', default=None) + + @asynccontextmanager + async def tenant_scope(workspace_uuid): + yield + + @asynccontextmanager + async def tenant_uow(workspace_uuid): + if current_session.get() is not None: + yield SimpleNamespace(session=current_session.get(), execute=current_session.get().execute) + return + async with session_factory.begin() as session: + token = current_session.set(session) + try: + yield SimpleNamespace(session=session, execute=session.execute) + finally: + current_session.reset(token) + + async def execute_async(statement): + if current_session.get() is not None: + connection = await current_session.get().connection() + return await connection.execute(statement) + async with manager.get_db_engine().begin() as connection: + return await connection.execute(statement) + + async def get_user(account_uuid): + async with session_factory() as session: + return await session.get(User, account_uuid) + + manager.tenant_scope = tenant_scope + manager.tenant_uow = tenant_uow + manager.execute_async = execute_async + manager.current_session = current_session.get + ap.user_service = SimpleNamespace(get_user_by_uuid=get_user) + ap.workspace_settings_service = WorkspaceSettingsService(ap) + ap.task_mgr = AsyncTaskManager(ap) + ap.storage_mgr = StorageMgr(ap) + monkeypatch.setattr(localstorage, 'LOCAL_STORAGE_PATH', str(tmp_path / 'objects')) + ap.storage_mgr.storage_provider = localstorage.LocalStorageProvider(ap) + context = RequestContext( + instance_uuid=workspace.instance_uuid, + placement_generation=1, + request_id='auxiliary-check', + auth_type='user_token', + principal=PrincipalContext(PrincipalType.ACCOUNT, account_uuid=owner.uuid), + workspace=WorkspaceContext(workspace.uuid, membership.uuid, 'owner', permissions_for_role('owner')), + ) + api = ApplicationAPI(ap, operations=operations()) + yield api, ap, context, session_factory + await ap.workspace_settings_service.flush_pending() + if ap.workspace_settings_service._writer_task is not None: + ap.workspace_settings_service._writer_task.cancel() + await asyncio.gather(ap.workspace_settings_service._writer_task, return_exceptions=True) + for task in ap.task_mgr.tasks: + if not task.task.done(): + task.task.cancel() + await asyncio.gather(*(task.task for task in ap.task_mgr.tasks), return_exceptions=True) + + +@pytest.mark.asyncio +async def test_file_upload_round_trip_and_workspace_isolation(foundation): + api, ap, context, _ = foundation + content = b'\x89PNG\r\n\x1a\n' + '中文图片'.encode() + uploaded = await api.execute('file.image.upload', context, {'filename': 'image.PNG', 'content': content}) + returned = await api.execute('file.image.get', context, uploaded) + assert returned == {'content': content, 'media_type': 'image/png'} + key = uploaded['file_key'] + assert localstorage._safe_resolve(localstorage.LOCAL_STORAGE_PATH, key) + foreign_key = key.replace(context.workspace_uuid, '00000000-0000-0000-0000-000000000000') + with pytest.raises(ApplicationAPIException): + await api.execute('file.image.get', context, {'file_key': foreign_key}) + viewer = dataclasses.replace( + context, + workspace=dataclasses.replace(context.workspace, role='viewer', permissions=permissions_for_role('viewer')), + ) + with pytest.raises(ApplicationAPIException) as denied: + await api.execute('file.image.upload', viewer, {'filename': 'image.png', 'content': content}) + assert denied.value.error.status_code == 403 + for filename in ('../a.png', 'a\\b.png', 'a.svg', 'bad\x00.png'): + with pytest.raises(ApplicationAPIException): + await api.execute('file.image.upload', context, {'filename': filename, 'content': content}) + + +@pytest.mark.asyncio +async def test_member_mutations_preserve_owner_and_invitation_secret(foundation): + api, ap, context, sessions = foundation + invite = await api.execute( + 'workspace.invitation.create', context, {'email': 'joined@example.com', 'role': 'viewer'} + ) + assert invite['delivery']['status'] == 'link_only' + assert 'token_hash' not in invite['invitation'] + async with sessions.begin() as session: + stored = await session.get(WorkspaceInvitation, invite['invitation']['uuid']) + assert stored.token_hash != invite['token'] + new_user = User( + uuid='joined-account', + user='joined', + normalized_email='joined@example.com', + password='test', + account_type='local', + ) + session.add(new_user) + await ap.workspace_collaboration_service.accept_invitation(invite['token'], new_user.uuid) + changed = await api.execute( + 'workspace.member.update', context, {'account_uuid': new_user.uuid, 'role': 'developer'} + ) + assert changed['role'] == 'developer' + members = await api.execute('workspace.member.list', context, {}) + assert {row['account_uuid'] for row in members} == {context.account_uuid, new_user.uuid} + with pytest.raises(ApplicationAPIException): + await api.execute('workspace.member.remove', context, {'account_uuid': context.account_uuid}) + await api.execute('workspace.member.remove', context, {'account_uuid': new_user.uuid}) + async with sessions() as session: + row = await session.scalar( + sqlalchemy.select(WorkspaceMembership).where(WorkspaceMembership.account_uuid == new_user.uuid) + ) + assert row.status == 'removed' + key_context = dataclasses.replace( + context, principal=PrincipalContext(PrincipalType.API_KEY, api_key_uuid='key'), auth_type='api_key' + ) + with pytest.raises(ApplicationAPIException): + await api.execute('workspace.member.list', key_context, {}) + + +@pytest.mark.asyncio +async def test_language_wizard_and_governance_persist_without_http(foundation): + api, ap, context, _ = foundation + assert (await api.execute('workspace.language.update', context, {'language': 'zh_Hans'}))['language'] == 'zh_Hans' + assert (await api.execute('workspace.language.get', context, {}))['language'] == 'zh_Hans' + await api.execute('system.wizard.progress.update', context, {'step': 2, 'bot_saved': True}) + assert (await api.execute('system.wizard.get', context, {}))['progress']['step'] == 2 + await api.execute('system.wizard.complete', context, {}) + assert await api.execute('system.wizard.get', context, {}) == {'status': 'completed', 'progress': None} + updated = await api.execute('settings.governance.update', context, {'level': 1, 'retention_days': 30}) + assert updated['configured_level'] == 1 + assert updated['retention_days'] == 30 + await ap.workspace_settings_service.flush_pending() + logs = await api.execute('settings.operation_logs.list', context, {}) + assert any(row['action'] == 'settings_update' for row in logs['records']) + unauthorized_role = dataclasses.replace( + context, + workspace=dataclasses.replace( + context.workspace, role='viewer', permissions=frozenset({Permission.AUDIT_VIEW.value}) + ), + ) + with pytest.raises(ApplicationAPIException) as denied: + await api.execute('settings.governance.update', unauthorized_role, {'level': 0}) + assert denied.value.error.status_code == 403 + + +@pytest.mark.asyncio +async def test_task_selection_excludes_other_workspace_and_hides_api_key_logs(foundation): + api, ap, context, _ = foundation + + async def work(): + return None + + owned = ap.task_mgr.create_user_task( + work(), instance_uuid=context.instance_uuid, workspace_uuid=context.workspace_uuid, placement_generation=1 + ) + owned.task_context.trace('secret internal diagnostic') + other = ap.task_mgr.create_user_task( + work(), instance_uuid=context.instance_uuid, workspace_uuid='other-workspace', placement_generation=1 + ) + await asyncio.gather(owned.task, other.task) + result = await api.execute('task.list', context, {}) + assert [task['id'] for task in result['tasks']] == [owned.id] + key_context = dataclasses.replace( + context, principal=PrincipalContext(PrincipalType.API_KEY, api_key_uuid='key'), auth_type='api_key' + ) + public = await api.execute('task.get', key_context, {'task_id': owned.id}) + assert 'secret internal diagnostic' not in str(public) + with pytest.raises(ApplicationAPIException): + await api.execute('task.get', context, {'task_id': other.id}) diff --git a/tests/unit_tests/api/test_application_context.py b/tests/unit_tests/api/test_application_context.py new file mode 100644 index 000000000..234c46cb8 --- /dev/null +++ b/tests/unit_tests/api/test_application_context.py @@ -0,0 +1,9 @@ +from langbot.pkg.api.context import ExecutionContext, RequestContext +from langbot.pkg.api.http.context import ExecutionContext as HTTPExecutionContext +from langbot.pkg.api.http.context import RequestContext as HTTPRequestContext + + +def test_application_context_types_are_http_independent_and_compatibly_reexported(): + assert RequestContext is HTTPRequestContext + assert ExecutionContext is HTTPExecutionContext + assert RequestContext.__module__ == 'langbot.pkg.api.context' diff --git a/tests/unit_tests/api/test_application_errors.py b/tests/unit_tests/api/test_application_errors.py new file mode 100644 index 000000000..530dca29d --- /dev/null +++ b/tests/unit_tests/api/test_application_errors.py @@ -0,0 +1,69 @@ +"""Application errors preserve business meaning and hide internal failure data.""" + +import pytest +from langbot_plugin.box.errors import BoxAdmissionError +from pydantic import BaseModel, ValidationError + +from langbot.pkg.agent.runner.errors import ( + RunnerExecutionError, + RunnerNotAuthorizedError, + RunnerNotFoundError, + RunnerProtocolError, +) +from langbot.pkg.api.application.errors import error_from_exception +from langbot.pkg.api.http.service.bot_errors import BotApplyError +from langbot.pkg.cloud.entitlements import EntitlementUnavailableError +from langbot.pkg.core.errors import TaskCapacityError +from langbot.pkg.provider.modelmgr.codex_errors import CodexProviderError +from langbot.pkg.utils.bounded_executor import BlockingWorkCapacityError +from langbot.pkg.workspace.collaboration import LastOwnerError, MembershipPermissionError +from langbot.pkg.workspace.errors import WorkspaceGenerationMismatchError + + +@pytest.mark.parametrize( + ('exc', 'code', 'status'), + [ + (RunnerExecutionError('runner', 'failed', error_code='tool_failed'), 'tool_failed', 422), + (RunnerNotAuthorizedError('runner', []), 'runner_not_authorized', 403), + (RunnerNotFoundError('runner'), 'runner_not_found', 409), + (RunnerProtocolError('runner', 'wrong protocol'), 'runner_protocol_error', 502), + (MembershipPermissionError('denied'), 'permission_denied', 403), + (LastOwnerError('last owner'), 'last_owner_required', 400), + (BoxAdmissionError('no grant'), 'managed_sandbox_unavailable', 403), + (EntitlementUnavailableError('expired'), 'entitlement_unavailable', 403), + (TaskCapacityError('full'), 'task_capacity_exceeded', 429), + (BlockingWorkCapacityError('full'), 'blocking_work_capacity_exceeded', 429), + (CodexProviderError('known safe error', 429, 'rate_limited'), 'rate_limited', 429), + (FileNotFoundError('/private/file'), 'resource_not_found', 404), + (WorkspaceGenerationMismatchError('stale placement'), 'workspace_unavailable', 409), + ], +) +def test_business_error_mapping(exc, code, status): + result = error_from_exception(exc) + assert (result.code, result.status_code) == (code, status) + + +def test_box_entitlement_and_saved_bot_identity_are_preserved(): + result = error_from_exception(EntitlementUnavailableError('expired'), operation_id='box.status') + assert (result.code, result.status_code) == ('managed_sandbox_unavailable', 403) + result = error_from_exception(BotApplyError('Saved configuration could not be applied', bot_uuid='bot-a')) + assert (result.code, result.status_code, result.details) == ('bot_apply_failed', 400, {'uuid': 'bot-a'}) + + +def test_unknown_and_pydantic_errors_do_not_expose_raw_configuration(): + class CredentialInput(BaseModel): + token: int + + with pytest.raises(ValidationError) as failure: + CredentialInput(token='sensitive-value') + assert 'sensitive-value' not in error_from_exception(failure.value).message + assert error_from_exception(RuntimeError('raw sensitive backend error')).message == 'Application operation failed' + + +def test_typed_assistant_and_migration_errors_keep_original_status(): + class ServiceError(ValueError): + code = 'stale_turn' + status = 409 + + result = error_from_exception(ServiceError('stale_turn')) + assert (result.code, result.status_code) == ('stale_turn', 409) diff --git a/tests/unit_tests/api/test_application_platform_setup.py b/tests/unit_tests/api/test_application_platform_setup.py new file mode 100644 index 000000000..4051cf48b --- /dev/null +++ b/tests/unit_tests/api/test_application_platform_setup.py @@ -0,0 +1,182 @@ +from __future__ import annotations + +import asyncio +import contextlib +import dataclasses +import inspect +import time +from types import SimpleNamespace +from unittest.mock import AsyncMock, Mock + +import lark_oapi +import pytest +from pydantic import TypeAdapter + +from langbot.pkg.api.application.api import ApplicationAPI +from langbot.pkg.api.application.contracts import ApplicationAPIException +from langbot.pkg.api.application.platform_setup import operations +from langbot.pkg.api.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext +from langbot.pkg.core.errors import TaskCapacityError +from langbot.pkg.platform.credential_setup import CredentialSetupError, _bind_session_scope, get_credential_setup + + +def context(**changes): + return RequestContext( + instance_uuid='instance-a', + workspace=WorkspaceContext('workspace-a', 'membership-a', 'developer', frozenset({'resource.manage'})), + placement_generation=1, + request_id='request-a', + auth_type='user-token', + principal=PrincipalContext(PrincipalType.ACCOUNT, account_uuid='account-a'), + **changes, + ) + + +@pytest.fixture +def ap(monkeypatch): + @contextlib.asynccontextmanager + async def tenant_scope(workspace): + yield + + def create_user_task(coro, **kwargs): + return SimpleNamespace(task=asyncio.create_task(coro)) + + application = SimpleNamespace( + task_mgr=SimpleNamespace(create_user_task=Mock(side_effect=create_user_task)), + workspace_service=SimpleNamespace( + get_execution_binding=AsyncMock(return_value=SimpleNamespace(instance_uuid='instance-a')) + ), + persistence_mgr=SimpleNamespace(tenant_scope=tenant_scope), + ) + monkeypatch.setattr('langbot.pkg.api.application.api.get_workspace_language', AsyncMock(return_value='en_US')) + monkeypatch.setattr(ApplicationAPI, '_record', AsyncMock()) + return application + + +@pytest.mark.asyncio +async def test_catalog_covers_all_credential_flows_and_checks_permission_before_start(ap): + api = ApplicationAPI(ap, operations=operations()) + assert len(api.operation_catalog()) == 18 + assert {op['permission'] for op in api.operation_catalog()} == {'resource.manage'} + viewer = dataclasses.replace( + context(), workspace=WorkspaceContext('workspace-a', 'membership-a', 'viewer', frozenset()) + ) + with pytest.raises(ApplicationAPIException) as exc: + await api.execute('platform.setup.lark.start', viewer, {}) + assert exc.value.error.code == 'permission_denied' + ap.task_mgr.create_user_task.assert_not_called() + assert not hasattr(ap, 'credential_setup_service') + + +@pytest.mark.asyncio +async def test_setup_service_is_shared_with_transport_and_does_not_need_request_globals(ap, monkeypatch): + release = asyncio.Event() + completed = asyncio.Event() + + async def register(*, on_qr_code, source): + on_qr_code({'url': 'https://example.test/qr'}) + await release.wait() + completed.set() + return {'client_id': 'app-a', 'client_secret': 'secret-a'} + + monkeypatch.setattr(lark_oapi, 'aregister_app', register) + api = ApplicationAPI(ap, operations=operations()) + ctx = context() + started = await api.execute('platform.setup.lark.start', ctx, {}) + service = get_credential_setup(ap) + assert get_credential_setup(ap) is service + assert await service.lark_status(started['session_id'], ctx) == {'status': 'waiting'} + release.set() + await completed.wait() + result = await api.execute('platform.setup.lark.status', ctx, {'session_id': started['session_id']}) + assert result == {'status': 'success', 'app_id': 'app-a', 'app_secret': 'secret-a'} + with pytest.raises(CredentialSetupError) as exc: + await service.lark_status(started['session_id'], ctx) + assert exc.value.status_code == 404 + + +@pytest.mark.asyncio +async def test_task_admission_error_is_preserved_as_domain_status(ap): + ap.task_mgr.create_user_task.side_effect = TaskCapacityError('capacity exhausted') + with pytest.raises(CredentialSetupError) as exc: + await get_credential_setup(ap).lark_start(context()) + assert exc.value.status_code == 429 + assert 'Too many active' in str(exc.value) + + +_CASES = [ + ('lark', '_create_app_sessions', {'app_id': 'id', 'app_secret': 'secret'}), + ( + 'weixin', + '_weixin_login_sessions', + { + 'qr_data_url': 'data:qr', + 'expire_at': 1.0, + 'token': 'token', + 'base_url': 'https://example.test', + 'account_id': 'id', + }, + ), + ('dingtalk', '_dingtalk_sessions', {'client_id': 'id', 'client_secret': 'secret'}), + ('wecombot', '_wecombot_sessions', {'botid': 'id', 'secret': 'secret'}), + ('qqofficial', '_qqofficial_sessions', {'appid': 'id', 'secret': 'secret', 'user_openid': 'user'}), +] + + +@pytest.mark.asyncio +@pytest.mark.parametrize('platform,store_name,credentials', _CASES) +async def test_each_platform_preserves_exact_owner_scoping_and_credentials(ap, platform, store_name, credentials): + service = get_credential_setup(ap) + status = getattr(service, f'{platform}_status') + cancel = getattr(service, f'{platform}_cancel') + store = inspect.getclosurevars(status).nonlocals[store_name] + session = {'status': 'success', 'created_at': time.time(), **credentials} + owner = context() + _bind_session_scope(session, owner) + store['session-a'] = session + foreign = dataclasses.replace(owner, principal=PrincipalContext(PrincipalType.ACCOUNT, account_uuid='account-b')) + for ctx in ( + foreign, + dataclasses.replace(owner, placement_generation=2), + dataclasses.replace(owner, workspace=dataclasses.replace(owner.workspace, workspace_uuid='workspace-b')), + ): + with pytest.raises(CredentialSetupError) as exc: + await status('session-a', ctx) + assert exc.value.status_code == 404 + with pytest.raises(CredentialSetupError): + await cancel('session-a', ctx) + assert store['session-a'] is session + result = await status('session-a', owner) + spec = next(op for op in operations() if op.operation_id == f'platform.setup.{platform}.status') + assert TypeAdapter(spec.output_type).validate_python(result, strict=True) == result + assert result == {'status': 'success', **credentials} + assert 'session-a' not in store + + +@pytest.mark.asyncio +@pytest.mark.parametrize('platform,store_name,credentials', _CASES) +async def test_each_platform_cancel_stops_its_owned_task(ap, platform, store_name, credentials): + service = get_credential_setup(ap) + status = getattr(service, f'{platform}_status') + store = inspect.getclosurevars(status).nonlocals[store_name] + task = SimpleNamespace(done=Mock(return_value=False), cancel=Mock()) + session = {'status': 'waiting', 'created_at': time.time(), 'task': task, **credentials} + _bind_session_scope(session, context()) + store['session-a'] = session + assert await getattr(service, f'{platform}_cancel')('session-a', context()) == {} + task.cancel.assert_called_once_with() + assert not store + + +@pytest.mark.asyncio +async def test_slack_cancel_checks_ownership_before_discard(ap): + from langbot.pkg.api.http.service.slack_setup import SlackSetupError + + ap.slack_setup_service = SimpleNamespace( + owned=Mock(side_effect=SlackSetupError('session_not_found')), discard=Mock() + ) + api = ApplicationAPI(ap, operations=operations()) + with pytest.raises(ApplicationAPIException) as exc: + await api.execute('platform.setup.slack.cancel', context(), {'session_id': 'foreign-session'}) + assert exc.value.error.status_code == 404 + ap.slack_setup_service.discard.assert_not_called() diff --git a/tests/unit_tests/api/test_application_plugins.py b/tests/unit_tests/api/test_application_plugins.py new file mode 100644 index 000000000..997fcaf0d --- /dev/null +++ b/tests/unit_tests/api/test_application_plugins.py @@ -0,0 +1,167 @@ +"""Plugin foundation contracts and Workspace isolation without a transport.""" + +from contextlib import asynccontextmanager +from io import BytesIO +from types import SimpleNamespace +from unittest.mock import AsyncMock, Mock +from zipfile import ZipFile + +import pytest +from pydantic import TypeAdapter, ValidationError + +from langbot.pkg.api.application import plugin_operations as ops +from langbot.pkg.api.context import ExecutionContext +from langbot.pkg.workspace.errors import WorkspaceGenerationMismatchError, WorkspaceNotFoundError + + +CONTEXT = SimpleNamespace(instance_uuid='instance-a', workspace_uuid='workspace-a', placement_generation=3) + + +def app(): + execution = ExecutionContext(instance_uuid='instance-a', workspace_uuid='workspace-a', placement_generation=3) + return SimpleNamespace( + plugin_connector=SimpleNamespace(require_workspace_context=AsyncMock(return_value=execution)), + instance_config=SimpleNamespace(data={'system': {'limitation': {}}}), + ) + + +@pytest.mark.asyncio +async def test_plugin_reads_mask_secrets_and_config_write_restores_persisted_values(): + ap = app() + raw = {'plugin_config': {'apiKey': 'runtime-secret'}, 'debug': {'plugin_debug_key': 'debug-secret'}} + ap.plugin_connector.get_plugin_info = AsyncMock(return_value=raw) + ap.plugin_connector.set_plugin_config = AsyncMock() + ap.persistence_mgr = SimpleNamespace( + execute_async=AsyncMock( + return_value=SimpleNamespace( + scalar_one_or_none=lambda: { + 'apiKey': 'stored-secret', + 'nested': {'token': 'nested-secret'}, + 'enabled': False, + } + ) + ) + ) + key = ops.PluginInput(author='a', plugin_name='b') + assert (await ops.get_plugin(ap, CONTEXT, key))['plugin']['plugin_config']['apiKey'] == '***' + assert raw['plugin_config']['apiKey'] == 'runtime-secret' + assert (await ops.get_config(ap, CONTEXT, key))['config']['nested']['token'] == '***' + await ops.update_config( + ap, + CONTEXT, + ops.ConfigInput( + author='a', plugin_name='b', config={'apiKey': '***', 'nested': {'token': '***'}, 'enabled': True} + ), + ) + assert ap.plugin_connector.set_plugin_config.await_args.args == ( + 'a', + 'b', + {'apiKey': 'stored-secret', 'nested': {'token': 'nested-secret'}, 'enabled': True}, + ) + with pytest.raises(ValueError, match='no existing value'): + await ops.update_config(ap, CONTEXT, ops.ConfigInput(author='a', plugin_name='b', config={'password': '***'})) + + +@pytest.mark.asyncio +async def test_queued_plugin_task_revalidates_generation_before_runtime_io(): + ap = app() + ap.plugin_connector.delete_plugin = AsyncMock() + queued = [] + scopes = [] + + @asynccontextmanager + async def tenant_scope(workspace): + scopes.append(workspace) + yield + + def create_task(coro, **metadata): + queued.append((coro, metadata)) + return SimpleNamespace(id=17) + + ap.persistence_mgr = SimpleNamespace(tenant_scope=tenant_scope) + ap.task_mgr = SimpleNamespace(create_user_task=create_task) + assert await ops.delete(ap, CONTEXT, ops.DeleteInput(author='a', plugin_name='b')) == {'task_id': 17} + assert TypeAdapter(ops.TaskResult).validate_python({'task_id': 17}, strict=True) == {'task_id': 17} + assert queued[0][1]['workspace_uuid'] == 'workspace-a' + assert queued[0][1]['placement_generation'] == 3 + ap.plugin_connector.require_workspace_context.side_effect = WorkspaceGenerationMismatchError('stale placement') + with pytest.raises(WorkspaceGenerationMismatchError): + await queued[0][0] + assert scopes == ['workspace-a'] + ap.plugin_connector.delete_plugin.assert_not_awaited() + + +@pytest.mark.asyncio +async def test_install_quota_counts_plugins_and_mcp_before_queueing(): + ap = app() + ap.instance_config.data['system']['limitation']['max_extensions'] = 2 + ap.plugin_connector.list_plugins = AsyncMock(return_value=[{}]) + ap.mcp_service = SimpleNamespace(get_mcp_servers=AsyncMock(return_value=[{}])) + ap.task_mgr = SimpleNamespace(create_user_task=Mock()) + with pytest.raises(ValueError, match='Maximum number'): + await ops.install_marketplace(ap, CONTEXT, ops.MarketplaceInput(plugin_author='a', plugin_name='b')) + ap.task_mgr.create_user_task.assert_not_called() + + +@pytest.mark.asyncio +async def test_asset_path_rejects_escape_before_runtime_read_and_returns_exact_bytes(): + ap = app() + ap.plugin_connector.get_plugin_assets = AsyncMock(return_value={'asset_base64': 'AP8K', 'mime_type': 'image/png'}) + for path in ('../secret', '/etc/passwd', '..\\secret'): + with pytest.raises(WorkspaceNotFoundError): + await ops.asset(ap, CONTEXT, ops.AssetInput(author='a', plugin_name='b', filepath=path)) + ap.plugin_connector.get_plugin_assets.assert_not_awaited() + result = await ops.asset(ap, CONTEXT, ops.AssetInput(author='a', plugin_name='b', filepath='image.png')) + assert result == {'content': b'\x00\xff\n', 'mime_type': 'image/png'} + assert ap.plugin_connector.get_plugin_assets.await_args.args == ('a', 'b', 'assets/image.png') + + +@pytest.mark.asyncio +async def test_config_files_are_workspace_scoped_and_size_checked(): + ap = app() + ap.storage_mgr = SimpleNamespace( + save_scoped=AsyncMock(return_value='scoped-key'), + is_scoped_object_key=Mock(return_value=False), + delete_scoped_object_key=AsyncMock(), + ) + assert await ops.upload_config_file(ap, CONTEXT, ops.FileInput(content=b'payload', filename='data.txt')) == { + 'file_key': 'scoped-key' + } + args = ap.storage_mgr.save_scoped.await_args + assert args.args == (CONTEXT,) + assert args.kwargs['owner'] == 'workspace-a' + assert args.kwargs['owner_type'] == 'plugin_config' + assert args.kwargs['value'] == b'payload' + with pytest.raises(ValueError, match='10MB'): + await ops.upload_config_file(ap, CONTEXT, ops.FileInput(content=b'x' * (10 * 1024 * 1024 + 1))) + with pytest.raises(ValueError, match='Invalid file key'): + await ops.delete_config_file(ap, CONTEXT, ops.FileKeyInput(file_key='foreign/raw')) + ap.storage_mgr.delete_scoped_object_key.assert_not_awaited() + + +@pytest.mark.asyncio +async def test_archive_preview_uses_archive_admission_and_component_metadata(): + ap = app() + archive = BytesIO() + with ZipFile(archive, 'w') as output: + output.writestr( + 'manifest.yaml', + 'metadata:\n author: example\n name: sample\nspec:\n components:\n tools:\n fromDirs: [components/tools]\n', + ) + output.writestr('components/tools/tool.yaml', 'name: test') + output.writestr('requirements.txt', 'httpx\n') + result = await ops.preview_local(ap, CONTEXT, ops.FileInput(content=archive.getvalue(), filename='sample.lbpkg')) + assert result['metadata'] == {'author': 'example', 'name': 'sample'} + assert result['component_counts'] == {'tools': 1} + assert result['requirements'] == ['httpx'] + with pytest.raises(ValueError): + await ops.preview_local(ap, CONTEXT, ops.FileInput(content=b'invalid')) + + +def test_install_inputs_do_not_coerce_force_flags_or_github_path_segments(): + with pytest.raises(ValidationError): + ops.LocalInstallInput(content=b'zip', administrator_force='true') + with pytest.raises(ValidationError): + ops.GithubAssetsInput(owner='a/b', repo='repo', release_id=1) + with pytest.raises(ValidationError): + ops.DeleteInput(author='a', plugin_name='b', delete_data='false') diff --git a/tests/unit_tests/api/test_application_streaming.py b/tests/unit_tests/api/test_application_streaming.py new file mode 100644 index 000000000..034f103c1 --- /dev/null +++ b/tests/unit_tests/api/test_application_streaming.py @@ -0,0 +1,173 @@ +from __future__ import annotations + +import asyncio +from contextlib import aclosing, asynccontextmanager +from contextvars import ContextVar +from types import SimpleNamespace +from unittest.mock import AsyncMock, Mock + +import pytest + +from langbot.pkg.api.application.api import ApplicationAPI +from langbot.pkg.api.application.contracts import ApplicationAPIException, Input, Operation +from langbot.pkg.api.authz import Permission +from langbot.pkg.api.context import PrincipalContext, PrincipalType, RequestContext, WorkspaceContext +from langbot.pkg.i18n import language +from langbot.pkg.utils.bounded_executor import current_blocking_work_scope + + +class Empty(Input): + pass + + +def context(workspace): + return RequestContext( + 'instance-a', + 1, + 'request-a', + 'user-token', + PrincipalContext(PrincipalType.ACCOUNT, account_uuid='account-a'), + WorkspaceContext(workspace, 'membership-a', 'developer', frozenset({'resource.view'})), + ) + + +@pytest.fixture +def host(monkeypatch): + tenant = ContextVar('test_stream_tenant', default=None) + + @asynccontextmanager + async def tenant_scope(workspace): + token = tenant.set(workspace) + try: + yield + finally: + tenant.reset(token) + + ap = SimpleNamespace( + persistence_mgr=SimpleNamespace(tenant_scope=tenant_scope), + tenant=tenant, + workspace_service=SimpleNamespace( + get_execution_binding=AsyncMock(return_value=SimpleNamespace(instance_uuid='instance-a')) + ), + logger=Mock(), + ) + monkeypatch.setattr( + 'langbot.pkg.api.application.api.get_workspace_language', + AsyncMock(side_effect=lambda ap, ws: 'zh_Hans' if ws == 'workspace-a' else 'ja_JP'), + ) + return ap + + +def api(host, handler): + return ApplicationAPI( + host, + operations=[Operation('test.stream', Permission.RESOURCE_VIEW, handler, Empty, dict[str, str], streaming=True)], + ) + + +def assert_consumer_unscoped(host): + assert host.tenant.get() is None + assert current_blocking_work_scope() is None + assert language.get() == 'en_US' + + +@pytest.mark.asyncio +async def test_interleaved_streams_isolate_tenant_language_and_executor_scope(host): + closed = set() + + async def handler(ap, ctx, payload): + async def source(): + try: + for _ in range(2): + assert host.tenant.get() == ctx.workspace_uuid + assert current_blocking_work_scope() == ctx.workspace_uuid + yield {'workspace': ctx.workspace_uuid, 'language': language.get()} + finally: + assert host.tenant.get() == ctx.workspace_uuid + closed.add(ctx.workspace_uuid) + + return source() + + application = api(host, handler) + first = application.stream('test.stream', context('workspace-a'), {}) + second = application.stream('test.stream', context('workspace-b'), {}) + async with aclosing(first), aclosing(second): + for _ in range(2): + assert await anext(first) == {'workspace': 'workspace-a', 'language': 'zh_Hans'} + assert_consumer_unscoped(host) + assert await anext(second) == {'workspace': 'workspace-b', 'language': 'ja_JP'} + assert_consumer_unscoped(host) + assert closed == {'workspace-a', 'workspace-b'} + + +@pytest.mark.asyncio +async def test_stream_can_be_closed_from_another_task_without_resetting_foreign_context(host): + closed = asyncio.Event() + + async def handler(ap, ctx, payload): + async def source(): + try: + yield {'item': 'first'} + await asyncio.Event().wait() + finally: + assert host.tenant.get() == ctx.workspace_uuid + closed.set() + + return source() + + stream = api(host, handler).stream('test.stream', context('workspace-a'), {}) + assert await anext(stream) == {'item': 'first'} + assert_consumer_unscoped(host) + await asyncio.create_task(stream.aclose()) + assert closed.is_set() + assert_consumer_unscoped(host) + + +@pytest.mark.asyncio +async def test_consumer_cancel_closes_source_and_restores_scopes(host): + started, closed = asyncio.Event(), asyncio.Event() + + async def handler(ap, ctx, payload): + async def source(): + try: + started.set() + await asyncio.Event().wait() + yield {} + finally: + closed.set() + + return source() + + stream = api(host, handler).stream('test.stream', context('workspace-a'), {}) + consumer = asyncio.create_task(anext(stream)) + await asyncio.wait_for(started.wait(), 2) + consumer.cancel() + with pytest.raises(asyncio.CancelledError): + await consumer + assert closed.is_set() + assert_consumer_unscoped(host) + + +@pytest.mark.asyncio +async def test_late_stream_failure_is_mapped_without_private_provider_text(host): + closed = asyncio.Event() + + async def handler(ap, ctx, payload): + async def source(): + try: + yield {'item': 'first'} + raise RuntimeError('private provider credential') + finally: + closed.set() + + return source() + + stream = api(host, handler).stream('test.stream', context('workspace-a'), {}) + async with aclosing(stream): + assert await anext(stream) == {'item': 'first'} + with pytest.raises(ApplicationAPIException) as exc: + await anext(stream) + assert exc.value.error.code == 'application_error' + assert 'private' not in exc.value.error.message + assert closed.is_set() + assert_consumer_unscoped(host) diff --git a/tests/unit_tests/api/test_application_workspace_streams.py b/tests/unit_tests/api/test_application_workspace_streams.py new file mode 100644 index 000000000..782e056a7 --- /dev/null +++ b/tests/unit_tests/api/test_application_workspace_streams.py @@ -0,0 +1,158 @@ +"""Streaming lifetimes: debug cancellation versus durable Assistant execution.""" + +from __future__ import annotations + +import asyncio +import contextvars +from contextlib import asynccontextmanager +from types import SimpleNamespace + +import pytest +from langbot_plugin.api.entities.builtin.provider.message import MessageChunk + +from langbot.pkg.api.application.service_operations import TurnInput, _assistant_service +from langbot.pkg.api.application.workspace_streams import AgentDebugInput, agent_debug, assistant_turn +from langbot.pkg.api.http.service.assistant import AssistantError +from tests.unit_tests.api.test_management_assistant import context + +pytest_plugins = ['tests.unit_tests.api.test_management_assistant'] + + +def scoped_app(debug): + active = contextvars.ContextVar('debug_stream_scope', default=None) + + @asynccontextmanager + async def tenant_scope(workspace_uuid): + token = active.set((workspace_uuid, asyncio.current_task())) + try: + yield + finally: + active.reset(token) + + return SimpleNamespace( + persistence_mgr=SimpleNamespace(tenant_scope=tenant_scope), + agent_service=SimpleNamespace(debug_agent=debug), + ), active + + +@pytest.mark.asyncio +async def test_debug_stream_delivers_callbacks_before_completion_and_propagates_failure(): + release = asyncio.Event() + + async def debug(ctx, agent_uuid, payload, on_result): + assert active.get() == (ctx.workspace_uuid, asyncio.current_task()) + await on_result({'text': payload['text']}) + await release.wait() + return {'agent_uuid': agent_uuid, 'status': 'completed'} + + ap, active = scoped_app(debug) + stream = await agent_debug(ap, context(), AgentDebugInput(agent_uuid='agent', payload={'text': 'partial'})) + assert await anext(stream) == {'kind': 'result', 'data': {'text': 'partial'}} + assert active.get() is None + release.set() + assert await anext(stream) == {'kind': 'completed', 'data': {'agent_uuid': 'agent', 'status': 'completed'}} + await stream.aclose() + + async def failed(*args, **kwargs): + raise ValueError('invalid debug input') + + ap.agent_service.debug_agent = failed + failed_stream = await agent_debug(ap, context(), AgentDebugInput(agent_uuid='agent', payload={})) + with pytest.raises(ValueError, match='invalid debug input'): + await anext(failed_stream) + + +@pytest.mark.asyncio +async def test_debug_stream_applies_backpressure_and_close_cancels_producer(): + completed_callbacks = 0 + saturated = asyncio.Event() + cancelled = asyncio.Event() + + async def debug(ctx, agent_uuid, payload, on_result): + nonlocal completed_callbacks + try: + for index in range(100): + if index == 33: + saturated.set() + await on_result({'index': index}) + completed_callbacks += 1 + finally: + cancelled.set() + + ap, _ = scoped_app(debug) + stream = await agent_debug(ap, context(), AgentDebugInput(agent_uuid='agent', payload={})) + assert (await anext(stream))['data']['index'] == 0 + await asyncio.wait_for(saturated.wait(), 1) + assert completed_callbacks == 33 + assert not cancelled.is_set() + await stream.aclose() + assert cancelled.is_set() + + +@pytest.mark.asyncio +async def test_assistant_stream_disconnect_preserves_run_and_bounded_queue(assistant): + _, ap, provider = assistant + service = _assistant_service(ap) + release = asyncio.Event() + + async def model_stream(**kwargs): + yield MessageChunk(role='assistant', content='first') + await release.wait() + for _ in range(100): + yield MessageChunk(role='assistant', content='x') + yield MessageChunk(role='assistant', content=' done', is_final=True) + + provider.invoke_llm_stream = model_stream + ctx = context() + conversation_id = (await service.create(ctx))['uuid'] + stream = await assistant_turn(ap, ctx, TurnInput(conversation_id=conversation_id, revision=0, text='Hi')) + try: + while (await asyncio.wait_for(anext(stream), 2))['kind'] != 'text': + pass + task = service._runs[conversation_id]['task'] + queue = service._runs[conversation_id]['queue'] + assert queue.maxsize == 32 + await stream.aclose() + assert not task.done() + assert _assistant_service(ap) is service + assert (await service.progress(ctx, conversation_id))['progress']['text'] == 'first' + release.set() + await asyncio.wait_for(task, 3) + completed = await service.get(ctx, conversation_id) + assert completed['status'] == 'ready' + assert completed['messages'][-1]['content'] == 'first' + 'x' * 100 + ' done' + assert queue.qsize() == 32 + frames = [queue.get_nowait() for _ in range(queue.qsize())] + assert frames[-1]['kind'] == 'completed' + with pytest.raises(AssistantError, match='conversation_not_found'): + await assistant_turn( + ap, context(account='other'), TurnInput(conversation_id=conversation_id, revision=1, text='steal') + ) + with pytest.raises(ValueError, match='Provide text or an approval decision'): + await assistant_turn( + ap, ctx, TurnInput(conversation_id=conversation_id, revision=1, text='both', approved=True) + ) + finally: + release.set() + await stream.aclose() + for run in list(service._runs.values()): + await asyncio.wait_for(run['task'], 3) + + +@pytest.mark.asyncio +async def test_debug_provider_cancellation_wakes_subscriber_without_timeout(): + finished = asyncio.Event() + + async def cancelled(*args, **kwargs): + try: + raise asyncio.CancelledError() + finally: + finished.set() + + ap, active = scoped_app(cancelled) + stream = await agent_debug(ap, context(), AgentDebugInput(agent_uuid='agent', payload={})) + with pytest.raises(asyncio.CancelledError): + await asyncio.wait_for(anext(stream), 0.5) + assert finished.is_set() + assert active.get() is None + await stream.aclose() diff --git a/tests/unit_tests/api/test_execution_application_api.py b/tests/unit_tests/api/test_execution_application_api.py new file mode 100644 index 000000000..b2af9e1a8 --- /dev/null +++ b/tests/unit_tests/api/test_execution_application_api.py @@ -0,0 +1,524 @@ +from __future__ import annotations + +import asyncio +import contextlib +import contextvars +from types import SimpleNamespace +from unittest.mock import AsyncMock, Mock + +import pytest +from langbot_plugin.entities.io.context import ActionContext, InstallationBinding +from langbot_plugin.entities.io.resp import ActionResponse + +from langbot.pkg.api.application.contracts import ApplicationAPIException +from langbot.pkg.api.application.execution import ExecutionAPI, ExecutionAPIContext +from langbot.pkg.plugin.handler import RuntimeConnectionHandler +from langbot.pkg.agent.runner.session_registry import get_session_registry + + +@pytest.fixture +def host(): + binding = InstallationBinding( + instance_uuid='instance-a', + workspace_uuid='workspace-a', + placement_generation=7, + installation_uuid='00000000-0000-4000-8000-000000000001', + runtime_revision=1, + artifact_digest='a' * 64, + ) + tenant = contextvars.ContextVar('execution_test_tenant', default=None) + + class Persistence: + execute_async = AsyncMock() + + @contextlib.asynccontextmanager + async def tenant_scope(self, workspace): + token = tenant.set(workspace) + try: + yield + finally: + tenant.reset(token) + + ap = SimpleNamespace( + deployment=SimpleNamespace(mode='cloud'), + persistence_mgr=Persistence(), + logger=Mock(), + workspace_service=SimpleNamespace(get_execution_binding=AsyncMock(return_value=binding)), + instance_config=SimpleNamespace(data={}), + ) + handler = RuntimeConnectionHandler(Mock(), AsyncMock(return_value=True), ap) + handler.register_installation_binding(binding, plugin_author='author-a', plugin_name='plugin-a') + ap.plugin_connector = SimpleNamespace(handler=handler) + return SimpleNamespace(ap=ap, handler=handler, binding=binding, api=ExecutionAPI(ap), tenant=tenant) + + +def run_session(host, **changes): + result = { + 'run_id': 'run-a', + 'plugin_identity': 'author-a/plugin-a', + 'authorization': { + 'workspace_id': host.binding.workspace_uuid, + 'available_apis': {}, + 'resources': {}, + 'authorized_ids': {}, + }, + } + result.update(changes) + return result + + +def stub_run(monkeypatch, session): + monkeypatch.setattr(get_session_registry(), 'get', AsyncMock(return_value=session)) + + +def install_action(host, name, action): + # The production constructor applies this wrapper after registration. + host.handler.actions[name] = action + host.handler._secure_plugin_actions() + + +@pytest.mark.asyncio +async def test_catalog_covers_all_core_business_actions_and_keeps_transport_internal(host): + excluded = {'initialize_plugin_settings', 'get_plugin_settings'} + expected = {name for name in host.handler.actions if not name.startswith('__') and name not in excluded} + assert {op.action for op in host.api.operations()} == expected + assert len(host.api.catalog()) == len(expected) + for operation in host.api.catalog(): + assert operation['input_schema']['additionalProperties'] is False + assert operation['output_schema'] + result = await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding)) + assert result['version'] + assert host.handler.current_action_context is None + assert host.tenant.get() is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'name', ['__ping', '__file_chunk', 'initialize_plugin_settings', 'get_plugin_settings', 'missing'] +) +async def test_runtime_control_actions_are_not_exposed(host, name): + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke(f'execution.{name}', ExecutionAPIContext(host.binding)) + assert exc.value.error.code == 'unknown_operation' + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'payload', + [ + {'run_id': None}, + {'run_id': 'another-run'}, + {'workspace_uuid': 'workspace-b'}, + {'caller_plugin_identity': 'other/plugin'}, + {'action_context': {}}, + {'placement_generation': 8}, + ], +) +async def test_payload_cannot_supply_authentication_fields(host, payload): + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding), payload) + assert exc.value.error.code == 'invalid_input' + assert host.handler.current_action_context is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize('field,value', [('runtime_revision', 2), ('artifact_digest', 'b' * 64)]) +async def test_stale_installation_is_rejected_before_business_dispatch(host, field, value): + altered = host.binding.model_copy(update={field: value}) + business = AsyncMock(return_value=ActionResponse.success({'version': 'test'})) + install_action(host, 'get_langbot_version', business) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(altered)) + assert exc.value.error.code == 'permission_denied' + business.assert_not_called() + assert host.handler.current_action_context is None + assert host.tenant.get() is None + + +@pytest.mark.asyncio +async def test_stale_generation_and_incomplete_binding_are_rejected(host): + host.ap.workspace_service.get_execution_binding.return_value = host.binding.model_copy( + update={'placement_generation': 8} + ) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding)) + assert exc.value.error.code == 'permission_denied' + incomplete = ActionContext(instance_uuid='instance-a', workspace_uuid='workspace-a', placement_generation=7) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(incomplete)) + assert exc.value.error.code == 'invalid_authentication' + + +@pytest.mark.asyncio +@pytest.mark.parametrize('session', [None, {'workspace': 'workspace-b'}, {'plugin': 'other/plugin'}]) +async def test_run_context_cannot_cross_workspace_or_plugin(host, monkeypatch, session): + actual = run_session(host) + if session is None: + actual = None + elif 'workspace' in session: + actual['authorization']['workspace_id'] = session['workspace'] + else: + actual['plugin_identity'] = session['plugin'] + stub_run(monkeypatch, actual) + business = AsyncMock(return_value=ActionResponse.success({'version': 'test'})) + install_action(host, 'get_langbot_version', business) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding, 'run-a')) + assert exc.value.error.code == 'permission_denied' + business.assert_not_called() + + +@pytest.mark.asyncio +async def test_run_identity_is_injected_and_cannot_fall_back_to_plugin_privileges(host, monkeypatch): + stub_run(monkeypatch, run_session(host)) + business = AsyncMock(return_value=ActionResponse.success({'version': 'test'})) + install_action(host, 'get_langbot_version', business) + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding, 'run-a')) + assert business.call_args.args[0] == {'run_id': 'run-a', 'caller_plugin_identity': 'author-a/plugin-a'} + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_bots', ExecutionAPIContext(host.binding, 'run-a')) + assert exc.value.error.code == 'permission_denied' + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_prompt', ExecutionAPIContext(host.binding)) + assert exc.value.error.code == 'permission_denied' + + +@pytest.mark.asyncio +async def test_resource_denial_uses_real_secured_host_action(host, monkeypatch): + stub_run(monkeypatch, run_session(host)) + host.ap.model_mgr = SimpleNamespace(get_model_by_uuid=AsyncMock()) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.invoke_llm', + ExecutionAPIContext(host.binding, 'run-a'), + { + 'llm_model_uuid': 'ungranted-model', + 'messages': [], + }, + ) + assert 'not authorized' in exc.value.error.message + host.ap.model_mgr.get_model_by_uuid.assert_not_called() + assert host.handler.current_action_context is None + + +@pytest.mark.asyncio +async def test_run_capability_denial_uses_real_secured_host_action(host, monkeypatch): + stub_run(monkeypatch, run_session(host)) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.state_get', ExecutionAPIContext(host.binding, 'run-a'), {'scope': 'conversation', 'key': 'x'} + ) + assert 'not authorized' in exc.value.error.message + assert host.handler.current_action_context is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'payload', + [ + {'llm_model_uuid': 1, 'messages': []}, + {'llm_model_uuid': 'model', 'messages': 'invalid'}, + {'llm_model_uuid': 'model', 'messages': [], 'extra_args': {'x': object()}}, + ], +) +async def test_input_type_validation_precedes_provider_calls(host, payload): + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.invoke_llm', ExecutionAPIContext(host.binding), payload) + assert exc.value.error.code == 'invalid_input' + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'response', + [ + ActionResponse.success({'tokens': 'wrong'}), + ActionResponse.success({}), + ActionResponse.success({'tokens': 1, 'unknown': object()}), + {'tokens': 1}, + ], +) +async def test_invalid_output_is_not_reported_as_success(host, response): + install_action(host, 'count_tokens', AsyncMock(return_value=response)) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.count_tokens', ExecutionAPIContext(host.binding), {'llm_model_uuid': 'model', 'messages': []} + ) + assert exc.value.error.code == 'invalid_output' + + +@pytest.mark.asyncio +async def test_failure_and_cancellation_restore_outer_context(host): + outer = host.binding.model_copy(update={'workspace_uuid': 'outer'}) + token = host.handler._current_action_context.set(outer) + try: + install_action(host, 'get_langbot_version', AsyncMock(side_effect=RuntimeError('private provider error'))) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding)) + assert exc.value.error.message == 'Execution operation failed' + assert host.handler.current_action_context is outer + install_action(host, 'get_langbot_version', AsyncMock(side_effect=asyncio.CancelledError())) + with pytest.raises(asyncio.CancelledError): + await host.api.invoke('execution.get_langbot_version', ExecutionAPIContext(host.binding)) + assert host.handler.current_action_context is outer + assert host.tenant.get() is None + finally: + host.handler._current_action_context.reset(token) + + +@pytest.mark.asyncio +async def test_stream_isolates_context_from_consumer_and_closes_on_early_exit(host): + closed = asyncio.Event() + + async def provider(data): + try: + assert host.handler.current_action_context == host.binding + assert host.tenant.get() == 'workspace-a' + yield ActionResponse.success({'chunk': {'text': 'first'}}) + await asyncio.Event().wait() + finally: + assert host.handler.current_action_context == host.binding + closed.set() + + install_action(host, 'invoke_llm_stream', provider) + stream = host.api.stream( + 'execution.invoke_llm_stream', ExecutionAPIContext(host.binding), {'llm_model_uuid': 'model', 'messages': []} + ) + async with contextlib.aclosing(stream): + assert await anext(stream) == {'chunk': {'text': 'first'}} + assert host.handler.current_action_context is None + assert host.tenant.get() is None + assert closed.is_set() + assert host.handler.current_action_context is None + assert host.tenant.get() is None + + +@pytest.mark.asyncio +async def test_stream_reports_late_failure_and_cancels_provider(host): + closed = asyncio.Event() + + async def provider(data): + try: + yield ActionResponse.success({'chunk': {'text': 'first'}}) + yield ActionResponse.error('provider refused request') + finally: + closed.set() + + install_action(host, 'invoke_llm_stream', provider) + async with contextlib.aclosing( + host.api.stream( + 'execution.invoke_llm_stream', + ExecutionAPIContext(host.binding), + {'llm_model_uuid': 'model', 'messages': []}, + ) + ) as stream: + assert await anext(stream) == {'chunk': {'text': 'first'}} + with pytest.raises(ApplicationAPIException, match='provider refused request'): + await anext(stream) + assert closed.is_set() + assert host.tenant.get() is None + + +@pytest.mark.asyncio +async def test_stream_cancellation_closes_provider(host): + started, closed = asyncio.Event(), asyncio.Event() + + async def provider(data): + try: + started.set() + await asyncio.Event().wait() + yield ActionResponse.success({'chunk': {}}) + finally: + closed.set() + + install_action(host, 'invoke_llm_stream', provider) + stream = host.api.stream( + 'execution.invoke_llm_stream', ExecutionAPIContext(host.binding), {'llm_model_uuid': 'model', 'messages': []} + ) + consumer = asyncio.create_task(anext(stream)) + await asyncio.wait_for(started.wait(), timeout=2) + consumer.cancel() + with pytest.raises(asyncio.CancelledError): + await consumer + assert closed.is_set() + assert host.handler.current_action_context is None + assert host.tenant.get() is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'operation,payload', + [ + ('call_tool', {'tool_name': 'ungranted-tool', 'parameters': {}}), + ('get_tool_detail', {'tool_name': 'ungranted-tool'}), + ('retrieve_knowledge', {'kb_id': 'ungranted-kb', 'query_text': 'test'}), + ('invoke_embedding', {'embedding_model_uuid': 'ungranted-model', 'texts': ['test']}), + ('invoke_rerank', {'rerank_model_uuid': 'ungranted-model', 'query': 'test', 'documents': ['test']}), + ('get_binary_storage', {'owner_type': 'workspace', 'key': 'key'}), + ], +) +async def test_resource_families_preserve_frozen_grants(host, monkeypatch, operation, payload): + stub_run(monkeypatch, run_session(host)) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke(f'execution.{operation}', ExecutionAPIContext(host.binding, 'run-a'), payload) + assert 'not authorized' in exc.value.error.message + assert host.handler.current_action_context is None + + +@pytest.mark.asyncio +@pytest.mark.parametrize( + 'operation,payload', + [ + ('history_page', {}), + ('event_get', {'event_id': 'event-a'}), + ('bind_box', {'box_id': 'box-a'}), + ('export_box_files', {'paths': ['report.txt']}), + ], +) +async def test_run_capability_families_cannot_fall_back_to_plugin_access(host, monkeypatch, operation, payload): + stub_run(monkeypatch, run_session(host)) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke(f'execution.{operation}', ExecutionAPIContext(host.binding, 'run-a'), payload) + assert 'not authorized' in exc.value.error.message + + +@pytest.mark.asyncio +async def test_box_export_preserves_structured_failure(host, monkeypatch): + from langbot.pkg.box.export_errors import export_error + + stub_run(monkeypatch, run_session(host)) + error = export_error('file_bytes', limit=100, actual=101, path='report.txt') + install_action( + host, + 'export_box_files', + AsyncMock( + return_value=ActionResponse.error( + str(error), + data={'error': error.error.model_dump(mode='json')}, + ) + ), + ) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.export_box_files', ExecutionAPIContext(host.binding, 'run-a'), {'paths': ['report.txt']} + ) + assert exc.value.error.details['error']['code'] == 'box.export.limit_exceeded' + assert exc.value.error.details['error']['details']['files_retained'] is True + + +@pytest.mark.asyncio +async def test_frozen_tool_source_cannot_be_replaced(host, monkeypatch): + session = run_session(host) + session['authorization']['authorized_ids'] = {'tool': {'tool-a'}} + session['authorization']['resources'] = {'tools': [{'tool_name': 'tool-a'}]} + stub_run(monkeypatch, session) + host.ap.tool_mgr = SimpleNamespace(execute_func_call=AsyncMock()) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.call_tool', ExecutionAPIContext(host.binding, 'run-a'), {'tool_name': 'tool-a', 'parameters': {}} + ) + assert 'source' in exc.value.error.message.lower() + host.ap.tool_mgr.execute_func_call.assert_not_called() + + +@pytest.mark.asyncio +async def test_runner_admin_operations_do_not_grant_admin_implicitly(host): + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke('execution.run_stats', ExecutionAPIContext(host.binding)) + assert 'not authorized' in exc.value.error.message + + +@pytest.mark.asyncio +async def test_query_identity_cannot_be_replaced_in_run_context(host, monkeypatch): + stub_run(monkeypatch, run_session(host)) + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.invoke_llm', + ExecutionAPIContext(host.binding, 'run-a'), + { + 'llm_model_uuid': 'model', + 'messages': [], + 'query_id': 'other-query', + }, + ) + assert exc.value.error.code == 'invalid_input' + + +@pytest.mark.asyncio +async def test_provider_cancellation_cannot_leave_stream_consumer_waiting(host): + async def provider(data): + raise asyncio.CancelledError() + yield + + install_action(host, 'invoke_llm_stream', provider) + stream = host.api.stream( + 'execution.invoke_llm_stream', ExecutionAPIContext(host.binding), {'llm_model_uuid': 'model', 'messages': []} + ) + with pytest.raises(asyncio.CancelledError): + await asyncio.wait_for(anext(stream), timeout=2) + assert host.tenant.get() is None + + +@pytest.mark.asyncio +async def test_concurrent_streams_keep_independent_installation_contexts(host): + other = host.binding.model_copy( + update={'workspace_uuid': 'workspace-b', 'installation_uuid': '00000000-0000-4000-8000-000000000002'} + ) + host.handler.register_installation_binding(other, plugin_author='author-a', plugin_name='plugin-a') + host.ap.workspace_service.get_execution_binding.side_effect = ( + lambda workspace, **kwargs: host.binding if workspace == 'workspace-a' else other + ) + + async def provider(data): + workspace = host.handler.current_action_context.workspace_uuid + for _ in range(2): + await asyncio.sleep(0) + assert host.handler.current_action_context.workspace_uuid == workspace + assert host.tenant.get() == workspace + yield ActionResponse.success({'chunk': {'workspace': workspace}}) + + install_action(host, 'invoke_llm_stream', provider) + + async def consume(binding): + results = [] + async with contextlib.aclosing( + host.api.stream( + 'execution.invoke_llm_stream', ExecutionAPIContext(binding), {'llm_model_uuid': 'model', 'messages': []} + ) + ) as stream: + async for chunk in stream: + assert host.handler.current_action_context is None + assert host.tenant.get() is None + results.append(chunk['chunk']['workspace']) + return results + + assert await asyncio.gather(consume(host.binding), consume(other)) == [['workspace-a'] * 2, ['workspace-b'] * 2] + + +@pytest.mark.asyncio +async def test_tool_parameters_reach_legacy_plugin_and_run_handlers(host, monkeypatch): + business = AsyncMock(return_value=ActionResponse.success({'result': 'ok'})) + install_action(host, 'call_tool', business) + await host.api.invoke( + 'execution.call_tool', ExecutionAPIContext(host.binding), {'tool_name': 'tool', 'parameters': {'x': 1}} + ) + assert business.call_args.args[0]['tool_parameters'] == {'x': 1} + assert 'parameters' not in business.call_args.args[0] + stub_run(monkeypatch, run_session(host)) + await host.api.invoke('execution.call_tool', ExecutionAPIContext(host.binding, 'run-a'), {'tool_name': 'tool'}) + assert business.call_args.args[0]['parameters'] == {} + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.call_tool', + ExecutionAPIContext(host.binding, 'run-a'), + {'tool_name': 'tool', 'tool_parameters': {}}, + ) + assert exc.value.error.code == 'invalid_input' + + +@pytest.mark.asyncio +async def test_platform_call_requires_bot_identity_outside_run(host): + with pytest.raises(ApplicationAPIException) as exc: + await host.api.invoke( + 'execution.call_platform_api', ExecutionAPIContext(host.binding), {'action': 'send_message'} + ) + assert exc.value.error.code == 'invalid_input'