feat(tenancy): add Workspace multi-tenant foundation (#2353)

* Document multi-tenant workspace architecture

* Add OSS and commercial workspace boundaries

* docs: redesign multi-tenant workspace architecture

* feat(tenancy): implement workspace isolation

* docs(tenancy): record verification evidence

* docs(tenancy): revise single-instance SaaS topology

* docs(tenancy): refine architecture options

* docs: finalize cloud v2 multi-tenant decisions

* feat(tenancy): establish cloud isolation foundations

* feat(tenancy): harden shared cloud runtime boundaries

* docs(tenancy): record final isolation verification

* fix(tenancy): close isolation and permission gaps

* docs(tenancy): record final isolation verification

* feat(tenancy): connect cloud workspace control plane

* fix(build): install git for pinned SDK

* docs(cloud): update control plane verification

* chore: update multi-tenant SDK pin

* fix(cloud): skip legacy model sync during startup

* test(cloud): preserve minimal model manager fixtures

* fix(cloud): preserve authenticated account context

* fix(cloud): reuse authenticated account for user info

* feat(cloud): complete Workspace settings navigation

* test(web): cover Workspace dropdown menu

* feat(web): place workspace controls in sidebar

* refactor(web): streamline workspace controls

* style(web): format workspace layout test

* fix(cloud): surface runtime and workspace plan status

* fix(plugin): keep runtime identity stable across restarts

* fix(ui): widen and center workspace switcher

* fix(ui): hide roles from workspace switcher

* fix(ui): align workspace switcher with sidebar entries

* feat(workspace): add in-product collaboration and direct Cloud launch

* style: format collaboration changes

* fix(workspace): bind collaboration APIs to tenant UoW

* fix(cloud): preserve Core-owned collaboration state

* test(cloud): require Space identity for invite registration

* feat(cloud): complete secure invitation experience

* style(web): format invitation flows

* fix(cloud): recover box runtime without unscoped skill reload

* feat(oss): enforce invitation account and owner billing flows

* style: format OSS account service

* test(oss): cover invitation logout handoff

* fix(oss): resolve workspace owner in scoped session

* feat(cloud): harden multi-tenant runtime resources

* fix(cloud): bound runtime restart storms

* fix(cloud): eliminate periodic runtime CPU spikes

* fix(cloud): enforce instance capacity ceilings

* fix(cloud): scope public login capability discovery

* fix(cloud): bound tenant maintenance and monitoring work

* fix(runtime): bound tenant resource amplification

* fix(deps): pin green multi-tenant plugin SDK

* fix(cloud): handle unavailable skill capability

* fix(security): require authentication for image file endpoint (H-2)

- Changed /api/v1/files/image from AuthType.NONE to USER_TOKEN_OR_API_KEY
- Added Permission.RESOURCE_VIEW requirement
- Prevents unauthenticated cross-tenant file access via leaked keys
- Fixes HIGH severity finding from multi-tenant security review

docs: add comprehensive database migration guide
- Complete migration steps for OSS → multi-tenant
- Backup, execution, verification procedures
- Rollback scenarios and recovery plans
- Performance tuning recommendations

* test: add comprehensive cross-tenant isolation tests

Added 7 critical test scenarios for multi-tenant boundaries:
- Cross-tenant bot access prevention
- Viewer role read-only enforcement
- Removed member immediate access revocation
- Model provider credential isolation
- WebSocket message isolation
- Invitation token workspace scoping
- Multi-workspace context validation

These tests address P0-2 coverage gaps for:
- workspaces.py (membership & invitation flows)
- user.py (authentication & authorization)
- websocket_chat.py (real-time isolation)
- plugins.py (resource access control)

docs: finalize database migration guide

* fix(security): resolve M-1, M-2, M-3 security findings

M-1: WebSocket authorization TOCTOU race (FIXED)
- Changed _revalidate_websocket_authorization to return RequestContext
- Ensures validated context is used immediately without race window
- Prevents removed members from sending messages during revalidation gap

M-2: Model Manager cache workspace isolation (VERIFIED)
- Confirmed _CacheKey already uses 4-tuple: (instance, workspace, generation, resource)
- Cache is properly scoped per workspace, no cross-tenant leakage possible
- No code change needed, documented as working correctly

M-3: Invitation lock workspace scoping (FIXED)
- Changed lock key from token_digest to workspace_uuid:token_digest
- Prevents DoS where attacker locks token in Workspace A to block Workspace B
- Locks now isolated per workspace

All MEDIUM severity findings from security review now resolved.

* fix(cloud): unblock tenant CI and enforce knowledge quotas

* fix(tenancy): scope rerank model sync

---------

Co-authored-by: dadachann <185672915+dadachann@users.noreply.github.com>
This commit is contained in:
RockChinQ
2026-07-30 21:43:35 +08:00
committed by GitHub
parent 463b120923
commit e1ac5e0fc8
468 changed files with 78320 additions and 13137 deletions
File diff suppressed because it is too large Load Diff
+97 -2
View File
@@ -5,7 +5,9 @@ import typing
import time
from ...core import app
from ...api.http.context import ExecutionContext
from ...entity.persistence import model as persistence_model
from ...workspace.errors import WorkspaceInvariantError
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from . import token
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -16,6 +18,20 @@ LLM_USAGE_QUERY_VARIABLE = '_llm_usage'
STREAM_USAGE_QUERY_VARIABLE = '_stream_usage'
def _ensure_same_execution_scope(
expected: ExecutionContext,
actual: ExecutionContext,
*,
resource: str,
) -> None:
if (
actual.instance_uuid != expected.instance_uuid
or actual.workspace_uuid != expected.workspace_uuid
or actual.placement_generation != expected.placement_generation
):
raise WorkspaceInvariantError(f'{resource} belongs to another Workspace execution scope')
def _store_llm_usage(query: pipeline_query.Query | None, usage_info: dict | None) -> None:
"""Store the latest provider usage on the query for upstream action handlers."""
if query is None or not usage_info:
@@ -39,24 +55,61 @@ class RuntimeProvider:
def __init__(
self,
execution_context: ExecutionContext,
provider_entity: persistence_model.ModelProvider,
token_mgr: token.TokenManager,
requester: ProviderAPIRequester,
):
if provider_entity.workspace_uuid != execution_context.workspace_uuid:
raise WorkspaceInvariantError('Provider belongs to another Workspace')
self.execution_context = execution_context
self.provider_entity = provider_entity
self.token_mgr = token_mgr
self.requester = requester
def _validate_invocation(
self,
model: RuntimeLLMModel | RuntimeEmbeddingModel | RuntimeRerankModel,
execution_context: ExecutionContext,
) -> None:
_ensure_same_execution_scope(self.execution_context, execution_context, resource='Provider invocation')
_ensure_same_execution_scope(self.execution_context, model.execution_context, resource='Runtime model')
if model.provider is not self:
raise WorkspaceInvariantError('Runtime model is attached to another provider')
def _resolve_llm_execution_context(
self,
query: pipeline_query.Query | None,
execution_context: ExecutionContext | None,
) -> ExecutionContext:
if query is not None:
from ...pipeline.pool import get_query_execution_context
query_context = get_query_execution_context(query)
if execution_context is not None:
_ensure_same_execution_scope(
query_context,
execution_context,
resource='Explicit LLM invocation context',
)
return query_context
if execution_context is None:
raise WorkspaceInvariantError('LLM invocation requires an ExecutionContext when query is absent')
return execution_context
async def invoke_llm(
self,
query: pipeline_query.Query,
query: pipeline_query.Query | None,
model: RuntimeLLMModel,
messages: typing.List[provider_message.Message],
funcs: typing.List[resource_tool.LLMTool] = None,
extra_args: dict[str, typing.Any] = {},
remove_think: bool = False,
execution_context: ExecutionContext | None = None,
) -> provider_message.Message:
"""Bridge method for invoking LLM with monitoring"""
invocation_context = self._resolve_llm_execution_context(query, execution_context)
self._validate_invocation(model, invocation_context)
# Start timing for monitoring
start_time = time.time()
input_tokens = 0
@@ -130,14 +183,17 @@ class RuntimeProvider:
async def invoke_llm_stream(
self,
query: pipeline_query.Query,
query: pipeline_query.Query | None,
model: RuntimeLLMModel,
messages: typing.List[provider_message.Message],
funcs: typing.List[resource_tool.LLMTool] = None,
extra_args: dict[str, typing.Any] = {},
remove_think: bool = False,
execution_context: ExecutionContext | None = None,
) -> provider_message.MessageChunk:
"""Bridge method for invoking LLM stream with monitoring"""
invocation_context = self._resolve_llm_execution_context(query, execution_context)
self._validate_invocation(model, invocation_context)
# Start timing for monitoring
start_time = time.time()
status = 'success'
@@ -212,6 +268,8 @@ class RuntimeProvider:
model: RuntimeEmbeddingModel,
input_text: typing.List[str],
extra_args: dict[str, typing.Any] = {},
*,
execution_context: ExecutionContext,
knowledge_base_id: str | None = None,
query_text: str | None = None,
session_id: str | None = None,
@@ -219,6 +277,7 @@ class RuntimeProvider:
call_type: str | None = None,
) -> typing.List[typing.List[float]]:
"""Bridge method for invoking embedding with monitoring"""
self._validate_invocation(model, execution_context)
# Start timing for monitoring
start_time = time.time()
prompt_tokens = 0
@@ -254,6 +313,7 @@ class RuntimeProvider:
try:
await self.requester.ap.monitoring_service.record_embedding_call(
execution_context,
model_name=model.model_entity.name,
prompt_tokens=prompt_tokens,
total_tokens=total_tokens,
@@ -276,8 +336,11 @@ class RuntimeProvider:
query: str,
documents: typing.List[str],
extra_args: dict[str, typing.Any] = {},
*,
execution_context: ExecutionContext,
) -> typing.List[dict]:
"""Bridge method for invoking rerank with monitoring"""
self._validate_invocation(model, execution_context)
start_time = time.time()
status = 'success'
@@ -316,9 +379,16 @@ class RuntimeLLMModel:
def __init__(
self,
execution_context: ExecutionContext,
model_entity: persistence_model.LLMModel,
provider: RuntimeProvider,
):
_ensure_same_execution_scope(provider.execution_context, execution_context, resource='LLM model')
if model_entity.workspace_uuid != execution_context.workspace_uuid:
raise WorkspaceInvariantError('LLM model belongs to another Workspace')
if model_entity.provider_uuid != provider.provider_entity.uuid:
raise WorkspaceInvariantError('LLM model references another provider')
self.execution_context = execution_context
self.model_entity = model_entity
self.provider = provider
@@ -334,9 +404,16 @@ class RuntimeEmbeddingModel:
def __init__(
self,
execution_context: ExecutionContext,
model_entity: persistence_model.EmbeddingModel,
provider: RuntimeProvider,
):
_ensure_same_execution_scope(provider.execution_context, execution_context, resource='Embedding model')
if model_entity.workspace_uuid != execution_context.workspace_uuid:
raise WorkspaceInvariantError('Embedding model belongs to another Workspace')
if model_entity.provider_uuid != provider.provider_entity.uuid:
raise WorkspaceInvariantError('Embedding model references another provider')
self.execution_context = execution_context
self.model_entity = model_entity
self.provider = provider
@@ -352,9 +429,16 @@ class RuntimeRerankModel:
def __init__(
self,
execution_context: ExecutionContext,
model_entity: persistence_model.RerankModel,
provider: RuntimeProvider,
):
_ensure_same_execution_scope(provider.execution_context, execution_context, resource='Rerank model')
if model_entity.workspace_uuid != execution_context.workspace_uuid:
raise WorkspaceInvariantError('Rerank model belongs to another Workspace')
if model_entity.provider_uuid != provider.provider_entity.uuid:
raise WorkspaceInvariantError('Rerank model references another provider')
self.execution_context = execution_context
self.model_entity = model_entity
self.provider = provider
@@ -379,6 +463,17 @@ class ProviderAPIRequester(metaclass=abc.ABCMeta):
async def initialize(self):
pass
async def aclose(self) -> None:
"""Release requester-owned clients when its runtime provider retires.
Most built-in requesters are currently stateless, but provider
extensions may own connection pools or background resources. Keeping
the lifecycle hook on the base class lets Workspace generation changes
and application shutdown retire them deterministically.
"""
return None
async def scan_models(self, api_key: str | None = None) -> dict[str, typing.Any] | list[dict[str, typing.Any]]:
"""Scan models supported by the provider.
@@ -8,6 +8,7 @@ import litellm
from litellm import acompletion, aembedding, arerank
from .. import errors, requester
from ....utils import httpclient
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
@@ -960,14 +961,17 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
rerank_url = f'{base_url}/{str(rerank_path).strip("/")}'
try:
async with httpx.AsyncClient(timeout=timeout) as client:
async with httpx.AsyncClient(
timeout=timeout,
event_hooks=httpclient.httpx_response_limit_hooks(),
) as client:
resp = await client.post(rerank_url, headers=headers, json=payload)
resp.raise_for_status()
data = resp.json()
data = await httpclient.parse_json_response(resp)
except httpx.HTTPStatusError as e:
body = ''
try:
body = e.response.text
body = await httpclient.response_text(e.response)
except Exception:
pass
raise errors.RequesterError(f'rerank 请求失败 (HTTP {e.response.status_code}): {body or str(e)}')
@@ -1003,10 +1007,14 @@ class LiteLLMRequester(requester.ProviderAPIRequester):
models_url = f'{base_url}/models'
try:
async with httpx.AsyncClient(trust_env=True, timeout=timeout) as client:
async with httpx.AsyncClient(
trust_env=True,
timeout=timeout,
event_hooks=httpclient.httpx_response_limit_hooks(),
) as client:
response = await client.get(models_url, headers=headers)
response.raise_for_status()
payload = response.json()
payload = await httpclient.parse_json_response(response)
models = []
for item in payload.get('data', []):
+30
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import abc
import asyncio
import typing
from typing import TYPE_CHECKING
@@ -11,6 +12,32 @@ if TYPE_CHECKING:
preregistered_runners: list[typing.Type[RequestRunner]] = []
_DEFAULT_SYNC_ITERATION_LIMIT = 100_000
_T = typing.TypeVar('_T')
def _next_sync(iterator: typing.Iterator[_T]) -> tuple[bool, _T | None]:
try:
return True, next(iterator)
except StopIteration:
return False, None
async def iterate_sync(
iterable: typing.Iterable[_T],
*,
max_items: int = _DEFAULT_SYNC_ITERATION_LIMIT,
) -> typing.AsyncGenerator[_T, None]:
"""Consume a blocking SDK iterator without stalling the event loop."""
iterator = iter(iterable)
for _ in range(max(max_items, 1)):
has_item, item = await asyncio.to_thread(_next_sync, iterator)
if not has_item:
return
yield typing.cast(_T, item)
raise RuntimeError('Synchronous provider stream exceeded the event limit')
def runner_class(name: str):
@@ -43,3 +70,6 @@ class RequestRunner(abc.ABC):
) -> typing.AsyncGenerator[provider_message.Message | provider_message.MessageChunk, None]:
"""运行请求"""
pass
async def aclose(self) -> None:
"""Release request-scoped resources after one runner invocation."""
+30 -7
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
import typing
import json
import base64
from langbot.pkg.provider import runner
from langbot.pkg.core import app
@@ -11,6 +10,16 @@ from langbot.pkg.utils import image
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.libs.coze_server_api.client import AsyncCozeAPIClient
_MAX_COZE_GENERATED_CHARS = 1024 * 1024
_MAX_COZE_MEDIA_BYTES = 10 * 1024 * 1024
def _append_bounded(current: str, addition: typing.Any) -> str:
addition = str(addition or '')
if len(current) + len(addition) > _MAX_COZE_GENERATED_CHARS:
raise ValueError('Coze response exceeds the runtime limit')
return current + addition
@runner.runner_class('coze-api')
class CozeAPIRunner(runner.RequestRunner):
@@ -77,7 +86,10 @@ class CozeAPIRunner(runner.RequestRunner):
content_parts.append({'type': 'text', 'text': ce.text})
elif ce.type == 'image_base64':
image_b64, image_format = await image.extract_b64_and_format(ce.image_base64)
file_bytes = base64.b64decode(image_b64)
file_bytes = await image.decode_base64_limited(
image_b64,
max_bytes=_MAX_COZE_MEDIA_BYTES,
)
file_id = await self._get_file_id(file_bytes)
content_parts.append({'type': 'image', 'file_id': file_id})
elif ce.type == 'file':
@@ -144,7 +156,7 @@ class CozeAPIRunner(runner.RequestRunner):
auto_save_history=self.auto_save_history,
stream=True,
):
self.ap.logger.debug(f'coze-chat-stream: {chunk}')
self.ap.logger.debug(f'coze-chat-stream: {str(chunk)[:1000]}')
event_type = chunk.get('event')
data = chunk.get('data', {})
@@ -153,11 +165,17 @@ class CozeAPIRunner(runner.RequestRunner):
if event_type == 'conversation.message.delta':
# 收集内容
if 'content' in data:
full_content += data.get('content', '')
full_content = _append_bounded(
full_content,
data.get('content', ''),
)
# 收集推理内容(如果有)
if 'reasoning_content' in data:
full_reasoning += data.get('reasoning_content', '')
full_reasoning = _append_bounded(
full_reasoning,
data.get('reasoning_content', ''),
)
elif event_type.split('.')[-1] == 'done': # 本地部署coze时,结束event不为done
# 保存会话ID
@@ -179,6 +197,8 @@ class CozeAPIRunner(runner.RequestRunner):
remove_think = self.pipeline_config.get('output', {}).get('misc', {}).get('remove-think', False)
if not remove_think:
content = f'<think>\n{full_reasoning}\n</think>\n{content}'.strip()
if len(content) > _MAX_COZE_GENERATED_CHARS:
raise ValueError('Coze response exceeds the runtime limit')
# 一次性返回完整内容
yield provider_message.Message(
@@ -227,7 +247,7 @@ class CozeAPIRunner(runner.RequestRunner):
auto_save_history=self.auto_save_history,
stream=True,
):
self.ap.logger.debug(f'coze-chat-stream-chunk: {chunk}')
self.ap.logger.debug(f'coze-chat-stream-chunk: {str(chunk)[:1000]}')
event_type = chunk.get('event')
data = chunk.get('data', {})
@@ -263,7 +283,7 @@ class CozeAPIRunner(runner.RequestRunner):
error_msg = f'Coze API错误: {data.get("message", "未知错误")}'
yield provider_message.MessageChunk(role='assistant', content=error_msg, finish_reason='error')
return
full_content += content
full_content = _append_bounded(full_content, content)
if message_idx % 8 == 0 or is_final:
if full_content:
yield provider_message.MessageChunk(role='assistant', content=full_content, is_final=is_final)
@@ -286,3 +306,6 @@ class CozeAPIRunner(runner.RequestRunner):
else:
async for msg in self._chat_messages(query):
yield msg
async def aclose(self) -> None:
await self.coze.close()
@@ -1,5 +1,6 @@
from __future__ import annotations
import asyncio
import typing
import re
@@ -10,6 +11,9 @@ from ...core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
_MAX_DASHSCOPE_RESPONSE_CHARS = 1024 * 1024
_MAX_DASHSCOPE_REFERENCES = 1024
class DashscopeAPIError(Exception):
"""Dashscope API 请求失败"""
@@ -19,6 +23,13 @@ class DashscopeAPIError(Exception):
super().__init__(self.message)
def _append_bounded(current: str, addition: typing.Any) -> str:
addition = str(addition or '')
if len(current) + len(addition) > _MAX_DASHSCOPE_RESPONSE_CHARS:
raise DashscopeAPIError('Dashscope response exceeds the runtime limit')
return current + addition
@runner.runner_class('dashscope-app-api')
class DashScopeAPIRunner(runner.RequestRunner):
"阿里云百炼DashsscopeAPI对话请求器"
@@ -111,18 +122,16 @@ class DashScopeAPIRunner(runner.RequestRunner):
if remove_think:
has_thoughts = False
# 发送对话请求
response = dashscope.Application.call(
api_key=self.api_key, # 智能体应用的API Key
app_id=self.app_id, # 智能体应用的ID
prompt=plain_text, # 用户输入的文本信息
stream=True, # 流式输出
incremental_output=True, # 增量输出,使用流式输出需要开启增量输出
session_id=query.session.using_conversation.uuid, # 会话ID用于,多轮对话
response = await asyncio.to_thread(
dashscope.Application.call,
api_key=self.api_key,
app_id=self.app_id,
prompt=plain_text,
stream=True,
incremental_output=True,
session_id=query.session.using_conversation.uuid,
enable_thinking=has_thoughts,
has_thoughts=has_thoughts,
# rag_options={ # 主要用于文件交互,暂不支持
# "session_file_ids": ["FILE_ID1"], # FILE_ID1 替换为实际的临时文件ID,逗号隔开多个
# }
)
idx_chunk = 0
try:
@@ -131,7 +140,7 @@ class DashScopeAPIRunner(runner.RequestRunner):
except AttributeError:
is_stream = False
if is_stream:
for chunk in response:
async for chunk in runner.iterate_sync(response):
if chunk.get('status_code') != 200:
raise DashscopeAPIError(
f'Dashscope API 请求失败: status_code={chunk.get("status_code")} message={chunk.get("message")} request_id={chunk.get("request_id")} '
@@ -145,15 +154,27 @@ class DashScopeAPIRunner(runner.RequestRunner):
if stream_think and stream_think[0].get('thought'):
if not think_start:
think_start = True
pending_content += f'<think>\n{stream_think[0].get("thought")}'
pending_content = _append_bounded(
pending_content,
f'<think>\n{stream_think[0].get("thought")}',
)
else:
# 继续输出 reasoning_content
pending_content += stream_think[0].get('thought')
pending_content = _append_bounded(
pending_content,
stream_think[0].get('thought'),
)
elif think_start and (not stream_think or stream_think[0].get('thought') == '') and not think_end:
think_end = True
pending_content += '\n</think>\n'
pending_content = _append_bounded(
pending_content,
'\n</think>\n',
)
if stream_output.get('text') is not None:
pending_content += stream_output.get('text')
pending_content = _append_bounded(
pending_content,
stream_output.get('text'),
)
# 是否是流式最后一个chunk
is_final = False if stream_output.get('finish_reason', False) == 'null' else True
@@ -162,12 +183,14 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 从模型传出的参考资料信息中提取用于替换的字典
if references_dict_list is not None:
for doc in references_dict_list:
for doc in references_dict_list[:_MAX_DASHSCOPE_REFERENCES]:
if doc.get('index_id') is not None:
references_dict[doc.get('index_id')] = doc.get('doc_name')
# 将参考资料替换到文本中
pending_content = self._replace_references(pending_content, references_dict)
if len(pending_content) > _MAX_DASHSCOPE_RESPONSE_CHARS:
raise DashscopeAPIError('Dashscope response exceeds the runtime limit')
if idx_chunk % 8 == 0 or is_final:
yield provider_message.MessageChunk(
@@ -178,7 +201,7 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 保存当前会话的session_id用于下次对话的语境
query.session.using_conversation.uuid = stream_output.get('session_id')
else:
for chunk in response:
async for chunk in runner.iterate_sync(response):
if chunk.get('status_code') != 200:
raise DashscopeAPIError(
f'Dashscope API 请求失败: status_code={chunk.get("status_code")} message={chunk.get("message")} request_id={chunk.get("request_id")} '
@@ -192,15 +215,27 @@ class DashScopeAPIRunner(runner.RequestRunner):
if stream_think and stream_think[0].get('thought'):
if not think_start:
think_start = True
pending_content += f'<think>\n{stream_think[0].get("thought")}'
pending_content = _append_bounded(
pending_content,
f'<think>\n{stream_think[0].get("thought")}',
)
else:
# 继续输出 reasoning_content
pending_content += stream_think[0].get('thought')
pending_content = _append_bounded(
pending_content,
stream_think[0].get('thought'),
)
elif think_start and (not stream_think or stream_think[0].get('thought') == '') and not think_end:
think_end = True
pending_content += '\n</think>\n'
pending_content = _append_bounded(
pending_content,
'\n</think>\n',
)
if stream_output.get('text') is not None:
pending_content += stream_output.get('text')
pending_content = _append_bounded(
pending_content,
stream_output.get('text'),
)
# 保存当前会话的session_id用于下次对话的语境
query.session.using_conversation.uuid = stream_output.get('session_id')
@@ -210,12 +245,14 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 从模型传出的参考资料信息中提取用于替换的字典
if references_dict_list is not None:
for doc in references_dict_list:
for doc in references_dict_list[:_MAX_DASHSCOPE_REFERENCES]:
if doc.get('index_id') is not None:
references_dict[doc.get('index_id')] = doc.get('doc_name')
# 将参考资料替换到文本中
pending_content = self._replace_references(pending_content, references_dict)
pending_content = self._replace_references(pending_content, references_dict)
if len(pending_content) > _MAX_DASHSCOPE_RESPONSE_CHARS:
raise DashscopeAPIError('Dashscope response exceeds the runtime limit')
yield provider_message.Message(
role='assistant',
@@ -240,18 +277,16 @@ class DashScopeAPIRunner(runner.RequestRunner):
biz_params.update(query.variables)
# 发送对话请求
response = dashscope.Application.call(
api_key=self.api_key, # 智能体应用的API Key
app_id=self.app_id, # 智能体应用的ID
prompt=plain_text, # 用户输入的文本信息
stream=True, # 流式输出
incremental_output=True, # 增量输出,使用流式输出需要开启增量输出
session_id=query.session.using_conversation.uuid, # 会话ID用于,多轮对话
biz_params=biz_params, # 工作流应用的自定义输入参数传递
flow_stream_mode='message_format', # 消息模式,输出/结束节点的流式结果
# rag_options={ # 主要用于文件交互,暂不支持
# "session_file_ids": ["FILE_ID1"], # FILE_ID1 替换为实际的临时文件ID,逗号隔开多个
# }
response = await asyncio.to_thread(
dashscope.Application.call,
api_key=self.api_key,
app_id=self.app_id,
prompt=plain_text,
stream=True,
incremental_output=True,
session_id=query.session.using_conversation.uuid,
biz_params=biz_params,
flow_stream_mode='message_format',
)
# 处理API返回的流式输出
@@ -262,7 +297,7 @@ class DashScopeAPIRunner(runner.RequestRunner):
is_stream = False
idx_chunk = 0
if is_stream:
for chunk in response:
async for chunk in runner.iterate_sync(response):
if chunk.get('status_code') != 200:
raise DashscopeAPIError(
f'Dashscope API 请求失败: status_code={chunk.get("status_code")} message={chunk.get("message")} request_id={chunk.get("request_id")} '
@@ -273,7 +308,10 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 获取流式传输的output
stream_output = chunk.get('output', {})
if stream_output.get('workflow_message') is not None:
pending_content += stream_output.get('workflow_message').get('message').get('content')
pending_content = _append_bounded(
pending_content,
stream_output.get('workflow_message').get('message').get('content'),
)
# if stream_output.get('text') is not None:
# pending_content += stream_output.get('text')
@@ -284,12 +322,14 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 从模型传出的参考资料信息中提取用于替换的字典
if references_dict_list is not None:
for doc in references_dict_list:
for doc in references_dict_list[:_MAX_DASHSCOPE_REFERENCES]:
if doc.get('index_id') is not None:
references_dict[doc.get('index_id')] = doc.get('doc_name')
# 将参考资料替换到文本中
pending_content = self._replace_references(pending_content, references_dict)
if len(pending_content) > _MAX_DASHSCOPE_RESPONSE_CHARS:
raise DashscopeAPIError('Dashscope response exceeds the runtime limit')
if idx_chunk % 8 == 0 or is_final:
yield provider_message.MessageChunk(
role='assistant',
@@ -301,7 +341,7 @@ class DashScopeAPIRunner(runner.RequestRunner):
query.session.using_conversation.uuid = stream_output.get('session_id')
else:
for chunk in response:
async for chunk in runner.iterate_sync(response):
if chunk.get('status_code') != 200:
raise DashscopeAPIError(
f'Dashscope API 请求失败: status_code={chunk.get("status_code")} message={chunk.get("message")} request_id={chunk.get("request_id")} '
@@ -312,7 +352,10 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 获取流式传输的output
stream_output = chunk.get('output', {})
if stream_output.get('text') is not None:
pending_content += stream_output.get('text')
pending_content = _append_bounded(
pending_content,
stream_output.get('text'),
)
is_final = False if stream_output.get('finish_reason', False) == 'null' else True
@@ -324,12 +367,14 @@ class DashScopeAPIRunner(runner.RequestRunner):
# 从模型传出的参考资料信息中提取用于替换的字典
if references_dict_list is not None:
for doc in references_dict_list:
for doc in references_dict_list[:_MAX_DASHSCOPE_REFERENCES]:
if doc.get('index_id') is not None:
references_dict[doc.get('index_id')] = doc.get('doc_name')
# 将参考资料替换到文本中
pending_content = self._replace_references(pending_content, references_dict)
pending_content = self._replace_references(pending_content, references_dict)
if len(pending_content) > _MAX_DASHSCOPE_RESPONSE_CHARS:
raise DashscopeAPIError('Dashscope response exceeds the runtime limit')
yield provider_message.Message(
role='assistant',
+178 -34
View File
@@ -1,10 +1,11 @@
from __future__ import annotations
import asyncio
import heapq
import typing
import json
import time
import uuid
import base64
import mimetypes
import os
import re
@@ -16,19 +17,40 @@ from langbot.pkg.provider import runner
from langbot.pkg.core import app
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.platform.message as platform_message
from langbot.pkg.utils import image
from langbot.pkg.utils import httpclient, image
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.libs.dify_service_api.v1 import client, errors
import httpx
# Module-level store for paused-workflow form state. The key isolates the bot,
# pipeline, adapter, and launcher; each value holds an insertion-ordered map of
# form_token -> form_data so one conversation can pause multiple workflows.
PendingFormKey = tuple[str, str, str, str, str]
# Module-level store for paused-workflow form state. The key includes the full
# execution scope before the bot, pipeline, adapter, and launcher dimensions;
# each value holds an insertion-ordered map of form_token -> form_data so one
# conversation can pause multiple workflows without crossing Workspaces or
# placement generations.
PendingFormKey = tuple[str, str, int, str, str, str, str, str]
_PENDING_FORMS: dict[PendingFormKey, 'OrderedDict[str, dict[str, typing.Any]]'] = {}
_PENDING_FORM_EXPIRY_HEAP: list[tuple[float, int, PendingFormKey, str]] = []
_PENDING_FORM_ACTIVE_COUNT = 0
_PENDING_FORM_REVISION = 0
_PENDING_FORM_DEFAULT_TTL = 30 * 60 # 30 minutes safety cap
_PENDING_FORM_MAX_SESSIONS = 4096
_PENDING_FORM_MAX_PER_SESSION = 16
_PENDING_FORM_HEAP_COMPACT_FLOOR = 64
_PENDING_FORM_HEAP_MAX_MULTIPLIER = 4
_PENDING_FORM_REVISION_KEY = '_langbot_cache_revision'
_STREAM_FORM_PLACEHOLDER = '\u200b'
_MAX_DIFY_UPLOAD_BYTES = 10 * 1024 * 1024
def _read_local_file_limited(path: str) -> bytes:
"""Read a local platform attachment without allowing an oversized allocation."""
if os.path.getsize(path) > _MAX_DIFY_UPLOAD_BYTES:
raise ValueError('Dify upload file exceeds the size limit')
with open(path, 'rb') as file:
content = file.read(_MAX_DIFY_UPLOAD_BYTES + 1)
if len(content) > _MAX_DIFY_UPLOAD_BYTES:
raise ValueError('Dify upload file exceeds the size limit')
return content
def _merge_stream_text(accumulated: str, incoming: typing.Any) -> str:
@@ -48,10 +70,13 @@ def _dify_user_from_query(query: pipeline_query.Query) -> str:
def _session_key_from_query(query: pipeline_query.Query) -> PendingFormKey:
"""Build a process-local pending-form key isolated by bot and pipeline."""
"""Build a process-local pending-form key isolated by execution scope."""
adapter = getattr(query, 'adapter', None)
adapter_type = f'{type(adapter).__module__}.{type(adapter).__qualname__}'
return (
str(getattr(query, 'instance_uuid', '') or ''),
str(getattr(query, 'workspace_uuid', '') or ''),
int(getattr(query, 'placement_generation', 0) or 0),
str(getattr(query, 'bot_uuid', '') or ''),
str(getattr(query, 'pipeline_uuid', '') or ''),
adapter_type,
@@ -60,22 +85,103 @@ def _session_key_from_query(query: pipeline_query.Query) -> PendingFormKey:
)
def _synchronize_pending_form_cache_if_externally_cleared() -> None:
"""Keep test/debug direct cache clears from retaining stale heap entries."""
global _PENDING_FORM_ACTIVE_COUNT
if _PENDING_FORMS:
return
_PENDING_FORM_EXPIRY_HEAP.clear()
_PENDING_FORM_ACTIVE_COUNT = 0
def _pending_form_entry_is_current(
expires_at: float,
revision: int,
session_key: PendingFormKey,
form_token: str,
) -> bool:
forms = _PENDING_FORMS.get(session_key)
if forms is None:
return False
stored = forms.get(form_token)
if stored is None:
return False
return stored.get(_PENDING_FORM_REVISION_KEY) == revision and stored.get('_expires_at') == expires_at
def _peek_valid_pending_form_expiry(
*,
pop: bool = False,
) -> tuple[float, int, PendingFormKey, str] | None:
while _PENDING_FORM_EXPIRY_HEAP:
entry = _PENDING_FORM_EXPIRY_HEAP[0]
if _pending_form_entry_is_current(*entry):
if pop:
heapq.heappop(_PENDING_FORM_EXPIRY_HEAP)
return entry
heapq.heappop(_PENDING_FORM_EXPIRY_HEAP)
return None
def _drop_pending_form(session_key: PendingFormKey, form_token: str) -> None:
global _PENDING_FORM_ACTIVE_COUNT
forms = _PENDING_FORMS.get(session_key)
if forms is None or forms.pop(form_token, None) is None:
return
_PENDING_FORM_ACTIVE_COUNT = max(_PENDING_FORM_ACTIVE_COUNT - 1, 0)
if not forms:
_PENDING_FORMS.pop(session_key, None)
def _drop_pending_form_session(session_key: PendingFormKey) -> None:
global _PENDING_FORM_ACTIVE_COUNT
forms = _PENDING_FORMS.pop(session_key, None)
if forms is not None:
_PENDING_FORM_ACTIVE_COUNT = max(
_PENDING_FORM_ACTIVE_COUNT - len(forms),
0,
)
def _compact_pending_form_expiry_heap_if_needed() -> None:
max_heap_entries = max(
_PENDING_FORM_HEAP_COMPACT_FLOOR,
_PENDING_FORM_ACTIVE_COUNT * _PENDING_FORM_HEAP_MAX_MULTIPLIER,
)
if len(_PENDING_FORM_EXPIRY_HEAP) <= max_heap_entries:
return
_PENDING_FORM_EXPIRY_HEAP[:] = [
(
float(stored['_expires_at']),
int(stored[_PENDING_FORM_REVISION_KEY]),
session_key,
form_token,
)
for session_key, forms in _PENDING_FORMS.items()
for form_token, stored in forms.items()
]
heapq.heapify(_PENDING_FORM_EXPIRY_HEAP)
def _prune_pending_forms(now: float | None = None) -> None:
_synchronize_pending_form_cache_if_externally_cleared()
if now is None:
now = time.time()
for session_key in list(_PENDING_FORMS.keys()):
forms = _PENDING_FORMS[session_key]
expired_tokens = [token for token, data in forms.items() if data.get('_expires_at', 0) <= now]
for token in expired_tokens:
forms.pop(token, None)
if not forms:
_PENDING_FORMS.pop(session_key, None)
while True:
entry = _peek_valid_pending_form_expiry()
if entry is None or entry[0] > now:
break
_, _, session_key, form_token = _peek_valid_pending_form_expiry(pop=True)
_drop_pending_form(session_key, form_token)
_compact_pending_form_expiry_heap_if_needed()
def _set_pending_form(session_key: PendingFormKey, form_data: dict[str, typing.Any]) -> None:
global _PENDING_FORM_ACTIVE_COUNT, _PENDING_FORM_REVISION
_prune_pending_forms()
if isinstance(session_key, tuple) and len(session_key) > 1:
form_data['pipeline_uuid'] = session_key[1]
if isinstance(session_key, tuple) and len(session_key) == 8:
form_data['pipeline_uuid'] = session_key[4]
stored = dict(form_data)
expiration_time = stored.get('expiration_time')
try:
@@ -83,11 +189,31 @@ def _set_pending_form(session_key: PendingFormKey, form_data: dict[str, typing.A
except (TypeError, ValueError):
expiration_ts = 0.0
stored['_expires_at'] = expiration_ts or (time.time() + _PENDING_FORM_DEFAULT_TTL)
_PENDING_FORM_REVISION += 1
stored[_PENDING_FORM_REVISION_KEY] = _PENDING_FORM_REVISION
form_token = str(stored.get('form_token') or '')
forms = _PENDING_FORMS.setdefault(session_key, OrderedDict())
# Re-insert at the end so this becomes the "latest" entry
forms.pop(form_token, None)
if forms.pop(form_token, None) is None:
_PENDING_FORM_ACTIVE_COUNT += 1
forms[form_token] = stored
heapq.heappush(
_PENDING_FORM_EXPIRY_HEAP,
(
stored['_expires_at'],
_PENDING_FORM_REVISION,
session_key,
form_token,
),
)
while len(forms) > _PENDING_FORM_MAX_PER_SESSION:
oldest_token = next(iter(forms))
_drop_pending_form(session_key, oldest_token)
if len(_PENDING_FORMS) > _PENDING_FORM_MAX_SESSIONS:
oldest_entry = _peek_valid_pending_form_expiry()
if oldest_entry is not None:
_drop_pending_form_session(oldest_entry[2])
_compact_pending_form_expiry_heap_if_needed()
def _get_pending_form_by_token(session_key: PendingFormKey, form_token: str) -> dict[str, typing.Any] | None:
@@ -139,11 +265,11 @@ def _clear_pending_form(session_key: PendingFormKey, form_token: str | None = No
if not forms:
return
if form_token is None:
_PENDING_FORMS.pop(session_key, None)
_drop_pending_form_session(session_key)
_compact_pending_form_expiry_heap_if_needed()
return
forms.pop(form_token, None)
if not forms:
_PENDING_FORMS.pop(session_key, None)
_drop_pending_form(session_key, form_token)
_compact_pending_form_expiry_heap_if_needed()
def _format_human_input_text(
@@ -716,6 +842,9 @@ class DifyServiceAPIRunner(runner.RequestRunner):
base_url=self.pipeline_config['ai']['dify-service-api']['base-url'],
)
async def aclose(self) -> None:
await self.dify_client.aclose()
def _process_thinking_content(
self,
content: str,
@@ -791,13 +920,16 @@ class DifyServiceAPIRunner(runner.RequestRunner):
async def download_file(file_url: str) -> tuple[bytes, str]:
"""Download file from url (supports data url)."""
async with httpx.AsyncClient() as client_session:
resp = await client_session.get(file_url)
client_session = httpclient.get_session()
async with client_session.get(file_url, timeout=120) as resp:
resp.raise_for_status()
content_type = (
resp.headers.get('content-type') or mimetypes.guess_type(file_url)[0] or 'application/octet-stream'
)
return resp.content, content_type
return (
await httpclient.read_limited(resp, max_bytes=_MAX_DIFY_UPLOAD_BYTES),
content_type,
)
def _detect_file_type(content_type: str) -> str:
"""Map MIME to dify file type."""
@@ -815,7 +947,10 @@ class DifyServiceAPIRunner(runner.RequestRunner):
plain_text += ce.text
elif ce.type == 'image_base64':
image_b64, image_format = await image.extract_b64_and_format(ce.image_base64)
file_bytes = base64.b64decode(image_b64)
file_bytes = await image.decode_base64_limited(
image_b64,
max_bytes=_MAX_DIFY_UPLOAD_BYTES,
)
image_id = await upload_file_bytes(f'img.{image_format}', file_bytes, f'image/{image_format}')
upload_files.append({'type': 'image', 'id': image_id})
elif ce.type == 'file_url':
@@ -835,7 +970,10 @@ class DifyServiceAPIRunner(runner.RequestRunner):
content_type = 'application/octet-stream'
if ';' in header:
content_type = header.split(';')[0][5:] or content_type
file_bytes = base64.b64decode(b64_data)
file_bytes = await image.decode_base64_limited(
b64_data,
max_bytes=_MAX_DIFY_UPLOAD_BYTES,
)
file_id = await upload_file_bytes(file_name, file_bytes, content_type)
file_type = _detect_file_type(content_type)
upload_files.append({'type': file_type, 'id': file_id})
@@ -860,15 +998,19 @@ class DifyServiceAPIRunner(runner.RequestRunner):
}
async def _download_file_for_form(self, file_url: str) -> tuple[bytes, str, str]:
async with httpx.AsyncClient() as client_session:
resp = await client_session.get(file_url)
client_session = httpclient.get_session()
async with client_session.get(file_url, timeout=120) as resp:
resp.raise_for_status()
content_type = (
resp.headers.get('content-type') or mimetypes.guess_type(file_url)[0] or 'application/octet-stream'
)
parsed = urlparse(file_url)
file_name = os.path.basename(parsed.path) or 'file'
return resp.content, content_type, file_name
return (
await httpclient.read_limited(resp, max_bytes=_MAX_DIFY_UPLOAD_BYTES),
content_type,
file_name,
)
async def _platform_file_to_dify(self, item: typing.Any, user: str) -> dict | None:
try:
@@ -885,13 +1027,15 @@ class DifyServiceAPIRunner(runner.RequestRunner):
content_type = header.split(';', 1)[0][5:] or content_type
return await self._upload_file_bytes_for_user(
file_name,
base64.b64decode(b64_data),
await image.decode_base64_limited(
b64_data,
max_bytes=_MAX_DIFY_UPLOAD_BYTES,
),
content_type,
user,
)
if item.path:
with open(item.path, 'rb') as f:
file_bytes = f.read()
file_bytes = await asyncio.to_thread(_read_local_file_limited, str(item.path))
content_type = mimetypes.guess_type(str(item.path))[0] or 'application/octet-stream'
file_name = item.name or os.path.basename(str(item.path)) or 'file'
return await self._upload_file_bytes_for_user(file_name, file_bytes, content_type, user)
@@ -1,5 +1,6 @@
from __future__ import annotations
import codecs
import typing
import json
import httpx
@@ -11,6 +12,44 @@ from ...core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
_MAX_LANGFLOW_LINE_CHARS = 1024 * 1024
_MAX_LANGFLOW_TOTAL_BYTES = 16 * 1024 * 1024
_MAX_LANGFLOW_RESPONSE_BYTES = 1024 * 1024
async def _iter_limited_lines(
response: httpx.Response,
) -> typing.AsyncGenerator[str, None]:
decoder = codecs.getincrementaldecoder('utf-8')('replace')
buffer = ''
total_bytes = 0
async for chunk in response.aiter_bytes(chunk_size=8192):
total_bytes += len(chunk)
if total_bytes > _MAX_LANGFLOW_TOTAL_BYTES:
raise ValueError('Langflow stream exceeds the runtime limit')
buffer += decoder.decode(chunk)
while '\n' in buffer:
line, buffer = buffer.split('\n', 1)
if len(line) > _MAX_LANGFLOW_LINE_CHARS:
raise ValueError('Langflow event exceeds the runtime limit')
yield line.rstrip('\r')
if len(buffer) > _MAX_LANGFLOW_LINE_CHARS:
raise ValueError('Langflow event exceeds the runtime limit')
buffer += decoder.decode(b'', final=True)
if buffer:
if len(buffer) > _MAX_LANGFLOW_LINE_CHARS:
raise ValueError('Langflow event exceeds the runtime limit')
yield buffer.rstrip('\r')
async def _read_limited_response(response: httpx.Response) -> bytes:
body = bytearray()
async for chunk in response.aiter_bytes(chunk_size=8192):
body.extend(chunk)
if len(body) > _MAX_LANGFLOW_RESPONSE_BYTES:
raise ValueError('Langflow response exceeds the runtime limit')
return bytes(body)
@runner.runner_class('langflow-api')
class LangflowAPIRunner(runner.RequestRunner):
@@ -99,7 +138,7 @@ class LangflowAPIRunner(runner.RequestRunner):
accumulated_content = ''
message_count = 0
async for line in response.aiter_lines():
async for line in _iter_limited_lines(response):
data_str = line
if data_str.startswith('data: '):
@@ -144,11 +183,15 @@ class LangflowAPIRunner(runner.RequestRunner):
yield provider_message.MessageChunk(role='assistant', content=accumulated_content, is_final=True)
else:
# 非流式请求
response = await client.post(url, json=payload, headers=headers, timeout=120.0)
response.raise_for_status()
# 解析响应
response_data = response.json()
async with client.stream(
'POST',
url,
json=payload,
headers=headers,
timeout=120.0,
) as response:
response.raise_for_status()
response_data = json.loads(await _read_limited_response(response))
# 提取消息内容
# 根据Langflow API文档,响应结构可能在outputs[0].outputs[0].outputs.message.message中
+19 -5
View File
@@ -11,6 +11,7 @@ import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.rag.context as rag_context
from ...pipeline.pool import get_query_execution_context
rag_combined_prompt_template = """
The following are relevant context entries retrieved from the knowledge base.
@@ -210,7 +211,7 @@ class LocalAgentRunner(runner.RequestRunner):
req_messages.append(
provider_message.Message(
role='system',
content=self.ap.box_service.get_system_guidance(query.query_id),
content=self.ap.box_service.get_system_guidance(query),
)
)
@@ -223,11 +224,15 @@ class LocalAgentRunner(runner.RequestRunner):
) -> list[modelmgr_requester.RuntimeLLMModel]:
"""Build ordered list of models to try: primary model + fallback models."""
candidates = []
execution_context = get_query_execution_context(query)
# Primary model
if query.use_llm_model_uuid:
try:
primary = await self.ap.model_mgr.get_model_by_uuid(query.use_llm_model_uuid)
primary = await self.ap.model_mgr.get_model_by_uuid(
execution_context,
query.use_llm_model_uuid,
)
candidates.append(primary)
except ValueError:
self.ap.logger.warning(f'Primary model {query.use_llm_model_uuid} not found')
@@ -236,7 +241,10 @@ class LocalAgentRunner(runner.RequestRunner):
fallback_uuids = (query.variables or {}).get('_fallback_model_uuids', [])
for fb_uuid in fallback_uuids:
try:
fb_model = await self.ap.model_mgr.get_model_by_uuid(fb_uuid)
fb_model = await self.ap.model_mgr.get_model_by_uuid(
execution_context,
fb_uuid,
)
candidates.append(fb_model)
except ValueError:
self.ap.logger.warning(f'Fallback model {fb_uuid} not found, skipping')
@@ -346,12 +354,13 @@ class LocalAgentRunner(runner.RequestRunner):
if kb_uuids and user_message_text:
# only support text for now
all_results: list[rag_context.RetrievalResultEntry] = []
execution_context = get_query_execution_context(query)
kb_engine_plugins: set[str] = set()
# Retrieve from each knowledge base
for kb_uuid in kb_uuids:
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(kb_uuid)
kb = await self.ap.rag_mgr.get_knowledge_base_by_uuid(execution_context, kb_uuid)
if not kb:
self.ap.logger.warning(f'Knowledge base {kb_uuid} not found, skipping')
@@ -364,6 +373,7 @@ class LocalAgentRunner(runner.RequestRunner):
kb_engine_plugins.add(engine_plugin_id)
result = await kb.retrieve(
execution_context,
user_message_text,
settings={
'bot_uuid': query.bot_uuid or '',
@@ -398,7 +408,10 @@ class LocalAgentRunner(runner.RequestRunner):
)
if all_results and rerank_model_uuid:
try:
rerank_model = await self.ap.model_mgr.get_rerank_model_by_uuid(rerank_model_uuid)
rerank_model = await self.ap.model_mgr.get_rerank_model_by_uuid(
execution_context,
rerank_model_uuid,
)
rerank_top_k = int(local_agent_config.get('rerank-top-k', 5))
doc_texts = []
@@ -411,6 +424,7 @@ class LocalAgentRunner(runner.RequestRunner):
model=rerank_model,
query=user_message_text,
documents=doc_texts_capped,
execution_context=execution_context,
)
scored = sorted(scores, key=lambda x: x.get('relevance_score', 0), reverse=True)
+15 -2
View File
@@ -12,6 +12,8 @@ from ...core import app
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
_MAX_N8N_RESPONSE_CHARS = 1024 * 1024
class N8nAPIError(Exception):
"""N8n API 请求失败"""
@@ -94,6 +96,8 @@ class N8nServiceAPIRunner(runner.RequestRunner):
else:
chunk_str = str(raw_chunk)
if len(full_text) + len(chunk_str) > _MAX_N8N_RESPONSE_CHARS:
raise N8nAPIError('n8n response exceeds the runtime limit')
full_text += chunk_str
buffer += chunk_str
@@ -112,7 +116,9 @@ class N8nServiceAPIRunner(runner.RequestRunner):
if obj.get('type') == 'item' and 'content' in obj:
chunk_idx += 1
content = obj['content']
content = str(obj['content'])
if len(full_content) + len(content) > _MAX_N8N_RESPONSE_CHARS:
raise N8nAPIError('n8n response exceeds the runtime limit')
full_content += content
elif obj.get('type') == 'end':
is_final = True
@@ -128,6 +134,8 @@ class N8nServiceAPIRunner(runner.RequestRunner):
except json.JSONDecodeError:
# buffer 末尾可能是一个不完整的 JSON,等待更多数据
break
except N8nAPIError:
raise
except Exception as e:
# 记录解析失败并继续接收后续 chunk
try:
@@ -255,7 +263,12 @@ class N8nServiceAPIRunner(runner.RequestRunner):
self.webhook_url, json=payload, headers=headers, auth=auth, timeout=self.timeout
) as response:
if response.status != 200:
error_text = await response.text()
error_text = (
await httpclient.read_limited(
response,
max_bytes=_MAX_N8N_RESPONSE_CHARS,
)
).decode('utf-8', errors='replace')
self.ap.logger.error(f'n8n webhook call failed: {response.status}, {error_text}')
raise Exception(f'n8n webhook call failed: {response.status}, {error_text}')
+75 -27
View File
@@ -1,8 +1,9 @@
from __future__ import annotations
import asyncio
import typing
import json
import base64
import logging
import tempfile
import os
@@ -11,10 +12,13 @@ from tboxsdk.model.file import File, FileType
from .. import runner
from ...core import app
from ...utils import image
from ...utils import bounded_executor, image
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
import langbot_plugin.api.entities.builtin.provider.message as provider_message
_MAX_TBOX_RESPONSE_CHARS = 1024 * 1024
_MAX_TBOX_MEDIA_BYTES = 10 * 1024 * 1024
class TboxAPIError(Exception):
"""TBox API 请求失败"""
@@ -24,6 +28,19 @@ class TboxAPIError(Exception):
super().__init__(self.message)
def _append_bounded(current: str, addition: typing.Any) -> str:
addition = str(addition or '')
if len(current) + len(addition) > _MAX_TBOX_RESPONSE_CHARS:
raise TboxAPIError('Tbox response exceeds the runtime limit')
return current + addition
def _write_temp_media(file_bytes: bytes, suffix: str) -> str:
with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp_file:
tmp_file.write(file_bytes)
return tmp_file.name
@runner.runner_class('tbox-app-api')
class TboxAPIRunner(runner.RequestRunner):
"蚂蚁百宝箱API对话请求器"
@@ -42,6 +59,7 @@ class TboxAPIRunner(runner.RequestRunner):
self.api_key = self.pipeline_config['ai']['tbox-app-api']['api-key']
# 初始化Tbox client
logging.getLogger('tbox.client').setLevel(logging.WARNING)
self.tbox_client = TboxClient(authorization=self.api_key)
async def _preprocess_user_message(self, query: pipeline_query.Query) -> tuple[str, list[str]]:
@@ -59,19 +77,29 @@ class TboxAPIRunner(runner.RequestRunner):
plain_text += ce.text
elif ce.type == 'image_base64':
image_b64, image_format = await image.extract_b64_and_format(ce.image_base64)
# 创建临时文件
file_bytes = base64.b64decode(image_b64)
file_bytes = await image.decode_base64_limited(
image_b64,
max_bytes=_MAX_TBOX_MEDIA_BYTES,
)
tmp_file_path: str | None = None
try:
with tempfile.NamedTemporaryFile(suffix=f'.{image_format}', delete=False) as tmp_file:
tmp_file.write(file_bytes)
tmp_file_path = tmp_file.name
file_upload_resp = self.tbox_client.upload_file(tmp_file_path)
tmp_file_path = await asyncio.to_thread(
_write_temp_media,
file_bytes,
f'.{image_format}',
)
file_upload_resp = await asyncio.to_thread(
self.tbox_client.upload_file,
tmp_file_path,
)
image_id = file_upload_resp.get('data', '')
image_ids.append(image_id)
finally:
# 清理临时文件
if os.path.exists(tmp_file_path):
os.unlink(tmp_file_path)
if tmp_file_path and os.path.exists(tmp_file_path):
await bounded_executor.run_blocking_cleanup(
os.unlink,
tmp_file_path,
)
elif isinstance(query.user_message.content, str):
plain_text = query.user_message.content
@@ -98,18 +126,23 @@ class TboxAPIRunner(runner.RequestRunner):
files = [File(file_id=image_id, type=FileType.IMAGE) for image_id in image_ids]
# 发送对话请求
response = self.tbox_client.chat(
app_id=self.app_id, # Tbox中智能体应用的ID
user_id=query.bot_uuid, # 用户ID
query=plain_text, # 用户输入的文本信息
stream=is_stream, # 是否流式输出
conversation_id=conversation_id, # 会话ID,为None时Tbox会自动创建一个新会话
files=files, # 图片内容
response = await asyncio.to_thread(
self.tbox_client.chat,
app_id=self.app_id,
user_id=query.bot_uuid,
query=plain_text,
stream=is_stream,
conversation_id=conversation_id,
files=files,
)
if is_stream:
# 解析Tbox流式输出内容,并发送给上游
for chunk in self._process_stream_message(response, query, remove_think):
async for chunk in self._process_stream_message(
response,
query,
remove_think,
):
yield chunk
else:
message = self._process_non_stream_message(response, query, remove_think)
@@ -127,13 +160,16 @@ class TboxAPIRunner(runner.RequestRunner):
thinking_content = payload.get('reasoningContent', [])
result = ''
if thinking_content and not remove_think:
result += f'<think>\n{thinking_content[0].get("text", "")}\n</think>\n'
result = _append_bounded(
result,
f'<think>\n{thinking_content[0].get("text", "")}\n</think>\n',
)
content = payload.get('result', [])
if content:
result += content[0].get('chunk', '')
result = _append_bounded(result, content[0].get('chunk', ''))
return result
def _process_stream_message(
async def _process_stream_message(
self, response: typing.Generator[dict], query: pipeline_query.Query, remove_think: bool
):
idx_msg = 0
@@ -141,7 +177,7 @@ class TboxAPIRunner(runner.RequestRunner):
conversation_id = None
think_start = False
think_end = False
for chunk in response:
async for chunk in runner.iterate_sync(response):
if chunk.get('type', '') == 'chunk':
"""
Tbox返回的消息内容chunk结构
@@ -149,7 +185,10 @@ class TboxAPIRunner(runner.RequestRunner):
"""
# 如果包含思考过程,拼接</think>
if think_start and not think_end:
pending_content += '\n</think>\n'
pending_content = _append_bounded(
pending_content,
'\n</think>\n',
)
think_end = True
payload = chunk.get('payload', {})
@@ -158,7 +197,10 @@ class TboxAPIRunner(runner.RequestRunner):
query.session.using_conversation.uuid = conversation_id
if payload.get('text'):
idx_msg += 1
pending_content += payload.get('text')
pending_content = _append_bounded(
pending_content,
payload.get('text'),
)
elif chunk.get('type', '') == 'thinking' and not remove_think:
"""
Tbox返回的思考过程chunk结构
@@ -170,9 +212,15 @@ class TboxAPIRunner(runner.RequestRunner):
content = payload.get('ext_data', {}).get('text')
if not think_start:
think_start = True
pending_content += f'<think>\n{content}'
pending_content = _append_bounded(
pending_content,
f'<think>\n{content}',
)
else:
pending_content += content
pending_content = _append_bounded(
pending_content,
content,
)
elif chunk.get('type', '') == 'error':
raise TboxAPIError(
f'Tbox API 请求失败: status_code={chunk.get("status_code")} message={chunk.get("message")} request_id={chunk.get("request_id")} '
+17 -8
View File
@@ -10,6 +10,15 @@ import langbot_plugin.api.entities.builtin.provider.message as provider_message
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from langbot.libs.weknora_api import client, errors
_MAX_WEKNORA_GENERATED_CHARS = 1024 * 1024
def _append_bounded(current: str, addition: typing.Any) -> str:
addition = str(addition or '')
if len(current) + len(addition) > _MAX_WEKNORA_GENERATED_CHARS:
raise errors.WeKnoraAPIError('WeKnora response exceeds the runtime limit')
return current + addition
@runner.runner_class('weknora-api')
class WeKnoraAPIRunner(runner.RequestRunner):
@@ -94,7 +103,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
web_search_enabled=web_search_enabled,
timeout=timeout,
):
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk))
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk)[:1000])
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
@@ -120,7 +129,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
elif response_type == 'answer':
if content:
full_answer += content
full_answer = _append_bounded(full_answer, content)
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
@@ -158,14 +167,14 @@ class WeKnoraAPIRunner(runner.RequestRunner):
knowledge_base_ids=knowledge_base_ids,
timeout=timeout,
):
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk))
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk)[:1000])
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
if response_type == 'answer':
if content:
full_answer += content
full_answer = _append_bounded(full_answer, content)
elif response_type == 'error':
raise errors.WeKnoraAPIError(f'WeKnora 服务错误: {content}')
@@ -207,7 +216,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
web_search_enabled=web_search_enabled,
timeout=timeout,
):
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk))
self.ap.logger.debug('weknora-agent-chunk: ' + str(chunk)[:1000])
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
@@ -235,7 +244,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
elif response_type == 'answer':
message_idx += 1
if content:
pending_answer += content
pending_answer = _append_bounded(pending_answer, content)
if done:
is_final = True
@@ -288,7 +297,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
knowledge_base_ids=knowledge_base_ids,
timeout=timeout,
):
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk))
self.ap.logger.debug('weknora-chat-chunk: ' + str(chunk)[:1000])
response_type = chunk.get('response_type', '')
content = chunk.get('content', '')
@@ -297,7 +306,7 @@ class WeKnoraAPIRunner(runner.RequestRunner):
if response_type == 'answer':
message_idx += 1
if content:
pending_answer += content
pending_answer = _append_bounded(pending_answer, content)
if done:
is_final = True
+355 -9
View File
@@ -1,42 +1,332 @@
from __future__ import annotations
import asyncio
import dataclasses
import heapq
import time
from ...core import app
from langbot_plugin.api.entities.builtin.provider import message as provider_message, prompt as provider_prompt
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
from ...api.http.context import ExecutionContext
from ...core import app
from ...pipeline.pool import (
ExecutionContextMismatchError,
ExecutionContextRequiredError,
bind_execution_context,
get_query_execution_context,
)
SessionKey = tuple[
str,
str,
int,
str,
str,
int | str,
]
SessionExpiryEntry = tuple[float, int, SessionKey]
_SESSION_EXPIRY_HEAP_MIN_LIMIT = 64
_SESSION_EXPIRY_HEAP_ACTIVE_MULTIPLIER = 4
def _query_session_key(query: pipeline_query.Query) -> tuple[SessionKey, ExecutionContext]:
execution_context = get_query_execution_context(query)
bot_uuid = getattr(query, 'bot_uuid', None)
if not isinstance(bot_uuid, str) or not bot_uuid.strip():
raise ExecutionContextRequiredError('Query.bot_uuid is required for session lookup')
execution_context = bind_execution_context(execution_context, bot_uuid=bot_uuid)
key: SessionKey = (
execution_context.instance_uuid,
execution_context.workspace_uuid,
execution_context.placement_generation,
bot_uuid,
query.launcher_type.value,
query.launcher_id,
)
return key, execution_context
class SessionManager:
"""会话管理器"""
ap: app.Application
session_list: list[provider_session.Session]
def __init__(self, ap: app.Application):
self.ap = ap
self.session_list = []
self._legacy_sessions: list[provider_session.Session] = []
self._session_index: dict[SessionKey, provider_session.Session] = {}
self._session_keys_by_workspace: dict[str, set[SessionKey]] = {}
self._session_expiry_heap: list[SessionExpiryEntry] = []
self._next_access_revision = 0
@property
def session_list(self) -> list[provider_session.Session]:
"""Compatibility view for API services that enumerate sessions."""
return [
*self._legacy_sessions,
*self._session_index.values(),
]
@session_list.setter
def session_list(self, sessions: list[provider_session.Session]) -> None:
"""Replace the cache while keeping the O(1) index consistent."""
session_values = list(sessions)
self._legacy_sessions = []
self._session_index = {}
self._session_keys_by_workspace = {}
self._session_expiry_heap = []
self._next_access_revision = 0
now = time.monotonic()
for session in session_values:
key = getattr(session, '_langbot_session_key', None)
if isinstance(key, tuple) and len(key) == 6:
self._session_index[key] = session
self._session_keys_by_workspace.setdefault(key[1], set()).add(key)
last_accessed = getattr(session, '_langbot_last_accessed', None)
if last_accessed is None:
last_accessed = now
self._touch_session(
session,
key,
float(last_accessed),
compact=False,
)
else:
self._legacy_sessions.append(session)
self._compact_session_expiry_heap(force=True)
def _retention_config(self) -> dict:
instance_config = getattr(getattr(self.ap, 'instance_config', None), 'data', {})
if not isinstance(instance_config, dict):
return {}
config = instance_config.get('system', {}).get('session_retention', {})
return config if isinstance(config, dict) else {}
def _positive_config_int(self, name: str, default: int) -> int:
try:
value = int(self._retention_config().get(name, default))
except (TypeError, ValueError):
value = default
return max(value, 1)
@staticmethod
def _session_is_idle(session: provider_session.Session) -> bool:
semaphore = getattr(session, '_semaphore', None)
concurrency = getattr(session, '_langbot_session_concurrency', None)
if semaphore is None or not isinstance(concurrency, int):
return True
return getattr(semaphore, '_value', -1) == concurrency
def _remove_session(self, session: provider_session.Session) -> None:
key = getattr(session, '_langbot_session_key', None)
if isinstance(key, tuple) and len(key) == 6 and self._session_index.get(key) is session:
self._session_index.pop(key, None)
workspace_keys = self._session_keys_by_workspace.get(key[1])
if workspace_keys is not None:
workspace_keys.discard(key)
if not workspace_keys:
self._session_keys_by_workspace.pop(key[1], None)
else:
try:
self._legacy_sessions.remove(session)
except ValueError:
pass
def _touch_session(
self,
session: provider_session.Session,
key: SessionKey,
now: float,
*,
compact: bool = True,
) -> None:
self._next_access_revision += 1
revision = self._next_access_revision
object.__setattr__(session, '_langbot_last_accessed', now)
object.__setattr__(session, '_langbot_access_revision', revision)
heapq.heappush(
self._session_expiry_heap,
(now, revision, key),
)
if compact:
self._compact_session_expiry_heap()
def _compact_session_expiry_heap(self, *, force: bool = False) -> None:
limit = max(
len(self._session_index) * _SESSION_EXPIRY_HEAP_ACTIVE_MULTIPLIER,
_SESSION_EXPIRY_HEAP_MIN_LIMIT,
)
if not force and len(self._session_expiry_heap) <= limit:
return
self._session_expiry_heap = [
(
float(getattr(session, '_langbot_last_accessed', 0.0)),
int(getattr(session, '_langbot_access_revision', 0)),
key,
)
for key, session in self._session_index.items()
]
heapq.heapify(self._session_expiry_heap)
def _pop_current_expiry_entry(
self,
) -> tuple[float, int, SessionKey, provider_session.Session] | None:
while self._session_expiry_heap:
last_accessed, revision, key = heapq.heappop(self._session_expiry_heap)
session = self._session_index.get(key)
if session is None:
continue
if getattr(session, '_langbot_access_revision', None) != revision:
continue
return last_accessed, revision, key, session
return None
def _prune_expired_sessions(self, now: float) -> None:
idle_ttl = self._positive_config_int('idle_ttl_seconds', 86400)
cutoff = now - idle_ttl
while self._session_expiry_heap:
last_accessed, _, _ = self._session_expiry_heap[0]
if last_accessed > cutoff:
break
current = self._pop_current_expiry_entry()
if current is None:
break
last_accessed, revision, key, session = current
if last_accessed > cutoff:
heapq.heappush(
self._session_expiry_heap,
(last_accessed, revision, key),
)
break
if self._session_is_idle(session):
self._remove_session(session)
continue
# The session became active without another cache lookup. Give it
# a fresh TTL instead of repeatedly examining the same expired
# entry or losing its future expiry record.
self._touch_session(session, key, now)
def _prune_workspace_capacity(
self,
workspace_uuid: str,
max_entries_per_workspace: int,
) -> None:
workspace_keys = self._session_keys_by_workspace.get(workspace_uuid, set())
overflow = len(workspace_keys) - max_entries_per_workspace + 1
if overflow <= 0:
return
idle_workspace_sessions = sorted(
(
session
for key in tuple(workspace_keys)
if (session := self._session_index.get(key)) is not None and self._session_is_idle(session)
),
key=lambda session: float(getattr(session, '_langbot_last_accessed', 0.0)),
)
for session in idle_workspace_sessions[:overflow]:
self._remove_session(session)
def _evict_oldest_idle_session(self, now: float) -> bool:
# At most one current entry per active session is examined. Stale heap
# revisions do not count and are discarded in O(log N).
current_probes = 0
max_probes = len(self._session_index)
while current_probes < max_probes:
current = self._pop_current_expiry_entry()
if current is None:
return False
_, _, key, session = current
current_probes += 1
if self._session_is_idle(session):
self._remove_session(session)
return True
self._touch_session(session, key, now)
return False
def _prune_sessions(self, now: float, workspace_uuid: str) -> None:
self._prune_expired_sessions(now)
max_entries_per_workspace = self._positive_config_int('max_entries_per_workspace', 200)
self._prune_workspace_capacity(
workspace_uuid,
max_entries_per_workspace,
)
max_entries = self._positive_config_int('max_entries', 2000)
overflow = len(self._session_index) - max_entries + 1
if overflow <= 0:
return
for _ in range(overflow):
if not self._evict_oldest_idle_session(now):
break
async def initialize(self):
pass
async def get_session(self, query: pipeline_query.Query) -> provider_session.Session:
"""获取会话"""
for session in self.session_list:
if query.launcher_type == session.launcher_type and query.launcher_id == session.launcher_id:
return session
session_key, execution_context = _query_session_key(query)
now = time.monotonic()
session = self._session_index.get(session_key)
if session is not None:
self._touch_session(session, session_key, now)
return session
self._prune_sessions(now, execution_context.workspace_uuid)
max_entries_per_workspace = self._positive_config_int('max_entries_per_workspace', 200)
workspace_entries = len(
self._session_keys_by_workspace.get(
execution_context.workspace_uuid,
(),
)
)
if workspace_entries >= max_entries_per_workspace:
raise RuntimeError(f'Workspace session cache capacity reached ({max_entries_per_workspace})')
max_entries = self._positive_config_int('max_entries', 2000)
if len(self._session_index) >= max_entries:
raise RuntimeError(f'Session cache capacity reached ({max_entries})')
session_concurrency = self.ap.instance_config.data['concurrency']['session']
session = provider_session.Session(
instance_uuid=execution_context.instance_uuid,
workspace_uuid=execution_context.workspace_uuid,
placement_generation=execution_context.placement_generation,
bot_uuid=query.bot_uuid,
launcher_type=query.launcher_type,
launcher_id=query.launcher_id,
sender_id=query.sender_id,
)
session_context = dataclasses.replace(
execution_context,
pipeline_uuid=None,
query_uuid=None,
)
# langbot-plugin 0.4.13 ignores Workspace fields. Preserve them until
# the Workspace-aware SDK becomes the minimum supported version.
object.__setattr__(session, 'instance_uuid', session_context.instance_uuid)
object.__setattr__(session, 'workspace_uuid', session_context.workspace_uuid)
object.__setattr__(
session,
'placement_generation',
session_context.placement_generation,
)
object.__setattr__(session, 'bot_uuid', query.bot_uuid)
object.__setattr__(session, '_execution_context', session_context)
object.__setattr__(session, '_langbot_session_key', session_key)
object.__setattr__(session, '_langbot_session_concurrency', session_concurrency)
session._semaphore = asyncio.Semaphore(session_concurrency)
self.session_list.append(session)
self._session_index[session_key] = session
self._session_keys_by_workspace.setdefault(
execution_context.workspace_uuid,
set(),
).add(session_key)
self._touch_session(session, session_key, now)
return session
async def get_conversation(
@@ -49,6 +339,17 @@ class SessionManager:
) -> provider_session.Conversation:
"""获取对话或创建对话"""
session_key, execution_context = _query_session_key(query)
if getattr(session, '_langbot_session_key', None) != session_key:
raise ExecutionContextMismatchError('Session does not belong to the Query execution scope')
execution_context = bind_execution_context(
execution_context,
bot_uuid=bot_uuid,
pipeline_uuid=pipeline_uuid,
)
if execution_context.bot_uuid != getattr(session, 'bot_uuid', None):
raise ExecutionContextMismatchError('Session bot_uuid does not match the Query execution scope')
if not session.conversations:
session.conversations = []
@@ -63,7 +364,11 @@ class SessionManager:
messages=prompt_messages,
)
if session.using_conversation is None or session.using_conversation.pipeline_uuid != pipeline_uuid:
if (
session.using_conversation is None
or session.using_conversation.pipeline_uuid != pipeline_uuid
or session.using_conversation.bot_uuid != bot_uuid
):
conversation = provider_session.Conversation(
prompt=prompt,
messages=[],
@@ -71,6 +376,47 @@ class SessionManager:
bot_uuid=bot_uuid,
)
session.conversations.append(conversation)
max_conversations = self._positive_config_int('max_conversations_per_session', 20)
if len(session.conversations) > max_conversations:
del session.conversations[:-max_conversations]
session.using_conversation = conversation
return session.using_conversation
def trim_conversation_messages(
self,
conversation: provider_session.Conversation,
*,
max_rounds: int,
) -> None:
"""Bound retained process-local history after a completed turn."""
try:
max_rounds = int(max_rounds)
except (TypeError, ValueError):
max_rounds = 10
max_rounds = max(max_rounds, 1)
max_messages = self._positive_config_int('max_messages_per_conversation', 100)
kept_reversed = []
user_rounds = 0
for message in reversed(conversation.messages):
if user_rounds >= max_rounds:
break
kept_reversed.append(message)
if getattr(message, 'role', None) == 'user':
user_rounds += 1
retained = list(reversed(kept_reversed))[-max_messages:]
# Binary payloads are needed for the current model call, but retaining
# them in process-local history makes a few image/file turns consume
# hundreds of MB. Historical URL and text references remain intact.
for message in retained:
content = getattr(message, 'content', None)
if not isinstance(content, list):
continue
for element in content:
if getattr(element, 'image_base64', None) is not None:
element.image_base64 = None
if getattr(element, 'file_base64', None) is not None:
element.file_base64 = None
conversation.messages = retained
@@ -11,7 +11,7 @@ async def is_box_backend_available(ap: Any) -> bool:
if not getattr(box_service, 'available', False):
return False
try:
status = await box_service.get_status()
status = await box_service.get_backend_status()
backend_info = status.get('backend', {})
return bool(backend_info.get('available', False))
except Exception:
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,50 @@
from __future__ import annotations
from typing import Any
MCP_STDIO_DISABLED_CODE = 'mcp_stdio_disabled'
MCP_STDIO_DISABLED_MESSAGE = 'Stdio MCP is disabled by instance policy'
class MCPStdioDisabledError(RuntimeError):
"""Raised when an instance-level policy refuses stdio MCP execution."""
code = MCP_STDIO_DISABLED_CODE
def __init__(self) -> None:
super().__init__(MCP_STDIO_DISABLED_MESSAGE)
def stdio_mcp_enabled(ap: Any) -> bool:
"""Return the independent instance-level stdio MCP feature gate.
The open-source default remains enabled for backwards compatibility. A
deployment can disable it with ``mcp.stdio.enabled: false`` (or
``MCP__STDIO__ENABLED=false``). Invalid values fail closed instead of
accidentally enabling local process execution.
"""
instance_config = getattr(ap, 'instance_config', None)
config = getattr(instance_config, 'data', None)
if not isinstance(config, dict):
return False
mcp_config = config.get('mcp', {})
if not isinstance(mcp_config, dict):
return False
stdio_config = mcp_config.get('stdio', {})
if not isinstance(stdio_config, dict):
return False
enabled = stdio_config.get('enabled', True)
return enabled if isinstance(enabled, bool) else False
def is_stdio_server(server_config: dict[str, Any] | None) -> bool:
return isinstance(server_config, dict) and str(server_config.get('mode') or '').strip().lower() == 'stdio'
def require_stdio_mcp_enabled(ap: Any, server_config: dict[str, Any] | None) -> None:
"""Fail closed for a stdio server before choosing Box or host transport."""
if is_stdio_server(server_config) and not stdio_mcp_enabled(ap):
raise MCPStdioDisabledError()
@@ -6,10 +6,12 @@ import os
import shutil
import shlex
import threading
from contextlib import suppress, AsyncExitStack
import weakref
from contextlib import suppress, AsyncExitStack, asynccontextmanager
from typing import TYPE_CHECKING, Any
import pydantic
from ....utils import bounded_executor
from mcp import ClientSession
from mcp.client.websocket import websocket_client
from ....box.workspace import (
@@ -27,7 +29,7 @@ if TYPE_CHECKING:
from .mcp import RuntimeMCPSession
_WORKSPACE_COPY_LOCKS: dict[str, threading.Lock] = {}
_WORKSPACE_COPY_LOCKS: weakref.WeakValueDictionary[str, threading.Lock] = weakref.WeakValueDictionary()
_WORKSPACE_COPY_LOCKS_GUARD = threading.Lock()
@@ -94,6 +96,60 @@ class MCPServerBoxConfig(pydantic.BaseModel):
_HANDSHAKE_ATTEMPT_TIMEOUT_SEC = 10.0
@asynccontextmanager
async def authenticated_websocket_client(url: str, headers: dict[str, str]):
"""MCP WebSocket transport with host-only Box relay headers.
The upstream MCP helper does not expose WebSocket handshake headers. This
mirrors that transport while keeping the Box control token out of the URL,
JSON-RPC payloads, and logs.
"""
import json
import anyio
import mcp.types as mcp_types
from mcp.shared.message import SessionMessage
from pydantic import ValidationError
from websockets.asyncio.client import connect as ws_connect
from websockets.typing import Subprotocol
read_stream_writer, read_stream = anyio.create_memory_object_stream(0)
write_stream, write_stream_reader = anyio.create_memory_object_stream(0)
async with ws_connect(
url,
subprotocols=[Subprotocol('mcp')],
additional_headers=dict(headers),
proxy=None,
) as websocket:
async def ws_reader():
async with read_stream_writer:
async for raw_text in websocket:
try:
message = mcp_types.JSONRPCMessage.model_validate_json(raw_text)
await read_stream_writer.send(SessionMessage(message))
except ValidationError as exc: # pragma: no cover - upstream parity
await read_stream_writer.send(exc)
async def ws_writer():
async with write_stream_reader:
async for session_message in write_stream_reader:
payload = session_message.message.model_dump(
by_alias=True,
mode='json',
exclude_none=True,
)
await websocket.send(json.dumps(payload))
async with anyio.create_task_group() as task_group:
task_group.start_soon(ws_reader)
task_group.start_soon(ws_writer)
yield read_stream, write_stream
task_group.cancel_scope.cancel()
class _TransferredStack:
"""Adapts an already-populated AsyncExitStack into an async context manager
so ownership of its resources can be transferred into another exit stack.
@@ -149,6 +205,7 @@ class BoxStdioSessionRuntime:
resolved_host_path = self.resolve_host_path() if host_path is ... else host_path
return BoxWorkspaceSession(
self.ap.box_service,
self.owner.execution_context,
self.owner._build_box_session_id(),
host_path=resolved_host_path,
host_path_mode=self.config.host_path_mode,
@@ -249,7 +306,11 @@ class BoxStdioSessionRuntime:
if install_cmd:
payload = self._wrap_process_payload_with_python_env(payload, process_cwd)
payload['process_id'] = self.process_id
await workspace.box_service.start_managed_process(workspace.session_id, payload)
await workspace.box_service.start_managed_process(
workspace.execution_context,
workspace.session_id,
payload,
)
except Exception:
self.owner.error_phase = MCPSessionErrorPhase.PROCESS_START
raise
@@ -259,7 +320,10 @@ class BoxStdioSessionRuntime:
f'process_id={self.process_id} (transport reconnect)'
)
websocket_url = workspace.get_managed_process_websocket_url(self.process_id)
(
websocket_url,
websocket_headers,
) = await workspace.get_managed_process_websocket_connection(self.process_id)
# Attach the WS transport + MCP session ONCE, on the owner's exit stack,
# in the same task as the serve loop that follows. websocket_client and
@@ -277,7 +341,12 @@ class BoxStdioSessionRuntime:
# attempt re-attaches to the same live process; once it has finished
# cold start the handshake succeeds and stays healthy.
try:
transport = await self.owner.exit_stack.enter_async_context(websocket_client(websocket_url))
transport_context = (
authenticated_websocket_client(websocket_url, websocket_headers)
if websocket_headers
else websocket_client(websocket_url)
)
transport = await self.owner.exit_stack.enter_async_context(transport_context)
read_stream, write_stream = transport
self.owner.session = await self.owner.exit_stack.enter_async_context(
ClientSession(read_stream, write_stream)
@@ -469,7 +538,11 @@ class BoxStdioSessionRuntime:
return
try:
process_host_root = os.path.join(self._shared_workspace_host_path(), '.mcp', self.process_id)
await asyncio.to_thread(shutil.rmtree, process_host_root, True)
await bounded_executor.run_blocking_cleanup(
shutil.rmtree,
process_host_root,
True,
)
except Exception as exc:
self.ap.logger.warning(
f'MCP server {self.server_name}: failed to clean staged workspace '
File diff suppressed because it is too large Load Diff
@@ -67,7 +67,11 @@ class PluginToolLoader(loader.ToolLoader):
async def invoke_tool(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any:
try:
return await self.ap.plugin_connector.call_tool(
name, parameters, session=query.session, query_id=query.query_id
name,
parameters,
session=query.session,
query_id=query.query_id,
query_uuid=query.query_uuid,
)
except Exception as e:
self.ap.logger.error(f'执行函数 {name} 时发生错误: {e}')
@@ -4,6 +4,7 @@ import re
import typing
from ....box import workspace as box_workspace
from ....api.http.context import ExecutionContext
if typing.TYPE_CHECKING:
from ....core import app
@@ -36,7 +37,15 @@ def get_visible_skills(ap: app.Application, query: pipeline_query.Query) -> dict
if skill_mgr is None:
return {}
visible_skills = getattr(skill_mgr, 'skills', {})
execution_context = ExecutionContext(
instance_uuid=str(getattr(query, 'instance_uuid', '') or ''),
workspace_uuid=str(getattr(query, 'workspace_uuid', '') or ''),
placement_generation=getattr(query, 'placement_generation', 0) or 0,
bot_uuid=getattr(query, 'bot_uuid', None),
pipeline_uuid=getattr(query, 'pipeline_uuid', None),
query_uuid=getattr(query, 'query_uuid', None),
)
visible_skills = skill_mgr.get_skills(execution_context)
bound_skills = get_bound_skill_names(query)
if bound_skills is None:
return visible_skills
@@ -192,5 +201,14 @@ def should_prepare_skill_python_env(package_root: str | None) -> bool:
return box_workspace.should_prepare_python_env(package_root)
def wrap_skill_command_with_python_env(command: str, *, mount_path: str = '/workspace') -> str:
return box_workspace.wrap_python_command_with_env(command, mount_path=mount_path).rstrip()
def wrap_skill_command_with_python_env(
command: str,
*,
mount_path: str = '/workspace',
state_path: str | None = None,
) -> str:
return box_workspace.wrap_python_command_with_env(
command,
mount_path=mount_path,
state_path=state_path,
).rstrip()
@@ -7,6 +7,7 @@ import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
from .. import loader
from .availability import is_box_backend_available
from ....api.http.context import ExecutionContext
# Align with Claude Code's Skill tool design:
# - activate: Activate a skill via Tool Call, returns SKILL.md content
@@ -72,12 +73,34 @@ class SkillToolLoader(loader.ToolLoader):
return self._sandbox_available
async def invoke_tool(self, name: str, parameters: dict, query) -> typing.Any:
require_sandbox = getattr(
getattr(self.ap, 'box_service', None),
'require_workspace_sandbox',
None,
)
if callable(require_sandbox):
await require_sandbox(self._execution_context(query))
if name == ACTIVATE_SKILL_TOOL_NAME:
return await self._invoke_activate_skill(parameters, query)
if name == REGISTER_SKILL_TOOL_NAME:
return await self._invoke_register_skill(parameters)
return await self._invoke_register_skill(parameters, query)
raise ValueError(f'Unknown skill tool: {name}')
@staticmethod
def _execution_context(query) -> ExecutionContext:
attached_context = getattr(query, '_execution_context', None)
if isinstance(attached_context, ExecutionContext):
return attached_context
return ExecutionContext(
instance_uuid=str(getattr(query, 'instance_uuid', '') or ''),
workspace_uuid=str(getattr(query, 'workspace_uuid', '') or ''),
placement_generation=getattr(query, 'placement_generation', 0) or 0,
bot_uuid=getattr(query, 'bot_uuid', None),
pipeline_uuid=getattr(query, 'pipeline_uuid', None),
query_uuid=getattr(query, 'query_uuid', None),
entitlement_revision=getattr(query, 'entitlement_revision', 0),
)
async def shutdown(self):
pass
@@ -128,14 +151,15 @@ class SkillToolLoader(loader.ToolLoader):
'content': result_content,
}
async def _invoke_register_skill(self, parameters: dict) -> typing.Any:
async def _invoke_register_skill(self, parameters: dict, query) -> typing.Any:
"""Register a skill from sandbox directory to data/skills/."""
sandbox_path = str(parameters.get('path', '') or '').strip()
if not sandbox_path:
raise ValueError('path is required')
# Resolve sandbox path to host path
host_path = self._resolve_workspace_directory(sandbox_path)
execution_context = self._execution_context(query)
host_path = self._resolve_workspace_directory(sandbox_path, execution_context)
# Get or create skill service
skill_service = getattr(self.ap, 'skill_service', None)
@@ -143,7 +167,7 @@ class SkillToolLoader(loader.ToolLoader):
raise ValueError('Skill service not available')
# Scan and register the skill
scanned = await skill_service.scan_directory_async(host_path)
scanned = await skill_service.scan_directory_async(execution_context, host_path)
# Override name if provided
skill_name = str(parameters.get('name') or scanned['name']).strip()
@@ -152,13 +176,14 @@ class SkillToolLoader(loader.ToolLoader):
# Create the skill
created = await skill_service.create_skill(
execution_context,
{
'name': skill_name,
'display_name': str(parameters.get('display_name') or scanned.get('display_name', '')).strip(),
'description': str(parameters.get('description') or scanned.get('description', '')).strip(),
'instructions': str(parameters.get('instructions') or scanned.get('instructions', '')),
'package_root': host_path,
}
},
)
return {
@@ -168,10 +193,19 @@ class SkillToolLoader(loader.ToolLoader):
'skill': created,
}
def _resolve_workspace_directory(self, sandbox_path: str) -> str:
def _resolve_workspace_directory(
self,
sandbox_path: str,
execution_context: ExecutionContext,
) -> str:
"""Resolve sandbox path to host filesystem path."""
box_service = getattr(self.ap, 'box_service', None)
workspace_root = getattr(box_service, 'default_workspace', None)
tenant_workspace = getattr(box_service, '_tenant_workspace', None)
workspace_root = (
tenant_workspace(execution_context)
if callable(tenant_workspace)
else getattr(box_service, 'default_workspace', None)
)
if not workspace_root:
raise ValueError('No default workspace configured')
+67 -15
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import typing
import time
import inspect
from typing import TYPE_CHECKING
import langbot_plugin.api.entities.builtin.resource.tool as resource_tool
@@ -9,6 +10,8 @@ from langbot_plugin.api.entities.events import pipeline_query
from . import loader as tool_loader
from .errors import ToolNotFoundError
from ...pipeline.pool import get_query_execution_context
from ...api.http.service.tenant import TenantContext
if TYPE_CHECKING:
from ...core import app
@@ -33,6 +36,36 @@ class ToolManager:
def __init__(self, ap: app.Application):
self.ap = ap
async def _bind_plugin_workspace(self, context: TenantContext) -> None:
"""Select the tenant before any plugin catalog lookup.
Tool discovery happens before invocation, so relying on ``call_tool``
to bind the Workspace is too late and can expose another task's
catalog in a shared Runtime.
"""
connector = getattr(self.ap, 'plugin_connector', None)
require_context = getattr(connector, 'require_workspace_context', None)
if require_context is None:
return
result = require_context(context)
if inspect.isawaitable(result):
await result
async def _workspace_sandbox_available(self, context: TenantContext) -> bool:
"""Resolve the Workspace capability before exposing sandbox tools."""
box_service = getattr(self.ap, 'box_service', None)
checker = getattr(box_service, 'is_workspace_sandbox_available', None)
if not callable(checker):
# Compatibility for OSS embedders and isolated manager tests. The
# BoxService execution path remains the final authority.
return True
try:
return bool(await checker(context))
except Exception:
return False
async def initialize(self):
from langbot.pkg.utils import importutil
from langbot.pkg.provider.tools import loaders
@@ -57,19 +90,24 @@ class ToolManager:
async def get_all_tools(
self,
context: TenantContext,
bound_plugins: list[str] | None = None,
bound_mcp_servers: list[str] | None = None,
include_skill_authoring: bool = False,
include_mcp_resource_tools: bool = True,
) -> list[resource_tool.LLMTool]:
await self._bind_plugin_workspace(context)
all_functions: list[resource_tool.LLMTool] = []
all_functions.extend(await self.native_tool_loader.get_tools())
if include_skill_authoring:
sandbox_available = await self._workspace_sandbox_available(context)
if sandbox_available:
all_functions.extend(await self.native_tool_loader.get_tools())
if include_skill_authoring and sandbox_available:
all_functions.extend(await self.skill_tool_loader.get_tools())
all_functions.extend(await self.plugin_tool_loader.get_tools(bound_plugins))
all_functions.extend(
await self.mcp_tool_loader.get_tools(
context,
bound_mcp_servers,
include_resource_tools=include_mcp_resource_tools,
)
@@ -79,11 +117,13 @@ class ToolManager:
async def get_tool_catalog(
self,
context: TenantContext,
bound_plugins: list[str] | None = None,
bound_mcp_servers: list[str] | None = None,
include_skill_authoring: bool = False,
include_mcp_resource_tools: bool = False,
) -> list[dict[str, typing.Any]]:
await self._bind_plugin_workspace(context)
catalog: list[dict[str, typing.Any]] = []
def append_tools(source: str, source_name: str, tools: list[resource_tool.LLMTool]) -> None:
@@ -99,13 +139,16 @@ class ToolManager:
}
)
append_tools('builtin', 'LangBot', await self.native_tool_loader.get_tools())
if include_skill_authoring:
sandbox_available = await self._workspace_sandbox_available(context)
if sandbox_available:
append_tools('builtin', 'LangBot', await self.native_tool_loader.get_tools())
if include_skill_authoring and sandbox_available:
append_tools('skill', 'LangBot', await self.skill_tool_loader.get_tools())
catalog.extend(await self.plugin_tool_loader.get_tool_catalog(bound_plugins))
if self.mcp_tool_loader:
for item in await self.mcp_tool_loader.get_tool_catalog(
context,
bound_mcp_servers,
include_resource_tools=include_mcp_resource_tools,
):
@@ -113,19 +156,24 @@ class ToolManager:
return catalog
async def get_tool_by_name(self, name: str) -> tool_loader.ToolLookupResult | None:
async def get_tool_by_name(self, context: TenantContext, name: str) -> tool_loader.ToolLookupResult | None:
"""Get tool by name from any active loader."""
for active_loader in (
self.native_tool_loader,
self.plugin_tool_loader,
self.mcp_tool_loader,
self.skill_tool_loader,
):
await self._bind_plugin_workspace(context)
sandbox_available = await self._workspace_sandbox_available(context)
if sandbox_available:
tool = await self.native_tool_loader.get_tool(name)
if tool:
return tool
for active_loader in (self.plugin_tool_loader,):
tool = await active_loader.get_tool(name)
if tool:
return tool
if sandbox_available:
tool = await self.skill_tool_loader.get_tool(name)
if tool:
return tool
return None
return await self.mcp_tool_loader.get_tool(context, name)
async def generate_tools_for_openai(self, use_funcs: list[resource_tool.LLMTool]) -> list:
tools = []
@@ -175,6 +223,7 @@ class ToolManager:
try:
await monitoring_service.record_tool_call(
get_query_execution_context(query),
tool_name=name,
tool_source=source,
duration=duration_ms,
@@ -231,7 +280,10 @@ class ToolManager:
async def execute_func_call(self, name: str, parameters: dict, query: pipeline_query.Query) -> typing.Any:
from langbot.pkg.telemetry import features as telemetry_features
if await self.native_tool_loader.has_tool(name):
execution_context = get_query_execution_context(query)
await self._bind_plugin_workspace(execution_context)
sandbox_available = await self._workspace_sandbox_available(execution_context)
if sandbox_available and await self.native_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'native')
return await self._invoke_tool_with_monitoring(
source='native',
@@ -249,7 +301,7 @@ class ToolManager:
query=query,
invoke=lambda: self.plugin_tool_loader.invoke_tool(name, parameters, query),
)
if await self.mcp_tool_loader.has_tool(name):
if await self.mcp_tool_loader.has_tool(execution_context, name):
telemetry_features.increment(query, 'tool_calls', 'mcp')
return await self._invoke_tool_with_monitoring(
source='mcp',
@@ -258,7 +310,7 @@ class ToolManager:
query=query,
invoke=lambda: self.mcp_tool_loader.invoke_tool(name, parameters, query),
)
if await self.skill_tool_loader.has_tool(name):
if sandbox_available and await self.skill_tool_loader.has_tool(name):
telemetry_features.increment(query, 'tool_calls', 'skill')
return await self._invoke_tool_with_monitoring(
source='skill',