feat(telemetry): payload v2 with feature usage counters and instance heartbeat

Per-query events now carry event_type='query' and a features JSON object:
- tool_calls by source (native/plugin/mcp/skill) via ToolManager
- tool_call_rounds, kb usage (count/engine plugins/retrieved entries) via local-agent
- sandbox execs/errors via BoxService
- activated_skills and bound mcp_servers snapshots

New instance_heartbeat event (startup + daily) reports anonymous instance
profile: deploy platform, database/vdb kind, box backend/availability,
adapter type names, and resource counts. Respects space.disable_telemetry.

All collection helpers are defensive and never break the pipeline.
Verified: ruff, 37 telemetry unit tests (13 new), 504 box/provider/pipeline tests.
This commit is contained in:
RockChinQ
2026-06-12 08:11:43 -04:00
parent bca710dbd4
commit dd96da895c
10 changed files with 488 additions and 0 deletions
@@ -13,6 +13,7 @@ from ....provider import runner as runner_module
import langbot_plugin.api.entities.events as events
from ....utils import importutil, constants, runner as runner_utils
from ....telemetry import features as telemetry_features
from ....provider import runners
import langbot_plugin.api.entities.builtin.provider.session as provider_session
import langbot_plugin.api.entities.builtin.pipeline.query as pipeline_query
@@ -201,7 +202,12 @@ class ChatMessageHandler(handler.MessageHandler):
runner_name, runner, query.pipeline_config
)
# Feature usage collected during query processing (tool calls,
# knowledge base usage, sandbox executions, activated skills, ...)
features = telemetry_features.collect_features(query)
payload = {
'event_type': 'query',
'query_id': query.query_id,
'adapter': adapter_name,
'runner': runner_name,
@@ -212,6 +218,7 @@ class ChatMessageHandler(handler.MessageHandler):
'instance_id': constants.instance_id,
'edition': constants.edition,
'pipeline_plugins': pipeline_plugins,
'features': features,
'error': locals().get('error_info', None),
'timestamp': datetime.utcnow().isoformat(),
}