* feat(speech): implement TTS and STT model layer and pipeline stages (LANG-251)
Co-authored-by: multica-agent <[email protected]>
* fix(speech): fix invoke signatures and provider lookup in speech stages and model testing
Co-authored-by: multica-agent <[email protected]>
* feat(speech): rename STT to ASR, improve TTS voice parameter handling and failure recovery
Co-authored-by: multica-agent <[email protected]>
* fix(persistence, lint, web): fix migration revision id, legacy table check, ruff duplicates, and frontend types
Co-authored-by: multica-agent <[email protected]>
* fix(speech): restore migration and CI contracts
Co-authored-by: multica-agent <[email protected]>
* fix(pipeline,ui): fix select dropdown width gap, add speech to guided tour, and configure show_if for speech options
Co-authored-by: multica-agent <[email protected]>
* feat(speech,openclaw): support silk audio transcoding, graceful fallback, duration estimation, and multi-lang deploy docs
Co-authored-by: multica-agent <[email protected]>
* fix(audio): enhance ffmpeg executable detection via venv and imageio_ffmpeg
Co-authored-by: multica-agent <[email protected]>
* fix(provider): improve LiteLLM error classification and avoid falling back to Unknown error
Co-authored-by: multica-agent <[email protected]>
* fix(openclaw): serialize voice_item attributes and improve transcoding fallback logging
Co-authored-by: multica-agent <[email protected]>
* fix(audio,openclaw): handle transient TMPDIR deletion and add size attribute to voice_item
Co-authored-by: multica-agent <[email protected]>
* fix(openclaw): download and convert inbound voice messages and align voice sending protocol
Co-authored-by: multica-agent <[email protected]>
* fix(models): redesign add model popover tabs layout and include ASR/TTS models in provider counts and scanning
Co-authored-by: multica-agent <[email protected]>
* fix(audio): decrypt and transcode wecombot voice to mp3, align openclaw weixin voice delivery
Co-authored-by: multica-agent <[email protected]>
* fix(wecombot): transcode outbound voice to silk and support voice_delivery mode
Co-authored-by: multica-agent <[email protected]>
* fix(discord,web): map audio attachments to voice for ASR, prefer base64 voice in webui
Co-authored-by: multica-agent <[email protected]>
* fix(telegram): deliver voice replies in streaming and non-streaming modes with quote origin
Co-authored-by: multica-agent <[email protected]>
* feat(dingtalk): route voice messages through pipeline ASR and send TTS replies as voice
Inbound DingTalk audio now arrives as a Voice component with its duration,
so the pipeline speech recognition switch decides how it is transcribed.
DingTalk's own recognition text is used only when the audio cannot be
downloaded.
Outbound Voice components are transcoded to ogg/opus, uploaded as voice
media and sent as native sampleAudio messages in both the omni and legacy
adapters. Audio over 60 seconds, or audio that cannot be transcoded or is
rejected as a voice message, is sent as a file instead. Voice parts no longer
leak into the reply text as "[Voice]".
Also estimate ogg durations from the granule position and ignore non-audio
MIME hints when extracting voice bytes, so ffmpeg can probe the input.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-authored-by: multica-agent <[email protected]>
* fix(ci): include speech model counts in provider responses and fix lint/image build
- Copy every count returned by get_provider_model_counts into the provider
payload instead of indexing asr_count/tts_count, and update the test fakes
to return the speech model counts.
- Install libc6-dev in the runtime image so the pilk audio extra can compile.
- Apply ruff and prettier formatting required by the lint jobs.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-authored-by: multica-agent <[email protected]>
* test(e2e): expect the speech step in the pipeline setup guide
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-authored-by: multica-agent <[email protected]>
---------
Co-authored-by: multica-agent <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
- Replace per-session mocks with lightweight shutdown counters
- Preserve the one-second timeout and 100-workspace coverage
- Assert that every session shuts down exactly once
- Deduplicate message and history resource URLs using stable catalog references
- Persist renewed URLs with concurrent refresh protection
- Refresh OneBot images via get_msg intervals or NapCat rkey TTL
- Migrate historical payloads and cover SQLite and PostgreSQL behavior
- Cache NapCat rkeys by type and refresh only when their TTL expires
- Renew image URLs through get_msg for other OneBot v11 implementations
- Refresh session views, Runner history, and each model request attempt
- Isolate caches by connection without downloading images or rewriting history
* fix(tools): require python3 for Box native file scripts
Unversioned `python` is still Python 2 on some host-rootfs nsjail
mounts. Falling back to it made read/write/edit scripts fail with
SyntaxError on f-strings (#2603).
Resolve only `python3`, and return a clear JSON error when it is
missing instead of invoking Python 2.
Fixes#2603
* style(tools): format Box interpreter error command
Co-authored-by: multica-agent <[email protected]>
---------
Co-authored-by: Li-john1021 <[email protected]>
Co-authored-by: Chin Rock <[email protected]>
Co-authored-by: multica-agent <[email protected]>
Use the chat provider to apply transport-specific metadata cleanup.
Keep finish_reason removal in the common message conversion path.
Verify streaming and non-streaming HTTP payloads through LiteLLM,
metadata preservation for other providers, and source message immutability.
Remove response-only finish_reason metadata when converting stored messages
into subsequent model requests.
Strip provider_specific_fields from Groq messages and tool calls to avoid
strict request validation failures after a tool response. Preserve metadata
required by other providers, including Gemini thought signatures, without
mutating the original message entities.
Cover streaming and non-streaming request construction with regression tests.
Core no longer assembles a whole trace locally. Every workflow node is uploaded
the moment it completes as its own `execution_node` record, and each execution
closes with exactly one `execution_chain` record; all records of one execution
carry the same `event_id`, so Space groups them back into a single chain.
- every execution starts at an inbound event: event-less Pipeline lanes and
WebUI/API debug runs synthesize their virtual inbound event at the boundary
and are marked debug=true, so a Runner is never a trace origin
- sampling is decided once per event from a stable hash of the event id (the
previous `int(id[:8], 16)` raised on every `platform:`-prefixed id and so
dropped every successful inbound-event chain) and is reported on each record
as sample{mode,denominator}; the default mode is now `all`
- the local trace buffer, the 120s TTL, the per-process trace cap and the shared
open-node mirror are gone, which also removes the truncated-chain upload and
the re-buffered-closed-trace leak they caused
- a broken chain is always uploaded, and its reason falls back to the first
failed node so the chain explains itself
- flush keeps records until the sender accepts them and counts what it drops;
telemetry health replaces the unused send-task queue
- heartbeat reports bound Agent / plugin-processor / event-binding counts
- bound Pipeline and Agent resources (plugins, MCP servers, knowledge bases,
tools with their Host source, skills, models and the Runner schema selectors
that picked them) are reported on the chain record
- failure text is bounded and redacted instead of passing exception text through
- the dead per-query feature counters and their call sites are removed
Wire contract: schema 2 (`execution_node` / `execution_chain`).
scan_skill_directory is disabled for Cloud-managed sandboxes, but the
register_skill tool was still advertised to the model whenever a backend was
available, so every call failed closed without a usable explanation. Advertise
register_skill only when the runtime may scan sandbox directories, and keep
activate available in both modes.
The workspace file tools (read/write/edit/glob/grep) ran a script through
`python`, which a sandbox built from a host rootfs may not provide: those hosts
ship `python3` only, and a read-only /usr prevents creating a `python` shim
inside the jail. Every file tool then failed with `/bin/sh: 1: python: not
found`. Resolve the interpreter with `command -v python3 || command -v python`,
matching the interpreter the Box service already uses for its own scripts.
* feat(mcp): surface OAuth-required server tests
* fix(mcp): show connection failure details in status cards
---------
Co-authored-by: RockChinQ <[email protected]>
execute_func_call returns list[ContentElement] for MCP tools, but the
runner assigned that list directly to the tool-message content. The
OpenAI chat-completions spec requires tool-message content to be a
string, so OpenAI-compatible endpoints return HTTP 500 when the raw
list is sent.
Serialize the list to a string before building the tool message, using
ContentElement.__str__ which returns the text payload for text elements
and a human-readable placeholder for images and files. Fixes#2457.
SessionManager clears image_base64 on past turns to save memory, and
exclude_none serialization drops the hollowed field entirely, so a
replayed history part can arrive as {'type': 'image_base64'} with no
payload. The converter accessed the missing key unconditionally and
raised KeyError on every turn after an image was sent.
Prefer the base64 payload when present, fall back to an image_url that
survived on the same element, and drop hollow parts otherwise (same
strategy as the existing file-part handling). Fixes#2469.
