mirror of
https://github.com/langbot-app/LangBot.git
synced 2026-11-04 14:52:12 +08:00
43c0ba7e44
* feat(speech): implement TTS and STT model layer and pipeline stages (LANG-251) Co-authored-by: multica-agent <[email protected]> * fix(speech): fix invoke signatures and provider lookup in speech stages and model testing Co-authored-by: multica-agent <[email protected]> * feat(speech): rename STT to ASR, improve TTS voice parameter handling and failure recovery Co-authored-by: multica-agent <[email protected]> * fix(persistence, lint, web): fix migration revision id, legacy table check, ruff duplicates, and frontend types Co-authored-by: multica-agent <[email protected]> * fix(speech): restore migration and CI contracts Co-authored-by: multica-agent <[email protected]> * fix(pipeline,ui): fix select dropdown width gap, add speech to guided tour, and configure show_if for speech options Co-authored-by: multica-agent <[email protected]> * feat(speech,openclaw): support silk audio transcoding, graceful fallback, duration estimation, and multi-lang deploy docs Co-authored-by: multica-agent <[email protected]> * fix(audio): enhance ffmpeg executable detection via venv and imageio_ffmpeg Co-authored-by: multica-agent <[email protected]> * fix(provider): improve LiteLLM error classification and avoid falling back to Unknown error Co-authored-by: multica-agent <[email protected]> * fix(openclaw): serialize voice_item attributes and improve transcoding fallback logging Co-authored-by: multica-agent <[email protected]> * fix(audio,openclaw): handle transient TMPDIR deletion and add size attribute to voice_item Co-authored-by: multica-agent <[email protected]> * fix(openclaw): download and convert inbound voice messages and align voice sending protocol Co-authored-by: multica-agent <[email protected]> * fix(models): redesign add model popover tabs layout and include ASR/TTS models in provider counts and scanning Co-authored-by: multica-agent <[email protected]> * fix(audio): decrypt and transcode wecombot voice to mp3, align openclaw weixin voice delivery Co-authored-by: multica-agent <[email protected]> * fix(wecombot): transcode outbound voice to silk and support voice_delivery mode Co-authored-by: multica-agent <[email protected]> * fix(discord,web): map audio attachments to voice for ASR, prefer base64 voice in webui Co-authored-by: multica-agent <[email protected]> * fix(telegram): deliver voice replies in streaming and non-streaming modes with quote origin Co-authored-by: multica-agent <[email protected]> * feat(dingtalk): route voice messages through pipeline ASR and send TTS replies as voice Inbound DingTalk audio now arrives as a Voice component with its duration, so the pipeline speech recognition switch decides how it is transcribed. DingTalk's own recognition text is used only when the audio cannot be downloaded. Outbound Voice components are transcoded to ogg/opus, uploaded as voice media and sent as native sampleAudio messages in both the omni and legacy adapters. Audio over 60 seconds, or audio that cannot be transcoded or is rejected as a voice message, is sent as a file instead. Voice parts no longer leak into the reply text as "[Voice]". Also estimate ogg durations from the granule position and ignore non-audio MIME hints when extracting voice bytes, so ffmpeg can probe the input. Co-Authored-By: Claude Opus 5.5 <[email protected]> Co-authored-by: multica-agent <[email protected]> * fix(ci): include speech model counts in provider responses and fix lint/image build - Copy every count returned by get_provider_model_counts into the provider payload instead of indexing asr_count/tts_count, and update the test fakes to return the speech model counts. - Install libc6-dev in the runtime image so the pilk audio extra can compile. - Apply ruff and prettier formatting required by the lint jobs. Co-Authored-By: Claude Opus 5.5 <[email protected]> Co-authored-by: multica-agent <[email protected]> * test(e2e): expect the speech step in the pipeline setup guide Co-Authored-By: Claude Opus 5.5 <[email protected]> Co-authored-by: multica-agent <[email protected]> --------- Co-authored-by: multica-agent <[email protected]> Co-authored-by: Claude Opus 5.5 <[email protected]>
82 lines
4.0 KiB
Docker
82 lines
4.0 KiB
Docker
FROM --platform=$BUILDPLATFORM node:22-alpine AS node
|
|
|
|
WORKDIR /app
|
|
|
|
COPY web ./web
|
|
|
|
RUN cd web && npm install && npx vite build
|
|
|
|
# Build nsjail from source so the image ships a self-contained sandbox backend
|
|
# that needs no host Docker socket. Pinned to a release tag for reproducibility.
|
|
# Multi-stage keeps the compile toolchain (bison/flex/protobuf-dev/libnl-dev)
|
|
# out of the final image; only the nsjail binary and its small runtime libs
|
|
# (libprotobuf, libnl-route-3) are carried over.
|
|
FROM python:3.12-slim-trixie AS nsjail-build
|
|
|
|
ARG NSJAIL_VERSION=3.6
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends \
|
|
ca-certificates git build-essential \
|
|
autoconf bison flex libtool pkg-config \
|
|
protobuf-compiler libprotobuf-dev libnl-route-3-dev \
|
|
&& git clone --depth 1 --branch "${NSJAIL_VERSION}" https://github.com/google/nsjail.git /nsjail \
|
|
&& make -C /nsjail \
|
|
&& install -m 0755 /nsjail/nsjail /usr/local/bin/nsjail \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
FROM python:3.12-slim-trixie
|
|
|
|
# Keep testing packages isolated to the SQLite release with the WAL-reset fix.
|
|
RUN printf 'deb http://deb.debian.org/debian forky main\n' > /etc/apt/sources.list.d/sqlite.list \
|
|
&& printf 'Package: libsqlite3-0\nPin: release n=forky\nPin-Priority: 990\n\nPackage: *\nPin: release n=forky\nPin-Priority: -1\n' > /etc/apt/preferences.d/sqlite \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends libsqlite3-0=3.53.4-2 \
|
|
&& python -c 'import sqlite3; assert sqlite3.sqlite_version == "3.53.4"' \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /app
|
|
|
|
COPY . .
|
|
|
|
ARG LANGBOT_BUILD_REVISION
|
|
RUN if [ -n "$LANGBOT_BUILD_REVISION" ]; then python3 scripts/stamp_build_revision.py --revision "$LANGBOT_BUILD_REVISION"; fi
|
|
|
|
COPY --from=node /app/web/dist ./web/dist
|
|
|
|
# nsjail binary built in the dedicated stage above. Self-contained sandbox
|
|
# backend; lets the Box runtime isolate code without a host Docker socket.
|
|
COPY --from=nsjail-build /usr/local/bin/nsjail /usr/local/bin/nsjail
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install -y --no-install-recommends gcc libc6-dev ca-certificates curl git gnupg ffmpeg \
|
|
# nsjail runtime libraries (the build toolchain stays in the nsjail-build
|
|
# stage; only these shared libs are needed to execute the binary).
|
|
&& apt-get install -y --no-install-recommends libprotobuf32t64 libnl-route-3-200 \
|
|
# Install the Docker CLI (client only) so the optional langbot_box
|
|
# service can drive the mounted host Docker socket and create sandbox
|
|
# containers. The same image powers langbot / plugin_runtime / box; only
|
|
# box uses the client. Arch-aware via dpkg so multi-arch builds work.
|
|
&& install -m 0755 -d /etc/apt/keyrings \
|
|
&& curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
|
|
&& chmod a+r /etc/apt/keyrings/docker.asc \
|
|
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" > /etc/apt/sources.list.d/docker.list \
|
|
&& apt-get update \
|
|
&& apt-get install -y --no-install-recommends docker-ce-cli \
|
|
# Install Node.js LTS so the sandbox (nsjail/Docker box) can run npx-based
|
|
# stdio MCP servers. node/npx land in /usr/bin, which is on the nsjail
|
|
# read-only mount whitelist (_READONLY_SYSTEM_MOUNTS), so they are bound
|
|
# into the sandbox chroot automatically. Without node, any npx-launched
|
|
# MCP server exits with return_code=127 (command not found).
|
|
&& curl -fsSL https://deb.nodesource.com/setup_22.x -o /tmp/nodesource_setup.sh \
|
|
&& bash /tmp/nodesource_setup.sh \
|
|
&& apt-get install -y --no-install-recommends nodejs \
|
|
&& rm -f /tmp/nodesource_setup.sh \
|
|
&& python -m pip install --no-cache-dir uv \
|
|
&& uv sync --extra seekdb --extra audio \
|
|
&& apt-get purge -y --auto-remove curl git gnupg \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& touch /.dockerenv
|
|
|
|
CMD [ "uv", "run", "--no-sync", "main.py" ]
|