Commit Graph

3 Commits

Author SHA1 Message Date
Junyan Chin a1e6eccdeb feat(box): bidirectional attachment transfer for sandbox (#2257)
* feat(box): bidirectional attachment transfer for sandbox

Materialize inbound attachments into the sandbox workspace so agents can
process user-sent files, and collect agent-produced files from the outbox
to attach them back to the reply.

- box(service): add materialize_inbound_attachments / collect_outbound
  attachments. Prefer direct host-filesystem read/write on the bind-mounted
  workspace (no size limit), falling back to chunked exec only for
  non-shared backends (e2b/remote). Clear per-query inbox/outbox dirs at
  turn start to avoid query_id-reuse collisions.
- provider(localagent): inject inbound attachment descriptors into the
  sandbox and append a system note telling the agent the inbox/outbox paths.
- pipeline(wrapper): collect outbox files on the final stream chunk and
  append them as attachment components to the response chain.
- web(debug-dialog): render File components with a download link when
  base64/url is present; add base64/path fields to the File entity.
- tests: cover inbound/outbound, large-file transfer without truncation,
  and stale-dir clearing (86 passing).

* feat(box): support voice/file attachment round-trip end-to-end

Extends the bidirectional attachment transfer to audio and arbitrary files
through the real webchat UI, and fixes the model-payload errors that
non-image attachments triggered.

- platform(websocket_adapter): resolve Voice/File component storage keys to
  base64 (previously only Image), so audio/documents reach the sandbox inbox.
- web(debug-dialog): accept audio/* and any file in the uploader (was
  image-only), classify by mimetype, upload Voice/File via the documents
  endpoint, and render non-image staged attachments as a chip.
- provider(litellmchat): drop non-image file parts (file_base64 / file_url)
  when building the OpenAI/LiteLLM payload. These come from Voice/File
  attachments — including ones replayed from conversation history — and the
  agent reads their bytes from the sandbox, not the model. Without this the
  provider rejects the request: 'invalid content type=file_base64'.
- provider(localagent): also strip those parts from the current user message
  alongside the sandbox-path note (model-facing clarity; the requester is the
  real safety net for history).
- tests: cover the requester strip/keep behavior (file dropped, image kept and
  reshaped to image_url, mixed history, plain-string content).

* test(box): cover inbound/outbound attachment helpers; fix ruff format

- ruff format localagent.py (CI ruff format --check was failing)
- add unit tests for ResponseWrapper outbound-attachment helpers (wrapper.py 78%->98%)
- add unit tests for LocalAgentRunner._inject_inbound_attachments
- add unit tests for WebSocketAdapter._process_image_components (0%->covered)

Lifts PR patch coverage from 68.97% to ~88% (>75% target).
2026-06-18 21:40:31 +08:00
Junyan Qin (Chin) d09b823c49 refactor: switch webchat from sse to websocket (#1808)
* refactor: switch webchat from sse to websocket

* perf: image preview dialog

* chore: remove console.log
2025-11-28 14:54:01 +08:00
devin-ai-integration[bot] 4f2ec195fc feat: add WebChat adapter for pipeline debugging (#1510)
* feat: add WebChat adapter for pipeline debugging

- Create WebChatAdapter for handling debug messages in pipeline testing
- Add HTTP API endpoints for debug message sending and retrieval
- Implement frontend debug dialog with session switching (private/group chat)
- Add Chinese i18n translations for debug interface
- Auto-create default WebChat bot during database initialization
- Support fixed session IDs: webchatperson and webchatgroup for testing

Co-Authored-By: Junyan Qin <Chin>, 秦骏言 in Chinese, you can call me my english name Rock Chin. <rockchinq@gmail.com>

* perf: ui for webchat

* feat: complete webchat backend

* feat: core chat apis

* perf: button style in pipeline card

* perf: log btn in bot card

* perf: webchat entities definition

* fix: bugs

* perf: web chat

* perf: dialog styles

* perf: styles

* perf: styles

* fix: group invalid in webchat

* perf: simulate real im message

* perf: group timeout toast

* feat(webchat): add supports for mentioning bot in group

* perf(webchat): at component styles

* perf: at badge display in message

* fix: linter errors

* fix: webchat was listed on adapter list

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Junyan Qin <Chin>, 秦骏言 in Chinese, you can call me my english name Rock Chin. <rockchinq@gmail.com>
2025-06-08 15:34:26 +08:00