Tracing ran inline with the request: every traced call did several database
round trips (dedupe probe, actor-name lookup, newest-hash read) and a
serialized insert. With tracing on, the WebUI's burst of parallel requests
(login alone touches a dozen endpoints) saturated the write path and stalled
the whole service.
Now the request path only builds a bounded dict and pushes it onto a queue
(measured ~0.005ms per trace, zero database work). A single background writer
coroutine drains the queue and persists sequentially, which also removes the
read-newest-hash race that used to produce false "chain broken" reports; the
previous asyncio lock is gone. The actor display-name lookup and the dedupe
key hash move into the writer as well. The queue is bounded: when traffic
outruns the writer the oldest pending trace is dropped instead of slowing the
request down.
Verified: migration tests pass, ruff clean, a focused writer test shows
enqueue is non-blocking, rows link with zero chain breaks and no internal
key leaks into the stored row.
- Chain integrity: the "read newest hash, then insert" pair in record() could be
interleaved by concurrent traces (the panel opens a dozen parallel GETs), so two
rows linked to the same predecessor and one looked like a broken chain even
though nobody edited anything. Serialize the pair with a per-service lock.
- Resource classification: generic verbs fell back to resource_type='resource', so
almost every read showed up as "View resource / Resource" (pipelines, bots,
providers, users, monitoring...). Derive the family from the registered route
identity (pipeline, bot, model_provider, llm_model, monitoring, user, workspace,
webhook, api_key, system).
- Frontend: render the resource family for the new types (8 locales) and keep the
resource_id badge added earlier.
Verified: backend integration 358 passed, ruff check/format clean; frontend tsc,
eslint, prettier and check-i18n all pass.
- Move the operation-traceability service and its HTTP routes into a standalone
pkg/operation_trace package; the Core controller package no longer depends on it
(routes stay auto-discovered by the controller package scan).
- Gate the hot path on one module-level boolean exposed as ap.operation_trace_active:
while no Workspace has tracing enabled the route wrapper performs zero trace work
and zero database round trips. The gate opens when a Workspace enables tracing and
is primed once at startup for already-enabled Workspaces.
- Record the installed extension identity for GitHub / marketplace / local / upload
installs (owner/repo, author/name, filename) so the trace no longer shows a
nameless "a plugin was installed" event and the card names the extension.
- Merge migrations 0032 + 0033 into a single 0032_operation_traceability revision
carrying the full table shape and an index set that matches the ORM.
Tests: unit 5165 passed / integration 358 passed; ruff check + format clean.
- Remove the unused request_summary local: the stored summary is derived by the
service from changes + the classified action, so handlers publishing
quart.g.operation_log_summary was a dead channel that never reached the row.
- Drop the unused datetime import in the operation log model.
- Apply ruff format to the touched files so `ruff format --check` passes.
Granularity follow-up: a record used to say only "installed an extension".
- Add resolve_resource_identity(): derive the acted-on resource from trusted route
params or the install-style payload (plugin_author/plugin_name), so every
resource family names its target. Unknown keys are ignored and sensitive keys
are skipped, keeping the audit trail bounded and secret-free.
- The route wrapper applies it to every traced request; the identity only fills
in when a handler did not publish a more precise one.
- Publish field-level before/after diffs for plugin config, skill, knowledge
base, MCP server, pipeline, provider and model updates, reusing
changed_fields()/build_summary() so secret-looking keys stay redacted and a
masked round-trip never shows up as a phantom change.
- Count hash mismatches / broken links over the whole filtered history instead
of only the returned page, so the badges no longer reset to zero on page 2.
- Classify failing record ids during the same scan and expose them through an
integrity=all|issues|hash_mismatch|chain_broken listing filter.
- Report scanned_count / scan_truncated so an approximate (row-capped) summary
is never mistaken for a clean one.
- Resolve the acted-on resource identity from route params or the request body,
giving every resource family a "which plugin/skill/knowledge base" trace.
- Render resource types and the two counters through i18n, keeping all eight
locale files key- and line-aligned.
- Export download failed because the URL builder appended the path to a
"/" base, producing a protocol-relative "//api/..." URL that the browser
read as host "api"; the request never reached the backend. Resolve the
base to the current origin, mirroring the other URL builders.
- Remove the orphan ``clear_prune`` action: the branch has no prune/clear
route and no route rule maps to it, so it could never be produced. Its
i18n key is removed too.
- Remove ``tamperedCount``, now unreferenced after the verification badge
was split into integrity/chain counters. The operationTrace catalogue is
now free of unproduced keys.
- Correct the settings controller docstring: only governance, operation
logs, filters and export routes exist; there is no on-demand delete.
Verified: no producer for clear_prune, classify() can no longer emit it,
operationTrace orphan scan returns none, tsc/check-i18n/prettier/py_compile
all pass.
Tracing is now opt-in: a Workspace records nothing until an owner or admin
explicitly turns it on, so auditing stays off the hot path by default.
View and configure remain limited to owners and admins: the audit.view
permission is granted to those roles only, non-owner/admin callers receive a
403 from the governance write route, and the panel already gates both on the
owner/admin membership role.
- Capture scope: treat the ``audit`` bucket (viewing the log / tracing
settings) as a read-level observation instead of recording it from the
mutation level. At "mutations only" the log no longer fills with GET page
views; views are recorded only at the read level.
- Coverage: classify plugin lifecycle (install / upgrade / config / page),
skills, knowledge bases and MCP servers, with dedicated actions and
resource types, so those operations are traced instead of falling into the
generic resource bucket. Route rules now support multi-fragment AND
matching and per-verb read/write actions.
- Panel: move "Download logs" next to "Refresh" in the toolbar.
- Verification: count hash mismatches and broken links separately instead of
collapsing both into one "tampered" signal.
- i18n: add the new action labels plus the split verification labels in all
eight locales.
Performance of the operation traceability surface:
- Cache the retention / row-budget / dedupe policy per Workspace and
invalidate it on governance change, so the hot write path no longer
issues three metadata round trips per recorded operation.
- Resolve the governance payload with one metadata query instead of four.
- Turn the dedupe probe into an existence check (LIMIT 1) instead of a
full COUNT over the dedupe window.
- Defer the row-budget COUNT to every N inserts; prune still enforces the
exact budget, and the maintenance loop keeps it bounded.
- Panel: fetch only the records page when paging or filtering instead of
re-requesting governance and the filter catalogue every time.
- Export: include the effective filters (and limit) in the artifact so a
downloaded CSV is self-describing.
Fixes and cleanup:
- Drop the unused OperationLogPruneReport import that broke `tsc`.
- i18n: remove the 9 redundant blank lines unique to en-US and format the
six non-compliant locale files with the project Prettier config; all
eight locales keep identical key sets, key order and placeholders.
A saved pipeline may carry the post-migration shape
{'ai': {'runner': {'id': ''}, 'runner_config': {}}} while no runner has been
selected yet. The planner matched the empty id against the certified
'plugin:author/name/component' form, failed, and returned a malformed-id
blocker, so the whole 'migrate all' batch stopped with both pipelines reported
as blocked even though there was no legacy section and nothing to convert.
A blank id with no legacy runner section now reports not_legacy: no work is
needed and no target is synthesized. A blank id that still coexists with a
legacy section remains an explicit invalid_runner_id blocker, because that
ambiguous state must be resolved by the operator rather than guessed.
The certified-archive admission gate treated any declared certificate it could
not resolve as an untrusted archive and rejected the install with
CERTIFIED_PLUGIN_OSS_FORCE_REQUIRED. OSS ships an empty
plugin.certification.trusted_public_keys ring and the marketplace signs every
package with its own issuer key, so all certified marketplace packages (for
example langbot-team/RunnerDemo and langbot-team/LocalAgent) failed at the
'validating plugin package' step before artifact storage.
Admission now distinguishes an unresolvable declaration from a configured trust
decision that fails:
- record certificate_id only when the key_id is actually present in the
configured ring, so an empty ring yields an unresolvable declaration;
- OSS degrades an unresolvable declaration to the existing oss_dev dedicated
profile with CERTIFIED_PLUGIN_OSS_UNTRUSTED_DEDICATED instead of blocking;
- a resolvable declaration that still fails keeps requiring the explicit
administrator force (CERTIFIED_PLUGIN_OSS_FORCE_REQUIRED);
- Cloud stays fail-closed and rejects before storage.
No shared-runtime privilege is granted when the issuer is not trusted, so this
withholds isolation rather than escalating it.
Resolve conflicts in the workspace assistant integration:
- DynamicFormItemComponent: keep HEAD's compact model selector and
`disabled` field support while adopting master's `sortModelsByCatalog`
ordering and `MODEL_SELECT_TRIGGER_CLASS`.
- i18n (en-US, ja-JP, zh-Hans): keep both the `assistant` namespace from
HEAD and master's `sidebarGuide` / `pipelineMigration` additions.
Add 0030_merge_assistant to join the assistant conversations branch with
the released chain so the migration graph converges on a single head.
Rename it from 0030_merge_assistant_conversations to stay within the 32
character revision limit enforced by test_migrations.
Also add assistant button docking (long-press drag, edge collapse to a
short blue rail, hover restore) with pure helpers in assistant-dock.ts
and unit coverage, plus auto-collapse for finished tool result cards.
Resolve the revision-graph conflict introduced by the release line.
Conflicts
- tests/integration/persistence/test_rag_document_identity.py: master had
independently introduced the same dynamic-head helper (`current_head`) plus
a `DOCUMENT_IDENTITY_REVISION` constant, and it explicitly upgrades to that
revision. Keep master's semantics (the explicit revision matters because
upgrading to the head now also traverses the TOTP branch) while keeping the
module-level alembic imports on this branch.
New migration
- 0030_merge_totp_into_release joins the TOTP branch's own merge revision
(0026_merge_totp_and_rag_identity) with the release head
(0029_merge_rag_identity). Both reached 0025_rag_document_identity without
including each other, which left Alembic with two heads and made
`upgrade head` fail with "Multiple head revisions are present".
Verification
- Single Alembic head confirmed (0030_merge_totp_into_release).
- tests/integration/persistence/test_rag_document_identity.py: 34 passed.
- Full fast integration suite: 356 passed, 84 skipped.
Address the migration and formatting failures reported on the TOTP branch.
Migrations
- Register totp_credentials and totp_recovery_codes in
_ALEMBIC_TENANT_TABLES. On a legacy PostgreSQL install these two tables
reference users.uuid, which only exists after 0009, so create_all() must
not run ahead of Alembic the way it did for the other tenant tables.
Without this the PostgreSQL migration test failed with
"column uuid referenced in foreign key constraint does not exist".
Tests
- Resolve the Alembic head dynamically in the RAG document identity
regression instead of pinning 0025_rag_document_identity. The TOTP and
RAG branches now meet at a merge revision, so the pinned value was no
longer the head. This matches the convention already used by
test_migrations_postgres.
Formatting
- Apply ruff format to the new backend modules and prettier to the locale
files and TOTP components, so the lint jobs pass.
Add a per-Account time-based one-time password (TOTP) second sign-in
factor, plus the owner/admin tooling needed to operate it.
Backend
- TotpService: enrolment, constant-time verification with a +/- one step
drift window, single-use recovery codes and disablement.
* shared secret is only ever persisted as a Fernet token whose key is
derived (HKDF-SHA256) from the instance JWT secret and gated by a
key_version epoch;
* recovery codes are only ever persisted as salted
PBKDF2-HMAC-SHA256 digests (600k iterations) and are single-use;
* the consumed counter advances monotonically so a captured code cannot
be replayed inside the same time window.
- New persistence entities and alembic migrations for credentials and
recovery codes.
- Login second-factor challenge, bound to the Account that passed the
password step.
- Owner/admin oversight endpoints to inspect and re-bind the second
factor of any Account.
Frontend
- Account settings panel for enrolling, managing, re-binding and
disabling the second factor.
- Login second-factor step and the matching client methods.
- Strings for all eight locales.
The shared secret never crosses the API boundary: enrolment returns a
server-rendered QR code (as a data: URL) and the plaintext secret is
discarded as soon as the image is produced.
* fix(pipeline): stop emitting empty final assistant messages
An empty final streaming chunk was appended as an empty message chain when no sandbox outbox attachment was present, so platforms received an empty reply. Only append the chain when attachments were actually collected, and gate the "Call ..." tool notice behind output.misc.track-function-calls so it no longer becomes the final chunk of a stream.
* fix(wecombot): do not close stream on a blank final snapshot
A blank final snapshot closed the WeCom stream with an empty bubble and pushed the real answer into a separate reply_text message. Skip blank final snapshots and keep the session open so the following non-blank chunk can finalize it. Covered by new regression tests.
Preserve the published timezone-less schema and epoch API contract across run lifecycle, deadlines, leases, heartbeat registry, events, transcripts and retention cutoffs. Exercise the real Host journal with asyncpg under a non-superuser role on metadata and published-migration schemas; 32 PostgreSQL regressions fail before the fix and pass after it.
Resolved one conflict in src/langbot/pkg/plugin/connector.py. Master added
archive admission (`_admit_plugin_archive` + the certification digest check
before Runtime apply) at the same points this branch added install-stage
reporting, so both sides are kept:
- the archive is admitted first, then the 'inspecting plugin package' stage is
reported
- the certification digest is verified before Runtime apply, then the
'installing or starting plugin' stage is reported
Verified after merge: py_compile + ruff, tsc --noEmit, 98/98 unit tests.
Addresses review feedback on the marketplace installed-state change.
- Progress: byte-derived progress no longer has time-based drift layered on
top, and fallback drift is clamped to the current stage range, so the bar
cannot exceed the download band. 90/100 bytes at 40s elapsed used to report
61% against a declared 5-45% band; it now reports 41%. Drift is measured
from when the stage was entered rather than from task start.
- Skills: a marketplace skill is no longer reported installed from a bare
skill name. The backend names skills from their own SKILL.md and records no
publisher, so alice/review and bob/review install identically; matching the
bare name marked every publisher's skill as installed. Skill cards now
resolve to not-installed until the installed skill carries a publisher.
- Stages: "installing plugin dependencies" and "launching plugin" described
work this task context cannot observe (installation persistence ran under
the former; the runtime installs dependencies and starts the plugin inside
apply_plugin_installation under the latter). They become "persisting the
installation" and "installing or starting plugin", and the frontend maps
that combined step to the dependency stage rather than the launch stage.
- Removed the per-dependency progress fields: the backend never populated
them, so the UI could never have displayed them.
The stage mapping and progress maths move to install-progress.ts, and the
installed-state matching to a React-free marketplace-installed.ts, so both
are covered by executable tests (+9).
Verified: ruff, tsc --noEmit, prettier --check, eslint (0 errors), 98/98 unit
tests.
Marketplace cards now reflect whether an extension is already installed in
the current workspace, and the installed-extension list gains a search box.
Backend (stream install progress):
- _read_httpx_response_limited gains an optional task_context: it publishes
download_total from Content-Length before the first chunk and updates
download_current / download_speed per chunk. The marketplace download path
previously had no progress reporting; it now matches the GitHub path.
- _marketplace_get forwards task_context to that helper.
- install_plugin resets the per-install counters so re-installing the same
plugin cannot inherit stale metadata, and reports human-readable stages:
preparing -> downloading -> inspecting -> storing -> installing
dependencies -> launching -> waiting for plugin to become ready.
Frontend (installed state):
- New marketplace-installed helper normalises the sidebar identities
(plugin: author/name, mcp: author__name, skill: bare name) into one
type:author/name index and resolves a card's installed state from it.
useMarketplaceInstalledIndex memoises on the sidebar lists, so a finished
install (which refreshes the sidebar) re-evaluates the cards automatically.
- PluginMarketCardVO carries installed / hasUpdate. An installed extension
turns its download affordance into a hollow green ring with a green check
in place, instead of adding a separate badge; the count slot switches to
the installed label. Cards with an available update use amber.
- PluginMarketComponent derives the annotated list and shares the index with
RecommendationLists.
Frontend (install task UI):
- mapActionToStage matches the new connector stage strings. The pre-download
stages are checked before the generic "install" match, because
"preparing plugin install" also contains "install".
- Stage progress ranges are non-overlapping; overall progress interpolates on
real byte counts while downloading and drifts monotonically elsewhere,
capped at 99%.
- The progress dialog and task queue expose the launching stage.
Frontend (installed list search):
- The installed list had no search at all. A query box in the page header
filters by label / name / author / description, case-insensitively, applied
before grouping so grouped and flat views both honour it.
- Search misses and an empty list now show distinct empty states, with a
clear action on a search miss.
- AsyncTask entity gains the optional created_at field.
i18n: new marketplace / install / search strings across all 8 locales.
Verified: ruff format + check, tsc --noEmit, prettier --check, eslint
(0 errors), and 89/89 frontend unit tests.
TyperBody