8.5 KiB
KnowledgeEngine shared-runtime source gate
All five KnowledgeEngine plugins declare shared placement (sharedRuntime: shared-runtime-v1 + componentModel: stateless-v1): LangRAG, LongTermMemory, DifyDatasetsConnector, FastGPTConnector and RAGFlowConnector. Placement and certification are separate: no signature, marketplace publication, Cloud admission, sandbox, real-provider execution, or production readiness is claimed here. The sections below record what is proven per plugin and what remains a release/certification gate; a gate that is not yet satisfied is listed as such rather than treated as a reason to stay dedicated.
LangRAG evidence and limits
The SDK 0.7.4 subprocess probe certification-tests/shared_subprocess_probe.py launches a separate Python worker, creates exactly one PluginRuntimeController / BasePlugin / each declared component object, attaches two distinct InstallationBinding Workspaces on the same WebSocket transport and asserts the object identities coincide inside that child. Both Workspaces successfully ingest and retrieve distinct markers with the same Host IDs; the Page snapshot succeeds for both. The Host vector/storage/embedding APIs are local scoped fixtures, not production services. Reproduce from repository root:
PYTHONDONTWRITEBYTECODE=1 /home/rock/work/ke-shared-test-venv/bin/python KnowledgeEngine/certification-tests/shared_subprocess_probe.py KnowledgeEngine/LangRAG
The single run does not establish revocation races, production persistence, config revision, or deterministic vector repair following ambiguous partial commits. The previous source correction preserves binding-local fences, invocation-bound telemetry, and settled CPU offload; telemetry is diagnostic only, not an accounting ledger. Keep release/certification gated on those separate checks.
LongTermMemory
The same real-child probe temporarily overrides the in-memory manifest to shared mode, attaches two Workspace slots and invokes both Page /summary endpoints with distinct task-local profile limits, proving the SDK object graph can host both slots. It does not exercise representative writes through Tool, Command, KnowledgeEngine and EventListener, nor interrupted nested episode operations, so those remain release gates. The manifest now declares shared placement, and the in-probe override is kept only so the probe stays runnable against older checkouts. Run the probe with KnowledgeEngine/LongTermMemory to reproduce the limited evidence.
External connectors
Dify, FastGPT, and RAGFlow now serialize mutations and fence ambiguous errors per knowledge base within the complete installation binding, without creating a tenant-bearing detached task; one ambiguous knowledge base never blocks its siblings sharing the installation. A durable plugin-storage intent precedes upload; Host file ID, KB ID, upstream ID, dataset, and state are recorded under an installation-bound key when returned. Deletion resolves only the exact mapping for the requested knowledge base — the canonical record, or a record that same knowledge base itself wrote and that carries the older Host revision's file UUID — and refuses missing/pending mappings and mismatched datasets; an absent exact mapping refuses the operation with no provider request and no storage write, and a tombstone is recorded only after upstream acknowledgement. A record written under another knowledge base's id is never adopted: historic collection-keyed data moves to the canonical key only through explicit reconciliation, never a scan-based copy. Only a dispatched mutation whose outcome is unknown fences: a deterministic pre-call validation error and ordinary pre-dispatch cancellation do not. FastGPT and RAGFlow order dispatch and release around that fence: the pre-written protection must be persisted before the mutation is dispatched (persist-before-dispatch is a precondition, not a best effort — a fence whose Host write fails aborts the mutation before any remote call, raises a dedicated error, and leaves no in-process marker behind), and the protection is released only after the provider result was confirmed and the Host result/mapping (or deletion tombstone) has been recorded, so the wrapped operation of a dispatch covers the request, the response validation and that durable write. A dispatch also refuses a knowledge base that is already fenced, so a later successful operation never clears an earlier unresolved operation's fence: a chain that reaches an unresolved step stops there (an unknown RAGFlow GraphRAG outcome no longer lets the RAPTOR trigger's success clear the fence). An unreadable fence store is not read as "no fence": the mutation is refused with the Host failure until the read succeeds, and the failed read is not cached. A mapping also records the connector's normalized upstream target, and deletion refuses a mapping recorded against a different target (or with none recorded) before any remote call. Fences live in Host storage under a knowledge-base-scoped key with an in-process cache, so a restarted worker still refuses that knowledge base, and deleting the knowledge base clears its fence; the plugins also implement BasePlugin.on_installation_revoked to drop per-binding process-local locks and fence cache on SDK revisions that dispatch it (never called by langbot-plugin 0.7.4). Shared placement is declared; integrated deletion still waits on the verification listed below.
The Host may already persist IngestionResult.document_id and pass that upstream ID on delete, but this source alone cannot prove which deployed Host revision is used. Provider DELETE acknowledgements here are not yet verified with an authoritative upstream list/readback, and an upload timeout after remote commit leaves a durable pending intent requiring manual reconciliation. Config KB tombstones must not precede cleanup of outstanding mappings; the Host needs to retain the file row until verified upstream absence. For RAGFlow, an acknowledged upload followed by failed parse needs reconciliation rather than blind reupload. Existing connector fixture tests written for deleting arbitrary unrecorded IDs now fail by design; cancellation fixtures with deliberately detached Host commits no longer model the SDK-owned revocable action. Replacement fixture tests cover per-knowledge-base fencing and sibling isolation, fence persistence across a worker restart, fence clearing on knowledge-base deletion, ambiguous provider/transport failures, non-fencing of deterministic validation errors and pre-dispatch cancellation, refusal (and retry) when the persisted fence store cannot be read, deletion refusing a mapping recorded for another upstream target, and the revocation hook. For FastGPT and RAGFlow the fixture tests additionally cover the fence lifecycle: an unpersistable pre-dispatch fence aborting the mutation before any provider request without leaving a stale fence, a response that fails validation (an undecodable body) keeping the knowledge base fenced and refusing the next mutation, a cancellation that lands after the provider acknowledgement but before the Host mapping/tombstone write keeping the fence, an unknown GraphRAG outcome stopping the RAGFlow chain before RAPTOR, and a later successful operation never clearing an earlier unresolved operation's fence. Real upstream A/B ingestion, deletion absence and cancellation/revision proof remain release gates.
The items above are certification and release gates, not placement blockers: an ambiguous upstream outcome needs the same manual reconciliation under dedicated placement, so they do not justify staying dedicated. All three connectors declare shared placement. No published certification is claimed for them yet.
The real-child two-binding probe certification-tests/shared_subprocess_probe.py now also takes a
connector directory: one controller/plugin/component graph attaches two InstallationBinding Workspaces
(asserting SAME_OBJECT_GRAPH=true inside the child) and runs the same Host ids through
create/ingest/retrieve/delete per binding against local Host and provider fixtures. Each binding is asserted
to ingest with its own document id and file bytes, to retrieve only hits carrying its own credentials, to keep
its own api_key under the same Host storage key, and to delete using only its own recorded upstream id, with
the provider call log partitioned per binding. certification-tests/verify_candidates.py now builds and
exercises all five KnowledgeEngine archives, LongTermMemory included. These remain local fixtures: live
provider acceptance is still a release gate.