mirrors/openclaw
0
0 Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-10-26 10:13:27 +08:00
Code Activity

test(core,ui): remove low-value tests (batch d480) (#182734)

Browse Source 
* test(core,ui): remove low-value tests (batch d480)

Retain the reviewed, CI-corrected subset of campaign batch d480: 24 test files, net -4125 lines, and one shrink-only timeout baseline update. Production code is unchanged.

Restore incompatible or insufficiently justified reductions to the pinned main. Preserve review and failure dispositions in PR #182734. Local proof is formatting, plain oxlint, and diff checks only; hosted CI owns runtime and type proof.

This squashed rebase preserves the reviewed final tree while allowing the line-cap hook to compare the actual batch against main, not an intermediate reduction. Original replayed commit messages follow for provenance; their historical scope and first-pass metrics are superseded by the final PR body.

test(gateway): deslop r0083 tests

Factor repeated session writes, list reads, dispatch gates, settings, and ACP fixtures without removing lifecycle or authority cases.

Remove the HTTP phased-output replay: commentary fallback and cursor identity remain covered by session-history-reads.test.ts; commentary/final sibling projection by chat-display-projection.media.test.ts; output_text normalization and control suppression by server.chat.gateway-server-chat-history.test.ts. Retain HTTP history and SSE boundary tests. Reduce same-path Accept variants while retaining each parser decision path.

CI-only campaign: formatting, plain oxlint, and diff checks pass. Runtime tests and coverage intentionally deferred to hosted CI. Reduction is 4964 to 4803 lines (3.2%), below the 35% target because remaining cases protect distinct behavior.

test(gateway): remove duplicate r0083 coverage layers

Replace randomized selector replay with existing resident resolution and scoped-locator proof in sessions-resolve-projection and session-row-projection. Keep active sentinel precedence, current child ownership, exact pagination, archive-zero, and owner-roster regressions. Search-scope and session-utils-owners cover spawned classification; archived projection owns promote/demote replay.

Retire cron handoff/rearm replay covered by server-cron owner integration tests; retain reserved replacement, cleanup uncertainty, and retry races. Retire incognito reset and initiating-admission replay covered by session-lifecycle-incognito and session-controller/session-lifecycle-admission owner tests.

Remove HTTP profile, forwarding, hidden-turn, parser-spy, and private-retention replay covered by chat-display-projection, server.chat-history-cursor, session-history-state, and focused accessor tests. Remove chat abort/pre-admission/root replay covered by chat.abort-persistence, sessions.abort-agent-scope, gateway-work-admission, and chat-send-pre-admission. Preserve legacy settings capture, stale rejection, and restart persistence ordering.

Trim project credential, attachment custody, failure-history, alias, and fixture-inventory replay covered by clone-identity.e2e, pending-input-media-custody, create.durability, workspace-policy, worktree-base, and service.gc. Keep actual deferred setup, abort, retry, sandbox, corrupt input, and prepublication GC custody.

CI-only: plain oxlint, oxfmt, and diff check pass; no test/build/coverage runs. Original shard 4964 lines to 3593 (27.6 percent), below the requested 35 percent; further deletion would remove distinct retained boundary or lifecycle proof.

test(infra): deslop r0084 tests

test(infra): consolidate r0084 overlapping coverage

Remove native TEMP-schema replays covered by sqlite-schema-facts.temp.test.ts: keeps TEMP trigger admission local, revokes admission for ordinary DDL, expires rolled-back TEMP facts, and does not suppress reentrant MAIN DDL. Retain native tracking-schema and read-scope behavior.

Consolidate stale-generation event rejection into projects model events only into their current run owner, and retained-routing rejection into reserves exclusive run ids for owner-only delivery and cleanup.

Remove SQL call-shape instrumentation and copied backup inventories while retaining observable integrity rejection, exact restored rows, unrelated-session preservation, and live Git scheduling proof. Exact migrated identity/socket bytes remain covered by client-credential-migration.test.ts and client-credential-migration.postgres.test.ts. CI-only: oxfmt, plain oxlint, and diff-check passed; no tests run. Reduction remains below the requested 35 percent because the retained lifecycle, custody, and durability cases cover distinct paths.

test(infra): trim r0084 repeated fixture layers

Reuse credential maintenance cleanup and compact fixed SQL seeds. Orphan rebuilding remains covered by client-credential-migration.test.ts: leaves copied on source rollback and rebuilds an orphan without merging newer cached tokens.

Remove native stale-cursor replay covered by sqlite-native-observer.test.ts: binds iterators immediately and holds custody until exhaustion or explicit return; preserves native cursor behavior when an invalidated iterator is stepped. Keep schema-level settlement and the unique TEMP tracking-write check; ordinary TEMP admission remains in sqlite-schema-facts.temp.test.ts.

Keep configured custom-owner and main-override relocation; remove their explicit standard-directory replay and fresh missing-source replay covered by doctor-state-migrations.test.ts. Keep effective credential preservation, selected-env exact transfer, independent order/override cases, and all integrity/authority/crash paths. CI-only: plain oxlint, oxfmt, and diff-check passed. No tests run; 35 percent reduction remains unmet.

test(chat): deslop r0081 tests

Consolidate repeated browser layout/style measurements and activity/turn fixture markup. Keep every test case, including viewport, engine, focus, scroll, and ordering boundaries. Gateway/history tests remain unchanged because their state and lifecycle paths are distinct.

test(chat): reduce r0081 fixture and assertion duplication

Consolidate same-path terminal variants and trim repeated layout readers, progress snapshots, closed menu fixtures, and history request payload assertions. Keep both terminal ownership branches and all layout assertions. Configured-main alias replay remains covered by session-key.test.ts (uses advertised main identity) and the cache owner's normalization path. Retain the keyed-commentary final regression from af210e1f3f, disabled-commentary delivery, and content-block tool identity extraction after preservation review.

test(chat): consolidate r0081 layout and event fixtures

Share layout snapshots, composer selectors, geometry assertions, and minimal transcript fixtures. Retain layout-specific image, virtual-row, scrolling, and hit-target checks. Collapse reconnect and late-abort variants into tables while preserving both outcomes and exact transcript assertions.

CI-only: plain oxlint, oxfmt and diff-check passed; no tests or coverage run. Original 5168 lines to 4439 (-14.1%); the 35% target remains unmet because no equivalent keeper was found for the distinct layout and event-ordering regressions.

test(agents): deslop r0085 tests

Consolidate transcript, subscription model, and recovery fixtures while preserving distinct authority, signature, streaming, and persistence paths. Fold equivalent thinking cases into tables without reducing their expanded cases.

The fresh-output table covers the removed duplicate frozen-history growth case. Retain explicit untruncated-frozen overflow coverage for the source-versus-replacement path. Validation is static only under the campaign CI-only instruction; no tests or coverage run.

test(agents): consolidate r0085 owner-boundary coverage

Replace repeated recovery stream and transcript setup with local fixtures, retain exact stream results and retry/notification boundaries, and remove redundant compaction call inventories.

Prompt-surface wiring remains covered by system-prompt.test.ts (resolves helper session keys to scoped prompt surfaces), embedded-agent-runner/system-prompt.test.ts (forwards the subagent prompt surface to embedded prompt rendering), and plugins/commands.test.ts (normalizes and filters structured agent prompt guidance by surface). Reconciler behavior remains in provider-stream.lifecycle.test.ts (registers provider streams with the resolved runtime lifecycle handle; first direct-compaction provider request).

Remove generic warning-cache, lease-reuse, and multi-result cap replays covered by infra/dedupe.test.ts, session-prompt-state.test.ts, tool-result-truncation.cache-ttl.test.ts, and tool-result-truncation.estimate.test.ts. Keep alias admission, source-versus-replacement frozen history, authority, lifecycle, spill and SQLite contracts. Static lint/format/diff checks only under the CI-only campaign instruction. Cumulative line reduction remains below the 35 percent target; no distinct boundary was removed to force the metric.

test(agents): factor r0085 cancellation and projection scenarios

Share paused preparation setup across admission, context-engine, model-resolution and budget-cancellation rows without removing their distinct authority or cleanup checks. Consolidate sanitizer and stale-signature fixtures, projection assertions and SQLite recovery setup; remove redundant native argument and prompt-setter echoes.

All changes remain in the three shard tests. Static formatting, plain oxlint and diff checks pass; no tests or builds run under CI-only instructions. Cumulative shard lines are 5083 to 4015, still below the requested 35 percent reduction. Keep the remaining distinct authority, persistence, spill and recovery behavior rather than delete it for the metric.

test(plugins): deslop r0086 tests

Consolidate provider alias and embedding fixtures, dimension assertions, and marketplace temp setup. Retain the same trust, redaction, cleanup, and protocol boundaries.

Remove duplicate metadata projection checks covered by projects without rediscovery and completes a planning view. Cross-process exclusion remains in reloads install records after waiting for another process, now asserting that a blocked callback never runs. Nested lease identity remains in the demand test.

CI-only campaign: plain oxlint, oxfmt, and git diff --check pass. No test, coverage, build, or Testbox runs. Reduction is below the 35 percent target; retain distinct source-capture, lifecycle recovery, and metadata boundary cases.

test(plugins): consolidate r0086 fixture layers

Replace fabricated marketplace response objects with real streams and share known-marketplace setup. Remove Git argv replays covered by git-source.test.ts: preserves each caller ref selection against a local repository.

Reuse existing HTTP and CLI child-process fixtures; preserve wire bodies, cache identity, cancellation, phase ordering, and process-bound plugin recovery. Consolidate metadata registry setup and remove redundant internal call-shape assertions.

The update-volume duplicate remains covered by keeps update source copies in TMPDIR without relocating retained native captures, plus caller-custody tests. Two dimension input variants use the same mismatch branch as retained single-item and late out-of-order batch cases. External-policy fake full/legacy branches are replaced by one observable hook result.

CI-only: oxfmt, plain oxlint, and git diff --check pass. No tests, builds, coverage, or Testbox runs. Total shard reduction remains below 35 percent; distinct ownership and recovery boundaries retained.

test(plugins): remove duplicate r0086 boundary layers

Remove packaged-discovery and exact-source-owner cases duplicated verbatim in provider-discovery.capture-lifetime.test.ts. Provider-policy-ownership.test.ts keeps bundled first-winner precedence; preserve root-basename fallback in the routing table.

Remove the embedding oversized-response replay. minimax-vlm.normalizes-api-key.test.ts bounds large successful response bodies through the same default readProviderJsonResponse owner, proving the exact cap, early stop, and cancellation. Keep plugin-specific error and deadline behavior.

Trim source-capture worker custody and payload-retry replays covered by plugin-generation-artifact.open-batch.test.ts and plugin-source-capture-recovery.test.ts. Preserve revoked cleanup authority and instance-removal retries. Keep metadata boot test focused on its scheduler contract.

Reduce lease outcome combinations to the unique successful-result-with-cleanup-failure case; retained process-cleanup and revocation cases cover operation failures, and demand tests cover default-path serialization.

CI-only: plain oxlint, oxfmt, and git diff --check pass. No tests/builds/coverage. Total reduction remains below 35 percent; no further proven duplicate layer found.

test(config): deslop r0087 tests

Consolidate recovery race fixtures, raw-file assertions, and write scenarios while retaining distinct authority, durability, and environment-provenance cases.

Remove the repeated prefix-recovery cap exercise: io.clobber-snapshot.test.ts retains concurrent cap, timestamp rotation, and one-warning coverage. The writer recovery case retains exact forensic bytes and permission hardening.

CI-only validation: plain oxlint, oxfmt, and diff-check passed. No tests or coverage run. Reduction remains below 35% because remaining cases cover distinct boundaries.

test(config): consolidate r0087 owner-boundary coverage

Remove repeated policy, include-provenance, model-map, locking, numeric-key, and recovery-policy exercises. Retain distinct writer authority, rollback, recovery errors, and all input-intent variants; share fixtures and reduce internal argument assertions.

Remaining proof: io.meta.test.ts defers unresolved restrictions and honors explicit allow-any; includes.test.ts records arrays, sibling overrides, delegates and allowed roots; io.write-prepare.test.ts covers touched model maps and explicit canonical maps; env-preserve.test.ts covers ambiguous references and escaped moves; io.paired-source-write.test.ts retains real environment-drift writes.

Lock and admission proof remains in io.write-lock.test.ts. Numeric-key proof remains in io.write-prepare.numeric-keys.test.ts. Committed revision proof remains in gateway/server.config-patch.test.ts (config.set pairs the committed config and revision while another writer waits). Root nonfinite and comment-warning integration stays in this shard.

Recovery duplicates are covered by io.snapshot.recovery.test.ts (replaced-backup identity drift), recovery-policy.test.ts (plugin-local invalid config), and io.observe-freshness.test.ts (newer persisted facts and worker-updated fields). Config journal fingerprint privacy remains in config-journal-snapshot.test.ts; the writer keeps committed-hash slot linkage. Authorized roster deletion remains in io.agent-deletion.test.ts.

CI-only: plain oxlint, oxfmt, and diff-check passed; no tests, coverage, or builds run. Total shard 5107 to 3981 lines, below the requested 35 percent reduction because further cuts would remove distinct behavior.

test(config): reduce remaining r0087 replay fixtures

Consolidate plugin-source persistence around its credential and authored-config outcomes, without repeating default materialization and notification inventories. MCP write behavior remains in mcp-config.test.ts; runtime publication in runtime-snapshot.test.ts; plugin defaults in validation.channel-metadata.test.ts. Keep both direct and MCP writes and both cold and reloaded runtime states.

Remove simple invalid-sibling repair replay covered by the retained nested-entry repair. Retire mutation-only revision, preflight-order, path-loss compensation, escaped-root, and shadowed-include replays: keepers are server.config-patch.test.ts committed revision, io.write-preflight.test-support.ts caller guard ordering, io.write-reread.test.ts compensation authority, retained include failed-refresh rollback, io.write-config.test.ts escaped literals, and include-write-boundary.test.ts same-path merges.

Retain the only direct write-time tilde restoration assertion and distinct recovery hardening, durability, and publication failures. Plain oxlint, oxfmt, and diff-check pass. No tests, coverage, or builds run under CI-only instructions. The requested overall 35 percent reduction remains unmet without removing distinct retained behavior.

test(state): deslop r0088 tests

Consolidate repeated fixture setup, exact anchor and migration assertions, and integrity gate setup without dropping any cases. Preserve worker publication, PostgreSQL migration, authority, recovery, and lifecycle interleavings. This already-consolidated shard cannot meet the 35% line target without removing distinct safety coverage.

test(state): remove duplicate r0088 coverage layers

Consolidate state v25-v28 upgrade proof into the retained v24 real-worker migration, preserving heads, leases, publication intent, index repair and admission metadata. Keep PostgreSQL-specific FTS and FIFO proofs; remove generic catalog, Doctor, auth and search replays covered by database-engine conformance suites.

Keeper map: worker idle renewal -> openclaw-agent-execution.idle; pooled factory refusal -> sqlite-worker-open-refusal; pinned WAL close -> openclaw-agent-db.integrity-cache; periodic maintenance -> sqlite-wal.agent-admission; publication cleanup outcomes -> openclaw-agent-worker-store.env-capture. Retain actual publication authority, drain, cancellation and failed lease cleanup checks.

GitHub bookkeeping -> github-repository-publication recovered-push and github-shared-publication-events; stale receipt -> state-domain-publication. Keep lazy schema insertion, source guards, closed-owner fencing and batch rollback. Native integrity prepared-existing and invalidated variants remain covered by openclaw-agent-execution.schema-admission; verified host proof remains in the retained two-leases case.

PostgreSQL keepers: model-catalog and post-ready conformance; mixed-agent-readers; gateway-session-startup and doctor-session-entry-state; session-search-product.postgres; transcript-projection-flush and transcript-identity-insert. Worker retention remains in store-retention. CI-only: formatting, plain oxlint and diff checks passed; no tests or coverage run. Original shard 4879 -> 3940 lines (19.2%); 35% target remains unmet without dropping distinct safety behavior.

test(state): consolidate r0088 lifecycle and integrity proof

Fold warm worker reopen into retained shutdown-lease coverage. Remove root-close integration replay: openclaw-agent-db-resources owns draining-root fencing and concurrent sibling settlement; openclaw-agent-db-open-authority and disposal own refusal before new state; the retained worker drain tests own accepted publication settlement.

Restrict native integrity matrix to actual failed-authority and live-revocation cases. Valid-database reuse, receipt custody and replacement are covered by openclaw-agent-db-validation-cache; empty-WAL and peer-lease semantics by openclaw-agent-db.integrity-lease. Retain real killed-owner corruption, legacy provenance, PID reuse and interrupted admission. Drop redundant peer return dumps and AbortSignal once-listener counts, preserving persisted results and terminal error identity.

Remove PostgreSQL close/reopen metadata replay covered by mixed-agent-readers, duplicate extension JSON covered by engine conformance, and the small FTS matrix replay covered by retained large and second-backend inserts. No production or support changes. Static gates only: formatting, plain oxlint and diff check passed. No tests or coverage run. Original shard 4879 to 3671 lines; the 35 percent target remains unmet rather than dropping distinct security or durability behavior.

test(gateway): consolidate r0090 lifecycle proof and fixtures

Keep branch representatives for finishing-result failures instead of replaying the stop/error cross product and cleanup wording variants. Preserve normal cancellation separately from drain-owned cancellation.

Remove executor outcome-unknown duplication covered by inference.test.ts: preserves independent authority and native failures through execute settlement. Keep exact claim rejection through the real adapter in stops an unconfirmed old launch before dispatching its replacement, replacing the mocked revalidation and same-helper size comparison.

Share abort providers, settlement observers, node adapter/workspace fixtures, and committed project setup proof. Retain explicit security, durability, replacement and shutdown interleavings. CI-only: oxlint, oxfmt and diff checks pass; no test runs.

test(gateway): deslop r0089 tests

Consolidate successful publication RPC assertions and repeated history message matchers without removing cases or changing authority and lifecycle scenarios. Keep separate proof for publication revocation/recovery, dispatch settlement, session durability, wire projection, and product authority boundaries. CI-only campaign: formatting, diff check, and plain oxlint passed; runtime and coverage verification deferred to hosted CI. Reduction is intentionally below the shard target because no safe redundant layer was established.

test(gateway): remove redundant r0089 proof layers

Retain real websocket reconnect, terminal ordering, publication authority, and early-checkout product proof. Remove duplicate lower-level assertions and exhaustive global order/attachment combinations. No production or shared helper changes.

Keepers: chat-send-attachments.owner.test.ts (unsandboxed cleanup before cancellation; managed-PDF pass-through cleanup) plus server.chat.gateway-server-chat-lifecycle.test.ts (sessions.abort terminalizes a restart-safe chat before agent adoption) replace the layered attachment-abort replay. chat-send-profile-binding.test.ts (preserves accepted native input across %s without adopting the retry socket; retains the original caller through initial authorization without an expected profile) owns receipt and profile rejection details. chat-send-admission-authority.test.ts and chat-send-admission.reads.test.ts retain admission revocation cases.

Store keepers: server.sessions.store-paths.test.ts (resolves and patches main alias to default agent main key), sessions-resolve-projection.test.ts (bounds exact resident selection without SQLite and sees committed label changes), server.sessions.effort-scopes.test.ts (write-scoped operators change and reset effort in an existing session), sessions-patch.test.ts (normalizes and clears fields; custom SVG persistence; archive attribution), session-row-selection.test.ts (maintains list order across metadata changes and archived-row rematerialization), sessions-patch-model-selection.test.ts (rejected patch preservation), and session-groups.test.ts (ordered catalogs, rename, delete). Retain successful write-scoped group RPCs and implicit model-runtime projection here.

Wire keepers: server-chat.stream-text-merge.test.ts (certified raw appends and uncapped boundaries), server-broadcast.live-text.test.ts (one recipient baseline and blocked appends), and remaining real wire assertions. Personal replay validation remains in github-repository-publication.test.ts (original personal selection/content replay); retained-index recovery remains in github-publication-git-index.test.ts (recovers the retained index when ambiguous CAS moved), while same-owner confirmation after restart remains here.

CI-only: no test, build, coverage, or Testbox execution. Plain oxlint, oxfmt, and diff check passed. Original shard 4421 -> 3928 lines (11.2%); 35% target remains unmet because further verified redundant layers were not established.

test(gateway): deslop r0089 tests

Reuse broadcast peers and publication transport fixtures; consolidate native replacement rows, caller setup, settlement helpers, and repeated RPC assertions.

Retained owner proof: sessions-github-read.target.test.ts covers visibility and awaited authority loss; sessions-github-read.test.ts covers pending personal-read authority. github-publication.test.ts covers advanced-branch recovery and github-publication-boundaries.test.ts covers changed worktree fingerprints. chat-send-admission-authority.test.ts covers read-only membership revocation; chat-send-attachments.owner.test.ts covers global attachment ownership; attempt.cwd-split.test.ts covers read-tool checkout gating.

Preserve distinct WebSocket reconnect/abort ordering, personal recovery effects and confirmation ownership, streamed retirement/cap behavior, global transcript isolation, and real checkout write/send revocation. CI-only: formatting, plain oxlint and diff check passed; no tests or coverage run. Original 4421 lines -> 3597 (18.6%); the requested 35% target remains unmet.

test(ui): deslop r0091 tests

Reuse the existing session-diff fixture and consolidate repeated audio, widget, and transcript test setup without removing behavior cases. Preserve focus, lifecycle, authority, and ordering assertions. CI-only campaign: formatting, plain oxlint, and diff checks passed; runtime and coverage proof deferred to hosted CI.

test(ui): consolidate r0091 cross-layer proof

Replace tool outcome cross-products with decision-branch cases and retain only the Raw-tab update contract locally. Per-file patch rendering and disclosure are covered by chat-tool-turn-outcome.e2e tests; item-only failure/blocked/unknown outcomes by chat-tool-item-outcomes.test-support. Keep skipped-only receipts and zero-exit failure regressions.

Drop redundant transcript geometry and stream-counter layers covered by chat-transcript-geometry, geometry-updates.browser, end-follow.browser, projection-scale and invalidation suites. Keep remaining restore/teardown/observer races and move row identity assertions into retained cases.

Retain the Review preference/native boundary and defer duplicate navigation/rename assertions to session-diff-panel.browser and chat-session-diff.e2e. Consolidate pin focus/toast/membership proof through ToolPreview and preserve long-duration audio formatting in the playback case. No production edits. CI-only: formatting, plain oxlint and diff check pass; no tests run.

test(ui): remove redundant r0091 fixture layers

Replace the local folding/menu/refresh replay with existing real-Gateway proof in chat-session-diff-folding.real-gateway.e2e.test.ts: folds actual sessions.diff source without concealing edits in expanded files. Keep path-key edge coverage with one representative per decision branch.

Delegate rendition retry schedule to chat-media-playback.test.ts: retries at the default delays until the rendition is ready, and stops after the bounded two-minute retry schedule. Consolidate media state setup and retain inline download, duration gating, long-duration formatting, focus and lifecycle assertions.

Remove cached-versus-fresh whole-object oracles and a duplicate direct Tool Search render. Retain explicit stream ownership, marker content, immutable prior state, raw sidebar identity and prepared failure behavior. CI-only: plain oxlint, formatting and diff-check passed; no tests run.

test(sessions): deslop r0093 tests

Share OAuth boundary profile setup, plugin loading, native key resolution, and rotation assertions without changing adapter cases or durable settlement checks.

The retained 'forks a catalog with request-isolated auth and provider mutations' test now also proves the inline models.json credentials previously covered by 'preserves models.json provider auth in a catalog fork'.

CI-only validation: oxfmt, plain oxlint, and git diff --check. No tests or coverage run. Conservative 112-line reduction; distinct security, migration, and lifecycle cases retained.

test(sessions): reduce r0093 duplicate proof and fixtures

Remove bounded-session mutation cases duplicated by same-named tests in session-manager-mutation-boundaries.test.ts, plus its bounded-assistant variant. Keep admitted-assistant persistence assertions in the stronger concurrent-reload case.

Consolidate manager/serialized provider-change fixtures and shrink model and SDK setup without dropping distinct auth, publication, routing or lifecycle behaviors. Drop SQL/internal call-shape assertions while retaining observable state and effect checks.

OAuth discovery recovery remains covered by third-party discovery settlement and xai-oauth.test.ts discovery failure/rediscovery tests; socket fencing remains in the bundled xAI matrix. No production or support-file changes.

CI-only proof: plain oxlint exits 0, oxfmt and diff check pass; no test runs. Cumulative shard LOC 4732 to 4132 (12.7%); remaining distinct boundaries prevent the requested 35% cut without coverage loss.

* test: remove obsolete truncation line-cap allowance
This commit is contained in:
Peter Steinberger authored and GitHub committed 2026-10-24 18:20:11 +00:00
1 parent 2e7668b2ea
commit f2fccfab05
26 files changed
+2401 -6527
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

config/max-lines-baseline.txt
-1
View File
@@ -183,7 +183,6 @@ src/agents/embedded-agent-runner/compact.hooks.test.ts
src/agents/embedded-agent-runner/run.overflow-compaction.harness.ts
src/agents/embedded-agent-runner/run/attempt-spawn-workspace.test-support.ts
src/agents/embedded-agent-runner/runs.ts
src/agents/embedded-agent-runner/tool-result-truncation.test.ts
src/agents/embedded-agent-runner/tool-result-truncation.ts
src/agents/embedded-agent-subscribe.handlers.tools.test.ts
src/agents/harness/native-hook-relay.test.ts
config/test-timeout-race-baseline.txt
+1 -1
View File
@@ -83,7 +83,7 @@ src/infra/push-apns.test.ts 1
src/infra/session-cost-usage-cache.worker.test.ts 5
src/infra/state-migrations.workspace-setup-sandbox.test.ts 1
src/meeting-bot/realtime-node-audio-transport.test.ts 1
src/plugins/openai-compatible-embedding-provider.test.ts 3
src/plugins/openai-compatible-embedding-provider.test.ts 2
src/plugins/plugin-metadata-lifecycle.test.ts 1
src/plugins/provider-self-hosted-discovery.test.ts 11
src/plugins/runtime/runtime-agent.ingress.integration.test.ts 1
src/agents/embedded-agent-runner/compact.hooks.test.ts
+137 -365
View File
@@ -1,6 +1,5 @@
// Hook integration coverage for direct and queued embedded compaction.
import { mkdtemp, realpath } from "node:fs/promises";
import { tmpdir } from "node:os";
import { realpath } from "node:fs/promises";
import { join } from "node:path";
import { expectDefined } from "@openclaw/normalization-core";
import type { AgentMessage, StreamFn } from "openclaw/plugin-sdk/agent-core";
@@ -19,7 +18,6 @@ import type { OpenClawConfig } from "../../config/types.openclaw.js";
import { delegateCompactionToRuntime } from "../../context-engine/delegate.js";
import { racePromiseWithAbortSignal } from "../../infra/abort-signal.js";
import { createPluginMetadataSnapshotFixture } from "../../plugins/plugin-metadata.test-support.js";
import { getModelProviderRuntimePluginHandle } from "../../plugins/provider-hook-runtime.js";
import type { CommandQueueEnqueueOptions } from "../../process/command-queue.types.js";
import type { SessionTurnAdmission } from "../../sessions/session-controller.admission.js";
import type { ReplyOperation } from "../../sessions/session-controller.contracts.js";
@@ -28,7 +26,6 @@ import { waitForSessionRunEnd } from "../../sessions/session-controller.native-r
import { isSessionRunActive } from "../../sessions/session-controller.queries.js";
import { createProcessSessionFixture } from "../bash-process-registry.test-helpers.js";
import { recordModelFallbackStop } from "../model-fallback-stop.js";
import { getModelProviderLocalServiceReconciler } from "../provider-local-service-reconcile.js";
import { createSessionMaintenanceOwner } from "../session-maintenance/coordinator.js";
import { agentSessionAutomaticCompaction } from "../sessions/agent-session-compaction.js";
import {
@@ -66,15 +63,12 @@ import {
getApiKeyForModelMock,
getMemorySearchManagerMock,
resolveMemorySearchConfigMock,
resolveSessionAgentIdMock,
getHistoryLimitFromSessionKeyMock,
hookRunner,
limitHistoryTurnsMock,
listRegisteredPluginAgentPromptGuidanceMock,
loadCompactHooksHarness,
maybeCompactAgentHarnessSessionMock,
resolveAgentHarnessPolicyMock,
registerProviderStreamForModelMock,
resolveProviderEntryApiKeyProfileReferenceMock,
resolveContextWindowInfoMock,
resolveCliBackendConfigMock,
@@ -125,17 +119,13 @@ const compactionFixture = useCompactHooksSessionFixture(TEST_SESSION_KEY);
let TEST_SESSION_FILE: string;
let TEST_WORKSPACE_DIR: string;
const TEST_CUSTOM_INSTRUCTIONS = "focus on decisions";
function plannedCompactionPluginSelections(
config: OpenClawConfig,
metadataSnapshot = createPluginMetadataSnapshotFixture({ plugins: [] }),
) {
function plannedCompactionPluginSelections(config: OpenClawConfig) {
const derive = expectDefined(
acquireAgentRunPreparedModelRuntimeMock.mock.calls[0]?.[1]?.deriveRuntimePluginSelections,
"admitted compaction selection recipe",
);
return derive({ config, metadataSnapshot });
return derive({ config, metadataSnapshot: createPluginMetadataSnapshotFixture({ plugins: [] }) });
}
function mockResolvedModel(supportsTools?: boolean, queued = false) {
resolveModelMock.mockReset();
resolveModelMock.mockImplementation(
@@ -181,17 +171,21 @@ function directCompactionArgs() {
};
}
function defaultSessionTarget() {
return {
agentId: "main",
sessionId: TEST_SESSION_ID,
sessionKey: TEST_SESSION_KEY,
storePath: TEST_STORE_PATH,
};
}
function wrappedCompactionArgs(overrides: Record<string, unknown> = {}) {
return {
sessionId: TEST_SESSION_ID,
sessionKey: TEST_SESSION_KEY,
sessionFile: TEST_SESSION_KEY,
sessionTarget: {
agentId: "main",
sessionId: TEST_SESSION_ID,
sessionKey: TEST_SESSION_KEY,
storePath: TEST_STORE_PATH,
},
sessionTarget: defaultSessionTarget(),
workspaceDir: TEST_WORKSPACE_DIR,
customInstructions: TEST_CUSTOM_INSTRUCTIONS,
...overrides,
@@ -222,6 +216,34 @@ function expectNoCalls(...mocks: Mock[]) {
}
const abortedCompaction = { ok: false, compacted: false, reason: "compaction aborted" };
const compacted = { ok: true, compacted: true };
const notCompacted = { ok: false, compacted: false };
const unchangedMaintenance = { changed: false, bytesFreed: 0, rewrittenEntries: 0 };
function lockedFailure(reason = "model_selection_locked") {
return { ...notCompacted, failure: { reason } };
}
function assertActiveWhile(active: () => boolean) {
return () => {
if (!active()) {
throw new Error("queued compaction host authority expired");
}
};
}
function subscriptionModel(provider = "openai", modelId = "fake") {
return {
...createCompactHooksResolvedModel(provider, modelId),
model: {
provider: "openai",
id: "fake",
api: "openai-chatgpt-responses",
baseUrl: "https://chatgpt.com/backend-api/codex",
input: [],
},
};
}
function useCodexPolicy() {
const policy = { runtime: "codex", runtimeSource: "model" };
@@ -419,9 +441,8 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
customInstructions: "preserve the committed state",
});
expect(contextEngineCompactMock).not.toHaveBeenCalled();
expect(createAgentSessionMock).toHaveBeenCalledOnce();
expect(sessionCompactImpl).toHaveBeenCalledOnce();
expect(result).toMatchObject({ ok: false, compacted: false });
expect(result).toMatchObject(notCompacted);
});
it("restricts endpoint tools and excludes private skills", async () => {
@@ -516,7 +537,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
},
},
});
expect(attemptServerEndpointCompactionMock).toHaveBeenCalledOnce();
expectCall(attemptServerEndpointCompactionMock, {
context: expect.objectContaining({
messages: [
@@ -568,8 +588,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
const result = await compactDirect({ compactionTimeoutReset });
expect(result).toMatchObject({ ok: true, compacted: true });
expect(hookRunner.runAfterCompaction).toHaveBeenCalledOnce();
expect(result).toMatchObject(compacted);
});
it("refreshes the delegated watchdog before delayed fallback setup", async () => {
@@ -599,8 +618,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
await pending;
}
await expect(pending).resolves.toMatchObject({ ok: true, compacted: true });
expect(createAgentSessionMock).toHaveBeenCalledTimes(2);
await expect(pending).resolves.toMatchObject(compacted);
expect(compactionTimeoutReset).toHaveBeenCalledTimes(6);
});
@@ -649,11 +667,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
sessionEntry: { sessionId: TEST_SESSION_ID, updatedAt: 1, pluginOwnerId: "stale-owner" },
});
expect(result).toMatchObject({
ok: false,
compacted: false,
failure: { reason: "model_selection_locked" },
});
expect(result).toMatchObject(lockedFailure());
expectNoCalls(resolveModelMock, sessionCompactImpl);
});
@@ -745,12 +759,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
([params]) => (params as { profileId?: string }).profileId,
),
).toEqual(["openai:missing", "openai:backup"]);
expect(resolveModelAsyncMock.mock.calls.map((call) => call[4])).toEqual(
expect.arrayContaining([
expect.objectContaining({ authProfileId: "openai:backup" }),
expect.objectContaining({ authProfileId: "openai:missing" }),
]),
);
expectCall(resolveEmbeddedAgentStreamMock, { authProfileId: "openai:backup" });
expectCall(buildAgentRuntimePlanMock, { sessionAuthProfileId: "openai:backup" });
});
@@ -788,10 +796,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
},
});
expectCall(resolveSandboxContextMock, {
sessionKey: other.scopeKey,
workspaceDir: join(TEST_WORKSPACE_DIR, "workspace"),
});
expect(sandboxQueries).toEqual([]);
expectCall(buildConfiguredAgentSystemPromptMock, {
runtimeInfo: expect.objectContaining({
@@ -804,27 +808,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
}
});
it.each([
["agent:main:subagent:worker", "subagent", "minimal", "Subagent compact command guidance."],
["agent:codex:acp:worker", "acp_backend", "full", "ACP compact command guidance."],
] as const)(
"rebuilds the %s prompt with its guidance",
async (sessionKey, promptSurface, promptMode, guidance) => {
const runtimeCwd = join(TEST_WORKSPACE_DIR, "task-repo");
await compactWorkspace({ sessionKey, cwd: runtimeCwd });
expect(listRegisteredPluginAgentPromptGuidanceMock).toHaveBeenCalledWith({
surface: promptSurface,
});
expectCall(buildConfiguredAgentSystemPromptMock, {
promptMode,
promptSurface,
runtimeCwd,
workspaceDir: join(TEST_WORKSPACE_DIR, "workspace"),
nativeCommandGuidanceLines: [guidance],
});
},
);
it.each([false, true])(
"renders the compaction bootstrap notice only for omitted files (%s)",
async (omitted) => {
@@ -856,7 +839,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
const prompt = created.session.agent.state.systemPrompt ?? "";
expect(prompt.includes("## Bootstrap Context Notice")).toBe(omitted);
expect(prompt.includes("Treat Project Context as partial")).toBe(omitted);
expect(created.session.setBaseSystemPrompt).toHaveBeenCalledWith(prompt);
},
);
@@ -935,11 +917,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
const result = await compactWorkspace({ config: { tools: { profile: "coding" } } });
expect(result.ok).toBe(true);
const toolOptions = mockCallArg(createOpenClawCodingToolsMock) as {
preparedModelRuntime: { metadataSnapshot: unknown };
};
expect(toolOptions.preparedModelRuntime.metadataSnapshot).toBe(metadataSnapshot);
expect(toolOptions).toMatchObject({
expect(mockCallArg(createOpenClawCodingToolsMock)).toMatchObject({
conversationCapabilityProfile: {
policy: { explicitToolAllowlist: expect.arrayContaining([toolName]) },
},
@@ -1192,8 +1170,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
const run = await compactWhileStreaming(10, "done");
await vi.advanceTimersByTimeAsync(11 * deltaEveryMs);
await expect(run.pending).resolves.toMatchObject({ ok: true, compacted: true });
expect(run.stream).toHaveBeenCalledOnce();
await expect(run.pending).resolves.toMatchObject(compacted);
});
it.each([
@@ -1216,7 +1193,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
compacted: false,
reason: expect.stringContaining("timed out"),
});
expect(run.stream).toHaveBeenCalledOnce();
});
// #164246: automatic summary timeouts still commit deterministic reduction.
@@ -1228,11 +1204,10 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
});
await vi.advanceTimersByTimeAsync(summaryCutoffMs - prepMs);
await expect(run.pending).resolves.toMatchObject({ ok: true, compacted: true });
await expect(run.pending).resolves.toMatchObject(compacted);
expect(
run.sessionManager.getBranch().findLast((entry) => entry.type === "compaction"),
).toMatchObject({ summary: expect.stringContaining("removed without a summary") });
expect(run.stream).toHaveBeenCalledOnce();
});
});
@@ -1248,23 +1223,16 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
config,
});
expect(result.ok, JSON.stringify(result)).toBe(true);
const expectedSelection = { provider: "anthropic", runtime: "codex" };
expect(plannedCompactionPluginSelections(config)).toContainEqual(
expect.objectContaining({
provider: "anthropic",
modelId: "claude-fallback",
runtime: "codex",
}),
expect.objectContaining({ ...expectedSelection, modelId: "claude-fallback" }),
);
const admittedConfig = defaultsConfig({
...config.agents?.defaults,
compaction: { model: "anthropic/reloaded-summary" },
});
expect(plannedCompactionPluginSelections(admittedConfig)).toContainEqual(
expect.objectContaining({
provider: "anthropic",
modelId: "reloaded-summary",
runtime: "codex",
}),
expect.objectContaining({ ...expectedSelection, modelId: "reloaded-summary" }),
);
});
@@ -1333,14 +1301,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
});
expect(result).toMatchObject({ ok: true, result: { summary: "oauth fallback summary" } });
findMockCall(
resolveModelMock,
([provider, modelId]) => provider === "openai" && modelId === "gpt-5.5",
);
findMockCall(
resolveModelMock,
([provider, modelId]) => provider === "openai" && modelId === "gpt-5.4-mini",
);
expectRecordFields(mockCallArg(resolveEmbeddedAgentStreamMock, 1), {
authProfileId: "openai:default",
});
@@ -1393,15 +1353,9 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
});
it("skips sync in await mode when postCompactionForce is false", async () => {
const sync = vi.fn(async () => {});
getMemorySearchManagerMock.mockResolvedValue({ manager: { sync } });
resolveMemorySearchConfigMock.mockReturnValue({
sources: ["sessions"],
sync: {
sessions: {
postCompactionForce: false,
},
},
sync: { sessions: { postCompactionForce: false } },
});
await compactTesting.runPostCompactionSideEffects({
@@ -1410,14 +1364,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
sessionFile: TEST_SESSION_FILE,
});
const resolveAgentArg = mockCallArg(resolveSessionAgentIdMock);
if (typeof resolveAgentArg !== "object" || resolveAgentArg === null) {
throw new Error("Expected session agent arguments");
}
expect(resolveAgentArg).toMatchObject({ sessionKey: TEST_SESSION_KEY });
expect(Reflect.get(resolveAgentArg, "config")).toBeTypeOf("object");
expect(getMemorySearchManagerMock).not.toHaveBeenCalled();
expect(sync).not.toHaveBeenCalled();
});
registerDirectProviderRefreshTests({
@@ -1426,27 +1373,6 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
sessionKey: TEST_SESSION_KEY,
sessionFile: () => TEST_SESSION_FILE,
});
it("carries the prepared provider reconciler into direct compaction", async () => {
const reconcile = vi.fn(async () => undefined);
const { resolvePreparedProviderRuntimeHandle } = await import("../runtime-plan/build.js");
vi.mocked(resolvePreparedProviderRuntimeHandle).mockImplementationOnce((params) => ({
provider: params.provider,
modelId: params.modelId,
workspaceDir: params.workspaceDir,
prepared: true,
plugin: { id: params.provider, label: "Fixture", auth: [], reconcileLocalService: reconcile },
}));
await expect(compactDirect()).resolves.toMatchObject({ ok: true });
const streamRegistration = mockCallArg(registerProviderStreamForModelMock) as {
model: object;
};
expect(getModelProviderLocalServiceReconciler(streamRegistration.model)).toBe(reconcile);
expect(getModelProviderRuntimePluginHandle(streamRegistration.model)).toBe(
buildAgentRuntimePlanMock.mock.calls[0]?.[0].providerRuntimeHandle,
);
});
it("compacts an overflow transcript anchored by a compaction summary", async () => {
sessionMessages.splice(
0,
@@ -1466,8 +1392,7 @@ describe("compactEmbeddedAgentSessionDirect hooks", () => {
const result = await compactDirect({ trigger: "overflow" });
expect(result).toMatchObject({ ok: true, compacted: true });
expect(sessionCompactImpl).toHaveBeenCalledOnce();
expect(result).toMatchObject(compacted);
});
it("never falls back from an explicit compaction.model", async () => {
@@ -1716,29 +1641,11 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
it("cancels only the target during runtime admission", async () => {
const sourceController = new AbortController();
const admissionStarted = createDeferred<AbortSignal | undefined>();
acquireAgentRunPreparedModelRuntimeMock.mockImplementationOnce((async (
_input: Record<string, unknown>,
options?: { abortSignal?: AbortSignal },
): Promise<never> => {
const signal = options?.abortSignal;
acquireAgentRunPreparedModelRuntimeMock.mockImplementationOnce(async (_input, options) => {
const signal = expectDefined(options?.abortSignal, "runtime admission abort signal");
admissionStarted.resolve(signal);
if (!signal) {
throw new Error("prepared runtime lease admission did not receive the caller signal");
}
return await new Promise<never>((_resolve, reject) => {
signal.addEventListener(
"abort",
() => {
const error = new Error("Prepared model runtime lease admission aborted", {
cause: signal.reason,
});
error.name = "AbortError";
reject(error);
},
{ once: true },
);
});
}) as never);
return await racePromiseWithAbortSignal(createDeferred<never>().promise, signal);
});
const pending = compactQueued({ abortSignal: sourceController.signal, trigger: "manual" });
const admittedSignal = expectDefined(
@@ -1754,57 +1661,18 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
expect(resolveContextEngineMock).not.toHaveBeenCalled();
});
it("releases the prepared runtime lease when host authority expires after admission", async () => {
const { AsyncWorkScope } = await import("../../shared/async-work-scope.js");
const parent = new AsyncWorkScope();
const admissionStarted = createDeferred();
const releaseAdmission = createDeferred();
const releaseLease = vi.fn(async () => {});
const defaultAcquire = expectDefined(
acquireAgentRunPreparedModelRuntimeMock.getMockImplementation(),
"default prepared runtime acquisition mock",
);
let hostActive = true;
acquireAgentRunPreparedModelRuntimeMock.mockImplementationOnce((async (
input: Parameters<typeof acquireAgentRunPreparedModelRuntimeMock>[0],
) => {
admissionStarted.resolve(undefined);
await releaseAdmission.promise;
const lease = await defaultAcquire(input);
return { ...lease, [Symbol.asyncDispose]: releaseLease };
}) as never);
const pending = parent.run(() =>
compactQueued(
{},
{
assertActive: () => {
if (!hostActive) {
throw new Error("queued compaction host authority expired");
}
},
},
),
);
await admissionStarted.promise;
hostActive = false;
releaseAdmission.resolve(undefined);
try {
await expect(pending).rejects.toThrow("queued compaction host authority expired");
} finally {
await AsyncWorkScope.runWhenAllIdle(
() => [parent],
() => parent.drain(),
);
}
expect(releaseLease).toHaveBeenCalledTimes(1);
expect(resolveContextEngineMock).not.toHaveBeenCalled();
});
it.each(["context-engine", "model"] as const)(
"stops preparation when host authority expires during %s resolution",
async (stage) => {
it.each([
["admission", false],
["context-engine", false],
["model", false],
["context-engine", true],
] as const)(
"stops preparation at %s after authority expires (budget cancellation: %s)",
async (stage, cancel) => {
const { AsyncWorkScope } = await import("../../shared/async-work-scope.js");
const parent = new AsyncWorkScope();
const controller = new AbortController();
const releaseLease = vi.fn(async () => {});
const resolutionStarted = createDeferred();
const releaseResolution = createDeferred();
let hostActive = true;
@@ -1812,98 +1680,79 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
resolutionStarted.resolve(undefined);
await releaseResolution.promise;
};
if (stage === "model") {
if (stage === "admission") {
const acquire = expectDefined(
acquireAgentRunPreparedModelRuntimeMock.getMockImplementation(),
"prepared runtime acquisition",
);
acquireAgentRunPreparedModelRuntimeMock.mockImplementationOnce(async (input) => {
await waitForResolution();
return { ...(await acquire(input)), [Symbol.asyncDispose]: releaseLease };
});
} else if (stage === "model") {
resolveModelAsyncMock.mockImplementationOnce(async (provider, modelId) => {
await waitForResolution();
return {
...createCompactHooksResolvedModel(provider, modelId),
model: {
provider: "openai",
id: "fake",
api: "openai-chatgpt-responses",
baseUrl: "https://chatgpt.com/backend-api/codex",
input: [],
},
};
return subscriptionModel(provider, modelId);
});
} else {
resolveContextEngineMock.mockImplementationOnce(async () => {
await waitForResolution();
return {
info: { ownsCompaction: true },
compact: contextEngineCompactMock,
};
return { info: { ownsCompaction: true }, compact: contextEngineCompactMock };
});
}
const pending = compactQueued(
stage === "model"
? {
provider: "openai",
model: "fake",
runtimeAuthPlan: {
providerForAuth: "openai",
authProfileProviderForAuth: "openai",
selectedAuthMode: "api-key",
modelRoute: openaiRoute(undefined, "fake"),
},
}
: {},
{
assertActive: () => {
if (!hostActive) {
throw new Error("queued compaction host authority expired");
}
},
},
const pending = parent.run(() =>
compactQueued(
stage === "model"
? {
provider: "openai",
model: "fake",
runtimeAuthPlan: {
providerForAuth: "openai",
authProfileProviderForAuth: "openai",
selectedAuthMode: "api-key",
modelRoute: openaiRoute(undefined, "fake"),
},
}
: cancel
? { abortSignal: controller.signal, trigger: "budget" }
: {},
{ assertActive: assertActiveWhile(() => hostActive) },
),
);
await resolutionStarted.promise;
hostActive = false;
if (cancel) {
controller.abort(new Error("request timed out"));
} else {
hostActive = false;
}
releaseResolution.resolve(undefined);
await expect(pending).rejects.toThrow("queued compaction host authority expired");
try {
if (cancel) {
await expect(pending).resolves.toEqual(abortedCompaction);
} else {
await expect(pending).rejects.toThrow("queued compaction host authority expired");
}
} finally {
await AsyncWorkScope.runWhenAllIdle(
() => [parent],
() => parent.drain(),
);
}
expect(resolveModelAsyncMock).toHaveBeenCalledTimes(stage === "model" ? 1 : 0);
expectNoCalls(selectAgentHarnessForPreparedModelProvidersMock, contextEngineCompactMock);
if (stage === "model") {
if (stage === "admission") {
expect(releaseLease).toHaveBeenCalledOnce();
expect(resolveContextEngineMock).not.toHaveBeenCalled();
}
if (stage === "model" || cancel) {
expectNoCalls(maybeCompactAgentHarnessSessionMock, enqueueCommandInLaneMock);
}
},
);
it("reports budget compaction cancellation during context-engine resolution", async () => {
const sourceController = new AbortController();
const resolutionStarted = createDeferred();
const releaseResolution = createDeferred();
resolveContextEngineMock.mockImplementationOnce(async () => {
resolutionStarted.resolve(undefined);
await releaseResolution.promise;
return {
info: { ownsCompaction: true },
compact: contextEngineCompactMock,
};
});
const pending = compactQueued({
abortSignal: sourceController.signal,
trigger: "budget",
});
await resolutionStarted.promise;
sourceController.abort(new Error("request timed out"));
releaseResolution.resolve(undefined);
await expect(pending).resolves.toEqual({
ok: false,
compacted: false,
reason: "compaction aborted",
});
expect(resolveModelAsyncMock).not.toHaveBeenCalled();
expect(selectAgentHarnessForPreparedModelProvidersMock).not.toHaveBeenCalled();
expect(contextEngineCompactMock).not.toHaveBeenCalled();
expect(enqueueCommandInLaneMock).not.toHaveBeenCalled();
});
it("resolves the durable session key before invoking an owning context engine", async () => {
const dir = await realpath(await mkdtemp(join(tmpdir(), "openclaw-compaction-session-key-")));
const dir = await realpath(compactionFixture.makeTempDir("openclaw-compaction-session-key-"));
const storePath = join(dir, "sessions.json");
const sessionId = "9d6c8436-7cb2-4bd5-a302-e33305bfc8c4";
const sessionKey = "agent:main:telegram:direct:reporter";
@@ -1966,42 +1815,20 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
},
sessionKey: "agent:marie-clawndo:dashboard:session-1",
sessionTarget: {
...defaultSessionTarget(),
agentId: "marie-clawndo",
sessionId: TEST_SESSION_ID,
sessionKey: "agent:marie-clawndo:dashboard:session-1",
storePath: TEST_STORE_PATH,
},
});
expect(result.ok).toBe(true);
expect(acquireAgentRunPreparedModelRuntimeMock).toHaveBeenCalledWith(
expect.objectContaining({ agentId: "marie-clawndo" }),
expect.objectContaining({ abortSignal: expect.any(AbortSignal) }),
);
});
it("disposes the context engine once when route materialization rejects", async () => {
const dispose = vi.fn(async () => {});
const { authStorage } = createCompactHooksResolvedModel();
resolveContextEngineMock.mockResolvedValue({
info: { ownsCompaction: true },
compact: contextEngineCompactMock,
dispose,
} as never);
mockContextEngine(true, false, { dispose });
resolveModelAsyncMock
.mockImplementationOnce(async (provider, modelId) => ({
logicalRef: { provider, model: modelId },
model: {
provider: "openai",
id: "fake",
api: "openai-chatgpt-responses",
baseUrl: "https://chatgpt.com/backend-api/codex",
input: [],
},
error: null,
authStorage,
modelRegistry: {},
}))
.mockImplementationOnce(async (provider, modelId) => subscriptionModel(provider, modelId))
.mockRejectedValueOnce(new Error("route materialization failed"));
await expect(
@@ -2012,14 +1839,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
providerForAuth: "openai",
authProfileProviderForAuth: "openai",
selectedAuthMode: "api-key",
modelRoute: {
provider: "openai",
modelId: "fake",
api: "openai-responses",
baseUrl: "https://api.openai.com/v1",
authRequirement: "api-key",
requestTransportOverrides: "none",
},
modelRoute: openaiRoute(undefined, "fake"),
},
}),
).rejects.toThrow("route materialization failed");
@@ -2250,25 +2070,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
);
expect(result.ok).toBe(true);
if (subscription) {
expect(mockCallArg(resolveModelAsyncMock, 0, 4)).toEqual(
expect.objectContaining({ authProfileId: "openai:token" }),
);
expect(resolveModelAsyncMock).toHaveBeenLastCalledWith(
"openai",
"gpt-5.5",
expect.any(String),
expect.objectContaining(
providerConfig("openai", expect.objectContaining({ api, baseUrl })),
),
expect.objectContaining({ authProfileMode: "token" }),
);
expectCall(contextEngineCompactMock, { tokenBudget: 272_000 });
} else {
expect(mockCallArg(resolveModelMock)).toBe("openai");
expectRecordFields(mockCallArg(resolveContextWindowInfoMock), {
provider: "openai",
modelId: "gpt-5.5",
});
}
expectCall(
maybeCompactAgentHarnessSessionMock,
@@ -2292,24 +2094,12 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
expect(contextEngineCompactMock.mock.invocationCallOrder[0]).toBeLessThan(
maybeCompactAgentHarnessSessionMock.mock.invocationCallOrder[0]!,
);
const compactArg = mockCallArg(contextEngineCompactMock) as {
runtimeContext?: Record<string, unknown>;
};
expectRecordFields(compactArg.runtimeContext, {
provider: "openai",
runtimeProvider: undefined,
model: "gpt-5.5",
});
},
);
it("reports deferred compaction scheduling failures", async () => {
const dispose = vi.fn(async () => {});
const maintain = vi.fn(async () => ({
changed: false,
bytesFreed: 0,
rewrittenEntries: 0,
}));
const maintain = vi.fn(async () => unchangedMaintenance);
resolveContextEngineMock.mockResolvedValue({
info: { ownsCompaction: true, turnMaintenanceMode: "background" },
compact: contextEngineCompactMock,
@@ -2391,15 +2181,9 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
failure: { reason: failureReason },
},
);
expect(maybeCompactAgentHarnessSessionMock).toHaveBeenCalledTimes(1);
expectCall(
maybeCompactAgentHarnessSessionMock,
{
provider: "openai",
model: "gpt-5.5",
agentHarnessId,
preflightRequired: true,
},
{ preflightRequired: true },
expect.objectContaining({ nativeCompactionRequest: "required_preflight" }),
);
expect(contextEngineCompactMock).toHaveBeenCalledTimes(nativeCapabilityUsed ? 1 : 0);
@@ -2421,11 +2205,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
agentHarnessId,
currentTokenCount: 333,
});
expect(result).toMatchObject({
ok: false,
compacted: false,
failure: { reason: "model_selection_locked" },
});
expect(result).toMatchObject(lockedFailure());
expect(maybeCompactAgentHarnessSessionMock).toHaveBeenCalledTimes(nativeCalls);
expect(contextEngineCompactMock).not.toHaveBeenCalled();
},
@@ -2433,7 +2213,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
it("runs native manual compaction before generic auth", async () => {
const result = await compactNative({ ...nativeCliArgs(), agentHarnessId: "claude-cli" });
expect(result).toMatchObject({ ok: true, compacted: true });
expect(result).toMatchObject(compacted);
expectCall(runCliAgentMock, { cliSessionId: "native-session", controlOperation: "compact" });
expectNoCalls(acquireAgentRunPreparedModelRuntimeMock, contextEngineCompactMock);
});
@@ -2471,7 +2251,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
expect(settled).toBe(false);
nativeTerminal.resolve(nativeResult);
await expect(resultPromise).resolves.toMatchObject({ ok: true, compacted: true });
await expect(resultPromise).resolves.toMatchObject(compacted);
} finally {
nativeTerminal.resolve(nativeResult);
await resultPromise.catch(() => undefined);
@@ -2586,7 +2366,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
expect(pending.signal?.aborted).toBe(true);
expect(pending.signal?.reason).toBe(replyOperation.abortSignal.reason);
pending.release.resolve(undefined);
await expect(resultPromise).resolves.toMatchObject({ ok: false, compacted: false });
await expect(resultPromise).resolves.toMatchObject(notCompacted);
expect(isEmbeddedAgentRunHandleActive(TEST_SESSION_ID)).toBe(false);
expect(isSessionRunActive(TEST_SESSION_ID)).toBe(true);
} finally {
@@ -2642,11 +2422,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
await entered.promise;
try {
const result = await compactQueued({ trigger: "manual" });
expect(result).toMatchObject(
active.foreignStore
? { ok: true, compacted: true }
: { ok: false, compacted: false, failure: { reason: "active_run" } },
);
expect(result).toMatchObject(active.foreignStore ? compacted : lockedFailure("active_run"));
expect(contextEngineCompactMock).toHaveBeenCalledTimes(active.foreignStore ? 1 : 0);
expect(maybeCompactAgentHarnessSessionMock).not.toHaveBeenCalled();
expect(existingHandle.abort).not.toHaveBeenCalled();
@@ -2662,7 +2438,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
async (format) => {
const dir =
format === "session-key"
? await realpath(await mkdtemp(join(tmpdir(), "openclaw-compaction-successor-mismatch-")))
? await realpath(compactionFixture.makeTempDir("openclaw-compaction-successor-mismatch-"))
: undefined;
const storePath = dir ? join(dir, "sessions.json") : TEST_STORE_PATH;
const activeTarget = { ...wrappedCompactionArgs().sessionTarget, storePath };
@@ -2708,7 +2484,7 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
const result = await compactQueued();
expect(result).toMatchObject({ ok: true, compacted: true });
expect(result).toMatchObject(compacted);
expect(contextEngineCompactMock).toHaveBeenCalledTimes(1);
});
it("prepares native harness auth without a host profile", async () => {
@@ -2731,18 +2507,14 @@ describe("compactEmbeddedAgentSession hooks (ownsCompaction engine)", () => {
expectCall(
maybeCompactAgentHarnessSessionMock,
{ runtimeAuthPlan: undefined },
expectedNativeCompactionOptions("after_context_engine"),
expect.objectContaining({ nativeCompactionRequest: "after_context_engine" }),
);
});
it.each(["SQLite marker", "partial structured target"] as const)(
"keeps a %s successor in the active store for legacy maintenance",
async (format) => {
const maintain = vi.fn(async (_params?: unknown) => ({
changed: false,
bytesFreed: 0,
rewrittenEntries: 0,
}));
const maintain = vi.fn(async (_params?: unknown) => unchangedMaintenance);
const delegatedSessionId =
format === "SQLite marker" ? "delegated-marker-session" : "delegated-session";
const storePath =
src/agents/embedded-agent-runner/thinking.test.ts
+329 -925
View File
File diff suppressed because it is too large. Load diff
src/agents/embedded-agent-runner/tool-result-truncation.test.ts
+275 -520
View File
File diff suppressed because it is too large. Load diff
src/agents/sessions/auth-storage.test.ts
+157 -248
View File
@@ -67,14 +67,7 @@ const { ensureAuthProfileStoreWithoutExternalProfiles, saveAuthProfileStore } =
authProfileStoreRuntime;
function providerBCredential(overrides: Partial<OAuthCredential> = {}) {
return createCredential({
provider: "provider-b",
access: "provider-b-access",
refresh: "provider-b-refresh",
expires: Date.now() + 600_000,
accountId: "acct-b",
...overrides,
});
return createCredential({ provider: "provider-b", accountId: "acct-b", ...overrides });
}
function saveCredential(agentDir: string, profileId: string, credential: OAuthCredential) {
@@ -83,6 +76,37 @@ function saveCredential(agentDir: string, profileId: string, credential: OAuthCr
});
}
function registerRefresh(storage: AuthStorage, providerId: string, credential: OAuthCredential) {
const refresh = vi.fn(async () => credential);
getAuthStorageOAuthProviderRegistry(storage).register(oauthProvider(providerId, refresh));
return refresh;
}
function expectPublished(
storage: AuthStorage,
agentDir: string,
providerId: string,
credential: OAuthCredential | undefined,
) {
expect(storage.get(providerId)).toEqual(credential);
expect(readProfile(agentDir, `${providerId}:default`)).toEqual(credential);
}
function changeCredential(
storage: AuthStorage,
providerId: string,
change: string,
replacement: OAuthCredential,
) {
if (change === "logout") {
storage.logout(providerId);
} else if (change === "replace") {
storage.set(providerId, replacement);
} else if (change === "unrelated") {
storage.set("other", { type: "api_key", key: "synthetic-other-new" });
}
}
afterEach(() => {
vi.restoreAllMocks();
clearAuthProfileMigrationDiagnostics();
@@ -94,7 +118,6 @@ describe("AuthStorage OAuth refresh ownership", () => {
it("refreshes local OAuth credentials while the shared owner requires migration", () =>
withAgent(async (agentDir, state) => {
const providerId = "test-oauth";
const credential = createCredential({ provider: providerId, expires: 1 });
const sharedStore = { version: 1, profiles: {} };
await state.writeJson("agents/main/agent/auth-profiles.json", {
version: 1,
@@ -103,25 +126,18 @@ describe("AuthStorage OAuth refresh ownership", () => {
},
});
writePersistedAuthProfileStoreRaw(sharedStore);
writePersistedAuthProfileStoreRaw(profileStore(credential), agentDir);
const storage = AuthStorage.forAgent(agentDir, {});
const refreshed = createCredential({
provider: providerId,
access: "synthetic-refreshed-access",
refresh: "synthetic-refreshed-refresh",
});
const refreshToken = vi.fn(async () => refreshed);
getAuthStorageOAuthProviderRegistry(storage).register(
oauthProvider(providerId, refreshToken),
writePersistedAuthProfileStoreRaw(
profileStore(createCredential({ provider: providerId, expires: 1 })),
agentDir,
);
const storage = AuthStorage.forAgent(agentDir, {});
const refreshed = createCredential({ provider: providerId });
registerRefresh(storage, providerId, refreshed);
const fallback = vi.fn(() => "synthetic-fallback-key");
storage.setFallbackResolver(fallback);
await expect(storage.getApiKey(providerId)).resolves.toBe(refreshed.access);
expect(refreshToken).toHaveBeenCalledOnce();
expect(loadPersistedAuthProfileStore(agentDir)?.profiles[`${providerId}:default`]).toEqual(
refreshed,
);
expectPublished(storage, agentDir, providerId, refreshed);
await expect(storage.getApiKey("anthropic")).rejects.toMatchObject({
code: "AUTH_PROFILE_MIGRATION_REQUIRED",
affectedProviders: ["anthropic"],
@@ -139,14 +155,15 @@ describe("AuthStorage OAuth refresh ownership", () => {
"blocks credential writes and OAuth refresh when the %s destination requires migration",
(agentId) =>
withAgent(async (agentDir, state) => {
const originalStore = profileStore(
createCredential({ provider: "test-oauth", expires: 1 }),
writePersistedAuthProfileStoreRaw(
profileStore(createCredential({ provider: "test-oauth", expires: 1 })),
agentDir,
);
writePersistedAuthProfileStoreRaw(originalStore, agentDir);
const storage = AuthStorage.forAgent(agentDir, {});
const refreshToken = vi.fn(async () => createCredential({ provider: "test-oauth" }));
getAuthStorageOAuthProviderRegistry(storage).register(
oauthProvider("test-oauth", refreshToken),
const refreshToken = registerRefresh(
storage,
"test-oauth",
createCredential({ provider: "test-oauth" }),
);
// A populated SQLite owner treats legacy files as leftover bytes. Once
// emptied, its matching legacy credential must block further credential use.
@@ -159,10 +176,6 @@ describe("AuthStorage OAuth refresh ownership", () => {
: profileStore(createCredential({ provider: "test-oauth" })),
);
await refreshAuthProfileFileState();
expect(() => assertAuthProfileMigrationReady(agentDir)).toThrow(
"requires legacy credential migration",
);
const write = vi.spyOn(authProfileStoreRuntime, "saveAuthProfileStoreWithPreparedOwner");
expect(() =>
storage.set("litellm", { type: "api_key", key: "synthetic-local-key" }),
).toThrow("requires legacy credential migration");
@@ -170,196 +183,121 @@ describe("AuthStorage OAuth refresh ownership", () => {
code: "AUTH_PROFILE_MIGRATION_REQUIRED",
});
expect(refreshToken).not.toHaveBeenCalled();
expect(write).not.toHaveBeenCalled();
expect(readPersistedAuthProfileStoreRaw(agentDir)).toEqual(migrationStore);
}, agentId),
);
it.each(["pending", "compare-and-swap", "observer"])(
"rejects a provider change during OAuth manager %s",
async (phase) => {
await withAgent(async (agentDir) => {
const profileId = "openai:oauth";
describe.each(["manager", "serialized"])("%s provider ownership", (backendKind) => {
it.each(["pending", "compare-and-swap", "observer"])("rejects changes at %s", (phase) =>
withAgent(async (agentDir) => {
const profileId = "openai:default";
const attempted = createCredential({ expires: 1, accountId: "acct-a" });
const providerB = providerBCredential(phase === "pending" ? { expires: 1 } : {});
const foreignFence = createOAuthRefreshFence({ profileId, credential: providerB });
saveCredential(
agentDir,
profileId,
const foreign = providerBCredential(phase === "pending" ? { expires: 1 } : {});
const foreignFence = createOAuthRefreshFence({ profileId, credential: foreign });
const initial =
phase === "compare-and-swap"
? attempted
: phase === "pending"
? foreignFence
: createOAuthRefreshFence({ profileId, credential: attempted }),
);
const buildApiKey = vi.fn(
async (_provider, credential: OAuthCredential) => credential.access,
);
const refreshCredential = vi.fn(async () => (phase === "observer" ? null : providerB));
mockOAuthProviderOperations({
buildApiKey,
refreshCredential,
canRefreshCredential: async () => {
if (phase === "compare-and-swap") {
saveCredential(agentDir, profileId, providerB);
}
return true;
},
});
const manager = createOAuthManager();
if (phase === "pending") {
const load = authProfileStoreRuntime.loadAuthProfileStoreWithoutExternalProfiles;
let reads = 0;
vi.spyOn(
authProfileStoreRuntime,
"loadAuthProfileStoreWithoutExternalProfiles",
).mockImplementation((...args) => {
const store = load(...args);
const credential = store.profiles[profileId];
if (
credential?.type === "oauth" &&
credential.provider === "provider-b" &&
isPendingOAuthRefreshFence(credential) &&
++reads > 1
) {
throw new Error("provider-B fence must not be observed");
}
return store;
: createOAuthRefreshFence({ profileId, credential: attempted });
const refresh = vi.fn(async () => (phase === "observer" ? null : foreign));
const resolve = vi.fn(async (credential: OAuthCredential) => credential.access);
let reads = 0;
const canRefresh = async (replace: (credential: OAuthCredential) => void) => {
if (phase === "compare-and-swap") {
replace(foreign);
}
return true;
};
if (backendKind === "manager") {
const replace = (credential: OAuthCredential) =>
saveCredential(agentDir, profileId, credential);
replace(initial);
mockOAuthProviderOperations({
buildApiKey: async (_provider, credential) => resolve(credential),
refreshCredential: refresh,
canRefreshCredential: () => canRefresh(replace),
});
} else if (phase === "observer") {
const load = authProfileStoreRuntime.loadAuthProfileStoreWithoutExternalProfilesAsync;
let ownReads = 0;
let foreignReads = 0;
vi.spyOn(
authProfileStoreRuntime,
"loadAuthProfileStoreWithoutExternalProfilesAsync",
).mockImplementation(async (...args) => {
const store = await load(...args);
const credential = store.profiles[profileId];
if (credential?.type === "oauth" && credential.provider === "openai") {
if (++ownReads === 2) {
saveCredential(agentDir, profileId, foreignFence);
if (phase === "observer") {
const load = authProfileStoreRuntime.loadAuthProfileStoreWithoutExternalProfilesAsync;
vi.spyOn(
authProfileStoreRuntime,
"loadAuthProfileStoreWithoutExternalProfilesAsync",
).mockImplementation(async (...args) => {
const store = await load(...args);
const credential = store.profiles[profileId];
if (
credential?.type === "oauth" &&
credential.provider === "openai" &&
++reads === 2
) {
replace(foreignFence);
}
} else if (
credential?.type === "oauth" &&
credential.provider === "provider-b" &&
isPendingOAuthRefreshFence(credential) &&
++foreignReads > 1
) {
throw new Error("provider-B fence observation continued");
}
return store;
});
return store;
});
}
await expect(
createOAuthManager().resolveOAuthAccess({
agentDir,
credential: attempted,
profileId,
store:
phase === "compare-and-swap"
? ensureAuthProfileStoreWithoutExternalProfiles(agentDir)
: profileStore(attempted, profileId),
}),
).resolves.toBeNull();
if (phase === "compare-and-swap") {
expect(readProfile(agentDir, profileId)).toEqual(foreign);
}
} else {
const { backend, set } = serializedBackend(
initial,
{
after: () => {
if (++reads === 1 && phase === "observer") {
set(foreignFence);
}
},
},
profileId,
);
await expect(
refreshSerializedOAuthCredential({
backend,
provider: "openai",
profileId,
label: "test serialized provider ownership",
timeoutMs: 1_000,
parse: (current) => JSON.parse(current ?? "{}") as Record<string, OAuthCredential>,
serialize: JSON.stringify,
readCredential: (data) => data[profileId],
writeCredential: (data, credential) => ({ ...data, [profileId]: credential }),
canRefresh: () => canRefresh(set),
refresh: async () => {
const credential = await refresh();
return credential ? { apiKey: credential.access, credential } : null;
},
resolve: async (credential) => ({ apiKey: await resolve(credential), credential }),
commit: () => {},
}),
).resolves.toBeNull();
}
const store =
phase === "compare-and-swap"
? ensureAuthProfileStoreWithoutExternalProfiles(agentDir)
: profileStore(attempted, profileId);
await expect(
manager.resolveOAuthAccess({ agentDir, credential: attempted, store, profileId }),
).resolves.toBeNull();
expect(buildApiKey).not.toHaveBeenCalled();
if (phase !== "observer") {
expect(refreshCredential).not.toHaveBeenCalled();
expect(resolve).not.toHaveBeenCalled();
if (backendKind === "serialized" || phase !== "observer") {
expect(refresh).not.toHaveBeenCalled();
}
if (phase === "compare-and-swap") {
expect(readProfile(agentDir, profileId)).toEqual(providerB);
}
});
},
);
it.each(["pending initial fence", "post-canRefresh replacement", "observer fence replacement"])(
"rejects a serialized provider change from a %s",
async (phase) => {
const profileId = "openai:default";
const foreign = createCredential({ provider: "provider-b", expires: 1 });
const foreignFence = createOAuthRefreshFence({ profileId, credential: foreign });
const initial =
phase === "pending initial fence"
? foreignFence
: phase === "observer fence replacement"
? createOAuthRefreshFence({ profileId, credential: createCredential({ expires: 1 }) })
: createCredential({ expires: 1 });
let calls = 0;
const { backend, set } = serializedBackend(
initial,
{
before: () => {
calls += 1;
if (
(phase === "pending initial fence" && calls > 1) ||
(phase === "observer fence replacement" && calls > 2)
) {
throw new Error("provider-B fence must not be observed");
}
},
after: () => {
if (phase === "observer fence replacement" && calls === 1) {
set(foreignFence);
}
},
},
profileId,
);
const refreshed = providerBCredential();
const refresh = vi.fn(async () => ({ apiKey: refreshed.access, credential: refreshed }));
const resolve = vi.fn(async (credential: OAuthCredential) => ({
apiKey: credential.access,
credential,
}));
await expect(
refreshSerializedOAuthCredential({
backend,
provider: "openai",
profileId,
label: "test serialized provider ownership",
timeoutMs: 1_000,
parse: (current) => JSON.parse(current ?? "{}") as Record<string, OAuthCredential>,
serialize: JSON.stringify,
readCredential: (data) => data[profileId],
writeCredential: (data, credential) => ({ ...data, [profileId]: credential }),
canRefresh: async () => {
if (phase === "post-canRefresh replacement") {
set(createCredential({ provider: "provider-b", expires: Date.now() + 600_000 }));
}
return true;
},
refresh,
resolve,
commit: () => {},
}),
).resolves.toBeNull();
expect(refresh).not.toHaveBeenCalled();
expect(resolve).not.toHaveBeenCalled();
},
);
it("runs provider I/O outside custom backend locks and fences peer retries", async () => {
const providerId = "test-oauth";
let lockDepth = 0;
const { backend, read } = serializedBackend(
createCredential({
provider: providerId,
access: "claimed-access",
refresh: "claimed-refresh",
expires: 1,
accountId: "acct-123",
}),
{
before: () => {
expect(lockDepth).toBe(0);
lockDepth += 1;
},
after: () => {
lockDepth -= 1;
},
},
);
});
it("hides custom-backend refresh claims and shares their settled credential with peers", async () => {
const providerId = "test-oauth";
const { backend, read } = serializedBackend(
createCredential({ provider: providerId, expires: 1, accountId: "acct-123" }),
);
const started = createDeferredCore();
const refresh = createDeferredCore<{ access: string; refresh: string; expires: number }>();
const refreshToken = vi.fn(() => {
expect(lockDepth).toBe(0);
started.resolve();
return refresh.promise;
});
@@ -371,8 +309,8 @@ describe("AuthStorage OAuth refresh ownership", () => {
await started.promise;
expect(read()).toContain("openclaw-oauth-refresh-fence:v1:");
expect(read()).not.toContain("claimed-access");
expect(read()).not.toContain("claimed-refresh");
expect(read()).not.toContain("access-token");
expect(read()).not.toContain("refresh-token");
expect(storage.get(providerId)).toBeUndefined();
expect(storage.has(providerId)).toBe(false);
expect(storage.list()).not.toContain(providerId);
@@ -401,12 +339,7 @@ describe("AuthStorage OAuth refresh ownership", () => {
it("preserves an expired credential when no refresh owner exists", async () => {
const providerId = "unowned-oauth";
const original = createCredential({
provider: providerId,
access: "unowned-access",
refresh: "unowned-refresh",
expires: 1,
});
const original = createCredential({ provider: providerId, expires: 1 });
const { backend, read } = serializedBackend(original);
const storage = AuthStorage.fromStorage(backend);
@@ -417,12 +350,7 @@ describe("AuthStorage OAuth refresh ownership", () => {
it("does not replay a failed generation and allows environment fallback after restart", async () => {
const providerId = "xai";
const { backend, read } = serializedBackend(
createCredential({
provider: providerId,
access: "failed-access",
refresh: "failed-refresh",
expires: 1,
}),
createCredential({ provider: providerId, expires: 1 }),
);
const refreshToken = vi.fn(async () => {
throw new Error("simulated provider rejection");
@@ -439,8 +367,8 @@ describe("AuthStorage OAuth refresh ownership", () => {
await expect(restarted.getApiKey(providerId)).resolves.toBe("environment-fallback");
expect(refreshToken).toHaveBeenCalledTimes(1);
expect(read()).toContain("openclaw-oauth-refresh-fence:v1:");
expect(read()).not.toContain("failed-access");
expect(read()).not.toContain("failed-refresh");
expect(read()).not.toContain("access-token");
expect(read()).not.toContain("refresh-token");
} finally {
vi.unstubAllEnvs();
}
@@ -479,13 +407,7 @@ describe("AuthStorage OAuth publication", () => {
) {
queued = true;
queueMicrotask(() => {
if (change === "logout") {
mutate.logout(providerId);
} else if (change === "replace") {
mutate.set(providerId, replacement);
} else if (change === "unrelated") {
mutate.set("other", { type: "api_key", key: "synthetic-other-new" });
}
changeCredential(mutate, providerId, change, replacement);
});
}
}
@@ -497,8 +419,7 @@ describe("AuthStorage OAuth publication", () => {
const expected =
change === "logout" ? undefined : change === "replace" ? replacement : refreshed;
expect(apiKey).toBe(expected?.access);
expect(storage.get(providerId)).toEqual(expected);
expect(durable?.[profileId]).toEqual(expected);
expectPublished(storage, agentDir, providerId, expected);
expect(refreshToken).toHaveBeenCalledTimes(
phase === "claim" && (change === "logout" || change === "replace") ? 0 : 1,
);
@@ -545,11 +466,7 @@ describe("AuthStorage OAuth publication", () => {
} else if (!changeQueued && durable.access === refreshed.access) {
changeQueued = true;
queueMicrotask(() => {
if (change === "logout") {
peer.logout(providerId);
} else if (change === "replace") {
peer.set(providerId, replacement);
}
changeCredential(peer, providerId, change, replacement);
});
}
}
@@ -562,8 +479,7 @@ describe("AuthStorage OAuth publication", () => {
const expected =
change === "logout" ? undefined : change === "replace" ? replacement : refreshed;
expect(apiKey).toBe(expected?.access);
expect(storage.get(providerId)).toEqual(expected);
expect(readProfile(agentDir, profileId)).toEqual(expected);
expectPublished(storage, agentDir, providerId, expected);
}),
);
@@ -606,8 +522,7 @@ describe("AuthStorage OAuth publication", () => {
const durable = readProfile(agentDir, profileId);
if (replace) {
expect(apiKey).toBe(replacement.access);
expect(storage.get(providerId)).toEqual(replacement);
expect(durable).toEqual(replacement);
expectPublished(storage, agentDir, providerId, replacement);
} else {
expect(apiKey).toBeUndefined();
expect(storage.get(providerId)).toBeUndefined();
@@ -989,18 +904,12 @@ describe("SQLite auth storage", () => {
const right = AuthStorage.forAgent(agentDir);
const refreshEntered = createDeferredCore();
const releaseRefresh = createDeferredCore();
let refreshCalls = 0;
let activeRefreshes = 0;
let maxActiveRefreshes = 0;
const provider = oauthProvider("test-oauth", async () => {
refreshCalls += 1;
activeRefreshes += 1;
maxActiveRefreshes = Math.max(maxActiveRefreshes, activeRefreshes);
const refresh = vi.fn(async () => {
refreshEntered.resolve();
await releaseRefresh.promise;
activeRefreshes -= 1;
return freshOAuth();
});
const provider = oauthProvider("test-oauth", refresh);
getAuthStorageOAuthProviderRegistry(left).register(provider);
getAuthStorageOAuthProviderRegistry(right).register(provider);
@@ -1036,7 +945,7 @@ describe("SQLite auth storage", () => {
} finally {
releaseRefresh.resolve();
}
expect([refreshCalls, maxActiveRefreshes]).toEqual([1, 1]);
expect(refresh).toHaveBeenCalledOnce();
await expect(right.getApiKey("test-oauth")).resolves.toBe("fake-fresh-access");
});
src/agents/sessions/model-registry.test.ts
+112 -209
View File
@@ -36,6 +36,10 @@ const providerConfig = (models: unknown[], config: Record<string, unknown> = {})
function catalog(providers: Record<string, unknown>) {
return { generatedBy: PLUGIN_MODEL_CATALOG_GENERATED_BY, providers };
}
const capturedCatalog = (pluginId: string, providers: Record<string, unknown>) => ({
pluginId,
contents: JSON.stringify(catalog(providers)),
});
const authResult = (apiKey?: string, headers?: Record<string, string | null>) => ({
ok: true,
@@ -58,14 +62,7 @@ describe("ModelRegistry", () => {
const { writeModelsJson, writeModelsJsonWithPluginCatalog, writeModelsJsonWithPluginCatalogs } =
installModelRegistryTestFixtures();
const writeRoot = (providers: Record<string, unknown> = {}) => writeModelsJson({ providers });
const nvidia = { baseUrl: "https://integrate.api.nvidia.com/v1", apiKey: "NVIDIA_API_KEY" };
const zai = {
baseUrl: "https://api.z.ai/api/paas/v4",
api: "openai-completions",
apiKey: "ZAI_API_KEY",
models: [{ id: "glm-5.1", name: "GLM 5.1" }],
};
const zai = providerConfig([{ id: "glm-5.1", name: "GLM 5.1" }]);
const custom = providerConfig([{ id: "authored-model", name: "Authored Model" }], {
apiKey: "authored-provider-test-key",
});
@@ -95,17 +92,6 @@ describe("ModelRegistry", () => {
});
const ownerSnapshot = (...ids: string[]) =>
pluginOwnerSnapshotEntries(ids.map((id) => ({ providerId: id, pluginId: id })));
async function generatedRegistry(
providers: Record<string, unknown>,
auth = AuthStorage.inMemory(),
) {
return createRegistry(
await writeCatalogs(providers),
{ pluginMetadataSnapshot: ownerSnapshot(...Object.keys(providers)) },
auth,
);
}
function oauthProviderConfig(name: string, apiKeyPrefix: string): ProviderConfigInput {
const credentials = () => ({
access: "test-token-placeholder",
@@ -148,18 +134,16 @@ describe("ModelRegistry", () => {
});
it("accepts Bedrock AWS SDK auth without apiKey", async () => {
const modelsPath = writeRoot({
"amazon-bedrock": providerConfig([{ id: "bedrock-model" }], {
baseUrl: "https://bedrock-runtime.us-east-1.amazonaws.com",
api: "bedrock-converse-stream",
auth: "aws-sdk",
const registry = createRegistry(
writeRoot({
"amazon-bedrock": providerConfig([{ id: "bedrock-model" }], {
api: "bedrock-converse-stream",
auth: "aws-sdk",
}),
}),
});
const registry = ModelRegistry.create(AuthStorage.inMemory(), modelsPath);
);
const model = registry.find("amazon-bedrock", "bedrock-model");
expect(registry.getError()).toBeUndefined();
expect(registry.getAvailable()).toEqual([model]);
await expect(registry.getApiKeyAndHeaders(model!)).resolves.toEqual(authResult());
expect(registry.getProviderAuthStatus("amazon-bedrock")).toEqual({
@@ -170,12 +154,16 @@ describe("ModelRegistry", () => {
});
it("forks a catalog with request-isolated auth and provider mutations", async () => {
const template = createRegistry(exampleCatalog());
const template = createRegistry(exampleCatalog("test-token-placeholder"));
const firstAuth = AuthStorage.inMemory();
const secondAuth = AuthStorage.inMemory();
const first = template.fork(firstAuth);
const second = template.fork(secondAuth);
await expect(first.getApiKeyForProvider("custom")).resolves.toBe("test-token-placeholder");
await expect(
first.getApiKeyAndHeaders(first.find("custom", "example-model")!),
).resolves.toEqual(authResult("test-token-placeholder"));
firstAuth.setRuntimeApiKey("custom", "first-runtime-key");
secondAuth.setRuntimeApiKey("custom", "second-runtime-key");
first.registerProvider("first-only", oauthProviderConfig("First only", "first"));
@@ -197,17 +185,6 @@ describe("ModelRegistry", () => {
expect(first.find("custom", "example-model")).toBeDefined();
});
it("preserves models.json provider auth in a catalog fork", async () => {
const template = createRegistry(exampleCatalog("test-token-placeholder"));
const fork = template.fork(AuthStorage.inMemory());
const model = fork.find("custom", "example-model");
await expect(fork.getApiKeyForProvider("custom")).resolves.toBe("test-token-placeholder");
await expect(fork.getApiKeyAndHeaders(model!)).resolves.toEqual(
authResult("test-token-placeholder"),
);
});
it("skips native presence while resolving configured HTTP credentials", async () => {
const auth = AuthStorage.inMemory({
custom: {
@@ -219,8 +196,6 @@ describe("ModelRegistry", () => {
const registry = createRegistry(exampleCatalog("synthetic-configured-api-key"), {}, auth);
const model = registry.find("custom", "example-model")!;
expect(auth.hasAuth("custom")).toBe(true);
await expect(auth.getApiKey("custom", { includeFallback: false })).resolves.toBeUndefined();
await expect(registry.getApiKeyForProvider("custom")).resolves.toBe(
"synthetic-configured-api-key",
);
@@ -250,52 +225,36 @@ describe("ModelRegistry", () => {
parameterCount: 4_000_000_000,
family: "meta",
maxTokens: 128_000,
contextWindow: 1_000_000,
contextTokens: 872_000,
};
const config: ProviderConfigInput = {
api: "openai-responses",
baseUrl: "https://models.example/v1",
models: [{ ...model, input: [...model.input] }],
};
const windows = [
["fixture-primary", 1_000_000, 872_000],
["fixture-secondary", 1_050_000, 922_000],
] as const;
const providers: Record<string, ProviderConfigInput> = Object.fromEntries(
windows.map(([provider, contextWindow, contextTokens]) => [
provider,
{
api: "openai-responses",
baseUrl: "https://models.example/v1",
models: [{ ...model, input: [...model.input], contextWindow, contextTokens }],
},
]),
);
const registry =
source === "persisted"
? createRegistry(writeRoot(providers))
? createRegistry(writeRoot({ fixture: config }))
: ModelRegistry.inMemory(AuthStorage.inMemory());
if (source === "registered") {
for (const [provider, config] of Object.entries(providers)) {
registry.registerProvider(provider, config);
}
registry.registerProvider("fixture", config);
}
expect(registry.getError()).toBeUndefined();
for (const candidate of [registry, registry.fork(AuthStorage.inMemory())]) {
for (const [provider, config] of Object.entries(providers)) {
expect(candidate.find(provider, "shared-model")).toMatchObject(config.models![0]!);
}
expect(candidate.find("fixture", model.id)).toMatchObject(model);
}
});
it("drops a malformed persisted family without rejecting the catalog", () => {
const registry = createRegistry(
writeRoot({
"family-fixture": {
api: "openai-responses",
baseUrl: "https://models.example/v1",
models: [
{ id: "spaced", family: "Open AI" },
{ id: "cased", family: " Meta " },
],
},
"family-fixture": providerConfig([
{ id: "spaced", family: "Open AI" },
{ id: "cased", family: " Meta " },
]),
}),
);
expect(registry.getError()).toBeUndefined();
expect(registry.find("family-fixture", "spaced")).not.toHaveProperty("family");
expect(registry.find("family-fixture", "cased")?.family).toBe("meta");
});
@@ -314,7 +273,6 @@ describe("ModelRegistry", () => {
),
});
const pluginCatalogs = loadPersistedPluginModelCatalogsReadOnly(dirname(modelsPath));
expect(pluginCatalogs).toHaveLength(1);
const stored = JSON.parse(pluginCatalogs[0]!.contents).providers[provider].models;
for (const id of ["unknown", "prompt-only"]) {
expect(stored.find((row: { id: string }) => row.id === id)).not.toHaveProperty(
@@ -327,15 +285,15 @@ describe("ModelRegistry", () => {
pluginMetadataSnapshot: pluginOwnerSnapshot(provider, provider),
});
for (const candidate of [registry, registry.fork(AuthStorage.inMemory())]) {
for (const [id, expected] of [
["unknown", { contextWindow: 128_000 }],
["prompt-only", { contextWindow: 777_000, contextTokens: 777_000 }],
] as const) {
expect(candidate.find(provider, id)).toMatchObject({
...expected,
contextWindowSource: "synthetic",
});
}
expect(candidate.find(provider, "unknown")).toMatchObject({
contextWindow: 128_000,
contextWindowSource: "synthetic",
});
expect(candidate.find(provider, "prompt-only")).toMatchObject({
contextWindow: 777_000,
contextTokens: 777_000,
contextWindowSource: "synthetic",
});
for (const [id, window] of [
["reported", 128_000],
["authored", 64_000],
@@ -350,21 +308,14 @@ describe("ModelRegistry", () => {
it("loads only lifecycle-captured generated catalogs when the root catalog is absent", () => {
const modelsPath = writeRoot();
rmSync(modelsPath);
const capturedCatalog = {
pluginId: "zai",
contents: JSON.stringify(catalog({ zai: { ...zai, apiKey: undefined } })),
};
const pluginMetadataSnapshot = ownerSnapshot("zai", "other");
const registry = createRegistry(modelsPath, {
includePluginCatalogs: true,
modelsJsonContents: null,
pluginCatalogs: [capturedCatalog],
pluginCatalogs: [capturedCatalog("zai", { zai })],
pluginMetadataSnapshot,
});
const fork = registry.fork(AuthStorage.inMemory());
expect(registry.getError()).toBeUndefined();
expect(registry.find("zai", "glm-5.1")?.name).toBe("GLM 5.1");
expect(registry.find("other", "unrelated-model")).toBeUndefined();
expect(registry.getProviderMetadataOwners()).toBe(pluginMetadataSnapshot.owners);
@@ -382,73 +333,38 @@ describe("ModelRegistry", () => {
});
it("loads richer generated catalog metadata without widening runtime inputs", async () => {
const registry = await generatedRegistry(
{
minimax: providerConfig([{ id: "video", input: ["text", "image", "video"] }], {
baseUrl: "https://api.minimaxi.com/v1",
apiKey: "MINIMAX_API_KEY",
}),
nvidia: providerConfig(
[
{ id: "audio", input: ["text", "image", "audio"] },
{ id: "audio-only", input: ["audio"] },
{ id: "explicit-empty", input: [] },
],
nvidia,
),
},
AuthStorage.inMemory({
minimax: { type: "api_key", key: "sk-minimax" },
nvidia: { type: "api_key", key: "sk-nvidia" },
}),
const modelsPath = await writeCatalogs({
fixture: providerConfig([
{ id: "mixed", input: ["text", "image", "video", "audio"] },
{ id: "unsupported", input: ["audio"] },
{ id: "empty", input: [] },
]),
});
const registry = createRegistry(
modelsPath,
{ pluginMetadataSnapshot: ownerSnapshot("fixture") },
AuthStorage.inMemory({ fixture: { type: "api_key", key: "fixture-key" } }),
);
expect(registry.getError()).toBeUndefined();
expect(registry.find("minimax", "video")?.input).toEqual(["text", "image"]);
expect(registry.find("nvidia", "audio")?.input).toEqual(["text", "image"]);
expect(registry.find("nvidia", "audio-only")).toBeUndefined();
expect(registry.find("nvidia", "explicit-empty")?.input).toEqual([]);
const availableRefs = registry.getAvailable().map((model) => `${model.provider}/${model.id}`);
expect(availableRefs).not.toContain("nvidia/audio-only");
expect(availableRefs).toContain("nvidia/explicit-empty");
expect(registry.find("fixture", "mixed")?.input).toEqual(["text", "image"]);
expect(registry.find("fixture", "unsupported")).toBeUndefined();
expect(registry.find("fixture", "empty")?.input).toEqual([]);
const availableIds = registry.getAvailable().map((model) => model.id);
expect(availableIds).not.toContain("unsupported");
expect(availableIds).toContain("empty");
});
it("isolates invalid persisted plugin catalogs from valid models", async () => {
const modelsPath = await writeModelsJsonWithPluginCatalogs({
root: {
providers: {
custom: providerConfig([{ id: "root-model", name: "Root Model" }], {
api: "openai-responses",
apiKey: "CUSTOM_API_KEY",
}),
},
},
pluginCatalogs: [
pluginCatalog("google", {
"google-vertex": {
baseUrl: "https://us-central1-aiplatform.googleapis.com/v1",
api: "google-vertex",
apiKey: "GOOGLE_API_KEY",
models: [{ id: "gemini-3.1-pro-preview", contextWindow: 0 }],
},
}),
pluginCatalog("zai", { zai }),
],
});
const registry = createRegistry(modelsPath, {
pluginMetadataSnapshot: pluginOwnerSnapshotEntries([
{ providerId: "google-vertex", pluginId: "google" },
{ providerId: "zai", pluginId: "zai" },
]),
});
expect(registry.getError()).toContain(
"Provider google-vertex, model gemini-3.1-pro-preview: invalid contextWindow",
const modelsPath = await writeCatalogs(
{ invalid: providerConfig([{ id: "invalid", contextWindow: 0 }]), zai },
{ providers: { custom } },
);
expect(registry.find("custom", "root-model")?.name).toBe("Root Model");
const registry = createRegistry(modelsPath, {
pluginMetadataSnapshot: ownerSnapshot("invalid", "zai"),
});
expect(registry.getError()).toContain("Provider invalid, model invalid: invalid contextWindow");
expect(registry.find("custom", "authored-model")?.name).toBe("Authored Model");
expect(registry.find("zai", "glm-5.1")?.name).toBe("GLM 5.1");
expect(registry.find("google-vertex", "gemini-3.1-pro-preview")).toBeUndefined();
expect(registry.find("invalid", "invalid")).toBeUndefined();
});
it.each([
@@ -610,16 +526,10 @@ describe("ModelRegistry", () => {
if (process.getuid?.() === 0) {
return;
}
const modelsPath = await writeModelsJsonWithPluginCatalog({
root: { providers: { custom } },
...pluginCatalog("anthropic", {
anthropic: providerConfig([{ id: "healthy-model", name: "Healthy Model" }], {
baseUrl: "https://anthropic.example/v1",
api: "anthropic-messages",
apiKey: "healthy-provider-test-key",
}),
}),
});
const modelsPath = await writeCatalogs(
{ anthropic: providerConfig([{ id: "healthy-model", name: "Healthy Model" }]) },
{ providers: { custom } },
);
const sourcePath = join(dirname(modelsPath), "plugins", "zai", PLUGIN_MODEL_CATALOG_FILE);
mkdirSync(dirname(sourcePath), { recursive: true });
writeFileSync(
@@ -666,24 +576,19 @@ describe("ModelRegistry source composition", () => {
vercelGatewayRouting: { order: ["provider-default"], only: ["shared-route"] },
};
const authoredHeaders = { "X-Authored-Provider": "root", "X-Authored-Model": "root" };
const authored = {
api: "openai-completions",
baseUrl: rootUrl,
apiKey: "authored-fixture-key",
headers: { "X-Authored-Provider": "root" },
models: [
const authored = providerConfig(
[
{ id: "shared", name: "Authored", maxTokens: 2048, headers: { "X-Authored-Model": "root" } },
{ id: "Shared", name: "Case-distinct authored model" },
],
};
const generated = {
api: "openai-completions",
baseUrl: catalogUrl,
apiKey: "cached-fixture-key",
auth: "aws-sdk",
authHeader: true,
headers: { "X-Cached-Provider": "cache" },
models: [
{
baseUrl: rootUrl,
apiKey: "authored-fixture-key",
headers: { "X-Authored-Provider": "root" },
},
);
const generated = providerConfig(
[
{
id: "shared",
name: "Generated",
@@ -693,7 +598,14 @@ describe("ModelRegistry source composition", () => {
},
{ id: "generated-only", maxTokens: 4096 },
],
};
{
baseUrl: catalogUrl,
apiKey: "cached-fixture-key",
auth: "aws-sdk",
authHeader: true,
headers: { "X-Cached-Provider": "cache" },
},
);
const getAuth = (registry: ModelRegistry, id = "shared", owner = provider) =>
registry.getApiKeyAndHeaders(registry.find(owner, id)!);
@@ -719,12 +631,7 @@ describe("ModelRegistry source composition", () => {
root === null ? null : { [options.authoredProvider ?? provider]: root },
AuthStorage.inMemory(options.credentials ?? {}),
{
pluginCatalogs: [
{
pluginId,
contents: JSON.stringify(catalog({ [provider]: cached })),
},
],
pluginCatalogs: [capturedCatalog(pluginId, { [provider]: cached })],
pluginMetadataSnapshot: pluginOwnerSnapshot(provider, pluginId),
},
);
@@ -867,7 +774,7 @@ describe("ModelRegistry source composition", () => {
...authored,
api,
authHeader: true,
models: [{ ...authored.models[0], baseUrl: catalogUrl }],
models: [{ id: "shared", baseUrl: catalogUrl, headers: { "X-Authored-Model": "root" } }],
},
generated: {
api: "openai-completions",
@@ -930,31 +837,27 @@ describe("ModelRegistry no-auth", () => {
params.storedKey ? { custom: { type: "api_key", key: "stored-key" } } : {},
);
const getApiKey = vi.spyOn(authStorage, "getApiKey");
const modelConfig = {
id: "model",
api: params.modelApi ?? "openai-completions",
headers: { authorization: params.authorization ?? "Bearer model", "X-Model": "model" },
};
const provider = {
baseUrl: "https://inference.example.invalid/v1",
api: params.providerApi ?? "openai-completions",
auth: params.auth,
...(params.inlineKey === false ? {} : { apiKey: "inline-key" }),
authHeader: true,
headers: {
Authorization: params.authorization ?? "Bearer provider",
"X-Provider": "provider",
const provider = providerConfig(
[
{
id: "model",
api: params.modelApi ?? "openai-completions",
headers: { authorization: params.authorization ?? "Bearer model", "X-Model": "model" },
},
],
{
api: params.providerApi ?? "openai-completions",
auth: params.auth,
...(params.inlineKey === false ? {} : { apiKey: "inline-key" }),
authHeader: true,
headers: {
Authorization: params.authorization ?? "Bearer provider",
"X-Provider": "provider",
},
},
models: [modelConfig],
};
const registry = capturedRegistry({ custom: provider }, authStorage, {
pluginCatalogs: [],
includePluginCatalogs: false,
});
const model = registry.find("custom", "model");
if (!model) {
throw new Error(registry.getError() ?? "Expected custom model");
}
);
const registry = capturedRegistry({ custom: provider }, authStorage);
const model = registry.find("custom", "model")!;
return { registry, model, getApiKey };
}
@@ -962,8 +865,8 @@ describe("ModelRegistry no-auth", () => {
"keeps registry no-auth terminal with provider adapter %s and an effective Completions route",
async (providerApi) => {
const { registry, model, getApiKey } = noAuthRegistry({
providerApi,
auth: "none",
providerApi,
storedKey: true,
});
const requestModel = {
src/agents/sessions/sdk.auth-migration.test.ts
+39 -99
View File
@@ -12,10 +12,7 @@ import {
replaceSessionEntry,
} from "../../config/sessions/session-accessor.js";
import * as contextWorker from "../../config/sessions/session-transcript-read-worker-runtime.js";
import {
getOwnedSessionTranscriptWriterFence,
SessionTranscriptWriterClaimReboundError,
} from "../../config/sessions/transcript-write-context.js";
import { SessionTranscriptWriterClaimReboundError } from "../../config/sessions/transcript-write-context.js";
import { snapshotFiles } from "../../infra/state-migrations.caller-mode.test-helpers.js";
import { autoMigrateLegacyState } from "../../infra/state-migrations.doctor.js";
import { inspectLegacyAgentDir } from "../../infra/state-migrations.legacy-sessions.js";
@@ -24,7 +21,6 @@ import { createDeferredCore } from "../../shared/deferred.js";
import { unregisterOpenClawAgentDatabase } from "../../state/openclaw-agent-db-registry.js";
import {
closeOpenClawAgentDatabaseByPathAsync,
inspectOpenClawAgentDatabaseOwner,
openOpenClawAgentDatabase,
} from "../../state/openclaw-agent-db.js";
import { runOpenClawAgentWorkerWrite } from "../../state/openclaw-agent-write-admission.js";
@@ -60,7 +56,6 @@ import { createAgentSession } from "./sdk.js";
import { sessionManagerReadInitialContext } from "./session-manager-current-turn.js";
import { SessionMetadataCommittedError } from "./session-manager-metadata-error.js";
import { sessionManagerAppendInitialMetadata } from "./session-manager-metadata.js";
import type { ModelChangeEntry, ThinkingLevelChangeEntry } from "./session-manager-types.js";
import * as writeAdmission from "./session-manager-write-admission.js";
import { SessionManager } from "./session-manager.js";
import { SettingsManager } from "./settings-manager.js";
@@ -101,10 +96,6 @@ describe("SDK credential migration", () => {
expect(() => assertAuthProfileMigrationReady(ownerDir)).toThrow(migrationMessage);
}
function expectKey(actual: string | undefined, expected: string) {
expect(actual === expected, "selected credential belongs to the expected owner").toBe(true);
}
async function stageLegacy(
state: OpenClawTestState,
legacy: AuthProfileStore,
@@ -156,14 +147,8 @@ describe("SDK credential migration", () => {
await writeFile(path.join(legacyDir, "bin", "fd"), "installed SDK tool");
openOpenClawAgentDatabase({ agentId: "main", env: state.env, path: databasePath });
await closeOpenClawAgentDatabaseByPathAsync(databasePath);
// The legacy standalone SDK fixture predates registry enrollment.
unregisterOpenClawAgentDatabase({ agentId: "main", env: state.env, path: databasePath });
expect(inspectOpenClawAgentDatabaseOwner(databasePath)).toEqual({
status: "owned",
agentId: "main",
});
// Payload inspection materializes native WAL sidecars before the snapshot.
expect(inspectLegacyAgentDir(legacyDir)).toEqual({ status: "payload" });
inspectLegacyAgentDir(legacyDir); // Materialize WAL sidecars before taking the snapshot.
const before = snapshotFiles(legacyDir);
const cfg = { agents: { entries: { worker: {} } }, plugins: { enabled: false } };
await state.writeConfig(cfg);
@@ -173,7 +158,6 @@ describe("SDK credential migration", () => {
doctorOnlyStateMigrations: true,
legacySessionSurfaces: EMPTY_LEGACY_SESSION_SURFACES,
});
expect(result.stepReceipts.find((entry) => entry.outcome === "refused")).toBeUndefined();
const receipt = result.stepReceipts.find((entry) => entry.id === "agent-dir");
expect(receipt).toMatchObject({
outcome: "deferred",
@@ -185,9 +169,6 @@ describe("SDK credential migration", () => {
path: legacyDir,
},
],
warnings: expect.arrayContaining([
expect.stringContaining("Keep using the existing store"),
]),
});
expect(snapshotFiles(legacyDir)).toEqual(before);
await expect(
@@ -215,12 +196,9 @@ describe("SDK credential migration", () => {
});
const model = makeProviderModelFixture({
id: "synthetic-model",
name: "Synthetic model",
api: "openai-completions",
provider: "arcee",
baseUrl: modelUrl,
contextWindow: 1000,
maxTokens: 1000,
});
const runtimeConfig = getRuntimeConfig();
const manager = SessionManager.inMemory();
@@ -247,16 +225,14 @@ describe("SDK credential migration", () => {
const result = Promise.resolve(stream(model, { messages: [] }, {}));
if (blocked) {
await expect(result).rejects.toThrow(migrationMessage);
expect(providerIo).not.toHaveBeenCalled();
} else {
await result;
}
expect(providerIo).toHaveBeenCalledTimes(blocked ? 0 : 1);
if (!blocked) {
const auth = await session.modelRegistry.getApiKeyAndHeaders(model);
expect(
auth.ok && auth.apiKey === credential,
"direct account credential selected",
).toBe(true);
expect(providerIo).toHaveBeenCalledOnce();
expect(await session.modelRegistry.getApiKeyAndHeaders(model)).toMatchObject({
ok: true,
apiKey: credential,
});
}
} finally {
session.dispose();
@@ -317,7 +293,7 @@ describe("SDK credential migration", () => {
"requires the active secrets runtime to materialize SecretRef credentials",
);
} else {
expectKey(await resolve(), provider === "arcee" ? localKey : unaffectedKey);
expect(await resolve()).toBe(unaffectedKey);
}
expect(fallback).not.toHaveBeenCalled();
},
@@ -349,7 +325,7 @@ describe("SDK credential migration", () => {
expect(fallback).not.toHaveBeenCalled();
clearAuthProfileMigrationDiagnostics();
storage.reload();
expectKey(await storage.getApiKey("arcee", { baseUrl }), importedKey);
expect(await storage.getApiKey("arcee", { baseUrl })).toBe(importedKey);
if (owner === "shared") {
const localKey = "synthetic-new-local-key";
storage.set("arcee", { type: "api_key", key: localKey });
@@ -358,7 +334,7 @@ describe("SDK credential migration", () => {
await state.writeJson(legacyPath, legacy);
await refreshAuthProfileFileState();
expectMigrationRequired();
expectKey(await storage.getApiKey("arcee", { baseUrl }), localKey);
expect(await storage.getApiKey("arcee", { baseUrl })).toBe(localKey);
}
});
},
@@ -372,7 +348,7 @@ describe("SDK credential migration", () => {
await importLegacy(state, legacy);
writeStore(legacy, agentDir);
const { storage, fallback } = authWithFallback(agentDir);
expectKey(await storage.getApiKey("arcee", { baseUrl }), key);
expect(await storage.getApiKey("arcee", { baseUrl })).toBe(key);
expect(fallback).not.toHaveBeenCalled();
},
{ UNRESOLVED_IMPORTED_ARCEE: undefined },
@@ -422,7 +398,7 @@ describe("SDK credential migration", () => {
});
await expect(storage.getApiKey("arcee", { baseUrl })).rejects.toMatchObject(migrationError);
expect(reloadedSelectedCredential).toBe(true);
expectKey(await storage.getApiKey("arcee", { baseUrl }), localKey);
expect(await storage.getApiKey("arcee", { baseUrl })).toBe(localKey);
});
});
@@ -443,7 +419,7 @@ describe("SDK credential migration", () => {
if (blocked) {
await expect(result).rejects.toMatchObject(migrationError);
} else {
expect((await result) === key, "unrelated canonical credential remains usable").toBe(true);
expect(await result).toBe(key);
}
});
});
@@ -488,16 +464,9 @@ function registryFor(...models: (typeof testModel)[]) {
registry.registerProvider(testModel.provider, {
api: testModel.api,
baseUrl: testModel.baseUrl,
models: models.map(({ id, name, api, reasoning, input, cost, contextWindow, maxTokens }) => ({
id,
name,
api,
reasoning,
input,
cost,
contextWindow: contextWindow ?? 32_768,
maxTokens,
})),
models: models.map((model) =>
Object.assign({}, model, { contextWindow: model.contextWindow ?? 32_768 }),
),
});
return registry;
}
@@ -573,23 +542,15 @@ it.each(["metadata", "context loading"] as const)(
const replacementBefore = await loadTranscriptEvents(replacement);
const originalBefore = await loadTranscriptEvents(original);
const manager = SessionManager.open(original, state.workspaceDir);
const completed: {
entry?: ModelChangeEntry | ThinkingLevelChangeEntry;
records?: Awaited<ReturnType<typeof loadTranscriptEvents>>;
watermark?: ReturnType<typeof readSessionTranscriptWatermark>;
} = {};
let committedId: string | undefined;
let committedEntry: ReturnType<SessionManager["getEntry"]>;
let committedRecords: Awaited<ReturnType<typeof loadTranscriptEvents>> | undefined;
const retargetAfterAppend = async (append: () => Promise<string>) => {
const id = await append();
const entry = manager.getEntry(id);
if (!entry || (entry.type !== "model_change" && entry.type !== "thinking_level_change")) {
throw new Error("Expected the real append's committed metadata entry");
}
completed.entry = structuredClone(entry);
// Retarget after the real append commits, before the factory exposes its session.
completed.records = loadTranscriptEventsSync(original);
completed.watermark = readSessionTranscriptWatermark(original);
committedId = await append();
committedEntry = structuredClone(manager.getEntry(committedId));
committedRecords = loadTranscriptEventsSync(original);
manager.setSessionTarget(replacement);
return id;
return committedId;
};
const appendInitial = manager[sessionManagerAppendInitialMetadata].bind(manager);
const readInitialContext = manager[sessionManagerReadInitialContext].bind(manager);
@@ -606,13 +567,7 @@ it.each(["metadata", "context loading"] as const)(
return context;
});
onTestFinished(() => intercepted.mockRestore());
const model = {
...testModel,
id: "sdk-metadata-fixture",
reasoning: true,
};
expect(getOwnedSessionTranscriptWriterFence()).toBeUndefined();
const model = { ...testModel, reasoning: true };
const error = await rejectedSession(
createTestSession(manager, state.workspaceDir, {
model,
@@ -624,24 +579,24 @@ it.each(["metadata", "context loading"] as const)(
}),
}),
);
expect(intercepted).toHaveBeenCalledOnce();
expect.soft(await loadTranscriptEvents(replacement)).toEqual(replacementBefore);
if (after === "context loading") {
expect(await loadTranscriptEvents(original)).toEqual(originalBefore);
expect(error).toBeInstanceOf(SessionTranscriptWriterClaimReboundError);
expect(error).not.toBeInstanceOf(SessionMetadataCommittedError);
expect(hasModelFallbackStop(error)).toBe(false);
} else {
expect(completed.entry).toBeDefined();
expect(completed.records).toHaveLength(3);
expect(await loadTranscriptEvents(original)).toEqual(completed.records);
const records = await loadTranscriptEvents(original);
const watermark = readSessionTranscriptWatermark(original);
expect(records).toHaveLength(3);
expect(records).toEqual(committedRecords);
expect(committedEntry).toMatchObject({ id: committedId, type: "thinking_level_change" });
expect(error).toBeInstanceOf(SessionMetadataCommittedError);
expect(error).toMatchObject({
committedEntry: completed.entry,
committedEntry,
committedTarget: original,
committedVersion: {
generation: completed.watermark?.generation,
rawSeq: completed.watermark?.maxSeq,
generation: watermark?.generation,
rawSeq: watermark?.maxSeq,
},
});
expect(hasModelFallbackStop(error)).toBe(true);
@@ -712,11 +667,7 @@ it.each([
const { session } = await pending;
const expected = selection === "branch" ? [first] : [];
expect(session.messages).toEqual(expected);
const next = {
role: "user" as const,
content: "Continue selected branch",
timestamp: 3,
};
const next = { role: "user" as const, content: "Continue selected branch", timestamp: 3 };
manager.appendMessage(next);
expect(manager.buildSessionContext().messages).toEqual([...expected, next]);
expect((await loadTranscriptEvents(target)).slice(0, before.length)).toEqual(before);
@@ -771,15 +722,15 @@ it("awaits extension entry, name, and label persistence before publishing their
const id = await api.appendEntryAsync("extension-state", { count: 1 });
await api.setSessionNameAsync("Awaited name");
await api.setLabelAsync(id, "bookmark");
expect(manager.getEntry(id)).toMatchObject({
expect(changes).toEqual(["Awaited name"]);
const reopened = await SessionManager.openAsync(target, state.workspaceDir);
expect(reopened.getEntry(id)).toMatchObject({
type: "custom",
customType: "extension-state",
data: { count: 1 },
});
expect(manager.getLabel(id)).toBe("bookmark");
expect(changes).toEqual(["Awaited name"]);
const reopened = await SessionManager.openAsync(target, state.workspaceDir);
expect(reopened.getEntry(id)).toEqual(manager.getEntry(id));
expect(manager.getLabel(id)).toBe("bookmark");
expect(reopened.getSessionName()).toBe("Awaited name");
expect(reopened.getLabel(id)).toBe("bookmark");
} finally {
@@ -882,7 +833,6 @@ it("does not publish a committed session name into a manager retargeted before c
}),
]);
await manager.setSessionTargetAsync(replacement);
expect(session.sessionManager).toBe(manager);
expect(manager.getSessionName()).toBe("Replacement name");
release.resolve();
await expect
@@ -936,12 +886,7 @@ it.each(["SDK initialization", "model transition"] as const)(
try {
process.chdir(firstDir);
const manager = SessionManager.open(relativeTarget, firstDir);
const capturedTarget = manager.getSessionTarget();
const reasoningModel = {
...testModel,
id: "cwd-reasoning",
reasoning: true,
};
const reasoningModel = { ...testModel, id: "cwd-reasoning", reasoning: true };
const plainModel = { ...reasoningModel, id: "cwd-plain", reasoning: false };
const modelRegistry = registryFor(reasoningModel, plainModel);
const create = async () =>
@@ -985,13 +930,8 @@ it.each(["SDK initialization", "model transition"] as const)(
await session.setModel(plainModel);
}
expect(intercepted).toHaveBeenCalledOnce();
expect(completed.records).toBeDefined();
expect(process.cwd()).toBe(secondDir);
const expectedTarget = { ...first, env: { OPENCLAW_STATE_DIR: state.stateDir } };
expect.soft(capturedTarget).toEqual(expectedTarget);
expect.soft(manager.getSessionTarget()).toEqual(expectedTarget);
expect(manager.getSessionId()).toBe(relativeTarget.sessionId);
const firstAfter = await loadTranscriptEvents(first);
expect(firstAfter.slice(0, completed.records?.length)).toEqual(completed.records);
const level = operation === "SDK initialization" ? "high" : "off";
src/gateway/server.chat-abort-dispatch-rejection.test.ts
+31 -43
View File
@@ -146,6 +146,14 @@ function textAttachment(bytes: string) {
};
}
async function history(socket: GatewaySocket, sessionKey = "main") {
const result = await rpcReq<{ messages: unknown[] }>(socket, "chat.history", { sessionKey });
expect(result.ok).toBe(true);
return result.payload?.messages;
}
const containingMessages = (...messages: Record<string, unknown>[]) =>
expect.arrayContaining(messages.map((message) => expect.objectContaining(message)));
describe("gateway WebSocket chat abort ownership", () => {
test("preserves committed native input across a profile-bound reconnect", async () => {
const scope = await prepareChatSession("native-reconnect", "native-reconnect-bound");
@@ -435,11 +443,8 @@ describe("gateway WebSocket chat abort ownership", () => {
const established = await rpcReq(socket, "chat.send", sendParameters);
expectRun(established, runId, "error", false);
expect(terminalStates).toEqual(yielded ? ["final", "error"] : ["error"]);
const history = await rpcReq<{ messages: unknown[] }>(socket, "chat.history", {
sessionKey: "main",
});
expect(history.payload?.messages).toContainEqual(
expect.objectContaining({
expect(await history(socket)).toEqual(
containingMessages({
role: "custom",
customType: "run-failed-before-reply",
content: expect.stringContaining("dispatch rejected before a late abort"),
@@ -450,8 +455,10 @@ describe("gateway WebSocket chat abort ownership", () => {
sessionKey: "main",
runId,
});
expect(lateAbort.ok).toBe(true);
expect(lateAbort.payload).toMatchObject({ ok: true, aborted: false, runIds: [] });
expect(lateAbort).toMatchObject({
ok: true,
payload: { ok: true, aborted: false, runIds: [] },
});
const replay = await rpcReq(socket, "chat.send", sendParameters);
expect(replay.ok).toBe(false);
@@ -564,19 +571,15 @@ describe("gateway WebSocket chat abort ownership", () => {
});
if (!runtimeTerminal) {
const settled = await rpcReq(socket, "agent.wait", { runId, timeoutMs: 1_000 });
expect(settled.ok).toBe(true);
expect(settled.payload).toMatchObject({
runId,
status: "error",
stopReason: "aborted",
endedAt: expect.any(Number),
expect(settled).toMatchObject({
ok: true,
payload: { runId, status: "error", stopReason: "aborted", endedAt: expect.any(Number) },
});
}
// The history RPC response follows every previously emitted chat
// event on this socket, so it exposes any contradictory late terminal.
const barrier = await rpcReq(socket, "chat.history", { sessionKey: "main" });
expect(barrier.ok).toBe(true);
await history(socket);
expect(terminalStates).toEqual([terminalState]);
} finally {
dispatchRelease.resolve();
@@ -693,14 +696,8 @@ describe("gateway WebSocket chat abort ownership", () => {
await withinTest(failureEntered.promise, signal);
await lifecycle;
// The same-socket response fences delivered events while the notice commit is held.
const heldHistory = await rpcReq<{ messages: unknown[] }>(socket, "chat.history", {
sessionKey: scope.sessionKey,
});
expect(heldHistory.ok).toBe(true);
expect(heldHistory.payload?.messages).not.toEqual(
expect.arrayContaining([
expect.objectContaining({ customType: "run-failed-before-reply" }),
]),
expect(await history(socket, scope.sessionKey)).not.toEqual(
containingMessages({ customType: "run-failed-before-reply" }),
);
expect.soft(terminalStates).toEqual([]);
failureRelease.resolve();
@@ -708,24 +705,20 @@ describe("gateway WebSocket chat abort ownership", () => {
expect.soft(terminalStates).toEqual(["error"]);
expect.soft(terminalFrame.payload?.errorMessage).toBe(warning);
expect(loadSessionEntry(scope)).toMatchObject({ status: "failed", lastRunId: runId });
const history = await rpcReq<{ messages: unknown[] }>(reader, "chat.history", {
sessionKey: scope.sessionKey,
});
expect(history.ok).toBe(true);
expect(history.payload?.messages).toEqual(
expect.arrayContaining([
expect.objectContaining({
expect(await history(reader, scope.sessionKey)).toEqual(
containingMessages(
{
role: "assistant",
content: expect.arrayContaining([
expect.objectContaining({ type: "text", text: "Running it now." }),
]),
}),
expect.objectContaining({
},
{
role: "custom",
customType: "run-failed-before-reply",
content: warning,
}),
]),
},
),
);
expect(loadTranscriptEventsSync(scope)).toContainEqual(
expect.objectContaining({
@@ -978,23 +971,18 @@ describe("gateway WebSocket chat abort ownership", () => {
const response = await send;
await sendWork;
await discardWork;
expect(response.ok).toBe(true);
expect(response.payload).toMatchObject({
const terminal = {
runId,
status: "timeout",
summary: "aborted",
stopReason: "rpc",
});
};
expect(response).toMatchObject({ ok: true, payload: terminal });
expect(responses).toEqual([
{
filePresent: false,
cleanupSettled: true,
payload: expect.objectContaining({
runId,
status: "timeout",
summary: "aborted",
stopReason: "rpc",
}),
payload: expect.objectContaining(terminal),
},
]);
expect(discardSpy).toHaveBeenCalledOnce();
src/gateway/server.sessions.reset-cleanup.test.ts
+8 -97
View File
@@ -6,18 +6,12 @@ import {
createDeferred,
withinTest,
} from "../../test/helpers/promise.js";
import {
isSessionEntryDataSql,
observeHostDataSql,
} from "../../test/helpers/sqlite-statement-execution-counter.js";
import { seedCanonicalAcpSessionMeta } from "../acp/runtime/session-meta-fixture.test-support.js";
import { createAcpMeta } from "../acp/runtime/session-meta-lifecycle.test-support.js";
import { readAcpSessionEntry } from "../acp/runtime/session-meta.js";
import { listRegisteredAgentHarnesses, registerAgentHarness } from "../agents/harness/registry.js";
import { restoreRegisteredAgentHarnesses } from "../agents/harness/registry.test-support.js";
import * as preparedModelRuntime from "../agents/prepared-model-runtime.js";
import { loadSessionEntry, upsertSessionEntryCore } from "../config/sessions/session-accessor.js";
import { memorySessionActorOwners } from "../config/sessions/session-actor-memory-owner.js";
import { RESET_PARENT_GRANT_FIXTURE } from "../config/sessions/session-lineage.test-support.js";
import type { InternalSessionEntry } from "../config/sessions/types.js";
import { peekSystemEvents } from "../infra/system-events.js";
@@ -36,7 +30,6 @@ import {
} from "../sessions/session-controller.mailbox.js";
import { createReplyOperation } from "../sessions/session-controller.operation.js";
import { runExclusiveSessionLifecycle } from "../sessions/session-lifecycle-admission.test-support.js";
import { resolveIncognitoOpenClawAgentSqlitePath } from "../state/openclaw-agent-db.paths.js";
import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js";
import { createRequestGate } from "./server-methods.request-gate.test-support.js";
import { disposeSessionReadContexts } from "./server-methods/sessions-read-cache.test-support.js";
@@ -65,8 +58,7 @@ import {
} from "./test/server-sessions.test-helpers.js";
const { createSessionStoreDir, seedActiveMainSession } = setupGatewaySessionsHandlerTestHarness();
const { retireSessionMcpRuntime: retireMcp, disposeSessionMcpRuntime: disposeMcp } =
bundleMcpRuntimeMocks;
const { disposeSessionMcpRuntime: disposeMcp } = bundleMcpRuntimeMocks;
afterEach(async () => {
await disposeSessionReadContexts();
@@ -199,14 +191,6 @@ function installAcpRuntimeBackendWithFreshSession() {
return prepareFreshSession;
}
const retirement = (retainAcrossReuse: boolean) => ({
sessionId: "sess-main",
reason: "gateway-session-cleanup",
preserveActiveLeases: true,
retainAcrossReuse,
onError: expect.any(Function),
});
function seedAcp(
sessionKey: string,
overrides: Partial<Parameters<typeof resolvedAcpMeta>[0]> = {},
@@ -257,37 +241,6 @@ const closedAcpKeys = () =>
([request]) => request?.sessionKey,
);
test("sessions.reset retires an unbound memory session without host session SQL", async () => {
await createSessionStoreDir();
const location = {
agentId: "main",
path: resolveIncognitoOpenClawAgentSqlitePath({ agentId: "main" }),
};
const sessionKey = "agent:main:dashboard:incognito-reset-source";
const scope = { agentId: "main", storePath: location.path, sessionKey };
try {
await upsertSessionEntryCore(scope, {
sessionId: "reset-source",
lifecycleRevision: "reset-source-lifecycle",
updatedAt: 1,
incognito: true,
});
const sql = observeHostDataSql();
try {
const result = await directSessionReq<{ deleted: boolean }>("sessions.reset", {
key: sessionKey,
});
expect(result).toMatchObject({ ok: true, payload: { deleted: true } });
expect(loadSessionEntry(scope)).toBeUndefined();
expect(sql.queries.filter(isSessionEntryDataSql)).toEqual([]);
} finally {
sql.restore();
}
} finally {
memorySessionActorOwners.closeDatabase(location);
}
});
test("sessions.reset preempts the controller run and clears queues", async () => {
const { activeRun, storePath } = await seedWaitingActiveMainSession();
const parentGrant = RESET_PARENT_GRANT_FIXTURE;
@@ -335,15 +288,10 @@ test("sessions.reset preempts the controller run and clears queues", async () =>
expectActiveRunCleanup("agent:main:main", ["main", "agent:main:main", "sess-main"], "main");
expect(peekSystemEvents("agent:main:main")).toStrictEqual([]);
expect(peekSystemEvents("agent:main:sess-main")).toStrictEqual([]);
expect(retireMcp).toHaveBeenCalledExactlyOnceWith(retirement(false));
expect(disposeMcp).toHaveBeenCalledWith("sess-main");
const browserCalls: unknown[][] =
browserSessionTabMocks.closeTrackedBrowserTabsForSessions.mock.calls;
expect(browserCalls).toHaveLength(1);
expect(browserCalls[0]?.[0]).toMatchObject({
sessionKeys: ["agent:main:main", "agent:main:sess-main"],
onWarn: expect.any(Function),
});
expect(browserSessionTabMocks.closeTrackedBrowserTabsForSessions).toHaveBeenCalledWith(
expect.objectContaining({ sessionKeys: ["agent:main:main", "agent:main:sess-main"] }),
);
expect(subagentLifecycleHookMocks.runSubagentEnded).toHaveBeenCalledExactlyOnceWith(
{
targetSessionKey: "agent:main:main",
@@ -475,29 +423,6 @@ test("sessions.reset clears retained native conversations from every execution o
}
});
test.each([false, true])(
"sessions.reset interrupts only non-initiating work (initiator: %s)",
async (initiator) => {
const { storePath } = await seedMainSession();
let interrupted = false;
let releaseAdmission = () => {};
const admission = await admitMainWork(storePath, () => {
interrupted = true;
if (!initiator) {
releaseAdmission();
}
});
releaseAdmission = admission.release;
try {
const reset = await (initiator ? admission.run(resetMainSession) : resetMainSession());
expect(reset.ok).toBe(true);
expect(interrupted).toBe(!initiator);
} finally {
admission.release();
}
},
);
test("sessions.reset rejects an active lifecycle mutation without interrupting admitted work", async () => {
const { storePath } = await seedMainSession();
let interrupted = false;
@@ -791,25 +716,11 @@ describe("acp-authority", () => {
const { closeAcpRuntimeForSession } = await import("./session-reset-acp.js");
const { storePath } = await seedActiveMainSession();
const sessionKey = "agent:main:main";
seedCanonicalAcpSessionMeta({
sessionKey,
meta: createAcpMeta({
backend: "acpx",
agent: "codex",
identity: {
state: "resolved",
acpxRecordId: sessionKey,
acpxSessionId: "original-backend",
source: "status",
lastUpdatedAt: Date.now(),
},
cwd: "/tmp/acp-session",
}),
});
acpRuntimeMocks.getAcpRuntimeBackend.mockReturnValue({
id: "acpx",
runtime: { prepareFreshSession: vi.fn(async () => {}) },
seedAcp(sessionKey, {
backendSessionId: "original-backend",
runtimeSessionName: "original-runtime",
});
installAcpRuntimeBackendWithFreshSession();
const scope = { agentId: "main", sessionKey, storePath };
const readState = () => ({
entry: loadSessionEntry(scope),
src/gateway/session-row-selection.test.ts
+156 -576
View File
@@ -1,6 +1,5 @@
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { subagentRuns } from "../agents/subagents/registry/subagent-registry-memory.js";
import { buildSubagentRunReadIndexFromRuns } from "../agents/subagents/registry/subagent-registry-queries.js";
import {
persistRegistryFixture,
saveSubagentRegistryToSqlite,
@@ -13,44 +12,42 @@ import {
patchSessionEntryCore,
replaceSessionEntrySync,
} from "../config/sessions/session-accessor.js";
import { writeSessionEntry } from "../config/sessions/session-accessor.sqlite-entry-store.js";
import type { SessionEntry } from "../config/sessions/types.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { resetPluginRuntimeStateForTest, setActivePluginRegistry } from "../plugins/runtime.js";
import * as sessionKeys from "../sessions/session-key-utils.js";
import { sessionChanges } from "../sessions/session-row-changes.js";
import { emitSessionTranscriptUpdate } from "../sessions/transcript-events.js";
import { registerOpenClawAgentDatabase } from "../state/openclaw-agent-db-registry.js";
import {
resolveOpenClawAgentSqlitePath,
runOpenClawAgentWriteTransaction,
} from "../state/openclaw-agent-db.js";
import { resolveOpenClawAgentSqlitePath } from "../state/openclaw-agent-db.js";
import { withOpenClawTestState } from "../test-utils/openclaw-test-state.js";
import { createSessionConversationTestRegistry } from "../test-utils/session-conversation-registry.js";
import { registerRpcSourceForTest } from "./server-methods/rpc-source-fixtures.test-support.js";
import { requestContext } from "./server-methods/sessions-read-cache.test-support.js";
import { listSessionFixture } from "./session-list.test-support.js";
import { retainSessionListForegroundWork } from "./session-projection-work.js";
import { indexSessionRow } from "./session-row-projection-identities.js";
import {
create as createSessionRow,
identity,
type Query,
type Row,
} from "./session-row-projection-record.js";
import { create as createSessionRow } from "./session-row-projection-record.js";
import { createSessionRowProjection } from "./session-row-projection.js";
import { holdSubagentExecution } from "./session-row-projection.read-fixture.test-support.js";
import { createSessionRowProjectionFixture } from "./session-row-projection.test-support.js";
import { createSessionRowEntrySelector, selectMatchingSessionRows } from "./session-row-scope.js";
import * as childOwners from "./session-utils-core.js";
import {
filterAndSortSessionEntries,
listProjectedSessions,
prepareProjectedSessionList,
prepareSessionRowSelection,
} from "./session-utils-list.js";
import { claimRpcSourceForTest } from "./test-helpers.rpc-source.js";
type ListOptions = Parameters<typeof listProjectedSessions>[0];
const listRows = (projection: ListOptions["projection"], opts: ListOptions["opts"], key?: string) =>
listProjectedSessions({ projection, opts, key });
function writeMain(sessionKey: string, entry: Partial<SessionEntry>) {
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{ sessionId: sessionKey, updatedAt: 1, ...entry },
);
}
const rowKeys = (result: Awaited<ReturnType<typeof listProjectedSessions>>) =>
result.sessions.map((row) => row.key);
beforeEach(() => {
setActivePluginRegistry(createSessionConversationTestRegistry());
});
@@ -59,109 +56,6 @@ afterEach(() => {
resetPluginRuntimeStateForTest();
});
it("preserves scoped key and ID selection against a scan over randomized federated rows", () => {
let seed = 1467;
const random = (bound: number) => {
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
return seed % bound;
};
const cfg = { agents: { entries: { main: {}, work: {}, other: {} } } };
const agents = Object.keys(cfg.agents.entries);
const paths = ["/synthetic/a", "/synthetic/b", "/synthetic/c"];
const scope = {
physicalPaths: (locator: string, agentId?: string) =>
locator === "/synthetic/shared"
? agentId
? [paths[agents.indexOf(agentId)]!]
: paths
: [locator],
};
for (let fixture = 0; fixture < 20; fixture++) {
const rows = new Map<string, Row>();
for (let index = 0; index < 80; index++) {
const agentId = agents[random(agents.length)]!;
const key = ["global", "unknown", `agent:${agentId}:key-${random(12)}`][random(3)]!;
const entry = { sessionId: `id-${random(12)}`, updatedAt: random(10) };
const row = {
...createSessionRow({
key,
agentId,
storeTarget: {
agentId: agents[random(agents.length)]!,
storePath: paths[random(paths.length)]!,
},
}),
entry,
};
rows.set(identity(row), row);
}
const indexes = {
byKey: new Map<string, Set<string>>(),
byAgent: new Map<string, Set<string>>(),
byStore: new Map<string, Set<string>>(),
byParent: new Map<string, Set<string>>(),
};
for (const row of rows.values()) {
indexSessionRow(row, indexes);
}
const matching = (query: Query, kind?: string) =>
selectMatchingSessionRows({ rows, indexes, scope }, query, kind);
const select = createSessionRowEntrySelector({
isActive: () => true,
runAsOwner: (read) => read(),
prepare: () => true,
state: () => ({
cfg,
scope,
...indexes,
rows,
dirty: new Set<string>(),
hasUnresolvedCategories: false,
matching,
acquire: (row) => row,
referenced: () => undefined,
}),
});
// The reference scans rows, independent of the production candidate indexes.
const scan = (query: Query, kind = "key") => {
const selectedPaths = query.storePath
? scope.physicalPaths(query.storePath, query.agentId)
: undefined;
const candidates =
!query.key && selectedPaths
? selectedPaths.flatMap((pathname) =>
[...rows.values()].filter((row) => row.storeTarget.storePath === pathname),
)
: [...rows.values()];
return candidates.filter(
(row) =>
(!query.key || (kind === "id" ? row.entry?.sessionId : row.key) === query.key) &&
(!query.agentId ||
row.agentId === query.agentId ||
((query.key || selectedPaths) && row.storeTarget.agentId === query.agentId)) &&
(!selectedPaths || selectedPaths.includes(row.storeTarget.storePath)),
);
};
for (let sample = 0; sample < 40; sample++) {
const query: Query = {
agentId: [...agents, undefined][random(4)],
storePath: [...paths, "/synthetic/shared", undefined][random(5)],
sortBy: null,
...(random(2)
? { sessionIdOrKey: random(2) ? `id-${random(12)}` : "global" }
: { key: random(2) ? `agent:main:key-${random(12)}` : undefined }),
};
expect(matching(query).map(identity)).toEqual(scan(query).map(identity));
const indexed = { ...query, key: query.sessionIdOrKey };
const keys = new Set([...scan(indexed, "id"), ...scan(indexed)].map((row) => row.key));
const expected = (
query.sessionIdOrKey ? [...keys].flatMap((key) => scan({ ...query, key })) : scan(query)
).filter((row) => !query.agentId || row.agentId === query.agentId);
expect(select(query).map(identity)).toEqual(expected.map(identity));
}
}
});
it("maintains list order across metadata changes and archived-row rematerialization", async () => {
await withOpenClawTestState({ scenario: "minimal" }, async () => {
const cfg = { agents: { entries: { main: {} } } };
@@ -171,147 +65,86 @@ it("maintains list order across metadata changes and archived-row rematerializat
[first, 2, 1],
[second, 1, 2],
] as const) {
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{ sessionId: sessionKey, updatedAt, lastInteractionAt },
);
writeMain(sessionKey, { updatedAt, lastInteractionAt });
}
const release = retainSessionListForegroundWork();
const projection = await createSessionRowProjection({ cfg, modelCatalog: [] });
const scan = vi.spyOn(projection, "selectEntries");
try {
const list = async (sortBy?: "lastInteractionAt" | "activity") =>
(await listProjectedSessions({ projection, opts: { limit: 1, sortBy } })).sessions;
(await listRows(projection, { limit: 1, sortBy })).sessions;
expect((await list())[0]?.key).toBe(first);
scan.mockClear();
emitSessionTranscriptUpdate({
target: { agentId: "main", sessionId: first, sessionKey: first },
});
expect((await list())[0]?.key).toBe(first);
expect(scan.mock.calls.filter(([query]) => !query?.key)).toHaveLength(0);
expect((await list("lastInteractionAt"))[0]?.key).toBe(second);
expect((await list("activity"))[0]?.key).toBe(second);
expect((await list())[0]?.key).toBe(first);
replaceSessionEntrySync(
{ agentId: "main", sessionKey: second },
{ sessionId: second, updatedAt: 3, label: "Changed metadata" },
);
writeMain(second, { updatedAt: 3, label: "Changed metadata" });
expect((await list())[0]).toMatchObject({ key: second, label: "Changed metadata" });
const third = "agent:main:third";
replaceSessionEntrySync(
{ agentId: "main", sessionKey: third },
{ sessionId: third, updatedAt: 4, lastInteractionAt: 3 },
);
writeMain(third, { updatedAt: 4, lastInteractionAt: 3 });
expect((await list())[0]?.key).toBe(third);
expect((await list("lastInteractionAt"))[0]?.key).toBe(third);
expect(
await listProjectedSessions({ projection, opts: { limit: 1, offset: 1 } }),
).toMatchObject({
expect(await listRows(projection, { limit: 1, offset: 1 })).toMatchObject({
totalCount: 3,
nextOffset: 2,
hasMore: true,
sessions: [{ key: second }],
});
replaceSessionEntrySync(
{ agentId: "main", sessionKey: third },
{ sessionId: third, updatedAt: 4, lastInteractionAt: 3, archivedAt: 5 },
);
expect((await list())[0]?.key).toBe(second);
const archived = await listProjectedSessions({ projection, opts: { archived: true } });
expect(archived.sessions.map((row) => row.key)).toEqual([third]);
replaceSessionEntrySync(
{ agentId: "main", sessionKey: third },
{ sessionId: third, updatedAt: 4, lastInteractionAt: 3 },
);
expect((await list())[0]?.key).toBe(third);
expect(await listProjectedSessions({ projection, opts: { archived: true } })).toMatchObject({
sessions: [],
totalCount: 0,
});
// Archive membership is presence, including a zero timestamp.
replaceSessionEntrySync(
{ agentId: "main", sessionKey: third },
{ sessionId: third, updatedAt: 4, lastInteractionAt: 3, archivedAt: 0 },
);
writeMain(third, { updatedAt: 4, lastInteractionAt: 3, archivedAt: 0 });
expect((await list())[0]?.key).toBe(second);
expect(
await listProjectedSessions({ projection, opts: { archived: true, limit: 1 } }),
).toMatchObject({ sessions: [{ key: third }], totalCount: 1, hasMore: false });
expect(await listRows(projection, { archived: true, limit: 1 })).toMatchObject({
sessions: [{ key: third }],
totalCount: 1,
hasMore: false,
});
const archivedQuery = { agentId: "main", key: third };
sessionChanges.emit({ all: true, scope: "catalog" });
await projection.ensureMaterialized();
expect(projection.capture(archivedQuery)?.materialized).toBeUndefined();
const rematerialized = await listProjectedSessions({
projection,
opts: { archived: true },
});
expect(rematerialized.sessions.map((row) => row.key)).toEqual([third]);
const rematerialized = await listRows(projection, { archived: true });
expect(rowKeys(rematerialized)).toEqual([third]);
expect(rematerialized.totalCount).toBe(1);
expect(projection.capture(archivedQuery)?.materialized).toBeDefined();
expect((await list())[0]?.key).toBe(second);
expect(scan.mock.calls.filter(([query]) => !query?.key)).toHaveLength(0);
await deleteSessionEntryLifecycle({
agentId: "main",
storePath: projection.capture({ agentId: "main", key: third })!.storeTarget.storePath,
archiveTranscript: false,
target: { canonicalKey: third, storeKeys: [third] },
});
const remaining = await listProjectedSessions({
projection,
opts: { archived: "all", sortBy: "activity" },
});
expect(remaining.sessions.map((row) => row.key)).toEqual([second, first]);
const remaining = await listRows(projection, { archived: "all", sortBy: "activity" });
expect(rowKeys(remaining)).toEqual([second, first]);
expect(remaining).toMatchObject({ totalCount: 2, nextOffset: null, hasMore: false });
// Deletion publishes topology; keyed updates must reuse the rebuilt scope.
scan.mockClear();
replaceSessionEntrySync(
{ agentId: "main", sessionKey: first },
{ sessionId: first, updatedAt: 2, lastInteractionAt: 1, pinnedAt: 10 },
);
writeMain(first, { updatedAt: 2, lastInteractionAt: 1, pinnedAt: 10 });
expect((await list())[0]?.key).toBe(first);
expect((await list("activity"))[0]?.key).toBe(second);
expect((await list("lastInteractionAt"))[0]?.key).toBe(first);
// A retained pin becomes effective only when the child is a sidebar root.
for (const sidebarRoot of [false, true, false]) {
replaceSessionEntrySync(
{ agentId: "main", sessionKey: first },
{
sessionId: first,
updatedAt: 2,
lastInteractionAt: 1,
pinnedAt: 10,
parentSessionKey: second,
sidebarRoot,
},
);
writeMain(first, {
updatedAt: 2,
lastInteractionAt: 1,
pinnedAt: 10,
parentSessionKey: second,
sidebarRoot,
});
expect((await list())[0]?.key).toBe(sidebarRoot ? first : second);
expect(
(await listProjectedSessions({ projection, opts: { pinned: true } })).sessions.map(
(row) => row.key,
),
(await listRows(projection, { pinned: true })).sessions.map((row) => row.key),
).toEqual(sidebarRoot ? [first] : []);
}
replaceSessionEntrySync(
{ agentId: "main", sessionKey: first },
{ sessionId: first, updatedAt: 2, lastInteractionAt: 1, label: "Unpinned" },
);
writeMain(first, { updatedAt: 2, lastInteractionAt: 1, label: "Unpinned" });
expect((await list())[0]?.key).toBe(second);
expect(scan.mock.calls.filter(([query]) => !query?.key)).toHaveLength(0);
projection.onSelectionChange(() => {
throw new Error("Synthetic selection observer failure");
});
replaceSessionEntrySync(
{ agentId: "main", sessionKey: first },
{ sessionId: first, updatedAt: 5, label: "Changed despite observer failure" },
);
writeMain(first, { updatedAt: 5, label: "Changed despite observer failure" });
expect((await list())[0]).toMatchObject({
key: first,
label: "Changed despite observer failure",
});
expect(() => projection.dispose()).not.toThrow();
projection.dispose();
expect(projection.selectEntries()).toEqual([]);
} finally {
scan.mockRestore();
projection.dispose();
release();
}
@@ -336,10 +169,7 @@ it("refreshes agent owner facets when the sidebar roster changes", async () => {
{ owner: { type: "agent", id: "owner" }, assignedBy: { type: "system", id: "test" } },
);
const acpKey = "agent:main:acp-owner";
replaceSessionEntrySync(
{ agentId: "main", sessionKey: acpKey },
{ sessionId: "acp-owner", updatedAt: 2, archivedAt: 3 },
);
writeMain(acpKey, { sessionId: "acp-owner", updatedAt: 2, archivedAt: 3 });
assignSessionOwner(
{ agentId: "main", sessionKey: acpKey },
{ owner: { type: "agent", id: "transport" }, assignedBy: { type: "system", id: "test" } },
@@ -351,17 +181,14 @@ it("refreshes agent owner facets when the sidebar roster changes", async () => {
});
try {
const list = (scoped = true) =>
listProjectedSessions({
projection,
opts: {
agentId: scoped ? "main" : undefined,
configuredAgentsOnly: true,
includeGlobal: true,
includeUnknown: true,
rowMode: "sidebar",
limit: 200,
archived: "all",
},
listRows(projection, {
agentId: scoped ? "main" : undefined,
configuredAgentsOnly: true,
includeGlobal: true,
includeUnknown: true,
rowMode: "sidebar",
limit: 200,
archived: "all",
});
// Read the cold ACP row before a scoped read can populate its owner projection.
expect((await list(false)).owners?.map(({ id }) => id)).toEqual(["owner"]);
@@ -390,33 +217,16 @@ it("selects an exact row before pagination while retaining discovery filters", a
const key = "agent:main:target";
for (let index = 0; index < 7; index++) {
const sessionKey = index === 0 ? key : `${key}-${index}`;
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{ sessionId: `transcript-${index}`, updatedAt: index + 1 },
);
writeMain(sessionKey, { sessionId: `transcript-${index}`, updatedAt: index + 1 });
}
const projection = await createSessionRowProjection({ cfg });
try {
const page = await listProjectedSessions({
projection,
opts: { agentId: "main", search: key, limit: 5 },
});
expect(page.sessions.map((row) => row.key)).not.toContain(key);
const exact = await listProjectedSessions({
projection,
key,
opts: { agentId: "main", limit: 1 },
});
const page = await listRows(projection, { agentId: "main", search: key, limit: 5 });
expect(rowKeys(page)).not.toContain(key);
const exact = await listRows(projection, { agentId: "main", limit: 1 }, key);
expect(exact.sessions).toMatchObject([{ key, sessionId: "transcript-0" }]);
replaceSessionEntrySync(
{ agentId: "main", sessionKey: key },
{ sessionId: "transcript-0", updatedAt: 8, archivedAt: 8 },
);
const hidden = await listProjectedSessions({
projection,
key,
opts: { agentId: "main", limit: 1 },
});
writeMain(key, { sessionId: "transcript-0", updatedAt: 8, archivedAt: 8 });
const hidden = await listRows(projection, { agentId: "main", limit: 1 }, key);
expect(hidden.sessions).toEqual([]);
} finally {
projection.dispose();
@@ -424,259 +234,94 @@ it("selects an exact row before pagination while retaining discovery filters", a
});
});
it("reuses resident key predicates across list requests and refreshes legacy spawned classification", async () => {
it("filters active sentinels only after selecting their physical store winner", async () => {
await withOpenClawTestState({ scenario: "minimal" }, async () => {
const cfg = { agents: { entries: { main: {} } } };
const keys = [
"agent:main:legacy-visible",
"agent:main:cron:job:run:one",
"agent:main:subagent:child",
"agent:main:matrix:channel:!Room:example.org:thread:$Event",
"agent:main:signal:group:OpaqueGroup",
"agent:main:sessions",
const cfg = { agents: { entries: { main: {}, ops: {} } } };
const projection = createSessionRowProjectionFixture({ cfg, store: {} });
const samples = [
["shadow-global", "global", "main", "fallback"],
["main-global", "global", "main", "primary"],
["ops-global", "global", "ops", "primary"],
["ops-shadow", "global", "ops", "fallback"],
["unknown-winner", "unknown", "ops", "primary"],
["unknown-shadow", "unknown", "main", "fallback"],
] as const;
for (const sessionKey of keys) {
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{ sessionId: sessionKey, updatedAt: 1 },
const rows = samples.map(([sessionId, key, agentId, storePath]) => {
const entry = { sessionId, updatedAt: 1 };
return {
...createSessionRow({ key, agentId, storeTarget: { agentId, storePath } }, entry),
entry,
};
});
projection.selectEntries = (query) => {
if (!query?.sessionIdOrKey) {
return rows;
}
const keys = new Set(
rows
.filter(
(row) =>
row.key === query.sessionIdOrKey || row.entry.sessionId === query.sessionIdOrKey,
)
.map((row) => row.key),
);
}
const projection = await createSessionRowProjection({ cfg });
return rows.filter((row) => keys.has(row.key));
};
projection.state.scope = () => ({
paths: new Map([
["primary", 0],
["fallback", 1],
]),
path: "(multiple)",
agentId: undefined,
configuredAgentIds: new Set(["main", "ops"]),
ownerAgentIds: new Set(["main", "ops"]),
});
const registrations = await Promise.all(
[
["main", "shadow-global"],
["ops", "ops-global"],
["main", "unknown-shadow"],
].map(async ([agentId, sessionId]) => {
const registration = registerRpcSourceForTest({
storeScope: samples.find(([id]) => id === sessionId)![3],
runId: sessionId!,
agentId,
sessionId: sessionId!,
sessionKey: `agent:${agentId}:run-alias:${sessionId}`,
});
const release = await claimRpcSourceForTest(registration.entry!);
return Object.assign({}, registration, { release });
}),
);
try {
await projection.ensureMaterialized();
const opts = { agentId: "main", excludeSubagents: true, archived: "all" as const };
const expected = [keys[0], keys[3], keys[4], keys[5]].toSorted();
for (let iteration = 0; iteration < 2; iteration++) {
const prepared = prepareSessionRowSelection(projection, opts);
const cron = vi.spyOn(sessionKeys, "isCronRunSessionKey");
const subagent = vi.spyOn(sessionKeys, "isSubagentSessionKey");
try {
expect(
filterAndSortSessionEntries(prepared)
.map(([key]) => key)
.toSorted(),
).toEqual(expected);
expect(cron).not.toHaveBeenCalled();
expect(subagent).not.toHaveBeenCalled();
} finally {
cron.mockRestore();
subagent.mockRestore();
}
}
const sessionKey = keys[0];
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{
sessionId: sessionKey,
updatedAt: 2,
spawnedBy: "agent:main:parent",
archivedAt: 2,
const { filters } = prepareProjectedSessionList({
projection,
context: requestContext(cfg),
opts: {
activeOnly: true,
configuredAgentsOnly: true,
includeGlobal: true,
includeUnknown: true,
},
);
const hidden = await listProjectedSessions({ projection, opts });
expect(hidden.sessions.map((row) => row.key).toSorted()).toEqual(
expected.filter((key) => key !== sessionKey),
);
for (const category of ["Research", " "]) {
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{
sessionId: sessionKey,
updatedAt: 2,
spawnedBy: "agent:main:parent",
archivedAt: 2,
category,
},
);
const grouped = await listProjectedSessions({ projection, opts });
expect(grouped.sessions.map((row) => row.key).toSorted()).toEqual(
category.trim() ? expected : expected.filter((key) => key !== sessionKey),
);
expect(
(
await listProjectedSessions({ projection, opts: { ...opts, archived: false } })
).sessions.some((row) => row.key === sessionKey),
).toBe(false);
}
replaceSessionEntrySync(
{ agentId: "main", sessionKey },
{ sessionId: sessionKey, updatedAt: 3, archivedAt: 2 },
);
const restored = await listProjectedSessions({ projection, opts });
expect(restored.sessions.map((row) => row.key).toSorted()).toEqual(expected);
now: Date.now(),
metadataPrepared: true,
});
// An active fallback cannot displace its inactive winner; other agents remain visible.
expect(filterAndSortSessionEntries(filters).map(([, entry]) => entry.sessionId)).toEqual([
"ops-global",
"unknown-shadow",
]);
} finally {
for (const registration of registrations) {
registration.release();
registration.cleanup();
}
projection.dispose();
}
});
});
it.each([false, true])(
"preserves sentinel precedence, ties, and resident order before filtering (activeOnly: %s)",
async (activeOnly) => {
await withOpenClawTestState({ scenario: "minimal" }, async () => {
const cfg = { agents: { entries: { main: {}, ops: {} } } };
const projection = createSessionRowProjectionFixture({ cfg, store: {} });
const parent = "agent:main:parent";
const now = Date.now();
projection.state.rowContext.subagentRuns = buildSubagentRunReadIndexFromRuns({
now,
runs: new Map<string, SubagentRunRecord>([
[
"ordinary-run",
{
runId: "ordinary-run",
childSessionKey: "agent:main:ordinary",
requesterSessionKey: parent,
requesterDisplayKey: parent,
task: "Synthetic child selection",
cleanup: "keep",
createdAt: now,
execution: { status: "running", startedAt: now },
},
],
]),
});
const samples = [
["shadow-global", "global", "main", "fallback"],
["ordinary", "agent:main:ordinary", "main", "primary"],
["main-global", "global", "main", "primary"],
["ops-global", "global", "ops", "primary"],
["ops-shadow", "global", "ops", "fallback"],
["unknown-winner", "unknown", "ops", "primary"],
["unknown-shadow", "unknown", "main", "fallback"],
["excluded", "agent:main:ordinary", "main", "excluded"],
["retired", "agent:retired:ordinary", "retired", "primary"],
] as const;
const rows = samples.map(([sessionId, key, agentId, storePath]) => {
const entry = {
sessionId,
updatedAt: 1,
...(["ordinary", "shadow-global", "unknown-shadow"].includes(sessionId)
? { spawnedBy: parent }
: {}),
};
return {
...createSessionRow(
{
key,
agentId,
storeTarget: { agentId, storePath },
},
entry,
),
entry,
};
});
projection.selectEntries = (query) => {
if (query?.sessionIdOrKey) {
const keys = new Set(
rows
.filter(
(row) =>
row.key === query.sessionIdOrKey || row.entry.sessionId === query.sessionIdOrKey,
)
.map((row) => row.key),
);
return rows.filter((row) => keys.has(row.key));
}
return query?.parentSessionKey
? rows.filter((row) => row.entry.spawnedBy === query.parentSessionKey)
: rows;
};
projection.state.scope = () => ({
paths: new Map([
["primary", 0],
["fallback", 1],
]),
path: "(multiple)",
agentId: undefined,
configuredAgentIds: new Set(["main", "ops"]),
ownerAgentIds: new Set(["main", "ops"]),
});
try {
const prepared = prepareSessionRowSelection(projection, {
activeOnly,
configuredAgentsOnly: true,
includeGlobal: true,
includeUnknown: true,
});
expect(prepared.entries("all").map(([, entry]) => entry.sessionId)).toEqual(
activeOnly
? ["ordinary", "main-global", "ops-global", "unknown-winner", "unknown-shadow"]
: ["ordinary", "main-global", "unknown-winner"],
);
for (const [key, entry] of prepared.entries("all")) {
const target = prepared.getTarget(key)!;
const original = rows.find((row) => row.entry === entry)!;
expect(target.entry).toBe(entry);
expect(target.storeTarget).toBe(original.storeTarget);
expect(target.storeKey ?? key).toBe(original.key);
}
const visible = filterAndSortSessionEntries({
...prepared,
entryFilter: (_key, entry) => entry.sessionId !== "main-global",
});
expect(visible.map(([, entry]) => entry.sessionId)).toEqual(
expect.not.arrayContaining(["main-global", "shadow-global", "ops-shadow"]),
);
expect(rows.map((row) => row.entry.sessionId)).toEqual(samples.map(([id]) => id));
const scoped = filterAndSortSessionEntries({
...prepared,
opts: { ...prepared.opts, agentId: "ops" },
});
expect(scoped.map(([, entry]) => entry.sessionId)).toEqual(
activeOnly ? ["main-global", "ops-global", "unknown-winner"] : ["main-global"],
);
const children = filterAndSortSessionEntries(
prepareSessionRowSelection(projection, { ...prepared.opts, spawnedBy: parent }),
);
expect(children.map(([, entry]) => entry.sessionId)).toEqual(["ordinary"]);
if (activeOnly) {
const context = requestContext(cfg);
const registrations = await Promise.all(
[
["main", "shadow-global"],
["ops", "ops-global"],
["main", "unknown-shadow"],
].map(async ([agentId, sessionId]) => {
const registration = registerRpcSourceForTest({
storeScope: samples.find(([sampleId]) => sampleId === sessionId)![3],
runId: sessionId!,
agentId,
sessionId: sessionId!,
sessionKey: `agent:${agentId}:run-alias:${sessionId}`,
});
const release = await claimRpcSourceForTest(registration.entry!);
return Object.assign({}, registration, { release });
}),
);
try {
const { filters } = prepareProjectedSessionList({
projection,
context,
opts: prepared.opts,
now,
metadataPrepared: true,
});
// The live fallback ID cannot displace its inactive physical winner.
// Other agents' sentinel rows and same-ID aliases remain independently visible.
expect(
filterAndSortSessionEntries(filters).map(([, entry]) => entry.sessionId),
).toEqual(["ops-global", "unknown-shadow"]);
} finally {
for (const registration of registrations) {
registration.release();
registration.cleanup();
}
}
}
} finally {
projection.dispose();
}
});
},
);
it("rejects duplicate ordinary keys introduced after store admission before filtering or pagination", async () => {
await withOpenClawTestState({ scenario: "minimal" }, async (state) => {
const cfg = { agents: { entries: { main: {} } } };
@@ -701,53 +346,30 @@ it("rejects duplicate ordinary keys introduced after store admission before filt
const projection = await createSessionRowProjection({ cfg });
try {
const opts = { configuredAgentsOnly: true };
expect((await listProjectedSessions({ projection, opts })).sessions).toHaveLength(2);
expect((await listRows(projection, opts)).sessions).toHaveLength(2);
const duplicate = { agentId: "main", storePath: secondary, sessionKey: key };
replaceSessionEntrySync(duplicate, { sessionId: "duplicate", updatedAt: Date.now() + 1 });
await expect(
listProjectedSessions({ projection, opts: { ...opts, limit: 1, offset: 1 } }),
).rejects.toThrow("duplicate rows resolve to canonical session key");
await expect(
listProjectedSessions({ projection, opts: { ...opts, spawnedBy: parent } }),
).rejects.toThrow("duplicate rows resolve to canonical session key");
await expect(listRows(projection, { ...opts, limit: 1, offset: 1 })).rejects.toThrow(
"duplicate rows resolve to canonical session key",
);
await expect(listRows(projection, { ...opts, spawnedBy: parent })).rejects.toThrow(
"duplicate rows resolve to canonical session key",
);
await deleteSessionEntryLifecycle({
agentId: "main",
storePath: secondary,
archiveTranscript: false,
target: { canonicalKey: key, storeKeys: [key] },
});
const result = await listProjectedSessions({ projection, opts });
expect(result.sessions.map((row) => row.key).toSorted()).toEqual([key, "agent:main:other"]);
const result = await listRows(projection, opts);
expect(rowKeys(result).toSorted()).toEqual([key, "agent:main:other"]);
} finally {
projection.dispose();
}
});
});
it("keeps ungrouped visible spawned work discoverable", async () => {
const result = await listSessionFixture({
cfg: { agents: { entries: { main: {} } } },
storePath: "/tmp/openclaw-visible-session-activity",
store: {
"agent:main:subagent:hidden": {
sessionId: "hidden",
updatedAt: 3,
spawnedBy: "agent:main:discussion",
},
"agent:main:dashboard:visible": {
sessionId: "visible",
updatedAt: 2,
spawnedBy: "agent:main:discussion",
},
"agent:main:discussion": { sessionId: "parent", updatedAt: 1 },
},
opts: { excludeSubagents: true, limit: 1 },
});
expect(result.sessions.map((row) => row.key)).toEqual(["agent:main:dashboard:visible"]);
expect(result).toMatchObject({ totalCount: 2, nextOffset: 1, hasMore: true });
});
it("lists indexed children without inspecting unrelated resident ownership", async () => {
it("pages children by their current controller and retains persistent children", async () => {
await withOpenClawTestState(
{ scenario: "minimal", env: { OPENCLAW_TEST_READ_SUBAGENT_RUNS_FROM_SQLITE: "1" } },
async () => {
@@ -758,18 +380,6 @@ it("lists indexed children without inspecting unrelated resident ownership", asy
const parent = "agent:main:parent";
const other = "agent:main:other";
const key = (name: string) => `agent:main:${name}`;
const unrelated = 96;
const entries = new Map<string, SessionEntry>(
Array.from({ length: unrelated }, (_, index) => [
key(`unrelated-${index}`),
{
sessionId: `unrelated-${index}`,
updatedAt: now,
archivedAt: 1,
parentSessionKey: key(`unrelated-parent-${index}`),
},
]),
);
for (const [name, fields] of [
["spawned", { spawnedBy: parent }],
["dashboard:persistent", { parentSessionKey: parent, status: "done", endedAt: 1 }],
@@ -777,25 +387,13 @@ it("lists indexed children without inspecting unrelated resident ownership", asy
["moved", { spawnedBy: parent }],
["runtime", {}],
] as const) {
entries.set(key(name), { sessionId: name, updatedAt: now, ...fields });
writeMain(key(name), { sessionId: name, updatedAt: now, ...fields });
}
const seed = (agentId: string, rows: Iterable<[string, SessionEntry]>) =>
runOpenClawAgentWriteTransaction(
(database) => {
for (const [sessionKey, entry] of rows) {
writeSessionEntry(database, sessionKey, entry, {
canonicalPreviousEntry: null,
previousEntry: null,
});
}
},
{ agentId },
);
seed("main", entries);
const crossAgent = "agent:other:child";
seed("other", [
[crossAgent, { sessionId: "cross-agent", updatedAt: now, spawnedBy: parent }],
]);
replaceSessionEntrySync(
{ agentId: "other", sessionKey: crossAgent },
{ sessionId: "cross-agent", updatedAt: now, spawnedBy: parent },
);
const run = (name: string, controller: string): SubagentRunRecord => ({
runId: name,
childSessionKey: key(name),
@@ -819,39 +417,22 @@ it("lists indexed children without inspecting unrelated resident ownership", asy
saveSubagentRegistryToSqlite(runs);
const release = retainSessionListForegroundWork();
const projection = await createSessionRowProjection({ cfg, modelCatalog: [] });
const ownership = vi.spyOn(childOwners, "resolveSessionChildOwners");
try {
const opts = { spawnedBy: parent, archived: "all" as const, limit: 2 };
const list = () => listProjectedSessions({ projection, opts });
const list = () => listRows(projection, opts);
const first = await list();
expect(first.sessions.map((row) => row.key)).toEqual([
key("dashboard:persistent"),
key("runtime"),
]);
expect(rowKeys(first)).toEqual([key("dashboard:persistent"), key("runtime")]);
expect(first).toMatchObject({ totalCount: 4, nextOffset: 2, hasMore: true });
const second = await listProjectedSessions({ projection, opts: { ...opts, offset: 2 } });
expect(second.sessions.map((row) => row.key)).toEqual([key("spawned"), crossAgent]);
const second = await listRows(projection, { ...opts, offset: 2 });
expect(rowKeys(second)).toEqual([key("spawned"), crossAgent]);
expect(second).toMatchObject({ totalCount: 4, nextOffset: null, hasMore: false });
ownership.mockClear();
expect(await list()).toEqual(first);
const visited = new Set(ownership.mock.calls.map(([params]) => params.key));
ownership.mockRestore();
const sentinel = await listProjectedSessions({
projection,
opts: { ...opts, spawnedBy: "global" },
});
const sentinel = await listRows(projection, { ...opts, spawnedBy: "global" });
expect(sentinel.sessions).toEqual([]);
expect([...visited].filter((sessionKey) => sessionKey.includes("unrelated-")).length).toBe(
0,
);
runs.set(runtime.runId, { ...runtime, controllerSessionKey: other });
subagentRuns.set(runtime.runId, runs.get(runtime.runId)!);
persistRegistryFixture(runs, [runtime.runId]);
const moved = await list();
expect(moved.sessions.map((row) => row.key)).toEqual([
key("dashboard:persistent"),
key("spawned"),
]);
expect(rowKeys(moved)).toEqual([key("dashboard:persistent"), key("spawned")]);
expect(moved.totalCount).toBe(3);
runs.set(runtime.runId, {
...runtime,
@@ -862,7 +443,6 @@ it("lists indexed children without inspecting unrelated resident ownership", asy
persistRegistryFixture(runs, [runtime.runId]);
expect((await list()).totalCount).toBe(4);
} finally {
ownership.mockRestore();
projection.dispose();
runtimeExecution[Symbol.dispose]();
movedExecution[Symbol.dispose]();
src/gateway/sessions-history-http.test.ts
+6 -310
View File
@@ -3,9 +3,6 @@ import os from "node:os";
import path from "node:path";
import { expectDefined } from "@openclaw/normalization-core/expect";
import { afterEach, describe, expect, test, vi } from "vitest";
import { HEARTBEAT_PROMPT } from "../auto-reply/heartbeat.js";
import { replaceSessionEntry } from "../config/sessions/session-accessor.js";
import * as sessionEntryRows from "../config/sessions/session-accessor.sqlite-status.js";
import { replaceTranscriptEvents } from "../config/sessions/session-accessor.sqlite-transcript-write.test-support.js";
import { memorySessionActorOwners } from "../config/sessions/session-actor-memory-owner.js";
import { commitMemory } from "../config/sessions/session-actor-memory-publication.test-support.js";
@@ -14,15 +11,10 @@ import type { SessionActorStorageWrites } from "../config/sessions/session-actor
import { resolveSqliteTargetFromSessionStorePath } from "../config/sessions/session-sqlite-target.js";
import { executeSqliteQuerySync, getNodeSqliteKysely } from "../infra/kysely-sync.js";
import { emitSessionTranscriptUpdate } from "../sessions/transcript-events.js";
import { persistUserTurnTranscript } from "../sessions/user-turn-transcript.test-support.js";
import type { DB as OpenClawAgentKyselyDatabase } from "../state/openclaw-agent-db.generated.js";
import { runOpenClawAgentWriteTransaction } from "../state/openclaw-agent-db.js";
import { resolveIncognitoOpenClawAgentSqlitePath } from "../state/openclaw-agent-db.paths.js";
import { setAvatar, setDisplayName } from "../state/user-profile-writes.worker.js";
import { ensureProfileForEmail } from "../state/user-profiles.js";
import { resolveCurrentUserProfileDisplay } from "./current-user-profile-display.js";
import * as sessionHistoryState from "./session-history-state.js";
import { SessionHistorySseState } from "./session-history-state.js";
import {
appendHistoryMessage,
expectErrorResponse,
@@ -32,7 +24,6 @@ import {
openHistoryStream,
readSessionHistoryBody,
type SessionHistoryBody,
type SessionHistoryMessage,
type SessionHistorySseStream,
} from "./sessions-history-http-read.test-support.js";
import {
@@ -97,20 +88,6 @@ async function appendMessage(
return { ...appended, message };
}
function persistHistoryTurn(
storePath: string,
input: Parameters<typeof persistUserTurnTranscript>[0]["input"],
) {
return persistUserTurnTranscript({
agentId: AGENT_ID,
sessionEntry: { sessionId: "sess-main", updatedAt: 1 },
sessionId: "sess-main",
sessionKey: SESSION_KEY,
storePath,
input,
});
}
function replaceHistory(
storePath: string,
events: Parameters<typeof replaceTranscriptEvents>[1],
@@ -122,23 +99,6 @@ function replaceHistory(
);
}
const sessionHistoryRowIdentity = (message: SessionHistoryMessage) =>
`${message["__openclaw"]?.seq}:${message.role}:${message.content?.[0]?.text}`;
function withMockedDateNow<T>(now: number, run: () => T): T {
using _ = vi.spyOn(Date, "now").mockReturnValue(now);
return run();
}
function currentProfileAvatarUrl(profileId: string): string {
const display = resolveCurrentUserProfileDisplay(profileId);
expect(display.kind).toBe("resolved");
if (display.kind !== "resolved") {
throw new Error("expected a resolved current profile display");
}
return display.avatarUrl;
}
function withGatewayHistoryStream(
run: (stream: SessionHistorySseStream, port: number) => Promise<void>,
query?: string,
@@ -243,30 +203,19 @@ describe("session history HTTP endpoints", () => {
await withGatewayHarness(async (harness) => {
const cases = {
"text/event-stream": [
"text/event-stream",
"TEXT/EVENT-STREAM",
" text/event-stream ",
"text/event-stream;",
"text/event-stream; charset=utf-8",
"text/event-stream;q=0.001",
"text/event-stream;Q=1.000",
" TEXT/EVENT-STREAM;Q=1.000;charset=UTF-8; ",
"text/event-stream;q=0, text/event-stream;q=0.5",
"text/event-stream;q=0, text/event-stream;charset=utf-8;q=0.5",
"text/event-stream;q=0.5;charset=utf-8",
],
"application/json": [
'text/event-stream; note="quoted,comma;semicolon\\\"quote"; q=0.5',
"text/event-stream;q=1, text/event-stream;charset=utf-8;q=0",
"text/event-stream;q=1;charset=utf-16",
"text/event-stream;charset=utf-16",
"text/event-streaming",
"text/event-streamx",
'application/json; note="text/event-stream"',
"text/*",
"*/*",
"text/event-stream;q=0",
"text/event-stream;q=0, */*;q=1",
"text/event-stream;q=0.1234",
"text/event-stream;q =0.5",
"text/event-stream;q= 0.5",
"text/event-stream;\u00a0q=0.5",
@@ -293,167 +242,14 @@ describe("session history HTTP endpoints", () => {
});
});
test("reads only the selected history entry for default and blank cursor queries", async () => {
const { storePath } = await seedSession({ text: "hello from history" });
const unrelatedPrompt = "Unrelated saved session prompt";
await replaceSessionEntry(
{ agentId: AGENT_ID, sessionKey: "agent:main:unrelated", storePath },
{
sessionId: "sess-unrelated",
updatedAt: 1,
skillsSnapshot: { prompt: unrelatedPrompt, skills: [] },
},
);
test("treats blank cursor queries as default history", async () => {
await seedSession({ text: "hello from history" });
await withGatewayHarness(async (harness) => {
using decode = vi.spyOn(sessionEntryRows, "parseSessionEntryJson");
for (const query of ["", "?cursor=", "?cursor=%20"]) {
decode.mockClear();
const context = `query ${JSON.stringify(query)}`;
const res = await fetchSessionHistory(harness.port, SESSION_KEY, { query });
expect(res.status, context).toBe(200);
const body = (await res.json()) as SessionHistoryBody;
expect(body, context).toMatchObject({
expect(await readSessionHistoryBody(harness.port, SESSION_KEY, { query })).toMatchObject({
sessionKey: SESSION_KEY,
messages: [{ content: [{ text: "hello from history" }], __openclaw: { seq: 1 } }],
});
expect(
decode.mock.calls.some(([row]) => row.entry_json.includes(unrelatedPrompt)),
context,
).toBe(false);
}
});
});
test("shares revisioned current-profile projection across REST and initial and inline SSE", async () => {
const { storePath } = await seedSession();
const profile = withMockedDateNow(1_800_000_000_000, () => {
const created = ensureProfileForEmail("session-history-profile@example.com");
setDisplayName(created.id, "Old Display Name");
expect(setAvatar(created.id, new Uint8Array([1, 2, 3]), "image/png").ok).toBe(true);
return created;
});
const oldAvatarUrl = currentProfileAvatarUrl(profile.id);
const persistAttributedTurn = async (id: string, senderName: string, text: string) => {
const turn = await persistHistoryTurn(storePath, {
idempotencyKey: `session-history-profile:${id}`,
sender: {
id: profile.id,
identity: { type: "profile", id: profile.id },
name: senderName,
username: "ada",
},
text,
});
expect(turn).toBeDefined();
return turn!;
};
const first = await persistAttributedTurn("first", "Historical Ada", "first turn");
const turn = (id: string, seq: number, name: string, text: string, avatarUrl: string) => ({
role: "user",
content: text,
__openclaw: {
id,
seq,
senderId: profile.id,
senderName: name,
senderUsername: "ada",
senderProfileAvatarUrl: avatarUrl,
},
});
await withGatewayHarness(async (harness) => {
const initialRest = await readSessionHistoryBody(harness.port, SESSION_KEY);
await using stream = await openHistoryStream(harness.port, SESSION_KEY);
const initialSse = await stream.next();
const oldExpected = turn(first.messageId, 1, "Historical Ada", "first turn", oldAvatarUrl);
expect(initialRest.messages?.[0]).toMatchObject(oldExpected);
expect(initialSse.messages?.[0]).toMatchObject(oldExpected);
withMockedDateNow(1_900_000_000_000, () => {
setDisplayName(profile.id, "Current Ada");
expect(setAvatar(profile.id, new Uint8Array([4, 5, 6]), "image/png").ok).toBe(true);
});
const newAvatarUrl = currentProfileAvatarUrl(profile.id);
expect(newAvatarUrl).not.toBe(oldAvatarUrl);
const inlineEventPromise = stream.next();
const second = await persistAttributedTurn("second", "Current Ada", "second turn");
const refreshEvent = await inlineEventPromise;
const expected = [
turn(first.messageId, 1, "Historical Ada", "first turn", newAvatarUrl),
turn(second.messageId, 2, "Current Ada", "second turn", newAvatarUrl),
];
for (const body of [refreshEvent, await readSessionHistoryBody(harness.port, SESSION_KEY)]) {
expect(body.messages).toMatchObject(expected);
}
});
});
test("attributes forwarded history only from structured provenance across transports and pages", async () => {
const { storePath } = await seedSession();
const sessionKey = SESSION_KEY;
const cases = [
{
body: "Verified sender body\n indented line",
sourceSessionKey: "agent:helper:ops",
senderLabel: "Forwarded from helper",
senderSession: { sessionKey: "agent:helper:ops", agentId: "helper" },
},
{
body: "Unverified sender body\n indented line",
sourceSessionKey: undefined,
senderLabel: "Forwarded agent message",
senderSession: undefined,
},
];
for (const entry of cases) {
const persisted = await persistHistoryTurn(storePath, {
text: `[Inter-session message] sourceSession=agent:grimwald:asserted sourceTool=sessions_send isUser=false\n${entry.body}`,
provenance: {
kind: "inter_session",
sourceTool: "sessions_send",
...(entry.sourceSessionKey ? { sourceSessionKey: entry.sourceSessionKey } : {}),
},
});
expect(persisted).toBeDefined();
}
await withGatewayHarness(async (harness) => {
const ws = await harness.openWs();
try {
expect((await connectReq(ws, { scopes: ["operator.read"] })).ok).toBe(true);
let cursor: string | undefined;
for (const [offset, entry] of cases.toReversed().entries()) {
const http = await readSessionHistoryBody(harness.port, sessionKey, {
query: `?limit=1${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ""}`,
});
const websocket = await rpcReq<{ messages: unknown[]; hasMore: boolean }>(
ws,
"chat.history",
{ sessionKey, limit: 1, offset },
);
expect(websocket.ok).toBe(true);
for (const page of [http, websocket.payload]) {
expect(page?.messages).toHaveLength(1);
expect(page?.messages?.[0]).toMatchObject({
role: "assistant",
content: entry.body,
senderLabel: entry.senderLabel,
...(entry.senderSession ? { senderSession: entry.senderSession } : {}),
});
if (!entry.senderSession) {
expect(page?.messages?.[0]).not.toHaveProperty("senderSession");
}
expect(page?.hasMore).toBe(offset < cases.length - 1);
}
if (offset < cases.length - 1) {
expect(http.nextCursor).toEqual(expect.any(String));
}
cursor = http.nextCursor;
}
} finally {
ws.close();
}
});
});
@@ -595,53 +391,6 @@ describe("session history HTTP endpoints", () => {
);
});
test("keeps repeated assistant replies from separate hidden user turns in REST and SSE history", async () => {
const sessionId = "sess-hidden-turn-replies";
const sessionKey = SESSION_KEY;
const { storePath } = await seedSession({ sessionId });
const timestamp = Date.UTC(2026, 7, 15, 9, 30, 0);
const assistantMessage = (text: string, model: string) => ({
...makeTranscriptAssistantMessage({ text, provider: "openclaw", model }),
timestamp,
});
const replies = ["First reply.", "Second reply.", "Third reply."];
await replaceHistory(
storePath,
replies.flatMap((text, index) => [
{ message: assistantMessage(text, "acp-runtime") },
{ message: { role: "user", content: index ? HEARTBEAT_PROMPT : "" } },
...(index === 2 ? [{ message: { role: "assistant", content: "HEARTBEAT_OK" } }] : []),
{ message: assistantMessage(text, "gateway-injected") },
]),
sessionId,
);
const expectedRows = [
[1, 3],
[4, 6],
[7, 10],
].flatMap((sequences, index) => sequences.map((seq) => `${seq}:assistant:${replies[index]}`));
await withGatewayHarness(async (harness) => {
const history = await readSessionHistoryBody(harness.port, sessionKey);
expect(history.messages?.map(sessionHistoryRowIdentity)).toEqual(expectedRows);
expect(history.messages?.map((message) => message.timestamp)).toEqual(
Array.from({ length: 6 }, () => timestamp),
);
expect(history.messages?.map((message) => message["__openclaw"]?.turnBoundary)).toEqual([
undefined,
undefined,
undefined,
true,
undefined,
true,
]);
await using stream = await openHistoryStream(harness.port, SESSION_KEY);
const streamedHistory = await stream.next();
expect(streamedHistory.messages?.map(sessionHistoryRowIdentity)).toEqual(expectedRows);
});
});
test("caps all-digit direct REST history limits that exceed safe integer range", async () => {
const { storePath } = await seedSession({ text: "first message" });
await appendText(storePath, "second message");
@@ -689,48 +438,6 @@ describe("session history HTTP endpoints", () => {
});
});
test("sanitizes phased output_text assistant history before returning it", async () => {
const { storePath } = await seedSession();
await withGatewayHarness(async (harness) => {
const visibleMessageId = "visible-phased-assistant";
await replaceHistory(storePath, [
{ id: "hidden-control", message: makeTranscriptAssistantMessage({ text: "NO_REPLY" }) },
{
id: visibleMessageId,
message: {
...makeTranscriptAssistantMessage({ text: "Done." }),
content: [
{ id: "item_commentary", phase: "commentary", text: "internal reasoning" },
{ id: "item_final", phase: "final_answer", text: "Done." },
].map(({ text, ...signature }) => ({
type: "output_text",
text,
textSignature: JSON.stringify({ v: 1, ...signature }),
})),
},
},
]);
const body = await readSessionHistoryBody(harness.port, SESSION_KEY);
expect(body.sessionKey).toBe(SESSION_KEY);
expect(body.messages).toHaveLength(2);
expect(body.messages?.[0]).toMatchObject({
content: [{ type: "text", text: "internal reasoning" }],
openclawStreamFallback: {
itemId: "item_commentary",
replacementText: "internal reasoning",
source: "segment",
},
});
expect(body.messages?.[1]?.content?.map((block) => block.text)).toEqual(["Done."]);
expect(body.messages?.[1]?.["__openclaw"]).toMatchObject({
id: visibleMessageId,
seq: 2,
});
});
});
test("includes updates committed while opening bounded SSE history", async () => {
const { storePath } = await seedSession({ text: "first message" });
@@ -750,9 +457,8 @@ describe("session history HTTP endpoints", () => {
});
});
test("bounds retained SSE history without truncating full history or live updates", async () => {
const retainedLimit = 1_000;
const initialCount = retainedLimit + 2;
test("serves complete history and live updates beyond the SSE retention window", async () => {
const initialCount = 1_002;
const { storePath } = await seedSession();
await replaceHistory(
storePath,
@@ -763,7 +469,6 @@ describe("session history HTTP endpoints", () => {
})),
);
using snapshotSpy = vi.spyOn(SessionHistorySseState.prototype, "snapshot");
const expectWindow = (
body: SessionHistoryBody | undefined,
length: number,
@@ -778,31 +483,22 @@ describe("session history HTTP endpoints", () => {
expect(body?.messages?.at(-1)?.content?.[0]?.text).toBe(last);
}
};
const retainedSnapshot = () => snapshotSpy.mock.results.at(-1)?.value;
await withGatewayHistoryStream(async (stream, port) => {
const initial = await stream.next();
expectWindow(initial, initialCount, "history message 1");
expectWindow(retainedSnapshot(), retainedLimit, "history message 3");
await using cursorStream = await openHistoryStream(
port,
SESSION_KEY,
`?cursor=${initialCount + 1}`,
);
expectWindow(await cursorStream.next(), initialCount);
expectWindow(retainedSnapshot(), retainedLimit, "history message 3");
const messageId = await appendText(storePath, "live history message");
const lastSequence = initial.messages?.at(-1)?.["__openclaw"]?.seq;
expect(lastSequence).toEqual(expect.any(Number));
await stream.expectMessage("live history message", (lastSequence ?? 0) + 1, messageId);
const liveRetained = snapshotSpy.mock.results.findLast((result) => {
const snapshot = result.value;
return snapshot?.messages?.at(-1)?.content?.[0]?.text === "live history message";
})?.value;
expectWindow(liveRetained, retainedLimit, undefined, "live history message");
expectWindow(await cursorStream.next(), initialCount, "history message 1");
expectWindow(retainedSnapshot(), retainedLimit);
const completeHistory = await vi.waitFor(() => readSessionHistoryBody(port, SESSION_KEY), {
interval: 25,
src/gateway/worker-environments/worker-turn-launcher-terminal-results.test.ts
+9 -20
View File
@@ -143,19 +143,13 @@ describe("worker turn launcher terminal results", () => {
providerFailure?: true;
}>([
{ stopReason: "stop", reconciliationFails: false },
{ stopReason: "error", reconciliationFails: false },
{ stopReason: "stop", reconciliationFails: true },
{ stopReason: "error", reconciliationFails: true },
{ stopReason: "error", reconciliationFails: false, providerFailure: true },
...[
"Browser cleanup timed out",
"Browser cleanup failed: rate limit exceeded",
"Browser disposal failed",
].map((cleanupFailure) => ({
stopReason: "stop" as const,
{
stopReason: "stop",
reconciliationFails: false,
cleanupFailure,
})),
cleanupFailure: "Browser cleanup failed: rate limit exceeded",
},
])(
"retains the ACKed finishing outcome after assistant $stopReason (reconciliation fails: $reconciliationFails; cleanup: $cleanupFailure; provider fallback: $providerFailure)",
async ({ stopReason, reconciliationFails, cleanupFailure, providerFailure }) => {
@@ -290,7 +284,6 @@ describe("worker turn launcher terminal results", () => {
}),
);
}),
syncWorkspace: vi.fn(),
reconcileWorkspace: vi.fn(async (request) => {
if (reconciliationFails) {
throw reconciliationError;
@@ -322,7 +315,7 @@ describe("worker turn launcher terminal results", () => {
const uninstall = installSessionPlacementAdmissionProvider(provider);
try {
const runLocal = vi.fn(localResult);
const execute = vi.fn(() => provider.executeTurn(runIdentity(runId), workerTurn, runLocal));
const execute = () => provider.executeTurn(runIdentity(runId), workerTurn, runLocal);
const config = {
...workerTurn.config,
agents: {
@@ -336,7 +329,7 @@ describe("worker turn launcher terminal results", () => {
};
let mediaTasks = getGeneratedMediaTaskIdsForSessionKey(fx.SESSION_KEY);
const candidateFailures: unknown[] = [];
const runCandidate = vi.fn((candidateProvider: string, model: string) => {
const runCandidate = (candidateProvider: string, model: string) => {
mediaTasks = getGeneratedMediaTaskIdsForSessionKey(fx.SESSION_KEY);
return runEmbeddedAgent({
...workerTurn,
@@ -348,7 +341,7 @@ describe("worker turn launcher terminal results", () => {
candidateFailures.push(error);
throw error;
});
});
};
const runOuterEntry = () =>
runEmbeddedAgentEntry({
selection: {
@@ -411,7 +404,6 @@ describe("worker turn launcher terminal results", () => {
await expectSettled();
if (providerFailure) {
expect(launchedModels).toEqual(["gpt-5.6-luna", "gpt-5.6-sol"]);
expect(runCandidate).toHaveBeenCalledTimes(2);
expect(observed).toMatchObject({ message: "Unexpected second worker execution" });
await expect(fs.stat(effectFile)).rejects.toMatchObject({ code: "ENOENT" });
return;
@@ -420,14 +412,13 @@ describe("worker turn launcher terminal results", () => {
.soft(launchedModels, "cleanup must not replay the committed worker effect")
.toEqual(["gpt-5.6-luna"]);
expect.soft(await fs.readFile(effectFile, "utf8")).toBe("effect\n");
expect.soft(runCandidate).toHaveBeenCalledOnce();
expect.soft(observed).toBe(candidateFailures[0]);
return;
}
expect(observed).toMatchObject({ message: expect.stringContaining(failure) });
if (reconciliationFails) {
expect(observed).toMatchObject({ cause: expect.any(WorkerWorkspaceReconciliationError) });
expect(observed).toMatchObject({ cause: { cause: reconciliationError } });
expect(observed).toMatchObject({ cause: expect.any(WorkerWorkspaceReconciliationError) });
expect(resolveModelFallbackError(observed)).toEqual({
kind: "coordination",
error: observed,
@@ -437,8 +428,7 @@ describe("worker turn launcher terminal results", () => {
expect(observed).toMatchObject({ message: failure });
expect(reconcileActivePlacement).not.toHaveBeenCalled();
}
expect(execute).toHaveBeenCalledOnce();
expect(tunnel.launchTurn).toHaveBeenCalledOnce();
expect(launchedModels).toHaveLength(1);
expect(runLocal).not.toHaveBeenCalled();
await expectSettled();
await expect(
@@ -764,7 +754,6 @@ describe("worker turn launcher terminal results", () => {
await completed.appendMessageAsync(
makeAgentAssistantMessage({
content: [{ type: "toolCall", id: "call-usage", name: "read", arguments: {} }],
provider: "openai",
model: "gpt-first-call",
stopReason: "toolUse",
timestamp: 21,
src/infra/agent-events.test.ts
+29 -54
View File
@@ -213,32 +213,27 @@ test("defers clearing an existing context until all overlapping owners exit", ()
});
test("reserves exclusive run ids for owner-only delivery and cleanup", () => {
const claimId = claimRun("run", { sessionKey: "worker" }, { exclusive: true, trackOwner: true })!;
expect(claimRun("run", { sessionKey: "local" }, { trackOwner: true })).toBeUndefined();
const seen: unknown[] = [];
onAgentEventForRun("run", ({ data }) => seen.push(data.text));
const event = (text: string) => ({ runId: "run", stream: "assistant", data: { text } });
emitAgentEventIfCurrent(event("local"));
emitAgentEventForOwner(event("worker"), claimId);
clearRun("run", readGeneration());
expect(getRun("run")?.sessionKey).toBe("worker");
releaseRun("run", claimId);
expect(getRun("run")).toBeUndefined();
emitAgentEventForOwner(event("late"), claimId);
expect(seen).toEqual(["worker"]);
});
test("drops stale explicit-generation events before shared listeners", () => {
const lifecycleGeneration = readGeneration();
registerRun("run", { sessionKey: "main", lifecycleGeneration });
const seen = captureEvents();
expect(
sendEvent("run", "lifecycle", { phase: "end" }, { lifecycleGeneration: "pre-restart" }),
).toBe(false);
expect(
sendEvent("run", "lifecycle", { phase: "start", startedAt: 1_000 }, { lifecycleGeneration }),
).toBe(true);
expect(seen).toMatchObject([{ seq: 1, data: { phase: "start" } }]);
withGeneration(readGeneration(), () => {
const claimId = claimRun(
"run",
{ sessionKey: "worker" },
{ exclusive: true, trackOwner: true },
)!;
expect(claimRun("run", { sessionKey: "local" }, { trackOwner: true })).toBeUndefined();
const seen: unknown[] = [];
onAgentEventForRun("run", ({ data }) => seen.push(data.text));
const event = (text: string) => ({ runId: "run", stream: "assistant", data: { text } });
emitAgentEventIfCurrent(event("local"));
emitAgentEventForOwner(event("worker"), claimId);
clearRun("run", readGeneration());
expect(getRun("run")?.sessionKey).toBe("worker");
releaseRun("run", claimId);
expect(getRun("run")).toBeUndefined();
emitAgentEventForOwner(event("late"), claimId);
expect(seen).toEqual(["worker"]);
rotateGeneration();
expect(emitAgentEventIfCurrent(event("rotated"))).toBe(false);
});
});
test("rejects inherited stale ownership and cannot reclaim the admitted replacement", () => {
@@ -847,28 +842,6 @@ describe("agent event routing after cancellation", () => {
expect(events[2]?.sessionKey).toBeUndefined();
});
it("never lets routing provenance revive a released claim or rotated execution", () => {
const generation = readGeneration();
const events = captureEvents(onAgentRuntimeEvent);
withGeneration(generation, () => {
const claim = claimRun(
"owned",
{ agentId: "delivery", sessionKey: "agent:delivery:owned" },
{ exclusive: true, trackOwner: true, ownsContext: true },
);
expect(claim).toBeDefined();
const event = { runId: "owned", stream: "tool", data: { phase: "result" } };
emitAgentEventForOwner(event, claim!);
clearRun("owned", generation, claim);
releaseRun("owned", claim);
expect(getRun("owned")).toBeUndefined();
emitAgentEventForOwner(event, claim!);
rotateGeneration();
expect(emitAgentEventIfCurrent(event)).toBe(false);
});
expect(events).toHaveLength(1);
});
it("extends a retained v2026.9.4 scope without replacing its execution owner", async () => {
const lifecycleGeneration = readGeneration();
// Released updaters can retain this exact shape while loading current runtime aliases.
@@ -1001,12 +974,14 @@ describe("live agent model projection", () => {
rotateGeneration();
clearRun(runId, generation);
registerRun(runId, { sessionKey: "agent:main:replacement" });
emitAgentEvent({
runId,
lifecycleGeneration: generation,
stream: "lifecycle",
data: { phase: "model", provider: "stale", model: "stale" },
});
expect(
emitAgentEventIfCurrent({
runId,
lifecycleGeneration: generation,
stream: "lifecycle",
data: { phase: "model", provider: "stale", model: "stale" },
}),
).toBe(false);
expect(getRun(runId)).not.toHaveProperty("activeModel");
});
src/infra/git-worker.lifecycle.test.ts
-29
View File
@@ -13,16 +13,10 @@ import * as worktreeGit from "../agents/worktrees/git.js";
import { runPublicationCommand } from "../gateway/github-publication-git-transport.js";
import { runWorkspaceInventoryCommandToFile } from "../gateway/worker-environments/workspace-sync-inventory.js";
import { ensureStagedInputDirectory, stagedInputDirectory } from "../media/staged-inputs.js";
import { emitChildProcessSpawnSample } from "../process/spawn-diagnostics.js";
import { createDeferredCore } from "../shared/deferred.js";
import { drainGlobalSingletonLifecycleState } from "../shared/global-singleton.js";
import { isPidAlive } from "../shared/pid-alive.js";
import { killPidIfAlive } from "../test-utils/process-tree.js";
import {
onDiagnosticEvent,
setDiagnosticsEnabledForProcess,
type DiagnosticEventPayload,
} from "./diagnostic-events.js";
import * as gitExec from "./git-exec.js";
import { installUnknownDirentFixture } from "./git-worker-dir.test-support.js";
import { runGitWorkerOperation, type GitWorkerOperationOptions } from "./git-worker.js";
@@ -431,18 +425,6 @@ describe("Git operation host lifecycle", () => {
const peerRoot = path.join(root, "peer");
await fs.mkdir(peerRoot);
const peer = await repository(peerRoot);
let now = 0;
vi.spyOn(performance, "now").mockImplementation(() => now);
setDiagnosticsEnabledForProcess(false);
emitChildProcessSpawnSample();
setDiagnosticsEnabledForProcess(true);
const spawns: Extract<DiagnosticEventPayload, { type: "diagnostic.child_process.spawn" }>[] =
[];
const stop = onDiagnosticEvent((event) => {
if (event.type === "diagnostic.child_process.spawn") {
spawns.push(event);
}
});
const entered = createDeferredCore();
const release = createDeferredCore();
const realRun = worktreeGit.runGitBytes;
@@ -489,18 +471,7 @@ describe("Git operation host lifecycle", () => {
} finally {
release.resolve();
await Promise.all(pending);
now = 60_000;
emitChildProcessSpawnSample();
stop();
setDiagnosticsEnabledForProcess(false);
emitChildProcessSpawnSample();
}
expect(
spawns
.map(({ operation }) => operation ?? "unknown")
.toSorted((left, right) => left.localeCompare(right)),
).toEqual(["checkout.diff", "repository.branches"]);
expect(spawns.every(({ family, count }) => family === "git" && count > 0)).toBe(true);
});
it.each(["cleanup-inspection", "snapshot"] as const)(
src/infra/state-migrations.shared-auth-store.test.ts
+28 -105
View File
@@ -8,7 +8,6 @@ import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import { resolveAgentDir } from "../agents/agent-scope-config.js";
import * as ownership from "../agents/auth-profiles/path-resolve.js";
import { prepareSharedAuthStoreOwnershipForNative } from "../agents/auth-profiles/path-resolve.native.js";
import * as persisted from "../agents/auth-profiles/persisted.js";
import { setAuthProfileOrder } from "../agents/auth-profiles/profiles.js";
import * as paths from "../agents/auth-profiles/shared-main-dir.js";
import * as bootstrap from "../agents/auth-profiles/shared-store-bootstrap.js";
@@ -18,7 +17,6 @@ import {
readSharedAuthProfileRows,
} from "../agents/auth-profiles/sqlite-read.js";
import * as sqlite from "../agents/auth-profiles/sqlite.js";
import * as authState from "../agents/auth-profiles/state.js";
import {
loadAuthProfileStoreWithoutExternalProfiles,
saveAuthProfileStore,
@@ -101,9 +99,6 @@ function receiptRow(db: EngineConnection) {
"SELECT source_sha256, source_record_count, status, removed_source FROM migration_sources WHERE target_table = 'auth_profile_stores'",
);
}
function count(db: EngineConnection, table: string, key: string, value: string) {
return row(db, `SELECT COUNT(*) AS count FROM ${table} WHERE ${key} = ?`, value)?.count;
}
function completedReceipt(source: unknown) {
return {
source_sha256: createHash("sha256").update(JSON.stringify(source)).digest("hex"),
@@ -321,43 +316,13 @@ describe("shared auth relocation lifecycle", () => {
`);
}
const sourceChecks = new Map<DatabaseSync, string[]>();
// oxlint-disable-next-line typescript/unbound-method -- Invoked with the intercepted native database receiver.
const prepare = DatabaseSync.prototype.prepare;
vi.spyOn(DatabaseSync.prototype, "prepare").mockImplementation(function (
this: DatabaseSync,
sql,
) {
const statement = prepare.call(this, sql);
if (this.location() !== sourcePath || !/^PRAGMA (?:quick|integrity)_check\b/u.test(sql)) {
return statement;
}
const all = statement.all.bind(statement);
statement.all = () => {
const checks = sourceChecks.get(this) ?? [];
checks.push(sql);
sourceChecks.set(this, checks);
return all();
};
return statement;
});
const result = await migrate(sourcePath, integrityCheck);
if (integrityCheck === "full") {
expect([...sourceChecks.values()]).toEqual([["PRAGMA integrity_check;"]]);
expect(result).toMatchObject({
changes: [],
warnings: [expect.stringMatching(/integrity_check.*idx_agent_cache_updated/su)],
});
} else {
expect([...sourceChecks.values()]).toEqual([
["PRAGMA quick_check;"],
[
"PRAGMA quick_check;",
"PRAGMA integrity_check('auth_profile_store');",
"PRAGMA integrity_check('auth_profile_state');",
],
]);
expect(result.warnings).toEqual([]);
expect(
await readSharedAuthProfileRows(captureOpenClawStateWorkerContext({ env: state.env })),
@@ -481,10 +446,8 @@ it.runIf(process.platform === "linux")(
},
);
it.each([false, true])("records ownership only (source=%s)", async (hasSource) => {
const fixture = await createEmptyFixture(hasSource);
expect(fs.existsSync(fixture.sourcePath)).toBe(hasSource);
it("records ownership for an empty source database without reporting relocation", async () => {
const fixture = await createEmptyFixture(true);
const detected = fixture.detect();
expect(detected.hasLegacy).toBe(true);
expect(await fixture.migrate(detected)).toEqual(NO_CHANGES);
@@ -493,65 +456,26 @@ it.each([false, true])("records ownership only (source=%s)", async (hasSource) =
});
});
it("moves exact rows, preserves every effective agent store, and records receipts", async () => {
it("preserves effective main and private-agent auth across relocation", async () => {
const fixture = await createFixture();
const effectiveBytes = (agentDir: string) => {
const effective = loadAuthProfileStoreWithoutExternalProfiles(agentDir);
return JSON.stringify({
credentials: persisted.buildPersistedAuthProfileSecretsStore(effective),
state: authState.buildPersistedAuthProfileState(effective),
const effective = () =>
[fixture.mainAgentDir, fixture.opsAgentDir].map((agentDir) => {
const { profiles, order, lastGood } = loadAuthProfileStoreWithoutExternalProfiles(agentDir);
return { profiles, order, lastGood };
});
};
const before = {
main: effectiveBytes(fixture.mainAgentDir),
ops: effectiveBytes(fixture.opsAgentDir),
};
const detected = fixture.detect();
expect(await fixture.migrate(detected)).toMatchObject({
const before = effective();
expect(await fixture.migrate(fixture.detect())).toMatchObject({
warnings: [],
changes: [expect.stringContaining("Relocated shared auth")],
});
expect({
store: sqlite.readPersistedAuthProfileStoreRaw(),
state: sqlite.readPersistedAuthProfileStateRaw(),
local: localAuth(fixture.mainAgentDir),
}).toEqual({
store: fixture.sharedStore,
state: fixture.sharedState,
local: { store: null, state: null },
});
expect({
main: effectiveBytes(fixture.mainAgentDir),
ops: effectiveBytes(fixture.opsAgentDir),
}).toEqual(before);
const database = stateDb.openOpenClawStateDatabase({ env: fixture.env }).db;
for (const [key, value] of [
["authProfiles.store", fixture.sharedStore],
["authProfiles.state", fixture.sharedState],
["auth.sharedStore", { location: "state-db" }],
] as const) {
expect(machineRow(database, key)).toEqual({ value_json: JSON.stringify(value) });
}
expect(
row(
database,
"SELECT COUNT(*) AS count FROM migration_sources WHERE migration_kind = ?",
"shared-auth-store-state-db",
),
).toEqual({ count: 2 });
expect(effective()).toEqual(before);
});
it.each(["standard", "custom", "main override"] as const)(
it.each(["custom", "main override"] as const)(
"relocates the configured inheritance owner through Doctor (%s)",
async (layout) => {
const fixture = await createEmptyFixture(false);
const ownerDir = path.join(
fixture.stateDir,
layout === "custom" ? "custom-owner" : "agents/ops/agent",
);
const ownerDir = path.join(fixture.stateDir, "custom-owner");
const peerDir = path.join(fixture.stateDir, "agents/dev/agent");
const config: OpenClawConfig = {
agents: {
@@ -755,11 +679,6 @@ it.each([
expect(result.warnings).toEqual(converges ? [] : [expect.stringMatching(/conflict.*Back up/)]);
if (!converges) {
expect(result.warnings[0]).toContain(conflict);
expect(result.warnings[0]).toContain("openclaw doctor --fix");
expect(result.warnings[0]).toContain(JSON.stringify(sourcePath));
expect(result.warnings[0]).toContain(
JSON.stringify(path.join(fixture.stateDir, "state", "openclaw.sqlite")),
);
expect(result.warnings[0]).not.toMatch(/shared-key|extra-key|different-key|legacyMetadata/);
}
expect(machineRow(migratedTarget, "authProfiles.store", true)).toEqual(targetRow);
@@ -769,6 +688,13 @@ it.each([
expect(fixture.detect().hasLegacy).toBe(!converges);
if (scenario === "changed credential") {
const warning = result.warnings[0]!;
for (const guidance of [
"openclaw doctor --fix",
sourcePath,
path.join(fixture.stateDir, "state", "openclaw.sqlite"),
]) {
expect(warning).toContain(guidance);
}
expect(warning).toContain(`${JSON.stringify(escapedProfileId)}: missing from target`);
expect(warning.indexOf(JSON.stringify(escapedProfileId))).toBeLessThan(
warning.indexOf('"openai:shared"'),
@@ -834,14 +760,10 @@ it.each(["absolute", "tilde"] as const)(
},
{ env: source.env },
);
expect(localAuth(source.agentDir)).toEqual({ store: source.store, state: source.state });
}
const detected = await detectLegacy(selected.env, () => selected.owner.home);
expect(detected).toMatchObject({
stateDir: selected.owner.stateDir,
stateSchema: { hasLegacy: false },
});
expect(detected.stateDir).toBe(selected.owner.stateDir);
const selectedSourcePath = sqlite.resolveAuthProfileDatabasePath(selected.agentDir);
expect
.soft(detected.sharedAuthStore)
@@ -981,14 +903,15 @@ for (const crashState of [
value_json: targetStoreJson,
});
}
expect(
["authProfiles.store", "authProfiles.state"].map((key) =>
count(migratedTarget, "config_machine_state", "state_key", key),
),
).toEqual([1, 1]);
expect(sharedAuth(fixture.env)).toEqual({
store: JSON.parse(targetStoreJson),
state: fixture.sharedState,
});
using cleanedSource = new DatabaseSync(sourcePath, { readOnly: true });
expect(count(cleanedSource, "auth_profile_store", "store_key", "primary")).toBe(0);
expect(count(cleanedSource, "auth_profile_state", "state_key", "primary")).toBe(0);
expect([readSource(cleanedSource, "store"), readSource(cleanedSource, "state")]).toEqual([
undefined,
undefined,
]);
});
}
src/plugins/marketplace.test.ts
+197 -335
View File
@@ -1,4 +1,5 @@
import "../test-utils/prepare-compiled-subprocesses.js";
import assert from "node:assert/strict";
import { constants } from "node:fs";
import fs from "node:fs/promises";
import os from "node:os";
@@ -7,8 +8,6 @@ import { configureFsSafeNative, getFsSafeNativeConfig } from "@openclaw/fs-safe/
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import * as jsonFiles from "../infra/json-files.js";
import { resolvePreferredOpenClawTmpDir } from "../infra/tmp-openclaw-dir.js";
import { withEnvAsync } from "../test-utils/env.js";
import { withTempDir } from "../test-utils/temp-dir.js";
import {
createMarketplaceInstallInput,
expectMarketplaceInstallSuccess,
@@ -76,8 +75,6 @@ function mockSuccessfulMarketplaceInstall() {
ok: true,
pluginId: "frontend-design",
targetDir: "/tmp/frontend-design",
version: "0.1.0",
extensions: ["index.ts"],
});
}
@@ -107,25 +104,6 @@ async function writeLocalMarketplaceFixture(rootDir: string) {
};
}
async function withKnownMarketplaceRegistry<T>(
marketplaces: Record<string, unknown> | ((homeDir: string) => Record<string, unknown>),
run: (homeDir: string) => Promise<T>,
): Promise<T> {
return await withTempDir("openclaw-marketplace-known-", async (homeDir) => {
const openClawHome = path.join(homeDir, "openclaw-home");
const registryPath = path.join(homeDir, ".claude", "plugins", "known_marketplaces.json");
await fs.mkdir(path.dirname(registryPath), { recursive: true });
await fs.mkdir(openClawHome, { recursive: true });
await fs.writeFile(
registryPath,
JSON.stringify(typeof marketplaces === "function" ? marketplaces(homeDir) : marketplaces),
);
return await withEnvAsync({ HOME: homeDir, OPENCLAW_HOME: openClawHome }, async () =>
run(homeDir),
);
});
}
function mockRemoteMarketplaceClone(params: {
manifest: unknown;
pluginDir?: string;
@@ -133,10 +111,9 @@ function mockRemoteMarketplaceClone(params: {
symlinkPath?: string;
}) {
runCommandWithTimeoutMock.mockImplementationOnce(async (argv: string[]) => {
const repoDir = argv.at(-1);
expect(typeof repoDir).toBe("string");
const repoDir = argv.at(-1)!;
await writeRemoteMarketplaceFixture({
repoDir: repoDir as string,
repoDir,
...params,
});
if (params.symlinkPath) {
@@ -144,7 +121,7 @@ function mockRemoteMarketplaceClone(params: {
"openclaw-marketplace-outside",
tempOutsideDirs,
);
const link = path.join(repoDir as string, params.symlinkPath);
const link = path.join(repoDir, params.symlinkPath);
await fs.mkdir(path.dirname(link), { recursive: true });
await fs.symlink(outsideDir, link);
}
@@ -152,21 +129,6 @@ function mockRemoteMarketplaceClone(params: {
});
}
function expectRemoteCloneCommand() {
expect(runCommandWithTimeoutMock).toHaveBeenCalledTimes(1);
const [argv, options] = runCommandWithTimeoutMock.mock.calls[0] ?? [];
expect(Array.isArray(argv)).toBe(true);
expect((argv as unknown[]).slice(0, 5)).toEqual([
"git",
"clone",
"--depth",
"1",
"https://github.com/owner/repo.git",
]);
expect(typeof (argv as unknown[])[5]).toBe("string");
expect(options).toEqual({ timeoutMs: 120_000 });
}
function expectLocalMarketplaceInstallResult(
result: unknown,
pluginDir: string,
@@ -177,14 +139,31 @@ function expectLocalMarketplaceInstallResult(
}
describe("marketplace plugins", () => {
let rootDir: string;
beforeEach(async () => {
rootDir = await fs.mkdtemp(path.join(os.tmpdir(), "openclaw-marketplace-test-"));
});
afterEach(async () => {
fetchWithSsrFGuardMock.mockClear();
installPluginFromPathMock.mockReset();
runCommandWithTimeoutMock.mockReset();
vi.unstubAllGlobals();
vi.unstubAllEnvs();
await cleanupTrackedTempDirsAsync(tempOutsideDirs);
await fs.rm(rootDir, { recursive: true, force: true });
});
async function known(marketplaces: Record<string, unknown>) {
vi.stubEnv("HOME", rootDir);
vi.stubEnv("OPENCLAW_HOME", path.join(rootDir, "openclaw-home"));
await fs.mkdir(path.join(rootDir, "openclaw-home"), { recursive: true });
const registryPath = path.join(rootDir, ".claude", "plugins", "known_marketplaces.json");
await fs.mkdir(path.dirname(registryPath), { recursive: true });
await fs.writeFile(registryPath, JSON.stringify(marketplaces));
return rootDir;
}
it.each([
[
"GitHub primary identifier and ref",
@@ -223,54 +202,31 @@ describe("marketplace plugins", () => {
{ type: " ", source: "git", url: 42, repo: " https://example.com/alias.git " },
gitSource("https://example.com/alias.git"),
],
[
"GitHub missing optional fields",
{ type: "github", repo: "owner/repo" },
githubSource("owner/repo"),
],
[
"Git explicit ref before branch and tag",
{
type: "git",
url: "https://example.com/repo.git",
ref: " pinned ",
branch: "main",
tag: "v1",
},
gitSource("https://example.com/repo.git", "pinned"),
],
] as const)("normalizes %s in local marketplace listings", async (_label, source, expected) => {
await withTempDir("openclaw-marketplace-test-", async (rootDir) => {
await writeMarketplaceManifest(rootDir, { plugins: [{ name: "example", source }] });
await writeMarketplaceManifest(rootDir, { plugins: [{ name: "example", source }] });
const result = await listMarketplacePlugins({ marketplace: rootDir });
const result = await listMarketplacePlugins({ marketplace: rootDir });
expect(result.ok).toBe(true);
if (!result.ok) {
throw new Error(result.error);
}
expect(result.manifest.plugins[0]?.source).toStrictEqual(expected);
expectNoNetwork();
});
assert(result.ok);
expect(result.manifest.plugins[0]?.source).toStrictEqual(expected);
expectNoNetwork();
});
it.each([
{ source: { type: "github", repo: " ", url: false }, error: 'github source missing "repo"' },
{ source: { type: "git", url: null, repo: "\t" }, error: 'git source missing "url"' },
])("preserves $error in local marketplace listings", async ({ source, error }) => {
await withTempDir("openclaw-marketplace-test-", async (rootDir) => {
const manifestPath = await writeMarketplaceManifest(rootDir, {
plugins: [{ name: "example", source }],
});
const result = await listMarketplacePlugins({ marketplace: rootDir });
expect(result).toEqual({
ok: false,
error: `invalid marketplace entry "example" in ${manifestPath}: ${error}`,
});
expectNoNetwork();
const manifestPath = await writeMarketplaceManifest(rootDir, {
plugins: [{ name: "example", source }],
});
const result = await listMarketplacePlugins({ marketplace: rootDir });
expect(result).toEqual({
ok: false,
error: `invalid marketplace entry "example" in ${manifestPath}: ${error}`,
});
expectNoNetwork();
});
it.each([false, true])("loads a symlinked manifest with oversized=%s", async (oversized) => {
@@ -278,57 +234,41 @@ describe("marketplace plugins", () => {
// Windows CI does not guarantee symlink support.
return;
}
await withTempDir("openclaw-marketplace-test-", async (rootDir) => {
const manifestPath = path.join(rootDir, ".claude-plugin", "marketplace.json");
await fs.mkdir(path.dirname(manifestPath), { recursive: true });
const targetPath = path.join(rootDir, "real-manifest.json");
await fs.writeFile(
targetPath,
oversized
? Buffer.alloc(16 * 1024 * 1024 + 1, "x")
: JSON.stringify(manifestFor(".", "symlinked-plugin")),
);
await fs.symlink(targetPath, manifestPath);
const manifestPath = path.join(rootDir, ".claude-plugin", "marketplace.json");
await fs.mkdir(path.dirname(manifestPath), { recursive: true });
const targetPath = path.join(rootDir, "real-manifest.json");
await fs.writeFile(
targetPath,
oversized
? Buffer.alloc(16 * 1024 * 1024 + 1, "x")
: JSON.stringify(manifestFor(".", "symlinked-plugin")),
);
await fs.symlink(targetPath, manifestPath);
const result = await listMarketplacePlugins({ marketplace: rootDir });
const result = await listMarketplacePlugins({ marketplace: rootDir });
if (oversized) {
expect(result).toEqual({ ok: false, error: "Marketplace manifest too large" });
} else {
expect(result.ok).toBe(true);
if (!result.ok) {
throw new Error(result.error);
}
expect(result.manifest.plugins).toHaveLength(1);
}
});
if (oversized) {
expect(result).toEqual({ ok: false, error: "Marketplace manifest too large" });
} else {
assert(result.ok);
expect(result.manifest.plugins).toHaveLength(1);
}
});
it("preserves the logical local install path instead of canonicalizing it", async () => {
await withTempDir("openclaw-marketplace-test-", async (rootDir) => {
const canonicalRootDir = await fs.realpath(rootDir);
const canonicalPluginDir = path.join(canonicalRootDir, "plugins", "frontend-design");
const { pluginDir, manifestPath } = await writeLocalMarketplaceFixture(rootDir);
mockSuccessfulMarketplaceInstall();
const { pluginDir, manifestPath } = await writeLocalMarketplaceFixture(rootDir);
mockSuccessfulMarketplaceInstall();
const result = await installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "frontend-design",
});
const result = await installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "frontend-design",
});
expectLocalMarketplaceInstallResult(result, pluginDir, manifestPath);
expect(installPluginInput().installPolicyRequest).toMatchObject({
kind: "plugin-dir",
requestedSpecifier: `frontend-design@${manifestPath}`,
source: { kind: "local-path", authority: "user", mutable: true, network: false },
});
if (canonicalPluginDir !== pluginDir) {
expect(
installPluginFromPathMock.mock.calls.some(
([input]) => (input as { path?: unknown } | undefined)?.path === canonicalPluginDir,
),
).toBe(false);
}
expectLocalMarketplaceInstallResult(result, pluginDir, manifestPath);
expect(installPluginInput().installPolicyRequest).toMatchObject({
kind: "plugin-dir",
requestedSpecifier: `frontend-design@${manifestPath}`,
source: { kind: "local-path", authority: "user", mutable: true, network: false },
});
});
@@ -338,187 +278,150 @@ describe("marketplace plugins", () => {
] as const)(
"rejects only ambiguous marketplace selections during %s",
async (mode, duplicateName) => {
await withTempDir("openclaw-marketplace-test-", async (rootDir) => {
const { pluginDir, manifestPath } = await writeLocalMarketplaceFixture(rootDir);
await writeMarketplaceManifest(rootDir, {
plugins: [
{ name: pluginName, source: relativePlugin },
{ name: duplicateName, source: "./plugins/other" },
{ name: "Frontend-design", source: relativePlugin },
],
});
mockSuccessfulMarketplaceInstall();
await expect(listMarketplacePlugins({ marketplace: manifestPath })).resolves.toMatchObject({
ok: true,
manifest: {
plugins: [{ name: pluginName }, { name: pluginName }, { name: "Frontend-design" }],
},
});
await expect(
installPluginFromMarketplace({ marketplace: manifestPath, plugin: pluginName, mode }),
).resolves.toEqual({
ok: false,
error: `plugin "${pluginName}" is ambiguous in marketplace ${manifestPath}: 2 entries share this name; remove or rename the duplicate entries`,
});
expect(installPluginFromPathMock).not.toHaveBeenCalled();
await expect(
installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "Frontend-design",
mode,
}),
).resolves.toMatchObject({ ok: true, marketplacePlugin: "Frontend-design" });
expect(installPluginInput()).toMatchObject({ path: pluginDir, mode });
expectNoNetwork();
const { pluginDir, manifestPath } = await writeLocalMarketplaceFixture(rootDir);
await writeMarketplaceManifest(rootDir, {
plugins: [
{ name: pluginName, source: relativePlugin },
{ name: duplicateName, source: "./plugins/other" },
{ name: "Frontend-design", source: relativePlugin },
],
});
mockSuccessfulMarketplaceInstall();
await expect(listMarketplacePlugins({ marketplace: manifestPath })).resolves.toMatchObject({
ok: true,
manifest: {
plugins: [{ name: pluginName }, { name: pluginName }, { name: "Frontend-design" }],
},
});
await expect(
installPluginFromMarketplace({ marketplace: manifestPath, plugin: pluginName, mode }),
).resolves.toEqual({
ok: false,
error: `plugin "${pluginName}" is ambiguous in marketplace ${manifestPath}: 2 entries share this name; remove or rename the duplicate entries`,
});
expect(installPluginFromPathMock).not.toHaveBeenCalled();
await expect(
installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "Frontend-design",
mode,
}),
).resolves.toMatchObject({ ok: true, marketplacePlugin: "Frontend-design" });
expect(installPluginInput()).toMatchObject({ path: pluginDir, mode });
expectNoNetwork();
},
);
it("does not publish a marketplace plugin after authority closes during artifact review", async () => {
await withTempDir("openclaw-marketplace-guard-", async (rootDir) => {
const { pluginDir, extensionsDir } = createBundleInstallFixtureFactory(() => rootDir)({
bundleFormat: "claude",
name: "Guarded Bundle",
});
const manifestPath = await writeMarketplaceManifest(rootDir, {
plugins: [{ name: "guarded-bundle", source: "./plugin-src" }],
});
const { installPluginFromPath } = await import("./install-package.js");
installPluginFromPathMock.mockImplementationOnce(installPluginFromPath);
let authorityActive = true;
const result = await installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "guarded-bundle",
extensionsDir,
onBeforePluginArtifactCommit: async () => {
authorityActive = false;
},
beforePersistentApply: () => {
if (!authorityActive) {
throw new Error("plugin installation authority closed");
}
},
});
expect(authorityActive).toBe(false);
expect(result).toMatchObject({
ok: false,
error: expect.stringContaining("plugin installation authority closed"),
});
await expect(fs.stat(path.join(extensionsDir, "guarded-bundle"))).rejects.toMatchObject({
code: "ENOENT",
});
await expect(fs.stat(pluginDir)).resolves.toBeDefined();
const { pluginDir, extensionsDir } = createBundleInstallFixtureFactory(() => rootDir)({
bundleFormat: "claude",
name: "Guarded Bundle",
});
const manifestPath = await writeMarketplaceManifest(rootDir, {
plugins: [{ name: "guarded-bundle", source: "./plugin-src" }],
});
const { installPluginFromPath } = await import("./install-package.js");
installPluginFromPathMock.mockImplementationOnce(installPluginFromPath);
let authorityActive = true;
const result = await installPluginFromMarketplace({
marketplace: manifestPath,
plugin: "guarded-bundle",
extensionsDir,
onBeforePluginArtifactCommit: async () => {
authorityActive = false;
},
beforePersistentApply: () => {
if (!authorityActive) {
throw new Error("plugin installation authority closed");
}
},
});
expect(authorityActive).toBe(false);
expect(result).toMatchObject({
ok: false,
error: expect.stringContaining("plugin installation authority closed"),
});
await expect(fs.stat(path.join(extensionsDir, "guarded-bundle"))).rejects.toMatchObject({
code: "ENOENT",
});
await expect(fs.stat(pluginDir)).resolves.toBeDefined();
});
it("resolves Claude-style plugin@marketplace shortcuts from known_marketplaces.json", async () => {
await withKnownMarketplaceRegistry(
(homeDir) => ({
"claude-plugins-official": {
source: { source: "github", repo: "anthropics/claude-plugins-official" },
installLocation: path.join(homeDir, ".claude", "plugins", "marketplaces", "official"),
},
}),
async () => {
expect(
await resolveMarketplaceInstallShortcut("superpowers@claude-plugins-official"),
).toEqual({
ok: true,
plugin: "superpowers",
marketplaceSource: "claude-plugins-official",
});
},
);
await known({ fixture: { installLocation: rootDir } });
expect(await resolveMarketplaceInstallShortcut("tool@fixture")).toEqual({
ok: true,
plugin: "tool",
marketplaceSource: "fixture",
});
});
const cyclicKnownMarketplaces = [
{
label: "self-referential",
marketplace: "loop",
marketplaces: {
loop: pathAlias("loop"),
},
cycle: "loop -> loop",
},
{
label: "two-marketplace",
marketplace: "alpha",
marketplaces: {
alpha: pathAlias("beta"),
beta: pathAlias("alpha"),
},
cycle: "alpha -> beta -> alpha",
},
] as const;
it.each(cyclicKnownMarketplaces)(
"rejects a $label known marketplace alias cycle while listing",
async ({ marketplace, marketplaces, cycle }) => {
await withKnownMarketplaceRegistry(marketplaces, async () => {
const registryRead = vi.spyOn(jsonFiles, "tryReadJson");
for (let read = 0; read <= Object.keys(marketplaces).length; read += 1) {
registryRead.mockResolvedValueOnce(marketplaces);
}
// Bound the unfixed recursive loader so the regression never leaves a runaway task.
registryRead.mockResolvedValueOnce({});
try {
const result = await listMarketplacePlugins({ marketplace });
expect(result).toEqual({
ok: false,
error: `known marketplace source cycle: ${cycle}`,
});
expect(registryRead).toHaveBeenCalledTimes(1);
expectNoNetwork();
} finally {
registryRead.mockRestore();
}
});
},
);
it("resolves independent known marketplace alias calls from one registry snapshot each", async () => {
await withKnownMarketplaceRegistry({}, async (homeDir) => {
const marketplaceRoot = path.join(homeDir, "known-marketplace");
const { pluginDir } = await writeLocalMarketplaceFixture(marketplaceRoot);
await fs.writeFile(
path.join(homeDir, ".claude", "plugins", "known_marketplaces.json"),
JSON.stringify({
alpha: pathAlias("beta"),
beta: { installLocation: marketplaceRoot, ...pathAlias("alpha") },
}),
);
mockSuccessfulMarketplaceInstall();
it.each([
["loop", { loop: pathAlias("loop") }, "loop -> loop"],
["alpha", { alpha: pathAlias("beta"), beta: pathAlias("alpha") }, "alpha -> beta -> alpha"],
] as const)(
"rejects known marketplace alias cycles from %s",
async (marketplace, marketplaces, cycle) => {
await known(marketplaces);
const registryRead = vi.spyOn(jsonFiles, "tryReadJson");
for (let read = 0; read <= Object.keys(marketplaces).length; read += 1) {
registryRead.mockResolvedValueOnce(marketplaces);
}
// Bound the unfixed recursive loader so the regression never leaves a runaway task.
registryRead.mockResolvedValueOnce({});
try {
const listed = await listMarketplacePlugins({ marketplace: "alpha" });
const installed = await installPluginFromMarketplace({
marketplace: "alpha",
plugin: "frontend-design",
expect(await listMarketplacePlugins({ marketplace })).toEqual({
ok: false,
error: `known marketplace source cycle: ${cycle}`,
});
expect(listed).toMatchObject({ ok: true, sourceLabel: "beta" });
expectLocalMarketplaceInstallResult(installed, pluginDir, "alpha");
expect(registryRead).toHaveBeenCalledTimes(2);
expect(registryRead).toHaveBeenCalledTimes(1);
expectNoNetwork();
} finally {
registryRead.mockRestore();
}
});
},
);
it("resolves independent known marketplace alias calls from one registry snapshot each", async () => {
const homeDir = await known({});
const marketplaceRoot = path.join(homeDir, "known-marketplace");
const { pluginDir } = await writeLocalMarketplaceFixture(marketplaceRoot);
await fs.writeFile(
path.join(homeDir, ".claude", "plugins", "known_marketplaces.json"),
JSON.stringify({
alpha: pathAlias("beta"),
beta: { installLocation: marketplaceRoot, ...pathAlias("alpha") },
}),
);
mockSuccessfulMarketplaceInstall();
const registryRead = vi.spyOn(jsonFiles, "tryReadJson");
try {
const listed = await listMarketplacePlugins({ marketplace: "alpha" });
const installed = await installPluginFromMarketplace({
marketplace: "alpha",
plugin: "frontend-design",
});
expect(listed).toMatchObject({ ok: true, sourceLabel: "beta" });
expectLocalMarketplaceInstallResult(installed, pluginDir, "alpha");
expect(registryRead).toHaveBeenCalledTimes(2);
expectNoNetwork();
} finally {
registryRead.mockRestore();
}
});
it("preserves ordinary source fallback for an invalid known marketplace alias", async () => {
await withKnownMarketplaceRegistry({ broken: { source: { source: "path" } } }, async () => {
const result = await listMarketplacePlugins({ marketplace: "broken" });
expect(result).toEqual({ ok: false, error: "unsupported marketplace source: broken" });
expectNoNetwork();
await known({ broken: { source: { source: "path" } } });
expect(await listMarketplacePlugins({ marketplace: "broken" })).toEqual({
ok: false,
error: "unsupported marketplace source: broken",
});
expectNoNetwork();
});
it.each([
@@ -545,15 +448,6 @@ describe("marketplace plugins", () => {
? /[\\/]repo[\\/]plugins[\\/]frontend-design\.tgz$/
: /[\\/]repo[\\/]plugins[\\/]frontend-design$/,
);
if (commit) {
expect(runCommandWithTimeoutMock).toHaveBeenCalledTimes(2);
expect(runCommandWithTimeoutMock.mock.calls.map(([argv]) => argv)).toEqual([
["git", "clone", "https://github.com/owner/repo.git", expect.any(String)],
["git", "switch", "--detach", "--", commit],
]);
} else {
expectRemoteCloneCommand();
}
expect(installPluginInput().installPolicyRequest).toMatchObject({
kind: extension ? "plugin-archive" : "plugin-git",
requestedSpecifier: `${pluginName}@${marketplace}`,
@@ -624,14 +518,6 @@ describe("marketplace archive downloads", () => {
.toSorted();
}
function expectFetchDownloadCall(url = "https://example.com/frontend-design.tgz") {
expect(fetchWithSsrFGuardMock.mock.calls[0]?.[0]).toMatchObject({
url,
timeoutMs: 120_000,
auditContext: "marketplace-plugin-download",
});
}
const archiveUrl = "https://example.com/frontend-design.tgz";
let rootDir: string;
let manifestPath: string;
@@ -655,24 +541,6 @@ describe("marketplace archive downloads", () => {
);
}
function fakeResponse(kind = "stream") {
const cancel = vi.fn(async () => undefined);
const arrayBuffer = vi.fn(async () => new Uint8Array([1, 2, 3]).buffer);
const reader = { read: vi.fn(), cancel: vi.fn(async () => undefined), releaseLock: vi.fn() };
const response = {
ok: true,
status: 200,
arrayBuffer,
headers: new Headers(
kind.endsWith("length")
? { "content-length": kind === "malformed length" ? "1e9" : "268435457" }
: {},
),
body: { getReader: () => reader, cancel },
} as unknown as Response;
return { response, reader, cancel, arrayBuffer };
}
function expectDownloadFailure(result: unknown, error: string) {
expect(result).toEqual({ ok: false, error: `failed to download ${archiveUrl}: ${error}` });
expect(installPluginFromPathMock).not.toHaveBeenCalled();
@@ -708,28 +576,26 @@ describe("marketplace archive downloads", () => {
["oversized length", "download too large: 268435457 bytes (limit: 268435456 bytes)"],
["drive-relative filename", "invalid download filename"],
] as const)("rejects %s responses before installing an archive", async (kind, error) => {
const fixture = fakeResponse(kind);
const cancel = vi.fn();
const response =
kind === "HTTP"
? streamingResponse(fixture.cancel, 503)
: kind === "empty"
? new Response(null)
: kind === "drive-relative filename"
? new Response("tgz-bytes")
: fixture.response;
kind === "empty"
? new Response(null)
: streamingResponse(cancel, kind === "HTTP" ? 503 : 200);
if (kind.endsWith("length")) {
response.headers.set("content-length", kind === "malformed length" ? "1e9" : "268435457");
}
const getReader = response.body ? vi.spyOn(response.body, "getReader") : undefined;
const release = respond(
response,
kind === "drive-relative filename" ? "https://cdn.example.com/C:plugin.tgz" : archiveUrl,
);
expectDownloadFailure(await install(), error);
expectFetchDownloadCall();
expect(release).toHaveBeenCalledTimes(1);
if (kind === "HTTP" || kind.endsWith("length")) {
expect(fixture.cancel).toHaveBeenCalledTimes(1);
expect(cancel).toHaveBeenCalledTimes(1);
}
if (kind.endsWith("length")) {
expect(fixture.reader.read).not.toHaveBeenCalled();
expect(fixture.reader.cancel).not.toHaveBeenCalled();
expect(getReader).not.toHaveBeenCalled();
}
});
@@ -751,7 +617,6 @@ describe("marketplace archive downloads", () => {
marketplacePlugin: "frontend-design",
marketplaceSource: manifestPath,
});
expectFetchDownloadCall();
expect(String(installPluginInput().path)).toMatch(/[\\/]frontend-design\.tgz$/);
expect(installPluginInput().installPolicyRequest).toMatchObject({
kind: "plugin-archive",
@@ -859,10 +724,7 @@ describe("marketplace archive downloads", () => {
expect(result).toEqual({ ok: false, error: "failed to download ***: Invalid URL" });
expect(fetchWithSsrFGuardMock).not.toHaveBeenCalled();
} else {
expect(result.ok).toBe(false);
if (result.ok) {
return;
}
assert(!result.ok);
expect(result.error).toContain(
"failed to download https://***:***@example.com/frontend-design.tgz:",
);
src/plugins/openai-compatible-embedding-provider.test.ts
+168 -351
View File
@@ -1,33 +1,35 @@
// Covers OpenAI-compatible embedding provider plugin behavior.
import "../test-utils/prepare-compiled-subprocesses.js";
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
import type { AddressInfo } from "node:net";
import { Agent, type Dispatcher } from "undici";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
createMemorySearchDeadlineControl,
MEMORY_SEARCH_DEADLINE_CONTROL,
} from "../../packages/memory-host-sdk/src/host/search-deadline-control.js";
import {
closeHttpServer,
listenLoopback,
readRequestText,
} from "../../test/helpers/gateway-integration-fixtures.js";
import { withTestTimeout } from "../../test/helpers/promise.js";
import type { ConfiguredProviderLocalServiceTarget } from "../agents/provider-local-service-target.js";
import type {
AcquireConfiguredProviderLocalService,
ConfiguredProviderLocalServiceTarget,
} from "../agents/provider-local-service-target.js";
import type { EmbeddingProviderCreateOptions } from "./embedding-providers.js";
import { openAICompatibleEmbeddingProviderAdapter } from "./openai-compatible-embedding-provider.js";
async function createProvider(options: EmbeddingProviderCreateOptions) {
async function createProvider(
options: EmbeddingProviderCreateOptions & {
acquireLocalService?: AcquireConfiguredProviderLocalService;
},
) {
const result = await openAICompatibleEmbeddingProviderAdapter.create(options);
if (!result.provider) {
throw new Error("expected OpenAI-compatible embedding provider");
}
const cacheKeyData = result.runtime?.cacheKeyData as
| { baseUrl?: string; headers?: Record<string, string> }
| undefined;
return {
provider: result.provider,
client: {
baseUrl: cacheKeyData?.baseUrl,
headers: cacheKeyData?.headers ?? {},
},
};
return { ...result, provider: result.provider };
}
type CapturedRequest = {
@@ -37,28 +39,7 @@ type CapturedRequest = {
body: Record<string, unknown>;
};
type FixtureResponse = {
object: "list";
data: Array<{
object?: "embedding";
embedding: number[];
index: number;
}>;
model?: string;
usage?: {
prompt_tokens?: number;
total_tokens?: number;
};
};
type OversizedStreamServer = {
baseUrl: string;
closed: Promise<void>;
getBodyBytesSent: () => number;
getPlannedBodyBytes: () => number;
};
const servers: Array<{ close: () => Promise<void> }> = [];
const servers: Server[] = [];
function createOptions(
overrides: Partial<EmbeddingProviderCreateOptions> = {},
@@ -71,45 +52,41 @@ function createOptions(
};
}
function configuredOptions(
provider: string,
config: NonNullable<
NonNullable<EmbeddingProviderCreateOptions["config"]["models"]>["providers"]
>[string],
overrides: Partial<EmbeddingProviderCreateOptions> = {},
) {
return createOptions({
provider,
config: { models: { providers: { [provider]: config } } },
...overrides,
});
}
function vectors(...embeddings: number[][]) {
return { data: embeddings.map((embedding, index) => ({ embedding, index })) };
}
async function readJsonBody(req: IncomingMessage): Promise<Record<string, unknown>> {
const chunks: Buffer[] = [];
for await (const chunk of req) {
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
}
const text = Buffer.concat(chunks).toString("utf8");
return JSON.parse(text) as Record<string, unknown>;
return JSON.parse(await readRequestText(req)) as Record<string, unknown>;
}
async function listen(server: Server, destroyConnections = false) {
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
server.off("error", reject);
resolve();
});
});
servers.push({
close: () =>
new Promise<void>((resolve, reject) => {
if (destroyConnections) {
server.closeAllConnections();
}
server.close((error) => (error ? reject(error) : resolve()));
}),
});
return `http://127.0.0.1:${(server.address() as AddressInfo).port}/v1`;
async function listen(server: Server) {
const baseUrl = await listenLoopback(server);
servers.push(server);
return `${baseUrl}/v1`;
}
function textEmbeddings({ body }: CapturedRequest, usage = false): FixtureResponse {
function textEmbeddings({ body }: CapturedRequest, usage = false) {
const texts = Array.isArray(body.input) ? body.input : [body.input];
return {
object: "list",
data: texts.map((text, index) => ({
object: "embedding",
embedding: [String(text).length, index + 0.25, 1],
index,
})),
model: String(body.model),
...(usage ? { usage: { prompt_tokens: texts.length, total_tokens: texts.length } } : {}),
};
}
@@ -129,13 +106,16 @@ function localProvider(baseUrl: string): EmbeddingProviderCreateOptions["config"
};
}
async function providerAt(baseUrl: string) {
return createProvider(createOptions({ remote: { baseUrl } }));
async function providerAt(
baseUrl: string,
overrides: Partial<EmbeddingProviderCreateOptions> = {},
) {
return createProvider(createOptions({ remote: { baseUrl }, ...overrides }));
}
async function startEmbeddingServer(params?: {
token?: string;
respond?: (request: CapturedRequest) => FixtureResponse | Record<string, unknown> | null;
respond?: (request: CapturedRequest) => Record<string, unknown> | null;
status?: number;
redirectUrl?: string;
}): Promise<{ baseUrl: string; requests: CapturedRequest[] }> {
@@ -163,15 +143,7 @@ async function startEmbeddingServer(params?: {
...(params?.redirectUrl ? { location: params.redirectUrl } : {}),
});
res.end(
JSON.stringify(
params?.respond
? params.respond(captured)
: {
object: "list",
data: [{ object: "embedding", embedding: [0.1, 0.2, 0.3], index: 0 }],
model: body.model,
},
),
JSON.stringify(params?.respond ? params.respond(captured) : vectors([0.1, 0.2, 0.3])),
);
} catch (error) {
res.writeHead(500, { "content-type": "application/json" });
@@ -188,10 +160,11 @@ const EMBEDDING_ERROR_BOUNDARY_BODY = `${EMBEDDING_ERROR_BOUNDARY_PREFIX}😀${"
8 * 1024 - EMBEDDING_ERROR_BOUNDARY_PREFIX.length - 4,
)}`;
async function startHangingErrorEmbeddingServer(): Promise<{
baseUrl: string;
closed: Promise<void>;
}> {
async function startStreamServer(
status: number,
write: (res: ServerResponse) => void,
contentType = "application/json",
) {
let resolveClosed: () => void = () => undefined;
const closed = new Promise<void>((resolve) => {
resolveClosed = resolve;
@@ -200,86 +173,20 @@ async function startHangingErrorEmbeddingServer(): Promise<{
void (async () => {
await readJsonBody(req);
res.on("close", resolveClosed);
res.writeHead(502, { "content-type": "text/plain" });
res.write(EMBEDDING_ERROR_BOUNDARY_BODY);
res.writeHead(status, { "content-type": contentType });
write(res);
})();
});
return {
baseUrl: await listen(server, true),
baseUrl: await listen(server),
closed,
};
}
async function startOversizedSuccessEmbeddingServer(): Promise<OversizedStreamServer> {
const chunk = Buffer.alloc(64 * 1024, 0x20);
const prefix = Buffer.from('{"data":[');
const plannedBodyBytes = 64 * 1024 * 1024;
let bodyBytesSent = 0;
let resolveClosed: () => void = () => undefined;
const closed = new Promise<void>((resolve) => {
resolveClosed = resolve;
});
const server = createServer((req: IncomingMessage, res: ServerResponse) => {
void (async () => {
await readJsonBody(req);
let closedAlready = false;
res.on("close", () => {
closedAlready = true;
resolveClosed();
});
res.writeHead(200, { "content-type": "application/json" });
const writeChunk = async (buffer: Buffer): Promise<boolean> => {
if (closedAlready) {
return false;
}
const accepted = res.write(buffer);
bodyBytesSent += buffer.byteLength;
if (accepted) {
return true;
}
return await new Promise<boolean>((resolve) => {
const cleanup = () => {
res.off("drain", onDrain);
res.off("close", onClose);
};
const onDrain = () => {
cleanup();
resolve(!closedAlready);
};
const onClose = () => {
cleanup();
resolve(false);
};
res.once("drain", onDrain);
res.once("close", onClose);
});
};
if (!(await writeChunk(prefix))) {
return;
}
const chunksToSend = Math.ceil((plannedBodyBytes - bodyBytesSent) / chunk.byteLength);
for (let i = 0; i < chunksToSend; i++) {
if (!(await writeChunk(chunk))) {
return;
}
}
res.end("]}");
})();
});
return {
baseUrl: await listen(server, true),
closed,
getBodyBytesSent: () => bodyBytesSent,
getPlannedBodyBytes: () => plannedBodyBytes,
};
}
afterEach(async () => {
const pending = servers.splice(0);
await Promise.all(pending.map((server) => server.close()));
await Promise.all(pending.map(closeHttpServer));
});
describe("openai-compatible generic embedding provider", () => {
@@ -287,22 +194,16 @@ describe("openai-compatible generic embedding provider", () => {
const server = await startEmbeddingServer();
for (const remoteOverride of [false, true]) {
const { provider } = await createProvider(
createOptions({
provider: "private-fixture",
config: {
models: {
providers: {
"private-fixture": {
api: "openai-completions",
baseUrl: remoteOverride ? "https://provider.example.test/v1" : server.baseUrl,
request: { allowPrivateNetwork: false },
models: [],
},
},
},
configuredOptions(
"private-fixture",
{
api: "openai-completions",
baseUrl: remoteOverride ? "https://provider.example.test/v1" : server.baseUrl,
request: { allowPrivateNetwork: false },
models: [],
},
...(remoteOverride ? { remote: { baseUrl: server.baseUrl } } : {}),
}),
remoteOverride ? { remote: { baseUrl: server.baseUrl } } : {},
),
);
await expect(provider.embed("private input")).rejects.toThrow(
/private\/internal\/special-use/,
@@ -317,9 +218,7 @@ describe("openai-compatible generic embedding provider", () => {
status: 307,
redirectUrl: `${target.baseUrl}/embeddings`,
});
const { provider } = await createProvider(
createOptions({ remote: { baseUrl: source.baseUrl } }),
);
const { provider } = await providerAt(source.baseUrl);
await expect(provider.embed("private input")).rejects.toThrow(/private\/internal\/special-use/);
expect(source.requests).toHaveLength(1);
expect(target.requests).toEqual([]);
@@ -327,9 +226,7 @@ describe("openai-compatible generic embedding provider", () => {
it.each([true, false])("preserves caller-owned HTTP deadlines: signal=%s", async (withSignal) => {
const server = await startEmbeddingServer();
const { provider } = await createProvider(
createOptions({ remote: { baseUrl: server.baseUrl } }),
);
const { provider } = await providerAt(server.baseUrl);
const requests: Dispatcher.DispatchOptions[] = [];
// oxlint-disable-next-line typescript/unbound-method -- The observer calls the original with the same Agent receiver.
const dispatch = Agent.prototype.dispatch;
@@ -349,8 +246,9 @@ describe("openai-compatible generic embedding provider", () => {
}),
).resolves.toEqual([[0.1, 0.2, 0.3]]);
expect(requests).toHaveLength(1);
expect(requests[0]?.headersTimeout).toBe(withSignal ? 0 : undefined);
expect(requests[0]?.bodyTimeout).toBe(withSignal ? 0 : undefined);
expect([requests[0]?.headersTimeout, requests[0]?.bodyTimeout]).toEqual(
withSignal ? [0, 0] : [undefined, undefined],
);
} finally {
spy.mockRestore();
}
@@ -367,16 +265,14 @@ describe("openai-compatible generic embedding provider", () => {
events.push("reconcile");
return { release };
});
const options = {
const { provider } = await createProvider({
...createOptions({
config: localProvider(server.baseUrl),
provider: "gpu-spark",
model: "gpu-spark/nomic-embed-text",
}),
acquireLocalService,
};
const { provider } = await createProvider(options);
});
const control = createMemorySearchDeadlineControl();
control.subscribe((action) => events.push(action));
const caller = new AbortController();
@@ -395,17 +291,15 @@ describe("openai-compatible generic embedding provider", () => {
it("does not lease a configured local service for a remote endpoint override", async () => {
const server = await startEmbeddingServer();
const acquireLocalService = vi.fn(async () => ({ release: vi.fn() }));
const options = createOptions({
config: localProvider("http://spark.local:11434/v1"),
provider: "gpu-spark",
model: "gpu-spark/nomic-embed-text",
remote: { baseUrl: server.baseUrl },
}) as EmbeddingProviderCreateOptions & {
acquireLocalService: typeof acquireLocalService;
};
options.acquireLocalService = acquireLocalService;
const { provider } = await createProvider(options);
const { provider } = await createProvider({
...createOptions({
config: localProvider("http://spark.local:11434/v1"),
provider: "gpu-spark",
model: "gpu-spark/nomic-embed-text",
remote: { baseUrl: server.baseUrl },
}),
acquireLocalService,
});
await expect(provider.embed("hello")).resolves.toEqual([0.1, 0.2, 0.3]);
expect(acquireLocalService).not.toHaveBeenCalled();
});
@@ -426,13 +320,7 @@ describe("openai-compatible generic embedding provider", () => {
}),
);
expect(result.runtime?.cacheKeyData).toMatchObject({
headers: {
accept: "application/json",
"content-type": "application/json",
"x-deployment": "tenant-a",
},
});
expect(result.runtime?.cacheKeyData).toMatchObject({ headers: { "x-deployment": "tenant-a" } });
expect(result.runtime?.cacheKeyData).not.toHaveProperty("authorization");
expect(
(result.runtime!.cacheKeyData as { headers?: Record<string, string> }).headers,
@@ -456,26 +344,22 @@ describe("openai-compatible generic embedding provider", () => {
};
for (const key of ["fixture-before", "fixture-after"]) {
const result = await openAICompatibleEmbeddingProviderAdapter.create(
createOptions({
provider: "custom",
model: "fixture-embedding",
dimensions: 3,
inputType: "default",
queryInputType: "query",
documentInputType: "document",
config: {
models: {
providers: {
custom: {
baseUrl: canonical.baseUrl,
headers: { "X-Access": key, "X-Deployment": "tenant-a" },
request: { auth: { mode: "header", headerName: " X-Access ", value: key } },
models: [],
},
},
},
configuredOptions(
"custom",
{
baseUrl: canonical.baseUrl,
headers: { "X-Access": key, "X-Deployment": "tenant-a" },
request: { auth: { mode: "header", headerName: " X-Access ", value: key } },
models: [],
},
}),
{
model: "fixture-embedding",
dimensions: 3,
inputType: "default",
queryInputType: "query",
documentInputType: "document",
},
),
);
const priorIdentity = {
...canonical,
@@ -487,12 +371,8 @@ describe("openai-compatible generic embedding provider", () => {
},
};
expect(result.runtime?.cacheKeyData).toEqual(canonical);
expect(result.runtime?.indexIdentityAliases).toEqual([
{
model: "fixture-embedding",
cacheKeyData: priorIdentity,
},
]);
expect(result.runtime?.indexIdentityAliases).toHaveLength(1);
expect(result.runtime?.indexIdentityAliases?.[0]?.model).toBe("fixture-embedding");
expect(JSON.stringify(result.runtime?.indexIdentityAliases?.[0]?.cacheKeyData)).toBe(
JSON.stringify(priorIdentity),
);
@@ -518,22 +398,16 @@ describe("openai-compatible generic embedding provider", () => {
"does not create an index identity alias for $name",
async ({ authHeaderName, headers, remote }) => {
const result = await openAICompatibleEmbeddingProviderAdapter.create(
createOptions({
provider: "custom",
config: {
models: {
providers: {
custom: {
baseUrl: "http://127.0.0.1:18001/v1",
headers,
request: { auth: { mode: "header", headerName: authHeaderName, value: "key" } },
models: [],
},
},
},
configuredOptions(
"custom",
{
baseUrl: "http://127.0.0.1:18001/v1",
headers,
request: { auth: { mode: "header", headerName: authHeaderName, value: "key" } },
models: [],
},
remote,
}),
{ remote },
),
);
expect(result.runtime?.indexIdentityAliases).toBeUndefined();
},
@@ -546,7 +420,7 @@ describe("openai-compatible generic embedding provider", () => {
respond: (request) => textEmbeddings(request, true),
});
const { provider, client } = await createProvider(
const { provider, runtime } = await createProvider(
createOptions({
model: "text-embedding-bge-m3",
dimensions: 3,
@@ -565,7 +439,7 @@ describe("openai-compatible generic embedding provider", () => {
model: "text-embedding-bge-m3",
dimensions: 3,
});
expect(client.baseUrl).toBe(server.baseUrl);
expect(runtime?.cacheKeyData).toMatchObject({ baseUrl: server.baseUrl });
expect(server.requests).toHaveLength(0);
const onUsage = vi.fn();
@@ -583,35 +457,21 @@ describe("openai-compatible generic embedding provider", () => {
expect(server.requests[0]).toMatchObject({
method: "POST",
url: "/v1/embeddings",
body: {
model: "text-embedding-bge-m3",
input: ["hello"],
dimensions: 3,
headers: {
"content-type": expect.stringContaining("application/json"),
accept: "application/json",
"x-local-runtime": "ollama",
},
});
expect(server.requests[0]?.body).not.toHaveProperty("encoding_format");
expect(server.requests[0]?.body).not.toHaveProperty("input_type");
expect(server.requests[0]?.headers).toMatchObject({
"content-type": expect.stringContaining("application/json"),
accept: "application/json",
"x-local-runtime": "ollama",
});
expect(server.requests[1]?.body).toEqual({
model: "text-embedding-bge-m3",
input: ["a", "abcd"],
dimensions: 3,
});
expect(server.requests.map(({ body }) => body)).toEqual([
{ model: "text-embedding-bge-m3", input: ["hello"], dimensions: 3 },
{ model: "text-embedding-bge-m3", input: ["a", "abcd"], dimensions: 3 },
]);
});
it("returns endpoint vectors unchanged when dimensions are not configured", async () => {
const server = await startEmbeddingServer({
respond: () => ({
object: "list",
data: [
{ object: "embedding", embedding: [0.1, 0.2, 0.3], index: 0 },
{ object: "embedding", embedding: [0.4, 0.5], index: 1 },
],
}),
respond: () => vectors([0.1, 0.2, 0.3], [0.4, 0.5]),
});
const result = await openAICompatibleEmbeddingProviderAdapter.create(
createOptions({ remote: { baseUrl: server.baseUrl } }),
@@ -626,40 +486,42 @@ describe("openai-compatible generic embedding provider", () => {
expect(server.requests[0]?.body).not.toHaveProperty("dimensions");
});
it("rejects vectors that do not match the requested dimensions", async () => {
// Simulate a server that ignores `dimensions` and returns the model's native size.
const native = await startEmbeddingServer({ respond: (request) => textEmbeddings(request) });
const { provider } = await createProvider(
createOptions({ dimensions: 2, remote: { baseUrl: native.baseUrl } }),
);
await expect(provider.embed("hello", { inputType: "query" })).rejects.toMatchObject({
code: "INVALID_EMBEDDING_RESPONSE",
expectedDimensions: 2,
actualDimensions: 3,
message:
"openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 1): expected 2 dimensions, got 3 at index 0; the endpoint did not honor the requested dimensions. Remove memory.search.outputDimensionality or request dimensions, or use a server that supports them.",
});
await expect(provider.embedBatch(["a", "b"], { inputType: "document" })).rejects.toMatchObject({
code: "INVALID_EMBEDDING_RESPONSE",
message:
"openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 2): expected 2 dimensions, got 3 at index 0; the endpoint did not honor the requested dimensions. Remove memory.search.outputDimensionality or request dimensions, or use a server that supports them.",
});
const { provider: wider } = await createProvider(
createOptions({ dimensions: 4, remote: { baseUrl: native.baseUrl } }),
);
await expect(wider.embed("hello")).rejects.toThrow(
"expected 4 dimensions, got 3 at index 0; the endpoint did not honor the requested dimensions. Remove memory.search.outputDimensionality or request dimensions, or use a server that supports them.",
);
expect(native.requests.map((request) => request.body.dimensions)).toEqual([2, 2, 4]);
});
it.each([
{ dimensions: 2, inputs: ["hello"], batch: false, index: 0 },
{ dimensions: 2, inputs: ["a", "b", "c"], batch: true, index: 2 },
])(
"rejects mismatched dimensions ($dimensions, batch=$batch, index=$index)",
async ({ dimensions, inputs, batch, index }) => {
const server = await startEmbeddingServer({
respond:
index === 2
? () => ({
data: [
{ embedding: [0, 0.6, 0.8], index: 2 },
{ embedding: [0.6, 0.8], index: 1 },
{ embedding: [1, 0], index: 0 },
],
})
: textEmbeddings,
});
const { provider } = await providerAt(server.baseUrl, { dimensions });
await expect(
batch
? provider.embedBatch(inputs, { inputType: "document" })
: provider.embed(inputs[0]!, { inputType: "query" }),
).rejects.toMatchObject({
code: "INVALID_EMBEDDING_RESPONSE",
expectedDimensions: dimensions,
actualDimensions: 3,
message: `openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: ${inputs.length}): expected ${dimensions} dimensions, got 3 at index ${index}; the endpoint did not honor the requested dimensions. Remove memory.search.outputDimensionality or request dimensions, or use a server that supports them.`,
});
expect(server.requests[0]?.body.dimensions).toBe(dimensions);
},
);
it("keeps malformed response errors distinct from dimension mismatches", async () => {
const server = await startEmbeddingServer({ respond: () => ({ data: [{}] }) });
const { provider } = await createProvider(
createOptions({ dimensions: 2, remote: { baseUrl: server.baseUrl } }),
);
const { provider } = await providerAt(server.baseUrl, { dimensions: 2 });
const error = await provider.embed("query").catch((cause: unknown) => cause);
expect(error).toMatchObject({
code: "INVALID_EMBEDDING_RESPONSE",
@@ -667,75 +529,34 @@ describe("openai-compatible generic embedding provider", () => {
});
expect(error).not.toHaveProperty("expectedDimensions");
expect(error).not.toHaveProperty("actualDimensions");
expect(server.requests).toHaveLength(1);
});
it("rejects a batch when any vector differs from the requested dimensions", async () => {
const server = await startEmbeddingServer({
respond: () => ({
object: "list",
data: [
{ object: "embedding", embedding: [0, 0.6, 0.8], index: 2 },
{ object: "embedding", embedding: [0.6, 0.8], index: 1 },
{ object: "embedding", embedding: [1, 0], index: 0 },
],
}),
});
const { provider } = await createProvider(
createOptions({ dimensions: 2, remote: { baseUrl: server.baseUrl } }),
);
await expect(provider.embedBatch(["a", "b", "c"])).rejects.toMatchObject({
code: "INVALID_EMBEDDING_RESPONSE",
message:
"openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 3): expected 2 dimensions, got 3 at index 2; the endpoint did not honor the requested dimensions. Remove memory.search.outputDimensionality or request dimensions, or use a server that supports them.",
});
});
it("bounds exact-limit embedding errors without splitting UTF-16 and cancels", async () => {
const server = await startHangingErrorEmbeddingServer();
const server = await startStreamServer(
502,
(res) => {
res.write(EMBEDDING_ERROR_BOUNDARY_BODY);
},
"text/plain",
);
const { provider } = await providerAt(server.baseUrl);
const outcome = await withTestTimeout(
provider.embed("hello").then(
() => ({ type: "resolved" as const }),
(error: unknown) => ({ type: "rejected" as const, error }),
),
const request = withTestTimeout(
provider.embed("hello"),
1_000,
"timed out waiting for bounded embedding error",
);
if (outcome.type !== "rejected") {
throw new Error(`expected embedding request to reject, got ${outcome.type}`);
}
expect(outcome.error).toBeInstanceOf(Error);
expect((outcome.error as Error).message).toBe(
`openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 1): HTTP 502: ${EMBEDDING_ERROR_BOUNDARY_PREFIX}... [truncated]`,
);
await expect(request).rejects.toBeInstanceOf(Error);
await expect(request).rejects.toMatchObject({
message: `openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 1): HTTP 502: ${EMBEDDING_ERROR_BOUNDARY_PREFIX}... [truncated]`,
});
await expect(
withTestTimeout(
server.closed.then(() => "closed" as const),
server.closed,
1_000,
"timed out waiting for embedding error server to close",
),
).resolves.toBe("closed");
});
it("bounds and cancels oversized successful embedding JSON bodies", async () => {
const server = await startOversizedSuccessEmbeddingServer();
const { provider } = await providerAt(server.baseUrl);
await expect(provider.embed("hello")).rejects.toThrow(
"openai-compatible embeddings failed (model: text-embedding-bge-m3, batch size: 1): JSON response exceeds 16777216 bytes",
);
await expect(
withTestTimeout(
server.closed.then(() => "closed" as const),
1_000,
"timed out waiting for oversized response server to close",
),
).resolves.toBe("closed");
expect(server.getBodyBytesSent()).toBeLessThan(server.getPlannedBodyBytes() / 2);
).resolves.toBeUndefined();
});
it("maps configured memory input_type labels onto query and document requests", async () => {
@@ -743,7 +564,6 @@ describe("openai-compatible generic embedding provider", () => {
const result = await openAICompatibleEmbeddingProviderAdapter.create(
createOptions({
model: "text-embedding-bge-m3",
inputType: " default ",
queryInputType: " query ",
documentInputType: " document ",
@@ -780,10 +600,7 @@ describe("openai-compatible generic embedding provider", () => {
const server = await startEmbeddingServer({
respond: ({ body }) => {
expect(body.input).toEqual(["ab"]);
return {
object: "list",
data: [{ object: "embedding", embedding: [2, 1], index: 0 }],
};
return vectors([2, 1]);
},
});
const { provider } = await providerAt(server.baseUrl);
src/plugins/plugin-lifecycle-lease.test.ts
+71 -215
View File
@@ -1,13 +1,14 @@
import { AsyncResource } from "node:async_hooks";
import { spawn, type ChildProcessByStdio } from "node:child_process";
import type { ChildProcessWithoutNullStreams } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import type { Readable, Writable } from "node:stream";
import { setImmediate as nextTurn } from "node:timers/promises";
import { pathToFileURL } from "node:url";
import { queryObjects } from "node:v8";
import { afterEach, describe, expect, it, vi } from "vitest";
import { createDeferred } from "../../test/helpers/promise.js";
import { stopChildProcess } from "../../test/helpers/stop-child-process.js";
import { runCliProcessChild } from "../cli/cli-process-child.test-helpers.js";
import { createPluginLifecycleLeaseTestClock } from "../gateway/config-reload.test-support.js";
import { sqliteWorkerPreloadEnv } from "../infra/sqlite-worker-preload.test-support.js";
import { closeOpenClawStateDatabaseForTest } from "../state/openclaw-state-db.js";
@@ -35,7 +36,7 @@ import {
} from "./plugin-lifecycle-lease.js";
import { seedInstalledPluginIndex } from "./test-helpers/installed-plugin-index.js";
type LeaseChild = ChildProcessByStdio<Writable, Readable, Readable>;
type LeaseChild = ChildProcessWithoutNullStreams;
type LeaseChildRun = {
child: LeaseChild;
ready: Promise<void>;
@@ -68,19 +69,6 @@ afterEach(() => {
closeOpenClawStateDatabaseForTest();
});
async function terminateLeaseChild(child: LeaseChild): Promise<void> {
await new Promise<void>((resolve) => {
const onClose = () => resolve();
child.once("close", onClose);
if (child.exitCode !== null || child.signalCode !== null) {
child.off("close", onClose);
resolve();
return;
}
child.kill("SIGKILL");
});
}
async function withLeaseChildren<T>(fn: (children: Set<LeaseChildRun>) => Promise<T>): Promise<T> {
const children = new Set<LeaseChildRun>();
try {
@@ -89,7 +77,7 @@ async function withLeaseChildren<T>(fn: (children: Set<LeaseChildRun>) => Promis
} finally {
await Promise.all(
Array.from(children, async ({ child, completed }) => {
await terminateLeaseChild(child);
await stopChildProcess(child, 5_000, { force: true });
await completed.catch(() => {});
}),
);
@@ -102,59 +90,47 @@ async function withLeaseChildren<T>(fn: (children: Set<LeaseChildRun>) => Promis
}
}
function runLeaseChild(
async function runLeaseChild(
children: Set<LeaseChildRun>,
scriptPath: string,
args: string[],
env?: NodeJS.ProcessEnv,
): LeaseChildRun {
const child = spawn(process.execPath, ["--import", "tsx", scriptPath, ...args], {
stdio: ["pipe", "pipe", "pipe"],
env,
});
child.stdout.setEncoding("utf8");
child.stderr.setEncoding("utf8");
): Promise<LeaseChildRun> {
let stdout = "";
let stderr = "";
let pendingLine = "";
const phases = new Set<string>();
const phaseWaiters = new Map<string, ReturnType<typeof createDeferred<void>>>();
child.stdout.on("data", (chunk: string) => {
stdout += chunk;
pendingLine += chunk;
const lines = pendingLine.split("\n");
pendingLine = lines.pop() ?? "";
for (const line of lines) {
phases.add(line);
phaseWaiters.get(line)?.resolve();
const started = createDeferred<LeaseChild>();
const completed = runCliProcessChild({
nodeArgs: ["--import", "tsx", scriptPath, ...args],
nodeArgsPolicy: "caller",
env: env ?? process.env,
onStdout(output) {
stdout = output;
for (const line of output.split("\n").slice(0, -1)) {
phases.add(line);
phaseWaiters.get(line)?.resolve();
}
},
interact(child) {
child.stderr.on("data", (chunk: string) => {
stderr += chunk;
});
child.stdin.end();
started.resolve(child);
},
}).then(({ code, signal }) => {
if (code !== 0) {
throw new Error(`lease child ${args[0]} exited ${code ?? signal}`);
}
});
child.stderr.on("data", (chunk: string) => {
stderr += chunk;
});
const completed = new Promise<void>((resolve, reject) => {
const fail = (phase: string, error: Error) =>
reject(
new Error(`failed to ${phase} lease child ${args[0]}: ${error.message}`, { cause: error }),
);
child.stdin.once("error", (error) => {
fail("signal", error);
});
child.once("error", (error) => {
fail("start", error);
});
child.once("close", (code, signal) => {
if (code === 0) {
resolve();
} else {
reject(new Error(`lease child ${args[0]} exited ${code ?? signal}`));
}
});
});
void completed.catch(() => {});
const child = await Promise.race([
started.promise,
completed.then(() => {
throw new Error("Lease child exited before starting");
}),
]);
const waitForPhase = (phase: string): Promise<void> => {
if (phases.has(phase)) {
return Promise.resolve();
@@ -241,8 +217,12 @@ describe("plugin lifecycle lease", () => {
const operations: Promise<unknown>[] = [];
try {
expect(hasPluginLifecycleLeaseDemand()).toBe(false);
const holder = withPluginLifecycleLease({ env: state.env, signal }, async () => {
await withPluginLifecycleLease({}, async () => {
const holder = withPluginLifecycleLease({ env: state.env, signal }, async (outer) => {
await withPluginLifecycleLease({}, async (nested) => {
expect(nested).toBe(outer);
expect(nested.databasePath).toBe(
path.resolve(state.stateDir, "state", "openclaw.sqlite"),
);
expect(hasPluginLifecycleLeaseDemand()).toBe(false);
});
entered.resolve();
@@ -286,51 +266,28 @@ describe("plugin lifecycle lease", () => {
},
);
it.each([
[false, false],
[true, false],
[false, true],
[true, true],
])(
"preserves operation and cleanup outcomes (run failure: %s, cleanup failure: %s)",
async (failRun, failCleanup) => {
await withOpenClawTestState({ label: "plugin-lifecycle-outcome" }, async (state) => {
const runError = new Error("operation failed");
const cleanupError = new Error("cleanup failed");
const value = {};
let cache: ReturnType<typeof getPluginCache> | undefined;
const operation = withPluginLifecycleLease({ env: state.env }, async () => {
it("returns the operation value while recording cleanup failure", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-outcome" }, async (state) => {
const cleanupError = new Error("cleanup failed");
const value = {};
let cache: ReturnType<typeof getPluginCache> | undefined;
await expect(
withPluginLifecycleLease({ env: state.env }, async () => {
cache = getPluginCache();
cacheCleanup(cache, "lease-cleanup", () => {
if (failCleanup) {
throw cleanupError;
}
throw cleanupError;
});
if (failRun) {
throw runError;
}
return value;
});
const [outcome] = await Promise.allSettled([operation]);
expect(cache).toBeDefined();
const cleanup = await retirePluginCache(cache!);
expect(cleanup.failures.map((failure) => failure.error)).toEqual(
failCleanup ? [cleanupError] : [],
);
if (failRun) {
expectRejected(outcome, runError);
} else {
expect(outcome.status).toBe("fulfilled");
if (outcome.status === "fulfilled") {
expect(outcome.value).toBe(value);
}
}
await expect(
withPluginLifecycleLease({ env: state.env, waitMs: 0 }, async () => value),
).resolves.toBe(value);
});
},
);
}),
).resolves.toBe(value);
expect(cache).toBeDefined();
const cleanup = await retirePluginCache(cache!);
expect(cleanup.failures.map((failure) => failure.error)).toEqual([cleanupError]);
await expect(
withPluginLifecycleLease({ env: state.env, waitMs: 0 }, async () => value),
).resolves.toBe(value);
});
});
it("joins process cleanup outcomes while preserving the original operation failure", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-cleanup-owners" }, async (state) => {
@@ -422,17 +379,14 @@ describe("plugin lifecycle lease", () => {
});
});
it.each([
["one state directory", false],
["an explicit database path across different state directories", true],
])("serializes lifecycle work sharing %s", async (_label, explicitPath) => {
it("serializes lifecycle work sharing an explicit database across state directories", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-lease" }, async (state) => {
const firstEntered = createDeferred();
const releaseFirst = createDeferred();
const events: string[] = [];
const leaseOptions = (caller: string) => ({
env: explicitPath ? { ...state.env, OPENCLAW_STATE_DIR: state.path(caller) } : state.env,
...(explicitPath ? { path: state.path("shared-plugin-lifecycle.sqlite") } : {}),
env: { ...state.env, OPENCLAW_STATE_DIR: state.path(caller) },
path: state.path("shared-plugin-lifecycle.sqlite"),
leaseMs: 1_000,
waitMs: 3_000,
});
@@ -463,11 +417,10 @@ describe("plugin lifecycle lease", () => {
it("reclaims process-bound lifecycle work after its process is killed", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-killed-owner" }, async (state) => {
await withLeaseChildren(async (children) => {
const leaseModuleUrl = pluginModuleUrl("plugin-lifecycle-lease.ts");
const script = await state.writeText(
"killed-lease.mts",
`
import { withPluginLifecycleLease } from ${JSON.stringify(leaseModuleUrl)};
import { withPluginLifecycleLease } from ${JSON.stringify(pluginModuleUrl("plugin-lifecycle-lease.ts"))};
const env = { ...process.env, OPENCLAW_STATE_DIR: process.argv[2] };
await withPluginLifecycleLease({ env, processBound: true }, async () => {
process.stdout.write("ready\\n");
@@ -475,10 +428,10 @@ describe("plugin lifecycle lease", () => {
});
`,
);
const holder = runLeaseChild(children, script, [state.stateDir]);
const holder = await runLeaseChild(children, script, [state.stateDir]);
await holder.ready;
await terminateLeaseChild(holder.child);
// SIGKILL is the expected result, already joined above.
await stopChildProcess(holder.child, 5_000, { force: true });
await holder.completed.catch(() => {});
children.delete(holder);
await expect(
withPluginLifecycleLease({ env: state.env, waitMs: 0 }, async () => "recovered"),
@@ -487,85 +440,6 @@ describe("plugin lifecycle lease", () => {
});
});
it("serializes lifecycle work across processes", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-processes" }, async (state) => {
await withLeaseChildren(async (children) => {
const secondMarker = state.path("second-entered");
const secondResult = state.path("second-result");
const leaseModuleUrl = pluginModuleUrl("plugin-lifecycle-lease.ts");
const childScript = await state.writeText(
"lease-child.mts",
`
import fs from "node:fs/promises";
import { createInterface } from "node:readline";
import { withPluginLifecycleLease } from ${JSON.stringify(leaseModuleUrl)};
const [role, stateDir, secondMarker, secondResult] = process.argv.slice(2);
const env = { ...process.env, OPENCLAW_STATE_DIR: stateDir };
const input = createInterface({ input: process.stdin });
const commands = input[Symbol.asyncIterator]();
try {
// Complete cold database/worker admission before either contested lease starts.
await withPluginLifecycleLease({ env, leaseMs: 1_000, waitMs: 5_000 }, async () => {});
process.stdout.write("ready\\n");
if ((await commands.next()).value !== "go") {
throw new Error("expected go command");
}
if (role === "second") {
try {
await withPluginLifecycleLease({ env, leaseMs: 1_000, waitMs: 0 }, async () => {
await fs.writeFile(secondMarker, "entered");
});
await fs.writeFile(secondResult, "acquired");
} catch (error) {
await fs.writeFile(secondResult, error?.code ?? String(error));
}
} else {
await withPluginLifecycleLease({ env, leaseMs: 1_000, waitMs: 5_000 }, async () => {
process.stdout.write("acquired\\n");
if ((await commands.next()).value !== "release") {
throw new Error("expected release command");
}
});
}
} finally {
input.close();
}
`,
);
const childArgs = [state.stateDir, secondMarker, secondResult];
const first = runLeaseChild(children, childScript, ["first", ...childArgs]);
await first.ready;
const second = runLeaseChild(children, childScript, ["second", ...childArgs]);
await second.ready;
first.child.stdin.write("go\n");
await first.waitForPhase("acquired");
second.child.stdin.end("go\n");
// Wait for the child to close so its result write is fully flushed before
// reading; file existence alone can race with the write after open().
await second.completed;
let assertionError: unknown;
try {
await expect(fs.readFile(secondResult, "utf8")).resolves.toBe(
"OPENCLAW_STATE_LEASE_HELD",
);
await expect(fs.access(secondMarker)).rejects.toMatchObject({ code: "ENOENT" });
} catch (error) {
assertionError = error;
} finally {
first.child.stdin.end("release\n");
}
await Promise.all([first.completed, second.completed]);
if (assertionError) {
throw assertionError instanceof Error
? assertionError
: new Error("cross-process lease assertion failed", { cause: assertionError });
}
});
});
});
it("reloads install records after waiting for another process", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-record-cache" }, async (state) => {
await withLeaseChildren(async (children) => {
@@ -635,7 +509,9 @@ if (process.versions.bun) {
await waitForMarker(goMarker);
if (pluginId === "beta") {
await assert.rejects(
withPluginLifecycleLease({ env, leaseMs: 1_000, waitMs: 0 }, async () => {}),
withPluginLifecycleLease({ env, leaseMs: 1_000, waitMs: 0 }, async () => {
assert.fail("blocked lifecycle callback must not run");
}),
{ code: "OPENCLAW_STATE_LEASE_HELD" },
);
process.stdout.write("held\\n");
@@ -668,13 +544,13 @@ if (process.versions.bun) {
`,
);
const alpha = runLeaseChild(
const alpha = await runLeaseChild(
children,
childScript,
["alpha", state.stateDir, alphaGoMarker, releaseAlphaMarker],
childEnv,
);
const beta = runLeaseChild(
const beta = await runLeaseChild(
children,
childScript,
["beta", state.stateDir, betaGoMarker, releaseAlphaMarker],
@@ -729,26 +605,6 @@ if (process.versions.bun) {
});
});
it("reuses the active lease for nested lifecycle work", async () => {
await withOpenClawTestState({ label: "plugin-lifecycle-reentrant" }, async (state) => {
const events: string[] = [];
await withPluginLifecycleLease(
{ env: state.env, leaseMs: 1_000, waitMs: 0 },
async (outerLease) => {
events.push("outer");
await withPluginLifecycleLease({}, async (innerLease) => {
events.push("inner");
expect(innerLease).toBe(outerLease);
expect(innerLease.databasePath).toBe(
path.resolve(state.stateDir, "state", "openclaw.sqlite"),
);
});
},
);
expect(events).toEqual(["outer", "inner"]);
});
});
it.each([
{ authority: "outer", explicitNestedEnv: false },
{ authority: "nested", explicitNestedEnv: false },
src/state/github-publication.worker.test.ts
+58 -173
View File
@@ -33,15 +33,11 @@ import {
prepareGitHubPublicationSource,
} from "../gateway/github-publication-source.js";
import {
claimRepositoryGitHubPublicationAsync,
readGitHubPublicationRequestAsync,
runGitHubPublicationMaintenanceAsync,
} from "../gateway/github-publication-store-async.js";
import { listRepositoryGitHubPublicationsInDatabase } from "../gateway/github-repository-publication-read.worker.js";
import {
claimRepositoryGitHubPublicationInDatabase,
insertRepositoryGitHubPublicationInDatabase,
} from "../gateway/github-repository-publication-store.worker.js";
import { insertRepositoryGitHubPublicationInDatabase } from "../gateway/github-repository-publication-store.worker.js";
import { repositoryGitHubPublicationDigest } from "../gateway/github-repository-publication.kernel.js";
import * as workerAdmission from "../infra/sqlite-worker-operation-admission.js";
import { sqliteWorkerOwnerProbe } from "../infra/sqlite-worker-owner-probe.test-support.js";
@@ -54,10 +50,7 @@ import {
waitForPublicationGate,
} from "./github-publication-fixtures.test-support.js";
import type { RepositoryGitHubPublicationRow as RepositoryPublicationRow } from "./github-publication-read.types.js";
import {
captureGitHubPublicationLifecycle,
githubPublicationReceipts,
} from "./github-publication-receipts.js";
import { githubPublicationReceipts } from "./github-publication-receipts.js";
import { prepareGitHubPublicationSessionLifecycle } from "./github-publication-session-lifecycles.js";
import {
insertGitHubPublicationSessionLifecycle,
@@ -85,10 +78,7 @@ import {
openOpenClawStateDatabase,
runOpenClawStateWriteTransaction,
} from "./openclaw-state-db.js";
import {
captureOpenClawStateReadContext,
captureOpenClawStateWorkerContext,
} from "./openclaw-state-worker-context.js";
import { captureOpenClawStateWorkerContext } from "./openclaw-state-worker-context.js";
import type { OpenClawStateWorkerContext } from "./openclaw-state-worker-context.types.js";
import { createSessionRepositoryWorkspaceInDatabase } from "./session-repository-workspaces.kernel.js";
import {
@@ -231,9 +221,7 @@ async function sourceFixture(
return { row, source };
}
it("creates the lazy shared schema and publishes lifecycle authority before its first insert reply", async ({
signal,
}) => {
it("creates the lazy shared schema and replaces a missing lifecycle binding on insert", async () => {
const request = {
sessionKey: "agent:main:first-shared-publication",
agentId: "main",
@@ -284,30 +272,7 @@ it("creates the lazy shared schema and publishes lifecycle authority before its
requestId: "first-shared-publication-request",
});
expect(lifecycle.readCurrent()).toBeUndefined();
const committed = createDeferredCore();
let atSettlement: ReturnType<typeof lifecycle.readCurrent>;
onTestFinished(
githubPublicationReceipts.subscribeFacts((change) => {
if (
change.kind === "committed" &&
change.receipt.source.identity === context.admission.identity.key
) {
expect(lifecycle.readCurrent).toThrow("lifecycle authority is unavailable");
committed.resolve();
}
if (change.kind === "settled" && change.identity === context.admission.identity.key) {
atSettlement = captureGitHubPublicationLifecycle(captureOpenClawStateReadContext(), {
publicationKind: "shared",
requestId: "first-shared-publication-request",
}).readCurrent();
}
}),
);
const reply = holdNextReply("githubPublications.insert", () => {
// An uncommitted insert cannot authorize publication from a missing binding.
expect(lifecycle.readCurrent()).toBeUndefined();
});
const inserting = insertGitHubPublicationRequestAsync(
const inserted = await insertGitHubPublicationRequestAsync(
{
request,
requestId: "first-shared-publication-request",
@@ -325,13 +290,6 @@ it("creates the lazy shared schema and publishes lifecycle authority before its
},
source,
);
try {
await waitForPublicationGate(signal, inserting, reply.ready, committed.promise);
expect(lifecycle.readCurrent).toThrow("lifecycle authority is unavailable");
} finally {
reply.release();
}
const inserted = await inserting;
expect(inserted).toMatchObject({
request_id: "first-shared-publication-request",
status: "requested",
@@ -344,7 +302,6 @@ it("creates the lazy shared schema and publishes lifecycle authority before its
requester_authority_json:
'{"version":1,"actor":{"kind":"system"},"scopes":["operator.admin"],"grant":null}',
};
expect(atSettlement).toEqual(expected);
expect(
(
await prepareGitHubPublicationSessionLifecycle({
@@ -542,30 +499,21 @@ it("publishes a committed receipt before a delayed ordinary worker reply", async
const row = seed("delayed-reply");
const reply = holdNextReply();
const { facts } = observeAuthority();
const authorityAtNotification: unknown[] = [];
const key = JSON.stringify(["repository", row.request_id]);
const committed = createDeferredCore();
const published = vi.fn((_receipt: PublicationMutationReceipt) => {
authorityAtNotification.push(facts.get(key));
const published = () => {
expect(facts.get(key)).toMatchObject({
kind: "postimage",
value: { request_id: row.request_id, status: "publishing", execution_id: "execution" },
});
committed.resolve();
});
};
const claim = mutatePublication(
scope,
{ operation: "claim", row, instanceId: "gateway", executionId: "execution" },
published,
);
await waitForPublicationGate(signal, claim, reply.ready, committed.promise);
expect(published).toHaveBeenCalledTimes(1);
expect(authorityAtNotification).toMatchObject([
{
kind: "postimage",
value: { request_id: row.request_id, status: "publishing", execution_id: "execution" },
},
]);
expect(published.mock.calls[0]?.[0]).toMatchObject({
kind: "repository",
rows: [{ request_id: row.request_id, status: "publishing", execution_id: "execution" }],
});
expectRows(row, [{ request_id: row.request_id, status: "publishing" }]);
reply.release();
await expect(claim).resolves.toMatchObject({ rows: [{ status: "publishing" }] });
@@ -588,90 +536,53 @@ it("publishes a committed receipt before a delayed ordinary worker reply", async
expectRows(row, [{ reported_at_ms: expect.any(Number) }]);
});
it.for(["superseded", "closed"] as const)(
"fences a late authority receipt after its owner is %s",
async (outcome, { signal }) => {
const lifecycleInput = {
publicationKind: "shared" as const,
requestId: `late-${outcome}-lifecycle`,
};
runOpenClawStateWriteTransaction(({ db }) =>
insertGitHubPublicationSessionLifecycle(db, {
...lifecycleInput,
lifecycleRevision: "original-revision",
}),
);
const lifecycle = await prepareGitHubPublicationSessionLifecycle(lifecycleInput);
let closed = false;
const scope = createScope({
...context,
assertPublicationCurrent: () => {
if (closed) {
throw new Error("Synthetic publication owner closed");
}
(context.assertPublicationCurrent ?? context.admission.assertCurrent)();
},
});
const row = seed(`late-${outcome}`);
const { facts, unknown } = observeAuthority();
const key = JSON.stringify(["repository", row.request_id]);
const receipt = holdNextReceipt();
const reply = holdNextReply();
const published = vi.fn();
const claim = mutatePublication(
scope,
{ operation: "claim", row, instanceId: "gateway", executionId: "original" },
published,
);
await waitForPublicationGate(signal, claim, receipt.ready, reply.ready);
if (outcome === "superseded") {
runOpenClawStateWriteTransaction((database) =>
claimRepositoryGitHubPublicationInDatabase(
database,
{
...row,
status: "publishing",
execution_id: "original",
gateway_instance_id: "gateway",
},
"gateway",
"replacement",
{
assertCurrent: context.admission.assertCurrent,
assertCustody: context.admission.assertCurrent,
},
),
);
expect(facts.get(key)).toMatchObject({
kind: "postimage",
value: { execution_id: "replacement" },
});
receipt.release();
expect(facts.get(key)).toEqual({ kind: "unknown" });
expect(published).toHaveBeenCalledTimes(1);
} else {
closed = true;
expect(receipt.release).toThrow("publication owner closed");
expect(facts.has(key)).toBe(false);
expect(unknown).toContain(context.admission.identity.key);
expect(published).not.toHaveBeenCalled();
}
reply.release();
await expect(claim).resolves.toMatchObject({ rows: [{ execution_id: "original" }] });
expectRows(row, [{ execution_id: outcome === "superseded" ? "replacement" : "original" }]);
if (outcome === "closed") {
expect(lifecycle.readCurrent).toThrow("lifecycle authority is unavailable");
await expect(prepareGitHubPublicationSessionLifecycle(lifecycleInput)).rejects.toThrow(
"lifecycle authority is unavailable",
);
} else {
expect(lifecycle.readCurrent()).toEqual({
lifecycle_revision: "original-revision",
requester_authority_json: null,
});
}
},
);
it("fences a late authority receipt after its owner closes", async ({ signal }) => {
const lifecycleInput = {
publicationKind: "shared" as const,
requestId: "late-closed-lifecycle",
};
runOpenClawStateWriteTransaction(({ db }) =>
insertGitHubPublicationSessionLifecycle(db, {
...lifecycleInput,
lifecycleRevision: "original-revision",
}),
);
const lifecycle = await prepareGitHubPublicationSessionLifecycle(lifecycleInput);
let closed = false;
const scope = createScope({
...context,
assertPublicationCurrent: () => {
if (closed) {
throw new Error("Synthetic publication owner closed");
}
(context.assertPublicationCurrent ?? context.admission.assertCurrent)();
},
});
const row = seed("late-closed");
const { facts, unknown } = observeAuthority();
const key = JSON.stringify(["repository", row.request_id]);
const receipt = holdNextReceipt();
const reply = holdNextReply();
const published = vi.fn();
const claim = mutatePublication(
scope,
{ operation: "claim", row, instanceId: "gateway", executionId: "original" },
published,
);
await waitForPublicationGate(signal, claim, receipt.ready, reply.ready);
closed = true;
expect(receipt.release).toThrow("publication owner closed");
expect(facts.has(key)).toBe(false);
expect(unknown).toContain(context.admission.identity.key);
expect(published).not.toHaveBeenCalled();
reply.release();
await expect(claim).resolves.toMatchObject({ rows: [{ execution_id: "original" }] });
expectRows(row, [{ execution_id: "original" }]);
expect(lifecycle.readCurrent).toThrow("lifecycle authority is unavailable");
await expect(prepareGitHubPublicationSessionLifecycle(lifecycleInput)).rejects.toThrow(
"lifecycle authority is unavailable",
);
});
it("revokes prepared sources when canonical deletion commits before its ordinary reply", async ({
signal,
@@ -724,32 +635,6 @@ it("revokes prepared sources when canonical deletion commits before its ordinary
expect(() => bindGitHubPublicationSource(source)).toThrow("source authority changed");
});
it("settles execution bookkeeping without source authority while distinguishing observed effects", async () => {
const row = seed("observed-effect");
const execution = await claimRepositoryGitHubPublicationAsync(row, "gateway", {
assertCustody: context.admission.assertCurrent,
assertAction() {
throw new Error("No action authority");
},
async prepareSource() {
throw new Error("No source authority");
},
});
await execution.updateHead("d".repeat(40));
await execution.recordEffect("push");
expectRows(row, [{ effect_state: "dispatched", pushed_head_commit: null }]);
await execution.recordEffect("push", { headCommit: "d".repeat(40) });
await execution.interrupt();
expectRows(row, [
{
status: "requested",
last_effect: "push",
effect_state: "observed",
pushed_head_commit: "d".repeat(40),
},
]);
});
it("refuses delayed observation from an execution replaced by a later claim", async () => {
const scope = createScope();
const row = seed("replaced-execution");
src/state/openclaw-agent-db.postgres.test.ts
+104 -485
View File
@@ -8,8 +8,6 @@ import { isRecord } from "@openclaw/normalization-core/record-coerce";
import pg from "pg";
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import * as doctorEntries from "../commands/doctor/shared/session-entry-rewrite.js";
import { migrateLegacySessionEntryState } from "../commands/doctor/shared/session-entry-shape.js";
import { reconcileSessionTranscriptIndexInTransaction } from "../config/sessions/session-transcript-index.js";
import type { TranscriptTurnBoundary } from "../config/sessions/transcript-entry-anchor.js";
import { createTranscriptEventInserter } from "../config/sessions/transcript-payload.js";
@@ -26,11 +24,9 @@ import { iterateSqliteQuerySync } from "../infra/kysely-sync.js";
import { openNodeSqliteDatabase } from "../infra/node-sqlite.js";
import { isPostgresSqlConnection, type EngineConnection } from "../infra/sql-connection.js";
import { retireSqliteDatabaseAdmissionForPath as retireAdmission } from "../infra/sqlite-database-admission.js";
import { readDatabasePathIdentitySync } from "../infra/sqlite-worker-identity.js";
import { migrateHistoricalTranscriptDirectives } from "../infra/state-migrations.transcript-directives.js";
import { OPENCLAW_AGENT_SCHEMA_VERSION as schemaVersion } from "./openclaw-agent-db-contract.js";
import { agentEngineSchema } from "./openclaw-agent-db-engine.js";
import { readOpenClawAgentDatabaseIdentity as readIdentity } from "./openclaw-agent-db-identity.js";
import { resolveOpenClawAgentSqlitePath } from "./openclaw-agent-db.paths.js";
import {
createDatabaseOpener,
@@ -86,21 +82,6 @@ function exportFts(database: { db: EngineConnection }, sessionId: string) {
);
}
async function inspectSchema(pathname: string, agentId?: string) {
await using inspector = owners.createAgentSchemaInspectionWorker();
const result = await inspector.inspect({
pathname,
...(agentId ? { agentId } : {}),
env: process.env,
supportedVersion: schemaVersion,
verifyCurrentSchemaShape: true,
inspectOwnership: true,
requireStartupMigrationReadiness: true,
});
expect(inspector).toMatchObject({ snapshotCount: 0, processCount: 1 });
return result;
}
async function queryRows<T extends pg.QueryResultRow = pg.QueryResultRow>(
sql: string,
values?: unknown[],
@@ -165,27 +146,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
});
beforeEach(() => vi.stubEnv("OPENCLAW_STATE_DIR", root));
it("reads persisted model catalogs through the worker without local application tables", async () => {
const { loadPersistedPluginModelCatalogs } =
await import("../agents/plugin-model-catalog-execution.js");
const agentId = "model-catalog";
const contents = JSON.stringify({ providers: { fixture: { models: [{ id: "synthetic" }] } } });
const database = openDatabase({ agentId });
database.write(
"INSERT INTO cache_entries (scope, key, value_json, updated_at) VALUES (?, ?, ?, ?)",
"plugin-model-catalog-v1",
"fixture",
contents,
1,
);
const databasePath = database.path;
closeDatabase(databasePath);
expect(await loadPersistedPluginModelCatalogs(path.dirname(databasePath), ["fixture"])).toEqual(
[{ pluginId: "fixture", contents }],
);
expectAnchorOnly(databasePath);
});
it("inspects clean PostgreSQL transcripts and preserves history requiring a file repair backup", async () => {
const target = await createSession("doctor-transcript", "doctor-transcripts", "main");
await owners.appendTranscriptMessage(target, {
@@ -228,69 +188,7 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
expectAnchorOnly(database.path);
});
it("scans and repairs legacy Doctor state while retaining opaque PostgreSQL JSON numbers", () => {
admit(() => {
using database = openDatabase({ agentId: "doctor" });
const sessionKey = "agent:doctor:main";
const raw =
'{"sessionId":"doctor-session","updatedAt":42,"status":"queued","pendingFinalDelivery":true,"pendingFinalDeliveryText":"saved reply","opaqueCount":9007199254740993,"opaque":{"value":123456789012345678901234567890,"label":"retained"}}';
database.run(
"INSERT INTO session_nodes (session_key, current_session_id, entry_json, updated_at) VALUES (?, ?, ?, ?)",
sessionKey,
"doctor-session",
raw,
42,
);
database.run(
"INSERT INTO session_windows (session_id, session_key, session_scope, created_at, updated_at) VALUES (?, ?, 'conversation', ?, ?)",
"doctor-session",
sessionKey,
42,
42,
);
const scope = { agentId: "doctor", storePath: database.path };
const expectedIdentity = readDatabasePathIdentitySync(database.path);
const scanned: Array<{ sessionKey: string; entry: Record<string, unknown> }> = [];
doctorEntries.scanDoctorSessionEntryRecords(
scope,
(record) => scanned.push(record),
expectedIdentity,
);
expect(scanned).toEqual([
{
sessionKey,
entry: expect.objectContaining({ status: "queued", pendingFinalDelivery: true }),
},
]);
expect(
doctorEntries.rewriteDoctorSessionEntries({
scope,
sessionKeys: [sessionKey],
expectedIdentity,
rawTransform: (entry, key, updatedAt) =>
migrateLegacySessionEntryState(entry, updatedAt, key),
}),
).toBe(1);
const repairedRaw = database.get(
"SELECT entry_json FROM session_nodes WHERE session_key = ?",
sessionKey,
)?.entry_json;
assert(typeof repairedRaw === "string");
expect(JSON.parse(repairedRaw)).toMatchObject({
sessionId: "doctor-session",
updatedAt: 42,
status: "interrupted",
pendingFinalDelivery: { kind: "replayable", text: "saved reply", createdAt: 42 },
opaque: { label: "retained" },
});
expect(repairedRaw).not.toContain("pendingFinalDeliveryText");
expect(repairedRaw).toContain("9007199254740993");
expect(repairedRaw).toContain("123456789012345678901234567890");
expectAnchorOnly(database.path);
});
});
it("opens default and explicit stores, retains identity, and closes both handles", () => {
it("closes default and explicit stores with paired handles and a PostgreSQL receipt", () => {
return admit(() => {
for (const options of [
{ agentId: "main" },
@@ -298,19 +196,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
]) {
const database = openDatabase(options);
assert(isPostgresSqlConnection(database.db));
const facts = postgresFacts(database.db);
expect(facts).toMatchObject({ store: "agent", version: schemaVersion });
expect(
database.get(
"SELECT agent_id, role, schema_version FROM schema_meta WHERE meta_key = 'primary'",
),
).toEqual({ agent_id: options.agentId, role: "agent", schema_version: schemaVersion });
const identity = readIdentity(database);
const retained = owners.retainOpenClawAgentDatabaseReadOnly(options);
assert(retained.found);
expect(retained.claim.identity).toBe(identity.identity);
expect(retained.claim.isCurrent()).toBe(true);
retained.claim.release();
const anchor = database.db.anchor;
closeReceipts.mockClear();
expect(owners.closeOpenClawAgentDatabaseByPath(database.path)).toBe(true);
@@ -319,12 +204,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
`agent database clean-close receipt: postgres-closed; agent=${options.agentId}`,
expect.objectContaining({ agentId: options.agentId, receipt: "postgres-closed" }),
);
const reopened = owners.openOpenClawAgentDatabase(options);
assert(isPostgresSqlConnection(reopened.db));
expect(postgresFacts(reopened.db)?.namespace).toBe(facts?.namespace);
expect(readIdentity(reopened).identity).toBe(identity.identity);
expect(readIdentity(reopened).incarnation).not.toBe(identity.incarnation);
closeDatabase(reopened.path);
}
});
});
@@ -341,10 +220,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
},
};
const assertMemorySearch = (db: EngineConnection) => {
const extensions = db.prepare("SELECT search_extensions FROM openclaw_schema_meta").get();
expect(JSON.parse(String(extensions?.search_extensions))).toEqual({
"memory.text": memorySearch,
});
expect(
db
.prepare(`SELECT id, text FROM memory_index_chunks
@@ -485,27 +360,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
server_major: facts.capabilities.serverMajor,
});
});
const indexes = await queryRows(
`SELECT indexname, indexdef FROM pg_indexes WHERE schemaname = $1
AND indexname IN ('idx_agent_session_windows_conversation', 'idx_agent_transcript_event_parent',
'idx_agent_session_nodes_last_interaction_at', 'idx_agent_session_reactions_message')`,
[facts.namespace],
);
expect(indexes).toEqual([
{
indexname: "idx_agent_session_windows_conversation",
indexdef: `CREATE INDEX idx_agent_session_windows_conversation ON ${facts.namespace}.session_windows USING btree (primary_conversation_id NULLS FIRST) WHERE (primary_conversation_id IS NOT NULL)`,
},
]);
expect(
await queryRows(
`SELECT p.proname FROM pg_proc p
JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = $1 AND p.proname IN
('openclaw_memory_index_sources_trigger', 'openclaw_memory_index_chunks_trigger')`,
[facts.namespace],
),
).toEqual([]);
expect(
await queryRows(`SELECT revision FROM "${facts.namespace}".memory_index_state`),
).toEqual([{ revision: "1" }]);
@@ -543,29 +397,6 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
}
});
it("discovers a cold explicit store owner through the selected reader worker", async () => {
const pathname = path.join(root, "explicit", "openclaw-agent.sqlite");
admit(() => {
owners.openOpenClawAgentDatabase({ agentId: "alternate", path: pathname });
owners.closeOpenClawAgentDatabaseByPath(pathname);
});
await owners.closeOpenClawAgentDatabaseByPathAsync(pathname);
const { readActiveOpenClawAgentDatabaseLeasesReadOnly } =
await import("./openclaw-agent-db-lease.js");
const activeLeases = () =>
admit(() =>
readActiveOpenClawAgentDatabaseLeasesReadOnly({ env: process.env }).filter(
(lease) => lease.path === pathname,
),
);
expect(activeLeases()).toEqual([]);
retireAdmission(pathname);
expect(await inspectSchema(pathname)).toMatchObject({
agentSchemaMeta: { agentId: "alternate", role: "agent", schemaVersion },
});
expect(activeLeases()).toEqual([]);
});
it("appends and hydrates plain transcript JSON and drains interleaved turns through the agent worker in FIFO order", async () => {
const {
appendTranscriptMessage,
@@ -601,29 +432,11 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
});
parentId = reply.messageId;
}
const hydrated = await prepareSessionTranscriptHydration(target).read();
expect(
hydrated.snapshot.events.filter((event) => isRecord(event) && event.type === "message"),
).toHaveLength(7);
expect(await owners.searchSessionTranscripts({ ...target, query: "progress" })).toMatchObject({
hits: [
{
sessionId: target.sessionId,
sessionKey: target.sessionKey,
role: "assistant",
snippet: expect.stringContaining("progress"),
},
],
indexing: false,
});
const database = openDatabase({ agentId: "actor" });
assert(isPostgresSqlConnection(database.db));
const facts = postgresFacts(database.db);
assert(facts);
expect(await inspectSchema(database.path, "actor")).toMatchObject({
version: schemaVersion,
agentSchemaMeta: { agentId: "actor", role: "agent", schemaVersion },
});
const hydrated = await prepareSessionTranscriptHydration(target).read();
const payloads = await queryRows<{ event_json: string; event_zstd: Buffer | null }>(
`SELECT event_json, event_zstd FROM "${facts.namespace}".transcript_events ORDER BY seq`,
);
@@ -685,223 +498,92 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
).toEqual([{ count: "0" }]);
});
it("searches active branches through the worker with session and role filters and lifecycle deletion", async () => {
const target = targetFor("search-branch");
const sibling = {
...target,
sessionId: "search-sibling",
sessionKey: "agent:actor:search-sibling",
it("preserves FTS row identity for nullable and repeated IDs across bounded inserts", async () => {
const count = 515;
const sessionId = `fts-identities-${count}`;
const options = { agentId: "actor" };
const database = openDatabase(options);
const postgres = database.db;
assert(isPostgresSqlConnection(postgres));
const facts = postgresFacts(postgres);
assert(facts);
// Cold restoration preserves these nullable/repeated stored IDs; normal
// transcript append filters absent IDs and deduplicates event identities.
const entries = Array.from({ length: count }, (_, index) => ({
messageId: index % 4 < 2 ? null : "repeated",
text: `ftskeytoken${index}`,
role: index % 2 ? "assistant" : "user",
timestamp: index,
}));
let statements = 0;
const observation = channel("openclaw.database.statement");
const observe = (value: unknown) => {
if (isRecord(value) && value.engine === "postgres") {
statements += 1;
}
};
for (const scope of [target, sibling]) {
await owners.upsertSessionEntryCore(scope, { sessionId: scope.sessionId, updatedAt: 1 });
}
await owners.appendTranscriptMessage(target, {
eventId: "search-user",
parentId: null,
message: { role: "user", content: "searchable alpha origin" },
});
await owners.appendTranscriptMessage(target, {
eventId: "search-assistant",
parentId: "search-user",
message: { role: "assistant", content: "searchable beta abandoned" },
});
await owners.appendTranscriptMessage(sibling, {
eventId: "search-sibling-user",
message: { role: "user", content: "searchable survivor" },
});
const search = {
agentId: target.agentId,
storePath: target.storePath,
env: process.env,
query: "searchable",
sessionKeys: [target.sessionKey, sibling.sessionKey],
};
const find = (overrides: Partial<Parameters<typeof owners.searchSessionTranscripts>[0]> = {}) =>
owners.searchSessionTranscripts({ ...search, ...overrides });
expect((await find({ role: "assistant" })).hits).toEqual([
expect.objectContaining({ messageId: "search-assistant" }),
]);
expect((await find({ sessionId: sibling.sessionId })).hits).toEqual([
expect.objectContaining({ messageId: "search-sibling-user" }),
]);
expect(
(await find({ sessionKeys: [target.sessionKey] })).hits
.map((hit) => hit.messageId)
.toSorted(),
).toEqual(["search-assistant", "search-user"]);
await owners.appendTranscriptEvent(target, {
type: "leaf",
id: "search-leaf",
parentId: "search-assistant",
targetId: "search-user",
});
expect((await find({ query: "beta" })).hits).toEqual([]);
const options = { agentId: target.agentId, env: process.env };
await owners.waitForSessionTranscriptIndexReconcile(options);
if ((await find()).indexing) {
await owners.reconcileSessionTranscriptIndexes(options);
}
expect(await find({ query: "alpha" })).toMatchObject({
hits: [expect.objectContaining({ messageId: "search-user" })],
indexing: false,
});
expect((await find({ query: "beta" })).hits).toEqual([]);
const removal = {
...target,
target: { canonicalKey: target.sessionKey, storeKeys: [target.sessionKey] },
archiveTranscript: false,
deleteTranscriptWithoutArchive: true,
};
await owners.upsertSessionEntryCore(sibling, { previousSessionId: target.sessionId });
expect(await owners.deleteSessionEntryLifecycle(removal)).toMatchObject({ deleted: true });
expect((await find({ query: "alpha" })).hits).toEqual([
expect.objectContaining({ sessionId: target.sessionId, messageId: "search-user" }),
]);
await owners.upsertSessionEntryCore(target, { sessionId: target.sessionId, updatedAt: 2 });
await owners.upsertSessionEntryCore(sibling, { previousSessionId: undefined });
expect(
await owners.deleteSessionEntryLifecycle({ ...removal, archiveTranscript: true }),
).toMatchObject({ deleted: true, archivedTranscripts: [expect.anything()] });
expect((await find()).hits).toEqual([
expect.objectContaining({ sessionId: sibling.sessionId, messageId: "search-sibling-user" }),
]);
});
it("indexes only accepted searchable messages from a transcript append batch", async () => {
const target = await createSession("fts-append-batch");
function event(id: string, parentId: string | null, role: string, content: string) {
return { type: "message", id, parentId, timestamp: 1, message: { role, content } };
}
admit(() =>
owners.runOpenClawAgentWriteTransaction(
(database) => {
expect(
owners.appendTranscriptEventsInTransaction(database, target, [
event("batch-user", null, "user", "batchneedle accepted origin"),
event("batch-user", null, "user", "batchneedle rejected duplicate"),
event("batch-tool", "batch-user", "toolResult", "batchneedle hidden tool"),
event("batch-reply", "batch-tool", "assistant", "batchneedle accepted reply"),
]),
).toBe(3);
},
{ agentId: target.agentId },
),
);
const search = (query: string) => owners.searchSessionTranscripts({ ...target, query });
expect((await search("batchneedle")).hits.map((hit) => hit.messageId).toSorted()).toEqual([
"batch-reply",
"batch-user",
]);
expect((await search("rejected")).hits).toEqual([]);
expect((await search("hidden")).hits).toEqual([]);
});
it.each([4, 515])(
"preserves FTS row identity for %i nullable and repeated message IDs in bounded statements",
async (count) => {
const sessionId = `fts-identities-${count}`;
const options = { agentId: "actor" };
const database = openDatabase(options);
const postgres = database.db;
assert(isPostgresSqlConnection(postgres));
const facts = postgresFacts(postgres);
assert(facts);
// Cold restoration preserves these nullable/repeated stored IDs; normal
// transcript append filters absent IDs and deduplicates event identities.
const entries = Array.from({ length: count }, (_, index) => ({
messageId: index % 4 < 2 ? null : "repeated",
text: `ftskeytoken${index}`,
role: index % 2 ? "assistant" : "user",
timestamp: index,
}));
let statements = 0;
const observation = channel("openclaw.database.statement");
const observe = (value: unknown) => {
if (isRecord(value) && value.engine === "postgres") {
statements += 1;
owners.runOpenClawAgentWriteTransaction(({ db }) => {
const insert = owners.createSessionTranscriptFtsInserter(db, sessionId);
observation.subscribe(observe);
try {
insert(entries);
} finally {
observation.unsubscribe(observe);
}
};
admit(() =>
owners.runOpenClawAgentWriteTransaction(({ db }) => {
const insert = owners.createSessionTranscriptFtsInserter(db, sessionId);
observation.subscribe(observe);
try {
insert(entries);
} finally {
observation.unsubscribe(observe);
}
}, options),
);
const persisted = await queryRows(
`SELECT r.id, r.id = f.rowid AS same_id, r.message_id AS owner_message_id,
}, options),
);
const persisted = await queryRows(
`SELECT r.id, r.id = f.rowid AS same_id, r.message_id AS owner_message_id,
f.message_id, f.text, f.role, f.timestamp
FROM "${facts.namespace}".session_transcript_fts_rows r
LEFT JOIN "${facts.namespace}".session_transcript_fts f ON f.rowid = r.id
WHERE r.session_id = $1 ORDER BY f.timestamp`,
[sessionId],
[sessionId],
);
const expected = entries.map(({ messageId, ...entry }) => ({
message_id: messageId,
...entry,
}));
expect(persisted.map(({ id: _id, ...row }) => row)).toEqual(
expected.map((entry) => ({ same_id: true, owner_message_id: entry.message_id, ...entry })),
);
const ids = persisted.map((row) => BigInt(row.id));
for (let index = 1; index < ids.length; index++) {
expect(ids[index]).toBeGreaterThan(ids[index - 1]!);
}
expect(admit(() => exportFts(database, sessionId))).toEqual(expected);
// Each observed PostgreSQL statement makes one bridge/server request.
expect(statements).toBe(2);
admit(() => {
// Keep the first bridge open so the pooled writer uses another backend.
const next = owners.openAgentEngineStore(
postgres.anchor,
options.agentId,
database.path,
true,
);
const expected = entries.map(({ messageId, ...entry }) => ({
message_id: messageId,
...entry,
}));
expect(persisted.map(({ id: _id, ...row }) => row)).toEqual(
expected.map((entry) => ({ same_id: true, owner_message_id: entry.message_id, ...entry })),
);
const ids = persisted.map((row) => BigInt(row.id));
for (let index = 1; index < ids.length; index++) {
expect(ids[index]).toBeGreaterThan(ids[index - 1]!);
}
expect(admit(() => exportFts(database, sessionId))).toEqual(expected);
for (const index of [0, 1, 2, 3]) {
const searched = await queryRows(
`SELECT message_id, text FROM "${facts.namespace}".session_transcript_fts
WHERE session_id = $1 AND openclaw_search @@
plainto_tsquery('"${facts.namespace}".openclaw_simple_unaccent', $2)`,
[sessionId, `ftskeytoken${index}`],
assert(isPostgresSqlConnection(next));
try {
expect(next.prepare("SELECT pg_backend_pid() AS pid").get()?.pid).not.toBe(
postgres.prepare("SELECT pg_backend_pid() AS pid").get()?.pid,
);
expect(searched).toEqual([
{ message_id: index < 2 ? null : "repeated", text: `ftskeytoken${index}` },
]);
next.runTransaction(() =>
owners.createSessionTranscriptFtsInserter(next, `${sessionId}-next`)(entries.slice(0, 2)),
);
const later = next
.prepare("SELECT id FROM session_transcript_fts_rows WHERE session_id = ? ORDER BY id")
.all(`${sessionId}-next`);
expect(later).toHaveLength(2);
expect(BigInt(String(later[0]!.id))).toBeGreaterThan(ids.at(-1)!);
expect(BigInt(String(later[1]!.id))).toBeGreaterThan(BigInt(String(later[0]!.id)));
} finally {
next.close();
}
// Each observed PostgreSQL statement makes one bridge/server request.
expect(statements).toBe(count === 4 ? 1 : 2);
if (count > 512) {
admit(() => {
// Keep the first bridge open so the pooled writer uses another backend.
const next = owners.openAgentEngineStore(
postgres.anchor,
options.agentId,
database.path,
true,
);
assert(isPostgresSqlConnection(next));
try {
expect(next.prepare("SELECT pg_backend_pid() AS pid").get()?.pid).not.toBe(
postgres.prepare("SELECT pg_backend_pid() AS pid").get()?.pid,
);
next.runTransaction(() =>
owners.createSessionTranscriptFtsInserter(
next,
`${sessionId}-next`,
)(entries.slice(0, 2)),
);
const later = next
.prepare(
"SELECT id FROM session_transcript_fts_rows WHERE session_id = ? ORDER BY id",
)
.all(`${sessionId}-next`);
expect(later).toHaveLength(2);
expect(BigInt(String(later[0]!.id))).toBeGreaterThan(ids.at(-1)!);
expect(BigInt(String(later[1]!.id))).toBeGreaterThan(BigInt(String(later[0]!.id)));
} finally {
next.close();
}
});
}
},
);
});
});
it("matches SQLite recent-search cutoffs and cold-export order for equal-timestamp batches", () => {
it("preserves recent-search cutoffs and cold-export order for equal-timestamp batches", () => {
const events = Array.from({ length: 130 }, (_, index) => ({
type: "message",
id: `ordered-${index}`,
@@ -909,99 +591,36 @@ describe.skipIf(!connection)("PostgreSQL agent database owner", () => {
timestamp: 1,
message: { role: "user", content: `chronologyneedle entry ${index}` },
}));
const results = (["sqlite", "postgres"] as const).map((engine) =>
engineSelection.withDatabaseEngineSelection(
{
effectiveEngines: resolveDatabaseFamilyEngines({ database: { engine } }),
postgres: { connection: connection!, schemaPrefix: prefix },
},
() =>
admit(() => {
const options = {
agentId: "actor",
...(engine === "sqlite"
? { path: path.join(root, "sqlite-fts-order", "agent.sqlite") }
: {}),
};
const target = {
...options,
sessionId: "fts-chronology",
sessionKey: "agent:actor:fts-chronology",
};
using database = openDatabase(options);
owners.runOpenClawAgentWriteTransaction((writer) => {
expect(
owners.appendTranscriptEventsInTransaction(writer, target, events, {
scheduleProjectionReconcile: false,
}),
).toBe(events.length);
}, options);
const exported = exportFts(database, target.sessionId);
expect(exported.map((row) => row.message_id)).toEqual(events.map((event) => event.id));
return [1, 3, 25].map((limit) => {
const result = owners.searchSessionTranscriptsReadOnlySync(
{ ...target, query: "chronologyneedle", order: "recent", limit },
{ ...options, path: database.path },
);
const ids = result.hits.map((hit) => hit.messageId);
expect(ids).toEqual(
events
.slice(-limit)
.toReversed()
.map((event) => event.id),
);
return ids;
});
admit(() => {
const options = { agentId: "actor" };
const target = {
...options,
sessionId: "fts-chronology",
sessionKey: "agent:actor:fts-chronology",
};
using database = openDatabase(options);
owners.runOpenClawAgentWriteTransaction((writer) => {
expect(
owners.appendTranscriptEventsInTransaction(writer, target, events, {
scheduleProjectionReconcile: false,
}),
),
);
expect(results[1]).toEqual(results[0]);
});
it("reads persisted auth through the selected owner and agent worker", async () => {
const store = {
version: 1,
profiles: { fixture: { type: "api_key", provider: "fixture", key: "synthetic" } },
};
for (const options of [
{ agentId: "main" },
{ agentId: "alternate", path: path.join(root, "explicit", "openclaw-agent.sqlite") },
]) {
const database = openDatabase(options);
admit(() =>
owners.runOpenClawAgentWriteTransaction(
({ db }) => owners.writeAuthProfileJsonCell(db, "store", "agent", store),
options,
),
).toBe(events.length);
}, options);
expect(exportFts(database, target.sessionId).map((row) => row.message_id)).toEqual(
events.map((event) => event.id),
);
expect(
admit(() => owners.inspectPersistedAuthProfileStoreRaw(path.dirname(database.path))),
).toEqual({ status: "readable", raw: store });
closeDatabase(database.path);
await owners.closeOpenClawAgentDatabaseByPathAsync(database.path);
const reader = owners.prepareAgentAuthProfileRowsRead({
databasePath: database.path,
agentId: options.agentId,
env: process.env,
});
try {
expect((await reader.read()).store).toEqual({ status: "readable", raw: store });
} finally {
await reader.dispose();
for (const limit of [1, 3, 25]) {
const result = owners.searchSessionTranscriptsReadOnlySync(
{ ...target, query: "chronologyneedle", order: "recent", limit },
{ ...options, path: database.path },
);
expect(result.hits.map((hit) => hit.messageId)).toEqual(
events
.slice(-limit)
.toReversed()
.map((event) => event.id),
);
}
}
const sharedMainPath = path.join(root, "agents", "main", "agent", "openclaw-agent.sqlite");
await owners.closeOpenClawAgentDatabaseByPathAsync(sharedMainPath);
retireAdmission(sharedMainPath);
expect(
await owners.prepareFreshSharedAuthStoreWriteAsync({
agentDir: undefined,
allowExplicitMain: false,
env: process.env,
}),
).toBe(true);
expect(owners.resolveSharedAuthStoreOwnership(process.env)).toEqual({
location: "legacy-main",
});
});
src/state/openclaw-agent-worker-store.test.ts
+42 -376
View File
@@ -1,7 +1,6 @@
import { channel } from "node:diagnostics_channel";
import fs from "node:fs";
import path from "node:path";
import { DatabaseSync } from "node:sqlite";
import { setImmediate as nextTurn } from "node:timers/promises";
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
@@ -10,15 +9,12 @@ import {
openFixtureReceiptChannel,
type FixtureReceiptChannel,
} from "../../test/helpers/fixture-receipts.js";
import { awaitGateBeforeSettlement, withinTest } from "../../test/helpers/promise.js";
import { withinTest } from "../../test/helpers/promise.js";
import { useSqliteWorkerFault } from "../../test/helpers/sqlite-worker-fault.js";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import { resolveRuntimeWorkerUrl } from "../infra/runtime-worker-url.js";
import { readSqliteDatabaseCleanClose } from "../infra/sqlite-database-admission.js";
import { SQLITE_IDLE_HANDLE_TTL_MS } from "../infra/sqlite-handle-lifecycle.js";
import { throwSqliteLifecycleErrors } from "../infra/sqlite-lifecycle-errors.js";
import { observeSqliteWalPeriodicWork } from "../infra/sqlite-wal-scheduler.test-support.js";
import * as sqliteWal from "../infra/sqlite-wal.js";
import { readDatabasePathIdentitySync } from "../infra/sqlite-worker-identity.js";
import * as admission from "../infra/sqlite-worker-operation-admission.js";
import { sqliteWorkerOwnerProbe as probe } from "../infra/sqlite-worker-owner-probe.test-support.js";
@@ -31,21 +27,12 @@ import {
} from "../process/gateway-work-admission.js";
import { createDeferredCore } from "../shared/deferred.js";
import { requireNativeSqliteDatabase } from "../test-utils/native-sqlite.js";
import {
closeOpenClawAgentDatabasesAsync,
retainAgentDatabase,
} from "./openclaw-agent-db-lifecycle.js";
import {
isAgentDatabaseClosing,
registerOpenClawAgentDatabaseAsyncResource,
revokeAgentDatabaseResources,
} from "./openclaw-agent-db-resources.js";
import { closeOpenClawAgentDatabasesAsync } from "./openclaw-agent-db-lifecycle.js";
import { revokeAgentDatabaseResources } from "./openclaw-agent-db-resources.js";
import { withOpenClawAgentDatabaseWrite } from "./openclaw-agent-db-write.js";
import {
openOpenClawAgentDatabase as openDatabase,
closeOpenClawAgentDatabasesForTest,
withOpenClawAgentDatabaseAdmission,
withOpenClawAgentDatabaseAsync,
} from "./openclaw-agent-db.js";
import type { OpenClawAgentDatabaseExecution } from "./openclaw-agent-execution-contract.js";
import { captureOpenClawAgentDatabaseExecution as captureExecution } from "./openclaw-agent-execution.js";
@@ -159,81 +146,8 @@ it("leaves agent worker checkpoints to maintenance instead of foreground commits
expect(await worker.execute({ type: "inspect", input: "wal_autocheckpoint" }, allow)).toBe(0);
});
it("renews idle retention on reborrow", async () => {
const { db, worker } = await setup();
const shared = openOpenClawStateDatabase();
const releaseState = retainOpenClawStateDatabaseForIdle(shared);
const readLeases = leaseReader(shared);
const hostLeases = readLeases();
const publish = (value: string) => worker.run((scope) => scope.execute(append(value)), allow);
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
try {
await publish("first");
const retainedLeases = readLeases();
expect(retainedLeases).toHaveLength(hostLeases.length + 1);
await vi.advanceTimersByTimeAsync(SQLITE_IDLE_HANDLE_TTL_MS - 1);
const borrowed = captureExecution(options);
executions.add(borrowed);
await publish("reborrowed");
expect(readLeases()).toEqual(retainedLeases);
await vi.advanceTimersByTimeAsync(SQLITE_IDLE_HANDLE_TTL_MS);
expect(readLeases()).toEqual(retainedLeases);
await borrowed.release();
executions.delete(borrowed);
await vi.advanceTimersByTimeAsync(SQLITE_IDLE_HANDLE_TTL_MS - 1);
expect(readLeases()).toEqual(retainedLeases);
await vi.advanceTimersByTimeAsync(1);
// A new request joins the expired generation's cleanup before reopening it.
await publish("after idle");
expect(readLeases()).toHaveLength(retainedLeases.length);
expect(readLeases()).not.toEqual(retainedLeases);
expectValues(db, ["first", "reborrowed", "after idle"]);
} finally {
vi.useRealTimers();
releaseState();
}
});
it("retains warm execution across fallback preparation", async () => {
const { db, worker } = await setup();
const firstThread = await worker.execute(append("before preparation"), allow);
const lifetime = captureExecution(options);
executions.add(lifetime);
await worker.close();
const entered = createDeferredCore();
const resume = createDeferredCore();
let next: OpenClawAgentSqliteWorkerStore<AgentOperations> | undefined;
const publication = (async () => {
entered.resolve();
await resume.promise;
next = await openWorker(db);
return next.run((scope) => scope.execute(append("after preparation")), allow);
})();
try {
await entered.promise;
markGatewayRestartDraining();
resume.resolve();
expect(await publication).toBe(firstThread);
expectValues(db, ["before preparation", "after preparation"]);
await next?.close();
expect(db.isOpen).toBe(true);
beginGatewayShutdownCleanup();
await lifetime.release();
expect(db.isOpen).toBe(false);
expect(readSqliteDatabaseCleanClose(options.path)).toBe(true);
} finally {
resume.resolve();
await Promise.allSettled([publication]);
await Promise.allSettled([next?.close(), lifetime.release(), worker.close()]);
executions.delete(lifetime);
resetGatewayWorkAdmission();
}
});
it.each([false, true])(
"releases shutdown leases unless retained by an accepted sequence (%s)",
"reuses retained publications across reopen and releases shutdown leases (%s)",
async (retainGeneration) => {
const { db, worker } = await setup();
const execution = retainGeneration ? captureExecution(options) : undefined;
@@ -244,6 +158,7 @@ it.each([false, true])(
const releaseState = retainOpenClawStateDatabaseForIdle(shared);
const readLeases = leaseReader(shared);
const hostLeases = readLeases();
let following = worker;
try {
const firstThread = await worker.run(async (scope) => {
const thread = await scope.execute(append("first"));
@@ -255,10 +170,16 @@ it.each([false, true])(
expect(secondThread).toBe(firstThread);
expect(retainedLeases).toHaveLength(hostLeases.length + 1);
expect(readLeases()).toEqual(retainedLeases);
await worker.run((scope) => scope.execute(append("third")), allow);
if (retainGeneration) {
await worker.close();
}
following = retainGeneration ? await openWorker(db) : worker;
expect(await following.run((scope) => scope.execute(append("third")), allow)).toBe(
firstThread,
);
expect(readLeases()).toEqual(retainedLeases);
beginGatewayShutdownCleanup();
const cleanupThread = await worker.execute(append("cleanup"), allow);
const cleanupThread = await following.execute(append("cleanup"), allow);
if (retainGeneration) {
expect(cleanupThread).toBe(firstThread);
expect(readLeases()).toEqual(retainedLeases);
@@ -266,12 +187,13 @@ it.each([false, true])(
expect(readLeases()).toEqual(hostLeases);
}
expectValues(db, ["first", "second", "third", "cleanup"]);
await worker.close();
await following.close();
await execution?.release();
expect(db.isOpen).toBe(false);
expect(readLeases()).toEqual([]);
expect(readSqliteDatabaseCleanClose(options.path)).toBe(true);
} finally {
await following.close();
await worker.close();
await execution?.release();
if (execution) {
@@ -283,136 +205,6 @@ it.each([false, true])(
},
);
it.each([false, true])(
"certifies native close only after its WAL is empty (pinned: %s)",
async (pinned) => {
const { db, worker } = await setup();
const releaseHost = retainAgentDatabase(db);
let reader: DatabaseSync | undefined;
try {
await worker.execute(append("worker commit"), () => {});
beginGatewayShutdownCleanup();
await worker.close();
if (pinned) {
reader = new DatabaseSync(options.path, { readOnly: true });
reader.exec("BEGIN");
reader.prepare("SELECT value FROM worker_proof").all();
}
db.exec("INSERT INTO worker_proof VALUES ('after worker checkpoint')");
const prepare = vi.spyOn(db, "prepare");
const exec = vi.spyOn(db, "exec");
await closeOpenClawAgentDatabasesAsync(root);
expect(prepare).not.toHaveBeenCalled();
// Retirement may change lock policy, but must not run host-thread data SQL.
expect(exec.mock.calls).toEqual([["PRAGMA busy_timeout = 0"]]);
expect(readSqliteDatabaseCleanClose(options.path)).toBe(!pinned);
resetGatewayWorkAdmission();
const reopened = openDatabase(options);
expect(reopened.db.prepare("SELECT value FROM worker_proof ORDER BY rowid").all()).toEqual([
{ value: "worker commit" },
{ value: "after worker checkpoint" },
]);
} finally {
reader?.close();
releaseHost();
resetGatewayWorkAdmission();
}
},
);
it("closes siblings while a publication retains its database", ({ signal }) =>
fixture.run(async () => {
const { db, worker } = await setup();
const heldPath = options.path;
const healthy = openDatabase({
agentId: "healthy",
path: path.join(root, "healthy.sqlite"),
});
const shared = openOpenClawStateDatabase();
const readLeases = leaseReader(shared);
const entered = createDeferredCore();
const release = createDeferredCore();
const healthyClosed = createDeferredCore();
const nativeClose = healthy.db.close.bind(healthy.db);
vi.spyOn(healthy.db, "close").mockImplementation(() => {
nativeClose();
healthyClosed.resolve();
});
const publication = worker.run(async (scope) => {
const result = await scope.execute(append("committed"));
entered.resolve();
await release.promise;
return result;
}, allow);
let closing: Promise<void> | undefined;
let closed = false;
try {
await withinTest(
awaitGateBeforeSettlement(
entered.promise,
publication,
"Worker publication was not retained",
),
signal,
);
expect(readLeases(heldPath)).toHaveLength(2);
expect(readLeases(healthy.path)).toHaveLength(1);
closing = closeOpenClawAgentDatabasesAsync(root).then(() => {
closed = true;
});
await withinTest(
awaitGateBeforeSettlement(
healthyClosed.promise,
closing,
"Root close settled before the healthy database closed",
),
signal,
);
expect(healthy.db.isOpen).toBe(false);
expect(readLeases(healthy.path)).toEqual([]);
expect(readSqliteDatabaseCleanClose(healthy.path)).toBe(true);
expect(db.isOpen).toBe(true);
expect(readLeases(heldPath)).toHaveLength(2);
expect(readSqliteDatabaseCleanClose(heldPath)).toBe(false);
expect(closed).toBe(false);
expect(isAgentDatabaseClosing(healthy)).toBe(true);
const later = { agentId: "late", path: path.join(root, "late.sqlite") };
expect(() => openDatabase(later)).toThrow("resources are closing");
await expect(withOpenClawAgentDatabaseAsync(later, (database) => database)).rejects.toThrow(
"resources are closing",
);
await expect(
withOpenClawAgentDatabaseAdmission(
later,
async (run) => run(() => {}),
(database) => database,
),
).rejects.toThrow("resources are closing");
expect(fs.existsSync(later.path)).toBe(false);
expect(readLeases(later.path)).toEqual([]);
expect(openDatabase(options).db).toBe(db);
expect(() =>
registerOpenClawAgentDatabaseAsyncResource({
agentId: "new",
path: path.join(root, "new.sqlite"),
revoke() {},
close: async () => {},
}),
).toThrow("resources are closing");
release.resolve();
await withinTest(publication, signal);
await withinTest(closing, signal);
expect(db.isOpen).toBe(false);
expect(readLeases(heldPath)).toEqual([]);
expect(readSqliteDatabaseCleanClose(heldPath)).toBe(true);
expect(isAgentDatabaseClosing(healthy)).toBe(false);
expect(openDatabase(later).db.isOpen).toBe(true);
} finally {
release.resolve();
await Promise.allSettled([publication, closing]);
}
}));
describe("pooled agent publication owner (captured source)", () => {
beforeEach(() => {
sourceMode = "captured";
@@ -579,110 +371,36 @@ describe("pooled agent publication owner (captured source)", () => {
expectValues(db, ["first", "following"]);
});
it("preserves command success despite binding cleanup failure", async () => {
const { db, worker } = await setup({ closeFailure: "fixture binding cleanup failed" });
expect(await worker.execute(append("committed"), allow)).toBeGreaterThan(0);
expectValues(db, ["committed"]);
it("preserves command admission errors when cleanup admission fails", async () => {
const { db, worker } = await setup({ cleanupAdmission: true });
const commandRefusal = new Error("controlled command admission refusal");
let refusals = 0;
const interception = probe.admission(admission, (request, grant, admit) => {
if (request.stage === "transaction") {
throw commandRefusal;
}
if (isRecord(request.facts) && request.facts.kind === "fixture-cleanup") {
refusals++;
throw new Error("controlled publication cleanup admission refusal");
}
admit(request, grant);
});
await expect(worker.execute(append("failed"), allow)).rejects.toBe(commandRefusal);
expect(refusals).toBe(1);
expectValues(db, []);
interception.mockRestore();
const following = await openWorker(db);
expect(await following.execute(append("follower"), allow)).toBeGreaterThan(0);
expectValues(db, ["committed", "follower"]);
expectValues(db, ["follower"]);
});
it.each(["ordinary", "admission"] as const)(
"preserves %s errors when cleanup admission fails",
async (failure) => {
const { db, worker } = await setup({ cleanupAdmission: true });
db.exec(`
CREATE TRIGGER fail_command BEFORE INSERT ON worker_proof
WHEN NEW.value = 'failed'
BEGIN
SELECT RAISE(ABORT, 'controlled command failure');
END
`);
const commandRefusal = new Error("controlled command admission refusal");
let commandRefusals = 0;
let refusals = 0;
const interception = probe.admission(admission, (request, grant, admit) => {
if (failure === "admission" && request.stage === "transaction") {
commandRefusals++;
throw commandRefusal;
}
if (isRecord(request.facts) && request.facts.kind === "fixture-cleanup") {
refusals++;
throw new Error("controlled publication cleanup admission refusal");
}
admit(request, grant);
});
const result = worker.execute(append("failed"), allow);
if (failure === "admission") {
await expect(result).rejects.toBe(commandRefusal);
} else {
await expect(result).rejects.toThrow("controlled command failure");
}
expect(commandRefusals).toBe(failure === "admission" ? 1 : 0);
expect(refusals).toBe(1);
expectValues(db, []);
interception.mockRestore();
const following = await openWorker(db);
expect(await following.execute(append("follower"), allow)).toBeGreaterThan(0);
expectValues(db, ["follower"]);
},
);
it("does not grant a cleanup transaction after a single read", async () => {
const { db, worker } = await setup({ closeWriteValue: "must not commit" });
await expect(worker.execute({ type: "inspect", input: undefined }, allow)).resolves.toBe(0);
expectValues(db, []);
});
it("refuses factory entry without terminating siblings", async () => {
const siblings = [];
const resume = createDeferredCore();
const held: Promise<void>[] = [];
try {
for (let index = 0; index < 4; index++) {
options = { ...options, path: path.join(root, "agent-" + index + ".sqlite") };
const sibling = await setup();
siblings.push(sibling);
const entered = createDeferredCore();
const work = sibling.worker.run(async (scope) => {
entered.resolve();
await resume.promise;
await scope.execute(append("survived"));
}, allow);
held.push(work);
void work.catch(allow);
await Promise.race([entered.promise, work]);
}
let refused = 0;
const interception = probe.admission(admission, (request, grant, admit) => {
if (request.stage === "open") {
refused++;
throw new Error("controlled opening revocation");
}
admit(request, grant);
});
options = { ...options, path: path.join(root, "refused.sqlite") };
const openMarker = path.join(root, "factory-entered");
const { worker: refusedWorker } = await setup({ openMarker });
await expect(
refusedWorker.run((scope) => scope.execute(append("refused")), allow),
).rejects.toThrow("controlled opening revocation");
interception.mockRestore();
expect(refused).toBe(1);
expect(fs.existsSync(openMarker)).toBe(false);
resume.resolve();
await Promise.all(held);
for (const { db } of siblings) {
expectValues(db, ["survived"]);
}
} finally {
resume.resolve();
await Promise.allSettled(held);
}
});
it("preserves commits and queued followers after cleanup refusal", async () => {
const { db, worker } = await setup();
const committed = createDeferredCore<number>();
@@ -780,49 +498,6 @@ describe("pooled agent publication owner (captured source)", () => {
}
throwSqliteLifecycleErrors(failures, "Publication fixture and cleanup failed");
});
it("queues a real periodic maintenance tick behind publication", ({ signal }) =>
fixture.run(async () => {
signal.throwIfAborted();
let capturing = false;
const scheduled = observeSqliteWalPeriodicWork(() => capturing);
const configure = sqliteWal.configureSqliteConnectionPragmas;
vi.spyOn(sqliteWal, "configureSqliteConnectionPragmas").mockImplementation((db, policy) => {
capturing = policy?.databasePath === options.path;
try {
return configure(db, policy);
} finally {
capturing = false;
}
});
const { db, worker } = await setup().finally(scheduled.restore);
const tick = scheduled.periodic;
db.exec(`INSERT INTO cache_entries(scope, key, blob, updated_at)
VALUES ('maintenance-proof', 'pages', zeroblob(4194304), 1);
DELETE FROM cache_entries WHERE scope = 'maintenance-proof';`);
const freePages = () =>
Number(db.prepare("PRAGMA freelist_count").get()?.freelist_count ?? 0);
const before = freePages();
expect(before).toBeGreaterThan(512);
const exec = vi.spyOn(db, "exec");
const transactionMarker = path.join(root, "maintenance-transaction");
const work = worker.run(
(scope) => scope.execute(append("published", { transactionMarker, delayMs: 250 })),
allow,
);
await waitForFixtureEntry(receipts, transactionMarker, work, signal);
const started = performance.now();
const maintenance = Promise.resolve(tick());
expect(performance.now() - started).toBeLessThan(100);
expect(exec.mock.calls.some(([sql]) => sql.includes("incremental_vacuum"))).toBe(false);
expect(freePages()).toBe(before);
await work;
await maintenance;
await withOpenClawAgentDatabaseWrite(options, allow, db);
expect(exec.mock.calls.some(([sql]) => sql.includes("incremental_vacuum"))).toBe(false);
expect(before - freePages()).toBeGreaterThan(0);
expect(before - freePages()).toBeLessThanOrEqual(512);
expectValues(db, ["published"]);
}));
});
type V2Worker = OpenClawAgentSqliteWorkerStoreV2<AgentOperations>;
@@ -855,17 +530,9 @@ async function expectReadmission(
expectValues(db, [...prior, "readmitted"]);
}
function observeTerminal(worker: V2Worker) {
const notify = vi.fn();
const terminal = createDeferredCore();
worker.terminalSignal.addEventListener(
"abort",
() => {
notify(worker.terminalSignal.reason);
terminal.resolve();
},
{ once: true },
);
return { notify, terminal: terminal.promise };
worker.terminalSignal.addEventListener("abort", () => terminal.resolve(), { once: true });
return terminal.promise;
}
it.each([false, true])(
@@ -925,7 +592,7 @@ it.for(["owner revocation", "crash", "exit", "resource revocation"] as const)(
},
)
: await openV2Worker();
const observed = observeTerminal(worker);
const terminal = observeTerminal(worker);
if (failure === "owner revocation") {
await expect(worker.execute(append("refused"), allow)).rejects.toThrow("V2 owner revoked");
} else if (failure === "resource revocation") {
@@ -944,7 +611,7 @@ it.for(["owner revocation", "crash", "exit", "resource revocation"] as const)(
return result;
}, allow);
expect(await withinTest(accepted, signal)).toBeGreaterThan(0);
await withinTest(observed.terminal, signal);
await withinTest(terminal, signal);
await revoked;
expect(terminalBeforePublication).toBe(false);
expect(committedAtNotification).toEqual([[{ value: "accepted" }]]);
@@ -955,14 +622,14 @@ it.for(["owner revocation", "crash", "exit", "resource revocation"] as const)(
{ status: "rejected", reason: expect.any(Error) },
{ status: "rejected", reason: expect.any(Error) },
]);
await withinTest(observed.terminal, signal);
await withinTest(terminal, signal);
expect(worker.terminalSignal.aborted).toBe(true);
expect(worker.terminalSignal.reason).toMatchObject({
message: expect.stringMatching(/worker|exit|crash/i),
});
}
await withinTest(observed.terminal, signal);
expect(observed.notify).toHaveBeenCalledExactlyOnceWith(expect.any(Error));
await withinTest(terminal, signal);
expect(worker.terminalSignal.reason).toBeInstanceOf(Error);
const reason: unknown = worker.terminalSignal.reason;
if (failure === "owner revocation") {
expect(reason).toMatchObject({ message: "V2 owner revoked" });
@@ -974,7 +641,6 @@ it.for(["owner revocation", "crash", "exit", "resource revocation"] as const)(
if (failure === "crash" || failure === "exit") {
await worker.close();
expect(worker.terminalSignal.reason).toBe(reason);
expect(observed.notify).toHaveBeenCalledTimes(1);
}
},
);
ui/src/pages/chat/chat-gateway.test.ts
+268 -389
View File
File diff suppressed because it is too large. Load diff
ui/src/pages/chat/components/chat-audio-player.test.ts
+122 -257
View File
@@ -9,6 +9,49 @@ import { CHAT_AUDIO_WAVEFORM_MAX_BYTES } from "./chat-audio-waveform.ts";
type ChatAudioPlayer = HTMLElementTagNameMap["openclaw-chat-audio-player"];
const audio = (player: ChatAudioPlayer) => player.querySelector("audio")!;
const downloadLink = (player: ChatAudioPlayer) =>
player.querySelector<HTMLAnchorElement>("a[download]")!;
const headerDownload = (player: ChatAudioPlayer) =>
player.querySelector<HTMLAnchorElement>(".chat-assistant-attachment-card__download");
const waveformElement = (player: ChatAudioPlayer) =>
player.querySelector(".chat-audio-player__waveform");
const waveformRects = (player: ChatAudioPlayer) =>
player.querySelectorAll(".chat-audio-player__waveform rect");
const seekInput = (player: ChatAudioPlayer) =>
player.querySelector<HTMLInputElement>(".chat-audio-player__seek")!;
const source = (player: ChatAudioPlayer) => player.querySelector("audio")?.getAttribute("src");
function pendingFetch() {
let resolve!: (response: Response) => void;
const fetchMock = vi.fn<typeof fetch>(
async () =>
await new Promise<Response>((respond) => {
resolve = respond;
}),
);
vi.stubGlobal("fetch", fetchMock);
return { fetchMock, resolve: (response: Response) => resolve(response) };
}
async function update(player: ChatAudioPlayer, properties: Partial<ChatAudioPlayer>) {
Object.assign(player, properties);
await player.updateComplete;
}
async function fail(player: ChatAudioPlayer) {
audio(player).dispatchEvent(new Event("error"));
await player.updateComplete;
}
function transcodePlayer() {
return createPlayer("voice", {
src: "/__openclaw__/assistant-media?source=voice.caf&mediaTicket=ticket",
sourceIdentity: "/tmp/voice.caf",
label: "voice.caf",
playback: "transcode",
});
}
function expectUnavailable(player: ChatAudioPlayer) {
expect(card(player, "--compact")).not.toBeNull();
expect(card(player, "__reason")).toBeNull();
expect(player.querySelector(".chat-audio-player")).toBeNull();
}
const control = (player: ChatAudioPlayer, suffix: string) =>
player.querySelector<HTMLButtonElement>(`.chat-audio-player__${suffix}`)!;
const card = (player: ChatAudioPlayer, suffix = "") =>
@@ -22,21 +65,26 @@ const audioResponse = () =>
status: 200,
headers: { "Content-Type": "audio/mpeg", "Content-Length": "4" },
});
function mockPlayback(media: HTMLMediaElement) {
let paused = true;
function mockPlayback(media: HTMLMediaElement, active = false, events = true, pauseEvent = events) {
let paused = !active;
Object.defineProperty(media, "paused", { configurable: true, get: () => paused });
return {
play: vi.spyOn(media, "play").mockImplementation(async () => {
paused = false;
media.dispatchEvent(new Event("play"));
if (!active) {
paused = false;
}
if (events) {
media.dispatchEvent(new Event("play"));
}
}),
pause: vi.spyOn(media, "pause").mockImplementation(() => {
paused = true;
media.dispatchEvent(new Event("pause"));
if (pauseEvent) {
media.dispatchEvent(new Event("pause"));
}
}),
};
}
function setMediaNumber(
media: HTMLMediaElement,
property: "currentTime" | "duration",
@@ -44,20 +92,15 @@ function setMediaNumber(
) {
Object.defineProperty(media, property, { configurable: true, writable: true, value });
}
function mockActivePlayback(media: HTMLMediaElement) {
let paused = false;
Object.defineProperty(media, "paused", { configurable: true, get: () => paused });
setMediaNumber(media, "currentTime", 20);
setMediaNumber(media, "duration", 80);
return {
play: vi.spyOn(media, "play").mockResolvedValue(undefined),
pause: vi.spyOn(media, "pause").mockImplementation(() => {
paused = true;
}),
};
return mockPlayback(media, true, false);
}
function mockPlay(media: HTMLMediaElement) {
Object.defineProperty(media, "paused", { configurable: true, value: true });
return vi.spyOn(media, "play").mockResolvedValue(undefined);
}
async function createPlayer(
label: string,
overrides: Partial<ChatAudioPlayer> = {},
@@ -124,39 +167,20 @@ afterEach(() => {
});
describe("ChatAudioPlayer", () => {
it("keeps the download action for normalized base64 audio", async () => {
const player = await createPlayer("inline", {
src: "data:audio/wav;base64,UklGRg==",
sourceIdentity: "inline-audio",
label: "inline.wav",
});
expect(
player
.querySelector<HTMLAnchorElement>(".chat-assistant-attachment-card__download")
?.getAttribute("href"),
).toBe("data:audio/wav;base64,UklGRg==");
});
it("keeps voice notes player-only and never opens the attachment sidebar", async () => {
const player = await createPlayer("voice-note");
player.voiceNote = true;
player.onExpand = vi.fn();
player.sizeBytes = 2048;
await player.updateComplete;
await update(player, { voiceNote: true, onExpand: vi.fn(), sizeBytes: 2048 });
expect(player.querySelector(".chat-assistant-attachment-card__header")).toBeNull();
expect(player.querySelector(".chat-attachment-file-icon, [data-openable]")).toBeNull();
expect(player.textContent).not.toContain("voice-note.mp3");
expect(player.querySelector('[role="group"]')?.getAttribute("aria-label")).toBe("Voice note");
player.querySelector<HTMLElement>(".chat-assistant-attachment-card")!.click();
expect(player.onExpand).not.toHaveBeenCalled();
audio(player).dispatchEvent(new Event("error"));
await player.updateComplete;
await fail(player);
expect(player.querySelector("audio, .chat-audio-player__toggle")).toBeNull();
expect(player.querySelectorAll("[role=status]")).toHaveLength(1);
expect(player.querySelector("[role=status]")?.textContent).toContain("Preview unavailable");
expect(player.querySelector("a[download]")).toMatchObject({
expect(downloadLink(player)).toMatchObject({
href: "https://example.com/voice-note.mp3",
target: "_blank",
rel: "noreferrer",
@@ -164,55 +188,38 @@ describe("ChatAudioPlayer", () => {
expect(card(player, "__expand")).toBeNull();
});
it("formats elapsed, total, and buffered media time from metadata", async () => {
const player = await createPlayer("timing");
expect(times(player)).toEqual(["0:00 / 0:00"]);
it("drives play, pause, seek, and keyboard state through the hidden audio element", async () => {
const inline = "data:audio/wav;base64,UklGRg==";
const player = await createPlayer("briefing", { src: inline });
expect(headerDownload(player)?.getAttribute("href")).toBe(inline);
const media = audio(player);
setMediaNumber(media, "currentTime", 65.9);
setMediaNumber(media, "duration", 3_665);
expect(times(player)).toEqual(["0:00 / 0:00"]);
Object.defineProperty(media, "buffered", {
configurable: true,
value: { length: 1, end: () => 1_832.5 },
});
setMediaNumber(media, "duration", 3_665);
setMediaNumber(media, "currentTime", 65.9);
media.dispatchEvent(new Event("loadedmetadata"));
await player.updateComplete;
expect(times(player)).toEqual(["1:05 / 61:05"]);
expect(
player
.querySelector<HTMLInputElement>(".chat-audio-player__seek")!
.style.getPropertyValue("--chat-audio-buffered"),
).toBe("50%");
});
it("drives play, pause, seek, and keyboard state through the hidden audio element", async () => {
const player = await createPlayer("briefing");
const media = audio(player);
expect(seekInput(player).style.getPropertyValue("--chat-audio-buffered")).toBe("50%");
setMediaNumber(media, "duration", 125);
setMediaNumber(media, "currentTime", 0);
const { play, pause } = mockPlayback(media);
media.dispatchEvent(new Event("loadedmetadata"));
await player.updateComplete;
expect(player.querySelector(".chat-audio-player__time")?.textContent).toContain("2:05");
expect(times(player)).toEqual(["0:00 / 2:05"]);
control(player, "toggle").click();
await waitForSolid(() => expect(play).toHaveBeenCalledOnce());
await waitForSolid(() =>
expect(control(player, "toggle")?.getAttribute("aria-label")).toBe("Pause"),
);
expect(
player
.querySelector<HTMLAnchorElement>(".chat-assistant-attachment-card__download")
?.getAttribute("download"),
).toBe("briefing.mp3");
const seek = player.querySelector<HTMLInputElement>(".chat-audio-player__seek")!;
expect(headerDownload(player)?.getAttribute("download")).toBe("briefing.mp3");
const seek = seekInput(player);
seek.value = "35";
seek.dispatchEvent(new Event("input", { bubbles: true }));
expect(media.currentTime).toBe(35);
for (const [key, time] of [
["ArrowRight", 40],
["ArrowUp", 45],
@@ -224,12 +231,13 @@ describe("ChatAudioPlayer", () => {
expect(event.defaultPrevented).toBe(true);
expect(media.currentTime).toBe(time);
}
const controls = player.querySelector<HTMLElement>(".chat-audio-player")!;
controls.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowRight", bubbles: true }));
expect(media.currentTime).toBe(40);
controls.dispatchEvent(new KeyboardEvent("keydown", { key: " ", bubbles: true }));
expect(pause).toHaveBeenCalledOnce();
await player.updateComplete;
expect(times(player)).toEqual(["0:40 / 2:05"]);
});
it("does not expand the attachment when muting through the changing SVG icon", async () => {
@@ -256,58 +264,37 @@ describe("ChatAudioPlayer", () => {
const status = first.querySelector('[role="status"]');
expect(status?.textContent).toBe("");
const firstMedia = audio(first);
let paused = true;
Object.defineProperty(firstMedia, "paused", { configurable: true, get: () => paused });
const play = vi.spyOn(firstMedia, "play").mockImplementation(async () => {
paused = false;
firstMedia.dispatchEvent(new Event("play"));
});
const pauseFirst = vi.spyOn(firstMedia, "pause").mockImplementation(() => {
paused = true;
});
const { play, pause: pauseFirst } = mockPlayback(firstMedia, false, true, false);
control(first, "toggle").click();
await first.updateComplete;
expect(play).toHaveBeenCalledOnce();
expect(control(first, "toggle")?.getAttribute("aria-label")).toBe("Pause");
firstMedia.dispatchEvent(new Event("error"));
await first.updateComplete;
expect(first.querySelector(".chat-audio-player__toggle")).toBeNull();
expect(card(first, "--compact")).not.toBeNull();
expect(card(first, "__reason")).toBeNull();
expect(first.querySelector(".chat-audio-player")).toBeNull();
expectUnavailable(first);
expect(first.querySelector('[role="status"]')).toBe(status);
expect(status?.textContent).toBe("broken.mp3: Preview unavailable");
expect(status?.getAttribute("aria-live")).toBe("polite");
expect(status?.getAttribute("aria-atomic")).toBe("true");
expect(
first
.querySelector<HTMLAnchorElement>(".chat-assistant-attachment-card__download")
?.getAttribute("download"),
).toBe("broken.mp3");
expect(headerDownload(first)?.getAttribute("download")).toBe("broken.mp3");
expect(pauseFirst).toHaveBeenCalledOnce();
pauseFirst.mockClear();
const second = await createPlayer("working");
audio(second).dispatchEvent(new Event("play"));
expect(pauseFirst).not.toHaveBeenCalled();
first.src = "https://example.com/broken.mp3?renewed";
await waitForSolid(() => expect(control(first, "toggle")).not.toBeNull());
expect(first.querySelector('[role="status"]')).toBe(status);
expect(status?.textContent).toBe("");
audio(first).dispatchEvent(new Event("error"));
await first.updateComplete;
await fail(first);
expect(status?.textContent).toBe("broken.mp3: Preview unavailable");
first.src = "https://example.com/replacement.mp3";
first.sourceIdentity = "media://replacement";
first.label = "replacement.mp3";
await waitForSolid(() => expect(control(first, "toggle")).not.toBeNull());
expect(status?.textContent).toBe("");
audio(first).dispatchEvent(new Event("error"));
await first.updateComplete;
await fail(first);
expect(status?.textContent).toBe("replacement.mp3: Preview unavailable");
});
@@ -332,7 +319,7 @@ describe("ChatAudioPlayer", () => {
outside.focus();
}
await player.updateComplete;
const download = player.querySelector<HTMLAnchorElement>("a[download]")!;
const download = downloadLink(player);
expect(download.getAttribute("download")).toBe("focus-" + target + ".mp3");
expect(document.activeElement).toBe(
target === "outside" || target === "moved" ? outside : download,
@@ -341,54 +328,13 @@ describe("ChatAudioPlayer", () => {
},
);
it("shows preparing, retries a 202 rendition, and then plays", async () => {
vi.useFakeTimers();
const fetchMock = vi
.fn<typeof fetch>()
.mockResolvedValueOnce(new Response(null, { status: 202 }))
.mockResolvedValueOnce(new Response(null, { status: 200 }));
vi.stubGlobal("fetch", fetchMock);
const player = await createPlayer("voice", {
src: "/__openclaw__/assistant-media?source=voice.caf&mediaTicket=ticket",
sourceIdentity: "/tmp/voice.caf",
label: "voice.caf",
playback: "transcode",
});
await Promise.resolve();
await vi.advanceTimersByTimeAsync(0);
await player.updateComplete;
expect(player.textContent).toContain("Preparing playback…");
expect(player.querySelector("[role=status]")?.textContent).toBe("");
await vi.advanceTimersByTimeAsync(2_000);
await player.updateComplete;
const media = audio(player);
let paused = true;
Object.defineProperty(media, "paused", { configurable: true, get: () => paused });
const play = vi.spyOn(media, "play").mockImplementation(async () => {
paused = false;
media.dispatchEvent(new Event("play"));
});
expect(media.getAttribute("src")).toContain("mediaTicket=ticket&playback=1");
control(player, "toggle").click();
await vi.advanceTimersByTimeAsync(0);
await player.updateComplete;
expect(play).toHaveBeenCalledOnce();
expect(fetchMock).toHaveBeenCalledTimes(2);
});
it("keeps header Download focused when the parent Expand callback changes", async () => {
const player = await createPlayer("callback-focus", { onExpand: vi.fn() });
const download = player.querySelector<HTMLAnchorElement>(
".chat-assistant-attachment-card__download",
)!;
const download = headerDownload(player)!;
download.focus();
expect(document.activeElement).toBe(download);
player.onExpand = vi.fn();
await player.updateComplete;
expect(document.activeElement).toBe(download);
});
@@ -419,12 +365,9 @@ describe("ChatAudioPlayer", () => {
: player.querySelector<HTMLElement>(`.chat-assistant-attachment-card__${target}`)!;
focused.focus();
expect(document.activeElement).toBe(focused);
respond(new Response(null, { status: 500 }));
await waitForSolid(() => expect(card(player, "--compact")).not.toBeNull());
expect(document.activeElement).toBe(
target === "outside" ? outside : player.querySelector("a[download]"),
);
expect(document.activeElement).toBe(target === "outside" ? outside : downloadLink(player));
},
);
@@ -432,62 +375,47 @@ describe("ChatAudioPlayer", () => {
vi.useFakeTimers();
const fetchMock = vi.fn<typeof fetch>(async () => new Response(null, { status: 202 }));
vi.stubGlobal("fetch", fetchMock);
const player = await createPlayer("voice", {
src: "/__openclaw__/assistant-media?source=voice.caf&mediaTicket=ticket",
sourceIdentity: "/tmp/voice.caf",
label: "voice.caf",
playback: "transcode",
});
const player = await transcodePlayer();
await vi.runAllTimersAsync();
await player.updateComplete;
expect(card(player, "--compact")).not.toBeNull();
expect(card(player, "__reason")).toBeNull();
expect(player.querySelector(".chat-audio-player")).toBeNull();
expect(
player
.querySelector<HTMLAnchorElement>(".chat-assistant-attachment-card__download")
?.getAttribute("href"),
).not.toContain("playback=1");
expectUnavailable(player);
expect(headerDownload(player)?.getAttribute("href")).not.toContain("playback=1");
const failedAttempts = fetchMock.mock.calls.length;
player.label = "renamed-voice.caf";
player.onMediaLoaded = vi.fn();
player.sizeBytes = 2048;
player.serverDurationMs = 4000;
await player.updateComplete;
await update(player, {
label: "renamed-voice.caf",
onMediaLoaded: vi.fn(),
sizeBytes: 2048,
serverDurationMs: 4000,
});
await vi.runAllTimersAsync();
expect(fetchMock).toHaveBeenCalledTimes(failedAttempts);
expect(player.querySelector("audio")).toBeNull();
expect(card(player, "--compact")).not.toBeNull();
fetchMock.mockResolvedValue(new Response(null, { status: 200 }));
player.src = "/__openclaw__/assistant-media?source=voice.caf&mediaTicket=recovered";
await player.updateComplete;
await vi.runAllTimersAsync();
await player.updateComplete;
expect(card(player, "--compact")).toBeNull();
expect(player.querySelector("audio")?.getAttribute("src")).toContain(
"mediaTicket=recovered&playback=1",
);
expect(source(player)).toContain("mediaTicket=recovered&playback=1");
});
it("renders decoded waveform peaks and scopes waveform cache reuse to auth context", async () => {
const intersect = observeIntersections();
const decodeAudioData = mockAudioContext();
let resolveFetch: ((response: Response) => void) | undefined;
const fetchMock = vi.fn<typeof fetch>(
async () =>
await new Promise<Response>((resolve) => {
resolveFetch = resolve;
}),
);
vi.stubGlobal("fetch", fetchMock);
const { fetchMock, resolve: resolveFetch } = pendingFetch();
vi.spyOn(URL, "createObjectURL").mockReturnValue("blob:waveform-audio");
vi.spyOn(URL, "revokeObjectURL").mockImplementation(() => undefined);
const player = await createPlayer("waveform-reuse");
expect(player.querySelector(".chat-audio-player__waveform")).toBeNull();
const initialPlay = mockPlay(audio(player));
control(player, "toggle").click();
await initialPlay.mock.results[0]!.value;
await Promise.resolve();
flush();
expect(waveformElement(player)).toBeNull();
expect(fetchMock).not.toHaveBeenCalled();
expect(decodeAudioData).not.toHaveBeenCalled();
initialPlay.mockRestore();
player.serverDurationMs = 4_000;
await player.updateComplete;
expect(fetchMock).not.toHaveBeenCalled();
@@ -495,24 +423,16 @@ describe("ChatAudioPlayer", () => {
await waitForSolid(() => expect(fetchMock).toHaveBeenCalledOnce());
const media = audio(player);
const { play } = mockPlayback(media);
resolveFetch?.(audioResponse());
await waitForSolid(() =>
expect(player.querySelectorAll(".chat-audio-player__waveform rect")).toHaveLength(96),
);
await waitForSolid(() => expect(waveformRects(player)).toHaveLength(96));
await player.updateComplete;
expect(
new Set(
Array.from(player.querySelectorAll(".chat-audio-player__waveform rect"), (rect) =>
rect.getAttribute("height"),
),
).size,
new Set(Array.from(waveformRects(player), (rect) => rect.getAttribute("height"))).size,
).toBeGreaterThan(1);
player.voiceNote = true;
await player.updateComplete;
const voiceHeights = Array.from(
player.querySelectorAll(".chat-audio-player__waveform rect"),
(rect) => Number(rect.getAttribute("height")),
const voiceHeights = Array.from(waveformRects(player), (rect) =>
Number(rect.getAttribute("height")),
);
expect(Math.min(...voiceHeights)).toBeGreaterThanOrEqual(6);
expect(Math.max(...voiceHeights)).toBeLessThanOrEqual(14);
@@ -520,29 +440,22 @@ describe("ChatAudioPlayer", () => {
expect(fetchMock).toHaveBeenCalledOnce();
expect(decodeAudioData).toHaveBeenCalledOnce();
expect(media.getAttribute("src")).toBe("blob:waveform-audio");
control(player, "toggle").click();
await waitForSolid(() => expect(play).toHaveBeenCalledOnce());
expect(media.getAttribute("src")).toBe("blob:waveform-audio");
expect(fetchMock).toHaveBeenCalledOnce();
player.authToken = "new-principal";
await player.updateComplete;
expect(media.getAttribute("src")).toBe("https://example.com/waveform-reuse.mp3");
player.remove();
const refreshed = document.createElement("openclaw-chat-audio-player");
refreshed.src = "https://example.com/waveform-reuse.mp3?mediaTicket=fresh";
refreshed.sourceIdentity = "media://waveform-reuse";
refreshed.authToken = "different-principal";
refreshed.label = "waveform-reuse.mp3";
refreshed.serverDurationMs = 4_000;
mountSolid(() => refreshed);
await refreshed.updateComplete;
const refreshed = await createPlayer("waveform-reuse", {
src: "https://example.com/waveform-reuse.mp3?mediaTicket=fresh",
authToken: "different-principal",
serverDurationMs: 4_000,
});
intersect();
const refreshedMedia = refreshed.querySelector("audio")!;
Object.defineProperty(refreshedMedia, "paused", { configurable: true, value: true });
vi.spyOn(refreshedMedia, "play").mockResolvedValue(undefined);
mockPlay(refreshedMedia);
await waitForSolid(() => expect(fetchMock).toHaveBeenCalledTimes(3));
resolveFetch?.(new Response(null, { status: 500 }));
});
@@ -576,16 +489,13 @@ describe("ChatAudioPlayer", () => {
const label = change === "unmount" ? "waveform-unmount" : "waveform-replacement";
const player = await createPlayer(label, { serverDurationMs: 4_000 });
intersect();
await waitForSolid(() =>
expect(player.querySelector(".chat-audio-player__waveform")).not.toBeNull(),
);
const waveform = player.querySelector(".chat-audio-player__waveform")!;
await waitForSolid(() => expect(waveformElement(player)).not.toBeNull());
const waveform = waveformElement(player)!;
const media = player.querySelector("audio")!;
expect(observe).toHaveBeenCalledExactlyOnceWith(waveform);
expect(disconnect).not.toHaveBeenCalled();
expect(media.getAttribute("src")).toBe("blob:waveform-disposal");
const { pause } = mockActivePlayback(media);
if (change === "unmount") {
player.remove();
await Promise.resolve();
@@ -627,21 +537,17 @@ describe("ChatAudioPlayer", () => {
intersect();
await waitForSolid(() => expect(fetchMock).toHaveBeenCalledOnce());
resolveFirstFetch?.(audioResponse());
await waitForSolid(() =>
expect(player.querySelector("audio")?.getAttribute("src")).toBe("blob:waveform-audio"),
);
await waitForSolid(() => expect(source(player)).toBe("blob:waveform-audio"));
const media = audio(player);
Object.defineProperty(media, "error", { configurable: true, value: {} as MediaError });
media.dispatchEvent(new Event("error"));
await player.updateComplete;
expect(card(player, "--compact")).not.toBeNull();
expect(media.isConnected).toBe(false);
player.src = "https://example.com/waveform-recovery.mp3?mediaTicket=recovered";
await player.updateComplete;
await waitForSolid(() =>
expect(player.querySelector("audio")?.getAttribute("src")).toBe(
expect(source(player)).toBe(
"https://example.com/waveform-recovery.mp3?mediaTicket=recovered",
),
);
@@ -650,25 +556,6 @@ describe("ChatAudioPlayer", () => {
expect(card(player, "--compact")).toBeNull();
});
it("skips waveform work without a server-probed duration", async () => {
const decodeAudioData = stubAudioContext();
const fetchMock = vi.fn<typeof fetch>();
vi.stubGlobal("fetch", fetchMock);
const player = await createPlayer("unknown-duration");
const media = audio(player);
Object.defineProperty(media, "paused", { configurable: true, value: true });
const play = vi.spyOn(media, "play").mockResolvedValue(undefined);
control(player, "toggle").click();
await play.mock.results[0]!.value;
await Promise.resolve();
flush();
expect(fetchMock).not.toHaveBeenCalled();
expect(decodeAudioData).not.toHaveBeenCalled();
expect(player.querySelector(".chat-audio-player__waveform")).toBeNull();
});
it("discards waveform peaks when decoded duration exceeds the server gate by 20 percent", async () => {
const decodeAudioData = vi.fn(async () => decodedAudio(121));
const AudioContextMock = vi.fn(function (options?: AudioContextOptions) {
@@ -684,17 +571,14 @@ describe("ChatAudioPlayer", () => {
const player = await createPlayer("duration-mismatch");
player.serverDurationMs = 100_000;
const media = audio(player);
Object.defineProperty(media, "paused", { configurable: true, value: true });
setMediaNumber(media, "duration", 100);
media.dispatchEvent(new Event("loadedmetadata"));
await player.updateComplete;
vi.spyOn(media, "play").mockResolvedValue(undefined);
mockPlay(media);
control(player, "toggle").click();
await waitForSolid(() => expect(media.getAttribute("src")).toBe("blob:duration-mismatch"));
expect(decodeAudioData).toHaveBeenCalledOnce();
expect(player.querySelector(".chat-audio-player__waveform")).toBeNull();
expect(waveformElement(player)).toBeNull();
expect(times(player)).toEqual(["0:00 / 1:40"]);
});
@@ -715,17 +599,14 @@ describe("ChatAudioPlayer", () => {
const player = await createPlayer("oversized-waveform");
player.serverDurationMs = 4_000;
const media = audio(player);
Object.defineProperty(media, "paused", { configurable: true, value: true });
const play = vi.spyOn(media, "play").mockResolvedValue(undefined);
const play = mockPlay(media);
control(player, "toggle").click();
expect(play).toHaveBeenCalledOnce();
await waitForSolid(() => expect(fetchMock).toHaveBeenCalledOnce());
await waitForSolid(() => expect(response.body?.locked).toBe(false));
expect(createObjectURL).not.toHaveBeenCalled();
expect(decodeAudioData).not.toHaveBeenCalled();
expect(player.querySelector(".chat-audio-player__waveform")).toBeNull();
expect(waveformElement(player)).toBeNull();
expect(media.getAttribute("src")).toBe("https://example.com/oversized-waveform.mp3");
});
@@ -743,31 +624,21 @@ describe("ChatAudioPlayer", () => {
player.serverDurationMs = 4_000;
const media = audio(player);
const { play } = mockPlayback(media);
control(player, "toggle").click();
await waitForSolid(() => expect(FailingAudioContext).toHaveBeenCalledOnce());
await Promise.resolve();
flush();
media.pause();
control(player, "toggle").click();
expect(play).toHaveBeenCalledTimes(2);
expect(media.getAttribute("src")).toBe("blob:context-fallback");
});
it("cancels the old source when identity changes during rendition preparation", async () => {
let resolveFetch: ((response: Response) => void) | undefined;
const fetchMock = vi.fn<typeof fetch>(
async () =>
await new Promise<Response>((resolve) => {
resolveFetch = resolve;
}),
);
vi.stubGlobal("fetch", fetchMock);
const { fetchMock, resolve: resolveFetch } = pendingFetch();
const player = await createPlayer("identity-before");
const media = audio(player);
const { play, pause } = mockActivePlayback(media);
player.src = "/__openclaw__/assistant-media?source=after.caf&mediaTicket=ticket";
player.sourceIdentity = "media://identity-after";
player.label = "after.caf";
@@ -776,7 +647,6 @@ describe("ChatAudioPlayer", () => {
expect(pause).toHaveBeenCalledOnce();
expect(fetchMock).not.toHaveBeenCalled();
await waitForSolid(() => expect(player.textContent).toContain("Preparing playback…"));
expect(pause).toHaveBeenCalledOnce();
expect(fetchMock).toHaveBeenCalledOnce();
expect(fetchMock.mock.calls[0]?.[0]).toBe(
@@ -796,7 +666,6 @@ describe("ChatAudioPlayer", () => {
const player = await createPlayer("disconnect-resume");
const media = audio(player);
const { play, pause } = mockActivePlayback(media);
player.src = "https://example.com/disconnect-resume-fresh.mp3";
await player.updateComplete;
media.dispatchEvent(new Event("error"));
@@ -807,7 +676,6 @@ describe("ChatAudioPlayer", () => {
mountSolid(() => player);
media.currentTime = 0;
media.dispatchEvent(new Event("loadedmetadata"));
expect(play).not.toHaveBeenCalled();
});
@@ -816,18 +684,15 @@ describe("ChatAudioPlayer", () => {
const firstMedia = audio(first);
const { play: playFirst, pause: pauseFirst } = mockActivePlayback(firstMedia);
firstMedia.dispatchEvent(new Event("play"));
first.src = "https://example.com/superseded-resume-fresh.mp3";
await first.updateComplete;
firstMedia.dispatchEvent(new Event("error"));
const second = await createPlayer("superseding-player");
audio(second).dispatchEvent(new Event("play"));
expect(pauseFirst).toHaveBeenCalledOnce();
second.remove();
firstMedia.currentTime = 0;
firstMedia.dispatchEvent(new Event("loadedmetadata"));
expect(playFirst).not.toHaveBeenCalled();
});
});
ui/src/pages/chat/components/chat-transcript-controller.test.tsx
+1 -147
View File
@@ -8,11 +8,7 @@ import { flush } from "../../../test-helpers/solid-settle.ts";
import { projectChatFields } from "../chat-field-projection.ts";
import { makeChatHost } from "../chat-host.test-support.ts";
import { createTestTranscript, stubAnimationFrames } from "../chat-view.test-helpers.ts";
import {
handleChatScrollTakeover,
saveChatSessionScrollPosition,
scheduleCommittedChatScroll,
} from "../scroll.ts";
import { handleChatScrollTakeover, saveChatSessionScrollPosition } from "../scroll.ts";
import { SIDEBAR_GEOMETRY_COMMIT_EVENT } from "../sidebar-layout.ts";
import {
render,
@@ -178,23 +174,6 @@ it.each([false, true])("re-observes re-stamped rows (foreign host=%s)", async (f
transcript.dispose();
});
it("measures newly inserted rows after Solid connects them", async () => {
transcriptDomState.detachedRowHeight = 0;
const props = threadProps("pane-commentary-insert", "agent:main:session-a", [
{ role: "assistant", content: "commentary", timestamp: 1_000 },
{ role: "user", content: "next turn", timestamp: 2_000 },
]);
const { container, transcript, draw } = threadFixture(props);
draw();
transcript.start();
transcript.update();
await flushDeferredRowPrune();
emitRowSizes(transcriptRows(container), 100);
draw();
expect(transcriptSize(container)).toBe(200);
});
it("updates the visible range on native scroll without a parent render", async () => {
const flushFrames = stubAnimationFrames();
const props = threadProps("local-geometry", "agent:main:local-geometry", messages(40, "message"));
@@ -223,37 +202,6 @@ it("updates the visible range on native scroll without a parent render", async (
expect(Number(transcriptRows(container)[0]?.dataset.index)).toBeGreaterThan(0);
});
it("reveals committed initial row sizes without waiting for an animation frame", async () => {
stubAnimationFrames();
vi.spyOn(HTMLElement.prototype, "clientHeight", "get").mockReturnValue(600);
const host = new TranscriptTestHost();
cleanup.push(() => host.remove());
const transcript = (host.transcript = createChatTranscript(() => "native-measured-tail"));
const rows = numberedContentRows(6);
const scroller = document.createElement("div");
scroller.className = "chat-thread";
host.renderTranscript = () => {
const content = renderReadySession(transcript, "native-measured-tail", rows);
if (scroller.firstChild !== content) {
scroller.replaceChildren(content);
}
return scroller;
};
document.body.append(host);
await host.settleUpdates();
const extent = requireElement(host, ".chat-thread-inner");
expect(transcriptRows(host).every((row) => observedElements.has(row))).toBe(true);
await Promise.resolve();
flush();
await host.settleUpdates();
expect(extent.style.visibility).toBe("");
expect(transcriptRows(host).every((row) => row.style.containIntrinsicBlockSize === "100px")).toBe(
true,
);
expect(host.textContent).toContain("row 5");
host.remove();
});
it("keeps retained MCP rows and the virtual row model atomic through teardown", async () => {
const teardownPending = createDeferred();
transcriptDomState.measuredRowHeight = 180;
@@ -409,22 +357,6 @@ it("retains history controls across callback and loading replacements", async ()
expect(first).toHaveBeenCalledOnce();
});
it("renders the history tail immediately after an empty disconnected shell", () => {
const flushFrames = stubAnimationFrames();
const props = threadProps("cold-load-tail", "agent:main:cold-load-tail", []);
const { container, transcript, draw } = threadFixture(props);
draw();
geometry(container.querySelector(".chat-thread")!, { clientHeight: 600, scrollHeight: 600 });
transcript.start();
transcript.update();
flushFrames();
draw({ ...props, messages: messages(40, "Cached history") });
expect(container.textContent).toContain("Cached history 39");
expect(container.textContent).not.toContain("Cached history 0");
});
it.each([true, false])("settles a non-overflowing restore (initially loading=%s)", (loading) => {
const flushFrames = stubAnimationFrames();
if (!loading) {
@@ -498,13 +430,10 @@ it("retries a growing restore without pane commits and renders the settled reade
transcript.scrollToOffset(2420, onSettled);
await host.settleUpdates();
const initialCommits = host.committedRenders;
const retryCommits: number[] = [];
for (const height of [600, 600, 600, 720, 720, 720, 720, 900, 900, 900, 900]) {
scrollHeight = height;
const before = host.committedRenders;
flushFrame();
await host.settleUpdates();
retryCommits.push(host.committedRenders - before);
expect(onSettled).not.toHaveBeenCalled();
}
scrollHeight = 4800;
@@ -514,7 +443,6 @@ it("retries a growing restore without pane commits and renders the settled reade
expect(container.scrollTop).toBe(2420);
expect(container.querySelector("output")?.textContent).toBe("Reading history");
expect.soft(host.committedRenders - initialCommits).toBe(0);
expect.soft(retryCommits).toEqual(Array(11).fill(0));
// jsdom does not emit native scroll events for scrollTop writes. Deliver the browser read-back.
scroll(container);
@@ -747,80 +675,6 @@ it("keeps smooth latest through settlement and idle until the target", async ()
expect(transcript.isProgrammaticScroll).toBe(false);
});
it.each([true, false])(
"leaves height-resize follow to page policy with reader lock=%s",
async (locked) => {
const flushFrames = stubAnimationFrames();
const policy = makeChatHost({ chatHasAutoScrolled: true });
const onViewportResize = vi.fn(() =>
scheduleCommittedChatScroll(policy, false, false, { source: "resize" }),
);
const transcript = createChatTranscript(() => `height-resize-${locked}`, {
onViewportResize,
canFollowEnd: () => !policy.chatFollowLocked,
onReaderScroll: (towardEnd) => handleChatScrollTakeover(policy, towardEnd),
});
const rows = numberedContentRows(12);
const { container } = await mountTestTranscript(`height-resize-${locked}`, rows, transcript);
const total = transcriptSize(container);
geometry(container, { clientHeight: 600, scrollHeight: total + 88 });
const scrollTo = vi.fn();
container.scrollTo = scrollTo;
policy.chatScrollElement = () => container;
policy.chatScrollToEnd = (options) => transcript.scrollToEnd(options);
emitSize(container, 600);
scroll(container, total - 600);
if (locked) {
container.dispatchEvent(new WheelEvent("wheel", { deltaY: -88 }));
}
expect(policy.chatFollowLocked).toBe(locked);
scrollTo.mockClear();
for (const height of [560, 640]) {
geometry(container, { clientHeight: height });
emitSize(container, height);
// The observer only reports geometry; follow waits for the policy frame.
expect(scrollTo).not.toHaveBeenCalled();
flushFrames();
if (locked) {
expect(scrollTo).not.toHaveBeenCalled();
expect(container.scrollTop).toBe(total - 600);
} else {
expect(scrollTo).toHaveBeenLastCalledWith({
top: total + 88 - height,
behavior: "auto",
});
}
scrollTo.mockClear();
}
expect(onViewportResize).toHaveBeenCalledTimes(2);
expect(policy.chatFollowLocked).toBe(locked);
},
);
it("keeps a reader above the padded real end stationary when a rendered row grows", async () => {
transcriptDomState.measuredRowHeight = 120;
const rows = numberedContentRows(12);
const { container, renderRows } = await mountTestTranscript("padded-row-resize", rows);
const total = transcriptSize(container);
container.style.padding = "28px 0 56px";
container.scrollTo = (options?: ScrollToOptions | number) => applyScroll(container, options);
geometry(container, { clientHeight: 600, scrollHeight: total + 84 });
emitSize(container, 600);
scroll(container, container.scrollHeight - container.clientHeight - 44);
const readerOffset = container.scrollTop;
expect(Math.max(total - container.clientHeight - readerOffset, 0)).toBe(0);
const row = requireElement(container, '[data-index="11"]');
expect(observedElements.has(row)).toBe(true);
geometry(row, { offsetHeight: 192 });
geometry(container, { scrollHeight: total + 84 + 72 });
emitSize(row, 192);
renderRows(rows);
expect(transcriptSize(container)).toBe(total + 72);
expect(container.scrollTop).toBe(readerOffset);
});
it.each([false, true])(
"keeps disclosure anchoring through geometry commit unless interrupted=%s",
async (interrupt) => {
ui/src/pages/chat/components/chat-transcript-message-index.test.ts
+53 -197
View File
@@ -1,4 +1,4 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { afterEach, describe, expect, it } from "vitest";
import { chatItemGroups } from "../chat-agent-run-grouping.ts";
import { resetWorkingProgress } from "../chat-progress.ts";
import { buildChatItems, type BuildChatItemsProps } from "../chat-thread-build.ts";
@@ -29,11 +29,9 @@ function makeTool(
turnBoundary?: boolean,
) {
return {
role: "toolResult",
...makeMessage(id, seq, { role: "toolResult", content: "Evidence", timestamp, runId }),
toolName: "read",
toolCallId: id,
content: "Evidence",
timestamp,
__openclaw: { id, seq, runId, turnBoundary },
};
}
@@ -56,11 +54,7 @@ function chatProps(overrides: Partial<BuildChatItemsProps> = {}): BuildChatItems
const chatItems = (overrides: Partial<BuildChatItemsProps> = {}) =>
buildChatItems(chatProps(overrides));
const chainOptions = {
sessionKey,
runWorking: false,
searchActive: false,
};
const chainOptions = { sessionKey, runWorking: false, searchActive: false };
const labels = { assistantName: "Assistant", userId: "reader", userName: "Reader" };
type Chain = ReturnType<typeof projectTranscriptChain>;
type Index = ReturnType<typeof projectTranscriptIndex>;
@@ -101,41 +95,6 @@ const replyFrameShape = (outcome: unknown, ...parts: unknown[]) => ({
parts,
});
it.each([false, true])("merges adjacent continuation work across runs while live=%s", (live) => {
const messages = [
history[0],
{ ...partial, content: "Checking the remaining work.", phase: "commentary" },
...["run-1", "announce:review", "announce:resume"].map((runId, index) =>
makeTool(`tool-${index}`, 3 + index, runId, 30 + index, true),
),
];
for (const persisted of [messages, structuredClone(messages)]) {
const items = chatItems({
messages: persisted,
runId: live ? "announce:resume" : null,
runWorking: live,
streamStartedAt: live ? 40 : null,
});
const options = { ...chainOptions, runWorking: live };
const chain = chainFor(items, options);
const parts = transcriptParts(chain);
const activities = kind(parts, "activity-run");
expect(activities).toHaveLength(1);
expect(activities[0]?.groups.map((group) => group.runId)).toEqual([
"run-1",
"announce:review",
"announce:resume",
]);
expect(messagesIn(kind(parts, "group"))).toContain(persisted[1]);
expect(parts.some((item) => item.kind === "stream-run")).toBe(live);
expect(chainFor(items, { ...options })).toBe(chain);
const index = indexFor(chain);
expect(new Set([0, 1, 2].map((tool) => index.messageRowKeysById.get(`tool-${tool}`)))).toEqual(
new Set([activities[0]?.key]),
);
}
});
it.each([
{ key: sessionKey, working: true, projected: false, untagged: false },
{ key: sessionKey, working: true, projected: true, untagged: false },
@@ -183,8 +142,6 @@ describe("completed reply frame ownership", () => {
];
}
const frames = framesOf;
it.each<[string, (string | undefined)[], boolean, string?, boolean?]>([
["missing prompt", ["run-1"], false],
["resumed run without prompt", ["run-u", "run-1"], false],
@@ -205,7 +162,7 @@ describe("completed reply frame ownership", () => {
const original = structuredClone(messages);
const options = { sessionKey: key };
const chain = projectHistory(messages, options);
const [frame] = frames(chain);
const [frame] = framesOf(chain);
expect(chain.transcriptItems).toEqual([
...(prompt ? [expect.objectContaining({ role: "user" })] : []),
frame,
@@ -239,7 +196,7 @@ describe("completed reply frame ownership", () => {
if (!prompt) {
for (const restored of [structuredClone(messages), [history[0], ...messages]]) {
const next = projectHistory(restored, options);
expect(frames(next)[0]?.key).toBe(frame?.key);
expect(framesOf(next)[0]?.key).toBe(frame?.key);
const nextIndex = indexFor(next);
for (const id of ["work-0", "final"]) {
expect(nextIndex.messageRowKeysById.get(id)).toBe(index.messageRowKeysById.get(id));
@@ -267,13 +224,13 @@ describe("completed reply frame ownership", () => {
});
messages.splice(prompt ? 2 : 1, 0, inserted);
const chain = projectHistory(messages);
expect(frames(chain).at(-1)?.outcome).toMatchObject(completedOutcome(messages.at(-1)));
expect(framesOf(chain).at(-1)?.outcome).toMatchObject(completedOutcome(messages.at(-1)));
const work = (forwarded ? chain.transcriptItems : transcriptParts(chain)).find(
(item) => item.kind === "work-group",
);
expect(work).toBeDefined();
if (forwarded) {
expect(messagesIn(frames(chain))).not.toContain(inserted);
expect(messagesIn(framesOf(chain))).not.toContain(inserted);
const standalone = phase ? kind(chain.transcriptItems, "group") : [work!];
expect(messagesIn(standalone)).toContain(inserted);
} else {
@@ -282,10 +239,12 @@ describe("completed reply frame ownership", () => {
kind(transcriptParts(chain), "group").filter((item) => item.role === "assistant"),
),
).toEqual([inserted, messages.at(-1)]);
expect(frames(chain).map((frame) => frame.runId)).toEqual(
expect(framesOf(chain).map((frame) => frame.runId)).toEqual(
runId === "run-1" ? ["run-1"] : ["run-other", "run-1"],
);
expect(runId === "run-1" ? frames(chain)[0]?.parts[0] : chain.transcriptItems[1]).toBe(work);
expect(runId === "run-1" ? framesOf(chain)[0]?.parts[0] : chain.transcriptItems[1]).toBe(
work,
);
}
});
@@ -298,7 +257,7 @@ describe("completed reply frame ownership", () => {
});
messages.splice(2, 0, steer);
const chain = projectHistory(messages);
const [before, frame] = frames(chain);
const [before, frame] = framesOf(chain);
expect(chain.transcriptItems).toEqual([
expect.objectContaining({ role: "user" }),
before,
@@ -368,119 +327,40 @@ it("reuses transcript structure for scroll renders and rebuilds for its inputs",
expect(indexFor(settled, expanded, { ...labels, userName: "Renamed" })).not.toBe(expandedIndex);
});
const work = vi.hoisted(() => ({ chainCalls: 0, chainItems: 0, positionItems: 0 }));
vi.mock("../chat-turn-presentation.ts", async (importOriginal) => {
const original = await importOriginal<typeof import("../chat-turn-presentation.ts")>();
return {
...original,
coalesceStreamRuns: (...args: Parameters<typeof original.coalesceStreamRuns>) => {
work.chainCalls++;
work.chainItems += args[0].length;
return original.coalesceStreamRuns(...args);
},
};
});
vi.mock("./chat-position-projection.ts", async (importOriginal) => {
const original = await importOriginal<typeof import("./chat-position-projection.ts")>();
return {
...original,
projectChatPositions: (...args: Parameters<typeof original.projectChatPositions>) => {
function* counted() {
for (const item of args[0]) {
work.positionItems++;
yield item;
}
}
return original.projectChatPositions(counted(), args[1], args[2], args[3]);
},
};
});
afterEach(() => resetChatThreadState());
function pastMessages(size: number, tagged = true): unknown[] {
return Array.from({ length: size }, (_, i) => {
const phase = i % (tagged ? 6 : 4);
const toolPhase = phase === 2 || phase === 4;
const finalPhase = phase === (tagged ? 5 : 3);
return {
role: phase === 0 ? "user" : toolPhase ? "toolResult" : "assistant",
content: phase === 0 ? "Question" : finalPhase ? "Answer" : "Work",
...(toolPhase ? { toolName: "read", toolCallId: `call-${i}` } : {}),
...(phase === 1 || (tagged && phase === 3)
? { phase: "commentary" }
: finalPhase
? { phase: "final_answer", stopReason: "stop" }
: {}),
timestamp: i + 1,
__openclaw: {
id: `history-${i}`,
seq: i + 1,
...(tagged ? { runId: `history-run-${Math.floor(i / 6)}` } : {}),
},
};
});
}
function streamInput(size: number): BuildChatItemsProps {
const messages = pastMessages(size);
const liveMessage = (id: string, offset: number, fields: Record<string, unknown>) =>
makeMessage(id, size + offset, { timestamp: size + offset, runId: "live", ...fields });
messages.push(
liveMessage("live-user", 1, { role: "user" }),
liveMessage("live-comment", 2, { phase: "commentary" }),
makeTool("live-tool", size + 3, "live", size + 3),
);
return chatProps({
paneId: "stream-projection",
messages,
stream: "Initial",
streamStartedAt: size + 4,
runId: "live",
runWorking: true,
});
}
it("keeps stream-only projection work bounded by the tail across history sizes", () => {
const measure = (size: number) => {
resetChatThreadState();
const input = streamInput(size);
const expanded = new Map<string, boolean>();
let initialChain: Chain | undefined;
const project = (stream: string) => {
const items = buildCachedChatItems({ ...input, stream });
const chain = chainFor(items, { ...chainOptions, runWorking: true });
initialChain ??= chain;
expect(chain.transcriptItems).toBe(initialChain.transcriptItems);
expect(chain.collapsedItems).toBe(initialChain.collapsedItems);
return indexFor(chain, expanded, labels);
};
const initial = project("Initial");
const previews = new Map();
const presentation = projectTranscriptRows(initial.rows, previews, []);
Object.assign(work, { chainCalls: 0, chainItems: 0, positionItems: 0 });
for (const stream of ["One", "One two", "One two three"]) {
const next = project(stream);
expect(next.rows).toBe(initial.rows);
expect(next.positionIndex.markers).toBe(initial.positionIndex.markers);
expect(projectTranscriptRows(next.rows, previews, [])).toBe(presentation);
}
return { ...work };
};
const small = measure(20);
const large = measure(400);
expect(large).toEqual(small);
expect(measure(1_000)).toEqual(small);
expect(large.chainCalls).toBe(0);
expect(large.positionItems).toBeLessThanOrEqual(6);
});
it.each([
{ name: "dashboard frame and queued tail", framed: true, searchActive: false },
{ name: "unframed stream without boundary", framed: false, searchActive: false },
{ name: "search projection", framed: true, searchActive: true },
])("recomputes streamed history: $name", ({ framed, searchActive }) => {
const input = streamInput(6);
input.messages = [...pastMessages(4, false), ...input.messages.slice(-3)];
const historical = (seq: number, fields: Record<string, unknown>) =>
makeMessage(`history-${seq - 1}`, seq, {
timestamp: seq,
__openclaw: { id: `history-${seq - 1}`, seq },
...fields,
});
const input = chatProps({
paneId: "stream-projection",
messages: [
historical(1, { role: "user", content: "Question" }),
historical(2, { content: "Work", phase: "commentary" }),
historical(3, {
role: "toolResult",
content: "Work",
toolName: "read",
toolCallId: "call-2",
}),
historical(4, { content: "Answer", phase: "final_answer", stopReason: "stop" }),
makeMessage("live-user", 7, { role: "user", timestamp: 7, runId: "live" }),
makeMessage("live-comment", 8, { phase: "commentary", timestamp: 8, runId: "live" }),
makeTool("live-tool", 9, "live", 9),
],
stream: "Initial",
streamStartedAt: 10,
runId: "live",
runWorking: true,
});
if (!framed) {
input.runId = null;
}
@@ -498,11 +378,8 @@ it.each([
}
const options = { runWorking: true, searchActive };
const expanded = new Map<string, boolean>();
const retained: {
chain: Chain;
index: Index;
snapshot: unknown;
}[] = [];
const previews = new Map();
const retained: { chain: Chain; index: Index; snapshot: unknown }[] = [];
const texts = [
" ",
"<thinking>Private planning</thinking>",
@@ -535,36 +412,6 @@ it.each([
setExpansionState(expanded, workKey, false);
}
const index = indexFor(chain, expanded);
const fullChain = chainFor([...items], options);
const fullIndex = indexFor(fullChain, expanded);
const currentChain = (value: Chain) => ({
...value,
collapsedItems: value.collapsedItems.map((item) =>
item.key === value.live?.item.key ? value.live.item : item,
),
transcriptItems: value.transcriptItems.map((item) =>
item.key === value.live?.item.key ? value.live.item : item,
),
});
const currentIndex = (value: Index) => ({
...value,
rows: value.rows.map((row) =>
row.kind === "item" && row.key === value.liveItem?.key
? { ...row, item: value.liveItem }
: row,
),
positionIndex: {
...value.positionIndex,
markers: value.positionIndex.markers.map((marker) => ({
...marker,
message:
value.positionIndex.liveMarkers?.find((candidate) => candidate.id === marker.id)
?.message ?? marker.message,
})),
},
});
expect(currentChain(chain)).toEqual(currentChain(fullChain));
expect(currentIndex(index)).toEqual(currentIndex(fullIndex));
expect(chainFor(items, options)).toBe(chain);
expect(indexFor(chain, expanded)).toBe(index);
if (live) {
@@ -580,9 +427,10 @@ it.each([
expect(owner.boundaryId).toBeUndefined();
}
if (framed && visible) {
const marker = index.positionIndex.markers.find((item) => item.id === "run:live");
expect(
currentIndex(index).positionIndex.markers.find((marker) => marker.id === "run:live")
?.message,
index.positionIndex.liveMarkers?.find((item) => item.id === marker?.id)?.message ??
marker?.message,
).toMatchObject({ content: [{ type: "text", text: stream }] });
}
if ([3, 4, 6, 9].includes(delta)) {
@@ -594,6 +442,9 @@ it.each([
expect(index.positionIndex.markerIdsByMessageId).toBe(
previous.index.positionIndex.markerIdsByMessageId,
);
expect(projectTranscriptRows(index.rows, previews, [])).toBe(
projectTranscriptRows(previous.index.rows, previews, []),
);
for (const [position, row] of index.rows.entries()) {
if (row.key === ownerKey) {
break;
@@ -707,11 +558,16 @@ describe("subagent handoff", () => {
(part) => part.kind === "group" && part.messages.every((source) => source.message !== prose),
);
const resumedRow = readRow("read-2", "announce:resume", 7);
const logs = logsOf([...waiting, resumedRow, answer]);
const resumed = projectHistory([...waiting, resumedRow, answer]);
const logs = framesOf(resumed).flatMap((frame) => kind(frame.parts, "activity-run"));
expect(logs).toHaveLength(1);
// The row the reader may have opened while waiting keeps its identity.
expect(logs[0]).toMatchObject({ key: `activity:${waitingRow?.key}` });
expect(logs[0]?.groups.map((group) => group.runId)).toEqual(["run-1", "announce:resume"]);
const index = indexFor(resumed);
expect(new Set(["read-1", "read-2"].map((id) => index.messageRowKeysById.get(id)))).toEqual(
new Set([framesOf(resumed)[0]?.key]),
);
const spoke = makeMessage("spoke", 6, {
timestamp: 45,
runId: "announce:resume",