mirrors/openclaw
0
0 Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-10-26 10:13:27 +08:00
Code Activity

fix: Control UI shows run commentary twice during long tool runs (#167054)

Browse Source 
* fix(gateway): retire live assistant text by exact occurrence identity

Control UI rendered run commentary twice mid-run: cumulative inFlightRun
snapshots replayed committed text beside paged history while bounded event
replay omitted most earlier commentary identities. Client text subtraction
could not reliably recover ownership from those partial views.

Give embedded runs, workers, CLI, ACP, Agents API and native Codex exact
assistant occurrence identities. Gateway retirement owns the unpersisted
live tail across snapshots, replacement deltas and display-final projection;
terminal delivery content remains complete. Native capture freezes occurrence
IDs before mirror persistence so later bytes cannot share an older receipt.
Decide silent-reply suppression from full available source, preserving the
existing trailing-token and directive policies.

Keep still-streaming unsaved text at the bottom of its run, below saved rows,
and give it a transcript position when saved. Steers remain below all run
output. Preserve completed commentary and saved-row causal order. Remove
client cumulative-prefix retirement, pending commentary handoffs, steer
splitting and redundant unreleased delta itemId/itemStartOffset fields.
Cap eviction may conservatively redisplay committed text; never infer
ownership to hide potentially unsaved output.

Builds on and supersedes the live-delta reconciliation from #158459; its regression suite is retained with ordering updated to the live-at-bottom invariant.

Rebased onto 278d66f245, preserving main's native settlement helpers,
transcript authority checks and terminal identity reader consolidation.
Keep native provisional-stream decisions with the occurrence tracker and
move existing type/test support to their adjacent owners without ratchets.

Proof: all 144 focused files pass one at a time, including the 22-paragraph
production-shaped regression, all 17 retained acceptance cases plus reconnect,
and the seeded real-cap/reused-identity model (891 ms). Core, UI-test,
extension and Gateway test typechecks, protocol registry/generation/native
checks, scoped core/UI/extension lint, formatting and diff checks pass.
Full-range and refinement P2 autoreviews found no actionable findings.

Co-authored-by: Dallin Romney <dallinromney@gmail.com>

* fix(ui): apply hidden replacement frames to the live stream

A Gateway replacement frame can now carry an unpersisted tail that is a
bare silent token after its saved prefix retires. The delta handler skipped
silent and heartbeat text before applying replacements, so the previous
baseline stayed visible beside its durable row.

Treat a replacement as authoritative even when its new text is hidden: it
retracts the live stream instead of being ignored. Non-replacing silent
deltas are still ignored.

Proof: the new chat-gateway case fails before the change (expected
'The token is ' to be '') and passes after; chat-gateway (87),
commentary-dedupe acceptance (18) and inflight history (44) suites pass;
UI typecheck, oxfmt and diff checks pass.

* fix(agents): reconcile run identity additions with main

Align identical runId fields with the positions already added on main so Git does not produce duplicate properties in the PR merge tree. Preserves exact assistant occurrence retirement and complete ACP transcript persistence.

* fix(gateway): preserve canonical streaming order across clients

Publish canonical assistant events before their derived chat projection so SDK live consumers receive text once. Keep inflight snapshot projection and buffer contracts in leaf owners, migrate retained UI fixtures to the unsaved-text contract, preserve real compaction event exports, and shrink obsolete mock baseline entries.

* fix(gateway): retain commentary correction ordering

Apply the shared chat projection before commentary item delivery and after canonical assistant delivery. Preserve correction-before-preamble rendering and duplicate-free SDK delivery together; both existing regressions pass.

---------

Co-authored-by: Dallin Romney <dallinromney@gmail.com>
This commit is contained in:
Peter SteinbergerandDallin Romney authored and GitHub committed 2026-10-08 04:39:55 -07:00
1 parent 4bf68b98ab
commit af210e1f3f
118 files changed
+6600 -3167
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

config/test-mock-exports-baseline.txt
-2
View File
@@ -5463,7 +5463,6 @@
["src/commands/agent-via-gateway.test.ts","../plugins/one-shot-diagnostics.js","731e5bb43dc7a002dfbcbdced135d87ee2d6692c7c64eca36091f202d612f19e",1]
["src/commands/agent-via-gateway.test.ts","./agent.js","ee8506e54283d7439912b570b7e8ef7e76fca54c37b21bffe963630e356e4348",1]
["src/commands/agent-via-gateway.test.ts","./agent/session.runtime.js","ae98be297e23e86be84254e2100d745d79bc1d79322bb1063a4ff4ff22df9a53",1]
["src/commands/agent.acp.test.ts","../agents/command/attempt-execution.runtime.js","281744fc97c890b20b1fce926e8b527cd88851f35e98e95f4fde33d3bff326f0",1]
["src/commands/agent.acp.test.ts","../agents/command/delivery.runtime.js","0712d95e72e1deabbb3c9422358b112989379e58cf9907b160d65bfac2505b10",1]
["src/commands/agent.acp.test.ts","../infra/agent-events.js","0aaf116f9e7afda89af95f9fa21177c5b0612569cb77d83334764f29a423c596",1]
["src/commands/agent.delivery-format.test.ts","../agents/command/delivery.runtime.js","e90b007d53ab845caac92ae7dc7faf2c3c385dd59e84be19aad7e5aed3b39288",1]
@@ -9556,7 +9555,6 @@
["src/plugins/web-provider-runtime-shared.test.ts","./runtime/load-context.js","9357987610fe409bfbe3cd61112f3f299a0ee7494629c2cc9ea7120a32aff150",1]
["src/plugins/wired-hooks-after-tool-call.e2e.test.ts","../infra/agent-events.js","5d64ca88850e72e5466c2c98ea13ba15d581dc72da0bcaa24c952fd3258806f5",1]
["src/plugins/wired-hooks-after-tool-call.e2e.test.ts","../plugins/hook-runner-global.js","6d9823e1f3c0ba539024acbfa6778307c43992be6b2b53cdf5aa0f585d74d084",1]
["src/plugins/wired-hooks-compaction.test.ts","../infra/agent-events.js","40f503b73546879041867c9975af80e082af495b2b8e15f003b76bd7585ba938",1]
["src/plugins/wired-hooks-compaction.test.ts","../plugins/hook-runner-global.js","6d9823e1f3c0ba539024acbfa6778307c43992be6b2b53cdf5aa0f585d74d084",1]
["src/process/background-work.test.ts","../logging/diagnostic-runtime.js","3005d9447dc22400dda2144b032ad7501a9cb7972627fc9a168b58643fd04d82",1]
["src/process/child-process-tree.reaping.test.ts","./kill-tree.js","883637170b915a6ed10435001c6b849d92ff07f0a4d1efdf3ab0950d962d05e1",1]
docs/gateway/protocol/handshake.md
+6
View File
@@ -238,6 +238,12 @@ the general protocol version. Frames stay under 64 KiB, except a negotiated
`worker.heartbeat`, `worker.transcript.commit`, `worker.live-event`,
`worker.inference.start`, and `worker.inference.cancel`.
Assistant messages in `worker.transcript.commit` may carry the live event's
`itemId`. The Gateway binds it to its generated commit key, removes it before
storage, and publishes the correlation only after the row commits. It is not
provider replay data. This worker-only field travels between the exact
builds admitted by the bundle, version, and feature checks above.
For an identity-audited attached run, the live turn capability can record the
credential, build, owner-epoch, and placement checks as one enforced admission
receipt. The receipt contains none of the credential, build hashes, tokens,
docs/gateway/protocol/transport.md
+19 -1
View File
@@ -59,7 +59,17 @@ Frame shapes:
- Response: `{type:"res", id, ok, payload|error}`
- Event: `{type:"event", event, payload, seq?, stateVersion?, recipientProfileId?}`
Live text uses append deltas after an initial recipient snapshot. An outer event
Live text uses append deltas after an initial recipient snapshot. The
`chat.history` field `inFlightRun.text` restores only the current run's unpersisted
assistant tail. Silent-reply suppression uses the full available source; the tail keeps existing directive and media normalization without being classified as a silent reply again. If the cap evicts source context, retained tail text stays visible conservatively. Cumulative snapshots or replacements that restore text evicted by the live buffer cap may redisplay committed text when ownership is unknown, preserving unsaved output.
When durable history or a keyed
commentary item takes ownership of live text, the Gateway sends an existing
`replace: true` frame before subsequent append deltas. Terminal result snapshots
retain their complete result in `message.content`. A buffer-backed terminal can
also supply the existing `message.openclawDisplayContent` projection: the same
unpersisted tail finalized for display, including an explicitly empty tail and
any Canvas blocks. The Control UI applies that projection before storing its
live view; delivery consumers retain the complete content. An outer event
sequence gap means a client may have lost part of that baseline: retire the
connection and reconnect before applying more deltas. If the frame revealing the
gap is a `chat` final, aborted, or error event, deliver its authoritative terminal
@@ -70,6 +80,14 @@ after reconnect; the Gateway sends a complete snapshot with the next text frame
for each observed run. Run-local payload sequences can skip numbers because text
is paced and coalesced; they are not the outer connection sequence.
Retirement uses assistant occurrence identity, never text-prefix matching.
Producers correlate live occurrences with committed transcript idempotency keys
or the host's exact source receipts. Native harnesses that persist part of a
still-streaming item retain its text-merge `itemId` and separately identify each
captured interval with `occurrenceId`. Unidentified text remains in the
live tail until it gains an identity-bearing commit or the run terminates;
equal durable text alone cannot hide it.
Clients that share one connection among several views must keep reconstruction
with their local stream owner. A new local listener may join after the wire
snapshot was delivered to another view. Seed it from that owner's current state,
docs/plugins/sdk-agent-harness/attempt-runtime.md
+12
View File
@@ -201,6 +201,18 @@ isolates observer failures, and `createAgentHarnessAttemptLifecycle` gates
lifecycle events and deduplicates execution phases. Native interruption,
completion decisions, output flushing, and cleanup remain backend-owned.
Assistant event `data.itemId` identifies the native text item for cumulative
snapshots and delta merging. A harness that persists part of an item while it
continues streaming also supplies `data.occurrenceId`, an opaque string identifying
the exact live interval. Freeze that occurrence when capturing the transcript
candidate, before awaiting persistence, and allocate a new occurrence for later
bytes without changing the native item ID or delta text. Successful transcript
publication supplies the captured IDs in `assistantItemIds`; failed writes do not
retire them. Retries retain every occurrence actually included in the candidate.
Without `occurrenceId`, retirement uses `itemId` or the host's source receipt.
Text without either identity stays live until terminal settlement; matching
durable text alone never proves ownership.
The private `openclaw/plugin-sdk/agent-harness-tool-runtime` provides correlated
execution promises and argument/start snapshots through
`createAgentHarnessToolExecutionRegistry` and
docs/plugins/sdk-runtime/agent.md
+30
View File
@@ -114,6 +114,36 @@ the next Plugin SDK major. See the [migration table](/plugins/sdk-migration/how-
for every replacement, return value, and the additive extension and provider
replay APIs. Transcript formats, schemas, and update behavior are unchanged.
## Native assistant persistence
Assistant producers should reuse the committed row's `idempotencyKey` as the
live `assistant` event's `itemId`. The Gateway retires that exact occurrence
when its run-owned commit is published, including corrections that arrive
after persistence. Separate occurrences need separate identities even when
their text is identical.
Native harnesses that publish committed assistant rows with
`publishSessionTranscriptUpdateByIdentity` from
`openclaw/plugin-sdk/session-transcript-runtime` can include
`update.assistantItemIds`. These are the exact `assistant` stream item IDs whose
live display the committed row replaces or supersedes. Capture the IDs before
awaiting persistence, and publish only after the row commits or an exact
idempotent persistence receipt confirms it. An empty array still identifies a
native row with no preceding streamed item. The persisted row's existing
idempotency key also identifies a later canonical assistant frame.
This field is display provenance, not terminal or run authorization. Existing
session and run ownership checks still apply. It is internal to the host's
transcript notification path: do not put it in the persisted message or public
gateway events. Independently owned keyed commentary and async rows omit it.
When steering commits a completed item, include only that item's ID; a later
unfinished item remains live even if its text repeats the committed row.
The Gateway does not infer ownership from text. An unkeyed producer's text
stays in the live tail until an identity-bearing commit can own it or the run
terminates. Such a producer can temporarily show a duplicate durable row;
the Gateway favors preserving unsaved text over guessing which occurrence to hide.
## Bounded model context
Use `await SessionManager.openModelContextAsync(...)` from
docs/web/control-ui.md
+8
View File
@@ -87,6 +87,14 @@ the total tool-call count. When no run duration is available, the heading reads
non-success outcomes remain visible even when collapsed, such as
**Worked for 2 minutes, 3 seconds · 2 failed**.
Still-streaming assistant text stays at the bottom of its run, below saved output, and
takes its transcript position once saved.
Steering keeps the current response intact. A steer appears below all server
output from the run it targets, including live text, restored text, commentary,
and saved assistant messages. These display rules do not rewrite the stored
transcript or split an assistant message around a steer.
Consecutive tool activity shares one expandable log, including when background
work resumes in a new run. Visible messages, media, and conversation markers
keep their place and separate logs; live response text and the working indicator
extensions/acpx/src/harness-attempt.ts
+7 -4
View File
@@ -246,6 +246,7 @@ export async function runAcpHarnessAttempt(params: {
}
};
const requestId = `${admission.entryId}:acp:${randomUUID()}`;
const assistantItemId = `${requestId}:acp:assistant`;
// Prose labels keep file paths and literal user text out of native slash-command dispatch.
const turn: AcpRuntimeTurnInput &
Pick<AcpxRuntimeTurnInput, "onPermissionRequest" | "assertActive"> = {
@@ -297,7 +298,10 @@ export async function runAcpHarnessAttempt(params: {
assertActive();
}
text += event.text;
const update = { stream: "assistant", data: { text, delta: event.text } };
const update = {
stream: "assistant",
data: { text, delta: event.text, itemId: assistantItemId },
};
emitAgentEvent({
runId: input.runId,
sessionKey,
@@ -357,13 +361,12 @@ export async function runAcpHarnessAttempt(params: {
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
};
const key = `${requestId}:acp:assistant`;
const written = await appendSessionTranscriptMessageByIdentityStrict({
...transcript,
config: input.config,
runId: input.runId,
updateMode: "inline",
message: { ...assistant, idempotencyKey: key },
message: { ...assistant, idempotencyKey: assistantItemId },
beforeFreshMessageCommit: createNativeSessionBindingAuthority(
[],
input.hostCapabilities.assertActive,
@@ -373,7 +376,7 @@ export async function runAcpHarnessAttempt(params: {
throw new Error("ACP assistant transcript was not committed");
}
assistant = written.result.message;
assistantIdempotencyKey = key;
assistantIdempotencyKey = assistantItemId;
terminalAnchor = written.result.anchor;
messages = (
await SessionManager.openModelContextAsync(transcript, {
extensions/agentsapi/agentsapi-attempt.test.ts
+125 -16
View File
@@ -13,12 +13,14 @@ import {
resetGlobalHookRunner,
} from "openclaw/plugin-sdk/plugin-test-runtime";
import { upsertSessionEntry } from "openclaw/plugin-sdk/session-store-runtime";
import * as transcriptRuntime from "openclaw/plugin-sdk/session-transcript-runtime";
import { useSessionStoreTempDirs } from "openclaw/plugin-sdk/sqlite-runtime-testing";
import { withOpenClawTestState } from "openclaw/plugin-sdk/test-state";
import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { afterAll, afterEach, assert, beforeEach, describe, expect, it, vi } from "vitest";
import { runAgentsApiAttempt, type AgentsApiPromptHistories } from "./agentsapi-attempt.js";
import type { AgentsApiBinding } from "./agentsapi-bindings.js";
import { AgentsApiClient, type AgentsApiItem } from "./agentsapi-client.js";
import { AgentsApiMessageProjection } from "./agentsapi-messages.js";
import { createHostedSession, createModel, createTurn } from "./agentsapi.test-support.js";
const { createSession, registerRun } = vi.hoisted(() => ({
@@ -40,29 +42,41 @@ vi.mock("openclaw/plugin-sdk/agent-harness-runtime", async (importOriginal) => {
});
const tempDirs = useSessionStoreTempDirs(afterAll, "agentsapi-completed-reply-");
const readItems = vi.fn<AgentsApiClient["items"]>();
const readTurn = vi.fn<AgentsApiClient["turn"]>();
beforeEach(() => {
vi.spyOn(AgentsApiClient.prototype, "create").mockResolvedValue("session-fixture");
vi.spyOn(AgentsApiClient.prototype, "items").mockResolvedValue([completedItem]);
vi.spyOn(AgentsApiClient.prototype, "turn").mockResolvedValue(completedTurn);
vi.spyOn(AgentsApiClient.prototype, "session").mockResolvedValue(hostedSession);
createSession.mockImplementation((options) => ({
function completedSession(
options: Parameters<typeof createSession>[0],
turn = completedTurn,
item = completedItem,
) {
return {
isAvailable: () => false,
isSettled: () => true,
wasSubmitted: () => true,
queueMessage: async () => {},
readUsageTurns: async () => [completedTurn],
readUsageTurns: async () => [turn],
run: async () => {
options.onSettled?.();
await options.onReconcile?.(completedTurn, [completedItem]);
return { turn: completedTurn, cancelled: false, terminatedByTool: false };
await options.onReconcile?.(turn, [item]);
return { turn, cancelled: false, terminatedByTool: false };
},
close: async () => {},
reconcileAfterClose: async () => {
await options.onReconcile?.(completedTurn, [completedItem]);
return completedTurn;
await options.onReconcile?.(turn, [item]);
return turn;
},
}));
};
}
beforeEach(() => {
vi.spyOn(AgentsApiClient.prototype, "create").mockResolvedValue("session-fixture");
readItems.mockReset().mockResolvedValue([completedItem]);
readTurn.mockReset().mockResolvedValue(completedTurn);
vi.spyOn(AgentsApiClient.prototype, "items").mockImplementation(readItems);
vi.spyOn(AgentsApiClient.prototype, "turn").mockImplementation(readTurn);
vi.spyOn(AgentsApiClient.prototype, "session").mockResolvedValue(hostedSession);
createSession.mockImplementation(completedSession);
});
afterEach(() => {
@@ -74,6 +88,85 @@ afterEach(() => {
});
describe("Agents API completed reply settlement", () => {
it("publishes exact committed item identities without acquiring a newer preview", async () => {
const fixture = await createAttempt();
const events: Parameters<NonNullable<AgentHarnessAttemptParamsV2["onAgentEvent"]>>[0][] = [];
const projection = new AgentsApiMessageProjection(
fixture.params,
"session-fixture",
(event) => {
events.push(event);
},
() => {},
);
const append = transcriptRuntime.appendSessionTranscriptMessageByIdentityStrict;
const appending = createDeferred<void>();
const release = createDeferred<void>();
vi.spyOn(
transcriptRuntime,
"appendSessionTranscriptMessageByIdentityStrict",
).mockImplementation(async (params) => {
appending.resolve();
await release.promise;
return append(params);
});
const publish = vi.spyOn(transcriptRuntime, "publishSessionTranscriptUpdateByIdentity");
const completion = projection.commit(completedTurn, [completedItem]);
try {
await appending.promise;
await projection.observe({
type: "agent.session.turn.item.added",
item: {
...completedItem,
id: "newer-preview",
status: "in_progress",
content: [{ type: "output_text", text: "Still working." }],
},
});
expect(events.at(-1)).toMatchObject({
stream: "assistant",
data: {
itemId: "agentsapi:session-fixture:turn-fixture:newer-preview",
text: "Still working.",
},
});
release.resolve();
await completion;
const manager = await SessionManager.openAsync(fixture.target, fixture.params.workspaceDir);
const saved = manager.getBranch().find((entry) => entry.type === "message");
assert(saved?.type === "message");
expect(saved.message).toMatchObject({ __openclaw: { runId: fixture.params.runId } });
expect(publish).toHaveBeenCalledExactlyOnceWith({
agentId: fixture.target.agentId,
sessionId: fixture.target.sessionId,
sessionKey: fixture.target.sessionKey,
storePath: fixture.target.storePath,
sessionEntry: undefined,
update: {
message: saved.message,
messageId: saved.id,
messageSeq: 1,
runId: fixture.params.runId,
assistantItemIds: [
"agentsapi:session-fixture:turn-fixture:answer-fixture",
"agentsapi:session-fixture:turn-fixture:reply",
],
},
});
expect(events.at(-1)).toMatchObject({
stream: "assistant",
data: {
itemId: "agentsapi:session-fixture:turn-fixture:reply",
text: "The completed answer.",
},
});
} finally {
release.resolve();
await completion;
}
});
it("retains and presents one durable completed reply when artifact listing fails", async () => {
const fixture = await createAttempt();
const failure = new Error("fixture artifact listing failed");
@@ -156,9 +249,12 @@ it("leaves PDF steering uncommitted so its next turn transfers the original in t
const started = createDeferred<void>();
const finish = createDeferred<void>();
const submitted: string[] = [];
const followupTurn = createTurn({ id: "turn-pdf-followup" });
const followupItem = { ...completedItem, id: "answer-pdf-followup", turn_id: followupTurn.id };
let firstTurn = true;
createSession.mockImplementation((options) => {
const activeTurn = firstTurn;
const turn = activeTurn ? completedTurn : followupTurn;
firstTurn = false;
return {
isAvailable: () => true,
@@ -167,7 +263,7 @@ it("leaves PDF steering uncommitted so its next turn transfers the original in t
queueMessage: async (_text, persistInput) => {
await persistInput?.();
},
readUsageTurns: async () => [completedTurn],
readUsageTurns: async () => [turn],
run: async (text, persistInput, onSubmitted) => {
await persistInput();
onSubmitted();
@@ -177,10 +273,10 @@ it("leaves PDF steering uncommitted so its next turn transfers the original in t
await finish.promise;
}
options.onSettled?.();
return { turn: completedTurn, cancelled: false, terminatedByTool: false };
return { turn, cancelled: false, terminatedByTool: false };
},
close: async () => {},
reconcileAfterClose: async () => completedTurn,
reconcileAfterClose: async () => turn,
};
});
const first = fixture.run();
@@ -207,6 +303,8 @@ it("leaves PDF steering uncommitted so its next turn transfers the original in t
}
expect((await first).terminal).toEqual({ kind: "ok" });
readTurn.mockResolvedValue(followupTurn);
readItems.mockResolvedValue([followupItem]);
fixture.params.runId = "pdf-followup-run";
fixture.params.prompt = prompt;
fixture.params.media = media;
@@ -382,6 +480,17 @@ describe("Agents API retry prompt history", () => {
retried.currentAttemptCompletedAssistant,
]);
const nextNativeTurn = createTurn({ id: "turn-next-user" });
const nextNativeItem = {
...completedItem,
id: "answer-next-user",
turn_id: nextNativeTurn.id,
};
readTurn.mockResolvedValue(nextNativeTurn);
readItems.mockResolvedValue([nextNativeItem]);
createSession.mockImplementationOnce((options) =>
completedSession(options, nextNativeTurn, nextNativeItem),
);
fixture.params.runId = "next-user-run";
fixture.params.prompt = "Start the next request.";
fixture.params.skipPreparedUserTurnMessage = false;
extensions/agentsapi/agentsapi-messages.test.ts
+76 -42
View File
@@ -8,50 +8,84 @@ import { createModel, createTurn } from "./agentsapi.test-support.js";
type AgentEvent = Parameters<NonNullable<AgentHarnessAttemptParamsV2["onAgentEvent"]>>[0];
describe("Agents API commentary projection", () => {
it("completes commentary after an identical final delta without replaying completion", async () => {
const { projection, events } = createProjection();
const item: AgentsApiItem = {
id: "commentary-fixture",
type: "message",
role: "assistant",
phase: "commentary",
status: "in_progress",
turn_id: "turn-fixture",
content: [{ type: "output_text", text: "" }],
};
const text = "Checking the command.";
it.each(["added", "done"])(
"hands off commentary identified at item.%s once",
async (phaseKnownAt) => {
const { projection, events } = createProjection();
const item: AgentsApiItem = {
id: "commentary-fixture",
type: "message",
role: "assistant",
phase: phaseKnownAt === "added" ? "commentary" : null,
status: "in_progress",
turn_id: "turn-fixture",
content: [{ type: "output_text", text: "" }],
};
const text = "Checking the command.\n\n pwd";
await projection.observe({ type: "agent.session.turn.item.added", item });
await projection.observe({
type: "agent.session.turn.output_text.delta",
item_id: item.id,
turn_id: "turn-fixture",
content_index: 0,
delta: text,
});
const completed: AgentsApiEvent = {
type: "agent.session.turn.item.done",
item: {
...item,
status: "completed",
content: [{ type: "output_text", text }],
},
};
await projection.observe(completed);
await projection.observe(completed);
await projection.observe({ type: "agent.session.turn.item.added", item });
await projection.observe({
type: "agent.session.turn.output_text.delta",
item_id: item.id,
turn_id: "turn-fixture",
content_index: 0,
delta: text,
});
const completed: AgentsApiEvent = {
type: "agent.session.turn.item.done",
item: {
...item,
phase: "commentary",
status: "completed",
content: [{ type: "output_text", text }],
},
};
await projection.observe(completed);
await projection.observe(completed);
const preamble = {
itemId: "agentsapi:session-fixture:turn-fixture:commentary-fixture",
kind: "preamble",
title: "Preamble",
progressText: text,
source: "agentsapi",
};
expect(events).toEqual([
{ stream: "item", data: { ...preamble, phase: "update" } },
{ stream: "item", data: { ...preamble, phase: "end" } },
]);
});
const itemId = "agentsapi:session-fixture:turn-fixture:commentary-fixture";
const preamble = {
itemId,
kind: "preamble",
title: "Preamble",
progressText: text,
source: "agentsapi",
};
expect(events).toEqual([
...(phaseKnownAt === "added"
? [{ stream: "item", data: { ...preamble, phase: "update" } }]
: [
{
stream: "assistant",
data: { itemId, text, delta: "", replaceable: true, replace: true },
},
{ stream: "assistant", data: { itemId, text: "", delta: "", replace: true } },
]),
{ stream: "item", data: { ...preamble, phase: "end" } },
]);
await projection.observe({
type: "agent.session.turn.item.done",
item: {
...item,
id: "final-fixture",
phase: "final_answer",
status: "completed",
content: [{ type: "output_text", text: "Done." }],
},
});
expect(events.at(-1)).toEqual({
stream: "assistant",
data: {
itemId: "agentsapi:session-fixture:turn-fixture:final-fixture",
text: "Done.",
delta: "",
replaceable: true,
replace: true,
},
});
},
);
});
describe("Agents API final usage accounting", () => {
extensions/agentsapi/agentsapi-messages.ts
+30 -7
View File
@@ -346,16 +346,23 @@ export class AgentsApiMessageProjection {
promptError: turn.error,
turnCompleted: isAgentsApiTerminalTurn(turn.status),
});
const replyItemId = `agentsapi:${this.remoteSessionId}:${turn.id}:reply`;
if (text) {
this.reply.lastAssistant = await this.append({
...assistant,
idempotencyKey: `agentsapi:${this.remoteSessionId}:${turn.id}`,
});
const assistantItemIds = this.visibleAssistantItemId
? [this.visibleAssistantItemId, replyItemId]
: [replyItemId];
this.reply.lastAssistant = await this.append(
{
...assistant,
idempotencyKey: `agentsapi:${this.remoteSessionId}:${turn.id}`,
},
assistantItemIds,
);
this.assertCurrent();
await this.params.onAssistantMessageStart?.();
this.assertCurrent();
}
await this.emitAssistantSnapshot(`agentsapi:${this.remoteSessionId}:${turn.id}:reply`, text);
await this.emitAssistantSnapshot(replyItemId, text);
if (text) {
this.assertCurrent();
await this.params.onPartialReply?.({ text });
@@ -500,6 +507,14 @@ export class AgentsApiMessageProjection {
const id = this.identity(state.turnId, state.item.id);
const text = joinTextParts(state.texts);
if (state.item.phase === "commentary") {
if (this.visibleAssistantItemId === id) {
this.visibleAssistantItemId = undefined;
state.lastAssistantText = undefined;
await this.emit({
stream: "assistant",
data: { itemId: id, text: "", delta: "", replace: true },
});
}
const phase = terminal ? "end" : "update";
if (
!text.trim() ||
@@ -657,8 +672,16 @@ export class AgentsApiMessageProjection {
this.assertCurrent();
}
private append<TMessage extends AgentMessage>(message: TMessage): Promise<TMessage> {
return appendAgentsApiTranscriptMessage(this.params, message, this.assertCurrent);
private append<TMessage extends AgentMessage>(
message: TMessage,
assistantItemIds?: readonly string[],
): Promise<TMessage> {
return appendAgentsApiTranscriptMessage(
this.params,
message,
this.assertCurrent,
assistantItemIds,
);
}
}
extensions/agentsapi/agentsapi-transcript.ts
+23 -2
View File
@@ -8,7 +8,10 @@ import type {
AnyAgentTool,
} from "openclaw/plugin-sdk/agent-harness-runtime";
import type { NativeSessionBindingAuthority } from "openclaw/plugin-sdk/agent-harness-session-runtime";
import { appendSessionTranscriptMessageByIdentityStrict } from "openclaw/plugin-sdk/session-transcript-runtime";
import {
appendSessionTranscriptMessageByIdentityStrict,
publishSessionTranscriptUpdateByIdentity,
} from "openclaw/plugin-sdk/session-transcript-runtime";
import { asOptionalRecord } from "openclaw/plugin-sdk/string-coerce-runtime";
import type { AgentsApiFunctionCall, AgentsApiItem } from "./agentsapi-client.js";
import {
@@ -199,11 +202,14 @@ export async function appendAgentsApiTranscriptMessage<TMessage extends AgentMes
params: AgentHarnessAttemptParamsV2,
message: TMessage,
assertCurrent: TranscriptAssertion,
assistantItemIds?: readonly string[],
): Promise<TMessage> {
await assertTranscriptCurrent(assertCurrent);
const target = requireAgentsApiSessionTarget(params);
const append = await appendSessionTranscriptMessageByIdentityStrict({
...requireAgentsApiSessionTarget(params),
...target,
config: params.config,
runId: params.runId,
message,
beforeFreshMessageCommit: assertCurrent,
});
@@ -211,6 +217,21 @@ export async function appendAgentsApiTranscriptMessage<TMessage extends AgentMes
if (append.kind !== "result") {
throw new Error("Agents API transcript append was refused");
}
if (assistantItemIds) {
await publishSessionTranscriptUpdateByIdentity({
...target,
update: {
message: append.result.message,
messageId: append.result.messageId,
...(append.result.anchor
? { messageSeq: append.result.anchor.activeMessagePosition + 1 }
: {}),
runId: params.runId,
assistantItemIds,
},
});
await assertTranscriptCurrent(assertCurrent);
}
return append.result.message;
}
extensions/codex/src/app-server/event-projector-assistant.ts
+29 -22
View File
@@ -17,6 +17,7 @@ import { CodexSteeringAssistantSegments } from "./event-projector-steering.js";
import { extractRawResponseItemText } from "./event-projector-values.js";
import type { CodexThreadItem, JsonObject } from "./protocol.js";
import type { CodexTranscriptCheckpointEntry } from "./transcript-checkpoint.js";
import { attachCodexAssistantItemIds } from "./upstream-prompt-provenance.js";
type AgentEvent = Parameters<NonNullable<EmbeddedRunAttemptParams["onAgentEvent"]>>[0];
type AnswerCandidateStatus = "candidate" | "superseded" | "selected";
@@ -54,8 +55,6 @@ export class CodexAssistantProjection {
private readonly rawPromotedAssistantItemIds = new Set<string>();
private assistantStarted = false;
private responseModel: string | undefined;
private streamedPartialAssistantItemId: string | undefined;
private streamedPartialAssistantItemReplaceable = false;
// Native handoffs retire earlier finals; terminal summaries contain only the last answer.
private persistableAssistantBarrier = 0;
// A completed answer mirrored before a steer is already durable. Do not
@@ -113,29 +112,20 @@ export class CodexAssistantProjection {
if (knownFinalAnswer) {
this.emitAnswerCandidate(itemId, "candidate");
}
const replace =
this.streamedPartialAssistantItemId !== undefined &&
this.streamedPartialAssistantItemId !== itemId;
// Codex defines final_answer as terminal text. Replacement mode is for
// phase-unknown/provisional items; append-only consumers cannot retract bytes.
if (replace && (!knownFinalAnswer || this.streamedPartialAssistantItemReplaceable)) {
this.streamedPartialAssistantItemReplaceable = true;
} else if (this.streamedPartialAssistantItemId === undefined) {
this.streamedPartialAssistantItemReplaceable = !knownFinalAnswer;
}
this.streamedPartialAssistantItemId = itemId;
const replaceable = this.streamedPartialAssistantItemReplaceable;
const replacement = replace && replaceable;
const { occurrenceId, replacement, replaceable } = this.steeringSegments.recordStream(
itemId,
knownFinalAnswer,
);
const streamPayload = {
text,
delta: replacement ? "" : delta,
...(replacement ? { replace: true as const } : {}),
};
this.steeringSegments.recordStream(itemId, replacement);
this.emitAgentEvent({
stream: "assistant",
data: {
itemId,
occurrenceId,
...streamPayload,
...(replaceable ? { replaceable: true as const } : {}),
},
@@ -290,7 +280,7 @@ export class CodexAssistantProjection {
text &&
phase !== "commentary" &&
candidateWasSupersededBeforeRaw &&
itemId !== this.streamedPartialAssistantItemId &&
itemId !== this.steeringSegments.streamedPartialAssistantItemId &&
!isIdlessTerminalAssistantAfterCompletedWork
) {
return;
@@ -345,6 +335,13 @@ export class CodexAssistantProjection {
});
}
collectTerminalAssistantItemIds(): string[] {
// A settled terminal row supersedes orphan previews as well as selected items.
return this.steeringSegments.collectOccurrenceIds(
this.assistantItemOrder.filter((itemId) => !this.isNonTerminalAssistantItem(itemId)),
);
}
private readCommentaryMessage(itemId: string): AssistantMessage | undefined {
const text = this.assistantTextByItem.get(itemId)?.trim();
const timestamp = this.assistantTimestampByItem.get(itemId);
@@ -379,7 +376,10 @@ export class CodexAssistantProjection {
if (!segment) {
return [];
}
const message = this.createAssistantMessage(segment.text, options);
const message = attachCodexAssistantItemIds(
this.createAssistantMessage(segment.text, options),
segment.occurrenceIds,
);
message.timestamp = segment.split
? this.nextTranscriptTimestamp()
: (this.assistantTimestampByItem.get(itemId) ?? message.timestamp);
@@ -495,7 +495,7 @@ export class CodexAssistantProjection {
}
private adoptSteeringPrefixForCompletion(item: AssistantCompletion | undefined): void {
const sourceId = this.streamedPartialAssistantItemId;
const sourceId = this.steeringSegments.streamedPartialAssistantItemId;
if (
item?.type !== "agentMessage" ||
!sourceId ||
@@ -518,7 +518,7 @@ export class CodexAssistantProjection {
this.supersedeVisibleAnswerCandidate();
}
this.assistantTextByItem.delete(sourceId);
this.streamedPartialAssistantItemId = item.id;
this.steeringSegments.streamedPartialAssistantItemId = item.id;
}
private rememberAssistantPhase(item: CodexThreadItem | undefined): void {
@@ -564,12 +564,19 @@ export class CodexAssistantProjection {
text: string;
phase: "update" | "end";
}): void {
const progressText = params.text.trim();
const cleared = this.steeringSegments.clearProvisionalStream(params.itemId);
if (cleared) {
this.emitAgentEvent({ stream: "assistant", data: cleared });
}
const progressText = params.text.trimEnd();
// Codex completes an item with the same text as its last delta. Channels
// need that boundary before their first notifying post, so agents must not
// collapse completion into a text-only duplicate or invent a timer instead.
const previous = this.lastCommentaryProgressEventByItem.get(params.itemId);
if (!progressText || (previous?.phase === params.phase && previous.text === progressText)) {
if (
!progressText.trim() ||
(previous?.phase === params.phase && previous.text === progressText)
) {
return;
}
this.lastCommentaryProgressEventByItem.set(params.itemId, {
extensions/codex/src/app-server/event-projector-result.ts
+7 -1
View File
@@ -27,6 +27,7 @@ import { CodexUsageProjection } from "./event-projector-usage.js";
import { readCodexProviderRefusal, type CodexProviderRefusal } from "./event-projector-values.js";
import type { CodexTurn, JsonValue } from "./protocol.js";
import { CodexTranscriptCheckpoint } from "./transcript-checkpoint.js";
import { attachCodexAssistantItemIds } from "./upstream-prompt-provenance.js";
import { resolveCodexPromptError } from "./usage-limit-error.js";
export type CodexAppServerToolTelemetry = Partial<
@@ -291,7 +292,12 @@ export abstract class CodexTurnProjection {
commentaryMessages,
toolMessages: this.toolTranscriptProjection.transcriptMessages,
steeringMessages: options?.steeringMessages,
lastAssistant,
lastAssistant: lastAssistant
? attachCodexAssistantItemIds(
lastAssistant,
this.assistantProjection.collectTerminalAssistantItemIds(),
)
: undefined,
turnTainted,
});
const turnFailed = completedTurn?.status === "failed";
extensions/codex/src/app-server/event-projector-steering.ts
+89 -4
View File
@@ -1,3 +1,4 @@
import { randomUUID } from "node:crypto";
import { isSilentReplyPayloadText } from "openclaw/plugin-sdk/reply-chunking";
type SteeringAssistantPrefix = {
@@ -6,26 +7,82 @@ type SteeringAssistantPrefix = {
completed: boolean;
split: boolean;
segment: number;
occurrenceCount: number;
};
type AssistantOccurrences = {
current?: string;
ids: string[];
ownedCount: number;
};
/** Owns the raw assistant cutoffs committed before each steer in one native turn. */
export class CodexSteeringAssistantSegments {
streamedPartialAssistantItemId: string | undefined;
private streamedPartialAssistantItemReplaceable = false;
private readonly visible = new Set<string>();
private readonly completed = new Map<string, number>();
private readonly handoffItems = new Set<string>();
private handoffGeneration = 0;
private readonly persisted = new Map<string, { text: string; nextSegment: number }>();
private readonly pending = new Map<string, SteeringAssistantPrefix>();
private readonly occurrences = new Map<string, AssistantOccurrences>();
private readonly receiptIds = new Map<string, readonly string[]>();
beginSnapshot(): void {
this.pending.clear();
}
recordStream(itemId: string, replacement: boolean): void {
recordStream(itemId: string, knownFinalAnswer: boolean) {
const replace =
this.streamedPartialAssistantItemId !== undefined &&
this.streamedPartialAssistantItemId !== itemId;
// Codex defines final_answer as terminal text. Replacement mode is for
// phase-unknown/provisional items; append-only consumers cannot retract bytes.
if (replace && (!knownFinalAnswer || this.streamedPartialAssistantItemReplaceable)) {
this.streamedPartialAssistantItemReplaceable = true;
} else if (this.streamedPartialAssistantItemId === undefined) {
this.streamedPartialAssistantItemReplaceable = !knownFinalAnswer;
}
this.streamedPartialAssistantItemId = itemId;
const replaceable = this.streamedPartialAssistantItemReplaceable;
const replacement = replace && replaceable;
if (replacement) {
this.visible.clear();
}
this.visible.add(itemId);
let occurrence = this.occurrences.get(itemId);
if (!occurrence) {
occurrence = { ids: [], ownedCount: 0 };
this.occurrences.set(itemId, occurrence);
}
if (!occurrence.current) {
occurrence.current = randomUUID();
occurrence.ids.push(occurrence.current);
}
return { occurrenceId: occurrence.current, replacement, replaceable };
}
clearProvisionalStream(itemId: string) {
if (
this.streamedPartialAssistantItemId === itemId &&
this.streamedPartialAssistantItemReplaceable
) {
this.streamedPartialAssistantItemId = undefined;
const occurrence = this.occurrences.get(itemId);
return {
itemId,
occurrenceId: occurrence?.current ?? occurrence?.ids.at(-1),
text: "",
delta: "",
replace: true as const,
};
}
return undefined;
}
collectOccurrenceIds(itemIds: readonly string[]): string[] {
return [...new Set(itemIds.flatMap((itemId) => this.occurrences.get(itemId)?.ids ?? []))];
}
isVisible(itemId: string): boolean {
@@ -71,7 +128,9 @@ export class CodexSteeringAssistantSegments {
itemId: string,
text: string | undefined,
completed: boolean,
): { itemId: string; text: string; split: boolean } | undefined {
):
| { itemId: string; text: string; split: boolean; occurrenceIds: readonly string[] }
| undefined {
if (text === undefined) {
return undefined;
}
@@ -83,8 +142,26 @@ export class CodexSteeringAssistantSegments {
const split = prefix !== undefined || !completed;
const segment = prefix?.nextSegment ?? 0;
const mirrorItemId = split ? `${itemId}:segment:${segment}` : itemId;
this.pending.set(mirrorItemId, { itemId, text, completed, split, segment });
return { itemId: mirrorItemId, text: remainder, split };
const occurrence = this.occurrences.get(itemId);
const unownedIds = occurrence?.ids.slice(occurrence.ownedCount) ?? [];
const occurrenceIds = unownedIds.length
? unownedIds
: (this.receiptIds.get(mirrorItemId) ?? []);
this.receiptIds.set(mirrorItemId, occurrenceIds);
this.pending.set(mirrorItemId, {
itemId,
text,
completed,
split,
segment,
occurrenceCount: occurrence?.ids.length ?? 0,
});
// The commit awaits I/O. Bytes arriving after this capture must belong to
// another occurrence even before the captured candidate becomes durable.
if (occurrence) {
delete occurrence.current;
}
return { itemId: mirrorItemId, text: remainder, split, occurrenceIds };
}
consume(mirrorItemId: string): { itemId: string; completed: boolean } | undefined {
@@ -93,6 +170,10 @@ export class CodexSteeringAssistantSegments {
return undefined;
}
this.pending.delete(mirrorItemId);
const occurrence = this.occurrences.get(prefix.itemId);
if (occurrence) {
occurrence.ownedCount = Math.max(occurrence.ownedCount, prefix.occurrenceCount);
}
if (prefix.split) {
// Each committed candidate owns its captured bytes even if a later write fails.
this.persisted.set(prefix.itemId, {
@@ -113,6 +194,10 @@ export class CodexSteeringAssistantSegments {
if (persisted) {
this.persisted.set(itemId, persisted);
}
const occurrence = this.occurrences.get(sourceId);
if (occurrence) {
this.occurrences.set(itemId, occurrence);
}
if (pending) {
pending.itemId = itemId;
}
extensions/codex/src/app-server/event-projector.assistant.test.ts
+21 -4
View File
@@ -169,8 +169,14 @@ describe("CodexAppServerEventProjector assistant projection", () => {
.map(([event]) => event)
.filter((event) => event.stream === "assistant"),
).toEqual([
{ stream: "assistant", data: { itemId: "msg-1", text: "hel", delta: "hel" } },
{ stream: "assistant", data: { itemId: "msg-1", text: "hello", delta: "lo" } },
{
stream: "assistant",
data: { itemId: "msg-1", text: "hel", delta: "hel", occurrenceId: expect.any(String) },
},
{
stream: "assistant",
data: { itemId: "msg-1", text: "hello", delta: "lo", occurrenceId: expect.any(String) },
},
]);
expect(result.assistantTexts).toEqual(["hello"]);
expect(result.messagesSnapshot.map((message) => message.role)).toEqual(["user", "assistant"]);
@@ -337,8 +343,19 @@ describe("CodexAppServerEventProjector assistant projection", () => {
.map((call) => call[0])
.filter((event) => event.stream === "assistant"),
).toEqual([
{ stream: "assistant", data: { itemId: "answer-1", text: summary, delta: summary } },
{ stream: "assistant", data: { itemId: "answer-2", text: coda, delta: coda } },
{
stream: "assistant",
data: {
itemId: "answer-1",
text: summary,
delta: summary,
occurrenceId: expect.any(String),
},
},
{
stream: "assistant",
data: { itemId: "answer-2", text: coda, delta: coda, occurrenceId: expect.any(String) },
},
]);
expect(result.assistantTexts).toEqual([summary, coda]);
expect(result.lastAssistant?.content).toEqual([
extensions/codex/src/app-server/event-projector.commentary.test.ts
+111 -76
View File
@@ -155,86 +155,121 @@ describe("CodexAppServerEventProjector commentary projection", () => {
).toBeUndefined();
});
it("streams commentary with a distinct completion even when the last delta has the same text", async () => {
const onAgentEvent = vi.fn();
const onPartialReply = vi.fn();
const commentaryText = [
"Checking the app-server stream",
"",
"| Intent | Command |",
"| --- | --- |",
"| Session only | `/model opus` |",
"",
"```text",
"BEFORE /model opus → persistent",
"AFTER /model opus → session only",
"```",
].join("\n");
const projector = await createProjector({
...(await createParams()),
onAgentEvent,
onPartialReply,
});
it.each(["started", "completed"])(
"hands off formatted commentary identified at item/%s",
async (phaseKnownAt) => {
const onAgentEvent = vi.fn();
const onPartialReply = vi.fn();
const commentaryText = [
"Checking the app-server stream",
"",
"| Intent | Command |",
"| --- | --- |",
"| Session only | `/model opus` |",
"",
"```text",
"BEFORE /model opus → persistent",
"AFTER /model opus → session only",
"```",
].join("\n");
const projector = await createProjector({
...(await createParams()),
onAgentEvent,
onPartialReply,
});
await projector.handleNotification(
forCurrentTurn("item/started", {
item: commentaryItem("msg-commentary"),
}),
);
await projector.handleNotification(agentMessageDelta("Checking", "msg-commentary"));
await projector.handleNotification(
agentMessageDelta(commentaryText.slice("Checking".length), "msg-commentary"),
);
// The completion boundary lets channels buffer their first notification;
// text-only dedupe must not erase it after the final identical snapshot.
await projector.handleNotification(
forCurrentTurn("item/completed", {
item: commentaryItem("msg-commentary", commentaryText),
}),
);
await projector.handleNotification(
turnCompleted([
{ type: "agentMessage", id: "msg-final", phase: "final_answer", text: "final answer" },
commentaryItem("msg-commentary", commentaryText),
]),
);
await projector.handleNotification(
forCurrentTurn("item/started", {
item:
phaseKnownAt === "started"
? commentaryItem("msg-commentary")
: { type: "agentMessage", id: "msg-commentary", text: "" },
}),
);
await projector.handleNotification(agentMessageDelta("Checking", "msg-commentary"));
await projector.handleNotification(
agentMessageDelta(commentaryText.slice("Checking".length), "msg-commentary"),
);
if (phaseKnownAt === "completed") {
expect(onAgentEvent.mock.calls.at(-1)?.[0]).toMatchObject({
stream: "assistant",
data: { itemId: "msg-commentary", text: commentaryText },
});
}
// The completion boundary lets channels buffer their first notification;
// text-only dedupe must not erase it after the final identical snapshot.
await projector.handleNotification(
forCurrentTurn("item/completed", {
item: commentaryItem("msg-commentary", commentaryText),
}),
);
if (phaseKnownAt === "completed") {
expect(
onAgentEvent.mock.calls
.map(([event]) => event)
.filter((event) => event.stream === "assistant" || event.stream === "item")
.slice(-2),
).toMatchObject([
{
stream: "assistant",
data: { itemId: "msg-commentary", text: "", delta: "", replace: true },
},
{
stream: "item",
data: { kind: "preamble", itemId: "msg-commentary", progressText: commentaryText },
},
]);
}
await projector.handleNotification(
turnCompleted([
{ type: "agentMessage", id: "msg-final", phase: "final_answer", text: "final answer" },
commentaryItem("msg-commentary", commentaryText),
]),
);
const progressEvents = onAgentEvent.mock.calls
.map((call) => call[0])
.filter((event) => event.stream === "item" && event.data.kind === "preamble");
const progressEvents = onAgentEvent.mock.calls
.map((call) => call[0])
.filter((event) => event.stream === "item" && event.data.kind === "preamble");
expect(onPartialReply).not.toHaveBeenCalled();
const preamble = {
itemId: "msg-commentary",
kind: "preamble",
title: "Preamble",
source: "codex-app-server",
};
expect(progressEvents.map((event) => event.data)).toEqual([
{ ...preamble, phase: "update", progressText: "Checking" },
{ ...preamble, phase: "update", progressText: commentaryText },
{ ...preamble, phase: "end", progressText: commentaryText },
]);
const result = projector.buildResult(buildEmptyToolTelemetry());
expect(result.assistantTexts).toEqual(["final answer"]);
const commentary = result.messagesSnapshot.find(
(message) =>
(message as { openclawStreamFallback?: { itemId?: unknown } }).openclawStreamFallback
?.itemId === "msg-commentary",
);
expect(commentary).toMatchObject({
role: "assistant",
content: [{ type: "text", text: commentaryText }],
openclawStreamFallback: {
replacementText: commentaryText,
source: "segment",
expect(onPartialReply).not.toHaveBeenCalled();
const preamble = {
itemId: "msg-commentary",
},
__openclaw: { mirrorIdentity: `${TURN_ID}:commentary:msg-commentary` },
});
expect((commentary as { phase?: unknown } | undefined)?.phase).toBeUndefined();
});
kind: "preamble",
title: "Preamble",
source: "codex-app-server",
};
expect(progressEvents.map((event) => event.data)).toEqual([
...(phaseKnownAt === "started"
? [
{ ...preamble, phase: "update", progressText: "Checking" },
{ ...preamble, phase: "update", progressText: commentaryText },
]
: []),
{ ...preamble, phase: "end", progressText: commentaryText },
]);
const result = projector.buildResult(buildEmptyToolTelemetry());
expect(result.assistantTexts).toEqual(["final answer"]);
const commentary = result.messagesSnapshot.find(
(message) =>
requireRecord(
requireRecord(message, "assistant snapshot").openclawStreamFallback ?? {},
"commentary fallback",
).itemId === "msg-commentary",
);
expect(commentary).toMatchObject({
role: "assistant",
content: [{ type: "text", text: commentaryText }],
openclawStreamFallback: {
replacementText: commentaryText,
source: "segment",
itemId: "msg-commentary",
},
__openclaw: { mirrorIdentity: `${TURN_ID}:commentary:msg-commentary` },
});
expect(requireRecord(commentary, "commentary message").phase).toBeUndefined();
},
);
it("omits durable commentary when the operator explicitly disables persistence", async () => {
const params = await createParams();
extensions/codex/src/app-server/run-attempt-finalize.ts
+19 -7
View File
@@ -40,6 +40,10 @@ import type { CodexAttemptTurnState } from "./run-attempt-turn-state.js";
import { clearCodexBindingForClient } from "./session-binding.js";
import { captureCodexSettledTurnFinalizationContext } from "./settled-turn-context.js";
import { normalizeCodexTrajectoryError, recordCodexTrajectoryCompletion } from "./trajectory.js";
import {
buildCodexMirrorDedupeIdentity,
buildCodexMirrorIdempotencyKey,
} from "./transcript-mirror-attestation.js";
import { codexTranscriptMirrorRuntime } from "./transcript-mirror.js";
import { readMirrorIdentity } from "./upstream-prompt-provenance.js";
import {
@@ -133,6 +137,16 @@ export async function finalizeCodexAttempt(
? turnRuntime.steeringQueueRef.current?.getAcceptedMessages()
: undefined,
});
const terminalAssistantSource = result.messagesSnapshot.find(
(message) => readMirrorIdentity(message) === `${activeTurnId}:assistant`,
);
const terminalSourceKey =
terminalAssistantSource?.role === "assistant"
? buildCodexMirrorIdempotencyKey(
`codex-app-server:${resourceState.thread.threadId}`,
buildCodexMirrorDedupeIdentity(terminalAssistantSource),
)
: undefined;
const projectedTerminal = attemptTerminal.project(result.terminal);
// Transport loss aborts in-flight work mechanically, but its terminal outcome
// must remain a failure unless the operator explicitly canceled the attempt.
@@ -295,9 +309,7 @@ export async function finalizeCodexAttempt(
for (const message of [
result.lastAssistant,
result.currentAttemptAssistant,
result.messagesSnapshot.find(
(candidate) => readMirrorIdentity(candidate) === `${activeTurnId}:assistant`,
),
terminalAssistantSource,
]) {
if (message?.role === "assistant") {
const providerRefusal = message.diagnostics?.some(
@@ -616,6 +628,7 @@ export async function finalizeCodexAttempt(
});
resourceState.trajectoryEndRecorded = true;
const terminalAssistantText = collectTerminalAssistantText(result);
const terminalAssistantItemId = assistantTranscriptIdempotencyKey ?? terminalSourceKey;
if (
terminalAssistantText &&
(assistantTranscriptIdempotencyKey ||
@@ -627,11 +640,10 @@ export async function finalizeCodexAttempt(
stream: "assistant",
data: {
text: terminalAssistantText,
// The receipt identifies the selected persisted occurrence, which can
// exclude candidates streamed before a native tool or sleep boundary.
...(assistantTranscriptIdempotencyKey
// Source identity survives a receipt wait timeout without claiming durability.
...(terminalAssistantItemId
? {
itemId: assistantTranscriptIdempotencyKey,
itemId: terminalAssistantItemId,
replace: true,
replaceable: true,
}
extensions/codex/src/app-server/run-attempt.settlement.test.ts
+9
View File
@@ -627,6 +627,15 @@ describe("Codex app-server terminal settlement", () => {
expect(result.contextEngineTerminalAnchor).toBeUndefined();
}
if (boundary === "final" && release === "after cutoff") {
const terminalAssistant = onAgentEvent.mock.calls
.map(([event]) => event)
.findLast((event) => event.stream === "assistant");
expect(terminalAssistant?.data).toEqual({
text: "Completed before checkpoint.",
itemId: "codex-app-server:thread-1:turn-1:assistant",
replace: true,
replaceable: true,
});
// Successor I/O and relay retirement must outlive the completed deadline simulation.
vi.useRealTimers();
let nextThreadId = "thread-1";
extensions/codex/src/app-server/transcript-mirror-attestation.ts
+4
View File
@@ -27,6 +27,10 @@ export function buildCodexMirrorDedupeIdentity(message: MirroredAgentMessage): s
return `${message.role}:${createHash("sha256").update(payload).digest("hex").slice(0, 16)}`;
}
export function buildCodexMirrorIdempotencyKey(scope: string, identity: string): string {
return `${scope}:${identity}`;
}
const MIRROR_ORIGIN_META_KEY = "mirrorOrigin" as const;
const MIRROR_SOURCE_FINGERPRINT_META_KEY = "mirrorSourceFingerprint" as const;
const CODEX_APP_SERVER_MIRROR_ORIGIN = "codex-app-server" as const;
extensions/codex/src/app-server/transcript-mirror-write.ts
+76 -49
View File
@@ -23,11 +23,17 @@ import {
attachCodexMirrorAttestation,
attachCodexMirrorRunId,
buildCodexMirrorDedupeIdentity,
buildCodexMirrorIdempotencyKey,
fingerprintCodexMirrorSourceMessage,
isMirroredAgentMessage,
readCodexMirrorSourceFingerprint,
type MirroredAgentMessage,
} from "./transcript-mirror-attestation.js";
import { attachCodexMirrorIdentity, readMirrorIdentity } from "./upstream-prompt-provenance.js";
import {
attachCodexMirrorIdentity,
readMirrorIdentity,
takeCodexAssistantItemIds,
} from "./upstream-prompt-provenance.js";
type MirroredUserMessage = Extract<AgentMessage, { role: "user" }>;
export type MirroredUserMessageReceipt = {
@@ -83,7 +89,9 @@ export async function mirror(params: {
};
}
const candidates = messages.map((message) => {
const candidates = messages.map((source) => {
const message = { ...source };
const assistantItemIds = takeCodexAssistantItemIds(message);
const dedupeIdentity = buildCodexMirrorDedupeIdentity(message);
const sourceFingerprint = fingerprintCodexMirrorSourceMessage(message);
const sourceUserIdempotencyKey =
@@ -94,13 +102,47 @@ export async function mirror(params: {
// the provider mirror identity so retries find the exact logical message.
const idempotencyKey =
sourceUserIdempotencyKey ??
(params.idempotencyScope ? `${params.idempotencyScope}:${dedupeIdentity}` : undefined);
return { dedupeIdentity, idempotencyKey, message, sourceFingerprint };
(params.idempotencyScope
? buildCodexMirrorIdempotencyKey(params.idempotencyScope, dedupeIdentity)
: undefined);
return { dedupeIdentity, idempotencyKey, message, sourceFingerprint, assistantItemIds };
});
const candidateIdempotencyKeys = candidates.flatMap(({ idempotencyKey }) =>
idempotencyKey ? [idempotencyKey] : [],
);
const transcriptTarget = resolveCodexMirrorTranscriptTarget(params);
const publishCommitted = async (update: {
lifecycleRevision?: string;
messageId: string;
message: AgentMessage;
messageSeq?: number;
assistantItemIds?: readonly string[];
}) => {
try {
// Commentary and tool rows share the turn but cannot claim terminal ownership.
const terminalOwner = params.terminalAssistantOwner;
const terminalRunId =
update.message.role === "assistant" &&
terminalOwner &&
readMirrorIdentity(update.message) === terminalOwner.mirrorIdentity
? terminalOwner.runId
: undefined;
await publishSessionTranscriptUpdateByIdentity({
...transcriptTarget,
update: {
...update,
...(params.agentId ? { agentId: params.agentId } : {}),
...(terminalRunId ? { runId: terminalRunId } : {}),
sessionKey: transcriptTarget.sessionKey,
},
});
} catch (error) {
// A failed notification cannot turn a committed row into a retryable write.
embeddedAgentLog.warn("failed to publish codex app-server transcript update", {
error: formatErrorMessage(error),
});
}
};
// A queued terminal must still match its prepared outcome before committing.
// Publication may trigger Stop afterward; that cannot erase a committed receipt.
const assertWritable = composeSessionTranscriptWriteAssertion([
@@ -108,16 +150,10 @@ export async function mirror(params: {
params.assertWriteCurrent,
]);
assertWritable();
const mirrorBatch = await withCodexSessionTranscriptMirrorWriteLock(
const result = await withCodexSessionTranscriptMirrorWriteLock(
{ ...transcriptTarget, config: params.config },
async (transcript) => {
assertWritable();
const nextAppendedUpdates: Array<{
lifecycleRevision?: string;
messageId: string;
message: AgentMessage;
messageSeq?: number;
}> = [];
const nextAssistantMirrorIdentitiesOwned = new Set<string>();
const recordAssistantOwnership = (identity: string) => {
nextAssistantMirrorIdentitiesOwned.add(identity);
@@ -131,7 +167,13 @@ export async function mirror(params: {
});
assertWritable();
const taint = { tainted: false };
for (const { dedupeIdentity, idempotencyKey, message, sourceFingerprint } of candidates) {
for (const {
dedupeIdentity,
idempotencyKey,
message,
sourceFingerprint,
assistantItemIds,
} of candidates) {
const sourceMessage = applyCodexTranscriptTaint(message, taint);
const mirrorIdentity = readMirrorIdentity(message);
const ownsRun = Boolean(
@@ -162,6 +204,18 @@ export async function mirror(params: {
const persistedAnchor = mirrorFacts.anchorsByIdempotencyKey.get(idempotencyKey);
if (persistedMessage && isMirroredAgentMessage(persistedMessage)) {
nextMessagesPresent.push(persistedMessage);
if (
assistantItemIds &&
persistedAnchor &&
readCodexMirrorSourceFingerprint(persistedMessage) === sourceFingerprint
) {
await publishCommitted({
messageId: persistedAnchor.entryId,
messageSeq: persistedAnchor.activeMessagePosition + 1,
message: persistedMessage,
assistantItemIds,
});
}
if (persistedMessage.role === "user" && persistedAnchor) {
nextUserMessageReceipts.push({
anchor: persistedAnchor,
@@ -273,8 +327,8 @@ export async function mirror(params: {
if (appended && message.role === "assistant") {
recordAssistantOwnership(dedupeIdentity);
}
params.assertCurrent?.();
if (!appended) {
params.assertCurrent?.();
continue;
}
const { messageId, message: appendedMessage } = appended;
@@ -294,14 +348,21 @@ export async function mirror(params: {
message: appendedMessage,
});
}
if (appended.appended) {
nextAppendedUpdates.push({
const committedItemIds =
isMirroredAgentMessage(appendedMessage) &&
readCodexMirrorSourceFingerprint(appendedMessage) === sourceFingerprint
? assistantItemIds
: undefined;
if (appended.appended || committedItemIds) {
await publishCommitted({
lifecycleRevision,
messageId,
message: appendedMessage,
...(messageSeq !== undefined ? { messageSeq } : {}),
...(committedItemIds ? { assistantItemIds: committedItemIds } : {}),
});
}
params.assertCurrent?.();
if (idempotencyKey) {
mirrorFacts.existingIdempotencyKeys.add(idempotencyKey);
if (appended.anchor) {
@@ -310,7 +371,6 @@ export async function mirror(params: {
}
}
return {
appendedUpdates: nextAppendedUpdates,
assistantMirrorIdentitiesOwned: [...nextAssistantMirrorIdentitiesOwned],
anchorsByMirrorIdentity: nextAnchorsByMirrorIdentity,
messagesPresent: nextMessagesPresent,
@@ -319,39 +379,6 @@ export async function mirror(params: {
},
);
params.assertCurrent?.();
const { appendedUpdates, ...result } = mirrorBatch;
for (const update of appendedUpdates) {
try {
// Commentary and tool rows share the Codex turn but cannot claim terminal run ownership.
const terminalOwner = params.terminalAssistantOwner;
const terminalRunId =
update.message.role === "assistant" &&
terminalOwner &&
readMirrorIdentity(update.message) === terminalOwner.mirrorIdentity
? terminalOwner.runId
: undefined;
await publishSessionTranscriptUpdateByIdentity({
...transcriptTarget,
update: {
lifecycleRevision: update.lifecycleRevision,
...(params.agentId ? { agentId: params.agentId } : {}),
message: update.message,
messageId: update.messageId,
...(update.messageSeq !== undefined ? { messageSeq: update.messageSeq } : {}),
...(terminalRunId ? { runId: terminalRunId } : {}),
sessionKey: transcriptTarget.sessionKey,
},
});
} catch (error) {
// The transcript append is already committed. A transient live-update
// failure must not make dispatch append a second assistant message.
embeddedAgentLog.warn("failed to publish codex app-server transcript update", {
error: formatErrorMessage(error),
});
}
}
return result;
}
extensions/codex/src/app-server/transcript-mirror.test.ts
+141 -3
View File
@@ -42,7 +42,10 @@ import {
readMirrorMessages,
readMirrorRaw,
} from "./transcript-mirror.test-harness.js";
import { attachCodexMirrorIdentity } from "./upstream-prompt-provenance.js";
import {
attachCodexAssistantItemIds,
attachCodexMirrorIdentity,
} from "./upstream-prompt-provenance.js";
import { buildCodexUserPromptMessage } from "./user-prompt-message.js";
const mirrorCodexAppServerTranscript = codexTranscriptMirrorRuntime.mirror;
@@ -933,7 +936,11 @@ describe("mirrorCodexAppServerTranscript", () => {
messages: [staleMessage],
idempotencyScope: "codex-app-server:thread-1",
});
const currentMessage = mirroredAssistant("current answer", "turn-1:assistant", Date.now() + 1);
publishSessionTranscriptUpdateByIdentityMock.mockClear();
const currentMessage = attachCodexAssistantItemIds(
mirroredAssistant("current answer", "turn-1:assistant", Date.now() + 1),
["current-item"],
);
const mirrorOutcome = await mirrorTranscriptBestEffort({
params: {
@@ -955,6 +962,7 @@ describe("mirrorCodexAppServerTranscript", () => {
expect(mirrorOutcome.assistantTranscriptOwned).toBe(false);
expect(mirrorOutcome.mirroredMessages).toEqual([]);
expect(publishSessionTranscriptUpdateByIdentityMock).not.toHaveBeenCalled();
});
it("attests the exact persisted payload after a message-write hook transforms it", async () => {
@@ -972,7 +980,10 @@ describe("mirrorCodexAppServerTranscript", () => {
]),
);
const target = await createSqliteMirrorTarget("openclaw-codex-mirror-attested-hook-");
const sourceMessage = mirroredAssistant("sensitive answer", "turn-1:assistant", Date.now());
const sourceMessage = attachCodexAssistantItemIds(
mirroredAssistant("sensitive answer", "turn-1:assistant", Date.now()),
["rewritten-item"],
);
const mirrorOutcome = await mirrorTranscriptBestEffort({
params: {
@@ -1000,6 +1011,11 @@ describe("mirrorCodexAppServerTranscript", () => {
{ role: "assistant", content: [{ type: "text", text: "[redacted by hook]" }] },
]);
expect(JSON.stringify(mirrorOutcome.mirroredMessages)).not.toContain("sensitive answer");
expect(publishSessionTranscriptUpdateByIdentityMock).toHaveBeenCalledWith(
expect.objectContaining({
update: expect.objectContaining({ assistantItemIds: ["rewritten-item"] }),
}),
);
});
it("returns the final mirrored row as the terminal anchor", async () => {
@@ -1057,6 +1073,127 @@ describe("mirrorCodexAppServerTranscript", () => {
describe("projected transcript persistence", () => {
registerCodexEventProjectorTestLifecycle();
it.each([false, true])(
"freezes receipt occurrences before continuation arrives (retry=%s)",
async (retry) => {
const target = await createSqliteMirrorTarget("openclaw-codex-captured-occurrence-");
const onAgentEvent = vi.fn<NonNullable<EmbeddedRunAttemptParams["onAgentEvent"]>>();
const onPartialReply = vi.fn<NonNullable<EmbeddedRunAttemptParams["onPartialReply"]>>();
const params: EmbeddedRunAttemptParams = {
...(await createProjectorParams()),
...target,
sessionTarget: target,
workspaceDir: path.dirname(target.storePath),
suppressNextUserMessagePersistence: true,
onAgentEvent,
onPartialReply,
};
const projector = new CodexAppServerEventProjector(params, "thread-1", "turn-1");
const item = { type: "agentMessage", id: "answer", phase: "final_answer", text: "" };
const delta = (text: string) =>
projector.handleNotification(
forCurrentTurn("item/agentMessage/delta", { itemId: item.id, delta: text }),
);
const events = () =>
onAgentEvent.mock.calls.flatMap(([event]) =>
event.stream === "assistant" ? [event.data] : [],
);
const mirrorParams = {
...target,
idempotencyScope: "codex-app-server:thread-1",
runId: params.runId,
runMirrorIdentityPrefix: "turn-1:",
onAssistantMessageOwned: (identity: string) =>
projector.markSteeringTranscriptMessagePersisted(identity),
};
await projector.handleNotification(forCurrentTurn("item/started", { item }));
await delta("A");
let captured = projector.buildSteeringTranscriptPrefix();
if (retry) {
await expect(
mirrorCodexAppServerTranscript({
...mirrorParams,
messages: captured,
assertCurrent: () => {
throw new Error("write owner unavailable");
},
}),
).rejects.toThrow("write owner unavailable");
}
// Capture precedes the awaited append; the receipt must not acquire these bytes.
await delta("B");
const [first, second] = events().map((event) => event.occurrenceId);
expect(first).toEqual(expect.any(String));
expect(second).toEqual(expect.any(String));
expect(second).not.toBe(first);
if (retry) {
captured = projector.buildSteeringTranscriptPrefix();
}
const capturedIds = retry ? [first, second] : [first];
await mirrorCodexAppServerTranscript({ ...mirrorParams, messages: captured });
expect(publishSessionTranscriptUpdateByIdentityMock).toHaveBeenLastCalledWith(
expect.objectContaining({
update: expect.objectContaining({ assistantItemIds: capturedIds }),
}),
);
publishSessionTranscriptUpdateByIdentityMock.mockClear();
await mirrorCodexAppServerTranscript({ ...mirrorParams, messages: captured });
expect(publishSessionTranscriptUpdateByIdentityMock).toHaveBeenCalledOnce();
expect(publishSessionTranscriptUpdateByIdentityMock).toHaveBeenLastCalledWith(
expect.objectContaining({
update: expect.objectContaining({ assistantItemIds: capturedIds }),
}),
);
await delta("C");
if (retry) {
expect(events()[2]?.occurrenceId).not.toBe(first);
expect(events()[2]?.occurrenceId).not.toBe(second);
} else {
expect(events()[2]?.occurrenceId).toBe(second);
}
expect(
events().map(({ itemId, text, delta: chunk }) => ({ itemId, text, delta: chunk })),
).toEqual([
{ itemId: "answer", text: "A", delta: "A" },
{ itemId: "answer", text: "AB", delta: "B" },
{ itemId: "answer", text: "ABC", delta: "C" },
]);
expect(onPartialReply.mock.calls.map(([payload]) => payload)).toEqual([
{ text: "A", delta: "A" },
{ text: "AB", delta: "B" },
{ text: "ABC", delta: "C" },
]);
await projector.handleNotification(turnCompleted([{ ...item, text: "ABC" }]));
await mirrorTranscriptBestEffort({
params,
result: projector.buildResult(buildEmptyToolTelemetry()),
agentId: target.agentId,
sessionKey: target.sessionKey,
notifyUserMessagePersisted: () => undefined,
cwd: params.workspaceDir,
threadId: "thread-1",
turnId: "turn-1",
});
expect(publishSessionTranscriptUpdateByIdentityMock).toHaveBeenLastCalledWith(
expect.objectContaining({
update: expect.objectContaining({
assistantItemIds: [...new Set(events().map((event) => event.occurrenceId))],
message: expect.objectContaining({
content: [{ type: "text", text: retry ? "C" : "BC" }],
}),
}),
}),
);
expect(await readMirrorMessages(target)).toEqual([
{ role: "assistant", text: retry ? "AB" : "A" },
{ role: "assistant", text: retry ? "C" : "BC" },
]);
const durable = await readMirrorRaw(target);
expect(durable).not.toContain("assistantItemIds");
expect(durable).not.toContain("occurrenceId");
},
);
it("retains each committed steering cutoff when a later mirror assertion fails", async () => {
const target = await createSqliteMirrorTarget("openclaw-codex-steering-partial-mirror-");
const params: EmbeddedRunAttemptParams = {
@@ -1500,6 +1637,7 @@ describe("deliverAsyncMessageBestEffort", () => {
"async-blocked",
"Blocked update.",
);
delivery.message = attachCodexAssistantItemIds(delivery.message, ["blocked-item"]);
await expect(deliverAsyncMessageBestEffort(delivery)).resolves.toBe("settled");
expect(onBlockReply).not.toHaveBeenCalled();
extensions/codex/src/app-server/transcript-mirror.user-idempotency.test.ts
+74 -8
View File
@@ -23,7 +23,11 @@ import {
fingerprintCodexMirrorSourceMessage,
} from "./transcript-mirror-attestation.js";
import { codexTranscriptMirrorRuntime } from "./transcript-mirror.js";
import { attachCodexMirrorIdentity } from "./upstream-prompt-provenance.js";
import { readMirrorMessages } from "./transcript-mirror.test-harness.js";
import {
attachCodexAssistantItemIds,
attachCodexMirrorIdentity,
} from "./upstream-prompt-provenance.js";
const sessionDirs = useSessionStoreTempDirs(afterAll, "codex-mirror-user-race-");
@@ -31,6 +35,7 @@ const transcriptRace = vi.hoisted(() => ({
competingMessage: undefined as unknown,
lookups: [] as Array<string | undefined>,
publish: vi.fn(),
beforeAppend: vi.fn<() => Promise<void>>().mockResolvedValue(undefined),
userAnchor: undefined as TranscriptEntryAnchor | undefined,
}));
@@ -54,21 +59,21 @@ vi.mock("openclaw/plugin-sdk/codex-session-transcript-runtime", async (importOri
) =>
await actual.withCodexSessionTranscriptMirrorWriteLock(params, async (locked) => {
const competingMessage = transcriptRace.competingMessage;
if (!competingMessage) {
return await run(locked);
}
transcriptRace.competingMessage = undefined;
const intercepted: CodexSessionTranscriptMirrorWriteLockContext = {
...locked,
readMessageFacts: async (factParams) => {
const staleFacts = await locked.readMessageFacts(factParams);
await locked.appendMessage({
message: competingMessage as AgentMessage,
idempotencyLookup: "scan",
});
if (competingMessage) {
await locked.appendMessage({
message: competingMessage as AgentMessage,
idempotencyLookup: "scan",
});
}
return staleFacts;
},
appendMessageWithMessageSequence: async (options) => {
await transcriptRace.beforeAppend();
transcriptRace.lookups.push(options.idempotencyLookup);
const result = await locked.appendMessageWithMessageSequence(options);
const appended = result.result;
@@ -89,9 +94,70 @@ afterEach(() => {
transcriptRace.competingMessage = undefined;
transcriptRace.lookups.length = 0;
transcriptRace.publish.mockReset();
transcriptRace.beforeAppend.mockReset().mockResolvedValue(undefined);
transcriptRace.userAnchor = undefined;
});
it("publishes each committed assistant before a later mirror append can stall", async () => {
const transcriptRuntime = await vi.importActual<
typeof import("openclaw/plugin-sdk/session-transcript-runtime")
>("openclaw/plugin-sdk/session-transcript-runtime");
// Publication must resolve its target without reentering the held writer queue.
transcriptRace.publish.mockImplementation(
transcriptRuntime.publishSessionTranscriptUpdateByIdentity,
);
const target = {
agentId: "main",
sessionId: "commit-publication",
sessionKey: "agent:main:commit-publication",
storePath: path.join(sessionDirs.make(), "openclaw-agent.sqlite"),
};
await upsertSessionEntry({
...target,
entry: { sessionId: target.sessionId, updatedAt: 1 },
});
const reachedSecond = Promise.withResolvers<void>();
const releaseSecond = Promise.withResolvers<void>();
transcriptRace.beforeAppend.mockResolvedValueOnce(undefined).mockImplementationOnce(async () => {
reachedSecond.resolve();
await releaseSecond.promise;
});
const mirror = codexTranscriptMirrorRuntime.mirror({
...target,
runId: "run-1",
idempotencyScope: "codex-app-server:thread-1",
messages: ["first", "second"].map((itemId, index) =>
attachCodexAssistantItemIds(
attachCodexMirrorIdentity(
makeAgentAssistantMessage({
content: [{ type: "text", text: itemId }],
timestamp: index,
}),
`turn-1:assistant:${itemId}`,
),
[itemId],
),
),
});
try {
await reachedSecond.promise;
expect(await readMirrorMessages(target)).toEqual([{ role: "assistant", text: "first" }]);
expect(transcriptRace.publish).toHaveBeenCalledOnce();
expect(transcriptRace.publish).toHaveBeenCalledWith(
expect.objectContaining({
update: expect.objectContaining({ assistantItemIds: ["first"], messageSeq: 1 }),
}),
);
} finally {
releaseSecond.resolve();
await mirror;
}
expect(await readMirrorMessages(target)).toEqual([
{ role: "assistant", text: "first" },
{ role: "assistant", text: "second" },
]);
});
it("adopts a competing indexed user without duplicating writes or slowing assistant mirrors", async () => {
const root = sessionDirs.make();
const target = {
extensions/codex/src/app-server/upstream-prompt-provenance.ts
+18
View File
@@ -13,6 +13,24 @@ function readCodexMeta(message: AgentMessage | undefined) {
: undefined;
}
const ASSISTANT_ITEM_IDS = Symbol("codexAssistantItemIds");
/** Follow snapshot copies without adding provider identities to durable messages. */
export function attachCodexAssistantItemIds<T extends AgentMessage>(
message: T,
itemIds: readonly string[],
): T {
return { ...message, [ASSISTANT_ITEM_IDS]: itemIds };
}
export function takeCodexAssistantItemIds(
message: AgentMessage & { [ASSISTANT_ITEM_IDS]?: readonly string[] },
): readonly string[] | undefined {
const itemIds = message[ASSISTANT_ITEM_IDS];
delete message[ASSISTANT_ITEM_IDS];
return itemIds;
}
function attachCodexMeta<T extends AgentMessage>(message: T, key: string, value: string): T {
return {
...message,
packages/gateway-protocol/src/schema/logs-chat.ts
-2
View File
@@ -438,8 +438,6 @@ export const ChatDeltaEventSchema = closedObject({
state: Type.Literal("delta"),
message: Type.Optional(Type.Unknown()),
deltaText: Type.String(),
itemId: Type.Optional(Type.String()),
itemStartOffset: Type.Optional(Type.Integer({ minimum: 0 })),
replace: Type.Optional(Type.Boolean()),
usage: Type.Optional(Type.Unknown()),
});
packages/gateway-protocol/src/schema/worker-admission.ts
+4 -1
View File
@@ -245,7 +245,10 @@ export const WorkerRuntimeContextMessageSchema = Type.Union([
export const WorkerTranscriptMessageSchema = Type.Union([
WorkerTranscriptUserMessageSchema,
WorkerRuntimeContextMessageSchema,
transcriptSchemas.contextAssistant,
closedObject({
...transcriptSchemas.contextAssistant.properties,
itemId: Type.Optional(WorkerIdentifierSchema),
}),
transcriptSchemas.toolResult,
]);
packages/sdk/src/index.e2e.test.ts
+24 -12
View File
@@ -10,7 +10,7 @@ import {
registerAgentRunContext,
} from "../../../src/infra/agent-run-registry.js";
import { withTimeout } from "../../../src/utils/with-timeout.js";
import { GatewayClientTransport, OpenClaw } from "./index.js";
import { GatewayClientTransport, OpenClaw, type OpenClawEvent } from "./index.js";
type JsonObject = Record<string, unknown>;
type FakeGatewayRequest = {
@@ -522,16 +522,14 @@ describe("OpenClaw SDK real Gateway e2e", () => {
const run = await oc.runs.get(runId);
const eventsPromise = (async () => {
const seen: string[] = [];
const sessionKeys: Array<string | undefined> = [];
const seen: OpenClawEvent[] = [];
for await (const event of run.events()) {
seen.push(event.type);
sessionKeys.push(event.sessionKey);
seen.push(event);
if (event.type === "run.completed") {
break;
}
}
return { seen, sessionKeys };
return seen;
})();
emitAgentEvent({
@@ -550,14 +548,28 @@ describe("OpenClaw SDK real Gateway e2e", () => {
data: { phase: "end", endedAt: 222 },
});
const { seen, sessionKeys } = await withTimeout(eventsPromise, 2_000, {
const seen = await withTimeout(eventsPromise, 2_000, {
message: "timed out waiting for real Gateway SDK events",
});
expect(seen).toEqual(["run.started", "assistant.delta", "run.completed"]);
expect(sessionKeys).toEqual([
"agent:main:dashboard:sdk-real-gateway",
"agent:main:dashboard:sdk-real-gateway",
"agent:main:dashboard:sdk-real-gateway",
expect(seen).toMatchObject([
{
type: "run.started",
sessionKey: "agent:main:dashboard:sdk-real-gateway",
data: { phase: "start" },
raw: { event: "agent" },
},
{
type: "assistant.delta",
sessionKey: "agent:main:dashboard:sdk-real-gateway",
data: { text: "hello from real gateway", delta: "hello from real gateway" },
raw: { event: "agent" },
},
{
type: "run.completed",
sessionKey: "agent:main:dashboard:sdk-real-gateway",
data: { phase: "end" },
raw: { event: "agent" },
},
]);
registerAgentRunContext(replayRunId, {
src/agents/cli-runner/assistant-identity.ts
+4
View File
@@ -0,0 +1,4 @@
/** Recovery attempts and the final correction share the run's cumulative transcript row. */
export function cliAssistantItemId(runId: string): string {
return `cli-assistant:${runId}`;
}
src/agents/cli-runner/cli-run-transcript.test.ts
+1
View File
@@ -132,6 +132,7 @@ it.each([
yielded,
});
expect(result.owned).toBe(true);
expect(result.idempotencyKey).toBe("cli-assistant:cli-media-run");
expect(prepareAssistantTranscriptMessage).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({ content: [{ type: "text", text: sourceText }] }),
sourceText,
src/agents/cli-runner/cli-run-transcript.ts
+2 -1
View File
@@ -45,6 +45,7 @@ import type { AgentMessage } from "../runtime/index.js";
import { withSessionManagerWrite } from "../sessions/session-manager-write-admission.js";
import { SessionManager } from "../sessions/session-manager.js";
import { buildAssistantMessage, buildUsageWithNoCost } from "../stream-message-shared.js";
import { cliAssistantItemId } from "./assistant-identity.js";
import type { PreparedCliRunContext, RunCliAgentParams } from "./types.js";
const log = createSubsystemLogger("agents/cli-runner");
@@ -154,7 +155,7 @@ export async function persistCliAssistantTranscript(params: {
return { owned: false };
}
try {
const idempotencyKey = `cli-assistant:${runParams.runId}`;
const idempotencyKey = cliAssistantItemId(runParams.runId);
const result = await appendExactAssistantMessageToSessionTranscript({
sessionKey: runParams.sessionKey,
agentId: runParams.agentId,
src/agents/cli-runner/execute-events.ts
+2
View File
@@ -24,6 +24,7 @@ import {
import { runAgentHarnessAfterToolCallHook } from "../harness/hook-helpers.js";
import { applyPluginTextReplacements } from "../plugin-text-transforms.js";
import { resolveCliToolTerminalReason } from "../run-termination.js";
import { cliAssistantItemId } from "./assistant-identity.js";
import type { CliToolTracking } from "./execute-tool-tracking.js";
import { normalizeCliToolName, stripOpenClawMcpToolPrefix } from "./tool-policy.js";
import type { PreparedCliRunContext } from "./types.js";
@@ -398,6 +399,7 @@ export function createCliEventHandlers(params: {
signalExecutionPhase("assistant_output_started");
}
emitLiveEvent("assistant", () => ({
itemId: cliAssistantItemId(runParams.runId),
text: applyPluginTextReplacements(text, context.backendResolved.textTransforms?.output),
delta: applyPluginTextReplacements(delta, context.backendResolved.textTransforms?.output),
}));
src/agents/command/acp-execution.ts
+1
View File
@@ -277,6 +277,7 @@ export async function runAcpAgentCommand(params: {
: undefined;
params.trackInternalModelRunTarget(internalTarget);
const transcriptResult = await attemptExecutionRuntime.persistAcpTurnTranscript({
assistantIdempotencyKey: params.runId,
body: params.body,
transcriptBody: params.transcriptBody,
inputProvenance: params.opts.inputProvenance,
src/agents/command/acp-lifecycle.ts
+1
View File
@@ -440,6 +440,7 @@ export function emitAcpAssistantDelta(params: { runId: string; text: string; del
runId: params.runId,
stream: "assistant",
data: {
itemId: params.runId,
text: params.text,
delta: params.delta,
},
src/agents/command/transcript-persistence.ts
+1 -1
View File
@@ -216,7 +216,7 @@ async function persistTextTurnTranscript(
runId: params.runId,
publishWhen: "always",
touchSessionEntry: true,
updateMode: "file-only",
updateMode: params.runId ? "inline" : "file-only",
expectedSessionId:
params.expectedSessionId ??
(params.sessionStore && params.storePath ? params.sessionId : undefined),
src/agents/embedded-agent-subscribe.commentary-display.test.ts
+292
View File
@@ -0,0 +1,292 @@
import type { AssistantMessage } from "openclaw/plugin-sdk/llm";
import { describe, expect, it, onTestFinished, vi } from "vitest";
import { createDeferred } from "../../test/helpers/promise.js";
import { projectInFlightRunSnapshot } from "../gateway/chat-inflight-snapshot.js";
import { createAgentEventTestHarness } from "../gateway/server-chat.agent-events.test-harness.js";
import { subscribeAgentEvents } from "../gateway/server-chat.agent-events.test-helpers.js";
import { createSubscribedSessionHarness } from "./embedded-agent-subscribe.e2e-harness.js";
import { createOpenAiResponsesTextBlock } from "./embedded-agent-subscribe.openai-responses.test-helpers.js";
import { textAssistant } from "./test-helpers/sparse-transcript.test-support.js";
type Options = Omit<Parameters<typeof createSubscribedSessionHarness>[0], "runId">;
type TestAssistant = Pick<AssistantMessage, "role" | "content"> &
Partial<Pick<AssistantMessage, "api" | "stopReason" | "openclawDelivery">> & {
phase?: "commentary" | "final_answer";
};
function commentaryHarness(options: Options) {
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"] });
vi.setSystemTime(new Date("2026-05-09T00:00:00.000Z"));
const gateway = createAgentEventTestHarness();
gateway.register("run", "main", "run");
const unsubscribe = subscribeAgentEvents((event) => {
if (event.runId === "run") {
return gateway.handler(event);
}
});
const finishing = createDeferred();
const finish = createDeferred();
const source = createSubscribedSessionHarness({
runId: "run",
...options,
onBeforeLifecycleTerminal: () => {
finishing.resolve();
return finish.promise;
},
});
onTestFinished(async () => {
finish.resolve();
await source.subscription.waitForPendingEvents();
source.subscription.unsubscribe();
await unsubscribe();
await gateway.handler.dispose();
gateway.chatRunState.clear();
vi.useRealTimers();
});
return {
...source,
broadcast: gateway.broadcast,
finishing: finishing.promise,
drain: unsubscribe.drain,
snapshot: async () => {
await unsubscribe.drain();
gateway.chatRunState.flushPendingText("run");
return projectInFlightRunSnapshot({ chatRunState: gateway.chatRunState, runId: "run" }).text;
},
};
}
function update(
emit: ReturnType<typeof createSubscribedSessionHarness>["emit"],
message: TestAssistant,
event: {
type: string;
contentIndex?: number;
delta?: string;
content?: string;
partial?: TestAssistant;
},
) {
emit({ type: "message_update", message, assistantMessageEvent: event });
}
function textMessage(
text: string,
api: "anthropic-messages" | "openai-completions",
): TestAssistant {
return { ...textAssistant(text), api };
}
function textBlock(text: string, id: string, phase: "commentary" | "final_answer") {
return { ...createOpenAiResponsesTextBlock({ text, id, phase }), type: "text" as const };
}
function toolBlock(id: string, name: string) {
return { type: "toolCall" as const, id, name, arguments: {} };
}
describe("commentary preambles", () => {
it.each([{ api: "anthropic-messages" }, { api: "openai-completions" }] satisfies Array<{
api: "anthropic-messages" | "openai-completions";
}>)(
"transfers the current $api display to formatted commentary while preserving raw output",
async ({ api }) => {
const onAgentEvent = vi.fn();
const { emit, subscription, snapshot, drain } = commentaryHarness({ onAgentEvent });
const narration = "Checking the workspace.\n\n```sh\n pwd\n```";
emit({ type: "message_start", message: textMessage("", api) });
update(emit, textMessage(narration, api), { type: "text_delta", delta: narration });
await subscription.waitForPendingEvents();
await drain();
const initial = onAgentEvent.mock.calls.find(([event]) => event.stream === "assistant")?.[0];
expect(initial?.data.text).toContain("Checking the workspace.");
expect(await snapshot()).toContain("Checking the workspace.");
const commentary: TestAssistant = {
role: "assistant",
api,
stopReason: "toolUse",
content: [textBlock(narration, "commentary-0", "commentary"), toolBlock("exec_0", "exec")],
};
update(emit, commentary, { type: "toolcall_start", contentIndex: 1, partial: commentary });
emit({ type: "message_end", message: commentary });
await subscription.waitForPendingEvents();
await drain();
const events = onAgentEvent.mock.calls.map(([event]) => event);
expect(events).toMatchObject([
{
stream: "assistant",
data: {
itemId: initial.data.itemId,
text: expect.stringContaining("Checking the workspace."),
},
},
{
stream: "item",
data: {
kind: "preamble",
itemId: "commentary-0",
phase: "update",
progressText: narration,
},
},
{
stream: "item",
data: { kind: "preamble", itemId: "commentary-0", phase: "end", progressText: narration },
},
]);
expect(await snapshot()).toBe("");
emit({ type: "message_start", message: textMessage("", api) });
update(emit, textMessage("Next answer.", api), { type: "text_delta", delta: "Next answer." });
await subscription.waitForPendingEvents();
await drain();
const next = onAgentEvent.mock.calls.at(-1)?.[0];
expect(next).toMatchObject({ stream: "assistant", data: { text: "Next answer." } });
expect(next.data.itemId).not.toBe(initial.data.itemId);
expect(await snapshot()).toBe("Next answer.");
},
);
it.each([
{ phaseKnownAt: "text_delta", deferred: false },
{ phaseKnownAt: "text_end", deferred: false },
{ phaseKnownAt: "text_end", deferred: true },
] satisfies Array<{ phaseKnownAt: "text_delta" | "text_end"; deferred: boolean }>)(
"preserves an explicit final-answer block when sibling commentary is identified at $phaseKnownAt (deferred=$deferred)",
async ({ phaseKnownAt, deferred }) => {
const onAgentEvent = vi.fn();
const { emit, subscription, snapshot, finishing, broadcast, drain } = commentaryHarness({
onAgentEvent,
...(deferred ? { onBeforeTerminalDelivery: () => undefined } : {}),
});
const answer: TestAssistant = {
role: "assistant",
api: "openai-responses",
content: [textBlock("Accepted answer.", "answer-0", "final_answer")],
};
emit({ type: "message_start", message: answer });
update(emit, answer, {
type: "text_delta",
contentIndex: 0,
delta: "Accepted answer.",
partial: answer,
});
const mixed: TestAssistant = {
...answer,
content: [...answer.content, textBlock("Checking a detail.", "commentary-0", "commentary")],
};
const partial: TestAssistant =
phaseKnownAt === "text_delta"
? mixed
: {
...answer,
content: [...answer.content, { type: "text", text: "Checking a detail." }],
};
update(emit, partial, {
type: "text_delta",
contentIndex: 1,
delta: "Checking a detail.",
partial,
});
if (phaseKnownAt === "text_end" && !deferred) {
expect(onAgentEvent.mock.calls.at(-1)?.[0]).toMatchObject({
stream: "assistant",
data: { text: expect.stringContaining("Checking a detail.") },
});
expect(await snapshot()).toBe("Accepted answer.\nChecking a detail.");
expect(broadcast.mock.calls.findLast(([event]) => event === "chat")?.[1]).toMatchObject({
message: {
content: [{ type: "text", text: "Accepted answer.\nChecking a detail." }],
},
});
broadcast.mockClear();
}
update(emit, mixed, {
type: "text_end",
contentIndex: 1,
content: "Checking a detail.",
partial: mixed,
});
await subscription.waitForPendingEvents();
await drain();
if (phaseKnownAt === "text_end" && !deferred) {
const correction = broadcast.mock.calls.findIndex(([event]) => event === "chat");
const preamble = broadcast.mock.calls.findIndex(
([event, payload]) => event === "agent" && payload.data?.kind === "preamble",
);
expect(correction).toBeGreaterThanOrEqual(0);
expect(broadcast.mock.calls[correction]?.[1]).toMatchObject({
replace: true,
deltaText: "Accepted answer.",
});
expect(preamble).toBeGreaterThan(correction);
}
const beforeMessageEnd = onAgentEvent.mock.calls.map(([event]) => event);
const current = beforeMessageEnd.findLast((event) => event.stream === "assistant");
if (deferred) {
expect(current).toBeUndefined();
} else {
expect(current?.data.text).toBe(
phaseKnownAt === "text_delta"
? "Accepted answer."
: "Accepted answer.\nChecking a detail.",
);
}
expect(await snapshot()).toBe("Accepted answer.");
const continued =
phaseKnownAt === "text_end" && !deferred
? {
...mixed,
content: [
...mixed.content,
textBlock("Continuing answer.", "answer-1", "final_answer"),
],
}
: mixed;
if (continued !== mixed) {
update(emit, continued, {
type: "text_delta",
contentIndex: 2,
delta: "Continuing answer.",
partial: continued,
});
await subscription.waitForPendingEvents();
await drain();
expect(await snapshot()).toBe("Accepted answer.\nContinuing answer.");
}
emit({ type: "message_end", message: continued });
await subscription.waitForPendingEvents();
await drain();
if (deferred) {
emit({ type: "agent_end", messages: [continued] });
await finishing;
await drain();
}
const events = onAgentEvent.mock.calls.map(([event]) => event);
const assistant = events.filter((event) => event.stream === "assistant");
const expectedAnswer =
continued === mixed ? "Accepted answer." : "Accepted answer.\nContinuing answer.";
expect(assistant.length).toBeGreaterThan(0);
if (phaseKnownAt === "text_delta") {
expect(assistant.every((event) => event.data.text === "Accepted answer.")).toBe(true);
} else {
expect(assistant).toContainEqual(
expect.objectContaining({
data: expect.objectContaining({ text: "Accepted answer.\nChecking a detail." }),
}),
);
expect(assistant).toContainEqual(
expect.objectContaining({
data: expect.objectContaining({ text: expectedAnswer, replace: true }),
}),
);
}
expect(events).toContainEqual(
expect.objectContaining({
stream: "item",
data: expect.objectContaining({
itemId: "commentary-0",
progressText: "Checking a detail.",
}),
}),
);
expect(await snapshot()).toBe(expectedAnswer);
},
);
});
src/agents/embedded-agent-subscribe.handlers.messages.update.ts
+1 -1
View File
@@ -584,7 +584,7 @@ export function handleMessageUpdate(
text: currentSourcePartial.text,
delta: releaseHeldSnapshot ? currentSourcePartial.text : deltaText,
replace: releaseHeldSnapshot || replace || undefined,
phase: deliveryPhase ?? assistantPhase,
phase: deliveryPhase,
},
{ emitPartialReply: !currentSourcePartial.hold },
);
src/agents/embedded-agent-subscribe.handlers.ts
+18
View File
@@ -2,6 +2,10 @@
* Dispatches serialized embedded-agent subscription events to specific handlers.
*/
import { isPromiseLike } from "@openclaw/normalization-core/promise-like";
import {
type AgentAssistantSourceReceipt,
bindAgentAssistantSource,
} from "../infra/agent-events.js";
import {
handleAgentEnd,
handleAgentStart,
@@ -12,6 +16,7 @@ import {
handleMessageStart,
handleMessageEnd,
} from "./embedded-agent-subscribe.handlers.messages.lifecycle.js";
import { isSubscribeTranscriptOnlyOpenClawAssistantMessage } from "./embedded-agent-subscribe.handlers.messages.stream.js";
import { handleMessageUpdate } from "./embedded-agent-subscribe.handlers.messages.update.js";
import { handleToolExecutionEnd } from "./embedded-agent-subscribe.handlers.tools.completion.js";
import { handleToolExecutionUpdate } from "./embedded-agent-subscribe.handlers.tools.progress.js";
@@ -23,6 +28,7 @@ import type { AgentSessionEvent } from "./sessions/index.js";
/** Create the serialized event dispatcher for subscribed embedded-agent sessions. */
export function createEmbeddedAgentSessionEventHandler(ctx: EmbeddedAgentSubscribeContext) {
let assistantSource: AgentAssistantSourceReceipt | undefined;
const scheduleEvent = (evt: AgentSessionEvent, handler: () => unknown): void | Promise<void> => {
// Tool-result delivery must settle before later assistant or terminal events;
// suppression flags would discard those events instead of preserving order.
@@ -60,6 +66,18 @@ export function createEmbeddedAgentSessionEventHandler(ctx: EmbeddedAgentSubscri
};
return (evt: AgentSessionEvent) => {
if (
(evt.type === "message_start" ||
evt.type === "message_update" ||
evt.type === "message_end") &&
evt.message.role === "assistant" &&
!isSubscribeTranscriptOnlyOpenClawAssistantMessage(evt.message)
) {
if (evt.type === "message_start" || !assistantSource) {
assistantSource = {};
}
bindAgentAssistantSource(evt.message, assistantSource);
}
// Model facts advance before persistence, independently of queued reply delivery.
ctx.captureModelEvent(evt);
// Capture tool facts before reply delivery can delay their lifecycle handlers.
src/agents/embedded-agent-subscribe.reply-delivery.ts
+111 -52
View File
@@ -6,7 +6,12 @@ import {
setReplyPayloadMetadata,
} from "../auto-reply/reply-payload.js";
import { isSilentReplyText, SILENT_REPLY_TOKEN } from "../auto-reply/tokens.js";
import { emitAgentEventIfCurrent } from "../infra/agent-events.js";
import {
emitAgentEventWithAssistantSourceIfCurrent,
readAgentAssistantSource,
type AgentAssistantProjection,
type AgentAssistantSourceReceipt,
} from "../infra/agent-events.js";
import { normalizeTextForComparison } from "./embedded-agent-helpers.js";
import type { BlockReplyPayload } from "./embedded-agent-payloads.js";
import { runBestEffortCallback } from "./embedded-agent-subscribe.callback.js";
@@ -26,6 +31,8 @@ import type { SubscribeEmbeddedAgentSessionParams } from "./embedded-agent-subsc
import type { AgentMessage } from "./runtime/index.js";
type AssistantStreamDelivery = {
assistantSource?: AgentAssistantSourceReceipt;
assistantProjection?: AgentAssistantProjection;
data: AssistantStreamData;
eventData?: AssistantStreamData;
emitPartialReply: boolean;
@@ -92,19 +99,56 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
// must stay distinct so a correction cannot overwrite an earlier attempt.
const streamId = randomUUID();
let messageIndex = -1;
let blockIndex = -1;
let assistantItemId = "";
let prefix = "";
let streamedText = "";
const raw = { text: "", prefix: "", blockIndex: -1 };
const display = { ...raw };
let finalText = "";
let finalized = false;
const publishAgentEvent = (event: EmbeddedAgentEvent) => {
if (
!emitAgentEventIfCurrent({
runId: params.runId,
lifecycleGeneration: params.lifecycleGeneration,
...event,
})
) {
const advanceStream = (
stream: typeof raw,
data: AssistantStreamData,
finalMessage: boolean,
): AssistantStreamData | undefined => {
if (stream.blockIndex !== state.assistantMessageIndex) {
stream.prefix = stream.text;
stream.blockIndex = state.assistantMessageIndex;
}
const text = finalMessage
? data.text
: stream.prefix && data.text
? `${stream.prefix}\n${data.text}`
: stream.prefix || data.text;
const replace = finalMessage ? !text.startsWith(stream.text) : data.replace === true;
const delta = finalMessage
? replace
? ""
: text.slice(stream.text.length)
: stream.prefix && stream.text.length === stream.prefix.length && data.delta
? `\n${data.delta}`
: data.delta;
const eventData =
text !== stream.text || data.mediaUrls?.length || data.managedMediaUrls?.length
? { ...data, text, delta, replace: replace || undefined, itemId: assistantItemId }
: undefined;
stream.text = text;
return eventData;
};
const publishAgentEvent = (
event: EmbeddedAgentEvent,
assistantSource: AgentAssistantSourceReceipt | undefined,
assistantProjection?: AgentAssistantProjection,
) => {
const owned = {
runId: params.runId,
lifecycleGeneration: params.lifecycleGeneration,
...event,
};
const emitted = emitAgentEventWithAssistantSourceIfCurrent(
owned,
assistantSource,
assistantProjection,
);
if (!emitted) {
return;
}
if (params.onAgentEvent) {
@@ -125,13 +169,12 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
scope.pending ||=
delivery.emitPartialReply && Boolean(params.onPartialReply) && state.shouldEmitPartialReplies;
const itemId = eventData?.itemId ?? "";
const progressText =
eventData?.phase === "commentary" ? eventData.text.replace(/\s+/g, " ").trim() : "";
const progressText = eventData?.phase === "commentary" ? eventData.text.trimEnd() : "";
const preamblePhase = delivery.finalMessage ? "end" : "update";
// Completion must survive an identical last delta: first-notification
// consumers wait for this boundary, not a timer or a repeated text snapshot.
const commentarySignature = `${preamblePhase}\0${progressText}`;
const event = progressText
const event = progressText.trim()
? {
stream: "item" as const,
data: {
@@ -152,7 +195,7 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
if (event.stream === "item") {
lastEmittedCommentaryByItem.set(itemId, commentarySignature);
}
publishAgentEvent(event);
publishAgentEvent(event, delivery.assistantSource, delivery.assistantProjection);
}
drainPartialReply(scope);
};
@@ -163,63 +206,75 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
if (state.unsubscribed) {
return;
}
const assistantSource = readAgentAssistantSource(state.lastAssistant);
let eventData: AssistantStreamData | undefined;
let assistantProjection: AgentAssistantProjection | undefined;
if (data.phase === "commentary") {
if (messageIndex === state.assistantMessageStartIndex && display.text !== finalText) {
display.prefix = display.text = finalText;
assistantProjection = { itemId: assistantItemId, text: finalText, replace: true };
// HTTP consumers retain append-only raw text. Only the Gateway display
// transfers provisional text to commentary, including deferred frames.
for (const scope of [streamScope, ...deferredAssistantScopes]) {
const delivery = scope.delivery;
if (
delivery &&
delivery.blockIndex >= messageIndex &&
delivery.data.phase !== "final_answer"
) {
delivery.assistantProjection = assistantProjection;
}
}
}
eventData = data;
} else {
if (messageIndex !== state.assistantMessageStartIndex) {
messageIndex = state.assistantMessageStartIndex;
blockIndex = state.assistantMessageIndex;
assistantItemId = `${streamId}:${messageIndex}`;
prefix = streamedText = "";
for (const stream of [raw, display]) {
stream.prefix = stream.text = "";
stream.blockIndex = state.assistantMessageIndex;
}
finalText = "";
finalized = false;
}
if (!finalized || options?.finalMessage) {
if (blockIndex !== state.assistantMessageIndex) {
prefix = streamedText;
blockIndex = state.assistantMessageIndex;
}
const text = options?.finalMessage
? data.text
: prefix && data.text
? `${prefix}\n${data.text}`
: prefix || data.text;
const replace = options?.finalMessage
? !text.startsWith(streamedText)
: data.replace === true;
const delta = options?.finalMessage
? replace
? ""
: text.slice(streamedText.length)
: prefix && streamedText.length === prefix.length && data.delta
? `\n${data.delta}`
: data.delta;
if (text !== streamedText || data.mediaUrls?.length || data.managedMediaUrls?.length) {
eventData = {
...data,
text,
delta,
replace: replace || undefined,
itemId: assistantItemId,
};
}
streamedText = text;
eventData = advanceStream(raw, data, options?.finalMessage === true);
const displayData = advanceStream(display, data, options?.finalMessage === true);
assistantProjection = {
itemId: assistantItemId,
text: display.text,
replace: displayData?.replace === true,
};
finalText =
data.phase === "final_answer"
? display.text
: display.text.startsWith(finalText)
? finalText
: "";
finalized = options?.finalMessage === true;
}
if (options?.finalMessage && state.lastAssistant?.stopReason === "error") {
const itemId = assistantItemId;
const text = streamedText;
const text = raw.text;
const displayText = display.text;
params.assistantErrorTranscript?.bindStream(state.lastAssistant, (visible) => {
clearAssistantStream();
publishAgentEvent({
stream: "assistant",
data: { itemId, text: visible ? text : "", delta: "", replace: true },
});
publishAgentEvent(
{
stream: "assistant",
data: { itemId, text: visible ? text : "", delta: "", replace: true },
},
assistantSource,
{ itemId, text: visible ? displayText : "", replace: true },
);
});
}
}
// Capture both coordinate domains before any callback can advance message state.
const delivery = {
assistantSource,
assistantProjection,
data,
eventData,
emitPartialReply: options?.emitPartialReply === true,
@@ -236,6 +291,7 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
isStreamAppend(previous) &&
isStreamAppend(delivery) &&
previous.blockIndex === delivery.blockIndex &&
previous.assistantSource === delivery.assistantSource &&
previous.data.phase === data.phase &&
previous.data.itemId === data.itemId &&
previous.emitPartialReply === delivery.emitPartialReply &&
@@ -417,6 +473,9 @@ export function createReplyDelivery({ params, state, log }: ReplyDeliveryParams)
if (delivery.eventData) {
delivery.eventData = { ...delivery.eventData, text: "", delta: "" };
}
if (delivery.assistantProjection) {
delivery.assistantProjection = { ...delivery.assistantProjection, text: "" };
}
}
}
}
src/agents/harness/acp-native-execution.process.test.ts
+47 -3
View File
@@ -14,7 +14,10 @@ import {
buildKnownAgentRunFailureReplyPayload,
} from "../../auto-reply/reply/agent-runner-failure-reply.js";
import type { OpenClawConfig } from "../../config/types.openclaw.js";
import { onAgentEventForRun, type AgentEventPayload } from "../../infra/agent-events.js";
import { projectInFlightRunSnapshot } from "../../gateway/chat-inflight-snapshot.js";
import { createAgentEventTestHarness } from "../../gateway/server-chat.agent-events.test-harness.js";
import { subscribeAgentEvents } from "../../gateway/server-chat.agent-events.test-helpers.js";
import type { AgentEventPayload } from "../../infra/agent-events.js";
import { hasErrnoCode } from "../../infra/errno.js";
import { formatErrorMessage } from "../../infra/errors.js";
import { upsertSessionEntry } from "../../plugin-sdk/session-store-runtime.js";
@@ -22,6 +25,10 @@ import {
appendSessionTranscriptMessageByIdentityStrict,
readVisibleSessionTranscriptMessageEntries,
} from "../../plugin-sdk/session-transcript-runtime.js";
import {
onInternalSessionTranscriptUpdate,
readSessionTranscriptRunId,
} from "../../sessions/transcript-events.js";
import { withOpenClawTestState } from "../../test-utils/openclaw-test-state.js";
import {
attemptFor,
@@ -213,7 +220,27 @@ it.for(["complete", "revoke"] as const)(
});
const attempt = await attemptFor(state, config, "opencode", "full");
const updates: AgentEventPayload[] = [];
const unsubscribe = onAgentEventForRun(attempt.input.runId, (event) => {
const gateway = createAgentEventTestHarness();
gateway.register(attempt.input.runId, attempt.target.sessionKey, attempt.input.runId);
const snapshot = () => {
gateway.chatRunState.flushPendingText(attempt.input.runId);
return projectInFlightRunSnapshot({
chatRunState: gateway.chatRunState,
runId: attempt.input.runId,
}).text;
};
const committedSnapshots: string[] = [];
const unsubscribeTranscript = onInternalSessionTranscriptUpdate((event) => {
if (readSessionTranscriptRunId(event.message) === attempt.input.runId) {
gateway.handler.retireTranscript(event);
committedSnapshots.push(snapshot());
}
});
const unsubscribe = subscribeAgentEvents(async (event) => {
if (event.runId !== attempt.input.runId) {
return;
}
await gateway.handler(event);
if (event.stream === "assistant") {
updates.push(event);
}
@@ -234,19 +261,36 @@ it.for(["complete", "revoke"] as const)(
await expect.poll(() => updates.at(-1)?.data.text).toBe("First chunk");
expect(finished).toBe(false);
expect(updates[0]?.sessionKey).toBe(attempt.target.sessionKey);
expect(snapshot()).toBe("First chunk");
if (completion === "revoke") {
attempt.close();
}
await fs.writeFile(path.join(native.peerDirectory, "prompt-reply-release"), "release");
const result = await run;
await unsubscribe.drain();
expect(result.terminal.kind).toBe(completion === "complete" ? "ok" : "failed");
expect(updates.map((event) => event.data.delta)).toEqual(
completion === "complete" ? ["First chunk", " second chunk"] : ["First chunk"],
);
if (completion === "complete") {
const transcript = await readVisibleSessionTranscriptMessageEntries(attempt.target);
const assistant = transcript.find((row) => row.role === "assistant");
expect(assistant?.idempotencyKey).toBe(result.assistantTranscriptIdempotencyKey);
expect(updates.map((event) => event.data.itemId)).toEqual([
assistant?.idempotencyKey,
assistant?.idempotencyKey,
]);
expect(committedSnapshots).toEqual([""]);
} else {
expect(committedSnapshots).toEqual([]);
}
} finally {
await fs.writeFile(path.join(native.peerDirectory, "prompt-reply-release"), "release");
await Promise.allSettled([run]);
unsubscribe();
await unsubscribe();
unsubscribeTranscript();
await gateway.handler.dispose();
gateway.chatRunState.clear();
attempt.close();
await native.service.stop?.(native.context);
}
src/agents/session-tool-result-guard.transcript-events.test.ts
+144
View File
@@ -5,6 +5,7 @@ import { upsertSessionEntry } from "openclaw/plugin-sdk/session-store-runtime";
import { closeOpenClawAgentDatabasesForTest } from "openclaw/plugin-sdk/sqlite-runtime-testing";
import { afterEach, assert, describe, expect, it, vi } from "vitest";
import { transformMessages } from "../../packages/ai/src/transcript-transform.js";
import { createDeferred } from "../../test/helpers/promise.js";
import { useAutoCleanupTempDirTracker } from "../../test/helpers/temp-dir.js";
import { makeTextToolResult } from "../../test/helpers/text-tool-result.js";
import { makeUserMessage } from "../../test/helpers/user-message.js";
@@ -17,6 +18,10 @@ import {
} from "../config/sessions/session-accessor.js";
import { applyAssistantDeliveryDirectives } from "../config/sessions/transcript-assistant-delivery.js";
import { withOwnedSessionTranscriptWrites } from "../config/sessions/transcript-write-context.js";
import { projectInFlightRunSnapshot } from "../gateway/chat-inflight-snapshot.js";
import { createAgentEventTestHarness } from "../gateway/server-chat.agent-events.test-harness.js";
import { subscribeAgentEvents } from "../gateway/server-chat.agent-events.test-helpers.js";
import type { AgentEventRuntimePayload } from "../infra/agent-events.js";
import { createAssistantMessageEventStream } from "../llm/utils/event-stream.js";
import {
initializeGlobalHookRunner,
@@ -38,6 +43,7 @@ import {
} from "../state/openclaw-agent-db.js";
import { createAssistantErrorTranscript } from "./assistant-error-transcript.js";
import { normalizeAssistantReplayContent } from "./embedded-agent-runner/replay-history.js";
import { createSubscribedSessionHarness } from "./embedded-agent-subscribe.e2e-harness.js";
import { runAgentHarnessBeforeMessageWriteHook } from "./harness/hook-helpers.js";
import { guardSessionManager } from "./session-tool-result-guard-wrapper.js";
import { installSessionToolResultGuard } from "./session-tool-result-guard.js";
@@ -126,6 +132,144 @@ afterEach(async () => {
});
describe("guardSessionManager transcript updates", () => {
async function openSourceProjection(deferred: boolean) {
const { sessionManager, target } = await openPersistedSessionManager();
const runId = `source-${target.sessionId}`;
const manager = guardSessionManager(sessionManager, { ...target, runId });
const gateway = createAgentEventTestHarness();
gateway.register(runId, target.sessionKey, runId);
const sourceEvents: AgentEventRuntimePayload[] = [];
const unsubscribeAgent = subscribeAgentEvents(async (event) => {
if (event.runId === runId) {
if (event.stream === "assistant") {
sourceEvents.push(event);
}
await gateway.handler(event);
}
});
const unsubscribeTranscript = onInternalSessionTranscriptUpdate((event) => {
if (event.sessionId === target.sessionId) {
gateway.handler.retireTranscript(event);
}
});
const finishing = createDeferred();
const finish = createDeferred();
const source = createSubscribedSessionHarness({
runId,
...(deferred ? { onBeforeTerminalDelivery: () => undefined } : {}),
onBeforeLifecycleTerminal: () => {
finishing.resolve();
return finish.promise;
},
});
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
const stream = (message: ReturnType<typeof assistantText>) => {
source.emit({ type: "message_start", message: { ...message, content: [] } });
source.emit({
type: "message_update",
message: { ...message },
assistantMessageEvent: {
type: "text_delta",
delta: message.content
.flatMap((block) => (block.type === "text" ? [block.text] : []))
.join("\n"),
},
});
};
const commit = (message: Parameters<typeof persistAgentSessionMessage>[1]) =>
persistAgentSessionMessage(manager, message, { invalidateSerializedPrefixCache: false });
const snapshot = async () => {
await unsubscribeAgent.drain();
gateway.chatRunState.flushPendingText(runId);
return projectInFlightRunSnapshot({ chatRunState: gateway.chatRunState, runId }).text;
};
return {
...source,
manager,
sourceEvents,
stream,
commit,
snapshot,
finishing,
async close() {
finish.resolve();
await source.subscription.waitForPendingEvents();
source.subscription.unsubscribe();
await unsubscribeAgent();
unsubscribeTranscript();
await gateway.handler.dispose();
gateway.chatRunState.clear();
vi.useRealTimers();
},
};
}
it.each(["superseded", "retained by steer"])(
"retires a committed deferred occurrence %s without hiding identical later output",
async (prior) => {
const h = await openSourceProjection(true);
const first = assistantText("[[reply_to_current]] Repeated answer.");
const second = assistantText("Repeated answer.");
try {
h.stream(first);
h.emit({ type: "message_end", message: first });
const firstId = await h.commit(first);
assert(firstId);
expect(h.manager.getEntry(firstId)).toMatchObject({
type: "message",
message: { role: "assistant" },
});
expect(h.sourceEvents).toEqual([]);
if (prior === "retained by steer") {
const steer = makeUserMessage("Continue with the same answer", 1);
h.emit({ type: "message_start", message: steer });
h.emit({ type: "message_end", message: steer });
await h.commit(steer);
}
h.stream(second);
h.emit({ type: "message_end", message: second });
h.emit({ type: "agent_end", messages: [first, second] });
await h.finishing.promise;
expect(h.sourceEvents.map((event) => event.data.text)).toEqual(
prior === "superseded" ? ["Repeated answer."] : ["Repeated answer.", "Repeated answer."],
);
expect(await h.snapshot()).toBe("Repeated answer.");
await h.commit(second);
expect(await h.snapshot()).toBe("");
} finally {
await h.close();
}
},
);
it("retires a prior committed source after the next identical item starts", async () => {
const h = await openSourceProjection(false);
const first = assistantText("[[reply_to_current]] Same answer.");
const second = assistantText("Same answer.");
try {
h.stream(first);
h.emit({ type: "message_end", message: first });
await h.subscription.waitForPendingEvents();
expect(await h.snapshot()).toBe("Same answer.");
h.stream(second);
await h.subscription.waitForPendingEvents();
expect(await h.snapshot()).toBe("Same answer.\n\nSame answer.");
await h.commit(first);
expect(await h.snapshot()).toBe("Same answer.");
h.emit({
type: "message_update",
message: assistantText("Same answer. Continued."),
assistantMessageEvent: { type: "text_delta", delta: " Continued." },
});
await h.subscription.waitForPendingEvents();
expect(await h.snapshot()).toBe("Same answer. Continued.");
} finally {
await h.close();
}
});
it("preserves prepared source and redaction when a concurrent append forces a retry", async () => {
const { root, sessionManager: manager, target } = await openPersistedSessionManager();
const baseId = manager.appendMessage(makeUserMessage("Compute a value", 1));
src/agents/session-tool-result-guard.ts
+7
View File
@@ -2,6 +2,7 @@ import { normalizeOptionalString } from "@openclaw/normalization-core/string-coe
import { publishTranscriptUpdate } from "../config/sessions/session-accessor.js";
import type { TranscriptEntryAnchor } from "../config/sessions/transcript-entry-anchor.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { bindAgentAssistantSource, readAgentAssistantSource } from "../infra/agent-events.js";
import type {
PluginHookBeforeMessageWriteEvent,
PluginHookBeforeMessageWriteResult,
@@ -321,6 +322,7 @@ export function installSessionToolResultGuard(
},
AppendReceipt
> {
const assistantSource = readAgentAssistantSource(acknowledgementSource);
const runOwnedMessage = attachSessionTranscriptRunId(message, transcriptRunId);
copyCodeModeSourceAppend(message, runOwnedMessage, sourceAppend);
const parentEntryId = sessionManager.getLeafId();
@@ -356,6 +358,11 @@ export function installSessionToolResultGuard(
: undefined;
// Destructive tool-side state commits only after this exact result is durable.
acknowledgeInternalToolResult(acknowledgementSource);
if (assistantSource && appended) {
assistantSource.committedMessageSeq =
messageSeq ?? (anchor ? anchor.activeMessagePosition + 1 : null);
bindAgentAssistantSource(persistedMessage, assistantSource);
}
// Update only committed state, before callbacks can re-enter or throw.
recordPendingReceipt(entryId, persistedMessage, viewWasSuperseded === true);
if (appended) {
src/auto-reply/reply/agent-runner-cli-dispatch.test.ts
+20 -6
View File
@@ -399,17 +399,31 @@ describe("runCliAgentWithLifecycle", () => {
});
const onActivity = vi.fn();
const onAssistantText = vi.fn<(text: string) => Promise<void>>(async () => undefined);
await runCliAgentWithLifecycle({
runId: "run-activity-assistant",
onActivity,
onAssistantText,
runParams: createRunParams("run-activity-assistant"),
const assistantEvents: Record<string, unknown>[] = [];
const stop = onAgentEvent((event) => {
if (event.runId === "run-activity-assistant" && event.stream === "assistant") {
assistantEvents.push(event.data);
}
});
try {
await runCliAgentWithLifecycle({
runId: "run-activity-assistant",
onActivity,
onAssistantText,
runParams: createRunParams("run-activity-assistant"),
});
} finally {
stop();
}
// Every real event stamps, independent of which callbacks are registered.
expect(onAssistantText).toHaveBeenCalledTimes(1);
expect(onActivity).toHaveBeenCalledTimes(2);
expect(assistantEvents).toEqual([
{ text: "Visible answer", delta: "Visible answer" },
{ itemId: "cli-assistant:run-activity-assistant", text: "Visible answer" },
]);
});
it("keeps the captured lifecycle generation on the start event", async () => {
src/auto-reply/reply/agent-runner-cli-dispatch.ts
+2 -1
View File
@@ -4,6 +4,7 @@ import { normalizeOptionalString } from "@openclaw/normalization-core/string-coe
import { Value } from "typebox/value";
import { AgentActivityItemSchema } from "../../../packages/gateway-protocol/src/schema/logs-chat.js";
import { runCliAgent } from "../../agents/cli-runner.js";
import { cliAssistantItemId } from "../../agents/cli-runner/assistant-identity.js";
import { stripOpenClawMcpToolPrefix } from "../../agents/cli-runner/tool-policy.js";
import type { RunCliAgentParams } from "../../agents/cli-runner/types.js";
import type { MediaImageLayout } from "../../agents/embedded-agent-runner/run/prompt-image-metadata.js";
@@ -484,7 +485,7 @@ async function runCliAgentWithLifecycleInternal(
emitAgentEvent({
runId: params.runId,
stream: "assistant",
data: { text: cliText },
data: { itemId: cliAssistantItemId(params.runId), text: cliText },
});
}
src/commands/agent.acp.test.ts
+55 -22
View File
@@ -6,6 +6,8 @@ import "./agent-command.test-mocks.js";
import * as acpManagerModule from "../acp/control-plane/manager.js";
import { AcpRuntimeError } from "../acp/runtime/errors.js";
import { deliverAgentCommandResult } from "../agents/command/delivery.runtime.js";
import { persistAcpTurnTranscript } from "../agents/command/transcript-persistence.js";
import { getTranscriptMessageRole } from "../agents/embedded-agent-runner/message-visibility.js";
import * as embeddedModule from "../agents/embedded-agent.js";
import { readAgentRunTerminalOutcome } from "../channels/turn/agent-run-terminal-outcome.js";
import * as configIoModule from "../config/io.js";
@@ -14,6 +16,10 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
import { withTempHomeCore as withTempHomeBase } from "../plugin-sdk/test-helpers/temp-home.js";
import { parseAgentSessionKey } from "../routing/session-key.js";
import type { RuntimeEnv } from "../runtime.js";
import {
onInternalSessionTranscriptUpdate,
type InternalSessionTranscriptUpdate,
} from "../sessions/transcript-events.js";
import { agentCommand } from "./agent.js";
import { createThrowingTestRuntime } from "./test-runtime-config-helpers.js";
@@ -49,10 +55,9 @@ const attemptExecutionMocks = vi.hoisted(() => ({
emitAcpLifecycleError: vi.fn(),
emitAcpPromptSubmitted: vi.fn(),
emitAcpRuntimeEvent: vi.fn(),
persistAcpTurnTranscript: vi.fn(async ({ sessionEntry }: { sessionEntry?: unknown }) => ({
kind: "persisted",
sessionEntry,
})),
persistAcpTurnTranscript: vi.fn<
typeof import("../agents/command/transcript-persistence.js").persistAcpTurnTranscript
>(async ({ sessionEntry }) => ({ kind: "persisted", sessionEntry })),
}));
vi.mock("../infra/agent-events.js", () => agentEventMocks);
@@ -82,10 +87,12 @@ vi.mock("../agents/command/delivery.runtime.js", () => ({
),
}));
// mock-isolation: ACP command routing does not execute embedded or CLI attempts.
vi.mock("../agents/command/attempt-execution.runtime.js", async () => {
const { buildAcpResult, resolveAcpLifecycleEndFields } = await vi.importActual<
typeof import("../agents/command/acp-lifecycle.js")
>("../agents/command/acp-lifecycle.js");
const { buildAcpResult, emitAcpAssistantDelta, resolveAcpLifecycleEndFields } =
await vi.importActual<typeof import("../agents/command/acp-lifecycle.js")>(
"../agents/command/acp-lifecycle.js",
);
const createAcpVisibleTextAccumulator = () => {
let text = "";
let silent = false;
@@ -121,20 +128,7 @@ vi.mock("../agents/command/attempt-execution.runtime.js", async () => {
emitAcpLifecycleError: attemptExecutionMocks.emitAcpLifecycleError,
emitAcpPromptSubmitted: attemptExecutionMocks.emitAcpPromptSubmitted,
emitAcpRuntimeEvent: attemptExecutionMocks.emitAcpRuntimeEvent,
emitAcpAssistantDelta: ({
runId,
text,
delta,
}: {
runId: string;
text: string;
delta: string;
}) =>
agentEventMocks.emitAgentEvent({
runId,
stream: "assistant",
data: { text, delta },
}),
emitAcpAssistantDelta,
buildAcpResult,
resolveAcpLifecycleEndFields,
persistAcpTurnTranscript: attemptExecutionMocks.persistAcpTurnTranscript,
@@ -277,12 +271,13 @@ function createRunTurnFromTextDeltas(chunks: string[]) {
}
function subscribeAssistantEvents() {
const assistantEvents: Array<{ text?: string; delta?: string }> = [];
const assistantEvents: Array<{ text?: string; delta?: string; itemId?: string }> = [];
const stop = agentEventMocks.onAgentEvent((evt) => {
if (evt.stream !== "assistant") {
return;
}
assistantEvents.push({
itemId: typeof evt.data?.itemId === "string" ? evt.data.itemId : undefined,
text: typeof evt.data?.text === "string" ? evt.data.text : undefined,
delta: typeof evt.data?.delta === "string" ? evt.data.delta : undefined,
});
@@ -468,6 +463,44 @@ describe("agentCommand ACP runtime routing", () => {
},
);
it("streams ACP visible text with the committed assistant occurrence identity", async () => {
await withAcpSessionEnv(async () => {
const updates: InternalSessionTranscriptUpdate[] = [];
const unsubscribe = onInternalSessionTranscriptUpdate((event) => {
if (getTranscriptMessageRole(event.message) === "assistant") {
updates.push(event);
}
});
attemptExecutionMocks.persistAcpTurnTranscript.mockImplementationOnce(
persistAcpTurnTranscript,
);
try {
const repeated = await runAcpTurnWithAssistantEvents(["bo", "ok"]);
expect(updates).toHaveLength(1);
const committed = updates[0];
if (!committed?.target) {
throw new Error("Expected an inline committed ACP assistant");
}
expect(committed.runId).toEqual(expect.any(String));
expect(committed.message).toMatchObject({
idempotencyKey: committed.runId,
__openclaw: { runId: committed.runId },
content: [{ type: "text", text: "book" }],
});
expect(await loadTranscriptEvents(committed.target)).toContainEqual(
expect.objectContaining({ type: "message", message: committed.message }),
);
expect(repeated.assistantEvents).toEqual([
{ itemId: committed.runId, text: "bo", delta: "bo" },
{ itemId: committed.runId, text: "book", delta: "ok" },
]);
expect(repeated.logLines.join("\n")).toContain("book");
} finally {
unsubscribe();
}
});
});
it("keeps no-reply ACP turns silent", async () => {
await withAcpSessionEnv(async () => {
const { assistantEvents, logLines } = await runAcpTurnWithAssistantEvents(["NO_REPLY"]);
src/config/sessions/session-accessor.types.ts
+1 -1
View File
@@ -343,7 +343,7 @@ export type TranscriptMessageAppendResult<TMessage> = {
/** Transcript update fields supplied by callers; the target is resolved here. */
export type TranscriptUpdatePayload = Partial<SessionTranscriptUpdate> &
Pick<InternalSessionTranscriptUpdate, "lifecycleRevision">;
Pick<InternalSessionTranscriptUpdate, "lifecycleRevision" | "assistantItemIds">;
export type LatestTranscriptAssistantText = {
id?: string;
src/gateway/agent-event-assistant-text.ts
+5
View File
@@ -4,6 +4,7 @@ type AssistantTextInput = {
text?: string;
delta?: string;
itemId?: string;
occurrenceId?: string;
replace?: boolean;
replaceable?: boolean;
managedMediaUrls?: string[];
@@ -107,6 +108,10 @@ export function resolveAssistantTextInput(data: unknown): AssistantTextInput | u
text: typeof record.text === "string" ? record.text : undefined,
delta: typeof record.delta === "string" ? record.delta : undefined,
itemId: typeof record.itemId === "string" && record.itemId ? record.itemId : undefined,
occurrenceId:
typeof record.occurrenceId === "string" && record.occurrenceId
? record.occurrenceId
: undefined,
replace: record.replace === true,
replaceable: record.replaceable === true,
...(Array.isArray(record.managedMediaUrls)
src/gateway/chat-abort.ts
+2 -45
View File
@@ -17,7 +17,6 @@ import type { OpenClawConfig } from "../config/types.openclaw.js";
import {
reserveAgentTerminalEvent,
getAgentEventLifecycleGeneration,
type AgentEventPayload,
} from "../infra/agent-events.js";
import {
releaseAgentRunDelegatedAuthority,
@@ -28,14 +27,10 @@ import type { ChatAbortDiagnosticReason } from "./chat-abort-diagnostics.js";
import { removeChatAbortControllerEntry } from "./chat-abort-lifecycle-internal.js";
import type { ChatAbortControllerEntry } from "./chat-abort.types.js";
import { appendChatCanvasBlocksToMessage } from "./chat-display-projection.canvas.js";
import { projectInFlightRunSnapshot, type InFlightRunSnapshot } from "./chat-inflight-snapshot.js";
import { resolveChatRunOwnerAgentId } from "./chat-run-owner.js";
import { projectLiveAssistantBufferedText } from "./live-chat-projector.js";
import type { GatewayBroadcastFn } from "./server-broadcast-types.js";
import {
createChatAbortMarker,
type ChatRunPlanSnapshot,
type ChatRunState,
} from "./server-chat-state.js";
import { createChatAbortMarker, type ChatRunState } from "./server-chat-state.js";
import { resolveRequestedSessionAgentId } from "./session-request-agent.js";
import {
resolveSessionSubscriptionKey,
@@ -55,44 +50,6 @@ export type RestartRecoveryCandidate = {
observedAt?: number;
};
export type InFlightRunSnapshot = {
runId: string;
text: string;
startedAt?: number;
/**
* True when the in-flight run is owned by the embedded-run registry and can
* only be cancelled through the session-owned abort path (sessions.abort),
* never through run-specific chat.abort. Control UI uses this to keep Stop
* routing session-scoped for recovered embedded runs.
*/
sessionAbortable?: boolean;
plan?: ChatRunPlanSnapshot;
events?: AgentEventPayload[];
};
export function projectInFlightRunSnapshot(params: {
chatRunState: Pick<ChatRunState, "resolveBuffer" | "runs">;
runId: string;
startedAtMs?: number;
sessionAbortable?: boolean;
}): InFlightRunSnapshot {
const run = params.chatRunState.runs.get(params.runId);
const projected = projectLiveAssistantBufferedText(
params.chatRunState.resolveBuffer(params.runId).text,
{ suppressLeadFragments: true },
);
const plan = run?.planSnapshot;
const events = run?.progressSnapshot?.events;
return {
runId: params.runId,
text: projected.suppress ? "" : projected.text,
...(params.startedAtMs === undefined ? {} : { startedAt: params.startedAtMs }),
...(params.sessionAbortable ? { sessionAbortable: true } : {}),
...(plan ? { plan } : {}),
...(events?.length ? { events } : {}),
};
}
type RegisteredChatAbortController = {
controller: AbortController;
markExecutionStarted: () => boolean;
src/gateway/chat-inflight-snapshot.ts
+37
View File
@@ -0,0 +1,37 @@
import type { AgentEventPayload } from "../infra/agent-events.js";
import type { ChatRunPlanSnapshot, ChatRunState } from "./server-chat-state.js";
export type InFlightRunSnapshot = {
runId: string;
text: string;
startedAt?: number;
/**
* True when the in-flight run is owned by the embedded-run registry and can
* only be cancelled through the session-owned abort path (sessions.abort),
* never through run-specific chat.abort. Control UI uses this to keep Stop
* routing session-scoped for recovered embedded runs.
*/
sessionAbortable?: boolean;
plan?: ChatRunPlanSnapshot;
events?: AgentEventPayload[];
};
export function projectInFlightRunSnapshot(params: {
chatRunState: Pick<ChatRunState, "resolveBuffer" | "runs">;
runId: string;
startedAtMs?: number;
sessionAbortable?: boolean;
}): InFlightRunSnapshot {
const run = params.chatRunState.runs.get(params.runId);
const projected = params.chatRunState.resolveBuffer(params.runId);
const plan = run?.planSnapshot;
const events = run?.progressSnapshot?.events;
return {
runId: params.runId,
text: projected.suppress ? "" : projected.text,
...(params.startedAtMs === undefined ? {} : { startedAt: params.startedAtMs }),
...(params.sessionAbortable ? { sessionAbortable: true } : {}),
...(plan ? { plan } : {}),
...(events?.length ? { events } : {}),
};
}
src/gateway/http-stream.test-support.ts
+71
View File
@@ -1,3 +1,4 @@
import { asOptionalObjectRecord } from "@openclaw/normalization-core/record-coerce";
import OpenAI from "openai";
import { expect } from "vitest";
import {
@@ -5,6 +6,11 @@ import {
buildAgentRunTerminalOutcomeFromLifecycleEvent,
} from "../agents/agent-run-terminal-outcome.js";
import { createAgentCommandLifecycle } from "../agents/command/lifecycle.js";
import {
createSubscribedSessionHarness,
emitToolRun,
} from "../agents/embedded-agent-subscribe.e2e-harness.js";
import { makeAgentAssistantMessage } from "../agents/test-helpers/agent-message-fixtures.js";
import { recordAgentRunTerminalOutcome } from "../channels/turn/agent-run-terminal-outcome.js";
import {
emitAgentEvent,
@@ -14,6 +20,71 @@ import {
type SseEvent = { event?: string; data: string };
export async function emitEmbeddedLateCommentary(opts: unknown): Promise<void> {
const runId = asOptionalObjectRecord(opts)?.runId;
if (typeof runId !== "string" || !runId) {
throw new Error("Expected a streaming HTTP run identity");
}
const { emit, subscription } = createSubscribedSessionHarness({ runId });
const commentary = "I will inspect the workspace.";
const finalText = "The check is complete.";
const message = (text: string) =>
makeAgentAssistantMessage({
api: "openai-completions",
content: [{ type: "text", text }],
});
try {
emit({ type: "message_start", message: message("") });
emit({
type: "message_update",
message: message(commentary),
assistantMessageEvent: { type: "text_delta", delta: commentary },
});
const completedCommentary = makeAgentAssistantMessage({
api: "openai-completions",
stopReason: "toolUse",
content: [
{
type: "text",
text: commentary,
textSignature: JSON.stringify({ v: 1, id: "commentary-http", phase: "commentary" }),
},
{ type: "toolCall", id: "read-http", name: "read", arguments: {} },
],
});
emit({
type: "message_update",
message: completedCommentary,
assistantMessageEvent: {
type: "toolcall_start",
contentIndex: 1,
partial: completedCommentary,
},
});
emit({ type: "message_end", message: completedCommentary });
emitToolRun({
emit,
toolName: "read",
toolCallId: "read-http",
args: {},
isError: false,
result: { content: [{ type: "text", text: "Read complete." }] },
});
const finalMessage = message(finalText);
emit({ type: "message_start", message: message("") });
emit({
type: "message_update",
message: finalMessage,
assistantMessageEvent: { type: "text_delta", delta: finalText },
});
emit({ type: "message_end", message: finalMessage });
emit({ type: "agent_end", messages: [completedCommentary, finalMessage] });
await subscription.waitForPendingEvents();
} finally {
subscription.unsubscribe();
}
}
export function parseSseEvents(text: string): SseEvent[] {
const events: SseEvent[] = [];
const lines = text.split("\n");
src/gateway/openai-http.test.ts
+20
View File
@@ -53,6 +53,7 @@ import {
emitIncompatibleAssistantReplacement,
emitCompatibleAssistantReplacement,
emitBufferedAssistantReplacement,
emitEmbeddedLateCommentary,
createOpenAiHttpTestClient,
parseSseDataLines,
readRawChatCompletionStream,
@@ -2131,6 +2132,25 @@ describe("OpenAI-compatible HTTP API (e2e)", () => {
},
);
it("keeps embedded late commentary append-only while the final answer streams", async () => {
agentCommandMock.mockImplementationOnce(emitEmbeddedLateCommentary);
const stream = await createOpenAiHttpTestClient(enabledPort).chat.completions.create({
model: "openclaw",
messages: [{ role: "user", content: "Inspect the workspace and summarize it." }],
stream: true,
});
const content: string[] = [];
const finishReasons: Array<string | null> = [];
for await (const chunk of stream) {
for (const choice of chunk.choices) {
content.push(choice.delta.content ?? "");
finishReasons.push(choice.finish_reason);
}
}
expect(content.join("")).toBe("I will inspect the workspace.\n\nThe check is complete.");
expect(finishReasons.at(-1)).toBe("stop");
});
it.each(
incompatibleReplacementCases.toSpliced(6, 0, {
name: "rewritten then explicitly restored",
src/gateway/openresponses-http.test.ts
+16
View File
@@ -31,6 +31,7 @@ import {
import {
incompatibleReplacementCases,
emitIncompatibleAssistantReplacement,
emitEmbeddedLateCommentary,
createOpenAiHttpTestClient,
parseSseEvents,
collectSseEventTypes,
@@ -336,6 +337,21 @@ describe("OpenResponses HTTP API (e2e)", () => {
},
);
it("keeps embedded late commentary append-only while the final answer streams", async () => {
agentCommandMock.mockImplementationOnce(emitEmbeddedLateCommentary);
const stream = createStream();
const content: string[] = [];
const terminals: string[] = [];
stream.on("response.output_text.delta", ({ delta }) => content.push(delta));
stream.on("response.failed", () => terminals.push("failed"));
stream.on("response.completed", () => terminals.push("completed"));
const response = await stream.finalResponse();
expect(content.join("")).toBe("I will inspect the workspace.\n\nThe check is complete.");
expect(response.output_text).toBe(content.join(""));
expect(terminals).toEqual(["completed"]);
expect(response.status).toBe("completed");
});
it.each(
incompatibleReplacementCases.filter(({ name }) =>
[
src/gateway/server-chat-buffer.ts
+297
View File
@@ -0,0 +1,297 @@
import { randomUUID } from "node:crypto";
import type { AgentAssistantSourceReceipt } from "../infra/agent-events.js";
import { mergeAssistantText, type AssistantTextSnapshot } from "./agent-event-assistant-text.js";
import {
capLiveAssistantText,
type createLiveAssistantTextProjection,
} from "./live-chat-projector.js";
export type ChatRunBufferState = {
rawBuffer?: string;
rawOffset?: number;
/** Positions are absolute; null ends retain identity facts after a source replacement. */
assistantItems?: Map<
string | symbol | undefined,
{ itemId?: string; committed?: true; start?: number; end?: number | null; scope?: number }
>;
assistantScope?: AssistantTextSnapshot["scope"];
assistantScopeOffset?: number;
assistantOccurrenceId?: string;
managedMediaUrls?: Set<string>;
display?: {
projector: ReturnType<typeof createLiveAssistantTextProjection>;
current: ReturnType<ReturnType<typeof createLiveAssistantTextProjection>["replace"]>;
pendingRawDelta?: string | null;
reset?: boolean;
unsentDelta: string | null;
sentText?: string;
};
};
type BufferInput = Parameters<typeof mergeAssistantText>[1];
const invalidateRange = (item: { start?: number; end?: number | null }) => {
delete item.start;
item.end = null;
};
export const bufferVisibleText = (record: ChatRunBufferState) => {
const text = record.rawBuffer ?? "";
const offset = record.rawOffset ?? 0;
const items = [...(record.assistantItems?.values() ?? [])]
.filter((item) => typeof item.end === "number" && item.end > offset)
.toSorted((a, b) => (a.start ?? 0) - (b.start ?? 0));
if (!items.some((item) => item.committed)) {
return text;
}
let visible = "";
let cursor = offset;
let gap = "";
let scope: number | undefined;
for (const item of items) {
if (typeof item.end !== "number") {
continue;
}
const start = Math.max(offset, item.start ?? 0);
gap += text.slice(cursor - offset, start - offset);
cursor = Math.max(cursor, item.end);
if (item.committed) {
continue;
}
let part = text.slice(start - offset, item.end - offset);
if (!part) {
continue;
}
if (!visible && (item.start ?? 0) > offset) {
// Retired native boundaries do not leave presentation padding in the tail.
part = part.replace(/^\n{1,2}/, "");
} else if (visible && scope !== item.scope) {
const trailing = visible.endsWith("\n\n") ? 2 : visible.endsWith("\n") ? 1 : 0;
const leading = part.startsWith("\n\n") ? 2 : part.startsWith("\n") ? 1 : 0;
visible += "\n".repeat(Math.max(0, 2 - trailing - leading));
} else if (visible) {
// Unscoped corrections can retain native padding between owned intervals.
visible += gap;
}
visible += part;
gap = "";
scope = item.scope;
}
return visible;
};
export const updateBuffer = (
record: ChatRunBufferState,
incoming: BufferInput,
source?: AgentAssistantSourceReceipt,
) => {
const sourceId = source ? (source.itemId ??= randomUUID()) : undefined;
const itemId = sourceId ?? incoming.occurrenceId ?? incoming.itemId;
// Embedded messages have their own merge scope; native occurrences share one.
const input = sourceId ? { ...incoming, itemId: sourceId } : incoming;
const display = record.display;
if (input.managedMediaUrls?.length) {
const urls = (record.managedMediaUrls ??= new Set<string>());
const previousSize = urls.size;
input.managedMediaUrls.forEach((url) => urls.add(url));
if (display && urls.size !== previousSize) {
display.reset = true;
}
}
const previous = record.rawBuffer ?? "";
const items = (record.assistantItems ??= new Map());
if (source) {
retireSource(record, source);
}
const item = items.get(itemId) ?? {};
if (
itemId !== undefined &&
itemId !== record.assistantOccurrenceId &&
item.committed &&
(item.end !== undefined ||
(bufferVisibleText(record).length > 0 && !(input.replace && input.replaceable)))
) {
// Closed occurrences cannot reopen over a newer live owner.
item.end ??= null;
items.set(itemId, item);
return previous;
}
const previousOccurrence = items.get(record.assistantOccurrenceId);
const anonymous = itemId ? items.get(undefined) : undefined;
if (anonymous) {
items.set(Symbol("anonymous assistant interval"), anonymous);
items.delete(undefined);
}
const sameItem = input.itemId && input.itemId === record.assistantScope?.itemId;
const retainedOffset = record.rawOffset ?? 0;
const previousEnd = retainedOffset + previous.length;
const canAppendRawDelta =
previous.length === 0 ||
(previousOccurrence !== undefined &&
!previousOccurrence.committed &&
previousOccurrence.end === previousEnd &&
(previousOccurrence.start ?? previousEnd) < previousEnd);
const oldScopeOffset = record.assistantScopeOffset;
const snapshot = mergeAssistantText(
{ text: previous, scope: record.assistantScope },
input,
"live",
);
if (
itemId &&
(itemId !== record.assistantOccurrenceId || (input.itemId && !sameItem)) &&
typeof item.end === "number"
) {
// A returning identity or a new native scope leaves the earlier interval intact.
items.set(Symbol("earlier occurrence interval"), { ...item });
invalidateRange(item);
}
const resetSource =
(!itemId && snapshot.appendedText === undefined) ||
(input.replace && input.replaceable && !sameItem);
const sourceOffset =
resetSource || (!input.itemId && snapshot.text === input.text)
? 0
: sameItem &&
input.text !== undefined &&
snapshot.scope &&
oldScopeOffset !== undefined &&
retainedOffset > oldScopeOffset
? Math.min(
retainedOffset,
oldScopeOffset - snapshot.scope.prefix.length - snapshot.scope.separatorLength,
)
: retainedOffset;
const end = sourceOffset + snapshot.text.length;
const scopeOffset = snapshot.scope
? sameItem && input.text === undefined
? oldScopeOffset
: sourceOffset + snapshot.scope.prefix.length + snapshot.scope.separatorLength
: undefined;
let commonEnd = Math.min(retainedOffset, end);
if (snapshot.appendedText !== undefined) {
commonEnd = previousEnd;
} else {
const limit = Math.min(previousEnd, end);
while (commonEnd < limit) {
const point = previous.codePointAt(commonEnd - retainedOffset);
if (point !== snapshot.text.codePointAt(commonEnd - sourceOffset)) {
break;
}
commonEnd += point !== undefined && point > 0xffff ? 2 : 1;
}
}
const replacing = commonEnd < previousEnd;
if (resetSource) {
for (const observed of items.values()) {
if (observed.end !== undefined) {
invalidateRange(observed);
}
}
} else if (replacing || input.replace) {
for (const observed of items.values()) {
if (typeof observed.end === "number") {
observed.end = Math.min(observed.end, commonEnd);
observed.start = Math.min(observed.start ?? 0, commonEnd);
}
}
}
let scope = sameItem ? (previousOccurrence?.scope ?? scopeOffset) : (scopeOffset ?? sourceOffset);
if (!snapshot.scope) {
let through = sourceOffset;
for (const observed of items.values()) {
if (
typeof observed.end === "number" &&
observed.end > through &&
observed.end <= commonEnd &&
observed.end > (observed.start ?? 0)
) {
through = observed.end;
scope = observed.scope;
}
}
}
const restoredUnknown =
!resetSource &&
input.text !== undefined &&
sourceOffset < retainedOffset &&
[...items.values()].some(
(observed) =>
(!snapshot.scope || observed.scope === scope) &&
typeof observed.end === "number" &&
observed.end > (observed.start ?? 0) &&
observed.itemId !== itemId,
);
if (restoredUnknown) {
// The cap discards content: identical retained states can require different
// ownership even for matching suffixes. Keep this replacement unowned rather
// than hiding unsaved text behind a receipt for an unknowable old prefix.
for (const observed of items.values()) {
if ((!snapshot.scope || observed.scope === scope) && observed.end !== undefined) {
invalidateRange(observed);
}
}
items.set(Symbol("restored unknown text"), { start: scopeOffset ?? sourceOffset, end, scope });
}
const start = resetSource
? (scopeOffset ?? 0)
: sameItem
? Math.max(scopeOffset ?? 0, Math.min(item.start ?? previousEnd, commonEnd))
: (scopeOffset ?? item.start ?? commonEnd);
item.start = restoredUnknown ? end : start;
item.end = end;
item.itemId = itemId;
item.scope = scope;
items.set(itemId, item);
record.assistantScope = snapshot.scope;
record.assistantScopeOffset = scopeOffset;
record.assistantOccurrenceId = itemId;
const text = capLiveAssistantText(snapshot);
record.rawOffset = sourceOffset + snapshot.text.length - text.length;
record.rawBuffer = text;
if (display) {
display.reset ||=
text.length !== snapshot.text.length ||
input.replace === true ||
replacing ||
item.committed === true;
display.pendingRawDelta =
snapshot.appendedText !== undefined &&
display.pendingRawDelta !== null &&
!display.reset &&
canAppendRawDelta
? (display.pendingRawDelta ?? "") + snapshot.appendedText
: null;
}
return text;
};
export const retireBuffer = (record: ChatRunBufferState, itemIds: readonly string[]) => {
const items = (record.assistantItems ??= new Map());
let changed = false;
for (const itemId of itemIds) {
const item = items.get(itemId);
changed ||= !item?.committed && typeof item?.end === "number";
items.set(itemId, { ...item, committed: true });
for (const observed of items.values()) {
if (observed.itemId === itemId) {
changed ||= !observed.committed && typeof observed.end === "number";
observed.committed = true;
}
}
}
if (changed && record.display) {
record.display.reset = true;
}
return changed;
};
export const retireSource = (
record: ChatRunBufferState | undefined,
source: AgentAssistantSourceReceipt,
) => {
if (!record || source.committedMessageSeq === undefined) {
return false;
}
return retireBuffer(record, [(source.itemId ??= randomUUID())]);
};
src/gateway/server-chat-state.model.test-support.ts
+162
View File
@@ -0,0 +1,162 @@
import {
normalizeLiveAssistantBufferedText,
projectLiveAssistantBufferedText,
} from "./live-chat-projector.js";
export type ModelOccurrence = {
id?: string;
native?: string;
group?: string;
text: string;
retired: boolean;
evicted?: boolean;
};
export type AvailableAssistantPart = {
owner?: ModelOccurrence;
separatorFor?: string;
text: string;
evicted?: boolean;
};
export type CharacterRun = { character: string; minimum: number; maximum: number };
export function assistantTextDifference(
actual: string,
expected: string,
context: string,
): string | undefined {
if (actual === expected) {
return undefined;
}
let divergence = 0;
while (
divergence < actual.length &&
divergence < expected.length &&
actual[divergence] === expected[divergence]
) {
divergence++;
}
return `${context} ${JSON.stringify({
actualLength: actual.length,
expectedLength: expected.length,
divergence,
actual: actual.slice(divergence, divergence + 40),
expected: expected.slice(divergence, divergence + 40),
})}`;
}
// Classification uses the complete available source. Retirement selects raw
// occurrence chunks; normalizing that tail deliberately does not classify it again.
export const projectAvailableParts = (
parts: AvailableAssistantPart[],
tail: string,
final: boolean,
managedMediaUrls: string[],
contextEvicted: boolean,
) => {
const source = parts.map((part) => part.text).join("");
const options = { final, managedMediaUrls };
const classified = projectLiveAssistantBufferedText(
normalizeLiveAssistantBufferedText(source, options),
{ suppressLeadFragments: !final },
);
const displayText = contextEvicted
? tail
: classified.suppress
? ""
: tail === source
? classified.text
: normalizeLiveAssistantBufferedText(tail, options);
return { ...classified, displayText };
};
export const paragraphSeparator = (before: string, after: string) => {
const trailing = before.endsWith("\n\n") ? 2 : before.endsWith("\n") ? 1 : 0;
const leading = after.startsWith("\n\n") ? 2 : after.startsWith("\n") ? 1 : 0;
return "\n".repeat(Math.max(0, 2 - trailing - leading));
};
export const joinOccurrences = (entries: ModelOccurrence[]) => {
let text = "";
let native: string | undefined;
for (const entry of entries) {
if (!entry.text) {
continue;
}
const group = entry.group ?? entry.native;
if (text && group !== undefined && group !== native) {
text += paragraphSeparator(text, entry.text);
}
text += entry.text;
native = group;
}
return text;
};
export const appendCharacterRuns = (runs: CharacterRun[], text: string, required: boolean) => {
for (let start = 0; start < text.length;) {
const character = text[start]!;
let end = start + 1;
while (end < text.length && text[end] === character) {
end++;
}
const length = end - start;
const previous = runs.at(-1);
if (previous?.character === character) {
previous.minimum += required ? length : 0;
previous.maximum += length;
} else {
runs.push({ character, minimum: required ? length : 0, maximum: length });
}
start = end;
}
};
// Recognize source order with only committed bytes and presentation separators
// optional. Run-length intervals avoid backtracking over the 600k fixtures.
export const preservesAvailableText = (source: CharacterRun[], actual: string) => {
const actualRuns: CharacterRun[] = [];
appendCharacterRuns(actualRuns, actual, true);
let positions = new Map<number, Array<[number, number]>>([[0, [[0, 0]]]]);
for (const run of source) {
const next = new Map<number, Array<[number, number]>>();
const retain = (index: number, low: number, high: number) => {
if (low <= high) {
const ranges = next.get(index) ?? [];
ranges.push([low, high]);
next.set(index, ranges);
}
};
for (const [index, ranges] of positions) {
const current = actualRuns[index];
for (const [low, high] of ranges) {
if (run.minimum === 0) {
retain(index, low, high);
}
if (current?.character !== run.character) {
continue;
}
const from = low + Math.max(1, run.minimum);
const through = Math.min(current.maximum, high + run.maximum);
retain(index, from, Math.min(through, current.maximum - 1));
if (from <= current.maximum && through === current.maximum) {
retain(index + 1, 0, 0);
}
}
}
for (const [index, ranges] of next) {
const merged: Array<[number, number]> = [];
for (const range of ranges.toSorted((left, right) => left[0] - right[0])) {
const previous = merged.at(-1);
if (previous && range[0] <= previous[1] + 1) {
previous[1] = Math.max(previous[1], range[1]);
} else {
merged.push(range);
}
}
next.set(index, merged);
}
positions = next;
if (positions.size === 0) {
return false;
}
}
return positions.has(actualRuns.length);
};
src/gateway/server-chat-state.projection.test.ts
+900 -3
View File
@@ -1,8 +1,511 @@
import { describe, expect, it, vi } from "vitest";
import {
INTERNAL_RUNTIME_CONTEXT_BEGIN,
INTERNAL_RUNTIME_CONTEXT_END,
} from "../agents/internal-runtime-context.js";
import { SILENT_REPLY_TOKEN } from "../auto-reply/tokens.js";
import type { AgentAssistantSourceReceipt } from "../infra/agent-events.js";
import * as codeRegions from "../shared/text/code-regions.js";
import { sanitizeChatHistoryMessage } from "./chat-display-projection.sanitize.js";
import { projectInFlightRunSnapshot } from "./chat-inflight-snapshot.js";
import { SUPPRESSED_CONTROL_REPLY_TOKENS } from "./control-reply-text.js";
import { capLiveAssistantText } from "./live-chat-projector.js";
import { createChatRunState } from "./server-chat-state.js";
import {
appendCharacterRuns,
assistantTextDifference,
joinOccurrences as join,
paragraphSeparator,
preservesAvailableText,
projectAvailableParts,
type AvailableAssistantPart as AvailablePart,
type CharacterRun,
type ModelOccurrence as Occurrence,
} from "./server-chat-state.model.test-support.js";
describe("live chat directive projection", () => {
it.each([
{
name: "bare trailing policy",
prefix: "The token is ",
suffix: SILENT_REPLY_TOKEN,
expected: SILENT_REPLY_TOKEN,
},
{
name: "punctuated literal suffix",
prefix: "The token is ",
suffix: `${SILENT_REPLY_TOKEN}.`,
expected: `${SILENT_REPLY_TOKEN}.`,
},
{ name: "standalone silent reply", prefix: "", suffix: SILENT_REPLY_TOKEN, expected: "" },
{
name: "terminal control lead fragment",
prefix: SILENT_REPLY_TOKEN.slice(0, 1),
suffix: SILENT_REPLY_TOKEN.slice(1, 2),
expected: "",
finalExpected: SILENT_REPLY_TOKEN.slice(1, 2),
},
{
name: "runtime directive crossing occurrences",
prefix: `Opening\n${INTERNAL_RUNTIME_CONTEXT_BEGIN}\nPrivate`,
suffix: ` context\n${INTERNAL_RUNTIME_CONTEXT_END}\nVisible`,
expected: " context\nVisible",
},
{
name: "media directive crossing occurrences",
prefix: "Opening\nMEDIA",
suffix: ":./model.png\nVisible",
expected: ":./model.png\nVisible",
},
{
name: "code literal crossing occurrences",
prefix: "`",
suffix: "[[reply_to_current]]`",
expected: "`",
},
{
name: "media URL retained after its directive prefix commits",
prefix: "Opening\nMEDIA:prose ./model.png ",
suffix: "https://example.com/a.png\n",
expected: "https://example.com/a.png\n",
},
])(
"decides suppression from full source for $name",
({ prefix, suffix, expected, finalExpected }) => {
const state = createChatRunState();
const runId = "source-classification";
if (prefix) {
state.updateBuffer(runId, { itemId: "native", occurrenceId: "a", text: prefix });
}
state.updateBuffer(runId, {
itemId: "native",
occurrenceId: "b",
text: prefix + suffix,
managedMediaUrls: ["./model.png"],
});
if (prefix && suffix === SILENT_REPLY_TOKEN) {
// Full live/history policy strips a bare trailing token. Once its context
// commits, the raw tail stays visible instead of being classified alone.
expect(state.resolveBuffer(runId).text).toBe(prefix.trimEnd());
expect(
sanitizeChatHistoryMessage({
role: "assistant",
content: [{ type: "text", text: prefix + suffix }],
}).message,
).toMatchObject({ content: [{ type: "text", text: prefix.trimEnd() }] });
}
if (prefix) {
state.retireBuffer(runId, ["a"]);
}
expect
.soft(state.resolveBuffer(runId))
.toMatchObject({ text: expected, suppress: !expected });
expect.soft(projectInFlightRunSnapshot({ chatRunState: state, runId }).text).toBe(expected);
const final = state.resolveBuffer(runId, { final: true });
expect(final.displayText ?? final.text).toBe(finalExpected ?? expected);
},
);
it("retires a keyed source after repeated capped complete snapshots", () => {
const state = createChatRunState();
const text = `${"a".repeat(100_000)}${"b".repeat(500_000)}`;
state.updateBuffer("reply", { itemId: "native", text });
state.updateBuffer("reply", { itemId: "native", text: `${text}!` });
expect(state.retireBuffer("reply", ["native"])).toBe(true);
expect(state.resolveBuffer("reply").text).toBe("");
});
it("matches a terminal-only native receipt by identity before accepting an identical new item", () => {
const state = createChatRunState();
state.retireBuffer("reply", ["superseded"]);
state.retireBuffer("reply", ["selected"]);
state.updateBuffer("reply", {
itemId: "selected",
text: "Same.",
replace: true,
replaceable: true,
});
expect(state.resolveBuffer("reply").text).toBe("");
state.updateBuffer("reply", { itemId: "new", text: "Same.", replace: true, replaceable: true });
expect(state.resolveBuffer("reply").text).toBe("Same.");
});
it("keeps a later item live when the previous paragraph commits after it starts", () => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "Saved." });
state.updateBuffer("reply", { itemId: "second", text: "Tail." });
expect(state.retireBuffer("reply", ["first"])).toBe(true);
expect(state.resolveBuffer("reply").text).toBe("Tail.");
state.updateBuffer("reply", { itemId: "second", text: "Tail. More." });
expect(state.resolveBuffer("reply").text).toBe("Tail. More.");
});
it.each([
{ sameNative: false, texts: ["First", "Middle", "Again"], visible: "First\n\nAgain" },
{ sameNative: true, texts: ["A", "AB", "ABC"], visible: "AC" },
])(
"keeps returning occurrences disjoint (sameNative=$sameNative)",
({ sameNative, texts, visible }) => {
const state = createChatRunState();
["first", "middle", "first"].forEach((occurrenceId, index) => {
state.updateBuffer("reply", {
itemId: sameNative ? "native" : occurrenceId,
occurrenceId,
text: texts[index],
});
});
state.retireBuffer("reply", ["middle"]);
expect(state.resolveBuffer("reply").text).toBe(visible);
state.retireBuffer("reply", ["first"]);
expect(state.resolveBuffer("reply").text).toBe("");
},
);
it("keeps an invalidated occurrence disjoint when its identity returns", () => {
const state = createChatRunState();
const runId = "reused-occurrence";
state.updateBuffer(runId, { itemId: "n", occurrenceId: "a", text: "A" });
state.updateBuffer(runId, {
itemId: "m",
occurrenceId: "m",
text: "X",
replace: true,
replaceable: true,
});
state.updateBuffer(runId, { itemId: "n", occurrenceId: "b", text: "B" });
state.updateBuffer(runId, { itemId: "n", occurrenceId: "a", text: "BC" });
state.retireBuffer(runId, ["m"]);
expect.soft(state.resolveBuffer(runId).text).toBe("BC");
expect.soft(projectInFlightRunSnapshot({ chatRunState: state, runId }).text).toBe("BC");
const final = state.resolveBuffer(runId, { final: true });
expect(final.displayText ?? final.text).toBe("BC");
});
it("retires an earlier unscoped cumulative occurrence without replaying its prefix", () => {
const state = createChatRunState();
state.updateBuffer("reply", { occurrenceId: "first", text: "A" });
state.updateBuffer("reply", { occurrenceId: "second", text: "AB" });
state.retireBuffer("reply", ["first"]);
expect(state.resolveBuffer("reply").text).toBe("B");
state.retireBuffer("reply", ["second"]);
expect(state.resolveBuffer("reply").text).toBe("");
});
it.each(["", "\n", "\n\n"])(
"keeps unscoped correction boundary %j after retirement",
(separator) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "A" });
state.updateBuffer("reply", { itemId: "discarded", text: "B" });
state.updateBuffer("reply", { occurrenceId: "corrected", text: `A${separator}X` });
state.updateBuffer("reply", { itemId: "last", text: "Last" });
state.retireBuffer("reply", ["last"]);
expect(state.resolveBuffer("reply").text).toBe(`A${separator}X`);
state.retireBuffer("reply", ["first"]);
expect(state.resolveBuffer("reply").text).toBe("X");
state.retireBuffer("reply", ["corrected"]);
expect(state.resolveBuffer("reply").text).toBe("");
},
);
it.each(["anonymous", "native"])(
"keeps %s deltas aligned after an empty active occurrence",
(kind) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "Visible." });
state.updateBuffer("reply", { itemId: "second", occurrenceId: "saved", text: "Saved" });
state.retireBuffer("reply", ["saved"]);
state.updateBuffer("reply", { itemId: "second", occurrenceId: "empty", text: "Saved" });
expect(state.resolveBuffer("reply").suppress).toBe(false);
const baseline = state.resolveBuffer("reply").text;
state.takeBufferDelta("reply", baseline);
state.updateBuffer("reply", {
...(kind === "native" ? { itemId: "second", occurrenceId: "empty" } : {}),
delta: "X",
});
const visible = state.resolveBuffer("reply").text;
const delta = state.takeBufferDelta("reply", visible);
const wire = delta?.replace ? delta.deltaText : baseline + (delta?.deltaText ?? "");
expect(visible).toBe("Visible.\n\nX");
expect(wire).toBe(visible);
},
);
it("retains an identified commit before its first ordinary text callback", () => {
const state = createChatRunState();
state.retireBuffer("reply", ["saved"]);
state.updateBuffer("reply", { itemId: "saved", text: "Saved." });
state.updateBuffer("reply", { itemId: "tail", text: "Tail." });
expect(state.resolveBuffer("reply").text).toBe("Tail.");
expect(state.resolveBuffer("reply", { final: true }).text).toBe("Saved.\n\nTail.");
});
it("retires an identified commit before a correction of the same occurrence", () => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "answer", text: "Draft" });
state.retireBuffer("reply", ["answer"]);
state.updateBuffer("reply", { itemId: "answer", text: "Corrected", replace: true });
expect(state.resolveBuffer("reply").text).toBe("");
expect(state.resolveBuffer("reply", { final: true }).text).toBe("Corrected");
});
it.each(["Saved.", "Saved.\n\n"])(
"keeps corrected text after a committed prefix %j and changing leading newlines",
(prefix) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "saved", text: prefix });
state.retireBuffer("reply", ["saved"]);
state.takeBufferDelta("reply", "");
state.updateBuffer("reply", { itemId: "tail", text: "\n" });
expect(state.resolveBuffer("reply").text).toBe("");
state.updateBuffer("reply", { itemId: "tail", delta: "\nDraft" });
expect(state.resolveBuffer("reply").text).toBe("Draft");
state.updateBuffer("reply", { itemId: "tail", text: "Corrected", replace: true });
expect(state.resolveBuffer("reply").text).toBe("Corrected");
expect(state.takeBufferDelta("reply", "Corrected")).toEqual({
deltaText: "Corrected",
replace: true,
});
},
);
it.each(["visible", "empty"])(
"retires only owned bytes when %s commits before the other occurrence",
(firstCommit) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "visible", text: "Saved." });
state.updateBuffer("reply", { itemId: "empty", text: "", delta: "" });
state.retireBuffer("reply", [firstCommit]);
expect(state.resolveBuffer("reply").text).toBe(firstCommit === "visible" ? "" : "Saved.");
state.retireBuffer("reply", [firstCommit === "visible" ? "empty" : "visible"]);
expect(state.resolveBuffer("reply").text).toBe("");
},
);
it.each([
{ kind: "item", observed: true },
{ kind: "item", observed: false },
{ kind: "source", observed: true },
{ kind: "source", observed: false },
])(
"keeps a newer tail when committed $kind callbacks arrive late (observed=$observed)",
({ kind, observed }) => {
const state = createChatRunState();
const first: AgentAssistantSourceReceipt = {};
const second: AgentAssistantSourceReceipt = {};
if (observed) {
state.updateBuffer(
"reply",
{ itemId: "first", text: "Saved." },
kind === "source" ? first : undefined,
);
}
if (kind === "source") {
first.committedMessageSeq = 2;
state.retireSource("reply", first);
} else {
state.retireBuffer("reply", ["first"]);
}
state.updateBuffer(
"reply",
{ itemId: "second", text: "Unsaved." },
kind === "source" ? second : undefined,
);
state.updateBuffer(
"reply",
{ itemId: "first", text: "Saved. corrected.", replace: true },
kind === "source" ? first : undefined,
);
expect(state.resolveBuffer("reply").text).toBe("Unsaved.");
state.updateBuffer(
"reply",
{ itemId: "second", text: "Unsaved. More.", delta: " More." },
kind === "source" ? second : undefined,
);
expect(state.resolveBuffer("reply").text).toBe("Unsaved. More.");
if (kind === "source") {
second.committedMessageSeq = 3;
state.retireSource("reply", second);
} else {
state.retireBuffer("reply", ["second"]);
}
state.updateBuffer(
"reply",
{ itemId: "first", text: "Saved. corrected again.", replace: true },
kind === "source" ? first : undefined,
);
expect(state.resolveBuffer("reply").text).toBe("");
expect(state.resolveBuffer("reply", { final: true }).text).toBe(
`${observed ? "Saved.\n\n" : ""}Unsaved. More.`,
);
},
);
it.each([false, true])(
"retires all source intervals when only its late callback sees the commit (returning=%s)",
(returning) => {
const state = createChatRunState();
const first: AgentAssistantSourceReceipt = {};
const second: AgentAssistantSourceReceipt = {};
state.updateBuffer("reply", { itemId: "first", text: "Saved." }, first);
state.updateBuffer("reply", { itemId: "second", text: "Unsaved." }, second);
if (returning) {
state.updateBuffer("reply", { itemId: "first", text: "Saved again." }, first);
state.updateBuffer("reply", { itemId: "tail", text: "Tail." }, {});
}
first.committedMessageSeq = 2;
state.updateBuffer("reply", { itemId: "first", text: "Corrected.", replace: true }, first);
expect(state.resolveBuffer("reply").text).toBe(returning ? "Unsaved.\n\nTail." : "Unsaved.");
},
);
it.each([undefined, "reused-display-id"])(
"keeps private occurrences in separate buffer scopes (displayItemId=%s)",
(itemId) => {
const state = createChatRunState();
const first: AgentAssistantSourceReceipt = {};
const second: AgentAssistantSourceReceipt = {};
state.updateBuffer("reply", { itemId, text: "Saved." }, first);
first.committedMessageSeq = 2;
state.retireSource("reply", first);
state.updateBuffer("reply", { itemId, text: "Unsaved.", delta: "" }, second);
expect(state.resolveBuffer("reply").text).toBe("Unsaved.");
second.committedMessageSeq = 3;
state.retireSource("reply", second);
state.updateBuffer("reply", { itemId, text: "Corrected.", replace: true }, second);
expect(state.resolveBuffer("reply").text).toBe("");
},
);
it.each([
{ identified: true, committedBeforeAppend: false },
{ identified: false, committedBeforeAppend: false },
{ identified: true, committedBeforeAppend: true },
{ identified: false, committedBeforeAppend: true },
])(
"keeps earlier unsaved text when a later item commits ($identified, $committedBeforeAppend)",
({ identified, committedBeforeAppend }) => {
const state = createChatRunState();
state.updateBuffer("reply", { ...(identified ? { itemId: "first" } : {}), text: "Unsaved." });
state.updateBuffer("reply", { itemId: "second", text: "Saved." });
if (committedBeforeAppend) {
state.retireBuffer("reply", ["second"]);
}
let wire = state.resolveBuffer("reply").text;
state.takeBufferDelta("reply", wire);
state.updateBuffer("reply", { delta: "More." });
if (!committedBeforeAppend) {
state.retireBuffer("reply", ["second"]);
}
const delta = state.takeBufferDelta("reply", state.resolveBuffer("reply").text);
if (delta) {
wire = delta.replace ? delta.deltaText : wire + delta.deltaText;
}
expect(wire).toBe("Unsaved.\n\nMore.");
expect(state.resolveBuffer("reply").text).toBe("Unsaved.\n\nMore.");
if (identified) {
state.retireBuffer("reply", ["first"]);
expect(state.resolveBuffer("reply").text).toBe("More.");
}
},
);
it.each([true, false])(
"keeps unidentified text outside an earlier occurrence's frontier (committedBefore=%s)",
(committedBefore) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "old", text: "Saved." });
if (committedBefore) {
state.retireBuffer("reply", ["old"]);
}
state.updateBuffer("reply", { text: "Saved.\n\nUnsaved.", replace: true });
if (!committedBefore) {
state.retireBuffer("reply", ["old"]);
}
expect(state.resolveBuffer("reply").text).toBe("Saved.\n\nUnsaved.");
},
);
it.each([true, false])(
"preserves exact retirement across an unidentified append (committedBefore=%s)",
(committedBefore) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "saved", text: "Saved." });
state.takeBufferDelta("reply", "Saved.");
if (committedBefore) {
state.retireBuffer("reply", ["saved"]);
state.takeBufferDelta("reply", "");
}
state.updateBuffer("reply", { delta: "More." });
if (!committedBefore) {
state.retireBuffer("reply", ["saved"]);
}
expect(state.resolveBuffer("reply").text).toBe("More.");
expect(state.takeBufferDelta("reply", "More.")?.deltaText).toBe("More.");
},
);
it.each([false, true])(
"consumes genuinely late committed text without manufacturing source bytes (snapshot=%s)",
(snapshot) => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "First" });
state.takeBufferDelta("reply", "First");
state.retireBuffer("reply", ["first"]);
expect(state.takeBufferDelta("reply", "")).toEqual({ deltaText: "", replace: true });
expect(state.resolveBuffer("reply", { final: true }).text).toBe("First");
state.updateBuffer("reply", {
itemId: "first",
delta: " note",
...(snapshot ? { text: "First note" } : {}),
});
expect(state.resolveBuffer("reply").text).toBe("");
expect(state.resolveBuffer("reply", { final: true }).text).toBe("First note");
expect(state.takeBufferDelta("reply", "")).toBeUndefined();
state.updateBuffer("reply", { itemId: "second", text: "Tail." });
expect(state.resolveBuffer("reply").text).toBe("Tail.");
},
);
it("replaces a trimmed identical tail when its earlier occurrence commits", () => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "Same." });
state.takeBufferDelta("reply", "Same.");
state.updateBuffer("reply", { itemId: "second", text: "Same." });
state.retireBuffer("reply", ["first"]);
expect(state.takeBufferDelta("reply", state.resolveBuffer("reply").text.trim())).toEqual({
deltaText: "Same.",
replace: true,
});
});
it("retires identical messages only by their committed item identities", () => {
const state = createChatRunState();
state.updateBuffer("reply", { itemId: "first", text: "Same." });
state.takeBufferDelta("reply", "Same.");
state.retireBuffer("reply", ["first"]);
expect(state.takeBufferDelta("reply", "")).toEqual({ deltaText: "", replace: true });
state.updateBuffer("reply", { itemId: "second", text: "Same." });
expect(state.resolveBuffer("reply").text).toBe("Same.");
expect(state.retireBuffer("reply", ["first"])).toBe(false);
expect(state.resolveBuffer("reply").text).toBe("Same.");
expect(state.retireBuffer("reply", ["second"])).toBe(true);
expect(state.resolveBuffer("reply").text).toBe("");
state.updateBuffer("reply", { itemId: "third", text: "Same." });
expect(state.resolveBuffer("reply").text).toBe("Same.");
});
it("keeps the retirement frontier across capped cumulative snapshots", () => {
const state = createChatRunState();
const committed = `${"x".repeat(500_020)}Saved.`;
state.updateBuffer("reply", { itemId: "first", text: committed });
expect(state.retireBuffer("reply", ["first"])).toBe(true);
state.updateBuffer("reply", { itemId: "second", text: "Tail." });
expect(state.resolveBuffer("reply").text).toBe("Tail.");
state.updateBuffer("reply", { itemId: "second", delta: " More." });
expect(state.resolveBuffer("reply").text).toBe("Tail. More.");
});
it("keeps settled literal directives without repeatedly parsing the growing reply", () => {
const regions = vi.spyOn(codeRegions, "findCodeRegions");
const ownership = vi.spyOn(codeRegions, "findCodeOwnership");
@@ -112,15 +615,409 @@ describe("live chat directive projection", () => {
expect(state.takeBufferDelta("reply", visible)).toEqual({ deltaText: visible, replace: true });
});
it("retires code ownership when the display cap removes its opening delimiter", () => {
it("keeps text visible when the cap evicts its classification context", () => {
const state = createChatRunState();
const prefix = "`[[reply_to_current]]`\n\nNext ";
state.updateBuffer("reply", { delta: prefix + "x".repeat(500_000 - prefix.length) });
expect(state.resolveBuffer("reply").text.startsWith(prefix)).toBe(true);
state.updateBuffer("reply", { delta: "!" });
const visible = state.resolveBuffer("reply").text;
expect(visible.startsWith("`\n\nNext ")).toBe(true);
expect(visible).not.toContain("[[reply_to_current]]");
expect(visible.startsWith(prefix.slice(1))).toBe(true);
expect(visible).toContain("[[reply_to_current]]");
expect(visible.endsWith("!")).toBe(true);
});
});
it("matches a seeded occurrence model across real caps and native replacements", () => {
const failures: string[] = [];
const managedMediaUrls = ["./model.png"];
const block = (marker: string, size: number, tag: string) => {
const suffix = `#${tag};`;
return marker.repeat(Math.max(0, size - suffix.length)) + suffix;
};
// Use the production cap policy directly; its limit is private to that module.
const capped = (text: string) => capLiveAssistantText({ text });
const seeds = [0x51a7, 0xbadc0de, 0xc0ffee];
for (const seed of seeds) {
const prefixVariant = (seed >>> 5) & 3;
const state = createChatRunState();
const runId = "occurrence-model";
const retired = new Set<string>();
let entries: Occurrence[] = [];
let available: AvailablePart[] = [];
let currentNative: string | undefined;
let currentGroup: string | undefined;
let scopeSerial = 0;
const occurrenceIds = new Set<string>();
const nativeIds = new Set<string>();
const evictedGroups = new Set<string>();
let serial = 0;
let step = 0;
let wire = "";
let randomState = seed;
let firstMismatch: string | undefined;
let mismatchCount = 0;
let crossedCap = false;
let contextEvicted = false;
const random = () => {
randomState ^= randomState << 13;
randomState ^= randomState >>> 17;
randomState ^= randomState << 5;
return randomState >>> 0;
};
const textBlock = () => {
const size = random() % 3 ? 1 + (random() % 20) : 100_000 + (random() % 200_001);
const token =
SUPPRESSED_CONTROL_REPLY_TOKENS[random() % SUPPRESSED_CONTROL_REPLY_TOKENS.length]!;
const alphabet = [
token,
`${token}.`,
`The token is ${token}`,
`The token is ${token}.`,
`${token}\nVisible`,
`Before ${token} after`,
token.slice(0, 3),
`\n${INTERNAL_RUNTIME_CONTEXT_BEGIN}\nPrivate`,
`\n${INTERNAL_RUNTIME_CONTEXT_END}\nVisible`,
"\nMEDIA:./model.png\nVisible",
"\nMEDIA",
];
return size < 100 && random() % 3 === 0
? alphabet[random() % alphabet.length]!
: block(String.fromCharCode(97 + (random() % 8)), size, `o${++serial}`);
};
const capAvailability = () => {
const supplied = available.map((part) => part.text).join("");
let removed = supplied.length - capped(supplied).length;
crossedCap ||= removed > 0;
contextEvicted ||= removed > 0;
available = available.flatMap((part) => {
if (removed === 0) {
return [part];
}
if (part.owner) {
part.owner.evicted = true;
if (part.owner.group) {
evictedGroups.add(part.owner.group);
}
}
const count = Math.min(removed, part.text.length);
removed -= count;
return count === part.text.length
? []
: [{ ...part, text: part.text.slice(count), evicted: true }];
});
};
const verify = (op: string) => {
const context = `seed=0x${seed.toString(16)} step=${step} op=${op}`;
const ranges = [...(state.runs.get(runId)?.assistantItems?.values() ?? [])]
.flatMap((item) => {
if (item.end === null && item.start !== undefined) {
throw new Error(`${context} invalidated range retains start=${item.start}`);
}
if (typeof item.end !== "number") {
return [];
}
if (typeof item.start !== "number" || item.start > item.end) {
throw new Error(`${context} invalid bounds ${JSON.stringify(item)}`);
}
return item.start < item.end
? [{ id: item.itemId, start: item.start, end: item.end }]
: [];
})
.toSorted((left, right) => left.start - right.start || left.end - right.end);
for (let index = 1; index < ranges.length; index++) {
const left = ranges[index - 1];
const right = ranges[index];
if (left && right && left.end > right.start) {
throw new Error(`${context} overlapping ranges ${JSON.stringify([left, right])}`);
}
}
const visible = (parts: AvailablePart[]) => {
const text = join(
parts.flatMap((part) =>
part.owner && !part.owner.retired ? [{ ...part.owner, text: part.text }] : [],
),
);
return available.some((part) => part.owner?.retired) &&
available.findIndex((part) => part.owner && !part.owner.retired && part.text) > 0
? text.replace(/^\n{1,2}/, "")
: text;
};
const tail = visible(available);
const projected = projectAvailableParts(
available,
tail,
false,
managedMediaUrls,
contextEvicted,
);
const terminal = projectAvailableParts(
available,
tail,
true,
managedMediaUrls,
contextEvicted,
);
const expected = projected.displayText;
const expectedFinal = terminal.displayText;
const ambiguous = available.some(
(part) =>
part.evicted ||
part.owner?.evicted ||
(part.owner?.group && evictedGroups.has(part.owner.group)),
);
const allowed: CharacterRun[] = [];
if (ambiguous) {
for (const part of available) {
appendCharacterRuns(allowed, part.text, Boolean(part.owner && !part.owner.retired));
}
}
const resolved = state.resolveBuffer(runId);
const live = resolved.suppress ? "" : resolved.text;
const delta = state.takeBufferDelta(runId, live);
if (delta) {
wire = delta.replace ? delta.deltaText : wire + delta.deltaText;
}
const final = state.resolveBuffer(runId, { final: true });
const views: Array<[string, string, string]> = [
["live", live, expected],
["delta", wire, expected],
["snapshot", projectInFlightRunSnapshot({ chatRunState: state, runId }).text, expected],
["display-final", final.displayText ?? (final.suppress ? "" : final.text), expectedFinal],
["complete-final", final.text, terminal.text],
];
const safety = new Map<string, boolean>();
for (const [view, actual, wanted] of views) {
const exact = !ambiguous || view === "complete-final";
let matches = actual === wanted;
if (!exact && !matches) {
matches = safety.get(actual) ?? preservesAvailableText(allowed, actual);
safety.set(actual, matches);
}
if (!matches) {
mismatchCount++;
firstMismatch ??= assistantTextDifference(
actual,
wanted,
`seed=${seed} step=${step} op=${op} view=${view} oracle=${exact ? "exact" : "safety"}`,
);
}
}
step++;
};
const emit = (entry: Occurrence, replace = false, replaceable = false) => {
const sameNative = currentNative === entry.native;
const group = sameNative ? currentGroup : `scope-${++scopeSerial}`;
entry.group = group;
const supplied = entries.filter((part) => part.group === group);
if (entry.id) {
occurrenceIds.add(entry.id);
}
if (entry.native) {
nativeIds.add(entry.native);
}
available =
replace && replaceable && !sameNative
? []
: sameNative
? available.filter((part) => !part.owner || part.owner.group !== group)
: available;
if (replace && replaceable && !sameNative) {
contextEvicted = false;
}
available = available.filter((part) => part.separatorFor !== group);
const prefix = available.map((part) => part.text).join("");
const suppliedText = supplied.map((part) => part.text).join("");
if (prefix && suppliedText) {
available.push({ separatorFor: group, text: paragraphSeparator(prefix, suppliedText) });
}
available.push(
...supplied.filter((part) => part.text).map((owner) => ({ owner, text: owner.text })),
);
currentNative = entry.native;
currentGroup = group;
capAvailability();
state.updateBuffer(runId, {
itemId: entry.native,
occurrenceId: entry.id,
text: suppliedText,
managedMediaUrls,
...(replace ? { replace: true } : {}),
...(replaceable ? { replaceable: true } : {}),
});
};
const commit = (id: string) => {
retired.add(id);
for (const entry of entries) {
if (entry.id === id) {
entry.retired = true;
}
}
state.retireBuffer(runId, [id]);
verify(`commit:${id}`);
};
const saved = block("a", 100_000, "A");
const hidden =
prefixVariant === 1
? saved
: prefixVariant === 2
? block("z", 100_000, "A")
: "a".repeat(75_000) + block("z", 25_000, "A");
const a: Occurrence = { id: "A", native: "native", text: hidden, retired: false };
const b: Occurrence = {
id: "B",
native: "native",
text: block("b", 250_000, "B0"),
retired: false,
};
entries.push(a);
emit(a);
verify("item:A:100000");
entries.push(b);
emit(b);
verify("occurrence:B:250000");
b.text += block("b", 250_000, "B1");
emit(b);
verify("append:B:250000");
commit("A");
const replacement = saved + b.text.slice(0, 450_000);
// Three known source edits produce the same replacement after identical
// retained windows: preserve all A, replace all A, or preserve its first 75k.
const retained = prefixVariant === 1 ? a.text.length : prefixVariant === 2 ? 0 : 75_000;
a.text = a.text.slice(0, retained);
b.text = replacement.slice(retained);
emit(b, true);
const shrankAfterEviction = crossedCap;
verify("same-item:shrink-after-eviction:550000");
for (const [id, native, text, reset] of [
["reuse-a", "reuse-n", "Old", true],
["reuse-m", "reuse-m", "Reset", true],
["reuse-b", "reuse-n", "Keep", false],
["reuse-a", "reuse-n", "New", false],
] satisfies Array<[string, string, string, boolean]>) {
const entry = { id, native, text, retired: false };
entries = reset ? [entry] : [...entries, entry];
emit(entry, reset, reset);
verify(`invalidate-return:${id}:${native}`);
}
commit("reuse-m");
const token = SUPPRESSED_CONTROL_REPLY_TOKENS[seed % SUPPRESSED_CONTROL_REPLY_TOKENS.length]!;
const semanticParts = [
["The token is ", `${token}.`],
["The token is ", token],
["", token],
[`${token}\n`, "Visible"],
["Before ", `${token} after`],
["Prefix ", token.slice(0, 3)],
[
`Text\n${INTERNAL_RUNTIME_CONTEXT_BEGIN}\nPrivate`,
` hidden\n${INTERNAL_RUNTIME_CONTEXT_END}\nVisible`,
],
["Text\nMEDIA", ":./model.png\nVisible"],
];
for (const [index, [prefix, suffix]] of semanticParts.entries()) {
const native = `semantic-${index}`;
const before = { id: `${native}-a`, native, text: prefix!, retired: false };
entries = [before];
emit(before, true, true);
verify(`semantic-prefix:${index}`);
const after = { id: `${native}-b`, native, text: suffix!, retired: false };
entries.push(after);
emit(after);
verify(`semantic-suffix:${index}`);
commit(before.id);
}
let reusedOccurrences = 0;
let reusedNatives = 0;
const occurrenceId = () => {
const pool = [...occurrenceIds].filter((id) => !retired.has(id));
const reused = pool.length && random() % 2 === 0 ? pool[random() % pool.length] : undefined;
if (reused !== undefined) {
reusedOccurrences++;
return reused;
}
return `occurrence-${++serial}`;
};
for (let turn = 0; turn < 60; turn++) {
const choice = random() % 8;
const current = entries.at(-1);
if (choice === 7 && current?.id) {
const candidates = entries.filter((entry) => entry.id !== undefined);
const target = candidates[random() % candidates.length];
if (target?.id) {
commit(target.id);
}
continue;
}
if (choice === 6) {
const text = textBlock();
const previous = available.findLast((part) => part.owner)?.owner;
const entry = { text, retired: false, group: previous?.group ?? previous?.native };
entries.push(entry);
available.push({ owner: entry, text });
currentNative = undefined;
capAvailability();
state.updateBuffer(runId, { delta: text });
verify(`anonymous-append:${text.length}`);
continue;
}
if (!current?.native || choice === 4 || choice === 5) {
const id = occurrenceId();
const pool = [...nativeIds].filter((native) => native !== currentNative);
const reused = pool.length && random() % 2 === 0 ? pool[random() % pool.length] : undefined;
const native = reused ?? `item-${++serial}`;
reusedNatives += Number(reused !== undefined);
const replacing = choice === 5;
if (replacing && !occurrenceIds.has(id) && random() % 2 === 0) {
commit(id);
}
const entry = { id, native, text: textBlock(), retired: retired.has(id) };
entries = replacing ? [entry] : [...entries, entry];
emit(entry, replacing, replacing);
verify(`${replacing ? "cross-item-replace" : "item-change"}:${id}:${entry.text.length}`);
continue;
}
if (choice === 3) {
const entry = {
id: occurrenceId(),
native: current.native,
text: textBlock(),
retired: false,
};
entries.push(entry);
emit(entry);
verify(`new-occurrence:${entry.id}:${entry.text.length}`);
continue;
}
if (choice === 0) {
const text = textBlock();
current.text += text;
emit(current);
verify(`append:${current.id}:${text.length}`);
} else {
const kind = random() % 4;
current.text =
kind === 0
? ""
: kind === 1
? current.text.slice(0, Math.floor(current.text.length / 2))
: kind === 2
? textBlock()
: current.text + textBlock();
emit(current, true);
verify(`same-item-replace:${kind}:${current.id}:${current.text.length}`);
}
}
expect(reusedOccurrences, `seed=${seed} never reused an occurrence`).toBeGreaterThan(0);
expect(reusedNatives, `seed=${seed} never reused a native item`).toBeGreaterThan(0);
expect(crossedCap, `seed=${seed} never crossed the real cap`).toBe(true);
expect(shrankAfterEviction, `seed=${seed} never shrank after eviction`).toBe(true);
if (firstMismatch) {
failures.push(`${firstMismatch} (first of ${mismatchCount} mismatches in ${step} steps)`);
}
}
if (failures.length) {
throw new Error(failures.join("\n"));
}
});
src/gateway/server-chat-state.test.ts
+4
View File
@@ -151,6 +151,8 @@ describe("createChatRunState", () => {
data: { kind: "preamble", progressText: "Inspecting" },
});
state.updateBuffer("run-1", { itemId: "reused", text: "Saved." });
state.retireBuffer("run-1", ["reused"]);
state.clearRun("run-1");
expect(state.registry.peek("run-1")?.clientRunId).toBe("client-1");
@@ -161,6 +163,8 @@ describe("createChatRunState", () => {
abortMarker: expect.any(Object),
toolRecipient: expect.any(Object),
});
state.updateBuffer("run-1", { itemId: "reused", text: "New reply." });
expect(state.resolveBuffer("run-1").text).toBe("New reply.");
});
it.each(["full", "summary"] as const)(
src/gateway/server-chat-state.ts
+84 -118
View File
@@ -1,13 +1,12 @@
import type { AgentPlanStep } from "../channels/streaming.js";
import type { AgentEventPayload } from "../infra/agent-events.js";
import { mergeAssistantText, type AssistantTextSnapshot } from "./agent-event-assistant-text.js";
import type { AgentEventPayload, AgentAssistantSourceReceipt } from "../infra/agent-events.js";
import type { ChatCanvasBlock } from "./chat-display-projection.canvas.js";
import {
capLiveAssistantText,
createLiveAssistantTextProjection,
normalizeLiveAssistantBufferedText,
projectLiveAssistantBufferedText,
} from "./live-chat-projector.js";
import * as assistantText from "./server-chat-buffer.js";
import type { ChatRunProgressSnapshot } from "./server-chat-progress-snapshot.js";
import { updateChatRunProgressSnapshot } from "./server-chat-progress-snapshot.js";
import {
@@ -87,37 +86,20 @@ type PendingLiveTextFlush = {
flush: () => void;
};
type LiveDisplayState = {
itemStartOffset?: number;
projector: ReturnType<typeof createLiveAssistantTextProjection>;
current: ReturnType<ReturnType<typeof createLiveAssistantTextProjection>["replace"]>;
pendingRawDelta?: string | null;
reset?: boolean;
unsentDelta: string | null;
sentText?: string;
};
type LiveDisplayState = NonNullable<assistantText.ChatRunBufferState["display"]>;
type ScopedLiveAssistantProjection = ReturnType<typeof projectLiveAssistantBufferedText> & {
itemId?: string;
itemStartOffset?: number;
};
type ChatRunRecord = {
export type ChatRunRecord = assistantText.ChatRunBufferState & {
lastActivityAt: number;
registrations?: ChatRunEntry[];
rawBuffer?: string;
buffer?: string;
bufferIsCurrent?: () => boolean;
/** Retire queued connection snapshots when this buffering generation is cleared. */
liveTextGroup?: AbortController;
liveTextEpoch?: object;
display?: LiveDisplayState;
planSnapshot?: ChatRunPlanSnapshot;
progressSnapshot?: ChatRunProgressSnapshot;
canvasBlocks?: ChatCanvasBlock[];
deltaSentAt?: number;
assistantScope?: AssistantTextSnapshot["scope"];
managedMediaUrls?: Set<string>;
agentText?: Partial<
Record<"assistant" | "thinking" | "preamble" | "answer_candidate", ChatRunAgentTextState>
>;
@@ -234,6 +216,8 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
return;
}
delete record.rawBuffer;
delete record.rawOffset;
delete record.assistantItems;
delete record.buffer;
delete record.bufferIsCurrent;
record.liveTextGroup?.abort();
@@ -245,6 +229,8 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
delete record.canvasBlocks;
delete record.deltaSentAt;
delete record.assistantScope;
delete record.assistantScopeOffset;
delete record.assistantOccurrenceId;
delete record.managedMediaUrls;
clearPendingLiveTextFlushes(record);
delete record.agentText;
@@ -259,91 +245,62 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
store.runs.clear();
};
const updateBuffer = (runId: string, input: Parameters<typeof mergeAssistantText>[1]) => {
const record = store.getOrCreate(runId);
const display = record.display;
if (input.managedMediaUrls?.length) {
const urls = (record.managedMediaUrls ??= new Set<string>());
const previousSize = urls.size;
input.managedMediaUrls.forEach((url) => urls.add(url));
if (display && urls.size !== previousSize) {
display.reset = true;
delete display.itemStartOffset;
}
}
const previousScope = record.assistantScope;
const snapshot = mergeAssistantText(
{ text: record.rawBuffer ?? "", scope: previousScope },
input,
"live",
);
if (display && (snapshot.scope !== previousScope || input.replace === true)) {
delete display.itemStartOffset;
}
record.assistantScope = snapshot.scope;
const text = capLiveAssistantText(snapshot);
if (display && text.length !== snapshot.text.length) {
delete display.itemStartOffset;
}
record.rawBuffer = text;
if (display) {
display.reset ||= text.length !== snapshot.text.length || input.replace === true;
display.pendingRawDelta =
snapshot.appendedText !== undefined && display.pendingRawDelta !== null
? (display.pendingRawDelta ?? "") + snapshot.appendedText
: null;
}
return text;
};
const resolveBuffer = (
runId: string,
options?: { final?: boolean },
): ScopedLiveAssistantProjection => {
): ReturnType<typeof projectLiveAssistantBufferedText> & { displayText?: string } => {
const record = store.runs.get(runId);
if (!record || record.bufferIsCurrent?.() === false) {
return projectLiveAssistantBufferedText("");
}
const withItemScope = <T extends { text: string; suppress: boolean }>(projected: T) => {
const scope = record.assistantScope;
if (!scope || options?.final) {
return projected;
}
const itemStartOffset =
record.display?.itemStartOffset ??
projectLiveAssistantBufferedText(
normalizeLiveAssistantBufferedText(scope.prefix + "\n".repeat(scope.separatorLength), {
managedMediaUrls: record.managedMediaUrls ? [...record.managedMediaUrls] : undefined,
}),
).text.length;
if (record.display) {
record.display.itemStartOffset = itemStartOffset;
const source = record.rawBuffer;
if (source === undefined) {
return projectLiveAssistantBufferedText(record.buffer ?? "");
}
const projectionOptions = {
...options,
managedMediaUrls: record.managedMediaUrls ? [...record.managedMediaUrls] : undefined,
};
const createProjector = () => createLiveAssistantTextProjection(projectionOptions);
const rawText = source;
const tail = assistantText.bufferVisibleText(record);
const contextEvicted = (record.rawOffset ?? 0) > 0;
const projectTail = (full: LiveDisplayState["current"]) => {
if (tail === source && !contextEvicted) {
return full;
}
// Missing source context cannot justify suppressing retained output.
const suppressed =
!contextEvicted &&
(options?.final
? projectLiveAssistantBufferedText(full.text, { suppressLeadFragments: false }).suppress
: full.suppress);
const text = suppressed
? ""
: contextEvicted
? tail
: normalizeLiveAssistantBufferedText(tail, projectionOptions);
return {
...projected,
itemId: scope.itemId,
itemStartOffset: Math.min(projected.text.length, itemStartOffset),
...full,
text,
suppress: suppressed || !text,
pendingLeadFragment: suppressed && full.pendingLeadFragment,
};
};
const rawText = record.rawBuffer;
if (rawText === undefined) {
return withItemScope(projectLiveAssistantBufferedText(record.buffer ?? ""));
}
const createProjector = () =>
createLiveAssistantTextProjection({
...options,
managedMediaUrls: record.managedMediaUrls ? [...record.managedMediaUrls] : undefined,
});
// Finalization releases ambiguous tails without changing the live projection.
// Delivery remains complete; display finalizes the same occurrence-owned tail.
if (options?.final) {
return withItemScope(createProjector().replace(rawText));
const complete = createProjector().replace(source);
return {
...complete,
...(tail !== source || contextEvicted ? { displayText: projectTail(complete).text } : {}),
};
}
let display = record.display;
if (!display) {
const projector = createProjector();
display = record.display = {
projector,
current: projector.replace(rawText),
current: projectTail(projector.replace(rawText)),
unsentDelta: null,
};
} else if (display.reset || display.pendingRawDelta !== undefined) {
@@ -360,13 +317,21 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
? rawText.slice(projector.source.length)
: null
: pendingRawDelta;
display.current =
const next = projectTail(
delta == null
? display.projector.replace(rawText)
: display.projector.append(delta, rawText);
if (display.current.delta === null) {
delete display.itemStartOffset;
}
: display.projector.append(delta, rawText),
);
// A source append is a display append only when it continues the emitted baseline.
const previous = display.current.suppress ? "" : display.current.text;
const visible = next.suppress ? "" : next.text;
next.delta =
next.delta !== null &&
visible.length === previous.length + next.delta.length &&
visible.startsWith(previous)
? next.delta
: null;
display.current = next;
display.unsentDelta =
display.unsentDelta !== null && display.current.delta !== null
? display.unsentDelta + display.current.delta
@@ -375,7 +340,7 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
delete display.reset;
}
record.buffer = display.current.text;
return withItemScope(display.current);
return display.current;
};
const takeBufferDelta = (runId: string, text: string) => {
@@ -389,29 +354,22 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
const visible = projected.suppress ? "" : projected.text;
const previous = display.sentText;
const append =
text === visible && previous !== undefined && display.unsentDelta !== null
? display.unsentDelta
: previous === undefined
? text
: text.startsWith(previous)
? text.slice(previous.length)
: null;
previous === undefined
? text
: display.unsentDelta === null
? null
: text === visible
? display.unsentDelta
: text.startsWith(previous)
? text.slice(previous.length)
: null;
display.sentText = text;
display.unsentDelta = text === visible ? "" : null;
if (append === "") {
return undefined;
}
const itemScope =
projected.itemId &&
projected.itemStartOffset !== undefined &&
projected.itemStartOffset <= text.length
? { itemId: projected.itemId, itemStartOffset: projected.itemStartOffset }
: {};
return {
deltaText: append === null ? text : append,
...itemScope,
...(append === null ? { replace: true as const } : {}),
};
display.unsentDelta = visible.startsWith(text) ? visible.slice(text.length) : null;
return append === null && (text !== "" || previous !== "")
? { deltaText: text, replace: true as const }
: append
? { deltaText: append }
: undefined;
};
return {
@@ -421,7 +379,15 @@ export function createChatRunState(isConnectionActive?: (connId: string) => bool
/** Acquire mutable state and record activity; readers use runs.get. */
getOrCreate: store.getOrCreate,
resolveBuffer,
updateBuffer,
updateBuffer: (
runId: string,
input: Parameters<typeof assistantText.updateBuffer>[1],
source?: AgentAssistantSourceReceipt,
) => assistantText.updateBuffer(store.getOrCreate(runId), input, source),
retireBuffer: (runId: string, itemIds: readonly string[]) =>
assistantText.retireBuffer(store.getOrCreate(runId), itemIds),
retireSource: (runId: string, source: AgentAssistantSourceReceipt) =>
assistantText.retireSource(store.runs.get(runId), source),
takeBufferDelta,
flushPendingText: (runId: string) => {
const record = store.runs.get(runId);
src/gateway/server-chat.commentary-history.test.ts
+439
View File
@@ -0,0 +1,439 @@
import { Value } from "typebox/value";
import { expect, it, vi } from "vitest";
import { ChatEventSchema } from "../../packages/gateway-protocol/src/schema/logs-chat.js";
import { extractText } from "../../ui/src/lib/chat/message-extract.ts";
import { handleChatGatewayEvent } from "../../ui/src/pages/chat/chat-gateway.ts";
import {
activeHistory,
createState,
type TestState,
} from "../../ui/src/pages/chat/chat-history.inflight.test-support.ts";
import { loadChatHistory } from "../../ui/src/pages/chat/chat-history.ts";
import { buildChatItems } from "../../ui/src/pages/chat/chat-thread-build.ts";
import { applySessionMessagePayload } from "../../ui/src/pages/chat/session-message-apply.ts";
import { handleAgentEvent } from "../../ui/src/pages/chat/tool-stream.ts";
import { createSubscribedSessionHarness } from "../agents/embedded-agent-subscribe.e2e-harness.js";
import { emitAgentEvent, type AgentEventPayload } from "../infra/agent-events.js";
import { projectInFlightRunSnapshot } from "./chat-inflight-snapshot.js";
import {
createAgentEventTestHarness,
widgetResult,
} from "./server-chat.agent-events.test-harness.js";
import { subscribeAgentEvents } from "./server-chat.agent-events.test-helpers.js";
function renderedItems(state: TestState) {
return buildChatItems({
paneId: "commentary-history",
sessionKey: state.sessionKey,
runId: state.chatRunId,
messages: state.chatMessages,
toolMessages: state.chatToolMessages,
streamSegments: state.chatStreamSegments,
stream: state.chatStream,
streamStartedAt: state.chatStreamStartedAt,
showToolCalls: true,
});
}
function renderedText(state: TestState) {
return renderedItems(state)
.flatMap((item) =>
item.kind === "group" && item.role !== "tool"
? item.messages.flatMap(({ message }) => extractText(message) || [])
: item.kind === "stream"
? [item.text]
: [],
)
.join("\n");
}
function renderedOrder(state: TestState) {
return [...renderedText(state).matchAll(/Step (\d+):/gu)].map((match) => Number(match[1]));
}
it("restores only the unowned tail with bounded commentary replay and continues live deltas", async () => {
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"] });
vi.setSystemTime(new Date("2026-05-09T00:00:00.000Z"));
vi.stubGlobal("window", globalThis);
const startedAtMs = Date.now();
const runId = "run-commentary-history";
const gateway = createAgentEventTestHarness();
gateway.register(runId, "main", runId);
const history = activeHistory(runId);
const live = createState(history);
live.chatRunId = runId;
const clients = [live];
gateway.broadcast.mockImplementation((event: string, payload: unknown) => {
if (event === "chat" && Value.Check(ChatEventSchema, payload)) {
for (const client of clients) {
handleChatGatewayEvent(client, payload);
}
}
});
const unsubscribe = subscribeAgentEvents(async (event) => {
if (event.runId !== runId) {
return;
}
await gateway.handler(event);
for (const client of clients) {
handleAgentEvent(client, { ...event, sessionKey: "main" });
}
});
const deliver = (stream: AgentEventPayload["stream"], data: AgentEventPayload["data"]) =>
emitAgentEvent({ runId, sessionKey: "main", stream, data });
const { emit, subscription } = createSubscribedSessionHarness({ runId });
const tool = (toolCallId: string, completed: boolean) => {
deliver("tool", { phase: "start", toolCallId, name: "exec", args: {} });
if (completed) {
deliver("tool", { phase: "result", toolCallId, name: "exec", result: "done" });
}
deliver("item", {
kind: "tool",
itemId: `tool:${toolCallId}`,
toolCallId,
name: "exec",
title: "Execute",
phase: completed ? "end" : "start",
status: completed ? "completed" : "running",
});
};
// Keep the reported 22-paragraph, 3658-character source shape synthetic.
const paragraphs = Array.from(
{ length: 22 },
(_, index) =>
`Step ${index + 1}: I will inspect the workspace.${index % 2 ? " " : "\n"}The configuration and tool results need another check before continuing. I will compare each tool output to the saved result.${index === 0 ? " Then I can move." : ""}`,
);
const textMessage = (text: string) => ({
role: "assistant",
api: "openai-completions",
content: [{ type: "text", text }],
});
const updateText = (text: string, delta: string) => {
emit({
type: "message_update",
message: textMessage(text),
assistantMessageEvent: { type: "text_delta", delta },
});
};
try {
for (const [index, text] of paragraphs.entries()) {
vi.advanceTimersByTime(250);
emit({ type: "message_start", message: textMessage("") });
updateText(text, text);
await subscription.waitForPendingEvents();
await unsubscribe.drain();
const itemId = `commentary-0-step-${index}`;
const commentary = {
...textMessage(text),
stopReason: "toolUse",
content: [
{
type: "text",
text,
textSignature: JSON.stringify({ v: 1, id: itemId, phase: "commentary" }),
},
{ type: "toolCall", id: "exec_0", name: "exec", arguments: {} },
],
};
emit({
type: "message_update",
message: commentary,
assistantMessageEvent: { type: "toolcall_start", contentIndex: 1, partial: commentary },
});
emit({ type: "message_end", message: commentary });
await subscription.waitForPendingEvents();
await unsubscribe.drain();
expect
.soft(renderedOrder(live))
.toEqual(Array.from({ length: index + 1 }, (_, step) => step + 1));
tool("exec_0", true);
if (index === 20) {
tool("tool_call:exec_0:exec:old", true);
}
if (index >= 18) {
history.messages!.push(
{
role: "assistant",
content: [{ type: "text", text }],
timestamp: Date.now(),
__openclaw: { id: `step-${index}`, seq: index * 10 + 2, runId },
openclawStreamFallback: { source: "segment", itemId, replacementText: text },
},
{
role: "assistant",
content: [commentary.content[1]],
timestamp: Date.now(),
__openclaw: { id: `step-${index}`, seq: index * 10 + 2, runId },
},
...Array.from({ length: 8 }, (_, nested) => ({
role: "toolResult",
toolCallId: nested ? `tool_call:exec_0:exec:${index * 8 + nested}` : "exec_0",
content: [{ type: "text", text: `result-${index}-${nested}` }],
timestamp: Date.now(),
__openclaw: { id: `result-${index}-${nested}`, seq: index * 10 + 3 + nested, runId },
})),
);
}
}
for (let index = 0; index < 15; index++) {
tool(`tool_call:exec_0:exec:${index}`, true);
}
tool("exec_0", false);
deliver("usage", { outputTokens: 1000 });
await unsubscribe.drain();
history.hasMore = true;
history.nextOffset = 40;
history.totalMessages = 220;
history.inFlightRun = projectInFlightRunSnapshot({
chatRunState: gateway.chatRunState,
runId,
startedAtMs,
});
expect(history.messages).toHaveLength(40);
expect(history.inFlightRun.events).toHaveLength(49);
expect(
history.inFlightRun.events?.filter((event) => event.data.kind === "preamble"),
).toMatchObject([{ data: { itemId: "commentary-0-step-21", phase: "end" } }]);
await loadChatHistory(live);
expect(live.chatRunId).toBe(runId);
expect(renderedOrder(live)).toEqual(Array.from({ length: 22 }, (_, index) => index + 1));
const normalized = renderedText(live).replace(/\s+/gu, " ");
expect(
paragraphs.map((text) => normalized.split(text.replace(/\s+/gu, " ")).length - 1),
).toEqual(paragraphs.map(() => 1));
const pending = "Step 23: Still investigating.";
vi.advanceTimersByTime(250);
emit({ type: "message_start", message: textMessage("") });
updateText(pending, pending);
await subscription.waitForPendingEvents();
await unsubscribe.drain();
history.inFlightRun = projectInFlightRunSnapshot({
chatRunState: gateway.chatRunState,
runId,
startedAtMs,
});
const restored = createState(history);
clients.push(restored);
await loadChatHistory(restored);
expect(renderedOrder(restored)).toEqual([19, 20, 21, 22, 23]);
expect(restored.chatStream).toBe(pending);
vi.advanceTimersByTime(250);
updateText(`${pending} Continuing after switch-back.`, " Continuing after switch-back.");
await subscription.waitForPendingEvents();
await unsubscribe.drain();
vi.advanceTimersByTime(250);
expect(renderedOrder(live)).toEqual(Array.from({ length: 23 }, (_, index) => index + 1));
expect(renderedOrder(restored)).toEqual([19, 20, 21, 22, 23]);
expect(restored.chatStream).toBe(`${pending} Continuing after switch-back.`);
expect(restored.chatRunId).toBe(runId);
} finally {
subscription.unsubscribe();
await unsubscribe();
await gateway.handler.dispose();
gateway.chatRunState.clear();
vi.useRealTimers();
vi.unstubAllGlobals();
}
});
it.each([
{
name: "without steer",
reload: false,
tail: "Live B.",
persistTail: false,
canvas: false,
includeSteer: false,
},
{
name: "saved after steer",
reload: false,
tail: "Live B.",
persistTail: false,
canvas: false,
steerFirst: true,
},
{ name: "live", reload: false, tail: "Live B.", persistTail: false, canvas: false },
{ name: "restored", reload: true, tail: "Live B.", persistTail: false, canvas: false },
{
name: "identical later occurrence",
reload: false,
tail: "Saved A.",
persistTail: false,
canvas: false,
},
{ name: "literal terminal tail", reload: false, tail: "[", persistTail: false, canvas: false },
{ name: "fully committed", reload: false, tail: "Live B.", persistTail: true, canvas: false },
{
name: "canvas-only display tail",
reload: false,
tail: "Live B.",
persistTail: true,
canvas: true,
},
])(
"displays each occurrence once when a run finishes ($name)",
async ({ reload, tail, persistTail, canvas, includeSteer = true, steerFirst = false }) => {
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"] });
vi.setSystemTime(new Date("2026-05-09T00:00:00.000Z"));
vi.stubGlobal("window", globalThis);
const runId = "steered-final";
const gateway = createAgentEventTestHarness();
gateway.register(runId, "main", runId);
const history = activeHistory(runId);
const original = {
role: "user",
content: "Original prompt",
timestamp: Date.now(),
__openclaw: { id: "prompt", seq: 1, idempotencyKey: `${runId}:user` },
};
const saved = {
role: "assistant",
content: [{ type: "text", text: "Saved A." }],
timestamp: Date.now() + (steerFirst ? 2 : 1),
__openclaw: { id: "saved-a", seq: steerFirst ? 3 : 2, runId, idempotencyKey: "item-a" },
};
const steer = {
role: "user",
content: "Steer prompt",
timestamp: Date.now() + (steerFirst ? 1 : 2),
__openclaw: {
id: "steer",
seq: steerFirst ? 2 : 3,
idempotencyKey: "steer:user",
steerTargetRunId: runId,
},
};
history.messages = [original];
const state = createState(history);
gateway.broadcast.mockImplementation((event: string, payload: unknown) => {
if (event === "chat" && Value.Check(ChatEventSchema, payload)) {
handleChatGatewayEvent(state, payload);
}
});
const applySteer = () =>
applySessionMessagePayload(state, { message: steer, clientRunId: runId }, true, {
kind: "live",
activeRunId: runId,
});
try {
await loadChatHistory(state);
if (includeSteer && steerFirst) {
applySteer();
}
await gateway.emit(runId, "assistant", { itemId: "item-a", text: "Saved A." });
gateway.chatRunState.flushPendingText(runId);
gateway.handler.retireTranscript({
sessionKey: "main",
message: { ...saved, idempotencyKey: "item-a" },
});
expect(gateway.chatRunState.resolveBuffer(runId).text).toBe("");
applySessionMessagePayload(state, { message: saved, runId }, true, {
kind: "live",
activeRunId: runId,
});
if (includeSteer && !steerFirst) {
applySteer();
}
await gateway.emit(runId, "assistant", { itemId: "item-b", text: tail }, { seq: 2 });
gateway.chatRunState.flushPendingText(runId);
if (reload) {
history.messages = [
original,
...(includeSteer && steerFirst ? [steer] : []),
saved,
...(includeSteer && !steerFirst ? [steer] : []),
];
history.inFlightRun = projectInFlightRunSnapshot({
chatRunState: gateway.chatRunState,
runId,
});
await loadChatHistory(state);
}
expect(renderedText(state)).toBe(
[
"Original prompt",
"Saved A.",
...(tail === "[" ? [] : [tail]),
...(includeSteer ? ["Steer prompt"] : []),
].join("\n"),
);
if (persistTail) {
const savedTail = {
role: "assistant",
content: [{ type: "text", text: tail }],
timestamp: Date.now() + 3,
__openclaw: { id: "saved-b", seq: 4, runId, idempotencyKey: "item-b" },
};
gateway.handler.retireTranscript({
sessionKey: "main",
message: { ...savedTail, idempotencyKey: "item-b" },
});
applySessionMessagePayload(state, { message: savedTail, runId }, true, {
kind: "live",
activeRunId: runId,
});
}
if (canvas) {
gateway.toolEventRecipients.add(runId, "control-ui");
await gateway.emit(
runId,
"tool",
{
phase: "result",
toolCallId: "widget",
name: "show_widget",
result: widgetResult("final-widget"),
},
{ seq: 3 },
);
handleAgentEvent(state, gateway.targetedAgent().at(-1)?.[1]);
}
await gateway.end(runId, 4);
// Delivery consumers still receive the complete terminal result.
expect(gateway.chat().at(-1)?.[1]).toMatchObject({
state: "final",
message: {
content: expect.arrayContaining([{ type: "text", text: `Saved A.\n\n${tail}` }]),
},
});
expect(renderedText(state)).toBe(
["Original prompt", "Saved A.", tail, ...(includeSteer ? ["Steer prompt"] : [])].join("\n"),
);
if (canvas) {
const widget = expect.objectContaining({
type: "canvas",
preview: expect.objectContaining({ title: "final-widget" }),
});
expect(gateway.chat().at(-1)?.[1]).toMatchObject({
message: {
content: expect.arrayContaining([widget]),
openclawDisplayContent: expect.arrayContaining([widget]),
},
});
expect(
renderedItems(state).flatMap((item) =>
item.kind === "group" ? item.messages.map(({ message }) => message) : [],
),
).toContainEqual(
expect.objectContaining({
content: expect.arrayContaining([
expect.objectContaining({
type: "canvas",
preview: expect.objectContaining({ title: "final-widget" }),
}),
]),
}),
);
}
expect(state.chatRunId).toBeNull();
} finally {
await gateway.handler.dispose();
gateway.chatRunState.clear();
vi.useRealTimers();
vi.unstubAllGlobals();
}
},
);
src/gateway/server-chat.ts
+130 -75
View File
@@ -16,13 +16,18 @@ import {
classifyAgentRunTerminalOutcome,
isDefinitiveRunLifecycle,
} from "../agents/agent-run-terminal-outcome.js";
import { getTranscriptMessageRole } from "../agents/embedded-agent-runner/message-visibility.js";
import { isActiveEmbeddedRunId } from "../agents/embedded-agent-runner/runs.js";
import { isTimeoutError, resolveFailoverReasonFromError } from "../agents/failover-error.js";
import { readToolValidationErrorSummary } from "../agents/tool-error-summary.js";
import { normalizeVerboseLevel } from "../auto-reply/thinking.js";
import { normalizeAgentPlanSteps } from "../channels/streaming.js";
import { getRuntimeConfig } from "../config/io.js";
import type { AgentEventPayload, AgentEventRuntimePayload } from "../infra/agent-events.js";
import {
readAgentAssistantSource,
type AgentEventPayload,
type AgentEventRuntimePayload,
} from "../infra/agent-events.js";
import { getAgentRunContext, getAgentRunContextOwnerStatus } from "../infra/agent-run-registry.js";
import { formatErrorMessage } from "../infra/errors.js";
import { boundedJsonUtf8Bytes } from "../infra/json-utf8-bytes.js";
@@ -33,6 +38,11 @@ import {
isSubagentSessionKey,
parseCronRunScopeSuffix,
} from "../sessions/session-key-utils.js";
import {
readSessionTranscriptRunId,
type InternalSessionTranscriptUpdate,
} from "../sessions/transcript-events.js";
import { ASSISTANT_DISPLAY_CONTENT_FIELD } from "../shared/assistant-display-content.js";
import { resolveAssistantEventPhase } from "../shared/chat-message-content.js";
import { setSafeTimeout } from "../utils/timer-delay.js";
import { resolveAssistantTextInput } from "./agent-event-assistant-text.js";
@@ -89,6 +99,7 @@ import { tryResolveSessionCompatibilityOwnerAgentId } from "./session-request-ag
import type { SessionRowReadView } from "./session-row-prepared-read.js";
import type { SessionRowProjection } from "./session-row-projection.js";
import { resolveSessionSubscriptionKeys } from "./session-subscription-keys.js";
import { readTranscriptMessageIdempotencyKey } from "./session-transcript-entry-message.js";
import { loadGatewaySessionEntryReadOnly } from "./session-utils.js";
import { formatForLog } from "./ws-log.js";
@@ -219,6 +230,7 @@ export type AgentEventHandlerOptions = {
};
type AgentEventHandler = ((event: AgentEventPayload) => void | Promise<void>) & {
retireTranscript: (event: InternalSessionTranscriptUpdate) => void;
dispose: () => Promise<void>;
};
@@ -676,7 +688,6 @@ export function createAgentEventHandler({
"chat",
sessionKey,
agentId,
run.assistantScope?.itemId,
opts?.controlUiVisible ?? true,
]);
const canvasBlocks = run.canvasBlocks;
@@ -728,11 +739,12 @@ export function createAgentEventHandler({
sourceRunId: string,
seq: number,
input: NonNullable<ReturnType<typeof resolveAssistantTextInput>>,
source: AgentEventRuntimePayload["assistantSource"],
opts?: { controlUiVisible?: boolean; isCurrent?: () => boolean; isHeartbeat?: boolean },
) => {
const run = chatRunState.getOrCreate(clientRunId);
const previousRawText = run.rawBuffer ?? "";
const mergedRawText = chatRunState.updateBuffer(clientRunId, input);
const mergedRawText = chatRunState.updateBuffer(clientRunId, input, source);
if (!mergedRawText && !previousRawText) {
return;
}
@@ -742,7 +754,7 @@ export function createAgentEventHandler({
broadcastChatDelta(sessionKey, agentId, clientRunId, sourceRunId, seq, "", opts);
return;
}
if (run.deltaSentAt !== undefined) {
if (run.deltaSentAt !== undefined && !input.replace) {
scheduleLiveTextFlush(run, "chat", LIVE_TEXT_PACING_MS - (now - run.deltaSentAt), () => {
if (run.bufferIsCurrent?.() === false) {
chatRunState.clearRun(clientRunId);
@@ -773,29 +785,6 @@ export function createAgentEventHandler({
broadcastChatDelta(sessionKey, agentId, clientRunId, sourceRunId, seq, mergedText, opts);
};
const resolveBufferedChatTextState = (
clientRunId: string,
sourceRunId: string,
options?: { final?: boolean; suppressLeadFragments?: boolean; isHeartbeat?: boolean },
) => {
const bufferedText = chatRunState
.resolveBuffer(clientRunId, { final: options?.final })
.text.trim();
const normalizedHeartbeatText = normalizeHeartbeatChatFinalText({
runId: clientRunId,
sourceRunId,
text: bufferedText,
isHeartbeat: options?.isHeartbeat,
});
const projected = projectLiveAssistantBufferedText(normalizedHeartbeatText.text.trim(), {
suppressLeadFragments: options?.suppressLeadFragments,
});
return {
text: projected.text.trim(),
shouldSuppressSilent: normalizedHeartbeatText.suppress || projected.suppress,
};
};
const flushBufferedChatDeltaIfNeeded = (
sessionKey: string,
agentId: string | undefined,
@@ -807,21 +796,9 @@ export function createAgentEventHandler({
firstAssistantTimingEntry?: ChatRunEntry;
isHeartbeat?: boolean;
},
resolved?: { text: string; shouldSuppressSilent: boolean },
) => {
cancelPendingChatDeltaFlush(clientRunId);
const streamed = resolved
? projectLiveAssistantBufferedText(resolved.text, { suppressLeadFragments: true })
: undefined;
const { text, shouldSuppressSilent } = streamed
? {
text: streamed.text.trim(),
shouldSuppressSilent: resolved?.shouldSuppressSilent || streamed.suppress,
}
: resolveBufferedChatTextState(clientRunId, sourceRunId, {
suppressLeadFragments: true,
isHeartbeat: opts?.isHeartbeat,
});
const { text, suppress } = chatRunState.resolveBuffer(clientRunId);
const shouldSuppressHeartbeatStreaming = shouldHideHeartbeatChatOutput(
clientRunId,
sourceRunId,
@@ -833,7 +810,7 @@ export function createAgentEventHandler({
// Suppression replaces a prior visible snapshot; omission would leave the UI
// materializing stale text at a message-less final. Empty untouched runs no-op.
const mergedText = shouldSuppressSilent ? "" : text;
const mergedText = suppress ? "" : text;
broadcastChatDelta(sessionKey, agentId, clientRunId, sourceRunId, seq, mergedText, opts);
};
@@ -913,18 +890,24 @@ export function createAgentEventHandler({
isHeartbeat?: boolean;
},
) => {
const { text, shouldSuppressSilent } = resolveBufferedChatTextState(clientRunId, sourceRunId, {
final: true,
suppressLeadFragments: false,
const terminalBuffer = chatRunState.resolveBuffer(clientRunId, { final: true });
const normalizedHeartbeatText = normalizeHeartbeatChatFinalText({
runId: clientRunId,
sourceRunId,
text: terminalBuffer.text.trim(),
isHeartbeat: opts?.isHeartbeat,
});
const projected = projectLiveAssistantBufferedText(normalizedHeartbeatText.text.trim(), {
suppressLeadFragments: false,
});
const text = projected.text.trim();
const shouldSuppressSilent =
normalizedHeartbeatText.suppress || (projected.suppress && !terminalBuffer.displayText);
const hasDisplayContent = Boolean(text || terminalBuffer.displayText);
// Flush any paced delta so streaming clients receive the complete text
// before the final event.
// Only flush if the buffered text differs from the last broadcast to avoid duplicates.
flushBufferedChatDeltaIfNeeded(sessionKey, opts?.agentId, clientRunId, sourceRunId, seq, opts, {
text,
shouldSuppressSilent,
});
flushBufferedChatDeltaIfNeeded(sessionKey, opts?.agentId, clientRunId, sourceRunId, seq, opts);
const spawnedBy = resolveSpawnedBy(sessionKey);
const terminalPayload = {
runId: clientRunId,
@@ -939,6 +922,16 @@ export function createAgentEventHandler({
role: "assistant",
content: text ? [{ type: "text", text }] : [],
timestamp: Date.now(),
...(terminalBuffer.displayText === undefined
? {}
: {
[ASSISTANT_DISPLAY_CONTENT_FIELD]: appendChatCanvasBlocks(
terminalBuffer.displayText
? [{ type: "text", text: terminalBuffer.displayText }]
: [],
canvasBlocks,
),
}),
...(opts?.assistantTranscriptIdempotencyKey
? {
__openclaw: {
@@ -968,7 +961,7 @@ export function createAgentEventHandler({
...(stopReason && { stopReason }),
...(jobState === "done" && opts?.yielded ? { yielded: true as const } : {}),
message:
(text && !shouldSuppressSilent) || canvasOnly
(hasDisplayContent && !shouldSuppressSilent) || canvasOnly
? createTerminalMessage(canvasBlocks)
: undefined,
};
@@ -983,7 +976,7 @@ export function createAgentEventHandler({
const payload = {
...terminalPayload,
state: "error" as const,
...(opts?.assistantTranscriptIdempotencyKey && text && !shouldSuppressSilent
...(opts?.assistantTranscriptIdempotencyKey && hasDisplayContent && !shouldSuppressSilent
? {
message: createTerminalMessage(chatRunState.runs.get(clientRunId)?.canvasBlocks ?? []),
}
@@ -1367,6 +1360,43 @@ export function createAgentEventHandler({
});
}
}
const emitAssistantChatProjection = () => {
if (!(isControlUiVisible || hasSessionMessageSubscribers) || !sessionKey) {
return;
}
const assistantLiveChatInput = evt.assistantProjection
? resolveAssistantTextInput({ ...evt.data, ...evt.assistantProjection })
: evt.stream === "assistant"
? resolveAssistantTextInput(evt.data)
: undefined;
const suppressAssistant = shouldSuppressAssistantEventForLiveChat(evt.data);
if (
!isAborted &&
assistantLiveChatInput &&
(!suppressAssistant || assistantLiveChatInput.itemId)
) {
emitChatDelta(
sessionKey,
sessionAgentId,
clientRunId,
evt.runId,
evt.seq,
suppressAssistant
? { ...assistantLiveChatInput, text: "", delta: "" }
: assistantLiveChatInput,
evt.assistantSource,
{
controlUiVisible: isControlUiVisible,
isCurrent,
isHeartbeat: heartbeatPolicy,
},
);
}
};
if (isItemEvent) {
// Retract reclassified live text before publishing its commentary item.
emitAssistantChatProjection();
}
if (isToolEvent) {
const toolPhase = typeof evt.data?.phase === "string" ? evt.data.phase : "";
if (toolPhase === "start") {
@@ -1554,30 +1584,10 @@ export function createAgentEventHandler({
if (isControlUiVisible && isToolEvent && !suppressHeartbeatToolEvents) {
sendNodeToolPayload(evt, sessionKey, sessionAgentId, agentPayload);
}
const assistantLiveChatInput =
evt.stream === "assistant" ? resolveAssistantTextInput(evt.data) : undefined;
const suppressAssistant = shouldSuppressAssistantEventForLiveChat(evt.data);
if (
!isAborted &&
assistantLiveChatInput &&
(!suppressAssistant || assistantLiveChatInput.itemId)
) {
emitChatDelta(
sessionKey,
sessionAgentId,
clientRunId,
evt.runId,
evt.seq,
suppressAssistant
? { ...assistantLiveChatInput, text: "", delta: "" }
: assistantLiveChatInput,
{
controlUiVisible: isControlUiVisible,
isCurrent,
isHeartbeat: heartbeatPolicy,
},
);
}
}
if (!isItemEvent) {
// Dual subscribers must receive canonical assistant text before its derived chat projection.
emitAssistantChatProjection();
}
if (lifecyclePhase === "error") {
@@ -1630,7 +1640,7 @@ export function createAgentEventHandler({
}
};
return createAgentEventAdmission({
const handler = createAgentEventAdmission({
handleEvent,
resolveEventSession,
isCurrent: (evt) =>
@@ -1643,5 +1653,50 @@ export function createAgentEventHandler({
pendingTerminalLifecycleErrors.clear();
},
});
return Object.assign(handler, {
retireTranscript: (event: InternalSessionTranscriptUpdate) => {
const sourceRunId = readSessionTranscriptRunId(event.message);
if (!sourceRunId || getTranscriptMessageRole(event.message) !== "assistant") {
return;
}
const link = chatRunState.registry.peek(sourceRunId);
const clientRunId = link?.clientRunId ?? sourceRunId;
const context = getAgentRunContext(sourceRunId);
const sessionKey = link?.sessionKey ?? context?.sessionKey;
const source = readAgentAssistantSource(event.message);
const mirrorKey = readTranscriptMessageIdempotencyKey(event.message);
const itemIds = [...(event.assistantItemIds ?? []), ...(mirrorKey ? [mirrorKey] : [])];
if (
(!source && itemIds.length === 0) ||
!sessionKey ||
sessionKey !== event.sessionKey ||
(context?.sessionId && context.sessionId !== event.sessionId)
) {
return;
}
const run = chatRunState.getOrCreate(clientRunId);
if (
run.bufferIsCurrent?.() === false ||
isChatAbortMarkerCurrent(run.abortMarker, link) ||
!(source
? chatRunState.retireSource(clientRunId, source)
: chatRunState.retireBuffer(clientRunId, itemIds))
) {
return;
}
run.liveTextEpoch = {};
flushBufferedChatDeltaIfNeeded(
sessionKey,
link?.agentId ?? context?.agentId,
clientRunId,
sourceRunId,
agentRunSeq.get(sourceRunId) ?? 0,
{
controlUiVisible: context?.isControlUiVisible,
isHeartbeat: context?.isHeartbeat,
},
);
},
});
}
/* oxlint-disable max-lines -- TODO: split this grandfathered oversized file. */
src/gateway/server-chat.wire.test.ts
+548 -10
View File
@@ -1,11 +1,21 @@
import { EventEmitter } from "node:events";
import { afterEach, expect, it, vi } from "vitest";
import path from "node:path";
import { afterAll, afterEach, expect, it, vi } from "vitest";
import { GATEWAY_CLIENT_CAPS } from "../../packages/gateway-protocol/src/client-info.js";
import { buildPreparedCliRunContext } from "../agents/cli-runner.test-helpers.js";
import { persistCliAssistantTranscript } from "../agents/cli-runner/cli-run-transcript.js";
import { createCliEventHandlers } from "../agents/cli-runner/execute-events.js";
import { createCliToolTracking } from "../agents/cli-runner/execute-tool-tracking.js";
import { SILENT_REPLY_TOKEN } from "../auto-reply/tokens.js";
import { upsertSessionEntryCore } from "../config/sessions/session-accessor.sqlite-entry.js";
import { type AgentEventRuntimePayload, resetAgentEventsForTest } from "../infra/agent-events.js";
import { onInternalSessionTranscriptUpdate } from "../sessions/transcript-events.js";
import { readAssistantDisplayContent } from "../shared/assistant-display-content.js";
import { useSessionStoreTempDirs } from "../test-utils/session-state-cleanup.js";
import { createAssistantTextStream } from "./agent-event-assistant-text.js";
import { abortChatRunById, registerChatAbortController } from "./chat-abort.js";
import { projectInFlightRunSnapshot } from "./chat-inflight-snapshot.js";
import { capLiveAssistantText } from "./live-chat-projector.js";
import { createGatewayBroadcaster } from "./server-broadcast.js";
import {
emitAgentEvent,
@@ -22,6 +32,8 @@ import { broadcastChatFinal } from "./server-methods/chat-broadcast.js";
import { GatewayClientRegistry } from "./server/client-registry.js";
import type { GatewayWsClient } from "./server/ws-types.js";
const sessionDirs = useSessionStoreTempDirs(afterAll, "openclaw-chat-wire-cli-");
function createHarness(
audience: Omit<Parameters<typeof createGatewayBroadcaster>[0], "clients"> = {},
) {
@@ -98,8 +110,6 @@ function connect(
data?: { text?: string; delta?: string };
state?: string;
deltaText?: string;
itemId?: string;
itemStartOffset?: number;
};
}> = [];
const socket = Object.assign(new EventEmitter(), {
@@ -141,8 +151,459 @@ const chatDeltaText = (frames: Frames) =>
.map((payload) => payload.deltaText)
.join("");
function expectCapturedTail(actual: string, tail: string, prefix: string) {
if (capLiveAssistantText({ text: prefix }).length === prefix.length) {
expect(actual).toBe(tail);
return;
}
// Cumulative snapshots may correct evicted bytes, so capped multi-occurrence
// text can retain a committed prefix; every unsaved character must survive.
expect(actual.endsWith(tail)).toBe(true);
expect(prefix.endsWith(actual.slice(0, -tail.length))).toBe(true);
expect(actual.length).toBeLessThanOrEqual(capLiveAssistantText({ text: prefix + tail }).length);
}
afterEach(() => vi.useRealTimers());
it("replaces the wire baseline when a cumulative snapshot restores capped context", () => {
const state = createChatRunState();
const runId = "restored-context";
state.updateBuffer(runId, { itemId: "a", text: "A".repeat(400_000) });
state.updateBuffer(runId, { itemId: "b", text: "A".repeat(100_002) });
state.updateBuffer(runId, { itemId: "b", text: "", replace: true });
const baseline = state.resolveBuffer(runId).text;
expect(baseline.length).toBe(399_996);
state.takeBufferDelta(runId, baseline);
state.updateBuffer(runId, { text: "A".repeat(400_001) });
const restored = state.resolveBuffer(runId).text;
const frame = state.takeBufferDelta(runId, restored);
const wire = frame?.replace ? frame.deltaText : baseline + (frame?.deltaText ?? "");
expect(wire.length).toBe(restored.length);
expect(wire === restored).toBe(true);
});
it("restores the wire baseline when an anonymous snapshot invalidates retired ownership", async () => {
await using harness = createHarness();
const { frames } = connect(harness.clients, "viewer", undefined, [
GATEWAY_CLIENT_CAPS.CHAT_ONLY_ASSISTANT_TEXT,
]);
const runId = "anonymous-reset";
const sessionKey = "agent:main:anonymous-reset";
const emit = harness.registerRun(runId, sessionKey);
await emit(1, "assistant", { itemId: "saved", text: "Saved." });
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
assistantItemIds: ["saved"],
message: {
role: "assistant",
content: [{ type: "text", text: "Saved." }],
__openclaw: { runId },
},
});
expect(payloads(frames, "chat").at(-1)).toMatchObject({ deltaText: "", replace: true });
await emit(2, "assistant", { text: "Saved. More." });
vi.advanceTimersByTime(75);
expect(payloads(frames, "chat").at(-1)).toMatchObject({
deltaText: "Saved. More.",
});
});
it.each([false, true])(
"keeps literal control tails through the terminal wire flush (capped=%s)",
async (capped) => {
await using harness = createHarness();
const { frames } = connect(harness.clients, "viewer", undefined, [
GATEWAY_CLIENT_CAPS.CHAT_ONLY_ASSISTANT_TEXT,
]);
const runId = "literal-control";
const sessionKey = "agent:main:literal-control";
const emit = harness.registerRun(runId, sessionKey);
const prefix = "The token is " + (capped ? " ".repeat(500_000) : "");
const tail = `${SILENT_REPLY_TOKEN}.`;
await emit(1, "assistant", { itemId: "native", occurrenceId: "a", text: prefix });
await emit(2, "assistant", { itemId: "native", occurrenceId: "b", text: prefix + tail });
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
assistantItemIds: ["a"],
message: {
role: "assistant",
content: [{ type: "text", text: prefix }],
__openclaw: { runId },
},
});
const display = harness.chatRunState.resolveBuffer(runId).text;
expect(display.trim()).toBe(tail);
await emitLifecycleEnd(harness.handler, runId, 3);
const delta = payloads(frames, "chat").findLast((payload) => payload.state === "delta");
expect
.soft(
readAssistantDisplayContent(delta?.message)[0]?.text === display,
"pending flush preserves the display tail",
)
.toBe(true);
const final = payloads(frames, "chat").at(-1);
expect(final?.state).toBe("final");
expect(
readAssistantDisplayContent(final?.message)[0]?.text === display,
"terminal frame retains display-only content",
).toBe(true);
},
);
it.each(["Hello new", ""])(
"keeps committed native replacement %j out of the live snapshot",
async (text) => {
await using harness = createHarness();
const sessionKey = "agent:main:native-receipt";
const emit = harness.registerRun("native-receipt", sessionKey);
await emit(1, "assistant", { itemId: "partial", text: "Hello" });
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
assistantItemIds: ["partial"],
message: {
role: "assistant",
content: [{ type: "text", text }],
idempotencyKey: "terminal-receipt",
__openclaw: { runId: "native-receipt" },
},
});
expect(harness.chatRunState.resolveBuffer("native-receipt").text).toBe("");
await emit(2, "assistant", {
itemId: "terminal-receipt",
text,
replace: true,
replaceable: true,
});
expect(harness.chatRunState.resolveBuffer("native-receipt").text).toBe("");
},
);
it("preserves an unfinished native replacement when an earlier matching prefix persists", async () => {
await using harness = createHarness();
const sessionKey = "agent:main:native-steer";
const emit = harness.registerRun("native-steer", sessionKey);
await emit(1, "assistant", { itemId: "completed", text: "Same.", replaceable: true });
await emit(2, "assistant", {
itemId: "unfinished",
text: "Same. tail",
replace: true,
replaceable: true,
});
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
assistantItemIds: ["completed"],
message: {
role: "assistant",
content: [{ type: "text", text: "Same." }],
idempotencyKey: "codex-app-server:thread:turn:assistant:completed",
__openclaw: { runId: "native-steer" },
},
});
expect(harness.chatRunState.resolveBuffer("native-steer").text).toBe("Same. tail");
});
it.each([
{ name: "short", prefix: "A", preceding: false },
{ name: "capped", prefix: "A".repeat(600_000), preceding: false },
{ name: "capped after another item", prefix: "A".repeat(600_000), preceding: true },
])(
"retires a captured occurrence while the same native item continues ($name)",
async ({ prefix, preceding }) => {
await using harness = createHarness();
const sessionKey = "agent:main:occurrence-tail";
const runId = "occurrence-tail";
const wireEmit = harness.registerRun(runId, sessionKey);
const rawStream = createAssistantTextStream(false);
const emit = async (seq: number, occurrenceId: string, text: string, delta: string) => {
const data = { itemId: "answer", occurrenceId, text, delta };
rawStream.update(data);
await wireEmit(seq, "assistant", data);
};
if (preceding) {
const data = { itemId: "earlier", text: "Earlier." };
rawStream.update(data);
await wireEmit(0, "assistant", data);
}
await emit(1, "captured", prefix, prefix);
await emit(2, "tail", `${prefix}B`, "B");
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
assistantItemIds: [...(preceding ? ["earlier"] : []), "captured"],
message: {
role: "assistant",
content: [{ type: "text", text: prefix }],
__openclaw: { runId },
},
});
const tail = projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text;
expectCapturedTail(tail, "B", prefix);
expect(harness.chatRunState.resolveBuffer(runId, { final: true }).text).toBe(
`${prefix}B`.slice(-500_000),
);
await emit(3, "tail", `${prefix}BC`, "C");
expectCapturedTail(
projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text,
"BC",
prefix,
);
expect(rawStream.streamedText).toBe(`${preceding ? "Earlier.\n\n" : ""}${prefix}BC`);
expect(harness.chatRunState.resolveBuffer(runId, { final: true }).text).toBe(
`${prefix}BC`.slice(-500_000),
);
},
);
it.each([
{ name: "plain", saved: "A", preceding: false },
{ name: "capped", saved: "A".repeat(600_000), preceding: false },
{ name: "earlier committed item", saved: "A", preceding: true },
])(
"preserves the next answer after clearing a native item ($name)",
async ({ saved, preceding }) => {
await using harness = createHarness();
const { frames } = connect(harness.clients, "control-ui");
const runId = "cleared-native-item";
const sessionKey = "agent:main:cleared-native-item";
const emit = harness.registerRun(runId, sessionKey);
const commit = (itemId: string, text: string) =>
harness.handler.retireTranscript({
sessionKey,
assistantItemIds: [itemId],
message: {
role: "assistant",
content: [{ type: "text", text }],
__openclaw: { runId },
},
});
if (preceding) {
await emit(0, "assistant", { itemId: "earlier", text: "Earlier." });
commit("earlier", "Earlier.");
}
await emit(1, "assistant", { itemId: "native", occurrenceId: "saved", text: saved });
commit("saved", saved);
await emit(2, "assistant", { itemId: "native", occurrenceId: "pending", text: `${saved}B` });
expectCapturedTail(
projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text,
"B",
saved,
);
await emit(3, "assistant", {
itemId: "native",
occurrenceId: "pending",
text: "",
replace: true,
});
expect(projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text).toBe("");
await emit(4, "assistant", { itemId: "answer", text: "Done" });
commit("saved", saved);
harness.chatRunState.flushPendingText(runId);
expect(projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text).toBe(
"Done",
);
expect(payloads(frames, "chat").at(-1)).toMatchObject({ deltaText: "Done" });
await emitLifecycleEnd(harness.handler, runId, 5);
expect(payloads(frames, "chat").at(-1)).toMatchObject({
state: "final",
message: { content: [{ type: "text", text: `${preceding ? "Earlier.\n\n" : ""}Done` }] },
});
},
);
it.each([
{ saved: "AAAA", replacement: "X" },
{ saved: "😀", replacement: "😁" },
])(
"keeps nonempty native replacement $replacement and the next item outside retired positions",
({ saved, replacement }) => {
const state = createChatRunState();
const runId = "nonempty-replacement";
state.updateBuffer(runId, { itemId: "native", occurrenceId: "saved", text: saved });
state.retireBuffer(runId, ["saved"]);
state.updateBuffer(runId, { itemId: "native", occurrenceId: "pending", text: `${saved}B` });
state.updateBuffer(runId, {
itemId: "native",
occurrenceId: "pending",
text: replacement,
replace: true,
});
expect.soft(state.resolveBuffer(runId).text).toBe(replacement);
state.updateBuffer(runId, { itemId: "next", text: "Done" });
expect(projectInFlightRunSnapshot({ chatRunState: state, runId }).text).toBe(
`${replacement}\n\nDone`,
);
const final = state.resolveBuffer(runId, { final: true });
expect(final.displayText ?? final.text).toBe(`${replacement}\n\nDone`);
},
);
it.each([
{
name: "fully evicted occurrence",
itemId: "native",
savedSize: 100_000,
tailSize: 500_000,
correctPrefix: false,
replace: true,
},
{
name: "matching retained suffix",
itemId: "native",
savedSize: 600_000,
tailSize: 100_000,
correctPrefix: true,
replace: true,
},
{
name: "unflagged cumulative snapshot",
itemId: "native",
savedSize: 600_000,
tailSize: 100_000,
correctPrefix: true,
replace: false,
},
{
name: "unscoped cumulative snapshot",
itemId: undefined,
savedSize: 600_000,
tailSize: 100_000,
correctPrefix: true,
replace: false,
},
])(
"preserves unsaved text after an ambiguous replacement ($name)",
({ itemId, savedSize, tailSize, correctPrefix, replace }) => {
const state = createChatRunState();
const runId = "evicted-retirement";
const saved = "a".repeat(savedSize);
const pending = "b".repeat(tailSize);
state.updateBuffer(runId, { itemId, occurrenceId: "saved", text: saved });
state.updateBuffer(runId, { itemId, occurrenceId: "pending", text: saved + pending });
state.retireBuffer(runId, ["saved"]);
const shorter = replace ? pending.slice(0, -50_000) : pending;
const replacement = (correctPrefix ? "x" + saved.slice(1) : saved) + shorter;
state.updateBuffer(runId, {
itemId,
occurrenceId: "pending",
text: replacement,
replace,
});
const available = capLiveAssistantText({ text: replacement });
// A correction at the evicted first character makes all replacement bytes
// unsaved even though the retained suffix matches the old committed occurrence.
const unsaved = correctPrefix ? available : shorter;
const availableSaved = available.slice(0, -unsaved.length);
const final = state.resolveBuffer(runId, { final: true });
const views: Array<[string, string]> = [
["live", state.resolveBuffer(runId).text],
["snapshot", projectInFlightRunSnapshot({ chatRunState: state, runId }).text],
["display-final", final.displayText ?? final.text],
];
for (const [view, actual] of views) {
expect(actual.endsWith(unsaved), `${view} retains every unsaved character`).toBe(true);
expect(actual.length, `${view} respects the display cap`).toBeLessThanOrEqual(
available.length,
);
const prefix = actual.slice(0, -unsaved.length);
expect(
availableSaved.endsWith(prefix),
`${view} only duplicates available committed text`,
).toBe(true);
}
},
);
it("keeps unidentified third-party text live through unrelated commits", async () => {
await using harness = createHarness();
const { frames } = connect(harness.clients, "existing", undefined, [
GATEWAY_CLIENT_CAPS.CHAT_ONLY_ASSISTANT_TEXT,
]);
const sessionKey = "agent:main:unkeyed-replacement";
const runId = "unkeyed-replacement";
const emit = harness.registerRun(runId, sessionKey);
const persist = (messageSeq: number) =>
harness.handler.retireTranscript({
sessionKey,
messageSeq,
message: {
role: "assistant",
content: [{ type: "text", text: "Saved." }],
__openclaw: { runId },
},
});
await emit(1, "assistant", { text: "Draft" });
persist(2);
const replacement = "Saved.\n\nNew answer";
await emit(2, "assistant", { text: replacement, replace: true });
expect(projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text).toBe(
replacement,
);
expect(payloads(frames, "chat").at(-1)).toMatchObject({
replace: true,
deltaText: replacement,
});
await emit(3, "assistant", { delta: " More." });
vi.advanceTimersByTime(75);
expect(payloads(frames, "chat").at(-1)).toMatchObject({ deltaText: " More." });
persist(3);
expect(projectInFlightRunSnapshot({ chatRunState: harness.chatRunState, runId }).text).toBe(
"Saved.\n\nNew answer More.",
);
});
it.each(["Tail.", " const value = 1;"])(
"replaces committed live text and continues tail %j for existing and returning clients",
async (tail) => {
await using harness = createHarness({ canReceiveSessionEvent: () => true });
const { frames } = connect(harness.clients, "existing", undefined, [
GATEWAY_CLIENT_CAPS.CHAT_ONLY_ASSISTANT_TEXT,
]);
const sessionKey = "agent:main:tail";
const emit = harness.registerRun("tail-run", sessionKey);
await emit(1, "assistant", { itemId: "saved", text: "Saved.", delta: "Saved." });
await emit(2, "assistant", { itemId: "tail", text: tail, delta: tail });
harness.handler.retireTranscript({
sessionKey,
messageSeq: 2,
message: {
role: "assistant",
idempotencyKey: "saved",
content: [{ type: "text", text: "Saved." }],
__openclaw: { runId: "tail-run" },
},
});
expect(payloads(frames, "chat").at(-1)).toMatchObject({
replace: true,
deltaText: tail,
message: { content: [{ type: "text", text: tail }] },
});
const { frames: returning } = connect(harness.clients, "returning", undefined, [
GATEWAY_CLIENT_CAPS.CHAT_ONLY_ASSISTANT_TEXT,
]);
expect(harness.chatRunState.resolveBuffer("tail-run").text).toBe(tail);
await emit(3, "assistant", { itemId: "tail", text: `${tail} More.`, delta: " More." });
vi.advanceTimersByTime(75);
expect(payloads(frames, "chat").at(-1)).toMatchObject({ deltaText: " More." });
expect(payloads(returning, "chat").at(-1)).toMatchObject({
message: { content: [{ type: "text", text: `${tail} More.` }] },
});
await emitLifecycleEnd(harness.handler, "tail-run", 4);
expect(payloads(frames, "chat").at(-1)).toMatchObject({
state: "final",
message: { content: [{ type: "text", text: `Saved.\n\n${tail} More.` }] },
});
expect(payloads(frames, "chat").findLast((payload) => payload.state === "delta")).toMatchObject(
{
deltaText: " More.",
},
);
},
);
it("keeps non-text agent events for chat-only clients", async () => {
const onBroadcast = vi.fn();
await using harness = createHarness({ onBroadcast, canReceiveSessionEvent: () => true });
@@ -216,8 +677,6 @@ it("sends append-only wire text while retaining snapshots for observers and late
expect.objectContaining({
message: expect.any(Object),
deltaText: "Hello",
itemId: "answer",
itemStartOffset: 0,
}),
expect.not.objectContaining({ message: expect.anything() }),
]);
@@ -244,16 +703,34 @@ it("sends append-only wire text while retaining snapshots for observers and late
expect(payloads(frames, "agent").at(-1)?.data?.text).toBe("Reset!");
await emit(6, "Other", "Other", undefined, "other");
expect(payloads(frames, "chat").at(-1)).toMatchObject({
itemId: "other",
itemStartOffset: "Reset!\n\n".length,
deltaText: "\n\nOther",
});
expect(projectInFlightRunSnapshot({ chatRunState, runId: "wire-run" }).text).toBe(
"Reset!\n\nOther",
);
await emit(7, "Reset! again", " again");
expect(payloads(frames, "agent").at(-1)?.data?.text).toBe("Reset! again");
handler.retireTranscript({
sessionKey: "agent:main:wire",
messageSeq: 1,
assistantItemIds: ["other"],
message: {
role: "assistant",
content: [{ type: "text", text: "Other" }],
__openclaw: { runId: "wire-run" },
},
});
expect(projectInFlightRunSnapshot({ chatRunState, runId: "wire-run" }).text).toBe(
"Reset!\n\nReset! again",
);
await emitLifecycleEnd(handler, "wire-run", 8);
expect(frames.at(-1)?.payload).toMatchObject({
state: "final",
message: { content: [{ type: "text", text: "Reset!\n\nOther\n\nReset! again" }] },
});
expect(readAssistantDisplayContent(frames.at(-1)?.payload.message)).toEqual([
{ type: "text", text: "Reset!\n\nReset! again" },
]);
expect(chatOnly.map(({ event, payload }) => ({ event, payload }))).toEqual(
frames
.filter(({ event, payload }) => event !== "agent" || payload.stream !== "assistant")
@@ -311,11 +788,15 @@ it.each(["immediate", "paced", "slow"])(
while (callbacks.length) {
callbacks.shift()?.();
}
const itemId = `cli-assistant:${runId}`;
expect(payloads(frames, "agent").map((payload) => payload.data)).toEqual([
{ text: "foo", delta: "foo" },
{ itemId, text: "foo", delta: "foo" },
...(mode === "immediate"
? [{ text: replacement, delta: "bar" }, { delta: "baz" }]
: [{ text: `${replacement}baz`, delta: "barbaz" }]),
? [
{ itemId, text: replacement, delta: "bar" },
{ itemId, delta: "baz" },
]
: [{ itemId, text: `${replacement}baz`, delta: "barbaz" }]),
]);
} finally {
try {
@@ -328,6 +809,63 @@ it.each(["immediate", "paced", "slow"])(
},
);
it("retires a real CLI reply before its corrected source snapshot arrives", async () => {
await using harness = createHarness();
vi.useRealTimers();
const root = sessionDirs.make();
const runId = "cli-committed-correction";
const target = {
agentId: "main",
sessionId: "cli-session",
sessionKey: `agent:main:${runId}`,
storePath: path.join(root, "agents", "main", "agent", "openclaw-agent.sqlite"),
};
await upsertSessionEntryCore(target, { sessionId: target.sessionId, updatedAt: Date.now() });
harness.registerRun(runId, target.sessionKey);
const context = buildPreparedCliRunContext({ runId, sessionTarget: target, workspaceDir: root });
context.params.storePath = target.storePath;
context.params.persistAssistantTranscript = true;
const cli = createCliEventHandlers({
context,
toolTracking: createCliToolTracking(context),
getRunState: () => ({ failed: false, error: undefined }),
});
const stopEvents = subscribeAgentEvents((event) => {
if (event.runId === runId) {
return harness.handler(event);
}
});
const stopTranscript = onInternalSessionTranscriptUpdate(harness.handler.retireTranscript);
try {
cli.emitCliAssistantDelta({ text: "Preview", delta: "Preview" });
await stopEvents.drain();
harness.chatRunState.flushPendingText(runId);
expect(harness.chatRunState.resolveBuffer(runId).text).toBe("Preview");
const persisted = await persistCliAssistantTranscript({
runParams: context.params,
text: "Selected",
modelId: context.modelId,
stopReason: "stop",
});
expect(persisted).toMatchObject({
owned: true,
terminalAnchor: { entryId: expect.any(String) },
});
expect(harness.chatRunState.resolveBuffer(runId).text).toBe("");
cli.emitCliAssistantDelta({ text: "Selected", delta: "" });
await stopEvents.drain();
harness.chatRunState.flushPendingText(runId);
expect(harness.chatRunState.resolveBuffer(runId).text).toBe("");
expect(harness.chatRunState.resolveBuffer(runId, { final: true }).text).toBe("Selected");
} finally {
stopTranscript();
await stopEvents();
resetAgentEventsForTest({ preserveListeners: true });
}
});
it.each([true, false])(
"re-baselines after an upstream sequence gap (visible=%s)",
async (visible) => {
src/gateway/server-methods/chat-history-budget.ts
+1 -1
View File
@@ -8,7 +8,7 @@ import { readTranscriptDisplayPosition } from "../../chat/transcript-display-pos
import type { AgentHistoryActivity } from "../../infra/agent-activity-events.js";
import { jsonUtf8Bytes, jsonUtf8BytesOrInfinity } from "../../infra/json-utf8-bytes.js";
import { logLargePayload } from "../../logging/diagnostic-payload.js";
import type { InFlightRunSnapshot } from "../chat-abort.js";
import type { InFlightRunSnapshot } from "../chat-inflight-snapshot.js";
import { readChatHistoryMessageId } from "../session-history-tail.js";
export const CHAT_HISTORY_MAX_SINGLE_MESSAGE_BYTES = 128 * 1024;
src/gateway/server-methods/chat-history-recovery.ts
+1 -1
View File
@@ -3,7 +3,7 @@ import {
resolveActiveEmbeddedRunOwner,
resolveActiveEmbeddedRunHandleSessionId,
} from "../../agents/embedded-agent-runner/runs.js";
import { projectInFlightRunSnapshot } from "../chat-abort.js";
import { projectInFlightRunSnapshot } from "../chat-inflight-snapshot.js";
import type { ChatRunState } from "../server-chat-state.js";
import type { RespondFn } from "./types.js";
src/gateway/server-runtime-subscriptions.test-support.ts
+118 -1
View File
@@ -3,6 +3,7 @@ import {
bindTestChannelParticipantAdmissionEvidence,
createChannelParticipantAdmissionEvidence,
} from "../../test/helpers/channel-admission-evidence.js";
import { createDeferred } from "../../test/helpers/promise.js";
import {
enqueueExecutionIdentityContextAtAdmission,
hasExecutionIdentityAdmissionSink,
@@ -14,8 +15,15 @@ import {
recordChannelAdmissionDecision,
} from "../channels/message-access/admission-evidence.js";
import type { OpenClawConfig } from "../config/types.openclaw.js";
import { emitAgentAuditEvent, emitAgentEvent } from "../infra/agent-events.js";
import {
type AgentEventPayload,
emitAgentAuditEvent,
emitAgentEvent,
getAgentEventLifecycleGeneration,
} from "../infra/agent-events.js";
import { claimAgentRunContext } from "../infra/agent-run-registry.js";
import type { SubsystemLogger } from "../logging/subsystem.js";
import { emitSessionTranscriptUpdate } from "../sessions/transcript-events.js";
import { createTestGatewayScheduler } from "../test-utils/gateway-scheduler-clock.js";
import { registerChatAbortController, type ChatAbortControllerEntry } from "./chat-abort.js";
import {
@@ -365,3 +373,112 @@ export function registerAuditSubscriptionTests(params: {
expect(auditTestState.stopped).toBe(1);
});
}
export function registerAssistantTailSubscriptionTests({
createParams,
installHandlerFactory,
start,
}: {
createParams: ReturnType<typeof createSubscriptionTestFixture>["createParams"];
installHandlerFactory: (
factory: typeof import("./server-chat.js").createAgentEventHandler,
) => void;
start: (params: Parameters<typeof startGatewayEventSubscriptions>[0]) => void;
}): void {
it.each([
{ source: "plain", committed: ["Saved paragraph."], initial: "Saved paragraph." },
{
source: "native raw directive",
committed: ["[[reply_to_current]]Saved paragraph."],
initial: "[[reply_to_current]]Saved paragraph.",
},
{
source: "native partial before directive",
committed: ["Saved[[reply_to_current]] paragraph."],
initial: "Saved",
},
{ source: "before first bytes", committed: ["Saved paragraph."], initial: undefined },
{
source: "identical receipts before first bytes",
committed: ["Saved paragraph.", "Saved paragraph."],
initial: undefined,
},
])(
"retires $source by identity before queued continuation without session subscribers",
async ({ committed, initial }) => {
const actual = await vi.importActual<typeof import("./server-chat.js")>("./server-chat.js");
vi.useFakeTimers({ toFake: ["Date", "setTimeout", "clearTimeout"] });
const params = createParams();
const runId = "run-persisted-tail";
const sessionKey = "agent:main:main";
const sessionId = "session-persisted-tail";
const lifecycleGeneration = getAgentEventLifecycleGeneration();
const registration = registerSubscriptionChatRun(params, {
runId,
sessionId,
sessionKey,
lifecycleGeneration,
});
claimAgentRunContext(runId, { lifecycleGeneration, sessionId, sessionKey });
const delivered = createDeferred();
installHandlerFactory((options) => {
const handler = actual.createAgentEventHandler(options);
return Object.assign(async (event: AgentEventPayload) => {
await handler(event);
if (
typeof event.data.text === "string" &&
event.data.text.endsWith("Unpersisted tail.")
) {
delivered.resolve();
}
}, handler);
});
try {
start(params);
emitAgentEvent(
initial === undefined
? { runId, stream: "lifecycle", data: { phase: "start", startedAt: Date.now() } }
: {
runId,
stream: "assistant",
data: { itemId: "saved-paragraph-0", text: initial, delta: initial },
},
);
for (const [index, text] of committed.entries()) {
emitSessionTranscriptUpdate({
sessionKey,
target: { agentId: "main", sessionId, sessionKey },
messageId: `saved-paragraph-${index}`,
messageSeq: index * 2 + 2,
message: {
role: "assistant",
idempotencyKey: `saved-paragraph-${index}`,
content: [{ type: "text", text }],
__openclaw: { runId },
},
});
}
for (const [index, text] of committed.entries()) {
emitAgentEvent({
runId,
stream: "assistant",
data: { itemId: `saved-paragraph-${index}`, text },
});
}
emitAgentEvent({
runId,
stream: "assistant",
data: { itemId: "new-paragraph", text: "Unpersisted tail." },
});
await delivered.promise;
expect(params.chatRunState.resolveBuffer(runId).text.trim()).toBe("Unpersisted tail.");
expect(params.chatRunState.resolveBuffer(runId, { final: true }).text).toBe(
`${"Saved paragraph.\n\n".repeat(committed.length)}Unpersisted tail.`,
);
} finally {
params.chatRunState.clear();
registration.cleanup();
}
},
);
}
src/gateway/server-runtime-subscriptions.test.ts
+9
View File
@@ -43,6 +43,7 @@ import {
readLifecycleState,
registerSubscriptionChatRun,
registerAuditSubscriptionTests,
registerAssistantTailSubscriptionTests,
} from "./server-runtime-subscriptions.test-support.js";
import type { SessionRowProjection } from "./session-row-projection.js";
@@ -1028,6 +1029,14 @@ describe("startGatewayEventSubscriptions", () => {
await waitForFast(() => expect(getActiveGatewayRootWorkCount()).toBe(0));
});
registerAssistantTailSubscriptionTests({
createParams,
installHandlerFactory: (factory) => agentEventHandlerMocks.create.mockImplementation(factory),
start: (params) => {
unsubs = startGatewayEventSubscriptions(params);
},
});
it("broadcasts progress-card retirement without session-list subscribers", () => {
const params = createParams();
unsubs = startGatewayEventSubscriptions(params);
src/gateway/server-runtime-subscriptions.ts
+16 -3
View File
@@ -34,7 +34,10 @@ import {
onSessionIdentityMutation,
onSessionLifecycleEvent,
} from "../sessions/session-lifecycle-events.js";
import { onInternalSessionTranscriptUpdate } from "../sessions/transcript-events.js";
import {
onInternalSessionTranscriptUpdate,
type InternalSessionTranscriptUpdate,
} from "../sessions/transcript-events.js";
import { runOutsideAsyncWorkScope } from "../shared/async-work-scope.js";
import {
createLazyPromise,
@@ -646,8 +649,18 @@ export function startGatewayEventSubscriptions(params: {
const transcriptUnsub = onInternalSessionTranscriptUpdate((evt) => {
sessionActivitySummaries.handleTranscript(evt);
void dispatchEventHandler({
loadHandler: getTranscriptUpdateHandler,
// Share the agent queue so a later cumulative update cannot outrun retirement.
const agentHandler = agentEventHandlerLoader.peek();
void dispatchEventHandler<InternalSessionTranscriptUpdate>({
loadHandler: agentHandler
? () =>
agentHandler
.then(
(handler) => handler.retireTranscript(evt),
() => undefined,
)
.then(getTranscriptUpdateHandler)
: getTranscriptUpdateHandler,
event: evt,
log: params.log,
failureMessage: "Transcript update dispatch failed",
src/gateway/worker-environments/transcript-commit.kernel.ts
+24 -6
View File
@@ -207,12 +207,30 @@ export function prepareTranscriptCommit(input: TranscriptCommitInput): PreparedT
const plan = (
result: ApplyTranscriptCommitResult,
nextMessageSeq = 0,
): PreparedTranscriptCommit => ({
result,
version: snapshot.version,
nextMessageSeq,
parentId: manager.getAppendParentId(),
});
): PreparedTranscriptCommit => {
let applied = result;
if (result.ok && result.messages.length > 0) {
const activeSequences = new Map(
manager
.getBranch()
.filter((event) => event.type === "message" || event.type === "compaction")
.map((event, index) => [event.id, index + 1]),
);
applied = {
...result,
messages: result.messages.map((message) => {
const messageSeq = activeSequences.get(message.messageId);
return messageSeq === undefined ? message : { ...message, messageSeq };
}),
};
}
return {
result: applied,
version: snapshot.version,
nextMessageSeq,
parentId: manager.getAppendParentId(),
};
};
if (input.recoverPersistedBatch) {
const recovered = resolvePersistedCommitAcrossDag({
baseLeafId: input.requestedBaseLeafId,
src/gateway/worker-environments/transcript-commit.runtime.ts
+25 -7
View File
@@ -33,6 +33,7 @@ import {
} from "../../sessions/transcript-events.js";
import { captureOpenClawAgentDatabaseExecution } from "../../state/openclaw-agent-execution.js";
import { openOpenClawAgentSqliteWorkerStore } from "../../state/openclaw-agent-worker-store.js";
import { readTranscriptMessageIdempotencyKey } from "../session-transcript-entry-message.js";
import type { WorkerConnectionIdentity } from "./connection-identity.js";
import { prepareWorkerTurnTranscriptMessage } from "./placement-turn-claim-events.js";
import type {
@@ -62,6 +63,7 @@ async function applyWorkerTranscriptCommit(params: {
config: OpenClawConfig;
identity: WorkerConnectionIdentity;
messages: readonly CommittedAgentMessage[];
assistantItemIds: ReadonlyMap<string, string>;
recoverPersistedBatch: boolean;
requestedBaseLeafId: string | null;
runId: string | null;
@@ -219,7 +221,13 @@ async function applyWorkerTranscriptCommit(params: {
}
for (const message of applied.messages) {
if (!message.appended) {
const itemId =
message.message.role === "assistant"
? params.assistantItemIds.get(readTranscriptMessageIdempotencyKey(message.message) ?? "")
: undefined;
// Pending recovery can find a committed row whose original publication was lost.
// A recovered sequence certifies active-branch membership; abandoned rows stay silent.
if (!message.appended && (!itemId || message.messageSeq === undefined)) {
continue;
}
const runId = resolveTerminalAssistantTranscriptRunId(message.message, params.runId);
@@ -228,6 +236,7 @@ async function applyWorkerTranscriptCommit(params: {
message: message.message,
messageId: message.messageId,
messageSeq: message.messageSeq,
...(itemId ? { assistantItemIds: [itemId] } : {}),
...(runId ? { runId } : {}),
});
}
@@ -257,13 +266,21 @@ export async function commitWorkerTranscript(
expectedLifecycleRevision: params.sessionTarget.expectedLifecycleRevision,
expectedWriterRunId: params.sessionTarget.expectedWriterRunId,
});
// Ingress validated the closed schema; clone every admitted field before transcript redaction.
const messages = params.request.messages.map((message, index) => ({
...structuredClone(message),
idempotencyKey: `worker-commit-${sha256Base64Url(
const assistantItemIds = new Map<string, string>();
// Correlation belongs to the commit receipt, never stored or provider-visible content.
const messages = params.request.messages.map((message, index) => {
const idempotencyKey = `worker-commit-${sha256Base64Url(
[sessionId, params.request.runEpoch, params.request.seq, index].join("\0"),
)}`,
}));
)}`;
if (message.role === "assistant") {
const { itemId, ...assistant } = structuredClone(message);
if (itemId) {
assistantItemIds.set(idempotencyKey, itemId);
}
return { ...assistant, idempotencyKey };
}
return { ...structuredClone(message), idempotencyKey };
});
const requestedBaseLeafId = params.request.baseLeafId;
params.assertCurrent();
const started = await store.begin(input, params.assertCurrent);
@@ -291,6 +308,7 @@ export async function commitWorkerTranscript(
config,
identity: params.identity,
messages,
assistantItemIds,
recoverPersistedBatch: started.kind === "recover",
requestedBaseLeafId,
runId: params.identity.runId,
src/gateway/worker-environments/transcript-commit.test-support.ts
+30 -1
View File
@@ -1,4 +1,8 @@
import type { WorkerTranscriptMessage } from "../../../packages/gateway-protocol/src/schema/worker-admission.js";
import { createHash } from "node:crypto";
import type {
WorkerTranscriptCommitParams,
WorkerTranscriptMessage,
} from "../../../packages/gateway-protocol/src/schema/worker-admission.js";
import { makeTextToolResult } from "../../../test/helpers/text-tool-result.js";
import { createZeroUsageFixture } from "../../agents/test-helpers/usage-fixtures.js";
import type { OpenClawConfig } from "../../config/types.openclaw.js";
@@ -103,3 +107,28 @@ export function createTurnMessages(userText = "Inspect the workspace"): WorkerTr
makeTextToolResult("call-read-1", "read", "Workspace ready.", false, 300),
];
}
export const SESSION_ID = "session-worker-transcript";
export const RUN_EPOCH = 7;
export function createRequest(
params: {
baseLeafId?: string | null;
messages?: WorkerTranscriptMessage[];
seq?: number;
} = {},
): WorkerTranscriptCommitParams {
return {
runEpoch: RUN_EPOCH,
seq: params.seq ?? 1,
baseLeafId: params.baseLeafId ?? null,
messages: params.messages ?? createTurnMessages(),
};
}
export function messageIdempotencyKey(seq: number, index: number): string {
const digest = createHash("sha256")
.update([SESSION_ID, RUN_EPOCH, seq, index].join("\0"))
.digest("base64url");
return `worker-commit-${digest}`;
}
src/gateway/worker-environments/transcript-commit.test.ts
+80 -27
View File
@@ -1,4 +1,3 @@
import { createHash } from "node:crypto";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
@@ -9,7 +8,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import {
WorkerTranscriptCommitParamsSchema,
WorkerTranscriptMessageSchema,
type WorkerTranscriptCommitParams,
type WorkerTranscriptMessage,
} from "../../../packages/gateway-protocol/src/schema/worker-admission.js";
import { createNoisyPngBuffer } from "../../../test/helpers/image-fixtures.js";
@@ -55,42 +53,22 @@ import {
} from "./transcript-commit.kernel.js";
import {
createInterruptedCommitter,
createRequest,
createTranscriptCommitIdentity,
createTurnMessages,
messageIdempotencyKey,
PROVIDER_REPLAY,
RUN_EPOCH,
SESSION_ID,
ZERO_USAGE,
} from "./transcript-commit.test-support.js";
type WorkerTranscriptCommitter = ReturnType<typeof createWorkerTranscriptCommitter>;
const SESSION_ID = "session-worker-transcript";
const SESSION_KEY = "agent:main:worker-transcript";
const RUN_EPOCH = 7;
const IDENTITY = createTranscriptCommitIdentity(SESSION_ID, RUN_EPOCH);
function createRequest(
params: {
baseLeafId?: string | null;
messages?: WorkerTranscriptMessage[];
seq?: number;
} = {},
): WorkerTranscriptCommitParams {
return {
runEpoch: RUN_EPOCH,
seq: params.seq ?? 1,
baseLeafId: params.baseLeafId ?? null,
messages: params.messages ?? createTurnMessages(),
};
}
function messageIdempotencyKey(seq: number, index: number): string {
const digest = createHash("sha256")
.update([SESSION_ID, RUN_EPOCH, seq, index].join("\0"))
.digest("base64url");
return `worker-commit-${digest}`;
}
function requireAppendableWorkerMessage(
message: unknown,
): Parameters<SessionManager["appendMessage"]>[0] {
@@ -440,6 +418,57 @@ describe("worker transcript commit application", () => {
}
});
it("recovers an interrupted terminal write after later transcript activity", async () => {
const updates: InternalSessionTranscriptUpdate[] = [];
unsubscribe = onInternalSessionTranscriptUpdate((update) => updates.push(update));
const interruptedCommitter = createInterruptedCommitter(
() => cfg,
ledgerStore,
"simulated commit-result interruption",
);
const request = createRequest();
expectDefined(
request.messages.find((message) => message.role === "assistant"),
"assistant occurrence",
).itemId = "worker-assistant-recovered";
await expect(interruptedCommitter.commit({ ...ADMITTED_OWNER, request })).rejects.toThrow(
"simulated commit-result interruption",
);
const afterInterruption = await SessionManager.openAsync(sessionTarget);
const committedEntryIds = afterInterruption.getEntries().map((entry) => entry.id);
expect(committedEntryIds).toHaveLength(request.messages.length);
const laterLeafId = await afterInterruption.appendMessageAsync({
role: "user",
content: [{ type: "text", text: "Later local activity" }],
timestamp: 400,
});
updates.length = 0;
const replay = await committer.commit({ ...ADMITTED_OWNER, request });
expect(replay).toEqual({
ok: true,
result: {
entryIds: committedEntryIds,
newLeafId: committedEntryIds.at(-1),
},
});
const reopened = await SessionManager.openAsync(sessionTarget);
expect(reopened.getEntries()).toHaveLength(request.messages.length + 1);
expect(reopened.getLeafId()).toBe(laterLeafId);
expect(updates).toEqual([
expect.objectContaining({
assistantItemIds: ["worker-assistant-recovered"],
message: expect.objectContaining({ idempotencyKey: messageIdempotencyKey(1, 1) }),
messageId: committedEntryIds[1],
messageSeq: 2,
}),
]);
await expect(committer.commit({ ...ADMITTED_OWNER, request })).resolves.toEqual(replay);
expect(updates).toHaveLength(1);
});
it("replays an interrupted terminal write after its branch is abandoned", async () => {
cfg = { ...cfg };
const initialManager = await SessionManager.openAsync(sessionTarget);
@@ -460,6 +489,10 @@ describe("worker transcript commit application", () => {
baseLeafId,
messages: createTurnMessages("my key is sk-abcdef1234567890xyz"),
});
expectDefined(
request.messages.find((message) => message.role === "assistant"),
"assistant occurrence",
).itemId = "worker-assistant-abandoned";
await expect(interruptedCommitter.commit({ ...ADMITTED_OWNER, request })).rejects.toThrow(
"simulated off-branch terminal interruption",
@@ -493,7 +526,13 @@ describe("worker transcript commit application", () => {
timestamp: 400,
});
const updates: Parameters<Parameters<typeof onSessionTranscriptUpdate>[0]>[0][] = [];
unsubscribe = onSessionTranscriptUpdate((update) => updates.push(update));
const internalUpdates: InternalSessionTranscriptUpdate[] = [];
const offPublic = onSessionTranscriptUpdate((update) => updates.push(update));
const offInternal = onInternalSessionTranscriptUpdate((update) => internalUpdates.push(update));
unsubscribe = () => {
offPublic();
offInternal();
};
const entriesBeforeReplay = structuredClone(afterInterruption.getEntries());
const sessionEntryBeforeReplay = structuredClone(
loadSessionEntry({ agentId: "main", sessionKey: SESSION_KEY, storePath }),
@@ -545,6 +584,7 @@ describe("worker transcript commit application", () => {
}
expect(replay.result.entryIds).not.toContain(duplicatePrefixId);
expect(updates).toEqual([]);
expect(internalUpdates).toEqual([]);
});
it("rolls back the entire transcript batch when commit authority is revoked", async () => {
@@ -691,6 +731,10 @@ describe("worker transcript commit application", () => {
};
expect(Buffer.byteLength(image.data)).toBeGreaterThan(64 * 1024);
const messages = createTurnMessages(literalUserText);
expectDefined(
messages.find((message) => message.role === "assistant"),
"assistant occurrence",
).itemId = "worker-assistant-occurrence";
const toolResult = messages[2]!;
if (toolResult.role !== "toolResult") {
throw new Error("missing read result");
@@ -787,6 +831,15 @@ describe("worker transcript commit application", () => {
});
expect(updates[0]?.message).not.toHaveProperty("openclawDelivery");
expect(updates[1]?.message).not.toHaveProperty("providerReplay");
expect(updates[1]?.message).not.toHaveProperty("itemId");
expect(updates[1]).not.toHaveProperty("assistantItemIds");
expect(reopened.getEntries()[1]).not.toHaveProperty("message.itemId");
expect(internalUpdates[1]).toMatchObject({
assistantItemIds: ["worker-assistant-occurrence"],
message: { idempotencyKey: messageIdempotencyKey(1, 1) },
messageId: first.result.entryIds[1],
messageSeq: 2,
});
expect(updates[1]).not.toHaveProperty("lifecycleRevision");
expect(internalUpdates.map((update) => update.lifecycleRevision)).toEqual(
updates.map(() => "worker-original-revision"),
src/infra/agent-events.test.ts
+35 -3
View File
@@ -1,11 +1,13 @@
import { afterEach, beforeEach, expect, onTestFinished, test, vi } from "vitest";
import { sessionChanges } from "../sessions/session-row-changes.js";
import {
type AgentAssistantSourceReceipt,
type AgentEventPayload,
type AgentEventRuntimePayload,
captureAgentRunLifecycleGeneration,
emitAgentAuditEvent,
emitAgentEventIfCurrent,
emitAgentEventWithAssistantSourceIfCurrent,
emitAgentEventForOwner,
emitAgentRunOutputTokens,
getAgentEventLifecycleGeneration,
@@ -243,6 +245,12 @@ test("rejects inherited stale ownership and cannot reclaim the admitted replacem
withAgentRunLifecycleGeneration(oldGeneration, () => {
expect(captureAgentRunLifecycleGeneration("descendant")).toBe(oldGeneration);
emit("run");
expect(
emitAgentEventWithAssistantSourceIfCurrent(
{ runId: "run", stream: "assistant", data: { text: "stale" } },
{},
),
).toBe(false);
});
claimAgentRunContext("run", { sessionKey: "new-session", lifecycleGeneration });
registerAgentRunContext("run", {
@@ -345,11 +353,35 @@ test("keeps hidden routing private while preserving lifecycle persistence identi
});
const received: AgentEventRuntimePayload[] = [];
onAgentRuntimeEvent((event) => received.push(event));
emit("run", "assistant", { text: "private" }, { sessionKey: "main" });
const data = { text: "private" };
const assistantSource: AgentAssistantSourceReceipt = {};
const assistantProjection = { itemId: "item", text: "display", replace: false };
emitAgentEventWithAssistantSourceIfCurrent(
{ runId: "run", stream: "assistant", data, sessionKey: "main" },
assistantSource,
assistantProjection,
);
const native = {
runId: "run",
stream: "assistant",
data,
sessionKey: "main",
assistantSource,
assistantProjection,
};
emitAgentEventIfCurrent(native);
emit("run", "lifecycle", { phase: "start", startedAt: 1_234 }, { sessionKey: "main" });
emit("run", "lifecycle", { phase: "error" });
expect(received.map((event) => event.sessionKey)).toEqual([undefined, "main", "main"]);
const event = received[1]!;
expect(received.map((event) => event.sessionKey)).toEqual([undefined, undefined, "main", "main"]);
expect(received[0]?.assistantSource).toBe(assistantSource);
expect(received[0]?.assistantProjection).toBe(assistantProjection);
expect({ ...received[0] }).not.toHaveProperty("assistantSource");
expect({ ...received[0] }).not.toHaveProperty("assistantProjection");
expect(received[1]?.assistantSource).toBeUndefined();
expect(received[1]?.assistantProjection).toBeUndefined();
expect({ ...received[1] }).not.toHaveProperty("assistantSource");
expect({ ...received[1] }).not.toHaveProperty("assistantProjection");
const event = received[2]!;
expect(event).toMatchObject({
lifecycleGeneration: getAgentEventLifecycleGeneration(),
projectSessionLifecycle: false,
src/infra/agent-events.ts
+64 -66
View File
@@ -9,6 +9,12 @@ import {
recordAgentEventRouting,
} from "./agent-event-execution-context.js";
import { hasInvalidLifecycleStartTimestamp } from "./agent-event-lifecycle.js";
import type {
AgentAssistantProjection,
AgentAssistantSourceReceipt,
AgentEventPayload,
AgentEventRuntimePayload,
} from "./agent-events.types.js";
import { createAgentRunStaleLifecycleError } from "./agent-lifecycle-error.js";
import {
getAgentRunContext,
@@ -22,71 +28,14 @@ import {
import type { AgentRunContext, AgentRunEventState } from "./agent-run-registry.types.js";
import { recordAgentRunOutputTokens } from "./agent-run-usage.js";
/** Payload for approval requests and their later resolution events. */
export type AgentApprovalEventData = {
phase: "requested" | "resolved";
kind: "exec" | "plugin" | "unknown";
status: "pending" | "unavailable" | "approved" | "denied" | "failed";
title: string;
itemId?: string;
toolCallId?: string;
approvalId?: string;
approvalSlug?: string;
command?: string;
host?: string;
reason?: string;
scope?: "turn" | "session";
message?: string;
};
/** Stream name for agent events delivered to gateway listeners and plugin host hooks. */
export type AgentEventStream =
| "lifecycle"
| "tool"
| "assistant"
| "usage"
| "error"
| "item"
| "plan"
| "approval"
| "command_output"
| "patch"
| "compaction"
| "thinking"
| (string & {});
/** Enriched event delivered to subscribers after sequencing and context stamping. */
export type AgentEventPayload = {
runId: string;
seq: number;
stream: AgentEventStream;
ts: number;
data: Record<string, unknown>;
/** Internal, non-enumerable gateway lifecycle generation that owns this run. */
lifecycleGeneration?: string;
sessionKey?: string;
/**
* sessionId the run was bound to when it started. Lifecycle persistence uses
* this to reject terminal events from a pre-`sessions.reset` run that would
* otherwise clobber the rotated session row resolved by the shared sessionKey.
*/
sessionId?: string;
agentId?: string;
};
/** Gateway-only routing metadata stamped onto events after public input validation. */
export type AgentEventRuntimePayload = AgentEventPayload & {
readonly admitLifecyclePublication?: () => boolean;
readonly controlUiVisible?: boolean;
readonly contextClaimId?: string;
readonly deliverySessionKey?: string;
readonly mainSessionRestartRecovery?: true;
readonly projectSessionLifecycle?: boolean;
readonly projectSessionMessages?: boolean;
readonly isHeartbeat?: boolean;
readonly verboseLevel?: AgentRunContext["verboseLevel"];
readonly registeredAt?: number;
};
export type {
AgentApprovalEventData,
AgentAssistantProjection,
AgentAssistantSourceReceipt,
AgentEventPayload,
AgentEventRuntimePayload,
AgentEventStream,
} from "./agent-events.types.js";
type AgentEventListener = (evt: AgentEventRuntimePayload) => void;
type AgentEventRegistration = {
@@ -97,6 +46,7 @@ type AgentEventListeners = Map<AgentEventListener, AgentEventRegistration>;
type AgentEventState = {
runs: Map<string, AgentRunEventState>;
assistantSources?: WeakMap<object, AgentAssistantSourceReceipt>;
listeners: AgentEventListeners;
runListeners: Map<string, AgentEventListeners>;
nextListenerId: number;
@@ -127,6 +77,22 @@ function getAgentEventState(): AgentEventState {
}));
}
/** Correlate copied model frames with their occurrence without changing transcript bytes. */
export function bindAgentAssistantSource(
message: object,
source: AgentAssistantSourceReceipt,
): void {
(getAgentEventState().assistantSources ??= new WeakMap()).set(message, source);
}
export function readAgentAssistantSource(
message: unknown,
): AgentAssistantSourceReceipt | undefined {
return message && typeof message === "object"
? getAgentEventState().assistantSources?.get(message)
: undefined;
}
registerAgentRunSequenceResetHandler((runId) => {
getAgentEventState().runs.delete(runId);
});
@@ -433,6 +399,8 @@ function enrichAgentEvent(
claimId?: string,
expectedContext?: AgentRunContext,
reservedPublication?: symbol,
assistantSource?: AgentAssistantSourceReceipt,
assistantProjection?: AgentAssistantProjection,
): AgentEventRuntimePayload | undefined {
const resolved = resolveAgentEventRouting(event, claimId, expectedContext);
if (!resolved || hasInvalidLifecycleStartTimestamp(event.stream, event.data)) {
@@ -544,6 +512,10 @@ function enrichAgentEvent(
},
});
}
Object.defineProperties(enriched, {
assistantSource: { value: assistantSource, enumerable: false },
assistantProjection: { value: assistantProjection, enumerable: false },
});
if (lifecycleGeneration) {
// Persistence needs restart ownership, but agent events are also spread into
// public payloads. Keep the internal generation readable without serializing it.
@@ -618,9 +590,19 @@ function dispatchAgentEvent(
claimId?: string,
expectedContext?: AgentRunContext,
reservedPublication?: symbol,
assistantSource?: AgentAssistantSourceReceipt,
assistantProjection?: AgentAssistantProjection,
): boolean {
const state = getAgentEventState();
const enriched = enrichAgentEvent(state, event, claimId, expectedContext, reservedPublication);
const enriched = enrichAgentEvent(
state,
event,
claimId,
expectedContext,
reservedPublication,
assistantSource,
assistantProjection,
);
if (!enriched) {
return false;
}
@@ -633,6 +615,22 @@ export function emitAgentEventIfCurrent(event: Omit<AgentEventPayload, "seq" | "
return dispatchAgentEvent(event);
}
/** Bind the source occurrence only after ordinary event ownership admission. */
export function emitAgentEventWithAssistantSourceIfCurrent(
event: Omit<AgentEventPayload, "seq" | "ts">,
assistantSource: AgentAssistantSourceReceipt | undefined,
assistantProjection?: AgentAssistantProjection,
): boolean {
return dispatchAgentEvent(
event,
undefined,
undefined,
undefined,
assistantSource,
assistantProjection,
);
}
/** Adds one completed model call, returning its accepted run total for local callbacks. */
export function emitAgentRunOutputTokens(params: {
runId: string;
src/infra/agent-events.types.ts
+82
View File
@@ -0,0 +1,82 @@
import type { AgentRunContext } from "./agent-run-registry.types.js";
/** Payload for approval requests and their later resolution events. */
export type AgentApprovalEventData = {
phase: "requested" | "resolved";
kind: "exec" | "plugin" | "unknown";
status: "pending" | "unavailable" | "approved" | "denied" | "failed";
title: string;
itemId?: string;
toolCallId?: string;
approvalId?: string;
approvalSlug?: string;
command?: string;
host?: string;
reason?: string;
scope?: "turn" | "session";
message?: string;
};
/** Stream name for agent events delivered to gateway listeners and plugin host hooks. */
export type AgentEventStream =
| "lifecycle"
| "tool"
| "assistant"
| "usage"
| "error"
| "item"
| "plan"
| "approval"
| "command_output"
| "patch"
| "compaction"
| "thinking"
| (string & {});
/** Enriched event delivered to subscribers after sequencing and context stamping. */
export type AgentEventPayload = {
runId: string;
seq: number;
stream: AgentEventStream;
ts: number;
data: Record<string, unknown>;
/** Internal, non-enumerable gateway lifecycle generation that owns this run. */
lifecycleGeneration?: string;
sessionKey?: string;
/**
* sessionId the run was bound to when it started. Lifecycle persistence uses
* this to reject terminal events from a pre-`sessions.reset` run that would
* otherwise clobber the rotated session row resolved by the shared sessionKey.
*/
sessionId?: string;
agentId?: string;
};
export type AgentAssistantSourceReceipt = {
/** Stable occurrence identity for buffer scopes and worker forwarding. */
itemId?: string;
/** Undefined until commit; null is durable without an available transcript position. */
committedMessageSeq?: number | null;
};
export type AgentAssistantProjection = {
itemId: string;
text: string;
replace: boolean;
};
/** Gateway-only routing metadata stamped onto events after public input validation. */
export type AgentEventRuntimePayload = AgentEventPayload & {
readonly admitLifecyclePublication?: () => boolean;
readonly assistantSource?: AgentAssistantSourceReceipt;
readonly assistantProjection?: AgentAssistantProjection;
readonly controlUiVisible?: boolean;
readonly contextClaimId?: string;
readonly deliverySessionKey?: string;
readonly mainSessionRestartRecovery?: true;
readonly projectSessionLifecycle?: boolean;
readonly projectSessionMessages?: boolean;
readonly isHeartbeat?: boolean;
readonly verboseLevel?: AgentRunContext["verboseLevel"];
readonly registeredAt?: number;
};
src/plugin-sdk/session-transcript-runtime.ts
+3
View File
@@ -550,6 +550,9 @@ export async function appendSessionTranscriptMessagesByIdentity<TMessage>(
/**
* Publishes a transcript update by scoped transcript target.
* `update.assistantItemIds` retires exact native display occurrences only after
* their row commits. It is internal provenance, not terminal/run authorization,
* and must be omitted for independently owned commentary and async rows.
*/
export async function publishSessionTranscriptUpdateByIdentity(
params: SessionTranscriptTargetParams & { update?: TranscriptUpdatePayload },
src/plugins/wired-hooks-compaction.test.ts
+2 -1
View File
@@ -19,7 +19,8 @@ vi.mock("../plugins/hook-runner-global.js", () => ({
getGlobalHookRunner: () => hookMocks.runner,
}));
vi.mock("../infra/agent-events.js", () => ({
vi.mock(import("../infra/agent-events.js"), async (importOriginal) => ({
...(await importOriginal()),
emitAgentEvent: hookMocks.emitAgentEvent,
emitAgentEventIfCurrent: vi.fn(() => true),
getAgentEventLifecycleGeneration: () => "test-generation",
src/sessions/transcript-events.test.ts
+2
View File
@@ -109,6 +109,7 @@ describe("transcript events", () => {
},
lifecycleRevision: " committed-revision ",
messageId: "msg-1",
assistantItemIds: ["native-item"],
});
expect(internalListener).toHaveBeenCalledWith({
@@ -123,6 +124,7 @@ describe("transcript events", () => {
sessionKey: "agent:main:main",
lifecycleRevision: "committed-revision",
messageId: "msg-1",
assistantItemIds: ["native-item"],
});
expect(publicListener).toHaveBeenCalledWith({
target: {
src/sessions/transcript-events.ts
+4 -1
View File
@@ -26,11 +26,13 @@ export type InternalSessionTranscriptUpdate = {
messageId?: string;
messageSeq?: number;
runId?: string;
/** Exact native display occurrences retired by this commit; never run authority. */
assistantItemIds?: readonly string[];
};
export type SessionTranscriptUpdate = Omit<
InternalSessionTranscriptUpdate,
"sessionFile" | "lifecycleRevision" | "target"
"sessionFile" | "lifecycleRevision" | "target" | "assistantItemIds"
> & {
target: Omit<SessionTranscriptUpdateTarget, "storePath">;
};
@@ -170,6 +172,7 @@ function normalizeSessionTranscriptUpdate(
...(messageId ? { messageId } : {}),
...(messageSeq !== undefined ? { messageSeq } : {}),
...(runId ? { runId } : {}),
...(update.assistantItemIds ? { assistantItemIds: update.assistantItemIds } : {}),
};
}
src/worker/embedded-agent-live.runtime.ts
+20 -5
View File
@@ -1,3 +1,4 @@
import { randomUUID } from "node:crypto";
import type { WorkerLiveEvent } from "../../packages/gateway-protocol/src/schema/worker-admission.js";
import {
mergeAgentRunAttemptTerminal,
@@ -10,6 +11,11 @@ import { hasModelFallbackStop } from "../agents/failover-error.js";
import type { AgentMessage } from "../agents/runtime/index.js";
import type { AgentSessionEvent } from "../agents/sessions/agent-session.js";
import { parseReplyDirectives } from "../auto-reply/reply/reply-directives.js";
import {
bindAgentAssistantSource,
readAgentAssistantSource,
type AgentAssistantSourceReceipt,
} from "../infra/agent-events.js";
import { formatErrorMessage } from "../infra/errors.js";
import {
resolveAssistantMessagePhase,
@@ -193,7 +199,7 @@ export function createWorkerLiveRuntime(client: WorkerLiveClient) {
};
let streamedText = "";
let streamedPhase: AssistantPhase | undefined;
let assistantMessageIndex = 0;
let assistantSource: AgentAssistantSourceReceipt | undefined;
let streamedThinking = "";
const emitAssistantSnapshot = (message: AgentMessage, complete = false) => {
const { text, phase } = readAssistantSnapshot(message);
@@ -213,15 +219,25 @@ export function createWorkerLiveRuntime(client: WorkerLiveClient) {
...(mediaUrls?.length ? { mediaUrls } : {}),
...(replace ? { replace: true as const } : {}),
...(phase ? { phase } : {}),
// Provider signatures can arrive only at text_end. Message lifecycle,
// not those late ids, owns this cumulative snapshot's stable scope.
itemId: `assistant-${assistantMessageIndex}`,
itemId: readAgentAssistantSource(message)?.itemId,
},
});
streamedText = text;
streamedPhase = phase;
};
const handleSessionEvent = (event: AgentSessionEvent) => {
if (
(event.type === "message_start" ||
event.type === "message_update" ||
event.type === "message_end") &&
event.message.role === "assistant"
) {
if (event.type === "message_start" || !assistantSource) {
assistantSource = { itemId: randomUUID() };
}
// Persistence still needs occurrence identity after optional previews degrade.
bindAgentAssistantSource(event.message, assistantSource);
}
// Disabled previews no longer need snapshots or diagnostics, but agent_end
// still owns the terminal result deferred until the transcript is durable.
if (!previewEnabled && event.type !== "agent_end") {
@@ -232,7 +248,6 @@ export function createWorkerLiveRuntime(client: WorkerLiveClient) {
return;
}
if (event.type === "message_start" && event.message.role === "assistant") {
assistantMessageIndex += 1;
streamedText = "";
streamedPhase = undefined;
streamedThinking = "";
src/worker/embedded-agent-transcript.runtime.ts
+5
View File
@@ -4,6 +4,7 @@ import type { WorkerInferenceContext } from "../../packages/gateway-protocol/src
import { WORKER_INFERENCE_MAX_CONTEXT_MESSAGES } from "../../packages/gateway-protocol/src/schema/worker-inference.js";
import type { AgentMessage } from "../agents/runtime/index.js";
import type { AgentSessionWriteSettlementRunner } from "../agents/sessions/agent-session.js";
import { readAgentAssistantSource } from "../infra/agent-events.js";
import {
hasRuntimeContextMarker,
isRuntimeContextMessage,
@@ -124,6 +125,10 @@ export function createWorkerTranscriptRuntime(
`Worker transcript cannot persist authoritative provider replay: ${projected.details.reason}.`,
);
}
const itemId = readAgentAssistantSource(message)?.itemId;
if (projected.message.role === "assistant" && itemId) {
projected.message.itemId = itemId;
}
if (!isWorkerTranscriptMessageFrameSafe(projected.message)) {
throw new Error("Worker transcript message exceeds the protocol payload limit.");
}
src/worker/transcript-message.test.ts
+62
View File
@@ -7,11 +7,15 @@ import {
WorkerTranscriptMessageSchema,
} from "../../packages/gateway-protocol/src/index.js";
import { WORKER_PROTOCOL_MAX_MEDIA_PAYLOAD_BYTES } from "../../packages/gateway-protocol/src/schema/worker-protocol-primitives.js";
import { guardSessionManager } from "../agents/session-tool-result-guard-wrapper.js";
import { SessionManager } from "../agents/sessions/session-manager.js";
import type { AssistantMessage, Context } from "../llm/types.js";
import { createDeferredCore } from "../shared/deferred.js";
import { createWorkerLiveRuntime } from "./embedded-agent-live.runtime.js";
import {
createWorkerTranscriptRuntime,
toWorkerInferenceContext,
type WorkerTranscriptClient,
} from "./embedded-agent-transcript.runtime.js";
import {
isWorkerTranscriptMessageFrameSafe,
@@ -96,6 +100,64 @@ function assistantWithReplay(
}
describe("worker transcript provider replay", () => {
it.each([true, false])(
"correlates identical assistant occurrences through guarded persistence (previews=%s)",
async (previews) => {
const commit = vi.fn<WorkerTranscriptClient["commit"]>(async () => {});
const transcript = createWorkerTranscriptRuntime({ commit });
const manager = guardSessionManager(SessionManager.inMemory(), {
onMessagePersisted: transcript.onMessagePersisted,
});
const liveItemIds: Array<string | undefined> = [];
const live = createWorkerLiveRuntime({
enqueuePreview: (event) => {
if (event.kind === "assistant") {
liveItemIds.push(event.payload.itemId);
}
return previews;
},
emitTerminal: async () => {},
});
live.handleSessionEvent({ type: "agent_start" });
for (let index = 0; index < 2; index++) {
const message = assistantWithReplay();
live.handleSessionEvent({ type: "message_start", message });
live.handleSessionEvent({
type: "message_update",
message,
assistantMessageEvent: { type: "text_delta", contentIndex: 0, delta: "visible" },
});
const completed = structuredClone(message);
live.handleSessionEvent({ type: "message_end", message: completed });
const entryId = await transcript.withSessionWriteSettlement(() =>
manager.appendMessageAsync(completed),
);
if (!entryId) {
throw new Error("Expected a persisted assistant entry");
}
expect(manager.getEntry(entryId)).not.toHaveProperty("message.itemId");
expect(toWorkerInferenceContext({ messages: [completed] })).not.toHaveProperty(
"context.messages.0.itemId",
);
}
const messages = commit.mock.calls.flatMap(([batch]) => batch);
const itemIds = messages.flatMap((message) =>
message.role === "assistant" ? [message.itemId] : [],
);
expect(itemIds).toEqual([expect.any(String), expect.any(String)]);
expect(new Set(itemIds).size).toBe(2);
expect(liveItemIds).toEqual(previews ? itemIds : []);
expect(
validateWorkerTranscriptCommitParams({
runEpoch: 1,
seq: 1,
baseLeafId: null,
messages,
}),
).toBe(true);
},
);
it.each(["accepted", "failed"] as const)(
"settles a submitted transcript commit after cancellation when it is %s",
async (outcome) => {
ui/src/e2e/chat-run-lifecycle.e2e.test.ts
+1 -1
View File
@@ -292,7 +292,7 @@ suite.define(() => {
],
inFlightRun: {
runId: "run-reconnected",
text: "Saved opening. Still working after reconnect.",
text: "Still working after reconnect.",
},
sessionInfo: {
activeRunIds: ["run-reconnected"],
ui/src/e2e/chat-send-ack-ownership.e2e.test.ts
+4 -1
View File
@@ -117,7 +117,10 @@ suite.define(() => {
state: "delta",
deltaText: " Still streaming.",
});
await page.locator(".chat-bubble").getByText("Still streaming.", { exact: true }).waitFor();
await page
.locator(".chat-bubble")
.getByText(`${progress} Still streaming.`, { exact: true })
.waitFor();
await composer.fill("");
await stop.click();
const aborted = await gateway.waitForRequest("chat.abort");
ui/src/lib/chat/chat-types.ts
+2 -17
View File
@@ -271,20 +271,8 @@ export type ChatStreamSegment = {
text: string;
ts: number;
runId?: string;
/** Persisted user send that causally precedes this transient output. */
afterBoundaryRunId?: string;
/** Persisted user send that causally follows this transient output. */
boundaryRunId?: string;
/** Ordering-only boundary with no renderable assistant text. */
boundaryMarker?: true;
/** Hidden durable replacement; cumulative text still owns the prefix baseline. */
persisted?: true;
/** Keyed item that consumed this cumulative occurrence; late updates cannot consume another. */
retiredItemId?: string;
/** In-flight handoff owned by the retired cumulative prefix, not its live display. */
pendingCommentary?: { text: string; prefixLength: number };
/** Visible cumulative prefix that must stay before this persisted commentary item. */
pendingCommentaryPrefixFor?: string;
toolCallId?: string;
itemId?: string;
};
@@ -293,11 +281,8 @@ export function streamSegmentHasItemId(segment: { itemId?: unknown }): boolean {
return typeof segment.itemId === "string" && segment.itemId.trim().length > 0;
}
export function streamSegmentUsesAccumulatedText(segment: {
itemId?: unknown;
boundaryMarker?: unknown;
}): boolean {
return segment.boundaryMarker !== true && !streamSegmentHasItemId(segment);
export function streamSegmentUsesAccumulatedText(segment: { itemId?: unknown }): boolean {
return !streamSegmentHasItemId(segment);
}
/** Advance the accumulated-text tracker only when the segment genuinely
ui/src/pages/chat/chat-commands.ts
-2
View File
@@ -420,8 +420,6 @@ export async function dispatchChatSlashCommand(
if (result.trackRunId && targetIsCurrent()) {
host.chatRunId = result.trackRunId;
host.chatStream = "";
host.chatStreamItemId = undefined;
host.chatStreamItemStartOffset = undefined;
host.chatSending = false;
}
ui/src/pages/chat/chat-gateway.test.ts
+189 -139
View File
@@ -37,8 +37,6 @@ import {
} from "./terminal-message-identity.ts";
import type { ToolStreamHost } from "./tool-stream-contract.ts";
import { buildToolStreamIdentity } from "./tool-stream-identity.ts";
import { createHost } from "./tool-stream.test-helpers.ts";
import { handleAgentEvent } from "./tool-stream.ts";
beforeEach(() => vi.useFakeTimers());
afterEach(() => {
@@ -237,6 +235,56 @@ it.each([
},
);
it("keeps a later final distinct after an unpositioned steer", () => {
const runId = "run-1";
const state = createState({
chatRunId: runId,
chatMessages: [
textMessage("user", "Ask", { id: "prompt", seq: 1, idempotencyKey: "run-1:user" }, 1),
textMessage(
"user",
"Earlier steer",
{
id: "earlier-steer",
seq: 2,
idempotencyKey: "earlier:user",
steerTargetRunId: runId,
},
2,
),
textMessage("assistant", "Earlier answer", { id: "earlier-answer", seq: 3, runId }, 3),
],
});
applySessionMessagePayload(
state,
{
clientRunId: runId,
messageId: "latest-steer",
message: textMessage(
"user",
"Latest steer",
{
idempotencyKey: "latest:user",
steerTargetRunId: runId,
},
4,
),
},
true,
{ kind: "live", activeRunId: runId },
);
receive(state, "final", {
message: textMessage("assistant", "New final", undefined, 5),
});
expect(state.chatMessages.map(extractText)).toEqual([
"Ask",
"Earlier steer",
"Earlier answer",
"Latest steer",
"New final",
]);
});
it("preserves receipt-less fallback ownership across cache before matching persistence", () => {
const runId = "unreceipted-run";
const user = textMessage("user", "Ask", { id: "user", seq: 1, runId });
@@ -263,62 +311,6 @@ it("preserves receipt-less fallback ownership across cache before matching persi
expect(restored.chatMessages).toEqual([user, prior, current]);
});
it.each([false, true])(
"completes an overtaken commentary item with formatting (persisted=%s)",
(persisted) => {
const text = "- first file\n- second file\n\n```python\n execute()\n```";
const state = Object.assign(
createState(),
createHost({
chatRunId: "run-1",
chatStream: text.slice(0, -4),
}),
);
handleAgentEvent(state, {
sessionKey: "main",
runId: "run-1",
seq: 1,
ts: 1,
stream: "item",
data: {
kind: "preamble",
phase: "end",
itemId: "commentary-1",
progressText: text.replace(/\s+/gu, " "),
},
});
expect(
visibleParts(state, true).map((part) => ({ text: part.text, itemId: part.itemId })),
).toEqual([{ text: text.replace(/\s+/gu, " "), itemId: "commentary-1" }]);
if (persisted) {
// Durable history projects the transcript text, not flattened progressText.
applySessionMessagePayload(
state,
{
runId: "run-1",
message: {
role: "assistant",
content: [{ type: "text", text }],
__openclaw: { id: "saved-commentary", seq: 1, runId: "run-1" },
openclawStreamFallback: { source: "segment", itemId: "commentary-1" },
},
},
true,
{ kind: "history-delta" },
);
expect(state.chatMessages.map(extractText)).toEqual([text]);
}
receive(state, "delta", { seq: 2, message: textMessage("assistant", text) });
expect(
visibleParts(state, true).map((part) => ({ text: part.text, itemId: part.itemId })),
).toEqual(persisted ? [] : [{ text, itemId: "commentary-1" }]);
if (persisted) {
expect(state.chatMessages.map(extractText)).toEqual([text]);
}
expect(state.chatStream).toBe(text);
},
);
type HistoryResult = {
messages: Array<unknown>;
thinkingLevel?: string;
@@ -584,23 +576,68 @@ describe("handleChatGatewayEvent", () => {
expect(state.chatRunStartup).toEqual({ state: "activity", runId: "run-1" });
});
it("appends one background final when three retained panes receive the same event", () => {
it.each([false, true])(
"appends one background final across three panes (display projection=%s)",
(projected) => {
const cache = new Map();
const states = ["one", "two", "three"].map((sessionKey) =>
createState({ chatMessagesBySession: cache, sessionKey }),
);
const payload: ChatEventPayload = chatEvent("final", {
sessionKey: "background",
message: projected
? {
...textMessage("assistant", "complete delivery result"),
openclawDisplayContent: [{ type: "text", text: "background final" }],
}
: textMessage("assistant", "background final"),
});
seedChatSnapshot(states[0]!, { sessionKey: "background" });
for (const state of states) {
handleChatGatewayEvent(state, payload);
}
expect(readChatMessagesFromCache(cache, states[0]!, { sessionKey: "background" })).toEqual([
textMessage("assistant", "background final"),
]);
},
);
it("keeps a background canvas-only final after its durable same-run text", () => {
const cache = new Map();
const states = ["one", "two", "three"].map((sessionKey) =>
createState({ chatMessagesBySession: cache, sessionKey }),
);
const payload: ChatEventPayload = chatEvent("final", {
sessionKey: "background",
message: textMessage("assistant", "background final"),
const state = createState({ sessionKey: "foreground", chatMessagesBySession: cache });
const target = { sessionKey: "background" };
const saved = textMessage("assistant", "Saved text", { id: "saved", seq: 2, runId: "run-1" });
const widget = {
type: "canvas",
rawText: null,
preview: {
kind: "canvas",
surface: "assistant_message",
render: "url",
url: "/__openclaw__/canvas/documents/background-widget/index.html",
},
};
cacheChatSessionSnapshot(cache, state, target, {
messages: [saved],
pagination: { hasMore: false, completeSnapshot: true },
sessionId: "cached-session",
});
seedChatSnapshot(states[0]!, { sessionKey: "background" });
for (const state of states) {
handleChatGatewayEvent(state, payload);
}
expect(readChatMessagesFromCache(cache, states[0]!, { sessionKey: "background" })).toEqual([
payload.message,
handleChatGatewayEvent(
state,
chatEvent("final", {
sessionKey: target.sessionKey,
message: {
role: "assistant",
content: [{ type: "text", text: "Saved text" }, widget],
openclawDisplayContent: [widget],
},
}),
);
expect(readChatMessagesFromCache(cache, state, target)).toEqual([
saved,
{ role: "assistant", content: [widget] },
]);
});
@@ -729,9 +766,8 @@ describe("handleChatGatewayEvent", () => {
it.each([
{
name: "renders the cumulative snapshot across a rolled-over stream boundary",
name: "adopts the complete snapshot when its preceding delta was missed",
previous: null,
segments: [{ text: "Live", ts: 1, runId: "run-1", boundaryRunId: "steer-run" }],
delta: " reply",
snapshot: "Live reply",
expected: "Live reply",
@@ -744,11 +780,18 @@ describe("handleChatGatewayEvent", () => {
replace: true,
expected: "",
},
])("$name", ({ previous, segments, delta, snapshot, replace, expected }) => {
{
name: "retires saved text when a replacement tail is a silent token",
previous: "The token is ",
delta: "",
snapshot: "NO_REPLY",
replace: true,
expected: "",
},
])("$name", ({ previous, delta, snapshot, replace, expected }) => {
const state = createState({
chatRunId: "run-1",
chatStream: previous,
...(segments ? { chatStreamSegments: segments } : {}),
});
const payload: ChatEventPayload = chatEvent("delta", {
deltaText: delta,
@@ -760,51 +803,6 @@ describe("handleChatGatewayEvent", () => {
expect(state.chatStream).toBe(expected);
});
it("reuses persisted text across live deltas and refreshes replaced messages", () => {
const persistedMessage = (id: string, text: string) => {
const readContent = vi.fn(() => [{ type: "text", text }]);
return {
readContent,
message: {
role: "assistant",
get content() {
return readContent();
},
__openclaw: { id, runId: "run-1" },
},
};
};
const first = persistedMessage("part-a", "A");
const second = persistedMessage("part-b", "B");
const state = createState({
chatRunId: "run-1",
chatMessages: [first.message, second.message],
});
const receiveDelta = (text: string) => {
receive(state, "delta", { message: textMessage("assistant", text) });
return visibleCurrentAssistantStreamTail(state, () => false);
};
expect(receiveDelta("ABC")).toBe("C");
expect(first.readContent).toHaveBeenCalled();
expect(second.readContent).toHaveBeenCalled();
first.readContent.mockClear();
second.readContent.mockClear();
for (const text of ["ABCD", "ABCDE", "ABCDEF"]) {
expect(receiveDelta(text)).toBe(text.slice(2));
}
expect(first.readContent).not.toHaveBeenCalled();
expect(second.readContent).not.toHaveBeenCalled();
const replacement = persistedMessage("part-b", "BC");
state.chatMessages = [first.message, replacement.message];
expect(receiveDelta("ABCDEFG")).toBe("DEFG");
expect(replacement.readContent).toHaveBeenCalled();
expect(first.readContent).not.toHaveBeenCalled();
expect(second.readContent).not.toHaveBeenCalled();
});
it("keeps a delivered legacy text-only assistant visible exactly once across stale history", async () => {
const final = {
text: "Delivered answer",
@@ -881,37 +879,48 @@ describe("handleChatGatewayEvent", () => {
});
});
it("replaces an exact keyed final-answer stream with the persisted terminal", () => {
it("preserves keyed commentary when a distinct terminal answer repeats its text", () => {
const user = textMessage("user", "Ask", undefined, 1);
const state = createState({ chatRunId: "run-1", chatMessages: [user] });
state.chatStreamSegments = [{ text: "Final answer.", ts: 2, itemId: "final-answer-1" }];
state.chatStreamSegments = [{ text: "Final answer.", ts: 2, itemId: "commentary-1" }];
receive(state, "final", {
message: textMessage("assistant", "Final answer.", undefined, 5),
});
expect(state.chatMessages).toHaveLength(2);
expect(state.chatMessages).toHaveLength(3);
expectTextMessage(state.chatMessages[0], "user", "Ask");
expectTextMessage(state.chatMessages[1], "assistant", "Final answer.");
expectTextMessage(state.chatMessages[2], "assistant", "Final answer.");
expect(state.chatMessages[1]).toMatchObject({
openclawStreamFallback: { source: "segment", itemId: "commentary-1" },
});
expect(state.chatMessages[2]).not.toHaveProperty("openclawStreamFallback");
expect(state.chatStreamSegments).toEqual([]);
});
it("preserves an already-recorded stream boundary for a persisted steer", () => {
it("preserves one durable keyed commentary row when a steered run finishes", () => {
const state = createState({
chatRunId: "run-1",
chatMessages: [
{ role: "user", content: [{ type: "text", text: "Ask" }], timestamp: 1 },
textMessage("user", "Ask", { idempotencyKey: "run-1:user" }, 1),
{
role: "assistant",
content: [{ type: "text", text: "Looking into it." }],
timestamp: 2,
openclawStreamFallback: {
itemId: "preamble-1",
runId: "run-1",
replacementText: "Looking into it.",
source: "segment",
},
},
textMessage("user", "Focus on deployment", { idempotencyKey: "steer-send-1:user" }, 3),
textMessage(
"user",
"Focus on deployment",
{ idempotencyKey: "steer-send-1:user", steerTargetRunId: "run-1" },
3,
),
],
});
state.chatStreamSegments = [
@@ -919,7 +928,7 @@ describe("handleChatGatewayEvent", () => {
text: "Looking into it.",
ts: 2,
itemId: "preamble-1",
boundaryRunId: "steer-send-1",
runId: "run-1",
},
];
@@ -934,27 +943,47 @@ describe("handleChatGatewayEvent", () => {
expectTextMessage(state.chatMessages[3], "assistant", "Final answer.");
});
it("keeps a terminal-only suffix after a steer with no post-boundary delta", () => {
it("keeps the complete terminal reply above a steer when no later delta arrived", () => {
const state = createState({
chatRunId: "run-1",
chatStream: "Before steer.",
chatStreamStartedAt: 2,
chatMessages: [
textMessage("user", "Ask", { idempotencyKey: "run-1:user" }, 1),
textMessage("user", "Steer", { idempotencyKey: "steer-1:user" }, 3),
textMessage(
"user",
"Steer",
{
idempotencyKey: "steer-1:user",
steerTargetRunId: "run-1",
},
3,
),
],
});
state.chatStreamSegments = [
{ text: "Before steer.", ts: 2, runId: "run-1", boundaryRunId: "steer-1" },
];
receive(state, "final", {
message: textMessage("assistant", "Before steer. Final unseen suffix.", undefined, 4),
});
expect(state.chatMessages).toHaveLength(4);
expect(state.chatMessages).toHaveLength(3);
expectTextMessage(state.chatMessages[0], "user", "Ask");
expectTextMessage(state.chatMessages[1], "assistant", "Before steer.");
expectTextMessage(state.chatMessages[2], "user", "Steer");
expectTextMessage(state.chatMessages[3], "assistant", "Final unseen suffix.");
expectTextMessage(state.chatMessages[1], "user", "Steer");
expectTextMessage(state.chatMessages[2], "assistant", "Before steer. Final unseen suffix.");
const rendered = buildChatItems({
paneId: "terminal-above-steer",
sessionKey: state.sessionKey,
runId: state.chatRunId,
messages: state.chatMessages,
toolMessages: [],
streamSegments: [],
stream: state.chatStream,
streamStartedAt: state.chatStreamStartedAt,
showToolCalls: true,
}).flatMap((item) =>
item.kind === "group" ? item.messages.map(({ message }) => extractText(message)) : [],
);
expect(rendered).toEqual(["Ask", "Before steer. Final unseen suffix.", "Steer"]);
});
it("clears keyed commentary when chatPersistCommentary is false", () => {
@@ -1220,13 +1249,34 @@ describe("handleChatGatewayEvent", () => {
type TerminalErrorFixture = {
stream?: string | null;
previous?: ReturnType<typeof textMessage>[];
segments?: Array<{ text: string; ts: number; toolCallId?: string }>;
segments?: ChatState["chatStreamSegments"];
message?: Record<string, unknown>;
expected: Array<readonly ["assistant" | "user", string]>;
verify?: (state: ChatState) => void;
};
it.each([
{
name: "keeps an interrupted answer once beside identically worded keyed commentary",
create(): TerminalErrorFixture {
const text = "Checking the workspace.";
return {
stream: text,
segments: [{ text, ts: 90, itemId: "commentary-1" }],
message: textMessage("assistant", text, undefined, 101),
expected: [
["assistant", text],
["assistant", text],
],
verify: (state) => {
expect(state.chatMessages[0]).toMatchObject({
openclawStreamFallback: { source: "segment", itemId: "commentary-1" },
});
expect(state.chatMessages[1]).not.toHaveProperty("openclawStreamFallback");
},
};
},
},
{
name: "keeps streamed text without appending the error payload message",
create(): TerminalErrorFixture {
ui/src/pages/chat/chat-gateway.ts
+72 -109
View File
@@ -1,11 +1,11 @@
import {
hasSessionProjectionAcceptedFinal,
isSessionProjectionErrorMessage,
readAssistantStreamSegmentIdentity,
readSessionMessageIdentity,
reduceSessionProjectionRunEvent,
} from "@openclaw/gateway-client/browser";
import { asNullableRecord as asRecord } from "@openclaw/normalization-core/record-coerce";
import { projectAssistantDisplayContent } from "../../../../src/shared/assistant-display-content.js";
import { t } from "../../i18n/index.ts";
import { isAssistantHeartbeatAckForDisplay } from "../../lib/chat/heartbeat-display.ts";
import { extractText } from "../../lib/chat/message-extract.ts";
@@ -35,20 +35,12 @@ import {
setChatRunError,
} from "./run-lifecycle.ts";
import { appendChatMessageToCache, readChatMessagesFromCache } from "./session-message-cache.ts";
import {
latestStreamBoundaryRunId,
reconcileTerminalStreamBoundary,
} from "./stream-causal-boundary.ts";
import { persistedSteerTargetRunId, replaceChatStream } from "./stream-causal-boundary.ts";
import {
appendTerminalAssistantMessage,
clearToolStreamSegments,
terminalMessageReplacesVisibleStream,
} from "./stream-reconciliation.ts";
import {
discardStreamSegmentIndexes,
prunePersistedCurrentAssistantItem,
reconcilePersistedAssistantStream,
} from "./stream-segment-pruning.ts";
import {
authoritativeHistoryAppliedForRun,
normalizeFinalAssistantMessage,
@@ -66,13 +58,6 @@ function normalizeAbortedAssistantMessage(message: unknown): Record<string, unkn
return candidate?.role === "assistant" && Array.isArray(candidate.content) ? candidate : null;
}
function hasPersistedAssistantItem(state: ChatState, runId: string, itemId: string): boolean {
return state.chatMessages.some((message) => {
const identity = readAssistantStreamSegmentIdentity(message);
return identity?.itemId === itemId && (!identity.runId || identity.runId === runId);
});
}
function formatGatewayErrorDetail(payload: ChatEventPayload): string | null {
const detail = payload.errorDetail;
if (!detail || detail.providerRuntimeFailureKind !== "auth_refresh") {
@@ -115,10 +100,17 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
if (!incoming) {
return null;
}
const wireMessage = asRecord(incoming.message);
const displayMessage =
wireMessage && incoming.state !== "delta" && incoming.state !== "status"
? projectAssistantDisplayContent(wireMessage)
: incoming.message;
const displayEvent =
displayMessage === incoming.message ? incoming : { ...incoming, message: displayMessage };
const payload =
incoming.state === "aborted" && incoming.stopReason === "auth-revoked"
? { ...incoming, errorMessage: t("chat.providerAccessRemoved") }
: incoming;
displayEvent.state === "aborted" && displayEvent.stopReason === "auth-revoked"
? { ...displayEvent, errorMessage: t("chat.providerAccessRemoved") }
: displayEvent;
const errorKind =
payload.state === "error" && payload.errorKind === "state_contention"
? "state_contention"
@@ -142,6 +134,25 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
__openclaw: { ...asRecord(incomingFinalMessage["__openclaw"]), id: injectedMessageId },
}
: incomingFinalMessage;
const terminalAfterSequence = (
messages: unknown[],
runId: string | null | undefined,
): number | null | undefined => {
// Display projections own an unpersisted tail; a durable row cannot claim
// that occurrence by run identity alone.
if (displayMessage !== incoming.message) {
return null;
}
const latestSteer = runId
? messages.findLast(
(entry) =>
readSessionMessageIdentity(entry)?.role === "user" &&
persistedSteerTargetRunId(entry) === runId,
)
: undefined;
// A known steer with no position still fences earlier same-run answers.
return latestSteer ? (readSessionMessageIdentity(latestSteer)?.sequence ?? null) : undefined;
};
const hadActiveRunBeforeEvent = state.chatRunId !== null;
const sessionMatches = visibleSessionMatches(state, payload.sessionKey, payload.agentId);
const activeRunMatches =
@@ -159,21 +170,29 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
? (payload.agentId ?? resolveUiDefaultAgentId(state))
: payload.agentId;
if (state.chatMessagesBySession) {
const cachedMessages = readChatMessagesFromCache(state.chatMessagesBySession, state, {
sessionKey: payload.sessionKey,
agentId: cacheAgentId,
});
if (
injectedMessageId &&
readChatMessagesFromCache(state.chatMessagesBySession, state, {
sessionKey: payload.sessionKey,
agentId: cacheAgentId,
})?.some((message) => readSessionMessageIdentity(message)?.id === injectedMessageId)
cachedMessages.some(
(message) => readSessionMessageIdentity(message)?.id === injectedMessageId,
)
) {
return null;
}
const afterSequence = terminalAfterSequence(cachedMessages, payload.runId);
appendChatMessageToCache(
state.chatMessagesBySession,
state,
{ sessionKey: payload.sessionKey, agentId: cacheAgentId },
finalMessage,
injectedMessageId ? { messageId: injectedMessageId } : payload,
injectedMessageId
? { messageId: injectedMessageId }
: afterSequence === undefined
? payload
: { ...payload, afterSequence },
);
}
}
@@ -200,16 +219,18 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
message: Record<string, unknown>,
visibleMessages: unknown[],
runId: string | null | undefined,
afterSequence?: number | null,
): void => {
const event = payload as ChatEventPayload & { messageId?: unknown; messageSeq?: unknown };
// A pre-steer durable reply cannot claim a later final from the same run.
// This is an identity fence; the thread builder owns presentation order.
const afterSequence = terminalAfterSequence(state.chatMessages, runId);
publishChatSessionProjectionMessages(state, visibleMessages, {
scope,
event: {
type: "messagePersisted",
message,
envelope: {
afterSequence,
...(afterSequence === undefined ? {} : { afterSequence }),
...(runId ? { runId } : {}),
...(event.messageId === undefined ? {} : { messageId: event.messageId }),
...(event.messageSeq === undefined ? {} : { messageSeq: event.messageSeq }),
@@ -348,7 +369,11 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
return null;
}
const terminalAfterBoundaryRunId = latestStreamBoundaryRunId(state);
if (activeRunMatches && displayMessage !== incoming.message) {
// An empty display tail is authoritative too; do not materialize an older live copy.
replaceChatStream(state, extractText(displayMessage) ?? null);
}
const materializeVisibleStream = (
materializeOpts: Parameters<typeof materializeVisibleAssistantStreamMessages>[2] = {},
) => materializeVisibleAssistantStreamMessages(state.chatMessages, state, materializeOpts);
@@ -359,7 +384,6 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
rememberLiveTerminalRun(
state.chatMessages.findLast((message) => transcriptRunId(message) === terminalRunId),
terminalRunId,
terminalAfterBoundaryRunId,
payload.state === "aborted"
? "aborted"
: projectedRun?.currentRun?.status === "timeout"
@@ -399,86 +423,32 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
reconcileChatRunStartup(state, { state: "activity", runId: payload.runId });
}
const next = payload.message == null ? null : (extractText(payload.message) ?? "");
if (
typeof next === "string" &&
!isSilentReplyStream(next) &&
!isAssistantHeartbeatAckForDisplay(payload.message)
) {
const itemStartOffset =
Number.isInteger(payload.itemStartOffset) &&
(payload.itemStartOffset ?? -1) >= 0 &&
(payload.itemStartOffset ?? Number.POSITIVE_INFINITY) <= next.length
? payload.itemStartOffset
: undefined;
if (payload.itemId !== undefined) {
state.chatStreamItemId = payload.itemId;
state.chatStreamItemStartOffset = itemStartOffset;
} else {
state.chatStreamItemId = undefined;
state.chatStreamItemStartOffset = undefined;
if (typeof next === "string") {
const hidden =
isSilentReplyStream(next) || isAssistantHeartbeatAckForDisplay(payload.message);
// A replacement retires the previous baseline even when its new text is hidden;
// ignoring it would keep already-saved text visible beside its durable row.
if (payload.replace) {
replaceChatStream(state, hidden ? "" : next);
} else if (!hidden) {
state.chatStream = next;
}
state.chatStream = next;
if (
payload.runId &&
payload.itemId &&
itemStartOffset !== undefined &&
hasPersistedAssistantItem(state, payload.runId, payload.itemId)
) {
prunePersistedCurrentAssistantItem(state, {
runId: payload.runId,
itemId: payload.itemId,
});
}
reconcilePersistedAssistantStream(state);
}
} else if (payload.state === "final") {
const finalMessage = normalizedFinalMessage;
if (authoritativeTerminalMatches) {
// History already owns this run's terminal message. Discard the live
// projection; terminal cleanup below clears its remaining stream.
} else if (finalMessage && !shouldHideAssistantChatMessage(finalMessage)) {
const visibleMessages = materializeVisibleStream();
const liveFinal = rememberLiveTerminalRun(finalMessage, terminalRunId);
publishVisibleTerminal(
finalMessage,
appendTerminalAssistantMessage(visibleMessages, liveFinal),
terminalRunId,
);
} else {
const boundary = finalMessage
? reconcileTerminalStreamBoundary(finalMessage, state)
: { kind: "none" as const };
if (boundary.kind === "split") {
// The tail follows a known transcript boundary; it cannot adopt the saved prefix.
discardStreamSegmentIndexes(state, boundary.replacedSegmentIndexes);
let visibleMessages = materializeVisibleStream({ includeCurrent: false });
if (boundary.tailMessage && !shouldHideAssistantChatMessage(boundary.tailMessage)) {
const liveTail = rememberLiveTerminalRun(
boundary.tailMessage,
terminalRunId,
boundary.afterBoundaryRunId,
);
// A retired commentary item keeps its own identity even when the answer
// repeats its text. The sequence fence reconciles only the later answer.
visibleMessages = appendTerminalAssistantMessage(visibleMessages, liveTail, {
preserveKeyedCommentary: boundary.preserveKeyedCommentary,
});
publishVisibleTerminal(
boundary.tailMessage,
visibleMessages,
terminalRunId,
boundary.afterSequence,
);
} else {
publishChatSessionProjectionMessages(state, visibleMessages, { scope });
}
} else if (finalMessage && !shouldHideAssistantChatMessage(finalMessage)) {
const visibleMessages = materializeVisibleStream();
const liveFinal = rememberLiveTerminalRun(
finalMessage,
terminalRunId,
terminalAfterBoundaryRunId,
);
publishVisibleTerminal(
finalMessage,
appendTerminalAssistantMessage(visibleMessages, liveFinal),
terminalRunId,
);
} else {
publishChatSessionProjectionMessages(state, materializeVisibleStream(), { scope });
}
publishChatSessionProjectionMessages(state, materializeVisibleStream(), { scope });
}
reconcileOwnedTerminalRun();
} else if (payload.state === "aborted") {
@@ -488,12 +458,7 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
replacementMessages: [normalizedMessage],
includeCurrent: false,
});
const liveAborted = rememberLiveTerminalRun(
normalizedMessage,
terminalRunId,
terminalAfterBoundaryRunId,
"aborted",
);
const liveAborted = rememberLiveTerminalRun(normalizedMessage, terminalRunId, "aborted");
publishVisibleTerminal(
normalizedMessage,
appendTerminalAssistantMessage(visibleMessages, liveAborted),
@@ -521,7 +486,6 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
state,
{
isHiddenStreamText: isHiddenAssistantStreamText,
persistCommentary: state.settings?.chatPersistCommentary !== false,
},
);
const visibleMessages = materializeVisibleStream({
@@ -530,7 +494,6 @@ export function handleChatGatewayEvent(state: ChatState, incoming?: ChatEventPay
const liveError = rememberLiveTerminalRun(
visiblePayloadMessage,
terminalRunId,
terminalAfterBoundaryRunId,
projectedRun?.currentRun?.status === "timeout" ? "timeout" : "error",
);
publishVisibleTerminal(
ui/src/pages/chat/chat-history-hydration.ts
+20 -17
View File
@@ -54,7 +54,7 @@ import {
} from "./performance.ts";
import type { ChatHistoryRunObservation } from "./run-lifecycle.ts";
import { applySessionMessagePayload } from "./session-message-apply.ts";
import { rolloverChatStream } from "./stream-causal-boundary.ts";
import { retainPersistedStreamPrefix } from "./stream-causal-boundary.ts";
import {
hasVisibleStreamParts,
historyReplacedVisibleStream,
@@ -63,6 +63,7 @@ import {
} from "./stream-reconciliation.ts";
import {
pruneHistoryReplacedStreamSegments,
prunePersistedAssistantStreamSegments,
prunePersistedToolStreamMessages,
} from "./stream-segment-pruning.ts";
import { reconcileAuthoritativeTerminalHistory } from "./terminal-message-identity.ts";
@@ -199,7 +200,6 @@ export async function hydrateChatHistory(
}
if (isHistoryCursor(response) && response.kind === "delta") {
const runProjectionsBeforeApply = readRunProjections(state, sessionKey, requestAgentId);
const activeStreamBeforeApply = state.chatRunId ? state.chatStream : null;
const runActive = isSessionRunActive(response.sessionInfo);
for (const payload of response.messages) {
applySessionMessagePayload(state, payload, runActive, { kind: "history-delta" });
@@ -234,7 +234,6 @@ export async function hydrateChatHistory(
runProjectionsBeforeApply,
currentRunProjections: historyProjection.runs,
resetStream: !state.chatRunId || state.chatRunId === previousRunId,
activeStreamBeforeReset: activeStreamBeforeApply,
});
commitCurrentChatHistorySnapshot(state, response.deltaCursor ?? null);
recordTiming("applied", {
@@ -337,7 +336,6 @@ export async function hydrateChatHistory(
state.chatVerboseLevel = res.verboseLevel ?? null;
state.chatQueueModeOverride = res.sessionInfo?.queueMode;
state.chatEffectiveQueueMode = res.sessionInfo?.effectiveQueueMode;
const activeStreamBeforeReset = state.chatRunId ? state.chatStream : null;
const resetStream = !state.chatRunId || state.chatRunId === previousRunId;
if (resetStream) {
const streamReconciliation = {
@@ -345,13 +343,18 @@ export async function hydrateChatHistory(
isHiddenAssistantMessage: shouldHideAssistantChatMessage,
isHiddenStreamText: isHiddenAssistantStreamText,
};
for (const message of state.chatMessages) {
prunePersistedAssistantStreamSegments(state, message);
}
const hasVisibleStream = hasVisibleStreamParts(state, streamReconciliation);
const historyReplacedStream = historyReplacedVisibleStream(
state.chatMessages,
state,
streamReconciliation,
);
pruneHistoryReplacedStreamSegments(state.chatMessages, state, streamReconciliation);
// The live snapshot already excludes durable bytes. Equal text can be a
// distinct new occurrence, so only item identity can retire its live copy.
const historyReplacedStream =
!res.inFlightRun &&
historyReplacedVisibleStream(state.chatMessages, state, streamReconciliation);
if (!res.inFlightRun) {
pruneHistoryReplacedStreamSegments(state.chatMessages, state, streamReconciliation);
}
const liveToolIds = state.toolStreamOrder ?? [];
if (
state.chatRunId &&
@@ -367,7 +370,7 @@ export async function hydrateChatHistory(
const liveToolStreamReplaced = liveToolIds.length === 0 || historyReplacedToolStream;
if (!hasVisibleStream || historyReplacedStream) {
if (state.chatRunId && historyReplacedStream) {
rolloverChatStream(state, { runId: state.chatRunId, persisted: true });
retainPersistedStreamPrefix(state);
}
if (liveToolStreamReplaced) {
maybeResetToolStream(state, { preserveStreamSegments: state.chatRunId !== null });
@@ -376,8 +379,6 @@ export async function hydrateChatHistory(
}
if (!state.chatRunId) {
state.chatStream = null;
state.chatStreamItemId = undefined;
state.chatStreamItemStartOffset = undefined;
state.chatStreamStartedAt = null;
}
recordTiming("stream-reset", {
@@ -391,8 +392,6 @@ export async function hydrateChatHistory(
);
maybeResetToolStream(state);
state.chatStream = null;
state.chatStreamItemId = undefined;
state.chatStreamItemStartOffset = undefined;
state.chatStreamStartedAt = null;
} else if (historyReplacedSomeToolStream) {
publishChatSessionProjectionMessages(
@@ -406,7 +405,12 @@ export async function hydrateChatHistory(
if (!visibleCurrentAssistantStreamTail(state, streamReconciliation.isHiddenStreamText)) {
state.chatStreamStartedAt = null;
}
pruneHistoryReplacedStreamSegments(state.chatMessages, state, streamReconciliation);
for (const message of state.chatMessages) {
prunePersistedAssistantStreamSegments(state, message);
}
if (!res.inFlightRun) {
pruneHistoryReplacedStreamSegments(state.chatMessages, state, streamReconciliation);
}
if (historyReplacedToolStream) {
maybeResetToolStream(state, { preserveStreamSegments: true });
} else {
@@ -424,7 +428,6 @@ export async function hydrateChatHistory(
runProjectionsBeforeApply,
currentRunProjections: historyProjection.runs,
resetStream,
activeStreamBeforeReset,
});
recordTiming("applied", {
ui/src/pages/chat/chat-history-stream.ts
+13 -88
View File
@@ -1,7 +1,6 @@
import { readSessionMessageIdentity } from "@openclaw/gateway-client/browser";
import { asNullableRecord, asOptionalRecord } from "@openclaw/normalization-core/record-coerce";
import type { GatewaySessionRow } from "../../api/types.ts";
import { accumulatedStreamText, advanceAccumulatedStreamText } from "../../lib/chat/chat-types.ts";
import { extractText } from "../../lib/chat/message-extract.ts";
import {
isHiddenAssistantStreamText,
@@ -25,10 +24,7 @@ import {
reconcileChatRunFromSessionRow,
setChatRunError,
} from "./run-lifecycle.ts";
import {
latestPersistedSteerBoundary,
resolveCumulativeAssistantTail,
} from "./stream-causal-boundary.ts";
import { replaceChatStream } from "./stream-causal-boundary.ts";
import { materializeVisibleStreamState } from "./stream-reconciliation.ts";
import { handleAgentEvent } from "./tool-stream.ts";
@@ -78,25 +74,6 @@ function resolveInFlightAssistantText(bufferedText: unknown): string | null {
: null;
}
function replayedCommentaryItemIds(
run: NonNullable<ChatHistoryResult["inFlightRun"]>,
): ReadonlySet<string> {
const itemIds = new Set<string>();
for (const event of run.events ?? []) {
const itemId = event.data.itemId;
if (
event.runId === run.runId &&
event.stream === "item" &&
event.data.kind === "preamble" &&
typeof itemId === "string" &&
itemId.trim()
) {
itemIds.add(itemId.trim());
}
}
return itemIds;
}
function runProjectionsUnchanged(
previous: ReturnType<typeof getChatSessionProjection>["runs"],
current: ReturnType<typeof getChatSessionProjection>["runs"],
@@ -132,18 +109,6 @@ export function readRunProjections(state: ChatState, sessionKey: string, agentId
).runs;
}
function mergeInFlightAssistantText(snapshot: string | null, live: string | null): string | null {
if (!snapshot || live?.startsWith(snapshot)) {
return live ?? snapshot;
}
if (!live || snapshot.startsWith(live)) {
return snapshot;
}
// Divergence: live deltas are newer than the bounded snapshot, so the local
// cumulative buffer wins — a lagging snapshot must not rewind streamed text.
return live;
}
export function applyHistoryRun(params: {
state: ChatState;
run: ChatHistoryResult["inFlightRun"];
@@ -153,7 +118,6 @@ export function applyHistoryRun(params: {
runProjectionsBeforeApply: ReturnType<typeof getChatSessionProjection>["runs"];
currentRunProjections: ReturnType<typeof getChatSessionProjection>["runs"];
resetStream: boolean;
activeStreamBeforeReset: string | null;
}): void {
const {
state,
@@ -164,7 +128,6 @@ export function applyHistoryRun(params: {
runProjectionsBeforeApply,
currentRunProjections,
resetStream,
activeStreamBeforeReset,
} = params;
const inFlightRunId = run?.runId?.trim();
if (!inFlightRunId || !run) {
@@ -270,13 +233,6 @@ export function applyHistoryRun(params: {
state.chatRunId === inFlightRunId &&
projectedInFlightRun?.status === "streaming" &&
runProjectionsUnchanged(previousRunProjections, currentRunProjections, inFlightRunId);
const retainsLiveStream =
sameRunContinued ||
(state.chatRunId === inFlightRunId &&
(activeStreamBeforeReset !== null ||
state.chatStreamSegments?.some(
(segment) => !segment.runId || segment.runId === inFlightRunId,
)));
const activeRunIds = sessionInfo?.activeRunIds;
const inFlightRunIsActive =
isSessionRunActive(sessionInfo ?? {}) &&
@@ -307,7 +263,7 @@ export function applyHistoryRun(params: {
const recoveringRun = state.chatRunId !== inFlightRunId;
// Canonical run projections change on every live delta or terminal.
// Their identity fences ABA races where a run starts and finishes while
// history is pending; deltas from this same live run must still merge.
// history is pending; the same live run retains its ordered text baseline.
adoptStartedChatRun(state, inFlightRunId, Date.now());
if (recoveringRun && sessionInfo) {
observeChatRunModel(state, inFlightRunId, sessionInfo);
@@ -319,50 +275,19 @@ export function applyHistoryRun(params: {
}
const snapshotStartedAt =
typeof run.startedAt === "number" && Number.isFinite(run.startedAt) ? run.startedAt : null;
const liveText = replacesOwnedRun
? null
: sameRunContinued
? mergeInFlightAssistantText(
resolveInFlightAssistantText(extractText(projectedInFlightRun?.message)),
activeStreamBeforeReset,
)
: activeStreamBeforeReset;
const mergedStream = mergeInFlightAssistantText(resolveInFlightAssistantText(run.text), liveText);
state.chatStream = mergedStream;
if (!retainsLiveStream || mergedStream !== activeStreamBeforeReset) {
state.chatStreamItemId = undefined;
state.chatStreamItemStartOffset = undefined;
const advancedDuringRead =
previousRunProjections[inFlightRunId]?.message !==
runProjectionsBeforeApply[inFlightRunId]?.message;
// Ordered live frames already establish their baseline, including shrinking
// replacements. A pending history snapshot cannot extend that baseline by text.
if (advancedDuringRead && !replacesOwnedRun) {
state.chatStream = resolveInFlightAssistantText(
extractText(runProjectionsBeforeApply[inFlightRunId]?.message),
);
} else {
replaceChatStream(state, resolveInFlightAssistantText(run.text));
}
state.chatStreamStartedAt = snapshotStartedAt ?? state.chatStreamStartedAt ?? Date.now();
// A retained pane gets its boundary from session.message. Only fresh adoption
// reconstructs it from history, with the persisted prefix as cumulative evidence.
const boundary = retainsLiveStream
? null
: latestPersistedSteerBoundary(state.chatMessages, inFlightRunId);
const tail =
state.chatStream === null
? null
: resolveCumulativeAssistantTail(
state.chatMessages,
state.chatStream,
inFlightRunId,
boundary?.index,
replayedCommentaryItemIds(run),
);
const prefix = state.chatStream?.slice(0, state.chatStream.length - (tail?.length ?? 0)) ?? "";
const accumulated = accumulatedStreamText(state.chatStreamSegments ?? []);
if (boundary || advanceAccumulatedStreamText(accumulated, prefix) !== accumulated) {
state.chatStreamSegments = [
...(state.chatStreamSegments ?? []),
{
text: prefix,
ts: state.chatStreamStartedAt,
runId: inFlightRunId,
...(boundary ? { boundaryRunId: boundary.runId } : {}),
...(prefix ? { persisted: true } : { boundaryMarker: true }),
},
];
}
const startup = run.events?.findLast(
(event) => event.runId === inFlightRunId && event.stream === "run_status",
);
ui/src/pages/chat/chat-history.inflight.test-support.ts
+110
View File
@@ -1,7 +1,14 @@
import { onTestFinished, vi } from "vitest";
import { createDeferred } from "../../../../test/helpers/promise.js";
import { extractText } from "../../lib/chat/message-extract.ts";
import { isHiddenAssistantStreamText } from "../../lib/chat/message-visibility.ts";
import { handleChatGatewayEvent } from "./chat-gateway.ts";
import type { ChatHistoryResult } from "./chat-history-snapshot.ts";
import type { ChatEventPayload } from "./chat-history.ts";
import { makeChatHost } from "./chat-host.test-support.ts";
import type { ChatHistoryHost } from "./chat-state-contract.ts";
import { buildChatItems } from "./chat-thread-build.ts";
import { visibleCurrentAssistantStreamTail } from "./stream-reconciliation.ts";
import type { ToolStreamEntry } from "./tool-stream-contract.ts";
import type { handleAgentEvent } from "./tool-stream.ts";
@@ -46,3 +53,106 @@ export function activeHistory(runId: string): ChatHistoryResult {
inFlightRun: { runId, text: "" },
};
}
export const message = (
role: string,
content: string,
metadata?: Record<string, unknown>,
timestamp?: number,
) => ({
role,
content,
...(metadata ? { __openclaw: metadata } : {}),
...(timestamp ? { timestamp } : {}),
});
export function emit(
state: TestState,
runId: string,
event: Omit<ChatEventPayload, "sessionKey" | "runId">,
) {
handleChatGatewayEvent(state, { sessionKey: "main", runId, ...event });
}
export function delayed(history: ChatHistoryResult) {
const response = createDeferred<ChatHistoryResult>();
const state = createState(history);
const request = vi.spyOn(state.client!, "request").mockReturnValue(response.promise);
return { state, response, request };
}
export const tail = (state: TestState) =>
visibleCurrentAssistantStreamTail(state, isHiddenAssistantStreamText);
export function renderedText(state: TestState) {
return buildChatItems({
paneId: "steer-regression",
sessionKey: state.sessionKey,
runId: state.chatRunId,
messages: state.chatMessages,
toolMessages: state.chatToolMessages,
streamSegments: state.chatStreamSegments,
stream: state.chatStream,
streamStartedAt: state.chatStreamStartedAt,
showToolCalls: true,
}).flatMap((item) =>
item.kind === "group"
? item.messages.map(({ message: entry }) => extractText(entry)?.trim())
: item.kind === "stream"
? [item.text.trim()]
: [],
);
}
export const event = (seq: number, stream: string, data: Record<string, unknown>) => ({
runId: "run-live",
seq,
stream,
ts: 899 + seq,
sessionKey: "main",
data,
});
export function failedHistory(): ChatHistoryResult {
return {
messages: [
message(
"user",
"Inspect the unavailable project",
{ id: "first-user", idempotencyKey: "run-first:user", seq: 1 },
1,
),
],
sessionInfo: {
key: "main",
kind: "direct",
updatedAt: 2,
status: "failed",
hasActiveRun: false,
lastRunId: "run-first",
lastRunError:
"ProjectCloneError: Git clone could not reach GitHub. Check the Gateway network connection and retry.",
},
};
}
export function steerPrompts(seq: number, timestamp = seq) {
return {
original: message("user", "Original prompt", { idempotencyKey: "active-run:user", seq: 1 }, 1),
steer: message(
"user",
"Steer prompt",
{
id: "steer",
idempotencyKey: "steer-run:user",
seq,
steerTargetRunId: "active-run",
},
timestamp,
),
};
}
export const toolEvent = (call: string) => ({
...event(2, "tool", {
toolCallId: call,
name: "read",
phase: "start",
args: { path: "README.md" },
}),
ts: 1000,
});
ui/src/pages/chat/chat-history.inflight.test.ts
+331 -133
View File
@@ -4,18 +4,25 @@ import { createDeferred } from "../../../../test/helpers/promise.js";
import type { GatewayBrowserClient } from "../../api/gateway.ts";
import { createChatSubmissions } from "../../app/chat-submissions.ts";
import { extractText } from "../../lib/chat/message-extract.ts";
import { isHiddenAssistantStreamText } from "../../lib/chat/message-visibility.ts";
import { handleChatGatewayEvent } from "./chat-gateway.ts";
import type { ChatHistoryResponse, ChatHistoryResult } from "./chat-history-snapshot.ts";
import { materializeVisibleAssistantStreamMessages } from "./chat-history-stream.ts";
import {
activeHistory,
createState,
delayed,
emit,
event,
failedHistory,
message,
renderedText,
steerPrompts,
tail,
toolEvent,
type TestState,
} from "./chat-history.inflight.test-support.ts";
import { loadChatHistory, type ChatEventPayload } from "./chat-history.ts";
import { loadChatHistory } from "./chat-history.ts";
import { activeChatRunStartupStatus, chatStartupStatusLabel } from "./chat-run-startup.ts";
import { buildChatItems } from "./chat-thread-build.ts";
import {
admitChatSubmission,
getChatSessionProjection,
@@ -30,55 +37,9 @@ import {
reconcileChatRunLifecycle,
} from "./run-lifecycle.ts";
import { applySessionMessagePayload } from "./session-message-apply.ts";
import { visibleCurrentAssistantStreamTail } from "./stream-reconciliation.ts";
import { handleAgentEvent } from "./tool-stream.ts";
import { buildInitialChatSubmission } from "./user-message-content.ts";
const message = (
role: string,
content: string,
metadata?: Record<string, unknown>,
timestamp?: number,
) => ({
role,
content,
...(metadata ? { __openclaw: metadata } : {}),
...(timestamp ? { timestamp } : {}),
});
function emit(
state: TestState,
runId: string,
event: Omit<ChatEventPayload, "sessionKey" | "runId">,
) {
handleChatGatewayEvent(state, { sessionKey: "main", runId, ...event });
}
function delayed(history: ChatHistoryResult) {
const response = createDeferred<ChatHistoryResult>();
const state = createState(history);
const request = vi.spyOn(state.client!, "request").mockReturnValue(response.promise);
return { state, response, request };
}
const tail = (state: TestState) =>
visibleCurrentAssistantStreamTail(state, isHiddenAssistantStreamText);
function renderedText(state: TestState) {
return buildChatItems({
paneId: "steer-regression",
sessionKey: state.sessionKey,
runId: state.chatRunId,
messages: state.chatMessages,
toolMessages: state.chatToolMessages,
streamSegments: state.chatStreamSegments,
stream: state.chatStream,
streamStartedAt: state.chatStreamStartedAt,
showToolCalls: true,
}).flatMap((item) =>
item.kind === "group"
? item.messages.map(({ message: entry }) => extractText(entry)?.trim())
: item.kind === "stream"
? [item.text.trim()]
: [],
);
}
async function loadWithTools(state: TestState) {
vi.stubGlobal("window", globalThis);
try {
@@ -88,54 +49,6 @@ async function loadWithTools(state: TestState) {
vi.unstubAllGlobals();
}
}
function failedHistory(): ChatHistoryResult {
return {
messages: [
message(
"user",
"Inspect the unavailable project",
{ id: "first-user", idempotencyKey: "run-first:user", seq: 1 },
1,
),
],
sessionInfo: {
key: "main",
kind: "direct",
updatedAt: 2,
status: "failed",
hasActiveRun: false,
lastRunId: "run-first",
lastRunError:
"ProjectCloneError: Git clone could not reach GitHub. Check the Gateway network connection and retry.",
},
};
}
function steerPrompts(seq: number, timestamp = seq) {
return {
original: message("user", "Original prompt", { idempotencyKey: "active-run:user", seq: 1 }, 1),
steer: message(
"user",
"Steer prompt",
{
id: "steer",
idempotencyKey: "steer-run:user",
seq,
steerTargetRunId: "active-run",
},
timestamp,
),
};
}
const toolEvent = (call: string) => ({
...event(2, "tool", {
toolCallId: call,
name: "read",
phase: "start",
args: { path: "README.md" },
}),
ts: 1000,
});
it("retires an interrupted run after missing its live terminal", async () => {
const history = activeHistory("run-interrupted");
const state = createState(history);
@@ -260,8 +173,9 @@ it("restores tools, preamble time, and usage from the active snapshot", async ()
);
});
it("restores cleared activity without replacing an owned run's live text", async () => {
it("restores cleared activity alongside an owned run's current text", async () => {
const history = activeHistory("run-live");
history.inFlightRun!.text = "The active response survived reconnect.";
history.inFlightRun!.events = [toolEvent("call-reconnected")];
const state = createState(history);
state.chatRunId = "run-live";
@@ -281,7 +195,7 @@ it("restores cleared activity without replacing an owned run's live text", async
});
it.each(["fresh mirror", "retained idempotency"])(
"keeps cumulative prefixes through history replacement (%s)",
"keeps complete persisted and live replies above a steer (%s)",
async (mode) => {
const history = activeHistory("active-run");
const identity = (seq: number) =>
@@ -315,45 +229,60 @@ it.each(["fresh mirror", "retained idempotency"])(
steer,
message("user", "Queued follow-up.", { idempotencyKey: "queued-run:user", seq: 6 }),
];
history.inFlightRun!.text = `${prefix} After steer.`;
const persistedText = history.messages.slice(0, -1).map(extractText);
history.inFlightRun!.text = "After steer.";
const persistedText = history.messages.slice(0, -2).map(extractText);
const state = createState(history);
if (mode === "retained idempotency") {
state.chatRunId = "active-run";
state.chatMessages = [original, steer];
state.chatStreamSegments = [{ text: prefix, ts: 2, runId: "active-run" }];
emit(state, "active-run", {
state: "delta",
replace: true,
message: message("assistant", history.inFlightRun!.text),
});
state.chatStreamSegments = [
{ text: prefix, ts: 2, runId: "active-run", boundaryRunId: "steer-run" },
];
}
await loadChatHistory(state);
expect(tail(state)).toBe("After steer.");
expect(renderedText(state)).toEqual([...persistedText, "After steer.", "Queued follow-up."]);
expect(renderedText(state)).toEqual([
...persistedText,
"After steer.",
"Steer prompt",
"Queued follow-up.",
]);
emit(state, "active-run", {
state: "delta",
deltaText: " Continued.",
message: message("assistant", `${prefix} After steer. Continued.`),
message: message("assistant", "After steer. Continued."),
});
expect(renderedText(state)).toEqual([
...persistedText,
"After steer. Continued.",
"Steer prompt",
"Queued follow-up.",
]);
expect(state.chatStream).toBe(`${prefix} After steer. Continued.`);
expect(state.chatStream).toBe("After steer. Continued.");
emit(state, "active-run", {
state: "final",
message: message("assistant", `${prefix} After steer. Continued. Final suffix.`),
message: {
...message("assistant", `${prefix} After steer. Continued. Final suffix.`),
openclawDisplayContent: [{ type: "text", text: "After steer. Continued. Final suffix." }],
},
});
const expected = [
...persistedText,
"After steer. Continued. Final suffix.",
"Steer prompt",
"Queued follow-up.",
];
expect(renderedText(state)).toEqual(expected);
expect(state.chatMessages.map(extractText)).toEqual(expected);
const transcriptOrder = [
...persistedText,
"Steer prompt",
"After steer. Continued. Final suffix.",
"Queued follow-up.",
];
expect(state.chatMessages.map(extractText)).toEqual(transcriptOrder);
expect(state.chatMessages.at(-2)).toMatchObject({ role: "assistant" });
reduceChatSessionProjection(
state,
@@ -368,11 +297,54 @@ it.each(["fresh mirror", "retained idempotency"])(
},
{ scope: readChatSessionProjectionScope(state), runActive: false },
);
expect(state.chatMessages.map(extractText)).toEqual([...expected, "Later authoritative user."]);
expect(state.chatMessages.map(extractText)).toEqual([
...transcriptOrder,
"Later authoritative user.",
]);
},
);
it("rolls over a live steer after active-run publication", async () => {
it.each([false, true])(
"keeps the steer below the whole assistant message (history=%s)",
async (reload) => {
const runId = "active-run";
const { original, steer } = steerPrompts(2);
const history = activeHistory(runId);
history.messages = [original, steer];
history.inFlightRun!.text = "Before steer. After steer.";
const state = createState(history);
state.chatMessages = [original];
state.chatRunId = runId;
emit(state, runId, {
state: "delta",
message: message("assistant", "Before steer."),
});
applySessionMessagePayload(
state,
{ clientRunId: runId, messageId: "steer", messageSeq: 2, message: steer },
true,
{ kind: "live", activeRunId: runId },
);
expect(renderedText(state)).toEqual(["Original prompt", "Before steer.", "Steer prompt"]);
if (reload) {
await loadChatHistory(state);
} else {
emit(state, runId, {
state: "delta",
deltaText: " After steer.",
message: message("assistant", "Before steer. After steer."),
});
}
expect(state.chatRunId).toBe(runId);
expect(renderedText(state)).toEqual([
"Original prompt",
"Before steer. After steer.",
"Steer prompt",
]);
},
);
it("keeps streaming one message after a live steer is published", async () => {
const history = activeHistory("active-run");
const { original, steer } = steerPrompts(2, 3);
history.messages = [original, steer];
@@ -392,18 +364,23 @@ it("rolls over a live steer after active-run publication", async () => {
});
expect(renderedText(state)).toEqual([
"Original prompt",
"Before steer.",
"Before steer. After steer.",
"Steer prompt",
"After steer.",
]);
});
it("retains the persisted replacement baseline for the next cumulative delta", () => {
it("keeps an identical new reply after the Gateway retires its persisted predecessor", () => {
const state = createState(activeHistory("active-run"));
state.chatMessages = [
message("user", "Original prompt", { idempotencyKey: "active-run:user", seq: 1 }),
];
emit(state, "active-run", { state: "delta", message: message("assistant", "Saved opening.") });
emit(state, "active-run", {
state: "delta",
replace: true,
deltaText: "",
message: message("assistant", ""),
});
applySessionMessagePayload(
state,
{
@@ -421,10 +398,234 @@ it("retains the persisted replacement baseline for the next cumulative delta", (
);
emit(state, "active-run", {
state: "delta",
deltaText: " Continued.",
message: message("assistant", "Saved opening. Continued."),
deltaText: "Saved opening.",
message: message("assistant", "Saved opening."),
});
expect(renderedText(state)).toEqual(["Original prompt", "Saved opening.", "Continued."]);
expect(renderedText(state)).toEqual(["Original prompt", "Saved opening.", "Saved opening."]);
});
it.each([false, true])(
"keeps an identical unpersisted tail through active history reload (retained steer=%s)",
async (steered) => {
const history = activeHistory("active-run");
const prompt = message("user", "Repeat the answer", {
id: "prompt",
seq: 1,
idempotencyKey: "active-run:user",
});
const steer = message("user", "Steer prompt", {
id: "steer",
seq: 3,
idempotencyKey: "steer-run:user",
steerTargetRunId: "active-run",
});
history.messages = [
prompt,
message("assistant", "Repeated answer.", { id: "saved", seq: 2, runId: "active-run" }),
...(steered ? [steer] : []),
];
history.inFlightRun!.text = "Repeated answer.";
const state = createState(history);
state.chatMessages = [prompt];
state.chatRunId = "active-run";
emit(state, "active-run", {
state: "delta",
message: message("assistant", "Repeated answer."),
});
if (steered) {
applySessionMessagePayload(
state,
{
clientRunId: "active-run",
messageId: "steer",
messageSeq: 3,
message: steer,
},
true,
{ kind: "live", activeRunId: "active-run" },
);
expect(renderedText(state)).toEqual([
"Repeat the answer",
"Repeated answer.",
"Steer prompt",
]);
}
await loadChatHistory(state);
expect(state.chatRunId).toBe("active-run");
const persisted = ["Repeat the answer", "Repeated answer."];
const steers = steered ? ["Steer prompt"] : [];
expect(renderedText(state)).toEqual([...persisted, "Repeated answer.", ...steers]);
emit(state, "active-run", {
state: "delta",
deltaText: " Continued.",
message: message("assistant", "Repeated answer. Continued."),
});
expect(renderedText(state)).toEqual([...persisted, "Repeated answer. Continued.", ...steers]);
},
);
it.each(["live replacement", "history adoption", "steer then history"])(
"preserves independent unkeyed preamble occurrences through %s",
async (delivery) => {
const runId = "active-run";
const text = "Checking the workspace.";
const now = Date.now();
const prompt = message(
"user",
"Check the workspace",
{
id: "prompt",
seq: 1,
idempotencyKey: `${runId}:user`,
},
now - 3,
);
const history = activeHistory(runId);
history.messages = [prompt];
history.inFlightRun!.text = text;
history.inFlightRun!.startedAt = now + 2;
const state = createState(history);
state.chatRunId = runId;
state.chatMessages = [prompt];
for (const seq of [1, 2, 2]) {
handleAgentEvent(state, {
sessionKey: "main",
runId,
seq,
ts: now - 3 + seq,
stream: "item",
data: { kind: "preamble", phase: "end", progressText: text },
});
}
const expected = ["Check the workspace", text, text];
if (delivery === "steer then history") {
emit(state, runId, { state: "delta", message: message("assistant", "Before steer.") });
const steer = message(
"user",
"Steer prompt",
{
id: "steer",
seq: 3,
idempotencyKey: "steer-run:user",
steerTargetRunId: runId,
},
now + 1,
);
history.messages.push(
message("assistant", "Before steer.", { id: "before", seq: 2, runId }, now),
steer,
);
applySessionMessagePayload(
state,
{ clientRunId: runId, messageId: "steer", messageSeq: 3, message: steer },
true,
{ kind: "live", activeRunId: runId },
);
expected.push("Before steer.");
}
if (delivery === "live replacement") {
emit(state, runId, {
state: "delta",
replace: true,
deltaText: text,
message: message("assistant", text),
});
} else {
await loadChatHistory(state);
}
expect(state.chatRunId).toBe(runId);
expect(renderedText(state)).toEqual([
...expected,
text,
...(delivery === "steer then history" ? ["Steer prompt"] : []),
]);
},
);
it.each([
{ name: "an empty committed tail", snapshot: "", newer: undefined, expectedTail: null },
{
name: "a different committed tail",
snapshot: "Next answer.",
newer: undefined,
expectedTail: "Next answer.",
},
{
name: "a newer live answer",
snapshot: "",
newer: "Live next answer.",
expectedTail: "Live next answer.",
},
{ name: "a newer live retraction", snapshot: "Old tail.", newer: "", expectedTail: null },
])(
"restores $name after a missed Gateway replacement",
async ({ snapshot, newer, expectedTail }) => {
const runId = "active-run";
const history = activeHistory(runId);
const prompt = message("user", "Continue working", { id: "prompt", seq: 1, runId });
history.messages = [
prompt,
message("assistant", "Saved reply.", { id: "saved", seq: 2, runId }),
];
history.inFlightRun!.text = snapshot;
const { state, response, request } = delayed(history);
state.chatMessages = [prompt];
state.chatRunId = runId;
emit(state, runId, { state: "delta", seq: 1, message: message("assistant", "Saved reply.") });
const loading = loadChatHistory(state);
expect(request).toHaveBeenCalledOnce();
if (newer !== undefined) {
emit(state, runId, {
state: "delta",
seq: 2,
replace: true,
deltaText: newer,
message: message("assistant", newer),
});
}
response.resolve(history);
await loading;
expect(state.chatRunId).toBe(runId);
expect(tail(state)).toBe(expectedTail);
expect(renderedText(state)).toEqual([
"Continue working",
"Saved reply.",
...(expectedTail ? [expectedTail] : []),
]);
},
);
it("keeps a shorter live replacement over pending history and continues its baseline", async () => {
const runId = "active-run";
const history = activeHistory(runId);
history.inFlightRun!.text = "Same.\n\nSame.";
const { state, response, request } = delayed(history);
emit(state, runId, {
state: "delta",
seq: 1,
message: message("assistant", "Same.\n\nSame."),
});
const loading = loadChatHistory(state);
expect(request).toHaveBeenCalledOnce();
emit(state, runId, {
state: "delta",
seq: 2,
replace: true,
deltaText: "Same.",
message: message("assistant", "Same."),
});
response.resolve(history);
await loading;
expect(tail(state)).toBe("Same.");
expect(renderedText(state)).toEqual(["Same."]);
emit(state, runId, {
state: "delta",
seq: 3,
deltaText: " More.",
message: message("assistant", "Same. More."),
});
expect(tail(state)).toBe("Same. More.");
expect(renderedText(state)).toEqual(["Same. More."]);
});
it("adopts the snapshot after remount replaces an unchanged run map", async () => {
@@ -450,19 +651,19 @@ it("adopts the snapshot after remount replaces an unchanged run map", async () =
it.each([
{
snapshot: "Saved opening. repeat",
snapshot: "repeat",
delta: "repeat",
live: "Saved opening. repeatrepeat",
live: "repeatrepeat",
expected: "repeatrepeat",
},
{
snapshot: "Saved opening. Buffered before reconnect. And live.",
snapshot: "Buffered before reconnect. And live.",
delta: " Buffered before reconnect.",
live: "Saved opening. Buffered before reconnect.",
expected: "Buffered before reconnect. And live.",
live: "Buffered before reconnect.",
expected: "Buffered before reconnect.",
},
])(
"reconciles an in-flight delta against snapshot $snapshot",
"keeps the ordered live baseline while snapshot $snapshot is pending",
async ({ snapshot, delta, live, expected }) => {
const history = activeHistory("run-reconnected");
history.messages = [
@@ -481,9 +682,15 @@ it.each([
response.resolve(history);
await loading;
expect(state.chatRunId).toBe("run-reconnected");
expect(state.chatStream).toBe(`Saved opening. ${expected}`);
expect(state.chatStream).toBe(expected);
expect(tail(state)).toBe(expected);
expect(state.chatMessages).toEqual(history.messages);
emit(state, "run-reconnected", {
state: "delta",
deltaText: " More.",
message: message("assistant", `${expected} More.`),
});
expect(state.chatStream).toBe(`${expected} More.`);
},
);
@@ -516,15 +723,6 @@ it("does not resurrect delayed history after a newer intervening run completes",
expect(getChatModelObservedRunId(state, history.sessionInfo)).toBeUndefined();
});
const event = (seq: number, stream: string, data: Record<string, unknown>) => ({
runId: "run-live",
seq,
stream,
ts: 899 + seq,
sessionKey: "main",
data,
});
it("retains newer live startup progress through delayed history", async () => {
const history = activeHistory("run-live");
history.inFlightRun!.events = [event(2, "run_status", { phase: "naming_worktree" })];
ui/src/pages/chat/chat-history.test.ts
+2 -12
View File
@@ -518,24 +518,14 @@ it("keeps a foreground tool when history persists a sibling's identical call id"
const backgroundMessage = addTool("run-background", "exec", { command: "background" }, 3, true);
state.toolStreamOrder = [foreground, background];
state.chatToolMessages = [foregroundMessage, backgroundMessage];
const foregroundSegment = {
text: "before foreground",
ts: 2,
runId: "run-foreground",
toolCallId,
};
state.chatStreamSegments = [
foregroundSegment,
{ text: "before background", ts: 3, runId: "run-background", toolCallId },
];
await loadChatHistory(state);
expect(state.chatRunId).toBe("run-foreground");
expect(state.chatStream).toBe("foreground still running");
// The snapshot owns the unchanged live tail; tool ownership stays run-scoped.
expect(state.chatStream).toBe("intentionally ignored on web");
expect(state.toolStreamOrder).toEqual([foreground]);
expect(state.toolStreamById.has(foreground)).toBe(true);
expect(state.toolStreamById.has(background)).toBe(false);
expect(state.chatToolMessages).toEqual([foregroundMessage]);
expect(state.chatStreamSegments).toEqual([foregroundSegment]);
});
describe("chat history run errors", () => {
ui/src/pages/chat/chat-history.ts
-2
View File
@@ -263,8 +263,6 @@ export type ChatEventPayload = {
retry?: NonNullable<Extract<ChatEvent, { state: "status" }>["retry"]>;
message?: unknown;
deltaText?: string;
itemId?: string;
itemStartOffset?: number;
replace?: boolean;
errorMessage?: string;
errorKind?: Extract<ChatEvent, { state: "error" }>["errorKind"];
ui/src/pages/chat/chat-pane-history.test.ts
+19 -10
View File
@@ -166,7 +166,7 @@ describe("chat pane native history pagination", () => {
},
);
it("preserves the steer split through the refresh callback and later cumulative deltas", async () => {
it("keeps all run output above the steer through refresh and later cumulative deltas", async () => {
const history = createDeferred<ChatHistoryResult>();
const request = vi.fn(() => history.promise);
const client = { request } as unknown as GatewayBrowserClient;
@@ -220,19 +220,18 @@ describe("chat pane native history pagination", () => {
activeRunId: "run-refresh",
});
delta("Saved opening. Still working.");
const expected = [
const liveExpected = [
"Start working.",
"Saved opening.",
"Saved opening. Still working.",
"Also check the result.",
"Still working.",
];
try {
expect(renderedText()).toEqual(expected);
expect(renderedText()).toEqual(liveExpected);
pane.render();
expect(pane.chatProps).toBeDefined();
pane.chatProps!.onRefresh();
expect(renderedText()).toEqual(expected);
expect(renderedText()).toEqual(liveExpected);
history.resolve({
messages: [
original,
@@ -251,12 +250,22 @@ describe("chat pane native history pagination", () => {
hasActiveRun: true,
activeRunIds: ["run-refresh"],
},
inFlightRun: { runId: "run-refresh", text: "Saved opening. Still working." },
inFlightRun: { runId: "run-refresh", text: "Still working." },
});
await vi.waitFor(() => expect(state.chatLoading).toBe(false));
expect(renderedText()).toEqual(expected);
delta("Saved opening. Still working. More progress.");
expect(renderedText()).toEqual([...expected.slice(0, -1), "Still working. More progress."]);
expect(renderedText()).toEqual([
"Start working.",
"Saved opening.",
"Still working.",
"Also check the result.",
]);
delta("Still working. More progress.");
expect(renderedText()).toEqual([
"Start working.",
"Saved opening.",
"Still working. More progress.",
"Also check the result.",
]);
} finally {
state.connected = false;
history.resolve({ messages: [] });
ui/src/pages/chat/chat-queue-steer-lifecycle.test.ts
+20 -13
View File
@@ -191,9 +191,12 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
const delivered = [original.content, "Already visible output.", queued.text];
expect.soft(snapshot(), "request dispatch").toEqual({ thread: delivered, queue: [] });
let continued = false;
let prefixPersisted = false;
const continueOutput = () => {
continued = true;
history.inFlightRun!.text = "Already visible output. Later output.";
history.inFlightRun!.text = prefixPersisted
? "Later output."
: "Already visible output. Later output.";
handleChatGatewayEvent(host, {
sessionKey,
runId: "active-run",
@@ -202,7 +205,13 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
});
};
const expected = () => ({
thread: [...delivered, ...(continued ? ["Later output."] : [])],
thread: [
original.content,
...(prefixPersisted
? ["Already visible output.", ...(continued ? ["Later output."] : [])]
: [continued ? "Already visible output. Later output." : "Already visible output."]),
queued.text,
],
queue: [],
});
if (ackMode === "retry") {
@@ -219,7 +228,7 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
transport = createDeferred();
retry!.click();
await retryRequested.promise;
expect.soft(snapshot(), "retry retains original boundary").toEqual(expected());
expect.soft(snapshot(), "retry retains original run ownership").toEqual(expected());
}
(ackMode === "retry" ? retryAck : ack).resolve({
runId: queued.sendRunId,
@@ -259,6 +268,8 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
__openclaw: { id: "pre-steer", seq: 2, runId: "active-run" },
},
];
prefixPersisted = true;
history.inFlightRun!.text = "";
await loadChatHistory(host);
const cached = readChatSessionSnapshot(host.chatMessagesBySession!, host, { sessionKey });
expect(cached).not.toBeNull();
@@ -292,10 +303,7 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
if (!continued) {
continueOutput();
}
expect.soft(snapshot(), "continued output").toEqual({
thread: [...delivered, "Later output."],
queue: [],
});
expect.soft(snapshot(), "continued output").toEqual(expected());
const steer = {
role: "user",
content: queued.text,
@@ -313,12 +321,11 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
});
history.messages = [...history.messages!, steer];
history.pendingInputs = { items: [], total: 0, queuedCount: 0 };
history.inFlightRun!.text = "Already visible output. Later output.";
history.inFlightRun!.text = prefixPersisted
? "Later output."
: "Already visible output. Later output.";
await loadChatHistory(host);
expect.soft(snapshot(), "persisted copy replaces optimistic").toEqual({
thread: [...delivered, "Later output."],
queue: [],
});
expect.soft(snapshot(), "persisted copy replaces optimistic").toEqual(expected());
history.messages = [
original,
{
@@ -345,7 +352,7 @@ it.each(["custody", "receipt", "retry", "remount"] as const)(
};
await loadChatHistory(host);
expect.soft(snapshot(), "finished history").toEqual({
thread: [...delivered, "Later output."],
thread: [original.content, "Already visible output.", "Later output.", queued.text],
queue: [],
});
} finally {
ui/src/pages/chat/chat-send-delivery.ts
-4
View File
@@ -70,7 +70,6 @@ import {
reconcileChatRunLifecycle,
} from "./run-lifecycle.ts";
import { scheduleChatScroll } from "./scroll.ts";
import { rolloverChatStream } from "./stream-causal-boundary.ts";
import { resetToolStream } from "./tool-stream-state.ts";
import { buildLocalUserMessage } from "./user-message-content.ts";
@@ -325,9 +324,6 @@ async function sendPreparedChatMessage(
scope: readChatSessionProjectionScope(host, { sessionKey, agentId: prepared.agentId }),
},
);
// The dispatched steer owns one live boundary even while custody and its
// transcript receipt are in flight. A retry cannot close that interval again.
rolloverChatStream(host, { runId: steerTargetRunId, boundaryRunId: runId });
}
}
const ack = await requestChatSend(host, {
ui/src/pages/chat/chat-state-contract.ts
-2
View File
@@ -72,8 +72,6 @@ export type ChatState = StreamCausalBoundaryState & {
/** Producer-cumulative text; visible tails derive from the segment baseline. */
chatStream: string | null;
/** Identified assistant item at the tail of the current cumulative stream. */
chatStreamItemId?: string;
chatStreamItemStartOffset?: number;
chatStreamStartedAt: number | null;
chatRunStartup?: ChatRunStartupState | null;
lastError: string | null;
ui/src/pages/chat/chat-state.test.ts
+117 -35
View File
@@ -145,11 +145,12 @@ describe("canonical session message recovery", () => {
const text = "I am checking the files and will report the result.";
const partial = text.slice(0, text.indexOf(" will report"));
const { state } = createSessionEventState({ chatRunId: runId });
const delta = (value: string) =>
const delta = (value: string, replace = false) =>
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
state: "delta",
...(replace ? { deltaText: value, replace: true } : {}),
message: { role: "assistant", content: [{ type: "text", text: value }] },
});
const item = (seq: number) =>
@@ -202,6 +203,7 @@ describe("canonical session message recovery", () => {
delta(partial);
expect(visible()).toEqual([{ role: "assistant", text: partial }]);
}
delta("", true);
item(10);
expect(visible()).toEqual(single);
if (persistence === "before tool") {
@@ -221,10 +223,10 @@ describe("canonical session message recovery", () => {
persist();
expect(visible()).toEqual(single);
}
delta(`${partial} will report`);
item(11);
expect(visible()).toEqual(single);
// The final chunk also brings a distinct, identically worded occurrence.
delta(`${text}\n\n${text}`);
// The next assistant message is a distinct, identically worded occurrence.
delta(text);
expect(visible()).toEqual([...single, ...single]);
if (persistence === "after final delta") {
persist();
@@ -243,11 +245,12 @@ describe("canonical session message recovery", () => {
const runId = "active-run";
const { state } = createSessionEventState({ chatRunId: runId });
let seq = 0;
const delta = (text: string) =>
const delta = (text: string, replace = false) =>
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
state: "delta",
...(replace ? { deltaText: text, replace: true } : {}),
message: { role: "assistant", content: text },
});
const tool = (id: string) =>
@@ -260,7 +263,8 @@ describe("canonical session message recovery", () => {
data: { phase: "start", toolCallId: id, name: "read", args: {} },
});
const first = "First observation.";
const persist = () =>
const persist = () => {
delta("", true);
emitGatewayEvent(state, "session.message", {
sessionKey: state.sessionKey,
sessionId: state.currentSessionId,
@@ -274,6 +278,7 @@ describe("canonical session message recovery", () => {
__openclaw: { id: "saved-first", seq: 1, runId },
},
});
};
delta(first);
if (persistFirst) {
persist();
@@ -286,7 +291,7 @@ describe("canonical session message recovery", () => {
persist();
}
for (const text of ["Next", "Next observation", "Next observation continues."]) {
delta(first + "\n\n" + text);
delta(text);
tool("overlap-" + seq);
expect(renderedTranscript(state).filter((entry) => entry.text)).toEqual([
{ role: "assistant", text: first },
@@ -589,6 +594,7 @@ describe("canonical session message recovery", () => {
chatRunId: runId,
});
let agentSeq = 0;
let persisted = false;
const delta = (snapshot: string, deltaText: string) => {
emitGatewayEvent(state, "agent", {
sessionKey: state.sessionKey,
@@ -598,6 +604,9 @@ describe("canonical session message recovery", () => {
stream: "assistant",
data: { text: snapshot, delta: deltaText },
});
if (persisted) {
return;
}
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
@@ -606,7 +615,15 @@ describe("canonical session message recovery", () => {
message: { role: "assistant", content: [{ type: "text", text: snapshot }] },
});
};
const persist = () =>
const persist = () => {
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
state: "delta",
deltaText: "",
replace: true,
message: { role: "assistant", content: [] },
});
emitGatewayEvent(state, "session.message", {
sessionKey: state.sessionKey,
runId,
@@ -619,6 +636,8 @@ describe("canonical session message recovery", () => {
__openclaw: { id: "durable-answer", seq: 2, runId },
},
});
persisted = true;
};
if (persistence === "before-stream") {
persist();
}
@@ -655,7 +674,7 @@ describe("canonical session message recovery", () => {
expect(state.chatRunId).toBe(runId);
expect(request.mock.calls.filter(([method]) => method === "chat.history")).toHaveLength(0);
// Replayed cumulative deltas must not revive the retired projection.
// Late raw assistant events have no new Gateway chat tail after persistence.
delta(text, text.slice(partial.length));
expect(renderedTranscript(state).filter((entry) => entry.text)).toEqual([
{ role: "assistant", text },
@@ -732,7 +751,7 @@ describe("canonical session message recovery", () => {
deltaCursor: "during-tool",
inFlightRun: {
runId,
text: commentary,
text: "",
startedAt: 1,
events: [
{
@@ -744,7 +763,7 @@ describe("canonical session message recovery", () => {
data: {
kind: "preamble",
itemId: "commentary-1",
progressText: commentary.replace(/\s+/gu, " "),
progressText: commentary,
},
},
{
@@ -846,7 +865,7 @@ describe("canonical session message recovery", () => {
seq: index + 4,
state: "delta",
deltaText: index === 0 ? partial : answer.slice(partial.length),
message: { role: "assistant", content: [{ type: "text", text: commentary + text }] },
message: { role: "assistant", content: [{ type: "text", text }] },
});
if (secondTool && index === 0) {
for (const phase of ["start", "result"]) {
@@ -869,6 +888,14 @@ describe("canonical session message recovery", () => {
}
}
if (persistence === "before-final") {
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
state: "delta",
deltaText: "",
replace: true,
message: { role: "assistant", content: [] },
});
emitGatewayEvent(state, "session.message", persistedPayload(savedAnswer));
}
for (const [index, phase] of ["finishing", "end"].entries()) {
@@ -883,7 +910,7 @@ describe("canonical session message recovery", () => {
}
const terminalMessage = {
role: "assistant",
content: [{ type: "text", text: commentary + answer }],
content: [{ type: "text", text: answer }],
};
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
@@ -898,7 +925,7 @@ describe("canonical session message recovery", () => {
{ role: "assistant", text: answer },
];
expect(visibleText()).toEqual(expected);
expect(terminalMessage.content).toEqual([{ type: "text", text: commentary + answer }]);
expect(terminalMessage.content).toEqual([{ type: "text", text: answer }]);
if (persistence === "after-final") {
emitGatewayEvent(state, "session.message", persistedPayload(savedAnswer));
expect(visibleText()).toEqual(expected);
@@ -933,14 +960,18 @@ describe("canonical session message recovery", () => {
data: { kind: "preamble", itemId, progressText: text },
});
}
const stream = (snapshot: string) =>
const stream = (snapshot: string, replace = false) =>
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
state: "delta",
...(replace ? { deltaText: snapshot, replace: true } : {}),
message: { role: "assistant", content: [{ type: "text", text: snapshot }] },
});
const persist = (id: string, seq: number, itemId?: string) =>
const persist = (id: string, seq: number, itemId?: string) => {
if (!itemId) {
stream("", true);
}
emitGatewayEvent(state, "session.message", {
sessionKey: state.sessionKey,
runId,
@@ -955,12 +986,13 @@ describe("canonical session message recovery", () => {
: {}),
},
});
};
persist("durable-commentary", 1, "commentary-one");
stream(text);
expect(renderedTranscript(state).map((entry) => entry.text)).toEqual([text, text, text]);
persist("first-answer", 2);
expect(renderedTranscript(state).map((entry) => entry.text)).toEqual([text, text, text]);
stream(`${text} ${text}`);
stream(text);
expect(renderedTranscript(state).map((entry) => entry.text)).toEqual([text, text, text, text]);
persist("second-answer", 3);
expect(renderedTranscript(state).map((entry) => entry.text)).toEqual([text, text, text, text]);
@@ -969,7 +1001,7 @@ describe("canonical session message recovery", () => {
});
it.each([false, true])(
"keeps cumulative output ordered across a steer (later commentary=%s)",
"keeps whole server messages above a steer (later commentary=%s)",
(laterCommentary) => {
const activeRunId = "active-run";
const steerRunId = "steer-request";
@@ -1004,6 +1036,30 @@ describe("canonical session message recovery", () => {
]);
expect(state.chatRunId).toBe(activeRunId);
expect(state.chatQueue).toEqual([]);
if (laterCommentary) {
// Embedded turns commit the prior assistant before admitting a durable steer.
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId: activeRunId,
state: "delta",
deltaText: "",
replace: true,
message: { role: "assistant", content: [] },
});
emitGatewayEvent(state, "session.message", {
sessionKey: state.sessionKey,
runId: activeRunId,
hasActiveRun: true,
messageId: "saved-before-steer",
messageSeq: 2,
message: {
role: "assistant",
content: [{ type: "text", text: "Before steer." }],
timestamp: 101,
__openclaw: { id: "saved-before-steer", seq: 2, runId: activeRunId },
},
});
}
reduceChatSessionProjection(state, {
type: "sendPending",
runId: steerRunId,
@@ -1024,7 +1080,7 @@ describe("canonical session message recovery", () => {
clientRunId: activeRunId,
hasActiveRun: true,
messageId: "persisted-steer-user",
messageSeq: 2,
messageSeq: laterCommentary ? 3 : 2,
message: {
role: "user",
content: [{ type: "text", text: "Steer prompt" }],
@@ -1032,7 +1088,7 @@ describe("canonical session message recovery", () => {
__openclaw: {
id: "persisted-steer-user",
idempotencyKey: `${steerRunId}:user`,
seq: 2,
seq: laterCommentary ? 3 : 2,
steerTargetRunId: activeRunId,
},
},
@@ -1041,7 +1097,6 @@ describe("canonical session message recovery", () => {
handlePageGatewayEvent(state, steerEvent);
expect(state.chatRunId).toBe(activeRunId);
const segmentsAfterRequestBoundary = state.chatStreamSegments;
expect(segmentsAfterRequestBoundary.at(-1)?.boundaryRunId).toBe(steerRunId);
expect(state.chatStreamSegments).toBe(segmentsAfterRequestBoundary);
expect(
state.chatMessages.filter((message) => extractText(message) === "Steer prompt"),
@@ -1050,24 +1105,41 @@ describe("canonical session message recovery", () => {
sessionKey: state.sessionKey,
runId: activeRunId,
state: "delta",
deltaText: " After steer.",
deltaText: laterCommentary ? "After steer." : " After steer.",
message: {
role: "assistant",
content: [{ type: "text", text: "Before steer. After steer." }],
content: [
{
type: "text",
text: laterCommentary ? "After steer." : "Before steer. After steer.",
},
],
},
});
expect(renderedTranscript(state)).toEqual([
{ role: "user", text: "Original prompt" },
{ role: "assistant", text: "Before steer." },
...(laterCommentary
? [
{ role: "assistant", text: "Before steer." },
{ role: "assistant", text: "After steer." },
]
: [{ role: "assistant", text: "Before steer. After steer." }]),
{ role: "user", text: "Steer prompt" },
{ role: "assistant", text: "After steer." },
]);
handlePageGatewayEvent(state, steerEvent);
expect(state.chatStreamSegments).toBe(segmentsAfterRequestBoundary);
if (laterCommentary) {
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId: activeRunId,
state: "delta",
deltaText: "",
replace: true,
message: { role: "assistant", content: [] },
});
emitGatewayEvent(state, "agent", {
sessionKey: state.sessionKey,
runId: activeRunId,
@@ -1085,16 +1157,18 @@ describe("canonical session message recovery", () => {
runId: activeRunId,
hasActiveRun: true,
messageId: "saved-commentary",
messageSeq: 3,
messageSeq: 4,
message: {
role: "assistant",
content: [{ type: "text", text: "After steer." }],
openclawStreamFallback: { source: "segment", itemId: "after-steer-commentary" },
__openclaw: { id: "saved-commentary", seq: 3, runId: activeRunId },
__openclaw: { id: "saved-commentary", seq: 4, runId: activeRunId },
},
});
}
const terminalText = "Before steer. After steer. Final unseen suffix.";
const terminalText = laterCommentary
? "Before steer. Final unseen suffix."
: "Before steer. After steer. Final unseen suffix.";
const terminalMessage = {
role: "assistant",
content: [{ type: "text", text: terminalText }],
@@ -1107,14 +1181,14 @@ describe("canonical session message recovery", () => {
});
expect(renderedTranscript(state)).toEqual([
{ role: "user", text: "Original prompt" },
{ role: "assistant", text: "Before steer." },
{ role: "user", text: "Steer prompt" },
...(laterCommentary
? [
{ role: "assistant", text: "Before steer." },
{ role: "assistant", text: "After steer." },
{ role: "assistant", text: " Final unseen suffix." },
]
: [{ role: "assistant", text: "After steer. Final unseen suffix." }]),
: []),
{ role: "assistant", text: terminalText },
{ role: "user", text: "Steer prompt" },
]);
expect(terminalMessage.content).toEqual([{ type: "text", text: terminalText }]);
},
@@ -1399,6 +1473,15 @@ describe("canonical session message recovery", () => {
messages: [originalPrompt, abandonedPartial, localUser, localFinal],
sessionId: state.currentSessionId,
});
emitGatewayEvent(state, "chat", {
sessionKey: state.sessionKey,
runId,
seq: 5,
state: "delta",
replace: true,
deltaText: "",
message: { role: "assistant", content: [] },
});
emitGatewayEvent(state, "session.message", {
sessionKey: state.sessionKey,
runId,
@@ -2017,7 +2100,7 @@ describe("canonical session message recovery", () => {
expect(state.chatStreamSegments).toEqual([]);
});
it("keeps pre-steer output above an earlier ordinary queued user", () => {
it("keeps the whole live message above ordinary queued input and its later steer", () => {
const activeRunId = "active-run";
const originalPrompt = {
role: "user",
@@ -2092,10 +2175,9 @@ describe("canonical session message recovery", () => {
expect(renderedTranscript(state)).toEqual([
{ role: "user", text: "Original prompt" },
{ role: "assistant", text: "Before steer." },
{ role: "assistant", text: "Before steer. After steer." },
{ role: "user", text: "Queued follow-up" },
{ role: "user", text: "Steer prompt" },
{ role: "assistant", text: "After steer." },
]);
});
ui/src/pages/chat/chat-steer-order.test.ts
+32 -7
View File
@@ -117,9 +117,10 @@ it.each(["tool", "item"] as const)(
});
expect(rows()).toEqual([...before, steer.content]);
history.messages = [original, steer];
history.inFlightRun!.text = "Already visible answer.";
await loadChatHistory(state);
expect(rows()).toEqual([...before, steer.content]);
// Producer clocks cannot pull newly received activity above its user boundary.
// The steer remains below all output from its target run despite clock skew.
emitTool("read-after-steer", 2, 500);
await vi.runOnlyPendingTimersAsync();
handleChatGatewayEvent(state, {
@@ -128,9 +129,15 @@ it.each(["tool", "item"] as const)(
state: "delta",
message: { role: "assistant", content: "Already visible answer. Continued." },
});
const continued = [...before, steer.content, "tool", "Continued."];
const continued = [
"Original prompt",
"tool",
"tool",
"Already visible answer. Continued.",
steer.content,
];
expect(rows()).toEqual(continued);
// Completion updates the original card in place, retaining its first boundary.
// Completion updates the original card in place.
emitTool("read-before-steer", 3, 30_000, true);
await vi.runOnlyPendingTimersAsync();
expect(rows()).toEqual(continued);
@@ -151,6 +158,7 @@ it.each(["tool", "item"] as const)(
});
expect(rows()).toEqual([...continued, secondSteer.content]);
history.messages = [original, steer, secondSteer];
history.inFlightRun!.text = "Already visible answer. Continued.";
await loadChatHistory(state);
expect(rows()).toEqual([...continued, secondSteer.content]);
handleChatGatewayEvent(state, {
@@ -162,14 +170,31 @@ it.each(["tool", "item"] as const)(
content: "Already visible answer. Continued. Finishing.",
},
});
expect(rows()).toEqual([...continued, secondSteer.content, "Finishing."]);
// The producer's terminal contains only the suffix after its saved cutoffs.
expect(rows()).toEqual([
"Original prompt",
"tool",
"tool",
"Already visible answer. Continued. Finishing.",
steer.content,
secondSteer.content,
]);
// No assistant row has committed, so the terminal owns the complete reply.
handleChatGatewayEvent(state, {
sessionKey: state.sessionKey,
runId: "active-run",
state: "final",
message: { role: "assistant", content: "Finishing. Done." },
message: {
role: "assistant",
content: "Already visible answer. Continued. Finishing. Done.",
},
});
expect(rows()).toEqual([...continued, secondSteer.content, "Finishing. Done."]);
expect(rows()).toEqual([
"Original prompt",
"tool",
"tool",
"Already visible answer. Continued. Finishing. Done.",
steer.content,
secondSteer.content,
]);
},
);
ui/src/pages/chat/chat-thread-build.ts
+165 -153
View File
@@ -1,4 +1,4 @@
import { readAssistantStreamSegmentIdentity } from "@openclaw/gateway-client/browser";
import { readSessionMessageIdentity } from "@openclaw/gateway-client/browser";
import { asNullableRecord as asRecord } from "@openclaw/normalization-core/record-coerce";
import { normalizeOptionalString } from "@openclaw/normalization-core/string-coerce";
import { composeTranscriptDisplay } from "../../../../src/chat/transcript-display-position.js";
@@ -62,7 +62,6 @@ import {
insertChatItemsByTimestamp,
sanitizeStreamText,
timestampAfterVisibleItems,
userTurnRunId,
transcriptPositionTimestamp,
type TurnInsertionBounds,
} from "./chat-thread-items.ts";
@@ -79,7 +78,7 @@ import {
} from "./chat-thread-run-identity.ts";
import { coalesceToolActivityMessages } from "./chat-tool-activity-coalesce.ts";
import { safeNormalizeMessage } from "./chat-turn-boundary.ts";
import { latestPersistedSteerBoundary } from "./stream-causal-boundary.ts";
import { persistedSteerTargetRunId } from "./stream-causal-boundary.ts";
import type { CompactionStatus } from "./tool-stream-contract.ts";
export type BuildChatItemsProps = ChatInputPlacementProps & {
@@ -122,16 +121,54 @@ function canvasAssistantItemKey(
return identity ? `canvas:${identity}` : `${fallback}:canvas`;
}
/** Move selected rows after their run output without changing saved transcript order. */
function orderSteersAfterRunOutput<T>(
items: T[],
steerTarget: (item: T) => string | null,
outputRuns: (item: T) => Iterable<string>,
): T[] {
const lastOutput = new Map<string, number>();
for (const [index, item] of items.entries()) {
for (const runId of outputRuns(item)) {
lastOutput.set(runId, index);
}
}
const deferred = new Map<number, T[]>();
return items.flatMap((item, index) => {
const target = steerTarget(item);
const after = target ? lastOutput.get(target) : undefined;
if (after !== undefined && after > index) {
const steers = deferred.get(after) ?? [];
steers.push(item);
deferred.set(after, steers);
return [];
}
return [item, ...(deferred.get(index) ?? [])];
});
}
export function buildChatItems(
props: BuildChatItemsProps,
inputOrder: ChatInputOrderState = { keys: [] },
): Array<ChatItem | MessageGroup> {
let items: ChatItem[] = [];
const outputRuns = new Map<string, Set<string>>();
const ownOutput = <T extends ChatItem>(item: T, sourceRunId: unknown): T => {
const runId = normalizeOptionalString(sourceRunId);
if (runId) {
const runs = outputRuns.get(item.key) ?? new Set<string>();
runs.add(runId);
outputRuns.set(item.key, runs);
}
return item;
};
const tools = props.toolMessages.filter(
(message): message is Record<string, unknown> => asRecord(message) !== null,
);
const toolItems = buildMessageItems(tools).map((item) => {
const projection: ChatProjection<typeof item> = { item };
const projection: ChatProjection<typeof item> = {
item: ownOutput(item, transcriptRunId(item.message)),
};
return {
projection,
runId: normalizeOptionalString(item.message.runId),
@@ -165,31 +202,32 @@ export function buildChatItems(
props.persistCommentary === false && (props.runWorking || props.runActive)
? normalizeOptionalString(resolveProgress().runId)
: undefined;
const history = composeTranscriptDisplay(
props.messages.filter(
(message) =>
!isAssistantHeartbeatAckForDisplay(message) &&
(props.persistCommentary !== false ||
!isKeyedAssistantStreamFallbackMessage(message) ||
(activeCommentaryRunId !== undefined &&
transcriptRunId(message) === activeCommentaryRunId)),
const history = orderSteersAfterRunOutput(
composeTranscriptDisplay(
props.messages.filter(
(message) =>
!isAssistantHeartbeatAckForDisplay(message) &&
(props.persistCommentary !== false ||
!isKeyedAssistantStreamFallbackMessage(message) ||
(activeCommentaryRunId !== undefined &&
transcriptRunId(message) === activeCommentaryRunId)),
),
),
(message) =>
readSessionMessageIdentity(message)?.role === "user"
? persistedSteerTargetRunId(message)
: null,
(message) => {
const runId =
readSessionMessageIdentity(message)?.role === "user" ? undefined : transcriptRunId(message);
return runId ? [runId] : [];
},
);
const searchFiltering = props.searchOpen === true && Boolean(props.searchQuery?.trim());
const hiddenHistoryKeys = new Set<string>();
const persistedCanvasIdentities = new Set<string>();
const normalizedHistory = history.map(safeNormalizeMessage);
const historyItems = buildMessageItems(history);
const persistedCommentaryKeys = new Map<string, string>();
for (const [index, message] of history.entries()) {
const identity = readAssistantStreamSegmentIdentity(message);
if (identity) {
persistedCommentaryKeys.set(
`${identity.runId ?? ""}\u0000${identity.itemId}`,
historyItems[index]!.key,
);
}
}
let canvasTurn: {
previews: { preview: CanvasToolPreview; item: (typeof historyItems)[number] }[];
lastMatchingAssistantIndex: number;
@@ -237,11 +275,16 @@ export function buildChatItems(
rawMessageTimestamp(msg) ?? Date.now(),
i,
);
items.push({
...divider,
compactionId: divider.key,
...(matchesLive && compactionKey ? { key: compactionKey } : {}),
});
items.push(
ownOutput(
{
...divider,
compactionId: divider.key,
...(matchesLive && compactionKey ? { key: compactionKey } : {}),
},
transcriptRunId(msg) ?? (matchesLive ? compaction?.runId : undefined),
),
);
hasPersistedCompaction ||= matchesLive;
continue;
}
@@ -258,7 +301,12 @@ export function buildChatItems(
if (role === "system") {
const text = extractTextCached(msg);
if (text?.trim()) {
items.push({ kind: "notice", key: itemKey, text, timestamp: normalized.timestamp });
items.push(
ownOutput(
{ kind: "notice", key: itemKey, text, timestamp: normalized.timestamp },
transcriptRunId(msg),
),
);
}
continue;
}
@@ -279,6 +327,7 @@ export function buildChatItems(
if (persistedCanvasSource && matchingCanvas) {
// Enrich the owned display row, including a later assistant shortcode,
// without changing transcript input or introducing a second widget card.
ownOutput(matchingCanvas.item, transcriptRunId(msg));
matchingCanvas.item.message = appendCanvasBlockToAssistantMessage(
matchingCanvas.item.message,
persistedCanvasSource.preview,
@@ -290,14 +339,19 @@ export function buildChatItems(
!matchingCanvas &&
(!searchFiltering || canvasTurns[i]!.lastMatchingAssistantIndex > i);
if (persistedCanvasSource && renderPersistedPreview) {
items.push({
kind: "message",
key: canvasAssistantItemKey(msg, persistedCanvasSource, itemKey),
message: createCanvasAssistantMessage(
persistedCanvasSource,
persistedCanvasSource.timestamp ?? transcriptPositionTimestamp(history, i),
items.push(
ownOutput(
{
kind: "message",
key: canvasAssistantItemKey(msg, persistedCanvasSource, itemKey),
message: createCanvasAssistantMessage(
persistedCanvasSource,
persistedCanvasSource.timestamp ?? transcriptPositionTimestamp(history, i),
),
},
transcriptRunId(msg),
),
});
);
}
if (!props.showToolCalls && isToolResult && !hasSessionsYieldCall(msg)) {
@@ -319,7 +373,11 @@ export function buildChatItems(
continue;
}
items.push(...projectChatSystemNotice(item, normalized));
items.push(
...projectChatSystemNotice(item, normalized).map((projected) =>
role === "user" ? projected : ownOutput(projected, transcriptRunId(msg)),
),
);
}
const currentRunId =
props.runId ??
@@ -375,14 +433,17 @@ export function buildChatItems(
canvasMaximumIndex,
);
if (assistant) {
items[assistant.index] = {
...assistant.item,
message: appendCanvasBlockToAssistantMessage(
assistant.item.message,
preview.preview,
preview.text,
),
};
items[assistant.index] = ownOutput(
{
...assistant.item,
message: appendCanvasBlockToAssistantMessage(
assistant.item.message,
preview.preview,
preview.text,
),
},
transcriptRunId(projection.item.message),
);
continue;
}
if (searchFiltering) {
@@ -401,11 +462,18 @@ export function buildChatItems(
preview.timestamp != null && nextTimestamp != null
? Math.min(preview.timestamp, nextTimestamp)
: preview.timestamp;
items.splice(insertionIndex, 0, {
kind: "message",
key: canvasAssistantItemKey(projection.item.message, preview, projection.item.key),
message: createCanvasAssistantMessage(preview, timestamp),
});
items.splice(
insertionIndex,
0,
ownOutput(
{
kind: "message",
key: canvasAssistantItemKey(projection.item.message, preview, projection.item.key),
message: createCanvasAssistantMessage(preview, timestamp),
},
transcriptRunId(projection.item.message),
),
);
}
items = items.filter(
(item) => item.kind !== "message" || hasRenderableNormalizedMessage(item.message),
@@ -414,31 +482,22 @@ export function buildChatItems(
if (compaction && compactionKey && !hasPersistedCompaction) {
const timestamp = compaction.startedAt ?? compaction.completedAt ?? Date.now();
projections.push({
item: {
...buildCompactionDividerItem(
{},
timestamp,
0,
compaction.phase === "complete" ? "complete" : "active",
),
key: compactionKey,
},
item: ownOutput(
{
...buildCompactionDividerItem(
{},
timestamp,
0,
compaction.phase === "complete" ? "complete" : "active",
),
key: compactionKey,
},
compaction.runId,
),
});
}
const afterBoundaryBySegment = new Map<ChatStreamSegment, string>();
let latestBoundaryRunId: string | undefined;
for (const segment of segments) {
const afterBoundaryRunId =
normalizeOptionalString(segment.afterBoundaryRunId) ?? latestBoundaryRunId;
if (afterBoundaryRunId) {
afterBoundaryBySegment.set(segment, afterBoundaryRunId);
}
latestBoundaryRunId = normalizeOptionalString(segment.boundaryRunId) ?? latestBoundaryRunId;
}
const keyedSegments = segments.filter(streamSegmentHasItemId);
const indexedSegments = segments.filter(
(segment) => !streamSegmentHasItemId(segment) && segment.boundaryMarker !== true,
);
const indexedSegments = segments.filter((segment) => !streamSegmentHasItemId(segment));
const toolLookup = createToolCallLookup<ChatProjection>();
for (const tool of toolItems) {
toolLookup.add(tool.runId, tool.callId, tool.projection);
@@ -446,76 +505,37 @@ export function buildChatItems(
// Empty user rows may have disappeared since canvas placement. Preserve the
// earlier current-turn fallback, but resolve exact bounds over rendered rows.
const projectionRunBounds = createRunTurnLookup(executionItems());
const resolveProjectionBounds = (
runId: unknown,
boundaryRunId?: string,
afterBoundaryRunId?: string,
): TurnInsertionBounds | undefined => {
const bounds =
const resolveProjectionBounds = (runId: unknown): TurnInsertionBounds | undefined =>
boundToPendingInputs(
resolveRunInsertionBounds(projectionRunBounds, runId, currentRunId, currentTurnBounds) ??
(!runId && activeInputKey ? currentTurnBounds : undefined);
const boundaryKey = boundaryRunId ? projectionRunBounds(boundaryRunId)?.afterKey : undefined;
const afterBounds = afterBoundaryRunId ? projectionRunBounds(afterBoundaryRunId) : undefined;
return boundToPendingInputs(
bounds || boundaryKey || afterBounds
? {
...bounds,
...(afterBounds?.afterKey ? { afterKey: afterBounds.afterKey } : {}),
...(afterBounds?.beforeKey ? { beforeKey: afterBounds.beforeKey } : {}),
...(boundaryKey ? { beforeKey: boundaryKey } : {}),
}
: undefined,
(!runId && activeInputKey ? currentTurnBounds : undefined),
);
};
if (!searchFiltering) {
if (props.archiveNotice) {
projections.push({ item: props.archiveNotice });
}
for (const notice of props.guardianNotices ?? []) {
projections.push({
item: buildGuardianNoticeItem(notice),
item: ownOutput(buildGuardianNoticeItem(notice), notice.runId),
bounds: resolveProjectionBounds(notice.runId),
});
}
}
const toolBeforeBoundaries = createToolCallLookup<string>();
const toolAfterBoundaries = createToolCallLookup<string>();
for (const segment of indexedSegments) {
const callId = normalizeOptionalString(segment.toolCallId);
const runId = normalizeOptionalString(segment.runId);
const boundaryRunId = normalizeOptionalString(segment.boundaryRunId);
const afterBoundaryRunId = afterBoundaryBySegment.get(segment);
if (boundaryRunId) {
toolBeforeBoundaries.add(runId, callId, boundaryRunId);
}
if (afterBoundaryRunId) {
toolAfterBoundaries.add(runId, callId, afterBoundaryRunId);
}
}
const appendStreamSegment = (segment: ChatStreamSegment, key: string, text: string) => {
const afterBoundaryRunId = afterBoundaryBySegment.get(segment);
const pendingItemId = normalizeOptionalString(segment.pendingCommentaryPrefixFor);
const persistedCommentaryKey = pendingItemId
? (persistedCommentaryKeys.get(
`${normalizeOptionalString(segment.runId) ?? ""}\u0000${pendingItemId}`,
) ?? persistedCommentaryKeys.get(`\u0000${pendingItemId}`))
: undefined;
const bounds = resolveProjectionBounds(
segment.runId,
segment.boundaryRunId,
afterBoundaryRunId,
);
projections.push({
item: {
kind: "stream",
key,
text,
startedAt: segment.ts,
isStreaming: false,
...optionalRunIdentity(segment.runId),
...optionalBoundaryIdentity(afterBoundaryRunId ?? segment.runId),
},
bounds: persistedCommentaryKey ? { ...bounds, beforeKey: persistedCommentaryKey } : bounds,
item: ownOutput(
{
kind: "stream",
key,
text,
startedAt: segment.ts,
isStreaming: false,
...optionalRunIdentity(segment.runId),
...optionalBoundaryIdentity(segment.runId),
},
segment.runId,
),
bounds: resolveProjectionBounds(segment.runId),
});
};
let previousAccumulatedStreamText: string | null = null;
@@ -550,13 +570,7 @@ export function buildChatItems(
}
const tool = toolItems[i];
if (tool && (props.showToolCalls || hasSessionsYieldCall(tool.projection.item.message))) {
const before =
normalizeOptionalString(tool.projection.item.message.boundaryRunId) ??
toolBeforeBoundaries.get(tool.runId, tool.callId);
const after =
normalizeOptionalString(tool.projection.item.message.afterBoundaryRunId) ??
toolAfterBoundaries.get(tool.runId, tool.callId);
tool.projection.bounds = resolveProjectionBounds(tool.runId, before, after);
tool.projection.bounds = resolveProjectionBounds(tool.runId);
projections.push(tool.projection);
}
}
@@ -582,7 +596,7 @@ export function buildChatItems(
startedAt: prompt.createdAtMs,
};
projections.push({
item: questionItem,
item: ownOutput(questionItem, prompt.runId),
bounds: prompt.runId ? resolveProjectionBounds(prompt.runId) : undefined,
});
}
@@ -614,22 +628,14 @@ export function buildChatItems(
if (props.runWorking !== true && props.stream === null && !showWorkingIndicator) {
clearWorkingProgress(props.sessionKey);
}
const persistedBoundary = currentRunId
? latestPersistedSteerBoundary(history, currentRunId)
: null;
const activeBoundaryRunId =
persistedBoundary &&
(!latestBoundaryRunId ||
history.findIndex((message) => userTurnRunId(message) === latestBoundaryRunId) <=
persistedBoundary.index)
? persistedBoundary.runId
: latestBoundaryRunId;
const activeTurnRunId = activeBoundaryRunId ?? currentRunId;
const activeTurnBounds = boundToPendingInputs(
(activeTurnRunId ? createRunTurnLookup(executionItems())(activeTurnRunId) : null) ??
(currentRunId ? createRunTurnLookup(executionItems())(currentRunId) : null) ??
(activeInputKey ? { afterKey: activeInputKey } : null),
);
const appendActiveRunItem = (item: ChatItem) => {
const appendActiveRunItem = (
item: Extract<ChatItem, { kind: "stream" | "reading-indicator" }>,
) => {
ownOutput(item, item.runId);
// Queued custody is a ceiling for the whole live response, not just its text.
// Moving its working indicator past that ceiling splits and remeasures the run.
if (activeTurnBounds) {
@@ -648,14 +654,12 @@ export function buildChatItems(
const liveRunId = props.runId ?? liveProgress.runId;
const liveStreamItem: ChatItem = {
kind: "stream",
key: activeBoundaryRunId
? `${liveProgress.key}:after:${activeBoundaryRunId}`
: liveProgress.key,
key: liveProgress.key,
text: visibleText,
startedAt: timestampAfterVisibleItems(items, props.streamStartedAt ?? Date.now()),
isStreaming: true,
...optionalRunIdentity(liveRunId),
...optionalBoundaryIdentity(activeBoundaryRunId ?? liveRunId),
...optionalBoundaryIdentity(liveRunId),
};
appendActiveRunItem(liveStreamItem);
}
@@ -668,7 +672,7 @@ export function buildChatItems(
key: workingProgress.key,
startedAt: workingProgress.startedAt,
...optionalRunIdentity(workingRunId),
...optionalBoundaryIdentity(activeBoundaryRunId ?? workingRunId),
...optionalBoundaryIdentity(workingRunId),
});
} else if (props.subagentWait && !initialHistoryLoad) {
// The handoff ended the parent's run, not its work. Carrying that run's
@@ -681,6 +685,14 @@ export function buildChatItems(
runId: props.subagentWait.runId,
});
}
items = orderSteersAfterRunOutput(
items,
(item) =>
item.kind === "message" && readSessionMessageIdentity(item.message)?.role === "user"
? persistedSteerTargetRunId(item.message)
: null,
(item) => outputRuns.get(item.key) ?? [],
);
// Place output against the complete transcript before search hides any rows.
// Pending/local inputs contribute people and turn boundaries just like history;
// queued future inputs must remain after the live output they do not own.
ui/src/pages/chat/chat-thread-grouping.test.ts
+39
View File
@@ -347,6 +347,45 @@ describe("content classification", () => {
});
describe("answer visibility across continuations", () => {
it.each([true, false])(
"keeps work with its answer above a trailing steer (working=%s)",
(runWorking) => {
const runId = "target-run";
const prompt = message("user", "Inspect the file", 1, {
__openclaw: { idempotencyKey: `${runId}:user` },
});
const evidence = tool("read-file", 2, { runId });
const answer = message("assistant", "File inspected", 3, { runId });
const queued = message("user", "Queued follow-up", 4, {
__openclaw: { idempotencyKey: "queued-run:user" },
});
const steer = message("user", "Also check permissions", 5, {
__openclaw: {
idempotencyKey: "steer-run:user",
steerTargetRunId: runId,
},
});
const ordered = [prompt, evidence, answer, queued, steer];
const groups = groupMessages(
ordered.map((entry, index) => ({
kind: "message",
key: `message:${index}`,
message: entry,
})),
);
const items = collapseCompletedTurnWork(groups, {
sessionKey,
runWorking,
session: { key: sessionKey, lastRunId: runId, status: "done", runtimeMs: 321 },
});
expect(visible(items)).toEqual(runWorking ? ordered : [prompt, answer, queued, steer]);
expect(work(items)).toEqual(runWorking ? [] : [evidence]);
if (!runWorking) {
expect(items[1]).toMatchObject({ kind: "work-group", replyRunId: runId, durationMs: 321 });
}
},
);
const terminalCases = [
{ name: "settled terminal", terminal: true, preserved: true },
{ name: "intermediate text", terminal: false, preserved: false },
ui/src/pages/chat/chat-thread-grouping.ts
+24 -45
View File
@@ -26,7 +26,7 @@ import {
hasForwardedSource,
isInterSessionMessage,
} from "./chat-turn-boundary.ts";
import { indexTurnContinuations, persistedSteerTargetRunId } from "./stream-causal-boundary.ts";
import { persistedSteerTargetRunId } from "./stream-causal-boundary.ts";
function assistantMessageKind(message: unknown, visibleContent: MessageGroup["visibleContent"]) {
return resolveAssistantReplyPhase(message) ?? (visibleContent === "none" ? "activity" : "reply");
@@ -501,44 +501,27 @@ export function collapseCompletedTurnWork(
turns.push(currentTurn);
}
const { continuationTurnIndexes, precedingContinuationTurnIndexes } = indexTurnContinuations(
turns,
turnUserMessages,
);
const terminalReplies = turns.map((turn, turnIndex) =>
continuationTurnIndexes.has(turnIndex) ? undefined : turn.findLast(isFinalReplyGroup),
);
const finalReplyIndexes = turns.map((turn, index) => {
const reply = terminalReplies[index];
return reply ? turn.lastIndexOf(reply) : -1;
});
for (let turnIndex = turns.length - 2; turnIndex >= 0; turnIndex -= 1) {
const continuationTurnIndex = continuationTurnIndexes.get(turnIndex);
if (!terminalReplies[turnIndex] && continuationTurnIndex !== undefined) {
terminalReplies[turnIndex] = terminalReplies[continuationTurnIndex];
}
}
const liveTurnIndexes = new Set<number>();
if (opts.runWorking) {
let liveTurnIndex = turns.length - 1;
liveTurnIndexes.add(liveTurnIndex);
for (;;) {
const precedingTurnIndex = precedingContinuationTurnIndexes.get(liveTurnIndex);
if (precedingTurnIndex === undefined) {
break;
}
liveTurnIndex = precedingTurnIndex;
liveTurnIndexes.add(liveTurnIndex);
}
}
const steerTarget = opts.runWorking
? turnUserMessages(turns.at(-1) ?? [])
.map(persistedSteerTargetRunId)
.findLast((runId) => runId !== null)
: undefined;
const targetTurnIndex = steerTarget
? turns.findIndex((turn) =>
turnUserMessages(turn).some((message) => userTurnRunId(message) === steerTarget),
)
: -1;
const liveTurnIndex = opts.runWorking
? targetTurnIndex >= 0
? targetTurnIndex
: turns.length - 1
: -1;
const result: Array<TurnRenderItem | WorkGroupRenderItem> = [];
for (const [turnIndex, turn] of turns.entries()) {
// In-flight content (stream runs, streaming groups) marks the turn live.
// While the run works, the trailing turn also stays expanded so activity
// is watchable until the terminal rebuild collapses it.
// A trailing steer names the still-working turn above it; it does not own that work.
const isLive =
liveTurnIndexes.has(turnIndex) ||
turnIndex === liveTurnIndex ||
turn.some(
(item) => item.kind === "stream-run" || (item.kind === "group" && item.isStreaming),
);
@@ -546,17 +529,17 @@ export function collapseCompletedTurnWork(
result.push(...turn);
continue;
}
const finalReplyIndex = finalReplyIndexes[turnIndex] ?? -1;
const terminalReply = terminalReplies[turnIndex];
const terminalReply = turn.findLast(isFinalReplyGroup);
// Without a final reply, the tool rows are the turn's only visible result.
// Keep them exposed instead of replacing the result with an opaque rollup.
if (!terminalReply) {
result.push(...turn);
continue;
}
const finalReplyIndex = turn.lastIndexOf(terminalReply);
// Partition the answer's output segment, including work after the last answer.
// Never move activity across a user, forwarded input, or structural marker.
let segmentStart = finalReplyIndex >= 0 ? finalReplyIndex : turn.length - 1;
let segmentStart = finalReplyIndex;
let segmentEnd = segmentStart;
while (segmentStart > 0 && isTurnOutputGroup(turn[segmentStart - 1]!)) {
segmentStart -= 1;
@@ -609,7 +592,7 @@ export function collapseCompletedTurnWork(
if (
index !== finalReplyIndex &&
isCollapsibleWorkGroup(item) &&
(finalReplyIndex < 0 || index < finalReplyIndex || !groupHasFailedResult(item))
(index < finalReplyIndex || !groupHasFailedResult(item))
) {
groups.push(item);
if (precedingAnswerKey) {
@@ -642,11 +625,9 @@ export function collapseCompletedTurnWork(
runtimeMs >= 0
? runtimeMs
: null;
const continuationBoundary = turns[continuationTurnIndexes.get(turnIndex) ?? -1]?.[0];
// A completed rollup may span automatic resumptions. Its reply owns the
// display only when reaching it crosses neither another answer’s run nor a steer.
// display only when earlier answers belong to that same run.
const replyRunId =
finalReplyIndex >= 0 &&
!hasForwardedSource(terminalReply) &&
!groups.some(hasForwardedSource) &&
answers
@@ -663,9 +644,7 @@ export function collapseCompletedTurnWork(
result.push({
kind: "work-group",
// The final reply survives older-history prepends; the first work row does not.
key: `work:${
finalReplyIndex >= 0 || !continuationBoundary ? terminalReply.key : continuationBoundary.key
}`,
key: `work:${terminalReply.key}`,
groups,
...(replyRunId ? { replyRunId } : {}),
...(previewAfterGroup.size > 0 ? { previewAfterGroup } : {}),
ui/src/pages/chat/chat-thread-input-order.test.ts
+33
View File
@@ -74,6 +74,39 @@ beforeEach(() => resetChatThreadState("input-order"));
afterEach(() => resetChatThreadState("input-order"));
describe("transcript input order", () => {
it("moves durable steers below their run's output without reordering other history", () => {
const messages = [
{ role: "user", content: "Original", __openclaw: { idempotencyKey: "run:user", seq: 1 } },
{ role: "assistant", content: "Before", __openclaw: { runId: "run", seq: 2 } },
{ role: "user", content: "Steer one", __openclaw: { steerTargetRunId: "run", seq: 3 } },
{ role: "assistant", content: "After", __openclaw: { runId: "run", seq: 4 } },
{ role: "user", content: "Steer two", __openclaw: { steerTargetRunId: "run", seq: 5 } },
{ role: "assistant", content: "Final", __openclaw: { runId: "run", seq: 6 } },
{ role: "user", content: "Next", __openclaw: { idempotencyKey: "next:user", seq: 7 } },
{ role: "assistant", content: "Other run", __openclaw: { runId: "next", seq: 8 } },
];
expect(visibleRows({ messages })).toEqual([
"Original",
"Before",
"After",
"Final",
"Steer one",
"Steer two",
"Next",
"Other run",
]);
expect(messages.map((message) => message.content)).toEqual([
"Original",
"Before",
"Steer one",
"After",
"Steer two",
"Final",
"Next",
"Other run",
]);
});
it.each(["steer", "interrupt"] as const)(
"keeps consecutive sends in submission order while a %s ACK is pending",
(queueMode) => {
ui/src/pages/chat/chat-thread-run-identity.ts
+1 -2
View File
@@ -112,8 +112,7 @@ export function resolveRunInsertionBounds(
}
const runBounds = findRunBounds(runId);
if (runId === currentRunId) {
// Active runs can span steers: the original prompt is a floor, not a ceiling.
return runBounds ? { afterKey: runBounds.afterKey } : currentTurnBounds;
return runBounds ?? currentTurnBounds;
}
if (runBounds || currentRunId == null) {
return runBounds;
Loaded 100 of 118 files, more files were not shown because too many files have changed in this diff. Show more