mirrors/pi
0
0 Fork 0
mirror of https://github.com/earendil-works/pi.git synced 2026-10-28 09:44:37 +08:00
Code Activity

feat(durable): windowed conversation view, ConversationTranscript and runFailure; remove watchEvents

Browse Source 
ConversationView.entries is a window over the fork-aware history with before;
conversation.watch(ctx, { from, min, max }) and observeConversation take the
window. ConversationTranscript merges windows into a client's transcript with
gaps, loadEarlier, fillGap and resume. pi.live.runFailure says why a run ended
without an answer. watchEvents and its event types are removed; the changelog
has the migration guide.

closes #11327
This commit is contained in:
Mario Zechner committed 2026-10-27 16:57:24 +00:00
1 parent 72363383ff
commit f3255de765
37 files changed
+3074 -2065
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

packages/durable/CHANGELOG.md
+72 -5
View File
@@ -8,23 +8,90 @@
- `Storage.commit(writes, adopt, context)` takes an `adopt(seq)` callback, the Session's in-memory adoption and publication of the batch: a storage calls it exactly once, synchronously, once the batch is durable and before its next commit starts, then resolves with `seq`, and never for a commit that fails before the batch is durable. Memory, SQLite, JSONL, and IndexedDB storages do; custom storages and wrappers must call or forward it, and the storage conformance suite checks it. A commit that rejects after its adoption is published and fails the Session without rolling the adopted changes back.
- `Storage.mintId()`'s contract is narrowed to IDs greater than every ID minted since the storage was opened and every ID stored, also across reopen. The bundled storages already behave so.
- Watches use Chord's `WatchHandle` and `WatchEnd` (Durable re-exports both, with `WatchChange` and `NativeWatchChange`). End reasons are `stopped`, `cancelled`, `ended` (was `session_closed`), `listener_error`, and `failed` (was `session_failed`); `retired` is gone (see below). An ended watch drops its listener and every reference into its Session, so a stopped watch that is still referenced keeps only its last value.
- Watch listeners take one change object before the context: `watch.start(async ({ value, ops, retired }, context) => …)` for document, conversation view (`Conversation.watch()`), and task graph watches, and `stream.start(async ({ value: events }, context) => …)` for `watchEvents()`. Migration: `start(async (value, ops, ctx) => …)` becomes `start(async ({ value, ops }, ctx) => …)`, and an event listener `(events, ctx)` becomes `({ value: events }, ctx)`.
- Watch listeners take one change object before the context: `watch.start(async ({ value, ops, retired }, context) => …)` for document, conversation view (`Conversation.watch()`), and task graph watches. Migration: `start(async (value, ops, ctx) => …)` becomes `start(async ({ value, ops }, ctx) => …)`.
- Document watches follow their address (kind, scope, and key) instead of one incarnation: `DocumentWatch<T>` is `WatchHandle<Readonly<T>>` (no `null`). A retirement no longer ends the watch with a last frame `[["r", null]]` and `retired`: the next call has `retired: true`, the last value the watch was given, and no ops (folded with what is pending; a change made in the retiring commit itself is not shown), and a document created at the address later arrives on the same watch as `[["r", value]]`, possibly in the same call. A task document's watch ends `ended` once the retirement call of the commit that makes its task terminal has returned; Session close or failure still ends it first if that call was not delivered yet (a watch never started). Acquiring an absent document still gives `undefined`. A watch pins the incarnation at its address, nothing while it is absent. Migration: test `change.retired` instead of a `null` value or a `retired` end, and stop a watch whose document may be retired for good, as it otherwise stays until the Session ends.
- Document, conversation view, and task graph watches deliver the commits made while their listener runs (or before `start()`) in one callback: the newest value, the commits' operations folded (appends to one string join, sets of one field keep the last) and the newest commit's Context. Past 10,000 pending operations the callback gets one root replacement `[["r", newest]]`. Before, each commit was its own callback and more than 100 pending commits became a root replacement. `watchEvents()` still delivers every event batch in order, with one `snapshot` past 100 pending batches.
- Document, conversation view, and task graph watches deliver the commits made while their listener runs (or before `start()`) in one callback: the newest value, the commits' operations folded (appends to one string join, sets of one field keep the last) and the newest commit's Context. Past 10,000 pending operations the callback gets one root replacement `[["r", newest]]`. Before, each commit was its own callback and more than 100 pending commits became a root replacement.
- Removed the Chord state getters `Session.documentState()`, `Conversation.viewState()`, and `Harness.taskGraph()`, and the `DocumentState` type; use `watchDoc()`, `Conversation.watch()`, and `Harness.watchTaskGraph()`. Durable no longer uses Chord's `replicatedState()`.
- Partial answers and running tool output are stored later than they are shown: their throttled commits are deferred (`CommitPersistence`, spec §4.1), so watches, views, and agent events see each at once, while Storage gets them together with the next commit that reads `pi.live` (the end of the answer, a tool settling, a run ending) and at the latest after `settings.progress.persistWithinMs` (default 1,000 ms). With 32 streams over SQLite this cuts storage transactions about 5.7x and disk writes about 2.9x for turns with parallel tools, 27x and 9x for long answers, and CPU by about a quarter. A crash now loses up to about a second of streamed content instead of 100 ms; a stream that crashes before its first store leaves no aborted entry, and that attempt's usage is not recorded. `subscribeCommits()` gets these commits only once they are stored, with exactly what was stored (held operations first; a document the commit retires shows as its retirement only, without the operations stored before it), and gets a publication for a deadline's store too; it no longer sees every partial. Migration: a `subscribeCommits()` listener that showed live progress should use `watchDoc()`, `Conversation.watch()`, or `watchEvents()`; one that mirrors Storage keeps working; set `settings.progress.persistWithinMs: 0` to store progress right after each commit, about as before (one storage transaction per partial).
- Partial answers and running tool output are stored later than they are shown: their throttled commits are deferred (`CommitPersistence`, spec §4.1), so watches and views see each at once, while Storage gets them together with the next commit that reads `pi.live` (the end of the answer, a tool settling, a run ending) and at the latest after `settings.progress.persistWithinMs` (default 1,000 ms). With 32 streams over SQLite this cuts storage transactions about 5.7x and disk writes about 2.9x for turns with parallel tools, 27x and 9x for long answers, and CPU by about a quarter. A crash now loses up to about a second of streamed content instead of 100 ms; a stream that crashes before its first store leaves no aborted entry, and that attempt's usage is not recorded. `subscribeCommits()` gets these commits only once they are stored, with exactly what was stored (held operations first; a document the commit retires shows as its retirement only, without the operations stored before it), and gets a publication for a deadline's store too; it no longer sees every partial. Migration: a `subscribeCommits()` listener that showed live progress should use `watchDoc()` or `Conversation.watch()`; one that mirrors Storage keeps working; set `settings.progress.persistWithinMs: 0` to store progress right after each commit, about as before (one storage transaction per partial).
- The change type of `FileSystem.watch()` in `@earendil-works/pi-durable/env` is renamed from `WatchChange` to `FileChange` (same shape: `{ paths }`, `{ overflow: true }`, or `{ error }`), so `WatchChange` names only a watch listener's change. Migration: import `type FileChange` from `@earendil-works/pi-durable/env` where `WatchChange` was imported from it.
- `HostCallback` gains `"telemetry"`: a `host_error` report about a telemetry adapter that threw while a commit was published. Exhaustive switches over `HostCallback` need a case for it.
- `ConversationView.entries` is a window over the transcript instead of the active transcript: the fork-aware history `Conversation.entries()` scans, in entry ID order, with head markers (compaction summaries, resets, handoffs) appended in place instead of cutting the entries before them. A watch starts with the newest 50 entries; new entries are appends, and past 100 the oldest leave in one splice down to 50, which also sets the new `ConversationView.before`, the ID of the entry just before the window (absent when the window starts the transcript). First values and overflow replacements (`[["r", view]]`) hold at most `max` entries, never the whole transcript. Migration: read the active transcript the model sees from `Conversation.context()`; render a transcript longer than the window with `ConversationTranscript`, or page older entries with `Conversation.entries()`; a client that rebuilt the head cut from the view's operations no longer gets one ([#11327](https://github.com/earendil-works/pi/issues/11327)).
- `Conversation.watch(context, window?)` and `Harness.observeConversation(id, observer, context, window?)` take a `ConversationWindow` (`from`, `min` default 50, `max` default `2 * min`): `min`/`max` bound the window, and `from` starts the first value at that entry ID when at most `max` entries are at or after it, otherwise at the newest `min`; it picks only that first value, so every entry committed after the attachment is appended, and a `from` past the newest entry starts empty. Bounds that are not integers, or `max` below `min`, reject with a `RangeError`. Each watch and observer has its own window, so the observers of one conversation no longer share one change object or one value; they share the document values ([#11327](https://github.com/earendil-works/pi/issues/11327)).
- `pi.live` is no longer empty after a run that ended without an answer: it keeps `runFailure` (below) until a later run starts. Code that compares an idle `pi.live` with `{}`, or tests for idleness that way, checks `run === undefined` instead ([#11327](https://github.com/earendil-works/pi/issues/11327)).
- Removed `watchEvents()` and its types `AgentEvent`, `AgentEventStream`, `SnapshotEvent`, `MessageChange`, and `ToolEventCall`, and spec §9.4. A renderer renders state instead of assembling it from events: everything the events carried that a renderer draws is in the conversation view, typed for producer and consumer, and new information becomes a field there rather than an event type every client must learn ([#11327](https://github.com/earendil-works/pi/issues/11327)). Migration, for a client of `watchEvents()`:
Before:
```ts
const stream = await watchEvents(harness, conversation.id, context);
initialize(stream.snapshot); // entries, run, generation, tools, compactions, inbox, agent, usage
stream.start(async ({ value: events }) => {
for (const event of events) {
if (event.type === "message_update") applyChanges(event.changes); // streamed text
else if (event.type === "message_end") appendEntry(event.entry);
else if (event.type === "tool_execution_update") applyOutput(event.toolCallId, event.output);
else if (event.type === "run_end") showIdle();
// ... and every other event type
}
});
// After a drop: attach again; the new snapshot replaces everything held.
```
After:
```ts
const transcript = new ConversationTranscript((query, limit, ctx) => conversation.entries(query, limit, undefined, ctx));
const watch = await conversation.watch(context);
render(transcript.merge(watch.value));
watch.start(async ({ value }) => render(transcript.merge(value)));
function render(view: TranscriptView): void {
const live = view.docs["pi.live"] as LiveState;
drawItems(view.items); // entries, a "load more" row per gap, "load earlier" at the top
if (live.generation?.message !== undefined) drawStreaming(live.generation.message);
for (const slot of live.tools ?? []) drawTool(slot); // status, running output, details, result entry
if (live.runFailure !== undefined) drawFailure(live.runFailure); // why the last run ended without an answer
setBusy(live.run !== undefined);
}
// A "load earlier" or "load more" row scrolled into view:
render(await transcript.loadEarlier(context));
render(await transcript.fillGap(gap, context));
// After a drop: only what was missed when it fits the window, otherwise the newest entries and a gap.
const resumed = await conversation.watch(context, transcript.resume);
```
The watch's operations carry the same deltas the events did (text and thinking appends, tool-call argument appends, output trims and appends), for a client that forwards them, for example to a remote replica. Where each event went:
| event | state |
|---|---|
| `snapshot` | the watch's first value, `watch.value`, its window bounded by `max` |
| `run_start`, `run_end` | `pi.live.run` appears, disappears |
| `turn_start`, `turn_end` | `pi.live.run.taskId` moves to the next generation; the task graph's `pi.generation` nodes, `completing` at their hold |
| `message_start`, `message_update` | `pi.live.generation.message`, the streamed answer, with its `usage` |
| `message_end`, `entry_appended` | the entry appended to `entries` |
| `tool_execution_start`, `tool_execution_update`, `tool_execution_end` | the slot in `pi.live.tools` or `nestedTools`: `status` pending, running, done; `output`, `details`, `diagnostics`; `entry` once done (absent after a fault); a nested call's `arguments`, its `summary`, and its result in the caller's `NestedCallDoc` |
| `auto_retry_start`, `auto_retry_end` | `pi.live.generation.retry` set, removed |
| `deferred_poll` | `pi.live.generation.deferred.pollAt` |
| `compaction_start`, `compaction_end` | a status in `pi.live.compactions` appears, disappears; `pi.live.compactionFailure` when it failed; the summary entry when it is placed |
| `inbox_update` | `pi.inbox` |
| `agent_changed`, `usage_changed` | `pi.agent`, `pi.usage` |
| `submission` | `Submission.status()` and `wait()`, `Harness.submission(id)`; queued items in `pi.inbox`, the run's inputs in `pi.live.run.inputs` |
| `task_failed` | the task graph (`completing` with the outcome's status), `Harness.getTask()`, and `onReport`'s `task_faulted`; for a run, `pi.live.runFailure`; for a compaction, `compactionFailure`; for a tool, a done slot without `entry` |
| a run that ended without an answer (`run_end` with an `unanswered` `submission`) | `pi.live.runFailure`: `reason` (`model_error`, `no_model`, `aborted`, `reset`, `faulted`, `orphaned`), `detail`, `taskId`, `inputs` |
Some event data has no state in the view. A tool's arguments after repair and `beforeTool` at its start: in the nested slot's `arguments` for a nested call, only in the tool task's checkpoint for a model-issued one. A submission's settlement: `Submission.status()`/`wait()` and `Harness.submission(id)`; an input settled without a run (withdrawn, `stale`, aborted while queued, `run_failed`) just leaves `pi.inbox`. A failed task's error: `Harness.getTask()` and `onReport`; the task graph keeps only live tasks, not terminal outcomes. A nested call's full result: its caller's `NestedCallDoc` (`watchDoc()`), which retires when the caller settles. Exact turn boundaries: the run's `taskId` and the task graph, with no explicit event. Whether a compaction did anything: its `CompactionResult` (`waitForTask()`), and the summary entry when it is placed.
### Added
- Deferred commits: `Session.commit()`, `Conversation.commit()`, `TaskRuntime.commit()`, and a tool's `api.commit()` take `{ persistence: { kind: "deferred", withinMs? } }`. A commit that changes only existing conversation documents with `history: "latest"` and `fork: "initial"` is then adopted in memory and shown to watches at once, and stored with the next commit that reads or writes the document, before the document leaves memory, at close, or within `withinMs` (default `settings.progress.persistWithinMs`, or `createSession(storage, { persistWithinMs })`); any other write makes it immediate. A crash loses what is held. See spec §4.1.
- Synchronous observers for adapters, such as a Chord source over a Durable view: `Session.observeDoc(token, ...scope, observer, context)`, `Harness.observeConversation(id, observer, context)`, and `Harness.observeTaskGraph(observer, context)` attach an `Observer<T>` (`advance(change, context)`, `end(failure)`) atomically on the Session line, like a watch, and return an `Observation<T>` (`value`, `detach()`; `observeDoc` gives `undefined` for an absent document). `advance` gets every change unfolded as memory adopts it, for a stored commit inside Storage's `adopt` and for a deferred one before it is stored, so it must only record it: no awaits, no Session calls, no throws. A document observer gets one call per change at its address (two for a commit that retires and recreates it), the task graph also a call outside any commit when a registry change or a failed migration changes a `blocked` reason, and a change can arrive before the attaching promise resolves. The attaching context governs only the attachment; a closed Session or an invalid argument rejects the promise. One that throws is reported as `listener_error`, detached, and ended with the error. `end` comes once, at Session close or failure, or after a task document's retirement with its task. Document, view, and task graph watches are built on them (spec §9.1).
- `ConversationTranscript` keeps what a client has of one conversation over any `ConversationView`: `merge(view)` returns the view with `items`, the client's whole transcript (entries, a `gap` where a later window did not continue it, and an `earlier` marker until the start is loaded), instead of the window; `loadEarlier(context)` and `fillGap(gap, context)` page entries through a loader such as `Conversation.entries()`, and `resume` is the window that continues it after a drop: `conversation.watch(context, transcript.resume)`. A merge costs O(items) and returns the same `items` array when it adds nothing; concurrent loads of the same page are shared ([#11327](https://github.com/earendil-works/pi/issues/11327)).
- `pi.live.runFailure` (`RunFailure`, `RunFailureReason`) records why the newest run ended without an answer, `model_error`, `no_model`, `aborted`, `reset`, `faulted`, or `orphaned`, with the run's task and inputs and a `detail` where one exists (the error text, the missing model, the fault, the orphan reason). The commit that ends the run sets it; a later commit that starts a run removes it ([#11327](https://github.com/earendil-works/pi/issues/11327)).
- `HarnessOptions.readOnly` opens a Harness as a viewer of a session another process drives: open leaves running tasks as committed instead of recovering them, every commit with writes rejects with `ReadOnlySession` before it reaches Storage, and `resume()` and every call that would run tasks reject with it.
### Changed
- Document, conversation view, task graph, and event watches use Chord's watch handles (`Watch`, and `NativeWatch` for event batches).
- Document, conversation view, and task graph watches use Chord's watch handles (`Watch`).
- A stored commit is published when Storage adopts it, inside `Storage.commit()`: the commit's telemetry events and Durable's own observers (document watches, conversation views, the task graph, the scheduler, and waits) run inside `adopt`, so two views never show one commit apart. `subscribeCommits()` listeners run one microtask later, outside the storage's synchronous step, before every wait the commit resolves and every abort signal it fires; they have no order relative to watch listeners and `onWake`. With a storage that works after adopting (JSONL's sidecar reclaim), all of these run before `Storage.commit()` resolves.
- `subscribeCommits()` listeners are read when a commit's dispatch runs: a listener added between adoption and the dispatch gets the commit only when one was registered at adoption; one added after the dispatch ran misses it.
- The abort signal of a newly marked running task fires after `subscribeCommits()` listeners instead of before them.
@@ -33,7 +100,7 @@
### Fixed
- Fixed a tool round with `control.handoff` ending the run on the tool-calling answer: the handoff's `pi.reset` entry is written and the same run continues with a new generation in the new context, so the submission settles with the eventual answer. Queued writes and selected steers are placed after the handoff text, follow-ups wait for the run's final boundary, and a queued reset still ends the run `unanswered` with reason `reset`. `terminate` remains the way to end a round without another model request.
- Fixed a conversation watcher (`watchEvents()`, conversation views) holding a copy of a streamed text or tool call per partial commit, through Chord append ops that kept the whole string alive; a 140 KB answer followed by a 140 KB tool call held well over 100 MB ([#9265](https://github.com/earendil-works/pi/issues/9265))
- Fixed a conversation watcher (conversation views) holding a copy of a streamed text or tool call per partial commit, through Chord append ops that kept the whole string alive; a 140 KB answer followed by a 140 KB tool call held well over 100 MB ([#9265](https://github.com/earendil-works/pi/issues/9265))
- Fixed a nested call's error summary keeping the call's whole output alive through a slice of it ([#9265](https://github.com/earendil-works/pi/issues/9265))
- Fixed task span error messages and codemode call previews keeping the whole text they were cut from alive ([#9265](https://github.com/earendil-works/pi/issues/9265))
- Fixed partial commits of a streamed answer copying every growing text, thinking and tool-call argument string in full on each commit (about 20 s of CPU and a 100 MB heap peak for a 500 KB tool call): they now append new text with Chord's `appendText()`, and apply the argument changes of their own parse of the streamed argument text ([#9265](https://github.com/earendil-works/pi/issues/9265))
packages/durable/README.md
+35 -25
View File
@@ -324,7 +324,7 @@ const testsPass = defineTool({
- The caller gets the nested call's `taskId`, `structuredOutput`, `details`, `diagnostics`, `usage`, and `isError`, not the `output` meant for the model. A blocked, invalid, failed, or aborted nested call returns an `isError` result. `executeTool()` itself rejects once `execute()` has returned, when the caller is aborted, or when the Harness closes; cancelling its `context` stops only the wait.
- Hooks see a nested call's `call.parent`, so a guard can treat calls from scripts differently.
- While the caller runs, its nested calls show in `docs["pi.live"].nestedTools`, with the arguments they run with and a `summary` (error, duration, usage) once done, and as tool events with `parentToolCallId` and `parentTaskId`. `progress: false` keeps a nested call's running output out of `pi.live`. Before the caller settles, nested calls it left running are aborted, also those an earlier attempt made. The caller's result message does not list its nested calls; their task records keep them.
- While the caller runs, its nested calls show in `docs["pi.live"].nestedTools`, with their caller's `parentCallId` and `parentTaskId`, the arguments they run with, and a `summary` (error, duration, usage) once done; a nested call's full result is in its caller's `NestedCallDoc` while the caller runs. `progress: false` keeps a nested call's running output out of `pi.live`. Before the caller settles, nested calls it left running are aborted, also those an earlier attempt made. The caller's result message does not list its nested calls; their task records keep them.
- Each nested call has a key, its position among the caller's nested calls unless the caller passes `{ key }`. A replay-safe caller that reruns after a crash and calls in the same order, or passes the same keys, gets the nested calls it already made back, finished or still running, instead of starting them again. A nested call the crash interrupted before its result was committed follows its own `replay` policy, so a replay-safe tool with external effects still needs its own idempotency. A caller that is not replay-safe never reruns, so after a restart its unfinished nested calls are abandoned, with everything they own, before they run again.
- A nested call's `usage` is counted under its own tool; passing it on in the caller's result counts it twice.
- Nested results are not kept: they live in the caller's task documents (`NestedCallDoc`, one per nested call, keyed by its key, holding `{ taskId, result? }`) only until the caller settles. The nested call's task record keeps only a small receipt. A stored result is exactly what `executeTool()` returns, without the model's `output`, so an image or a `bash` tail is stored once.
@@ -479,8 +479,9 @@ Everything a UI needs is committed state. `watch()` returns a watch of the conve
const watch = await root.watch(context);
render(watch.value); // the state at attachment
watch.start(async ({ value, ops }) => {
// value.entries: the active transcript
// value.docs["pi.live"]: the running generation (streamed partial, retry, deferred) and tool calls (output, details)
// value.entries: a window over the transcript, newest last; value.before: the entry just before it
// value.docs["pi.live"]: the run, the running generation (streamed partial, retry, deferred), tool calls (output,
// details), compactions, and why the last run ended without an answer (runFailure)
// value.docs["pi.inbox"], value.docs["pi.usage"], value.docs["pi.agent"], value.docs["pi.provider"]
render(value);
await send(ops); // for example to a remote client that applies them
@@ -488,11 +489,33 @@ watch.start(async ({ value, ops }) => {
// later: await watch.stop();
```
The view's `entries` are a window, so neither the first value nor a reconnect costs the whole transcript. A watch starts with the newest 50 entries; new entries are appended, and once the window would hold more than 100, its oldest leave in one splice, leaving the newest 50. `watch(context, { min, max })` changes the bounds. `from` picks where the first value starts (as `transcript.resume` does after a drop); every entry committed after the watch attached is appended, so a `from` past the newest entry starts empty and receives what comes next. Compaction summaries and resets are entries like any other, in place; a summary's `head` says where the entries it summarizes end. `conversation.context()` gives the active transcript the model sees.
`ConversationTranscript` keeps what a client has seen beyond the window: call `merge()` with every value of the view, and render the `items` it returns, the client's whole transcript with a "load earlier" marker at the top and a "load more" gap wherever the client missed entries, after a stall or a reconnect:
```typescript
import { ConversationTranscript } from "@earendil-works/pi-durable";
const transcript = new ConversationTranscript((query, limit, ctx) => root.entries(query, limit, undefined, ctx));
const watch = await root.watch(context);
render(transcript.merge(watch.value));
watch.start(async ({ value }) => render(transcript.merge(value)));
// When a marker scrolls into view:
render(await transcript.loadEarlier(context)); // the page before the oldest entry
render(await transcript.fillGap(gap, context)); // the next page of a gap item
// Reconnect after a drop: only what was missed, when it fits the window, otherwise the newest entries and a gap.
const resumed = await root.watch(context, transcript.resume);
```
Entries never change once committed, so a merge only adds; one that adds nothing returns the same `items` array. See `test/examples/31-transcript.ts`.
Watches are Chord's `WatchHandle` (exported by `@earendil-works/chord`; Durable re-exports it). The listener gets one change per call, `{ value, ops, retired }`, and the newest commit's context. A watch ends once, and `closed` (or `stop()`) gives the reason: `stopped`, `cancelled` (its acquisition context was cancelled), `ended` (the Harness closed, or a task document's task ended), `listener_error`, or `failed` (the Harness failed). An ended watch keeps its last value and no longer references the Harness.
Adapters that must change in the same step as Durable's memory, such as a Chord source over a view, attach a synchronous observer instead of a watch: `harness.observeConversation(id, observer, context)`, `harness.observeTaskGraph(observer, context)`, and `harness.observeDoc(token, ...scope, observer, context)` return the current `value` and `detach()`, and call `observer.advance(change, context)` with every later change, unfolded, as memory adopts it: inside the storage's commit for a stored commit, at once for a deferred one (before it is stored). A document observer gets one call per change at its address, so a commit that retires and recreates the document gives two; the task graph also advances without a commit when a registry change alters a task's `blocked` reason. A change can arrive before the `observeXXX()` promise resolves; its ops apply from the observation's `value`. `observer.end(failure)` comes once, when the Harness closes or fails, or, for a task document, right after the retirement of the commit that ends its task. `advance` only records the change: no awaits, no Harness calls, no throws, and quick, since every commit waits for it. One that throws is reported, detached, and ended with the error. Watches are built on these observers.
Commits that land while a callback runs are delivered together in the next callback: the newest view and the operations of all of them, folded (appends to one string join, sets of one field keep the last). Past 10,000 pending operations the callback gets one root replacement `[["r", view]]` holding the whole newest view instead. A client that joins late or reconnects starts from the current view; nothing is replayed.
Commits that land while a callback runs are delivered together in the next callback: the newest view and the operations of all of them, folded (appends to one string join, sets of one field keep the last). Past 10,000 pending operations the callback gets one root replacement `[["r", view]]` holding the whole newest view instead, its window included. A client that joins late or reconnects starts from the current view; nothing is replayed.
Partial answers and tool output are committed at most every 100 ms by default, and watches see each commit at once. Storage gets them later and together, as deferred commits (see [Your Own State](#your-own-state)): with the next commit that reads `pi.live`, such as the end of the answer or of a tool, and at the latest after `settings.progress.persistWithinMs` (default 1,000 ms). A crash loses at most about that window of an answer or of tool output; a stream that crashes in its first second may leave no aborted entry, and that attempt's usage is not recorded. `settings.progress` changes the intervals; a host whose storage is remote can commit less often, for example `{ partialIntervalMs: 500, outputIntervalMs: 500 }`.
@@ -515,6 +538,8 @@ await root.submit({ type: "write", entry: { kind: "app.note", data: "user opened
If a run fails, queued items stay in the inbox until the next input or queued write places them, oldest first.
When a run ends without an answer, `docs["pi.live"].runFailure` says why until the next run starts: `model_error` and `no_model` (with the error text or the missing model in `detail`), `aborted`, `reset`, or `faulted` and `orphaned` (the run's task could not go on). The run's inputs are in it too, settled `unanswered`; the settlement's reason is the same, except that an orphaned run's inputs carry why it was orphaned (`missing_task`, `task_too_old`, `migration_failed`) and `no_model` has no `detail` there (spec §8.2).
An input is placed as a `pi.user` entry. To show it as something else, give it an `entry` with your kind and data; the model still sees `content` as the user message:
```typescript
@@ -582,23 +607,7 @@ When a provider rejects a request because the context is too long, generation co
When a compaction that a generation waits for at the blocking threshold fails, the generation fails too, without sending a request: its inputs settle `unanswered` with `model_error` and detail `Compaction failed: <error>`. One that finds nothing to compact or is declined lets the request go ahead.
Running compactions are listed in `docs["pi.live"].compactions` with their reason, attempt, and retry backoff. The newest failure stays in `docs["pi.live"].compactionFailure` (task, reason, error) until a compaction produces a summary. The agent events add `compaction_start` and `compaction_end` (with `error` when the compaction failed), and `compactions` and `compactionFailure` fields in the snapshot.
## Agent Events (Experimental)
For consumers that want coding-agent style events (`message_start`, `message_update`, `tool_execution_start`, ...) instead of structural state:
```typescript
import { watchEvents } from "@earendil-works/pi-durable";
const stream = await watchEvents(harness, root.id, context);
initialize(stream.snapshot); // entries, run, in-flight generation, tools, compactions, inbox, agent, usage
stream.start(async ({ value: events }) => {
for (const event of events) console.log(JSON.stringify(event));
});
```
Events are derived from commits, one batch per commit, and apply on top of the snapshot. Message and tool updates carry deltas: text and thinking appends, appended tool-call argument text, and output trims and appends. When a consumer falls more than 100 batches behind, it receives a fresh `snapshot` event instead. See `test/examples/18-json.ts` for the full stream of one run.
Running compactions are listed in `docs["pi.live"].compactions` with their reason, attempt, and retry backoff. The newest failure stays in `docs["pi.live"].compactionFailure` (task, reason, error) until a compaction produces a summary.
## Hooks
@@ -831,7 +840,7 @@ The package root loads TypeBox, because the tool task validates arguments with p
Three rules decide what an error does:
1. **A storage error ends the Harness.** The call that hit it rejects with the error. Everything after it, and every pending wait, rejects with `SessionFailed` (its `cause` is that error); watches and event streams end with `failed`; running tasks get their abort signal; `onReport` gets the error once. The Harness then closes itself. Nothing is retried; a custom storage retries its own transient failures. Reopen to recover: commits are atomic, so the store is consistent, and resubmitting with the same `requestId` finds a submission that was already admitted.
1. **A storage error ends the Harness.** The call that hit it rejects with the error. Everything after it, and every pending wait, rejects with `SessionFailed` (its `cause` is that error); watches end with `failed`; running tasks get their abort signal; `onReport` gets the error once. The Harness then closes itself. Nothing is retried; a custom storage retries its own transient failures. Reopen to recover: commits are atomic, so the store is consistent, and resubmitting with the same `requestId` finds a submission that was already admitted.
2. **An invalid read fails only that call:** a storage read throws `StorageRequestError` for a bad cursor, an unknown conversation, or history a document does not keep. A read you cancelled fails only itself too. The exemption covers reads only: any error from `commit()` or `mintId()`, `StorageRequestError` included, ends the Harness as in rule 1.
3. **Your code fails only its own unit:** a throwing task phase faults that task, a throwing tool gets an error result, a throwing hook is reported and skipped (`beforeTool` blocks the call), and a throwing commit callback rolls back that commit. Listeners that throw, or return a rejected promise, are reported and the others still run. The exception is a host `RegistryReader` whose `snapshot()` throws: the Harness cannot schedule without it, so it fails the Harness like a storage error.
@@ -859,10 +868,10 @@ node --conditions=source --experimental-strip-types test/examples/13-chat.ts
| [15-real-model](test/examples/15-real-model.ts) | Streaming an answer from OpenAI |
| [16-coding-tools](test/examples/16-coding-tools.ts) | A tool-using turn on JSONL storage |
| [17-print](test/examples/17-print.ts) | Print mode: submit a prompt, print the answer |
| [18-json](test/examples/18-json.ts) | JSON mode: agent events or raw view operations, on SQLite, JSONL, or memory |
| [18-json](test/examples/18-json.ts) | JSON mode: raw view operations, or appended entries and `pi.live` per change, on SQLite, JSONL, or memory |
| [19-inbox](test/examples/19-inbox.ts) | Steers, follow-ups, writes, and withdrawal while busy |
| [20-late-join](test/examples/20-late-join.ts) | Attaching a view and an event stream mid-run |
| [21-subagent-foreground](test/examples/21-subagent-foreground.ts) | A replay-safe subagent tool whose child the call owns, with the child's events under the call |
| [20-late-join](test/examples/20-late-join.ts) | Attaching a view mid-run: the running tool's output, then the streamed answer |
| [21-subagent-foreground](test/examples/21-subagent-foreground.ts) | A replay-safe subagent tool whose child the call owns, with the child's transcript under the call |
| [22-subagent-background](test/examples/22-subagent-background.ts) | Persistent subagents: spawn, steer, stop, list, answers reported back, restart-safe |
| [23-child-tasks](test/examples/23-child-tasks.ts) | A checkout that owns and waits for four payments: failFast, abort, restart |
| [24-compaction](test/examples/24-compaction.ts) | A long chat compacted in the background, manually, and after a context overflow |
@@ -872,6 +881,7 @@ node --conditions=source --experimental-strip-types test/examples/13-chat.ts
| [28-sandbox-per-conversation](test/examples/28-sandbox-per-conversation.ts) | An environment per conversation, looked up from an app document |
| [29-tool-override](test/examples/29-tool-override.ts) | A same-name bash for some conversations, and a wrapper that times whichever bash won |
| [30-reload-and-restart](test/examples/30-reload-and-restart.ts) | Reloading an extension mid-call, and stored choices surviving a restart |
| [31-transcript](test/examples/31-transcript.ts) | A client's transcript over the windowed view: live appends, a stall with a gap, loading it and earlier pages, and resuming with `from` |
| [00](test/examples/00-conversation.ts)–[12](test/examples/12-recovery.ts) | The layers underneath: sessions, documents, forks, watches, the Harness, agent configuration, reload, extension state, tasks, recovery |
Examples that call OpenAI need `OPENAI_API_KEY`; most use the faux provider otherwise.
packages/durable/docs/spec.md
+202 -201
View File
@@ -315,8 +315,10 @@ Context derivation:
kind ahead of the baseline therefore keeps the baseline from leading.
Stored entries and `contributions` keep the committed order.
Views carry raw active entries. UI reduction and model-context reduction are
separate consumers. Older stored history is available through the owning
The conversation view (section 9.3) carries a window over the visible
entries in ID order, head markers in place; `Conversation.context()` derives
the active transcript. UI reduction and model-context reduction are separate
consumers. Entries outside the window are available through the owning
`Conversation` object's cursor-based `entries()` scan.
### 2.2 Public Harness surface
@@ -685,8 +687,8 @@ interface Conversation {
/** `{ background: true }` also aborts background work under the conversation (section 5.4). */
abort(context: Context, options?: { readonly background?: boolean }): Promise<void>;
waitForIdle(context: Context): Promise<void>;
/** The structural view (section 9.3) as a watch. */
watch(context: Context): Promise<ConversationWatch>;
/** The structural view (section 9.3) as a watch; `window` bounds its entries. */
watch(context: Context, window?: ConversationWindow): Promise<ConversationWatch>;
}
interface Harness extends Session {
@@ -716,10 +718,10 @@ interface Harness extends Session {
waitForIdle(context: Context): Promise<void>;
/** Session total of every conversation's `pi.usage` (section 8.6). */
usage(context: Context): Promise<UsageState>;
/** Live tasks as a structural view (section 9.5), as a watch. */
/** Live tasks as a structural view (section 9.4), as a watch. */
watchTaskGraph(context: Context): Promise<TaskGraphWatch>;
/** Synchronous observers of a conversation view and of the task graph, for adapters (section 9.1). */
observeConversation(id: ConversationId, observer: Observer<ConversationView>, context: Context): Promise<Observation<ConversationView>>;
observeConversation(id: ConversationId, observer: Observer<ConversationView>, context: Context, window?: ConversationWindow): Promise<Observation<ConversationView>>;
observeTaskGraph(observer: Observer<TaskGraph>, context: Context): Promise<Observation<TaskGraph>>;
}
@@ -924,7 +926,7 @@ replace its model configuration without reopening the Harness. Settings are sync
modes are read on the Session line. `stream` is forwarded to generation
requests; `stream.maxRetries` are provider retries inside one request, while
`retry` governs durable generation attempts (section 8.3). Settings changes make
no commit and emit no event.
no commit and change no view.
```ts
// The user's settings, read live through getters.
@@ -1025,7 +1027,7 @@ wait aborts only that waiter.
Conversation handles are stateless; compare them by `id`. Hosts discover
conversations through lookups and scans. There is no creation listener; the task
graph view (section 9.5), `inspect()`, and the idle waits report live work.
graph view (section 9.4), `inspect()`, and the idle waits report live work.
`submit()` returns after durable admission, not settlement. An input submission
creates a user message with the Harness clock's timestamp at placement, which is
@@ -1867,7 +1869,7 @@ at its stored version. Anything else, such as an entry, a task state, a
submission, another document, a document's creation, migration, or retirement,
makes it immediate, a normal commit. A held commit is adopted in memory by
pointer swap, resolves, and is published to the Session's own observers: document
watches, the conversation view and agent events (section 9). Its stored
watches and the conversation view (section 9). Its stored
version and delta count stay; its operations wait, folded per incarnation
(Chord's `PendingOps`), and the incarnation is pinned in the cache, so idle
eviction skips it at any retention.
@@ -2404,7 +2406,7 @@ runtime state, not a persisted task status. A blocked task is not active in
`wake()` (section 5.6): only a registry change, or `retryTask()` for a parked
one, can let it run, and either makes `onWake` report it active, so a host need
not poll it. The task graph shows the reason and follows registry changes
(section 9.5). Document migration remains
(section 9.4). Document migration remains
access-driven.
Aborting a blocked task cannot run its abort handler, because that code is
@@ -2694,7 +2696,7 @@ An input is placed as one entry whose `model` is the user message of its
`content`: a `pi.user` entry, or, with `entry: { kind, data }`, an entry of that
kind carrying that data. Only `kind` and `data` are taken from `entry`; admission
rejects a kind that is not a non-empty string or starts with `pi.`. Nothing in
the Harness reads input entries by kind; context, compaction, events, and run
the Harness reads input entries by kind; context, compaction, and run
accounting go by `model` and the submission records.
A write of only `kind` and `data`, with no `model`, `head`, or `edits` (present
@@ -3076,8 +3078,8 @@ await conversation.configure({ extensions: { add: [Venv] } }, context);
| boundaries and idle admission, on the Session line | queue modes | per boundary (section 6) |
| tool task, `prepare`, `runtime.env()` | environment | at each call (section 2.2) |
Registry installs and settings changes make no commit, so they emit nothing
(section 9.4). A conversation on the default selection changes when the
Registry installs and settings changes make no commit, so no view changes
(section 9.3). A conversation on the default selection changes when the
settings or the installed extensions change; its next preparation appends the
system delta (section 7.4). A UI showing resolved tools or a resolved model
resolves again through `Conversation.agent()`.
@@ -3689,7 +3691,7 @@ that submission's result. Provider call IDs are not unique across a Session and
must not serve as request IDs. Aborting or abnormally
terminalizing the tool task cascades through the owned scope. The tool reports
the child in its running details, for example `api.details({ conversationId })`,
so a UI that sees the call can attach to the child's view or events.
so a UI that sees the call can attach to the child's view.
```ts
export const Subagent = defineExtension({
@@ -4206,7 +4208,7 @@ type LiveState = {
* IDs ascend.
*/
nestedTools?: {
/** `<parentCallId>/<key>`: the call's ID in tool events, unique because keys contain no `/`. */
/** `<parentCallId>/<key>`: the call's ID, unique because keys contain no `/`. */
callId: string;
parentCallId: string;
/** The calling tool task, model-issued or nested; the Harness relates slots by task ID. */
@@ -4252,6 +4254,25 @@ type LiveState = {
* after the task ended.
*/
compactionFailure?: CompactionFailure;
/**
* Why the newest run ended without an answer, so a client can show it after the run ended. Set in the commit that
* ends the run; removed when a later commit starts a run.
*/
runFailure?: RunFailure;
};
/** The newest run that ended without an answer. */
type RunFailure = {
/** The run's task at its end. */
taskId: TaskId;
/** The run's inputs, settled `unanswered`. */
inputs: SubmissionId[];
reason: "model_error" | "no_model" | "aborted" | "reset" | "faulted" | "orphaned";
/**
* `model_error`: the error text; `no_model`: which model is missing; `faulted`: the error message; `orphaned`: why
* the task could not run (`missing_task`, `task_too_old`, `migration_failed`).
*/
detail?: string;
};
type CompactionFailure = {
@@ -4321,7 +4342,26 @@ that outcome holds as `completing` (section 5.5), or the scheduler's cleanup
It stays small: the summary text lives in the task's placement, never here. Partials are
normalized to strict JSON before assignment. Every terminal path of a run task
removes `run`, `generation`, and `tools` in the commit that settles the run's
inputs. `tx.settleSubmission()` stages each input's new status
inputs.
A run that ends without an answer records why in `runFailure`, in the commit
that settles its inputs `unanswered`:
| reason | the run ended because | input settlement |
|---|---|---|
| `model_error` | the response failed after any retries, or a blocking or overflow compaction it needed failed; `detail` is the error text | `model_error` with the text |
| `no_model` | no model is configured, or the configured one is not available; `detail` names it | `no_model` |
| `aborted` | its task was aborted (`Conversation.abort()`, `abortTask()`) | `aborted` |
| `reset` | a queued reset started a new context at the post-tools boundary (a tool's handoff continues the run) | `reset` |
| `faulted` | the scheduler faulted its task (section 5.4); `detail` is the error message | `faulted` with the message |
| `orphaned` | its task was aborted while no definition could take it; `detail` is the reason | the reason |
The next commit that starts a run removes it. A run started in the same commit,
the queue's next run under `afterFailedRun: "continue"` (section 6) or after a
reset, keeps it, so the failure is shown until a later run starts. A run that
ends with an answer, including a round whose tools all ask to terminate, leaves
it alone. Inputs settled without a run (withdrawn, stale, aborted while
queued, `run_failed`) are not run failures; their submission records say why. `tx.settleSubmission()` stages each input's new status
and resolves the transaction's latest candidate submission record, falling back
to committed state, during assembly, like task-document validation (section 3.3),
so it is not a caller table read and works after the commit's first table write.
@@ -4922,7 +4962,7 @@ interface Session {
// ... and the other five scopes (section 3.3), the observer before the context
}
interface Harness {
observeConversation(id: ConversationId, observer: Observer<ConversationView>, context: Context): Promise<Observation<ConversationView>>;
observeConversation(id: ConversationId, observer: Observer<ConversationView>, context: Context, window?: ConversationWindow): Promise<Observation<ConversationView>>;
observeTaskGraph(observer: Observer<TaskGraph>, context: Context): Promise<Observation<TaskGraph>>;
}
```
@@ -4936,7 +4976,7 @@ the graph at most one call, and a document observer one call per change at its
address: a commit that retires the document and creates it again gives two, the
retirement and then the new incarnation's `[["r", value]]`. The task graph also
advances without a commit, when a registry change or a failed migration changes
a task's `blocked` reason (section 9.5): that call runs on the Session line,
a task's `blocked` reason (section 9.4): that call runs on the Session line,
outside any `adopt`, with a context of its own. A document observer follows its
address as a document watch does, and `observeDoc()` gives `undefined` for an
absent one. `end` comes once: when the Session closes (`undefined`) or fails
@@ -4960,8 +5000,9 @@ and returns: no awaits, no Session calls, no throws, and quick, as its time is
part of every commit; so does `end` when a task ends. One that throws anyway is
reported as `listener_error`, detached, and ended with the error; the commit and
the other observers are not affected. The change object is plain data, shared by
every observer of a view and by the watches built on it, and must not be
mutated.
every observer of a document or the task graph and by the watches built on it,
and must not be mutated; each observer of a conversation view gets its own, for
its own window (section 9.3), sharing the document values.
### 9.2 `watchDoc`
@@ -5033,227 +5074,188 @@ rejection is observed.
### 9.3 Conversation view
The public view is a fixed structural mount of the conversation's active
The public view is a fixed structural mount of a window over the conversation's
transcript and its built-in documents:
```ts
type ConversationView = {
readonly conversation: ConversationRecord;
/** Raw active entries, as `ContextView.entries` (section 2.1): the head marker, then the non-head entries from its head. */
/**
* A window over the transcript, its fork-aware history (`Conversation.entries()`), in entry ID order. Head markers
* such as compaction summaries sit where they were written; their `head` tells where the entries they summarize end.
*/
readonly entries: readonly EntryRecord[];
/** ID of the entry just before the window; absent when the window starts the transcript. */
readonly before?: EntryId;
/** `pi.agent`, `pi.live`, `pi.inbox`, `pi.provider`, and `pi.usage`, keyed by kind; absent documents are absent. */
readonly docs: Readonly<Record<string, JsonObject>>;
};
type ConversationWindow = {
/**
* Where the first value's window starts, inclusive, such as right after the newest entry a client has
* (`ConversationTranscript.resume`). It selects only that first value.
*/
readonly from?: EntryId;
/** Default 50; an integer, at least 1. */
readonly min?: number;
/** Default `2 * min`; an integer, at least `min`. */
readonly max?: number;
};
interface Conversation {
watch(context: Context, window?: ConversationWindow): Promise<ConversationWatch>;
}
interface Harness {
observeConversation(id: ConversationId, observer: Observer<ConversationView>, context: Context, window?: ConversationWindow): Promise<Observation<ConversationView>>;
}
```
These kinds and their fields are public protocol. Third-party documents are
exposed through their own Chord objects, not mounted.
exposed through their own Chord objects, not mounted. Everything a renderer
draws is state in this value: the transcript in `entries`, the run and its
presentation in `pi.live` (`run`, `generation`, `tools`, `nestedTools`,
`compactions`, `compactionFailure`, `runFailure`, section 8.2), the queue in
`pi.inbox`, the agent in `pi.agent`, spend in `pi.usage`. A renderer renders
the newest value; it never assembles state from a stream of events.
**The window.** Each watch and observer has its own window. It starts with the
newest `min` entries and the ID of the entry before them in `before`. With
`from`, it starts at `from` instead, when at most `max` entries are at or after
it; then `before` is the newest entry before `from`, and a window that resumes
right after the newest entry starts empty. Otherwise it starts at the newest
`min`, and `before` shows the client the entries it missed. `from` only picks
that first value: every entry committed after the attachment is appended,
whatever its ID, so a `from` past the newest entry, such as one a client
computed against another replica, gives an empty window whose `before` is the
newest entry, and then every entry that follows. The window has a stable start,
so new entries are appends. When it would hold more than `max`
entries, its oldest leave in one splice down to `min`, and `before` moves to
the last one that left:
```text
min 3, max 5; the window holds 4..7, before 3
commit appends 8: ["p", ["entries"], 4, 0, [8]] window 4..8
commit appends 9: ["p", ["entries"], 5, 0, [9]], ["p", ["entries"], 0, 3, []], ["s", ["before"], 6]
window 7..9, before 6
```
A commit that appends more than `max` entries at once keeps its newest `min`.
A first value, and an overflow replacement (section 9.2), therefore hold at
most `max` entries, never the whole transcript. Entry IDs are ordered but not
contiguous: a client never counts entries or infers ranges from IDs. `before`
tells whether a window continues what the client has, and
`Conversation.entries()` with `minEntryId`/`maxEntryId` and its cursor fetches
what lies in between; resuming from right after the newest entry and
`maxEntryId: oldest - 1` are exclusive bounds, the only arithmetic on IDs a
client does, and `ConversationTranscript` does both. A window with
bounds that are not integers, or `max` below `min`, rejects with a
`RangeError`.
Head markers stay in place: a compaction summary, a reset, or a handoff is an
appended entry like any other. The view does not derive the active
transcript; `Conversation.context()` does (section 2.1).
The Harness keeps at most one mount per conversation. The first `watch()` or
event attachment builds its revision on the Session line from the
committed active transcript and documents; the mount is dropped when its last
observer builds its documents on the Session line, and each attachment reads
its window there with bounded scans; the mount is dropped when its last
observer detaches. A mount derives each next revision from the Session's
commits as memory adopts them, for a stored commit inside Storage's `adopt`
(section 4) and for a held one when memory adopts it (section 4.1), like
`watchDoc()`: it shows committed state,
including deferred commits before they are stored (section 4.1). One publication that
touches the conversation yields one Chord batch:
`watchDoc()`: it shows committed state, including deferred commits before they
are stored (section 4.1). One publication that touches the conversation yields
one Chord batch per observer: the document operations, applied once and
shared, then that observer's entry splices:
```text
document op ["s", ["generation", "message"], value]
-> view op ["s", ["docs", "pi.live", "generation", "message"], value]
```
An appended entry without a head is a splice at the end of `entries`. An
appended head marker `H` makes the entries `H` followed by the current non-head
entries at or after `H.head`: the kept entries stay, and splices remove the
others and insert `H` at the front. Every kept entry is already mounted because
Harness writers never target a head before the current range: resets and
handoffs head themselves, compaction summaries target a kept entry of the
current range, and head writes that reach further back are stale (section 6). A raw head write that targets further back shows only the
mounted entries (section 12).
Document creation and retirement set and delete the `docs` key. Parent entries
through `parent.at` are immutable, so a child's mount follows only its own
conversation. A publication that touches nothing mounted creates no revision; a
redundant nonempty batch remains a real publication. Mounted document revisions
are structurally shared under the trusted immutability contract. The mount
performs no semantic projection and owns no persistence.
are structurally shared under the trusted immutability contract, and an entry
that stays in a window keeps its identity across revisions. The cost of a
commit is its operations plus, per observer that gets entries, one copy of its
window, at most `max` entries. The mount performs no semantic projection and
owns no persistence.
`Conversation.watch()` exposes the mount through the serialized watch of
section 9.2, with folded pending commits and a full-value overflow replacement;
`Harness.observeConversation()` attaches a synchronous observer of the same mount
(section 9.1). A view never retires, so its changes never set `retired`.
### 9.4 Agent events (experimental)
`watchEvents(harness, conversationId, context): Promise<AgentEventStream>`,
exported from the package root, is an experimental adapter that
translates one conversation's committed publications into agent events shaped
like the coding agent's `AgentSessionEvent`s. It owns no tracker or persistence,
covers exactly one conversation (not its owned subtree), and emits an event only
after the commit that makes it true. Its protocol may change without notice.
**A client's transcript.** A window only ever adds to what a client has seen:
entries never change once committed, and those that leave the window stay with
the client. `ConversationTranscript` keeps them, as a layer over any view,
wherever it comes from (a watch, or a replica of it on another host):
```ts
/**
* Which call a tool event is about. `toolCallId` is the provider's ID for a model-issued call and
* `<parent call ID>/<key>` for a nested one; consumers match it and never parse it. `taskId` is the call's tool task,
* absent for a call that never got one. A nested call also names its caller by call ID and by tool task.
*/
type ToolEventCall = {
toolCallId: string;
toolName: string;
taskId?: TaskId;
parentToolCallId?: string;
parentTaskId?: TaskId;
};
type TranscriptItem =
| { readonly kind: "entry"; readonly entry: EntryRecord }
/** Entries strictly between `after` and `upTo` are missing: render "load more". */
| { readonly kind: "gap"; readonly after: EntryId; readonly upTo: EntryId }
/** The transcript continues above: render "load earlier". */
| { readonly kind: "earlier" };
type AgentEvent =
| {
type: "snapshot";
entries: readonly EntryRecord[];
run?: { inputs: readonly SubmissionId[] };
/** Current generation attempt: its in-flight partial, retry backoff, or deferred poll. */
generation?: { attempt: number; message?: AssistantMessage; retry?: { at: number; error: string }; deferred?: { pollAt: number } };
tools: readonly ToolSlot[];
/** `pi.live.nestedTools` (section 8.2). */
nestedTools: readonly NestedToolSlot[];
/** `pi.live.compactions` (section 8.2). */
compactions: readonly CompactionStatus[];
/** `pi.live.compactionFailure` (section 8.2). */
compactionFailure?: CompactionFailure;
inbox: readonly { id: SubmissionId; mode: InboxItem["mode"] }[];
/** `pi.agent`; `{}` when absent. */
agent: AgentState;
usage: UsageState;
}
| { type: "run_start"; inputs: readonly SubmissionId[] }
| { type: "run_end"; inputs: readonly SubmissionId[] }
| { type: "turn_start" }
| { type: "turn_end" }
| { type: "message_start"; message: Message }
| { type: "message_update"; usage: Usage; changes: readonly MessageChange[] }
| { type: "message_end"; entry: EntryRecord }
| ({ type: "tool_execution_start"; args: JsonObject } & ToolEventCall)
| ({
type: "tool_execution_update";
/** A front trim and then an append of the retained window, or its replacement. */
output?: { trimStart?: number; append?: string } | { set: string };
details?: JsonValue;
diagnostics?: readonly ToolDiagnostic[];
} & ToolEventCall)
/** A model-issued call ends with its `entry`, a nested call with its `result`; see below for when neither. */
| ({ type: "tool_execution_end"; entry?: EntryRecord; result?: NestedToolExecutionResult } & ToolEventCall)
| { type: "inbox_update"; items: readonly { id: SubmissionId; mode: InboxItem["mode"] }[] }
| { type: "submission"; record: SubmissionRecord }
| { type: "auto_retry_start"; attempt: number; at: number; errorMessage: string }
| { type: "auto_retry_end"; attempt: number }
| { type: "deferred_poll"; pollAt: number }
| { type: "entry_appended"; entry: EntryRecord }
| { type: "agent_changed"; agent: AgentState }
| { type: "usage_changed"; usage: UsageState }
| { type: "task_failed"; taskId: TaskId; kind: string; message: string }
| { type: "compaction_start"; taskId: TaskId; reason: CompactionReason; blocking: boolean }
/** The task's receipt tells whether it produced a summary; the summary entry has its own events. */
| { type: "compaction_end"; taskId: TaskId; reason: CompactionReason; error?: string };
/** The conversation view as a renderer sees it: `items` instead of the window. */
type TranscriptView = Omit<ConversationView, "entries" | "before"> & { readonly items: readonly TranscriptItem[] };
/** One change to the in-flight assistant message, relative to that message. */
type MessageChange =
| { type: "text_start" | "thinking_start" | "toolcall_start"; contentIndex: number; block: AssistantMessage["content"][number] }
| { type: "text_delta" | "thinking_delta"; contentIndex: number; delta: string }
| { type: "toolcall_delta"; contentIndex: number; path: readonly (string | number)[]; delta: string }
| { type: "block"; contentIndex: number; block: AssistantMessage["content"][number] }
| { type: "message"; message: AssistantMessage };
type EntryLoader = (query: Omit<EntryQuery, "conversationId">, limit: number, context: Context) => Promise<Page<EntryRecord, Cursor>>;
interface AgentEventStream {
/** The `snapshot` event at attachment. */
readonly snapshot: Extract<AgentEvent, { type: "snapshot" }>;
/** Each call's `value` is one commit's batch, or a `snapshot` after an overflow (Chord's `NativeWatchChange`). */
start(listener: (change: { readonly value: readonly AgentEvent[] }, context: Context) => Promise<void>): void;
stop(): Promise<WatchEnd>;
readonly closed: Promise<WatchEnd>;
class ConversationTranscript {
constructor(load: EntryLoader, options?: { readonly pageSize?: number }); // default 50
/** Call with every value of the view. */
merge(view: ConversationView): TranscriptView;
/** The page before the oldest entry held. */
loadEarlier(context: Context): Promise<TranscriptView>;
/** The next page of a gap, from its start. */
fillGap(gap: Extract<TranscriptItem, { kind: "gap" }>, context: Context): Promise<TranscriptView>;
/** The ID the transcript reaches: its newest entry, or a later empty window's `before`. */
readonly newest: EntryId | undefined;
/** `{ from }` right after `newest`, `{}` for an empty transcript, undefined before the first merge. */
readonly resume: ConversationWindow | undefined;
}
```
Every event is relative to the `snapshot` and the events before it, never to
Durable's document layout. `snapshot` replaces everything a consumer holds; every
other event applies on top of it. Attachment captures the snapshot and
registers for later publications atomically on the Session line.
```ts
const transcript = new ConversationTranscript((query, limit, ctx) => conversation.entries(query, limit, undefined, ctx));
const watch = await conversation.watch(context);
render(transcript.merge(watch.value));
watch.start(async ({ value }) => render(transcript.merge(value)));
// A marker scrolled into view:
render(await transcript.loadEarlier(context));
// Reconnect after a drop:
const resumed = await conversation.watch(context, transcript.resume);
```
Events derive from committed changes:
A window whose `before` lies past the newest entry the client has does not
continue it: after a stall, an overflow, or a reconnect, the entries in between
become a gap. While the client keeps up there is never one: the window only
appends, and its splice removes entries the client already holds.
`fillGap()` loads `pageSize` entries from the gap's start and shrinks it, or
removes it once nothing is missing; `loadEarlier()` loads the page before the
oldest entry held, and the `earlier` item disappears once a page reaches the
transcript's start. A window, or a page, that covers part of a gap fills that
part. Items are in ID order: `earlier` first, then entries, each gap before
the entry after it.
- `run_start`/`run_end`: `pi.live.run` appears, is removed, or is replaced by a
successor run whose first input differs. Steers joining the current run and
handovers between its tasks are not run events. The inputs' outcomes are
`submission` events.
- `turn_start`: a `pi.generation` task is created. `turn_end`: a generation's
outcome is committed, when it holds `completing` or becomes terminal,
whichever comes first, so a successor created at hold starts after it.
- `message_start`: the first committed partial of an attempt, or, for a message
entry without a committed partial, the entry itself; `message_end` follows for
every entry with `model` messages. Generation commits a partial only once it has
content, and every built-in path that clears one appends its entry, so each
started message ends with its entry. Entries without messages are
`entry_appended`.
- `message_update`: the Chord operations on the committed partial, translated:
an insertion into `content` starts blocks, an append to a block's `text` or
`thinking` is a delta, an append to a string inside a tool call's `arguments`
is a `toolcall_delta` with the path relative to `arguments`, any other change
inside a block sends that `block`, which already holds the batch's later
changes to it, and any other change to `content` sends the whole `message`.
Each update carries the partial's current `usage`, as the coding agent's JSON
mode does; a change of only `usage` sends no changes. Only deltas travel, so throttled
partials cause no write amplification on the wire.
- `tool_execution_start`: a slot becomes `running`; `args` come from the tool
task's intent checkpoint in the same commit. A slot that becomes `done`
without running (a call not offered, invalid arguments, a block, a fault
before intent) gets only `tool_execution_end`. `tool_execution_update`: output
appends, front trims, and replacements of the slot's retained window, and the
slot's current `details` and `diagnostics` when they change; removed ones, as
when a safe replay restarts the tool, send `null` and `[]`.
`tool_execution_end`: the slot becomes `done`, with its `pi.tool-result`
entry, which carries the diagnostics, or without one after a fault or
orphan. An unfinished slot that disappears because its run ended ends with the
result entry appended in the same commit, as for the unstarted calls of an
aborted round, directly before its `message_start`, or without an entry.
Nested slots produce the same events with `parentToolCallId`; a nested call
ends with its `result`, taken from the `NestedCallDoc` value its terminal
commit published, or without one after a fault or orphan or when its slot
disappears unfinished. Snapshots show done nested slots with their summary,
not their result; a late subscriber reads a live caller's results from its
`NestedCallDoc`, keyed by call key (the slot's `callId` after `<parentCallId>/`).
- `inbox_update`, `agent_changed`, `usage_changed`: the document changed; a
retired one reads as its initial value, as in a snapshot. Registry installs
and settings changes make no commit and emit nothing; a UI showing resolved
tools or a resolved model resolves again through `Conversation.agent()`.
- `auto_retry_start`/`auto_retry_end`, `deferred_poll`: `pi.live.generation`
gains or drops `retry`, or gains `deferred` or moves its `pollAt`.
- `task_failed`: a task of the conversation settles `faulted` or `orphaned`.
- `compaction_start`/`compaction_end`: a status appears in or disappears from
`pi.live.compactions`. `compaction_end` carries `error` when the same commit
recorded the task in `pi.live.compactionFailure`. A retry backoff shows only
in the snapshot's `compactions`. A queued summary is placed later with its own
`message_start`/`message_end` and `submission` events.
`items` is a plain array. A merge costs O(items), never re-sorts or
re-fetches, and one that adds nothing returns the previous `items` array; with
the same `docs` and `conversation` it returns the previous view itself.
Concurrent `loadEarlier()` calls, and concurrent `fillGap()` calls for one gap,
share one load, also when one passes a gap descriptor that has shrunk since; a
load that rejects, or whose loader throws, is not kept, so the next call loads
again. A page without a next cursor, or without items, ends the range: the gap
closes, or the transcript's start is reached. An empty window adds nothing.
`loadEarlier()` before the first merge rejects; after the start is reached, and
`fillGap()` for a gap that is gone, return the current view.
Keeping the scroll position when items are inserted above is the renderer's
job.
One commit produces one batch, in this order: `tool_execution_start`,
`message_start` of a first partial, `message_update`, `tool_execution_update`,
and retry/deferred events; then entries in append order with their
`message_start`/`message_end` or `entry_appended`, where a tool result's
`tool_execution_end` directly precedes its `message_start`, as in the coding
agent, and the `tool_execution_end` of nested calls, children first, precede
the entries; then the `tool_execution_end` of tools ending without an entry;
then `compaction_end`, `task_failed`, `turn_end`, `run_end`; then `submission`
events in ID order; then `inbox_update`, `agent_changed`, and `usage_changed`;
and last `compaction_start`, `run_start`, and `turn_start`. A stream buffers at most 100 undelivered
batches; adding another replaces every undelivered batch with one `snapshot` of
the newest committed state. The stream therefore converges but does not promise
every transition; `Submission.wait()` reports exact outcomes. A reconnecting
consumer attaches again and starts from its `snapshot`; nothing is replayed.
Transport backpressure and disconnect policy belong to the consumer.
Print mode awaits its own input `Submission` and prints its answer. A TUI
renders `ConversationView`; events may drive transient animation.
### 9.5 Task graph view
### 9.4 Task graph view
The task graph view shows every live task of the Session as one structural
value, for UIs and debugging: what runs, what waits for what, and which
@@ -5574,8 +5576,7 @@ may not have committed, so a host that resubmits after reopening reuses its
and every wait that starts later rejects with `SessionFailed`, whose `cause` is
the first error; so does every Storage call still underway when it returns,
whatever it returned.
- Watches and event streams end with `{ reason: "failed", error }` and keep
their last value.
- Watches end with `{ reason: "failed", error }` and keep their last value.
- Running task invocations get their abort signal. `HarnessOptions.onReport`
gets the error once, after the Session is sealed, so a report handler that
calls back finds it failed.
@@ -5785,8 +5786,8 @@ These are contracts, not invitations to add defensive machinery:
messages, can misjudge; the Harness estimate uses entry order (section 8.3).
- **Raw head writes into the past:** a head written directly with `tx.appendEntry()`
that targets an entry before the conversation's active range changes model
context, but a mounted view keeps only the entries it already holds until the
mount is rebuilt. Use a write submission, whose stale check rejects it.
context in ways a queued write would not. Use a write submission, whose stale
check rejects it.
- **Double-counted subagent spend:** a tool that runs an owned conversation must
not report that conversation's usage in its result; the child's `pi.usage`
already counts it, and subtree sums would count it twice.
packages/durable/package.json
+1
View File
@@ -131,6 +131,7 @@
"bench:document-cache": "node --conditions=source --experimental-strip-types test/session-document-cache-bench.ts",
"bench:document-append": "node --conditions=source --experimental-strip-types test/document-append-bench.ts",
"bench:codemode": "node --conditions=source --experimental-strip-types test/codemode-bench.ts",
"bench:conversation-view": "node --conditions=source --experimental-strip-types --expose-gc test/conversation-view-bench.ts",
"prepublishOnly": "npm run clean && npm run build"
},
"keywords": [
packages/durable/src/harness/context.ts
-11
View File
@@ -253,17 +253,6 @@ function settle(
return { settled: [...settled, ...orderToolResults(open.slice(0, last))], open: open.slice(last) };
}
/** The raw active entries within captured bounds, without deriving model context. */
export async function activeEntries(
storage: Storage,
conversationId: ConversationId,
bounds: ContextBounds | undefined,
context: Context,
): Promise<readonly EntryRecord[]> {
if (bounds === undefined) return [];
return selectActive(bounds.head, await scanRange(storage, conversationId, rangeQuery(bounds), context));
}
/** The visible range of `bounds`: from the head marker's head, or transcript start, through the tail. */
function rangeQuery(bounds: ContextBounds): Omit<EntryQuery, "conversationId"> {
const head = bounds.head;
packages/durable/src/harness/events.ts
-512
View File
@@ -1,512 +0,0 @@
import {
type Context,
type JsonValue,
NativeWatch,
type NativeWatchChange,
type WatchEnd,
} from "@earendil-works/chord";
import type { Op, Path } from "@earendil-works/chord/delta";
import type { AssistantMessage, Message, Usage } from "@earendil-works/pi-ai";
import type { ObservedCommit } from "../session/session.ts";
import type {
CommitChange,
ConversationId,
EntryRecord,
JsonObject,
SubmissionId,
SubmissionRecord,
TaskId,
} from "../types.ts";
import { AgentDoc } from "./agent.ts";
import type { InboxItem, InboxState } from "./inbox.ts";
import type { CompactionFailure, CompactionStatus, LiveState, NestedToolSlot, ToolSlot } from "./live.ts";
import { NestedCallDoc, type NestedCallState } from "./tool.ts";
import type { AgentState, CompactionReason, Harness, NestedToolExecutionResult, ToolDiagnostic } from "./types.ts";
import { UsageDoc, type UsageState } from "./usage.ts";
import { scanAll } from "./util.ts";
import { type ConversationView, conversationViews } from "./view.ts";
type Block = AssistantMessage["content"][number];
type QueuedItem = { id: SubmissionId; mode: InboxItem["mode"] };
/** One change to the in-flight assistant message, relative to that message. */
export type MessageChange =
| { type: "text_start" | "thinking_start" | "toolcall_start"; contentIndex: number; block: Block }
| { type: "text_delta" | "thinking_delta"; contentIndex: number; delta: string }
| { type: "toolcall_delta"; contentIndex: number; path: readonly (string | number)[]; delta: string }
| { type: "block"; contentIndex: number; block: Block }
| { type: "message"; message: AssistantMessage };
export type SnapshotEvent = {
type: "snapshot";
entries: readonly EntryRecord[];
run?: { inputs: readonly SubmissionId[] };
/** Current generation attempt: its in-flight partial, retry backoff, or deferred poll. */
generation?: {
attempt: number;
message?: AssistantMessage;
retry?: { at: number; error: string };
deferred?: { pollAt: number };
};
tools: readonly ToolSlot[];
/** `pi.live.nestedTools`: nested calls of running tool calls. */
nestedTools: readonly NestedToolSlot[];
/** `pi.live.compactions`: live compactions with their attempt and retry backoff. */
compactions: readonly CompactionStatus[];
/** `pi.live.compactionFailure`: the newest compaction failure since a compaction last produced a summary. */
compactionFailure?: CompactionFailure;
inbox: readonly QueuedItem[];
/** `pi.agent`; `{}` when absent. */
agent: AgentState;
usage: UsageState;
};
/** Experimental agent event, shaped like the coding agent's session events (spec §9.4). */
export type AgentEvent =
| SnapshotEvent
| { type: "run_start"; inputs: readonly SubmissionId[] }
| { type: "run_end"; inputs: readonly SubmissionId[] }
| { type: "turn_start" }
| { type: "turn_end" }
| { type: "message_start"; message: Message }
/** `usage` is the partial's current usage, as in the coding agent's JSON mode. */
| { type: "message_update"; usage: Usage; changes: readonly MessageChange[] }
| { type: "message_end"; entry: EntryRecord }
| ({ type: "tool_execution_start"; args: JsonObject } & ToolEventCall)
| ({
type: "tool_execution_update";
/** A front trim and then an append of the retained window, or its replacement. */
output?: { trimStart?: number; append?: string } | { set: string };
details?: JsonValue;
diagnostics?: readonly ToolDiagnostic[];
} & ToolEventCall)
/**
* A model-issued call ends with its result `entry`, a nested call with its `result`. Both are absent when the tool
* task faulted or was orphaned, or when the call's slot left `pi.live` unfinished: its run ended, or its parent
* settled first.
*/
| ({ type: "tool_execution_end"; entry?: EntryRecord; result?: NestedToolExecutionResult } & ToolEventCall)
| { type: "inbox_update"; items: readonly QueuedItem[] }
| { type: "submission"; record: SubmissionRecord }
| { type: "auto_retry_start"; attempt: number; at: number; errorMessage: string }
| { type: "auto_retry_end"; attempt: number }
| { type: "deferred_poll"; pollAt: number }
| { type: "entry_appended"; entry: EntryRecord }
| { type: "agent_changed"; agent: AgentState }
| { type: "usage_changed"; usage: UsageState }
| { type: "task_failed"; taskId: TaskId; kind: string; message: string }
| { type: "compaction_start"; taskId: TaskId; reason: CompactionReason; blocking: boolean }
/**
* The task's receipt tells whether it produced a summary; the summary entry has its own events. `error` is set when
* the compaction failed, faulted, or was orphaned, as in `pi.live.compactionFailure`.
*/
| { type: "compaction_end"; taskId: TaskId; reason: CompactionReason; error?: string };
/**
* Which call a tool event is about. `toolCallId` is the call's ID in the transcript and in tool events: the provider's
* ID for a model-issued call, `<parent call ID>/<key>` for a nested one. Match events by it; do not parse it. `taskId`
* is the call's tool task, absent for a call that never got one (not offered, or not started in a sequential round).
* A nested call also names the call that made it, by call ID and by tool task.
*/
export type ToolEventCall = {
toolCallId: string;
toolName: string;
taskId?: TaskId;
parentToolCallId?: string;
parentTaskId?: TaskId;
};
/** Serialized stream of one conversation's event batches, one per commit. */
export interface AgentEventStream {
/** The `snapshot` event at attachment. */
readonly snapshot: SnapshotEvent;
/** Install the sole listener; each call's `value` is one commit's batch, or a `snapshot` after an overflow. */
start(listener: (change: NativeWatchChange<readonly AgentEvent[]>, context: Context) => Promise<void>): void;
stop(): Promise<WatchEnd>;
readonly closed: Promise<WatchEnd>;
}
/** The typed parts of a view the events read. */
type Parts = {
live: LiveState;
inbox: InboxState | undefined;
agent: AgentState | undefined;
usage: UsageState | undefined;
};
function parts(view: ConversationView): Parts {
return {
live: (view.docs["pi.live"] ?? {}) as LiveState,
inbox: view.docs["pi.inbox"] as InboxState | undefined,
agent: view.docs["pi.agent"] as AgentState | undefined,
usage: view.docs["pi.usage"] as UsageState | undefined,
};
}
function snapshotOf(view: ConversationView): SnapshotEvent {
const { live, inbox, agent, usage } = parts(view);
return {
type: "snapshot",
entries: view.entries,
...(live.run === undefined ? {} : { run: { inputs: live.run.inputs } }),
...(live.generation === undefined ? {} : { generation: live.generation as SnapshotEvent["generation"] }),
tools: live.tools ?? [],
nestedTools: live.nestedTools ?? [],
compactions: live.compactions ?? [],
...(live.compactionFailure === undefined ? {} : { compactionFailure: live.compactionFailure }),
inbox: queued(inbox),
agent: agent ?? AgentDoc.definition.initial(),
usage: usage ?? UsageDoc.definition.initial(),
};
}
function queued(inbox: InboxState | undefined): QueuedItem[] {
return (inbox?.items ?? []).map((item) => ({ id: item.id, mode: item.mode }));
}
/**
* Experimental: attach to one conversation's agent events (spec §9.4). The snapshot and the registration for later
* commits are captured atomically on the Session line; overflow replaces undelivered batches with one snapshot.
*/
export async function watchEvents(
harness: Harness,
conversationId: ConversationId,
context: Context,
): Promise<AgentEventStream> {
const views = conversationViews(harness);
let watch!: NativeWatch<readonly AgentEvent[]>;
let snapshot!: SnapshotEvent;
await views.attach(
conversationId,
async (initial, release, storage) => {
// Generations whose held outcome already ended their turn, read on the line with the snapshot.
const query = { conversationId, kind: "pi.generation", status: "completing" } as const;
const completing = await scanAll((cursor) => storage.scanTasks(query, 100, cursor, context));
const held = new Set<TaskId>(completing.map((record) => record.id));
let current = initial;
snapshot = snapshotOf(initial);
// Batches are the watch's values, each delivered (folding would drop events); an overflow delivers a snapshot of
// the newest view instead.
watch = new NativeWatch<readonly AgentEvent[]>(
[],
release,
(error) => views.report(error),
() => [snapshotOf(current)],
);
return {
publication: (before, after, ops, publication, commitContext) => {
current = after;
const events = translate(conversationId, before, after, ops, publication, held);
if (events.length > 0) watch.advance(events, commitContext);
},
end: (failure) => watch.end(failure),
};
},
context,
);
// Like a watch, the acquisition context governs the stream's lifetime.
const signal = context.abortSignal;
if (signal?.aborted) {
watch.cancel();
throw signal.reason;
}
if (signal !== undefined) watch.observeCancellation(signal);
return {
snapshot,
start: (listener) => watch.start(listener),
stop: () => watch.stop(),
closed: watch.closed,
};
}
type TaskChange = Extract<CommitChange, { type: "task" }>;
/** The tool result for `callId` among `entries`. */
function resultOf(entries: readonly EntryRecord[], callId: string): EntryRecord | undefined {
return entries.find((entry) => {
const message = entry.model?.[0];
return message?.role === "toolResult" && message.toolCallId === callId;
});
}
/** Every event one publication causes, in the order of spec §9.4. */
function translate(
conversationId: ConversationId,
before: ConversationView,
after: ConversationView,
viewOps: readonly Op[],
publication: ObservedCommit,
held: Set<TaskId>,
): AgentEvent[] {
const entries: EntryRecord[] = [];
const tasks = new Map<TaskId, TaskChange["value"]>();
const submissions: SubmissionRecord[] = [];
// Nested results this commit stored, by nested task ID; read now, since the caller's documents may retire next.
const nestedResults = new Map<TaskId, NestedToolExecutionResult>();
for (const change of publication.changes) {
if (
change.type === "document" &&
change.record.kind === NestedCallDoc.definition.kind &&
change.value !== null &&
change.conversationId === conversationId
) {
const state = change.value as NestedCallState;
if (state.taskId !== undefined && state.result !== undefined) {
nestedResults.set(state.taskId, state.result as unknown as NestedToolExecutionResult);
}
}
if (change.type === "entry" && change.value.conversationId === conversationId) entries.push(change.value);
if (change.type === "task" && change.value.conversationId === conversationId)
tasks.set(change.value.id, change.value);
if (change.type === "submission" && change.value.conversationId === conversationId) {
submissions.push(change.value);
}
}
if (viewOps.length === 0 && entries.length === 0 && tasks.size === 0 && submissions.length === 0) return [];
// Entries are appended in ID order; submission records are published in the order the commit first touched them.
submissions.sort((a, b) => a.id - b.id);
const was = parts(before);
const now = parts(after);
const events: AgentEvent[] = [];
// Progress: tool starts, the in-flight message, tool updates, retry and deferred state.
const slotsBefore = new Map((was.live.tools ?? []).map((slot) => [slot.callId, slot]));
const slots = now.live.tools ?? [];
const nestedBefore = new Map((was.live.nestedTools ?? []).map((slot) => [slot.taskId, slot]));
const nested = now.live.nestedTools ?? [];
for (const slot of [...slots, ...nested]) {
const previous = "parentCallId" in slot ? nestedBefore.get(slot.taskId) : slotsBefore.get(slot.callId);
if (slot.status !== "running" || previous?.status === "running") continue;
// A nested slot carries the arguments the call runs with; a model-issued call's are in its intent checkpoint.
const checkpoint = slot.taskId === undefined ? undefined : tasks.get(slot.taskId)?.state.checkpoint;
const args =
"parentCallId" in slot
? slot.arguments
: ((checkpoint as { arguments?: JsonObject } | undefined)?.arguments ?? {});
events.push({ type: "tool_execution_start", ...callOf(slot), args });
}
const partialBefore = was.live.generation?.message as AssistantMessage | undefined;
const partial = now.live.generation?.message as AssistantMessage | undefined;
if (partial !== undefined && partialBefore === undefined) events.push({ type: "message_start", message: partial });
else if (partial !== undefined && partial !== partialBefore) {
events.push({ type: "message_update", usage: partial.usage, changes: messageChanges(viewOps, partial) });
}
for (const [index, slot] of slots.entries()) {
const previous = slotsBefore.get(slot.callId);
if (slot.status !== "running" || previous?.status !== "running") continue;
const update = toolUpdate(viewOps, ["tools", index], slot, previous);
if (update === undefined) continue;
events.push({ type: "tool_execution_update", ...callOf(slot), ...update });
}
for (const [index, slot] of nested.entries()) {
const previous = nestedBefore.get(slot.taskId);
if (slot.status !== "running" || previous?.status !== "running") continue;
const update = toolUpdate(viewOps, ["nestedTools", index], slot, previous);
if (update === undefined) continue;
events.push({ type: "tool_execution_update", ...callOf(slot), ...update });
}
const generation = now.live.generation;
const generationBefore = was.live.generation;
if (generation?.retry !== undefined && generationBefore?.retry === undefined) {
const { at, error } = generation.retry;
events.push({ type: "auto_retry_start", attempt: generation.attempt, at, errorMessage: error });
}
if (generationBefore?.retry !== undefined && generation?.retry === undefined) {
events.push({ type: "auto_retry_end", attempt: generationBefore.attempt });
}
if (generation?.deferred !== undefined && generation.deferred.pollAt !== generationBefore?.deferred?.pollAt) {
events.push({ type: "deferred_poll", pollAt: generation.deferred.pollAt });
}
// Tools that end in this commit: a slot that becomes done, one created done (a call not offered), or an unfinished
// one that vanishes because its run ended. A done slot that vanishes ended earlier.
const toolEnds: Extract<AgentEvent, { type: "tool_execution_end" }>[] = [];
const endTool = (slot: ToolSlot, entryId: number | undefined): void => {
const entry = entries.find((candidate) => candidate.id === entryId);
toolEnds.push({ type: "tool_execution_end", ...callOf(slot), ...(entry === undefined ? {} : { entry }) });
};
for (const previous of slotsBefore.values()) {
if (previous.status === "done") continue;
const slot = slots.find((candidate) => candidate.callId === previous.callId);
if (slot?.status === "done") endTool(slot, slot.entry);
// A slot whose run ended in this commit may have had its result appended with it, as for unstarted calls.
else if (slot === undefined) endTool(previous, resultOf(entries, previous.callId)?.id);
}
for (const slot of slots) {
if (slot.status === "done" && !slotsBefore.has(slot.callId)) endTool(slot, slot.entry);
}
// Nested calls that end in this commit, the same way, children before the calls that made them: the lists hold
// parents first, so walk them backwards.
const endNested = (slot: NestedToolSlot): void => {
const result = nestedResults.get(slot.taskId);
events.push({ type: "tool_execution_end", ...callOf(slot), ...(result === undefined ? {} : { result }) });
};
const nestedNow = new Map(nested.map((slot) => [slot.taskId, slot]));
for (const previous of [...nestedBefore.values()].reverse()) {
if (previous.status === "done") continue;
const slot = nestedNow.get(previous.taskId);
if (slot?.status === "done" || slot === undefined) endNested(slot ?? previous);
}
for (const slot of [...nested].reverse()) {
if (slot.status === "done" && !nestedBefore.has(slot.taskId)) endNested(slot);
}
// Entries in append order; a tool's end directly precedes its result's message, as in the coding agent.
let assistantAppended = false;
for (const entry of entries) {
events.push(...toolEnds.filter((end) => end.entry === entry));
const message = entry.model?.[0];
if (message === undefined) {
events.push({ type: "entry_appended", entry });
continue;
}
// A streamed answer already started with its first partial.
const streamed = message.role === "assistant" && partialBefore !== undefined && !assistantAppended;
if (message.role === "assistant") assistantAppended = true;
if (!streamed) events.push({ type: "message_start", message });
events.push({ type: "message_end", entry });
}
// Ends without a result entry: a faulted or orphaned tool, or one whose run ended.
events.push(...toolEnds.filter((end) => end.entry === undefined));
// Compaction ends, task failures, then turn and run ends.
const compactionsBefore = was.live.compactions ?? [];
const compactions = now.live.compactions ?? [];
const failure = now.live.compactionFailure;
for (const { taskId, reason } of compactionsBefore) {
if (!compactions.some((status) => status.taskId === taskId)) {
const error = failure?.taskId === taskId ? failure.error : undefined;
events.push({ type: "compaction_end", taskId, reason, ...(error === undefined ? {} : { error }) });
}
}
// A generation's turn ends when its outcome is committed: at a `completing` hold or at terminal, whichever comes
// first, so a successor created at the hold starts after it.
let turnEnded = false;
for (const task of tasks.values()) {
const status = task.state.status;
if (task.kind === "pi.generation" && status === "completing" && !held.has(task.id)) {
held.add(task.id);
turnEnded = true;
}
if (status !== "terminal") continue;
if (task.kind === "pi.generation" && !held.delete(task.id)) turnEnded = true;
const outcome = task.state.outcome;
if (outcome.status === "faulted" || outcome.status === "orphaned") {
const message = outcome.status === "faulted" ? outcome.error.message : outcome.reason;
events.push({ type: "task_failed", taskId: task.id, kind: task.kind, message });
}
}
if (turnEnded) events.push({ type: "turn_end" });
const run = now.live.run;
const runBefore = was.live.run;
const runChanged = run?.inputs[0] !== runBefore?.inputs[0];
if (runBefore !== undefined && runChanged) events.push({ type: "run_end", inputs: runBefore.inputs });
// Submissions, document state, then what began.
for (const record of submissions) events.push({ type: "submission", record });
if (now.inbox !== was.inbox) events.push({ type: "inbox_update", items: queued(now.inbox) });
// A retired document reads as its initial value, as in a snapshot.
if (now.agent !== was.agent) {
events.push({ type: "agent_changed", agent: now.agent ?? AgentDoc.definition.initial() });
}
if (now.usage !== was.usage)
events.push({ type: "usage_changed", usage: now.usage ?? UsageDoc.definition.initial() });
for (const { taskId, reason, blocking } of compactions) {
if (!compactionsBefore.some((status) => status.taskId === taskId)) {
events.push({ type: "compaction_start", taskId, reason, blocking });
}
}
if (run !== undefined && runChanged) events.push({ type: "run_start", inputs: run.inputs });
if (run !== undefined && run.taskId !== runBefore?.taskId && tasks.get(run.taskId)?.kind === "pi.generation") {
events.push({ type: "turn_start" });
}
return events;
}
const PARTIAL_PATH = ["docs", "pi.live", "generation", "message"];
/** Translate the view operations on the in-flight message into message changes (spec §9.4). */
function messageChanges(viewOps: readonly Op[], message: AssistantMessage): MessageChange[] {
const changes: MessageChange[] = [];
// A block sent whole already holds every later change to it in this batch.
const whole = new Set<number>();
for (const op of viewOps) {
// View operations never replace the root.
const path = op[1] as Path;
if (!startsWith(path, PARTIAL_PATH)) {
// The whole message or generation was replaced.
if (startsWith(PARTIAL_PATH, path)) return [{ type: "message", message }];
continue;
}
const rest = path.slice(PARTIAL_PATH.length);
if (rest[0] === "usage") continue;
if (rest[0] !== "content") return [{ type: "message", message }];
if (rest.length === 1) {
if (op[0] !== "p" || op[3] !== 0) return [{ type: "message", message }];
for (const [offset, block] of (op[4] as unknown as Block[]).entries()) {
const type =
block.type === "text" ? "text_start" : block.type === "thinking" ? "thinking_start" : "toolcall_start";
changes.push({ type, contentIndex: op[2] + offset, block });
}
continue;
}
const contentIndex = rest[1] as number;
const field = rest[2];
if (whole.has(contentIndex)) continue;
if (op[0] === "a" && rest.length === 3 && (field === "text" || field === "thinking")) {
changes.push({ type: field === "text" ? "text_delta" : "thinking_delta", contentIndex, delta: op[2] });
} else if (op[0] === "a" && field === "arguments") {
changes.push({ type: "toolcall_delta", contentIndex, path: rest.slice(3), delta: op[2] });
} else {
whole.add(contentIndex);
changes.push({ type: "block", contentIndex, block: message.content[contentIndex]! });
}
}
return changes;
}
/** The identifying fields of a slot's tool events. */
function callOf(slot: ToolSlot | NestedToolSlot): ToolEventCall {
return {
toolCallId: slot.callId,
toolName: slot.name,
...(slot.taskId === undefined ? {} : { taskId: slot.taskId }),
...("parentCallId" in slot ? { parentToolCallId: slot.parentCallId, parentTaskId: slot.parentTaskId } : {}),
};
}
/** Output, details, and diagnostics changes of a running slot at `at` in `pi.live`, from the view operations on it. */
function toolUpdate(
viewOps: readonly Op[],
at: readonly ["tools" | "nestedTools", number],
slot: ToolSlot | NestedToolSlot,
previous: ToolSlot | NestedToolSlot,
): Omit<Extract<AgentEvent, { type: "tool_execution_update" }>, "type" | keyof ToolEventCall> | undefined {
const outputPath = ["docs", "pi.live", ...at, "output"];
let trimStart = 0;
let append = "";
let set = false;
for (const op of viewOps) {
if (!startsWith(op[1] as Path, outputPath)) continue;
if (op[0] === "t") trimStart += op[2];
else if (op[0] === "a") append += op[2];
else set = true;
}
let output: { trimStart?: number; append?: string } | { set: string } | undefined;
if (set || (slot.output !== previous.output && trimStart === 0 && append === ""))
output = { set: slot.output ?? "" };
else if (trimStart > 0 || append !== "") {
output = { ...(trimStart > 0 ? { trimStart } : {}), ...(append === "" ? {} : { append }) };
}
// A safe replay clears a running slot's progress: removed details send `null`, removed diagnostics `[]`.
const update = {
...(output === undefined ? {} : { output }),
...(slot.details === previous.details ? {} : { details: slot.details ?? null }),
...(slot.diagnostics === previous.diagnostics ? {} : { diagnostics: slot.diagnostics ?? [] }),
};
return Object.keys(update).length === 0 ? undefined : update;
}
function startsWith(path: Path, prefix: readonly (string | number)[]): boolean {
return prefix.length <= path.length && prefix.every((segment, index) => path[index] === segment);
}
packages/durable/src/harness/generation.ts
+17 -8
View File
@@ -21,7 +21,6 @@ import type {
EntryId,
NextTaskState,
SubmissionId,
SubmissionSettlement,
TaskId,
TaskOutcome,
TaskRuntime,
@@ -31,7 +30,7 @@ import type {
import { addTools, sectionExtension } from "./agent.ts";
import { compactionPolicy, createCompaction, estimateContext, selectCut } from "./compaction.ts";
import { applyBoundary, type Boundary, prepareBoundary, withdrawQueuedInputs } from "./inbox.ts";
import { endRun, LiveDoc, type LiveState, PROGRESS_COMMIT, type ToolSlot } from "./live.ts";
import { endRun, LiveDoc, type LiveState, PROGRESS_COMMIT, type RunEnd, type ToolSlot } from "./live.ts";
import { PartialChanges, type PartialMessage } from "./partial-changes.ts";
import { planSystemEntries, renderSections, replaySections } from "./prompt.ts";
import { ensureProviderSessionId } from "./provider.ts";
@@ -396,7 +395,12 @@ async function failNoModel(runtime: Runtime, ref: ModelRef | undefined, context:
await runtime.commit(async (tx) => {
const failure = await prepareFailure(tx, runtime);
const live = await tx.doc(LiveDoc, runtime.conversationId);
await endFailedRun(tx, runtime, live, failure, { status: "unanswered", reason: "no_model" });
await endFailedRun(tx, runtime, live, failure, {
status: "unanswered",
reason: "no_model",
detail: message,
settlement: { status: "unanswered", reason: "no_model" },
});
return { status: "terminal", outcome: { status: "failed", error: { message, detail: { reason: "no_model" } } } };
}, context);
}
@@ -420,14 +424,14 @@ async function endFailedRun(
runtime: Runtime,
live: Draft<LiveState>,
failure: Failure,
settlement: SubmissionSettlement,
end: Extract<RunEnd, { status: "unanswered" }>,
): Promise<void> {
const conversationId = runtime.conversationId;
endRun(tx, live, runtime.taskId, settlement);
endRun(tx, live, runtime.taskId, end);
if (failure.policy === "withdraw") await withdrawQueuedInputs(tx, conversationId, "run_failed");
if (failure.boundary === undefined) return;
const { users } = await applyBoundary(tx, failure.boundary, "final", runtime.now());
if (users.length > 0) await startRun(tx, conversationId, live, users);
if (users.length > 0) await startRun(tx, conversationId, live, users, { keepFailure: true });
}
/**
@@ -747,7 +751,7 @@ async function finishToolRound(
if (reset) {
// The queued reset cut the run's context before an answer.
endRun(tx, live, runtime.taskId, { status: "unanswered", reason: "reset" });
if (users.length > 0) await startRun(tx, conversationId, live, users);
if (users.length > 0) await startRun(tx, conversationId, live, users, { keepFailure: true });
} else {
delete live.tools;
delete live.nestedTools;
@@ -772,14 +776,19 @@ async function appendAssistant(
return tx.appendEntry(AssistantEntry, conversationId, { model: [message] });
}
/** Start a run for `inputs`, placed input submissions: a new generation takes `pi.live.run`. */
/**
* Start a run for `inputs`, placed input submissions: a new generation takes `pi.live.run`. Removes `runFailure`, unless
* `keepFailure`: a run started in the commit that ended the failed one keeps its failure.
*/
export async function startRun(
tx: Tx,
conversationId: ConversationId,
live: Draft<LiveState>,
inputs: SubmissionId[],
options?: { readonly keepFailure?: boolean },
): Promise<void> {
live.run = { taskId: await createGeneration(tx, conversationId, { submissionIds: inputs }), inputs };
if (options?.keepFailure !== true) delete live.runFailure;
}
/** A generation owned by its conversation. */
packages/durable/src/harness/harness.ts
+5 -4
View File
@@ -74,7 +74,7 @@ import type {
} from "./types.ts";
import { addUsageState, UsageDoc, type UsageState } from "./usage.ts";
import { scanAll } from "./util.ts";
import { type ConversationView, ConversationViews } from "./view.ts";
import { type ConversationView, ConversationViews, type ConversationWindow } from "./view.ts";
const SCAN_PAGE_SIZE = 256;
@@ -178,8 +178,8 @@ class ConversationImpl<Tool extends ToolRegistration> implements Conversation {
return this.#host.tasks.waitForIdle(this.id, context);
}
watch(context: Context): Promise<ConversationWatch> {
return this.#host.views.watch(this.id, context);
watch(context: Context, window?: ConversationWindow): Promise<ConversationWatch> {
return this.#host.views.watch(this.id, context, window);
}
}
@@ -384,9 +384,10 @@ class HarnessImpl<Tool extends ToolRegistration> extends SessionImpl implements
id: ConversationId,
observer: Observer<ConversationView>,
context: Context,
window?: ConversationWindow,
): Promise<Observation<ConversationView>> {
this.assertUsable();
return this.#host.views.observe(id, observer, context);
return this.#host.views.observe(id, observer, context, window);
}
root(
packages/durable/src/harness/live.ts
+66 -6
View File
@@ -44,7 +44,7 @@ export type ToolSlot = SlotProgress & {
/**
* Presentation of one nested call: a call a running tool made through `executeTool()`. `callId` is
* `<parentCallId>/<key>`, the call's ID in tool events and unique because keys contain no `/`. The Harness relates
* `<parentCallId>/<key>`, the call's ID, unique because keys contain no `/`. The Harness relates
* slots by task ID: `taskId` is the call's tool task, `parentTaskId` the calling one, model-issued or nested.
*/
export type NestedToolSlot = SlotProgress & {
@@ -87,6 +87,32 @@ export type CompactionFailure = {
error: string;
};
/**
* Why a run ended without an answer:
* - `model_error`: the model's response failed, after any retries, or the compaction a request needed failed;
* - `no_model`: no model is configured, or the configured one is not available;
* - `aborted`: the run was aborted (`Conversation.abort()`, `abortTask()`);
* - `reset`: a queued reset started a new context at the post-tools boundary, before an answer (a tool's handoff
* continues the run);
* - `faulted`: the run's task threw where it cannot recover, or storage rejected its commit;
* - `orphaned`: the run's task was aborted while no registered definition could take it.
*/
export type RunFailureReason = "model_error" | "no_model" | "aborted" | "reset" | "faulted" | "orphaned";
/** The newest run that ended without an answer (spec §8.2). */
export type RunFailure = {
/** The task that ended the run: the run's task at its end. */
taskId: TaskId;
/** The run's inputs, settled `unanswered`. */
inputs: SubmissionId[];
reason: RunFailureReason;
/**
* `model_error`: the error text; `no_model`: which model is missing; `faulted`: the error message; `orphaned`: why
* the task could not run (`missing_task`, `task_too_old`, `migration_failed`).
*/
detail?: string;
};
/** Built-in live conversation state: run control and presentation of the current generation and tool round. */
export type LiveState = {
/** Run control: the task that settles the run's inputs, and those inputs; present exactly while busy. */
@@ -115,6 +141,12 @@ export type LiveState = {
* removes it; one that is aborted or finds nothing to compact leaves it.
*/
compactionFailure?: CompactionFailure;
/**
* Why the newest run ended without an answer, so a client can show it after the run ended. Set in the commit that
* ends the run; removed when a later commit starts a run. A run started in the same commit, by
* `afterFailedRun: "continue"` or after a reset, keeps it.
*/
runFailure?: RunFailure;
};
export const LiveDoc = defineDoc<LiveState>({
@@ -147,13 +179,36 @@ const RUN_TASK_KINDS: ReadonlySet<string> = new Set(["pi.generation"]);
const TOOL_TASK_KIND = "pi.tool";
const COMPACTION_TASK_KIND = "pi.compaction";
/** How a run ends: with an answer, or without one, settling its inputs `unanswered` and recording why. */
export type RunEnd =
| { readonly status: "done"; readonly answer: EntryId }
| {
readonly status: "unanswered";
readonly reason: RunFailureReason;
readonly detail?: string;
/** The inputs' settlement; default `unanswered` with `reason` and `detail`. */
readonly settlement?: SubmissionSettlement;
};
/**
* End the run owned by `taskId`: settle each of its inputs and remove `run`. Always removes `generation`, `tools`, and
* `nestedTools`, whose presentation belongs to the ending run.
* End the run owned by `taskId`: settle each of its inputs, remove `run`, and, without an answer, record
* `runFailure`. Always removes `generation`, `tools`, and `nestedTools`, whose presentation belongs to the ending run.
*/
export function endRun(tx: Tx, live: Draft<LiveState>, taskId: TaskId, settlement: SubmissionSettlement): void {
export function endRun(tx: Tx, live: Draft<LiveState>, taskId: TaskId, end: RunEnd): void {
if (live.run?.taskId === taskId) {
for (const id of live.run.inputs) tx.settleSubmission(id, settlement);
const inputs = [...live.run.inputs];
if (end.status === "done") {
for (const id of inputs) tx.settleSubmission(id, { status: "done", answer: end.answer });
} else {
const { reason, detail } = end;
const settlement = end.settlement ?? {
status: "unanswered",
reason,
...(detail === undefined ? {} : { detail }),
};
for (const id of inputs) tx.settleSubmission(id, settlement);
live.runFailure = { taskId, inputs, reason, ...(detail === undefined ? {} : { detail }) };
}
delete live.run;
}
delete live.generation;
@@ -279,6 +334,11 @@ export async function settleSchedulerOutcome(
record.id,
outcome.status === "faulted"
? { status: "unanswered", reason: "faulted", detail: outcome.error.message }
: { status: "unanswered", reason: outcome.reason },
: {
status: "unanswered",
reason: "orphaned",
detail: outcome.reason,
settlement: { status: "unanswered", reason: outcome.reason },
},
);
}
packages/durable/src/harness/task-graph.ts
+2 -2
View File
@@ -15,7 +15,7 @@ import type {
import { type BlockedReason, type BlockedSubject, blockedSubject } from "./scheduler.ts";
import { closedError, scanAll } from "./util.ts";
/** A live task's durable status without its checkpoint and outcome payloads (spec §9.5). */
/** A live task's durable status without its checkpoint and outcome payloads (spec §9.4). */
export type TaskGraphState =
| { readonly status: "pending" | "running"; readonly phase: string }
| {
@@ -47,7 +47,7 @@ export type TaskGraphNode = {
readonly conversations: readonly ConversationId[];
};
/** Every live task of the Session (spec §9.5). */
/** Every live task of the Session (spec §9.4). */
export type TaskGraph = {
/** Every live task, keyed by its decimal ID. */
readonly tasks: Readonly<Record<string, TaskGraphNode>>;
packages/durable/src/harness/transcript.ts
+285
View File
@@ -0,0 +1,285 @@
import type { Context } from "@earendil-works/chord";
import { idFromNumber } from "../ids.ts";
import type { Cursor, EntryId, EntryQuery, EntryRecord, Page } from "../types.ts";
import type { ConversationView, ConversationWindow } from "./view.ts";
/** One row of a client's transcript. */
export type TranscriptItem =
| { readonly kind: "entry"; readonly entry: EntryRecord }
/** Entries strictly between `after` and `upTo` are missing: render "load more" and call `fillGap()`. */
| { readonly kind: "gap"; readonly after: EntryId; readonly upTo: EntryId }
/** The transcript continues above: render "load earlier" and call `loadEarlier()`. Always the first item. */
| { readonly kind: "earlier" };
/** The conversation view as a renderer sees it: `items`, the client's whole transcript, instead of the window. */
export type TranscriptView = Omit<ConversationView, "entries" | "before"> & {
readonly items: readonly TranscriptItem[];
};
/** Pages a conversation's entries, as `Conversation.entries()` does without a cursor. */
export type EntryLoader = (
query: Omit<EntryQuery, "conversationId">,
limit: number,
context: Context,
) => Promise<Page<EntryRecord, Cursor>>;
export type ConversationTranscriptOptions = {
/** Entries `loadEarlier()` and `fillGap()` load per call; default 50. */
readonly pageSize?: number;
};
type Gap = Extract<TranscriptItem, { kind: "gap" }>;
type EntryItem = Extract<TranscriptItem, { kind: "entry" }>;
const EARLIER: TranscriptItem = Object.freeze({ kind: "earlier" });
/**
* What a client has of one conversation: every entry it got from a view or fetched, and the gaps between (spec §9.3).
* A layer over any `ConversationView`, wherever it comes from: call `merge()` with every value of the view; it returns
* the view with the client's whole transcript as `items` instead of the window. Entries never change once committed,
* so the transcript only grows; a window that dropped its oldest entries keeps them here.
*
* A window whose `before` lies past the newest entry the client has does not continue it, after a stall, an overflow,
* or a reconnect: the entries in between become a gap. Nothing counts entries or computes ranges from entry IDs, which
* have holes; only exclusive bounds are turned into the inclusive bounds of `EntryQuery`. After a drop, a new watch
* with `resume` as its window continues the transcript.
*
* A merge costs O(items) and never re-sorts or re-fetches; a merge, `loadEarlier()`, or `fillGap()` that adds nothing
* returns the previous `items` array. Concurrent calls of `loadEarlier()`, or of `fillGap()` for the same gap, share one
* load, so a placeholder that stays visible does not fetch twice.
*/
export class ConversationTranscript {
readonly #load: EntryLoader;
readonly #pageSize: number;
/** Every entry held, in ID order, as items. */
#entries: EntryItem[] = [];
readonly #ids = new Set<EntryId>();
/** Gaps in ID order; each is an immutable item, replaced when it shrinks. */
#gaps: Gap[] = [];
/** Whether anything was merged yet. */
#started = false;
/** Exclusive lower bound of the known range: entries at or below it are not loaded; `-Infinity` at the start. */
#low = Number.NEGATIVE_INFINITY;
/** Inclusive upper bound of the known range: the newest entry, or a later empty window's `before`. */
#tail = Number.NEGATIVE_INFINITY;
#items: readonly TranscriptItem[] = [];
#last: ConversationView | undefined;
#view: TranscriptView | undefined;
#earlier: Promise<TranscriptView> | undefined;
readonly #filling = new Map<string, Promise<TranscriptView>>();
constructor(load: EntryLoader, options?: ConversationTranscriptOptions) {
const pageSize = options?.pageSize ?? 50;
if (!Number.isSafeInteger(pageSize) || pageSize < 1) {
throw new RangeError(`Page size must be an integer >= 1, got ${pageSize}`);
}
this.#load = load;
this.#pageSize = pageSize;
}
/**
* The ID the transcript reaches: its newest entry, or the `before` of a later window that was empty. Undefined before
* the first merge and for an empty transcript.
*/
get newest(): EntryId | undefined {
return this.#tail === Number.NEGATIVE_INFINITY ? undefined : idFromNumber<EntryId>(this.#tail);
}
/**
* The window that resumes this transcript after a drop: `{ from }` right after `newest`, so a new watch returns only
* what was missed when it fits its window, otherwise its newest entries and a gap; `{}` for an empty transcript;
* undefined before the first merge. `conversation.watch(context, transcript.resume)`.
*/
get resume(): ConversationWindow | undefined {
if (!this.#started) return undefined;
const newest = this.newest;
return newest === undefined ? {} : { from: idFromNumber<EntryId>(newest + 1) };
}
/** Take one value of the view: its window's entries join the transcript. */
merge(view: ConversationView): TranscriptView {
this.#last = view;
const { entries, before } = view;
const hi = entries.at(-1)?.id ?? before ?? Number.NEGATIVE_INFINITY;
this.#cover(before ?? Number.NEGATIVE_INFINITY, hi, entries);
return this.#result();
}
/** Load the page before the oldest entry held. Without anything earlier, returns the current view. */
loadEarlier(context: Context): Promise<TranscriptView> {
const last = this.#last;
if (last === undefined) return Promise.reject(new Error("ConversationTranscript: merge a view first"));
if (this.#low === Number.NEGATIVE_INFINITY) return Promise.resolve(this.#result());
if (this.#earlier !== undefined) return this.#earlier;
const low = this.#low;
// The loader runs in a later microtask, so even its synchronous throw settles a promise that is already shared.
const loading = Promise.resolve().then(async () => {
const query = { maxEntryId: idFromNumber<EntryId>(low), order: "descending" } as const;
const page = await this.#load(query, this.#pageSize, context);
const items = [...page.items].reverse();
const oldest = items[0];
// No next page, or an empty one: the transcript starts here.
if (page.next === undefined || oldest === undefined) this.#cover(Number.NEGATIVE_INFINITY, low, items);
else this.#cover(oldest.id - 1, low, items);
return this.#result();
});
this.#earlier = loading;
const settled = (): void => {
if (this.#earlier === loading) this.#earlier = undefined;
};
loading.then(settled, settled);
return loading;
}
/**
* Load the next page of `gap`, a gap item of `items`, shrinking it from its start, or removing it once nothing is
* missing. A gap that has shrunk since loads the gap it became; one that is no longer there returns the current view.
*/
fillGap(gap: Gap, context: Context): Promise<TranscriptView> {
const current =
this.#gaps.find((each) => each.after === gap.after && each.upTo === gap.upTo) ??
this.#gaps.find((each) => each.after < gap.upTo && gap.after < each.upTo);
if (current === undefined) {
const last = this.#last;
if (last === undefined) return Promise.reject(new Error("ConversationTranscript: merge a view first"));
return Promise.resolve(this.#result());
}
// Keyed by the gap the call resolved to, so a stale descriptor and the current one share the load.
const key = `${current.after}:${current.upTo}`;
const running = this.#filling.get(key);
if (running !== undefined) return running;
const filling = Promise.resolve().then(async () => {
const query = {
minEntryId: idFromNumber<EntryId>(current.after + 1),
maxEntryId: idFromNumber<EntryId>(current.upTo - 1),
order: "ascending",
} as const;
const page = await this.#load(query, this.#pageSize, context);
const newest = page.items.at(-1);
// No next page, or an empty one: nothing else is missing.
if (page.next === undefined || newest === undefined) this.#cover(current.after, current.upTo - 1, page.items);
else this.#cover(current.after, newest.id, page.items);
return this.#result();
});
this.#filling.set(key, filling);
const settled = (): void => {
if (this.#filling.get(key) === filling) this.#filling.delete(key);
};
filling.then(settled, settled);
return filling;
}
/**
* Record that every entry with an ID in (`lo`, `hi`] is known, `added` being those entries in ID order: hold the new
* ones, shrink the gaps they fill, extend the known range, and open a gap where the range does not continue it.
*/
#cover(lo: number, hi: number, added: readonly EntryRecord[]): void {
const fresh = added.filter((entry) => !this.#ids.has(entry.id));
for (const entry of fresh) this.#ids.add(entry.id);
const held = this.#entries;
// New entries past every held one: the items grow at their end.
const appended = fresh.length === 0 || held.length === 0 || fresh[0]!.id > held.at(-1)!.entry.id;
const items = fresh.map((entry): EntryItem => ({ kind: "entry", entry }));
if (items.length > 0) this.#entries = appended ? held.concat(items) : mergeById(held, items);
let rebuild = false;
if (!this.#started || (held.length === 0 && this.#gaps.length === 0 && lo >= this.#tail)) {
// The first range, or one after an empty transcript: it is all the client knows.
rebuild = !this.#started || lo !== this.#low;
this.#started = true;
this.#low = lo;
this.#tail = Math.max(this.#tail, hi);
} else if (lo > this.#tail) {
// Past the newest known ID: what lies between is missing.
const upTo = added[0]?.id ?? lo + 1;
this.#gaps = [...this.#gaps, gapOf(this.#tail, upTo)];
this.#tail = hi;
rebuild = true;
} else if (hi !== Number.NEGATIVE_INFINITY && hi < this.#low) {
// Below the oldest known ID: what lies between is missing.
const upTo = held.find((item) => item.entry.id > hi)?.entry.id ?? this.#low + 1;
this.#gaps = [gapOf(hi, upTo), ...this.#gaps];
this.#low = lo;
rebuild = true;
} else {
const gaps = this.#subtract(lo, hi);
if (gaps !== this.#gaps) {
this.#gaps = gaps;
rebuild = true;
}
if (lo < this.#low) {
this.#low = lo;
rebuild = true;
}
if (hi > this.#tail) this.#tail = hi;
}
if (rebuild || !appended) this.#items = this.#build();
else if (items.length > 0) this.#items = this.#items.concat(items);
}
/** The gaps without (`lo`, `hi`]; the same array when none changes, as when the range is empty. */
#subtract(lo: number, hi: number): Gap[] {
if (hi <= lo) return this.#gaps;
let changed = false;
const gaps: Gap[] = [];
for (const gap of this.#gaps) {
// Missing: IDs in (after, upTo); known now: IDs in (lo, hi].
if (hi <= gap.after || lo + 1 >= gap.upTo) {
gaps.push(gap);
continue;
}
changed = true;
if (lo > gap.after) {
const firstKnown = this.#entries.find((item) => item.entry.id > lo)?.entry.id ?? lo + 1;
gaps.push(gapOf(gap.after, Math.min(gap.upTo, firstKnown)));
}
if (hi + 1 < gap.upTo) gaps.push(gapOf(hi, gap.upTo));
}
return changed ? gaps : this.#gaps;
}
/** Items in ID order: "earlier" until the start is known, then entries with each gap before the entry after it. */
#build(): TranscriptItem[] {
const items: TranscriptItem[] = this.#low === Number.NEGATIVE_INFINITY ? [] : [EARLIER];
const gaps = this.#gaps;
let next = 0;
for (const item of this.#entries) {
while (next < gaps.length && gaps[next]!.after < item.entry.id) items.push(gaps[next++]!);
items.push(item);
}
while (next < gaps.length) items.push(gaps[next++]!);
return items;
}
/** The view with the current items; the previous one when nothing changed. */
#result(): TranscriptView {
const last = this.#last!;
const previous = this.#view;
if (
previous !== undefined &&
previous.items === this.#items &&
previous.docs === last.docs &&
previous.conversation === last.conversation
) {
return previous;
}
this.#view = { conversation: last.conversation, docs: last.docs, items: this.#items };
return this.#view;
}
}
function gapOf(after: number, upTo: number): Gap {
return Object.freeze({ kind: "gap", after: idFromNumber<EntryId>(after), upTo: idFromNumber<EntryId>(upTo) });
}
/** Two ID-ordered item lists merged in one pass. */
function mergeById(left: readonly EntryItem[], right: readonly EntryItem[]): EntryItem[] {
const merged: EntryItem[] = [];
let i = 0;
let j = 0;
while (i < left.length && j < right.length) {
merged.push(left[i]!.entry.id < right[j]!.entry.id ? left[i++]! : right[j++]!);
}
while (i < left.length) merged.push(left[i++]!);
while (j < right.length) merged.push(right[j++]!);
return merged;
}
packages/durable/src/harness/types.ts
+10 -8
View File
@@ -47,7 +47,7 @@ import type {
} from "../types.ts";
import type { TaskGraph, TaskGraphWatch } from "./task-graph.ts";
import type { UsageState } from "./usage.ts";
import type { ConversationView } from "./view.ts";
import type { ConversationView, ConversationWindow } from "./view.ts";
/** Provider and model ID resolved through pi-ai `Models`. */
export type ModelRef = {
@@ -795,10 +795,11 @@ export interface Conversation {
*/
waitForIdle(context: Context): Promise<void>;
/**
* The structural view (spec §9.3) as a serialized watch; commits made while its listener runs arrive folded. Built on
* the same mount as `Harness.observeConversation()`.
* The structural view (spec §9.3) as a serialized watch; commits made while its listener runs arrive folded. Its
* `entries` are a window over the transcript, as `window` says: the newest 50 by default, then appends, dropping the
* oldest past 100. Built on the same mount as `Harness.observeConversation()`.
*/
watch(context: Context): Promise<ConversationWatch>;
watch(context: Context, window?: ConversationWindow): Promise<ConversationWatch>;
}
export type ConversationWatch = WatchHandle<ConversationView>;
@@ -856,23 +857,24 @@ export interface Harness extends Session {
/** Session total: every conversation's `pi.usage` summed. */
usage(context: Context): Promise<UsageState>;
/**
* Every live task with its owner edge, status, and owned conversations (spec §9.5), as a serialized watch; commits
* Every live task with its owner edge, status, and owned conversations (spec §9.4), as a serialized watch; commits
* made while its listener runs arrive folded.
*/
watchTaskGraph(context: Context): Promise<TaskGraphWatch>;
/**
* Attach a synchronous `Observer` to a conversation's structural view (spec §9.3), atomically on the Session line,
* for adapters: `advance` gets each next revision with its operations as memory adopts the commit, inside Storage's
* `adopt` for a stored one and before it is stored for a deferred one, so it only records it. `context` governs only
* the attachment.
* `adopt` for a stored one and before it is stored for a deferred one, so it only records it. `window` as for
* `Conversation.watch()`; each observer has its own. `context` governs only the attachment.
*/
observeConversation(
id: ConversationId,
observer: Observer<ConversationView>,
context: Context,
window?: ConversationWindow,
): Promise<Observation<ConversationView>>;
/**
* Attach a synchronous `Observer` to the task graph (spec §9.5), as `observeConversation()` does to a view. A
* Attach a synchronous `Observer` to the task graph (spec §9.4), as `observeConversation()` does to a view. A
* `blocked` reason that changes without a commit (a registry change, a failed migration) advances it on the Session
* line outside any commit and `adopt`, with the background context.
*/
packages/durable/src/harness/view.ts
+168 -64
View File
@@ -1,12 +1,16 @@
import { type Context, type JsonValue, Watch, type WatchChange, type WatchHandle } from "@earendil-works/chord";
import { withoutAbortSignal } from "@earendil-works/chord/context";
import { applyImmutable, type NonEmptyPath, type Op, type Path } from "@earendil-works/chord/delta";
import { idFromNumber } from "../ids.ts";
import { contained, guardObserver, type ObservedCommit, observation, type SessionImpl } from "../session/session.ts";
import type {
ConversationDocToken,
ConversationId,
ConversationRecord,
Cursor,
DocumentId,
EntryId,
EntryQuery,
EntryRecord,
JsonObject,
Observation,
@@ -14,34 +18,50 @@ import type {
Storage,
} from "../types.ts";
import { AgentDoc } from "./agent.ts";
import { activeEntries, captureContextBounds } from "./context.ts";
import { InboxDoc } from "./inbox.ts";
import { LiveDoc } from "./live.ts";
import { ProviderDoc } from "./provider.ts";
import { UsageDoc } from "./usage.ts";
import { closedError } from "./util.ts";
/** Structural mount of one conversation's active transcript and built-in documents (spec §9.3). */
/** Structural view of one conversation: a window over its transcript and its built-in documents (spec §9.3). */
export type ConversationView = {
readonly conversation: ConversationRecord;
/** Raw active entries, as `ContextView.entries`: the head marker, then the non-head entries from its head. */
/**
* A window over the transcript, its fork-aware history (`Conversation.entries()`), in entry ID order. Head markers
* such as compaction summaries sit where they were written; their `head` tells where the entries they summarize end.
*/
readonly entries: readonly EntryRecord[];
/** ID of the entry just before the window; absent when the window starts the transcript. */
readonly before?: EntryId;
/** Built-in conversation documents keyed by kind; absent documents are absent. */
readonly docs: Readonly<Record<string, JsonObject>>;
};
/** Receives each next revision of a mount, and the Session's close. */
/**
* Which entries a view's window holds (spec §9.3). The window holds between `min` and `max` entries: it starts with
* the newest `min` entries, takes appends, and once it would hold more than `max`, drops its oldest in one splice down
* to `min`. With `from`, it starts at `from` instead when at most `max` entries are at or after it.
*/
export type ConversationWindow = {
/**
* Where the first value's window starts, inclusive, such as right after the newest entry a client has
* (`ConversationTranscript.resume`). It selects only that first value: every entry committed after the attachment is
* appended, whatever its ID. A `from` past the newest entry starts the window empty, with `before` the newest entry.
*/
readonly from?: EntryId;
/** Default 50; an integer, at least 1. */
readonly min?: number;
/** Default `2 * min`; an integer, at least `min`. */
readonly max?: number;
};
/** Default `ConversationWindow.min`. */
const DEFAULT_WINDOW_MIN = 50;
/** Receives each next revision of one observer's window, and the Session's close. */
export type ViewObserver = {
/** Each next revision; one change object is shared by every observer of the mount. */
advance?(change: WatchChange<ConversationView>, context: Context): void;
/** Every publication, after the mount took it; `ops` are the mount's, possibly none. */
publication?(
before: ConversationView,
after: ConversationView,
ops: readonly Op[],
publication: ObservedCommit,
context: Context,
): void;
advance(change: WatchChange<ConversationView>, context: Context): void;
/** The Session closed, or failed with `failure`. */
end(failure: { readonly error: unknown } | undefined): void;
};
@@ -54,13 +74,26 @@ const MOUNTED = [
UsageDoc,
] as unknown as readonly ConversationDocToken<JsonObject>[];
const MOUNTED_KINDS: ReadonlySet<string> = new Set(MOUNTED.map((token) => token.definition.kind));
const SCAN_PAGE_SIZE = 256;
/** One conversation's mount: its current revision, the document incarnations it shows, and its observers. */
type Mount = {
/** One observer of a mount and its window: the bounds and the revision it was last given. */
type Windowed = {
readonly min: number;
readonly max: number;
value: ConversationView;
readonly observer: ViewObserver;
};
/**
* One conversation's mount: the documents it shows, shared by every observer, and the observers, each with its own
* window over the entries.
*/
type Mount = {
readonly conversation: ConversationRecord;
docs: Readonly<Record<string, JsonObject>>;
/** Mounted incarnation and definition version per kind; another incarnation or version is set whole. */
readonly docs: Map<string, { id: DocumentId; version: number }>;
readonly observers: Set<ViewObserver>;
readonly incarnations: Map<string, { id: DocumentId; version: number }>;
readonly observers: Set<Windowed>;
};
const VIEWS = new WeakMap<object, ConversationViews>();
@@ -72,10 +105,28 @@ export function conversationViews(harness: object): ConversationViews {
return views;
}
/** A window with its defaults applied. */
type WindowBounds = { readonly from: EntryId | undefined; readonly min: number; readonly max: number };
/** `window` with its defaults; throws a `RangeError` for bounds that are not integers or `max` below `min`. */
function resolveWindow(window: ConversationWindow | undefined): WindowBounds {
const min = window?.min ?? DEFAULT_WINDOW_MIN;
if (!Number.isSafeInteger(min) || min < 1) throw new RangeError(`Window min must be an integer >= 1, got ${min}`);
const max = window?.max ?? 2 * min;
if (!Number.isSafeInteger(max) || max < min) {
throw new RangeError(`Window max must be an integer >= min (${min}), got ${max}`);
}
const from = window?.from;
if (from !== undefined && !Number.isSafeInteger(from)) {
throw new RangeError(`Window from must be an entry ID, got ${from}`);
}
return { from, min, max };
}
/**
* The Harness's conversation view mounts: at most one per conversation, built on the Session line by its first
* observer and dropped with its last. Each mount advances from the Session's commits as memory adopts them, deferred ones
* (`CommitPersistence`) before they are stored.
* (`CommitPersistence`) before they are stored, and hands each observer the next revision of its own window.
*/
export class ConversationViews {
readonly #session: SessionImpl;
@@ -94,9 +145,9 @@ export class ConversationViews {
session.subscribeClose(() => {
this.#closed = true;
for (const mount of this.#mounts.values())
for (const observer of [...mount.observers]) {
for (const windowed of [...mount.observers]) {
contained(
() => observer.end(this.#session.failure),
() => windowed.observer.end(this.#session.failure),
(error) => this.report(error),
);
}
@@ -117,10 +168,12 @@ export class ConversationViews {
id: ConversationId,
observer: Observer<ConversationView>,
context: Context,
window?: ConversationWindow,
): Promise<Observation<ConversationView>> {
let value!: ConversationView;
const attached = await this.attach(
id,
window,
(current, release) => {
value = current;
return guardObserver(observer, release, (error) => this.report(error));
@@ -136,9 +189,14 @@ export class ConversationViews {
}
/** A serialized watch of the view, an observer of its mount; cancelling `context` stops it. */
async watch(id: ConversationId, context: Context): Promise<WatchHandle<ConversationView>> {
async watch(
id: ConversationId,
context: Context,
window?: ConversationWindow,
): Promise<WatchHandle<ConversationView>> {
const { observer } = await this.attach(
id,
window,
(value, release) => new Watch<ConversationView>(value, release, (error) => this.#session.report(error)),
context,
);
@@ -152,27 +210,32 @@ export class ConversationViews {
}
/**
* Register an observer created from the current revision, atomically on the Session line: it sees every later
* publication and nothing earlier. `create` may read committed Storage, still on the line. `release` drops it, and
* the mount with its last observer.
* Register an observer created from its window's first revision, atomically on the Session line: it sees every later
* publication and nothing earlier. `release` drops it, and the mount with its last observer.
*/
attach<O extends ViewObserver>(
async attach<O extends ViewObserver>(
id: ConversationId,
create: (value: ConversationView, release: () => void, storage: Storage) => O | Promise<O>,
window: ConversationWindow | undefined,
create: (value: ConversationView, release: () => void) => O,
context: Context,
): Promise<{ observer: O; detach: () => void }> {
const bounds = resolveWindow(window);
return this.#session.readOnLine(async () => {
const mount = this.#mounts.get(id) ?? (await this.#build(id, context));
const { entries, before } = await loadWindow(this.#storage, id, bounds, context);
const value = view(mount.conversation, entries, before, mount.docs);
let windowed: Windowed | undefined;
const detach = (): void => {
mount.observers.delete(observer);
if (windowed !== undefined) mount.observers.delete(windowed);
if (mount.observers.size === 0 && this.#mounts.get(id) === mount) this.#mounts.delete(id);
};
const observer = await create(mount.value, detach, this.#storage);
const observer = create(value, detach);
// Close or cancellation may begin while the mount hydrates; register nothing then.
if (this.#closed) throw closedError(this.#session);
context.abortSignal?.throwIfAborted();
windowed = { min: bounds.min, max: bounds.max, value, observer };
this.#mounts.set(id, mount);
mount.observers.add(observer);
mount.observers.add(windowed);
return { observer, detach };
});
}
@@ -180,8 +243,6 @@ export class ConversationViews {
async #build(id: ConversationId, context: Context): Promise<Mount> {
const conversation = await this.#storage.conversation(id, context);
if (conversation === undefined) throw new Error(`Conversation ${id} does not exist`);
const bounds = await captureContextBounds(this.#storage, id, context);
const entries = await activeEntries(this.#storage, id, bounds, context);
const docs: Record<string, JsonObject> = {};
const incarnations = new Map<string, { id: DocumentId; version: number }>();
for (const token of MOUNTED) {
@@ -190,13 +251,60 @@ export class ConversationViews {
docs[token.definition.kind] = loaded.value;
incarnations.set(token.definition.kind, { id: loaded.record.id, version: loaded.version });
}
return { value: { conversation, entries, docs }, docs: incarnations, observers: new Set() };
return { conversation, docs, incarnations, observers: new Set() };
}
}
/** A view revision; `before` is a key only when set. */
function view(
conversation: ConversationRecord,
entries: readonly EntryRecord[],
before: EntryId | undefined,
docs: Readonly<Record<string, JsonObject>>,
): ConversationView {
return before === undefined ? { conversation, entries, docs } : { conversation, entries, before, docs };
}
/**
* Derive the mount's operations from one publication, apply them, and hand the revision to every observer; one that
* throws is reported through `report`, and the others still get it.
* A new observer's window, read on the Session line with bounded scans: the entries from `from` when at most `max` are
* at or after it, otherwise the newest `min`, and the entry before them.
*/
async function loadWindow(
storage: Storage,
conversationId: ConversationId,
bounds: WindowBounds,
context: Context,
): Promise<{ entries: readonly EntryRecord[]; before: EntryId | undefined }> {
const { from, min, max } = bounds;
if (from !== undefined) {
const since = await scanUpTo(storage, { conversationId, minEntryId: from, order: "ascending" }, max + 1, context);
if (since.length <= max) {
const query = { conversationId, maxEntryId: idFromNumber<EntryId>(from - 1) };
const previous = (await storage.scanEntries(query, 1, undefined, context)).items[0];
return { entries: since, before: previous?.id };
}
}
const newest = await scanUpTo(storage, { conversationId }, min + 1, context);
const before = newest.length > min ? newest.pop()!.id : undefined;
return { entries: newest.reverse(), before };
}
/** Up to `limit` entries of `query`, following cursors past short pages. */
async function scanUpTo(storage: Storage, query: EntryQuery, limit: number, context: Context): Promise<EntryRecord[]> {
const entries: EntryRecord[] = [];
let cursor: Cursor | undefined;
do {
const page = await storage.scanEntries(query, Math.min(SCAN_PAGE_SIZE, limit - entries.length), cursor, context);
entries.push(...page.items);
cursor = page.next;
} while (cursor !== undefined && entries.length < limit);
return entries;
}
/**
* Derive the mount's operations from one publication and hand each observer the next revision of its window; one that
* throws is reported through `report`, and the others still get theirs. Documents are applied once and shared; each
* window takes the appended entries and, past its `max`, drops its oldest down to its `min` in one splice.
*/
function advance(
id: ConversationId,
@@ -206,55 +314,51 @@ function advance(
report: (error: unknown) => void,
): void {
const docOps: Op[] = [];
const entryOps: Op[] = [];
let entries = mount.value.entries;
// Entry writes are published in ID order.
const appended: EntryRecord[] = [];
for (const change of publication.changes) {
if (change.type === "entry" && change.value.conversationId === id) {
const entry = change.value;
const value = entry as unknown as JsonValue;
if (entry.head === undefined) {
entryOps.push(["p", ["entries"], entries.length, 0, [value]]);
entries = [...entries, entry];
continue;
}
// A head marker keeps the non-head entries from its head, which are always a suffix, and goes in front.
const target = entry.head;
let kept = entries.findIndex((candidate) => candidate.head === undefined && candidate.id >= target);
if (kept < 0) kept = entries.length;
entryOps.push(["p", ["entries"], 0, kept, [value]]);
entries = [entry, ...entries.slice(kept)];
appended.push(change.value);
continue;
}
if (change.type !== "document" || change.conversationId !== id) continue;
const kind = change.record.kind;
if (!MOUNTED_KINDS.has(kind) || change.record.key !== undefined) continue;
const path: NonEmptyPath = ["docs", kind];
const mounted = mount.docs.get(kind);
const mounted = mount.incarnations.get(kind);
if (change.value === null) {
if (mounted?.id !== change.record.id) continue;
mount.docs.delete(kind);
mount.incarnations.delete(kind);
docOps.push(["d", path]);
} else if (mounted?.id === change.record.id && mounted.version === change.version) {
for (const op of change.ops) docOps.push(prefixed(op, path));
} else {
mount.docs.set(kind, { id: change.record.id, version: change.version! });
mount.incarnations.set(kind, { id: change.record.id, version: change.version! });
docOps.push(["s", path, change.value]);
}
}
const before = mount.value;
const ops = [...docOps, ...entryOps];
if (docOps.length === 0 && appended.length === 0) return;
if (docOps.length > 0) mount.docs = applyImmutable({ docs: mount.docs }, docOps).docs;
const frameContext = withoutAbortSignal(context);
if (ops.length > 0) {
const value = docOps.length === 0 ? mount.value : applyImmutable(mount.value, docOps);
mount.value = entries === value.entries ? value : { ...value, entries };
const change: WatchChange<ConversationView> = { value: mount.value, ops, retired: false };
for (const observer of [...mount.observers]) {
contained(() => observer.advance?.(change, frameContext), report);
const inserted = appended as unknown as JsonValue[];
for (const windowed of [...mount.observers]) {
const previous = windowed.value;
let entries = previous.entries;
let before = previous.before;
let ops: Op[] = docOps;
if (appended.length > 0) {
ops = [...docOps, ["p", ["entries"], entries.length, 0, inserted]];
entries = entries.concat(appended);
if (entries.length > windowed.max) {
const drop = entries.length - windowed.min;
before = entries[drop - 1]!.id;
entries = entries.slice(drop);
ops.push(["p", ["entries"], 0, drop, []], ["s", ["before"], before]);
}
}
}
for (const observer of [...mount.observers]) {
contained(() => observer.publication?.(before, mount.value, ops, publication, frameContext), report);
windowed.value = view(mount.conversation, entries, before, mount.docs);
const change: WatchChange<ConversationView> = { value: windowed.value, ops, retired: false };
contained(() => windowed.observer.advance(change, frameContext), report);
}
}
packages/durable/src/index.ts
+10 -9
View File
@@ -25,14 +25,6 @@ export {
type SummaryRequest,
} from "./harness/compaction.ts";
export { defineExtension, defineTool, hook, section, wrapSection, wrapTool } from "./harness/define.ts";
export {
type AgentEvent,
type AgentEventStream,
type MessageChange,
type SnapshotEvent,
type ToolEventCall,
watchEvents,
} from "./harness/events.ts";
export {
type GenerationCheckpoint,
type GenerationInput,
@@ -48,6 +40,8 @@ export {
type LiveState,
type NestedToolSlot,
type NestedToolSummary,
type RunFailure,
type RunFailureReason,
type ToolSlot,
} from "./harness/live.ts";
export { ProviderDoc, type ProviderState } from "./harness/provider.ts";
@@ -63,6 +57,13 @@ export {
type ToolTaskInput,
type ToolTaskResult,
} from "./harness/tool.ts";
export {
ConversationTranscript,
type ConversationTranscriptOptions,
type EntryLoader,
type TranscriptItem,
type TranscriptView,
} from "./harness/transcript.ts";
export type {
AfterFailedRun,
Agent,
@@ -123,7 +124,7 @@ export type {
Wrap,
} from "./harness/types.ts";
export { UsageDoc, type UsageState } from "./harness/usage.ts";
export type { ConversationView } from "./harness/view.ts";
export type { ConversationView, ConversationWindow } from "./harness/view.ts";
export { createSession } from "./session/session.ts";
export { MemoryStorage } from "./storage/memory.ts";
export { defineTask } from "./tasks.ts";
packages/durable/test/conversation-transcript.test.ts
+356
View File
@@ -0,0 +1,356 @@
// Tests for `ConversationTranscript`, the client-side layer that merges view windows into one transcript (#11327).
import {
ConversationTranscript,
type ConversationView,
type Cursor,
type EntryId,
type EntryLoader,
type EntryRecord,
MemoryStorage,
type TranscriptItem,
type TranscriptView,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { allEntries, chatSetup, openChat } from "./chat-support.ts";
import { context } from "./session-support.ts";
import { deferred } from "./task-support.ts";
const CONVERSATION = { id: 1 } as unknown as ConversationView["conversation"];
/** Entries with IDs that have holes: every third ID is missing. */
function transcript(count: number): EntryRecord[] {
const entries: EntryRecord[] = [];
for (let id = 1; entries.length < count; id++) {
if (id % 3 !== 0) entries.push({ id, conversationId: CONVERSATION.id, kind: "note" } as unknown as EntryRecord);
}
return entries;
}
/** A loader over `entries` with `Conversation.entries()` semantics, counting its calls. */
function loader(entries: readonly EntryRecord[]): EntryLoader & { calls: number } {
const load = async (query: Parameters<EntryLoader>[0], limit: number) => {
load.calls++;
const matching = entries.filter(
(entry) =>
(query.minEntryId === undefined || entry.id >= query.minEntryId) &&
(query.maxEntryId === undefined || entry.id <= query.maxEntryId),
);
const ordered = query.order === "ascending" ? matching : [...matching].reverse();
const items = ordered.slice(0, limit);
return { items, ...(ordered.length > limit ? { next: "more" as unknown as Cursor } : {}) };
};
load.calls = 0;
return load;
}
/** The view of a window: `entries` with the entry before them in `all`. */
function windowOf(
all: readonly EntryRecord[],
first: number,
last: number,
docs: ConversationView["docs"] = {},
): ConversationView {
const before = all[first - 1]?.id;
return {
conversation: CONVERSATION,
entries: all.slice(first, last + 1),
...(before === undefined ? {} : { before }),
docs,
};
}
/** Items as a compact list: entry IDs, `gap after..upTo`, and `earlier`. */
function shape(view: TranscriptView): (number | string)[] {
return view.items.map((item) =>
item.kind === "entry" ? item.entry.id : item.kind === "gap" ? `gap ${item.after}..${item.upTo}` : "earlier",
);
}
const idsOf = (entries: readonly EntryRecord[]) => entries.map((entry) => entry.id);
const gapOf = (view: TranscriptView) =>
view.items.find((item): item is Extract<TranscriptItem, { kind: "gap" }> => item.kind === "gap")!;
describe("ConversationTranscript", () => {
it("shows a first window under an `earlier` marker, or alone when it starts the transcript", () => {
const all = transcript(20);
const later = new ConversationTranscript(loader(all));
expect(shape(later.merge(windowOf(all, 15, 19)))).toEqual(["earlier", ...idsOf(all.slice(15))]);
expect(later.newest).toBe(all[19]!.id);
const whole = new ConversationTranscript(loader(all));
expect(shape(whole.merge(windowOf(all, 0, 19)))).toEqual(idsOf(all));
});
it("appends, keeps entries the window dropped, and returns the same items when nothing is added", () => {
const all = transcript(30);
const helper = new ConversationTranscript(loader(all));
const docsA = { a: { n: 1 } };
const first = helper.merge(windowOf(all, 10, 19, docsA));
const grown = helper.merge(windowOf(all, 10, 21, docsA));
expect(shape(grown)).toEqual(["earlier", ...idsOf(all.slice(10, 22))]);
// The window dropped its oldest; the transcript keeps them.
const spliced = helper.merge(windowOf(all, 18, 21, docsA));
expect(spliced.items).toBe(grown.items);
expect(spliced).toBe(grown);
// A docs-only change: a new view, the same items.
const docs = { a: { n: 2 } };
const streamed = helper.merge({ ...windowOf(all, 18, 21), docs });
expect(streamed).not.toBe(grown);
expect(streamed.items).toBe(grown.items);
expect(streamed.docs).toBe(docs);
expect(first.items).not.toBe(grown.items);
});
it("opens a gap when a window does not continue the transcript, and fills it page by page", async () => {
const all = transcript(200);
const load = loader(all);
const helper = new ConversationTranscript(load, { pageSize: 50 });
helper.merge(windowOf(all, 150, 159));
// A stall: the next window starts well past the newest entry held.
const stalled = helper.merge(windowOf(all, 190, 199));
const gap = gapOf(stalled);
expect(gap).toEqual({ kind: "gap", after: all[159]!.id, upTo: all[190]!.id });
expect(shape(stalled)).toEqual([
"earlier",
...idsOf(all.slice(150, 160)),
`gap ${all[159]!.id}..${all[190]!.id}`,
...idsOf(all.slice(190)),
]);
// 30 entries are missing; one page loads them all and closes the gap.
const filled = await helper.fillGap(gap, context);
expect(shape(filled)).toEqual(["earlier", ...idsOf(all.slice(150))]);
expect(load.calls).toBe(1);
});
it("shrinks a gap from its start while more than a page is missing", async () => {
const all = transcript(200);
const helper = new ConversationTranscript(loader(all), { pageSize: 10 });
helper.merge(windowOf(all, 100, 109));
let view = helper.merge(windowOf(all, 190, 199));
view = await helper.fillGap(gapOf(view), context);
expect(gapOf(view)).toEqual({ kind: "gap", after: all[119]!.id, upTo: all[190]!.id });
while (view.items.some((item) => item.kind === "gap")) view = await helper.fillGap(gapOf(view), context);
expect(shape(view)).toEqual(["earlier", ...idsOf(all.slice(100))]);
});
it("loads earlier pages until the start, then drops the marker", async () => {
const all = transcript(45);
const load = loader(all);
const helper = new ConversationTranscript(load, { pageSize: 20 });
helper.merge(windowOf(all, 40, 44));
let view = await helper.loadEarlier(context);
expect(shape(view)).toEqual(["earlier", ...idsOf(all.slice(20))]);
// A page without a next cursor reaches the start.
view = await helper.loadEarlier(context);
expect(shape(view)).toEqual(idsOf(all));
const calls = load.calls;
expect(await helper.loadEarlier(context)).toBe(view);
expect(load.calls).toBe(calls);
});
it("shares one load between concurrent calls of loadEarlier, and of fillGap for one gap", async () => {
const all = transcript(100);
const inner = loader(all);
const gate = deferred();
const load: EntryLoader = async (query, limit, callContext) => {
await gate.promise;
return inner(query, limit, callContext);
};
const helper = new ConversationTranscript(load);
helper.merge(windowOf(all, 40, 49));
const stalled = helper.merge(windowOf(all, 90, 99));
const earlier = [helper.loadEarlier(context), helper.loadEarlier(context)];
const fills = [helper.fillGap(gapOf(stalled), context), helper.fillGap(gapOf(stalled), context)];
expect(earlier[0]).toBe(earlier[1]);
expect(fills[0]).toBe(fills[1]);
gate.resolve();
await Promise.all([...earlier, ...fills]);
expect(inner.calls).toBe(2);
expect(shape(await helper.loadEarlier(context))).toEqual(idsOf(all));
});
it("resumes after `newest`: an empty window right after it adds nothing; one past it opens a gap", () => {
const all = transcript(30);
const helper = new ConversationTranscript(loader(all));
const view = helper.merge(windowOf(all, 10, 19));
const resumed: ConversationView = { conversation: CONVERSATION, entries: [], before: helper.newest!, docs: {} };
expect(helper.merge(resumed).items).toBe(view.items);
// A window that is empty but starts past `newest`: the entries up to its `before` are missing.
const past = helper.merge({ ...resumed, before: all[25]!.id });
expect(shape(past).at(-1)).toBe(`gap ${all[19]!.id}..${all[25]!.id + 1}`);
expect(helper.newest).toBe(all[25]!.id);
// The next window continues it.
expect(shape(helper.merge(windowOf(all, 26, 27))).slice(-3)).toEqual([
`gap ${all[19]!.id}..${all[25]!.id + 1}`,
all[26]!.id,
all[27]!.id,
]);
});
it("fills a gap from a window that covers it", () => {
const all = transcript(100);
const helper = new ConversationTranscript(loader(all));
helper.merge(windowOf(all, 40, 49));
helper.merge(windowOf(all, 90, 99));
// A wider window, such as a new watch with a larger `max`, covers the second half of the gap.
const view = helper.merge(windowOf(all, 70, 99));
expect(shape(view)).toEqual([
"earlier",
...idsOf(all.slice(40, 50)),
`gap ${all[49]!.id}..${all[70]!.id}`,
...idsOf(all.slice(70)),
]);
});
it("rejects loadEarlier before the first merge and returns the view for a gap that is gone", async () => {
const all = transcript(10);
const helper = new ConversationTranscript(loader(all));
await expect(helper.loadEarlier(context)).rejects.toThrow("merge a view first");
const view = helper.merge(windowOf(all, 0, 9));
const stale = { kind: "gap", after: 1 as EntryId, upTo: 2 as EntryId } as const;
expect(await helper.fillGap(stale, context)).toBe(view);
});
});
describe("ConversationTranscript loads", () => {
it("retries a load whose loader threw synchronously, for loadEarlier and fillGap", async () => {
const all = transcript(100);
const inner = loader(all);
let offline = true;
let calls = 0;
const load: EntryLoader = (query, limit, callContext) => {
calls++;
// An adapter that checks its connection before it returns a promise.
if (offline) throw new Error("temporarily offline");
return inner(query, limit, callContext);
};
const helper = new ConversationTranscript(load);
helper.merge(windowOf(all, 40, 49));
const stalled = helper.merge(windowOf(all, 90, 99));
await expect(helper.loadEarlier(context)).rejects.toThrow("temporarily offline");
await expect(helper.fillGap(gapOf(stalled), context)).rejects.toThrow("temporarily offline");
offline = false;
expect(shape(await helper.loadEarlier(context))).toEqual([
...idsOf(all.slice(0, 50)),
`gap ${all[49]!.id}..${all[90]!.id}`,
...idsOf(all.slice(90)),
]);
expect(shape(await helper.fillGap(gapOf(stalled), context))).toEqual(idsOf(all));
expect(calls).toBe(4);
});
it("retries a load that rejected", async () => {
const all = transcript(30);
const inner = loader(all);
let fail = true;
const load: EntryLoader = async (query, limit, callContext) => {
if (fail) throw new Error("timeout");
return inner(query, limit, callContext);
};
const helper = new ConversationTranscript(load);
helper.merge(windowOf(all, 20, 29));
await expect(helper.loadEarlier(context)).rejects.toThrow("timeout");
fail = false;
expect(shape(await helper.loadEarlier(context))).toEqual(idsOf(all));
});
it("keeps one gap for an empty window whose `before` lies inside it", () => {
const all = transcript(100);
const helper = new ConversationTranscript(loader(all));
helper.merge(windowOf(all, 40, 49));
const before = helper.merge(windowOf(all, 90, 99));
const view = helper.merge({ conversation: CONVERSATION, entries: [], before: all[65]!.id, docs: before.docs });
expect(view.items).toBe(before.items);
expect(view.items.filter((item) => item.kind === "gap")).toHaveLength(1);
});
it("shares one load between a stale gap descriptor and the gap it shrank into", async () => {
const all = transcript(100);
const inner = loader(all);
const gate = deferred();
const load: EntryLoader = async (query, limit, callContext) => {
await gate.promise;
return inner(query, limit, callContext);
};
const helper = new ConversationTranscript(load, { pageSize: 10 });
helper.merge(windowOf(all, 10, 19));
const stale = gapOf(helper.merge(windowOf(all, 90, 99)));
gate.resolve();
const current = gapOf(await helper.fillGap(stale, context));
expect(current).not.toEqual(stale);
const loads = inner.calls;
const [a, b] = [helper.fillGap(stale, context), helper.fillGap(current, context)];
expect(a).toBe(b);
await a;
expect(inner.calls).toBe(loads + 1);
});
it("treats an empty page with a next cursor as the end of the range", async () => {
const all = transcript(60);
// A remote loader that always returns a cursor.
const inner = loader(all);
const load: EntryLoader = async (query, limit, callContext) => ({
...(await inner(query, limit, callContext)),
next: "more" as unknown as Cursor,
});
const helper = new ConversationTranscript(load, { pageSize: 100 });
helper.merge(windowOf(all, 10, 19));
let view = helper.merge(windowOf(all, 40, 49));
view = await helper.fillGap(gapOf(view), context);
// The first page held every missing entry; the next one is empty and closes the gap.
view = await helper.fillGap(gapOf(view), context);
expect(gapOf(view)).toBeUndefined();
await helper.loadEarlier(context);
view = await helper.loadEarlier(context);
expect(shape(view)).toEqual(idsOf(all.slice(0, 50)));
});
it("resumes with `resume`: undefined before a merge, `{}` for an empty transcript, `{ from }` after the newest", () => {
const all = transcript(30);
const helper = new ConversationTranscript(loader(all));
expect(helper.resume).toBeUndefined();
helper.merge({ conversation: CONVERSATION, entries: [], docs: {} });
expect(helper.resume).toEqual({});
helper.merge(windowOf(all, 0, 9));
expect(helper.resume).toEqual({ from: all[9]!.id + 1 });
});
});
describe("ConversationTranscript over a Harness", () => {
it("follows the walkthrough: live appends, a stall, fillGap, loadEarlier, and a resume with `from`", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const note = () => root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
for (let index = 0; index < 120; index++) await note();
const helper = new ConversationTranscript((query, limit, callContext) =>
root.entries(query, limit, undefined, callContext),
);
const window = { min: 10, max: 20 };
const watch = await root.watch(context, window);
let view = helper.merge(watch.value);
watch.start(async ({ value }) => {
view = helper.merge(value);
});
for (let index = 0; index < 15; index++) await note();
await new Promise((resolve) => setTimeout(resolve, 0));
expect(view.items.some((item) => item.kind === "gap")).toBe(false);
expect(view.items.filter((item) => item.kind === "entry")).toHaveLength(25);
// A drop: the client misses 40 entries, more than `max`, and resumes after the newest it has.
await watch.stop();
for (let index = 0; index < 40; index++) await note();
const resumed = await root.watch(context, { ...window, ...helper.resume });
view = helper.merge(resumed.value);
expect(view.items.filter((item) => item.kind === "gap")).toHaveLength(1);
while (view.items.some((item) => item.kind === "gap")) view = await helper.fillGap(gapOf(view), context);
while (view.items[0]?.kind === "earlier") view = await helper.loadEarlier(context);
const entries = view.items.flatMap((item) => (item.kind === "entry" ? [item.entry.id] : []));
expect(entries).toEqual(idsOf(await allEntries(root)));
// A short drop resumes without a gap.
for (let index = 0; index < 3; index++) await note();
const again = await root.watch(context, { ...window, ...helper.resume });
expect(again.value.entries).toHaveLength(3);
view = helper.merge(again.value);
expect(view.items.some((item) => item.kind !== "entry")).toBe(false);
await resumed.stop();
await again.stop();
await harness.close(context);
});
});
packages/durable/test/conversation-view-bench.ts
+234
View File
@@ -0,0 +1,234 @@
/**
* Conversation view benchmark: a long conversation (thousands of entries) and the watches of its view.
*
* node --conditions=source --experimental-strip-types --expose-gc test/conversation-view-bench.ts
*
* Scenarios, each in its own child process so heap numbers are its own, on SQLite so history stays out of the heap:
* - `start`: acquire a watch of a conversation of N entries (the mount is built every time; the watch is stopped).
* - `append`: K started watches; commit entries one per commit, or in bursts of 50 per commit, and wait for delivery.
* - `resume`: a watch with `from` 20 entries back, as a client that reconnects after a short drop.
* - `stream`: K started watches; commits that change only `pi.live` (a streamed answer), no entries.
* - `long-lived`: one started watch while 20,000 entries are appended; heap after GC, and whether the first entry
* delivered is still reachable.
*/
import { execFile } from "node:child_process";
import { mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import type { Context } from "@earendil-works/chord";
import { BACKGROUND_CONTEXT as context } from "@earendil-works/chord/context";
import { createModels } from "@earendil-works/pi-ai/models";
import {
type Conversation,
type ConversationView,
createRegistry,
type EntryId,
Harness,
LiveDoc,
type Storage,
} from "../src/index.ts";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
type Scenario =
| { kind: "start"; entries: number }
| { kind: "append"; entries: number; watches: number; burst: number; commits: number }
| { kind: "resume"; entries: number }
| { kind: "stream"; entries: number; watches: number; commits: number }
| { kind: "long-lived"; entries: number };
const TEXT = "The quick brown fox jumps over the lazy dog. ".repeat(10);
type Watch = Awaited<ReturnType<Conversation["watch"]>>;
/** `watch(context, options)`; an older Harness ignores the options. */
type WatchWith = (context: Context, options?: { from?: EntryId }) => Promise<Watch>;
async function main(): Promise<void> {
const [, , encoded] = process.argv;
if (encoded !== undefined) {
const result = await run(JSON.parse(encoded) as Scenario);
process.stdout.write(`${JSON.stringify(result)}\n`);
return;
}
const scenarios: Scenario[] = [
{ kind: "start", entries: 5000 },
{ kind: "resume", entries: 5000 },
{ kind: "append", entries: 5000, watches: 1, burst: 1, commits: 2000 },
{ kind: "append", entries: 5000, watches: 4, burst: 1, commits: 2000 },
{ kind: "append", entries: 5000, watches: 4, burst: 50, commits: 200 },
{ kind: "stream", entries: 5000, watches: 4, commits: 5000 },
{ kind: "long-lived", entries: 20000 },
];
const rows: Record<string, unknown>[] = [];
for (const scenario of scenarios) {
const { stdout } = await promisify(execFile)(
process.execPath,
[...process.execArgv, fileURLToPath(import.meta.url), JSON.stringify(scenario)],
{ maxBuffer: 1 << 20 },
);
rows.push({ scenario: label(scenario), ...JSON.parse(stdout.trim().split("\n").at(-1)!) });
}
console.table(rows);
}
function label(scenario: Scenario): string {
switch (scenario.kind) {
case "start":
return `watch start, ${scenario.entries} entries`;
case "resume":
return `resume 20 back, ${scenario.entries} entries`;
case "append":
return `append x${scenario.burst} per commit, ${scenario.watches} watches`;
case "stream":
return `pi.live only, ${scenario.watches} watches`;
case "long-lived":
return `long-lived watch, ${scenario.entries} appends`;
}
}
function gc(): void {
(globalThis as { gc?: () => void }).gc?.();
}
function heapMiB(): number {
gc();
gc();
return Math.round((process.memoryUsage().heapUsed / 2 ** 20) * 10) / 10;
}
async function appendEntries(conversation: Conversation, count: number, perCommit: number): Promise<void> {
for (let done = 0; done < count; done += perCommit) {
await conversation.commit(async (tx) => {
for (let index = done; index < Math.min(count, done + perCommit); index++) {
await tx.appendEntry(conversation.id, { kind: "note", data: { text: `${index} ${TEXT}` } });
}
}, context);
}
}
/** Resolve once every watch delivered a value whose newest entry is `id`. */
function delivered(watches: readonly Watch[], id: () => EntryId | undefined): Promise<void> {
return new Promise((resolve) => {
const check = (): void => {
const target = id();
if (watches.every((watch) => watch.value.entries.at(-1)?.id === target)) resolve();
else setTimeout(check, 1);
};
check();
});
}
async function run(scenario: Scenario): Promise<Record<string, number>> {
const directory = mkdtempSync(join(tmpdir(), "pi-durable-view-bench-"));
let storage: Storage | undefined;
try {
storage = await openNodeSqliteStorage(join(directory, "session.sqlite"));
const models = createModels();
const harness = await Harness.open(storage, { models, registry: createRegistry() }, context);
const root = await harness.root(context);
await appendEntries(root, scenario.entries, 500);
const newest = async (): Promise<EntryId> => (await root.entries({}, 1, undefined, context)).items[0]!.id;
const watchWith = root.watch.bind(root) as unknown as WatchWith;
const heapBefore = heapMiB();
const cpu = process.cpuUsage();
const started = performance.now();
const result: Record<string, number> = {};
if (scenario.kind === "start" || scenario.kind === "resume") {
const from = scenario.kind === "resume" ? (((await newest()) - 19) as EntryId) : undefined;
const times: number[] = [];
let size = 0;
for (let round = 0; round < 20; round++) {
const at = performance.now();
const watch = await watchWith(context, from === undefined ? undefined : { from });
times.push(performance.now() - at);
size = watch.value.entries.length;
await watch.stop();
}
times.sort((a, b) => a - b);
result.p50Ms = round(times[10]!);
result.windowEntries = size;
} else if (scenario.kind === "append") {
const watches = await Promise.all(Array.from({ length: scenario.watches }, () => root.watch(context)));
for (const watch of watches) watch.start(async () => {});
let last: EntryId | undefined;
const at = performance.now();
for (let index = 0; index < scenario.commits; index++) {
const ids = await root.commit(async (tx) => {
const appended: EntryId[] = [];
for (let n = 0; n < scenario.burst; n++) {
appended.push((await tx.appendEntry(root.id, { kind: "note", data: { text: TEXT } })).id);
}
return appended;
}, context);
last = ids.at(-1);
}
await delivered(watches, () => last);
result.perCommitUs = round(((performance.now() - at) * 1000) / scenario.commits);
result.windowEntries = watches[0]!.value.entries.length;
result.heapWithWatchesMiB = heapMiB();
for (const watch of watches) await watch.stop();
} else if (scenario.kind === "stream") {
const watches = await Promise.all(Array.from({ length: scenario.watches }, () => root.watch(context)));
let calls = 0;
for (const watch of watches)
watch.start(async () => {
calls++;
});
const at = performance.now();
for (let index = 0; index < scenario.commits; index++) {
await root.commit(async (tx) => {
(await tx.doc(LiveDoc, root.id)).compactionFailure = {
taskId: 1 as never,
reason: "manual",
error: `${index}`,
};
}, context);
}
// Every watch delivered the final value, not just one of them.
const last = `${scenario.commits - 1}`;
const done = () =>
watches.every(
(watch) =>
(watch.value.docs["pi.live"] as { compactionFailure?: { error: string } } | undefined)
?.compactionFailure?.error === last,
);
while (calls < scenario.watches || !done()) await new Promise((resolve) => setTimeout(resolve, 1));
result.perCommitUs = round(((performance.now() - at) * 1000) / scenario.commits);
for (const watch of watches) await watch.stop();
} else {
const watch = await root.watch(context);
let first: WeakRef<object> | undefined;
watch.start(async ({ value }: { value: ConversationView }) => {
const entry = value.entries.at(-1);
if (first === undefined && entry !== undefined) first = new WeakRef(entry);
});
await appendEntries(root, 1, 1);
const id = await newest();
await delivered([watch], () => id);
await appendEntries(root, scenario.entries, 1);
const last = await newest();
await delivered([watch], () => last);
result.windowEntries = watch.value.entries.length;
result.heapWithWatchMiB = heapMiB();
// A WeakRef's target survives the job that created or read it; collect again in a later task.
await new Promise((resolve) => setTimeout(resolve, 10));
gc();
result.firstEntryCollected = first?.deref() === undefined ? 1 : 0;
await watch.stop();
}
const usage = process.cpuUsage(cpu);
result.cpuMs = round((usage.user + usage.system) / 1000);
result.wallMs = round(performance.now() - started);
result.heapBeforeMiB = heapBefore;
await harness.close(context);
return result;
} finally {
rmSync(directory, { recursive: true, force: true });
}
}
function round(value: number): number {
return Math.round(value * 100) / 100;
}
await main();
packages/durable/test/examples/18-json.ts
+18 -27
View File
@@ -1,11 +1,11 @@
// JSON mode: stream one prompt's run as JSON lines, like `pi --mode json`. Two modes:
// --events (default): the experimental agent events of watchEvents(), starting with a `snapshot` event.
// --ops: the raw ConversationView frames, starting with the view itself; each later line holds one commit's ops.
// JSON mode: stream one prompt's run as JSON lines, like `pi --mode json`. Two modes, both of the ConversationView:
// --ops (default): the raw frames, starting with the view itself; each later line holds the ops of the commits since.
// --state: per call, the entries appended since the previous one and the current `pi.live`, state a renderer draws.
// --storage sqlite (default) | jsonl | memory: sqlite writes a temporary database file, jsonl a temporary directory;
// neither is deleted, and the path is printed to stderr at the end.
// Uses OpenAI when OPENAI_API_KEY is set, and a scripted faux model otherwise.
// Run from packages/durable:
// node --conditions=source --experimental-strip-types test/examples/18-json.ts --events "What is in this directory?"
// node --conditions=source --experimental-strip-types test/examples/18-json.ts --state "What is in this directory?"
import { mkdtemp } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
@@ -14,22 +14,14 @@ import { createModels } from "@earendil-works/pi-ai/models";
import { fauxAssistantMessage, fauxProvider, fauxText, fauxToolCall } from "@earendil-works/pi-ai/providers/faux";
import { openaiProvider } from "@earendil-works/pi-ai/providers/openai";
import { NodeExecutionEnv } from "../../src/env/node.ts";
import {
createRegistry,
defineExtension,
Harness,
MemoryStorage,
type Storage,
section,
watchEvents,
} from "../../src/index.ts";
import { createRegistry, defineExtension, Harness, MemoryStorage, type Storage, section } from "../../src/index.ts";
import { openNodeJsonlStorage } from "../../src/storage/jsonl/node.ts";
import { openNodeSqliteStorage } from "../../src/storage/sqlite/node.ts";
import { createBashTool, createReadTool } from "../../src/tools/index.ts";
const context = BACKGROUND_CONTEXT;
const args = process.argv.slice(2);
const mode = args.includes("--ops") ? "ops" : "events";
const mode = args.includes("--state") ? "state" : "ops";
const storageIndex = args.indexOf("--storage");
const storageKind = storageIndex < 0 ? "sqlite" : args[storageIndex + 1];
const prompt =
@@ -77,20 +69,19 @@ const env = new NodeExecutionEnv({ cwd: process.cwd() });
const harness = await Harness.open(storage, { models, registry, env: () => env }, context);
const root = await harness.root(context, { agent: { model } });
// Attach before submitting, so the stream covers the whole run.
let stop: () => Promise<unknown>;
if (mode === "events") {
const stream = await watchEvents(harness, root.id, context);
print(stream.snapshot);
stream.start(async ({ value: events }) => {
for (const event of events) print(event);
});
stop = () => stream.stop();
} else {
const watch = await root.watch(context);
// Attach before submitting, so the watch covers the whole run.
const watch = await root.watch(context);
if (mode === "ops") {
print({ view: watch.value });
watch.start(async ({ ops }) => print({ ops }));
stop = () => watch.stop();
} else {
let newest = watch.value.entries.at(-1)?.id ?? 0;
print({ entries: watch.value.entries, live: watch.value.docs["pi.live"] });
watch.start(async ({ value }) => {
const entries = value.entries.filter((entry) => entry.id > newest);
newest = value.entries.at(-1)?.id ?? newest;
print({ ...(entries.length === 0 ? {} : { entries }), live: value.docs["pi.live"] });
});
}
const submission = await root.submit({ type: "input", content: prompt }, context);
@@ -98,6 +89,6 @@ await submission.wait(context);
await harness.waitForIdle(context);
// Let the last batch reach the listener before stopping.
await new Promise((resolve) => setTimeout(resolve, 0));
await stop();
await watch.stop();
await harness.close(context);
if (location !== undefined) console.error(`${storageKind} storage: ${location}`);
packages/durable/test/examples/20-late-join.ts
+14 -29
View File
@@ -1,5 +1,5 @@
// Late join: a client attaches while a run is already underway. It gets the current state first, the conversation
// view or the snapshot event, and then only what changes after that.
// view, and then only what changes after that.
// Run from packages/durable:
// node --conditions=source --experimental-strip-types test/examples/20-late-join.ts
import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context";
@@ -13,7 +13,6 @@ import {
Harness,
type LiveState,
MemoryStorage,
watchEvents,
} from "../../src/index.ts";
const context = BACKGROUND_CONTEXT;
@@ -54,7 +53,8 @@ const submission = await root.submit({ type: "input", content: "Count to ten, th
// Join while the tool is halfway through.
await pause(500);
// Structural client: the view holds the committed transcript and documents, including the running tool's output.
// The view holds the newest entries and the documents, including the running tool's output and, later, the answer
// being streamed.
const view = await root.watch(context);
const live = view.value.docs["pi.live"] as LiveState;
console.log(
@@ -62,38 +62,23 @@ console.log(
view.value.entries.map((entry) => entry.kind),
);
console.log("view tool slot:", live.tools?.[0]?.status, JSON.stringify(live.tools?.[0]?.output));
let shown = "";
view.start(async ({ value }) => {
const slot = (value.docs["pi.live"] as LiveState).tools?.[0];
const state = value.docs["pi.live"] as LiveState;
const slot = state.tools?.[0];
if (slot?.status === "running") console.log("view output now:", JSON.stringify(slot.output));
});
// Event client: the snapshot event carries the same state; later events apply on top of it.
const stream = await watchEvents(harness, root.id, context);
console.log(
"snapshot tools:",
stream.snapshot.tools.map((slot) => `${slot.name} ${slot.status}`),
);
let output = stream.snapshot.tools[0]?.output ?? "";
stream.start(async ({ value: events }) => {
for (const event of events) {
if (event.type === "tool_execution_update" && event.output !== undefined) {
output =
"set" in event.output
? event.output.set
: output.slice(event.output.trimStart ?? 0) + (event.output.append ?? "");
console.log("event output now:", JSON.stringify(output));
} else if (event.type === "message_update") {
const deltas = event.changes.flatMap((change) => (change.type === "text_delta" ? [change.delta] : []));
console.log("event text delta:", JSON.stringify(deltas.join("")));
} else {
console.log("event:", event.type);
}
}
const partial = state.generation?.message?.content.find((block) => block.type === "text")?.text ?? "";
if (partial !== shown && partial !== "") console.log("view answer now:", JSON.stringify(partial));
shown = partial;
if (state.run === undefined)
console.log(
"view entries:",
value.entries.map((entry) => entry.kind),
);
});
await submission.wait(context);
await harness.waitForIdle(context);
await pause(0);
await stream.stop();
await view.stop();
await harness.close(context);
packages/durable/test/examples/21-subagent-foreground.ts
+20 -18
View File
@@ -1,6 +1,6 @@
// A foreground subagent tool: the parent's tool call creates a child conversation it owns, runs one task there, and
// returns the child's answer. Aborting the tool call aborts the child. A UI finds the child through the tool's running
// details and shows its events under the call.
// details and shows its view under the call.
// Uses OpenAI when OPENAI_API_KEY is set, and a scripted faux model otherwise.
// Run from packages/durable:
// node --conditions=source --experimental-strip-types test/examples/21-subagent-foreground.ts
@@ -11,7 +11,6 @@ import { createModels } from "@earendil-works/pi-ai/models";
import { fauxAssistantMessage, fauxProvider, fauxText, fauxToolCall } from "@earendil-works/pi-ai/providers/faux";
import { openaiProvider } from "@earendil-works/pi-ai/providers/openai";
import {
type AgentEvent,
AssistantEntry,
type ConversationId,
configure,
@@ -19,11 +18,12 @@ import {
defineExtension,
defineTool,
type EntryId,
type EntryRecord,
type Extension,
Harness,
type LiveState,
MemoryStorage,
type ToolExecutionApi,
watchEvents,
} from "../../src/index.ts";
const context = BACKGROUND_CONTEXT;
@@ -99,26 +99,28 @@ registry.install(Subagent);
const harness = await Harness.open(new MemoryStorage(), { models, registry }, context);
const root = await harness.root(context, { agent: { model } });
// ─── UI: the parent's events, with each subagent's events indented under its call ───
// ─── UI: the parent's transcript, with each subagent's indented under its call ───
const print = (indent: string, event: AgentEvent): void => {
if (event.type === "message_end" && event.entry.kind === "pi.assistant") {
const message = event.entry.model?.[0] as AssistantMessage;
const text = message.content.flatMap((c) => (c.type === "text" ? [c.text] : [])).join("");
if (text !== "") console.log(`${indent}assistant: ${text}`);
} else if (event.type === "tool_execution_start") {
console.log(`${indent}tool ${event.toolName}(${JSON.stringify(event.args)})`);
/** Print an assistant entry's text and tool calls. */
const print = (indent: string, entry: EntryRecord): void => {
const message = entry.model?.[0];
if (entry.kind !== "pi.assistant" || message?.role !== "assistant") return;
for (const block of (message as AssistantMessage).content) {
if (block.type === "text" && block.text !== "") console.log(`${indent}assistant: ${block.text}`);
if (block.type === "toolCall") console.log(`${indent}tool ${block.name}(${JSON.stringify(block.arguments)})`);
}
};
const attached = new Set<ConversationId>();
const attach = async (id: ConversationId, indent: string): Promise<void> => {
attached.add(id);
const stream = await watchEvents(harness, id, context);
stream.start(async ({ value: events }) => {
for (const event of events) {
print(indent, event);
if (event.type !== "tool_execution_update") continue;
const child = (event.details as { conversationId?: ConversationId } | undefined)?.conversationId;
const watch = await (await harness.conversation(id, context))!.watch(context);
let newest = watch.value.entries.at(-1)?.id ?? 0;
watch.start(async ({ value }) => {
for (const entry of value.entries) if (entry.id > newest) print(indent, entry);
newest = value.entries.at(-1)?.id ?? newest;
// A running subagent call names its child conversation in its details.
for (const slot of (value.docs["pi.live"] as LiveState).tools ?? []) {
const child = (slot.details as { conversationId?: ConversationId } | undefined)?.conversationId;
if (child !== undefined && !attached.has(child)) await attach(child, `${indent} `);
}
});
@@ -131,6 +133,6 @@ const submission = await root.submit(
);
await submission.wait(context);
await harness.waitForIdle(context);
// Event callbacks run after their commit; let the last ones print.
// Watch callbacks run after their commit; let the last ones print.
await new Promise((resolve) => setTimeout(resolve, 0));
await harness.close(context);
packages/durable/test/examples/22-subagent-background.ts
+7 -6
View File
@@ -26,7 +26,6 @@ import {
Harness,
LiveDoc,
type TaskId,
watchEvents,
} from "../../src/index.ts";
import { openNodeSqliteStorage } from "../../src/storage/sqlite/node.ts";
@@ -283,11 +282,13 @@ function contentText(content: string | readonly { type: string; text?: string }[
/** Print the main conversation's messages as they are committed. Subagents work in their own conversations. */
async function follow(harness: Harness, id: ConversationId): Promise<void> {
const stream = await watchEvents(harness, id, context);
stream.start(async ({ value: events }) => {
for (const event of events) {
if (event.type !== "message_end") continue;
const message = event.entry.model?.[0];
const watch = await (await harness.conversation(id, context))!.watch(context);
let newest = watch.value.entries.at(-1)?.id ?? 0;
watch.start(async ({ value }) => {
const entries = value.entries.filter((entry) => entry.id > newest);
newest = value.entries.at(-1)?.id ?? newest;
for (const entry of entries) {
const message = entry.model?.[0];
if (message?.role === "user") {
const text = contentText(message.content);
// Input is either the user or a subagent's report, which arrives whenever the subagent is done.
packages/durable/test/examples/31-transcript.ts
+106
View File
@@ -0,0 +1,106 @@
// A client's transcript over the windowed conversation view: `ConversationTranscript` merges every value of a watch
// into the client's whole transcript, marks entries a stall skipped as a gap, and loads them, and earlier pages, on
// demand. A client that drops resumes after the newest entry it has.
// Run from packages/durable:
// node --conditions=source --experimental-strip-types test/examples/31-transcript.ts
import { BACKGROUND_CONTEXT } from "@earendil-works/chord/context";
import { createModels } from "@earendil-works/pi-ai/models";
import {
ConversationTranscript,
createRegistry,
Harness,
MemoryStorage,
type TranscriptItem,
type TranscriptView,
} from "../../src/index.ts";
const context = BACKGROUND_CONTEXT;
const harness = await Harness.open(
new MemoryStorage(),
{ models: createModels(), registry: createRegistry() },
context,
);
const root = await harness.root(context);
let count = 0;
/** Commit `n` chat lines, one entry each. */
const chat = async (n: number): Promise<void> => {
for (let index = 0; index < n; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "chat", data: `line ${++count}` }), context);
}
};
const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
/** One line per item: ranges of entries, gaps, and the "load earlier" marker. */
function render(label: string, view: TranscriptView): void {
const rows: string[] = [];
let run: [number, number] | undefined;
const flush = () => {
if (run !== undefined) rows.push(run[0] === run[1] ? `${run[0]}` : `${run[0]}..${run[1]}`);
run = undefined;
};
for (const item of view.items) {
if (item.kind === "entry") {
const line = Number(String(item.entry.data).slice("line ".length));
run = run === undefined ? [line, line] : [run[0], line];
continue;
}
flush();
rows.push(item.kind === "gap" ? "[load more]" : "[load earlier]");
}
flush();
console.log(`${label.padEnd(28)} ${rows.join(" ")}`);
}
const firstGap = (view: TranscriptView) =>
view.items.find((item): item is Extract<TranscriptItem, { kind: "gap" }> => item.kind === "gap");
await chat(200);
// Attach: the window holds the newest `min` entries; the transcript shows "load earlier" above them.
const transcript = new ConversationTranscript((query, limit, ctx) => root.entries(query, limit, undefined, ctx), {
pageSize: 20,
});
const window = { min: 10, max: 20 };
let watch = await root.watch(context, window);
let view = transcript.merge(watch.value);
render("attach", view);
watch.start(async ({ value }) => {
view = transcript.merge(value);
});
// Live: appends. The window drops its oldest past `max`; the transcript keeps them.
await chat(15);
await settle();
render("15 more, live", view);
// The user scrolls up: one page.
view = await transcript.loadEarlier(context);
render("load earlier", view);
// A drop: the client misses 40 entries, more than `max`, and resumes after the newest it has. The window starts at the
// newest `min` again, and `before` shows the transcript a gap.
await watch.stop();
await chat(40);
watch = await root.watch(context, { ...window, ...transcript.resume });
view = transcript.merge(watch.value);
render("resume after 40 missed", view);
watch.start(async ({ value }) => {
view = transcript.merge(value);
});
// "load more" fills the gap a page at a time.
for (let gap = firstGap(view); gap !== undefined; gap = firstGap(view)) {
view = await transcript.fillGap(gap, context);
render("fill gap", view);
}
// A short drop resumes without a gap: the window starts right after the newest entry.
await watch.stop();
await chat(3);
watch = await root.watch(context, { ...window, ...transcript.resume });
console.log(`${"resume after 3 missed".padEnd(28)} window: ${watch.value.entries.length} entries`);
view = transcript.merge(watch.value);
render("", view);
await watch.stop();
await harness.close(context);
packages/durable/test/harness-compaction.test.ts
+77 -84
View File
@@ -13,7 +13,6 @@ import {
type UserMessage,
} from "@earendil-works/pi-ai";
import {
type AgentEvent,
type CompactionPolicy,
type CompactionResult,
CompactionTask,
@@ -24,11 +23,11 @@ import {
type EntryRecord,
type Harness,
LiveDoc,
type LiveState,
MemoryStorage,
ProviderDoc,
type TaskId,
UsageDoc,
watchEvents,
} from "@earendil-works/pi-durable";
import { afterEach, describe, expect, it } from "vitest";
import { selectCut, serializeConversation } from "../src/harness/compaction.ts";
@@ -36,6 +35,7 @@ import { orderToolResults } from "../src/harness/context.ts";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, type ChatSetup, chatSetup, openChat, textOf, toolsNamed, waitFor } from "./chat-support.ts";
import { addHooks, addSection, addTask, addTool } from "./harness-support.ts";
import { viewHistory } from "./live-support.ts";
import { ControlledStorage, context } from "./session-support.ts";
import { aborted, type Deferred, deferred } from "./task-support.ts";
@@ -1006,40 +1006,37 @@ describe("background threshold compaction", () => {
await chat.harness.close(context);
});
it("records a failure in pi.live, the snapshot, and compaction_end until a summary is produced", async () => {
it("records a failure in pi.live, in the commit that ends the compaction, until a summary is produced", async () => {
const chat = await open({ contextWindow: 2000 });
await history(chat);
chat.setup.settings.compaction = BACKGROUND;
const events: AgentEvent[] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const views = await viewHistory(chat.harness, chat.root.id);
chat.faux.summaries.push(failure("bad request"));
await turn(chat, text("u4", 100), text("a4", 100));
await waitFor(() => events.some((event) => event.type === "compaction_end"));
await waitFor(async () => (await live(chat)).compactionFailure !== undefined);
const recorded = (await live(chat)).compactionFailure;
expect(recorded).toMatchObject({ reason: "threshold", error: "Summarization failed: bad request" });
expect(events.find((event) => event.type === "compaction_end")).toEqual({
type: "compaction_end",
taskId: recorded!.taskId,
reason: "threshold",
error: "Summarization failed: bad request",
});
const reopened = await watchEvents(chat.harness, chat.root.id, context);
expect(reopened.snapshot.compactionFailure).toEqual(recorded);
await reopened.stop();
// A compaction that produces a summary removes the failure, and ends without an error.
// The value that drops the compaction's status records its failure.
const ended = (taskId: TaskId) => {
const states = views.live();
const listed = states.findIndex((state) => state.compactions?.some((status) => status.taskId === taskId));
return states.find(
(state, index) => index > listed && !state.compactions?.some((status) => status.taskId === taskId),
)!;
};
expect(ended(recorded!.taskId).compactionFailure).toEqual(recorded);
// A late watch starts with it.
const late = await chat.root.watch(context);
expect((late.value.docs["pi.live"] as LiveState).compactionFailure).toEqual(recorded);
await late.stop();
// A compaction that produces a summary removes the failure in the commit that ends it.
chat.setup.settings.compaction = MANUAL;
chat.faux.summaries.push(summary());
const manual = await chat.root.compact(undefined, context);
expect((await result(chat, manual)).status).toBe("completed");
expect((await live(chat)).compactionFailure).toBeUndefined();
await waitFor(() => events.some((event) => event.type === "compaction_end" && event.taskId === manual));
expect(events.find((event) => event.type === "compaction_end" && event.taskId === manual)).not.toHaveProperty(
"error",
);
await stream.stop();
expect(ended(manual).compactionFailure).toBeUndefined();
views.detach();
await chat.harness.close(context);
});
@@ -1179,21 +1176,22 @@ describe("blocking threshold compaction", () => {
chat.setup.settings.compaction = BLOCKING;
const reached = deferred();
chat.faux.summaries.push(gated(deferred(), summary(), reached));
const events: AgentEvent[] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const views = await viewHistory(chat.harness, chat.root.id);
const input = await chat.root.submit({ type: "input", content: text("u4", 200) }, context);
await reached.promise;
const [child] = await compactionTasks(chat);
await chat.root.abort(context);
expect(await input.wait(context)).toMatchObject({ status: "unanswered", reason: "aborted" });
expect((await chat.harness.getTask(child!.id, context))?.state).toMatchObject({ outcome: { status: "aborted" } });
await waitFor(() => events.some((event) => event.type === "run_end"));
const types = events.map((event) => event.type);
expect(types.indexOf("compaction_end")).toBeLessThan(types.indexOf("run_end"));
await stream.stop();
views.detach();
// The compaction's status leaves before the run ends.
const states = views.live();
const busy = states.findIndex((state) => state.compactions !== undefined);
const compactionEnd = states.findIndex((state, index) => index > busy && state.compactions === undefined);
const runEnd = states.findIndex((state, index) => index > busy && state.run === undefined);
expect(busy).toBeGreaterThan(0);
expect(compactionEnd).toBeLessThan(runEnd);
expect(states[runEnd]!.runFailure).toMatchObject({ inputs: [input.id], reason: "aborted" });
await chat.harness.close(context);
});
@@ -1448,9 +1446,11 @@ describe("compaction estimates and interactions", () => {
expect(view.messages.slice(1).map(userText)).toEqual([text("u3", 100), text("a3", 100)]);
// The parent is untouched.
expect((await kinds(chat.root)).includes("pi.compaction")).toBe(false);
// The fork's view keeps the parent entries the summary kept.
// The fork's view shows the parent entries in ID order, the summary after them, its head on a parent entry.
const watch = await fork.watch(context);
expect(watch.value.entries).toEqual(view.entries);
expect(watch.value.entries).toEqual(await allEntries(fork));
expect(watch.value.entries.at(-1)).toEqual(view.head);
expect(watch.value.entries.some((entry) => entry.id === u3.id)).toBe(true);
await watch.stop();
await chat.harness.close(context);
});
@@ -1565,21 +1565,17 @@ function defineChildTask(gate: Deferred) {
});
}
describe("compaction events and live status", () => {
it("reports start and end, and the retry backoff in a late joiner's snapshot", async () => {
describe("compaction live status", () => {
it("lists a compaction from start to end, with its retry backoff in a late joiner's view", async () => {
const chat = await open();
await history(chat);
chat.setup.settings.retry = { enabled: true, maxRetries: 2, baseDelayMs: 60_000 };
const events: AgentEvent[] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const views = await viewHistory(chat.harness, chat.root.id);
chat.faux.summaries.push(failure("overloaded"));
const id = await chat.root.compact(undefined, context);
await waitFor(async () => (await live(chat)).compactions?.[0]?.retry !== undefined);
const late = await watchEvents(chat.harness, chat.root.id, context);
expect(late.snapshot.compactions).toEqual([
const late = await chat.root.watch(context);
expect((late.value.docs["pi.live"] as LiveState).compactions).toEqual([
{
taskId: id,
reason: "manual",
@@ -1590,12 +1586,12 @@ describe("compaction events and live status", () => {
]);
await late.stop();
await chat.harness.abortTask(id, context);
await waitFor(() => events.some((event) => event.type === "compaction_end"));
expect(events.filter((event) => event.type.startsWith("compaction_"))).toEqual([
{ type: "compaction_start", taskId: id, reason: "manual", blocking: false },
{ type: "compaction_end", taskId: id, reason: "manual" },
]);
await stream.stop();
await waitFor(async () => (await live(chat)).compactions === undefined);
views.detach();
const listed = views.live().flatMap((state) => state.compactions ?? []);
expect(listed[0]).toEqual({ taskId: id, reason: "manual", blocking: false, attempt: 1 });
// An abort is not a failure.
expect(views.live().at(-1)!.compactionFailure).toBeUndefined();
await chat.harness.close(context);
});
@@ -1893,15 +1889,21 @@ describe("compaction and the inbox", () => {
await chat.harness.close(context);
});
it("keeps the kept entries mounted in the conversation view", async () => {
it("appends the summary to the conversation view in place, its head on a kept entry", async () => {
const chat = await open();
await history(chat);
const watch = await chat.root.watch(context);
const before = watch.value.entries;
watch.start(async () => {});
chat.faux.summaries.push(summary());
await result(chat, await chat.root.compact(undefined, context));
await waitFor(() => watch.value.entries[0]?.kind === "pi.compaction");
expect(watch.value.entries).toEqual((await chat.root.context(context)).entries);
await waitFor(() => watch.value.entries.at(-1)?.kind === "pi.compaction");
// Nothing leaves the window: the summary is appended where it was written.
expect(watch.value.entries.slice(0, before.length)).toEqual(before);
expect(watch.value.entries).toEqual(await allEntries(chat.root));
const head = (await chat.root.context(context)).head!;
expect(watch.value.entries.at(-1)).toEqual(head);
expect(watch.value.entries.some((entry) => entry.id === head.head)).toBe(true);
await watch.stop();
await chat.harness.close(context);
});
@@ -2028,41 +2030,36 @@ describe("compaction edge cases", () => {
await chat.harness.close(context);
});
it("orders the events of a summary placed at once", async () => {
it("places a summary at once in the commit that ends its compaction", async () => {
const chat = await open();
await history(chat);
const events: AgentEvent[] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const views = await viewHistory(chat.harness, chat.root.id);
chat.faux.summaries.push(summary());
await result(chat, await chat.root.compact(undefined, context));
await waitFor(() => events.some((event) => event.type === "compaction_end"));
const end = events.findIndex((event) => event.type === "compaction_end");
const batch = events.slice(end - 2, end + 3).map((event) => event.type);
expect(batch).toEqual(["message_start", "message_end", "compaction_end", "submission", "usage_changed"]);
await stream.stop();
views.detach();
const states = views.live();
const listed = states.findIndex((state) => state.compactions !== undefined);
const ended = states.findIndex((state, index) => index > listed && state.compactions === undefined);
const before = views.views[ended - 1]!;
const after = views.views[ended]!;
expect(after.entries.slice(before.entries.length).map((entry) => entry.kind)).toEqual(["pi.compaction"]);
await chat.harness.close(context);
});
it("puts compaction_start last in the batch of the preparation commit", async () => {
it("lists a background compaction in the preparation commit, with the entries it appends", async () => {
const chat = await open({ contextWindow: 2000 });
await history(chat);
chat.setup.settings.compaction = BACKGROUND;
const events: AgentEvent[][] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push([...batch]);
});
const views = await viewHistory(chat.harness, chat.root.id);
// A new section makes the preparation commit append a system entry next to the compaction's status.
addSection(chat.setup.registry, "extra", () => "extra");
chat.faux.summaries.push(gated(deferred(), summary()));
await turn(chat, text("u4", 100), "a4");
await waitFor(() => events.some((batch) => batch.some((event) => event.type === "compaction_start")));
const batch = events.find((candidate) => candidate.some((event) => event.type === "compaction_start"))!;
expect(batch.map((event) => event.type)).toEqual(["message_start", "message_end", "compaction_start"]);
await stream.stop();
await waitFor(async () => (await live(chat)).compactions !== undefined);
views.detach();
const started = views.live().findIndex((state) => state.compactions !== undefined);
const appended = views.views[started]!.entries.slice(views.views[started - 1]!.entries.length);
expect(appended.map((entry) => entry.kind)).toEqual(["pi.system"]);
await chat.harness.abortTask((await compactionTasks(chat))[0]!.id, context);
await chat.harness.close(context);
});
@@ -2283,9 +2280,11 @@ describe("compaction pinning, silent overflow, and late policy changes", () => {
chat.faux.summaries.push(gated(deferred(), summary(), reached));
const id = await chat.root.compact(undefined, context);
await reached.promise;
const stream = await watchEvents(chat.harness, chat.root.id, context);
expect(stream.snapshot.compactions).toEqual([{ taskId: id, reason: "manual", blocking: false, attempt: 1 }]);
await stream.stop();
const watch = await chat.root.watch(context);
expect((watch.value.docs["pi.live"] as LiveState).compactions).toEqual([
{ taskId: id, reason: "manual", blocking: false, attempt: 1 },
]);
await watch.stop();
await chat.harness.abortTask(id, context);
await chat.harness.close(context);
});
@@ -2356,11 +2355,6 @@ describe("blocked compaction", () => {
({ harness, root } = await openChat(await openNodeSqliteStorage(path), setup));
harness.resume();
const chat = { harness, root, setup, faux: script(setup) };
const events: AgentEvent[] = [];
const stream = await watchEvents(chat.harness, chat.root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const inspection = await chat.harness.inspect(context);
expect(inspection.tasks.find((task) => task.record.id === id)?.state).toEqual({
kind: "blocked",
@@ -2372,8 +2366,7 @@ describe("blocked compaction", () => {
reason: "task_too_old",
});
expect((await live(chat)).compactions).toBeUndefined();
await waitFor(() => events.some((event) => event.type === "compaction_end"));
await stream.stop();
expect((await live(chat)).compactionFailure).toEqual({ taskId: id, reason: "manual", error: "task_too_old" });
await chat.harness.close(context);
});
});
packages/durable/test/harness-events.test.ts
-779
View File
@@ -1,779 +0,0 @@
import { GCProfiler, getHeapStatistics, setFlagsFromString } from "node:v8";
import { runInNewContext } from "node:vm";
import type { Draft } from "@earendil-works/chord";
import {
type AssistantMessage,
type FauxResponseStep,
fauxAssistantMessage,
fauxText,
fauxThinking,
fauxToolCall,
Type,
} from "@earendil-works/pi-ai";
import {
AgentDoc,
type AgentEvent,
type AgentEventStream,
type Conversation,
defineTool,
type Harness,
LiveDoc,
type LiveState,
MemoryStorage,
type MessageChange,
type SnapshotEvent,
UsageDoc,
watchEvents,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { type ChatSetup, chatSetup, openChat, textOf, waitFor } from "./chat-support.ts";
import { addTool } from "./harness-support.ts";
import { context, documentChanges, observeCommits } from "./session-support.ts";
import { aborted, deferred } from "./task-support.ts";
/** Attach and start an event stream that records every delivered batch. */
async function listen(harness: Harness, conversation: Conversation) {
const stream = await watchEvents(harness, conversation.id, context);
const batches: AgentEvent[][] = [];
stream.start(async ({ value: events }) => {
batches.push([...events]);
});
return { stream, batches, events: () => batches.flat() };
}
async function drained(): Promise<void> {
await new Promise((resolve) => setTimeout(resolve, 0));
}
function slow(): ChatSetup {
return chatSetup({ tokensPerSecond: 400, tokenSize: { min: 1, max: 1 } });
}
/** Rebuild the streamed text of block 0 from `message_start` and the text deltas that follow it. */
function streamedText(events: readonly AgentEvent[]): string {
let text = "";
for (const event of events) {
if (event.type === "message_start" && event.message.role === "assistant") {
const block = event.message.content[0];
text = block?.type === "text" ? block.text : "";
}
if (event.type !== "message_update") continue;
for (const change of event.changes) {
if (change.type === "text_start" && change.contentIndex === 0 && change.block.type === "text") {
text = change.block.text;
}
if (change.type === "text_delta" && change.contentIndex === 0) text += change.delta;
}
}
return text;
}
type Mutable = { [key: string]: unknown };
/** Apply message changes to a copy of `message`, as an events-only consumer would. */
function applyChanges(message: AssistantMessage, changes: readonly MessageChange[]): AssistantMessage {
let next = structuredClone(message);
for (const change of changes) {
if (change.type === "message") next = structuredClone(change.message);
else if (change.type === "block") next.content[change.contentIndex] = structuredClone(change.block);
else if (change.type.endsWith("_start") && "block" in change) {
next.content.splice(change.contentIndex, 0, structuredClone(change.block));
} else if (change.type === "text_delta" || change.type === "thinking_delta") {
const block = next.content[change.contentIndex] as unknown as Mutable;
const field = change.type === "text_delta" ? "text" : "thinking";
block[field] = `${block[field] as string}${change.delta}`;
} else if (change.type === "toolcall_delta") {
let target = (next.content[change.contentIndex] as unknown as Mutable).arguments as Mutable;
for (const segment of change.path.slice(0, -1)) target = target[segment] as Mutable;
const last = change.path.at(-1)!;
target[last] = `${target[last] as string}${change.delta}`;
}
}
return next;
}
/** Committed partials of `conversation`'s generation, one per commit that has one. */
function partialsOf(harness: Harness): AssistantMessage[] {
const partials: AssistantMessage[] = [];
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
const message = (change.value as LiveState | null)?.generation?.message as AssistantMessage | undefined;
if (change.record.kind === "pi.live" && message !== undefined) partials.push(message);
}
});
return partials;
}
describe("agent events", () => {
it("streams a run as lifecycle events and text deltas that rebuild the answer", async () => {
const setup = slow();
const text = "streamed answer text ".repeat(20);
setup.faux.setResponses([fauxAssistantMessage([fauxText(text)])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
expect(stream.snapshot).toMatchObject({ type: "snapshot", entries: [], tools: [], inbox: [] });
const partials: string[] = [];
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
const message = (change.value as LiveState | null)?.generation?.message as AssistantMessage | undefined;
if (change.record.kind === "pi.live" && message !== undefined) partials.push(textOf(message) ?? "");
}
});
const submission = await root.submit({ type: "input", content: "hi" }, context);
await submission.wait(context);
await drained();
const all = events();
const types = all.map((event) => event.type).filter((type, index, list) => type !== list[index - 1]);
expect(types).toEqual([
"message_start",
"message_end",
"submission",
"run_start",
"turn_start",
"message_start",
"message_update",
"message_end",
"turn_end",
"run_end",
"submission",
"usage_changed",
]);
expect(all.find((event) => event.type === "run_start")).toEqual({ type: "run_start", inputs: [submission.id] });
// After each event, the rebuilt text equals the committed partial of that commit.
const rebuilt = all.flatMap((event, index) =>
(event.type === "message_start" && event.message.role === "assistant") || event.type === "message_update"
? [streamedText(all.slice(0, index + 1))]
: [],
);
expect(rebuilt).toEqual(partials);
expect(partials.length).toBeGreaterThan(1);
const end = all.filter((event) => event.type === "message_end").at(-1)!;
expect(end.type === "message_end" && textOf(end.entry!.model![0])).toBe(text);
await stream.stop();
await harness.close(context);
});
it("reports tool start, output appends, and the result entry", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "print",
description: "Prints",
parameters: Type.Object({ n: Type.Number() }),
execute: async (_args, api) => {
api.output("one\n");
await new Promise((resolve) => setTimeout(resolve, 150));
api.output("two\n");
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("print", { n: 1 }, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const submission = await root.submit({ type: "input", content: "go" }, context);
// The output events rebuild the slot's retained window.
const output = (): string => {
let text = "";
for (const event of events()) {
if (event.type !== "tool_execution_update" || event.output === undefined) continue;
if ("set" in event.output) text = event.output.set;
else text = text.slice(event.output.trimStart ?? 0) + (event.output.append ?? "");
}
return text;
};
await waitFor(() => output() === "one\ntwo\n");
gate.resolve();
await submission.wait(context);
await drained();
const tool = events().filter((event) => event.type.startsWith("tool_execution"));
expect(tool[0]).toEqual({
type: "tool_execution_start",
toolCallId: "c1",
toolName: "print",
taskId: expect.any(Number),
args: { n: 1 },
});
const end = tool.at(-1)!;
expect(end).toMatchObject({ type: "tool_execution_end", toolCallId: "c1", entry: { kind: "pi.tool-result" } });
// As in the coding agent, the tool ends directly before its result message.
const all = events();
const endIndex = all.indexOf(end);
expect(all.slice(endIndex, endIndex + 3).map((event) => event.type)).toEqual([
"tool_execution_end",
"message_start",
"message_end",
]);
expect(all[endIndex + 1]).toMatchObject({ message: { role: "toolResult", toolCallId: "c1" } });
// Two turns: the tool round, and the answer.
expect(events().filter((event) => event.type === "turn_start")).toHaveLength(2);
expect(events().filter((event) => event.type === "turn_end")).toHaveLength(2);
await stream.stop();
await harness.close(context);
});
it("reports queued submissions, inbox changes, and retries", async () => {
const setup = chatSetup();
const release = deferred();
const held: FauxResponseStep = async (_request, options) => {
await Promise.race([release.promise, aborted(options!.signal!)]);
return fauxAssistantMessage([fauxText("first")]);
};
const error = fauxAssistantMessage([], { stopReason: "error", errorMessage: "503 Service Unavailable" });
setup.faux.setResponses([held, error, fauxAssistantMessage([fauxText("second")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
setup.settings.retry = { enabled: true, maxRetries: 1, baseDelayMs: 1 };
const { stream, events } = await listen(harness, root);
await root.submit({ type: "input", content: "a" }, context);
const followUp = await root.submit({ type: "input", content: "f" }, context);
await drained();
expect(events().filter((event) => event.type === "inbox_update")).toEqual([
{ type: "inbox_update", items: [{ id: followUp.id, mode: "followUp" }] },
]);
release.resolve();
await followUp.wait(context);
await drained();
const types = events().map((event) => event.type);
expect(types).toContain("auto_retry_start");
expect(types).toContain("auto_retry_end");
expect(types.filter((type) => type === "run_start")).toHaveLength(2);
expect(
events()
.filter((event) => event.type === "submission")
.map((event) => event.record.status),
).toEqual(["placed", "queued", "done", "placed", "done"]);
await stream.stop();
await harness.close(context);
});
it("delivers each of 100 pending batches in its own call, in order, unlike state watches", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const stream = await watchEvents(harness, root.id, context);
for (let index = 0; index < 100; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note", data: { index } }), context);
}
const batches: AgentEvent[][] = [];
stream.start(async ({ value: events }) => {
batches.push([...events]);
});
await drained();
expect(batches).toHaveLength(100);
expect(
batches.map((events) => events.map((event) => (event.type === "entry_appended" ? event.entry.data : event))),
).toEqual(Array.from({ length: 100 }, (_, index) => [{ index }]));
await stream.stop();
await harness.close(context);
});
it("replaces undelivered batches with one snapshot after 100 pending batches", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const stream = await watchEvents(harness, root.id, context);
for (let index = 0; index < 101; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
}
const batches: AgentEvent[][] = [];
stream.start(async ({ value: events }) => {
batches.push([...events]);
});
await drained();
expect(batches).toHaveLength(1);
expect(batches[0]).toHaveLength(1);
expect(batches[0]![0]).toMatchObject({ type: "snapshot" });
expect((batches[0]![0] as Extract<AgentEvent, { type: "snapshot" }>).entries).toHaveLength(101);
await stream.stop();
await harness.close(context);
});
it("rebuilds every committed partial of thinking, text, and tool-call arguments from message changes", async () => {
const setup = chatSetup({ tokensPerSecond: 150, tokenSize: { min: 1, max: 1 } });
const message = fauxAssistantMessage(
[
fauxThinking("thinking about it ".repeat(10)),
fauxText("some text ".repeat(10)),
fauxToolCall("missing", { path: "a/long/path/".repeat(10), note: "x".repeat(60) }, { id: "c1" }),
],
{ stopReason: "toolUse" },
);
setup.faux.setResponses([message, fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const partials = partialsOf(harness);
const { stream, events } = await listen(harness, root);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await drained();
const rebuilt: AssistantMessage[] = [];
let current: AssistantMessage | undefined;
// The streamed tool-calling message, up to its end; the short final answer commits no partial.
for (const event of events()) {
if (event.type === "message_end" && event.entry?.kind === "pi.assistant") break;
if (event.type === "message_start" && event.message.role === "assistant") current = event.message;
else if (event.type === "message_update") current = applyChanges(current!, event.changes);
else continue;
rebuilt.push(current!);
}
expect(partials.length).toBeGreaterThan(2);
expect(rebuilt.map((partial) => partial.content)).toEqual(partials.map((partial) => partial.content));
const types = new Set(
events()
.flatMap((event) => (event.type === "message_update" ? event.changes : []))
.map((c) => c.type),
);
expect(types.has("thinking_delta") || types.has("text_delta")).toBe(true);
await stream.stop();
await harness.close(context);
});
it("rebuilds a sliding tail window from output trims and appends", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "tail",
description: "Prints lines",
parameters: Type.Object({}),
outputLimits: { maxLines: 3, retain: "tail" },
execute: async (_args, api) => {
for (let line = 0; line < 6; line++) {
api.output(`line ${line}\n`);
await new Promise((resolve) => setTimeout(resolve, 120));
}
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("tail", {}, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const outputs: string[] = [];
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
const output = (change.value as LiveState | null)?.tools?.[0]?.output;
if (change.record.kind === "pi.live" && output !== undefined && output !== outputs.at(-1)) {
outputs.push(output);
}
}
});
const { stream, events } = await listen(harness, root);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => outputs.at(-1)?.endsWith("line 5\n") === true);
gate.resolve();
await submission.wait(context);
await drained();
const rebuilt: string[] = [];
let text = "";
for (const event of events()) {
if (event.type !== "tool_execution_update" || event.output === undefined) continue;
if ("set" in event.output) text = event.output.set;
else text = text.slice(event.output.trimStart ?? 0) + (event.output.append ?? "");
rebuilt.push(text);
}
expect(rebuilt).toEqual(outputs);
expect(
events().some((event) => event.type === "tool_execution_update" && "trimStart" in (event.output ?? {})),
).toBe(true);
await stream.stop();
await harness.close(context);
});
it("emits one exact batch when a run ends and a queued follow-up starts the next", async () => {
const setup = chatSetup();
const release = deferred();
const held: FauxResponseStep = async (_request, options) => {
await Promise.race([release.promise, aborted(options!.signal!)]);
return fauxAssistantMessage([fauxText("first")]);
};
setup.faux.setResponses([held, fauxAssistantMessage([fauxText("second")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const input = await root.submit({ type: "input", content: "a" }, context);
const followUp = await root.submit({ type: "input", content: "f" }, context);
const { stream, batches } = await listen(harness, root);
release.resolve();
await followUp.wait(context);
await drained();
const boundary = batches.find((batch) => batch.some((event) => event.type === "run_end"))!;
expect(boundary.map((event) => event.type)).toEqual([
"message_start",
"message_end",
"message_start",
"message_end",
"turn_end",
"run_end",
"submission",
"submission",
"inbox_update",
"usage_changed",
"run_start",
"turn_start",
]);
expect(boundary.filter((event) => event.type === "submission").map((event) => event.record.id)).toEqual([
input.id,
followUp.id,
]);
await stream.stop();
await harness.close(context);
});
it("ends a call that never runs and a tool aborted with its generation", async () => {
const setup = chatSetup();
addTool(
setup.registry,
defineTool({
name: "wait",
description: "Waits until aborted",
parameters: Type.Object({}),
execute: (_args, _api, callContext) => aborted(callContext.abortSignal!),
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("ghost", {}, { id: "c1" }), fauxToolCall("wait", {}, { id: "c2" })], {
stopReason: "toolUse",
}),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => events().some((event) => event.type === "tool_execution_start"));
// Aborting the generation aborts its round: the tool ends with its aborted result first (spec §8.5).
await harness.abortTask((await harness.snapshot(LiveDoc, root.id, context))!.run!.taskId, context);
await submission.wait(context);
await drained();
// The call not offered ends after its calling message and directly before its result message.
const round = events().map((event) =>
event.type === "message_end"
? `end:${event.entry.kind}`
: event.type === "message_start"
? `start:${event.message.role}`
: event.type,
);
const ghostEnd = round.indexOf("tool_execution_end");
expect(round.slice(ghostEnd - 1, ghostEnd + 3)).toEqual([
"end:pi.assistant",
"tool_execution_end",
"start:toolResult",
"end:pi.tool-result",
]);
const tool = events().filter((event) => event.type.startsWith("tool_execution"));
expect(tool.map((event) => [event.type, "toolCallId" in event && event.toolCallId, "entry" in event])).toEqual([
["tool_execution_end", "c1", true],
["tool_execution_start", "c2", false],
["tool_execution_end", "c2", true],
]);
await stream.stop();
await harness.close(context);
});
it("reports a steer without run events, a reset as an appended entry, and nothing for other conversations", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "hold",
description: "Waits",
parameters: Type.Object({}),
execute: async () => {
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("hold", {}, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const other = await harness.createConversation({ ownership: { kind: "ownerless" } }, context);
const { stream, events, batches } = await listen(harness, root);
const input = await root.submit({ type: "input", content: "a" }, context);
await waitFor(() => events().some((event) => event.type === "tool_execution_start"));
await root.submit({ type: "input", content: "s", whenBusy: "steer" }, context);
const before = batches.length;
await other.commit((tx) => tx.appendEntry(other.id, { kind: "note" }), context);
await drained();
expect(batches.length).toBe(before);
gate.resolve();
await input.wait(context);
await root.reset(undefined, context);
await drained();
expect(events().filter((event) => event.type === "run_start")).toHaveLength(1);
expect(events().filter((event) => event.type === "run_end")).toHaveLength(1);
expect(batches.at(-1)!.map((event) => event.type)).toEqual(["entry_appended", "submission"]);
await stream.stop();
await harness.close(context);
});
it("rejects an attachment cancelled while it waits for the Session line", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const release = deferred();
const blocking = root.commit(async () => {
await release.promise;
}, context);
const controller = new AbortController();
const attaching = watchEvents(harness, root.id, { ...context, abortSignal: controller.signal });
controller.abort(new Error("cancelled"));
release.resolve();
await blocking;
await expect(attaching).rejects.toThrow("cancelled");
await harness.close(context);
});
it("applies deltas after an overflow snapshot that holds an in-flight partial", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const partial = fauxAssistantMessage([fauxText("hel")]);
const stream = await watchEvents(harness, root.id, context);
await root.commit(async (tx) => {
(await tx.doc(LiveDoc, root.id)).generation = {
attempt: 1,
message: JSON.parse(JSON.stringify(partial)),
};
}, context);
for (let index = 0; index < 101; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
}
const batches: AgentEvent[][] = [];
stream.start(async ({ value: events }) => {
batches.push([...events]);
});
await drained();
const snapshot = batches[0]![0] as SnapshotEvent;
expect(snapshot.type).toBe("snapshot");
await root.commit(async (tx) => {
const block = (await tx.doc(LiveDoc, root.id)).generation!.message!.content[0] as { text: string };
block.text += "lo";
}, context);
await drained();
const update = batches.at(-1)![0]!;
if (update.type !== "message_update") throw new Error(`Unexpected ${update.type}`);
expect(update.changes).toEqual([{ type: "text_delta", contentIndex: 0, delta: "lo" }]);
const rebuilt = applyChanges(snapshot.generation!.message!, update.changes);
expect(rebuilt.content).toEqual([{ type: "text", text: "hello" }]);
await stream.stop();
await harness.close(context);
});
it("reports usage-only updates, cleared tool progress, tools ending without entries, and deferred polls", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const { stream, batches } = await listen(harness, root);
const change = (edit: (live: Draft<LiveState>) => void) =>
root.commit(async (tx) => edit(await tx.doc(LiveDoc, root.id)), context);
const partial = JSON.parse(JSON.stringify(fauxAssistantMessage([fauxText("partial")])));
await change((live) => {
live.generation = { attempt: 1, message: partial };
});
// A usage-only change sends the usage and no changes.
await change((live) => {
live.generation!.message!.usage.input = 42;
});
await change((live) => {
live.generation = { attempt: 1, deferred: { pollAt: 1 } };
});
await change((live) => {
live.generation!.deferred!.pollAt = 2;
});
await change((live) => {
live.tools = [{ callId: "c1", name: "t", status: "running", details: { n: 1 }, diagnostics: [] }];
});
// A safe replay clears the running slot's progress.
await change((live) => {
delete live.tools![0]!.details;
delete live.tools![0]!.diagnostics;
});
// A fault marks the slot done without an entry.
await change((live) => {
live.tools![0]!.status = "done";
});
await root.commit((tx) => tx.retireDoc(UsageDoc, root.id), context);
await root.commit((tx) => tx.retireDoc(AgentDoc, root.id), context);
await drained();
expect(batches.map((batch) => batch.filter((event) => event.type !== "task_failed"))).toEqual([
[{ type: "message_start", message: partial }],
[{ type: "message_update", usage: { ...partial.usage, input: 42 }, changes: [] }],
[{ type: "deferred_poll", pollAt: 1 }],
[{ type: "deferred_poll", pollAt: 2 }],
[{ type: "tool_execution_start", toolCallId: "c1", toolName: "t", args: {} }],
[{ type: "tool_execution_update", toolCallId: "c1", toolName: "t", details: null, diagnostics: [] }],
[{ type: "tool_execution_end", toolCallId: "c1", toolName: "t" }],
// Retired documents read as their initial values.
[{ type: "usage_changed", usage: { models: {}, tools: {} } }],
[{ type: "agent_changed", agent: {} }],
]);
await stream.stop();
await harness.close(context);
});
it("ends the stream with the Harness", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const stream: AgentEventStream = await watchEvents(harness, root.id, context);
await harness.close(context);
expect(await stream.closed).toEqual({ reason: "ended" });
});
it("starts a message at the first committed partial and ends it with the converted entry on abort", async () => {
const setup = chatSetup({ tokensPerSecond: 100, tokenSize: { min: 1, max: 1 } });
setup.faux.setResponses([fauxAssistantMessage([fauxText("x".repeat(400))])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const submission = await root.submit({ type: "input", content: "hi" }, context);
await waitFor(() =>
events().some((event) => event.type === "message_start" && event.message.role === "assistant"),
);
const live = await harness.snapshot(LiveDoc, root.id, context);
await harness.abortTask(live!.run!.taskId, context);
await submission.wait(context);
await drained();
const assistantEnds = events().filter(
(event) => event.type === "message_end" && event.entry?.kind === "pi.assistant",
);
expect(assistantEnds).toHaveLength(1);
expect(
events().filter((event) => event.type === "message_start" && event.message.role === "assistant"),
).toHaveLength(1);
await stream.stop();
await harness.close(context);
});
});
const gc = (() => {
setFlagsFromString("--expose-gc");
return runInNewContext("gc") as () => void;
})();
/** Bytes `run` allocates, counting what collections during it freed. */
async function allocatedBytes(run: () => Promise<void>): Promise<number> {
gc();
const profiler = new GCProfiler();
profiler.start();
const before = getHeapStatistics().used_heap_size;
await run();
const after = getHeapStatistics().used_heap_size;
let freed = 0;
for (const collection of profiler.stop().statistics) {
freed += collection.beforeGC.heapStatistics.usedHeapSize - collection.afterGC.heapStatistics.usedHeapSize;
}
return after - before + freed;
}
/** Heap bytes in use after a full collection. */
async function heapInUse(): Promise<number> {
for (let round = 0; round < 3; round++) {
await drained();
gc();
}
return getHeapStatistics().used_heap_size;
}
// #9265
describe("streamed partials", () => {
it("streams a tool-call partial as toolcall_delta appends at its argument paths", async () => {
const setup = chatSetup({ tokensPerSecond: 500, tokenSize: { min: 1, max: 1 } });
setup.settings.progress = { partialIntervalMs: 20, outputIntervalMs: 100 };
const args = { path: "src/file.ts", content: "const a = 1;\n".repeat(150) };
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("missing", args, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const partials = partialsOf(harness);
const { stream, events } = await listen(harness, root);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await drained();
const changes = events().flatMap((event) => (event.type === "message_update" ? event.changes : []));
const deltas = changes.filter((change) => change.type === "toolcall_delta");
expect(deltas.length).toBeGreaterThan(10);
expect(deltas.every((change) => ["path", "content"].includes(String(change.path)))).toBe(true);
// The call is resent whole only when a member appears (at most once per argument); growing strings are appends.
// With the raw argument text on the block, every commit resent the whole call.
expect(changes.filter((change) => change.type === "message")).toEqual([]);
expect(changes.filter((change) => change.type === "block").length).toBeLessThanOrEqual(2);
// The committed partials carry only the parsed arguments, never the raw argument text.
for (const partial of partials) {
for (const block of partial.content) {
expect(Object.keys(block).sort()).toEqual(["arguments", "id", "name", "type"]);
}
}
await stream.stop();
await harness.close(context);
});
it("commits a growing text and tool call at a cost that follows the new text, not the string lengths", async () => {
/** Bytes allocated while streaming one answer of `size` characters of text and as much tool-call content. */
const stream = async (size: number) => {
const setup = chatSetup({ tokensPerSecond: 64_000, tokenSize: { min: 64, max: 64 } });
setup.settings.progress = { partialIntervalMs: 1, outputIntervalMs: 100 };
// Keep `pi.live` cached between commits, as the default retention does. Evicted after each use (the suite's
// evicting run), every commit replays the document's delta chain: a cost of eviction, not of appending.
setup.settings.documentRetentionMs = 60_000;
const text = 'streamed text with "quotes" and \\ escapes\n'.repeat(size / 40);
setup.faux.setResponses([
fauxAssistantMessage(
[fauxText(text), fauxToolCall("missing", { path: "a.ts", content: text }, { id: "c1" })],
{
stopReason: "toolUse",
},
),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
let commits = 0;
observeCommits(harness, () => commits++);
const bytes = await allocatedBytes(async () => {
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
});
await harness.close(context);
return { bytes, commits };
};
const small = await stream(40_000);
const large = await stream(400_000);
expect(large.commits).toBeGreaterThan(small.commits * 5);
// Per commit, the cost stays flat: comparing the whole strings on every commit made it grow with their length.
expect(large.bytes / large.commits / (small.bytes / small.commits)).toBeLessThan(2);
}, 60_000);
it("keeps a watcher's memory proportional to a long streamed text and tool call, not to their commits", async () => {
const setup = chatSetup({ tokensPerSecond: 16_000, tokenSize: { min: 16, max: 16 } });
setup.settings.progress = { partialIntervalMs: 1, outputIntervalMs: 100 };
const line = (index: number) => `line ${index}: some streamed source text with "quotes" and \\ escapes\n`;
const text = Array.from({ length: 2_000 }, (_, index) => line(index)).join("");
const content = Array.from({ length: 2_000 }, (_, index) => line(index)).join("");
setup.faux.setResponses([
fauxAssistantMessage([fauxText(text), fauxToolCall("missing", { path: "a.ts", content }, { id: "c1" })], {
stopReason: "toolUse",
}),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, batches } = await listen(harness, root);
const before = await heapInUse();
const submission = await root.submit({ type: "input", content: "go" }, context);
const partialContent = async () =>
(await harness.snapshot(LiveDoc, root.id, context))?.generation?.message?.content;
const shown = async (index: number): Promise<number> => {
const block = (await partialContent())?.[index];
if (block?.type === "text") return block.text.length;
return block?.type === "toolCall" ? String(block.arguments.content ?? "").length : 0;
};
const retained: number[] = [];
// Measure while the text, then the tool call, is almost complete: the watcher's view holds the most then.
await waitFor(async () => (await shown(0)) > text.length * 0.95, 30_000);
retained.push((await heapInUse()) - before);
await waitFor(async () => (await shown(1)) > content.length * 0.95, 30_000);
retained.push((await heapInUse()) - before);
await submission.wait(context);
const commits = batches.length;
expect(commits).toBeGreaterThan(500);
// The two 140 KB strings, the watcher's recorded batches and the rest of the stream: about 3 MB. Sliced append ops
// made the watcher's view hold a copy of the text per commit: well over 100 MB.
for (const bytes of retained) expect(bytes).toBeLessThan(20_000_000);
await stream.stop();
await harness.close(context);
}, 60_000);
});
packages/durable/test/harness-generation.test.ts
+28 -8
View File
@@ -155,7 +155,8 @@ describe("generation", () => {
expect((await harness.waitForTask(taskId, context)).state.outcome).toEqual({ status: "aborted" });
// The earlier settlement stays; the run's own settlement leaves it unchanged.
expect(await submission.status(context)).toMatchObject({ status: "unanswered", reason: "withdrawn" });
expect(await live(harness, root)).toEqual({});
// A run ends without an answer even when its inputs were settled already.
expect(await live(harness, root)).toEqual({ runFailure: { taskId, inputs: [submission.id], reason: "aborted" } });
await harness.close(context);
});
@@ -179,8 +180,18 @@ describe("generation", () => {
error: { message: "Model faux/missing is not available", detail: { reason: "no_model" } },
},
});
expect(await live(harness, root)).toEqual({});
expect(await live(harness, plain)).toEqual({});
expect(await live(harness, root)).toEqual({
runFailure: {
taskId: tasks.items[0]!.id,
inputs: [expect.any(Number)],
reason: "no_model",
detail: "Model faux/missing is not available",
},
});
expect((await live(harness, plain))?.runFailure).toMatchObject({
reason: "no_model",
detail: "No model is configured",
});
await harness.close(context);
});
@@ -304,7 +315,7 @@ describe("generation", () => {
const converted = entries[1]!.model![0] as AssistantMessage;
expect(converted.stopReason).toBe("aborted");
expect(textOf(converted)!.startsWith(textOf(partial as Message)!)).toBe(true);
expect(await live(harness, root)).toEqual({});
expect(await live(harness, root)).toEqual({ runFailure: { taskId, inputs: [submission.id], reason: "aborted" } });
expect((await harness.waitForTask(taskId, context)).state.outcome).toEqual({ status: "aborted" });
await harness.close(context);
});
@@ -321,7 +332,7 @@ describe("generation", () => {
await harness.abortTask(taskId, context);
expect(await submission.wait(context)).toMatchObject({ status: "unanswered", reason: "aborted" });
expect(setup.faux.state.cancelledDeferred).toHaveLength(1);
expect(await live(harness, root)).toEqual({});
expect(await live(harness, root)).toEqual({ runFailure: { taskId, inputs: [submission.id], reason: "aborted" } });
await harness.close(context);
});
@@ -345,7 +356,7 @@ describe("generation", () => {
await harness.abortTask(taskId, context);
expect(await submission.wait(context)).toMatchObject({ status: "unanswered", reason: "aborted" });
expect(setup.reports).toContainEqual(new Error("cancel failed"));
expect(await live(harness, root)).toEqual({});
expect(await live(harness, root)).toEqual({ runFailure: { taskId, inputs: [submission.id], reason: "aborted" } });
await harness.close(context);
});
@@ -674,7 +685,14 @@ describe("generation", () => {
expect(entries.map((entry) => entry.kind)).toEqual(["pi.user", "pi.assistant"]);
expect(entries[1]!.model![0]).toMatchObject({ role: "assistant", stopReason: "aborted" });
expect(textOf(entries[1]!.model![0])).toBe("partial");
expect(await live(harness, root)).toEqual({});
expect(await live(harness, root)).toEqual({
runFailure: {
taskId: expect.any(Number),
inputs: [submission.id],
reason: "faulted",
detail: expect.stringContaining("non-JSON function"),
},
});
const tasks = await harness.commit((tx) => tx.scanTasks({ conversationId: root.id }, 10), context);
expect(tasks.items[0]!.state).toMatchObject({ status: "terminal", outcome: { status: "faulted" } });
await harness.close(context);
@@ -715,7 +733,9 @@ describe("generation", () => {
status: "unanswered",
reason: "task_too_old",
});
expect(await live(harness, root)).toEqual({});
expect(await live(harness, root)).toEqual({
runFailure: { taskId, inputs: [submissionId], reason: "orphaned", detail: "task_too_old" },
});
await harness.close(context);
});
packages/durable/test/harness-input-entries.test.ts
+18 -14
View File
@@ -12,7 +12,6 @@ import {
Type,
} from "@earendil-works/pi-ai";
import {
type AgentEvent,
type Conversation,
ConversationBusy,
defineTool,
@@ -20,14 +19,15 @@ import {
type Harness,
InboxDoc,
LiveDoc,
type LiveState,
MemoryStorage,
type SubmissionDraft,
watchEvents,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, type ChatSetup, chatSetup, openChat, textOf, waitFor } from "./chat-support.ts";
import { addSection, addTool } from "./harness-support.ts";
import { viewHistory } from "./live-support.ts";
import { context } from "./session-support.ts";
import { aborted, type Deferred, deferred } from "./task-support.ts";
@@ -263,11 +263,7 @@ describe("data-only writes", () => {
const requests: string[][] = [];
setup.faux.setResponses([requestLog(requests, HOLD), requestLog(requests, answer("done"))]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const events: AgentEvent[] = [];
const stream = await watchEvents(harness, root.id, context);
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const history = await viewHistory(harness, root.id);
const input = await root.submit({ type: "input", content: "a" }, context);
await toolRunning(harness, root);
const chat = { kind: "chat.message", data: { author: "bob", text: "lunch?" } };
@@ -281,12 +277,13 @@ describe("data-only writes", () => {
["pi.assistant"],
["chat.message", { author: "bob", text: "lunch?" }],
]);
await waitFor(() => events.some((event) => event.type === "entry_appended"));
const appended = events.find((event) => event.type === "entry_appended");
expect(appended).toMatchObject({
entry: { kind: "chat.message", id: settled.status === "done" && settled.entry },
// The view shows it while the run goes on.
const shown = history.views.find((view) => view.entries.some((entry) => entry.kind === "chat.message"));
expect(shown?.entries.at(-1)).toMatchObject({
kind: "chat.message",
id: settled.status === "done" && settled.entry,
});
expect(events.some((event) => event.type === "run_end")).toBe(false);
expect((shown?.docs["pi.live"] as LiveState).run?.inputs).toEqual([input.id]);
// A repeated request ID returns the same write without a second entry.
expect((await root.submit({ type: "write", entry: chat, requestId: "bob:1" }, context)).id).toBe(write.id);
@@ -303,7 +300,7 @@ describe("data-only writes", () => {
["system:", "user:a"],
["system:", "user:a", "assistant:", "toolResult:held"],
]);
await stream.stop();
history.detach();
await harness.close(context);
});
@@ -374,7 +371,14 @@ describe("data-only writes", () => {
await harness.waitForIdle(context);
expect((await f.status(context)).status).toBe("queued");
expect((await inboxItems(harness, root)).map((item) => item.id)).toEqual([f.id]);
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({
runFailure: {
taskId: expect.any(Number),
inputs: [input.id],
reason: "model_error",
detail: "invalid request",
},
});
expect(setup.faux.state.callCount).toBe(1);
// The next input places the waiting mention first.
const g = await root.submit({ type: "input", content: "g" }, context);
packages/durable/test/harness-nested-tools-restart.test.ts
+5 -15
View File
@@ -11,7 +11,6 @@ import {
Type,
} from "@earendil-works/pi-ai";
import {
type AgentEvent,
defineExtension,
defineTask,
defineTool,
@@ -23,12 +22,12 @@ import {
ToolResultEntry,
ToolTask,
type ToolTaskInput,
watchEvents,
} from "@earendil-works/pi-durable";
import { afterEach, describe, expect, it } from "vitest";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, type ChatSetup, chatSetup, openChat } from "./chat-support.ts";
import { addHooks, addTool } from "./harness-support.ts";
import { nestedResultOf, nestedResults, viewHistory } from "./live-support.ts";
import { context } from "./session-support.ts";
import { aborted, deferred, eventually } from "./task-support.ts";
@@ -75,16 +74,6 @@ function results(entries: Awaited<ReturnType<typeof allEntries>>): ToolResultMes
return entries.filter((entry) => ToolResultEntry.is(entry)).map((entry) => entry.model![0] as ToolResultMessage);
}
/** Collect tool end events from now on. */
async function collectEnds(harness: Harness, conversationId: Parameters<Harness["conversation"]>[0]) {
const ends: Extract<AgentEvent, { type: "tool_execution_end" }>[] = [];
const stream = await watchEvents(harness, conversationId, context);
stream.start(async ({ value: batch }) => {
for (const event of batch) if (event.type === "tool_execution_end") ends.push(event);
});
return { ends, stop: () => stream.stop() };
}
function codes(result: NestedToolExecutionResult | undefined): string[] {
return (result?.diagnostics ?? []).flatMap((diagnostic) => (diagnostic.code === undefined ? [] : [diagnostic.code]));
}
@@ -142,18 +131,19 @@ describe("nested tool calls across a restart", () => {
await opened.harness.close(context);
opened = await open(path, setup);
const { ends, stop } = await collectEnds(opened.harness, opened.root.id);
const history = await viewHistory(opened.harness, opened.root.id);
const nested = nestedResults(opened.harness);
opened.harness.resume();
expect((await (await opened.harness.submission(id, context))!.wait(context)).status).toBe("done");
await opened.harness.waitForIdle(context);
await stop();
history.detach();
expect(runs).toEqual({ started: 1, gated: 0 });
const [started, gate] = await nestedTasks(opened.harness);
for (const task of [started, gate]) {
expect(task).toMatchObject({ abandonOnRestart: true, abortRequested: true, abortReason: "restart" });
expect(task!.state).toMatchObject({ status: "terminal", outcome: { status: "aborted" } });
}
const end = (callId: string) => ends.find((event) => event.toolCallId === callId)?.result;
const end = (callId: string) => nestedResultOf(history.live(), nested, callId);
expect(codes(end("c1/1"))).toEqual(["interrupted"]);
expect(end("c1/1")).toMatchObject({ isError: true, details: { step: 1 } });
expect(end("c1/1")).not.toHaveProperty("structuredOutput");
packages/durable/test/harness-nested-tools.test.ts
+62 -102
View File
@@ -3,6 +3,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import type { JsonValue } from "@earendil-works/chord";
import { withAbortSignal } from "@earendil-works/chord/context";
import { applyImmutable } from "@earendil-works/chord/delta";
import {
type AssistantMessage,
fauxAssistantMessage,
@@ -13,12 +14,13 @@ import {
type Usage,
} from "@earendil-works/pi-ai";
import {
type AgentEvent,
AssistantEntry,
type ConversationView,
defineTool,
type EntryRecord,
type Harness,
LiveDoc,
type LiveState,
MemoryStorage,
NestedCallDoc,
type NestedToolExecutionResult,
@@ -30,12 +32,12 @@ import {
type ToolTaskInput,
type ToolTaskResult,
UsageDoc,
watchEvents,
} from "@earendil-works/pi-durable";
import { afterEach, describe, expect, it } from "vitest";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, type ChatSetup, chatSetup, openChat } from "./chat-support.ts";
import { addHooks, addTool } from "./harness-support.ts";
import { endedAt, nestedResultOf, nestedResults, slotsOf, startOrder, viewHistory } from "./live-support.ts";
import { context } from "./session-support.ts";
import { aborted, deferred, eventually } from "./task-support.ts";
@@ -372,15 +374,12 @@ describe("nested tool calls", () => {
);
setup.faux.setResponses([call("batch"), DONE]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
const ends: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
ends.push(...batch.filter((event) => event.type === "tool_execution_end"));
});
const history = await viewHistory(harness, root.id);
const nested = nestedResults(harness);
const settled = await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
expect(settled.status).toBe("done");
await harness.waitForIdle(context);
await stream.stop();
history.detach();
const [result] = results(await allEntries(root));
expect(text(result)).toBe("returned early");
const [, child] = await toolTasks(harness);
@@ -389,16 +388,13 @@ describe("nested tool calls", () => {
outcome: { status: "aborted", result: { kind: "nested" } },
});
// The aborted call's result keeps its details; programs get no partial output, which only the model read.
const childEnd = ends.find((event) => event.type === "tool_execution_end" && event.toolCallId === "c1/1");
expect(childEnd).toMatchObject({ result: { isError: true, details: { step: 1 } } });
expect((childEnd as { result?: NestedToolExecutionResult }).result).not.toHaveProperty("structuredOutput");
// The nested call ends, with its result, before its caller.
expect(ends.map((event) => [event.type === "tool_execution_end" && event.toolCallId, "result" in event])).toEqual(
[
["c1/1", true],
["c1", false],
],
);
const childResult = nestedResultOf(history.live(), nested, "c1/1");
expect(childResult).toMatchObject({ isError: true, details: { step: 1 } });
expect(childResult).not.toHaveProperty("structuredOutput");
// The nested call ends before its caller.
const live = history.live();
expect(endedAt(live, "c1/1")).toBeGreaterThan(0);
expect(endedAt(live, "c1/1")).toBeLessThan(endedAt(live, "c1"));
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
await harness.close(context);
});
@@ -429,7 +425,7 @@ describe("nested tool calls", () => {
await harness.close(context);
});
it("reports nested calls in pi.live.nestedTools and as tool events with their parent", async () => {
it("reports nested calls in pi.live.nestedTools with their parent", async () => {
const setup = chatSetup();
const release = deferred();
const running = deferred();
@@ -452,12 +448,9 @@ describe("nested tool calls", () => {
);
setup.faux.setResponses([call("batch"), DONE]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
expect(stream.snapshot.nestedTools).toEqual([]);
const events: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const history = await viewHistory(harness, root.id);
expect(history.live()[0]!.nestedTools).toBeUndefined();
const nested = nestedResults(harness);
const submission = await root.submit({ type: "input", content: "go" }, context);
await running.promise;
const live = await harness.snapshot(LiveDoc, root.id, context);
@@ -477,32 +470,19 @@ describe("nested tool calls", () => {
release.resolve();
await submission.wait(context);
await harness.waitForIdle(context);
await stream.stop();
const tools = events.flatMap((event) =>
event.type === "tool_execution_start" || event.type === "tool_execution_end"
? [[event.type, event.toolCallId, event.parentToolCallId]]
: [],
);
expect(tools).toEqual([
["tool_execution_start", "c1", undefined],
["tool_execution_start", "c1/1", "c1"],
["tool_execution_end", "c1/1", "c1"],
["tool_execution_end", "c1", undefined],
]);
const nestedEnd = events.find((event) => event.type === "tool_execution_end" && event.toolCallId === "c1/1");
expect(nestedEnd).toMatchObject({ result: { structuredOutput: "slow done", isError: false } });
// Events name the call's task and, for a nested call, the calling one.
history.detach();
const states = history.live();
expect(startOrder(states)).toEqual(["c1", "c1/1"]);
expect(endedAt(states, "c1/1")).toBeLessThan(endedAt(states, "c1"));
expect(nestedResultOf(states, nested, "c1/1")).toMatchObject({ structuredOutput: "slow done", isError: false });
// Slots name the call's task and, for a nested call, the calling one.
const [parent, child] = await toolTasks(harness);
const parentEnd = events.find((event) => event.type === "tool_execution_end" && event.toolCallId === "c1");
expect(nestedEnd).toMatchObject({ taskId: child!.id, parentTaskId: parent!.id });
expect(parentEnd).toMatchObject({ taskId: parent!.id });
expect(parentEnd).not.toHaveProperty("parentTaskId");
expect(
events.some(
(event) =>
event.type === "tool_execution_update" && event.toolCallId === "c1/1" && event.parentToolCallId === "c1",
),
).toBe(true);
expect(slotsOf(states, "c1/1").find((slot) => slot !== undefined)).toMatchObject({
taskId: child!.id,
parentCallId: "c1",
parentTaskId: parent!.id,
});
expect(slotsOf(states, "c1").find((slot) => slot?.taskId !== undefined)).toMatchObject({ taskId: parent!.id });
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
await harness.close(context);
});
@@ -535,7 +515,9 @@ describe("nested tool calls", () => {
record!.state.status === "terminal" ? record!.state.outcome : undefined;
expect(outcome(child)).toMatchObject({ status: "aborted", result: { kind: "nested" } });
expect(outcome(parent)).toMatchObject({ status: "aborted", result: { kind: "model" } });
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({
runFailure: expect.objectContaining({ reason: "aborted" }),
});
await harness.close(context);
});
@@ -831,29 +813,19 @@ describe("nested tool calls", () => {
);
setup.faux.setResponses([call("outer"), DONE]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
const tools: (string | undefined)[][] = [];
stream.start(async ({ value: batch }) => {
for (const event of batch) {
if (event.type === "tool_execution_start" || event.type === "tool_execution_end") {
tools.push([event.type, event.toolCallId, event.parentToolCallId]);
}
}
});
const history = await viewHistory(harness, root.id);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await harness.waitForIdle(context);
await stream.stop();
history.detach();
expect(text(results(await allEntries(root))[0])).toBe("echo deep");
const [outer, inner, deep] = await toolTasks(harness);
expect([inner!.owner, deep!.owner]).toEqual([outer!.id, inner!.id]);
expect(tools).toEqual([
["tool_execution_start", "c1", undefined],
["tool_execution_start", "c1/1", "c1"],
["tool_execution_start", "c1/1/x", "c1/1"],
["tool_execution_end", "c1/1/x", "c1/1"],
["tool_execution_end", "c1/1", "c1"],
["tool_execution_end", "c1", undefined],
]);
const states = history.live();
expect(startOrder(states)).toEqual(["c1", "c1/1", "c1/1/x"]);
expect(slotsOf(states, "c1/1/x").find((slot) => slot !== undefined)).toMatchObject({ parentCallId: "c1/1" });
// Children end first.
expect(endedAt(states, "c1/1/x")).toBeLessThan(endedAt(states, "c1/1"));
expect(endedAt(states, "c1/1")).toBeLessThan(endedAt(states, "c1"));
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
await harness.close(context);
});
@@ -889,31 +861,31 @@ describe("nested tool calls", () => {
const { harness, root } = await openChat(new MemoryStorage(), setup);
const submission = await root.submit({ type: "input", content: "go" }, context);
await Promise.all([running.one.promise, running.two.promise]);
const stream = await watchEvents(harness, root.id, context);
expect(stream.snapshot.nestedTools.map((slot) => [slot.callId, slot.status])).toEqual([
const watch = await root.watch(context);
const nestedOf = (view: ConversationView) => (view.docs["pi.live"] as LiveState).nestedTools ?? [];
expect(nestedOf(watch.value).map((slot) => [slot.callId, slot.status])).toEqual([
["c1/1", "running"],
["c1/2", "running"],
]);
const updates: [string, unknown][] = [];
stream.start(async ({ value: batch }) => {
for (const event of batch) {
if (event.type === "tool_execution_update" && event.output !== undefined) {
updates.push([event.toolCallId, event.output]);
}
}
// Each value's operations rebuild the next one, the late subscriber's starting value included.
let replayed = watch.value;
const outputs: [string | undefined, string | undefined][] = [];
watch.start(async ({ value, ops }) => {
replayed = applyImmutable(replayed, ops);
expect(replayed).toEqual(value);
const [one, two] = nestedOf(value);
outputs.push([one?.output, two?.output]);
});
// The second slot writes first, so a wrong slot index in the update path would show.
wrote.two.resolve();
await eventually(() => updates.length === 1);
await eventually(() => outputs.at(-1)?.[1] === "two\n");
expect(outputs.at(-1)).toEqual([undefined, "two\n"]);
wrote.one.resolve();
await eventually(() => updates.length === 2);
await eventually(() => outputs.at(-1)?.[0] === "one\n");
expect(outputs.at(-1)).toEqual(["one\n", "two\n"]);
release.resolve();
await submission.wait(context);
await stream.stop();
expect(updates).toEqual([
["c1/2", { set: "two\n" }],
["c1/1", { set: "one\n" }],
]);
await watch.stop();
await harness.close(context);
});
@@ -988,17 +960,13 @@ describe("nested tool calls", () => {
);
setup.faux.setResponses([call("batch"), DONE]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
const ends: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
ends.push(...batch.filter((event) => event.type === "tool_execution_end"));
});
const history = await viewHistory(harness, root.id);
const nested = nestedResults(harness);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await harness.waitForIdle(context);
await stream.stop();
history.detach();
// The abort's result is built from the slot, which never got the output.
const end = ends.find((event) => event.type === "tool_execution_end" && event.toolCallId === "c1/1");
const result = (end as { result?: NestedToolExecutionResult }).result!;
const result = nestedResultOf(history.live(), nested, "c1/1")!;
expect(result).toMatchObject({ isError: true });
expect(result).not.toHaveProperty("structuredOutput");
expect(result).not.toHaveProperty("details");
@@ -1038,11 +1006,6 @@ describe("nested tool calls", () => {
);
setup.faux.setResponses([call("batch"), DONE]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
const starts: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
starts.push(...batch.filter((event) => event.type === "tool_execution_start"));
});
const submission = await root.submit({ type: "input", content: "go" }, context);
const slot = async () => (await harness.snapshot(LiveDoc, root.id, context))!.nestedTools![0]!;
await hookEntered.promise;
@@ -1053,10 +1016,7 @@ describe("nested tool calls", () => {
release.resolve();
await submission.wait(context);
await harness.waitForIdle(context);
await stream.stop();
// The start event carries the arguments the call runs with; the task input keeps the ones it was made with.
const start = starts.find((event) => event.type === "tool_execution_start" && event.toolCallId === "c1/1");
expect(start).toMatchObject({ args: { text: large } });
// The slot showed the arguments the call runs with; the task input keeps the ones it was made with.
const nested = (await toolTasks(harness)).find((task) => (task.input as ToolTaskInput).kind === "nested");
expect(nested!.input as ToolTaskInput).toMatchObject({ call: { arguments: { text: "asked" } } });
await harness.close(context);
packages/durable/test/harness-ownership.test.ts
+1 -1
View File
@@ -1153,7 +1153,7 @@ describe("owned conversations from tools and supervisors", () => {
const childConversation = (await harness.conversation(child!, context))!;
await childConversation.waitForIdle(context);
const childLive = await harness.snapshot(LiveDoc, child!, context);
expect(childLive).toEqual({});
expect(childLive).toEqual({ runFailure: expect.objectContaining({ reason: "aborted" }) });
const childInputs = (await harness.inspect(context)).submissions.filter((s) => s.conversationId === child);
expect(childInputs).toEqual([]);
await harness.waitForIdle(context);
packages/durable/test/harness-run-failure.test.ts
+156
View File
@@ -0,0 +1,156 @@
// `pi.live.runFailure`: why the newest run ended without an answer, as state a renderer shows (#11327).
import { type FauxResponseStep, fauxAssistantMessage, fauxText, fauxToolCall, Type } from "@earendil-works/pi-ai";
import {
type AfterFailedRun,
type Conversation,
defineTool,
type Harness,
LiveDoc,
type LiveState,
MemoryStorage,
type RunFailure,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { chatSetup, openChat, unanswered } from "./chat-support.ts";
import { addTool } from "./harness-support.ts";
import { context } from "./session-support.ts";
import { deferred } from "./task-support.ts";
async function live(harness: Harness, conversation: Conversation): Promise<Readonly<LiveState>> {
return (await harness.snapshot(LiveDoc, conversation.id, context)) ?? {};
}
/** A response that fails for good once `gate` opens; `reached` resolves when it was requested. */
function failingOnce() {
const gate = deferred();
const reached = deferred();
const step: FauxResponseStep = async () => {
reached.resolve();
await gate.promise;
return fauxAssistantMessage([], { stopReason: "error", errorMessage: "Invalid request" });
};
return { step, gate, reached };
}
describe("pi.live.runFailure", () => {
it("records a model error with its text, and a later run clears it", async () => {
const setup = chatSetup();
setup.faux.setResponses([
fauxAssistantMessage([], { stopReason: "error", errorMessage: "Invalid request" }),
fauxAssistantMessage([fauxText("fine")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const submission = await root.submit({ type: "input", content: "hi" }, context);
const settled = await submission.wait(context);
expect(settled).toMatchObject({ status: "unanswered", reason: "model_error", detail: "Invalid request" });
const failure = (await live(harness, root)).runFailure;
expect(failure).toEqual({
taskId: expect.any(Number),
inputs: [submission.id],
reason: "model_error",
detail: "Invalid request",
} satisfies RunFailure);
const next = await root.submit({ type: "input", content: "again" }, context);
// The commit that starts the next run removes it.
expect((await live(harness, root)).runFailure).toBeUndefined();
expect((await next.wait(context)).status).toBe("done");
expect(await live(harness, root)).toEqual({});
await harness.close(context);
});
it("records a missing model with which one", async () => {
const setup = chatSetup();
const { harness, root } = await openChat(new MemoryStorage(), setup);
await root.configure({ model: { provider: "faux", modelId: "gone" } }, context);
const submission = await root.submit({ type: "input", content: "hi" }, context);
const settled = await submission.wait(context);
expect(settled).toMatchObject({ status: "unanswered", reason: "no_model" });
// The settlement is unchanged: no detail.
expect(settled).not.toHaveProperty("detail");
expect((await live(harness, root)).runFailure).toMatchObject({
inputs: [submission.id],
reason: "no_model",
detail: "Model faux/gone is not available",
});
await harness.close(context);
});
it("records an abort", async () => {
const setup = chatSetup();
const busy = unanswered();
setup.faux.setResponses([busy.step]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const submission = await root.submit({ type: "input", content: "hi" }, context);
await busy.reached;
const taskId = (await live(harness, root)).run!.taskId;
await root.abort(context);
expect(await submission.wait(context)).toMatchObject({ status: "unanswered", reason: "aborted" });
expect((await live(harness, root)).runFailure).toEqual({ taskId, inputs: [submission.id], reason: "aborted" });
await harness.close(context);
});
it("records a reset that cut the run before an answer, kept by the run its queued input starts", async () => {
const setup = chatSetup();
const gate = deferred();
const started = deferred();
addTool(
setup.registry,
defineTool({
name: "hold",
description: "Waits",
parameters: Type.Object({}),
execute: async () => {
started.resolve();
await gate.promise;
return {};
},
}),
);
const second = unanswered();
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("hold", {}, { id: "c1" })], { stopReason: "toolUse" }),
second.step,
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const first = await root.submit({ type: "input", content: "go" }, context);
await started.promise;
await root.reset(undefined, context);
const followUp = await root.submit({ type: "input", content: "next", whenBusy: "followUp" }, context);
gate.resolve();
expect(await first.wait(context)).toMatchObject({ status: "unanswered", reason: "reset" });
await second.reached;
const state = await live(harness, root);
// The follow-up's run started in the commit that ended the first; the failure stays until a later run starts.
expect(state.run?.inputs).toEqual([followUp.id]);
expect(state.runFailure).toMatchObject({ inputs: [first.id], reason: "reset" });
await root.abort(context);
expect((await live(harness, root)).runFailure).toMatchObject({ inputs: [followUp.id], reason: "aborted" });
await harness.close(context);
});
it("keeps the failure while a run continues the queue in the same commit, and replaces it with the next", async () => {
for (const policy of ["continue", "keep"] as const satisfies readonly AfterFailedRun[]) {
const setup = chatSetup();
setup.settings.afterFailedRun = policy;
const failing = failingOnce();
const busy = unanswered();
setup.faux.setResponses([failing.step, busy.step, fauxAssistantMessage([fauxText("third")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const first = await root.submit({ type: "input", content: "first" }, context);
await failing.reached.promise;
const second = await root.submit({ type: "input", content: "second", whenBusy: "followUp" }, context);
failing.gate.resolve();
await first.wait(context);
if (policy === "continue") {
await busy.reached;
const state = await live(harness, root);
expect(state.run?.inputs).toEqual([second.id]);
expect(state.runFailure).toMatchObject({ inputs: [first.id], reason: "model_error" });
} else {
await harness.waitForIdle(context);
expect((await live(harness, root)).runFailure).toMatchObject({ inputs: [first.id] });
}
await harness.close(context);
}
});
});
packages/durable/test/harness-storage-failure.test.ts
+6 -6
View File
@@ -11,7 +11,6 @@ import {
MemoryStorage,
StorageRequestError,
type TaskId,
watchEvents,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { safeNow } from "../src/harness/harness.ts";
@@ -95,8 +94,8 @@ describe("a failed Storage call", () => {
const task = harness.waitForTask(id, context);
const idle = harness.waitForIdle(context);
const watch = (await harness.watchDoc(LiveDoc, root.id, context))!;
const stream = await watchEvents(harness, root.id, context);
stream.start(async () => {});
const view = await root.watch(context);
view.start(async () => {});
const failure = new Error("disk gone");
storage.failNextCommit(failure);
@@ -107,7 +106,7 @@ describe("a failed Storage call", () => {
await expect(task).rejects.toMatchObject(failed);
await expect(idle).rejects.toMatchObject(failed);
expect(await watch.closed).toEqual({ reason: "failed", error: failure });
expect(await stream.closed).toEqual({ reason: "failed", error: failure });
expect(await view.closed).toEqual({ reason: "failed", error: failure });
// Every later call, and a wait that starts now.
await expect(harness.getTask(id, context)).rejects.toMatchObject(failed);
await expect(harness.waitForIdle(context)).rejects.toMatchObject(failed);
@@ -487,8 +486,9 @@ describe("listener and report errors", () => {
const failure = new Error("observer broke");
await views.attach(
root.id,
async () => ({
publication: () => {
undefined,
() => ({
advance: () => {
throw failure;
},
end: () => {},
packages/durable/test/harness-structured-output.test.ts
+1 -12
View File
@@ -20,7 +20,6 @@ import {
type ToolRegistration,
ToolResultEntry,
ToolTask,
watchEvents,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { NodeExecutionEnv } from "../src/env/node.ts";
@@ -163,25 +162,15 @@ describe("structured output", () => {
if (result !== undefined && change.ops.length > 0) stored.push(result as unknown as JsonValue);
}
});
const stream = await watchEvents(harness, root.id, context);
const ends: NestedToolExecutionResult[] = [];
stream.start(async ({ value: batch }) => {
for (const event of batch) {
if (event.type === "tool_execution_end" && event.parentToolCallId !== undefined) {
ends.push(event.result as NestedToolExecutionResult);
}
}
});
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await harness.waitForIdle(context);
await stream.stop();
await harness.close(context);
expect(received[0]).toMatchObject({ structuredOutput: "a", isError: false });
expect(received[1]).toMatchObject({ structuredOutput: null, isError: false });
expect(received[0]).not.toHaveProperty("output");
// A client reads a live caller's nested results from the same documents.
expect(stored).toEqual(received);
expect(ends).toEqual(received);
});
it("gives callers a schema tool's validated structured output, which the transcript does not store", async () => {
packages/durable/test/harness-structured.test.ts
+79 -63
View File
@@ -11,9 +11,9 @@ import {
Type,
} from "@earendil-works/pi-ai";
import {
type AgentEvent,
type Conversation,
type ConversationId,
type ConversationView,
createSession,
defineDoc,
defineTask,
@@ -26,18 +26,19 @@ import {
type NextTaskState,
type Registry,
type Storage,
type TaskGraph,
type TaskId,
type TaskOptions,
type TaskRuntime,
type ToolRegistration,
ToolResultEntry,
type Tx,
watchEvents,
} from "@earendil-works/pi-durable";
import { afterEach, describe, expect, it } from "vitest";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, chatSetup, openChat, waitFor } from "./chat-support.ts";
import { addHooks, addTask, addTool } from "./harness-support.ts";
import { generations, graphHistory, slotsOf, viewHistory } from "./live-support.ts";
import { ControlledStorage, context } from "./session-support.ts";
import { aborted, type Deferred, deferred, openTasks, settled } from "./task-support.ts";
@@ -1251,7 +1252,9 @@ describe("tool rounds", () => {
expect(await submission.wait(context)).toMatchObject({ status: "unanswered", reason: "aborted" });
const results = (await allEntries(root)).filter((entry) => ToolResultEntry.is(entry));
expect(results.map((entry) => (entry.model![0] as ToolResultMessage).toolCallId)).toEqual(["c1", "c2"]);
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({});
expect(await harness.snapshot(LiveDoc, root.id, context)).toEqual({
runFailure: { taskId: generation, inputs: [submission.id], reason: "aborted" },
});
expect((await harness.waitForTask(generation, context)).state.outcome.status).toBe("aborted");
await harness.close(context);
});
@@ -1286,7 +1289,7 @@ describe("tool rounds", () => {
await harness.close(context);
});
it("ends a turn at the generation's hold, before its successor's turn starts", async () => {
it("hands run control to the successor while the generation holds, and keeps it in the graph until it ends", async () => {
const setup = chatSetup();
addTask(setup.registry, Node);
addTool(
@@ -1308,21 +1311,23 @@ describe("tool rounds", () => {
setup.faux.setResponses([toolCalls(["noop", "c1"]), fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
opened = harness;
const stream = await watchEvents(harness, root.id, context);
const events: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const graph = await graphHistory(harness);
const history = await viewHistory(harness, root.id);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
const first = (await harness.commit((tx) => tx.scanTasks({ kind: "pi.generation" }, 1), context)).items[0]!;
expect((await state(harness, first.id)).status).toBe("completing");
const turns = () => events.filter((event) => event.type.startsWith("turn_")).map((event) => event.type);
await waitFor(() => turns().length === 4);
expect(turns()).toEqual(["turn_start", "turn_end", "turn_start", "turn_end"]);
// The first generation's outcome is committed at its hold: the successor already runs the run, and the graph shows
// the first one completing beside it.
const handed = graph.graphs.find((value) => generations(value).length === 2)!;
const [held, successor] = generations(handed);
expect(held).toMatchObject({ id: first.id, state: { status: "completing" } });
expect(successor!.state.status).not.toBe("completing");
expect(history.live().some((live) => live.run?.taskId === successor!.id)).toBe(true);
open("hooked");
expect(await outcomeOf(harness, first.id)).toBe("completed");
await waitFor(() => turns().length === 4);
await stream.stop();
await waitFor(() => graph.graphs.at(-1)!.tasks[String(first.id)] === undefined);
graph.detach();
history.detach();
await harness.close(context);
});
@@ -1396,7 +1401,14 @@ describe("tool rounds", () => {
opened = reopened.harness;
const again = (await reopened.harness.submission(submission.id, context))!;
expect(await again.wait(context)).toMatchObject({ status: "unanswered", reason: "faulted" });
expect(await reopened.harness.snapshot(LiveDoc, reopened.root.id, context)).toEqual({});
expect(await reopened.harness.snapshot(LiveDoc, reopened.root.id, context)).toEqual({
runFailure: {
taskId: expect.any(Number),
inputs: [submission.id],
reason: "faulted",
detail: "Value contains a non-JSON function; expected strict JSON",
},
});
expect((await allEntries(reopened.root)).map((entry) => entry.kind)).toEqual(["pi.user", "pi.assistant"]);
await reopened.harness.close(context);
});
@@ -1757,25 +1769,12 @@ describe("recovery through ownership edges", () => {
});
});
describe("tool rounds and events", () => {
async function listen(harness: Harness, conversation: Conversation) {
const stream = await watchEvents(harness, conversation.id, context);
const events: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
return { stream, events };
}
/** Event types, with tool ends and message ends labelled by call ID and whether an entry came along. */
function labels(events: readonly AgentEvent[]): string[] {
return events.flatMap((event) => {
if (event.type === "tool_execution_end") return [`end:${event.toolCallId}:${event.entry !== undefined}`];
if (event.type === "message_end") {
const message = event.entry.model?.[0];
return [message?.role === "toolResult" ? `result:${message.toolCallId}` : `message:${message?.role}`];
}
return [];
describe("tool rounds as the view shows them", () => {
/** The tool result entries of a view, by call ID, in transcript order. */
function resultsOf(view: ConversationView): string[] {
return view.entries.flatMap((entry) => {
const message = entry.model?.[0];
return message?.role === "toolResult" ? [message.toolCallId] : [];
});
}
@@ -1786,25 +1785,36 @@ describe("tool rounds and events", () => {
addTool(setup.registry, defineTool({ ...blockingTool("two").registration, name: "two" }));
setup.faux.setResponses([toolCalls(["one", "c1"], ["two", "c2"], ["two", "c3"])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const history = await viewHistory(harness, root.id);
const submission = await root.submit({ type: "input", content: "go" }, context);
await one.started;
await harness.abortTask((await harness.snapshot(LiveDoc, root.id, context))!.run!.taskId, context);
await submission.wait(context);
await waitFor(() => labels(events).includes("result:c3"));
expect(labels(events).filter((label) => !label.startsWith("message:"))).toEqual([
"end:c1:true",
"result:c1",
"end:c2:true",
"result:c2",
"end:c3:true",
"result:c3",
history.detach();
// The started call ends with its result entry; the commit that ends the run appends the unstarted calls' results,
// in call order, and removes the round.
const states = history.live();
const c1Done = slotsOf(states, "c1").find((slot) => slot?.status === "done");
const last = history.views.at(-1)!;
expect(resultsOf(last)).toEqual(["c1", "c2", "c3"]);
const c1Result = last.entries.find(
(entry) =>
entry.model?.[0]?.role === "toolResult" && (entry.model[0] as ToolResultMessage).toolCallId === "c1",
);
expect(c1Done?.entry).toBe(c1Result?.id);
const ending = history.views.findIndex((view) => resultsOf(view).includes("c2"));
expect(resultsOf(history.views[ending]!)).toEqual(["c1", "c2", "c3"]);
expect(states[ending]!.run).toBeUndefined();
expect(states[ending]!.tools).toBeUndefined();
expect(states[ending - 1]!.tools?.map((slot) => [slot.callId, slot.status])).toEqual([
["c1", "done"],
["c2", "pending"],
["c3", "pending"],
]);
await stream.stop();
await harness.close(context);
});
it("emits one turn_end per generation, also for a stream attached while it holds", async () => {
it("shows a held generation as completing to an observer attached while it holds", async () => {
const setup = chatSetup();
addTask(setup.registry, Node);
addTool(
@@ -1825,19 +1835,22 @@ describe("tool rounds and events", () => {
setup.faux.setResponses([toolCalls(["noop", "c1"]), fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
opened = harness;
const early = await listen(harness, root);
const early = await graphHistory(harness);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
const late = await listen(harness, root);
const late = await graphHistory(harness);
open("hooked");
const [first] = (await harness.commit((tx) => tx.scanTasks({ kind: "pi.generation" }, 1), context)).items;
await harness.waitForTask(first!.id, context);
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
await waitFor(() => late.events.some((event) => event.type === "entry_appended"));
const turns = (events: readonly AgentEvent[]) => events.filter((event) => event.type === "turn_end").length;
expect(turns(early.events)).toBe(2);
expect(turns(late.events)).toBe(0);
await early.stream.stop();
await late.stream.stop();
// The early observer saw both generations; the late one starts with the held one, completing, and sees it leave.
const seen = (graphs: readonly TaskGraph[]) =>
new Set(graphs.flatMap((graph) => generations(graph).map((node) => node.id)));
expect(seen(early.graphs).size).toBe(2);
expect(generations(late.graphs[0]!)).toEqual([
expect.objectContaining({ id: first!.id, state: expect.objectContaining({ status: "completing" }) }),
]);
expect(late.graphs.at(-1)!.tasks[String(first!.id)]).toBeUndefined();
early.detach();
late.detach();
await harness.close(context);
});
@@ -1861,7 +1874,6 @@ describe("tool rounds and events", () => {
);
setup.faux.setResponses([toolCalls(["delegate", "c1"]), fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(async () => (await harness.snapshot(LiveDoc, root.id, context))?.tools?.[0]?.status === "done");
const live = (await harness.snapshot(LiveDoc, root.id, context))!;
@@ -1869,16 +1881,14 @@ describe("tool rounds and events", () => {
expect(live.tools![0]!.entry).toBeDefined();
expect((await state(harness, tool)).status).toBe("completing");
expect((await state(harness, live.run!.taskId)).status).toBe("waiting");
await waitFor(() => labels(events).includes("end:c1:true"));
expect(await settled(submission.wait(context))).toBe(false);
open("sub");
expect((await submission.wait(context)).status).toBe("done");
expect(await outcomeOf(harness, tool)).toBe("completed");
await stream.stop();
await harness.close(context);
});
it("holds a faulted tool's slot and task_failed until the work it owns drained", async () => {
it("holds a faulted tool's slot and its graph node until the work it owns drained", async () => {
const setup = chatSetup();
addTask(setup.registry, Node);
script("held", {
@@ -1901,20 +1911,26 @@ describe("tool rounds and events", () => {
);
setup.faux.setResponses([toolCalls(["broken", "c1"]), fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { stream, events } = await listen(harness, root);
const history = await viewHistory(harness, root.id);
const graph = await graphHistory(harness);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => log.includes("abort:held"));
const live = (await harness.snapshot(LiveDoc, root.id, context))!;
const tool = live.tools![0]!.taskId!;
expect(await state(harness, tool)).toMatchObject({ status: "completing", outcome: { status: "faulted" } });
expect(live.tools![0]!.status).not.toBe("done");
expect(events.some((event) => event.type === "task_failed")).toBe(false);
// The graph shows the held fault; the task is not terminal yet.
expect(graph.graphs.at(-1)!.tasks[String(tool)]?.state).toEqual({ status: "completing", outcome: "faulted" });
open("abort.held");
expect((await submission.wait(context)).status).toBe("done");
expect(await outcomeOf(harness, tool)).toBe("faulted");
await waitFor(() => events.some((event) => event.type === "task_failed"));
expect(labels(events)).toContain("end:c1:false");
await stream.stop();
await waitFor(() => graph.graphs.at(-1)!.tasks[String(tool)] === undefined);
// The slot ends done without a result entry.
const done = slotsOf(history.live(), "c1").find((slot) => slot?.status === "done");
expect(done).toBeDefined();
expect(done).not.toHaveProperty("entry");
history.detach();
graph.detach();
await harness.close(context);
});
});
packages/durable/test/harness-view-run.test.ts
+607
View File
@@ -0,0 +1,607 @@
// What a renderer sees of a run through the conversation view: the transcript window and `pi.live`, every change a
// commit makes, rebuilt from operations alone. This replaced the event stream (`watchEvents`, #11327).
import { GCProfiler, getHeapStatistics, setFlagsFromString } from "node:v8";
import { runInNewContext } from "node:vm";
import type { Draft } from "@earendil-works/chord";
import { applyImmutable, type Op } from "@earendil-works/chord/delta";
import {
type AssistantMessage,
type FauxResponseStep,
fauxAssistantMessage,
fauxText,
fauxThinking,
fauxToolCall,
Type,
} from "@earendil-works/pi-ai";
import {
type Conversation,
type ConversationView,
defineTool,
type Harness,
LiveDoc,
type LiveState,
MemoryStorage,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { type ChatSetup, chatSetup, openChat, textOf, waitFor } from "./chat-support.ts";
import { addTool } from "./harness-support.ts";
import { slotsOf, viewHistory } from "./live-support.ts";
import { context, documentChanges, observeCommits } from "./session-support.ts";
import { aborted, deferred } from "./task-support.ts";
type Frame = { value: ConversationView; ops: readonly Op[] };
/**
* Start a watch of `conversation` that records every call and, with `check`, checks that its operations turn the value
* before it into its value. The frames are what a renderer that keeps up gets.
*/
async function listen(conversation: Conversation, check = true) {
const watch = await conversation.watch(context);
const frames: Frame[] = [];
let replayed = watch.value;
let mismatches = 0;
watch.start(async ({ value, ops }) => {
if (check) {
replayed = applyImmutable(replayed, ops);
if (JSON.stringify(replayed) !== JSON.stringify(value)) mismatches++;
}
frames.push({ value, ops });
});
return { watch, frames, mismatches: () => mismatches };
}
const liveOf = (view: ConversationView) => (view.docs["pi.live"] ?? {}) as LiveState;
async function drained(): Promise<void> {
await new Promise((resolve) => setTimeout(resolve, 0));
}
function slow(): ChatSetup {
return chatSetup({ tokensPerSecond: 400, tokenSize: { min: 1, max: 1 } });
}
/** Committed partials of the Harness's generations, one per commit that has one. */
function partialsOf(harness: Harness): AssistantMessage[] {
const partials: AssistantMessage[] = [];
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
const message = (change.value as LiveState | null)?.generation?.message as AssistantMessage | undefined;
if (change.record.kind === "pi.live" && message !== undefined) partials.push(message);
}
});
return partials;
}
/** The distinct in-flight partials in `states`, in order. */
function partialsIn(states: readonly LiveState[]): AssistantMessage[] {
const partials: AssistantMessage[] = [];
for (const state of states) {
const message = state.generation?.message as AssistantMessage | undefined;
if (message !== undefined && message !== partials.at(-1)) partials.push(message);
}
return partials;
}
describe("a run as the conversation view shows it", () => {
it("streams the answer as operations that rebuild every committed partial, then appends it", async () => {
const setup = slow();
const text = "streamed answer text ".repeat(20);
setup.faux.setResponses([fauxAssistantMessage([fauxText(text)])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const partials = partialsOf(harness);
const history = await viewHistory(harness, root.id);
const { watch, mismatches } = await listen(root);
const submission = await root.submit({ type: "input", content: "hi" }, context);
await submission.wait(context);
await drained();
history.detach();
expect(mismatches()).toBe(0);
const states = history.live();
expect(partials.length).toBeGreaterThan(1);
expect(partialsIn(states).map(textOf)).toEqual(partials.map(textOf));
// The run shows from the input's commit until the commit that appends the answer and clears the partial.
const started = states.findIndex((state) => state.run !== undefined);
expect(states[started]!.run).toEqual({ taskId: expect.any(Number), inputs: [submission.id] });
expect(history.views[started]!.entries.at(-1)!.kind).toBe("pi.user");
const ended = states.findIndex((state, index) => index > started && state.run === undefined);
expect(states[ended]!.generation).toBeUndefined();
expect(textOf(history.views[ended]!.entries.at(-1)!.model![0])).toBe(text);
expect(states.at(-1)).toEqual({});
await watch.stop();
await harness.close(context);
});
it("shows a tool call's slot from pending through its running output to its result entry", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "print",
description: "Prints",
parameters: Type.Object({ n: Type.Number() }),
execute: async (_args, api) => {
api.output("one\n");
await new Promise((resolve) => setTimeout(resolve, 150));
api.output("two\n");
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("print", { n: 1 }, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const history = await viewHistory(harness, root.id);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => slotsOf(history.live(), "c1").at(-1)?.output === "one\ntwo\n");
gate.resolve();
await submission.wait(context);
history.detach();
const slots = slotsOf(history.live(), "c1");
const listed = slots.findIndex((slot) => slot !== undefined);
// Listed pending with the tool-calling answer, in the same commit.
expect(slots[listed]).toEqual({ callId: "c1", name: "print", status: "pending", taskId: expect.any(Number) });
expect(history.views[listed]!.entries.at(-1)!.kind).toBe("pi.assistant");
const outputs = slots.flatMap((slot) => (slot?.status === "running" ? [slot.output ?? ""] : []));
expect([...new Set(outputs)]).toEqual(["", "one\n", "one\ntwo\n"]);
// Done with its result entry, appended in the same commit.
const done = slots.findIndex((slot) => slot?.status === "done");
expect(slots[done]).toEqual({
callId: "c1",
name: "print",
status: "done",
taskId: expect.any(Number),
entry: history.views[done]!.entries.at(-1)!.id,
});
expect(history.views[done]!.entries.at(-1)!.model![0]).toMatchObject({ role: "toolResult", toolCallId: "c1" });
// The run's two generations, the tool round and the answer, hand the run over.
const runTasks = new Set(history.live().flatMap((state) => (state.run === undefined ? [] : [state.run.taskId])));
expect(runTasks.size).toBe(2);
await harness.close(context);
});
it("shows queued inputs in the inbox, a retry backoff, and the next run", async () => {
const setup = chatSetup();
const release = deferred();
const held: FauxResponseStep = async (_request, options) => {
await Promise.race([release.promise, aborted(options!.signal!)]);
return fauxAssistantMessage([fauxText("first")]);
};
const error = fauxAssistantMessage([], { stopReason: "error", errorMessage: "503 Service Unavailable" });
setup.faux.setResponses([held, error, fauxAssistantMessage([fauxText("second")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
setup.settings.retry = { enabled: true, maxRetries: 1, baseDelayMs: 1 };
const history = await viewHistory(harness, root.id);
const first = await root.submit({ type: "input", content: "a" }, context);
const followUp = await root.submit({ type: "input", content: "f" }, context);
const inbox = (view: ConversationView) =>
(view.docs["pi.inbox"] as { items: { id: number; mode: string }[] }).items;
expect(inbox(history.views.at(-1)!)).toEqual([expect.objectContaining({ id: followUp.id, mode: "followUp" })]);
release.resolve();
await followUp.wait(context);
history.detach();
const states = history.live();
expect(states.some((state) => state.generation?.retry?.error === "503 Service Unavailable")).toBe(true);
const retried = states.findIndex((state) => state.generation?.retry !== undefined);
expect(states.slice(retried).some((state) => state.generation?.attempt === 2)).toBe(true);
const runs = states.flatMap((state, index) =>
state.run !== undefined && state.run.inputs !== states[index - 1]?.run?.inputs ? [state.run.inputs] : [],
);
expect(runs.at(0)).toEqual([first.id]);
expect(runs.at(-1)).toEqual([followUp.id]);
expect(inbox(history.views.at(-1)!)).toEqual([]);
await harness.close(context);
});
it("rebuilds every committed partial of thinking, text, and tool-call arguments from the view's operations", async () => {
const setup = chatSetup({ tokensPerSecond: 150, tokenSize: { min: 1, max: 1 } });
const message = fauxAssistantMessage(
[
fauxThinking("thinking about it ".repeat(10)),
fauxText("some text ".repeat(10)),
fauxToolCall("missing", { path: "a/long/path/".repeat(10), note: "x".repeat(60) }, { id: "c1" }),
],
{ stopReason: "toolUse" },
);
setup.faux.setResponses([message, fauxAssistantMessage([fauxText("done")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const partials = partialsOf(harness);
const views: ConversationView[] = [];
const ops: Op[][] = [];
const observation = await harness.observeConversation(
root.id,
{
advance: (change) => {
views.push(change.value);
ops.push([...change.ops]);
},
end: () => {},
},
context,
);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
observation.detach();
// Replaying the operations alone, from the observation's value, gives every committed partial.
let replayed = observation.value;
const rebuilt: unknown[] = [];
for (const batch of ops) {
replayed = applyImmutable(replayed, batch);
const partial = liveOf(replayed).generation?.message;
if (partial !== undefined && partial !== rebuilt.at(-1)) rebuilt.push(partial);
}
expect(replayed).toEqual(views.at(-1));
expect(partials.length).toBeGreaterThan(2);
expect(rebuilt.map((partial) => (partial as AssistantMessage).content)).toEqual(
partials.map((partial) => partial.content),
);
// Growing text and thinking travel as appends.
const appends = ops.flat().filter((op) => op[0] === "a" && op[1][2] === "generation");
expect(appends.length).toBeGreaterThan(2);
await harness.close(context);
});
it("rebuilds a sliding tail window from output trims and appends", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "tail",
description: "Prints lines",
parameters: Type.Object({}),
outputLimits: { maxLines: 3, retain: "tail" },
execute: async (_args, api) => {
for (let line = 0; line < 6; line++) {
api.output(`line ${line}\n`);
await new Promise((resolve) => setTimeout(resolve, 120));
}
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("tail", {}, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const outputs: string[] = [];
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
const output = (change.value as LiveState | null)?.tools?.[0]?.output;
if (change.record.kind === "pi.live" && output !== undefined && output !== outputs.at(-1)) {
outputs.push(output);
}
}
});
const { watch, frames, mismatches } = await listen(root);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => outputs.at(-1)?.endsWith("line 5\n") === true);
gate.resolve();
await submission.wait(context);
await drained();
await watch.stop();
expect(mismatches()).toBe(0);
const shown: string[] = [];
for (const frame of frames) {
const output = liveOf(frame.value).tools?.[0]?.output;
if (output !== undefined && output !== shown.at(-1)) shown.push(output);
}
expect(shown).toEqual(outputs);
expect(frames.some((frame) => frame.ops.some((op) => op[0] === "t"))).toBe(true);
await harness.close(context);
});
it("ends a run and starts the queued follow-up's in one value", async () => {
const setup = chatSetup();
const release = deferred();
const held: FauxResponseStep = async (_request, options) => {
await Promise.race([release.promise, aborted(options!.signal!)]);
return fauxAssistantMessage([fauxText("first")]);
};
setup.faux.setResponses([held, fauxAssistantMessage([fauxText("second")])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const input = await root.submit({ type: "input", content: "a" }, context);
const followUp = await root.submit({ type: "input", content: "f" }, context);
const history = await viewHistory(harness, root.id);
release.resolve();
await followUp.wait(context);
history.detach();
const states = history.live();
const boundary = states.findIndex((state) => state.run?.inputs[0] === followUp.id);
expect(states[boundary - 1]!.run?.inputs).toEqual([input.id]);
const before = history.views[boundary - 1]!;
const after = history.views[boundary]!;
const appended = after.entries.slice(before.entries.length);
expect(appended.map((entry) => [entry.kind, textOf(entry.model?.[0])])).toEqual([
["pi.assistant", "first"],
["pi.user", "f"],
]);
expect((after.docs["pi.inbox"] as { items: unknown[] }).items).toEqual([]);
await harness.close(context);
});
it("ends a call that never runs with its result at once, and a tool aborted with its generation", async () => {
const setup = chatSetup();
addTool(
setup.registry,
defineTool({
name: "wait",
description: "Waits until aborted",
parameters: Type.Object({}),
execute: (_args, _api, callContext) => aborted(callContext.abortSignal!),
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("ghost", {}, { id: "c1" }), fauxToolCall("wait", {}, { id: "c2" })], {
stopReason: "toolUse",
}),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const history = await viewHistory(harness, root.id);
const submission = await root.submit({ type: "input", content: "go" }, context);
await waitFor(() => slotsOf(history.live(), "c2").some((slot) => slot?.status === "running"));
// Aborting the generation aborts its round: the tool ends with its aborted result first (spec §8.5).
await harness.abortTask((await harness.snapshot(LiveDoc, root.id, context))!.run!.taskId, context);
await submission.wait(context);
history.detach();
const states = history.live();
// The call not offered is listed done, with its result entry, in the commit of its calling answer.
const listed = slotsOf(states, "c1").findIndex((slot) => slot !== undefined);
const ghost = slotsOf(states, "c1")[listed]!;
expect(ghost.status).toBe("done");
const listing = history.views[listed]!.entries;
expect(listing.map((entry) => entry.kind).slice(-2)).toEqual(["pi.assistant", "pi.tool-result"]);
expect(ghost.entry).toBe(listing.at(-1)!.id);
// The aborted tool ends with its result entry, then the run ends without an answer.
const done = slotsOf(states, "c2").findIndex((slot) => slot?.status === "done");
expect(slotsOf(states, "c2")[done]!.entry).toBe(history.views[done]!.entries.at(-1)!.id);
expect(states.at(-1)!.runFailure).toMatchObject({ inputs: [submission.id], reason: "aborted" });
await harness.close(context);
});
it("keeps a steer in the current run, shows nothing for other conversations, and appends a reset", async () => {
const setup = chatSetup();
const gate = deferred();
addTool(
setup.registry,
defineTool({
name: "hold",
description: "Waits",
parameters: Type.Object({}),
execute: async () => {
await gate.promise;
return {};
},
}),
);
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("hold", {}, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const other = await harness.createConversation({ ownership: { kind: "ownerless" } }, context);
const history = await viewHistory(harness, root.id);
const input = await root.submit({ type: "input", content: "a" }, context);
await waitFor(() => slotsOf(history.live(), "c1").some((slot) => slot?.status === "running"));
const steer = await root.submit({ type: "input", content: "s", whenBusy: "steer" }, context);
const count = history.views.length;
await other.commit((tx) => tx.appendEntry(other.id, { kind: "note" }), context);
expect(history.views.length).toBe(count);
gate.resolve();
await input.wait(context);
await root.reset(undefined, context);
history.detach();
const states = history.live();
// One run: the steer joined it.
const runs = new Set(states.flatMap((state) => (state.run === undefined ? [] : [state.run.inputs[0]])));
expect([...runs]).toEqual([input.id]);
expect(states.some((state) => state.run?.inputs.includes(steer.id))).toBe(true);
// The reset is appended in place, with no run.
expect(history.views.at(-1)!.entries.at(-1)!.kind).toBe("pi.reset");
expect(states.at(-1)!.run).toBeUndefined();
await harness.close(context);
});
it("converts the streamed partial into one aborted answer entry when the run is aborted", async () => {
const setup = chatSetup({ tokensPerSecond: 100, tokenSize: { min: 1, max: 1 } });
setup.faux.setResponses([fauxAssistantMessage([fauxText("x".repeat(400))])]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const history = await viewHistory(harness, root.id);
const submission = await root.submit({ type: "input", content: "hi" }, context);
await waitFor(() => history.live().some((state) => state.generation?.message !== undefined));
const live = await harness.snapshot(LiveDoc, root.id, context);
await harness.abortTask(live!.run!.taskId, context);
await submission.wait(context);
history.detach();
const states = history.live();
const ended = states.findIndex((state, index) => index > 0 && state.run === undefined);
expect(states[ended - 1]!.generation?.message).toBeDefined();
const appended = history.views[ended]!.entries.slice(history.views[ended - 1]!.entries.length);
expect(appended.map((entry) => entry.kind)).toEqual(["pi.assistant"]);
expect(appended[0]!.model![0]).toMatchObject({ role: "assistant", stopReason: "aborted" });
expect(states[ended]!.runFailure).toMatchObject({ reason: "aborted" });
await harness.close(context);
});
it("applies later operations to an overflow replacement that holds an in-flight partial", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const partial = fauxAssistantMessage([fauxText("hel")]);
const watch = await root.watch(context, { min: 1, max: 1 });
const change = (edit: (live: Draft<LiveState>) => void) =>
root.commit(async (tx) => edit(await tx.doc(LiveDoc, root.id)), context);
await change((live) => {
live.generation = { attempt: 1, message: JSON.parse(JSON.stringify(partial)) };
});
// Past 10,000 pending operations: an append, a splice, and a `before` per commit.
for (let index = 0; index < 3400; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
}
const frames: Frame[] = [];
watch.start(async ({ value, ops }) => {
frames.push({ value, ops });
});
await drained();
expect(frames[0]!.ops).toEqual([["r", frames[0]!.value]]);
await change((live) => {
(live.generation!.message!.content[0] as { text: string }).text += "lo";
});
await drained();
const update = frames.at(-1)!;
expect(update.ops).toEqual([["a", ["docs", "pi.live", "generation", "message", "content", 0, "text"], "lo"]]);
const rebuilt = applyImmutable(frames[0]!.value, update.ops);
expect(liveOf(rebuilt).generation?.message?.content).toEqual([{ type: "text", text: "hello" }]);
await watch.stop();
await harness.close(context);
}, 30_000);
});
const gc = (() => {
setFlagsFromString("--expose-gc");
return runInNewContext("gc") as () => void;
})();
/** Bytes `run` allocates, counting what collections during it freed. */
async function allocatedBytes(run: () => Promise<void>): Promise<number> {
gc();
const profiler = new GCProfiler();
profiler.start();
const before = getHeapStatistics().used_heap_size;
await run();
const after = getHeapStatistics().used_heap_size;
let freed = 0;
for (const collection of profiler.stop().statistics) {
freed += collection.beforeGC.heapStatistics.usedHeapSize - collection.afterGC.heapStatistics.usedHeapSize;
}
return after - before + freed;
}
/** Heap bytes in use after a full collection. */
async function heapInUse(): Promise<number> {
for (let round = 0; round < 3; round++) {
await drained();
gc();
}
return getHeapStatistics().used_heap_size;
}
// #9265
describe("streamed partials", () => {
it("streams a tool-call partial as appends at its argument paths", async () => {
const setup = chatSetup({ tokensPerSecond: 500, tokenSize: { min: 1, max: 1 } });
setup.settings.progress = { partialIntervalMs: 20, outputIntervalMs: 100 };
const args = { path: "src/file.ts", content: "const a = 1;\n".repeat(150) };
setup.faux.setResponses([
fauxAssistantMessage([fauxToolCall("missing", args, { id: "c1" })], { stopReason: "toolUse" }),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const partials = partialsOf(harness);
const ops: Op[] = [];
const observation = await harness.observeConversation(
root.id,
{ advance: (change) => ops.push(...change.ops), end: () => {} },
context,
);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
observation.detach();
const MESSAGE = ["docs", "pi.live", "generation", "message"];
// Operations on the in-flight message, with their paths relative to it.
const onMessage = ops.flatMap((op) => {
const path = op[0] === "r" ? [] : (op[1] as readonly (string | number)[]);
return MESSAGE.every((segment, index) => path[index] === segment)
? [{ kind: op[0], path: path.slice(MESSAGE.length) }]
: [];
});
const argumentAppends = onMessage.filter((op) => op.kind === "a" && op.path[2] === "arguments");
expect(argumentAppends.length).toBeGreaterThan(10);
expect(argumentAppends.every((op) => ["path", "content"].includes(String(op.path.at(-1))))).toBe(true);
// The call is set whole only when a member appears (at most once per argument); growing strings are appends.
// With the raw argument text on the block, every commit resent the whole call.
const blockSets = onMessage.filter((op) => op.kind !== "a" && op.path.length === 2);
expect(blockSets.length).toBeLessThanOrEqual(2);
// The committed partials carry only the parsed arguments, never the raw argument text.
for (const partial of partials) {
for (const block of partial.content) {
expect(Object.keys(block).sort()).toEqual(["arguments", "id", "name", "type"]);
}
}
await harness.close(context);
});
it("commits a growing text and tool call at a cost that follows the new text, not the string lengths", async () => {
/** Bytes allocated while streaming one answer of `size` characters of text and as much tool-call content. */
const stream = async (size: number) => {
const setup = chatSetup({ tokensPerSecond: 64_000, tokenSize: { min: 64, max: 64 } });
setup.settings.progress = { partialIntervalMs: 1, outputIntervalMs: 100 };
// Keep `pi.live` cached between commits, as the default retention does. Evicted after each use (the suite's
// evicting run), every commit replays the document's delta chain: a cost of eviction, not of appending.
setup.settings.documentRetentionMs = 60_000;
const text = 'streamed text with "quotes" and \\ escapes\n'.repeat(size / 40);
setup.faux.setResponses([
fauxAssistantMessage(
[fauxText(text), fauxToolCall("missing", { path: "a.ts", content: text }, { id: "c1" })],
{
stopReason: "toolUse",
},
),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
let commits = 0;
observeCommits(harness, () => commits++);
const bytes = await allocatedBytes(async () => {
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
});
await harness.close(context);
return { bytes, commits };
};
const small = await stream(40_000);
const large = await stream(400_000);
expect(large.commits).toBeGreaterThan(small.commits * 5);
// Per commit, the cost stays flat: comparing the whole strings on every commit made it grow with their length.
expect(large.bytes / large.commits / (small.bytes / small.commits)).toBeLessThan(2);
}, 60_000);
it("keeps a watcher's memory proportional to a long streamed text and tool call, not to their commits", async () => {
const setup = chatSetup({ tokensPerSecond: 16_000, tokenSize: { min: 16, max: 16 } });
setup.settings.progress = { partialIntervalMs: 1, outputIntervalMs: 100 };
const line = (index: number) => `line ${index}: some streamed source text with "quotes" and \\ escapes\n`;
const text = Array.from({ length: 2_000 }, (_, index) => line(index)).join("");
const content = Array.from({ length: 2_000 }, (_, index) => line(index)).join("");
setup.faux.setResponses([
fauxAssistantMessage([fauxText(text), fauxToolCall("missing", { path: "a.ts", content }, { id: "c1" })], {
stopReason: "toolUse",
}),
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const { watch, frames } = await listen(root, false);
const before = await heapInUse();
const submission = await root.submit({ type: "input", content: "go" }, context);
const partialContent = async () =>
(await harness.snapshot(LiveDoc, root.id, context))?.generation?.message?.content;
const shown = async (index: number): Promise<number> => {
const block = (await partialContent())?.[index];
if (block?.type === "text") return block.text.length;
return block?.type === "toolCall" ? String(block.arguments.content ?? "").length : 0;
};
const retained: number[] = [];
// Measure while the text, then the tool call, is almost complete: the watcher's view holds the most then.
await waitFor(async () => (await shown(0)) > text.length * 0.95, 30_000);
retained.push((await heapInUse()) - before);
await waitFor(async () => (await shown(1)) > content.length * 0.95, 30_000);
retained.push((await heapInUse()) - before);
await submission.wait(context);
const commits = frames.length;
expect(commits).toBeGreaterThan(500);
// The two 140 KB strings, the watcher's recorded frames and the rest of the stream: about 3 MB. Sliced append ops
// made the watcher's view hold a copy of the text per commit: well over 100 MB.
for (const bytes of retained) expect(bytes).toBeLessThan(20_000_000);
await watch.stop();
await harness.close(context);
}, 60_000);
});
packages/durable/test/harness-view.test.ts
+264 -33
View File
@@ -1,10 +1,17 @@
import { mkdtemp, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { setFlagsFromString } from "node:v8";
import { runInNewContext } from "node:vm";
import { applyImmutable, type Op } from "@earendil-works/chord/delta";
import { type AssistantMessage, type FauxResponseStep, fauxAssistantMessage, fauxText } from "@earendil-works/pi-ai";
import {
AgentDoc,
type Conversation,
type ConversationView,
type ConversationWindow,
defineDoc,
type EntryRecord,
type Harness,
InboxDoc,
LiveDoc,
@@ -13,6 +20,7 @@ import {
UsageDoc,
} from "@earendil-works/pi-durable";
import { describe, expect, it } from "vitest";
import { openNodeSqliteStorage } from "../src/storage/sqlite/node.ts";
import { allEntries, chatSetup, openChat, waitFor } from "./chat-support.ts";
import { context } from "./session-support.ts";
import { aborted, deferred } from "./task-support.ts";
@@ -22,8 +30,8 @@ type Frame = { value: ConversationView; ops: readonly Op[] };
const MOUNTED = ["pi.agent", "pi.inbox", "pi.live", "pi.provider", "pi.usage"];
/** Start a watch of `conversation` that records its acquisition revision and every delivered frame. */
async function record(conversation: Conversation) {
const watch = await conversation.watch(context);
async function record(conversation: Conversation, window?: ConversationWindow) {
const watch = await conversation.watch(context, window);
const initial = watch.value;
const frames: Frame[] = [];
watch.start(async ({ value, ops }) => {
@@ -33,21 +41,31 @@ async function record(conversation: Conversation) {
}
/** A freshly built view of `conversation`, for comparing with an advanced one. */
async function fresh(conversation: Conversation): Promise<ConversationView> {
const watch = await conversation.watch(context);
async function fresh(conversation: Conversation, window?: ConversationWindow): Promise<ConversationView> {
const watch = await conversation.watch(context, window);
const value = watch.value;
await watch.stop();
return value;
}
/** The view as committed state defines it, read without any mount: the active entries and the built-in documents. */
async function committed(harness: Harness, conversation: Conversation, record: ConversationView["conversation"]) {
/**
* The view as committed state defines it, read without any mount: the newest `size` entries, the entry before them, and
* the built-in documents.
*/
async function committed(
harness: Harness,
conversation: Conversation,
record: ConversationView["conversation"],
size: number,
) {
const docs: Record<string, unknown> = {};
for (const token of [AgentDoc, LiveDoc, InboxDoc, ProviderDoc, UsageDoc] as const) {
const value = await harness.snapshot(token as typeof LiveDoc, conversation.id, context);
if (value !== undefined) docs[token.definition.kind] = value;
}
return { conversation: record, entries: (await conversation.context(context)).entries, docs };
const newest = [...(await conversation.entries({}, size + 1, undefined, context)).items];
const before = newest.length > size ? newest.pop()!.id : undefined;
return { conversation: record, entries: newest.reverse(), ...(before === undefined ? {} : { before }), docs };
}
/** Replay every frame's operations from `initial`, checking each delivered revision on the way. */
@@ -86,7 +104,7 @@ function touches(harness: Harness, conversation: Conversation): { count: number
}
describe("conversation view", () => {
it("hydrates the active entries and the built-in documents", async () => {
it("hydrates the newest entries and the built-in documents", async () => {
const setup = chatSetup();
setup.faux.setResponses([answer("hello")]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
@@ -94,6 +112,7 @@ describe("conversation view", () => {
const view = await fresh(root);
expect(view.conversation).toEqual({ id: root.id });
expect(view.entries).toEqual(await allEntries(root));
expect(view.before).toBeUndefined();
expect(Object.keys(view.docs).sort()).toEqual(MOUNTED);
expect(view.docs["pi.live"]).toEqual({});
expect(view.docs["pi.inbox"]).toEqual({ items: [] });
@@ -118,7 +137,8 @@ describe("conversation view", () => {
await harness.waitForIdle(context);
await drained();
expect(frames).toHaveLength(touching.count);
expect(replay(initial, frames)).toEqual(await committed(harness, root, initial.conversation));
const last = replay(initial, frames);
expect(last).toEqual(await committed(harness, root, initial.conversation, last.entries.length));
expect(frames[0]!.ops).toContainEqual(["p", ["entries"], 0, 0, [expect.objectContaining({ kind: "pi.user" })]]);
// Document operations keep their exact shape under the mount path.
expect(frames.flatMap((frame) => frame.ops)).toContainEqual([
@@ -148,7 +168,7 @@ describe("conversation view", () => {
await harness.close(context);
});
it("cuts the entries at a head marker, keeping the entries from its head", async () => {
it("keeps head markers in place: a summary and a reset are appended entries", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const note = (kind: string) => root.commit((tx) => tx.appendEntry(root.id, { kind }), context);
await note("a");
@@ -160,28 +180,21 @@ describe("conversation view", () => {
await root.reset(undefined, context);
await drained();
const kinds = frames.map((frame) => frame.value.entries.map((entry) => entry.kind));
expect(kinds).toEqual([["summary", "b", "c"], ["summary", "b", "c", "d"], ["pi.reset"]]);
expect(frames[0]!.ops).toEqual([["p", ["entries"], 0, 1, [summary]]]);
expect(replay(initial, frames)).toEqual(await committed(harness, root, initial.conversation));
expect(kinds).toEqual([
["a", "b", "c", "summary"],
["a", "b", "c", "summary", "d"],
["a", "b", "c", "summary", "d", "pi.reset"],
]);
expect(frames[0]!.ops).toEqual([["p", ["entries"], 3, 0, [summary]]]);
// The summary's `head` tells a renderer where the entries it summarizes end.
expect(frames[0]!.value.entries[3]!.head).toBe(b.id);
const last = replay(initial, frames);
expect(last).toEqual(await committed(harness, root, initial.conversation, 6));
await stop();
await harness.close(context);
});
it("keeps only mounted entries for a raw head write that targets before the active range", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const old = await root.commit((tx) => tx.appendEntry(root.id, { kind: "old" }), context);
await root.reset(undefined, context);
const { frames, stop } = await record(root);
await root.commit((tx) => tx.appendEntry(root.id, { kind: "summary", head: old.id }), context);
await drained();
// Model context now starts at `old` again, but the mount never held it (spec §12); a rebuilt mount shows it.
expect(frames[0]!.value.entries.map((entry) => entry.kind)).toEqual(["summary"]);
await stop();
expect((await fresh(root)).entries.map((entry) => entry.kind)).toEqual(["summary", "old"]);
await harness.close(context);
});
it("cuts a fork's view into its inherited entries", async () => {
it("shows a fork's inherited entries in ID order, its head markers in place", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const a = await root.commit((tx) => tx.appendEntry(root.id, { kind: "a" }), context);
const b = await root.commit((tx) => tx.appendEntry(root.id, { kind: "b" }), context);
@@ -190,8 +203,8 @@ describe("conversation view", () => {
await fork.commit((tx) => tx.appendEntry(fork.id, { kind: "summary", head: b.id }), context);
await drained();
expect(initial.entries.map((entry) => entry.id)).toEqual([a.id, b.id]);
expect(frames[0]!.value.entries.map((entry) => entry.kind)).toEqual(["summary", "b"]);
expect(frames[0]!.value).toEqual(await committed(harness, fork, initial.conversation));
expect(frames[0]!.value.entries.map((entry) => entry.kind)).toEqual(["a", "b", "summary"]);
expect(frames[0]!.value).toEqual(await committed(harness, fork, initial.conversation, 3));
await stop();
await harness.close(context);
});
@@ -243,7 +256,8 @@ describe("conversation view", () => {
expect(frames).toHaveLength(1);
expect(frames[0]!.ops[0]?.[0]).not.toBe("r");
expect(applyImmutable(initial, frames[0]!.ops)).toEqual(frames[0]!.value);
expect(frames[0]!.value.entries).toHaveLength(101);
// The default window took 100 appends, then dropped its oldest 51 at the 101st, down to 50.
expect(frames[0]!.value).toEqual(await committed(harness, root, initial.conversation, 50));
await watch.stop();
await harness.close(context);
});
@@ -309,7 +323,9 @@ describe("conversation view", () => {
const failing = await root.watch(context);
const { frames, stop } = await record(root);
const shared = await root.watch(context);
expect(failing.value).toBe(shared.value);
// Each observer has its own window; the documents are the mount's.
expect(failing.value).toEqual(shared.value);
expect(failing.value.docs).toBe(shared.value.docs);
await shared.stop();
failing.start(async () => {
throw new Error("listener failed");
@@ -347,8 +363,223 @@ describe("conversation view", () => {
await drained();
expect(frames).toHaveLength(1);
expect(frames[0]!.value.entries.map((entry) => entry.kind)).toEqual(["a", "b"]);
expect(replay(initial, frames)).toEqual(await committed(harness, root, initial.conversation));
expect(replay(initial, frames)).toEqual(await committed(harness, root, initial.conversation, 2));
await stop();
await harness.close(context);
});
});
/** Append one entry of `kind` per commit; returns the entries. */
async function notes(conversation: Conversation, count: number, kind = "note"): Promise<EntryRecord[]> {
const entries: EntryRecord[] = [];
for (let index = 0; index < count; index++) {
entries.push(await conversation.commit((tx) => tx.appendEntry(conversation.id, { kind, data: index }), context));
}
return entries;
}
const ids = (entries: readonly EntryRecord[]) => entries.map((entry) => entry.id);
const gc = (() => {
setFlagsFromString("--expose-gc");
return runInNewContext("gc") as () => void;
})();
describe("conversation view window", () => {
it("starts with the newest `min` entries and the entry before them, or the whole short transcript", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 10);
const view = await fresh(root, { min: 4 });
expect(ids(view.entries)).toEqual(ids(written.slice(6)));
expect(view.before).toBe(written[5]!.id);
// The default window: the newest 50, here all ten, starting the transcript.
const all = await fresh(root);
expect(ids(all.entries)).toEqual(ids(written));
expect(all.before).toBeUndefined();
expect("before" in all).toBe(false);
await harness.close(context);
});
it("takes appends up to `max`, then drops its oldest in one splice down to `min`", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 4);
const { initial, frames, stop } = await record(root, { min: 3, max: 5 });
expect(ids(initial.entries)).toEqual(ids(written.slice(1)));
const appended = await notes(root, 3);
await drained();
expect(frames.map((frame) => frame.value.entries.length)).toEqual([4, 5, 3]);
const [e5, e6, e7] = appended as [EntryRecord, EntryRecord, EntryRecord];
expect(frames[0]!.ops).toEqual([["p", ["entries"], 3, 0, [e5]]]);
expect(frames[2]!.ops).toEqual([
["p", ["entries"], 5, 0, [e7]],
["p", ["entries"], 0, 3, []],
["s", ["before"], written[3]!.id],
]);
expect(ids(frames[2]!.value.entries)).toEqual(ids([e5, e6, e7]));
expect(frames[2]!.value.before).toBe(written[3]!.id);
const last = replay(initial, frames);
expect(last).toEqual(await committed(harness, root, initial.conversation, 3));
// Entries that stay keep their identity: a splice moves them, it never copies them.
expect(frames[2]!.value.entries[0]).toBe(frames[1]!.value.entries[3]);
await stop();
await harness.close(context);
});
it("keeps the newest `min` of a commit that appends more than `max` at once", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
await notes(root, 2);
const { initial, frames, stop } = await record(root, { min: 2, max: 3 });
const burst = await root.commit(async (tx) => {
const entries: EntryRecord[] = [];
for (let index = 0; index < 5; index++) entries.push(await tx.appendEntry(root.id, { kind: "burst" }));
return entries;
}, context);
await drained();
expect(frames).toHaveLength(1);
expect(ids(frames[0]!.value.entries)).toEqual(ids(burst.slice(3)));
expect(frames[0]!.value.before).toBe(burst[2]!.id);
expect(replay(initial, frames)).toEqual(await committed(harness, root, initial.conversation, 2));
await stop();
await harness.close(context);
});
it("starts at `from` when the entries since fit in `max`, otherwise at the newest `min`", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 30);
const window = { min: 5, max: 10 };
// Ten entries since `from`: all of them, with the entry before `from`.
const within = await fresh(root, { ...window, from: written[20]!.id });
expect(ids(within.entries)).toEqual(ids(written.slice(20)));
expect(within.before).toBe(written[19]!.id);
// Fewer than `min` since `from`: only those.
const short = await fresh(root, { ...window, from: written[28]!.id });
expect(ids(short.entries)).toEqual(ids(written.slice(28)));
expect(short.before).toBe(written[27]!.id);
// Resuming after the newest entry: an empty window right after it.
const resumed = await fresh(root, { ...window, from: (written[29]!.id + 1) as EntryRecord["id"] });
expect(resumed.entries).toEqual([]);
expect(resumed.before).toBe(written[29]!.id);
// Eleven since `from`: more than `max`, so the newest `min`; `before` shows the client a gap.
const beyond = await fresh(root, { ...window, from: written[19]!.id });
expect(ids(beyond.entries)).toEqual(ids(written.slice(25)));
expect(beyond.before).toBe(written[24]!.id);
// From the transcript's start.
const whole = await fresh(root, { min: 40, from: written[0]!.id });
expect(ids(whole.entries)).toEqual(ids(written));
expect(whole.before).toBeUndefined();
await harness.close(context);
});
it("advances a resumed window by appends from its start", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 5);
const { initial, frames, stop } = await record(root, { from: (written[4]!.id + 1) as EntryRecord["id"] });
const later = await notes(root, 2);
await drained();
expect(ids(frames.at(-1)!.value.entries)).toEqual(ids(later));
expect(frames.at(-1)!.value.before).toBe(written[4]!.id);
expect(replay(initial, frames)).toEqual(frames.at(-1)!.value);
await stop();
await harness.close(context);
});
it("uses `from` only for the first value: a `from` past the newest entry starts empty and takes every append", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 3);
const from = (written[2]!.id + 1000) as EntryRecord["id"];
const { initial, frames, stop } = await record(root, { from });
expect(initial.entries).toEqual([]);
expect(initial.before).toBe(written[2]!.id);
// An entry below `from` is still appended: the window continues from its attachment.
const [next] = await notes(root, 1);
expect(next!.id).toBeLessThan(from);
await drained();
expect(frames.at(-1)!.value.entries).toEqual([next]);
expect(frames.at(-1)!.value.before).toBe(written[2]!.id);
await stop();
await harness.close(context);
});
it("rejects window bounds that are not integers or put `max` below `min`", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
await expect(root.watch(context, { min: 0 })).rejects.toThrow(RangeError);
await expect(root.watch(context, { min: 1.5 })).rejects.toThrow(RangeError);
await expect(root.watch(context, { min: 10, max: 9 })).rejects.toThrow(RangeError);
await expect(root.watch(context, { from: Number.NaN as EntryRecord["id"] })).rejects.toThrow(RangeError);
const recorder = { advance: () => {}, end: () => {} };
await expect(harness.observeConversation(root.id, recorder, context, { max: 0 })).rejects.toThrow(RangeError);
// `max` may equal `min`.
await (await root.watch(context, { min: 3, max: 3 })).stop();
await harness.close(context);
});
it("gives each observer its own window over one mount", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const written = await notes(root, 6);
const changes: ConversationView[] = [];
const observation = await harness.observeConversation(
root.id,
{ advance: ({ value }) => changes.push(value), end: () => {} },
context,
{ min: 2, max: 3 },
);
const wide = await record(root, { min: 5 });
expect(ids(observation.value.entries)).toEqual(ids(written.slice(4)));
expect(ids(wide.initial.entries)).toEqual(ids(written.slice(1)));
const [next] = await notes(root, 1);
await drained();
expect(ids(changes[0]!.entries)).toEqual(ids([...written.slice(4), next!]));
expect(ids(wide.frames[0]!.value.entries)).toEqual(ids([...written.slice(1), next!]));
expect(changes[0]!.docs).toBe(wide.frames[0]!.value.docs);
observation.detach();
await wide.stop();
await harness.close(context);
});
it("bounds an overflow replacement by `max`", async () => {
const { harness, root } = await openChat(new MemoryStorage(), chatSetup());
const watch = await root.watch(context, { min: 5, max: 6 });
// Past 10,000 pending operations, the next call is one root replacement: here an append per commit, and a splice
// and a `before` every other commit.
for (let index = 0; index < 6000; index++) {
await root.commit((tx) => tx.appendEntry(root.id, { kind: "note" }), context);
}
const frames: Frame[] = [];
watch.start(async ({ value, ops }) => {
frames.push({ value, ops });
});
await drained();
expect(frames).toHaveLength(1);
expect(frames[0]!.ops).toEqual([["r", frames[0]!.value]]);
expect(frames[0]!.value.entries.length).toBeLessThanOrEqual(6);
await watch.stop();
await harness.close(context);
}, 30_000);
it("lets a long-lived watch drop the entries that left its window", async () => {
// On SQLite, which keeps no entry objects, so only the watch can hold the first one.
const directory = await mkdtemp(join(tmpdir(), "pi-durable-view-"));
const storage = await openNodeSqliteStorage(join(directory, "session.sqlite"));
const { harness, root } = await openChat(storage, chatSetup());
const watch = await root.watch(context, { min: 5, max: 10 });
let first: WeakRef<EntryRecord> | undefined;
let count = 0;
watch.start(async ({ value }) => {
count = value.entries.length;
if (first === undefined && value.entries[0] !== undefined) first = new WeakRef(value.entries[0]);
});
await notes(root, 1);
await waitFor(() => first !== undefined);
await notes(root, 200);
await drained();
expect(count).toBeLessThanOrEqual(10);
for (let round = 0; round < 3 && first!.deref() !== undefined; round++) {
await drained();
gc();
}
expect(first!.deref()).toBeUndefined();
await watch.stop();
await harness.close(context);
await rm(directory, { recursive: true, force: true });
});
});
packages/durable/test/live-support.ts
+110
View File
@@ -0,0 +1,110 @@
// What tests of renderers' state read: every `pi.live` value of a conversation, and the results of nested calls.
import {
type ConversationId,
type ConversationView,
type Harness,
type LiveState,
NestedCallDoc,
type NestedCallState,
type NestedToolExecutionResult,
type NestedToolSlot,
type TaskGraph,
type TaskGraphNode,
type TaskId,
type ToolSlot,
} from "@earendil-works/pi-durable";
import { context, documentChanges, observeCommits } from "./session-support.ts";
/**
* Every view of `conversationId` from now on, one per commit that changes it, unfolded: a synchronous observer, so no
* transition is skipped. `live()` lists their `pi.live` values.
*/
export async function viewHistory(harness: Harness, conversationId: ConversationId) {
const views: ConversationView[] = [];
const observation = await harness.observeConversation(
conversationId,
{ advance: ({ value }) => views.push(value), end: () => {} },
context,
);
views.push(observation.value);
return {
views,
live: (): LiveState[] => views.map((view) => (view.docs["pi.live"] ?? {}) as LiveState),
detach: () => observation.detach(),
};
}
/** A model-issued or nested slot, read through the fields of both. */
type Slot = ToolSlot & Partial<Pick<NestedToolSlot, "parentCallId" | "parentTaskId" | "arguments" | "summary">>;
/** The slots of tool call `callId`, model-issued or nested, in each `pi.live` value; `undefined` where it is absent. */
export function slotsOf(history: readonly LiveState[], callId: string): (Slot | undefined)[] {
return history.map(
(live) => [...(live.tools ?? []), ...(live.nestedTools ?? [])].find((slot) => slot.callId === callId) as Slot,
);
}
/** The index of the first value where call `callId` ended: its slot is done, or gone after it was listed. */
export function endedAt(history: readonly LiveState[], callId: string): number {
const slots = slotsOf(history, callId);
const listed = slots.findIndex((slot) => slot !== undefined);
if (listed < 0) return -1;
return slots.findIndex((slot, index) => index >= listed && (slot === undefined || slot.status === "done"));
}
/** The calls whose slot went `running`, in the order they started. */
export function startOrder(history: readonly LiveState[]): string[] {
const started: string[] = [];
for (const live of history) {
for (const slot of [...(live.tools ?? []), ...(live.nestedTools ?? [])]) {
if (slot.status === "running" && !started.includes(slot.callId)) started.push(slot.callId);
}
}
return started;
}
/**
* The result of every nested call from now on, by its tool task, read from the `NestedCallDoc` its terminal commit
* writes (spec §7.3), the place a client reads a live caller's nested results from.
*/
export function nestedResults(harness: Harness): Map<TaskId, NestedToolExecutionResult> {
const results = new Map<TaskId, NestedToolExecutionResult>();
observeCommits(harness, (publication) => {
for (const change of documentChanges(publication)) {
if (change.record.kind !== NestedCallDoc.definition.kind || change.value === null) continue;
const state = change.value as NestedCallState;
if (state.taskId !== undefined && state.result !== undefined) {
results.set(state.taskId, state.result as unknown as NestedToolExecutionResult);
}
}
});
return results;
}
/** The result of nested call `callId`, found by the task its slot names. */
export function nestedResultOf(
history: readonly LiveState[],
results: ReadonlyMap<TaskId, NestedToolExecutionResult>,
callId: string,
): NestedToolExecutionResult | undefined {
const slot = slotsOf(history, callId).find((each) => each?.taskId !== undefined);
return slot?.taskId === undefined ? undefined : results.get(slot.taskId);
}
/** Every task graph from now on, one per change, unfolded. */
export async function graphHistory(harness: Harness) {
const graphs: TaskGraph[] = [];
const observation = await harness.observeTaskGraph(
{ advance: ({ value }) => graphs.push(value), end: () => {} },
context,
);
graphs.push(observation.value);
return { graphs, detach: () => observation.detach() };
}
/** The live `pi.generation` nodes of `graph`, in task ID order. */
export function generations(graph: TaskGraph): TaskGraphNode[] {
return Object.values(graph.tasks)
.filter((node) => node.kind === "pi.generation")
.sort((a, b) => a.id - b.id);
}
packages/durable/test/session-document-retention.test.ts
+12 -13
View File
@@ -3,21 +3,21 @@
import type { Context } from "@earendil-works/chord";
import { fauxAssistantMessage, fauxText, fauxToolCall, Type } from "@earendil-works/pi-ai";
import {
type AgentEvent,
type ConversationId,
defineDoc,
defineDocFamily,
defineTool,
type HarnessSettings,
type LiveState,
MemoryStorage,
type Seq,
type StorageWrite,
watchEvents,
} from "@earendil-works/pi-durable";
import { describe, expect, it, onTestFinished } from "vitest";
import { SessionImpl, setDefaultDocumentRetention } from "../src/session/session.ts";
import { chatSetup, openChat } from "./chat-support.ts";
import { addTool } from "./harness-support.ts";
import { viewHistory } from "./live-support.ts";
import { ControlledStorage, context, createConversation } from "./session-support.ts";
import { deferred, eventually } from "./task-support.ts";
@@ -509,7 +509,7 @@ describe("document cache retention", () => {
await harness.close(context);
});
it("gives the same events with documents evicted right after use as with documents kept", async () => {
it("gives the same views with documents evicted right after use as with documents kept", async () => {
const run = async (documentRetentionMs: number): Promise<string[]> => {
const setup = chatSetup();
setup.settings.documentRetentionMs = documentRetentionMs;
@@ -534,19 +534,18 @@ describe("document cache retention", () => {
fauxAssistantMessage([fauxText("done")]),
]);
const { harness, root } = await openChat(new MemoryStorage(), setup);
const stream = await watchEvents(harness, root.id, context);
const events: AgentEvent[] = [];
stream.start(async ({ value: batch }) => {
events.push(...batch);
});
const history = await viewHistory(harness, root.id);
await (await root.submit({ type: "input", content: "go" }, context)).wait(context);
await harness.waitForIdle(context);
await stream.stop();
history.detach();
await harness.close(context);
// Ids and times differ between runs; the sequence of event types and tool outputs must not.
return events.map((event) =>
event.type === "tool_execution_update" ? `update:${JSON.stringify(event.output)}` : event.type,
);
// Ids and times differ between runs; the sequence of entries, run states, and tool outputs must not.
return history.views.map((view) => {
const live = (view.docs["pi.live"] ?? {}) as LiveState;
const tools = (live.tools ?? []).map((slot) => `${slot.callId}:${slot.status}:${slot.output ?? ""}`);
const kinds = view.entries.map((entry) => entry.kind).join(",");
return `${kinds} run:${live.run !== undefined} generation:${live.generation !== undefined} ${tools.join(" ")}`;
});
};
expect(await run(0)).toEqual(await run(600_000));
});
packages/durable/test/spec-usage.test.ts
+20
View File
@@ -6,7 +6,9 @@
import type { Context, Draft } from "@earendil-works/chord";
import { type AssistantMessage, type Models, type ToolCall, Type } from "@earendil-works/pi-ai";
import {
type Conversation,
type ConversationId,
ConversationTranscript,
configure,
createRegistry,
defineDoc,
@@ -27,6 +29,7 @@ import {
type TaskId,
type ToolExecutionApi,
ToolTask,
type TranscriptView,
type Tx,
type UserInput,
wrapTool,
@@ -75,6 +78,8 @@ declare const manager: SettingsManager;
declare const Anchor: ReturnType<typeof defineTask<null, { phase: "hold" }, null>>;
declare const Reporter: ReturnType<typeof defineTask<null, { phase: "report" }, null>>;
declare const subagentTool: ReturnType<typeof createReadTool>;
declare const conversation: Conversation;
declare function render(view: TranscriptView): void;
/** Never called: the declared names above have no values. */
function examples() {
@@ -351,6 +356,21 @@ function examples() {
});
},
// Section 9.3: a client's transcript over the windowed view.
transcript: async () => {
const transcript = new ConversationTranscript((query, limit, ctx) =>
conversation.entries(query, limit, undefined, ctx),
);
const watch = await conversation.watch(context);
render(transcript.merge(watch.value));
watch.start(async ({ value }) => render(transcript.merge(value)));
// A marker scrolled into view:
render(await transcript.loadEarlier(context));
// Reconnect after a drop:
const resumed = await conversation.watch(context, transcript.resume);
return resumed;
},
// Section 4: table reads before the first table write; documents stay usable.
tableRules: async () => {
await session.commit(async (tx) => {