mirror of
https://github.com/vercel-labs/fx.git
synced 2026-10-28 10:15:17 +08:00
Merge remote-tracking branch 'origin/main' into fazxes/turn-end-saved
# Conflicts: # sdk/fx-harness.js # sdk/fx-sdk.js # sdk/tests/test-types.mjs
This commit is contained in:
80 files changed
+8059
-386
No files matched your search
@@ -390,6 +390,10 @@ jobs:
|
||||
bun tests/tla/drivers/drive.mjs cutoff --walks 20 --seed 5
|
||||
bun tests/tla/drivers/drive.mjs leased --walks 40 --seed 12
|
||||
bun tests/tla/drivers/drive.mjs queue --walks 40 --seed 4
|
||||
bun tests/tla/drivers/drive-intake.mjs --walks 60 --seed 1
|
||||
bun tests/tla/drivers/drive-intake.mjs --walks 30 --seed 2 --durability world
|
||||
bun tests/tla/drivers/drive-images.mjs --walks 40 --seed 1
|
||||
bun tests/tla/drivers/drive-images.mjs --walks 20 --seed 2 --backend wasm
|
||||
bun tests/tla/drivers/drive-agents.mjs --walks 60 --seed 11
|
||||
bun tests/tla/drivers/drive-tenants.mjs --walks 60 --seed 11
|
||||
bun tests/tla/drivers/drive-steer.mjs ideal --walks 20 --seed 2
|
||||
|
||||
+4
-1
@@ -27,6 +27,8 @@ The SDK has two WebAssembly surfaces and one shared JavaScript host layer:
|
||||
| A durability over a World the app supplies; `local()` and `vercel()` are built on it | `sdk/durable/world.mjs` |
|
||||
| TypeScript declarations for every entry point | `sdk/types/`, checked by `sdk/tests/test-types.mjs` |
|
||||
| Checkpoint format, including the libfx version, tool set, and model each checkpoint records | `src/core/agent/runtime/checkpoint.zig` |
|
||||
| Images a session saves once, by name: naming in journal events and checkpoints, and the two-phase open | `src/core/session/session_codec.zig`, `src/acp/libfx_journal.zig`, and `src/acp/server.zig` |
|
||||
| A session's image store (the internal `persistenceImages` key) and storing before sending | `sdk/fx-sdk.js`; a durable session's images are World streams in `sdk/durable.js` |
|
||||
| Commands required by CI | `.github/workflows/ci.yml` |
|
||||
|
||||
Do not treat the demos or this file as the implementation contract. When prose and behavior disagree, establish the current behavior from code and focused tests before changing either.
|
||||
@@ -52,6 +54,7 @@ Do not treat the demos or this file as the implementation contract. When prose a
|
||||
| Public export, option, event, or result shape | Update `sdk/types/` and run `node sdk/tests/test-types.mjs` after `npm ci --prefix sdk/node` |
|
||||
| `createFxAgent()`, ACP translation, core session persistence, streaming, or cancellation | Core build, core Node tests, and the browser test when browser behavior is involved |
|
||||
| Live Gateway request or model-catalog translation | Core tests plus the opt-in live smoke test when a credential is available |
|
||||
| How a session saves images: naming, the image store, staging at open | Core and terminal builds, `bun sdk/tests/test-journal.mjs native wasm` and `wasm native`, `test-checkpoint.mjs` both ways, the Zig tests of `checkpoint.zig` and `journal.zig`, then `tests/tla/sdk/check.sh` and `bun tests/tla/drivers/drive-images.mjs` with `--backend native` and `wasm`, which walks the graph of `ImageStore.tla`; change the spec with the code. For durable sessions, also the durable sessions row |
|
||||
| Terminal adapter, input encoding, resize, cleanup, config, or prompt history | Terminal build and the headless terminal suite |
|
||||
| Terminal session persistence or browser device login | Terminal build plus `sdk/tests/test-term-session-resume.mjs` or `sdk/tests/test-term-login.mjs` |
|
||||
| Browser workspace metadata, permissions, execution, limits, or cancellation | Terminal build plus `sdk/node/test-term-workspace.mjs` |
|
||||
@@ -59,7 +62,7 @@ Do not treat the demos or this file as the implementation contract. When prose a
|
||||
| `encodeXtermKeyEvent()` or `xtermAdapter()` only | `sdk/node/test-xterm-adapter.mjs` |
|
||||
| Core debugger query behavior or automation state | `sdk/tests/test-core-browser.mjs` |
|
||||
| Terminal demo asset references, integrity, or cache policy | Package into a fresh temporary directory and inspect the generated HTML, manifest, and headers |
|
||||
| Durable sessions: leases, fencing, takeovers, tool reruns, the UI stream | `sdk/tests/test-durable-core.mjs` on memory, local, world, and queue (the core with a scripted harness; world is an app-built World passed to `world()`, queue one whose queue delivers as Vercel Queues does, with `queueLeases`), `sdk/tests/test-durable.mjs` on memory, local, and world, then `tests/tla/sdk/check.sh` and `bun tests/tla/drivers/drive.mjs send`, `lookup`, `write`, `cutoff` (deadline cut-offs up to their bound), `leased` (a World without liveness, where each worker renews a short lease), and `queue` (a World whose queue shows whether a worker lives, so leases are held without renewals), which walk the real code along the TLC state graphs of `DurableSession.tla`; change the spec with the code. For agent names and routing, also `bun tests/tla/drivers/drive-agents.mjs`, which walks `AgentRouting.tla`'s graph with each process in a worker thread of its own. For what a delivery may run, on whose agent and store, and input limits, also `bun tests/tla/drivers/drive-tenants.mjs`, which walks `TenantIsolation.tla`'s graph with four tenants and a forger sharing one route. With a Vercel access token, also run both against Vercel's World: `test-durable.mjs vercel native` and `drive.mjs --durability vercel`, configured as `sdk/tests/vercel-storage.mjs` describes |
|
||||
| Durable sessions: leases, fencing, takeovers, tool reruns, the UI stream | `sdk/tests/test-durable-core.mjs` on memory, local, world, and queue (the core with a scripted harness; world is an app-built World passed to `world()`, queue one whose queue delivers as Vercel Queues does, with `queueLeases`), `sdk/tests/test-durable.mjs` on memory, local, and world, then `tests/tla/sdk/check.sh` and `bun tests/tla/drivers/drive.mjs send`, `lookup`, `write`, `cutoff` (deadline cut-offs up to their bound), `leased` (a World without liveness, where each worker renews a short lease), and `queue` (a World whose queue shows whether a worker lives, so leases are held without renewals), which walk the real code along the TLC state graphs of `DurableSession.tla`; change the spec with the code. For how a session object takes prompts in (images and their image writes, cancels, and closing the agent), also run `bun tests/tla/drivers/drive-intake.mjs` on memory and `--durability world`, which walks the graph of `PromptIntake.tla`. For agent names and routing, also `bun tests/tla/drivers/drive-agents.mjs`, which walks `AgentRouting.tla`'s graph with each process in a worker thread of its own. For what a delivery may run, on whose agent and store, and input limits, also `bun tests/tla/drivers/drive-tenants.mjs`, which walks `TenantIsolation.tla`'s graph with four tenants and a forger sharing one route. With a Vercel access token, also run both against Vercel's World: `test-durable.mjs vercel native` and `drive.mjs --durability vercel`, configured as `sdk/tests/vercel-storage.mjs` describes |
|
||||
| Steering: interrupts, waits during tools, joins, hand-backs on cancel, image handoffs, steer ids | The libfx steering tests in `src/acp/libfx_steering.zig`, `sdk/tests/test-agent-steering.mjs` and `sdk/tests/test-durable.mjs` on native and wasm, then `tests/tla/sdk/check.sh` and `bun tests/tla/drivers/drive-steer.mjs ideal` and `faults`, with `--backend wasm` too, which walk the real code along the TLC state graphs of `DurableSteer.tla`; change the spec with the code |
|
||||
| Supported public behavior or setup | Update `sdk/README.md` in the same change |
|
||||
|
||||
|
||||
+3
-2
@@ -55,9 +55,10 @@ agent.session(sessionId).prompt("Start with the schema.");
|
||||
|
||||
- **One turn at a time.** A prompt sent while a turn runs waits behind it, even when it reaches a different server.
|
||||
- **Turns finish on their own.** A turn runs to the end whether or not anyone reads its events, and a turn nobody reads holds no connection open: its view starts reading only once you use its events, `readable` or `result`. Stop it with `session.cancel()`, or with the `signal` you passed to `prompt()`.
|
||||
- **Steering.** `session.steer(input)` steers the running turn the way the fx CLI does. While the model streams a reply, the steer cuts it short and the turn continues with it. While a tool runs, the steer waits for the tool. A steer sent while a turn waits to start joins that turn, and when no turn runs or waits, the steer becomes the next turn. A steer with images can't join a turn, so the running turn ends at its next safe point and the steer runs next.
|
||||
- **Steering.** `session.steer(input)` steers the running turn the way the fx CLI does. While the model streams a reply, the steer cuts it short and the turn continues with it. While a tool runs, the steer waits for the tool. A steer sent while a turn waits to start joins that turn, and when no turn runs or waits, the steer becomes the next turn. A steer with images can't join a turn, so the running turn ends at its next safe point and the steer runs next. Its images are saved once with the session before the steer is sent, as a prompt's are, and a steer whose image can't be saved throws the store's error.
|
||||
- **Retries are safe.** Give a prompt or a steer a `messageId`, and a repeated request with the same id runs nothing new. A repeated prompt follows the original turn, and its result has `repeated: true`. One id names one input, so a steer that reuses a prompt's id is rejected with `FX_MESSAGE_ID_IN_USE`. A steer's `user_message` event carries its id as `id` where it joined a turn.
|
||||
- **Inputs have a size limit.** A prompt or steer whose input, with the session's `context` and settings, comes to more than 8 MiB of JSON is rejected with `FX_INPUT_TOO_LARGE` before anything is stored. The model can't take a larger one.
|
||||
- **A refused prompt is never stored.** `prompt()` checks a prompt as the engine's `prompt()` does and throws a `TypeError` or `RangeError` for one libfx would refuse, such as an empty prompt, base64 that isn't canonical, an image whose bytes aren't its type, or more than 8 images. A prompt whose `signal` aborts before the prompt is stored isn't stored either, and its turn ends `cancelled`.
|
||||
|
||||
Each conversation can bring its own settings, which apply to the turns that session object starts:
|
||||
|
||||
@@ -407,7 +408,7 @@ agent.session().prompt([
|
||||
]);
|
||||
```
|
||||
|
||||
A session stores each prompt as JSON before it runs, so a session prompt takes image data as a base64 string. `createFxEngine()` also takes bytes, a `Blob` or a `File`. Each prompt takes up to 8 PNG, JPEG, GIF or WebP images, at most 3.75 MiB each and 6 MiB together. Pass `resizeImage` to downscale or convert prompt images before they're sent.
|
||||
A session stores each prompt as JSON before it runs, so a session prompt takes image data as a base64 string. `createFxEngine()` also takes bytes, a `Blob` or a `File`. A session saves each image once, named by its SHA-256, before it stores the prompt, and the saved conversation refers to it by that name, so images don't count toward the 4 MiB the session's own checkpoints hold. Images a host tool returns are saved the same way; one that can't be saved stays in the tool's result, held inline. A session keeps its newest 4,096 images, up to 512 MiB of them; an older image leaves the conversation, and the model sees a note in its place. A checkpoint you take with `checkpoint()` carries the images' bytes, within its 4 MiB. An engine with a `persistence` store of its own keeps images inside its records and checkpoints, so they count toward that 4 MiB; for a conversation with many images, use a session with [durability](#durability). libfx never deletes a saved image; it lasts as long as the session's storage. Each prompt takes up to 8 PNG, JPEG, GIF or WebP images, at most 3.75 MiB each and 6 MiB together. Pass `resizeImage` to downscale or convert prompt images before they're sent.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
|
||||
+328
-35
@@ -75,14 +75,129 @@ const recentIdsKept = 256;
|
||||
// The longest a message waits for a live holder before it is delivered again.
|
||||
const maxBackstopSeconds = 300;
|
||||
|
||||
// Base64 of a Uint8Array: the runtime's own encoder where it has one, and
|
||||
// otherwise four letters from the table for each three bytes. Both are fast
|
||||
// enough for a large image; btoa over String.fromCharCode is not.
|
||||
const base64Letters = encoder.encode("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/");
|
||||
const toBase64 = (bytes) => {
|
||||
let text = "";
|
||||
for (let index = 0; index < bytes.length; index += 0x8000) {
|
||||
text += String.fromCharCode(...bytes.subarray(index, index + 0x8000));
|
||||
if (typeof bytes.toBase64 === "function") return bytes.toBase64();
|
||||
const letters = new Uint8Array(Math.ceil(bytes.length / 3) * 4);
|
||||
const whole = bytes.length - (bytes.length % 3);
|
||||
let at = 0;
|
||||
for (let index = 0; index < whole; index += 3) {
|
||||
const group = (bytes[index] << 16) | (bytes[index + 1] << 8) | bytes[index + 2];
|
||||
letters[at] = base64Letters[group >> 18];
|
||||
letters[at + 1] = base64Letters[(group >> 12) & 63];
|
||||
letters[at + 2] = base64Letters[(group >> 6) & 63];
|
||||
letters[at + 3] = base64Letters[group & 63];
|
||||
at += 4;
|
||||
}
|
||||
return btoa(text);
|
||||
if (whole < bytes.length) {
|
||||
// One or two bytes left: two or three letters, then "=" padding.
|
||||
const two = whole + 1 < bytes.length;
|
||||
const group = (bytes[whole] << 16) | (two ? bytes[whole + 1] << 8 : 0);
|
||||
letters[at] = base64Letters[group >> 18];
|
||||
letters[at + 1] = base64Letters[(group >> 12) & 63];
|
||||
letters[at + 2] = two ? base64Letters[(group >> 6) & 63] : 61;
|
||||
letters[at + 3] = 61;
|
||||
}
|
||||
return decoder.decode(letters);
|
||||
};
|
||||
const fromBase64 = (text) => Uint8Array.from(atob(text), (char) => char.charCodeAt(0));
|
||||
const fromBase64 = (text) => {
|
||||
if (typeof Uint8Array.fromBase64 === "function") return Uint8Array.fromBase64(text);
|
||||
const binary = atob(text);
|
||||
const bytes = new Uint8Array(binary.length);
|
||||
for (let index = 0; index < binary.length; index += 1) bytes[index] = binary.charCodeAt(index);
|
||||
return bytes;
|
||||
};
|
||||
|
||||
// A session keeps each prompt image once, in its image store, named by the
|
||||
// lowercase hex SHA-256 of its bytes, as libfx's core names it; the log only
|
||||
// names it.
|
||||
async function imageName(bytes) {
|
||||
const digest = new Uint8Array(await globalThis.crypto.subtle.digest("SHA-256", bytes));
|
||||
let name = "";
|
||||
for (const byte of digest) name += byte.toString(16).padStart(2, "0");
|
||||
return name;
|
||||
}
|
||||
const base64url = (text) => toBase64(encoder.encode(text)).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||
// The key a harness's store gives its image store under, `{ put, get }`:
|
||||
// `persistenceImages` in fx-sdk.js, which engines read it from.
|
||||
const persistenceImages = Symbol.for("libfx.persistenceImages");
|
||||
|
||||
// A view of the bytes in an ArrayBuffer or typed array, or null.
|
||||
const byteView = (value) => (value instanceof ArrayBuffer ? new Uint8Array(value)
|
||||
: ArrayBuffer.isView(value) ? new Uint8Array(value.buffer, value.byteOffset, value.byteLength) : null);
|
||||
|
||||
// A prompt's input as the queue and the log hold it: JSON, so image data is
|
||||
// base64 text. With `named`, the session stores each image in its image
|
||||
// store and the log names it (`storePromptImages`); `images` says whether any
|
||||
// image has data to store.
|
||||
//
|
||||
// Each image block is copied once and only the copy is read, since an
|
||||
// accessor can return something else the next time it is read, and the log
|
||||
// must hold what was checked. The blocks are walked by index for the same
|
||||
// reason: an Array subclass can make map() return anything. Pure.
|
||||
function promptInputForLog(input, { named = false } = {}) {
|
||||
let images = false;
|
||||
const blocks = [];
|
||||
for (let index = 0; index < input.length; index += 1) {
|
||||
const block = input[index];
|
||||
if (block?.type !== "image") {
|
||||
blocks.push(block);
|
||||
continue;
|
||||
}
|
||||
const image = { ...block };
|
||||
if (image.data !== undefined && typeof image.data !== "string") throw new TypeError("a durable prompt carries image data as a base64 string");
|
||||
if (typeof image.data === "string") images = named;
|
||||
blocks.push(image);
|
||||
}
|
||||
return { input: blocks, images };
|
||||
}
|
||||
|
||||
// Stores each image a prompt's input holds in the session's image store,
|
||||
// before the input that names it is stored, and returns the input naming
|
||||
// them: `{ type: "image", mimeType, sourceRef?, data: { sha256 } }`. An older
|
||||
// libfx refuses data it can't read, so a worker that predates names fails
|
||||
// the turn rather than run it without its images. A store that fails rejects
|
||||
// with its own error. With `images` null it only names them, for a prompt the
|
||||
// log already holds.
|
||||
async function storePromptImages(images, input) {
|
||||
const blocks = [];
|
||||
for (const block of input) {
|
||||
if (block?.type !== "image" || block.data === undefined) {
|
||||
blocks.push(block);
|
||||
continue;
|
||||
}
|
||||
const { data, ...rest } = block;
|
||||
const bytes = fromBase64(data);
|
||||
const sha256 = await imageName(bytes);
|
||||
if (images) await images.put(sha256, bytes);
|
||||
blocks.push({ ...rest, data: { sha256 } });
|
||||
}
|
||||
return blocks;
|
||||
}
|
||||
|
||||
// A prompt's input with each named image read back from the session's image
|
||||
// store, as base64, which is how a harness has always received image data.
|
||||
// `missing` names the first image the store does not hold.
|
||||
async function resolvePromptImages(images, input) {
|
||||
if (!Array.isArray(input)) return { input };
|
||||
const blocks = [];
|
||||
for (const block of input) {
|
||||
const name = block?.type === "image" ? block.data?.sha256 : undefined;
|
||||
if (typeof name !== "string") {
|
||||
blocks.push(block);
|
||||
continue;
|
||||
}
|
||||
const bytes = await images.get(name);
|
||||
if (!bytes) return { missing: name };
|
||||
// Bytes that aren't the ones named are not the image its caller sent.
|
||||
if (await imageName(bytes) !== name) return { missing: name, corrupt: true };
|
||||
blocks.push({ ...block, data: toBase64(bytes) });
|
||||
}
|
||||
return { input: blocks };
|
||||
}
|
||||
|
||||
/**
|
||||
* The durability that keeps sessions in this process only. Agents given the
|
||||
@@ -451,7 +566,7 @@ function createMemoryBackend({ maxDurationMs, reserveMs } = {}) {
|
||||
const sessionFor = (id) => {
|
||||
let session = sessions.get(id);
|
||||
if (!session) {
|
||||
session = { entries: [], head: null, ui: [], uiWaiters: new Set(), watchers: new Set() };
|
||||
session = { entries: [], head: null, ui: [], uiWaiters: new Set(), watchers: new Set(), images: new Map() };
|
||||
sessions.set(id, session);
|
||||
}
|
||||
return session;
|
||||
@@ -530,6 +645,14 @@ function createMemoryBackend({ maxDurationMs, reserveMs } = {}) {
|
||||
session.watchers.add(fn);
|
||||
return () => session.watchers.delete(fn);
|
||||
},
|
||||
images: {
|
||||
async put(name, bytes) {
|
||||
if (!session.images.has(name)) session.images.set(name, bytes.slice());
|
||||
},
|
||||
async get(name) {
|
||||
return session.images.get(name)?.slice();
|
||||
},
|
||||
},
|
||||
ui: {
|
||||
async write(lines) {
|
||||
session.ui.push(...lines);
|
||||
@@ -580,6 +703,9 @@ const logStep = "fx.log";
|
||||
// the run's id, then `_user_` and the base64url of the namespace
|
||||
// "libfx-ui".
|
||||
const uiStreamOf = (runId) => `${runId.replace(/^wrun_/, "strm_")}_user_bGliZngtdWk`;
|
||||
// The stream that holds an image, named as the UI stream is, with the
|
||||
// namespace "libfx-image-" and the image's name.
|
||||
const imageStreamOf = (runId, name) => `${runId.replace(/^wrun_/, "strm_")}_user_${base64url(`libfx-image-${name}`)}`;
|
||||
const queuePrefix = "__libfx_wkf_workflow_";
|
||||
// The name of an agent created without one, and of whatever a message or a
|
||||
// session from before names belonged to.
|
||||
@@ -867,6 +993,8 @@ function createWorldBackend(world, durability) {
|
||||
async session(runId) {
|
||||
// The newest slot this worker has seen.
|
||||
let known = 0;
|
||||
// The images this worker knows the run's image store holds.
|
||||
const storedImages = new Set();
|
||||
// Where the World has the in-band writer fence, chained entries are
|
||||
// written in-band, each expecting the World's count of in-band writes
|
||||
// at the entry it continues, so the World refuses one that continues a
|
||||
@@ -1047,6 +1175,44 @@ function createWorldBackend(world, durability) {
|
||||
set.add(fn);
|
||||
return () => set.delete(fn);
|
||||
},
|
||||
// Each image is a stream of the run, named after it, so it is read by
|
||||
// name wherever checkpoints fall. Two writers of one image each add
|
||||
// its bytes once; a read takes the first chunk that matches the name.
|
||||
images: {
|
||||
async put(name, bytes) {
|
||||
if (storedImages.has(name)) return;
|
||||
// Refused here, before the prompt is stored, rather than by every
|
||||
// worker that reads the image back.
|
||||
if (typeof world.streams.getChunks !== "function") throw new TypeError("a World needs streams.getChunks() to keep a session's images");
|
||||
await ensureRun();
|
||||
const stream = imageStreamOf(runId, name);
|
||||
let present = false;
|
||||
try {
|
||||
present = Number((await world.streams.getInfo(runId, stream))?.tailIndex ?? -1) >= 0;
|
||||
} catch (error) {
|
||||
if (!missingRun(error)) throw error;
|
||||
}
|
||||
if (!present) await world.streams.write(runId, stream, bytes);
|
||||
storedImages.add(name);
|
||||
},
|
||||
async get(name) {
|
||||
let page;
|
||||
try {
|
||||
page = await world.streams.getChunks(runId, imageStreamOf(runId, name), { limit: 8 });
|
||||
} catch (error) {
|
||||
if (missingRun(error)) return undefined;
|
||||
throw error;
|
||||
}
|
||||
for (const chunk of page?.data ?? []) {
|
||||
const bytes = byteView(chunk?.data);
|
||||
if (bytes && bytes.byteLength > 0 && await imageName(bytes) === name) {
|
||||
storedImages.add(name);
|
||||
return bytes.slice();
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
},
|
||||
ui: {
|
||||
async write(lines) {
|
||||
await ensureRun();
|
||||
@@ -1589,6 +1755,8 @@ class SessionWorker {
|
||||
const ui = uiWriter(log, (error) => agent.emit("ui.error", { sessionId: this.sessionId, error: error?.name ?? "Error" }), lease.epoch);
|
||||
// When the lease runs out, as this worker last renewed it.
|
||||
let leaseExpiresAt = lease.expiresAt;
|
||||
// An image read that threw while the engine opened.
|
||||
let imageReadFailed = null;
|
||||
const store = {
|
||||
load: async () => ({
|
||||
...(state.checkpoint?.data ? { checkpoint: { through: state.checkpoint.through ?? "0", data: fromBase64(state.checkpoint.data) } } : {}),
|
||||
@@ -1597,6 +1765,20 @@ class SessionWorker {
|
||||
head: head ?? state.head,
|
||||
}),
|
||||
append: ({ idempotencyKey, data, marks }) => chained({ k: "record", key: idempotencyKey, data: toBase64(data), marks }).then((cursor) => ({ cursor })),
|
||||
// The engine stores each prompt image before it sends it; a prompt
|
||||
// from the log names images already here.
|
||||
[persistenceImages]: {
|
||||
put: (name, bytes) => log.images.put(name, bytes),
|
||||
get: async (name) => {
|
||||
try {
|
||||
return await log.images.get(name);
|
||||
} catch (error) {
|
||||
// The store failed to answer, which says nothing of the image.
|
||||
imageReadFailed = error;
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
},
|
||||
saveCheckpoint: async ({ through, data }) => {
|
||||
// What a fold that starts at this checkpoint would otherwise miss:
|
||||
// inputs before it that no turn took or that the open turn holds
|
||||
@@ -1686,6 +1868,10 @@ class SessionWorker {
|
||||
harnessRun = runOf(state.openTurn ?? state.pending[0]);
|
||||
harness = await agent.openHarness(this.sessionId, store, harnessRun, ready);
|
||||
} catch (error) {
|
||||
// The image store failed to answer while the engine read the
|
||||
// session's images: as when the log can't be read, the delivery
|
||||
// fails and its message runs again; nothing failed the turn.
|
||||
if (imageReadFailed) throw imageReadFailed;
|
||||
// A harness that cannot start fails the turn waiting on it, once:
|
||||
// the open turn, or after it failed, the next prompt.
|
||||
const waiting = state.openTurn && !state.openFailed ? state.openTurn.id : state.pending[0]?.messageId;
|
||||
@@ -1763,8 +1949,21 @@ class SessionWorker {
|
||||
harness = await agent.openHarness(this.sessionId, store, wanted, ready);
|
||||
harnessRun = wanted;
|
||||
}
|
||||
// A prompt names the images it stored; the engine gets their bytes.
|
||||
const withImages = await resolvePromptImages(log.images, next.input);
|
||||
if (withImages.missing) {
|
||||
// A history without the image is not the one its caller sent.
|
||||
await log.append({ k: "ended", messageId });
|
||||
const message = withImages.corrupt
|
||||
? `the session's image store holds other bytes under image ${withImages.missing}`
|
||||
: `the session's image store does not hold image ${withImages.missing}`;
|
||||
ui.push({ type: "turn_end", messageId, stopReason: "error", usage: {}, error: { name: "Error", message } });
|
||||
await ui.settle();
|
||||
state = this.fold(await log.read());
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
turn = harness.prompt(next.input ?? "", { turnId: messageId, yieldAt });
|
||||
turn = harness.prompt(withImages.input ?? "", { turnId: messageId, yieldAt });
|
||||
} catch (error) {
|
||||
await log.append({ k: "stopped", messageId });
|
||||
return unusable(error);
|
||||
@@ -2227,6 +2426,9 @@ function turnView({ messageId, resumeRequest = null, start }) {
|
||||
* sets over the agent's options for the turns it starts, as JSON the core
|
||||
* stores with each prompt and passes to `open()`, or undefined for none.
|
||||
* It throws a TypeError for options it refuses.
|
||||
* - `checkPrompt(input)` (optional): throws a TypeError or RangeError for
|
||||
* a prompt input the harness would refuse. A session's `prompt()` calls it
|
||||
* before it stores anything, so a prompt it refuses is never stored.
|
||||
*
|
||||
* A harness session has `prompt(input, { turnId, yieldAt })` and
|
||||
* `resume({ yieldAt, rerun })`, each returning a turn, where `rerun: false`
|
||||
@@ -2244,7 +2446,8 @@ function turnView({ messageId, resumeRequest = null, start }) {
|
||||
* end. Either way the steer runs as the next turn. A turn stops before a
|
||||
* model request once `yieldAt` has passed, with the stop reason `yielded`.
|
||||
*
|
||||
* An input the harness refuses is a turn that ends with an error.
|
||||
* An input the harness refuses only once a turn runs it is a turn that ends
|
||||
* with an error.
|
||||
* `prompt()` and `resume()` throw only when the harness cannot run a turn,
|
||||
* and a turn whose harness stopped under it rejects with a
|
||||
* `FX_HARNESS_STOPPED` code; either way the same message runs the session
|
||||
@@ -2442,7 +2645,16 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
await created.queue.send(queued, { idempotencyKey });
|
||||
seeded = true;
|
||||
};
|
||||
const startTurn = (type, messageId, input, retried = false, aborted = false) => {
|
||||
// Settles once the latest prompt from this session object with images
|
||||
// to store is stored or has failed; null while none has. Each later
|
||||
// prompt waits for it, so they are stored in the order made.
|
||||
let storing = null;
|
||||
// `ready`, when given, resolves the input once it can be stored; only a
|
||||
// prompt that is stored calls it.
|
||||
// `aborted()` says whether the prompt's signal has aborted. One that
|
||||
// aborted before the prompt's send began never stores it; `progress.sent`
|
||||
// is set as the send begins, after which an abort cancels the prompt.
|
||||
const startTurn = (type, messageId, input, retried = false, aborted = () => false, ready = null, progress = null) => {
|
||||
let accept;
|
||||
const accepted = new Promise((resolve, reject) => { accept = { resolve, reject }; });
|
||||
void accepted.catch(() => {});
|
||||
@@ -2451,20 +2663,27 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
resumeRequest: type === "resume" ? messageId : null,
|
||||
start: async () => {
|
||||
try {
|
||||
const read = input === undefined || ready === null || aborted() ? input : await ready();
|
||||
const message = { type, messageId, ...(input === undefined ? {} : { input: read }) };
|
||||
const queue = () => {
|
||||
if (progress) progress.sent = true;
|
||||
return send(message, { inRequest: type === "prompt" });
|
||||
};
|
||||
const { created, id: sessionId } = await resolved;
|
||||
const log = await created.session(sessionId);
|
||||
let attaching = type === "resume";
|
||||
let from = null;
|
||||
if (retried && !aborted) {
|
||||
// The viewer's start is read before the message is queued, so
|
||||
// every line this message's own turn writes comes after it,
|
||||
// and an end for this id before it can only be from a turn
|
||||
// this id already ran. The log, for that answer, is read
|
||||
// alongside the one queue send.
|
||||
from = await log.ui.length();
|
||||
// The viewer's start is read before the message is queued, so
|
||||
// every line this message's own turn writes comes after it, and
|
||||
// an end for this id before it can only be from a turn this id
|
||||
// already ran.
|
||||
if (retried) from = await log.ui.length();
|
||||
if (retried && !aborted()) {
|
||||
// The log, for that answer, is read alongside the one queue
|
||||
// send.
|
||||
const [state] = await Promise.all([
|
||||
log.read().then(foldSessionLog),
|
||||
send({ type, messageId, ...(input === undefined ? {} : { input }) }, { inRequest: type === "prompt" }).then(() => accept.resolve({ messageId, sessionId })),
|
||||
queue().then(() => accept.resolve({ messageId, sessionId })),
|
||||
]);
|
||||
// One the log does not know yet is this send's own. When the
|
||||
// log says it ran and nothing shows how, either this send's
|
||||
@@ -2477,9 +2696,8 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
}
|
||||
if (retried) {
|
||||
// A turn this id already ran answers from the log, without
|
||||
// queueing it again. The viewer's start is read first, so a
|
||||
// queueing it again. The viewer's start was read first, so a
|
||||
// turn that ends after it has its end in view.
|
||||
from = await log.ui.length();
|
||||
const outcome = await retryOutcome(log, foldSessionLog(await log.read()), messageId, from, { sent: false });
|
||||
if (outcome.follow) attaching = true;
|
||||
else if (!outcome.fresh) {
|
||||
@@ -2488,15 +2706,17 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
return { log, from, attaching: true, endWithinMs: retryEndWaitMs };
|
||||
}
|
||||
}
|
||||
if (aborted && !attaching) {
|
||||
// A new prompt whose signal had aborted is never stored.
|
||||
accept.resolve({ messageId, sessionId });
|
||||
return { settled: { messageId, stopReason: "cancelled", usage: {} } };
|
||||
}
|
||||
// The viewer starts where the stream is now, before the
|
||||
// message can produce anything.
|
||||
from ??= await log.ui.length();
|
||||
await send({ type, messageId, ...(input === undefined ? {} : { input }) }, { inRequest: type === "prompt" });
|
||||
if (aborted()) {
|
||||
// A prompt whose signal aborted before its send began is never
|
||||
// stored; a retry of one the log holds only follows its turn.
|
||||
accept.resolve({ messageId, sessionId });
|
||||
if (!attaching) return { settled: { messageId, stopReason: "cancelled", usage: {} } };
|
||||
return { log, from, attaching };
|
||||
}
|
||||
await queue();
|
||||
accept.resolve({ messageId, sessionId });
|
||||
return { log, from, attaching };
|
||||
} catch (error) {
|
||||
@@ -2513,25 +2733,83 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
* Queues `input` as the session's next turn and returns a view of it.
|
||||
* With `messageId`, a retried call is the same turn: it continues a
|
||||
* turn still running and answers one that ended, without running it
|
||||
* again.
|
||||
* again. Each image is stored once in the session's image store and
|
||||
* the log names it. A later prompt from this session object waits for
|
||||
* an earlier one's images. An input the harness would refuse throws
|
||||
* here and is never stored. A prompt whose signal aborts before it is
|
||||
* stored is never stored either: it ends cancelled.
|
||||
*/
|
||||
prompt(input, promptOptions = {}) {
|
||||
if (closed) throw new Error(`${label} is closed`);
|
||||
const messageId = promptOptions.messageId ?? newId("msg");
|
||||
if (!validId(messageId)) throw new TypeError(`messageId must be ${idRule}`);
|
||||
if (typeof input !== "string" && !Array.isArray(input)) throw new TypeError("prompt input must be a string or an array of content blocks");
|
||||
// The input travels through the queue and the log as JSON.
|
||||
if (Array.isArray(input) && input.some((block) => block?.type === "image" && block.data !== undefined && typeof block.data !== "string")) {
|
||||
throw new TypeError("a durable prompt carries image data as a base64 string");
|
||||
}
|
||||
admit(input);
|
||||
const signal = promptOptions.signal;
|
||||
if (signal !== undefined && typeof signal?.addEventListener !== "function") throw new TypeError("prompt signal must be an AbortSignal");
|
||||
const turn = startTurn("prompt", messageId, input, promptOptions.messageId !== undefined, signal?.aborted === true);
|
||||
const aborted = signal?.aborted === true;
|
||||
const retried = promptOptions.messageId !== undefined;
|
||||
// The input travels through the queue and the log as JSON. Its
|
||||
// images go to the session's image store first and the log names
|
||||
// them; a prompt aborted before the call stores nothing, so it
|
||||
// keeps them as base64.
|
||||
const named = !aborted;
|
||||
const stored = Array.isArray(input) ? promptInputForLog(input, { named }) : { input, images: false };
|
||||
// The copy the log will hold is what is checked: a getter can't
|
||||
// show the check one value and the log another.
|
||||
admit(stored.input);
|
||||
plugged.checkPrompt?.(stored.input);
|
||||
const before = storing;
|
||||
const ready = before || stored.images ? async () => {
|
||||
// The prompt waits for an earlier one and its image writes only
|
||||
// until its signal aborts; what it waited for may still land, but
|
||||
// the prompt is not stored.
|
||||
let release = () => {};
|
||||
const abortion = signal === undefined ? null : new Promise((resolve) => {
|
||||
signal.addEventListener("abort", resolve, { once: true });
|
||||
release = () => signal.removeEventListener("abort", resolve);
|
||||
});
|
||||
const unlessAborted = (promise) => {
|
||||
if (abortion === null) return promise;
|
||||
void promise.catch(() => {});
|
||||
return Promise.race([promise, abortion]);
|
||||
};
|
||||
try {
|
||||
await unlessAborted(before ?? Promise.resolve());
|
||||
if (signal?.aborted) return null;
|
||||
let read = stored.input;
|
||||
if (stored.images) {
|
||||
const { created, id: sessionId } = await resolved;
|
||||
const log = await created.session(sessionId);
|
||||
// A retry of a prompt the log already holds is answered from
|
||||
// the log: its images are named, not written again.
|
||||
const known = retried && knownMessage(foldSessionLog(await log.read()), { type: "prompt", messageId });
|
||||
if (signal?.aborted) return null;
|
||||
read = await unlessAborted(storePromptImages(known ? null : log.images, read));
|
||||
if (signal?.aborted) return null;
|
||||
}
|
||||
// An agent closed meanwhile stores nothing more.
|
||||
if (closed) throw new Error(`${label} is closed`);
|
||||
return read;
|
||||
} finally {
|
||||
release();
|
||||
}
|
||||
} : null;
|
||||
const progress = { sent: false };
|
||||
const turn = startTurn("prompt", messageId, stored.input, retried, () => signal?.aborted === true, ready, progress);
|
||||
if (stored.images && !aborted) {
|
||||
// A later prompt waits for this one and every one before it: one
|
||||
// withdrawn while it waited settles before them.
|
||||
const settled = Promise.all([before, turn.accepted.then(() => {}, () => {})]).then(() => {});
|
||||
storing = settled;
|
||||
void settled.then(() => { if (storing === settled) storing = null; });
|
||||
}
|
||||
if (signal !== undefined && !signal.aborted) {
|
||||
// The prompt's own cancel: it stops this turn whether it runs or
|
||||
// still waits, and never a later one.
|
||||
const stop = () => { void send({ type: "cancel", messageId: newId("cnl"), target: messageId }).catch(() => {}); };
|
||||
// still waits, and never a later one. A prompt whose send had not
|
||||
// begun is never stored, so it needs none.
|
||||
const stop = () => {
|
||||
if (progress.sent) void send({ type: "cancel", messageId: newId("cnl"), target: messageId }).catch(() => {});
|
||||
};
|
||||
signal.addEventListener("abort", stop, { once: true });
|
||||
void turn.result.finally(() => signal.removeEventListener("abort", stop)).catch(() => {});
|
||||
}
|
||||
@@ -2633,7 +2911,16 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
const messageId = steerOptions.messageId ?? newId("str");
|
||||
if (!validId(messageId)) throw new TypeError(`messageId must be ${idRule}`);
|
||||
admit(input);
|
||||
await send({ type: "steer", messageId, input });
|
||||
// A steer's images go in the session's image store before the steer
|
||||
// that names them, as a prompt's do; storing one again under its
|
||||
// name is harmless, so a retry needs no check.
|
||||
let named = input;
|
||||
if (Array.isArray(input) && input.some((block) => block?.type === "image" && typeof block.data === "string")) {
|
||||
const { created, id: sessionId } = await resolved;
|
||||
named = await storePromptImages((await created.session(sessionId)).images, input);
|
||||
if (closed) throw new Error(`${label} is closed`);
|
||||
}
|
||||
await send({ type: "steer", messageId, input: named });
|
||||
return { messageId };
|
||||
},
|
||||
/** Cancels the running turn; dropping a turn's view does not. */
|
||||
@@ -2652,6 +2939,12 @@ export function createDurableAgentFactory({ harness, defaultDurability, defaultS
|
||||
head: state.head,
|
||||
}),
|
||||
append: async () => { throw new Error("a checkpoint reads the session without changing it"); },
|
||||
// The records name the session's images; the engine reads them
|
||||
// from its image store to export them.
|
||||
[persistenceImages]: {
|
||||
get: (name) => log.images.get(name),
|
||||
put: async () => { throw new Error("a checkpoint reads the session without changing it"); },
|
||||
},
|
||||
};
|
||||
const harness = await agent.openHarness(sessionId, readOnly, null);
|
||||
try {
|
||||
|
||||
+6
-1
@@ -1,7 +1,7 @@
|
||||
// fx as a harness of the durable core in durable.js: `createFxAgent` is that
|
||||
// core running fx engines. Everything the core needs to know about fx is
|
||||
// here; the core itself knows only the harness contract.
|
||||
import { coreAnswered, engineInternals, journalMarksWanted, normalizeInstructions, normalizeModelChoice, resumeOutputShown } from "./fx-sdk.js";
|
||||
import { checkPromptInput, coreAnswered, engineInternals, journalMarksWanted, normalizeInstructions, normalizeModelChoice, resumeOutputShown } from "./fx-sdk.js";
|
||||
import { createCredentialMemory, credentialMemory } from "./subscription.js";
|
||||
|
||||
const encoder = new TextEncoder();
|
||||
@@ -28,6 +28,11 @@ export function fxHarness({ createEngine, defaultApiKey = async () => undefined
|
||||
return {
|
||||
...(checkpoint === undefined ? {} : { seed: snapshotOf(checkpointBytesOf(checkpoint)) }),
|
||||
settings: sessionSettings,
|
||||
// What a session's prompt() refuses before storing anything: whatever
|
||||
// this agent's engines would refuse.
|
||||
checkPrompt(input) {
|
||||
checkPromptInput(input, { resizeImage: engineOptions.resizeImage });
|
||||
},
|
||||
async open({ sessionId, store, context, settings, durability, ready }) {
|
||||
const apiKey = engineOptions.apiKey ?? await defaultApiKey(durability);
|
||||
const engine = await createEngine({
|
||||
|
||||
+211
-26
@@ -37,7 +37,10 @@ const maxUnreadEvents = 256;
|
||||
// Prompt images travel as raw bytes beside the ACP frame and are base64
|
||||
// encoded only in the model request. An image may use 5 MiB of encoded
|
||||
// request data, and a prompt's images 8 MiB, so the raw limits are 3/4 of that.
|
||||
// The kernel still validates content and media type.
|
||||
// The kernel still validates content and media type, and holds a prompt to
|
||||
// these limits too: `max_libfx_prompt_images` and
|
||||
// `max_libfx_prompt_images_bytes` in src/core/images/image_attachments.zig
|
||||
// must match them.
|
||||
const maxPromptImages = 8;
|
||||
const maxPromptImageDataBytes = 5 * 1024 * 1024;
|
||||
const maxPromptImagesDataBytes = 8 * 1024 * 1024;
|
||||
@@ -45,6 +48,7 @@ const maxPromptImageBytes = (maxPromptImageDataBytes / 4) * 3;
|
||||
const maxPromptImagesBytes = (maxPromptImagesDataBytes / 4) * 3;
|
||||
// Matches the native attachment table: one prompt's images or one checkpoint.
|
||||
const maxPendingAttachments = 8;
|
||||
const maxPendingAttachmentBytes = 8 * 1024 * 1024;
|
||||
const maxOutboundAttachments = 4;
|
||||
// Matches the core's kernel checkpoint limit (max_checkpoint_bytes).
|
||||
const maxCheckpointBytes = 4 * 1024 * 1024;
|
||||
@@ -234,6 +238,14 @@ export function normalizeAgentOptions(value) {
|
||||
// public surface.
|
||||
export const engineInternals = Symbol.for("libfx.engineInternals");
|
||||
export const journalMarksWanted = Symbol.for("libfx.journalMarks");
|
||||
// A persistence's image store, `{ put(name, bytes), get(name) }`, which keeps
|
||||
// each image once, named by the SHA-256 of its bytes, so records and
|
||||
// checkpoints only name it. Durable sessions give one to the engines they
|
||||
// run; without it, images stay inline.
|
||||
export const persistenceImages = Symbol.for("libfx.persistenceImages");
|
||||
// `{ images, bytes }`: a smaller image window than the core's, the newest
|
||||
// images a session with an image store keeps, so tests can reach its edge.
|
||||
const imageWindowLimits = Symbol.for("libfx.imageWindow");
|
||||
// Marks an error the core answered a request with, as opposed to one from a
|
||||
// core that stopped answering.
|
||||
export const coreAnswered = Symbol.for("libfx.coreAnswered");
|
||||
@@ -279,8 +291,26 @@ const journalOpenErrorCodes = new Map([
|
||||
["libfx journal events are out of order", "FX_JOURNAL_INVALID"],
|
||||
["libfx journal is too large", "FX_JOURNAL_TOO_LARGE"],
|
||||
["libfx journal holds more than 1024 turns", "FX_JOURNAL_TOO_LARGE"],
|
||||
["libfx journal images are too large", "FX_JOURNAL_TOO_LARGE"],
|
||||
["Invalid libfx journal images", "FX_JOURNAL_INVALID"],
|
||||
["libfx image does not match its name", "FX_JOURNAL_IMAGE_MISSING"],
|
||||
]);
|
||||
|
||||
// An image's name in a persistence's image store: the lowercase hex SHA-256
|
||||
// of its bytes, as the core names it.
|
||||
async function imageName(bytes) {
|
||||
const digest = new Uint8Array(await globalThis.crypto.subtle.digest("SHA-256", bytes));
|
||||
let name = "";
|
||||
for (const byte of digest) name += byte.toString(16).padStart(2, "0");
|
||||
return name;
|
||||
}
|
||||
|
||||
function imageMissingError(name, cause) {
|
||||
const error = journalLoadError(`libfx journal names an image the store does not hold: ${name}`, "FX_JOURNAL_IMAGE_MISSING", cause);
|
||||
error.image = name;
|
||||
return error;
|
||||
}
|
||||
|
||||
/**
|
||||
* A persistence store that keeps a session's records and its latest
|
||||
* checkpoint in memory, for tests and for hosts that copy them elsewhere.
|
||||
@@ -1986,6 +2016,62 @@ function normalizePromptInput(input, { deferImageLimits = false } = {}) {
|
||||
return prompt;
|
||||
}
|
||||
|
||||
// The image types libfx's core takes, each known by how its bytes start.
|
||||
const imageSignatures = [
|
||||
["image/png", 0, [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]],
|
||||
["image/jpeg", 0, [0xff, 0xd8, 0xff]],
|
||||
["image/gif", 0, [0x47, 0x49, 0x46, 0x38, 0x37, 0x61]],
|
||||
["image/gif", 0, [0x47, 0x49, 0x46, 0x38, 0x39, 0x61]],
|
||||
["image/webp", 0, [0x52, 0x49, 0x46, 0x46]],
|
||||
];
|
||||
const imageTypes = new Set(imageSignatures.map(([type]) => type));
|
||||
const webpMarker = [0x57, 0x45, 0x42, 0x50];
|
||||
|
||||
// The image type `bytes` start as, or null.
|
||||
function imageTypeOf(bytes) {
|
||||
const startsWith = (offset, signature) => bytes.length >= offset + signature.length && signature.every((byte, index) => bytes[offset + index] === byte);
|
||||
for (const [type, offset, signature] of imageSignatures) {
|
||||
if (!startsWith(offset, signature)) continue;
|
||||
if (type === "image/webp" && !startsWith(8, webpMarker)) continue;
|
||||
return type;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Refuses image bytes that are not the type they declare, as libfx's core does.
|
||||
function checkImageBytesType(bytes, mimeType, index) {
|
||||
const type = imageTypeOf(bytes);
|
||||
if (type === null) throw new TypeError(`image prompt block ${index} is not a PNG, JPEG, GIF or WebP image`);
|
||||
if (type !== mimeType) throw new TypeError(`image prompt block ${index} holds ${type} data, not ${mimeType}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks a prompt's input as libfx takes it and returns it normalized:
|
||||
* every check `prompt()` makes, and the ones libfx's core makes on images,
|
||||
* which would otherwise refuse the prompt only once it reached the core: a
|
||||
* supported image type, and bytes or base64 that are that type. A Blob is not
|
||||
* read, so its bytes are checked once read (`checkImageBytesType`). With
|
||||
* `resizeImage`, image bytes are checked after it runs instead. Throws a
|
||||
* TypeError or RangeError.
|
||||
*/
|
||||
export function checkPromptInput(input, { resizeImage } = {}) {
|
||||
const resized = resizeImage !== undefined;
|
||||
const normalized = normalizePromptInput(input, { deferImageLimits: resized });
|
||||
// The core takes no prompt without text or an image; a resource adds neither.
|
||||
if (!normalized.some((block) => block.type === "image" || (block.type === "text" && block.text.length > 0))) {
|
||||
throw new TypeError("prompt is empty");
|
||||
}
|
||||
// resizeImage decides the final image sizes, so they are counted after it runs.
|
||||
checkPromptFrameSize(normalized, !resized);
|
||||
normalized.forEach((block, index) => {
|
||||
if (block.type !== "image" || (resized && block.data !== undefined)) return;
|
||||
if (!imageTypes.has(block.mimeType)) throw new TypeError(`image prompt block ${index} mimeType must be image/png, image/jpeg, image/gif or image/webp`);
|
||||
if (block.source === "bytes") checkImageBytesType(block.data, block.mimeType, index);
|
||||
else if (block.source === "base64") checkImageBytesType(base64ToBytes(block.data.slice(0, 16)), block.mimeType, index);
|
||||
});
|
||||
return normalized;
|
||||
}
|
||||
|
||||
// Image bytes travel beside the frame as attachment references, but the model
|
||||
// request carries them base64 encoded and the native host caps that request at
|
||||
// 8 MiB. Images therefore count at their encoded size, the same budget they
|
||||
@@ -2071,6 +2157,11 @@ async function materializePromptImages(blocks, isCancelled, resizeImage) {
|
||||
}
|
||||
if (promptFrameSize(prepared) > maxPromptFrameBytes) prepared = omitReferencedImageData(prepared);
|
||||
checkPromptFrameSize(prepared, true);
|
||||
// A file's bytes, or what resizeImage made, are checked once there are
|
||||
// some, after the size limits.
|
||||
prepared.forEach((image, index) => {
|
||||
if (image.type === "image" && image.bytes !== undefined) checkImageBytesType(image.bytes, image.mimeType, index);
|
||||
});
|
||||
return prepared;
|
||||
}
|
||||
|
||||
@@ -2201,7 +2292,10 @@ export function normalizeInstructions(value) {
|
||||
return instructions;
|
||||
}
|
||||
|
||||
function hostToolContent(value) {
|
||||
// `stored` maps an image's base64 data to its name in the image store, for
|
||||
// the images stored before the result goes to the core: their blocks carry
|
||||
// the name, so the core names them too.
|
||||
function hostToolContent(value, stored = null) {
|
||||
if (value?.type === "libfx.tool-result") {
|
||||
if (typeof value.text !== "string" || !Array.isArray(value.images) || value.images.length > 8) {
|
||||
throw new TypeError("invalid typed tool result");
|
||||
@@ -2213,11 +2307,13 @@ function hostToolContent(value) {
|
||||
if ((!referenceOnly && typeof image.data !== "string") || typeof image.mimeType !== "string" || image.mimeType.length === 0 || image.mimeType.length > 128 || (image.data?.length > maxPromptImageDataBytes && sourceRef === undefined)) {
|
||||
throw new TypeError("invalid tool image");
|
||||
}
|
||||
const data = !referenceOnly && image.data.length <= maxPromptImageDataBytes ? image.data : undefined;
|
||||
return {
|
||||
type: "image",
|
||||
...(!referenceOnly && image.data.length <= maxPromptImageDataBytes ? { data: image.data } : {}),
|
||||
...(data === undefined ? {} : { data }),
|
||||
mimeType: image.mimeType,
|
||||
...(sourceRef === undefined ? {} : { sourceRef }),
|
||||
...(data !== undefined && stored?.has(data) ? { stored: stored.get(data) } : {}),
|
||||
};
|
||||
});
|
||||
if (promptImageDataBytes(images) > maxPromptImagesDataBytes) images = omitReferencedImageData(images);
|
||||
@@ -2228,7 +2324,7 @@ function hostToolContent(value) {
|
||||
content = JSON.stringify({ text: value.text, images });
|
||||
}
|
||||
if (encoder.encode(content).length > maxPromptImagesDataBytes) throw new RangeError("typed tool result exceeds the result limit");
|
||||
return { content, rich: true, isError: value.isError === true };
|
||||
return { content, rich: true, isError: value.isError === true, images };
|
||||
}
|
||||
if (typeof value === "string") return { content: value, rich: false };
|
||||
if (value === undefined) return { content: "null", rich: false };
|
||||
@@ -2317,6 +2413,11 @@ export async function createFxEngine(options = {}) {
|
||||
// waits for it: the core before an action others can see, such as a tool
|
||||
// call with effects, or a caller for its input.
|
||||
const persistence = options.persistence ?? null;
|
||||
// A persistence with an image store keeps each prompt image once, by
|
||||
// name, and the core names it in records and checkpoints instead of
|
||||
// holding its bytes.
|
||||
const imageStorage = persistence?.[persistenceImages] ?? null;
|
||||
const imageStore = imageStorage !== null;
|
||||
const wantsMarks = persistence?.marks === true || persistence?.[journalMarksWanted] === true;
|
||||
const checkpointAfterBytes = options.checkpointAfterBytes ?? defaultCheckpointAfterBytes;
|
||||
// The store's last cursor, and the last seq its records hold.
|
||||
@@ -2625,6 +2726,30 @@ export async function createFxEngine(options = {}) {
|
||||
}
|
||||
throw new Error("transport retry exhausted");
|
||||
};
|
||||
// A typed tool result's images go in the image store by name before the
|
||||
// result reaches the core, as a prompt's do, so the session's records name
|
||||
// them. The tool has already run, so an image the store refuses is not
|
||||
// lost: it stays in the result, and the records hold it inline.
|
||||
const storedToolContent = async (value) => {
|
||||
const normalized = hostToolContent(value);
|
||||
if (!imageStore || !normalized.images?.some((image) => image.data !== undefined)) return normalized;
|
||||
const stored = new Map();
|
||||
for (const image of normalized.images) {
|
||||
if (image.data === undefined || stored.has(image.data)) continue;
|
||||
let bytes;
|
||||
// Data that is not base64 is the core's to refuse.
|
||||
try { bytes = base64ToBytes(image.data); } catch { continue; }
|
||||
const name = await imageName(bytes);
|
||||
try {
|
||||
await imageStorage.put(name, bytes);
|
||||
stored.set(image.data, name);
|
||||
} catch {
|
||||
// Not stored: the result keeps the image, and the records hold it
|
||||
// inline.
|
||||
}
|
||||
}
|
||||
return stored.size ? hostToolContent(value, stored) : normalized;
|
||||
};
|
||||
const executeHostTool = async (name, input, requestedSessionId, toolCallId) => {
|
||||
const execute = hostTools.executors.get(name);
|
||||
const issuerSessionId = requestedSessionId ?? sessionId;
|
||||
@@ -2658,7 +2783,7 @@ export async function createFxEngine(options = {}) {
|
||||
});
|
||||
const value = await Promise.race([execution, aborted]);
|
||||
if (!controller.signal.aborted) {
|
||||
const normalized = hostToolContent(value);
|
||||
const normalized = await storedToolContent(value);
|
||||
content = normalized.content;
|
||||
rich = normalized.rich;
|
||||
isError = normalized.isError === true;
|
||||
@@ -2667,7 +2792,7 @@ export async function createFxEngine(options = {}) {
|
||||
isError = true;
|
||||
if (error?.toolResult?.type === "libfx.tool-result") {
|
||||
try {
|
||||
const normalized = hostToolContent(error.toolResult);
|
||||
const normalized = await storedToolContent(error.toolResult);
|
||||
content = normalized.content;
|
||||
rich = normalized.rich;
|
||||
} catch {
|
||||
@@ -2746,6 +2871,25 @@ export async function createFxEngine(options = {}) {
|
||||
if (!(bytes instanceof Uint8Array) || bytes.byteLength === 0) throw new Error("fx returned an invalid checkpoint");
|
||||
return new Uint8Array(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
||||
}
|
||||
// Stores each prompt image in the persistence's image store, by name,
|
||||
// before the prompt that carries it is sent: nothing the core writes may
|
||||
// name an image the store does not hold. Returns the blocks with their
|
||||
// names, or null when the prompt stopped meanwhile.
|
||||
async function storePromptImages(blocks, stopped) {
|
||||
const named = [];
|
||||
for (const block of blocks) {
|
||||
if (block.type !== "image" || block.bytes === undefined) {
|
||||
named.push(block);
|
||||
continue;
|
||||
}
|
||||
const name = await imageName(block.bytes);
|
||||
if (stopped()) return null;
|
||||
await imageStorage.put(name, block.bytes);
|
||||
if (stopped()) return null;
|
||||
named.push({ ...block, stored: name });
|
||||
}
|
||||
return named;
|
||||
}
|
||||
const sendPrompt = (blocks, resuming = false, followUp = null, turnId = null, rerun = [], yieldAt = undefined, outputShown = false) => {
|
||||
// A journaled turn given a time to stop at yields before its first model
|
||||
// request after it, keeping the turn open.
|
||||
@@ -2760,7 +2904,8 @@ export async function createFxEngine(options = {}) {
|
||||
type: "image",
|
||||
mimeType: block.mimeType,
|
||||
...(block.sourceRef === undefined ? {} : { sourceRef: block.sourceRef }),
|
||||
...(block.bytes === undefined ? {} : { _meta: { fx: { attachment: ids[next++] } } }),
|
||||
// An image the store holds carries its name, so the core names it too.
|
||||
...(block.bytes === undefined ? {} : { _meta: { fx: { attachment: ids[next++], ...(block.stored === undefined ? {} : { stored: block.stored }) } } }),
|
||||
});
|
||||
// A follow-up's turn places it; the core records it as accepted first
|
||||
// when the host could not. A journaled turn's first progress records its id.
|
||||
@@ -2878,24 +3023,62 @@ export async function createFxEngine(options = {}) {
|
||||
}
|
||||
if (journal) {
|
||||
const { events, checkpoint } = loaded;
|
||||
const params = { sessionId, ...(options.inputsDurable === true ? { inputsDurable: true } : {}) };
|
||||
// One call: attaching again would drop the first attachment.
|
||||
const attachments = [];
|
||||
if (events.length > 0) {
|
||||
const bytes = encoder.encode(JSON.stringify(withoutSupersededProgress(events)));
|
||||
if (bytes.byteLength > maxJournalBytes) throw journalLoadError("libfx journal is too large", "FX_JOURNAL_TOO_LARGE");
|
||||
attachments.push(["journalAttachment", bytes]);
|
||||
}
|
||||
if (checkpoint) attachments.push(["snapshotAttachment", checkpoint]);
|
||||
const ids = attachments.length ? attachBytes(attachments.map(([, bytes]) => bytes)) : [];
|
||||
attachments.forEach(([name], index) => { params[name] = ids[index]; });
|
||||
const opened = await request("libfx/journal_open", params).catch((error) => {
|
||||
const journalBytes = events.length > 0 ? encoder.encode(JSON.stringify(withoutSupersededProgress(events))) : null;
|
||||
if (journalBytes && journalBytes.byteLength > maxJournalBytes) throw journalLoadError("libfx journal is too large", "FX_JOURNAL_TOO_LARGE");
|
||||
const openFailed = (error) => {
|
||||
const message = error?.message ?? "";
|
||||
if (/newer fx/.test(message)) throw new FxJournalVersionError(message);
|
||||
const code = journalOpenErrorCodes.get(message);
|
||||
if (code) throw journalLoadError(message, code, error);
|
||||
throw error;
|
||||
});
|
||||
};
|
||||
const openOnce = () => {
|
||||
const params = {
|
||||
sessionId,
|
||||
...(options.inputsDurable === true ? { inputsDurable: true } : {}),
|
||||
...(imageStore ? { imageStore: true } : {}),
|
||||
...(imageStore && options[imageWindowLimits] ? { imageWindow: options[imageWindowLimits] } : {}),
|
||||
};
|
||||
// One call: attaching again would drop the first attachment.
|
||||
const attachments = [];
|
||||
if (journalBytes) attachments.push(["journalAttachment", journalBytes]);
|
||||
if (checkpoint) attachments.push(["snapshotAttachment", checkpoint]);
|
||||
const ids = attachments.length ? attachBytes(attachments.map(([, bytes]) => bytes)) : [];
|
||||
attachments.forEach(([name], index) => { params[name] = ids[index]; });
|
||||
return request("libfx/journal_open", params).catch(openFailed);
|
||||
};
|
||||
// Stages the images a history names, read from the store, in batches
|
||||
// the attachment table holds. A store that lacks one fails the open:
|
||||
// the history without it is not the one saved.
|
||||
const stageImages = async (names) => {
|
||||
let batch = [];
|
||||
let batchBytes = 0;
|
||||
const flush = async () => {
|
||||
if (batch.length === 0) return;
|
||||
const ids = attachBytes(batch.map(({ bytes }) => bytes));
|
||||
const images = batch.map(({ name }, index) => ({ name, attachment: ids[index] }));
|
||||
batch = [];
|
||||
batchBytes = 0;
|
||||
await request("libfx/journal_images", { sessionId, images }).catch(openFailed);
|
||||
};
|
||||
for (const name of new Set(names.map(String))) {
|
||||
const stored = imageStore ? await imageStorage.get(name) : undefined;
|
||||
if (stored === undefined || stored === null) throw imageMissingError(name);
|
||||
const bytes = persistenceBytes(stored, "an image the image store returns");
|
||||
if (batch.length === maxPendingAttachments || batchBytes + bytes.byteLength > maxPendingAttachmentBytes) await flush();
|
||||
batch.push({ name, bytes });
|
||||
batchBytes += bytes.byteLength;
|
||||
}
|
||||
await flush();
|
||||
};
|
||||
// A history that names images opens once they are staged from the
|
||||
// store: the core answers with the names it needs, all at once.
|
||||
let opened = await openOnce();
|
||||
if (Array.isArray(opened?.imagesNeeded)) {
|
||||
await stageImages(opened.imagesNeeded);
|
||||
opened = await openOnce();
|
||||
if (Array.isArray(opened?.imagesNeeded)) throw imageMissingError(String(opened.imagesNeeded[0]));
|
||||
}
|
||||
compareCheckpointMeta(opened?.checkpointMeta);
|
||||
resumable = opened?.resumable === true;
|
||||
openTurnId = resumable && validTurnId(opened?.openTurnId) ? opened.openTurnId : null;
|
||||
@@ -3211,13 +3394,12 @@ export async function createFxEngine(options = {}) {
|
||||
const resizeImage = options.resizeImage;
|
||||
const resuming = promptOptions[resumeTurn] === true;
|
||||
const followUp = promptOptions[followUpInput] ?? null;
|
||||
const normalized = resuming ? [] : normalizePromptInput(input, { deferImageLimits: resizeImage !== undefined });
|
||||
const normalized = resuming ? [] : checkPromptInput(input, { resizeImage });
|
||||
const hasPixels = normalized.some((block) => block.type === "image" && block.data !== undefined);
|
||||
// Blob reads and resizeImage run before the prompt frame is sent.
|
||||
const asyncImages = hasPixels && (resizeImage !== undefined ||
|
||||
// Blob reads, resizeImage and storing images run before the prompt
|
||||
// frame is sent.
|
||||
const asyncImages = hasPixels && (resizeImage !== undefined || imageStore ||
|
||||
normalized.some((block) => block.type === "image" && block.source === "blob"));
|
||||
// resizeImage decides the final image sizes, so they are counted after it runs.
|
||||
checkPromptFrameSize(normalized, resizeImage === undefined);
|
||||
// Snapshot caller-owned bytes, which could change before an async send.
|
||||
const prompt = asyncImages
|
||||
? normalized.map((block) => block.type === "image" && block.source === "bytes" ? { ...block, data: block.data.slice() } : block)
|
||||
@@ -3422,8 +3604,11 @@ export async function createFxEngine(options = {}) {
|
||||
}) : null;
|
||||
let response;
|
||||
if (asyncImages) {
|
||||
const stopped = () => cancelled || closing;
|
||||
response = Promise.race([
|
||||
Promise.resolve().then(() => materializePromptImages(prompt, () => cancelled || closing, resizeImage)),
|
||||
Promise.resolve()
|
||||
.then(() => materializePromptImages(prompt, stopped, resizeImage))
|
||||
.then((prepared) => prepared && imageStore ? storePromptImages(prepared, stopped) : prepared),
|
||||
imagePrepCancelled,
|
||||
]).then((prepared) => {
|
||||
cancelImagePrep = null;
|
||||
|
||||
@@ -608,9 +608,9 @@ for (const resizeImage of [undefined, (image) => image]) {
|
||||
}
|
||||
|
||||
// The SDK decodes base64 itself, so non-canonical base64 throws before the
|
||||
// turn starts. Kernel-side content validation stays authoritative: a sniffed
|
||||
// media type that contradicts the declaration fails the turn with a typed error
|
||||
// rather than reaching the model.
|
||||
// turn starts, and so does an image whose bytes aren't the type it declares,
|
||||
// or a type libfx doesn't take, as the kernel would refuse it. A Blob's bytes
|
||||
// are checked once read, which rejects the turn before the prompt is sent.
|
||||
{
|
||||
const gateway = mockGateway();
|
||||
const agent = await createAgent(gateway, { model: "sdk/vision-model" });
|
||||
@@ -621,17 +621,27 @@ for (const resizeImage of [undefined, (image) => image]) {
|
||||
);
|
||||
}
|
||||
const jpegBytes = Buffer.from([0xff, 0xd8, 0xff, 0xe0, 1, 2, 3, 4]).toString("base64");
|
||||
const mismatch = agent.prompt([{ type: "image", data: jpegBytes, mimeType: "image/png" }]);
|
||||
await assert.rejects(mismatch.result, /Invalid image prompt block/);
|
||||
const mislabeled = (error) => error instanceof TypeError && /holds image\/jpeg data, not image\/png/.test(error.message);
|
||||
assert.throws(() => agent.prompt([{ type: "image", data: jpegBytes, mimeType: "image/png" }]), mislabeled);
|
||||
assert.throws(() => agent.prompt([{ type: "image", data: Buffer.from(jpegBytes, "base64"), mimeType: "image/png" }]), mislabeled);
|
||||
assert.throws(
|
||||
() => agent.prompt([{ type: "image", data: Buffer.from("not an image"), mimeType: "image/png" }]),
|
||||
(error) => error instanceof TypeError && /is not a PNG, JPEG, GIF or WebP image/.test(error.message),
|
||||
);
|
||||
const blobMismatch = agent.prompt([
|
||||
{ type: "image", data: new Blob([Buffer.from(jpegBytes, "base64")], { type: "image/png" }) },
|
||||
]);
|
||||
await assert.rejects(blobMismatch.result, /Invalid image prompt block/);
|
||||
await assert.rejects(blobMismatch.result, mislabeled);
|
||||
for (const mimeType of ["image/svg+xml", "text/plain", "image/png\n"]) {
|
||||
const referenceOnly = agent.prompt([{ type: "image", mimeType, sourceRef: "host:unsupported-mime" }]);
|
||||
await assert.rejects(referenceOnly.result, /Invalid image prompt block/);
|
||||
assert.throws(
|
||||
() => agent.prompt([{ type: "image", mimeType, sourceRef: "host:unsupported-mime" }]),
|
||||
(error) => error instanceof TypeError && /mimeType must be image\/png, image\/jpeg, image\/gif or image\/webp/.test(error.message),
|
||||
);
|
||||
}
|
||||
assert.throws(() => agent.prompt(""), (error) => error instanceof TypeError && /prompt is empty/.test(error.message));
|
||||
assert.equal(gateway.state.chatBodies.length, 0);
|
||||
// The agent still takes a valid prompt.
|
||||
assert.equal((await runPrompt(agent, [{ type: "image", data: pngData, mimeType: "image/png" }])).stopReason, "end_turn");
|
||||
await agent.close();
|
||||
}
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
//
|
||||
// node sdk/tests/test-durable-core.mjs [memory|local|world|queue]
|
||||
import { strict as assert } from "node:assert";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { mkdtemp, readdir, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { createDurableAgentFactory, foldSessionLog, memory, wakeHandler } from "../durable.js";
|
||||
@@ -964,6 +964,48 @@ if (durabilityKind === "queue") {
|
||||
await agent.close();
|
||||
}
|
||||
});
|
||||
|
||||
test("an image prompt run in the request reads its image from the store, and no queued copy holds it", async () => {
|
||||
const internals = (agent) => agent[Symbol.for("libfx.durableInternals")];
|
||||
for (const maxDurationMs of [60_000, 300]) {
|
||||
const dir = await mkdtemp(join(tmpdir(), "libfx-in-request-image-"));
|
||||
dirs.push(dir);
|
||||
const waited = [];
|
||||
const { durability, broker } = await queueWorldAt(dir, { queueLeases: true, runInRequest: true, maxDurationMs, reserveMs: 100, waitUntil: (work) => waited.push(work) });
|
||||
const { agent, seen } = recordingAgent(durability);
|
||||
const session = agent.session();
|
||||
// With the short deadline, a step outlasts the request.
|
||||
if (maxDurationMs < 1000) {
|
||||
const slow = holdStep("image-1");
|
||||
void slow.holding.then(() => setTimeout(slow.release, 600));
|
||||
}
|
||||
const messageId = `turn-image-${maxDurationMs}`;
|
||||
const turn = session.prompt([{ type: "text", text: "look" }, image], { messageId });
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
assert.equal(waited.length, 1, "the request ran the turn");
|
||||
assert.deepEqual(seen, [[{ type: "text", text: "look" }, image]], "with its image, read once");
|
||||
const sent = broker.sent.filter((message) => message.payload?.input?.messageId === messageId);
|
||||
assert.equal(sent[0].attempt, 0, "the backstop waited");
|
||||
if (maxDurationMs < 1000) assert.ok(sent.length > 1, "a turn longer than the request went on in a fresh message");
|
||||
for (const message of sent) {
|
||||
const payload = JSON.stringify(message.payload);
|
||||
assert.ok(!payload.includes(png), "a queued copy holds none of the image");
|
||||
}
|
||||
await waited[0];
|
||||
for (let tries = 0; internals(agent).liveWorkers() > 0; tries += 1) {
|
||||
assert.ok(tries < 200, "the request's worker never let go");
|
||||
await sleep(10);
|
||||
}
|
||||
// The backstop, delivered at last, runs nothing.
|
||||
broker.lapse(sent[0]);
|
||||
for (let tries = 0; broker.held(() => true).some((message) => message.id === sent[0].id); tries += 1) {
|
||||
assert.ok(tries < 200, "the backstop was never acknowledged");
|
||||
await sleep(10);
|
||||
}
|
||||
assert.equal(seen.length, 1, "the backstop ran nothing");
|
||||
await agent.close();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
test("a reconnect reads the stream from its cursor's line, not its start", async () => {
|
||||
@@ -1052,6 +1094,421 @@ test("each prompt's turn runs with its session object's settings, resumed or not
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
// A real 1x1 PNG, as a caller would send it.
|
||||
const png = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jP0cAAAAASUVORK5CYII=";
|
||||
const pngBytes = Uint8Array.from(atob(png), (char) => char.charCodeAt(0));
|
||||
const image = { type: "image", data: png, mimeType: "image/png" };
|
||||
// An image block of its own: the PNG with byte `n` after it, so its name is
|
||||
// new and storing it writes it.
|
||||
const imageOf = (n) => ({ type: "image", data: Buffer.from([...pngBytes, n]).toString("base64"), mimeType: "image/png" });
|
||||
// An agent whose harness records each prompt input it is handed, and runs an
|
||||
// image prompt as the word "image"; with `refuseFirst`, its first prompt
|
||||
// throws, as a harness that cannot run a turn does.
|
||||
function recordingAgent(durability, { refuseFirst = false } = {}) {
|
||||
const seen = [];
|
||||
let refusals = refuseFirst ? 1 : 0;
|
||||
const agent = createDurableAgentFactory({
|
||||
harness: () => {
|
||||
const base = scriptedHarness()();
|
||||
return {
|
||||
open: async (options) => {
|
||||
const session = await base.open(options);
|
||||
return {
|
||||
...session,
|
||||
prompt: (input, opts) => {
|
||||
seen.push(input);
|
||||
if (refusals > 0) {
|
||||
refusals -= 1;
|
||||
throw new Error("the harness stopped");
|
||||
}
|
||||
return session.prompt(typeof input === "string" ? input : "image", opts);
|
||||
},
|
||||
get openTurn() { return session.openTurn; },
|
||||
};
|
||||
},
|
||||
};
|
||||
},
|
||||
defaultDurability: async () => memory(),
|
||||
})({ durability });
|
||||
return { agent, seen };
|
||||
}
|
||||
const kinds = (seen) => seen.map((input) => (typeof input === "string" ? input : "image"));
|
||||
|
||||
test("a durable prompt carries image data as base64: bytes and files throw from prompt(), and nothing is queued", async () => {
|
||||
const { agent, seen } = recordingAgent(await durabilityFor());
|
||||
const session = agent.session();
|
||||
const refused = [
|
||||
{ type: "image", data: 42, mimeType: "image/png" },
|
||||
{ type: "image", data: new Uint8Array(pngBytes), mimeType: "image/png" },
|
||||
{ type: "image", data: new Uint8Array(pngBytes).buffer, mimeType: "image/png" },
|
||||
{ type: "image", data: new Blob([pngBytes], { type: "image/png" }) },
|
||||
{ type: "image", data: new File([pngBytes], "shot.png", { type: "image/png" }) },
|
||||
];
|
||||
for (const block of refused) {
|
||||
assert.throws(
|
||||
() => session.prompt([{ type: "text", text: "look" }, block]),
|
||||
(error) => error instanceof TypeError && /^a durable prompt carries image data as a base64 string$/.test(error.message),
|
||||
);
|
||||
}
|
||||
const turn = session.prompt([{ type: "text", text: "look" }, image, { ...image, sourceRef: "uploads:shot" }]);
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
assert.deepEqual(seen, [[{ type: "text", text: "look" }, image, { ...image, sourceRef: "uploads:shot" }]], "base64 passes as given");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a prompt storing its image is stored before the prompts made after it, and a failed image write stores nothing", async () => {
|
||||
const fault = { writeMs: 150 };
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), fault));
|
||||
const session = agent.session();
|
||||
const first = session.prompt([imageOf(1)]);
|
||||
const second = session.prompt("second");
|
||||
assert.deepEqual([(await first.result).stopReason, (await second.result).stopReason], ["end_turn", "end_turn"]);
|
||||
assert.deepEqual(kinds(seen), ["image", "second"]);
|
||||
fault.failWrites = 1;
|
||||
const failing = session.prompt([imageOf(2)]);
|
||||
const after = session.prompt("after");
|
||||
await assert.rejects(failing.accepted, /^Error: the store is full$/);
|
||||
await assert.rejects(failing.result, /^Error: the store is full$/);
|
||||
assert.equal((await after.result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["image", "second", "after"], "the failed write stored no turn, and the prompt behind it ran");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a prompt's signal or the agent closing while its image is stored keeps its turn from running", async () => {
|
||||
const fault = { writeMs: 400 };
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), fault));
|
||||
const session = agent.session();
|
||||
const controller = new AbortController();
|
||||
const cancelled = session.prompt([imageOf(3)], { messageId: "turn-store", signal: controller.signal });
|
||||
await sleep(20);
|
||||
const aborted = Date.now();
|
||||
controller.abort();
|
||||
await cancelled.accepted;
|
||||
assert.equal((await cancelled.result).stopReason, "cancelled");
|
||||
assert.ok(Date.now() - aborted < 300, "the prompt stopped waiting for its image write at once");
|
||||
// Neither the prompt nor a cancel naming it was stored: its id runs anew.
|
||||
assert.equal((await session.prompt("after", { messageId: "turn-store" }).result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["after"]);
|
||||
fault.writeMs = 150;
|
||||
const closing = session.prompt([imageOf(4)]);
|
||||
await agent.close();
|
||||
await assert.rejects(closing.accepted, /agent is closed/);
|
||||
await assert.rejects(closing.result, /agent is closed/);
|
||||
assert.deepEqual(kinds(seen), ["after"]);
|
||||
});
|
||||
|
||||
test("a prompt withdrawn while it waits keeps the prompts behind it waiting for the ones before it", async () => {
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), { writeMs: 300 }));
|
||||
const session = agent.session();
|
||||
const first = session.prompt([imageOf(5)], { messageId: "turn-first" });
|
||||
const controller = new AbortController();
|
||||
const second = session.prompt([imageOf(6)], { messageId: "turn-second", signal: controller.signal });
|
||||
const third = session.prompt("third", { messageId: "turn-third" });
|
||||
await sleep(20);
|
||||
controller.abort();
|
||||
assert.equal((await second.result).stopReason, "cancelled");
|
||||
assert.equal((await first.result).stopReason, "end_turn");
|
||||
assert.equal((await third.result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["image", "third"], "the prompt behind the withdrawn one still came after the one before it");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("an image prompt its first worker could not start runs from the log with the same base64, once", async () => {
|
||||
const { agent, seen } = recordingAgent(await durabilityFor(), { refuseFirst: true });
|
||||
const session = agent.session();
|
||||
const shot = { ...image, sourceRef: "uploads:shot" };
|
||||
assert.equal((await session.prompt([shot], { messageId: "turn-img" }).result).stopReason, "end_turn");
|
||||
assert.deepEqual(seen, [[shot], [shot]]);
|
||||
const again = await agent.session(session.id).prompt([shot], { messageId: "turn-img" }).result;
|
||||
assert.equal(again.repeated, true);
|
||||
assert.equal(seen.length, 2, "the retry ran nothing");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("an image block is read once, so the log holds what was checked", async () => {
|
||||
const { agent, seen } = recordingAgent(await durabilityFor());
|
||||
const session = agent.session();
|
||||
// Accessors that answer one way when checked and another way after.
|
||||
let dataReads = 0;
|
||||
let mimeReads = 0;
|
||||
const shifting = {
|
||||
type: "image",
|
||||
get data() { dataReads += 1; return dataReads === 1 ? png : new Uint8Array([1, 2, 3]); },
|
||||
mimeType: "image/png",
|
||||
sourceRef: "uploads:shifting",
|
||||
};
|
||||
const relabelled = {
|
||||
type: "image",
|
||||
data: png,
|
||||
get mimeType() { mimeReads += 1; return mimeReads === 1 ? "image/png" : undefined; },
|
||||
};
|
||||
assert.equal((await session.prompt([{ type: "text", text: "look" }, shifting, relabelled]).result).stopReason, "end_turn");
|
||||
assert.deepEqual(seen.at(-1), [{ type: "text", text: "look" }, { ...image, sourceRef: "uploads:shifting" }, image]);
|
||||
assert.deepEqual([dataReads, mimeReads], [1, 1]);
|
||||
// An Array subclass whose map() returns something else still stores its blocks.
|
||||
class Remapped extends Array { map() { return "not the caller's blocks"; } }
|
||||
const blocks = Remapped.from([{ type: "text", text: "remapped" }, image]);
|
||||
assert.equal((await session.prompt(blocks).result).stopReason, "end_turn");
|
||||
assert.deepEqual(seen.at(-1), [{ type: "text", text: "remapped" }, image]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
const pngName = Buffer.from(await crypto.subtle.digest("SHA-256", pngBytes)).toString("hex");
|
||||
// Every image stream's name holds this: "_user_", then the base64url of the
|
||||
// namespace "libfx-image-".
|
||||
const imageStreamTag = `_user_${Buffer.from("libfx-image-").toString("base64url")}`;
|
||||
|
||||
// The durability with a faulty image store: with `fault.lose`, every image
|
||||
// reads back as missing; with `fault.corrupt`, every image reads back with
|
||||
// its last byte changed; `fault.failWrites` fails that many image writes;
|
||||
// `fault.writeMs` makes each image write wait that long first. The test may
|
||||
// change `fault` as it goes. A World's faults are in its streams, under the
|
||||
// code that names images.
|
||||
function faultyImages(durability, fault = {}) {
|
||||
fault.failWrites ??= 0;
|
||||
const slowly = async () => { if (fault.writeMs) await sleep(fault.writeMs); };
|
||||
const refuse = () => {
|
||||
fault.failWrites -= 1;
|
||||
throw new Error("the store is full");
|
||||
};
|
||||
const flipped = (bytes) => {
|
||||
if (!(bytes instanceof Uint8Array) || bytes.byteLength === 0) return bytes;
|
||||
const copy = bytes.slice();
|
||||
copy[copy.length - 1] ^= 0xff;
|
||||
return copy;
|
||||
};
|
||||
if (typeof durability.create === "function") {
|
||||
return {
|
||||
...durability,
|
||||
create() {
|
||||
const backend = durability.create();
|
||||
return {
|
||||
...backend,
|
||||
async session(id) {
|
||||
const log = await backend.session(id);
|
||||
return {
|
||||
...log,
|
||||
images: {
|
||||
put: async (name, bytes) => {
|
||||
await slowly();
|
||||
return fault.failWrites > 0 ? refuse() : log.images.put(name, bytes);
|
||||
},
|
||||
get: async (name) => {
|
||||
if (fault.lose) return undefined;
|
||||
const bytes = await log.images.get(name);
|
||||
return fault.corrupt ? flipped(bytes) : bytes;
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
const isImage = (name) => String(name).includes(imageStreamTag);
|
||||
const bound = (target, value) => (typeof value === "function" ? value.bind(target) : value);
|
||||
return {
|
||||
...durability,
|
||||
async world() {
|
||||
const world = await durability.world();
|
||||
const streams = new Proxy(world.streams, {
|
||||
get(target, prop) {
|
||||
const value = Reflect.get(target, prop);
|
||||
if (prop === "write") {
|
||||
return async (runId, name, chunk) => {
|
||||
if (!isImage(name)) return value.call(target, runId, name, chunk);
|
||||
await slowly();
|
||||
return fault.failWrites > 0 ? refuse() : value.call(target, runId, name, chunk);
|
||||
};
|
||||
}
|
||||
if (prop === "getChunks") {
|
||||
return async (runId, name, options) => {
|
||||
if (fault.lose && isImage(name)) return { data: [], cursor: null, hasMore: false, done: false };
|
||||
const result = await value.call(target, runId, name, options);
|
||||
if (!fault.corrupt || !isImage(name) || !Array.isArray(result?.data)) return result;
|
||||
return { ...result, data: result.data.map((chunk) => ({ ...chunk, data: flipped(chunk?.data) })) };
|
||||
};
|
||||
}
|
||||
return bound(target, value);
|
||||
},
|
||||
});
|
||||
return new Proxy(world, { get: (target, prop) => (prop === "streams" ? streams : bound(target, Reflect.get(target, prop))) });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Every file under a World's folder, for checking what it stored. A lock or
|
||||
// temporary file gone by the time it is read is skipped.
|
||||
async function filesUnder(dir) {
|
||||
const files = [];
|
||||
for (const entry of await readdir(dir, { withFileTypes: true, recursive: true })) {
|
||||
if (!entry.isFile()) continue;
|
||||
try {
|
||||
files.push(await readFile(join(entry.parentPath ?? entry.path, entry.name)));
|
||||
} catch (error) {
|
||||
if (error?.code !== "ENOENT") throw error;
|
||||
}
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
test("a durable prompt stores each image once in the session's image store, and the log only names it", async () => {
|
||||
const durability = await durabilityFor();
|
||||
const { agent, seen } = recordingAgent(durability);
|
||||
const session = agent.session();
|
||||
const turn = session.prompt([
|
||||
{ type: "text", text: "look" },
|
||||
image,
|
||||
{ ...image, sourceRef: "uploads:shot" },
|
||||
image,
|
||||
]);
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
assert.deepEqual(seen, [[{ type: "text", text: "look" }, image, { ...image, sourceRef: "uploads:shot" }, image]]);
|
||||
if (typeof durability.create === "function") {
|
||||
const log = await durability.create().session(session.id);
|
||||
const entries = (await log.readSince("0")).map(({ entry }) => entry);
|
||||
assert.deepEqual(entries.find((entry) => entry.k === "input")?.input, [
|
||||
{ type: "text", text: "look" },
|
||||
{ type: "image", mimeType: "image/png", data: { sha256: pngName } },
|
||||
{ type: "image", mimeType: "image/png", sourceRef: "uploads:shot", data: { sha256: pngName } },
|
||||
{ type: "image", mimeType: "image/png", data: { sha256: pngName } },
|
||||
]);
|
||||
assert.ok(!JSON.stringify(entries).includes(png), "no entry holds the image's base64");
|
||||
assert.deepEqual(await log.images.get(pngName), pngBytes);
|
||||
}
|
||||
await agent.close();
|
||||
if (typeof durability.create !== "function") {
|
||||
// On disk: the image's bytes once, in its stream, and its base64 nowhere.
|
||||
const files = await filesUnder(dirs.at(-1));
|
||||
assert.ok(!files.some((bytes) => bytes.includes(Buffer.from(png))), "no file holds the image's base64");
|
||||
assert.equal(files.filter((bytes) => bytes.includes(Buffer.from(pngBytes))).length, 1, "one file holds the image's bytes");
|
||||
}
|
||||
});
|
||||
|
||||
test("an image the session's store no longer holds fails its turn, naming it", async () => {
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), { lose: true }));
|
||||
const session = agent.session();
|
||||
const result = await session.prompt([{ type: "text", text: "look" }, image]).result;
|
||||
assert.equal(result.stopReason, "error");
|
||||
assert.match(result.error.message, new RegExp(`does not hold image ${pngName}`));
|
||||
assert.deepEqual(seen, [], "the harness never ran it");
|
||||
assert.equal((await session.prompt("after").result).stopReason, "end_turn");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("an image whose stored bytes changed fails its turn, naming it", async () => {
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), { corrupt: true }));
|
||||
const session = agent.session();
|
||||
const result = await session.prompt([{ type: "text", text: "look" }, image]).result;
|
||||
assert.equal(result.stopReason, "error");
|
||||
// A World's read skips bytes that don't match the name, so to it the image
|
||||
// is missing; a store that returns them fails the check after the read.
|
||||
assert.match(result.error.message, new RegExp(`(holds other bytes under|does not hold) image ${pngName}`));
|
||||
assert.deepEqual(seen, [], "the harness never ran it");
|
||||
assert.equal((await session.prompt("after").result).stopReason, "end_turn");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a World whose streams lack getChunks refuses an image prompt before storing it, and text still runs", async () => {
|
||||
const durability = await durabilityFor();
|
||||
// memory() keeps images itself; the others keep them in World streams.
|
||||
if (typeof durability.world !== "function") return;
|
||||
const bound = (target, value) => (typeof value === "function" ? value.bind(target) : value);
|
||||
const withoutGetChunks = {
|
||||
...durability,
|
||||
async world() {
|
||||
const world = await durability.world();
|
||||
const streams = new Proxy(world.streams, { get: (target, prop) => (prop === "getChunks" ? undefined : bound(target, Reflect.get(target, prop))) });
|
||||
return new Proxy(world, { get: (target, prop) => (prop === "streams" ? streams : bound(target, Reflect.get(target, prop))) });
|
||||
},
|
||||
};
|
||||
const { agent, seen } = recordingAgent(withoutGetChunks);
|
||||
const session = agent.session();
|
||||
const refused = session.prompt([{ type: "text", text: "look" }, image]);
|
||||
await assert.rejects(refused.accepted, (error) => error instanceof TypeError && /getChunks/.test(error.message));
|
||||
await assert.rejects(refused.result);
|
||||
assert.equal((await session.prompt("text only").result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["text only"]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a failed image write rejects the prompt before anything is stored", async () => {
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), { failWrites: 1 }));
|
||||
const session = agent.session();
|
||||
const refused = session.prompt([{ type: "text", text: "look" }, image], { messageId: "refused-img" });
|
||||
// The store's own error.
|
||||
const storeFailed = (error) => error.message === "the store is full";
|
||||
await assert.rejects(refused.accepted, storeFailed);
|
||||
await assert.rejects(refused.result, storeFailed);
|
||||
assert.equal((await session.prompt([{ type: "text", text: "again" }, image]).result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["image"], "only the second prompt was stored and ran");
|
||||
// The refused prompt was never stored, so its retry is a new turn.
|
||||
const retried = await session.prompt([{ type: "text", text: "look" }, image], { messageId: "refused-img" }).result;
|
||||
assert.equal(retried.stopReason, "end_turn");
|
||||
assert.notEqual(retried.repeated, true);
|
||||
assert.deepEqual(kinds(seen), ["image", "image"]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a failed image write rejects a steer before it is sent", async () => {
|
||||
const fault = {};
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), fault));
|
||||
const session = agent.session();
|
||||
assert.equal((await session.prompt("first").result).stopReason, "end_turn");
|
||||
fault.failWrites = 1;
|
||||
// The store's own error.
|
||||
await assert.rejects(session.steer([{ type: "text", text: "look" }, image]), (error) => error.message === "the store is full");
|
||||
// The steer was never sent, so no turn ran for it before the next prompt.
|
||||
assert.equal((await session.prompt("after").result).stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["first", "after"]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a retried image prompt the log already holds is answered without writing its images again", async () => {
|
||||
const fault = {};
|
||||
const { agent, seen } = recordingAgent(faultyImages(await durabilityFor(), fault));
|
||||
const session = agent.session();
|
||||
assert.equal((await session.prompt([{ type: "text", text: "look" }, image], { messageId: "img-once" }).result).stopReason, "end_turn");
|
||||
// From now on every image write fails, so a retry that wrote again would fail.
|
||||
fault.failWrites = Infinity;
|
||||
const again = await agent.session(session.id).prompt([{ type: "text", text: "look" }, image], { messageId: "img-once" }).result;
|
||||
assert.equal(again.repeated, true);
|
||||
assert.equal(again.stopReason, "end_turn");
|
||||
assert.deepEqual(kinds(seen), ["image"], "the retry ran nothing");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("base64 is the same with and without the runtime's own encoder and decoder", async () => {
|
||||
const reference = (bytes) => btoa(String.fromCharCode(...bytes));
|
||||
const inputs = [...Array.from({ length: 9 }, (_, n) => pngBytes.subarray(0, n + 1)), Uint8Array.from({ length: 3000 }, (_, i) => (i * 131) & 255)];
|
||||
const run = async () => {
|
||||
const { agent, seen } = recordingAgent(await durabilityFor());
|
||||
const session = agent.session();
|
||||
// Each image is decoded to be stored, and encoded again for the harness.
|
||||
for (const bytes of inputs) assert.equal((await session.prompt([{ type: "image", data: reference(bytes), mimeType: "image/png" }]).result).stopReason, "end_turn");
|
||||
// Each turn's records went through the log as base64 and back.
|
||||
const words = JSON.parse(decoder.decode(await session.checkpoint()));
|
||||
await agent.close();
|
||||
return { seen, words };
|
||||
};
|
||||
const native = await run();
|
||||
const encode = Uint8Array.prototype.toBase64;
|
||||
const decode = Uint8Array.fromBase64;
|
||||
delete Uint8Array.prototype.toBase64;
|
||||
delete Uint8Array.fromBase64;
|
||||
let fallback;
|
||||
try {
|
||||
fallback = await run();
|
||||
} finally {
|
||||
if (encode) Uint8Array.prototype.toBase64 = encode;
|
||||
if (decode) Uint8Array.fromBase64 = decode;
|
||||
}
|
||||
for (const result of [native, fallback]) {
|
||||
assert.deepEqual(result.seen.map((input) => input[0].data), inputs.map(reference));
|
||||
assert.equal(result.words.length, inputs.length * 3);
|
||||
}
|
||||
});
|
||||
|
||||
test("a worker lives only while a delivery runs it", async () => {
|
||||
const agent = createAgent({ durability: await durabilityFor() });
|
||||
const internals = agent[Symbol.for("libfx.durableInternals")];
|
||||
|
||||
+348
-11
@@ -5,7 +5,8 @@
|
||||
// node --experimental-wasm-jspi sdk/tests/test-durable.mjs [memory|local] [native|wasm]
|
||||
import { strict as assert } from "node:assert";
|
||||
import { spawn } from "node:child_process";
|
||||
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, readdir, readFile, rm } from "node:fs/promises";
|
||||
import { createServer } from "node:http";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
@@ -61,6 +62,8 @@ const crashSteer = (kind) => (kind === "image"
|
||||
? [{ type: "text", text: "use lookup with this" }, { type: "image", data: onePixelPng, mimeType: "image/png" }]
|
||||
: "use lookup");
|
||||
const resumeNotice = "Resuming from unexpected session interruption.";
|
||||
// The user message the gateway adds after a tool result's images.
|
||||
const toolImageFollowUp = "Attached image(s) from the tool result.";
|
||||
|
||||
const textOf = (message) => typeof message.content === "string"
|
||||
? message.content
|
||||
@@ -81,7 +84,7 @@ const answer = (text) => [{ type: "text-delta", id: "answer", delta: text }, fin
|
||||
// lookup call's input a little at a time for a few seconds; anything else is
|
||||
// echoed.
|
||||
function framesFor(prompt) {
|
||||
const users = userTexts(prompt).filter((text) => text !== resumeNotice);
|
||||
const users = userTexts(prompt).filter((text) => text !== resumeNotice && text !== toolImageFollowUp);
|
||||
const text = users.at(-1) ?? "";
|
||||
const results = toolResults(prompt);
|
||||
const turnResults = results.filter((part) => String(part.toolCallId).startsWith(`call-${users.length}-`));
|
||||
@@ -121,8 +124,11 @@ const server = createServer((request, response) => {
|
||||
request.on("end", () => {
|
||||
if (request.method === "GET") {
|
||||
response.writeHead(200, { "content-type": "application/json" });
|
||||
// It takes images, for the steers that carry one.
|
||||
response.end(JSON.stringify({ object: "list", data: [{ id: "durable/model", type: "language", tags: ["vision", "file-input"] }] }));
|
||||
response.end(JSON.stringify({ object: "list", data: [
|
||||
// It takes images, for the steers that carry one.
|
||||
{ id: "durable/model", type: "language", tags: ["vision", "file-input"] },
|
||||
{ id: "durable/vision-model", type: "language", tags: ["tool-use", "vision", "file-input"] },
|
||||
] }));
|
||||
return;
|
||||
}
|
||||
const prompt = JSON.parse(body).prompt;
|
||||
@@ -1671,19 +1677,332 @@ test("an option the backend rejects fails the first turn once", async () => {
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a prompt the engine refuses ends its own turn once, and the session goes on", async () => {
|
||||
const agent = createFxAgent(agentOptions(await durabilityFor()));
|
||||
// A 1x1 PNG, as bytes, for the image tests below.
|
||||
const pngBase64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jP0cAAAAASUVORK5CYII=";
|
||||
const pngBytes = () => Uint8Array.from(atob(pngBase64), (char) => char.charCodeAt(0));
|
||||
|
||||
test("a prompt the engine would refuse throws, stores nothing, and the session goes on", async () => {
|
||||
const agent = createFxAgent(agentOptions(await durabilityFor(), { model: "durable/vision-model" }));
|
||||
const session = agent.session();
|
||||
const empty = await session.prompt("").result;
|
||||
assert.equal(empty.stopReason, "error");
|
||||
const malformed = await session.prompt([{ type: "text" }]).result;
|
||||
assert.equal(malformed.stopReason, "error");
|
||||
const { text, result } = await collect(session.prompt("hello again"));
|
||||
const image = (data, mimeType = "image/png") => ({ type: "image", data, mimeType });
|
||||
for (const [input, refusal] of [
|
||||
["", /prompt is empty/],
|
||||
[[{ type: "text" }], /text prompt block 0 requires text/],
|
||||
[[{ type: "text", text: "look" }, image("not base64!")], /image prompt block 1 requires canonical base64 data/],
|
||||
[[{ type: "text", text: "look" }, image(pngBytes())], /a durable prompt carries image data as a base64 string/],
|
||||
[[{ type: "text", text: "look" }, image(pngBase64, "image/jpeg")], /image prompt block 1 holds image\/png data, not image\/jpeg/],
|
||||
[[{ type: "text", text: "look" }, image(pngBase64, "image/gif")], /image prompt block 1 holds image\/png data, not image\/gif/],
|
||||
[[{ type: "text", text: "look" }, image(Buffer.from([1, 2, 3, 4]).toString("base64"))], /image prompt block 1 is not a PNG, JPEG, GIF or WebP image/],
|
||||
[[{ type: "text", text: "look" }, image(pngBase64, "image/bmp")], /image prompt block 1 mimeType must be image\/png, image\/jpeg, image\/gif or image\/webp/],
|
||||
[[{ type: "text", text: "look" }, ...Array.from({ length: 9 }, () => image(pngBase64))], /prompt cannot contain more than 8 images/],
|
||||
]) {
|
||||
assert.throws(() => session.prompt(input, { messageId: "refused" }), refusal);
|
||||
}
|
||||
// Nothing was stored under that id, so a prompt with it runs.
|
||||
const { text, result } = await collect(session.prompt("hello again", { messageId: "refused" }));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.equal(text, "echo: hello again");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a durable prompt's images reach the model as the base64 it was given", async () => {
|
||||
const png = pngBase64;
|
||||
const agent = createFxAgent(agentOptions(await durabilityFor(), { model: "durable/vision-model" }));
|
||||
const { text, result } = await collect(agent.session().prompt([
|
||||
{ type: "text", text: "look" },
|
||||
{ type: "image", data: png, mimeType: "image/png" },
|
||||
{ type: "image", data: png, mimeType: "image/png", sourceRef: "uploads:shot" },
|
||||
]));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
// The model hears each image's placeholder alongside its text.
|
||||
assert.equal(text, "echo: look\n[Image #1]\n[Image #2]");
|
||||
const files = chats.at(-1).prompt
|
||||
.filter((message) => message.role === "user" && Array.isArray(message.content))
|
||||
.flatMap((message) => message.content)
|
||||
.filter((part) => part.type === "file");
|
||||
const file = { type: "file", mediaType: "image/png", data: { type: "data", data: png } };
|
||||
assert.deepEqual(files, [file, file]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("an image read that fails while the engine opens runs the turn again instead of failing it", async () => {
|
||||
const base = await durabilityFor();
|
||||
// memory() hands out its sessions directly, so one image read can fail.
|
||||
if (typeof base.create !== "function") return;
|
||||
const fault = { failReads: 0 };
|
||||
const durability = {
|
||||
...base,
|
||||
create() {
|
||||
const backend = base.create();
|
||||
return {
|
||||
...backend,
|
||||
async session(id) {
|
||||
const log = await backend.session(id);
|
||||
return {
|
||||
...log,
|
||||
images: {
|
||||
put: (name, bytes) => log.images.put(name, bytes),
|
||||
get: async (name) => {
|
||||
if (fault.failReads > 0) {
|
||||
fault.failReads -= 1;
|
||||
throw new Error("the image store did not answer");
|
||||
}
|
||||
return log.images.get(name);
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
const png = pngBase64;
|
||||
const first = createFxAgent(agentOptions(durability, { model: "durable/vision-model" }));
|
||||
const session = first.session();
|
||||
assert.equal((await session.prompt([{ type: "text", text: "look" }, { type: "image", data: png, mimeType: "image/png" }]).result).stopReason, "end_turn");
|
||||
await first.close();
|
||||
// A new agent's engine reads the session's image as it opens; that read
|
||||
// fails once.
|
||||
fault.failReads = 1;
|
||||
const second = createFxAgent(agentOptions(durability, { model: "durable/vision-model" }));
|
||||
const { result } = await collect(second.session(session.id).prompt("and now"));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.equal(fault.failReads, 0, "the failed read happened");
|
||||
const files = chats.at(-1).prompt
|
||||
.filter((message) => message.role === "user" && Array.isArray(message.content))
|
||||
.flatMap((message) => message.content)
|
||||
.filter((part) => part.type === "file");
|
||||
assert.deepEqual(files.map((part) => part.data?.data), [png]);
|
||||
await second.close();
|
||||
});
|
||||
|
||||
test("an engine given an image the log names refuses it, so a worker that can't read names fails the turn", async () => {
|
||||
// The log holds a stored image as `data: { sha256 }`, which an engine that
|
||||
// predates names can't read, with or without a sourceRef: such a worker
|
||||
// fails the turn instead of running it without the image.
|
||||
const { durability: _durability, tools: _tools, ...engineOptions } = agentOptions(null, { model: "durable/vision-model" });
|
||||
const engine = await createFxEngine(engineOptions);
|
||||
const named = { type: "image", mimeType: "image/png", data: { sha256: "a".repeat(64) } };
|
||||
for (const block of [named, { ...named, sourceRef: "uploads:shot" }]) {
|
||||
assert.throws(() => engine.prompt([{ type: "text", text: "look" }, block]), /requires base64 data, bytes, or a Blob/);
|
||||
}
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test("a durable prompt checks the copy its log holds, so a getter can't pass one image and store another", async () => {
|
||||
const agent = createFxAgent(agentOptions(await durabilityFor(), { model: "durable/vision-model" }));
|
||||
let reads = 0;
|
||||
const block = { type: "image", mimeType: "image/png" };
|
||||
Object.defineProperty(block, "data", { enumerable: true, get: () => (reads++ === 0 ? pngBase64 : Buffer.from([1, 2, 3, 4]).toString("base64")) });
|
||||
const { result } = await collect(agent.session().prompt([{ type: "text", text: "look" }, block]));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.equal(reads, 1, "the block was read once");
|
||||
const files = chats.at(-1).prompt.flatMap((message) => Array.isArray(message.content) ? message.content : []).filter((part) => part.type === "file");
|
||||
assert.deepEqual(files.map((part) => part.data?.data), [pngBase64]);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a durable prompt cancelled while its image is stored is never stored, and sends no cancel", async () => {
|
||||
const base = await durabilityFor();
|
||||
// memory() hands out its sessions directly, so its image writes can be held.
|
||||
if (typeof base.create !== "function") return;
|
||||
let release;
|
||||
const held = new Promise((resolveHeld) => { release = resolveHeld; });
|
||||
const durability = {
|
||||
...base,
|
||||
create() {
|
||||
const backend = base.create();
|
||||
return {
|
||||
...backend,
|
||||
async session(id) {
|
||||
const log = await backend.session(id);
|
||||
return { ...log, images: { put: async (name, bytes) => { await held; return log.images.put(name, bytes); }, get: (name) => log.images.get(name) } };
|
||||
},
|
||||
};
|
||||
},
|
||||
};
|
||||
const agent = createFxAgent(agentOptions(durability, { model: "durable/vision-model" }));
|
||||
const session = agent.session();
|
||||
const controller = new AbortController();
|
||||
const turn = session.prompt([{ type: "text", text: "look" }, { type: "image", data: pngBase64, mimeType: "image/png" }], { messageId: "withdrawn", signal: controller.signal });
|
||||
await new Promise((resolveWait) => setTimeout(resolveWait, 20));
|
||||
controller.abort();
|
||||
// The prompt stops waiting for its image write at once.
|
||||
await turn.accepted;
|
||||
assert.equal((await turn.result).stopReason, "cancelled");
|
||||
release();
|
||||
// Neither the prompt nor a cancel naming it was stored: one stored would
|
||||
// answer a prompt with its id from the log, and a cancel would end it.
|
||||
const { text, result } = await collect(session.prompt("hello again", { messageId: "withdrawn" }));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.equal(text, "echo: hello again");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("an engine refuses the same prompts in prompt(), before its core sees them", async () => {
|
||||
const { durability: _durability, tools: _tools, ...engineOptions } = agentOptions(null, { model: "durable/vision-model" });
|
||||
const engine = await createFxEngine(engineOptions);
|
||||
assert.throws(() => engine.prompt(""), /prompt is empty/);
|
||||
assert.throws(() => engine.prompt([{ type: "text", text: "look" }, { type: "image", data: pngBytes(), mimeType: "image/jpeg" }]), /holds image\/png data, not image\/jpeg/);
|
||||
await assert.rejects(engine.prompt([{ type: "text", text: "look" }, { type: "image", data: new File([pngBytes()], "shot.jpg", { type: "image/jpeg" }) }]).result, /holds image\/png data, not image\/jpeg/);
|
||||
const { result } = await collect(engine.prompt("hello"));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
await engine.close();
|
||||
});
|
||||
|
||||
test("a durable session saves each image once, and a new agent sends it again", async () => {
|
||||
// A PNG header, then filler, so its bytes are easy to look for.
|
||||
const header = Uint8Array.from(atob("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jP0cAAAAASUVORK5CYII="), (char) => char.charCodeAt(0));
|
||||
const bytes = new Uint8Array(300_000);
|
||||
bytes.set(header);
|
||||
for (let index = header.length; index < bytes.length; index += 1) bytes[index] = (index * 31 + 7) & 255;
|
||||
const base64 = Buffer.from(bytes).toString("base64");
|
||||
const durability = await durabilityFor();
|
||||
const holdsImage = (data) => data.includes(Buffer.from(base64.slice(1000, 1100))) || data.includes(Buffer.from(bytes.subarray(1000, 1064)));
|
||||
const savedData = (entry) => (entry.k === "record" || entry.k === "checkpoint" ? Buffer.from(entry.data, "base64") : Buffer.from(JSON.stringify(entry)));
|
||||
// A remote World keeps nothing on this machine, so what reaches it is
|
||||
// watched instead: each entry the log appends, and each stream write.
|
||||
const sent = { entries: [], imageWrites: 0 };
|
||||
if (remote) {
|
||||
const open = durability.world.bind(durability);
|
||||
durability.world = async () => {
|
||||
const world = await open();
|
||||
const create = world.events.create.bind(world.events);
|
||||
world.events.create = async (runId, event, ...rest) => {
|
||||
const input = event?.eventData?.input;
|
||||
if (input instanceof Uint8Array) {
|
||||
try {
|
||||
const entry = JSON.parse(Buffer.from(input).toString("utf8"));
|
||||
if (typeof entry?.k === "string") sent.entries.push(entry);
|
||||
} catch {}
|
||||
}
|
||||
return create(runId, event, ...rest);
|
||||
};
|
||||
const write = world.streams.write.bind(world.streams);
|
||||
world.streams.write = async (runId, stream, data, ...rest) => {
|
||||
if (data instanceof Uint8Array && Buffer.from(data).equals(Buffer.from(bytes))) sent.imageWrites += 1;
|
||||
return write(runId, stream, data, ...rest);
|
||||
};
|
||||
return world;
|
||||
};
|
||||
}
|
||||
const first = createFxAgent(agentOptions(durability, { model: "durable/vision-model" }));
|
||||
const session = first.session();
|
||||
// Records that only name the image stay small, so the second prompt is
|
||||
// `bulky`: its turn's end saves a checkpoint, which must name the images too.
|
||||
for (const text of ["look", bulky]) {
|
||||
const { result } = await collect(session.prompt([{ type: "text", text }, { type: "image", data: base64, mimeType: "image/png" }]));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
}
|
||||
// A turn's checkpoint is saved after its end, and close() waits for it.
|
||||
await first.close();
|
||||
if (durabilityKind === "memory" || remote) {
|
||||
// Every entry, records and checkpoints decoded: names only.
|
||||
const entries = remote ? sent.entries : (await (await durability.create().session(session.id)).readSince("0")).map(({ entry }) => entry);
|
||||
assert.ok(entries.some((entry) => entry.k === "checkpoint"));
|
||||
assert.ok(!entries.map(savedData).some(holdsImage), "no entry holds the image");
|
||||
}
|
||||
if (remote) {
|
||||
assert.equal(sent.imageWrites, 1, "the World got the image's bytes once");
|
||||
} else if (durabilityKind !== "memory") {
|
||||
const files = [];
|
||||
for (const entry of await readdir(dirs.at(-1), { withFileTypes: true, recursive: true })) {
|
||||
if (!entry.isFile()) continue;
|
||||
try {
|
||||
files.push(await readFile(join(entry.parentPath ?? entry.path, entry.name)));
|
||||
} catch (error) {
|
||||
if (error?.code !== "ENOENT") throw error;
|
||||
}
|
||||
}
|
||||
assert.ok(!files.some((data) => data.includes(Buffer.from(base64.slice(1000, 1100)))), "no file holds the image as base64");
|
||||
assert.equal(files.filter((data) => data.includes(Buffer.from(bytes.subarray(1000, 1064)))).length, 1, "one file holds the image's bytes");
|
||||
}
|
||||
|
||||
const second = createFxAgent(agentOptions(durabilityKind === "memory" ? durability : await durabilityFor({ dir: dirs.at(-1) }), { model: "durable/vision-model" }));
|
||||
const { result } = await collect(second.session(session.id).prompt("and now"));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
const files = chats.at(-1).prompt
|
||||
.filter((message) => message.role === "user" && Array.isArray(message.content))
|
||||
.flatMap((message) => message.content)
|
||||
.filter((part) => part.type === "file");
|
||||
assert.deepEqual(files.map((part) => part.data?.data), [base64, base64]);
|
||||
await second.close();
|
||||
});
|
||||
|
||||
// A PNG of 300 KB with filler after its header, so its bytes are easy to
|
||||
// look for, as base64.
|
||||
const shotBytes = (() => {
|
||||
const header = Uint8Array.from(atob(onePixelPng), (char) => char.charCodeAt(0));
|
||||
const bytes = new Uint8Array(300_000);
|
||||
bytes.set(header);
|
||||
for (let index = header.length; index < bytes.length; index += 1) bytes[index] = (index * 17 + 3) & 255;
|
||||
return bytes;
|
||||
})();
|
||||
const shotBase64 = Buffer.from(shotBytes).toString("base64");
|
||||
const shotName = createHash("sha256").update(shotBytes).digest("hex");
|
||||
const holdsShot = (data) => data.includes(Buffer.from(shotBase64.slice(1000, 1100))) || data.includes(Buffer.from(shotBytes.subarray(1000, 1064)));
|
||||
const entryData = (entry) => (entry.k === "record" || entry.k === "checkpoint" ? Buffer.from(entry.data, "base64") : Buffer.from(JSON.stringify(entry)));
|
||||
// Returns the screenshot as a typed result.
|
||||
const screenshot = {
|
||||
name: "screenshot",
|
||||
description: "Takes a screenshot",
|
||||
idempotent: true,
|
||||
inputSchema: { type: "object", properties: { key: { type: "string" } } },
|
||||
execute: async () => ({ type: "libfx.tool-result", text: "the screen", images: [{ type: "image", data: shotBase64, mimeType: "image/png" }] }),
|
||||
};
|
||||
const requestFiles = (prompt) => prompt
|
||||
.filter((message) => message.role === "user" && Array.isArray(message.content))
|
||||
.flatMap((message) => message.content)
|
||||
.filter((part) => part.type === "file")
|
||||
.map((part) => part.data?.data ?? part.data);
|
||||
|
||||
test("a durable session saves a tool's image once, its log only names it, and a new agent sends it again", async () => {
|
||||
const durability = await durabilityFor();
|
||||
const withShots = { model: "durable/vision-model", tools: [lookup, send, save, screenshot] };
|
||||
const first = createFxAgent(agentOptions(durability, withShots));
|
||||
const session = first.session();
|
||||
const { result } = await collect(session.prompt("use screenshot"));
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.deepEqual(requestFiles(chats.at(-1).prompt), [shotBase64], "the model saw the screenshot");
|
||||
await first.close();
|
||||
if (durabilityKind === "memory") {
|
||||
const log = await durability.create().session(session.id);
|
||||
const entries = (await log.readSince("0")).map(({ entry }) => entry);
|
||||
assert.ok(!entries.map(entryData).some(holdsShot), "no entry holds the screenshot");
|
||||
assert.ok(Buffer.from(await log.images.get(shotName)).equals(Buffer.from(shotBytes)), "the image store holds it by name");
|
||||
}
|
||||
|
||||
const second = createFxAgent(agentOptions(durabilityKind === "memory" ? durability : await durabilityFor({ dir: dirs.at(-1) }), withShots));
|
||||
const { result: after } = await collect(second.session(session.id).prompt("and now"));
|
||||
assert.equal(after.stopReason, "end_turn");
|
||||
assert.deepEqual(requestFiles(chats.at(-1).prompt), [shotBase64], "a new agent sends the screenshot again");
|
||||
await second.close();
|
||||
});
|
||||
|
||||
test("a durable steer saves its image once, and its log entry only names it", async () => {
|
||||
const durability = await durabilityFor();
|
||||
const agent = createFxAgent(agentOptions(durability, { model: "durable/vision-model" }));
|
||||
const session = agent.session();
|
||||
const before = slow.length;
|
||||
const turn = session.prompt("slow reply");
|
||||
await turn.accepted;
|
||||
await until(() => slow.length > before && slow.at(-1).sent > 0, "the reply to stream");
|
||||
await session.steer([{ type: "text", text: "match the shot" }, { type: "image", data: shotBase64, mimeType: "image/png" }], { messageId: "m-shot" });
|
||||
await turn.result;
|
||||
await linesUntil(session, (line) => line.type === "turn_end" && line.messageId === "m-shot", "the steer's turn");
|
||||
const steered = chats.findLast(({ prompt }) => userTexts(prompt).some((text) => text.includes("match the shot")));
|
||||
assert.deepEqual(requestFiles(steered.prompt), [shotBase64], "the steer's image reached the model");
|
||||
await agent.close();
|
||||
if (durabilityKind === "memory") {
|
||||
const log = await durability.create().session(session.id);
|
||||
const entries = (await log.readSince("0")).map(({ entry }) => entry);
|
||||
const entry = entries.find((item) => item.k === "input" && item.messageId === "m-shot");
|
||||
assert.ok(entry, "the steer is in the log");
|
||||
assert.deepEqual(entry.input.find((block) => block.type === "image").data, { sha256: shotName });
|
||||
assert.ok(!entries.map(entryData).some(holdsShot), "no entry holds the image");
|
||||
assert.ok(Buffer.from(await log.images.get(shotName)).equals(Buffer.from(shotBytes)), "the image store holds it by name");
|
||||
}
|
||||
});
|
||||
|
||||
test("the fx harness tells a refused turn from one whose core exited under it", async () => {
|
||||
const { fxHarness } = await import("../fx-harness.js");
|
||||
const { coreAnswered, engineInternals } = await import("../fx-sdk.js");
|
||||
@@ -1840,6 +2159,24 @@ test("a checkpoint restores the conversation in a new agent", async () => {
|
||||
await restored.close();
|
||||
});
|
||||
|
||||
test("a checkpoint after an image prompt holds the image, and a new agent sends it", async () => {
|
||||
const png = pngBase64;
|
||||
const agent = createFxAgent(agentOptions(await durabilityFor(), { model: "durable/vision-model" }));
|
||||
const look = [{ type: "text", text: "look" }, { type: "image", data: png, mimeType: "image/png" }];
|
||||
assert.equal((await agent.prompt(look).result).stopReason, "end_turn");
|
||||
// The session's records name the image; the checkpoint carries its bytes.
|
||||
const checkpoint = await agent.checkpoint();
|
||||
await agent.close();
|
||||
const restored = createFxAgent(agentOptions(await durabilityFor(), { checkpoint, model: "durable/vision-model" }));
|
||||
assert.equal((await restored.prompt("what was in it").result).stopReason, "end_turn");
|
||||
const files = chats.at(-1).prompt
|
||||
.filter((message) => message.role === "user" && Array.isArray(message.content))
|
||||
.flatMap((message) => message.content)
|
||||
.filter((part) => part.type === "file");
|
||||
assert.deepEqual(files.map((part) => part.data?.data), [png]);
|
||||
await restored.close();
|
||||
});
|
||||
|
||||
// LIBFX_TEST_ONLY runs only the tests whose names contain it.
|
||||
const only = process.env.LIBFX_TEST_ONLY;
|
||||
const selected = only ? tests.filter(([name]) => name.includes(only)) : tests;
|
||||
|
||||
+590
-5
@@ -19,7 +19,10 @@ const usage = { inputTokens: { total: 1 }, outputTokens: { total: 1 } };
|
||||
const textOf = (message) => typeof message.content === "string"
|
||||
? message.content
|
||||
: message.content.filter((part) => part.type === "text").map((part) => part.text).join("");
|
||||
const lastUserText = (prompt) => textOf(prompt.filter((message) => message.role === "user").at(-1));
|
||||
// The user message the gateway adds after a tool result's images.
|
||||
const toolImageFollowUp = "Attached image(s) from the tool result.";
|
||||
const isPrompt = (message) => message.role === "user" && textOf(message) !== toolImageFollowUp;
|
||||
const lastUserText = (prompt) => textOf(prompt.filter(isPrompt).at(-1));
|
||||
const toolResults = (prompt) => prompt
|
||||
.flatMap((message) => Array.isArray(message.content) ? message.content : [])
|
||||
.filter((part) => part.type === "tool-result");
|
||||
@@ -70,8 +73,13 @@ function persisted(journal) {
|
||||
// "gather everything" calls bulky bulkySteps times, one call per response.
|
||||
// Any other prompt is answered with its own text.
|
||||
const bulkySteps = 24;
|
||||
// An image prompt's text ends with a marker per image, which the scripts
|
||||
// ignore.
|
||||
function scriptText(prompt) {
|
||||
return lastUserText(prompt).replace(/(\n\[Image #\d+\])+$/u, "");
|
||||
}
|
||||
function framesFor(prompt) {
|
||||
const text = lastUserText(prompt);
|
||||
const text = scriptText(prompt);
|
||||
// "keep using the tool" asks for a lookup until three results are in, so an
|
||||
// agent that ignored a fence would keep calling it.
|
||||
if (text === "keep using the tool") {
|
||||
@@ -113,6 +121,35 @@ function framesFor(prompt) {
|
||||
{ type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage },
|
||||
];
|
||||
}
|
||||
// "album N" shows N album pages at once in one response, then says it saw
|
||||
// them.
|
||||
const pages = /^album (\d+)$/u.exec(text);
|
||||
if (pages) {
|
||||
if (toolResults(prompt.slice(prompt.findLastIndex(isPrompt) + 1)).length === 0) {
|
||||
return [
|
||||
...Array.from({ length: Number(pages[1]) }, (_, page) => ({ type: "tool-call", toolCallId: `call-page-${page}`, toolName: "album", input: { page } })),
|
||||
{ type: "finish", finishReason: { unified: "tool-calls", raw: "tool-calls" }, usage },
|
||||
];
|
||||
}
|
||||
return [
|
||||
{ type: "text-delta", id: "answer", delta: "saw the album" },
|
||||
{ type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage },
|
||||
];
|
||||
}
|
||||
// "screenshot N" takes screenshot N once in its turn, then says it saw it.
|
||||
const shot = /^screenshot (\d+)$/u.exec(text);
|
||||
if (shot) {
|
||||
if (toolResults(prompt.slice(prompt.findLastIndex(isPrompt) + 1)).length === 0) {
|
||||
return [
|
||||
{ type: "tool-call", toolCallId: `call-shot-${shot[1]}`, toolName: "screenshot", input: { shot: Number(shot[1]) } },
|
||||
{ type: "finish", finishReason: { unified: "tool-calls", raw: "tool-calls" }, usage },
|
||||
];
|
||||
}
|
||||
return [
|
||||
{ type: "text-delta", id: "answer", delta: `saw ${shot[1]}` },
|
||||
{ type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage },
|
||||
];
|
||||
}
|
||||
if (text === "use the tool" && toolResults(prompt).length === 0) {
|
||||
return [
|
||||
{ type: "tool-call", toolCallId: "call-1", toolName: "lookup", input: { key: "alpha" } },
|
||||
@@ -145,7 +182,7 @@ const server = createServer((request, response) => {
|
||||
Object.defineProperty(prompt, "toolChoice", { value: toolChoice?.type ?? null });
|
||||
requests.push(prompt);
|
||||
sessionHeaders.push([request.headers["x-session-id"], request.headers["x-session-affinity"]]);
|
||||
if (lastUserText(prompt) === "fail please") {
|
||||
if (scriptText(prompt) === "fail please") {
|
||||
response.writeHead(400, { "content-type": "application/json" });
|
||||
response.end(JSON.stringify({ error: { message: "refused for the test", type: "invalid_request_error" } }));
|
||||
return;
|
||||
@@ -178,6 +215,35 @@ const bulky = {
|
||||
inputSchema: { type: "object", properties: { part: { type: "number" } } },
|
||||
execute: async ({ part }) => `${part}:${"x".repeat(60_000)}`,
|
||||
};
|
||||
// Returns screenshot `shot` as a typed result: a PNG distinct per shot.
|
||||
const screenshot = {
|
||||
name: "screenshot",
|
||||
description: "Takes a screenshot",
|
||||
inputSchema: { type: "object", properties: { shot: { type: "number" } }, required: ["shot"] },
|
||||
execute: async ({ shot }) => ({
|
||||
type: "libfx.tool-result",
|
||||
text: `shot ${shot}`,
|
||||
images: [{ type: "image", data: base64Of(screenshotOf(shot)), mimeType: "image/png" }],
|
||||
}),
|
||||
};
|
||||
// Returns album page `page`: a result's 8 images, small and distinct from
|
||||
// every other page's.
|
||||
const albumImage = (index) => {
|
||||
const bytes = new Uint8Array(71);
|
||||
bytes.set(Buffer.from(pngData, "base64"));
|
||||
new DataView(bytes.buffer).setUint32(67, index);
|
||||
return bytes;
|
||||
};
|
||||
const album = {
|
||||
name: "album",
|
||||
description: "Shows an album page",
|
||||
inputSchema: { type: "object", properties: { page: { type: "number" } }, required: ["page"] },
|
||||
execute: async ({ page }) => ({
|
||||
type: "libfx.tool-result",
|
||||
text: `page ${page}`,
|
||||
images: Array.from({ length: 8 }, (_, index) => ({ type: "image", data: base64Of(albumImage(page * 8 + index)), mimeType: "image/png" })),
|
||||
}),
|
||||
};
|
||||
const options = (backend, journal, extra = {}) => ({
|
||||
backend,
|
||||
nativeAddon: addon,
|
||||
@@ -805,7 +871,7 @@ test("a journal that does not fold is refused", async () => {
|
||||
const invalid = (pattern) => (error) => pattern.test(error.message) && error.code === "FX_JOURNAL_INVALID";
|
||||
const duplicated = eventJournal([...journal.events, journal.events.at(-1)]);
|
||||
await assert.rejects(createFxEngine(options(targetBackend, duplicated)), invalid(/libfx journal events are out of order/));
|
||||
const newer = eventJournal([{ ...journal.events[0], v: 3 }]);
|
||||
const newer = eventJournal([{ ...journal.events[0], v: 4 }]);
|
||||
await assert.rejects(createFxEngine(options(targetBackend, newer)), /libfx journal was written by a newer fx/);
|
||||
const garbled = eventJournal([{ v: 1, seq: 1, turn: 1, type: "turn_committed", data: { kind: "nope" } }]);
|
||||
await assert.rejects(createFxEngine(options(targetBackend, garbled)), invalid(/Invalid libfx journal/));
|
||||
@@ -1181,7 +1247,7 @@ test("a turn left open resumes under the tools the new agent has", async () => {
|
||||
});
|
||||
|
||||
test("a journal from a newer libfx is refused", async () => {
|
||||
const future = eventJournal([{ v: 3, seq: 1, turn: 1, type: "turn_progress", data: {} }]);
|
||||
const future = eventJournal([{ v: 4, seq: 1, turn: 1, type: "turn_progress", data: {} }]);
|
||||
const newer = (error) => error instanceof FxJournalVersionError && error.code === "FX_JOURNAL_VERSION";
|
||||
await assert.rejects(createFxEngine(options(sourceBackend, future)), newer);
|
||||
const record = (format) => ({
|
||||
@@ -1454,6 +1520,487 @@ test("an image prompt after a crashed image turn gets the next image id", async
|
||||
assert.match(lastUserText(requests.at(-1)), /\[Image #2\]$/);
|
||||
});
|
||||
|
||||
// Images for the image store's tests: a PNG header, then filler that differs
|
||||
// per salt, so each image has its own name.
|
||||
const visionCatalog = { modelCatalog: [{ id: "journal/model", type: "language", tags: ["tool-use", "vision", "file-input"] }] };
|
||||
const pngOf = (size, salt) => {
|
||||
const bytes = new Uint8Array(size);
|
||||
bytes.set(Buffer.from(pngData, "base64"));
|
||||
for (let index = 67; index < size; index += 1) bytes[index] = (index * 31 + salt) & 255;
|
||||
return bytes;
|
||||
};
|
||||
const base64Of = (bytes) => Buffer.from(bytes).toString("base64");
|
||||
const imageBlock = (bytes) => ({ type: "image", data: bytes, mimeType: "image/png" });
|
||||
const nameOf = async (bytes) => Buffer.from(await crypto.subtle.digest("SHA-256", bytes)).toString("hex");
|
||||
// Whether saved bytes hold any of an image, raw or as base64.
|
||||
const holdsImage = (data, bytes) => Buffer.from(data).includes(Buffer.from(bytes.subarray(200, 264))) ||
|
||||
Buffer.from(data).includes(Buffer.from(base64Of(bytes).slice(400, 480)));
|
||||
const savedData = (store) => [...store.records.map((record) => record.data), ...(store.checkpoint ? [store.checkpoint.data] : [])];
|
||||
const requestImages = (prompt) => prompt
|
||||
.flatMap((message) => Array.isArray(message.content) ? message.content : [])
|
||||
.filter((part) => part.type === "file")
|
||||
.map((part) => part.data?.data ?? part.data);
|
||||
// libfx's own key for a persistence's image store, which durable sessions
|
||||
// give the engines they run.
|
||||
const persistenceImages = Symbol.for("libfx.persistenceImages");
|
||||
// `persistence` with an image store that keeps each image once, by name, in
|
||||
// `images`, as a durable session gives its engines.
|
||||
const withImageStore = (persistence, images = new Map()) => Object.assign(persistence, {
|
||||
images,
|
||||
[persistenceImages]: {
|
||||
put: async (name, bytes) => { if (!images.has(name)) images.set(name, bytes.slice()); },
|
||||
get: async (name) => images.get(name)?.slice(),
|
||||
},
|
||||
});
|
||||
const createImagePersistence = () => withImageStore(createMemoryPersistence());
|
||||
|
||||
test("a store keeps each prompt image once, and records and checkpoints only name it", async () => {
|
||||
const store = createImagePersistence();
|
||||
const first = pngOf(300_000, 1);
|
||||
const second = pngOf(200_000, 2);
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: store, checkpointAfterBytes: 0 }));
|
||||
await run(source, [{ type: "text", text: "one" }, imageBlock(first)]);
|
||||
await run(source, [{ type: "text", text: "two" }, imageBlock(second), imageBlock(first)]);
|
||||
// A turn without images: its events name none.
|
||||
await run(source, "no image here");
|
||||
await source.close();
|
||||
assert.equal(store.images.size, 2);
|
||||
assert.deepEqual([...store.images.keys()].sort(), [await nameOf(first), await nameOf(second)].sort());
|
||||
assert.ok(store.checkpoint, "a checkpoint was saved");
|
||||
for (const data of savedData(store)) {
|
||||
assert.ok(!holdsImage(data, first) && !holdsImage(data, second), "saved data holds no image bytes");
|
||||
}
|
||||
// Events that name an image are version 3. A progress that extends another
|
||||
// (`base`) is version 2 and leaves out the user's turn, so it names none;
|
||||
// every other event stays version 1.
|
||||
const versions = storedEvents(store).map((event) => [JSON.stringify(event).includes("inline_ref"), event.v, event.base !== undefined || event.type === "tool_result"]);
|
||||
assert.ok(versions.some(([named]) => named), "some events name images");
|
||||
for (const [named, version, delta] of versions) {
|
||||
assert.ok(!(named && delta), "a delta names no image");
|
||||
assert.equal(version, named ? 3 : delta ? 2 : 1);
|
||||
}
|
||||
assert.ok(versions.some(([named]) => !named), "the text-only turn's events name none");
|
||||
|
||||
const target = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: store }));
|
||||
requests.length = 0;
|
||||
const { result } = await run(target, "four");
|
||||
await target.close();
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(first), base64Of(second), base64Of(first)]);
|
||||
});
|
||||
|
||||
test("a session whose store lost an image is refused, naming it", async () => {
|
||||
const store = createImagePersistence();
|
||||
const kept = pngOf(50_000, 4);
|
||||
const lost = pngOf(60_000, 5);
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: store, checkpointAfterBytes: 0 }));
|
||||
await run(source, [{ type: "text", text: "one" }, imageBlock(kept), imageBlock(lost)]);
|
||||
await source.close();
|
||||
const name = await nameOf(lost);
|
||||
store.images.delete(name);
|
||||
await assert.rejects(
|
||||
createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: store })),
|
||||
(error) => error.code === "FX_JOURNAL_IMAGE_MISSING" && error.image === name && error.message.includes(name),
|
||||
);
|
||||
});
|
||||
|
||||
test("a persistence without an image store keeps writing images inline", async () => {
|
||||
const image = pngOf(20_000, 6);
|
||||
const journal = eventJournal();
|
||||
const agent = await createFxEngine(options(sourceBackend, journal, visionCatalog));
|
||||
await run(agent, [{ type: "text", text: "inline" }, imageBlock(image)]);
|
||||
await agent.close();
|
||||
// No event is the version that names stored images; a progress that
|
||||
// extends another is version 2, every other event version 1.
|
||||
for (const event of journal.events) assert.equal(event.v, event.base !== undefined || event.type === "tool_result" ? 2 : 1);
|
||||
assert.ok(JSON.stringify(journal.events).includes(base64Of(image)));
|
||||
assert.ok(!JSON.stringify(journal.events).includes("inline_ref"));
|
||||
});
|
||||
|
||||
test("an image turn that crashed after a delta reopens with its image", async () => {
|
||||
// The turn's first progress names the image. The one after the tool step
|
||||
// extends it and leaves out the user's turn, so a crash right after that
|
||||
// one brings the image back from the first.
|
||||
const image = pngOf(50_000, 8);
|
||||
const images = new Map();
|
||||
const journal = eventJournal();
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: withImageStore(persisted(journal), images) }));
|
||||
assert.equal((await run(source, [{ type: "text", text: "use the tool" }, imageBlock(image)])).text, "tool said beta");
|
||||
await source.close();
|
||||
const progress = journal.events.filter((event) => event.type === "turn_progress");
|
||||
assert.equal(progress[0].v, 3, "the first progress names the image");
|
||||
const stepped = progress.find((event) => event.steps === 1);
|
||||
assert.ok(stepped?.base !== undefined && stepped.v === 2, "the progress after the tool extends it");
|
||||
assert.ok(!JSON.stringify(stepped).includes("inline_ref"), "and names no image");
|
||||
const survived = journal.events.slice(0, journal.events.indexOf(stepped) + 1);
|
||||
const resumed = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: withImageStore(persisted(eventJournal(survived)), images) }));
|
||||
requests.length = 0;
|
||||
const turn = resumed.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await resumed.close();
|
||||
assert.deepEqual(requestImages(requests[0]), [base64Of(image)], "the image came back from the first progress");
|
||||
assert.equal(toolResults(requests[0]).length, 1, "the step came back from the delta");
|
||||
});
|
||||
|
||||
// A smaller image window than the core's, so a test reaches its edge.
|
||||
const imageWindow = Symbol.for("libfx.imageWindow");
|
||||
const droppedNote = /\[Image #1 is no longer in this session, which keeps only its newest images\.\]/;
|
||||
|
||||
for (const [label, limits] of [["images", { images: 2 }], ["bytes", { bytes: 2 * 40_000 }]]) {
|
||||
test(`a session keeps its newest images within its window of ${label}, and an older one leaves as a note`, async () => {
|
||||
const store = createImagePersistence();
|
||||
const pictures = [pngOf(40_000, 11), pngOf(40_000, 12), pngOf(40_000, 13)];
|
||||
const window = { [imageWindow]: limits };
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: store, checkpointAfterBytes: 0, ...window }));
|
||||
for (const [index, picture] of pictures.entries()) {
|
||||
await run(source, [{ type: "text", text: `look ${index}` }, imageBlock(picture)]);
|
||||
}
|
||||
// The third turn's request still carries every image; once it entered the
|
||||
// history, its image pushed the first out, and the next request carries
|
||||
// the two newest and a note where the first was.
|
||||
assert.deepEqual(requestImages(requests.at(-1)), pictures.map(base64Of));
|
||||
await run(source, "and then?");
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(pictures[1]), base64Of(pictures[2])]);
|
||||
assert.match(JSON.stringify(requests.at(-1)), droppedNote);
|
||||
await source.close();
|
||||
assert.ok(savedData(store).some((data) => Buffer.from(data).includes("\"dropped\":true")), "the checkpoint keeps the first image's name alone");
|
||||
|
||||
// A store that lost the first image still opens the session, which reads
|
||||
// only the images its window keeps.
|
||||
store.images.delete(await nameOf(pictures[0]));
|
||||
const read = [];
|
||||
const { get } = store[persistenceImages];
|
||||
store[persistenceImages].get = async (name) => {
|
||||
read.push(name);
|
||||
return get(name);
|
||||
};
|
||||
const target = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: store, ...window }));
|
||||
await run(target, "and now?");
|
||||
await target.close();
|
||||
assert.deepEqual(read.sort(), [await nameOf(pictures[1]), await nameOf(pictures[2])].sort());
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(pictures[1]), base64Of(pictures[2])]);
|
||||
assert.match(JSON.stringify(requests.at(-1)), droppedNote);
|
||||
});
|
||||
}
|
||||
|
||||
test("a turn that fails before it enters the history drops no image", async () => {
|
||||
const store = createImagePersistence();
|
||||
const pictures = [pngOf(40_000, 21), pngOf(40_000, 22), pngOf(40_000, 23)];
|
||||
const agent = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: store, [imageWindow]: { images: 2 } }));
|
||||
await run(agent, [{ type: "text", text: "look 0" }, imageBlock(pictures[0])]);
|
||||
await run(agent, [{ type: "text", text: "look 1" }, imageBlock(pictures[1])]);
|
||||
const failed = await run(agent, [{ type: "text", text: "fail please" }, imageBlock(pictures[2])]);
|
||||
assert.notEqual(failed.result.stopReason, "end_turn");
|
||||
// The failed turn is not in the history, so the window still holds both.
|
||||
await run(agent, "and then?");
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(pictures[0]), base64Of(pictures[1])]);
|
||||
assert.doesNotMatch(JSON.stringify(requests.at(-1)), /no longer in this session/);
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a turn that crashed with a full image window reopens with all of its images", async () => {
|
||||
// An open stages the whole window and the open turn's images, which the
|
||||
// window leaves out: one prompt's, up to 8 of them.
|
||||
const tiny = (n) => {
|
||||
const bytes = pngOf(80, 0);
|
||||
new DataView(bytes.buffer).setUint32(70, n);
|
||||
return bytes;
|
||||
};
|
||||
let made = 0;
|
||||
const prompt = (text) => [{ type: "text", text }, ...Array.from({ length: 8 }, () => imageBlock(tiny(made++)))];
|
||||
const images = new Map();
|
||||
const journal = eventJournal();
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: withImageStore(persisted(journal), images) }));
|
||||
for (let turn = 0; turn < 4096 / 8; turn += 1) await run(source, prompt(`look ${turn}`));
|
||||
const open = prompt("use the tool");
|
||||
assert.equal((await run(source, open)).text, "tool said beta");
|
||||
await source.close();
|
||||
assert.equal(images.size, 4096 + 8);
|
||||
// A crash after the last turn's tool step leaves it open, with the window
|
||||
// full of the turns before it.
|
||||
const stepped = journal.events.findLast((event) => event.type === "turn_progress" && event.steps === 1);
|
||||
const survived = journal.events.slice(0, journal.events.indexOf(stepped) + 1);
|
||||
const read = new Set();
|
||||
const reading = withImageStore(persisted(eventJournal(survived)), images);
|
||||
const { get } = reading[persistenceImages];
|
||||
reading[persistenceImages].get = async (name) => { read.add(name); return get(name); };
|
||||
const resumed = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: reading }));
|
||||
requests.length = 0;
|
||||
const turn = resumed.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await resumed.close();
|
||||
assert.equal(read.size, 4096 + 8, "the open read the window and the open turn's images");
|
||||
const sent = requestImages(requests[0]);
|
||||
for (const block of open.slice(1)) assert.ok(sent.includes(base64Of(block.data)), "the open turn kept its images");
|
||||
});
|
||||
|
||||
test("a failed image write fails the prompt before it is sent", async () => {
|
||||
const store = createImagePersistence();
|
||||
store[persistenceImages].put = async () => { throw new Error("disk full"); };
|
||||
const agent = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: store }));
|
||||
requests.length = 0;
|
||||
const turn = agent.prompt([{ type: "text", text: "look" }, imageBlock(pngOf(1_000, 7))]);
|
||||
// The store's own error.
|
||||
await assert.rejects(turn.result, (error) => error.message === "disk full");
|
||||
assert.equal(requests.length, 0, "the prompt never reached the model");
|
||||
assert.equal(storedEvents(store).length, 0, "nothing was recorded");
|
||||
const { result } = await run(agent, "text only");
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
await agent.close();
|
||||
});
|
||||
|
||||
test("a session saved inline opens behind an image store, and only new images are stored", async () => {
|
||||
const old = pngOf(40_000, 8);
|
||||
const fresh = pngOf(30_000, 9);
|
||||
const journal = eventJournal();
|
||||
const before = await createFxEngine(options(sourceBackend, journal, visionCatalog));
|
||||
await run(before, [{ type: "text", text: "old" }, imageBlock(old)]);
|
||||
await before.close();
|
||||
|
||||
const images = new Map();
|
||||
const upgraded = withImageStore(persisted(journal), images);
|
||||
const after = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: upgraded }));
|
||||
requests.length = 0;
|
||||
await run(after, [{ type: "text", text: "new" }, imageBlock(fresh)]);
|
||||
await after.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(old), base64Of(fresh)]);
|
||||
assert.deepEqual([...images.keys()], [await nameOf(fresh)]);
|
||||
|
||||
const again = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: upgraded }));
|
||||
requests.length = 0;
|
||||
await run(again, "again");
|
||||
await again.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(old), base64Of(fresh)]);
|
||||
});
|
||||
|
||||
test("images too large for a checkpoint are named instead", async () => {
|
||||
// Two 2.1 MiB images: more than one checkpoint holds raw.
|
||||
const images = [pngOf(2_200_000, 10), pngOf(2_200_000, 11)];
|
||||
const prompt = [{ type: "text", text: "large" }, ...images.map(imageBlock)];
|
||||
|
||||
// Without an image store, the core refuses the checkpoint.
|
||||
const inline = createMemoryPersistence();
|
||||
const without = await createFxEngine(options(sourceBackend, undefined, { ...visionCatalog, persistence: inline, checkpointAfterBytes: 0 }));
|
||||
await run(without, prompt);
|
||||
await without.close();
|
||||
assert.equal(inline.checkpoint, null);
|
||||
|
||||
const store = createImagePersistence();
|
||||
const saved = [];
|
||||
const named = await createFxEngine(options(sourceBackend, undefined, {
|
||||
...visionCatalog, persistence: store, checkpointAfterBytes: 0,
|
||||
onEvent: (event) => { if (event.type === "checkpoint.save") saved.push(event.type); },
|
||||
}));
|
||||
await run(named, prompt);
|
||||
await named.close();
|
||||
assert.deepEqual(saved, ["checkpoint.save"]);
|
||||
assert.ok(store.checkpoint.data.byteLength < 64 * 1024, `checkpoint is ${store.checkpoint.data.byteLength} bytes`);
|
||||
|
||||
const reopened = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: store }));
|
||||
requests.length = 0;
|
||||
const { result } = await run(reopened, "and now");
|
||||
await reopened.close();
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.deepEqual(requestImages(requests.at(-1)), images.map(base64Of));
|
||||
});
|
||||
|
||||
test("a session holds more images than a checkpoint could, turn after turn", async () => {
|
||||
const store = createImagePersistence();
|
||||
const images = Array.from({ length: 5 }, (_, index) => pngOf(1024 * 1024, 20 + index));
|
||||
let saves = 0;
|
||||
const countSaves = { onEvent: (event) => { if (event.type === "checkpoint.save") saves += 1; } };
|
||||
for (const [index, image] of images.entries()) {
|
||||
// Each turn reopens the session, as a new process would.
|
||||
const agent = await createFxEngine(options(index % 2 ? targetBackend : sourceBackend, undefined, { ...visionCatalog, ...countSaves, persistence: store, checkpointAfterBytes: 0 }));
|
||||
const { result } = await run(agent, [{ type: "text", text: `turn ${index}` }, imageBlock(image)]);
|
||||
await agent.close();
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
}
|
||||
assert.equal(saves, images.length, "every turn saved a checkpoint");
|
||||
const last = await createFxEngine(options(targetBackend, undefined, { ...visionCatalog, persistence: store }));
|
||||
requests.length = 0;
|
||||
await run(last, "all of them");
|
||||
await last.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), images.map(base64Of));
|
||||
});
|
||||
|
||||
const screenshotOf = (shot) => pngOf(300_000, 100 + shot);
|
||||
const withScreenshots = { ...visionCatalog, tools: [lookup, screenshot] };
|
||||
const toolDroppedNote = /\[1 image from this result is no longer in this session, which keeps only its newest images\.\]/;
|
||||
|
||||
test("a store keeps each tool image once, and records and checkpoints only name it", async () => {
|
||||
const store = createImagePersistence();
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withScreenshots, persistence: store, checkpointAfterBytes: 0 }));
|
||||
for (const shot of [1, 2, 3]) assert.equal((await run(source, `screenshot ${shot}`)).text, `saw ${shot}`);
|
||||
await source.close();
|
||||
const shots = [1, 2, 3].map(screenshotOf);
|
||||
assert.deepEqual([...store.images.keys()].sort(), (await Promise.all(shots.map(nameOf))).sort());
|
||||
assert.ok(store.checkpoint, "a checkpoint was saved");
|
||||
for (const data of savedData(store)) assert.ok(shots.every((bytes) => !holdsImage(data, bytes)), "saved data holds no tool image bytes");
|
||||
// An event that names an image is version 3, a progress that extends
|
||||
// another and names a new tool image too.
|
||||
const events = storedEvents(store);
|
||||
assert.ok(events.some((event) => event.base !== undefined && JSON.stringify(event).includes("inline_ref")), "a delta names its tool image");
|
||||
for (const event of events) {
|
||||
const named = JSON.stringify(event).includes("inline_ref");
|
||||
assert.equal(event.v, named ? 3 : event.base !== undefined || event.type === "tool_result" ? 2 : 1);
|
||||
}
|
||||
|
||||
const target = await createFxEngine(options(targetBackend, undefined, { ...withScreenshots, persistence: store }));
|
||||
requests.length = 0;
|
||||
const { result } = await run(target, "after the shots");
|
||||
await target.close();
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.deepEqual(requestImages(requests.at(-1)), shots.map(base64Of));
|
||||
});
|
||||
|
||||
test("a tool image leaves the window oldest first, as a note, and stays out after a reopen", async () => {
|
||||
const store = createImagePersistence();
|
||||
const window = { [imageWindow]: { images: 2 } };
|
||||
const shots = [1, 2, 3].map(screenshotOf);
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withScreenshots, ...window, persistence: store, checkpointAfterBytes: 0 }));
|
||||
for (const shot of [1, 2, 3]) await run(source, `screenshot ${shot}`);
|
||||
requests.length = 0;
|
||||
await run(source, "what is left");
|
||||
await source.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [shots[1], shots[2]].map(base64Of));
|
||||
assert.match(JSON.stringify(requests.at(-1)), toolDroppedNote);
|
||||
assert.ok(savedData(store).some((data) => Buffer.from(data).includes("\"dropped\":true")), "the first shot keeps its name alone");
|
||||
|
||||
const target = await createFxEngine(options(targetBackend, undefined, { ...withScreenshots, ...window, persistence: store }));
|
||||
requests.length = 0;
|
||||
await run(target, "and after reopening");
|
||||
await target.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [shots[1], shots[2]].map(base64Of));
|
||||
assert.match(JSON.stringify(requests.at(-1)), toolDroppedNote);
|
||||
});
|
||||
|
||||
test("a tool image the store refuses stays in the session inline", async () => {
|
||||
const store = createImagePersistence();
|
||||
store[persistenceImages].put = async () => { throw new Error("store is full"); };
|
||||
const agent = await createFxEngine(options(sourceBackend, undefined, { ...withScreenshots, persistence: store }));
|
||||
requests.length = 0;
|
||||
const { text, result } = await run(agent, "screenshot 4");
|
||||
await agent.close();
|
||||
assert.equal(result.stopReason, "end_turn");
|
||||
assert.equal(text, "saw 4");
|
||||
// The tool ran, so its image is not lost: the model saw it, and the
|
||||
// records hold it inline.
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(screenshotOf(4))]);
|
||||
assert.ok(savedData(store).some((data) => holdsImage(data, screenshotOf(4))), "the records hold the image inline");
|
||||
assert.equal(store.images.size, 0);
|
||||
|
||||
const reopened = await createFxEngine(options(targetBackend, undefined, { ...withScreenshots, persistence: store }));
|
||||
requests.length = 0;
|
||||
await run(reopened, "after the refusal");
|
||||
await reopened.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [base64Of(screenshotOf(4))]);
|
||||
});
|
||||
|
||||
test("a turn that crashed after its screenshot's progress reopens with the screenshot", async () => {
|
||||
const journal = eventJournal();
|
||||
const images = new Map();
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withScreenshots, persistence: withImageStore(persisted(journal), images) }));
|
||||
assert.equal((await run(source, "screenshot 5")).text, "saw 5");
|
||||
await source.close();
|
||||
// The progress after the tool step extends the first, and names the shot.
|
||||
const stepped = journal.events.find((event) => event.type === "turn_progress" && event.steps === 1);
|
||||
assert.ok(stepped?.base !== undefined && stepped.v === 3 && JSON.stringify(stepped).includes("inline_ref"), "the delta names the screenshot");
|
||||
assert.ok(!JSON.stringify(stepped).includes(base64Of(screenshotOf(5)).slice(0, 400)), "and does not hold it");
|
||||
const survived = journal.events.slice(0, journal.events.indexOf(stepped) + 1);
|
||||
const resumed = await createFxEngine(options(targetBackend, undefined, { ...withScreenshots, persistence: withImageStore(persisted(eventJournal(survived)), images) }));
|
||||
requests.length = 0;
|
||||
const turn = resumed.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await resumed.close();
|
||||
assert.deepEqual(requestImages(requests[0]), [base64Of(screenshotOf(5))], "the screenshot came back from the store");
|
||||
});
|
||||
|
||||
test("a turn that crashed right after its screenshot's result reopens with the screenshot, without running the tool again", async () => {
|
||||
const journal = eventJournal();
|
||||
const images = new Map();
|
||||
let shots = 0;
|
||||
const counting = { ...screenshot, execute: async (input) => { shots += 1; return screenshot.execute(input); } };
|
||||
const withCounting = { ...visionCatalog, tools: [lookup, counting] };
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withCounting, persistence: withImageStore(persisted(journal), images) }));
|
||||
assert.equal((await run(source, "screenshot 9")).text, "saw 9");
|
||||
await source.close();
|
||||
// The result names the screenshot as the call finished, before any
|
||||
// progress holds the step.
|
||||
const result = journal.events.find((event) => event.type === "tool_result");
|
||||
assert.ok(result?.v === 3 && result.data.images?.[0]?.inline_ref === await nameOf(screenshotOf(9)), "the result names the screenshot");
|
||||
assert.ok(!JSON.stringify(result).includes(base64Of(screenshotOf(9)).slice(0, 400)), "and does not hold it");
|
||||
const survived = journal.events.slice(0, journal.events.indexOf(result) + 1);
|
||||
assert.ok(!survived.some((event) => event.type === "turn_progress" && event.steps === 1), "no progress holds the step yet");
|
||||
const resumed = await createFxEngine(options(targetBackend, undefined, { ...withCounting, persistence: withImageStore(persisted(eventJournal(survived)), images) }));
|
||||
requests.length = 0;
|
||||
const turn = resumed.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await resumed.close();
|
||||
assert.equal(shots, 1, "the tool ran once");
|
||||
assert.deepEqual(requestImages(requests[0]), [base64Of(screenshotOf(9))], "the screenshot came back from the store");
|
||||
});
|
||||
|
||||
test("a tool image the store refused leaves the window like any other, as a note", async () => {
|
||||
const store = createImagePersistence();
|
||||
const refusedName = await nameOf(screenshotOf(6));
|
||||
const { put } = store[persistenceImages];
|
||||
store[persistenceImages].put = async (name, bytes) => {
|
||||
if (name === refusedName) throw new Error("store is full");
|
||||
return put(name, bytes);
|
||||
};
|
||||
const window = { [imageWindow]: { images: 2 } };
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withScreenshots, ...window, persistence: store, checkpointAfterBytes: 0 }));
|
||||
for (const shot of [6, 7, 8]) assert.equal((await run(source, `screenshot ${shot}`)).text, `saw ${shot}`);
|
||||
requests.length = 0;
|
||||
await run(source, "what is left");
|
||||
await source.close();
|
||||
// The refused shot, held inline, is the oldest image: it leaves first.
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [7, 8].map((shot) => base64Of(screenshotOf(shot))));
|
||||
assert.match(JSON.stringify(requests.at(-1)), toolDroppedNote);
|
||||
assert.ok(!holdsImage(store.checkpoint.data, screenshotOf(6)), "the checkpoint no longer holds it");
|
||||
|
||||
const target = await createFxEngine(options(targetBackend, undefined, { ...withScreenshots, ...window, persistence: store }));
|
||||
requests.length = 0;
|
||||
await run(target, "and after reopening");
|
||||
await target.close();
|
||||
assert.deepEqual(requestImages(requests.at(-1)), [7, 8].map((shot) => base64Of(screenshotOf(shot))));
|
||||
assert.match(JSON.stringify(requests.at(-1)), toolDroppedNote);
|
||||
});
|
||||
|
||||
test("a turn whose tools returned more images than a window and a prompt hold reopens with all of them", async () => {
|
||||
// 514 pages of 8: 4,112 images, more than the 4,096 a session keeps and
|
||||
// the 8 of one prompt, all an open stages unless it reports more missing.
|
||||
const pages = 514;
|
||||
const journal = eventJournal();
|
||||
const images = new Map();
|
||||
let staged = 0;
|
||||
const withImages = (persistence) => {
|
||||
const counted = withImageStore(persistence, images);
|
||||
const { get } = counted[persistenceImages];
|
||||
counted[persistenceImages].get = async (name) => { staged += 1; return get(name); };
|
||||
return counted;
|
||||
};
|
||||
const withAlbum = { ...visionCatalog, tools: [album] };
|
||||
const source = await createFxEngine(options(sourceBackend, undefined, { ...withAlbum, persistence: withImages(persisted(journal)) }));
|
||||
assert.equal((await run(source, `album ${pages}`)).text, "saw the album");
|
||||
await source.close();
|
||||
assert.equal(images.size, pages * 8);
|
||||
const stepped = journal.events.find((event) => event.type === "turn_progress" && event.steps === 1);
|
||||
assert.ok(stepped, "a progress holds the album's step");
|
||||
const survived = journal.events.slice(0, journal.events.indexOf(stepped) + 1);
|
||||
const resumed = await createFxEngine(options(targetBackend, undefined, { ...withAlbum, persistence: withImages(persisted(eventJournal(survived))) }));
|
||||
assert.equal(staged, pages * 8, "the open staged every image of the open turn");
|
||||
const turn = resumed.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await resumed.close();
|
||||
});
|
||||
|
||||
test("a journal past the turn limit is refused by name", async () => {
|
||||
const journal = eventJournal();
|
||||
const agent = await createFxEngine(options(sourceBackend, journal));
|
||||
@@ -1619,6 +2166,44 @@ test("a tool receives the model's call id as its execution id, and its session's
|
||||
await agent.close();
|
||||
assert.deepEqual(seen, [{ keys: ["executionId", "sessionId", "signal"], executionId: "call-1", sessionId, signal: true }]);
|
||||
});
|
||||
|
||||
test("a yielded turn resumed in place after its checkpoint reopens from any cut", async () => {
|
||||
// The store keeps a copy of itself after each append, as a crash there
|
||||
// would leave it.
|
||||
const store = createMemoryPersistence();
|
||||
const states = [];
|
||||
const append = store.append.bind(store);
|
||||
store.append = async (input) => {
|
||||
const out = await append(input);
|
||||
states.push({
|
||||
records: store.records.map((record) => ({ ...record, data: record.data.slice() })),
|
||||
checkpoint: store.checkpoint && { through: store.checkpoint.through, data: store.checkpoint.data.slice() },
|
||||
});
|
||||
return out;
|
||||
};
|
||||
const agent = await createFxEngine(options(sourceBackend, undefined, { persistence: store, checkpointAfterBytes: 0 }));
|
||||
const first = agent.prompt("use the tool", { yieldAt: Date.now() });
|
||||
for await (const _ of first) {}
|
||||
assert.equal((await first.result).stopReason, "yielded");
|
||||
for (let wait = 0; wait < 100 && !store.checkpoint; wait += 1) await new Promise((resolve) => setTimeout(resolve, 20));
|
||||
assert.ok(store.checkpoint, "a checkpoint holding the open turn was saved at the yield");
|
||||
const before = states.length;
|
||||
const turn = agent.resume();
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await agent.close();
|
||||
const cuts = states.slice(before).filter((state) => state.checkpoint);
|
||||
assert.ok(cuts.length > 0, "the resumed turn appended after the checkpoint");
|
||||
for (const [index, state] of cuts.entries()) {
|
||||
const copy = createMemoryPersistence();
|
||||
copy.records.push(...state.records);
|
||||
await copy.saveCheckpoint(state.checkpoint);
|
||||
const reopened = await createFxEngine(options(targetBackend, undefined, { persistence: copy })).catch((error) => {
|
||||
throw new Error(`the cut after resumed append ${index + 1} (checkpoint through ${state.checkpoint.through}, ${state.records.length} records) did not reopen: ${error.code}`);
|
||||
});
|
||||
await reopened.close();
|
||||
}
|
||||
});
|
||||
try {
|
||||
for (const { name, body } of cases) {
|
||||
await body();
|
||||
|
||||
@@ -51,6 +51,18 @@ try {
|
||||
{ type: "tool-call", toolCallId: "images1", toolName: "screenshots", input: {} },
|
||||
{ type: "finish", finishReason: { unified: "tool-calls", raw: "tool-calls" } },
|
||||
);
|
||||
if (requests === 3) {
|
||||
// The bridge carries at most 8 MiB, so this request left out the
|
||||
// older image instead of failing, with a note, and kept the newer.
|
||||
assert.ok(init.body.length <= 8 * 1024 * 1024 - 144 * 1024);
|
||||
const body = Buffer.from(init.body).toString("utf8");
|
||||
assert.equal(body.split(data).length - 1, 1, "only the newer image stays");
|
||||
assert.match(body, /not sent to keep the request under the provider's size limit/);
|
||||
return sse(
|
||||
{ type: "text-delta", delta: "read the text" },
|
||||
{ type: "finish", finishReason: { unified: "stop", raw: "stop" } },
|
||||
);
|
||||
}
|
||||
assert.equal(requests, 2);
|
||||
assert.ok(init.body.length < 8 * 1024 * 1024);
|
||||
assert.ok(Buffer.from(init.body).toString("base64").length > 8 * 1024 * 1024);
|
||||
@@ -76,10 +88,14 @@ try {
|
||||
assert.equal(toolCalls, 1);
|
||||
assert.equal(requests, 2);
|
||||
|
||||
// The retained images plus this prompt exceed the unchanged raw request budget.
|
||||
await assert.rejects(agent.prompt("x".repeat(2 * 1024 * 1024)).result, /HostStreamBackpressure/);
|
||||
assert.equal(requests, 2, "an oversized raw request must not reach host fetch");
|
||||
console.log("native image framing passed: accepted images survive base64 framing; oversized raw requests stay bounded");
|
||||
// The retained images plus this prompt exceed what the bridge carries: the
|
||||
// oldest images leave the request, which reaches host fetch within it.
|
||||
assert.equal((await agent.prompt("x".repeat(2 * 1024 * 1024)).result).stopReason, "end_turn");
|
||||
assert.equal(requests, 3);
|
||||
// A request past the bridge even without its images stays bounded.
|
||||
await assert.rejects(agent.prompt("z".repeat(6.5 * 1024 * 1024)).result, /HostStreamBackpressure/);
|
||||
assert.equal(requests, 3, "an oversized request must not reach host fetch");
|
||||
console.log("native image framing passed: accepted images survive base64 framing; requests past the bridge leave out their oldest images");
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
await agent?.close();
|
||||
|
||||
@@ -27,7 +27,7 @@ const sdkDir = fileURLToPath(new URL("..", import.meta.url));
|
||||
const fixturesDir = fileURLToPath(new URL("./types/", import.meta.url));
|
||||
|
||||
// fx-sdk.js exports these for libfx's own modules; they are not public API.
|
||||
const wasmInternals = new Set(["coreAnswered", "engineInternals", "journalMarksWanted", "normalizeAgentOptions", "normalizeInstructions", "normalizeModelChoice", "resumeOutputShown"]);
|
||||
const wasmInternals = new Set(["checkPromptInput", "coreAnswered", "engineInternals", "journalMarksWanted", "normalizeAgentOptions", "normalizeInstructions", "normalizeModelChoice", "persistenceImages", "resumeOutputShown"]);
|
||||
|
||||
// Each specifier as a consumer resolves it, the declaration file it must
|
||||
// reach, and the module whose runtime exports that file must match. node.cjs
|
||||
|
||||
@@ -205,6 +205,7 @@ export async function worlds(): Promise<void> {
|
||||
async write() {},
|
||||
async writeMulti() {},
|
||||
async get() { return new ReadableStream<Uint8Array>(); },
|
||||
async getChunks() { return { data: [], cursor: null, hasMore: false }; },
|
||||
async getInfo() { return { tailIndex: -1 }; },
|
||||
},
|
||||
async queue() { return { messageId: null }; },
|
||||
|
||||
Vendored
+4
-1
@@ -15,9 +15,12 @@ export interface World {
|
||||
list(params: unknown): unknown;
|
||||
};
|
||||
readonly streams: {
|
||||
write(runId: string, name: string, chunk: string): unknown;
|
||||
/** A session's view is written as text; each prompt image as one chunk of bytes. */
|
||||
write(runId: string, name: string, chunk: string | Uint8Array): unknown;
|
||||
writeMulti?(runId: string, name: string, chunks: string[]): unknown;
|
||||
get(runId: string, name: string, startIndex?: number): unknown;
|
||||
/** Read for a session's images: a page of chunks, `{ data: { data }[] }`. */
|
||||
getChunks(runId: string, name: string, options?: { limit?: number; cursor?: string }): unknown;
|
||||
getInfo(runId: string, name: string): unknown;
|
||||
};
|
||||
queue(queueName: string, message: unknown, options?: unknown): unknown;
|
||||
|
||||
@@ -863,7 +863,11 @@ export type FxStreamLine = (
|
||||
export interface FxSession {
|
||||
/** The session's id; for a new session, null until its first prompt is accepted. */
|
||||
readonly id: string | null;
|
||||
/** Queues `input` as the session's next turn and returns a view of that turn. */
|
||||
/**
|
||||
* Queues `input` as the session's next turn and returns a view of that turn. Throws a
|
||||
* TypeError or RangeError for an input libfx refuses, and stores nothing. A prompt whose
|
||||
* `signal` aborts before it is stored is never stored, and ends `cancelled`.
|
||||
*/
|
||||
prompt(input: FxSessionPromptInput, options?: FxPromptOptions): FxTurn;
|
||||
/** Continues a turn a stopped process left open; its result has the stop reason `idle` when there was none. */
|
||||
resume(): FxResumeTurn;
|
||||
|
||||
+516
-6
@@ -13,6 +13,7 @@ const debug_trace = @import("../core/shared/debug_trace.zig");
|
||||
const session_adapter = @import("../core/session/session_adapter.zig");
|
||||
const checkpoint_codec = @import("../core/agent/runtime/checkpoint.zig");
|
||||
const execution_memory = @import("../core/agent/execution_memory.zig");
|
||||
const runtime_execution_memory = @import("../core/agent/runtime/execution_memory.zig");
|
||||
const tool_dispatch = @import("../core/tooling/tool_dispatch.zig");
|
||||
|
||||
const Allocator = std.mem.Allocator;
|
||||
@@ -20,7 +21,7 @@ const Allocator = std.mem.Allocator;
|
||||
pub const Event = journal.Event;
|
||||
pub const max_load_bytes = journal.max_load_bytes;
|
||||
/// What the model is told when libfx continues a turn a crash left open.
|
||||
const resume_notice = "Resuming from unexpected session interruption.";
|
||||
const resume_notice = runtime_execution_memory.resume_notice;
|
||||
|
||||
/// A host's id for a turn, kept in place so copying a journal copies it.
|
||||
pub const TurnId = struct {
|
||||
@@ -98,13 +99,23 @@ pub const Journal = struct {
|
||||
/// The pending resume is a turn that yielded: it continues without being
|
||||
/// told it was interrupted.
|
||||
pending_resume_yielded: bool = false,
|
||||
/// The host keeps an image store: events and snapshots name the images
|
||||
/// in `stored_images` instead of holding their bytes.
|
||||
image_store: bool = false,
|
||||
/// The newest images the session keeps when the host keeps a store;
|
||||
/// older ones keep only their names (`dropOutsideWindow`).
|
||||
image_window: session_codec.ImageWindow = .{},
|
||||
/// The images the host's store holds: those it said it stored before it
|
||||
/// sent their prompt, and those an open resolved from it. Owned by the
|
||||
/// session's allocator.
|
||||
stored_images: session_codec.StoredImages = .{},
|
||||
/// The calls of the pending resume's answering step that no result was
|
||||
/// recorded for: the ones answered as possibly run. Owned by the
|
||||
/// session's allocator.
|
||||
pending_resume_ambiguous: []types.ToolCall = &.{},
|
||||
/// The last progress this journal sent in the open turn, so the next can
|
||||
/// carry only what is new. Its hashes are owned by the session's
|
||||
/// allocator; `stale` once its turn ended.
|
||||
/// allocator; `stale` once its turn ended or a snapshot took the turn in.
|
||||
last_progress: ?LastProgress = null,
|
||||
|
||||
const LastProgress = struct {
|
||||
@@ -114,6 +125,7 @@ pub const Journal = struct {
|
||||
};
|
||||
|
||||
pub fn deinit(self: *Journal, alloc: Allocator) void {
|
||||
self.stored_images.deinit(alloc);
|
||||
if (self.last_progress) |*last| last.hashes.deinit(alloc);
|
||||
self.last_progress = null;
|
||||
self.dropPendingResume(alloc);
|
||||
@@ -308,7 +320,10 @@ pub const Journal = struct {
|
||||
try params.writer.writeAll("{\"sessionId\":");
|
||||
try jsonrpc.writeJsonStr(session_id, ¶ms.writer);
|
||||
try params.writer.writeAll(",\"events\":[");
|
||||
const next = try self.cursor.write(¶ms.writer, event);
|
||||
const next = if (self.storedImages()) |stored|
|
||||
try self.cursor.writeNamingImages(¶ms.writer, event, stored)
|
||||
else
|
||||
try self.cursor.write(¶ms.writer, event);
|
||||
try params.writer.writeByte(']');
|
||||
if (barrier) try params.writer.writeAll(",\"barrier\":true");
|
||||
try params.writer.writeByte('}');
|
||||
@@ -335,6 +350,80 @@ pub const Journal = struct {
|
||||
}
|
||||
}
|
||||
|
||||
/// The images events and snapshots name, when the host keeps a store.
|
||||
pub fn storedImages(self: *const Journal) ?*const session_codec.StoredImages {
|
||||
return if (self.image_store) &self.stored_images else null;
|
||||
}
|
||||
|
||||
/// Records a prompt image the host stored before it sent it, under
|
||||
/// `name`, which counts only when it is the image's digest. A name that
|
||||
/// does not match leaves the image inline.
|
||||
pub fn noteStoredImage(
|
||||
self: *Journal,
|
||||
alloc: Allocator,
|
||||
image: types.ImageAttachment,
|
||||
name: session_codec.ImageName,
|
||||
) Allocator.Error!void {
|
||||
if (!self.image_store) return;
|
||||
const digest = image.snapshot_sha256 orelse return;
|
||||
if (image.inline_data == null or !std.mem.eql(u8, digest, &name)) {
|
||||
debug_trace.logf("session", "event=libfx_image_not_stored id={d} reason=name_mismatch", .{image.id});
|
||||
return;
|
||||
}
|
||||
try self.stored_images.add(alloc, name);
|
||||
}
|
||||
|
||||
/// Keeps the session within its image window when the host keeps a
|
||||
/// store: each image of `history` (owned by `history_alloc`) older than
|
||||
/// the window loses its bytes and keeps its name. A turn's images join the
|
||||
/// window once the turn enters the history; until then they stay outside
|
||||
/// it, as an open keeps the open turn's (`keepNeededImages`), so a session
|
||||
/// opened at any point holds the images the running one held.
|
||||
pub fn dropOutsideWindow(
|
||||
self: *Journal,
|
||||
alloc: Allocator,
|
||||
history_alloc: Allocator,
|
||||
history: []types.HistoryTurn,
|
||||
) Allocator.Error!void {
|
||||
if (!self.image_store) return;
|
||||
var held: std.ArrayList(HeldImage) = .empty;
|
||||
defer held.deinit(alloc);
|
||||
var sizes: std.ArrayList(?usize) = .empty;
|
||||
defer sizes.deinit(alloc);
|
||||
for (history) |*turn| {
|
||||
if (userOf(turn)) |user| for (user.images) |*image| {
|
||||
if (try countSize(alloc, &sizes, windowSize(image.*))) try held.append(alloc, .{ .prompt = image });
|
||||
};
|
||||
if (executionOf(turn)) |execution| for (execution.tool_steps) |step| for (step.tool_results) |result| for (result.tool_images) |*image| {
|
||||
if (try countSize(alloc, &sizes, toolWindowSize(image.*))) try held.append(alloc, .{ .tool = image });
|
||||
};
|
||||
}
|
||||
const outside = self.image_window.oldestOutside(sizes.items);
|
||||
var dropped: usize = 0;
|
||||
for (held.items[0..outside]) |image| switch (image) {
|
||||
.prompt => |prompt| {
|
||||
const bytes = prompt.inline_data orelse continue;
|
||||
history_alloc.free(bytes);
|
||||
prompt.inline_data = null;
|
||||
dropped += 1;
|
||||
},
|
||||
.tool => |tool| {
|
||||
if (tool.droppedFromSession()) continue;
|
||||
if (tool.name == null) {
|
||||
tool.name = try inlineToolImageName(alloc, tool.data) orelse {
|
||||
debug_trace.logf("session", "event=libfx_image_kept reason=inline_tool_image_not_base64 bytes={d}", .{tool.data.len});
|
||||
continue;
|
||||
};
|
||||
}
|
||||
const empty = try history_alloc.alloc(u8, 0);
|
||||
history_alloc.free(tool.data);
|
||||
tool.data = empty;
|
||||
dropped += 1;
|
||||
},
|
||||
};
|
||||
if (dropped > 0) debug_trace.logf("session", "event=libfx_images_dropped count={d} reason=image_window max_images={d} max_bytes={d}", .{ dropped, self.image_window.max_images, self.image_window.max_bytes });
|
||||
}
|
||||
|
||||
/// Records that a call of the running batch finished, with what the
|
||||
/// model sees of it. Nothing waits for it to be stored: a crash before
|
||||
/// the next progress answers the call with it instead of with "may have
|
||||
@@ -369,6 +458,14 @@ pub const Journal = struct {
|
||||
self.steers_waiting == 0 and self.follow_ups_waiting == 0;
|
||||
}
|
||||
|
||||
/// Records that a snapshot holding the open turn went to the host. A load
|
||||
/// from it starts the turn from the snapshot, without the progress record
|
||||
/// the snapshot folded, so the turn's next progress carries all of it
|
||||
/// instead of extending that record.
|
||||
pub fn noteOpenTurnSnapshot(self: *Journal) void {
|
||||
if (self.last_progress) |*last| last.stale = true;
|
||||
}
|
||||
|
||||
/// Records that the open turn ended without a history entry. A turn
|
||||
/// with no progress on record needs no event.
|
||||
pub fn clearProgress(self: *Journal, alloc: Allocator, session_id: []const u8) !void {
|
||||
@@ -410,12 +507,189 @@ fn dupePendingInputs(alloc: Allocator, inputs: []const journal.PendingInput) All
|
||||
return owned;
|
||||
}
|
||||
|
||||
fn userOf(turn: *types.HistoryTurn) ?*types.UserTurn {
|
||||
return switch (turn.*) {
|
||||
.assistant => |*entry| &entry.user,
|
||||
.interrupted => |*entry| &entry.user,
|
||||
.compacted_summary => null,
|
||||
};
|
||||
}
|
||||
|
||||
fn executionOf(turn: *types.HistoryTurn) ?*types.ExecutionMemory {
|
||||
return switch (turn.*) {
|
||||
.assistant => |*entry| &entry.execution,
|
||||
.interrupted => |*entry| &entry.execution,
|
||||
.compacted_summary => null,
|
||||
};
|
||||
}
|
||||
|
||||
/// An image the window counts: a prompt's, or one a tool returned, counted
|
||||
/// together in the order they entered the session.
|
||||
const HeldImage = union(enum) {
|
||||
prompt: *types.ImageAttachment,
|
||||
tool: *types.ToolImage,
|
||||
};
|
||||
|
||||
const MissingImages = std.array_hash_map.Auto(session_codec.ImageName, usize);
|
||||
|
||||
/// What the image window counts for one image.
|
||||
const WindowSize = union(enum) {
|
||||
/// Its bytes: those it holds, or for an image an open has yet to stage,
|
||||
/// the size its record gives (`unstaged`).
|
||||
bytes: usize,
|
||||
/// One the session dropped, which keeps every older image out.
|
||||
dropped,
|
||||
/// One with neither bytes nor a name, which only a host reference
|
||||
/// describes, and so holds nothing.
|
||||
none,
|
||||
};
|
||||
|
||||
/// What the window counts for `image`, as the session holds it.
|
||||
fn windowSize(image: types.ImageAttachment) WindowSize {
|
||||
if (image.inline_data) |bytes| return .{ .bytes = bytes.len };
|
||||
return if (image.droppedFromSession()) .dropped else .none;
|
||||
}
|
||||
|
||||
/// What the window counts for a tool image: its decoded bytes, whether the
|
||||
/// host stored it or the session holds it inline, as one the store refused
|
||||
/// or an older libfx saved. One with only a host reference holds nothing.
|
||||
fn toolWindowSize(image: types.ToolImage) WindowSize {
|
||||
if (image.droppedFromSession()) return .dropped;
|
||||
if (image.data.len == 0) return .none;
|
||||
return .{ .bytes = std.base64.standard.Decoder.calcSizeForSlice(image.data) catch image.data.len };
|
||||
}
|
||||
|
||||
/// The name of a tool image's bytes, which one held inline takes when it
|
||||
/// leaves the window, so it is saved dropped like a stored one; null for
|
||||
/// bytes that are not base64.
|
||||
fn inlineToolImageName(alloc: Allocator, data: []const u8) Allocator.Error!?session_codec.ImageName {
|
||||
const size = std.base64.standard.Decoder.calcSizeForSlice(data) catch return null;
|
||||
const bytes = try alloc.alloc(u8, size);
|
||||
defer alloc.free(bytes);
|
||||
std.base64.standard.Decoder.decode(bytes, data) catch return null;
|
||||
return session_codec.imageNameOf(bytes);
|
||||
}
|
||||
|
||||
/// An image an open has yet to stage, with the size its record gives.
|
||||
const Unstaged = struct { name: session_codec.ImageName, size: usize };
|
||||
|
||||
/// The name and size of `image` when it is one an open has yet to stage: it
|
||||
/// reads such an image as one without bytes, which `missing` names.
|
||||
fn unstaged(image: types.ImageAttachment, missing: *const MissingImages) ?Unstaged {
|
||||
if (!image.droppedFromSession()) return null;
|
||||
const name = session_codec.imageName(image.snapshot_sha256.?) orelse return null;
|
||||
return .{ .name = name, .size = missing.get(name) orelse return null };
|
||||
}
|
||||
|
||||
/// `unstaged` for a tool image, which an open reads by name without bytes.
|
||||
fn unstagedTool(image: types.ToolImage, missing: *const MissingImages) ?Unstaged {
|
||||
if (!image.droppedFromSession()) return null;
|
||||
const name = image.name.?;
|
||||
return .{ .name = name, .size = missing.get(name) orelse return null };
|
||||
}
|
||||
|
||||
/// Appends what the window counts for one image to `sizes`, oldest first,
|
||||
/// and returns whether it counted.
|
||||
fn countSize(alloc: Allocator, sizes: *std.ArrayList(?usize), size: WindowSize) Allocator.Error!bool {
|
||||
switch (size) {
|
||||
.bytes => |bytes| try sizes.append(alloc, bytes),
|
||||
.dropped => try sizes.append(alloc, null),
|
||||
.none => return false,
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/// Leaves in `images.missing` only the names of images the window keeps, so
|
||||
/// the host stages none the session would drop. The window counts the
|
||||
/// history alone, as it did when the last turn entered it
|
||||
/// (`Journal.dropOutsideWindow`); the open turn's images, its prompt's and
|
||||
/// its tools', are always needed.
|
||||
fn keepNeededImages(
|
||||
alloc: Allocator,
|
||||
images: *session_codec.ImageResolver,
|
||||
history: []const types.HistoryTurn,
|
||||
open_user: ?types.UserTurn,
|
||||
open_execution: ?types.ExecutionMemory,
|
||||
open_finished: []const journal.Finished,
|
||||
window: session_codec.ImageWindow,
|
||||
) Allocator.Error!void {
|
||||
var sizes: std.ArrayList(?usize) = .empty;
|
||||
defer sizes.deinit(alloc);
|
||||
var names: std.ArrayList(?session_codec.ImageName) = .empty;
|
||||
defer names.deinit(alloc);
|
||||
for (history) |turn| {
|
||||
const user, const execution = switch (turn) {
|
||||
.assistant => |entry| .{ entry.user, entry.execution },
|
||||
.interrupted => |entry| .{ entry.user, entry.execution },
|
||||
.compacted_summary => continue,
|
||||
};
|
||||
try countOpenedImages(alloc, &sizes, &names, user.images, &images.missing);
|
||||
try countOpenedToolImages(alloc, &sizes, &names, execution, &images.missing);
|
||||
}
|
||||
const outside = window.oldestOutside(sizes.items);
|
||||
var needed: MissingImages = .empty;
|
||||
errdefer needed.deinit(images.alloc);
|
||||
for (names.items[outside..], sizes.items[outside..]) |name, size| {
|
||||
if (name) |value| try needed.put(images.alloc, value, size.?);
|
||||
}
|
||||
if (open_user) |user| for (user.images) |image| {
|
||||
if (unstaged(image, &images.missing)) |found| try needed.put(images.alloc, found.name, found.size);
|
||||
};
|
||||
if (open_execution) |execution| for (execution.tool_steps) |step| for (step.tool_results) |result| for (result.tool_images) |image| {
|
||||
if (unstagedTool(image, &images.missing)) |found| try needed.put(images.alloc, found.name, found.size);
|
||||
};
|
||||
// A result the open turn recorded before a crash answers its call again.
|
||||
for (open_finished) |finished| for (finished.images) |image| {
|
||||
if (unstagedTool(image, &images.missing)) |found| try needed.put(images.alloc, found.name, found.size);
|
||||
};
|
||||
if (outside > 0) debug_trace.logf("session", "event=libfx_image_window_open outside={d} needed={d}", .{ outside, needed.count() });
|
||||
images.missing.deinit(images.alloc);
|
||||
images.missing = needed;
|
||||
}
|
||||
|
||||
/// Counts the images of `user_images` an open read, each with its name when
|
||||
/// it is one the host has yet to stage.
|
||||
fn countOpenedImages(
|
||||
alloc: Allocator,
|
||||
sizes: *std.ArrayList(?usize),
|
||||
names: *std.ArrayList(?session_codec.ImageName),
|
||||
user_images: []const types.ImageAttachment,
|
||||
missing: *const MissingImages,
|
||||
) Allocator.Error!void {
|
||||
for (user_images) |image| {
|
||||
const pending = unstaged(image, missing);
|
||||
if (!try countSize(alloc, sizes, if (pending) |found| .{ .bytes = found.size } else windowSize(image))) continue;
|
||||
try names.append(alloc, if (pending) |found| found.name else null);
|
||||
}
|
||||
}
|
||||
|
||||
/// Counts the tool images of `execution` an open read, as `countOpenedImages`
|
||||
/// counts a prompt's.
|
||||
fn countOpenedToolImages(
|
||||
alloc: Allocator,
|
||||
sizes: *std.ArrayList(?usize),
|
||||
names: *std.ArrayList(?session_codec.ImageName),
|
||||
execution: types.ExecutionMemory,
|
||||
missing: *const MissingImages,
|
||||
) Allocator.Error!void {
|
||||
for (execution.tool_steps) |step| for (step.tool_results) |result| for (result.tool_images) |image| {
|
||||
const pending = unstagedTool(image, missing);
|
||||
if (!try countSize(alloc, sizes, if (pending) |found| .{ .bytes = found.size } else toolWindowSize(image))) continue;
|
||||
try names.append(alloc, if (pending) |found| found.name else null);
|
||||
};
|
||||
}
|
||||
|
||||
/// Rebuilds a fresh session from the host's events, after `snapshot` when
|
||||
/// the host stored one, and starts its journal after them. A turn a crash
|
||||
/// left open becomes the pending resume, with every call it left running
|
||||
/// answered as possibly run, for the model to decide about unless the host
|
||||
/// has the resume run it again (`rerunCalls`); `session_alloc` owns it.
|
||||
/// With `inputs_durable`, the host stores every input before handing it in.
|
||||
/// Named images resolve through `images`, which holds what the host staged;
|
||||
/// when one is missing the session stays unopened and `error.ImagesMissing`
|
||||
/// leaves every missing name in `images.missing`. With `image_store` the
|
||||
/// journal names stored images from then on, and the session keeps only the
|
||||
/// newest images `window` holds: an older one is neither needed nor kept.
|
||||
pub fn open(
|
||||
alloc: Allocator,
|
||||
session_alloc: Allocator,
|
||||
@@ -424,16 +698,19 @@ pub fn open(
|
||||
events_json: []const u8,
|
||||
snapshot: ?[]const u8,
|
||||
inputs_durable: bool,
|
||||
images: *session_codec.ImageResolver,
|
||||
image_store: bool,
|
||||
window: session_codec.ImageWindow,
|
||||
) !struct { Journal, Opened } {
|
||||
if (!runtime.agent.fresh or runtime.agent.history.items.len != 0) return error.AgentNotFresh;
|
||||
var base: journal.Base = .{};
|
||||
var base: journal.Base = .{ .images = images };
|
||||
var decoded: ?checkpoint_codec.Decoded = null;
|
||||
defer if (decoded) |*value| value.deinit(alloc);
|
||||
var saved_meta: checkpoint_codec.Meta = .{};
|
||||
errdefer saved_meta.free(alloc);
|
||||
if (snapshot) |bytes| {
|
||||
const parsed = try journal.parseSnapshot(bytes);
|
||||
decoded = checkpoint_codec.decode(alloc, parsed.checkpoint) catch |err| switch (err) {
|
||||
decoded = checkpoint_codec.decodeWithImages(alloc, parsed.checkpoint, images) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
error.UnsupportedCheckpointVersion => return error.UnsupportedJournalVersion,
|
||||
else => return error.InvalidJournal,
|
||||
@@ -444,10 +721,19 @@ pub fn open(
|
||||
.cursor = .{ .next_seq = parsed.seq + 1, .turn = parsed.turn },
|
||||
.last_turn_id = parsed.last_turn_id,
|
||||
.open_turn = parsed.open_turn,
|
||||
.images = images,
|
||||
};
|
||||
}
|
||||
var folded = try journal.foldFrom(alloc, events_json, base);
|
||||
defer folded.deinit(alloc);
|
||||
if (image_store and images.missing.count() > 0) {
|
||||
try keepNeededImages(alloc, images, folded.history, if (folded.open_turn) |turn| turn.user else null, if (folded.open_turn) |turn| turn.execution else null, folded.finished, window);
|
||||
}
|
||||
// A history missing an image it keeps is not the one saved: open none.
|
||||
if (images.missing.count() > 0) {
|
||||
debug_trace.logf("session", "event=libfx_journal_images_missing count={d}", .{images.missing.count()});
|
||||
return error.ImagesMissing;
|
||||
}
|
||||
if (folded.history.len > 0) try runtime.agent.restoreHistory(alloc, folded.history);
|
||||
if (decoded) |value| runtime.agent.turn_usage = value.usage;
|
||||
var state: Journal = .{
|
||||
@@ -460,10 +746,16 @@ pub fn open(
|
||||
.last_turn_id = TurnId.of(folded.last_turn_id),
|
||||
.inputs_durable = inputs_durable,
|
||||
.steers_waiting = folded.pending_inputs.len,
|
||||
.image_store = image_store,
|
||||
.image_window = window,
|
||||
};
|
||||
errdefer state.deinit(session_alloc);
|
||||
const checkpoint = folded.open_turn orelse
|
||||
// An image the open resolved is in the store; one it read inline is not.
|
||||
if (image_store) for (images.resolved.keys()) |name| try state.stored_images.add(session_alloc, name);
|
||||
try state.dropOutsideWindow(alloc, alloc, runtime.agent.history.items);
|
||||
const checkpoint = folded.open_turn orelse {
|
||||
return .{ state, .{ .turns = folded.history.len, .resumable = false, .last_turn_id = state.last_turn_id, .checkpoint_meta = saved_meta } };
|
||||
};
|
||||
|
||||
debug_trace.logf("session", "event=libfx_journal_open_turn turn={d} running_calls={d} finished_calls={d} resolved=pending_resume", .{ folded.cursor.turn, folded.running_calls.len, folded.finished.len });
|
||||
var scratch = std.heap.ArenaAllocator.init(alloc);
|
||||
@@ -571,6 +863,7 @@ fn answerRunning(scratch: Allocator, turn: *session_codec.RecoveryCheckpoint, ca
|
||||
.tool_name = @constCast(call.name),
|
||||
.status = if (recorded) |entry| (if (entry.failed) .failure else .success) else .failure,
|
||||
.output = @constCast(output),
|
||||
.tool_images = if (recorded) |entry| entry.images else &.{},
|
||||
.output_bytes = output.len,
|
||||
.stored_output_bytes = output.len,
|
||||
};
|
||||
@@ -835,6 +1128,29 @@ test "a progress after its turn's first carries only what is new, and the next t
|
||||
try std.testing.expect(std.mem.find(u8, capture.frames.items[next..], "\"user\"") != null);
|
||||
}
|
||||
|
||||
test "a progress after a snapshot that holds its turn carries all of it" {
|
||||
const alloc = std.testing.allocator;
|
||||
var capture: TestCapture = .{};
|
||||
defer capture.frames.deinit(alloc);
|
||||
var writer = jsonrpc.Writer.initCallback(&capture, TestCapture.write);
|
||||
var session: Journal = .{ .writer = &writer };
|
||||
defer session.deinit(alloc);
|
||||
var progress = try testProgress(alloc);
|
||||
defer progress.deinit(alloc);
|
||||
|
||||
try session.appendProgress(alloc, alloc, "session", progress, "fake/model");
|
||||
// A load from the snapshot has no record for the next progress to extend.
|
||||
session.noteOpenTurnSnapshot();
|
||||
const next = capture.frames.items.len;
|
||||
try session.appendProgress(alloc, alloc, "session", progress, "fake/model");
|
||||
try std.testing.expect(std.mem.find(u8, capture.frames.items[next..], "\"base\"") == null);
|
||||
try std.testing.expect(std.mem.find(u8, capture.frames.items[next..], "\"user\"") != null);
|
||||
// The progress after that one extends it again.
|
||||
const after = capture.frames.items.len;
|
||||
try session.appendProgress(alloc, alloc, "session", progress, "fake/model");
|
||||
try std.testing.expect(std.mem.find(u8, capture.frames.items[after..], "\"base\":2,") != null);
|
||||
}
|
||||
|
||||
test "a resumed turn runs again the calls the host asks for and leaves the rest to the model" {
|
||||
const alloc = std.testing.allocator;
|
||||
var calls = [_]types.ToolCall{
|
||||
@@ -883,3 +1199,197 @@ test "a resume cancelled during a rerun keeps that call and the later ones answe
|
||||
// A step index past the turn's steps runs nothing.
|
||||
try std.testing.expectEqual(@as(usize, 0), try rerunCalls(alloc, &answered, 5, TestRerunner{ .ids = &ids, .ran = &ran }, null));
|
||||
}
|
||||
|
||||
/// An inline image of `size` bytes for the window tests, every field owned by
|
||||
/// `alloc`; `types.freeImageAttachment` frees it.
|
||||
fn windowTestImage(alloc: Allocator, id: usize, size: usize) !types.ImageAttachment {
|
||||
const bytes = try alloc.alloc(u8, size);
|
||||
errdefer alloc.free(bytes);
|
||||
@memset(bytes, 'x');
|
||||
const path = try alloc.dupe(u8, "inline://image");
|
||||
errdefer alloc.free(path);
|
||||
const media_type = try alloc.dupe(u8, "image/png");
|
||||
errdefer alloc.free(media_type);
|
||||
const digest = session_codec.imageNameOf(bytes);
|
||||
return .{ .id = id, .path = path, .media_type = media_type, .snapshot_sha256 = try alloc.dupe(u8, &digest), .inline_data = bytes };
|
||||
}
|
||||
|
||||
test "the image window drops the oldest history images, and not for an open turn's" {
|
||||
const alloc = std.testing.allocator;
|
||||
var capture: TestCapture = .{};
|
||||
defer capture.frames.deinit(alloc);
|
||||
var writer = jsonrpc.Writer.initCallback(&capture, TestCapture.write);
|
||||
var session: Journal = .{ .writer = &writer, .image_store = true, .image_window = .{ .max_images = 2 } };
|
||||
defer session.deinit(alloc);
|
||||
var images: [3]types.ImageAttachment = undefined;
|
||||
for (&images, 0..) |*image, index| image.* = try windowTestImage(alloc, index + 1, 10);
|
||||
defer for (images) |image| types.freeImageAttachment(alloc, image);
|
||||
var history: [3]types.HistoryTurn = undefined;
|
||||
for (&history, &images) |*turn, *image| {
|
||||
turn.* = .{ .assistant = .{ .user = .{ .text = @constCast("look"), .images = image[0..1] }, .assistant = @constCast("ok") } };
|
||||
}
|
||||
|
||||
// Two images at most: the two newest of history.
|
||||
try session.dropOutsideWindow(alloc, alloc, &history);
|
||||
try std.testing.expect(images[0].droppedFromSession());
|
||||
try std.testing.expect(images[1].inline_data != null and images[2].inline_data != null);
|
||||
|
||||
// An open turn has not entered the history, so its image moves nothing.
|
||||
var pending = try testProgress(alloc);
|
||||
pending.user.images = try alloc.alloc(types.ImageAttachment, 1);
|
||||
pending.user.images[0] = try windowTestImage(alloc, 4, 10);
|
||||
session.pending_resume = pending;
|
||||
try session.dropOutsideWindow(alloc, alloc, &history);
|
||||
try std.testing.expect(images[1].inline_data != null and images[2].inline_data != null);
|
||||
try std.testing.expect(session.pending_resume.?.user.images[0].inline_data != null);
|
||||
}
|
||||
|
||||
/// A one-image slice a tool returned, `size` bytes as base64, named as the
|
||||
/// host stored it unless `named` is false; `types.freeToolImages` frees it.
|
||||
fn windowTestToolImages(alloc: Allocator, size: usize, named: bool) ![]types.ToolImage {
|
||||
const bytes = try alloc.alloc(u8, size);
|
||||
defer alloc.free(bytes);
|
||||
@memset(bytes, 't');
|
||||
const encoded = try alloc.alloc(u8, std.base64.standard.Encoder.calcSize(size));
|
||||
errdefer alloc.free(encoded);
|
||||
_ = std.base64.standard.Encoder.encode(encoded, bytes);
|
||||
const mime_type = try alloc.dupe(u8, "image/png");
|
||||
errdefer alloc.free(mime_type);
|
||||
const images = try alloc.alloc(types.ToolImage, 1);
|
||||
images[0] = .{ .data = encoded, .mime_type = mime_type, .name = if (named) session_codec.imageNameOf(bytes) else null };
|
||||
return images;
|
||||
}
|
||||
|
||||
fn windowTestToolStep(results: *[1]types.PersistedToolResult, steps: *[1]types.ToolExecutionStep, images: []types.ToolImage) types.ExecutionMemory {
|
||||
results.* = .{.{
|
||||
.tool_images = images,
|
||||
.tool_call_id = @constCast("call"),
|
||||
.tool_name = @constCast("screenshot"),
|
||||
.status = .success,
|
||||
.output = @constCast("took it"),
|
||||
.output_bytes = 7,
|
||||
.stored_output_bytes = 7,
|
||||
}};
|
||||
steps.* = .{.{ .tool_results = results }};
|
||||
return .{ .tool_steps = steps };
|
||||
}
|
||||
|
||||
test "the image window counts a turn's tool images after its prompt's, inline ones too, and drops them by name" {
|
||||
const alloc = std.testing.allocator;
|
||||
var capture: TestCapture = .{};
|
||||
defer capture.frames.deinit(alloc);
|
||||
var writer = jsonrpc.Writer.initCallback(&capture, TestCapture.write);
|
||||
var session: Journal = .{ .writer = &writer, .image_store = true, .image_window = .{ .max_images = 2 } };
|
||||
defer session.deinit(alloc);
|
||||
var prompts: [2]types.ImageAttachment = undefined;
|
||||
for (&prompts, 0..) |*image, index| image.* = try windowTestImage(alloc, index + 1, 10);
|
||||
defer for (prompts) |image| types.freeImageAttachment(alloc, image);
|
||||
var tools: [2][]types.ToolImage = undefined;
|
||||
for (&tools) |*images| images.* = try windowTestToolImages(alloc, 10, true);
|
||||
defer for (tools) |images| types.freeToolImages(alloc, images);
|
||||
// A tool image the session holds inline, as one the store refused, the
|
||||
// oldest tool image here.
|
||||
const unstored = try windowTestToolImages(alloc, 12, false);
|
||||
defer types.freeToolImages(alloc, unstored);
|
||||
var unstored_bytes: [12]u8 = undefined;
|
||||
try std.base64.standard.Decoder.decode(&unstored_bytes, unstored[0].data);
|
||||
var results: [3][1]types.PersistedToolResult = undefined;
|
||||
var steps: [3][1]types.ToolExecutionStep = undefined;
|
||||
var history: [3]types.HistoryTurn = undefined;
|
||||
history[0] = .{ .assistant = .{ .user = .{ .text = @constCast("one"), .images = prompts[0..1] }, .assistant = @constCast("ok"), .execution = windowTestToolStep(&results[0], &steps[0], unstored) } };
|
||||
history[1] = .{ .assistant = .{ .user = .{ .text = @constCast("two"), .images = prompts[1..2] }, .assistant = @constCast("ok"), .execution = windowTestToolStep(&results[1], &steps[1], tools[0]) } };
|
||||
history[2] = .{ .assistant = .{ .user = .{ .text = @constCast("three") }, .assistant = @constCast("ok"), .execution = windowTestToolStep(&results[2], &steps[2], tools[1]) } };
|
||||
|
||||
// Two images at most: the newest two tool images, in the order they
|
||||
// entered; every older one leaves, keeping its name. The inline one
|
||||
// counts as any image does, and takes the name of its bytes.
|
||||
try session.dropOutsideWindow(alloc, alloc, &history);
|
||||
try std.testing.expect(prompts[0].droppedFromSession());
|
||||
try std.testing.expect(unstored[0].droppedFromSession());
|
||||
try std.testing.expectEqualSlices(u8, &session_codec.imageNameOf(&unstored_bytes), &unstored[0].name.?);
|
||||
try std.testing.expect(prompts[1].droppedFromSession());
|
||||
try std.testing.expect(!tools[0][0].droppedFromSession() and tools[0][0].data.len > 0);
|
||||
try std.testing.expect(!tools[1][0].droppedFromSession() and tools[1][0].data.len > 0);
|
||||
}
|
||||
|
||||
test "an open needs the newest tool images its window keeps, and every one of the open turn" {
|
||||
const alloc = std.testing.allocator;
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
// Three tool images the host has not staged yet, read as names alone.
|
||||
var names: [3]session_codec.ImageName = undefined;
|
||||
var images: [3][1]types.ToolImage = undefined;
|
||||
for (&names, &images, 0..) |*name, *image, index| {
|
||||
name.* = session_codec.imageNameOf(&.{@as(u8, @intCast(index))});
|
||||
image.* = .{.{ .data = @constCast(&[_]u8{}), .mime_type = @constCast("image/png"), .name = name.* }};
|
||||
try resolver.missing.put(alloc, name.*, 10);
|
||||
}
|
||||
var results: [3][1]types.PersistedToolResult = undefined;
|
||||
var steps: [3][1]types.ToolExecutionStep = undefined;
|
||||
var history: [2]types.HistoryTurn = undefined;
|
||||
for (&history, 0..) |*turn, index| {
|
||||
turn.* = .{ .assistant = .{ .user = .{ .text = @constCast("look") }, .assistant = @constCast("ok"), .execution = windowTestToolStep(&results[index], &steps[index], &images[index]) } };
|
||||
}
|
||||
const open_execution = windowTestToolStep(&results[2], &steps[2], &images[2]);
|
||||
|
||||
try keepNeededImages(alloc, &resolver, &history, null, open_execution, &.{}, .{ .max_images = 1 });
|
||||
try std.testing.expectEqual(@as(usize, 2), resolver.missing.count());
|
||||
try std.testing.expect(!resolver.missing.contains(names[0]));
|
||||
try std.testing.expect(resolver.missing.contains(names[1]) and resolver.missing.contains(names[2]));
|
||||
}
|
||||
|
||||
test "an open needs the images of a result its open turn recorded before a crash" {
|
||||
const alloc = std.testing.allocator;
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
const name = session_codec.imageNameOf("recorded shot");
|
||||
try resolver.missing.put(alloc, name, 13);
|
||||
var images = [_]types.ToolImage{.{ .data = @constCast(&[_]u8{}), .mime_type = @constCast("image/png"), .name = name }};
|
||||
const finished = [_]journal.Finished{.{ .id = @constCast("call-1"), .failed = false, .output = @constCast("took it"), .images = &images }};
|
||||
|
||||
try keepNeededImages(alloc, &resolver, &.{}, null, null, &finished, .{ .max_images = 1 });
|
||||
try std.testing.expectEqual(@as(usize, 1), resolver.missing.count());
|
||||
try std.testing.expectEqual(@as(?usize, 13), resolver.missing.get(name));
|
||||
}
|
||||
|
||||
test "an open needs only the named images its window keeps, and every image of the open turn" {
|
||||
const alloc = std.testing.allocator;
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
// Four named images the host has not staged yet, oldest first.
|
||||
var names: [4]session_codec.ImageName = undefined;
|
||||
var images: [4]types.ImageAttachment = undefined;
|
||||
for (&names, &images, 0..) |*name, *image, index| {
|
||||
name.* = session_codec.imageNameOf(&.{@as(u8, @intCast(index))});
|
||||
image.* = .{ .id = index + 1, .path = @constCast("inline://image"), .media_type = @constCast("image/png"), .snapshot_sha256 = name };
|
||||
}
|
||||
const reset = struct {
|
||||
fn run(target: *session_codec.ImageResolver, all: []const session_codec.ImageName) !void {
|
||||
target.missing.clearRetainingCapacity();
|
||||
for (all) |name| try target.missing.put(target.alloc, name, 10);
|
||||
}
|
||||
}.run;
|
||||
var history: [3]types.HistoryTurn = undefined;
|
||||
for (&history, images[0..3]) |*turn, *image| {
|
||||
turn.* = .{ .assistant = .{ .user = .{ .text = @constCast("look"), .images = image[0..1] }, .assistant = @constCast("ok") } };
|
||||
}
|
||||
|
||||
// The window counts the history alone: its two newest, and the open
|
||||
// turn's image as well.
|
||||
try reset(&resolver, &names);
|
||||
try keepNeededImages(alloc, &resolver, &history, .{ .text = @constCast("now"), .images = images[3..4] }, null, &.{}, .{ .max_images = 2 });
|
||||
try std.testing.expectEqual(@as(usize, 3), resolver.missing.count());
|
||||
try std.testing.expect(!resolver.missing.contains(names[0]));
|
||||
|
||||
// An open turn larger than the window keeps all of its images.
|
||||
try reset(&resolver, &names);
|
||||
try keepNeededImages(alloc, &resolver, history[0..2], .{ .text = @constCast("now"), .images = images[2..4] }, null, &.{}, .{ .max_images = 1 });
|
||||
try std.testing.expectEqual(@as(usize, 3), resolver.missing.count());
|
||||
try std.testing.expect(!resolver.missing.contains(names[0]));
|
||||
}
|
||||
@@ -140,7 +140,7 @@ fn prepareHost(
|
||||
defer inherited.deinit();
|
||||
const h = mcp_host.Host.create(alloc, io_mod.getIo(), configs, &inherited, .{
|
||||
.client_version = build_options.app_version,
|
||||
.credentials = .detect(alloc, home),
|
||||
.credentials = .{ .detect = home },
|
||||
.questions = questions,
|
||||
.url_completions = url_completions,
|
||||
}, builtin_tools.registry) catch |err| switch (err) {
|
||||
|
||||
+74
-3
@@ -134,10 +134,48 @@ fn promptInputFailure(err: anyerror) anyerror!TerminalOutcome {
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Image prompt exceeds size limit",
|
||||
} },
|
||||
error.PromptImagesTooLarge => .{ .rpc_error = .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Prompt images exceed the libfx limit",
|
||||
} },
|
||||
else => err,
|
||||
};
|
||||
}
|
||||
|
||||
/// Whether `images` fit one libfx prompt: at most
|
||||
/// `image_attachments.max_libfx_prompt_images` of them, holding at most
|
||||
/// `max_libfx_prompt_images_bytes` together. An open stages its open turn's
|
||||
/// images on top of the session's image window, and counts on this.
|
||||
fn fitsLibfxPrompt(images: []const PendingPromptImage) bool {
|
||||
if (images.len > image_attachments.max_libfx_prompt_images) return false;
|
||||
var bytes: usize = 0;
|
||||
for (images) |image| bytes += image.bytes.len;
|
||||
return bytes <= image_attachments.max_libfx_prompt_images_bytes;
|
||||
}
|
||||
|
||||
test "a libfx prompt holds at most its limit of images and image bytes" {
|
||||
const alloc = std.testing.allocator;
|
||||
var media_type = "image/png".*;
|
||||
var byte: [1]u8 = .{0x89};
|
||||
var images: [image_attachments.max_libfx_prompt_images + 1]PendingPromptImage = @splat(.{ .id = 1, .bytes = &byte, .media_type = &media_type });
|
||||
try std.testing.expect(fitsLibfxPrompt(images[0..image_attachments.max_libfx_prompt_images]));
|
||||
try std.testing.expect(!fitsLibfxPrompt(&images));
|
||||
// Reference-only images count toward the number too.
|
||||
var references = images;
|
||||
for (&references) |*image| image.bytes = &.{};
|
||||
try std.testing.expect(!fitsLibfxPrompt(&references));
|
||||
|
||||
const bytes = try alloc.alloc(u8, image_attachments.max_libfx_prompt_images_bytes + 1);
|
||||
defer alloc.free(bytes);
|
||||
const exact = [_]PendingPromptImage{.{ .id = 1, .bytes = bytes[1..], .media_type = &media_type }};
|
||||
try std.testing.expect(fitsLibfxPrompt(&exact));
|
||||
const over = [_]PendingPromptImage{
|
||||
.{ .id = 1, .bytes = bytes[0..1024], .media_type = &media_type },
|
||||
.{ .id = 2, .bytes = bytes[1024..], .media_type = &media_type },
|
||||
};
|
||||
try std.testing.expect(!fitsLibfxPrompt(&over));
|
||||
}
|
||||
|
||||
fn promptExecutionFailure(err: anyerror) anyerror!TerminalOutcome {
|
||||
return switch (err) {
|
||||
error.ModelImageCapabilityUnavailable,
|
||||
@@ -880,8 +918,10 @@ pub fn handlePrompt(
|
||||
if (session.store == null and session.wasm_state == null and session.v2 == null) {
|
||||
// libfx kernel session: images stay in memory and ride the kernel
|
||||
// checkpoint, so no filesystem snapshot backend is needed.
|
||||
if (!fitsLibfxPrompt(prompt_input.pending_images)) return promptInputFailure(error.PromptImagesTooLarge);
|
||||
prompt_input.captureImagesInline(alloc) catch |err|
|
||||
return promptInputFailure(err);
|
||||
if (session.journal) |*journal| try prompt_input.noteStoredImages(state.alloc, journal);
|
||||
} else {
|
||||
if (comptime host_target.is_wasm) return promptInputFailure(error.UnsupportedPromptImage);
|
||||
var temporary_snapshot_dir: ?[]u8 = null;
|
||||
@@ -1488,6 +1528,8 @@ const PendingPromptImage = struct {
|
||||
bytes: []u8,
|
||||
media_type: []u8,
|
||||
source_ref: ?[]u8 = null,
|
||||
/// The name the host stored the bytes under before it sent them.
|
||||
stored_name: ?session_codec.ImageName = null,
|
||||
|
||||
fn deinit(self: *PendingPromptImage, alloc: Allocator) void {
|
||||
alloc.free(self.bytes);
|
||||
@@ -1587,6 +1629,16 @@ const ParsedPromptInput = struct {
|
||||
self.images = images;
|
||||
}
|
||||
|
||||
/// Tells `journal` which captured images the host stored before it sent
|
||||
/// them, so its events name them. `captureImagesInline` keeps the order.
|
||||
fn noteStoredImages(self: *const ParsedPromptInput, session_alloc: Allocator, journal: *libfx_journal.Journal) Allocator.Error!void {
|
||||
std.debug.assert(self.images.len == self.pending_images.len);
|
||||
for (self.images, self.pending_images) |image, pending| {
|
||||
const name = pending.stored_name orelse continue;
|
||||
try journal.noteStoredImage(session_alloc, image, name);
|
||||
}
|
||||
}
|
||||
|
||||
fn retainImageSnapshots(self: *ParsedPromptInput) void {
|
||||
self.retain_image_snapshots = true;
|
||||
}
|
||||
@@ -1770,6 +1822,11 @@ fn parsePromptInputWithFirstImageId(
|
||||
break :bytes try alloc.alloc(u8, 0);
|
||||
};
|
||||
errdefer alloc.free(decoded);
|
||||
// A host with an image store names the bytes it stored first.
|
||||
const stored_name: ?session_codec.ImageName = if (acp_types.fxMetaField(block.object, "stored")) |value| name: {
|
||||
if (value != .string or decoded.len == 0) return error.InvalidPromptImage;
|
||||
break :name session_codec.imageName(value.string) orelse return error.InvalidPromptImage;
|
||||
} else null;
|
||||
|
||||
const image_id = std.math.add(usize, first_image_id, pending_images.items.len) catch
|
||||
return error.ImageIdOverflow;
|
||||
@@ -1788,6 +1845,7 @@ fn parsePromptInputWithFirstImageId(
|
||||
.bytes = decoded,
|
||||
.media_type = media_type,
|
||||
.source_ref = source_ref,
|
||||
.stored_name = stored_name,
|
||||
});
|
||||
} else if (std.mem.eql(u8, block_type.string, "resource")) {
|
||||
if (block.object.get("resource")) |resource| {
|
||||
@@ -2609,9 +2667,9 @@ fn completeAuthorizedToolCallTransport(
|
||||
}
|
||||
|
||||
/// Records a finished call's result in a journaled libfx session, so a
|
||||
/// crash before the next progress answers it with what the model sees
|
||||
/// instead of with "may have partly run". Nothing waits for it. A cancelled
|
||||
/// call did not finish and stays unanswered; a result's images are not kept.
|
||||
/// crash before the next progress answers it with what the model sees,
|
||||
/// its images too, instead of with "may have partly run". Nothing waits for
|
||||
/// it. A cancelled call did not finish and stays unanswered.
|
||||
fn recordJournalToolResult(ctx: *AcpContext, call: ToolCall, result: ToolExecutionResult) void {
|
||||
if (result.cancelled) return;
|
||||
const session = if (ctx.state.active_session) |*value| value else return;
|
||||
@@ -2623,6 +2681,7 @@ fn recordJournalToolResult(ctx: *AcpContext, call: ToolCall, result: ToolExecuti
|
||||
.name = call.name,
|
||||
.failed = result.status == .failure,
|
||||
.output = result.model_output,
|
||||
.images = if (result.tool_result_memory) |memory| memory.tool_images else &.{},
|
||||
}) catch |err| {
|
||||
debug_trace.logf("session", "event=libfx_journal_tool_result_dropped call={s} err={s} answered_by=next_progress", .{ call.id, @errorName(err) });
|
||||
};
|
||||
@@ -2946,6 +3005,7 @@ fn persistAcpHistoryTurn(
|
||||
session.session_rt.commitPreparedHistoryEntry(alloc, prepared);
|
||||
prepared_owned = false;
|
||||
if (current_prompt_input) |prompt_input| prompt_input.retainImageSnapshots();
|
||||
try keepImageWindow(alloc, session);
|
||||
if (session.wasm_state != null) try sessions.commitWasmSessionLocked(alloc, session);
|
||||
return;
|
||||
}
|
||||
@@ -2965,6 +3025,7 @@ fn persistAcpHistoryTurn(
|
||||
session.session_rt.commitPreparedHistoryEntry(alloc, prepared);
|
||||
prepared_owned = false;
|
||||
if (current_prompt_input) |prompt_input| prompt_input.retainImageSnapshots();
|
||||
try keepImageWindow(alloc, session);
|
||||
return;
|
||||
};
|
||||
try writable.prepareHistoryTurnForCommit(alloc, &prepared);
|
||||
@@ -2988,6 +3049,15 @@ fn persistAcpHistoryTurn(
|
||||
if (current_prompt_input) |prompt_input| prompt_input.retainImageSnapshots();
|
||||
}
|
||||
|
||||
/// Keeps a libfx session within its image window once a turn enters its
|
||||
/// history (owned by `alloc`): its images are the newest, and older ones past
|
||||
/// the window give up their bytes. A turn that fails before it is recorded
|
||||
/// drops nothing, so a reopen keeps the same images.
|
||||
fn keepImageWindow(alloc: Allocator, session: *server.ActiveSessionState) Allocator.Error!void {
|
||||
const journal = if (session.journal) |*value| value else return;
|
||||
try journal.dropOutsideWindow(alloc, alloc, session.session_rt.agent.history.items);
|
||||
}
|
||||
|
||||
fn commitContextCompaction(
|
||||
raw_ctx: *anyopaque,
|
||||
summary: types.CompactedSummaryHistoryTurn,
|
||||
@@ -3254,6 +3324,7 @@ fn pushToolLifecycle(raw_ctx: *anyopaque, event: types.ToolLifecycleEvent) !void
|
||||
.id = started.id.call_id,
|
||||
.name = started.tool_name,
|
||||
.arguments_json = started.arguments_json orelse "{}",
|
||||
.resolved_display_target = started.resolved_display_target,
|
||||
});
|
||||
},
|
||||
.terminal => |terminal| {
|
||||
|
||||
+188
-5
@@ -51,6 +51,7 @@ const host_tool_runtime = @import("../core/tooling/host_tool_runtime.zig");
|
||||
const tool_dispatch = @import("../core/tooling/tool_dispatch.zig");
|
||||
const agent_checkpoint = @import("../core/agent/runtime/checkpoint.zig");
|
||||
const host_attachments = @import("../core/hosts/host_attachments.zig");
|
||||
const image_attachments = @import("../core/images/image_attachments.zig");
|
||||
const libfx_steering = @import("libfx_steering.zig");
|
||||
const journal_events = @import("../core/agent/runtime/journal.zig");
|
||||
const libfx_journal = @import("libfx_journal.zig");
|
||||
@@ -87,6 +88,7 @@ const AcpMethod = enum {
|
||||
libfx_follow_up,
|
||||
libfx_snapshot,
|
||||
libfx_journal_open,
|
||||
libfx_journal_images,
|
||||
mcp_message,
|
||||
unknown,
|
||||
|
||||
@@ -112,6 +114,7 @@ const AcpMethod = enum {
|
||||
if (std.mem.eql(u8, method, "libfx/follow_up")) return .libfx_follow_up;
|
||||
if (std.mem.eql(u8, method, "libfx/snapshot")) return .libfx_snapshot;
|
||||
if (std.mem.eql(u8, method, "libfx/journal_open")) return .libfx_journal_open;
|
||||
if (std.mem.eql(u8, method, "libfx/journal_images")) return .libfx_journal_images;
|
||||
if (std.mem.eql(u8, method, "mcp/message")) return .mcp_message;
|
||||
return .unknown;
|
||||
}
|
||||
@@ -140,6 +143,7 @@ const AcpMethod = enum {
|
||||
.libfx_restore,
|
||||
.libfx_snapshot,
|
||||
.libfx_journal_open,
|
||||
.libfx_journal_images,
|
||||
.mcp_message,
|
||||
.unknown,
|
||||
=> true,
|
||||
@@ -148,7 +152,7 @@ const AcpMethod = enum {
|
||||
|
||||
fn isLibfx(self: AcpMethod) bool {
|
||||
return switch (self) {
|
||||
.libfx_checkpoint, .libfx_restore, .libfx_new, .libfx_steer, .libfx_withdraw, .libfx_handoff, .libfx_follow_up, .libfx_snapshot, .libfx_journal_open => true,
|
||||
.libfx_checkpoint, .libfx_restore, .libfx_new, .libfx_steer, .libfx_withdraw, .libfx_handoff, .libfx_follow_up, .libfx_snapshot, .libfx_journal_open, .libfx_journal_images => true,
|
||||
else => false,
|
||||
};
|
||||
}
|
||||
@@ -268,6 +272,13 @@ pub const ActiveSessionState = struct {
|
||||
steering: libfx_steering.Runtime = .{},
|
||||
/// Set when the host passed a journal; every state change is sent to it.
|
||||
journal: ?libfx_journal.Journal = null,
|
||||
/// Images the host staged from its store for the journal it opens next.
|
||||
/// Owned by the server allocator; cleared once the journal opens.
|
||||
staged_images: session_codec.StagedImages = .{},
|
||||
/// What the last open that found images missing needs staged, all told:
|
||||
/// an open turn's tool images can take a session past one window and one
|
||||
/// prompt, and what a running session held must reopen.
|
||||
staging_needed: StagedTotals = .{},
|
||||
|
||||
pub fn retainGrant(self: *ActiveSessionState, alloc: Allocator, tool_name: []const u8, target_path: []const u8) !void {
|
||||
for (self.session_grants) |grant| {
|
||||
@@ -753,6 +764,7 @@ fn destroyActiveSession(state: *ServerState) void {
|
||||
active.tool_identities.deinit(state.alloc);
|
||||
active.steering.deinit(state.alloc);
|
||||
if (active.journal) |*journal| journal.deinit(state.alloc);
|
||||
active.staged_images.deinit(state.alloc);
|
||||
types.freePermissionGrantSlice(state.alloc, active.session_grants);
|
||||
if (comptime !host_target.is_wasm) {
|
||||
if (active.mcp) |runtime| {
|
||||
@@ -1448,6 +1460,7 @@ fn dispatch(state: *ServerState, alloc: Allocator, msg: *jsonrpc.Message) !void
|
||||
.libfx_follow_up => handleKernelFollowUp(state, alloc, msg),
|
||||
.libfx_snapshot => handleKernelSnapshot(state, alloc, msg),
|
||||
.libfx_journal_open => handleKernelJournalOpen(state, alloc, msg),
|
||||
.libfx_journal_images => handleKernelJournalImages(state, alloc, msg),
|
||||
else => state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.method_not_found,
|
||||
.message = "Method not available in the web core yet",
|
||||
@@ -1473,6 +1486,7 @@ fn dispatch(state: *ServerState, alloc: Allocator, msg: *jsonrpc.Message) !void
|
||||
.libfx_follow_up => handleKernelFollowUp(state, alloc, msg),
|
||||
.libfx_snapshot => handleKernelSnapshot(state, alloc, msg),
|
||||
.libfx_journal_open => handleKernelJournalOpen(state, alloc, msg),
|
||||
.libfx_journal_images => handleKernelJournalImages(state, alloc, msg),
|
||||
.initialize,
|
||||
.request_cancel,
|
||||
.session_cancel,
|
||||
@@ -1891,6 +1905,21 @@ fn handleKernelJournalOpen(
|
||||
.message = "Invalid libfx journal params",
|
||||
}),
|
||||
} else false;
|
||||
// A host with an image store stores each prompt image before it sends
|
||||
// it, so the journal names stored images instead of holding their bytes.
|
||||
const image_store = if (parsed.value.object.get("imageStore")) |value| switch (value) {
|
||||
.bool => |enabled| enabled,
|
||||
else => return state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Invalid libfx journal params",
|
||||
}),
|
||||
} else false;
|
||||
const window = imageWindowParam(parsed.value.object.get("imageWindow")) orelse return state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Invalid libfx journal params",
|
||||
});
|
||||
var images: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &active.staged_images };
|
||||
defer images.deinit();
|
||||
const journal, const opened = libfx_journal.open(
|
||||
alloc,
|
||||
state.alloc,
|
||||
@@ -1899,8 +1928,21 @@ fn handleKernelJournalOpen(
|
||||
events orelse "[]",
|
||||
snapshot,
|
||||
inputs_durable,
|
||||
&images,
|
||||
image_store,
|
||||
window,
|
||||
) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
// The host stages them from its store, then opens again.
|
||||
error.ImagesMissing => {
|
||||
var needed: StagedTotals = .{ .images = active.staged_images.images.count(), .bytes = active.staged_images.bytes };
|
||||
for (images.missing.values()) |size| {
|
||||
needed.images +|= 1;
|
||||
needed.bytes +|= size;
|
||||
}
|
||||
active.staging_needed = needed;
|
||||
return writeImagesNeeded(state, alloc, msg.id, images.missing.keys());
|
||||
},
|
||||
error.AgentNotFresh => return state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_request,
|
||||
.message = "A libfx journal opens only on a new session",
|
||||
@@ -1922,6 +1964,9 @@ fn handleKernelJournalOpen(
|
||||
};
|
||||
defer opened.checkpoint_meta.free(alloc);
|
||||
active.journal = journal;
|
||||
// The history holds its own copies of every staged image it named.
|
||||
active.staged_images.clear(state.alloc);
|
||||
active.staging_needed = .{};
|
||||
// The host runs the follow-ups the journal held, after any resume.
|
||||
const follow_ups = active.journal.?.takeFollowUps();
|
||||
defer journal_events.freePendingInputs(state.alloc, follow_ups);
|
||||
@@ -1977,6 +2022,124 @@ fn handleKernelJournalOpen(
|
||||
try state.writer.writeResponse(alloc, msg.id, response.written());
|
||||
}
|
||||
|
||||
/// Answers an open whose saved history names images the host has not
|
||||
/// staged: every such name, so one round of staging serves the next open.
|
||||
fn writeImagesNeeded(state: *ServerState, alloc: Allocator, id: ?jsonrpc.RequestId, names: []const session_codec.ImageName) !void {
|
||||
var response: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer response.deinit();
|
||||
try response.writer.writeAll("{\"imagesNeeded\":[");
|
||||
for (names, 0..) |name, index| {
|
||||
if (index > 0) try response.writer.writeByte(',');
|
||||
try response.writer.print("\"{s}\"", .{&name});
|
||||
}
|
||||
try response.writer.writeAll("]}");
|
||||
try state.writer.writeResponse(alloc, id, response.written());
|
||||
}
|
||||
|
||||
/// The most images, and image bytes, a host may stage for one open: those of
|
||||
/// the largest image window, the newest images a session keeps, and those of
|
||||
/// the open turn, which the window leaves out and which holds one prompt's
|
||||
/// images at most.
|
||||
/// An open that needs more, because its open turn's tools returned images
|
||||
/// beyond one prompt's, may stage what it reported missing
|
||||
/// (`ActiveSessionState.staging_needed`).
|
||||
const max_staged_images = (session_codec.ImageWindow{}).max_images + image_attachments.max_libfx_prompt_images;
|
||||
const max_staged_image_bytes = (session_codec.ImageWindow{}).max_bytes + image_attachments.max_libfx_prompt_images_bytes;
|
||||
|
||||
const StagedTotals = struct { images: usize = 0, bytes: usize = 0 };
|
||||
|
||||
/// The image window an open asks for, `{images, bytes}`, each at most the
|
||||
/// default: a host may keep fewer images, never more than an open stages.
|
||||
/// Null when the value is not one.
|
||||
fn imageWindowParam(value: ?std.json.Value) ?session_codec.ImageWindow {
|
||||
const defaults: session_codec.ImageWindow = .{};
|
||||
const object = switch (value orelse return defaults) {
|
||||
.object => |object| object,
|
||||
else => return null,
|
||||
};
|
||||
var window = defaults;
|
||||
var iterator = object.iterator();
|
||||
while (iterator.next()) |entry| {
|
||||
const limit = switch (entry.value_ptr.*) {
|
||||
.integer => |integer| std.math.cast(usize, integer) orelse return null,
|
||||
else => return null,
|
||||
};
|
||||
if (limit == 0) return null;
|
||||
if (std.mem.eql(u8, entry.key_ptr.*, "images")) {
|
||||
window.max_images = @min(limit, defaults.max_images);
|
||||
} else if (std.mem.eql(u8, entry.key_ptr.*, "bytes")) {
|
||||
window.max_bytes = @min(limit, defaults.max_bytes);
|
||||
} else return null;
|
||||
}
|
||||
return window;
|
||||
}
|
||||
|
||||
/// Stages images from the host's store for the journal the session opens
|
||||
/// next. `images` is `[{name, attachment}]`; each attachment holds the bytes
|
||||
/// whose SHA-256 is the name.
|
||||
fn handleKernelJournalImages(
|
||||
state: *ServerState,
|
||||
alloc: Allocator,
|
||||
msg: *const jsonrpc.Message,
|
||||
) !void {
|
||||
const invalid: jsonrpc.RpcError = .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Invalid libfx journal images",
|
||||
};
|
||||
const too_large: jsonrpc.RpcError = .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "libfx journal images are too large",
|
||||
};
|
||||
var parsed = libfxSessionId(alloc, msg) catch return state.writer.writeError(alloc, msg.id, invalid);
|
||||
defer parsed.deinit();
|
||||
const active = activeLibfxSession(state, parsed.value) orelse
|
||||
return state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "Unknown libfx session",
|
||||
});
|
||||
const store = state.cfg.host_attachments orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
const list = parsed.value.object.get("images") orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
if (list != .array or list.array.items.len == 0) return state.writer.writeError(alloc, msg.id, invalid);
|
||||
|
||||
active.session_write_mutex.lockUncancelable(io_mod.getIo());
|
||||
defer active.session_write_mutex.unlock(io_mod.getIo());
|
||||
if (active.journal != null) return state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_request,
|
||||
.message = "libfx journal is already open",
|
||||
});
|
||||
for (list.array.items) |item| {
|
||||
if (item != .object) return state.writer.writeError(alloc, msg.id, invalid);
|
||||
const name_value = item.object.get("name") orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
if (name_value != .string) return state.writer.writeError(alloc, msg.id, invalid);
|
||||
const name = session_codec.imageName(name_value.string) orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
const reference = item.object.get("attachment") orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
const attachment = host_attachments.idFromJson(reference) orelse return state.writer.writeError(alloc, msg.id, invalid);
|
||||
if (active.staged_images.images.count() >= @max(max_staged_images, active.staging_needed.images)) return state.writer.writeError(alloc, msg.id, too_large);
|
||||
const bytes = store.take(state.alloc, attachment, image_attachments.max_image_bytes) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
error.AttachmentUnavailable => return state.writer.writeError(alloc, msg.id, invalid),
|
||||
error.AttachmentTooLarge => return state.writer.writeError(alloc, msg.id, too_large),
|
||||
};
|
||||
if (bytes.len > @max(max_staged_image_bytes, active.staging_needed.bytes) -| active.staged_images.bytes) {
|
||||
state.alloc.free(bytes);
|
||||
return state.writer.writeError(alloc, msg.id, too_large);
|
||||
}
|
||||
active.staged_images.add(state.alloc, name, bytes) catch |err| {
|
||||
state.alloc.free(bytes);
|
||||
return switch (err) {
|
||||
error.OutOfMemory => error.OutOfMemory,
|
||||
error.ImageNameMismatch => state.writer.writeError(alloc, msg.id, .{
|
||||
.code = ErrorCode.invalid_params,
|
||||
.message = "libfx image does not match its name",
|
||||
}),
|
||||
};
|
||||
};
|
||||
}
|
||||
const written = try alloc.print("{{\"staged\":{d}}}", .{active.staged_images.images.count()});
|
||||
defer alloc.free(written);
|
||||
try state.writer.writeResponse(alloc, msg.id, written);
|
||||
}
|
||||
|
||||
/// A snapshot of the session for its journal to store, when it is quiet (no
|
||||
/// turn open and no follow-up waiting) or at a turn that yielded with nothing
|
||||
/// waiting for it, which the snapshot then carries. `null` otherwise; the
|
||||
@@ -2009,8 +2172,14 @@ fn handleKernelSnapshot(
|
||||
const yielded = journal.yieldedQuiet();
|
||||
if (!journal.quiet() and !yielded) return state.writer.writeResponse(alloc, msg.id, none);
|
||||
// A session past a snapshot's bounds gets none until compaction brings it
|
||||
// back within them; the host learns why.
|
||||
const checkpoint = active.session_rt.agent.checkpoint(alloc, meta) catch |err| switch (err) {
|
||||
// back within them; the host learns why. Images the host stored are
|
||||
// named, not held.
|
||||
const stored = journal.storedImages();
|
||||
const encoded = if (stored) |names|
|
||||
active.session_rt.agent.checkpointNamingImages(alloc, meta, names)
|
||||
else
|
||||
active.session_rt.agent.checkpoint(alloc, meta);
|
||||
const checkpoint = encoded catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
error.CheckpointTooLarge => return writeSnapshotSkipped(state, alloc, msg.id, if (active.session_rt.agent.history.items.len > journal_events.max_history_turns) .too_many_turns else .too_large),
|
||||
else => return writeSnapshotSkipped(state, alloc, msg.id, .invalid),
|
||||
@@ -2019,8 +2188,21 @@ fn handleKernelSnapshot(
|
||||
const at_seq = journal.cursor.next_seq - 1;
|
||||
var open_turn: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer open_turn.deinit();
|
||||
if (yielded) try journal_events.writeOpenTurn(&open_turn.writer, journal.open_turn_id.slice(), journal.pending_resume.?);
|
||||
const bytes = journal_events.encodeSnapshot(alloc, at_seq, journal.cursor.turn, journal.last_turn_id.slice(), if (yielded) open_turn.written() else null, checkpoint) catch |err| switch (err) {
|
||||
var open_turn_names_images = false;
|
||||
if (yielded) {
|
||||
if (stored) |names| {
|
||||
open_turn_names_images = try journal_events.writeOpenTurnNamingImages(&open_turn.writer, journal.open_turn_id.slice(), journal.pending_resume.?, names);
|
||||
} else {
|
||||
try journal_events.writeOpenTurn(&open_turn.writer, journal.open_turn_id.slice(), journal.pending_resume.?);
|
||||
// An image the session dropped is written by name even here.
|
||||
open_turn_names_images = session_codec.checkpointNamesImages(journal.pending_resume.?, null, null);
|
||||
}
|
||||
}
|
||||
const snapshot_bytes = if (open_turn_names_images)
|
||||
journal_events.encodeSnapshotNamingImages(alloc, at_seq, journal.cursor.turn, journal.last_turn_id.slice(), open_turn.written(), checkpoint)
|
||||
else
|
||||
journal_events.encodeSnapshot(alloc, at_seq, journal.cursor.turn, journal.last_turn_id.slice(), if (yielded) open_turn.written() else null, checkpoint);
|
||||
const bytes = snapshot_bytes catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
error.SnapshotTooLarge => return writeSnapshotSkipped(state, alloc, msg.id, .too_large),
|
||||
};
|
||||
@@ -2031,6 +2213,7 @@ fn handleKernelSnapshot(
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
error.AttachmentStoreFull => return state.writer.writeResponse(alloc, msg.id, none),
|
||||
};
|
||||
if (yielded) journal.noteOpenTurnSnapshot();
|
||||
const written = try alloc.print("{{\"snapshotAttachment\":{d},\"atSeq\":{d}}}", .{ attachment, at_seq });
|
||||
defer alloc.free(written);
|
||||
try state.writer.writeResponse(alloc, msg.id, written);
|
||||
|
||||
+200
-23
@@ -26,6 +26,7 @@ const session_runtime = @import("../core/session/session.zig");
|
||||
const agent_execution_memory = @import("../core/agent/execution_memory.zig");
|
||||
const tool_dispatch = @import("../core/tooling/tool_dispatch.zig");
|
||||
const tool_call_presentation = @import("tool_call_presentation.zig");
|
||||
const runtime_tool_presentation = @import("../core/agent/runtime/tool_presentation.zig");
|
||||
const mcp_runtime = @import("../core/mcp/mcp_runtime.zig");
|
||||
const mcp_host = @import("../core/mcp_host/host.zig");
|
||||
const mcp_contract = @import("../core/mcp/mcp_contract.zig");
|
||||
@@ -729,7 +730,9 @@ pub fn handleLoadWasmSession(state: *server.ServerState, alloc: Allocator, msg:
|
||||
|
||||
if (state.active_session) |*active| {
|
||||
if (sameSessionId(active.session_id, session_id)) {
|
||||
for (active.session_rt.agent.history.items) |turn| try sendHistoryTurnAsUpdates(state, alloc, session_id, turn);
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
for (active.session_rt.agent.history.items) |turn| try sendHistoryTurnAsUpdates(state, alloc, session_id, &sources, turn);
|
||||
try sendActiveSessionInfoUpdate(state, alloc);
|
||||
return writeLoadSessionResponse(state, alloc, msg, active.model, observer_hooks);
|
||||
}
|
||||
@@ -792,7 +795,9 @@ pub fn handleLoadWasmSession(state: *server.ServerState, alloc: Allocator, msg:
|
||||
sid_owned = false;
|
||||
model_owned = false;
|
||||
session_rt_owned = false;
|
||||
for (state.active_session.?.session_rt.agent.history.items) |turn| try sendHistoryTurnAsUpdates(state, alloc, session_id, turn);
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
for (state.active_session.?.session_rt.agent.history.items) |turn| try sendHistoryTurnAsUpdates(state, alloc, session_id, &sources, turn);
|
||||
try sendActiveSessionInfoUpdate(state, alloc);
|
||||
try writeLoadSessionResponse(state, alloc, msg, state.active_session.?.model, observer_hooks);
|
||||
}
|
||||
@@ -972,13 +977,17 @@ fn handleRestoreSession(
|
||||
}
|
||||
if (previous_host) |h| h.destroy();
|
||||
server.enableSubagentHost(state);
|
||||
// One map, so a recovered read can name a call from an earlier turn.
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
if (kind.replaysHistory()) {
|
||||
try sendActiveHistoryUpdates(state, alloc, session_id);
|
||||
try sendActiveHistoryUpdates(state, alloc, session_id, &sources);
|
||||
}
|
||||
try sendPendingRecoveryUpdate(
|
||||
state,
|
||||
alloc,
|
||||
session_id,
|
||||
&sources,
|
||||
if (active.writable) |*writable| writable.state.recovery_checkpoint else null,
|
||||
);
|
||||
try sendActiveSessionInfoUpdate(state, alloc);
|
||||
@@ -1138,13 +1147,17 @@ fn handleRestoreSession(
|
||||
session_rt_owned = false;
|
||||
session_mcp_owned = false;
|
||||
session_host_owned = false;
|
||||
// One map, so a recovered read can name a call from an earlier turn.
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
if (kind.replaysHistory()) {
|
||||
try sendActiveHistoryUpdates(state, alloc, session_id);
|
||||
try sendActiveHistoryUpdates(state, alloc, session_id, &sources);
|
||||
}
|
||||
try sendPendingRecoveryUpdate(
|
||||
state,
|
||||
alloc,
|
||||
session_id,
|
||||
&sources,
|
||||
if (state.active_session.?.writable) |*value| value.state.recovery_checkpoint else null,
|
||||
);
|
||||
try sendActiveSessionInfoUpdate(state, alloc);
|
||||
@@ -1291,11 +1304,12 @@ fn sendPendingRecoveryUpdate(
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
checkpoint: ?session_codec.RecoveryCheckpoint,
|
||||
) !void {
|
||||
const recovery = checkpoint orelse return;
|
||||
try sendUserHistoryTurn(state, alloc, session_id, recovery.user);
|
||||
try sendExecutionHistory(state, alloc, session_id, recovery.execution);
|
||||
try sendExecutionHistory(state, alloc, session_id, sources, recovery.execution);
|
||||
if (recovery.assistant_source.len > 0) {
|
||||
try sendAgentHistoryChunk(state, alloc, session_id, recovery.assistant_source);
|
||||
}
|
||||
@@ -1880,19 +1894,25 @@ fn parseListCursor(raw: []const u8) !session_store.ResumableSessionContinuation
|
||||
return .{ .updated_at_ms = updated_at_ms, .id = id };
|
||||
}
|
||||
|
||||
fn sendActiveHistoryUpdates(state: *server.ServerState, alloc: Allocator, session_id: []const u8) !void {
|
||||
fn sendActiveHistoryUpdates(
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
) !void {
|
||||
const active = &state.active_session.?;
|
||||
if (active.store) |store| {
|
||||
const Visitor = struct {
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
|
||||
pub fn append(self: *@This(), turn: types.HistoryTurn) !void {
|
||||
try sendHistoryTurnAsUpdates(self.state, self.alloc, self.session_id, turn);
|
||||
try sendHistoryTurnAsUpdates(self.state, self.alloc, self.session_id, self.sources, turn);
|
||||
}
|
||||
};
|
||||
var visitor = Visitor{ .state = state, .alloc = alloc, .session_id = session_id };
|
||||
var visitor = Visitor{ .state = state, .alloc = alloc, .session_id = session_id, .sources = sources };
|
||||
return store.visitConversationHistory(alloc, session_id, &visitor);
|
||||
}
|
||||
// Every saved turn, as v1 replays it, not only those in memory.
|
||||
@@ -1901,20 +1921,27 @@ fn sendActiveHistoryUpdates(state: *server.ServerState, alloc: Allocator, sessio
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
|
||||
pub fn append(self: *@This(), turn: types.HistoryTurn) !void {
|
||||
try sendHistoryTurnAsUpdates(self.state, self.alloc, self.session_id, turn);
|
||||
try sendHistoryTurnAsUpdates(self.state, self.alloc, self.session_id, self.sources, turn);
|
||||
}
|
||||
};
|
||||
var visitor = Visitor{ .state = state, .alloc = alloc, .session_id = session_id };
|
||||
var visitor = Visitor{ .state = state, .alloc = alloc, .session_id = session_id, .sources = sources };
|
||||
return v2.visitHistory(alloc, &visitor);
|
||||
}
|
||||
for (active.session_rt.agent.history.items) |turn| {
|
||||
try sendHistoryTurnAsUpdates(state, alloc, session_id, turn);
|
||||
try sendHistoryTurnAsUpdates(state, alloc, session_id, sources, turn);
|
||||
}
|
||||
}
|
||||
|
||||
fn sendHistoryTurnAsUpdates(state: *server.ServerState, alloc: Allocator, session_id: []const u8, turn: types.HistoryTurn) !void {
|
||||
fn sendHistoryTurnAsUpdates(
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
turn: types.HistoryTurn,
|
||||
) !void {
|
||||
switch (turn) {
|
||||
.assistant => |assistant| try sendUserHistoryTurn(state, alloc, session_id, assistant.user),
|
||||
.interrupted => |interrupted| try sendUserHistoryTurn(state, alloc, session_id, interrupted.user),
|
||||
@@ -1923,11 +1950,11 @@ fn sendHistoryTurnAsUpdates(state: *server.ServerState, alloc: Allocator, sessio
|
||||
|
||||
switch (turn) {
|
||||
.assistant => |assistant| {
|
||||
try sendExecutionHistory(state, alloc, session_id, assistant.execution);
|
||||
try sendExecutionHistory(state, alloc, session_id, sources, assistant.execution);
|
||||
try sendAgentHistoryChunk(state, alloc, session_id, assistant.assistant);
|
||||
},
|
||||
.interrupted => |i| {
|
||||
try sendExecutionHistory(state, alloc, session_id, i.execution);
|
||||
try sendExecutionHistory(state, alloc, session_id, sources, i.execution);
|
||||
if (i.assistant) |assistant| {
|
||||
if (assistant.len > 0) try sendAgentHistoryChunk(state, alloc, session_id, assistant);
|
||||
}
|
||||
@@ -2012,6 +2039,7 @@ fn sendExecutionHistory(
|
||||
state: *server.ServerState,
|
||||
alloc: Allocator,
|
||||
session_id: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
execution: types.ExecutionMemory,
|
||||
) !void {
|
||||
if (execution.isEmpty()) return;
|
||||
@@ -2022,6 +2050,11 @@ fn sendExecutionHistory(
|
||||
tool_call_presentation.activeToolRegistry(state),
|
||||
activeSessionMcp(state),
|
||||
if (state.active_session) |*active| &active.tool_identities else null,
|
||||
.{
|
||||
.alloc = alloc,
|
||||
.workspace_root = if (state.active_session) |*active| active.workspace_root else state.workspace_root,
|
||||
.sources = sources,
|
||||
},
|
||||
execution,
|
||||
);
|
||||
for (frames) |frame| {
|
||||
@@ -2066,14 +2099,37 @@ fn findToolResultIndex(results: []const types.PersistedToolResult, call_id: []co
|
||||
return null;
|
||||
}
|
||||
|
||||
/// Calls whose saved output later replayed reads name, kept across the turns
|
||||
/// of one replay.
|
||||
const ReplaySources = struct {
|
||||
alloc: Allocator,
|
||||
workspace_root: []const u8,
|
||||
sources: *runtime_tool_presentation.ReplayedSavedOutputSources,
|
||||
};
|
||||
|
||||
fn planToolCallFrame(
|
||||
arena: Allocator,
|
||||
registry: tool_dispatch.Registry,
|
||||
mcp: ?*mcp_runtime.McpRuntime,
|
||||
identities: ?*tool_call_identities.Record,
|
||||
call: types.ToolCall,
|
||||
replay_sources: ReplaySources,
|
||||
replayed_call: types.ToolCall,
|
||||
result: ?types.PersistedToolResult,
|
||||
) !ReplayToolCall {
|
||||
var call = replayed_call;
|
||||
call.resolved_display_target = try runtime_tool_presentation.readToolResultDisplayTarget(
|
||||
arena,
|
||||
registry,
|
||||
call,
|
||||
replay_sources.sources,
|
||||
);
|
||||
if (result) |saved| try replay_sources.sources.record(
|
||||
replay_sources.alloc,
|
||||
registry,
|
||||
replay_sources.workspace_root,
|
||||
call,
|
||||
saved,
|
||||
);
|
||||
const name = tool_call_presentation.acpToolName(call.name);
|
||||
// MCP identity comes from the live catalog, or from the identity recorded
|
||||
// when the session made the call; it is never guessed from the exposed
|
||||
@@ -2136,6 +2192,7 @@ fn planExecutionReplay(
|
||||
registry: tool_dispatch.Registry,
|
||||
mcp: ?*mcp_runtime.McpRuntime,
|
||||
identities: ?*tool_call_identities.Record,
|
||||
replay_sources: ReplaySources,
|
||||
execution: types.ExecutionMemory,
|
||||
) ![]ReplayFrame {
|
||||
var frames: std.ArrayList(ReplayFrame) = .empty;
|
||||
@@ -2151,7 +2208,7 @@ fn planExecutionReplay(
|
||||
matched[i] = true;
|
||||
break :blk step.tool_results[i];
|
||||
} else null;
|
||||
try frames.append(arena, .{ .tool_call = try planToolCallFrame(arena, registry, mcp, identities, call, result) });
|
||||
try frames.append(arena, .{ .tool_call = try planToolCallFrame(arena, registry, mcp, identities, replay_sources, call, result) });
|
||||
}
|
||||
// Results without a surviving call record (e.g. trimmed history) still
|
||||
// replay as completed frames so the transcript stays faithful.
|
||||
@@ -2162,7 +2219,7 @@ fn planExecutionReplay(
|
||||
.name = result.tool_name,
|
||||
.arguments_json = "{}",
|
||||
};
|
||||
try frames.append(arena, .{ .tool_call = try planToolCallFrame(arena, registry, mcp, identities, call, result) });
|
||||
try frames.append(arena, .{ .tool_call = try planToolCallFrame(arena, registry, mcp, identities, replay_sources, call, result) });
|
||||
}
|
||||
steering_index = try appendSteeringReplay(arena, &frames, execution.steering, steering_index, step_index + 1);
|
||||
}
|
||||
@@ -2701,7 +2758,7 @@ test "ACP history excludes typed summaries without filtering original user text"
|
||||
var state = try initAcpSessionTestState(arena, workspace, capture);
|
||||
defer state.deinit();
|
||||
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .compacted_summary = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .compacted_summary = .{
|
||||
.summary = @constCast("internal summary"),
|
||||
.removed_turn_count = 1,
|
||||
.compaction_count = 1,
|
||||
@@ -2709,7 +2766,7 @@ test "ACP history excludes typed summaries without filtering original user text"
|
||||
try std.testing.expectEqual(@as(u64, 0), try capture.length(io_mod.getIo()));
|
||||
|
||||
const original = "Explain <context_handoff> without hiding my question.";
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .assistant = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .assistant = .{
|
||||
.user = .{ .text = @constCast(original) },
|
||||
.assistant = @constCast("original reply"),
|
||||
} });
|
||||
@@ -2743,7 +2800,7 @@ test "ACP interrupted history replay hides model-only abort context" {
|
||||
var state = try initAcpSessionTestState(arena, workspace, capture);
|
||||
defer state.deinit();
|
||||
var completed_tool_names = [_][]u8{@constCast("read_file")};
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .interrupted = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .interrupted = .{
|
||||
.user = .{ .text = @constCast("inspect the project") },
|
||||
.completed_tool_names = completed_tool_names[0..],
|
||||
} });
|
||||
@@ -2809,7 +2866,7 @@ test "ACP history replay emits structured tool call frames" {
|
||||
.tool_calls = calls[0..],
|
||||
.tool_results = results[0..],
|
||||
}};
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .assistant = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .assistant = .{
|
||||
.user = .{ .text = @constCast("read and write") },
|
||||
.assistant = @constCast("All done."),
|
||||
.execution = .{ .tool_steps = steps[0..] },
|
||||
@@ -2857,7 +2914,7 @@ test "ACP history replay leaves resultless tool calls pending" {
|
||||
.{ .id = "call_orphan", .name = "read_file", .arguments_json = "{\"path\":\"a.txt\"}" },
|
||||
};
|
||||
var steps = [_]types.ToolExecutionStep{.{ .tool_calls = calls[0..] }};
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .interrupted = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .interrupted = .{
|
||||
.user = .{ .text = @constCast("inspect") },
|
||||
.execution = .{ .tool_steps = steps[0..] },
|
||||
} });
|
||||
@@ -2895,7 +2952,7 @@ test "ACP history replay redacts sensitive tool arguments" {
|
||||
.stored_output_bytes = 2,
|
||||
}};
|
||||
var steps = [_]types.ToolExecutionStep{.{ .tool_calls = calls[0..], .tool_results = results[0..] }};
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", .{ .assistant = .{
|
||||
try sendTestHistoryTurn(&state, arena, "session-1", .{ .assistant = .{
|
||||
.user = .{ .text = @constCast("run it") },
|
||||
.assistant = @constCast("done"),
|
||||
.execution = .{ .tool_steps = steps[0..] },
|
||||
@@ -2940,11 +2997,14 @@ test "execution replay plan interleaves assistant text and orphan results" {
|
||||
.tool_results = results[0..],
|
||||
}};
|
||||
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(arena);
|
||||
const frames = try planExecutionReplay(
|
||||
arena,
|
||||
@import("../builtins/tools.zig").registry,
|
||||
null,
|
||||
null,
|
||||
.{ .alloc = arena, .workspace_root = "/workspace", .sources = &sources },
|
||||
.{ .tool_steps = steps[0..] },
|
||||
);
|
||||
try std.testing.expectEqual(@as(usize, 3), frames.len);
|
||||
@@ -2958,6 +3018,123 @@ test "execution replay plan interleaves assistant text and orphan results" {
|
||||
try std.testing.expect(frames[2].tool_call.raw_input != null);
|
||||
}
|
||||
|
||||
fn sendTestHistoryTurn(state: *server.ServerState, alloc: Allocator, session_id: []const u8, turn: types.HistoryTurn) !void {
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
return sendHistoryTurnAsUpdates(state, alloc, session_id, &sources, turn);
|
||||
}
|
||||
|
||||
test "execution replay titles reads with the call whose saved output they read" {
|
||||
const alloc = std.testing.allocator;
|
||||
var arena_state = std.heap.ArenaAllocator.init(alloc);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
var run_calls = [_]types.ToolCall{
|
||||
.{ .id = "run", .name = "shell", .arguments_json = "{\"action\":\"run\",\"command\":\"zig build test\"}" },
|
||||
};
|
||||
var run_results = [_]types.PersistedToolResult{.{
|
||||
.tool_call_id = @constCast("run"),
|
||||
.tool_name = @constCast("shell"),
|
||||
.status = .success,
|
||||
.output = @constCast("{\"exit_code\":0}"),
|
||||
.output_bytes = 15,
|
||||
.stored_output_bytes = 15,
|
||||
.command_output_replay = .{ .available = .{ .handle = replay, .framed_bytes = 64 } },
|
||||
}};
|
||||
var read_calls = [_]types.ToolCall{
|
||||
.{ .id = "read", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}" },
|
||||
};
|
||||
var read_results = [_]types.PersistedToolResult{.{
|
||||
.tool_call_id = @constCast("read"),
|
||||
.tool_name = @constCast("read_tool_result"),
|
||||
.status = .success,
|
||||
.output = @constCast("error: x"),
|
||||
.output_bytes = 8,
|
||||
.stored_output_bytes = 8,
|
||||
}};
|
||||
var steps = [_]types.ToolExecutionStep{
|
||||
.{ .tool_calls = run_calls[0..], .tool_results = run_results[0..] },
|
||||
.{ .tool_calls = read_calls[0..], .tool_results = read_results[0..] },
|
||||
};
|
||||
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(alloc);
|
||||
const registry = @import("../builtins/tools.zig").registry;
|
||||
const replay_sources: ReplaySources = .{ .alloc = alloc, .workspace_root = "/workspace", .sources = &sources };
|
||||
const frames = try planExecutionReplay(arena, registry, null, null, replay_sources, .{ .tool_steps = steps[0..] });
|
||||
try std.testing.expectEqual(@as(usize, 2), frames.len);
|
||||
try std.testing.expectEqualStrings("Searching zig build test output for \"error:\"", frames[1].tool_call.title);
|
||||
|
||||
// A later turn replayed with the same sources still names the command.
|
||||
var later_calls = [_]types.ToolCall{
|
||||
.{ .id = "later", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\"}}" },
|
||||
};
|
||||
var later_steps = [_]types.ToolExecutionStep{.{ .tool_calls = later_calls[0..] }};
|
||||
const later = try planExecutionReplay(arena, registry, null, null, replay_sources, .{ .tool_steps = later_steps[0..] });
|
||||
try std.testing.expectEqualStrings("Reading more output from zig build test", later[0].tool_call.title);
|
||||
}
|
||||
|
||||
test "ACP load titles a recovered read with the call from an earlier turn" {
|
||||
const alloc = std.testing.allocator;
|
||||
var tmp = std.testing.tmpDir(.{});
|
||||
defer tmp.cleanup();
|
||||
var arena_state = std.heap.ArenaAllocator.init(alloc);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const workspace = try io_mod.dirRealpathAlloc(alloc, tmp.dir, ".");
|
||||
defer alloc.free(workspace);
|
||||
var capture = try tmp.dir.createFile(io_mod.getIo(), "history.jsonl", .{ .read = true });
|
||||
defer capture.close(io_mod.getIo());
|
||||
var state = try initAcpSessionTestState(arena, workspace, capture);
|
||||
defer state.deinit();
|
||||
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
var run_calls = [_]types.ToolCall{
|
||||
.{ .id = "run", .name = "shell", .arguments_json = "{\"action\":\"run\",\"command\":\"zig build test\"}" },
|
||||
};
|
||||
var run_results = [_]types.PersistedToolResult{.{
|
||||
.tool_call_id = @constCast("run"),
|
||||
.tool_name = @constCast("shell"),
|
||||
.status = .success,
|
||||
.output = @constCast("{\"exit_code\":0}"),
|
||||
.output_bytes = 15,
|
||||
.stored_output_bytes = 15,
|
||||
.command_output_replay = .{ .available = .{ .handle = replay, .framed_bytes = 64 } },
|
||||
}};
|
||||
var run_steps = [_]types.ToolExecutionStep{.{ .tool_calls = run_calls[0..], .tool_results = run_results[0..] }};
|
||||
var read_calls = [_]types.ToolCall{
|
||||
.{ .id = "read", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}" },
|
||||
};
|
||||
var read_steps = [_]types.ToolExecutionStep{.{ .tool_calls = read_calls[0..] }};
|
||||
|
||||
var sources: runtime_tool_presentation.ReplayedSavedOutputSources = .{};
|
||||
defer sources.deinit(arena);
|
||||
try sendHistoryTurnAsUpdates(&state, arena, "session-1", &sources, .{ .assistant = .{
|
||||
.user = .{ .text = @constCast("run the tests") },
|
||||
.assistant = @constCast("done"),
|
||||
.execution = .{ .tool_steps = run_steps[0..] },
|
||||
} });
|
||||
try sendPendingRecoveryUpdate(&state, arena, "session-1", &sources, .{
|
||||
.turn_id = 2,
|
||||
.user = .{ .text = @constCast("find the failure") },
|
||||
.assistant_source = @constCast(""),
|
||||
.execution = .{ .tool_steps = read_steps[0..] },
|
||||
.cause = .response_interrupted,
|
||||
.action = .paused,
|
||||
.authority = .{ .provider = .gateway, .model = @constCast("test-model") },
|
||||
.requested_fast_mode = false,
|
||||
.fast_mode = false,
|
||||
.max_provider_attempts = 1,
|
||||
.consumed_provider_attempts = 1,
|
||||
});
|
||||
|
||||
const captured = try readCaptured(&capture, alloc);
|
||||
defer alloc.free(captured);
|
||||
try std.testing.expect(std.mem.find(u8, captured, "Searching zig build test output for \\\"error:\\\"") != null);
|
||||
}
|
||||
|
||||
test "ACP load maps one-off child denial to invalid params" {
|
||||
const alloc = std.testing.allocator;
|
||||
var tmp = std.testing.tmpDir(.{});
|
||||
|
||||
@@ -38,8 +38,13 @@ pub fn mapToolKind(tool_name: []const u8) acp_types.ToolCallKind {
|
||||
}
|
||||
|
||||
pub fn describeToolTitle(registry: tool_dispatch.Registry, arena: Allocator, call: ToolCall) ![]const u8 {
|
||||
if (registry.lookup(call.name) != null) {
|
||||
if (registry.lookup(call.name)) |tool| {
|
||||
if (try tool_presentation.formatSubagentPlainAction(arena, call, .identity)) |title| return title;
|
||||
// A read's input is an opaque handle, so its verb alone says nothing
|
||||
// about what it reads.
|
||||
if (tool.executor_kind == .read_tool_result) {
|
||||
return tool_presentation.formatPlainAction(arena, .{ .tool_registry = registry, .call = call });
|
||||
}
|
||||
}
|
||||
if (tool_presentation.isProviderSearchAlias(call.name)) {
|
||||
return tool_presentation.formatPlainAction(arena, .{
|
||||
@@ -161,6 +166,25 @@ test "describeToolCall flags internal discovery and names MCP identity" {
|
||||
try std.testing.expect(!titled.meta.internal);
|
||||
}
|
||||
|
||||
test "read titles say what they read while other tools keep their verb" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const registry = builtin_tools.registry;
|
||||
const read: ToolCall = .{
|
||||
.id = "read",
|
||||
.name = "read_tool_result",
|
||||
.arguments_json = "{\"request\":{\"handle\":\"fx-command-replay-0123.bin\",\"query\":\"error:\"}}",
|
||||
.resolved_display_target = "zig build test output for \"error:\"",
|
||||
};
|
||||
try std.testing.expectEqualStrings("Searching zig build test output for \"error:\"", describeToolCall(registry, arena, read, null).title);
|
||||
var unresolved = read;
|
||||
unresolved.resolved_display_target = null;
|
||||
try std.testing.expectEqualStrings("Searching command output", describeToolCall(registry, arena, unresolved, null).title);
|
||||
const file: ToolCall = .{ .id = "file", .name = "read_file", .arguments_json = "{\"path\":\"a.txt\"}" };
|
||||
try std.testing.expectEqualStrings("Reading", describeToolCall(registry, arena, file, null).title);
|
||||
}
|
||||
|
||||
test "toolUpdateContentText clips long output and guards unsafe bytes" {
|
||||
const long = text_utils.repeat("x", 500);
|
||||
const clipped = toolUpdateContentText(false, long);
|
||||
|
||||
@@ -666,7 +666,7 @@ pub fn loadHost(
|
||||
}
|
||||
const h = try mcp_host.Host.create(alloc, io_mod.getIo(), loaded.configs.items, &inherited, .{
|
||||
.client_version = build_options.app_version,
|
||||
.credentials = .detect(alloc, home),
|
||||
.credentials = .{ .detect = home },
|
||||
.questions = extras.questions,
|
||||
.notify = extras.notify,
|
||||
}, builtins);
|
||||
|
||||
@@ -830,8 +830,8 @@ pub const read_tool_result = ToolSpec{
|
||||
.requires_approval = false,
|
||||
.action_label = "Reading",
|
||||
.completed_action_label = "Read",
|
||||
.label_arg_kind = .path,
|
||||
.label_arg_default = "tool result",
|
||||
.label_arg_default = "saved output",
|
||||
.presentation_fn = read_tool_result_impl.presentation,
|
||||
.permission_target_kind = .none,
|
||||
.decode = read_tool_result_impl.decode,
|
||||
.validate = read_tool_result_impl.validate,
|
||||
@@ -1707,8 +1707,9 @@ test "built-in read_tool_result owns product metadata schema and callbacks" {
|
||||
try std.testing.expect(!read_tool_result.requires_approval);
|
||||
try std.testing.expectEqualStrings("Reading", read_tool_result.action_label);
|
||||
try std.testing.expectEqualStrings("Read", read_tool_result.completed_action_label);
|
||||
try std.testing.expectEqual(tool_dispatch.LabelArgKind.path, read_tool_result.label_arg_kind);
|
||||
try std.testing.expectEqualStrings("tool result", read_tool_result.label_arg_default);
|
||||
try std.testing.expectEqual(tool_dispatch.LabelArgKind.none, read_tool_result.label_arg_kind);
|
||||
try std.testing.expectEqualStrings("saved output", read_tool_result.label_arg_default);
|
||||
try std.testing.expect(read_tool_result.presentation_fn.? == read_tool_result_impl.presentation);
|
||||
try std.testing.expectEqual(tool_dispatch.PermissionTargetKind.none, read_tool_result.permission_target_kind);
|
||||
try std.testing.expect(read_tool_result.decode == read_tool_result_impl.decode);
|
||||
try std.testing.expect(read_tool_result.validate.? == read_tool_result_impl.validate);
|
||||
|
||||
@@ -2,6 +2,7 @@ const std = @import("std");
|
||||
const testing_allocator = @import("../../shared/testing_allocator.zig");
|
||||
const types = @import("../../shared/types.zig");
|
||||
const checkpoint_codec = @import("checkpoint.zig");
|
||||
const session_codec = @import("../../session/session_codec.zig");
|
||||
const runtime_prompt_context = @import("prompt_context.zig");
|
||||
const debug_trace = @import("../../shared/debug_trace.zig");
|
||||
const text_utils = @import("../../shared/text_utils.zig");
|
||||
@@ -44,6 +45,17 @@ pub const Agent = struct {
|
||||
return checkpoint_codec.encode(alloc, self.history.items, self.turn_usage, meta);
|
||||
}
|
||||
|
||||
/// Like `checkpoint`, but names the images a libfx host's store holds
|
||||
/// instead of holding their bytes.
|
||||
pub fn checkpointNamingImages(
|
||||
self: *const Agent,
|
||||
alloc: Allocator,
|
||||
meta: checkpoint_codec.Meta,
|
||||
stored: *const session_codec.StoredImages,
|
||||
) checkpoint_codec.Error![]u8 {
|
||||
return checkpoint_codec.encodeNamingImages(alloc, self.history.items, self.turn_usage, meta, stored);
|
||||
}
|
||||
|
||||
/// Restores `bytes` into this fresh agent. Returns what the checkpoint
|
||||
/// recorded about the agent that saved it, owned by `alloc`
|
||||
/// (`Meta.free`).
|
||||
|
||||
@@ -11,6 +11,12 @@ const magic = "FXCP";
|
||||
/// Its little-endian payload is:
|
||||
/// u32 json_len | json | u32 blob_count | (u32 blob_len | blob)*
|
||||
const version: u16 = 2;
|
||||
/// Version 3 has version 2's layout, and its JSON may name images instead of
|
||||
/// holding their bytes: ones the host stored (`inline_ref`), or ones the
|
||||
/// session dropped. Only a checkpoint that names one is version 3, so an fx
|
||||
/// that reads only version 2 reports a newer checkpoint instead of an
|
||||
/// invalid one.
|
||||
const naming_version: u16 = 3;
|
||||
/// Version 1 embedded inline image bytes as base64 inside a JSON-only payload.
|
||||
const json_only_version: u16 = 1;
|
||||
const header_bytes: usize = 4 + 2 + 2 + 4 + Sha256.digest_length;
|
||||
@@ -89,12 +95,34 @@ pub const Decoded = struct {
|
||||
};
|
||||
|
||||
/// `meta` is recorded when any of its fields is set; each must fit
|
||||
/// `max_meta_field_bytes`.
|
||||
/// `max_meta_field_bytes`. The checkpoint holds every image's bytes.
|
||||
pub fn encode(
|
||||
alloc: Allocator,
|
||||
history: []const types.HistoryTurn,
|
||||
usage: types.Usage,
|
||||
meta: Meta,
|
||||
) Error![]u8 {
|
||||
return encodeWith(alloc, history, usage, meta, null);
|
||||
}
|
||||
|
||||
/// Like `encode`, but names the images `stored` holds instead of holding
|
||||
/// their bytes.
|
||||
pub fn encodeNamingImages(
|
||||
alloc: Allocator,
|
||||
history: []const types.HistoryTurn,
|
||||
usage: types.Usage,
|
||||
meta: Meta,
|
||||
stored: *const session_codec.StoredImages,
|
||||
) Error![]u8 {
|
||||
return encodeWith(alloc, history, usage, meta, stored);
|
||||
}
|
||||
|
||||
fn encodeWith(
|
||||
alloc: Allocator,
|
||||
history: []const types.HistoryTurn,
|
||||
usage: types.Usage,
|
||||
meta: Meta,
|
||||
stored: ?*const session_codec.StoredImages,
|
||||
) Error![]u8 {
|
||||
std.debug.assert(meta.fits());
|
||||
if (history.len > max_history_turns) return error.CheckpointTooLarge;
|
||||
@@ -105,12 +133,18 @@ pub fn encode(
|
||||
defer blobs.deinit(alloc);
|
||||
const image_blobs: session_codec.ImageBlobs = .{ .alloc = alloc, .items = &blobs };
|
||||
var blob_section_bytes: usize = 0;
|
||||
var names_images = false;
|
||||
|
||||
json.writer.writeAll("{\"history\":[") catch return error.OutOfMemory;
|
||||
for (history, 0..) |turn, index| {
|
||||
if (index > 0) json.writer.writeByte(',') catch return error.OutOfMemory;
|
||||
const first_new_blob = blobs.items.len;
|
||||
session_codec.writeHistoryTurnWithImageBlobs(&json.writer, turn, image_blobs) catch |err| switch (err) {
|
||||
names_images = names_images or session_codec.turnNamesImages(turn, stored);
|
||||
const written = if (stored) |names|
|
||||
session_codec.writeHistoryTurnNamingImages(&json.writer, turn, .{ .stored = names, .blobs = image_blobs })
|
||||
else
|
||||
session_codec.writeHistoryTurnWithImageBlobs(&json.writer, turn, image_blobs);
|
||||
written catch |err| switch (err) {
|
||||
error.InvalidSessionFormat => return error.InvalidCheckpoint,
|
||||
else => return error.OutOfMemory,
|
||||
};
|
||||
@@ -138,7 +172,7 @@ pub fn encode(
|
||||
std.debug.assert(cursor == out.len);
|
||||
|
||||
@memcpy(out[0..magic.len], magic);
|
||||
std.mem.writeInt(u16, out[4..6], version, .little);
|
||||
std.mem.writeInt(u16, out[4..6], if (names_images) naming_version else version, .little);
|
||||
std.mem.writeInt(u16, out[6..8], 0, .little);
|
||||
std.mem.writeInt(u32, out[8..12], @intCast(payload_len), .little);
|
||||
Sha256.hash(out[header_bytes..], out[12..header_bytes], .{});
|
||||
@@ -161,12 +195,19 @@ fn writeSection(out: []u8, cursor: *usize, bytes: []const u8) void {
|
||||
}
|
||||
|
||||
pub fn decode(alloc: Allocator, bytes: []const u8) Error!Decoded {
|
||||
return decodeWithImages(alloc, bytes, null);
|
||||
}
|
||||
|
||||
/// Like `decode`, but resolves named images through `names`. Without it a
|
||||
/// checkpoint that names images is invalid. A name `names` cannot resolve is
|
||||
/// recorded there, and the decoded images it names have no bytes.
|
||||
pub fn decodeWithImages(alloc: Allocator, bytes: []const u8, names: ?*session_codec.ImageResolver) Error!Decoded {
|
||||
if (bytes.len < header_bytes or bytes.len > max_checkpoint_bytes) {
|
||||
return error.CorruptCheckpoint;
|
||||
}
|
||||
if (!std.mem.eql(u8, bytes[0..magic.len], magic)) return error.CorruptCheckpoint;
|
||||
const checkpoint_version = std.mem.readInt(u16, bytes[4..6], .little);
|
||||
if (checkpoint_version != version and checkpoint_version != json_only_version) {
|
||||
if (checkpoint_version != version and checkpoint_version != json_only_version and checkpoint_version != naming_version) {
|
||||
return error.UnsupportedCheckpointVersion;
|
||||
}
|
||||
if (std.mem.readInt(u16, bytes[6..8], .little) != 0) {
|
||||
@@ -180,7 +221,7 @@ pub fn decode(alloc: Allocator, bytes: []const u8) Error!Decoded {
|
||||
return error.CorruptCheckpoint;
|
||||
}
|
||||
const payload = bytes[header_bytes..];
|
||||
if (checkpoint_version == json_only_version) return decodeJson(alloc, payload, null);
|
||||
if (checkpoint_version == json_only_version) return decodeJson(alloc, payload, .{});
|
||||
|
||||
var reader: PayloadReader = .{ .bytes = payload };
|
||||
const json = reader.section() orelse return error.CorruptCheckpoint;
|
||||
@@ -192,7 +233,11 @@ pub fn decode(alloc: Allocator, bytes: []const u8) Error!Decoded {
|
||||
for (blobs) |*blob| blob.* = reader.section() orelse return error.CorruptCheckpoint;
|
||||
if (reader.remaining() != 0) return error.CorruptCheckpoint;
|
||||
var blob_reader: session_codec.ImageBlobReader = .{ .blobs = blobs };
|
||||
return decodeJson(alloc, json, &blob_reader);
|
||||
// Only a version 3 checkpoint names images.
|
||||
return decodeJson(alloc, json, .{
|
||||
.blobs = &blob_reader,
|
||||
.names = if (checkpoint_version == naming_version) names else null,
|
||||
});
|
||||
}
|
||||
|
||||
/// Reads length-prefixed sections from a version 2 payload. Returned slices
|
||||
@@ -221,7 +266,7 @@ const PayloadReader = struct {
|
||||
}
|
||||
};
|
||||
|
||||
fn decodeJson(alloc: Allocator, json: []const u8, image_blobs: ?*session_codec.ImageBlobReader) Error!Decoded {
|
||||
fn decodeJson(alloc: Allocator, json: []const u8, images: session_codec.ImageSources) Error!Decoded {
|
||||
const parsed = std.json.parseFromSlice(
|
||||
std.json.Value,
|
||||
alloc,
|
||||
@@ -244,13 +289,13 @@ fn decodeJson(alloc: Allocator, json: []const u8, image_blobs: ?*session_codec.I
|
||||
alloc.free(history);
|
||||
}
|
||||
for (history_value.array.items, 0..) |turn_value, index| {
|
||||
history[index] = session_codec.parseHistoryTurnWithImageBlobs(alloc, turn_value, image_blobs) catch |err| switch (err) {
|
||||
history[index] = session_codec.parseHistoryTurnWithImages(alloc, turn_value, images) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => return error.InvalidCheckpoint,
|
||||
};
|
||||
decoded_count += 1;
|
||||
}
|
||||
if (image_blobs) |blobs| {
|
||||
if (images.blobs) |blobs| {
|
||||
if (!blobs.consumedAll()) return error.InvalidCheckpoint;
|
||||
}
|
||||
const usage = std.json.parseFromValueLeaky(types.Usage, alloc, usage_value, .{}) catch
|
||||
@@ -327,9 +372,9 @@ test "kernel checkpoint refuses meta that is not short text" {
|
||||
"{\"history\":[],\"usage\":{},\"meta\":{\"model\":7}}",
|
||||
"{\"history\":[],\"usage\":{},\"meta\":{\"model\":\"" ++ text_utils.repeat("m", max_meta_field_bytes + 1) ++ "\"}}",
|
||||
};
|
||||
for (cases) |json| try std.testing.expectError(error.InvalidCheckpoint, decodeJson(alloc, json, null));
|
||||
for (cases) |json| try std.testing.expectError(error.InvalidCheckpoint, decodeJson(alloc, json, .{}));
|
||||
// Fields it does not know are left alone.
|
||||
var decoded = try decodeJson(alloc, "{\"history\":[],\"usage\":{},\"meta\":{\"model\":\"m\",\"later\":1}}", null);
|
||||
var decoded = try decodeJson(alloc, "{\"history\":[],\"usage\":{},\"meta\":{\"model\":\"m\",\"later\":1}}", .{});
|
||||
defer decoded.deinit(alloc);
|
||||
try std.testing.expectEqualStrings("m", decoded.meta.model);
|
||||
try std.testing.expectEqualStrings("", decoded.meta.libfx_version);
|
||||
@@ -425,7 +470,7 @@ test "kernel checkpoint rejects corruption and unsupported versions" {
|
||||
|
||||
const unsupported = try alloc.dupe(u8, bytes);
|
||||
defer alloc.free(unsupported);
|
||||
std.mem.writeInt(u16, unsupported[4..6], version + 1, .little);
|
||||
std.mem.writeInt(u16, unsupported[4..6], naming_version + 1, .little);
|
||||
try std.testing.expectError(
|
||||
error.UnsupportedCheckpointVersion,
|
||||
decode(alloc, unsupported),
|
||||
@@ -577,3 +622,244 @@ test "kernel checkpoint requires each image blob once, in order" {
|
||||
resealForTest(unreferenced);
|
||||
try std.testing.expectError(error.InvalidCheckpoint, decode(alloc, unreferenced));
|
||||
}
|
||||
|
||||
test "kernel checkpoint names stored images and keeps the bytes of the rest" {
|
||||
const alloc = std.testing.allocator;
|
||||
const stored_png = "\x89PNG\r\n\x1a\nstored-in-host";
|
||||
const kept_png = "\x89PNG\r\n\x1a\nnever-stored";
|
||||
var stored_image: TestImageHistory = undefined;
|
||||
stored_image.init(stored_png);
|
||||
var kept_image: TestImageHistory = undefined;
|
||||
kept_image.init(kept_png);
|
||||
const history = [_]types.HistoryTurn{ stored_image.history[0], kept_image.history[0] };
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, stored_image.digest_hex);
|
||||
|
||||
const bytes = try encodeNamingImages(alloc, &history, .{}, .{}, &stored);
|
||||
defer alloc.free(bytes);
|
||||
try std.testing.expectEqual(naming_version, std.mem.readInt(u16, bytes[4..6], .little));
|
||||
// The stored image is named, not held; the other keeps its raw blob.
|
||||
try std.testing.expect(std.mem.find(u8, bytes, stored_png) == null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "\"inline_ref\":\"" ++ stored_image.digest_hex ++ "\"") != null);
|
||||
try std.testing.expect(std.mem.endsWith(u8, bytes, kept_png));
|
||||
|
||||
// Without the host's images it cannot be read.
|
||||
try std.testing.expectError(error.InvalidCheckpoint, decode(alloc, bytes));
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, stored_image.digest_hex, try alloc.dupe(u8, stored_png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var decoded = try decodeWithImages(alloc, bytes, &resolver);
|
||||
defer decoded.deinit(alloc);
|
||||
try std.testing.expectEqualStrings(stored_png, decoded.history[0].assistant.user.images[0].inline_data.?);
|
||||
try std.testing.expectEqualStrings(kept_png, decoded.history[1].assistant.user.images[0].inline_data.?);
|
||||
try std.testing.expectEqual(@as(usize, 0), resolver.missing.count());
|
||||
try std.testing.expect(resolver.resolved.contains(stored_image.digest_hex));
|
||||
}
|
||||
|
||||
test "kernel checkpoint names a stored image with its size" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\nsized-in-host";
|
||||
var fixture: TestImageHistory = undefined;
|
||||
fixture.init(png);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, fixture.digest_hex);
|
||||
const bytes = try encodeNamingImages(alloc, &fixture.history, .{}, .{}, &stored);
|
||||
defer alloc.free(bytes);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, std.fmt.comptimePrint("\"inline_size\":{d}", .{png.len})) != null);
|
||||
|
||||
// A staged image whose size is not the one recorded is refused.
|
||||
const sized = "\"inline_size\":";
|
||||
const resized = try alloc.dupe(u8, bytes);
|
||||
defer alloc.free(resized);
|
||||
const at = std.mem.find(u8, resized, sized).? + sized.len;
|
||||
resized[at] += 1;
|
||||
resealForTest(resized);
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, fixture.digest_hex, try alloc.dupe(u8, png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
try std.testing.expectError(error.InvalidCheckpoint, decodeWithImages(alloc, resized, &resolver));
|
||||
}
|
||||
|
||||
test "kernel checkpoint keeps a dropped image's name alone, at the naming version" {
|
||||
const alloc = std.testing.allocator;
|
||||
var fixture: TestImageHistory = undefined;
|
||||
fixture.init("\x89PNG\r\n\x1a\ndropped-from-the-session");
|
||||
fixture.images[0].inline_data = null;
|
||||
try std.testing.expect(fixture.images[0].droppedFromSession());
|
||||
|
||||
// Even where no image is named, a dropped one needs the naming version.
|
||||
const bytes = try encode(alloc, &fixture.history, .{}, .{});
|
||||
defer alloc.free(bytes);
|
||||
try std.testing.expectEqual(naming_version, std.mem.readInt(u16, bytes[4..6], .little));
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "\"dropped\":true") != null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "\"inline_ref\"") == null);
|
||||
|
||||
var decoded = try decode(alloc, bytes);
|
||||
defer decoded.deinit(alloc);
|
||||
const image = decoded.history[0].assistant.user.images[0];
|
||||
try std.testing.expect(image.droppedFromSession());
|
||||
try std.testing.expectEqualStrings(&fixture.digest_hex, image.snapshot_sha256.?);
|
||||
}
|
||||
|
||||
/// A history of one turn whose one tool call returned one image, held as
|
||||
/// base64 and named as the host stored it.
|
||||
const TestToolImageHistory = struct {
|
||||
digest_hex: [Sha256.digest_length * 2]u8,
|
||||
encoded: [64]u8,
|
||||
images: [1]types.ToolImage,
|
||||
calls: [1]types.ToolCall,
|
||||
results: [1]types.PersistedToolResult,
|
||||
steps: [1]types.ToolExecutionStep,
|
||||
history: [1]types.HistoryTurn,
|
||||
|
||||
fn init(self: *TestToolImageHistory, png: []const u8) void {
|
||||
var digest: [Sha256.digest_length]u8 = undefined;
|
||||
Sha256.hash(png, &digest, .{});
|
||||
self.digest_hex = std.fmt.bytesToHex(digest, .lower);
|
||||
const data = std.base64.standard.Encoder.encode(&self.encoded, png);
|
||||
self.images = .{.{ .data = @constCast(data), .mime_type = @constCast("image/png"), .name = self.digest_hex }};
|
||||
self.calls = .{.{ .id = "call-1", .name = "screenshot", .arguments_json = "{}" }};
|
||||
self.results = .{.{
|
||||
.tool_images = &self.images,
|
||||
.tool_call_id = @constCast("call-1"),
|
||||
.tool_name = @constCast("screenshot"),
|
||||
.status = .success,
|
||||
.output = @constCast("took it"),
|
||||
.output_bytes = 7,
|
||||
.stored_output_bytes = 7,
|
||||
}};
|
||||
self.steps = .{.{ .tool_calls = &self.calls, .tool_results = &self.results }};
|
||||
self.history = .{.{ .assistant = .{
|
||||
.user = .{ .text = @constCast("look") },
|
||||
.assistant = @constCast("seen"),
|
||||
.execution = .{ .tool_steps = &self.steps },
|
||||
} }};
|
||||
}
|
||||
};
|
||||
|
||||
test "kernel checkpoint names a stored tool image and reads it back from the host's copy" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\ntool-image-in-host";
|
||||
var fixture: TestToolImageHistory = undefined;
|
||||
fixture.init(png);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
|
||||
const bytes = try encodeNamingImages(alloc, &fixture.history, .{}, .{}, &stored);
|
||||
defer alloc.free(bytes);
|
||||
try std.testing.expectEqual(naming_version, std.mem.readInt(u16, bytes[4..6], .little));
|
||||
// Named and sized, never held.
|
||||
try std.testing.expect(std.mem.find(u8, bytes, fixture.images[0].data) == null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "\"inline_ref\":\"" ++ fixture.digest_hex ++ "\"") != null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, std.fmt.comptimePrint("\"inline_size\":{d}", .{png.len})) != null);
|
||||
|
||||
// Without the host's images it cannot be read.
|
||||
try std.testing.expectError(error.InvalidCheckpoint, decode(alloc, bytes));
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, fixture.digest_hex, try alloc.dupe(u8, png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var decoded = try decodeWithImages(alloc, bytes, &resolver);
|
||||
defer decoded.deinit(alloc);
|
||||
const image = decoded.history[0].assistant.execution.tool_steps[0].tool_results[0].tool_images[0];
|
||||
try std.testing.expectEqualStrings(fixture.images[0].data, image.data);
|
||||
try std.testing.expectEqualStrings(&fixture.digest_hex, &image.name.?);
|
||||
try std.testing.expectEqual(@as(usize, 0), resolver.missing.count());
|
||||
}
|
||||
|
||||
test "kernel checkpoint without a store keeps a tool image's bytes, and a dropped one's name alone" {
|
||||
const alloc = std.testing.allocator;
|
||||
var fixture: TestToolImageHistory = undefined;
|
||||
fixture.init("\x89PNG\r\n\x1a\ntool-image-kept");
|
||||
const bytes = try encode(alloc, &fixture.history, .{}, .{});
|
||||
defer alloc.free(bytes);
|
||||
try std.testing.expectEqual(version, std.mem.readInt(u16, bytes[4..6], .little));
|
||||
try std.testing.expect(std.mem.find(u8, bytes, fixture.images[0].data) != null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "\"inline_ref\"") == null);
|
||||
|
||||
fixture.images[0].data = fixture.images[0].data[0..0];
|
||||
try std.testing.expect(fixture.images[0].droppedFromSession());
|
||||
const dropped = try encode(alloc, &fixture.history, .{}, .{});
|
||||
defer alloc.free(dropped);
|
||||
try std.testing.expectEqual(naming_version, std.mem.readInt(u16, dropped[4..6], .little));
|
||||
try std.testing.expect(std.mem.find(u8, dropped, "\"dropped\":true") != null);
|
||||
try std.testing.expect(std.mem.find(u8, dropped, "\"inline_size\"") == null);
|
||||
var decoded = try decode(alloc, dropped);
|
||||
defer decoded.deinit(alloc);
|
||||
const image = decoded.history[0].assistant.execution.tool_steps[0].tool_results[0].tool_images[0];
|
||||
try std.testing.expect(image.droppedFromSession());
|
||||
try std.testing.expectEqualStrings(&fixture.digest_hex, &image.name.?);
|
||||
}
|
||||
|
||||
test "kernel checkpoint naming no stored image stays version 2, byte for byte" {
|
||||
const alloc = std.testing.allocator;
|
||||
var fixture: TestImageHistory = undefined;
|
||||
fixture.init("\x89PNG\r\n\x1a\nnot-in-the-store");
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, session_codec.imageNameOf("some other image"));
|
||||
|
||||
const plain = try encode(alloc, &fixture.history, .{}, .{});
|
||||
defer alloc.free(plain);
|
||||
const named = try encodeNamingImages(alloc, &fixture.history, .{}, .{}, &stored);
|
||||
defer alloc.free(named);
|
||||
try std.testing.expectEqualSlices(u8, plain, named);
|
||||
}
|
||||
|
||||
test "kernel checkpoint records a named image the host did not stage" {
|
||||
const alloc = std.testing.allocator;
|
||||
var fixture: TestImageHistory = undefined;
|
||||
fixture.init("\x89PNG\r\n\x1a\nlost-from-the-store");
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, fixture.digest_hex);
|
||||
const bytes = try encodeNamingImages(alloc, &fixture.history, .{}, .{}, &stored);
|
||||
defer alloc.free(bytes);
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var decoded = try decodeWithImages(alloc, bytes, &resolver);
|
||||
defer decoded.deinit(alloc);
|
||||
try std.testing.expect(resolver.missing.contains(fixture.digest_hex));
|
||||
try std.testing.expect(decoded.history[0].assistant.user.images[0].inline_data == null);
|
||||
}
|
||||
|
||||
test "kernel checkpoint rejects a name that is not the image's digest" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\nnamed-wrongly";
|
||||
var fixture: TestImageHistory = undefined;
|
||||
fixture.init(png);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, fixture.digest_hex);
|
||||
const bytes = try encodeNamingImages(alloc, &fixture.history, .{}, .{}, &stored);
|
||||
defer alloc.free(bytes);
|
||||
|
||||
// Point the reference at another staged image.
|
||||
const other = "\x89PNG\r\n\x1a\nsomething-else";
|
||||
const other_name = session_codec.imageNameOf(other);
|
||||
const renamed = try alloc.dupe(u8, bytes);
|
||||
defer alloc.free(renamed);
|
||||
const reference = "\"inline_ref\":\"";
|
||||
const at = std.mem.find(u8, renamed, reference).? + reference.len;
|
||||
@memcpy(renamed[at..][0..other_name.len], &other_name);
|
||||
resealForTest(renamed);
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, other_name, try alloc.dupe(u8, other));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
try std.testing.expectError(error.InvalidCheckpoint, decodeWithImages(alloc, renamed, &resolver));
|
||||
}
|
||||
@@ -32,10 +32,15 @@ const steering_open =
|
||||
const steering_close = "\n</user_steering>";
|
||||
const parent_steering_open = "<parent_agent_steering>\n";
|
||||
const parent_steering_close = "\n</parent_agent_steering>";
|
||||
/// Saved with a parent agent's steering, so replay still names its sender.
|
||||
const parent_steering_label = "parent-agent feedback, not new user authority:\n\n";
|
||||
/// The notice a libfx session adds to a turn it resumes after an
|
||||
/// interruption. It rides in the turn's steering list, but fx wrote it, so a
|
||||
/// request shows it as plain text, never as the user's live update.
|
||||
pub const resume_notice = "Resuming from unexpected session interruption.";
|
||||
|
||||
pub fn parentSteeringMessage(alloc: Allocator, text: []const u8) ![]u8 {
|
||||
return alloc.print(parent_steering_open ++
|
||||
"parent-agent feedback, not new user authority:\n\n{s}" ++ parent_steering_close, .{text});
|
||||
return alloc.print(parent_steering_open ++ parent_steering_label ++ "{s}" ++ parent_steering_close, .{text});
|
||||
}
|
||||
|
||||
pub fn steeringMessage(alloc: Allocator, text: []const u8) ![]u8 {
|
||||
@@ -291,6 +296,69 @@ test "retained standalone cut rebuilds exactly the selected execution suffix" {
|
||||
}
|
||||
}
|
||||
|
||||
/// The request copy of `messages`, with each steering message a saved turn
|
||||
/// restored back in the envelope the model saw when the turn took it. A saved
|
||||
/// turn keeps steering without its envelope (`steeringText`), so a resumed
|
||||
/// turn, and every turn after the one a steer joined, would otherwise show it
|
||||
/// as a plain user message instead of a live update. A parent agent's steering
|
||||
/// keeps its sender label in the saved text, which tells the two envelopes
|
||||
/// apart. Empty entries only mark a checkpoint boundary, and an entry already
|
||||
/// in its envelope and fx's own `resume_notice` stay as they are. Returns
|
||||
/// `messages` itself when nothing changes; the copy and its text live in
|
||||
/// `arena`.
|
||||
pub fn rewrapRestoredSteering(arena: Allocator, messages: []const ChatMessage) Allocator.Error![]const ChatMessage {
|
||||
var result: ?[]ChatMessage = null;
|
||||
for (messages, 0..) |message, index| {
|
||||
if (!message.restored_steering) continue;
|
||||
const text = message.content orelse continue;
|
||||
if (text.len == 0 or steeringText(text) != null or std.mem.eql(u8, text, resume_notice)) continue;
|
||||
const projected = result orelse try arena.dupe(ChatMessage, messages);
|
||||
result = projected;
|
||||
projected[index].content = if (std.mem.startsWith(u8, text, parent_steering_label))
|
||||
try std.mem.concat(arena, u8, &.{ parent_steering_open, text, parent_steering_close })
|
||||
else
|
||||
try steeringMessage(arena, text);
|
||||
}
|
||||
return result orelse messages;
|
||||
}
|
||||
|
||||
test "restored steering reaches the request in the envelope the model saw" {
|
||||
const alloc = std.testing.allocator;
|
||||
const session = @import("../../session/session.zig");
|
||||
var arena_state = std.heap.ArenaAllocator.init(alloc);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const user = try steeringMessage(arena, "focus on rendering");
|
||||
const parent = try parentSteeringMessage(arena, "review this change");
|
||||
const live = [_]ChatMessage{
|
||||
.{ .role = .user, .content = user },
|
||||
.{ .role = .user, .content = parent },
|
||||
};
|
||||
const memory = try buildExecutionMemory(alloc, &live);
|
||||
defer types.freeExecutionMemory(alloc, memory);
|
||||
// A saved turn comes back as steering's bare text, flagged as steering.
|
||||
var restored: std.ArrayList(ChatMessage) = .empty;
|
||||
try session.appendExecutionMemoryChatMessages(arena, &restored, memory);
|
||||
try std.testing.expectEqual(@as(usize, 2), restored.items.len);
|
||||
try std.testing.expect(restored.items[0].restored_steering and steeringText(restored.items[0].content.?) == null);
|
||||
|
||||
const projected = try rewrapRestoredSteering(arena, restored.items);
|
||||
try std.testing.expectEqualStrings(user, projected[0].content.?);
|
||||
try std.testing.expectEqualStrings(parent, projected[1].content.?);
|
||||
// The session's own messages keep the bare text a saved turn holds.
|
||||
try std.testing.expectEqualStrings("focus on rendering", restored.items[0].content.?);
|
||||
|
||||
// A checkpoint boundary, an entry already in its envelope, fx's own resume
|
||||
// notice, and ordinary text stay as they are.
|
||||
const unchanged = [_]ChatMessage{
|
||||
.{ .role = .user, .content = "", .restored_steering = true },
|
||||
.{ .role = .user, .content = user, .restored_steering = true },
|
||||
.{ .role = .user, .content = resume_notice, .restored_steering = true },
|
||||
.{ .role = .user, .content = "plain prompt" },
|
||||
};
|
||||
try std.testing.expect((try rewrapRestoredSteering(arena, &unchanged)).ptr == (&unchanged).ptr);
|
||||
}
|
||||
|
||||
/// Steering the user typed during the turn, which a cancelled turn keeps.
|
||||
fn isSteering(message: ChatMessage) bool {
|
||||
if (message.restored_steering) return true;
|
||||
|
||||
@@ -36,6 +36,9 @@
|
||||
//! such as a compaction during the turn. Events that use `base` or are a
|
||||
//! `tool_result` are version 2; every other event stays version 1, so an fx
|
||||
//! that reads only version 1 still loads a journal without them.
|
||||
//! An event that names an image instead of holding its bytes, one the host
|
||||
//! stored or one the session dropped, is version 3, so an fx that reads only
|
||||
//! up to version 2 refuses it as newer.
|
||||
//!
|
||||
//! A `turn_progress` event lists the inputs its model request carries for
|
||||
//! the first time in `inputs`, so placement is stored with the text it
|
||||
@@ -47,15 +50,23 @@ const std = @import("std");
|
||||
const types = @import("../../shared/types.zig");
|
||||
const testing_allocator = @import("../../shared/testing_allocator.zig");
|
||||
const session_codec = @import("../../session/session_codec.zig");
|
||||
const image_data = @import("../../images/image_data.zig");
|
||||
const checkpoint_codec = @import("checkpoint.zig");
|
||||
const compactor = @import("../../compactor/compactor.zig");
|
||||
|
||||
const Allocator = std.mem.Allocator;
|
||||
|
||||
/// The newest event version this fx reads and writes.
|
||||
pub const version: u8 = 2;
|
||||
/// The version of an event that needs none of version 2's additions.
|
||||
pub const version: u8 = 3;
|
||||
/// The version of an event that needs none of the later additions.
|
||||
const plain_version: u8 = 1;
|
||||
/// The version of a `tool_result` and of a progress that extends another.
|
||||
const delta_version: u8 = 2;
|
||||
/// The version of an event that names an image instead of holding its
|
||||
/// bytes, one the host stored (`inline_ref`) or one the session dropped, so an
|
||||
/// fx that reads only up to version 2 reports a newer journal instead of an
|
||||
/// invalid one.
|
||||
const naming_version: u8 = 3;
|
||||
/// The most event bytes one load accepts, and the most bytes one snapshot
|
||||
/// holds: each travels as one host attachment, and an attachment holds at
|
||||
/// most one kernel checkpoint. Longer sessions need snapshots.
|
||||
@@ -112,6 +123,9 @@ pub const ToolResult = struct {
|
||||
name: []const u8,
|
||||
failed: bool,
|
||||
output: []const u8,
|
||||
/// The images the result returned, which a reopened turn gives back to
|
||||
/// the call with its output.
|
||||
images: []const types.ToolImage = &.{},
|
||||
};
|
||||
|
||||
/// The longest turn id a host may give.
|
||||
@@ -130,11 +144,13 @@ pub const Event = union(EventType) {
|
||||
turn_yielded,
|
||||
tool_result: ToolResult,
|
||||
|
||||
/// The version `Cursor.write` gives the event.
|
||||
fn eventVersion(self: Event) u8 {
|
||||
/// The version `Cursor.write` gives the event: `naming_version` when it
|
||||
/// names an image, one `stored` holds or one the session dropped.
|
||||
fn eventVersion(self: Event, stored: ?*const session_codec.StoredImages) u8 {
|
||||
if (eventNamesImages(self, stored)) return naming_version;
|
||||
return switch (self) {
|
||||
.tool_result => version,
|
||||
.turn_progress => |progress| if (progress.delta != null) version else plain_version,
|
||||
.tool_result => delta_version,
|
||||
.turn_progress => |progress| if (progress.delta != null) delta_version else plain_version,
|
||||
else => plain_version,
|
||||
};
|
||||
}
|
||||
@@ -154,8 +170,18 @@ pub const Cursor = struct {
|
||||
|
||||
/// Writes `event` as one JSON object and returns the cursor after it.
|
||||
pub fn write(self: Cursor, writer: *std.Io.Writer, event: Event) !Cursor {
|
||||
return self.writeWith(writer, event, null);
|
||||
}
|
||||
|
||||
/// Like `write`, but names the images `stored` holds instead of writing
|
||||
/// their bytes.
|
||||
pub fn writeNamingImages(self: Cursor, writer: *std.Io.Writer, event: Event, stored: *const session_codec.StoredImages) !Cursor {
|
||||
return self.writeWith(writer, event, stored);
|
||||
}
|
||||
|
||||
fn writeWith(self: Cursor, writer: *std.Io.Writer, event: Event, stored: ?*const session_codec.StoredImages) !Cursor {
|
||||
try writer.print("{{\"v\":{d},\"seq\":{d},\"turn\":{d},\"type\":\"{s}\"", .{
|
||||
event.eventVersion(),
|
||||
event.eventVersion(stored),
|
||||
self.next_seq,
|
||||
self.turn,
|
||||
@tagName(event),
|
||||
@@ -163,14 +189,22 @@ pub const Cursor = struct {
|
||||
switch (event) {
|
||||
.turn_committed => |turn| {
|
||||
try writer.writeAll(",\"data\":");
|
||||
try session_codec.writeHistoryTurn(writer, turn);
|
||||
if (stored) |names|
|
||||
try session_codec.writeHistoryTurnNamingImages(writer, turn, .{ .stored = names })
|
||||
else
|
||||
try session_codec.writeHistoryTurn(writer, turn);
|
||||
},
|
||||
.turn_progress => |progress| {
|
||||
try writer.writeAll(",\"data\":");
|
||||
if (progress.delta) |delta| {
|
||||
if (delta.base_seq >= self.next_seq) return error.InvalidDurableField;
|
||||
try session_codec.writeRecoveryCheckpointDelta(writer, progress.checkpoint, delta.from);
|
||||
if (stored) |names|
|
||||
try session_codec.writeRecoveryCheckpointDeltaNamingImages(writer, progress.checkpoint, delta.from, .{ .stored = names })
|
||||
else
|
||||
try session_codec.writeRecoveryCheckpointDelta(writer, progress.checkpoint, delta.from);
|
||||
try writer.print(",\"base\":{d},\"steps\":{d}", .{ delta.base_seq, progress.checkpoint.execution.tool_steps.len });
|
||||
} else if (stored) |names| {
|
||||
try session_codec.writeRecoveryCheckpointNamingImages(writer, progress.checkpoint, .{ .stored = names });
|
||||
} else {
|
||||
try session_codec.writeRecoveryCheckpoint(writer, progress.checkpoint);
|
||||
}
|
||||
@@ -197,7 +231,10 @@ pub const Cursor = struct {
|
||||
try writer.writeAll(",\"data\":[");
|
||||
for (history, 0..) |turn, index| {
|
||||
if (index > 0) try writer.writeByte(',');
|
||||
try session_codec.writeHistoryTurn(writer, turn);
|
||||
if (stored) |names|
|
||||
try session_codec.writeHistoryTurnNamingImages(writer, turn, .{ .stored = names })
|
||||
else
|
||||
try session_codec.writeHistoryTurn(writer, turn);
|
||||
}
|
||||
try writer.writeByte(']');
|
||||
},
|
||||
@@ -211,13 +248,17 @@ pub const Cursor = struct {
|
||||
},
|
||||
.turn_yielded => {},
|
||||
.tool_result => |result| {
|
||||
try writer.writeAll(",\"data\":");
|
||||
try std.json.Stringify.value(.{
|
||||
.id = result.id,
|
||||
.name = result.name,
|
||||
.status = if (result.failed) "failure" else "success",
|
||||
.output = result.output,
|
||||
}, .{}, writer);
|
||||
try writer.writeAll(",\"data\":{\"id\":");
|
||||
try std.json.Stringify.value(result.id, .{}, writer);
|
||||
try writer.writeAll(",\"name\":");
|
||||
try std.json.Stringify.value(result.name, .{}, writer);
|
||||
try writer.print(",\"status\":\"{s}\",\"output\":", .{if (result.failed) "failure" else "success"});
|
||||
try std.json.Stringify.value(result.output, .{}, writer);
|
||||
if (result.images.len > 0) {
|
||||
try writer.writeAll(",\"images\":");
|
||||
try session_codec.writeToolImages(writer, result.images, stored != null);
|
||||
}
|
||||
try writer.writeByte('}');
|
||||
},
|
||||
}
|
||||
try writer.writeByte('}');
|
||||
@@ -228,6 +269,22 @@ pub const Cursor = struct {
|
||||
}
|
||||
};
|
||||
|
||||
/// Whether writing `event` names an image: one `stored` holds, or one the
|
||||
/// session dropped.
|
||||
fn eventNamesImages(event: Event, stored: ?*const session_codec.StoredImages) bool {
|
||||
return switch (event) {
|
||||
.turn_committed => |turn| session_codec.turnNamesImages(turn, stored),
|
||||
// A delta leaves out the user's turn: it names only what its new
|
||||
// tool results do.
|
||||
.turn_progress => |progress| session_codec.checkpointNamesImages(progress.checkpoint, if (progress.delta) |delta| delta.from else null, stored),
|
||||
.history_replaced => |history| for (history) |turn| {
|
||||
if (session_codec.turnNamesImages(turn, stored)) break true;
|
||||
} else false,
|
||||
.tool_result => |result| session_codec.toolImagesNameImages(result.images, stored),
|
||||
.tool_intent, .turn_progress_cleared, .input_accepted, .input_withdrawn, .turn_yielded => false,
|
||||
};
|
||||
}
|
||||
|
||||
/// A snapshot of a quiet prefix: `FXSN`, a version byte, the last `seq` it
|
||||
/// covers and the turn after it (each a little-endian u64), the id of the
|
||||
/// last turn (a length byte, zero for none, then its bytes), then a
|
||||
@@ -239,6 +296,9 @@ pub const Cursor = struct {
|
||||
const snapshot_magic = "FXSN";
|
||||
const snapshot_version_quiet: u8 = 1;
|
||||
const snapshot_version: u8 = 2;
|
||||
/// Version 2's layout, with an open turn that names images: ones the host
|
||||
/// stored, or ones the session dropped.
|
||||
const snapshot_naming_version: u8 = 3;
|
||||
const snapshot_header_bytes = snapshot_magic.len + 1 + 8 + 8 + 1;
|
||||
|
||||
pub const Snapshot = struct {
|
||||
@@ -264,6 +324,31 @@ pub fn encodeSnapshot(
|
||||
last_turn_id: ?[]const u8,
|
||||
open_turn: ?[]const u8,
|
||||
checkpoint: []const u8,
|
||||
) error{ OutOfMemory, SnapshotTooLarge }![]u8 {
|
||||
return encodeSnapshotWith(alloc, seq, turn, last_turn_id, open_turn, checkpoint, false);
|
||||
}
|
||||
|
||||
/// Like `encodeSnapshot`, for an `open_turn` that names at least one image
|
||||
/// (`session_codec.namesImages`).
|
||||
pub fn encodeSnapshotNamingImages(
|
||||
alloc: Allocator,
|
||||
seq: u64,
|
||||
turn: u64,
|
||||
last_turn_id: ?[]const u8,
|
||||
open_turn: []const u8,
|
||||
checkpoint: []const u8,
|
||||
) error{ OutOfMemory, SnapshotTooLarge }![]u8 {
|
||||
return encodeSnapshotWith(alloc, seq, turn, last_turn_id, open_turn, checkpoint, true);
|
||||
}
|
||||
|
||||
fn encodeSnapshotWith(
|
||||
alloc: Allocator,
|
||||
seq: u64,
|
||||
turn: u64,
|
||||
last_turn_id: ?[]const u8,
|
||||
open_turn: ?[]const u8,
|
||||
checkpoint: []const u8,
|
||||
open_turn_names_images: bool,
|
||||
) error{ OutOfMemory, SnapshotTooLarge }![]u8 {
|
||||
const id = last_turn_id orelse "";
|
||||
std.debug.assert(id.len <= max_turn_id_bytes);
|
||||
@@ -271,7 +356,12 @@ pub fn encodeSnapshot(
|
||||
if (open_turn) |open| if (open.len > std.math.maxInt(u32)) return error.SnapshotTooLarge;
|
||||
const bytes = try alloc.alloc(u8, snapshot_header_bytes + id.len + open_bytes + checkpoint.len);
|
||||
@memcpy(bytes[0..snapshot_magic.len], snapshot_magic);
|
||||
bytes[snapshot_magic.len] = if (open_turn == null) snapshot_version_quiet else snapshot_version;
|
||||
bytes[snapshot_magic.len] = if (open_turn == null)
|
||||
snapshot_version_quiet
|
||||
else if (open_turn_names_images)
|
||||
snapshot_naming_version
|
||||
else
|
||||
snapshot_version;
|
||||
std.mem.writeInt(u64, bytes[snapshot_magic.len + 1 ..][0..8], seq, .little);
|
||||
std.mem.writeInt(u64, bytes[snapshot_magic.len + 9 ..][0..8], turn, .little);
|
||||
bytes[snapshot_header_bytes - 1] = @intCast(id.len);
|
||||
@@ -291,8 +381,9 @@ pub fn parseSnapshot(bytes: []const u8) LoadError!Snapshot {
|
||||
return error.InvalidJournal;
|
||||
}
|
||||
const found_version = bytes[snapshot_magic.len];
|
||||
if (found_version > snapshot_version) return error.UnsupportedJournalVersion;
|
||||
if (found_version != snapshot_version and found_version != snapshot_version_quiet) return error.InvalidJournal;
|
||||
if (found_version > snapshot_naming_version) return error.UnsupportedJournalVersion;
|
||||
if (found_version == 0) return error.InvalidJournal;
|
||||
const has_open_turn = found_version != snapshot_version_quiet;
|
||||
const seq = std.mem.readInt(u64, bytes[snapshot_magic.len + 1 ..][0..8], .little);
|
||||
const turn = std.mem.readInt(u64, bytes[snapshot_magic.len + 9 ..][0..8], .little);
|
||||
if (seq == 0 or turn == 0 or seq > max_safe_position or turn > max_safe_position) return error.InvalidJournal;
|
||||
@@ -300,7 +391,7 @@ pub fn parseSnapshot(bytes: []const u8) LoadError!Snapshot {
|
||||
if (id_len > max_turn_id_bytes or bytes.len - snapshot_header_bytes < id_len) return error.InvalidJournal;
|
||||
var at: usize = snapshot_header_bytes + id_len;
|
||||
var open_turn: ?[]const u8 = null;
|
||||
if (found_version == snapshot_version) {
|
||||
if (has_open_turn) {
|
||||
if (bytes.len - at < 4) return error.InvalidJournal;
|
||||
const open_len = std.mem.readInt(u32, bytes[at..][0..4], .little);
|
||||
at += 4;
|
||||
@@ -320,6 +411,28 @@ pub fn parseSnapshot(bytes: []const u8) LoadError!Snapshot {
|
||||
/// Writes the turn that yielded, for a version 2 snapshot: its id, when the
|
||||
/// host gave one, and its progress as a `turn_progress` carries it.
|
||||
pub fn writeOpenTurn(writer: *std.Io.Writer, turn_id: ?[]const u8, progress: session_codec.RecoveryCheckpoint) !void {
|
||||
return writeOpenTurnWith(writer, turn_id, progress, null);
|
||||
}
|
||||
|
||||
/// Like `writeOpenTurn`, but names the images `stored` holds instead of
|
||||
/// writing their bytes. Returns whether it named any, a dropped one too,
|
||||
/// which makes the snapshot version 3 (`encodeSnapshotNamingImages`).
|
||||
pub fn writeOpenTurnNamingImages(
|
||||
writer: *std.Io.Writer,
|
||||
turn_id: ?[]const u8,
|
||||
progress: session_codec.RecoveryCheckpoint,
|
||||
stored: *const session_codec.StoredImages,
|
||||
) !bool {
|
||||
try writeOpenTurnWith(writer, turn_id, progress, stored);
|
||||
return session_codec.checkpointNamesImages(progress, null, stored);
|
||||
}
|
||||
|
||||
fn writeOpenTurnWith(
|
||||
writer: *std.Io.Writer,
|
||||
turn_id: ?[]const u8,
|
||||
progress: session_codec.RecoveryCheckpoint,
|
||||
stored: ?*const session_codec.StoredImages,
|
||||
) !void {
|
||||
try writer.writeByte('{');
|
||||
if (turn_id) |id| {
|
||||
std.debug.assert(id.len > 0 and id.len <= max_turn_id_bytes);
|
||||
@@ -328,7 +441,10 @@ pub fn writeOpenTurn(writer: *std.Io.Writer, turn_id: ?[]const u8, progress: ses
|
||||
try writer.writeByte(',');
|
||||
}
|
||||
try writer.writeAll("\"progress\":");
|
||||
try session_codec.writeRecoveryCheckpoint(writer, progress);
|
||||
if (stored) |names|
|
||||
try session_codec.writeRecoveryCheckpointNamingImages(writer, progress, .{ .stored = names })
|
||||
else
|
||||
try session_codec.writeRecoveryCheckpoint(writer, progress);
|
||||
try writer.writeByte('}');
|
||||
}
|
||||
|
||||
@@ -387,14 +503,22 @@ pub const Finished = struct {
|
||||
id: []u8,
|
||||
failed: bool,
|
||||
output: []u8,
|
||||
/// Owned. An image the host has yet to stage has no bytes, and the
|
||||
/// fold's resolver records it as missing.
|
||||
images: []types.ToolImage = &.{},
|
||||
};
|
||||
|
||||
fn freeFinished(alloc: Allocator, finished: []Finished) void {
|
||||
freeFinishedEntries(alloc, finished);
|
||||
alloc.free(finished);
|
||||
}
|
||||
|
||||
fn freeFinishedEntries(alloc: Allocator, finished: []const Finished) void {
|
||||
for (finished) |entry| {
|
||||
alloc.free(entry.id);
|
||||
alloc.free(entry.output);
|
||||
types.freeToolImages(alloc, entry.images);
|
||||
}
|
||||
alloc.free(finished);
|
||||
}
|
||||
|
||||
pub fn freePendingInputs(alloc: Allocator, inputs: []PendingInput) void {
|
||||
@@ -443,14 +567,12 @@ fn announcedCall(calls: []const std.json.Value, id: []const u8) bool {
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Owned copies of the newest recorded result of each call in `calls`.
|
||||
fn finishedOf(alloc: Allocator, calls: []const types.ToolCall, results: []const std.json.Value) LoadError![]Finished {
|
||||
/// Owned copies of the newest recorded result of each call in `calls`, with
|
||||
/// the images each names resolved through `sources`.
|
||||
fn finishedOf(alloc: Allocator, calls: []const types.ToolCall, results: []const std.json.Value, sources: session_codec.ImageSources) LoadError![]Finished {
|
||||
var finished: std.ArrayList(Finished) = .empty;
|
||||
errdefer {
|
||||
for (finished.items) |entry| {
|
||||
alloc.free(entry.id);
|
||||
alloc.free(entry.output);
|
||||
}
|
||||
freeFinishedEntries(alloc, finished.items);
|
||||
finished.deinit(alloc);
|
||||
}
|
||||
for (calls) |call| {
|
||||
@@ -463,10 +585,19 @@ fn finishedOf(alloc: Allocator, calls: []const types.ToolCall, results: []const
|
||||
errdefer alloc.free(id);
|
||||
const output = try alloc.dupe(u8, result.object.get("output").?.string);
|
||||
errdefer alloc.free(output);
|
||||
const images: []types.ToolImage = if (result.object.get("images")) |list|
|
||||
session_codec.parseToolImages(alloc, list.array.items, sources) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => return error.InvalidJournal,
|
||||
}
|
||||
else
|
||||
&.{};
|
||||
errdefer types.freeToolImages(alloc, images);
|
||||
try finished.append(alloc, .{
|
||||
.id = id,
|
||||
.failed = std.mem.eql(u8, result.object.get("status").?.string, "failure"),
|
||||
.output = output,
|
||||
.images = images,
|
||||
});
|
||||
}
|
||||
return finished.toOwnedSlice(alloc);
|
||||
@@ -489,6 +620,10 @@ pub const Base = struct {
|
||||
last_turn_id: ?[]const u8 = null,
|
||||
/// The turn that yielded, as `writeOpenTurn` wrote it.
|
||||
open_turn: ?[]const u8 = null,
|
||||
/// Resolves the images events name. Without it an event that names one
|
||||
/// is invalid; a name it cannot resolve is recorded there, and the fold
|
||||
/// is then only good for learning which images the host must stage.
|
||||
images: ?*session_codec.ImageResolver = null,
|
||||
};
|
||||
|
||||
/// Rebuilds a session from `events_json`, a JSON array of events in order.
|
||||
@@ -551,7 +686,7 @@ pub fn foldFrom(alloc: Allocator, events_json: []const u8, base: Base) LoadError
|
||||
if (event.kind != .turn_yielded) yielded = false;
|
||||
switch (event.kind) {
|
||||
.turn_committed => {
|
||||
const turn = try parseTurn(alloc, event.data orelse return error.InvalidJournal);
|
||||
const turn = try parseTurn(alloc, event.data orelse return error.InvalidJournal, base.images);
|
||||
// The same rule the session applies when it commits the turn.
|
||||
if (replacesHistory(turn)) {
|
||||
for (history.items) |old| types.freeHistoryTurn(alloc, old);
|
||||
@@ -629,6 +764,7 @@ pub fn foldFrom(alloc: Allocator, events_json: []const u8, base: Base) LoadError
|
||||
if (!std.mem.eql(u8, status, "success") and !std.mem.eql(u8, status, "failure")) return error.InvalidJournal;
|
||||
const output = data.object.get("output") orelse return error.InvalidJournal;
|
||||
if (output != .string) return error.InvalidJournal;
|
||||
if (data.object.get("images")) |images| if (images != .array) return error.InvalidJournal;
|
||||
// One for a call no intent announced answers nothing.
|
||||
if (announcedCall(running.items, id)) try results.append(alloc, data);
|
||||
}
|
||||
@@ -666,7 +802,7 @@ pub fn foldFrom(alloc: Allocator, events_json: []const u8, base: Base) LoadError
|
||||
replacement.deinit(alloc);
|
||||
}
|
||||
try replacement.ensureTotalCapacity(alloc, data.array.items.len);
|
||||
for (data.array.items) |item| replacement.appendAssumeCapacity(try parseTurn(alloc, item));
|
||||
for (data.array.items) |item| replacement.appendAssumeCapacity(try parseTurn(alloc, item, base.images));
|
||||
for (history.items) |turn| types.freeHistoryTurn(alloc, turn);
|
||||
history.deinit(alloc);
|
||||
history = replacement;
|
||||
@@ -680,7 +816,7 @@ pub fn foldFrom(alloc: Allocator, events_json: []const u8, base: Base) LoadError
|
||||
}
|
||||
|
||||
var open_turn: ?session_codec.RecoveryCheckpoint = null;
|
||||
if (open) |data| open_turn = if (data == .null) return error.InvalidJournal else session_codec.parseRecoveryCheckpoint(alloc, data) catch |err| switch (err) {
|
||||
if (open) |data| open_turn = if (data == .null) return error.InvalidJournal else session_codec.parseRecoveryCheckpointWithImages(alloc, data, .{ .names = base.images }) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => return error.InvalidJournal,
|
||||
};
|
||||
@@ -697,7 +833,7 @@ pub fn foldFrom(alloc: Allocator, events_json: []const u8, base: Base) LoadError
|
||||
return err;
|
||||
};
|
||||
errdefer types.freeToolCallSlice(alloc, unanswered);
|
||||
const finished = try finishedOf(alloc, unanswered, results.items);
|
||||
const finished = try finishedOf(alloc, unanswered, results.items, .{ .names = base.images });
|
||||
errdefer freeFinished(alloc, finished);
|
||||
const pending = try pendingInputsOf(alloc, if (open_turn != null) inputs.items else &.{}, .steer);
|
||||
errdefer freePendingInputs(alloc, pending);
|
||||
@@ -863,8 +999,8 @@ fn replacesHistory(turn: types.HistoryTurn) bool {
|
||||
};
|
||||
}
|
||||
|
||||
fn parseTurn(alloc: Allocator, value: std.json.Value) LoadError!types.HistoryTurn {
|
||||
return session_codec.parseHistoryTurn(alloc, value) catch |err| switch (err) {
|
||||
fn parseTurn(alloc: Allocator, value: std.json.Value, images: ?*session_codec.ImageResolver) LoadError!types.HistoryTurn {
|
||||
return session_codec.parseHistoryTurnWithImages(alloc, value, .{ .names = images }) catch |err| switch (err) {
|
||||
error.OutOfMemory => error.OutOfMemory,
|
||||
else => error.InvalidJournal,
|
||||
};
|
||||
@@ -1277,7 +1413,7 @@ test "a journal from a newer fx or with a malformed event is rejected" {
|
||||
const alloc = std.testing.allocator;
|
||||
try std.testing.expectError(
|
||||
error.UnsupportedJournalVersion,
|
||||
fold(alloc, "[{\"v\":3,\"seq\":1,\"turn\":1,\"type\":\"turn_progress_cleared\"}]"),
|
||||
fold(alloc, "[{\"v\":4,\"seq\":1,\"turn\":1,\"type\":\"turn_progress_cleared\"}]"),
|
||||
);
|
||||
const malformed = [_][]const u8{
|
||||
"{}",
|
||||
@@ -1656,7 +1792,7 @@ test "a snapshot that is not one, or comes from a newer fx, is refused" {
|
||||
try std.testing.expectError(error.InvalidJournal, parseSnapshot(zero));
|
||||
const newer = try encodeSnapshot(alloc, 3, 2, null, null, "");
|
||||
defer alloc.free(newer);
|
||||
newer[snapshot_magic.len] = snapshot_version + 1;
|
||||
newer[snapshot_magic.len] = snapshot_naming_version + 1;
|
||||
try std.testing.expectError(error.UnsupportedJournalVersion, parseSnapshot(newer));
|
||||
}
|
||||
|
||||
@@ -1698,3 +1834,199 @@ test "a turn's first progress names it, and the turn keeps that name until it en
|
||||
defer alloc.free(wrong);
|
||||
try std.testing.expectError(error.InvalidJournal, fold(alloc, wrong));
|
||||
}
|
||||
|
||||
fn testImageUser(alloc: Allocator, png: []const u8) !types.UserTurn {
|
||||
const name = session_codec.imageNameOf(png);
|
||||
const images = try alloc.alloc(types.ImageAttachment, 1);
|
||||
images[0] = .{
|
||||
.id = 1,
|
||||
.path = try alloc.dupe(u8, "inline://image-1"),
|
||||
.media_type = try alloc.dupe(u8, "image/png"),
|
||||
.snapshot_sha256 = try alloc.dupe(u8, &name),
|
||||
.inline_data = try alloc.dupe(u8, png),
|
||||
};
|
||||
return .{ .text = try alloc.dupe(u8, "[Image #1]"), .images = images };
|
||||
}
|
||||
|
||||
fn testImageTurn(alloc: Allocator, png: []const u8) !types.HistoryTurn {
|
||||
return .{ .assistant = .{ .user = try testImageUser(alloc, png), .assistant = try alloc.dupe(u8, "seen") } };
|
||||
}
|
||||
|
||||
test "a journal names the images the host stored and folds them back from its copies" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\njournal-named";
|
||||
const name = session_codec.imageNameOf(png);
|
||||
const turn = try testImageTurn(alloc, png);
|
||||
defer types.freeHistoryTurn(alloc, turn);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, name);
|
||||
|
||||
var out: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer out.deinit();
|
||||
try out.writer.writeByte('[');
|
||||
_ = try (Cursor{}).writeNamingImages(&out.writer, .{ .turn_committed = turn }, &stored);
|
||||
try out.writer.writeByte(']');
|
||||
const events = out.written();
|
||||
// The event names the image and holds none of its bytes.
|
||||
try std.testing.expect(std.mem.startsWith(u8, events, "[{\"v\":3,"));
|
||||
try std.testing.expect(std.mem.find(u8, events, "\"inline_ref\":\"" ++ name ++ "\"") != null);
|
||||
try std.testing.expect(std.mem.find(u8, events, "journal-named") == null);
|
||||
try std.testing.expect(std.mem.find(u8, events, "\"inline_data\"") == null);
|
||||
|
||||
try std.testing.expectError(error.InvalidJournal, fold(alloc, events));
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, name, try alloc.dupe(u8, png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var folded = try foldFrom(alloc, events, .{ .images = &resolver });
|
||||
defer folded.deinit(alloc);
|
||||
try std.testing.expectEqualStrings(png, folded.history[0].assistant.user.images[0].inline_data.?);
|
||||
try std.testing.expectEqual(@as(usize, 0), resolver.missing.count());
|
||||
try std.testing.expect(resolver.resolved.contains(name));
|
||||
}
|
||||
|
||||
test "an event that names no image is version 1, byte for byte" {
|
||||
const alloc = std.testing.allocator;
|
||||
const turn = try testImageTurn(alloc, "\x89PNG\r\n\x1a\nnot-stored");
|
||||
defer types.freeHistoryTurn(alloc, turn);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, session_codec.imageNameOf("another image"));
|
||||
|
||||
var plain: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer plain.deinit();
|
||||
_ = try (Cursor{}).write(&plain.writer, .{ .turn_committed = turn });
|
||||
var named: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer named.deinit();
|
||||
_ = try (Cursor{}).writeNamingImages(&named.writer, .{ .turn_committed = turn }, &stored);
|
||||
try std.testing.expectEqualStrings(plain.written(), named.written());
|
||||
try std.testing.expect(std.mem.startsWith(u8, named.written(), "{\"v\":1,"));
|
||||
}
|
||||
|
||||
test "one fold names every image the host did not stage" {
|
||||
const alloc = std.testing.allocator;
|
||||
const first_png = "\x89PNG\r\n\x1a\nfirst-missing";
|
||||
const second_png = "\x89PNG\r\n\x1a\nsecond-missing";
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, session_codec.imageNameOf(first_png));
|
||||
try stored.add(alloc, session_codec.imageNameOf(second_png));
|
||||
|
||||
var out: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer out.deinit();
|
||||
try out.writer.writeByte('[');
|
||||
var cursor: Cursor = .{};
|
||||
const first = try testImageTurn(alloc, first_png);
|
||||
defer types.freeHistoryTurn(alloc, first);
|
||||
cursor = try cursor.writeNamingImages(&out.writer, .{ .turn_committed = first }, &stored);
|
||||
try out.writer.writeByte(',');
|
||||
var progress = try testProgress(alloc, "");
|
||||
defer progress.deinit(alloc);
|
||||
types.freeUserTurn(alloc, progress.user);
|
||||
progress.user = try testImageUser(alloc, second_png);
|
||||
_ = try cursor.writeNamingImages(&out.writer, .{ .turn_progress = .{ .checkpoint = progress } }, &stored);
|
||||
try out.writer.writeByte(']');
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var folded = try foldFrom(alloc, out.written(), .{ .images = &resolver });
|
||||
defer folded.deinit(alloc);
|
||||
try std.testing.expectEqual(@as(usize, 2), resolver.missing.count());
|
||||
try std.testing.expect(resolver.missing.contains(session_codec.imageNameOf(first_png)));
|
||||
try std.testing.expect(resolver.missing.contains(session_codec.imageNameOf(second_png)));
|
||||
}
|
||||
|
||||
test "a recorded tool result keeps its images, naming the stored one" {
|
||||
const alloc = std.testing.allocator;
|
||||
const stored_png = image_data.testPngHeader(1, 1);
|
||||
const inline_png = image_data.testPngHeader(2, 1);
|
||||
const name = session_codec.imageNameOf(&stored_png);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, name);
|
||||
var stored_data: [std.base64.standard.Encoder.calcSize(stored_png.len)]u8 = undefined;
|
||||
var inline_data: [std.base64.standard.Encoder.calcSize(inline_png.len)]u8 = undefined;
|
||||
const images = [_]types.ToolImage{
|
||||
.{ .data = @constCast(std.base64.standard.Encoder.encode(&stored_data, &stored_png)), .mime_type = @constCast("image/png"), .name = name },
|
||||
.{ .data = @constCast(std.base64.standard.Encoder.encode(&inline_data, &inline_png)), .mime_type = @constCast("image/png") },
|
||||
};
|
||||
var progress = try testProgress(alloc, "take two shots");
|
||||
defer progress.deinit(alloc);
|
||||
const calls = [_]types.ToolCall{.{ .id = "call-a", .name = "screenshot", .arguments_json = "{}" }};
|
||||
|
||||
var out: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer out.deinit();
|
||||
try out.writer.writeByte('[');
|
||||
var cursor: Cursor = .{};
|
||||
cursor = try cursor.writeNamingImages(&out.writer, .{ .turn_progress = .{ .checkpoint = progress } }, &stored);
|
||||
try out.writer.writeByte(',');
|
||||
cursor = try cursor.writeNamingImages(&out.writer, .{ .tool_intent = &calls }, &stored);
|
||||
try out.writer.writeByte(',');
|
||||
_ = try cursor.writeNamingImages(&out.writer, .{ .tool_result = .{ .id = "call-a", .name = "screenshot", .failed = false, .output = "two shots", .images = &images } }, &stored);
|
||||
try out.writer.writeByte(']');
|
||||
const bytes = out.written();
|
||||
// It names the stored image, which makes it a naming event, and holds the
|
||||
// other's bytes.
|
||||
try std.testing.expect(std.mem.find(u8, bytes, "{\"v\":3,\"seq\":3,\"turn\":1,\"type\":\"tool_result\"") != null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, images[0].data) == null);
|
||||
try std.testing.expect(std.mem.find(u8, bytes, images[1].data) != null);
|
||||
|
||||
// An open that has yet to stage the stored image names it as missing.
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
var discover: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer discover.deinit();
|
||||
var unstaged = try foldFrom(alloc, bytes, .{ .images = &discover });
|
||||
defer unstaged.deinit(alloc);
|
||||
try std.testing.expect(discover.missing.contains(name));
|
||||
|
||||
// Staged, the call is answered with both images.
|
||||
try staged.add(alloc, name, try alloc.dupe(u8, &stored_png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var folded = try foldFrom(alloc, bytes, .{ .images = &resolver });
|
||||
defer folded.deinit(alloc);
|
||||
try std.testing.expectEqual(@as(usize, 0), resolver.missing.count());
|
||||
try std.testing.expectEqual(@as(usize, 1), folded.finished.len);
|
||||
try std.testing.expectEqualStrings("two shots", folded.finished[0].output);
|
||||
try std.testing.expectEqual(@as(usize, 2), folded.finished[0].images.len);
|
||||
try std.testing.expectEqualStrings(images[0].data, folded.finished[0].images[0].data);
|
||||
try std.testing.expectEqualStrings(images[1].data, folded.finished[0].images[1].data);
|
||||
}
|
||||
|
||||
test "a yielded snapshot names its open turn's stored images" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\nyielded-turn";
|
||||
const name = session_codec.imageNameOf(png);
|
||||
var stored: session_codec.StoredImages = .{};
|
||||
defer stored.deinit(alloc);
|
||||
try stored.add(alloc, name);
|
||||
var progress = try testProgress(alloc, "");
|
||||
defer progress.deinit(alloc);
|
||||
types.freeUserTurn(alloc, progress.user);
|
||||
progress.user = try testImageUser(alloc, png);
|
||||
|
||||
var open_out: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer open_out.deinit();
|
||||
try std.testing.expect(try writeOpenTurnNamingImages(&open_out.writer, "turn-y", progress, &stored));
|
||||
try std.testing.expect(std.mem.find(u8, open_out.written(), "yielded-turn") == null);
|
||||
const snapshot = try encodeSnapshotNamingImages(alloc, 5, 2, null, open_out.written(), "");
|
||||
defer alloc.free(snapshot);
|
||||
try std.testing.expectEqual(snapshot_naming_version, snapshot[snapshot_magic.len]);
|
||||
const parsed = try parseSnapshot(snapshot);
|
||||
|
||||
var staged: session_codec.StagedImages = .{};
|
||||
defer staged.deinit(alloc);
|
||||
try staged.add(alloc, name, try alloc.dupe(u8, png));
|
||||
var resolver: session_codec.ImageResolver = .{ .alloc = alloc, .staged = &staged };
|
||||
defer resolver.deinit();
|
||||
var folded = try foldFrom(alloc, "[]", .{ .cursor = .{ .next_seq = 6, .turn = 2 }, .open_turn = parsed.open_turn, .images = &resolver });
|
||||
defer folded.deinit(alloc);
|
||||
try std.testing.expectEqualStrings(png, folded.open_turn.?.user.images[0].inline_data.?);
|
||||
try std.testing.expect(folded.yielded);
|
||||
}
|
||||
@@ -4622,24 +4622,24 @@ test "context overflow recovery is typed safe and bounded" {
|
||||
));
|
||||
}
|
||||
|
||||
/// Size retries allowed in one turn. Each lowers the learned limit, so the
|
||||
/// next request leaves out more of its oldest images.
|
||||
const max_request_size_retries: u8 = 2;
|
||||
|
||||
/// A size rejection is retried without compaction only when the rejected
|
||||
/// request carried images the next one can leave out. Anything else falls
|
||||
/// through to context overflow recovery.
|
||||
/// A size rejection is retried without compaction while the rejected request
|
||||
/// carried images, and fewer than the request refused before it, if any.
|
||||
/// Each retry lowers the learned limit below the size just refused, which
|
||||
/// only leaving out more images can meet, so the count falls with every
|
||||
/// retry and the retries end. Anything else falls through to context
|
||||
/// overflow recovery.
|
||||
fn shouldRetryWithFewerImages(
|
||||
failure: agent_stream_provider.Failure,
|
||||
images_sent: usize,
|
||||
retries_used: u8,
|
||||
images_last_refused: ?usize,
|
||||
replay_safe: bool,
|
||||
cancelled: bool,
|
||||
) bool {
|
||||
const fewer = if (images_last_refused) |last| images_sent < last else true;
|
||||
return !cancelled and
|
||||
replay_safe and
|
||||
images_sent > 0 and
|
||||
retries_used < max_request_size_retries and
|
||||
fewer and
|
||||
failure.kind == .request_too_large;
|
||||
}
|
||||
|
||||
@@ -4687,15 +4687,20 @@ test "size rejections retry with fewer images only while images remain" {
|
||||
const cases = [_]struct {
|
||||
failure: agent_stream_provider.Failure,
|
||||
images_sent: usize = 3,
|
||||
retries_used: u8 = 0,
|
||||
images_last_refused: ?usize = null,
|
||||
replay_safe: bool = true,
|
||||
cancelled: bool = false,
|
||||
expected: bool,
|
||||
}{
|
||||
.{ .failure = too_large, .expected = true },
|
||||
.{ .failure = too_large, .retries_used = max_request_size_retries - 1, .expected = true },
|
||||
.{ .failure = too_large, .retries_used = max_request_size_retries, .expected = false },
|
||||
// Any number of retries, while each leaves out more images.
|
||||
.{ .failure = too_large, .images_sent = 3, .images_last_refused = 5, .expected = true },
|
||||
.{ .failure = too_large, .images_sent = 1, .images_last_refused = 2, .expected = true },
|
||||
// One that left out none stops, so the retries always end.
|
||||
.{ .failure = too_large, .images_sent = 3, .images_last_refused = 3, .expected = false },
|
||||
.{ .failure = too_large, .images_sent = 4, .images_last_refused = 3, .expected = false },
|
||||
.{ .failure = too_large, .images_sent = 0, .expected = false },
|
||||
.{ .failure = too_large, .images_sent = 0, .images_last_refused = 1, .expected = false },
|
||||
.{ .failure = too_large, .replay_safe = false, .expected = false },
|
||||
.{ .failure = too_large, .cancelled = true, .expected = false },
|
||||
.{ .failure = overflow_text, .expected = false },
|
||||
@@ -4703,7 +4708,7 @@ test "size rejections retry with fewer images only while images remain" {
|
||||
for (cases) |case| try std.testing.expectEqual(case.expected, shouldRetryWithFewerImages(
|
||||
case.failure,
|
||||
case.images_sent,
|
||||
case.retries_used,
|
||||
case.images_last_refused,
|
||||
case.replay_safe,
|
||||
case.cancelled,
|
||||
));
|
||||
@@ -7265,7 +7270,9 @@ fn processQueuedPromptLoop(
|
||||
var assistant_prefill_recovery_used = false;
|
||||
var skip_next_preflight_refresh = false;
|
||||
var context_overflow_recovery: ContextOverflowRecoveryState = .ready;
|
||||
var request_size_retries: u8 = 0;
|
||||
var request_size_retries: usize = 0;
|
||||
// The images the last request refused for its size carried.
|
||||
var request_images_last_refused: ?usize = null;
|
||||
var recovery_has_unexecuted_tool_start = false;
|
||||
var successful_recovery_strategy: ?model_response_recovery.Strategy = null;
|
||||
defer {
|
||||
@@ -7459,11 +7466,12 @@ fn processQueuedPromptLoop(
|
||||
subagent_request_messages.len,
|
||||
});
|
||||
}
|
||||
const result_request_messages = try project_read_tool_result_request_messages(
|
||||
// Images the session dropped leave every request as notes.
|
||||
const result_request_messages = try image_attachments.withholdDroppedImages(overlay_arena, try project_read_tool_result_request_messages(
|
||||
overlay_arena,
|
||||
read_tool_result_request_eligible,
|
||||
subagent_request_messages,
|
||||
);
|
||||
));
|
||||
if (!tool_image_capabilities_resolved and request_capabilities.image_input_support == .unknown) {
|
||||
for (result_request_messages) |message| {
|
||||
const memory = message.tool_result_memory orelse continue;
|
||||
@@ -7487,9 +7495,12 @@ fn processQueuedPromptLoop(
|
||||
break :native projection.messages;
|
||||
} else result_request_messages;
|
||||
const image_projection = try runtime_gateway_step.projectToolImageMessages(overlay_arena, materialized_messages, request_capabilities.image_input_support, vision_policy.route == .fallback, config.max_tool_result_bytes);
|
||||
const request_messages = try with_replyable_conversation_tail(overlay_arena, image_projection.messages);
|
||||
if (request_messages.ptr != image_projection.messages.ptr) {
|
||||
debug_trace.eventf("history", "assistant_tail_continued", step_ctx, "messages={d}", .{image_projection.messages.len});
|
||||
// Saved turns keep steering without its envelope; the model gets it
|
||||
// back as the live update it saw when the turn took it.
|
||||
const steering_messages = try runtime_execution_memory.rewrapRestoredSteering(overlay_arena, image_projection.messages);
|
||||
const request_messages = try with_replyable_conversation_tail(overlay_arena, steering_messages);
|
||||
if (request_messages.ptr != steering_messages.ptr) {
|
||||
debug_trace.eventf("history", "assistant_tail_continued", step_ctx, "messages={d}", .{steering_messages.len});
|
||||
}
|
||||
// Tool images withheld because the catalog could not confirm image
|
||||
// input are invisible in the transcript otherwise. Tell the user
|
||||
@@ -7543,6 +7554,20 @@ fn processQueuedPromptLoop(
|
||||
.max_output_tokens = model_capabilities.requestOutputTokens(request_capabilities),
|
||||
.budget = .{ .cancel_flag = config.cancel_flag },
|
||||
};
|
||||
// The request re-sends every image the session keeps, and
|
||||
// building it whole would hold all of them, encoded, at once.
|
||||
// Images that pass every limit by their own size leave before the
|
||||
// build; `fitRequestImages` fits the rest to this model's limit.
|
||||
const past_limit = try request_image_budget.withholdImagesPastLimit(
|
||||
overlay_arena,
|
||||
request_data.messages,
|
||||
request_image_budget.largestRequestBytes(agent.requestByteLimit(gateway_model)),
|
||||
config.max_tool_result_bytes,
|
||||
);
|
||||
if (past_limit.withheld > 0) {
|
||||
debug_trace.eventf("images", "request_images_withheld_before_build", step_ctx, "withheld={d}", .{past_limit.withheld});
|
||||
request_data.messages = past_limit.messages;
|
||||
}
|
||||
var prepared_request_body: ?[]const u8 = null;
|
||||
var request_cost_for_attempt: ?runtime_prompt_context.RequestCost = null;
|
||||
var request_images_for_attempt: usize = 0;
|
||||
@@ -7555,7 +7580,10 @@ fn processQueuedPromptLoop(
|
||||
deps.agent_stream_provider,
|
||||
&request_data,
|
||||
built_request_body,
|
||||
.{ .max_request_bytes = agent.requestByteLimit(gateway_model) orelse request_image_budget.default_max_request_bytes },
|
||||
.{ .max_request_bytes = request_image_budget.maxRequestBytes(
|
||||
agent.requestByteLimit(gateway_model),
|
||||
deps.agent_stream_provider.max_request_bytes,
|
||||
) },
|
||||
config.max_tool_result_bytes,
|
||||
step_ctx,
|
||||
);
|
||||
@@ -7719,7 +7747,14 @@ fn processQueuedPromptLoop(
|
||||
.tool_steps = finalization.compacted_execution.tool_steps + active_cut.tool_steps,
|
||||
.steering = finalization.compacted_execution.steering + active_cut.steering,
|
||||
};
|
||||
if (overflow_pending) context_overflow_recovery = .used;
|
||||
if (overflow_pending) {
|
||||
context_overflow_recovery = .used;
|
||||
// The shorter history can fit more images than the
|
||||
// request last refused for its size carried, so size
|
||||
// retries count again from the next refusal. This
|
||||
// happens once per step, so the retries still end.
|
||||
request_images_last_refused = null;
|
||||
}
|
||||
compactor.traceEvent(
|
||||
step_ctx,
|
||||
.installed,
|
||||
@@ -8365,12 +8400,13 @@ fn processQueuedPromptLoop(
|
||||
if (request_cost_for_attempt) |rejected| if (shouldRetryWithFewerImages(
|
||||
failure,
|
||||
request_images_for_attempt,
|
||||
request_size_retries,
|
||||
request_images_last_refused,
|
||||
streamReplaySafe(&stream_ctx),
|
||||
config.cancel_flag.load(.seq_cst),
|
||||
)) {
|
||||
const limit = loweredRequestByteLimit(rejected.serialized_bytes);
|
||||
agent.lowerRequestByteLimit(gateway_model, limit);
|
||||
request_images_last_refused = request_images_for_attempt;
|
||||
request_size_retries += 1;
|
||||
debug_trace.eventf("images", "request_size_retry", step_ctx, "model={s} rejected_bytes={d} images_sent={d} limit_bytes={d} retry={d}", .{
|
||||
gateway_model,
|
||||
@@ -9862,8 +9898,19 @@ fn processQueuedPromptLoop(
|
||||
ToolCall,
|
||||
prepared_tool_calls.len,
|
||||
);
|
||||
const saved_output_sources: runtime_tool_presentation.TurnSavedOutputSources = .{
|
||||
.registry = deps.tool_registry,
|
||||
.workspace_root = config.workspace_root,
|
||||
.turns = &.{ history_messages.items, within_turn_suffix.items },
|
||||
};
|
||||
for (prepared_tool_calls, 0..) |prepared_call, i| {
|
||||
effective_tool_calls[i] = prepared_call.call();
|
||||
effective_tool_calls[i].resolved_display_target = try runtime_tool_presentation.readToolResultDisplayTarget(
|
||||
arena,
|
||||
deps.tool_registry,
|
||||
effective_tool_calls[i],
|
||||
saved_output_sources,
|
||||
);
|
||||
}
|
||||
const pending_assistant: ChatMessage = .{
|
||||
.role = .assistant,
|
||||
@@ -10136,6 +10183,7 @@ fn processQueuedPromptLoop(
|
||||
for (prepared_tool_calls, 0..) |prepared_tool_call, tool_call_index| {
|
||||
if (tool_call_index < parallel_skip_until) continue;
|
||||
var tool_call = prepared_tool_call.call();
|
||||
tool_call.resolved_display_target = effective_tool_calls[tool_call_index].resolved_display_target;
|
||||
const root_live_permission_mode = snapshotRootPermissionMode(deps);
|
||||
const root_action_permission_mode = permissionModeForAction(
|
||||
job.permission_mode,
|
||||
|
||||
@@ -24,6 +24,13 @@ pub const Limits = struct {
|
||||
max_images: usize = default_max_request_images,
|
||||
};
|
||||
|
||||
/// The size a request must fit: the limit learned from the provider, else
|
||||
/// the default, and never more than the transport can carry.
|
||||
pub fn maxRequestBytes(learned: ?usize, transport: ?usize) usize {
|
||||
const limit = learned orelse default_max_request_bytes;
|
||||
return if (transport) |cap| @min(limit, cap) else limit;
|
||||
}
|
||||
|
||||
/// Returns how many of the oldest images must leave a request of
|
||||
/// `request_bytes` for it to fit `limits`. `image_bytes` holds each image's
|
||||
/// serialized size, oldest first. Returns `image_bytes.len` when the request
|
||||
@@ -54,6 +61,37 @@ pub fn withholdOldestImages(
|
||||
limits: Limits,
|
||||
tool_text_limit: usize,
|
||||
) Allocator.Error!Projection {
|
||||
const sizes = try imageSizes(arena, messages);
|
||||
return withhold(arena, messages, oldestImagesToWithhold(sizes, request_bytes, limits), tool_text_limit);
|
||||
}
|
||||
|
||||
/// A size no request is fit below: the default, or a larger limit learned
|
||||
/// from a provider's rejection. A transport may lower either, never raise it.
|
||||
pub fn largestRequestBytes(learned: ?usize) usize {
|
||||
return @max(default_max_request_bytes, learned orelse 0);
|
||||
}
|
||||
|
||||
/// Leaves out the oldest images a request of at most `max_request_bytes`
|
||||
/// can't carry by their serialized size alone, before the request is built.
|
||||
/// A request re-sends every image the session keeps, so building it whole
|
||||
/// would hold all of them, encoded, at once. A body is never smaller than
|
||||
/// its images, so this withholds no image `withholdOldestImages` would keep
|
||||
/// for a limit of at most `max_request_bytes`.
|
||||
pub fn withholdImagesPastLimit(
|
||||
arena: Allocator,
|
||||
messages: []const ChatMessage,
|
||||
max_request_bytes: usize,
|
||||
tool_text_limit: usize,
|
||||
) Allocator.Error!Projection {
|
||||
const sizes = try imageSizes(arena, messages);
|
||||
var images_bytes: usize = 0;
|
||||
for (sizes) |size| images_bytes +|= size;
|
||||
const withheld = oldestImagesToWithhold(sizes, images_bytes, .{ .max_request_bytes = max_request_bytes });
|
||||
return withhold(arena, messages, withheld, tool_text_limit);
|
||||
}
|
||||
|
||||
/// Each image's serialized size, oldest first, allocated in `arena`.
|
||||
fn imageSizes(arena: Allocator, messages: []const ChatMessage) Allocator.Error![]const usize {
|
||||
var sizes: std.ArrayList(usize) = .empty;
|
||||
for (messages) |message| {
|
||||
for (message.images) |image| {
|
||||
@@ -65,7 +103,11 @@ pub fn withholdOldestImages(
|
||||
for (memory.tool_images) |image| try sizes.append(arena, image.data.len);
|
||||
}
|
||||
}
|
||||
const withheld = oldestImagesToWithhold(sizes.items, request_bytes, limits);
|
||||
return sizes.items;
|
||||
}
|
||||
|
||||
/// `messages` with its `withheld` oldest images replaced by notes.
|
||||
fn withhold(arena: Allocator, messages: []const ChatMessage, withheld: usize, tool_text_limit: usize) Allocator.Error!Projection {
|
||||
if (withheld == 0) return .{ .messages = messages };
|
||||
|
||||
const projected = try arena.dupe(ChatMessage, messages);
|
||||
@@ -127,6 +169,15 @@ test "oldest images leave until the request fits" {
|
||||
try std.testing.expectEqual(@as(usize, 0), oldestImagesToWithhold(&.{}, 50_000, limits));
|
||||
}
|
||||
|
||||
test "a request fits the transport's limit as well as the provider's" {
|
||||
try std.testing.expectEqual(default_max_request_bytes, maxRequestBytes(null, null));
|
||||
try std.testing.expectEqual(@as(usize, 4000), maxRequestBytes(4000, null));
|
||||
// A host transport that carries less lowers either limit.
|
||||
try std.testing.expectEqual(@as(usize, 8000), maxRequestBytes(null, 8000));
|
||||
try std.testing.expectEqual(@as(usize, 4000), maxRequestBytes(4000, 8000));
|
||||
try std.testing.expectEqual(@as(usize, 3000), maxRequestBytes(4000, 3000));
|
||||
}
|
||||
|
||||
test "oldest images leave when a request carries too many" {
|
||||
const limits = Limits{ .max_request_bytes = 1_000_000, .max_images = 2 };
|
||||
const sizes = [_]usize{ 10, 10, 10, 10 };
|
||||
@@ -186,6 +237,46 @@ test "withholding keeps history and names how to reload each image" {
|
||||
try std.testing.expectEqual(@as([*]const ChatMessage, &messages), unchanged.messages.ptr);
|
||||
}
|
||||
|
||||
test "images no request can carry leave before it is built, and none it can" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
var tool_images: [6]types.ToolImage = undefined;
|
||||
for (&tool_images) |*image| image.* = try testToolImage(arena, 4000);
|
||||
const messages = [_]ChatMessage{
|
||||
.{ .role = .tool, .content = "<path>a.png</path>", .tool_result_memory = .{ .tool_images = tool_images[0..3], .tool_image_handle = "image-result-a" } },
|
||||
.{ .role = .tool, .content = "<path>b.png</path>", .tool_result_memory = .{ .tool_images = tool_images[3..], .tool_image_handle = "image-result-b" } },
|
||||
};
|
||||
|
||||
// 24,000 bytes of images against 10,000: the oldest four leave, whatever
|
||||
// the text adds, and the newest two stay for the exact fit.
|
||||
const before = try withholdImagesPastLimit(arena, &messages, 10_000, 4096);
|
||||
try std.testing.expectEqual(@as(usize, 4), before.withheld);
|
||||
try std.testing.expectEqual(@as(usize, 0), before.messages[0].tool_result_memory.?.tool_images.len);
|
||||
try std.testing.expectEqual(@as(usize, 2), before.messages[1].tool_result_memory.?.tool_images.len);
|
||||
// Fitting the whole body withholds at least as many.
|
||||
const sizes = [_]usize{ 4000, 4000, 4000, 4000, 4000, 4000 };
|
||||
for ([_]usize{ 24_000, 24_100, 30_000 }) |body_bytes| {
|
||||
try std.testing.expect(oldestImagesToWithhold(&sizes, body_bytes, .{ .max_request_bytes = 10_000 }) >= before.withheld);
|
||||
}
|
||||
|
||||
// Images that fit by size stay for the exact fit to decide.
|
||||
const fits = try withholdImagesPastLimit(arena, &messages, 30_000, 4096);
|
||||
try std.testing.expectEqual(@as(usize, 0), fits.withheld);
|
||||
try std.testing.expectEqual(@as([*]const ChatMessage, &messages), fits.messages.ptr);
|
||||
|
||||
// The image count limit applies too.
|
||||
const many: [default_max_request_images + 5]usize = @splat(1);
|
||||
try std.testing.expectEqual(@as(usize, 5), oldestImagesToWithhold(&many, many.len, .{ .max_request_bytes = largestRequestBytes(null) }));
|
||||
}
|
||||
|
||||
test "no learned limit lowers the bound a request is fit below" {
|
||||
try std.testing.expectEqual(default_max_request_bytes, largestRequestBytes(null));
|
||||
try std.testing.expectEqual(default_max_request_bytes, largestRequestBytes(1000));
|
||||
// A rejected request that was mostly text can teach a larger limit.
|
||||
try std.testing.expectEqual(default_max_request_bytes + 1, largestRequestBytes(default_max_request_bytes + 1));
|
||||
}
|
||||
|
||||
test "withheld snapshot attachments point at their saved file" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
|
||||
@@ -225,9 +225,12 @@ test "terminal assistant completion continues with steering admitted during the
|
||||
resumed_job,
|
||||
);
|
||||
|
||||
// The steer comes back as the live update the model saw, not as a plain
|
||||
// user message.
|
||||
try expectBodyContainsInOrder(&resumed_gateway, 0, &.{
|
||||
"user prompt",
|
||||
"Original answer",
|
||||
"user_steering",
|
||||
"change direction",
|
||||
"Updated answer",
|
||||
"follow up",
|
||||
|
||||
@@ -14,6 +14,7 @@ const debug_trace = @import("../../shared/debug_trace.zig");
|
||||
const diff = @import("../../output/diff.zig");
|
||||
const file_mutation_contract = @import("../../tooling/file_mutation_contract.zig");
|
||||
const tool_result_errors = @import("../../tooling/tool_result_errors.zig");
|
||||
const read_result_presentation = @import("../../tooling/read_tool_result_presentation.zig");
|
||||
const command_result_mapping = if (@import("builtin").is_test)
|
||||
@import("../../tooling/command_result_mapping.zig")
|
||||
else
|
||||
@@ -751,6 +752,198 @@ pub fn fileMutationDisplayPath(
|
||||
};
|
||||
}
|
||||
|
||||
/// The handles under which a tool result keeps its full output or images.
|
||||
fn savedOutputHandles(
|
||||
output_handle: ?[]const u8,
|
||||
tool_image_handle: ?[]const u8,
|
||||
command_output_replay: ?types.CommandOutputReplay,
|
||||
) [3]?[]const u8 {
|
||||
return .{
|
||||
output_handle,
|
||||
tool_image_handle,
|
||||
if (command_output_replay) |replay| switch (replay) {
|
||||
.available => |descriptor| descriptor.handle,
|
||||
.unavailable => null,
|
||||
} else null,
|
||||
};
|
||||
}
|
||||
|
||||
/// Finds which call saved an output among a running prompt's messages: the
|
||||
/// newest tool result that keeps the handle, and the call that produced it.
|
||||
pub const TurnSavedOutputSources = struct {
|
||||
registry: tool_dispatch.Registry,
|
||||
workspace_root: []const u8,
|
||||
/// Message lists in conversation order, searched newest first.
|
||||
turns: []const []const types.ChatMessage,
|
||||
|
||||
/// Returns the terminal-safe name of the call that saved `handle`, or null
|
||||
/// when no message still holds it. The arena owns the returned text.
|
||||
pub fn find(self: TurnSavedOutputSources, arena: Allocator, handle: []const u8) !?[]const u8 {
|
||||
var turn_index = self.turns.len;
|
||||
while (turn_index > 0) {
|
||||
turn_index -= 1;
|
||||
const messages = self.turns[turn_index];
|
||||
var message_index = messages.len;
|
||||
while (message_index > 0) {
|
||||
message_index -= 1;
|
||||
const message = messages[message_index];
|
||||
if (message.role != .tool) continue;
|
||||
const memory = message.tool_result_memory orelse continue;
|
||||
if (!holdsHandle(savedOutputHandles(memory.output_handle, memory.tool_image_handle, memory.command_output_replay), handle)) continue;
|
||||
const source = self.findCall(message.tool_call_id orelse return null) orelse return null;
|
||||
return tooling_presentation.savedOutputSourceLabel(arena, self.registry, self.workspace_root, source);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
fn findCall(self: TurnSavedOutputSources, call_id: []const u8) ?ToolCall {
|
||||
var turn_index = self.turns.len;
|
||||
while (turn_index > 0) {
|
||||
turn_index -= 1;
|
||||
const messages = self.turns[turn_index];
|
||||
var message_index = messages.len;
|
||||
while (message_index > 0) {
|
||||
message_index -= 1;
|
||||
const message = messages[message_index];
|
||||
if (message.role != .assistant) continue;
|
||||
for (message.tool_calls) |call| {
|
||||
if (std.mem.eql(u8, call.id, call_id)) return call;
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/// Names of the calls whose saved output later `read_tool_result` calls read,
|
||||
/// recorded while a saved conversation replays in order and keyed by output
|
||||
/// handle. Owns its keys and names; free them with `deinit`.
|
||||
pub const ReplayedSavedOutputSources = struct {
|
||||
map: std.StringHashMapUnmanaged([]const u8) = .empty,
|
||||
|
||||
pub fn deinit(self: *ReplayedSavedOutputSources, alloc: Allocator) void {
|
||||
var it = self.map.iterator();
|
||||
while (it.next()) |entry| {
|
||||
alloc.free(entry.key_ptr.*);
|
||||
alloc.free(entry.value_ptr.*);
|
||||
}
|
||||
self.map.deinit(alloc);
|
||||
}
|
||||
|
||||
/// Records `call` under each handle that keeps `result`'s output. A later
|
||||
/// call saving the same handle wins.
|
||||
pub fn record(
|
||||
self: *ReplayedSavedOutputSources,
|
||||
alloc: Allocator,
|
||||
registry: tool_dispatch.Registry,
|
||||
workspace_root: []const u8,
|
||||
call: ToolCall,
|
||||
result: types.PersistedToolResult,
|
||||
) Allocator.Error!void {
|
||||
const handles = savedOutputHandles(result.output_handle, result.tool_image_handle, result.command_output_replay);
|
||||
for (handles) |maybe_handle| {
|
||||
if (maybe_handle != null) break;
|
||||
} else return;
|
||||
const label = (try tooling_presentation.savedOutputSourceLabel(alloc, registry, workspace_root, call)) orelse return;
|
||||
defer alloc.free(label);
|
||||
for (handles) |maybe_handle| {
|
||||
const handle = maybe_handle orelse continue;
|
||||
const owned_handle = try alloc.dupe(u8, handle);
|
||||
errdefer alloc.free(owned_handle);
|
||||
const owned_label = try alloc.dupe(u8, label);
|
||||
errdefer alloc.free(owned_label);
|
||||
const gop = try self.map.getOrPut(alloc, owned_handle);
|
||||
if (gop.found_existing) {
|
||||
alloc.free(owned_handle);
|
||||
alloc.free(gop.value_ptr.*);
|
||||
}
|
||||
gop.value_ptr.* = owned_label;
|
||||
}
|
||||
}
|
||||
|
||||
/// The name of the call that saved `handle`, borrowed from the map, for
|
||||
/// `readToolResultDisplayTarget`.
|
||||
pub fn find(self: *const ReplayedSavedOutputSources, _: Allocator, handle: []const u8) error{}!?[]const u8 {
|
||||
return self.map.get(handle);
|
||||
}
|
||||
};
|
||||
|
||||
fn holdsHandle(handles: [3]?[]const u8, handle: []const u8) bool {
|
||||
for (handles) |candidate| {
|
||||
if (std.mem.eql(u8, candidate orelse continue, handle)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Row text for a `read_tool_result` call: what it reads, and the earlier
|
||||
/// call whose output that is when `sources.find(arena, handle)` still knows
|
||||
/// it. Returns null for other tools. The arena owns the returned text.
|
||||
pub fn readToolResultDisplayTarget(
|
||||
arena: Allocator,
|
||||
registry: tool_dispatch.Registry,
|
||||
call: ToolCall,
|
||||
sources: anytype,
|
||||
) !?[]const u8 {
|
||||
const spec = registry.lookup(call.name) orelse return null;
|
||||
if (spec.executor_kind != .read_tool_result) return null;
|
||||
const source = if (try read_result_presentation.requestedHandle(arena, call.arguments_json)) |handle|
|
||||
try sources.find(arena, handle)
|
||||
else
|
||||
null;
|
||||
return try read_result_presentation.displayTarget(arena, call.arguments_json, source);
|
||||
}
|
||||
|
||||
test "read_tool_result rows name the earlier call whose output they read" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const registry = test_builtin_tools.advertisement_set.registry;
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
const earlier_turn = [_]types.ChatMessage{
|
||||
.{ .role = .assistant, .tool_calls = &.{.{
|
||||
.id = "read-1",
|
||||
.name = "read_file",
|
||||
.arguments_json = "{\"path\":\"src/main.zig\"}",
|
||||
}} },
|
||||
.{ .role = .tool, .tool_call_id = "read-1", .tool_result_memory = .{ .output_handle = "result-read_file-1.txt" } },
|
||||
};
|
||||
const current_turn = [_]types.ChatMessage{
|
||||
.{ .role = .assistant, .tool_calls = &.{.{
|
||||
.id = "run-1",
|
||||
.name = "shell",
|
||||
.arguments_json = "{\"action\":\"run\",\"command\":\"zig build test\"}",
|
||||
}} },
|
||||
.{ .role = .tool, .tool_call_id = "run-1", .tool_result_memory = .{
|
||||
.command_output_replay = .{ .available = .{ .handle = replay, .framed_bytes = 64 } },
|
||||
} },
|
||||
};
|
||||
const sources: TurnSavedOutputSources = .{
|
||||
.registry = registry,
|
||||
.workspace_root = "/workspace",
|
||||
.turns = &.{ earlier_turn[0..], current_turn[0..] },
|
||||
};
|
||||
const cases = [_]struct { arguments_json: []const u8, expected: ?[]const u8 }{
|
||||
.{ .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"start_byte\":9}}", .expected = "more output from zig build test" },
|
||||
.{ .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}", .expected = "zig build test output for \"error:\"" },
|
||||
// The model may drop the `.txt` that `validate` restores.
|
||||
.{ .arguments_json = "{\"request\":{\"handle\":\"result-read_file-1\"}}", .expected = "more output from src/main.zig" },
|
||||
.{ .arguments_json = "{\"request\":{\"handle\":\"fx-command-replay-ffff.bin\"}}", .expected = null },
|
||||
.{ .arguments_json = "{\"request\":{\"handle\":\"T5\"}}", .expected = "earlier tool call T5" },
|
||||
};
|
||||
for (cases) |case| {
|
||||
const call: ToolCall = .{ .id = "page", .name = "read_tool_result", .arguments_json = case.arguments_json };
|
||||
const target = try readToolResultDisplayTarget(arena, registry, call, sources);
|
||||
if (case.expected) |expected| {
|
||||
try std.testing.expectEqualStrings(expected, target orelse return error.TestExpectedDisplayTarget);
|
||||
} else {
|
||||
try std.testing.expect(target == null);
|
||||
}
|
||||
}
|
||||
const other: ToolCall = .{ .id = "grep", .name = "grep_files", .arguments_json = "{\"pattern\":\"x\"}" };
|
||||
try std.testing.expect(try readToolResultDisplayTarget(arena, registry, other, sources) == null);
|
||||
}
|
||||
|
||||
pub noinline fn startToolVisibleLifecycle(
|
||||
hooks: *const AgentRuntimeDeps,
|
||||
arena: Allocator,
|
||||
@@ -778,6 +971,7 @@ pub noinline fn startToolVisibleLifecycle(
|
||||
.activity_kind = activity_kind,
|
||||
.arguments_json = redacted_arguments,
|
||||
.place_after_current_transcript = false,
|
||||
.resolved_display_target = call.resolved_display_target,
|
||||
} });
|
||||
if (activity_line) |line| {
|
||||
try hooks.push_tool_lifecycle(hooks.ctx, .{ .progress = .{
|
||||
|
||||
@@ -465,6 +465,9 @@ pub const Provider = struct {
|
||||
build_request_fn: ?BuildRequestFn = null,
|
||||
/// Pure provider-owned slicing when one reply becomes separate history units.
|
||||
project_replay_fn: ?*const fn (Allocator, ?types.ProviderReplay, []const types.ToolCall, bool, bool) anyerror!?types.ProviderReplay = null,
|
||||
/// The largest request body this provider's transport can carry, when it
|
||||
/// has its own limit below any provider's.
|
||||
max_request_bytes: ?usize = null,
|
||||
|
||||
/// Borrows unchanged payloads; projected payloads live in the caller's arena.
|
||||
pub fn projectReplay(self: Provider, arena: Allocator, replay: ?types.ProviderReplay, calls: []const types.ToolCall, text: bool, reasoning: bool) !?types.ProviderReplay {
|
||||
|
||||
@@ -1366,6 +1366,7 @@ fn formatToolAction(
|
||||
|
||||
const presentation = tool_dispatch.presentationForArgs(spec.*, args);
|
||||
const value = display_target orelse
|
||||
call.resolved_display_target orelse
|
||||
tool_presentation.resolvedSkillName(call, presentation) orelse
|
||||
tool_dispatch.presentationLabelValue(presentation, args) orelse
|
||||
presentation.label_arg_default;
|
||||
|
||||
@@ -1770,7 +1770,8 @@ pub const State = struct {
|
||||
var arena: std.heap.ArenaAllocator = .init(std.heap.c_allocator);
|
||||
defer arena.deinit();
|
||||
const a = arena.allocator();
|
||||
const statuses = lease.host.runtime.statuses(a) catch return "";
|
||||
// The status line never shows sign-in state, so it doesn't wait for it.
|
||||
const statuses = lease.host.runtime.statusesNow(a) catch return "";
|
||||
sink.mutex.lockUncancelable(io_mod.getIo());
|
||||
const news: ?[]const u8 = if (sink.news) |text|
|
||||
(if (now_ms - sink.news_at_ms < mcp_menu.news_ms) a.dupe(u8, text) catch null else null)
|
||||
@@ -2045,7 +2046,10 @@ pub const State = struct {
|
||||
try out.appendSlice(a, &verb_completions);
|
||||
const lease = self.leaseHost() orelse return out.items;
|
||||
defer lease.deinit();
|
||||
const statuses = try lease.host.runtime.statuses(a);
|
||||
// Rebuilt twice a second on the UI thread, so it doesn't wait for
|
||||
// saved credentials: a description says "signed in" from the next
|
||||
// rebuild after they are restored.
|
||||
const statuses = try lease.host.runtime.statusesNow(a);
|
||||
for (statuses, lease.host.runtime.servers()) |status, server| {
|
||||
if (!mcp_command_provider.isValidServerName(status.name)) continue;
|
||||
const description = try mcp_verbs.statusText(a, status);
|
||||
|
||||
@@ -4530,6 +4530,9 @@ pub fn Runtime(comptime App: type) type {
|
||||
|
||||
workspace_root: []const u8,
|
||||
map: std.StringHashMapUnmanaged(Label) = .empty,
|
||||
/// Calls whose saved output later `read_tool_result` rows read, so
|
||||
/// those rows name them as the live session did.
|
||||
saved_output_sources: tool_presentation.ReplayedSavedOutputSources = .{},
|
||||
|
||||
fn deinit(self: *HistoricalSessionLabels, alloc: Allocator) void {
|
||||
var it = self.map.iterator();
|
||||
@@ -4539,6 +4542,7 @@ pub fn Runtime(comptime App: type) type {
|
||||
if (entry.value_ptr.reflow) |value| alloc.free(value);
|
||||
}
|
||||
self.map.deinit(alloc);
|
||||
self.saved_output_sources.deinit(alloc);
|
||||
}
|
||||
};
|
||||
|
||||
@@ -5066,6 +5070,13 @@ pub fn Runtime(comptime App: type) type {
|
||||
result: types.PersistedToolResult,
|
||||
labels: *HistoricalSessionLabels,
|
||||
) !void {
|
||||
try labels.saved_output_sources.record(
|
||||
app.alloc,
|
||||
app.toolAdvertisementSet().registry,
|
||||
labels.workspace_root,
|
||||
call,
|
||||
result,
|
||||
);
|
||||
if (try writeAnsweredQuestionResult(app, sink, call, result)) return;
|
||||
if (try writeCommittedFilePresentation(app, sink, call, result)) return;
|
||||
|
||||
@@ -5080,7 +5091,13 @@ pub fn Runtime(comptime App: type) type {
|
||||
|
||||
var action_arena = std.heap.ArenaAllocator.init(app.alloc);
|
||||
defer mem_utils.deinit_arena(action_arena);
|
||||
const session_target = historicalSessionTarget(action_arena.allocator(), labels, call);
|
||||
const action_target = historicalSessionTarget(action_arena.allocator(), labels, call) orelse
|
||||
try tool_presentation.readToolResultDisplayTarget(
|
||||
action_arena.allocator(),
|
||||
app.toolAdvertisementSet().registry,
|
||||
call,
|
||||
&labels.saved_output_sources,
|
||||
);
|
||||
const command_decision = if (is_command)
|
||||
try tool_presentation.commandOutcomeDecision(
|
||||
action_arena.allocator(),
|
||||
@@ -5100,7 +5117,7 @@ pub fn Runtime(comptime App: type) type {
|
||||
try app.describeToolActionDeniedWithAdvertised(
|
||||
action_arena.allocator(),
|
||||
call,
|
||||
session_target,
|
||||
action_target,
|
||||
if (context_deferred)
|
||||
types.context_deferred_tool_status_label
|
||||
else
|
||||
@@ -5111,7 +5128,7 @@ pub fn Runtime(comptime App: type) type {
|
||||
try app.describeToolActionDeniedWithAdvertised(
|
||||
action_arena.allocator(),
|
||||
call,
|
||||
session_target,
|
||||
action_target,
|
||||
tool_admission.permissionDeniedStatusLabel(reason),
|
||||
&.{},
|
||||
)
|
||||
@@ -5119,7 +5136,7 @@ pub fn Runtime(comptime App: type) type {
|
||||
try app.describeToolActionDeniedWithAdvertised(
|
||||
action_arena.allocator(),
|
||||
call,
|
||||
session_target,
|
||||
action_target,
|
||||
decision.label,
|
||||
&.{},
|
||||
)
|
||||
@@ -5143,13 +5160,13 @@ pub fn Runtime(comptime App: type) type {
|
||||
break :success try app.describeToolActionCompletedWithAdvertised(
|
||||
action_arena.allocator(),
|
||||
call,
|
||||
display_target orelse session_target,
|
||||
display_target orelse action_target,
|
||||
&.{},
|
||||
);
|
||||
} else try app.describeToolActionDeniedWithAdvertised(
|
||||
action_arena.allocator(),
|
||||
call,
|
||||
session_target,
|
||||
action_target,
|
||||
try tooling_presentation.subagentFailureLabel(action_arena.allocator(), call, result.output),
|
||||
&.{},
|
||||
);
|
||||
@@ -8259,6 +8276,86 @@ test "execution replay records session labels for tty-launched sessions" {
|
||||
);
|
||||
}
|
||||
|
||||
test "execution replay names the call whose saved output a later read opens" {
|
||||
const alloc = std.testing.allocator;
|
||||
var app = try TestApp.init(alloc, "/workspace");
|
||||
defer app.deinit();
|
||||
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
const run_output = "{\"exit_code\":0,\"full_output_handle\":\"" ++ replay ++ "\"}";
|
||||
const file_output = "<path>src/main.zig</path>";
|
||||
var source_calls = [_]types.ToolCall{
|
||||
.{ .id = "call_run", .name = "shell", .arguments_json = "{\"action\":\"run\",\"command\":\"zig build test\"}" },
|
||||
.{ .id = "call_file", .name = "read_file", .arguments_json = "{\"path\":\"src/main.zig\"}" },
|
||||
};
|
||||
var source_results = [_]types.PersistedToolResult{
|
||||
.{
|
||||
.tool_call_id = @constCast("call_run"),
|
||||
.tool_name = @constCast("shell"),
|
||||
.status = .success,
|
||||
.output = @constCast(run_output),
|
||||
.output_bytes = run_output.len,
|
||||
.stored_output_bytes = run_output.len,
|
||||
.command_output_replay = .{ .available = .{ .handle = replay, .framed_bytes = 64 } },
|
||||
},
|
||||
.{
|
||||
.tool_call_id = @constCast("call_file"),
|
||||
.tool_name = @constCast("read_file"),
|
||||
.status = .success,
|
||||
.output = @constCast(file_output),
|
||||
.output_handle = @constCast("result-read_file-1.txt"),
|
||||
.output_bytes = file_output.len,
|
||||
.stored_output_bytes = file_output.len,
|
||||
},
|
||||
};
|
||||
var source_steps = [_]types.ToolExecutionStep{.{
|
||||
.tool_calls = source_calls[0..],
|
||||
.tool_results = source_results[0..],
|
||||
}};
|
||||
var read_calls = [_]types.ToolCall{
|
||||
.{ .id = "page", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"start_byte\":9}}" },
|
||||
.{ .id = "query", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}" },
|
||||
.{ .id = "file", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"result-read_file-1\"}}" },
|
||||
.{ .id = "unknown", .name = "read_tool_result", .arguments_json = "{\"request\":{\"handle\":\"result-gone.txt\"}}" },
|
||||
};
|
||||
var read_results: [read_calls.len]types.PersistedToolResult = undefined;
|
||||
for (&read_results, read_calls) |*result, call| {
|
||||
result.* = .{
|
||||
.tool_call_id = @constCast(call.id),
|
||||
.tool_name = @constCast(call.name),
|
||||
.status = .success,
|
||||
.output = @constCast("page"),
|
||||
.output_bytes = 4,
|
||||
.stored_output_bytes = 4,
|
||||
};
|
||||
}
|
||||
var read_steps = [_]types.ToolExecutionStep{.{
|
||||
.tool_calls = read_calls[0..],
|
||||
.tool_results = read_results[0..],
|
||||
}};
|
||||
const history = [_]types.HistoryTurn{
|
||||
.{ .assistant = .{
|
||||
.user = .{ .text = @constCast("run the tests") },
|
||||
.assistant = @constCast(""),
|
||||
.execution = .{ .tool_steps = source_steps[0..] },
|
||||
} },
|
||||
.{ .assistant = .{
|
||||
.user = .{ .text = @constCast("find the failure") },
|
||||
.assistant = @constCast(""),
|
||||
.execution = .{ .tool_steps = read_steps[0..] },
|
||||
} },
|
||||
};
|
||||
|
||||
try Runtime(TestApp).replayHistory(&app, &history);
|
||||
|
||||
const statuses = app.completed_tool_statuses.items;
|
||||
try std.testing.expectEqual(@as(usize, 6), statuses.len);
|
||||
try std.testing.expectEqualStrings("● Completed read_tool_result more output from zig build test\n", statuses[2]);
|
||||
try std.testing.expectEqualStrings("● Completed read_tool_result zig build test output for \"error:\"\n", statuses[3]);
|
||||
try std.testing.expectEqualStrings("● Completed read_tool_result more output from src/main.zig\n", statuses[4]);
|
||||
try std.testing.expectEqualStrings("● Completed read_tool_result\n", statuses[5]);
|
||||
}
|
||||
|
||||
test "resume projection stores reflow metadata for session action rows" {
|
||||
const alloc = std.testing.allocator;
|
||||
var app = try TestApp.init(alloc, "/workspace");
|
||||
|
||||
@@ -14,6 +14,11 @@ const pathing = @import("../workspace/pathing.zig");
|
||||
const text_utils = @import("../shared/text_utils.zig");
|
||||
|
||||
pub const max_image_bytes: usize = 20 * 1024 * 1024;
|
||||
/// The most images, and image bytes together, one libfx prompt holds: the
|
||||
/// limits `sdk/fx-sdk.js` takes a prompt within (`maxPromptImages` and
|
||||
/// `maxPromptImagesBytes`), which the core holds a prompt to as well.
|
||||
pub const max_libfx_prompt_images: usize = 8;
|
||||
pub const max_libfx_prompt_images_bytes: usize = 6 * 1024 * 1024;
|
||||
const max_encoded_image_bytes: usize = 5 * 1024 * 1024;
|
||||
pub const image_too_large_notice = "image exceeds the 20 MiB limit";
|
||||
pub const image_preparation_failed_notice = "Unable to prepare this image for upload. Use a smaller image.";
|
||||
@@ -848,6 +853,62 @@ pub fn attachmentEncodedBytes(attachment: types.ImageAttachment) ?usize {
|
||||
return image_data.encodedImageBytes(size);
|
||||
}
|
||||
|
||||
/// Leaves the images the session dropped out of a request, each with a note
|
||||
/// in its place, so no later projection or provider sees an image without
|
||||
/// bytes. Returns `messages` itself when none were dropped, otherwise a copy
|
||||
/// allocated in `arena`. History is not modified.
|
||||
pub fn withholdDroppedImages(arena: std.mem.Allocator, messages: []const types.ChatMessage) ![]const types.ChatMessage {
|
||||
var result: ?[]types.ChatMessage = null;
|
||||
for (messages, 0..) |message, index| {
|
||||
if (message.tool_result_memory) |memory| {
|
||||
var dropped_tools: usize = 0;
|
||||
for (memory.tool_images) |image| {
|
||||
if (image.droppedFromSession()) dropped_tools += 1;
|
||||
}
|
||||
if (dropped_tools > 0) {
|
||||
const projected = result orelse try arena.dupe(types.ChatMessage, messages);
|
||||
result = projected;
|
||||
const kept = try arena.alloc(types.ToolImage, memory.tool_images.len - dropped_tools);
|
||||
var next: usize = 0;
|
||||
for (memory.tool_images) |image| {
|
||||
if (image.droppedFromSession()) continue;
|
||||
kept[next] = image;
|
||||
next += 1;
|
||||
}
|
||||
var withheld = memory;
|
||||
withheld.tool_images = kept;
|
||||
projected[index].tool_result_memory = withheld;
|
||||
const notice = if (dropped_tools == 1)
|
||||
"[1 image from this result is no longer in this session, which keeps only its newest images.]\n"
|
||||
else
|
||||
try arena.print("[{d} images from this result are no longer in this session, which keeps only its newest images.]\n", .{dropped_tools});
|
||||
projected[index].content = try std.mem.concat(arena, u8, &.{ notice, message.content orelse "" });
|
||||
}
|
||||
}
|
||||
var dropped: usize = 0;
|
||||
for (message.images) |image| {
|
||||
if (image.droppedFromSession()) dropped += 1;
|
||||
}
|
||||
if (dropped == 0) continue;
|
||||
const projected = result orelse try arena.dupe(types.ChatMessage, messages);
|
||||
result = projected;
|
||||
const kept = try arena.alloc(types.ImageAttachment, message.images.len - dropped);
|
||||
var notice: std.Io.Writer.Allocating = .init(arena);
|
||||
var next: usize = 0;
|
||||
for (message.images) |image| {
|
||||
if (image.droppedFromSession()) {
|
||||
notice.writer.print("[Image #{d} is no longer in this session, which keeps only its newest images.]\n", .{image.id}) catch return error.OutOfMemory;
|
||||
} else {
|
||||
kept[next] = image;
|
||||
next += 1;
|
||||
}
|
||||
}
|
||||
projected[index].images = kept;
|
||||
projected[index].content = try std.mem.concat(arena, u8, &.{ notice.written(), projected[index].content orelse "" });
|
||||
}
|
||||
return result orelse messages;
|
||||
}
|
||||
|
||||
fn writeWithheldAttachmentNotice(
|
||||
writer: *std.Io.Writer,
|
||||
attachment: types.ImageAttachment,
|
||||
@@ -3058,6 +3119,50 @@ test "capture keeps a byte-oversized original for request recovery" {
|
||||
try std.testing.expectEqual(@as(usize, 1), try countSnapshotFiles(snapshot_dir));
|
||||
}
|
||||
|
||||
test "requests leave out images the session dropped, each with a note" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const digest = text_utils.repeat("a", 64);
|
||||
const images = [_]types.ImageAttachment{
|
||||
.{ .id = 1, .path = @constCast("inline://image-1"), .media_type = @constCast("image/png"), .snapshot_sha256 = @constCast(digest) },
|
||||
.{ .id = 2, .path = @constCast("inline://image-2"), .media_type = @constCast("image/png"), .snapshot_sha256 = @constCast(digest), .inline_data = @constCast("png") },
|
||||
};
|
||||
const messages = [_]types.ChatMessage{
|
||||
.{ .role = .user, .content = "[Image #1] [Image #2]", .images = &images },
|
||||
.{ .role = .assistant, .content = "ok" },
|
||||
};
|
||||
const projected = try withholdDroppedImages(arena, &messages);
|
||||
try std.testing.expectEqual(@as(usize, 1), projected[0].images.len);
|
||||
try std.testing.expectEqual(@as(usize, 2), projected[0].images[0].id);
|
||||
try std.testing.expectEqualStrings("[Image #1 is no longer in this session, which keeps only its newest images.]\n[Image #1] [Image #2]", projected[0].content.?);
|
||||
// A request without dropped images passes through untouched.
|
||||
const kept = [_]types.ChatMessage{.{ .role = .user, .content = "[Image #2]", .images = images[1..] }};
|
||||
try std.testing.expect((try withholdDroppedImages(arena, &kept)).ptr == (&kept).ptr);
|
||||
}
|
||||
|
||||
test "requests leave out tool images the session dropped, with one note per result" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
const name = text_utils.repeat("b", 64)[0..64].*;
|
||||
const tool_images = [_]types.ToolImage{
|
||||
.{ .data = @constCast(&[_]u8{}), .mime_type = @constCast("image/png"), .name = name },
|
||||
.{ .data = @constCast("aGVsbG8="), .mime_type = @constCast("image/png"), .name = name },
|
||||
.{ .data = @constCast(&[_]u8{}), .mime_type = @constCast("image/png"), .name = name },
|
||||
};
|
||||
const messages = [_]types.ChatMessage{
|
||||
.{ .role = .tool, .content = "took it", .tool_call_id = "call-1", .tool_name = "screenshot", .tool_result_memory = .{ .tool_images = &tool_images } },
|
||||
};
|
||||
const projected = try withholdDroppedImages(arena, &messages);
|
||||
const memory = projected[0].tool_result_memory.?;
|
||||
try std.testing.expectEqual(@as(usize, 1), memory.tool_images.len);
|
||||
try std.testing.expectEqualStrings("aGVsbG8=", memory.tool_images[0].data);
|
||||
try std.testing.expectEqualStrings("[2 images from this result are no longer in this session, which keeps only its newest images.]\ntook it", projected[0].content.?);
|
||||
// The session's own messages keep their images.
|
||||
try std.testing.expectEqual(@as(usize, 3), messages[0].tool_result_memory.?.tool_images.len);
|
||||
}
|
||||
|
||||
test "requests leave out attachments over the model pixel limit and name their saved file" {
|
||||
var arena_state = std.heap.ArenaAllocator.init(std.testing.allocator);
|
||||
defer arena_state.deinit();
|
||||
|
||||
@@ -45,12 +45,6 @@ pub const Server = struct {
|
||||
missing: ?[]const u8 = null,
|
||||
};
|
||||
|
||||
/// Looks up the credential saved for a server; owned by `alloc`.
|
||||
pub const Saved = struct {
|
||||
context: *anyopaque,
|
||||
get: *const fn (context: *anyopaque, alloc: Allocator, name: []const u8, url: []const u8) Allocator.Error!?[]u8,
|
||||
};
|
||||
|
||||
pub const Resolved = struct {
|
||||
servers: []Server,
|
||||
/// What the engine runs, in order: `engine[i]` is `servers[engine_server[i]]`.
|
||||
@@ -58,10 +52,6 @@ pub const Resolved = struct {
|
||||
engine_server: []usize,
|
||||
};
|
||||
|
||||
/// Allocates everything with `arena`, which must outlive the engine: the
|
||||
/// engine copies each config but keeps pointing at a stdio server's
|
||||
/// environment. A saved credential is copied into `arena`; zero it once the
|
||||
/// engine has its own copy.
|
||||
/// The command and its arguments as one line, quoting what a shell would split.
|
||||
fn commandLine(arena: Allocator, command: []const u8, args: []const []const u8) Allocator.Error![]const u8 {
|
||||
var out: std.ArrayList(u8) = .empty;
|
||||
@@ -80,11 +70,14 @@ fn commandLine(arena: Allocator, command: []const u8, args: []const []const u8)
|
||||
return out.items;
|
||||
}
|
||||
|
||||
/// Allocates everything with `arena`, which must outlive the engine: the
|
||||
/// engine copies each config but keeps pointing at a stdio server's
|
||||
/// environment. Saved sign-in credentials are not read here: the runtime
|
||||
/// hands them to the engine from its own thread.
|
||||
pub fn resolve(
|
||||
arena: Allocator,
|
||||
configs: []const McpServerConfig,
|
||||
inherited: *const std.process.Environ.Map,
|
||||
saved: ?Saved,
|
||||
) Allocator.Error!Resolved {
|
||||
const servers = try arena.alloc(Server, configs.len);
|
||||
var engine: std.ArrayList(mcp.engine.Config) = .empty;
|
||||
@@ -126,9 +119,6 @@ pub fn resolve(
|
||||
if (s.bridge and auth.resource == null) if (s.url) |url| {
|
||||
auth.resource = try arena.print("{s}/", .{origin(url)});
|
||||
};
|
||||
if (saved) |lookup| if (s.url) |url| {
|
||||
auth.saved = try lookup.get(lookup.context, arena, s.name, url);
|
||||
};
|
||||
try engine.append(arena, .{ .name = s.name, .transport = transport, .auth = auth });
|
||||
try engine_server.append(arena, i);
|
||||
}
|
||||
@@ -227,14 +217,7 @@ test "stdio and HTTP servers become engine configs; the rest are held back with
|
||||
.{ .name = "keyless", .transport = .http, .url = "https://k.example/mcp", .header_env = &missing_header },
|
||||
.{ .name = "carried", .transport = .http, .url = "http://acp.invalid/x", .source = .acp, .acp_server_id = @constCast("x") },
|
||||
};
|
||||
const Lookup = struct {
|
||||
fn get(_: *anyopaque, alloc: Allocator, name: []const u8, url: []const u8) Allocator.Error!?[]u8 {
|
||||
if (std.mem.eql(u8, name, "linear") and std.mem.eql(u8, url, "https://mcp.linear.app/mcp")) return try alloc.dupe(u8, "{\"access\":\"a\"}");
|
||||
return null;
|
||||
}
|
||||
};
|
||||
var context: u8 = 0;
|
||||
const r = try resolve(arena, &configs, &inherited, .{ .context = &context, .get = Lookup.get });
|
||||
const r = try resolve(arena, &configs, &inherited);
|
||||
|
||||
try testing.expectEqual(@as(usize, 3), r.engine.len);
|
||||
try testing.expectEqualSlices(usize, &.{ 0, 1, 2 }, r.engine_server);
|
||||
@@ -250,7 +233,8 @@ test "stdio and HTTP servers become engine configs; the rest are held back with
|
||||
try testing.expectEqualStrings("t1", linear.transport.http.headers[0].value);
|
||||
try testing.expectEqualStrings("c1", linear.auth.client_id.?);
|
||||
try testing.expectEqualStrings("s1", linear.auth.client_secret.?);
|
||||
try testing.expectEqualStrings("{\"access\":\"a\"}", linear.auth.saved.?);
|
||||
// The runtime's thread hands over saved credentials later.
|
||||
try testing.expect(linear.auth.saved == null);
|
||||
try testing.expectEqual(@as(?u16, 33418), r.servers[2].callback_port);
|
||||
|
||||
try testing.expectEqual(@as(?Held, .disabled), r.servers[3].held);
|
||||
@@ -272,7 +256,7 @@ test "Slack signs in as fx's Slack app through the bridge, unless the config nam
|
||||
.{ .name = "mine", .transport = .http, .url = "https://mcp.slack.com/mcp", .auth = .{ .client_id = @constCast("my-app") } },
|
||||
.{ .name = "linear", .transport = .http, .url = "https://mcp.linear.app/mcp" },
|
||||
};
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited, null);
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited);
|
||||
try testing.expectEqualStrings(slack.client_id, r.engine[0].auth.client_id.?);
|
||||
try testing.expect(r.servers[0].bridge);
|
||||
try testing.expectEqualStrings("https://mcp.slack.com/", r.engine[0].auth.resource.?);
|
||||
@@ -292,7 +276,7 @@ test "a stdio server starts in the directory its config names" {
|
||||
.{ .name = "proj", .command = "node", .args = &.{"s.js"}, .cwd = @constCast("/work/app") },
|
||||
.{ .name = "here", .command = "node", .args = &.{"s.js"} },
|
||||
};
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited, null);
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited);
|
||||
try testing.expectEqualStrings("/work/app", r.engine[0].transport.stdio.cwd.?);
|
||||
try testing.expect(r.engine[1].transport.stdio.cwd == null);
|
||||
}
|
||||
@@ -307,7 +291,7 @@ test "a token from the environment becomes the Authorization header" {
|
||||
.{ .name = "plain", .transport = .http, .url = "https://mcp.plain.com/mcp", .bearer_token_env = @constCast("PLAIN_KEY") },
|
||||
.{ .name = "unset", .transport = .http, .url = "https://u.example/mcp", .bearer_token_env = @constCast("UNSET_KEY") },
|
||||
};
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited, null);
|
||||
const r = try resolve(arena_state.allocator(), &configs, &inherited);
|
||||
try testing.expectEqual(@as(usize, 1), r.engine.len);
|
||||
try testing.expectEqualStrings("Authorization", r.engine[0].transport.http.headers[0].name);
|
||||
try testing.expectEqualStrings("Bearer k1", r.engine[0].transport.http.headers[0].value);
|
||||
|
||||
@@ -39,15 +39,13 @@ pub const Store = struct {
|
||||
return .{ .home = home, .backend = if (keychain) .keychain else .file };
|
||||
}
|
||||
|
||||
/// The credential saved for `name` at `url`, or null. The caller owns it
|
||||
/// and frees it with `secret.zeroAndFree`.
|
||||
pub fn get(s: Store, alloc: Allocator, name: []const u8, url: []const u8) !?[]u8 {
|
||||
/// Every saved credential from one read of the store, so a host with
|
||||
/// many servers reads the Keychain once, not once per server. The
|
||||
/// caller frees it with `Snapshot.deinit`.
|
||||
pub fn snapshot(s: Store, alloc: Allocator) !Snapshot {
|
||||
var locked = try s.lock();
|
||||
defer locked.release();
|
||||
var doc = try s.read(alloc, &locked.dir);
|
||||
defer doc.deinit(alloc);
|
||||
const i = doc.find(name, url) orelse return null;
|
||||
return try alloc.dupe(u8, doc.entries.items[i].credential);
|
||||
return .{ .doc = try s.read(alloc, &locked.dir) };
|
||||
}
|
||||
|
||||
/// Saves `credential` for `name` at `url`, replacing any earlier one.
|
||||
@@ -162,6 +160,24 @@ pub const Store = struct {
|
||||
}
|
||||
};
|
||||
|
||||
/// The saved credentials as one read of the store found them.
|
||||
pub const Snapshot = struct {
|
||||
doc: Doc,
|
||||
|
||||
/// The credential saved for `name` at `url`, or null. Borrowed from the
|
||||
/// snapshot until `deinit`.
|
||||
pub fn get(s: *const Snapshot, name: []const u8, url: []const u8) ?[]const u8 {
|
||||
const i = s.doc.find(name, url) orelse return null;
|
||||
return s.doc.entries.items[i].credential;
|
||||
}
|
||||
|
||||
/// Zeroes the credentials and frees them.
|
||||
pub fn deinit(s: *Snapshot, alloc: Allocator) void {
|
||||
s.doc.deinit(alloc);
|
||||
s.* = undefined;
|
||||
}
|
||||
};
|
||||
|
||||
fn readFile(alloc: Allocator, dir: *io_mod.VerifiedDir) !?[]u8 {
|
||||
var file = dir.dir.openFile(io_mod.getIo(), file_name, .{
|
||||
.mode = .read_only,
|
||||
@@ -278,6 +294,33 @@ fn testHome(tmp: *testing.TmpDir) ![]u8 {
|
||||
return io_mod.dirRealpathAlloc(testing.allocator, tmp.dir, ".");
|
||||
}
|
||||
|
||||
/// The credential saved for `name` at `url` from a fresh read, or null.
|
||||
/// The caller frees it with `secret.zeroAndFree`.
|
||||
fn savedFor(store: Store, name: []const u8, url: []const u8) !?[]u8 {
|
||||
var saved = try store.snapshot(testing.allocator);
|
||||
defer saved.deinit(testing.allocator);
|
||||
const credential = saved.get(name, url) orelse return null;
|
||||
return try testing.allocator.dupe(u8, credential);
|
||||
}
|
||||
|
||||
test "one snapshot holds every server's credential" {
|
||||
var tmp = testing.tmpDir(.{});
|
||||
defer tmp.cleanup();
|
||||
const home = try testHome(&tmp);
|
||||
defer testing.allocator.free(home);
|
||||
const store: Store = .{ .home = home, .backend = .file };
|
||||
try store.put(testing.allocator, "linear", "https://mcp.linear.app/mcp", "{\"access\":\"l1\"}");
|
||||
try store.put(testing.allocator, "plain", "https://mcp.plain.com/mcp", "{\"access\":\"p1\"}");
|
||||
|
||||
var saved = try store.snapshot(testing.allocator);
|
||||
defer saved.deinit(testing.allocator);
|
||||
try testing.expectEqualStrings("{\"access\":\"l1\"}", saved.get("linear", "https://mcp.linear.app/mcp").?);
|
||||
try testing.expectEqualStrings("{\"access\":\"p1\"}", saved.get("plain", "https://mcp.plain.com/mcp").?);
|
||||
// A credential is kept per name and URL.
|
||||
try testing.expectEqual(@as(?[]const u8, null), saved.get("plain", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]const u8, null), saved.get("slack", "https://mcp.slack.com/mcp"));
|
||||
}
|
||||
|
||||
test "a credential is kept per server name and URL, replaced, and removed" {
|
||||
var tmp = testing.tmpDir(.{});
|
||||
defer tmp.cleanup();
|
||||
@@ -285,21 +328,21 @@ test "a credential is kept per server name and URL, replaced, and removed" {
|
||||
defer testing.allocator.free(home);
|
||||
const store: Store = .{ .home = home, .backend = .file };
|
||||
|
||||
try testing.expectEqual(@as(?[]u8, null), try store.get(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try savedFor(store, "linear", "https://mcp.linear.app/mcp"));
|
||||
try store.put(testing.allocator, "linear", "https://mcp.linear.app/mcp", "{\"access\":\"a1\"}");
|
||||
try store.put(testing.allocator, "linear", "https://other.example/mcp", "{\"access\":\"b1\"}");
|
||||
try store.put(testing.allocator, "linear", "https://mcp.linear.app/mcp", "{\"access\":\"a2\"}");
|
||||
|
||||
const first = (try store.get(testing.allocator, "linear", "https://mcp.linear.app/mcp")).?;
|
||||
const first = (try savedFor(store, "linear", "https://mcp.linear.app/mcp")).?;
|
||||
defer secret.zeroAndFree(testing.allocator, first);
|
||||
try testing.expectEqualStrings("{\"access\":\"a2\"}", first);
|
||||
const other = (try store.get(testing.allocator, "linear", "https://other.example/mcp")).?;
|
||||
const other = (try savedFor(store, "linear", "https://other.example/mcp")).?;
|
||||
defer secret.zeroAndFree(testing.allocator, other);
|
||||
try testing.expectEqualStrings("{\"access\":\"b1\"}", other);
|
||||
|
||||
try testing.expect(try store.remove(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expect(!try store.remove(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try store.get(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try savedFor(store, "linear", "https://mcp.linear.app/mcp"));
|
||||
}
|
||||
|
||||
test "v1's logins are taken over once, and a sign-out isn't undone" {
|
||||
@@ -325,16 +368,16 @@ test "v1's logins are taken over once, and a sign-out isn't undone" {
|
||||
"{\"server_identity\":\"bound\",\"token_type\":\"DPoP\"," ++ entry ++ "}]}");
|
||||
}
|
||||
const store: Store = .{ .home = home, .backend = .file };
|
||||
const got = (try store.get(testing.allocator, "linear", "https://mcp.linear.app/mcp")).?;
|
||||
const got = (try savedFor(store, "linear", "https://mcp.linear.app/mcp")).?;
|
||||
defer secret.zeroAndFree(testing.allocator, got);
|
||||
try testing.expectEqualStrings(
|
||||
\\{"access":"v1-access","refresh":"v1-refresh","expires_at_ms":null,"token_endpoint":"https://linear.app/oauth/token","revocation_endpoint":null,"resource":"https://mcp.linear.app/mcp","client_id":"c1","client_secret":null,"method":"none"}
|
||||
, got);
|
||||
// A token bound to a key isn't one MCP-v2 can send.
|
||||
try testing.expectEqual(@as(?[]u8, null), try store.get(testing.allocator, "bound", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try savedFor(store, "bound", "https://mcp.linear.app/mcp"));
|
||||
// Signed out, it stays signed out: v1's copy isn't taken again.
|
||||
try testing.expect(try store.remove(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try store.get(testing.allocator, "linear", "https://mcp.linear.app/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try savedFor(store, "linear", "https://mcp.linear.app/mcp"));
|
||||
}
|
||||
|
||||
test "the credential file is private, and an unreadable one is replaced on the next save" {
|
||||
@@ -350,9 +393,9 @@ test "the credential file is private, and an unreadable one is replaced on the n
|
||||
try testing.expectEqual(@as(std.posix.mode_t, 0o600), stat.permissions.toMode() & 0o777);
|
||||
|
||||
try tmp.dir.writeFile(testing.io, .{ .sub_path = path, .data = "not json" });
|
||||
try testing.expectEqual(@as(?[]u8, null), try store.get(testing.allocator, "plain", "https://mcp.plain.com/mcp"));
|
||||
try testing.expectEqual(@as(?[]u8, null), try savedFor(store, "plain", "https://mcp.plain.com/mcp"));
|
||||
try store.put(testing.allocator, "plain", "https://mcp.plain.com/mcp", "{\"access\":\"p2\"}");
|
||||
const saved = (try store.get(testing.allocator, "plain", "https://mcp.plain.com/mcp")).?;
|
||||
const saved = (try savedFor(store, "plain", "https://mcp.plain.com/mcp")).?;
|
||||
defer secret.zeroAndFree(testing.allocator, saved);
|
||||
try testing.expectEqualStrings("{\"access\":\"p2\"}", saved);
|
||||
}
|
||||
@@ -48,6 +48,6 @@ pub const Fixture = struct {
|
||||
}
|
||||
|
||||
pub fn options(f: *const Fixture) Options {
|
||||
return .{ .client_version = "test", .credentials = .{ .home = f.home, .backend = .file } };
|
||||
return .{ .client_version = "test", .credentials = .{ .store = .{ .home = f.home, .backend = .file } } };
|
||||
}
|
||||
};
|
||||
@@ -18,7 +18,6 @@ const host_target = @import("../hosts/target.zig");
|
||||
const debug_trace = @import("../shared/debug_trace.zig");
|
||||
const text_utils = @import("../shared/text_utils.zig");
|
||||
const url_opener = @import("../hosts/url_opener.zig");
|
||||
const secret = @import("../auth/secret.zig");
|
||||
const config = @import("config.zig");
|
||||
const credentials = @import("credentials.zig");
|
||||
const loopback = @import("login.zig");
|
||||
@@ -132,7 +131,11 @@ pub const Notify = struct {
|
||||
|
||||
pub const Options = struct {
|
||||
client_version: []const u8,
|
||||
credentials: credentials.Store,
|
||||
/// Where sign-in credentials are kept. `.detect` names the home whose
|
||||
/// store the owner thread picks with `credentials.Store.detect`, so
|
||||
/// `start` never waits for the Keychain; the owner replaces it with
|
||||
/// that store before it handles any request.
|
||||
credentials: union(enum) { detect: []const u8, store: credentials.Store },
|
||||
notify: ?Notify = null,
|
||||
/// The questions fx's UI on this surface can ask, declared to servers so
|
||||
/// they only ask those.
|
||||
@@ -260,6 +263,9 @@ pub const Runtime = struct {
|
||||
/// Guarded by `mutex`: the latest statuses, in `snapshot_arena`.
|
||||
snapshot: []Status = &.{},
|
||||
snapshot_arena: std.heap.ArenaAllocator,
|
||||
/// Set once the owner gave the servers their saved credentials and
|
||||
/// published that, so `statuses` never reports a premature signed_in.
|
||||
credentials_restored: std.Io.Event = .unset,
|
||||
|
||||
/// Owner thread only.
|
||||
calls: std.AutoHashMapUnmanaged(engine.Handle, *Request) = .empty,
|
||||
@@ -280,7 +286,8 @@ pub const Runtime = struct {
|
||||
/// owner thread. `configs` and `inherited` are read only during the call.
|
||||
/// `gpa` must be thread-safe, and `r` must not move until `deinit`: the
|
||||
/// engine and login tasks point at it. Nothing connects until a server
|
||||
/// is needed.
|
||||
/// is needed, and saved credentials are read on the owner thread, so
|
||||
/// this never waits for the Keychain.
|
||||
pub fn start(
|
||||
r: *Runtime,
|
||||
gpa: Allocator,
|
||||
@@ -302,17 +309,15 @@ pub const Runtime = struct {
|
||||
errdefer r.arena.deinit();
|
||||
errdefer r.snapshot_arena.deinit();
|
||||
const arena = r.arena.allocator();
|
||||
r.resolved = try config.resolve(arena, configs, inherited, .{ .context = r, .get = savedCredential });
|
||||
r.resolved = try config.resolve(arena, configs, inherited);
|
||||
r.flags = try arena.alloc(Flags, r.resolved.servers.len);
|
||||
for (r.flags) |*f| f.* = .{};
|
||||
for (r.resolved.engine, r.resolved.engine_server) |c, i| r.flags[i].signed_in = c.auth.saved != null;
|
||||
try r.engine.init(io, gpa, .{ .context = r, .allow = allow, .log = log }, .{
|
||||
.client = .{ .info = .{ .name = "fx", .version = options.client_version }, .capabilities = declaredCapabilities(options.questions) },
|
||||
}, r.resolved.engine);
|
||||
errdefer r.engine.deinit();
|
||||
// The engine has its own copies now.
|
||||
for (r.resolved.engine) |c| if (c.auth.saved) |s| std.crypto.secureZero(u8, @constCast(s));
|
||||
// The rest may point into `configs`, which the caller may free now.
|
||||
// The engine has its own copies now, and these may point into
|
||||
// `configs`, which the caller may free now.
|
||||
r.resolved.engine = &.{};
|
||||
try r.publish();
|
||||
r.thread = try std.Thread.spawn(.{}, run, .{r});
|
||||
@@ -340,8 +345,19 @@ pub const Runtime = struct {
|
||||
r.* = undefined;
|
||||
}
|
||||
|
||||
/// Every configured server's status, copied into `alloc`.
|
||||
/// Every configured server's status, copied into `alloc`. Waits for the
|
||||
/// owner to restore saved credentials, so `signed_in` is settled.
|
||||
pub fn statuses(r: *Runtime, alloc: Allocator) Allocator.Error![]Status {
|
||||
_ = r.active.fetchAdd(1, .acq_rel);
|
||||
defer _ = r.active.fetchSub(1, .acq_rel);
|
||||
r.credentials_restored.waitUncancelable(r.io);
|
||||
return r.statusesNow(alloc);
|
||||
}
|
||||
|
||||
/// `statuses` without the wait, for the UI thread's periodic refreshes,
|
||||
/// which must not hold up a frame: until the owner restored saved
|
||||
/// credentials, every server reads as not signed in.
|
||||
pub fn statusesNow(r: *Runtime, alloc: Allocator) Allocator.Error![]Status {
|
||||
_ = r.active.fetchAdd(1, .acq_rel);
|
||||
defer _ = r.active.fetchSub(1, .acq_rel);
|
||||
r.mutex.lockUncancelable(r.io);
|
||||
@@ -514,6 +530,7 @@ pub const Runtime = struct {
|
||||
// ---- the owner thread ----
|
||||
|
||||
fn run(r: *Runtime) void {
|
||||
r.restoreCredentials();
|
||||
while (true) {
|
||||
const stopping = r.drain();
|
||||
r.flush();
|
||||
@@ -650,7 +667,7 @@ pub const Runtime = struct {
|
||||
const s = r.resolved.servers[i];
|
||||
if (s.transport != .http) return r.finish(req, .{ .failed = error.NotHttp });
|
||||
try r.engine.signOut(s.name);
|
||||
_ = r.options.credentials.remove(r.gpa, s.name, s.url.?) catch |err| {
|
||||
_ = r.options.credentials.store.remove(r.gpa, s.name, s.url.?) catch |err| {
|
||||
debug_trace.logf("mcp", "MCP credential not removed server={s} err={s}", .{ s.name, @errorName(err) });
|
||||
};
|
||||
r.flags[i] = .{};
|
||||
@@ -931,11 +948,11 @@ pub const Runtime = struct {
|
||||
out.deinit();
|
||||
}
|
||||
if (try r.engine.saveCredential(s.name, &out.writer)) {
|
||||
r.options.credentials.put(r.gpa, s.name, url, out.written()) catch |err| {
|
||||
r.options.credentials.store.put(r.gpa, s.name, url, out.written()) catch |err| {
|
||||
debug_trace.logf("mcp", "MCP credential not saved server={s} err={s}", .{ s.name, @errorName(err) });
|
||||
};
|
||||
} else {
|
||||
_ = r.options.credentials.remove(r.gpa, s.name, url) catch |err| {
|
||||
_ = r.options.credentials.store.remove(r.gpa, s.name, url) catch |err| {
|
||||
debug_trace.logf("mcp", "MCP credential not removed server={s} err={s}", .{ s.name, @errorName(err) });
|
||||
};
|
||||
}
|
||||
@@ -1157,17 +1174,32 @@ pub const Runtime = struct {
|
||||
return out;
|
||||
}
|
||||
|
||||
fn savedCredential(context: *anyopaque, alloc: Allocator, name: []const u8, url: []const u8) Allocator.Error!?[]u8 {
|
||||
const r: *Runtime = @ptrCast(@alignCast(context));
|
||||
const found = r.options.credentials.get(r.gpa, name, url) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => {
|
||||
debug_trace.logf("mcp", "MCP credential not loaded server={s} err={s}", .{ name, @errorName(err) });
|
||||
return null;
|
||||
},
|
||||
} orelse return null;
|
||||
defer secret.zeroAndFree(r.gpa, found);
|
||||
return try alloc.dupe(u8, found);
|
||||
/// Before any request: picks the store when `start` left that to the
|
||||
/// owner, reads it once, and gives each HTTP server its saved
|
||||
/// credential, so no server starts without it. A store that can't be
|
||||
/// read leaves every server signed out, as one without credentials.
|
||||
fn restoreCredentials(r: *Runtime) void {
|
||||
defer r.credentials_restored.set(r.io);
|
||||
const store: credentials.Store = switch (r.options.credentials) {
|
||||
.store => |s| s,
|
||||
.detect => |home| .detect(r.gpa, home),
|
||||
};
|
||||
r.options.credentials = .{ .store = store };
|
||||
var saved = store.snapshot(r.gpa) catch |err| {
|
||||
debug_trace.logf("mcp", "MCP credentials not loaded err={s}", .{@errorName(err)});
|
||||
return;
|
||||
};
|
||||
defer saved.deinit(r.gpa);
|
||||
for (r.resolved.engine_server) |i| {
|
||||
const s = r.resolved.servers[i];
|
||||
const credential = saved.get(s.name, s.url orelse continue) orelse continue;
|
||||
r.engine.restoreCredential(s.name, credential) catch |err| {
|
||||
debug_trace.logf("mcp", "MCP credential not loaded server={s} err={s}", .{ s.name, @errorName(err) });
|
||||
continue;
|
||||
};
|
||||
r.flags[i].signed_in = true;
|
||||
}
|
||||
r.publish() catch |err| debug_trace.logf("mcp", "MCP status not published err={s}", .{@errorName(err)});
|
||||
}
|
||||
|
||||
/// Permission is fx's tool admission, before the call reaches the
|
||||
@@ -1373,6 +1405,34 @@ test "a cancelled call ends, and stopping ends a call that never answers" {
|
||||
try testing.expectEqual(@as(?Error, error.Stopped), got);
|
||||
}
|
||||
|
||||
test "saved credentials load on the owner thread, and statuses wait for them" {
|
||||
var f: Fixture = undefined;
|
||||
try f.init();
|
||||
defer f.deinit();
|
||||
const store: credentials.Store = .{ .home = f.home, .backend = .file };
|
||||
try store.put(testing.allocator, "web", "https://web.example.invalid/mcp", "{\"access\":\"w1\"}");
|
||||
try store.put(testing.allocator, "moved", "https://old.example.invalid/mcp", "{\"access\":\"m1\"}");
|
||||
const configs = [_]mcp_contract.McpServerConfig{
|
||||
.{ .name = "web", .transport = .http, .url = "https://web.example.invalid/mcp" },
|
||||
// A credential saved for another URL is not this server's.
|
||||
.{ .name = "moved", .transport = .http, .url = "https://new.example.invalid/mcp" },
|
||||
.{ .name = "none", .transport = .http, .url = "https://none.example.invalid/mcp" },
|
||||
.{ .name = "fake", .command = f.server },
|
||||
};
|
||||
var r: Runtime = undefined;
|
||||
try r.start(testing.allocator, testing.io, &configs, &f.inherited, f.options());
|
||||
defer r.deinit();
|
||||
|
||||
var arena: std.heap.ArenaAllocator = .init(testing.allocator);
|
||||
defer arena.deinit();
|
||||
const got = try r.statuses(arena.allocator());
|
||||
try testing.expect(got[0].signed_in);
|
||||
try testing.expect(!got[1].signed_in);
|
||||
try testing.expect(!got[2].signed_in);
|
||||
try testing.expect(!got[3].signed_in);
|
||||
try testing.expectEqual(State.idle, got[0].state);
|
||||
}
|
||||
|
||||
test "held servers say why, and unknown names and stdio logins are refused" {
|
||||
var f: Fixture = undefined;
|
||||
try f.init();
|
||||
|
||||
@@ -598,11 +598,13 @@ fn serializeRecord(
|
||||
}
|
||||
|
||||
fn parseRecord(alloc: Allocator, line: []const u8) !ParsedRecord {
|
||||
// Strings may point into `line`, which outlives the parse; everything
|
||||
// returned is copied before `parsed` is freed.
|
||||
var parsed = try std.json.parseFromSlice(
|
||||
std.json.Value,
|
||||
alloc,
|
||||
line,
|
||||
.{ .allocate = .alloc_always },
|
||||
.{ .allocate = .alloc_if_needed },
|
||||
);
|
||||
defer parsed.deinit();
|
||||
if (parsed.value != .object or parsed.value.object.count() != 4) {
|
||||
@@ -662,6 +664,9 @@ fn processCompleteLine(
|
||||
entries: *std.ArrayList(LoadedPromptHistoryEntry),
|
||||
) !void {
|
||||
if (line.len + 1 > max_record_bytes) return;
|
||||
// Most lines belong to other workspaces; skip them before the full parse,
|
||||
// which costs far more and runs before the first frame.
|
||||
if (!mayBelongToWorkspace(line, workspace_root)) return;
|
||||
var record = parseRecord(alloc, line) catch |err| {
|
||||
debug_trace.logf(
|
||||
"prompt_history",
|
||||
@@ -682,6 +687,30 @@ fn processCompleteLine(
|
||||
};
|
||||
}
|
||||
|
||||
/// False only when `line` can't be a record for `workspace_root`: every
|
||||
/// `"workspace_root":"` in it has a value with no escape that differs from
|
||||
/// it, and a value with no escape decodes to its own bytes. That key can't
|
||||
/// occur inside a string, where each quote is escaped. Anything else, such
|
||||
/// as an escaped value or a key written with spaces, goes to the full parse,
|
||||
/// which decides the line as before.
|
||||
fn mayBelongToWorkspace(line: []const u8, workspace_root: []const u8) bool {
|
||||
const key = "\"workspace_root\":\"";
|
||||
var found = false;
|
||||
var i: usize = 0;
|
||||
// Hopping between quotes is a vector scan; a substring search costs
|
||||
// more than the lines it skips.
|
||||
while (std.mem.findScalarPos(u8, line, i, '"')) |quote| {
|
||||
i = quote + 1;
|
||||
if (!std.mem.startsWith(u8, line[quote..], key)) continue;
|
||||
found = true;
|
||||
const start = quote + key.len;
|
||||
const end = std.mem.findAnyPos(u8, line, start, "\"\\") orelse return true;
|
||||
if (line[end] == '\\' or std.mem.eql(u8, line[start..end], workspace_root)) return true;
|
||||
i = end + 1;
|
||||
}
|
||||
return !found;
|
||||
}
|
||||
|
||||
fn repairIncompleteTail(file: std.Io.File) !void {
|
||||
const length = try file.length(io_mod.getIo());
|
||||
if (length == 0) return;
|
||||
@@ -951,6 +980,53 @@ test "reverse load filters workspace bounds results and preserves chronology" {
|
||||
try std.testing.expectEqualStrings("other", other[0].text);
|
||||
}
|
||||
|
||||
test "the workspace prefilter skips only lines that can't be the workspace's" {
|
||||
const root = "/tmp/workspace-a";
|
||||
try std.testing.expect(mayBelongToWorkspace("{\"schema_version\":1,\"timestamp_ms\":1,\"workspace_root\":\"/tmp/workspace-a\",\"text\":\"x\"}", root));
|
||||
try std.testing.expect(!mayBelongToWorkspace("{\"schema_version\":1,\"timestamp_ms\":1,\"workspace_root\":\"/tmp/workspace-b\",\"text\":\"x\"}", root));
|
||||
// A prefix of the root, or the root inside a longer path, is another workspace.
|
||||
try std.testing.expect(!mayBelongToWorkspace("{\"workspace_root\":\"/tmp/workspace\",\"text\":\"x\"}", root));
|
||||
try std.testing.expect(!mayBelongToWorkspace("{\"workspace_root\":\"/tmp/workspace-a/sub\",\"text\":\"x\"}", root));
|
||||
// Text that quotes a matching key doesn't count: its quotes are escaped.
|
||||
try std.testing.expect(!mayBelongToWorkspace("{\"workspace_root\":\"/tmp/workspace-b\",\"text\":\"\\\"workspace_root\\\":\\\"/tmp/workspace-a\\\"\"}", root));
|
||||
// Otherwise the full parse decides: an escaped value, a key with spaces,
|
||||
// a second key, a value with no end, or a line with no key at all.
|
||||
try std.testing.expect(mayBelongToWorkspace("{\"workspace_root\":\"\\/tmp\\/workspace-a\"}", root));
|
||||
try std.testing.expect(mayBelongToWorkspace("{\"workspace_root\" : \"/tmp/workspace-b\"}", root));
|
||||
try std.testing.expect(mayBelongToWorkspace("{\"workspace_root\":\"/tmp/workspace-b\",\"workspace_root\":\"/tmp/workspace-a\"}", root));
|
||||
try std.testing.expect(mayBelongToWorkspace("{\"workspace_root\":\"/tmp/workspace-a", root));
|
||||
try std.testing.expect(mayBelongToWorkspace("{malformed}", root));
|
||||
}
|
||||
|
||||
test "reverse load keeps a record whose workspace is written with escapes or spaces" {
|
||||
const alloc = std.testing.allocator;
|
||||
var tmp = std.testing.tmpDir(.{});
|
||||
defer tmp.cleanup();
|
||||
const home = try io_mod.dirRealpathAlloc(alloc, tmp.dir, ".");
|
||||
defer alloc.free(home);
|
||||
|
||||
const plain = try fixtureLine(alloc, 1, "/tmp/workspace-a", "plain");
|
||||
defer alloc.free(plain);
|
||||
const escaped = try fixtureLine(alloc, 2, "\\/tmp\\/workspace-a", "escaped");
|
||||
defer alloc.free(escaped);
|
||||
const other = try fixtureLine(alloc, 3, "/tmp/workspace-b", "other");
|
||||
defer alloc.free(other);
|
||||
const bytes = try std.mem.concat(alloc, u8, &.{
|
||||
plain,
|
||||
escaped,
|
||||
"{\"schema_version\":1, \"timestamp_ms\":4, \"workspace_root\" : \"/tmp/workspace-a\", \"text\":\"spaced\"}\n",
|
||||
other,
|
||||
});
|
||||
defer alloc.free(bytes);
|
||||
try writeFixture(home, bytes);
|
||||
|
||||
var store = try Store.initFromHome(alloc, home);
|
||||
defer store.deinit(alloc);
|
||||
const entries = try store.loadRecentForWorkspace(alloc, "/tmp/workspace-a", 100);
|
||||
defer freeLoadedEntries(alloc, entries);
|
||||
try expectTexts(entries, &.{ "plain", "escaped", "spaced" });
|
||||
}
|
||||
|
||||
test "reverse load scans beyond one mebibyte of newer interleaved workspace records" {
|
||||
const alloc = std.testing.allocator;
|
||||
var tmp = std.testing.tmpDir(.{});
|
||||
|
||||
@@ -561,6 +561,196 @@ pub const ImageBlobReader = struct {
|
||||
}
|
||||
};
|
||||
|
||||
/// An image's name in a libfx host's store: the lowercase hex SHA-256 of its
|
||||
/// bytes, as `ImageAttachment.snapshot_sha256` holds it for an inline image.
|
||||
const image_name_len = 2 * Sha256.digest_length;
|
||||
pub const ImageName = [image_name_len]u8;
|
||||
|
||||
/// `text` as an image name, or null when it is not 64 lowercase hex digits.
|
||||
pub fn imageName(text: []const u8) ?ImageName {
|
||||
if (text.len != image_name_len) return null;
|
||||
for (text) |char| switch (char) {
|
||||
'0'...'9', 'a'...'f' => {},
|
||||
else => return null,
|
||||
};
|
||||
return text[0..image_name_len].*;
|
||||
}
|
||||
|
||||
pub fn imageNameOf(bytes: []const u8) ImageName {
|
||||
var digest: [Sha256.digest_length]u8 = undefined;
|
||||
Sha256.hash(bytes, &digest, .{});
|
||||
return std.fmt.bytesToHex(digest, .lower);
|
||||
}
|
||||
|
||||
/// The images a libfx host's store holds, by name.
|
||||
pub const StoredImages = struct {
|
||||
names: std.array_hash_map.Auto(ImageName, void) = .empty,
|
||||
|
||||
pub fn deinit(self: *StoredImages, alloc: Allocator) void {
|
||||
self.names.deinit(alloc);
|
||||
self.* = undefined;
|
||||
}
|
||||
|
||||
pub fn add(self: *StoredImages, alloc: Allocator, name: ImageName) Allocator.Error!void {
|
||||
try self.names.put(alloc, name, {});
|
||||
}
|
||||
|
||||
/// Whether `image` has bytes the store holds under its name.
|
||||
pub fn holds(self: *const StoredImages, image: types.ImageAttachment) bool {
|
||||
if (image.inline_data == null) return false;
|
||||
const name = imageName(image.snapshot_sha256 orelse return false) orelse return false;
|
||||
return self.names.contains(name);
|
||||
}
|
||||
};
|
||||
|
||||
/// Whether writing `user` names an image instead of holding its bytes, which
|
||||
/// only the naming format can: one `stored` holds, or one the session
|
||||
/// dropped, which keeps only its name with or without a store.
|
||||
pub fn namesImages(user: session.UserTurn, stored: ?*const StoredImages) bool {
|
||||
for (user.images) |image| {
|
||||
if (image.droppedFromSession()) return true;
|
||||
if (stored) |names| if (names.holds(image)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
pub fn turnNamesImages(turn: session.HistoryTurn, stored: ?*const StoredImages) bool {
|
||||
return switch (turn) {
|
||||
.assistant => |entry| namesImages(entry.user, stored) or executionNamesImages(entry.execution, .{}, stored),
|
||||
.interrupted => |entry| namesImages(entry.user, stored) or executionNamesImages(entry.execution, .{}, stored),
|
||||
.compacted_summary => false,
|
||||
};
|
||||
}
|
||||
|
||||
/// Whether writing the items of `execution` after `from` names a tool image:
|
||||
/// one the host stored, when `stored` is given, or one the session dropped.
|
||||
pub fn executionNamesImages(execution: session.ExecutionMemory, from: ExecutionCounts, stored: ?*const StoredImages) bool {
|
||||
for (execution.tool_steps[from.tool_steps..]) |step| for (step.tool_results) |result| {
|
||||
if (toolImagesNameImages(result.tool_images, stored)) return true;
|
||||
};
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Whether writing `images` names one: one the host stored, when `stored` is
|
||||
/// given, or one the session dropped.
|
||||
pub fn toolImagesNameImages(images: []const types.ToolImage, stored: ?*const StoredImages) bool {
|
||||
for (images) |image| {
|
||||
if (image.droppedFromSession() or (stored != null and image.name != null)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Whether writing `checkpoint` names an image; with `delta`, only the
|
||||
/// execution items after it, which is all a delta progress writes.
|
||||
pub fn checkpointNamesImages(checkpoint: RecoveryCheckpoint, delta: ?ExecutionCounts, stored: ?*const StoredImages) bool {
|
||||
if (delta == null and namesImages(checkpoint.user, stored)) return true;
|
||||
return executionNamesImages(checkpoint.execution, delta orelse .{}, stored);
|
||||
}
|
||||
|
||||
/// The images a libfx session with an image store keeps: its newest, at most
|
||||
/// `max_images` of them holding at most `max_bytes`. An open stages these from
|
||||
/// the host's store, with the open turn's, which the window leaves out. Older
|
||||
/// images leave the session and keep only their names, so a session never
|
||||
/// grows past what it can reopen.
|
||||
pub const ImageWindow = struct {
|
||||
max_images: usize = 4096,
|
||||
max_bytes: usize = 512 * 1024 * 1024,
|
||||
|
||||
/// How many of the oldest images leave the window. `sizes` holds each
|
||||
/// image's bytes, oldest first, or null for one that already left, which
|
||||
/// keeps it and every older one out. Once an image does not fit, it and
|
||||
/// every older one leave.
|
||||
pub fn oldestOutside(self: ImageWindow, sizes: []const ?usize) usize {
|
||||
var kept: usize = 0;
|
||||
var bytes: usize = 0;
|
||||
var start = sizes.len;
|
||||
while (start > 0) : (start -= 1) {
|
||||
const size = sizes[start - 1] orelse break;
|
||||
if (kept == self.max_images or size > self.max_bytes - bytes) break;
|
||||
kept += 1;
|
||||
bytes += size;
|
||||
}
|
||||
return start;
|
||||
}
|
||||
};
|
||||
|
||||
/// Writes images the host stored as `inline_ref` names. Every other image
|
||||
/// keeps its bytes: in `blobs`, a libfx kernel checkpoint's blob section,
|
||||
/// when given, else inline.
|
||||
const ImageNames = struct {
|
||||
stored: *const StoredImages,
|
||||
blobs: ?ImageBlobs = null,
|
||||
};
|
||||
|
||||
/// Images a libfx host staged from its store for an open, by name. Owns the
|
||||
/// bytes.
|
||||
pub const StagedImages = struct {
|
||||
images: std.array_hash_map.Auto(ImageName, []u8) = .empty,
|
||||
bytes: usize = 0,
|
||||
|
||||
pub fn deinit(self: *StagedImages, alloc: Allocator) void {
|
||||
for (self.images.values()) |bytes| alloc.free(bytes);
|
||||
self.images.deinit(alloc);
|
||||
self.* = undefined;
|
||||
}
|
||||
|
||||
/// Stages `bytes`, `alloc`-owned, as `name`. A staged image must be
|
||||
/// exactly the bytes its name is the hash of. On success it owns `bytes`,
|
||||
/// freeing them when it already holds the name; on error the caller
|
||||
/// still owns them.
|
||||
pub fn add(self: *StagedImages, alloc: Allocator, name: ImageName, bytes: []u8) error{ OutOfMemory, ImageNameMismatch }!void {
|
||||
if (!std.mem.eql(u8, &imageNameOf(bytes), &name)) return error.ImageNameMismatch;
|
||||
if (self.images.contains(name)) {
|
||||
alloc.free(bytes);
|
||||
return;
|
||||
}
|
||||
try self.images.put(alloc, name, bytes);
|
||||
self.bytes += bytes.len;
|
||||
}
|
||||
|
||||
pub fn clear(self: *StagedImages, alloc: Allocator) void {
|
||||
for (self.images.values()) |bytes| alloc.free(bytes);
|
||||
self.images.clearRetainingCapacity();
|
||||
self.bytes = 0;
|
||||
}
|
||||
};
|
||||
|
||||
/// Resolves `inline_ref` names to staged bytes while a libfx journal or
|
||||
/// checkpoint is read. A name it cannot resolve is recorded in `missing` with
|
||||
/// the size its record gives, and parses as an image without bytes, so one
|
||||
/// read names every image an open needs; a caller that needs any of them
|
||||
/// discards what it read.
|
||||
pub const ImageResolver = struct {
|
||||
alloc: Allocator,
|
||||
staged: *const StagedImages,
|
||||
missing: std.array_hash_map.Auto(ImageName, usize) = .empty,
|
||||
/// The names it resolved, which the store therefore holds.
|
||||
resolved: std.array_hash_map.Auto(ImageName, void) = .empty,
|
||||
|
||||
pub fn deinit(self: *ImageResolver) void {
|
||||
self.missing.deinit(self.alloc);
|
||||
self.resolved.deinit(self.alloc);
|
||||
self.* = undefined;
|
||||
}
|
||||
|
||||
/// A copy of the bytes named `name`, owned by `alloc`, or null when none
|
||||
/// were staged. `size` is the image's size as its record gives it.
|
||||
fn resolve(self: *ImageResolver, alloc: Allocator, name: ImageName, size: usize) Allocator.Error!?[]u8 {
|
||||
const bytes = self.staged.images.get(name) orelse {
|
||||
try self.missing.put(self.alloc, name, size);
|
||||
return null;
|
||||
};
|
||||
try self.resolved.put(self.alloc, name, {});
|
||||
return try alloc.dupe(u8, bytes);
|
||||
}
|
||||
};
|
||||
|
||||
/// Where a parse finds image bytes kept outside the image object.
|
||||
pub const ImageSources = struct {
|
||||
blobs: ?*ImageBlobReader = null,
|
||||
names: ?*ImageResolver = null,
|
||||
};
|
||||
|
||||
pub fn writeHistoryTurn(writer: *std.Io.Writer, turn: session.HistoryTurn) !void {
|
||||
return writeHistoryTurnTo(writer, turn, {});
|
||||
}
|
||||
@@ -575,8 +765,15 @@ pub fn writeHistoryTurnWithImageBlobs(
|
||||
return writeHistoryTurnTo(writer, turn, image_blobs);
|
||||
}
|
||||
|
||||
/// `image_blobs` is `void` to embed inline image bytes or `ImageBlobs` to
|
||||
/// collect them. The choice is comptime so durable writers keep their error set.
|
||||
/// Like `writeHistoryTurn`, but names the images `names.stored` holds
|
||||
/// instead of writing their bytes.
|
||||
pub fn writeHistoryTurnNamingImages(writer: *std.Io.Writer, turn: session.HistoryTurn, names: ImageNames) !void {
|
||||
return writeHistoryTurnTo(writer, turn, names);
|
||||
}
|
||||
|
||||
/// `image_blobs` is `void` to embed inline image bytes, `ImageBlobs` to
|
||||
/// collect them, or `ImageNames` to name stored ones. The choice is comptime
|
||||
/// so durable writers keep their error set.
|
||||
fn writeHistoryTurnTo(writer: *std.Io.Writer, turn: session.HistoryTurn, image_blobs: anytype) !void {
|
||||
switch (turn) {
|
||||
.compacted_summary => |entry| {
|
||||
@@ -596,7 +793,7 @@ fn writeHistoryTurnTo(writer: *std.Io.Writer, turn: session.HistoryTurn, image_b
|
||||
try writer.writeAll(",\"provider_replay\":");
|
||||
try std.json.Stringify.value(entry.provider_replay, .{}, writer);
|
||||
try writer.writeAll(",\"execution\":");
|
||||
try writeExecutionMemoryJson(writer, entry.execution, .durable);
|
||||
try writeExecutionMemoryJsonFrom(writer, entry.execution, .durable, .{}, image_blobs);
|
||||
try writer.writeByte('}');
|
||||
},
|
||||
.interrupted => |entry| {
|
||||
@@ -623,7 +820,7 @@ fn writeHistoryTurnTo(writer: *std.Io.Writer, turn: session.HistoryTurn, image_b
|
||||
}
|
||||
if (hasDurableExecutionMemory(entry.execution)) {
|
||||
try writer.writeAll(",\"execution\":");
|
||||
try writeExecutionMemoryJson(writer, entry.execution, .durable);
|
||||
try writeExecutionMemoryJsonFrom(writer, entry.execution, .durable, .{}, image_blobs);
|
||||
}
|
||||
if (entry.cancelled_command) |presentation| {
|
||||
try writer.writeAll(",\"cancelled_command\":");
|
||||
@@ -667,6 +864,16 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
image_blobs: ?*ImageBlobReader,
|
||||
) !session.HistoryTurn {
|
||||
return parseHistoryTurnWithImages(alloc, value, .{ .blobs = image_blobs });
|
||||
}
|
||||
|
||||
/// Like `parseHistoryTurn`, but reads the image bytes a libfx journal or
|
||||
/// checkpoint keeps outside each image from `sources`.
|
||||
pub fn parseHistoryTurnWithImages(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
sources: ImageSources,
|
||||
) !session.HistoryTurn {
|
||||
const kind = try requireString(try requireObject(value), "kind");
|
||||
if (std.mem.eql(u8, kind, "compacted_summary")) {
|
||||
@@ -724,11 +931,11 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
if (std.mem.eql(u8, kind, "assistant")) {
|
||||
const shape = try exactVariantObject(value, &.{ "kind", "user", "assistant", "execution" }, &.{ "kind", "user", "assistant", "execution", "provider_replay" });
|
||||
const object = shape.object;
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, image_blobs);
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeUserTurn(alloc, user);
|
||||
const assistant = try parseRequiredDurableBytes(alloc, object, "assistant");
|
||||
errdefer mem_utils.free(alloc, assistant);
|
||||
const execution = try parseExecutionMemory(alloc, object.get("execution") orelse return error.InvalidSessionFormat);
|
||||
const execution = try parseExecutionMemoryWith(alloc, object.get("execution") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeExecutionMemory(alloc, execution);
|
||||
const replay = try parseProviderReplay(alloc, object.get("provider_replay") orelse .null);
|
||||
return .{ .assistant = .{
|
||||
@@ -745,7 +952,7 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
&.{ "kind", "user", "log_path", "expect_url", "url", "background_record_id", "assistant", "execution" },
|
||||
);
|
||||
const object = shape.object;
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, image_blobs);
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeUserTurn(alloc, user);
|
||||
const legacy_assistant = if (shape.extended)
|
||||
try parseOptionalDurableBytes(alloc, object.get("assistant") orelse return error.InvalidSessionFormat)
|
||||
@@ -753,7 +960,7 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
null;
|
||||
defer if (legacy_assistant) |owned| mem_utils.free(alloc, owned);
|
||||
const execution = if (shape.extended)
|
||||
try parseExecutionMemory(alloc, object.get("execution") orelse return error.InvalidSessionFormat)
|
||||
try parseExecutionMemoryWith(alloc, object.get("execution") orelse return error.InvalidSessionFormat, sources)
|
||||
else
|
||||
session.ExecutionMemory{};
|
||||
errdefer session.freeExecutionMemory(alloc, execution);
|
||||
@@ -788,7 +995,7 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
has_terminal_reason,
|
||||
has_cancellation_origin,
|
||||
);
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, image_blobs);
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeUserTurn(alloc, user);
|
||||
const assistant = try parseOptionalDurableBytes(alloc, object.get("assistant") orelse return error.InvalidSessionFormat);
|
||||
errdefer if (assistant) |owned| mem_utils.free(alloc, owned);
|
||||
@@ -809,7 +1016,7 @@ pub fn parseHistoryTurnWithImageBlobs(
|
||||
else
|
||||
.turn;
|
||||
const execution = if (has_execution)
|
||||
try parseExecutionMemory(alloc, object.get("execution") orelse return error.InvalidSessionFormat)
|
||||
try parseExecutionMemoryWith(alloc, object.get("execution") orelse return error.InvalidSessionFormat, sources)
|
||||
else
|
||||
session.ExecutionMemory{};
|
||||
errdefer session.freeExecutionMemory(alloc, execution);
|
||||
@@ -1068,7 +1275,13 @@ fn parsePermissionState(
|
||||
}
|
||||
|
||||
pub fn writeRecoveryCheckpoint(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint) !void {
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, null);
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, null, {});
|
||||
}
|
||||
|
||||
/// Like `writeRecoveryCheckpoint`, but names the images `names.stored` holds
|
||||
/// instead of writing their bytes.
|
||||
pub fn writeRecoveryCheckpointNamingImages(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint, names: ImageNames) !void {
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, null, names);
|
||||
}
|
||||
|
||||
/// How many items of each list in an open turn's execution an earlier
|
||||
@@ -1083,10 +1296,16 @@ pub const ExecutionCounts = struct {
|
||||
/// execution items after `from`: what a progress adds to the one before it,
|
||||
/// which holds the rest.
|
||||
pub fn writeRecoveryCheckpointDelta(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint, from: ExecutionCounts) !void {
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, from);
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, from, {});
|
||||
}
|
||||
|
||||
fn writeRecoveryCheckpointFrom(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint, delta: ?ExecutionCounts) !void {
|
||||
/// Like `writeRecoveryCheckpointDelta`, but names the tool images the host
|
||||
/// stored instead of writing their bytes.
|
||||
pub fn writeRecoveryCheckpointDeltaNamingImages(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint, from: ExecutionCounts, names: ImageNames) !void {
|
||||
return writeRecoveryCheckpointFrom(writer, checkpoint, from, names);
|
||||
}
|
||||
|
||||
fn writeRecoveryCheckpointFrom(writer: *std.Io.Writer, checkpoint: RecoveryCheckpoint, delta: ?ExecutionCounts, image_mode: anytype) !void {
|
||||
if (checkpoint.disposition == .history_only) return error.InvalidDurableField;
|
||||
if (delta) |from| {
|
||||
const execution = checkpoint.execution;
|
||||
@@ -1100,12 +1319,12 @@ fn writeRecoveryCheckpointFrom(writer: *std.Io.Writer, checkpoint: RecoveryCheck
|
||||
});
|
||||
if (delta == null) {
|
||||
try writer.writeAll(",\"user\":");
|
||||
try writeUserTurn(writer, checkpoint.user, {});
|
||||
try writeUserTurn(writer, checkpoint.user, image_mode);
|
||||
}
|
||||
try writer.writeAll(",\"assistant_source\":");
|
||||
try writeDurableBytes(writer, checkpoint.assistant_source);
|
||||
try writer.writeAll(",\"execution\":");
|
||||
try writeExecutionMemoryJsonFrom(writer, checkpoint.execution, .durable, delta orelse .{});
|
||||
try writeExecutionMemoryJsonFrom(writer, checkpoint.execution, .durable, delta orelse .{}, image_mode);
|
||||
try writer.writeAll(",\"cause\":");
|
||||
try writeJsonString(writer, @tagName(checkpoint.cause));
|
||||
try writer.writeAll(",\"action\":");
|
||||
@@ -1346,6 +1565,12 @@ fn decodeStateImpl(alloc: Allocator, source: *std.Io.Reader, limits: DecodeLimit
|
||||
}
|
||||
|
||||
pub fn parseRecoveryCheckpoint(alloc: Allocator, value: std.json.Value) !RecoveryCheckpoint {
|
||||
return parseRecoveryCheckpointWithImages(alloc, value, .{});
|
||||
}
|
||||
|
||||
/// Like `parseRecoveryCheckpoint`, but reads the image bytes a libfx journal
|
||||
/// keeps outside each image from `sources`.
|
||||
pub fn parseRecoveryCheckpointWithImages(alloc: Allocator, value: std.json.Value, sources: ImageSources) !RecoveryCheckpoint {
|
||||
const raw_object = try requireObject(value);
|
||||
const version = try requireU64(raw_object, "version");
|
||||
const legacy_route = raw_object.contains("route_identity");
|
||||
@@ -1384,11 +1609,11 @@ pub fn parseRecoveryCheckpoint(alloc: Allocator, value: std.json.Value) !Recover
|
||||
}),
|
||||
else => return error.InvalidDurableField,
|
||||
};
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, null);
|
||||
const user = try parseUserTurn(alloc, object.get("user") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeUserTurn(alloc, user);
|
||||
const assistant_source = try parseDurableBytes(alloc, object.get("assistant_source") orelse return error.InvalidSessionFormat);
|
||||
errdefer alloc.free(assistant_source);
|
||||
const execution = try parseExecutionMemory(alloc, object.get("execution") orelse return error.InvalidSessionFormat);
|
||||
const execution = try parseExecutionMemoryWith(alloc, object.get("execution") orelse return error.InvalidSessionFormat, sources);
|
||||
errdefer session.freeExecutionMemory(alloc, execution);
|
||||
const authority = if (legacy_route) TurnAuthority{
|
||||
.provider = .gateway,
|
||||
@@ -1544,10 +1769,15 @@ fn parseTurnAuthority(alloc: Allocator, value: std.json.Value) !TurnAuthority {
|
||||
}
|
||||
|
||||
fn writeUserTurn(writer: *std.Io.Writer, user: session.UserTurn, image_blobs: anytype) !void {
|
||||
const collect_blobs = comptime switch (@TypeOf(image_blobs)) {
|
||||
void => false,
|
||||
ImageBlobs => true,
|
||||
else => @compileError("image_blobs must be void or ImageBlobs"),
|
||||
const Mode = @TypeOf(image_blobs);
|
||||
comptime switch (Mode) {
|
||||
void, ImageBlobs, ImageNames => {},
|
||||
else => @compileError("image_blobs must be void, ImageBlobs or ImageNames"),
|
||||
};
|
||||
const blobs: ?ImageBlobs = switch (Mode) {
|
||||
ImageBlobs => image_blobs,
|
||||
ImageNames => image_blobs.blobs,
|
||||
else => null,
|
||||
};
|
||||
try writer.writeAll("{\"text\":");
|
||||
try writeDurableBytes(writer, user.text);
|
||||
@@ -1563,15 +1793,24 @@ fn writeUserTurn(writer: *std.Io.Writer, user: session.UserTurn, image_blobs: an
|
||||
try writer.writeAll(",\"snapshot_sha256\":");
|
||||
try writeOptionalDurableBytes(writer, image.snapshot_sha256);
|
||||
// Inline image bytes appear only for sessions without a filesystem
|
||||
// snapshot backend, so durable sessions keep their exact format.
|
||||
// snapshot backend, so durable sessions keep their exact format. An
|
||||
// image the host stored is named instead; its name is the digest,
|
||||
// and its size lets an open choose the images it keeps unread.
|
||||
if (image.inline_data) |inline_data| {
|
||||
if (collect_blobs) {
|
||||
try writer.print(",\"inline_blob\":{d}", .{image_blobs.items.items.len});
|
||||
try image_blobs.items.append(image_blobs.alloc, inline_data);
|
||||
if (Mode == ImageNames and image_blobs.stored.holds(image)) {
|
||||
try writer.writeAll(",\"inline_ref\":");
|
||||
try writeJsonString(writer, image.snapshot_sha256.?);
|
||||
try writer.print(",\"inline_size\":{d}", .{inline_data.len});
|
||||
} else if (blobs) |collected| {
|
||||
try writer.print(",\"inline_blob\":{d}", .{collected.items.items.len});
|
||||
try collected.items.append(collected.alloc, inline_data);
|
||||
} else {
|
||||
try writer.writeAll(",\"inline_data\":");
|
||||
try writeDurableBytes(writer, inline_data);
|
||||
}
|
||||
} else if (image.droppedFromSession()) {
|
||||
// An image the session dropped keeps only its digest.
|
||||
try writer.writeAll(",\"dropped\":true");
|
||||
}
|
||||
if (image.source_ref) |source_ref| {
|
||||
if (!image_data.validSourceRef(source_ref)) return error.InvalidSessionFormat;
|
||||
@@ -1608,16 +1847,19 @@ pub noinline fn writeExecutionMemoryJson(
|
||||
execution: session.ExecutionMemory,
|
||||
format: ExecutionMemoryJsonFormat,
|
||||
) !void {
|
||||
return writeExecutionMemoryJsonFrom(writer, execution, format, .{});
|
||||
return writeExecutionMemoryJsonFrom(writer, execution, format, .{}, {});
|
||||
}
|
||||
|
||||
/// `writeExecutionMemoryJson` with each list starting after `from` items.
|
||||
/// The caller checks that every list holds at least that many.
|
||||
/// The caller checks that every list holds at least that many. `image_mode`
|
||||
/// is as for `writeHistoryTurnTo`; only `ImageNames` changes how tool images
|
||||
/// are written.
|
||||
fn writeExecutionMemoryJsonFrom(
|
||||
writer: *std.Io.Writer,
|
||||
execution: session.ExecutionMemory,
|
||||
format: ExecutionMemoryJsonFormat,
|
||||
from: ExecutionCounts,
|
||||
image_mode: anytype,
|
||||
) !void {
|
||||
std.debug.assert(from.tool_steps <= execution.tool_steps.len and from.files <= execution.files.len and from.steering <= execution.steering.len);
|
||||
try writer.writeAll(if (format == .durable)
|
||||
@@ -1626,7 +1868,7 @@ fn writeExecutionMemoryJsonFrom(
|
||||
"{\"schema_version\":3,\"tool_steps\":[");
|
||||
for (execution.tool_steps[from.tool_steps..], 0..) |step, i| {
|
||||
if (i > 0) try writer.writeByte(',');
|
||||
try writeExecutionToolStep(writer, step, format);
|
||||
try writeExecutionToolStep(writer, step, format, image_mode);
|
||||
}
|
||||
try writer.writeAll("],\"files\":[");
|
||||
for (execution.files[from.files..], 0..) |file, i| {
|
||||
@@ -1651,7 +1893,7 @@ fn writeExecutionMemoryJsonFrom(
|
||||
try writer.writeByte('}');
|
||||
}
|
||||
|
||||
fn writeExecutionToolStep(writer: *std.Io.Writer, step: session.ToolExecutionStep, format: ExecutionMemoryJsonFormat) !void {
|
||||
fn writeExecutionToolStep(writer: *std.Io.Writer, step: session.ToolExecutionStep, format: ExecutionMemoryJsonFormat, image_mode: anytype) !void {
|
||||
try writer.writeAll("{\"assistant\":");
|
||||
try writeOptionalExecutionBytes(writer, step.assistant, format);
|
||||
if (format == .durable) {
|
||||
@@ -1666,7 +1908,7 @@ fn writeExecutionToolStep(writer: *std.Io.Writer, step: session.ToolExecutionSte
|
||||
try writer.writeAll("],\"tool_results\":[");
|
||||
for (step.tool_results, 0..) |result, result_index| {
|
||||
if (result_index > 0) try writer.writeByte(',');
|
||||
try writeExecutionToolResult(writer, result, format);
|
||||
try writeExecutionToolResult(writer, result, format, image_mode);
|
||||
}
|
||||
try writer.writeAll("]}");
|
||||
}
|
||||
@@ -1740,7 +1982,7 @@ pub fn executionHashes(alloc: Allocator, checkpoint: RecoveryCheckpoint) !Execut
|
||||
for (results) |*result| result.created_at_ms = 0;
|
||||
var stable = step;
|
||||
stable.tool_results = results;
|
||||
try writeExecutionToolStep(&out.writer, stable, .durable);
|
||||
try writeExecutionToolStep(&out.writer, stable, .durable, {});
|
||||
hash.* = std.hash.Wyhash.hash(0, out.written());
|
||||
}
|
||||
const files = try alloc.alloc(u64, execution.files.len);
|
||||
@@ -1824,6 +2066,7 @@ fn writeExecutionToolResult(
|
||||
writer: *std.Io.Writer,
|
||||
result: session.PersistedToolResult,
|
||||
format: ExecutionMemoryJsonFormat,
|
||||
image_mode: anytype,
|
||||
) !void {
|
||||
if (format == .durable and
|
||||
result.review_feedback and
|
||||
@@ -1891,7 +2134,7 @@ fn writeExecutionToolResult(
|
||||
try writeDurableBytes(writer, handle);
|
||||
} else if (result.tool_images.len > 0) {
|
||||
try writer.writeAll(",\"tool_images\":");
|
||||
writePersistedToolImages(writer, result.tool_images) catch |err| switch (err) {
|
||||
writeToolImagesTo(writer, result.tool_images, @TypeOf(image_mode) == ImageNames) catch |err| switch (err) {
|
||||
error.InvalidSourceRef, error.InvalidImage => return error.InvalidSessionFormat,
|
||||
else => |write_err| return write_err,
|
||||
};
|
||||
@@ -2095,7 +2338,7 @@ fn writeExecutionFileEvidence(
|
||||
try writer.writeByte('}');
|
||||
}
|
||||
|
||||
fn parseUserTurn(alloc: Allocator, value: std.json.Value, image_blobs: ?*ImageBlobReader) !session.UserTurn {
|
||||
fn parseUserTurn(alloc: Allocator, value: std.json.Value, sources: ImageSources) !session.UserTurn {
|
||||
const source = try requireObject(value);
|
||||
const object = if (source.count() == 2)
|
||||
try exactObject(value, &.{ "text", "images" })
|
||||
@@ -2137,7 +2380,8 @@ fn parseUserTurn(alloc: Allocator, value: std.json.Value, image_blobs: ?*ImageBl
|
||||
image.get("snapshot_sha256") orelse .null,
|
||||
);
|
||||
errdefer if (snapshot_sha256) |sha256_bytes| mem_utils.free(alloc, sha256_bytes);
|
||||
const inline_data = try parseInlineImageBytes(alloc, image, image_blobs);
|
||||
const inline_image = try parseInlineImageBytes(alloc, image, sources);
|
||||
const inline_data = inline_image.bytes;
|
||||
errdefer if (inline_data) |inline_bytes| mem_utils.free(alloc, inline_bytes);
|
||||
const source_ref = if (image.get("source_ref")) |_| source: {
|
||||
const source_bytes = try requireString(image, "source_ref");
|
||||
@@ -2145,7 +2389,22 @@ fn parseUserTurn(alloc: Allocator, value: std.json.Value, image_blobs: ?*ImageBl
|
||||
break :source try alloc.dupe(u8, source_bytes);
|
||||
} else null;
|
||||
errdefer if (source_ref) |source_bytes| alloc.free(source_bytes);
|
||||
if (inline_data) |inline_bytes| {
|
||||
const dropped = if (image.get("dropped")) |dropped_value| switch (dropped_value) {
|
||||
.bool => |flag| if (flag) true else return error.InvalidSessionFormat,
|
||||
else => return error.InvalidSessionFormat,
|
||||
} else false;
|
||||
if (dropped) {
|
||||
// An image the session dropped keeps only the digest naming it.
|
||||
if (inline_image.name != null or inline_data != null or snapshot_path != null) return error.InvalidSessionFormat;
|
||||
_ = imageName(snapshot_sha256 orelse return error.InvalidSessionFormat) orelse return error.InvalidSessionFormat;
|
||||
} else if (inline_image.name) |name| {
|
||||
// A named image is the host's stored copy of exactly the bytes
|
||||
// its digest names; with none staged, `sources` records it.
|
||||
if (snapshot_path != null) return error.InvalidSessionFormat;
|
||||
const digest = snapshot_sha256 orelse return error.InvalidSessionFormat;
|
||||
if (!std.mem.eql(u8, digest, &name)) return error.InvalidSessionFormat;
|
||||
if (inline_data) |inline_bytes| try validateInlineImageBytes(inline_bytes, media_type);
|
||||
} else if (inline_data) |inline_bytes| {
|
||||
// Inline images carry their own bytes: no snapshot file, but the
|
||||
// digest stays so request-time verification is unchanged.
|
||||
if (snapshot_path != null or snapshot_sha256 == null) return error.InvalidSessionFormat;
|
||||
@@ -2165,19 +2424,43 @@ fn parseUserTurn(alloc: Allocator, value: std.json.Value, image_blobs: ?*ImageBl
|
||||
return .{ .text = text, .images = images, .work_id = work_id };
|
||||
}
|
||||
|
||||
/// Reads an inline image's bytes from `inline_data` or, inside a libfx kernel
|
||||
/// checkpoint, from the raw blob named by `inline_blob`. Caller owns the result.
|
||||
/// An image's bytes as a parse found them, and its name when the object
|
||||
/// named it. A named image with no bytes was not staged.
|
||||
const InlineImage = struct {
|
||||
bytes: ?[]u8 = null,
|
||||
name: ?ImageName = null,
|
||||
};
|
||||
|
||||
/// Reads an inline image's bytes from `inline_data`, from the raw blob of a
|
||||
/// libfx kernel checkpoint named by `inline_blob`, or from the host's staged
|
||||
/// copy named by `inline_ref`. Caller owns the bytes.
|
||||
fn parseInlineImageBytes(
|
||||
alloc: Allocator,
|
||||
image: std.json.ObjectMap,
|
||||
image_blobs: ?*ImageBlobReader,
|
||||
) !?[]u8 {
|
||||
sources: ImageSources,
|
||||
) !InlineImage {
|
||||
if (image.get("inline_ref")) |ref_value| {
|
||||
const resolver = sources.names orelse return error.InvalidSessionFormat;
|
||||
if (image.get("inline_data") != null or image.get("inline_blob") != null or ref_value != .string) {
|
||||
return error.InvalidSessionFormat;
|
||||
}
|
||||
const name = imageName(ref_value.string) orelse return error.InvalidSessionFormat;
|
||||
const size = try requireUsize(image, "inline_size");
|
||||
if (size == 0 or size > image_attachments.max_image_bytes) return error.InvalidSessionFormat;
|
||||
const bytes = try resolver.resolve(alloc, name, size);
|
||||
if (bytes) |staged| if (staged.len != size) {
|
||||
alloc.free(staged);
|
||||
return error.InvalidSessionFormat;
|
||||
};
|
||||
return .{ .bytes = bytes, .name = name };
|
||||
}
|
||||
if (image.get("inline_size") != null) return error.InvalidSessionFormat;
|
||||
const blob_value = image.get("inline_blob") orelse
|
||||
return parseOptionalDurableBytes(alloc, image.get("inline_data") orelse .null);
|
||||
const blobs = image_blobs orelse return error.InvalidSessionFormat;
|
||||
return .{ .bytes = try parseOptionalDurableBytes(alloc, image.get("inline_data") orelse .null) };
|
||||
const blobs = sources.blobs orelse return error.InvalidSessionFormat;
|
||||
if (image.get("inline_data") != null or blob_value != .integer) return error.InvalidSessionFormat;
|
||||
const index = std.math.cast(usize, blob_value.integer) orelse return error.InvalidSessionFormat;
|
||||
return try alloc.dupe(u8, try blobs.take(index));
|
||||
return .{ .bytes = try alloc.dupe(u8, try blobs.take(index)) };
|
||||
}
|
||||
|
||||
fn imageAttachmentObject(value: std.json.Value) !std.json.ObjectMap {
|
||||
@@ -2197,6 +2480,9 @@ fn imageAttachmentObject(value: std.json.Value) !std.json.ObjectMap {
|
||||
std.mem.eql(u8, entry.key_ptr.*, "snapshot_sha256") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "inline_data") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "inline_blob") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "inline_ref") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "inline_size") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "dropped") or
|
||||
std.mem.eql(u8, entry.key_ptr.*, "source_ref"))
|
||||
{
|
||||
continue;
|
||||
@@ -2222,6 +2508,11 @@ fn validateInlineImageBytes(bytes: []const u8, media_type: []const u8) !void {
|
||||
}
|
||||
|
||||
fn parseExecutionMemory(alloc: Allocator, value: std.json.Value) !session.ExecutionMemory {
|
||||
return parseExecutionMemoryWith(alloc, value, .{});
|
||||
}
|
||||
|
||||
/// `parseExecutionMemory`, reading named tool images from `sources`.
|
||||
fn parseExecutionMemoryWith(alloc: Allocator, value: std.json.Value, sources: ImageSources) !session.ExecutionMemory {
|
||||
const source = try requireObject(value);
|
||||
const schema_version = try requireU64(source, "schema_version");
|
||||
const object = switch (schema_version) {
|
||||
@@ -2235,6 +2526,7 @@ fn parseExecutionMemory(alloc: Allocator, value: std.json.Value) !session.Execut
|
||||
alloc,
|
||||
object.get("tool_steps") orelse return error.InvalidSessionFormat,
|
||||
schema_version,
|
||||
sources,
|
||||
);
|
||||
errdefer session.freeExecutionMemory(alloc, .{ .tool_steps = tool_steps });
|
||||
const files = try parseFiles(alloc, object.get("files") orelse return error.InvalidSessionFormat);
|
||||
@@ -2352,6 +2644,7 @@ fn parseToolSteps(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
schema_version: u64,
|
||||
sources: ImageSources,
|
||||
) ![]session.ToolExecutionStep {
|
||||
if (value != .array) return error.InvalidSessionFormat;
|
||||
if (value.array.items.len == 0) return &.{};
|
||||
@@ -2377,6 +2670,7 @@ fn parseToolSteps(
|
||||
alloc,
|
||||
object.get("tool_results") orelse return error.InvalidSessionFormat,
|
||||
schema_version,
|
||||
sources,
|
||||
);
|
||||
errdefer session.freePersistedToolResults(alloc, tool_results);
|
||||
try session.repairPersistedToolArguments(alloc, tool_calls, tool_results, .schema_v3);
|
||||
@@ -2480,6 +2774,7 @@ fn parseToolResults(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
schema_version: u64,
|
||||
sources: ImageSources,
|
||||
) ![]session.PersistedToolResult {
|
||||
if (value != .array) return error.InvalidSessionFormat;
|
||||
if (value.array.items.len == 0) return &.{};
|
||||
@@ -2503,7 +2798,7 @@ fn parseToolResults(
|
||||
}
|
||||
};
|
||||
for (value.array.items, 0..) |result_value, i| {
|
||||
results[i] = try parseToolResult(alloc, result_value, schema_version);
|
||||
results[i] = try parseToolResultWith(alloc, result_value, schema_version, sources);
|
||||
parsed_count += 1;
|
||||
}
|
||||
return results;
|
||||
@@ -2513,6 +2808,15 @@ fn parseToolResult(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
schema_version: u64,
|
||||
) !session.PersistedToolResult {
|
||||
return parseToolResultWith(alloc, value, schema_version, .{});
|
||||
}
|
||||
|
||||
fn parseToolResultWith(
|
||||
alloc: Allocator,
|
||||
value: std.json.Value,
|
||||
schema_version: u64,
|
||||
sources: ImageSources,
|
||||
) !session.PersistedToolResult {
|
||||
const v1_keys = &.{
|
||||
"tool_call_id",
|
||||
@@ -2630,7 +2934,7 @@ fn parseToolResult(
|
||||
result_shape.extended,
|
||||
);
|
||||
errdefer presentations.deinit(alloc);
|
||||
var images = try parseToolResultImages(alloc, object, &text.output);
|
||||
var images = try parseToolResultImages(alloc, object, &text.output, sources);
|
||||
errdefer images.deinit(alloc);
|
||||
return .{
|
||||
.tool_images = images.items,
|
||||
@@ -2775,16 +3079,41 @@ fn parseToolResultPresentations(
|
||||
|
||||
/// Writes typed image blocks with persistence-owned source_ref spelling.
|
||||
pub fn writePersistedToolImages(writer: *std.Io.Writer, images: []const types.ToolImage) !void {
|
||||
return writeToolImagesTo(writer, images, false);
|
||||
}
|
||||
|
||||
/// `writePersistedToolImages`, but with `naming`, an image the host stored is
|
||||
/// written as its name (`inline_ref`) and decoded size instead of its bytes.
|
||||
/// An image the session dropped is written as its name alone either way.
|
||||
/// Writes a tool result's `images` as a list: with `naming`, each the host
|
||||
/// stored by its name, and every other with its bytes; one the session
|
||||
/// dropped by its name alone. `parseToolImages` reads it back.
|
||||
pub fn writeToolImages(writer: *std.Io.Writer, images: []const types.ToolImage, naming: bool) !void {
|
||||
return writeToolImagesTo(writer, images, naming);
|
||||
}
|
||||
|
||||
fn writeToolImagesTo(writer: *std.Io.Writer, images: []const types.ToolImage, naming: bool) !void {
|
||||
try writer.writeByte('[');
|
||||
for (images, 0..) |image, index| {
|
||||
if (index > 0) try writer.writeByte(',');
|
||||
if (image.source_ref) |source_ref| {
|
||||
if (!image_data.validSourceRef(source_ref)) return error.InvalidSourceRef;
|
||||
} else if (image.data.len == 0) return error.InvalidImage;
|
||||
} else if (image.data.len == 0 and image.name == null) return error.InvalidImage;
|
||||
try writer.writeAll("{\"type\":\"image\",\"mimeType\":");
|
||||
try writeJsonString(writer, image.mime_type);
|
||||
try writer.writeAll(",\"data\":");
|
||||
try writeJsonString(writer, image.data);
|
||||
if (image.droppedFromSession()) {
|
||||
try writer.writeAll(",\"inline_ref\":");
|
||||
try writeJsonString(writer, &image.name.?);
|
||||
try writer.writeAll(",\"dropped\":true");
|
||||
} else if (naming and image.name != null) {
|
||||
const size = std.base64.standard.Decoder.calcSizeForSlice(image.data) catch return error.InvalidImage;
|
||||
try writer.writeAll(",\"inline_ref\":");
|
||||
try writeJsonString(writer, &image.name.?);
|
||||
try writer.print(",\"inline_size\":{d}", .{size});
|
||||
} else {
|
||||
try writer.writeAll(",\"data\":");
|
||||
try writeJsonString(writer, image.data);
|
||||
}
|
||||
if (image.source_ref) |source_ref| {
|
||||
try writer.writeAll(",\"source_ref\":");
|
||||
try writeJsonString(writer, source_ref);
|
||||
@@ -2825,6 +3154,106 @@ pub fn parsePersistedToolImages(alloc: Allocator, content: []const std.json.Valu
|
||||
return image_data.parseToolImages(alloc, normalized[0..initialized]);
|
||||
}
|
||||
|
||||
/// `parsePersistedToolImages`, but an image written by name (`inline_ref`)
|
||||
/// reads its bytes from the host's staged copy in `sources.names`. One the
|
||||
/// session dropped, or one an open has yet to stage, parses with its name and
|
||||
/// without bytes; the open's `missing` tells the two apart. Caller owns the
|
||||
/// images.
|
||||
/// Reads a list `writeToolImages` wrote. Caller owns the images. A name
|
||||
/// `sources.names` has no bytes for reads as an image without bytes, and the
|
||||
/// resolver records it as missing.
|
||||
pub fn parseToolImages(alloc: Allocator, content: []const std.json.Value, sources: ImageSources) ![]types.ToolImage {
|
||||
return parseToolImagesWith(alloc, content, sources);
|
||||
}
|
||||
|
||||
fn parseToolImagesWith(alloc: Allocator, content: []const std.json.Value, sources: ImageSources) ![]types.ToolImage {
|
||||
for (content) |item| {
|
||||
if (item == .object and item.object.contains("inline_ref")) break;
|
||||
} else return parsePersistedToolImages(alloc, content);
|
||||
if (content.len > image_data.max_tool_images) return error.ImageLimitExceeded;
|
||||
var images: std.ArrayList(types.ToolImage) = .empty;
|
||||
defer {
|
||||
for (images.items) |image| freeToolImage(alloc, image);
|
||||
images.deinit(alloc);
|
||||
}
|
||||
for (content) |item| {
|
||||
if (item != .object) return error.InvalidImage;
|
||||
if (!item.object.contains("inline_ref")) {
|
||||
// An image written inline keeps its own admission; an unsupported
|
||||
// one parses as none, which the caller notes.
|
||||
const inline_images = try parsePersistedToolImages(alloc, &[_]std.json.Value{item});
|
||||
images.ensureUnusedCapacity(alloc, inline_images.len) catch |err| {
|
||||
types.freeToolImages(alloc, inline_images);
|
||||
return err;
|
||||
};
|
||||
images.appendSliceAssumeCapacity(inline_images);
|
||||
alloc.free(inline_images);
|
||||
continue;
|
||||
}
|
||||
const image = try parseNamedToolImage(alloc, item.object, sources);
|
||||
images.append(alloc, image) catch |err| {
|
||||
freeToolImage(alloc, image);
|
||||
return err;
|
||||
};
|
||||
}
|
||||
return images.toOwnedSlice(alloc);
|
||||
}
|
||||
|
||||
fn parseNamedToolImage(alloc: Allocator, object: std.json.ObjectMap, sources: ImageSources) !types.ToolImage {
|
||||
var keys = object.iterator();
|
||||
while (keys.next()) |entry| {
|
||||
for ([_][]const u8{ "type", "mimeType", "inline_ref", "inline_size", "dropped", "source_ref" }) |known| {
|
||||
if (std.mem.eql(u8, entry.key_ptr.*, known)) break;
|
||||
} else return error.InvalidSessionFormat;
|
||||
}
|
||||
if (!std.mem.eql(u8, try requireString(object, "type"), "image")) return error.InvalidSessionFormat;
|
||||
const mime_type = try requireString(object, "mimeType");
|
||||
if (!image_data.supportedMediaType(mime_type)) return error.InvalidSessionFormat;
|
||||
const name = imageName(try requireString(object, "inline_ref")) orelse return error.InvalidSessionFormat;
|
||||
const source_ref: ?[]const u8 = if (object.get("source_ref")) |value| ref: {
|
||||
if (value != .string or !image_data.validSourceRef(value.string)) return error.InvalidSourceRef;
|
||||
break :ref value.string;
|
||||
} else null;
|
||||
if (object.get("dropped")) |value| {
|
||||
if (value != .bool or !value.bool or object.contains("inline_size")) return error.InvalidSessionFormat;
|
||||
return ownedToolImage(alloc, try alloc.dupe(u8, ""), mime_type, source_ref, name);
|
||||
}
|
||||
const resolver = sources.names orelse return error.InvalidSessionFormat;
|
||||
const size = try requireUsize(object, "inline_size");
|
||||
if (size == 0 or size > image_data.max_encoded_image_bytes) return error.InvalidSessionFormat;
|
||||
const resolved = try resolver.resolve(alloc, name, size);
|
||||
const bytes = resolved orelse return ownedToolImage(alloc, try alloc.dupe(u8, ""), mime_type, source_ref, name);
|
||||
defer alloc.free(bytes);
|
||||
if (bytes.len != size) return error.InvalidSessionFormat;
|
||||
const encoded = try alloc.alloc(u8, std.base64.standard.Encoder.calcSize(bytes.len));
|
||||
_ = std.base64.standard.Encoder.encode(encoded, bytes);
|
||||
// The same admission as an image written inline.
|
||||
image_data.validateImage(alloc, encoded, mime_type) catch |err| {
|
||||
alloc.free(encoded);
|
||||
return err;
|
||||
};
|
||||
return ownedToolImage(alloc, encoded, mime_type, source_ref, name);
|
||||
}
|
||||
|
||||
/// A tool image owning `data`, which it takes, and copies of the rest.
|
||||
fn ownedToolImage(alloc: Allocator, data: []u8, mime_type: []const u8, source_ref: ?[]const u8, name: ImageName) Allocator.Error!types.ToolImage {
|
||||
errdefer alloc.free(data);
|
||||
const owned_type = try alloc.dupe(u8, mime_type);
|
||||
errdefer alloc.free(owned_type);
|
||||
return .{
|
||||
.data = data,
|
||||
.mime_type = owned_type,
|
||||
.source_ref = if (source_ref) |value| try alloc.dupe(u8, value) else null,
|
||||
.name = name,
|
||||
};
|
||||
}
|
||||
|
||||
fn freeToolImage(alloc: Allocator, image: types.ToolImage) void {
|
||||
alloc.free(image.data);
|
||||
alloc.free(image.mime_type);
|
||||
if (image.source_ref) |source_ref| alloc.free(source_ref);
|
||||
}
|
||||
|
||||
const ParsedToolResultImages = struct {
|
||||
handle: ?[]u8,
|
||||
items: []types.ToolImage,
|
||||
@@ -2840,6 +3269,7 @@ fn parseToolResultImages(
|
||||
alloc: Allocator,
|
||||
object: std.json.ObjectMap,
|
||||
output: *[]u8,
|
||||
sources: ImageSources,
|
||||
) !ParsedToolResultImages {
|
||||
const handle = if (object.get("tool_image_handle")) |value|
|
||||
try parseOptionalDurableBytes(alloc, value)
|
||||
@@ -2848,9 +3278,11 @@ fn parseToolResultImages(
|
||||
errdefer if (handle) |owned| mem_utils.free(alloc, owned);
|
||||
const items = if (object.get("tool_images")) |images| items: {
|
||||
if (images != .array) return error.InvalidSessionFormat;
|
||||
const parsed = parsePersistedToolImages(alloc, images.array.items) catch |err| {
|
||||
const parsed = parseToolImagesWith(alloc, images.array.items, sources) catch |err| {
|
||||
if (err == error.OutOfMemory) return error.OutOfMemory;
|
||||
if (err == error.InvalidSourceRef) return error.InvalidSessionFormat;
|
||||
// A named image that cannot be read is a format error, not an
|
||||
// image to leave out: the record says the store holds it.
|
||||
if (err == error.InvalidSourceRef or err == error.InvalidSessionFormat) return error.InvalidSessionFormat;
|
||||
const notice = try alloc.print(
|
||||
"{s}\n[Saved tool image unavailable: {s}]",
|
||||
.{ output.*, @errorName(err) },
|
||||
@@ -3552,7 +3984,7 @@ test "durable tool image validation preserves the session format error contract"
|
||||
.provider_native = false,
|
||||
.created_at_ms = 0,
|
||||
.tool_images = &images,
|
||||
}, .durable));
|
||||
}, .durable, {}));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4054,7 +4486,7 @@ test "review feedback durable codec rejects invalid provenance and unknown field
|
||||
result.review_feedback = true;
|
||||
var encoded: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer encoded.deinit();
|
||||
try writeExecutionToolResult(&encoded.writer, result, .durable);
|
||||
try writeExecutionToolResult(&encoded.writer, result, .durable, {});
|
||||
var parsed = try std.json.parseFromSlice(std.json.Value, alloc, encoded.written(), .{});
|
||||
defer parsed.deinit();
|
||||
const cases = [_]struct { status: []const u8, provider_native: bool, marker: std.json.Value }{
|
||||
@@ -4080,7 +4512,7 @@ test "review feedback durable codec rejects invalid provenance and unknown field
|
||||
invalid.provider_native = native;
|
||||
try std.testing.expectError(
|
||||
error.InvalidSessionFormat,
|
||||
writeExecutionToolResult(&encoded.writer, invalid, .durable),
|
||||
writeExecutionToolResult(&encoded.writer, invalid, .durable, {}),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -4092,7 +4524,7 @@ test "review feedback defaults old durable results without inspecting output" {
|
||||
result.output = @constCast("Security review held this action.");
|
||||
var encoded: std.Io.Writer.Allocating = .init(alloc);
|
||||
defer encoded.deinit();
|
||||
try writeExecutionToolResult(&encoded.writer, result, .durable);
|
||||
try writeExecutionToolResult(&encoded.writer, result, .durable, {});
|
||||
var parsed = try std.json.parseFromSlice(std.json.Value, alloc, encoded.written(), .{});
|
||||
defer parsed.deinit();
|
||||
try std.testing.expect(parsed.value.object.swapRemove("review_feedback"));
|
||||
@@ -5088,6 +5520,42 @@ test "inline image bytes round trip through the history turn codec" {
|
||||
}
|
||||
}
|
||||
|
||||
test "the image window keeps the newest images that fit, and drops every older one" {
|
||||
const window: ImageWindow = .{ .max_images = 3, .max_bytes = 100 };
|
||||
try std.testing.expectEqual(@as(usize, 0), window.oldestOutside(&.{}));
|
||||
try std.testing.expectEqual(@as(usize, 0), window.oldestOutside(&.{ 10, 20, 30 }));
|
||||
// Three images at most: the oldest of four leaves.
|
||||
try std.testing.expectEqual(@as(usize, 1), window.oldestOutside(&.{ 10, 20, 30, 40 }));
|
||||
// 100 bytes at most: 60 does not fit beside 40 and 30, so it and the
|
||||
// 5 before it leave, though 5 alone would fit.
|
||||
try std.testing.expectEqual(@as(usize, 2), window.oldestOutside(&.{ 5, 60, 30, 40 }));
|
||||
// An image already dropped keeps every older one out.
|
||||
try std.testing.expectEqual(@as(usize, 2), window.oldestOutside(&.{ 10, null, 30, 40 }));
|
||||
// An image larger than the whole window leaves too.
|
||||
try std.testing.expectEqual(@as(usize, 1), window.oldestOutside(&.{101}));
|
||||
}
|
||||
|
||||
test "a dropped image parses only as a name without bytes" {
|
||||
const alloc = std.testing.allocator;
|
||||
const digest = "\"" ++ text_utils.repeat("a", image_name_len) ++ "\"";
|
||||
const prefix = "{\"text\":\"[Image #1]\",\"images\":[{\"id\":1,\"path\":\"inline://image-1\",\"media_type\":\"image/png\",\"snapshot_path\":null,\"snapshot_sha256\":";
|
||||
var parsed = try std.json.parseFromSlice(std.json.Value, alloc, prefix ++ digest ++ ",\"dropped\":true}]}", .{});
|
||||
defer parsed.deinit();
|
||||
const user = try parseUserTurn(alloc, parsed.value, .{});
|
||||
defer types.freeUserTurn(alloc, user);
|
||||
try std.testing.expect(user.images[0].droppedFromSession());
|
||||
for ([_][]const u8{
|
||||
prefix ++ digest ++ ",\"dropped\":false}]}",
|
||||
prefix ++ "\"not-a-digest\",\"dropped\":true}]}",
|
||||
prefix ++ "null,\"dropped\":true}]}",
|
||||
prefix ++ digest ++ ",\"dropped\":true,\"inline_data\":\"iVBORw0KGgo=\"}]}",
|
||||
}) |json| {
|
||||
var invalid = try std.json.parseFromSlice(std.json.Value, alloc, json, .{});
|
||||
defer invalid.deinit();
|
||||
try std.testing.expectError(error.InvalidSessionFormat, parseUserTurn(alloc, invalid.value, .{}));
|
||||
}
|
||||
}
|
||||
|
||||
test "inline image parsing rejects missing digest and mismatched bytes" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\ninline-bytes";
|
||||
@@ -5646,13 +6114,13 @@ test "persisted image source refs reject malformed metadata and preserve the byt
|
||||
for (invalid) |value| {
|
||||
user_ref.* = value;
|
||||
tool_ref.* = value;
|
||||
try std.testing.expectError(error.InvalidSessionFormat, parseUserTurn(alloc, user.value, null));
|
||||
try std.testing.expectError(error.InvalidSessionFormat, parseUserTurn(alloc, user.value, .{}));
|
||||
try std.testing.expectError(error.InvalidSourceRef, parsePersistedToolImages(alloc, tools.value.array.items));
|
||||
}
|
||||
const boundary = text_utils.repeat("é", 256);
|
||||
user_ref.* = .{ .string = boundary };
|
||||
tool_ref.* = .{ .string = boundary };
|
||||
const decoded_user = try parseUserTurn(alloc, user.value, null);
|
||||
const decoded_user = try parseUserTurn(alloc, user.value, .{});
|
||||
defer session.freeUserTurn(alloc, decoded_user);
|
||||
const decoded_tools = try parsePersistedToolImages(alloc, tools.value.array.items);
|
||||
defer types.freeToolImages(alloc, decoded_tools);
|
||||
|
||||
@@ -246,6 +246,10 @@ pub const ToolLifecycleEvent = union(enum) {
|
||||
activity_kind: ToolActivityKind,
|
||||
arguments_json: ?[]const u8 = null,
|
||||
place_after_current_transcript: bool = false,
|
||||
/// The call's `ToolCall.resolved_display_target`, for hosts that
|
||||
/// title a call when it starts. Borrowed for this push only; copies
|
||||
/// omit it.
|
||||
resolved_display_target: ?[]const u8 = null,
|
||||
},
|
||||
progress: struct {
|
||||
id: ToolLifecycleId,
|
||||
@@ -957,6 +961,10 @@ pub const ToolCall = struct {
|
||||
provenance: ToolExecutionProvenance = .fx_local,
|
||||
/// Borrowed only for this action. Copies and durable/provider encodings omit it.
|
||||
resolved_skill: ?*const skill_contract.PreparedSkill = null,
|
||||
/// What this action works on, resolved from earlier results, such as the
|
||||
/// command whose saved output a `read_tool_result` call reads. Borrowed
|
||||
/// only for this action. Copies and durable/provider encodings omit it.
|
||||
resolved_display_target: ?[]const u8 = null,
|
||||
};
|
||||
|
||||
pub const WebSearchProgress = union(enum) {
|
||||
@@ -1310,11 +1318,23 @@ pub const ExecutionMemory = struct {
|
||||
|
||||
/// Image bytes returned by a tool; never a grant to read a user file.
|
||||
pub const ToolImage = struct {
|
||||
/// Owned encoded bytes; empty only when a valid source_ref is present.
|
||||
/// Owned encoded bytes; empty only when a valid source_ref is present, or
|
||||
/// when the session dropped the image (`droppedFromSession`).
|
||||
data: []u8,
|
||||
mime_type: []u8,
|
||||
/// Owned opaque host reference, never filesystem or network authority.
|
||||
source_ref: ?[]u8 = null,
|
||||
/// The image's name in a libfx host's store, the lowercase hex SHA-256
|
||||
/// of its decoded bytes, when the host stored them before it sent the
|
||||
/// result. Writers that name stored images write this instead of `data`.
|
||||
name: ?[64]u8 = null,
|
||||
|
||||
/// Whether the session no longer holds this image: a libfx session with
|
||||
/// an image store keeps only its newest images, and an older one keeps
|
||||
/// its name, without its bytes.
|
||||
pub fn droppedFromSession(self: ToolImage) bool {
|
||||
return self.data.len == 0 and self.name != null;
|
||||
}
|
||||
};
|
||||
|
||||
pub fn freeToolImages(alloc: std.mem.Allocator, images: []const ToolImage) void {
|
||||
@@ -1346,6 +1366,7 @@ pub fn dupeToolImages(alloc: std.mem.Allocator, images: []const ToolImage) ![]To
|
||||
.data = data,
|
||||
.mime_type = mime_type,
|
||||
.source_ref = if (image.source_ref) |source_ref| try alloc.dupe(u8, source_ref) else null,
|
||||
.name = image.name,
|
||||
};
|
||||
initialized += 1;
|
||||
}
|
||||
@@ -1498,6 +1519,13 @@ pub const ImageAttachment = struct {
|
||||
inline_data: ?[]u8 = null,
|
||||
/// Owned opaque host reference, never filesystem or network authority.
|
||||
source_ref: ?[]u8 = null,
|
||||
|
||||
/// Whether the session no longer holds this image: a libfx session with
|
||||
/// an image store keeps only its newest images, and an older one keeps
|
||||
/// its digest, which names it in the host's store, without its bytes.
|
||||
pub fn droppedFromSession(self: ImageAttachment) bool {
|
||||
return self.inline_data == null and self.snapshot_path == null and self.snapshot_sha256 != null;
|
||||
}
|
||||
};
|
||||
|
||||
pub const UserTurn = struct {
|
||||
@@ -3084,12 +3112,19 @@ test "dupeToolCall preserves argument integrity" {
|
||||
try std.testing.expectEqual(ToolArgumentIntegrity.malformed_json, copy.argument_integrity);
|
||||
}
|
||||
|
||||
test "dupeToolCall drops action scoped skill bindings" {
|
||||
test "dupeToolCall drops action scoped bindings" {
|
||||
const skill: skill_contract.PreparedSkill = .{ .skill = .{ .name = "workflow", .description = "", .path = "/skills/workflow", .source = .global_fx } };
|
||||
const source: ToolCall = .{ .id = "skill", .name = "skill", .arguments_json = "{\"location\":\"/skills/workflow\"}", .resolved_skill = &skill };
|
||||
const source: ToolCall = .{
|
||||
.id = "skill",
|
||||
.name = "skill",
|
||||
.arguments_json = "{\"location\":\"/skills/workflow\"}",
|
||||
.resolved_skill = &skill,
|
||||
.resolved_display_target = "more output from zig build test",
|
||||
};
|
||||
const copy = try dupeToolCall(std.testing.allocator, source);
|
||||
defer freeToolCall(std.testing.allocator, copy);
|
||||
try std.testing.expect(copy.resolved_skill == null);
|
||||
try std.testing.expect(copy.resolved_display_target == null);
|
||||
try std.testing.expectEqualStrings(source.arguments_json, copy.arguments_json);
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
const std = @import("std");
|
||||
const result_store = @import("../session/result_store.zig");
|
||||
const command_replay_store = @import("../session/command_replay_store.zig");
|
||||
const compactor = @import("../compactor/compactor.zig");
|
||||
const text_utils = @import("../shared/text_utils.zig");
|
||||
const tool_dispatch = @import("tool_dispatch.zig");
|
||||
|
||||
const Allocator = std.mem.Allocator;
|
||||
|
||||
/// Longest search text a row repeats, after terminal-safe encoding.
|
||||
const max_query_label_bytes = 60;
|
||||
|
||||
/// A handle as `validate` resolves it: without surrounding whitespace, plus
|
||||
/// the `.txt` a stored result or image handle may omit.
|
||||
const HandleNormalization = struct {
|
||||
trimmed: []const u8,
|
||||
suffix: []const u8,
|
||||
};
|
||||
|
||||
pub fn classifyHandleNormalization(handle: []const u8) HandleNormalization {
|
||||
const trimmed = std.mem.trim(u8, handle, " \t\r\n");
|
||||
const suffix = if ((std.mem.startsWith(u8, trimmed, "result-") or result_store.isImageHandle(trimmed)) and
|
||||
std.mem.findScalar(u8, trimmed, '.') == null)
|
||||
".txt"
|
||||
else
|
||||
"";
|
||||
return .{ .trimmed = trimmed, .suffix = suffix };
|
||||
}
|
||||
|
||||
/// What a call asks for, read the way the tool's `decode` and `validate` read it,
|
||||
/// so its row can say what the agent is reading.
|
||||
const RequestView = struct {
|
||||
kind: Kind,
|
||||
/// The handle without surrounding whitespace. Empty for a history search.
|
||||
handle: []const u8 = "",
|
||||
query: []const u8 = "",
|
||||
|
||||
const Kind = enum { range, query, images, record, history_search };
|
||||
};
|
||||
|
||||
fn requestView(args: std.json.ObjectMap) ?RequestView {
|
||||
const object = if (args.get("request")) |request| switch (request) {
|
||||
.object => |value| value,
|
||||
else => return null,
|
||||
} else args;
|
||||
if (object.get("search") != null) return .{ .kind = .history_search };
|
||||
const handle = switch (object.get("handle") orelse return null) {
|
||||
.string => |value| std.mem.trim(u8, value, " \t\r\n"),
|
||||
else => return null,
|
||||
};
|
||||
if (handle.len == 0) return null;
|
||||
var record_buffer: compactor.RecordFileBuffer = undefined;
|
||||
if (compactor.recordFile(&record_buffer, handle) != null) return .{ .kind = .record, .handle = handle };
|
||||
if (result_store.isImageHandle(handle)) return .{ .kind = .images, .handle = handle };
|
||||
if (object.get("query")) |query| {
|
||||
if (query == .string and query.string.len > 0) {
|
||||
return .{ .kind = .query, .handle = handle, .query = query.string };
|
||||
}
|
||||
}
|
||||
return .{ .kind = .range, .handle = handle };
|
||||
}
|
||||
|
||||
fn parseArguments(alloc: Allocator, arguments_json: []const u8) Allocator.Error!?std.json.Parsed(std.json.Value) {
|
||||
const parsed = std.json.parseFromSlice(std.json.Value, alloc, arguments_json, .{}) catch |err| return switch (err) {
|
||||
error.OutOfMemory => error.OutOfMemory,
|
||||
else => null,
|
||||
};
|
||||
if (parsed.value != .object) {
|
||||
parsed.deinit();
|
||||
return null;
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
fn outputNoun(handle: []const u8, command: []const u8, other: []const u8) []const u8 {
|
||||
return if (command_replay_store.isReplayHandle(handle)) command else other;
|
||||
}
|
||||
|
||||
fn recordNoun(kind: u8) []const u8 {
|
||||
return switch (std.ascii.toUpper(kind)) {
|
||||
'M' => "turn",
|
||||
'T' => "tool call",
|
||||
else => "compaction",
|
||||
};
|
||||
}
|
||||
|
||||
fn readPresentation(
|
||||
action_label: []const u8,
|
||||
completed_action_label: []const u8,
|
||||
label_arg_default: []const u8,
|
||||
) tool_dispatch.CallPresentation {
|
||||
return .{
|
||||
.activity_kind = .read,
|
||||
.action_label = action_label,
|
||||
.completed_action_label = completed_action_label,
|
||||
.label_arg_kind = .none,
|
||||
.label_arg_default = label_arg_default,
|
||||
};
|
||||
}
|
||||
|
||||
/// Labels a call by what it reads instead of one shared "tool result" label.
|
||||
/// The resolved row text from `displayTarget` replaces the default object.
|
||||
pub fn presentation(args: std.json.ObjectMap) ?tool_dispatch.CallPresentation {
|
||||
const view = requestView(args) orelse return null;
|
||||
return switch (view.kind) {
|
||||
.range => readPresentation("Reading", "Read", outputNoun(view.handle, "more command output", "more output")),
|
||||
.query => readPresentation("Searching", "Searched", outputNoun(view.handle, "command output", "saved output")),
|
||||
.images => readPresentation("Reloading", "Reloaded", "saved images"),
|
||||
.record => readPresentation("Opening", "Opened", "earlier conversation"),
|
||||
.history_search => readPresentation("Searching", "Searched", "earlier conversation"),
|
||||
};
|
||||
}
|
||||
|
||||
/// The saved output a call reads, with the suffix `validate` restores, or null
|
||||
/// for a history search, a saved record, or arguments `decode` rejects. The
|
||||
/// caller owns the returned handle.
|
||||
pub fn requestedHandle(alloc: Allocator, arguments_json: []const u8) Allocator.Error!?[]u8 {
|
||||
var parsed = (try parseArguments(alloc, arguments_json)) orelse return null;
|
||||
defer parsed.deinit();
|
||||
const view = requestView(parsed.value.object) orelse return null;
|
||||
switch (view.kind) {
|
||||
.record, .history_search => return null,
|
||||
.range, .query, .images => {},
|
||||
}
|
||||
const normalization = classifyHandleNormalization(view.handle);
|
||||
return try std.mem.concat(alloc, u8, &.{ normalization.trimmed, normalization.suffix });
|
||||
}
|
||||
|
||||
/// The row text for what a call reads: the output of the earlier call that
|
||||
/// `source` names, the text it searches for, or the saved record it opens.
|
||||
/// Returns null when the default object from `presentation` already says it.
|
||||
/// `source` must be terminal-safe. The caller owns the returned text.
|
||||
pub fn displayTarget(
|
||||
alloc: Allocator,
|
||||
arguments_json: []const u8,
|
||||
source: ?[]const u8,
|
||||
) Allocator.Error!?[]u8 {
|
||||
var parsed = (try parseArguments(alloc, arguments_json)) orelse return null;
|
||||
defer parsed.deinit();
|
||||
const view = requestView(parsed.value.object) orelse return null;
|
||||
switch (view.kind) {
|
||||
.range => {
|
||||
const name = source orelse return null;
|
||||
return try alloc.print("more output from {s}", .{name});
|
||||
},
|
||||
.query => {
|
||||
var query = try text_utils.encodeTerminalSafe(alloc, view.query, max_query_label_bytes);
|
||||
defer query.deinit(alloc);
|
||||
if (source) |name| return try alloc.print("{s} output for \"{s}\"", .{ name, query.bytes });
|
||||
return try alloc.print("{s} for \"{s}\"", .{
|
||||
outputNoun(view.handle, "command output", "saved output"),
|
||||
query.bytes,
|
||||
});
|
||||
},
|
||||
// `requestView` admits only a record ID: one letter, then digits.
|
||||
.record => return try alloc.print("earlier {s} {c}{s}", .{
|
||||
recordNoun(view.handle[0]),
|
||||
std.ascii.toUpper(view.handle[0]),
|
||||
view.handle[1..],
|
||||
}),
|
||||
.images, .history_search => return null,
|
||||
}
|
||||
}
|
||||
|
||||
fn expectPresentation(arguments_json: []const u8, action_label: []const u8, completed_action_label: []const u8, label_arg_default: []const u8) !void {
|
||||
var parsed = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, arguments_json, .{});
|
||||
defer parsed.deinit();
|
||||
const resolved = presentation(parsed.value.object) orelse return error.TestExpectedPresentation;
|
||||
try std.testing.expectEqual(tool_dispatch.LabelArgKind.none, resolved.label_arg_kind);
|
||||
try std.testing.expectEqualStrings(action_label, resolved.action_label);
|
||||
try std.testing.expectEqualStrings(completed_action_label, resolved.completed_action_label);
|
||||
try std.testing.expectEqualStrings(label_arg_default, resolved.label_arg_default);
|
||||
}
|
||||
|
||||
fn expectDisplayTarget(arguments_json: []const u8, source: ?[]const u8, expected: ?[]const u8) !void {
|
||||
const alloc = std.testing.allocator;
|
||||
const target = try displayTarget(alloc, arguments_json, source);
|
||||
defer if (target) |value| alloc.free(value);
|
||||
if (expected) |text| {
|
||||
try std.testing.expectEqualStrings(text, target orelse return error.TestExpectedDisplayTarget);
|
||||
} else {
|
||||
try std.testing.expect(target == null);
|
||||
}
|
||||
}
|
||||
|
||||
test "read_tool_result rows name what each request reads" {
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
try expectPresentation("{\"request\":{\"handle\":\"" ++ replay ++ "\"}}", "Reading", "Read", "more command output");
|
||||
try expectPresentation("{\"handle\":\"result-x.txt\",\"start_byte\":9}", "Reading", "Read", "more output");
|
||||
try expectPresentation("{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}", "Searching", "Searched", "command output");
|
||||
try expectPresentation("{\"handle\":\"result-x\",\"query\":\"error:\"}", "Searching", "Searched", "saved output");
|
||||
// An empty query reads a range, as `decode` treats it.
|
||||
try expectPresentation("{\"handle\":\"result-x\",\"query\":\"\"}", "Reading", "Read", "more output");
|
||||
try expectPresentation("{\"request\":{\"handle\":\"image-result-abc\"}}", "Reloading", "Reloaded", "saved images");
|
||||
try expectPresentation("{\"request\":{\"handle\":\" t5 \"}}", "Opening", "Opened", "earlier conversation");
|
||||
try expectPresentation("{\"request\":{\"search\":[\"retry budget\"]}}", "Searching", "Searched", "earlier conversation");
|
||||
|
||||
var parsed = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, "{\"request\":\"handle\"}", .{});
|
||||
defer parsed.deinit();
|
||||
try std.testing.expect(presentation(parsed.value.object) == null);
|
||||
}
|
||||
|
||||
test "read_tool_result display targets name the source, query, and record" {
|
||||
const replay = "fx-command-replay-0123.bin";
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"" ++ replay ++ "\"}}", "zig build test", "more output from zig build test");
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"" ++ replay ++ "\"}}", null, null);
|
||||
try expectDisplayTarget(
|
||||
"{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}",
|
||||
"zig build test",
|
||||
"zig build test output for \"error:\"",
|
||||
);
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"" ++ replay ++ "\",\"query\":\"error:\"}}", null, "command output for \"error:\"");
|
||||
try expectDisplayTarget("{\"handle\":\"result-x\",\"query\":\"needle\"}", null, "saved output for \"needle\"");
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"t5\"}}", "ignored", "earlier tool call T5");
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"M12\"}}", null, "earlier turn M12");
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"l2\"}}", null, "earlier compaction L2");
|
||||
try expectDisplayTarget("{\"request\":{\"handle\":\"image-result-abc\"}}", "vision", null);
|
||||
try expectDisplayTarget("{\"request\":{\"search\":\"retry\"}}", null, null);
|
||||
try expectDisplayTarget("not json", "zig build test", null);
|
||||
}
|
||||
|
||||
test "read_tool_result display targets keep search text terminal-safe and short" {
|
||||
const alloc = std.testing.allocator;
|
||||
const target = (try displayTarget(alloc, "{\"handle\":\"result-x\",\"query\":\"\\u001b[31mred" ++ text_utils.repeat("x", 120) ++ "\"}", null)).?;
|
||||
defer alloc.free(target);
|
||||
try std.testing.expect(std.mem.findScalar(u8, target, 0x1b) == null);
|
||||
try std.testing.expect(target.len < "saved output for \"\"".len + max_query_label_bytes + 1);
|
||||
}
|
||||
|
||||
test "read_tool_result requested handles match what validate opens" {
|
||||
const alloc = std.testing.allocator;
|
||||
const restored = (try requestedHandle(alloc, "{\"request\":{\"handle\":\" result-shell-abc \"}}")).?;
|
||||
defer alloc.free(restored);
|
||||
try std.testing.expectEqualStrings("result-shell-abc.txt", restored);
|
||||
const replay = (try requestedHandle(alloc, "{\"handle\":\"fx-command-replay-0123.bin\",\"query\":\"x\"}")).?;
|
||||
defer alloc.free(replay);
|
||||
try std.testing.expectEqualStrings("fx-command-replay-0123.bin", replay);
|
||||
try std.testing.expect(try requestedHandle(alloc, "{\"request\":{\"handle\":\"T5\"}}") == null);
|
||||
try std.testing.expect(try requestedHandle(alloc, "{\"request\":{\"search\":[\"x\"]}}") == null);
|
||||
try std.testing.expect(try requestedHandle(alloc, "[]") == null);
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
const std = @import("std");
|
||||
const debug_trace = @import("../shared/debug_trace.zig");
|
||||
const image_data = @import("../images/image_data.zig");
|
||||
const types = @import("../shared/types.zig");
|
||||
const tool_dispatch = @import("tool_dispatch.zig");
|
||||
@@ -21,6 +22,7 @@ pub fn parseRichResult(alloc: Allocator, bytes: []const u8, text_limit: usize, i
|
||||
else => return .{ .failure = try alloc.print("Host tool image could not be used: {s}", .{@errorName(err)}) },
|
||||
};
|
||||
errdefer types.freeToolImages(alloc, images);
|
||||
try nameStoredImages(alloc, images, values.array.items);
|
||||
const omitted = images.len != values.array.items.len;
|
||||
const source = if (omitted) try alloc.print("[Unsupported tool images were omitted.]\n{s}", .{text.string}) else text.string;
|
||||
defer if (omitted) alloc.free(source);
|
||||
@@ -28,6 +30,38 @@ pub fn parseRichResult(alloc: Allocator, bytes: []const u8, text_limit: usize, i
|
||||
return .{ .rich = .{ .text = body, .images = images, .is_error = is_error } };
|
||||
}
|
||||
|
||||
/// Names each image a libfx host stored before it sent the result: one whose
|
||||
/// decoded bytes hash to a `stored` mark the result carries. A mark that
|
||||
/// matches no image leaves the images inline.
|
||||
fn nameStoredImages(alloc: Allocator, images: []types.ToolImage, values: []const std.json.Value) Allocator.Error!void {
|
||||
var marks: usize = 0;
|
||||
for (values) |item| {
|
||||
if (item == .object and item.object.get("stored") != null) marks += 1;
|
||||
}
|
||||
if (marks == 0) return;
|
||||
var named: usize = 0;
|
||||
for (images) |*image| {
|
||||
if (image.data.len == 0) continue;
|
||||
const size = std.base64.standard.Decoder.calcSizeForSlice(image.data) catch continue;
|
||||
const bytes = try alloc.alloc(u8, size);
|
||||
defer alloc.free(bytes);
|
||||
std.base64.standard.Decoder.decode(bytes, image.data) catch continue;
|
||||
var digest: [std.crypto.hash.sha2.Sha256.digest_length]u8 = undefined;
|
||||
std.crypto.hash.sha2.Sha256.hash(bytes, &digest, .{});
|
||||
const name = std.fmt.bytesToHex(digest, .lower);
|
||||
for (values) |item| {
|
||||
if (item != .object) continue;
|
||||
const mark = item.object.get("stored") orelse continue;
|
||||
if (mark == .string and std.mem.eql(u8, mark.string, &name)) {
|
||||
image.name = name;
|
||||
named += 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (named < marks) debug_trace.logf("tools", "event=host_tool_images_not_stored marks={d} named={d} reason=name_mismatch", .{ marks, named });
|
||||
}
|
||||
|
||||
/// Replaces binary fields in a caller-owned parsed MCP block with delivery notes.
|
||||
/// Images themselves travel through the typed result, never JSON text.
|
||||
pub fn projectMediaForText(alloc: std.mem.Allocator, content: *std.json.Value) std.mem.Allocator.Error!void {
|
||||
@@ -55,3 +89,31 @@ fn projectMediaBlock(alloc: std.mem.Allocator, item: *std.json.Value) std.mem.Al
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
test "a host tool image is named only when its stored mark is the digest of its bytes" {
|
||||
const alloc = std.testing.allocator;
|
||||
const png = "\x89PNG\r\n\x1a\nhost-tool-shot";
|
||||
var encoded: [64]u8 = undefined;
|
||||
const data = std.base64.standard.Encoder.encode(&encoded, png);
|
||||
var digest: [std.crypto.hash.sha2.Sha256.digest_length]u8 = undefined;
|
||||
std.crypto.hash.sha2.Sha256.hash(png, &digest, .{});
|
||||
const name = std.fmt.bytesToHex(digest, .lower);
|
||||
for ([_]struct { mark: []const u8, named: bool }{
|
||||
.{ .mark = &name, .named = true },
|
||||
.{ .mark = "0000000000000000000000000000000000000000000000000000000000000000", .named = false },
|
||||
}) |case| {
|
||||
const json = try alloc.print("{{\"text\":\"shot\",\"images\":[{{\"type\":\"image\",\"data\":\"{s}\",\"mimeType\":\"image/png\",\"stored\":\"{s}\"}}]}}", .{ data, case.mark });
|
||||
defer alloc.free(json);
|
||||
const result = try parseRichResult(alloc, json, 1024, false);
|
||||
const rich = result.rich;
|
||||
defer {
|
||||
alloc.free(rich.text);
|
||||
types.freeToolImages(alloc, rich.images);
|
||||
}
|
||||
try std.testing.expectEqual(@as(usize, 1), rich.images.len);
|
||||
// A mark that does not match leaves the image inline, never lost.
|
||||
try std.testing.expectEqualStrings(data, rich.images[0].data);
|
||||
try std.testing.expectEqual(case.named, rich.images[0].name != null);
|
||||
if (case.named) try std.testing.expectEqualStrings(&name, &rich.images[0].name.?);
|
||||
}
|
||||
}
|
||||
@@ -672,12 +672,53 @@ pub fn formatPlainAction(alloc: Allocator, input: ToolActionInput) ![]const u8 {
|
||||
return alloc.print("{s} {s}", .{ presentation.action_label, try formatWebSearchActionDetail(scratch, args) });
|
||||
}
|
||||
const value = input.display_target orelse
|
||||
call.resolved_display_target orelse
|
||||
resolvedSkillName(call, presentation) orelse
|
||||
tool_dispatch.presentationLabelValue(presentation, args) orelse
|
||||
presentation.label_arg_default;
|
||||
return alloc.print("{s} {s}", .{ presentation.action_label, value });
|
||||
}
|
||||
|
||||
/// Names an earlier call in the row of a later call that reads its saved
|
||||
/// output: the command it ran, the path or URL it read, or the name of an MCP
|
||||
/// tool. Returns null when the call has none of these, or when its command
|
||||
/// cannot be described, so a label never fails the turn that shows it. The
|
||||
/// text is terminal-safe. The caller owns the returned allocation.
|
||||
pub fn savedOutputSourceLabel(
|
||||
alloc: Allocator,
|
||||
registry: tool_dispatch.Registry,
|
||||
workspace_root: []const u8,
|
||||
call: ToolCall,
|
||||
) Allocator.Error!?[]const u8 {
|
||||
const command_activity = formatRunCommandActivity(alloc, registry, workspace_root, call) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => return null,
|
||||
};
|
||||
if (command_activity) |activity| return activity.detail;
|
||||
const spec = registry.lookup(call.name) orelse {
|
||||
if (isProviderSearchAlias(call.name)) return null;
|
||||
return try encodeSourceLabel(alloc, call.name);
|
||||
};
|
||||
var scratch_state = std.heap.ArenaAllocator.init(alloc);
|
||||
defer mem_utils.deinit_arena(scratch_state);
|
||||
const args = tool_args.parseToolArgsObject(scratch_state.allocator(), call.arguments_json) catch |err| switch (err) {
|
||||
error.OutOfMemory => return error.OutOfMemory,
|
||||
else => return null,
|
||||
};
|
||||
const presentation = tool_dispatch.presentationForArgs(spec.*, args);
|
||||
switch (presentation.label_arg_kind) {
|
||||
.path, .url => {},
|
||||
else => return null,
|
||||
}
|
||||
const value = tool_dispatch.presentationLabelValue(presentation, args) orelse return null;
|
||||
return try encodeSourceLabel(alloc, value);
|
||||
}
|
||||
|
||||
fn encodeSourceLabel(alloc: Allocator, raw: []const u8) ![]const u8 {
|
||||
const encoded = try text_utils.encodeTerminalSafe(alloc, raw, max_run_command_activity_bytes);
|
||||
return encoded.bytes;
|
||||
}
|
||||
|
||||
/// The caller owns the returned allocation and must free it with `alloc`.
|
||||
pub fn formatPermissionLabel(alloc: Allocator, registry: tool_dispatch.Registry, call: ToolCall) ![]const u8 {
|
||||
var scratch_state = std.heap.ArenaAllocator.init(alloc);
|
||||
@@ -1502,3 +1543,44 @@ test "tool presentation frees all formatted output with a normal allocator" {
|
||||
fn expectContains(text: []const u8, needle: []const u8) !void {
|
||||
try std.testing.expect(std.mem.find(u8, text, needle) != null);
|
||||
}
|
||||
|
||||
test "read_tool_result rows use the resolved source and the request kind" {
|
||||
const alloc = std.testing.allocator;
|
||||
const registry = test_builtin_tools.advertisement_set.registry;
|
||||
const page: ToolCall = .{
|
||||
.id = "page",
|
||||
.name = "read_tool_result",
|
||||
.arguments_json = "{\"request\":{\"handle\":\"fx-command-replay-0123.bin\"}}",
|
||||
};
|
||||
const generic = try formatPlainAction(alloc, .{ .tool_registry = registry, .call = page });
|
||||
defer alloc.free(generic);
|
||||
try std.testing.expectEqualStrings("Reading more command output", generic);
|
||||
|
||||
var resolved = page;
|
||||
resolved.resolved_display_target = "more output from zig build test";
|
||||
const named = try formatPlainAction(alloc, .{ .tool_registry = registry, .call = resolved });
|
||||
defer alloc.free(named);
|
||||
try std.testing.expectEqualStrings("Reading more output from zig build test", named);
|
||||
}
|
||||
|
||||
test "saved output sources name commands, paths, URLs, and MCP tools" {
|
||||
const alloc = std.testing.allocator;
|
||||
const registry = test_builtin_tools.advertisement_set.registry;
|
||||
const cases = [_]struct { call: ToolCall, expected: ?[]const u8 }{
|
||||
.{ .call = .{ .id = "run", .name = "shell", .arguments_json = "{\"action\":\"run\",\"command\":\"zig build test\"}" }, .expected = "zig build test" },
|
||||
.{ .call = .{ .id = "file", .name = "read_file", .arguments_json = "{\"path\":\"src/\\u001b[2Jmain.zig\"}" }, .expected = "src/\\x1b[2Jmain.zig" },
|
||||
.{ .call = .{ .id = "fetch", .name = "web_fetch", .arguments_json = "{\"url\":\"https://fx.sh/docs\"}" }, .expected = "https://fx.sh/docs" },
|
||||
.{ .call = .{ .id = "mcp", .name = "mcp_linear_list_issues", .arguments_json = "{}" }, .expected = "mcp_linear_list_issues" },
|
||||
.{ .call = .{ .id = "grep", .name = "grep_files", .arguments_json = "{\"pattern\":\"needle\"}" }, .expected = null },
|
||||
.{ .call = .{ .id = "bad", .name = "read_file", .arguments_json = "not json" }, .expected = null },
|
||||
};
|
||||
for (cases) |case| {
|
||||
const label = try savedOutputSourceLabel(alloc, registry, "/workspace", case.call);
|
||||
defer if (label) |value| alloc.free(value);
|
||||
if (case.expected) |expected| {
|
||||
try std.testing.expectEqualStrings(expected, label orelse return error.TestExpectedSourceLabel);
|
||||
} else {
|
||||
try std.testing.expect(label == null);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4833,6 +4833,8 @@ test "semantic code block preserves indentation on wrapped continuation rows" {
|
||||
}
|
||||
|
||||
test {
|
||||
_ = @import("core/agent/runtime/journal.zig");
|
||||
_ = @import("acp/libfx_journal.zig");
|
||||
_ = @import("mcp/mcp.zig");
|
||||
_ = @import("core/mcp_host/runtime.zig");
|
||||
_ = @import("napi_fetch_state.zig");
|
||||
|
||||
@@ -297,6 +297,16 @@ pub const Engine = struct {
|
||||
return s.saveCredential(w);
|
||||
}
|
||||
|
||||
/// Gives server `name` the credential `saveCredential` wrote, as
|
||||
/// `Config.auth.saved` does, for a host that reads it after `init`.
|
||||
/// `saved` is copied. Call it before anything starts the server: a
|
||||
/// running server keeps the sign-in it has.
|
||||
pub fn restoreCredential(e: *Engine, name: []const u8, saved: []const u8) Error!void {
|
||||
const slot = try e.find(name);
|
||||
std.debug.assert(slot.server == null);
|
||||
slot.config.auth.saved = try slot.arena.allocator().dupe(u8, saved);
|
||||
}
|
||||
|
||||
/// Wakes a `next` that waits, from any thread: it returns null early, so
|
||||
/// the host can act on what it was woken for.
|
||||
pub fn wake(e: *Engine) void {
|
||||
@@ -637,6 +647,29 @@ test "wake from another thread ends a waiting next early" {
|
||||
try std.testing.expect(e.now() - started < 5_000);
|
||||
}
|
||||
|
||||
test "a credential restored after init is the one the server starts with" {
|
||||
const Fake = struct {
|
||||
fn allow(_: *anyopaque, _: []const u8, _: []const u8, _: ?[]const u8) bool {
|
||||
return true;
|
||||
}
|
||||
fn log(_: *anyopaque, _: []const u8, _: []const u8) void {}
|
||||
};
|
||||
var context: u8 = 0;
|
||||
var e: Engine = undefined;
|
||||
const servers = [_]Config{.{ .name = "web", .transport = .{ .http = .{ .url = "https://web.example.invalid/mcp" } } }};
|
||||
try e.init(std.testing.io, std.testing.allocator, .{ .context = &context, .allow = Fake.allow, .log = Fake.log }, .{
|
||||
.client = .{ .info = .{ .name = "test", .version = "0" } },
|
||||
}, &servers);
|
||||
defer e.deinit();
|
||||
|
||||
var saved = "{\"access\":\"w1\"}".*;
|
||||
try e.restoreCredential("web", &saved);
|
||||
// The engine keeps its own copy.
|
||||
@memset(&saved, 0);
|
||||
try std.testing.expectEqualStrings("{\"access\":\"w1\"}", e.slots.items[0].config.auth.saved.?);
|
||||
try std.testing.expectError(error.UnknownServer, e.restoreCredential("nope", "{}"));
|
||||
}
|
||||
|
||||
test "a stdio server's directory is part of its config, so a new one restarts it" {
|
||||
const argv = [_][]const u8{ "node", "s.js" };
|
||||
const a: Config = .{ .name = "x", .transport = .{ .stdio = .{ .argv = &argv, .cwd = "/a" } } };
|
||||
|
||||
+23
-4
@@ -4,6 +4,7 @@ const acp_server = @import("acp/server.zig");
|
||||
const jsonrpc = @import("acp/jsonrpc.zig");
|
||||
const gateway_provider = @import("core/gateway/gateway_provider.zig");
|
||||
const provider_set = @import("core/gateway/provider_set.zig");
|
||||
const stream_provider = @import("core/agent/stream_provider.zig");
|
||||
const agent_steps = @import("core/config/agent_steps.zig");
|
||||
const context_contract = @import("core/workspace/context_contract.zig");
|
||||
const host = @import("core/hosts/host.zig");
|
||||
@@ -31,6 +32,24 @@ const max_input_bytes = 8 * 1024 * 1024;
|
||||
const max_output_bytes = 8 * 1024 * 1024;
|
||||
const max_output_message_bytes = 64 * 1024 * 1024;
|
||||
const max_fetch_request_bytes = 8 * 1024 * 1024;
|
||||
/// The largest request body a provider sends over the fetch bridge: its
|
||||
/// budget less room for the request's method, URL, and headers, which hold
|
||||
/// at most the URL and API key this addon accepts and the other headers.
|
||||
const max_bridge_request_body_bytes = max_fetch_request_bytes - (max_url_bytes + max_api_key_bytes + 64 * 1024);
|
||||
|
||||
/// `provider` with its requests fit to the fetch bridge, so they leave out
|
||||
/// their oldest images instead of failing.
|
||||
fn bridgeBound(provider: stream_provider.Provider) stream_provider.Provider {
|
||||
var bound = provider;
|
||||
bound.max_request_bytes = max_bridge_request_body_bytes;
|
||||
return bound;
|
||||
}
|
||||
|
||||
fn configuredBridgeBundle(raw: ?*anyopaque, definition: *const configured_provider.Definition) provider_set.Bundle {
|
||||
var bundle = host_chat_completions.bundle(raw, definition);
|
||||
if (bundle.agent_stream) |stream| bundle.agent_stream = bridgeBound(stream);
|
||||
return bundle;
|
||||
}
|
||||
const max_fetch_response_bytes = 8 * 1024 * 1024;
|
||||
// One attachment holds one prompt image or one kernel checkpoint, the larger.
|
||||
const max_attachment_bytes = agent_checkpoint.max_checkpoint_bytes;
|
||||
@@ -686,23 +705,23 @@ const Runtime = struct {
|
||||
.presentation = builtin_gateway.provider_bundle.presentation,
|
||||
.auth_strategy = .vercel,
|
||||
.fallback_model_capabilities_fn = builtin_gateway.provider_bundle.fallback_model_capabilities_fn,
|
||||
.agent_stream = host_stream_provider.provider(&self.stream_context),
|
||||
.agent_stream = bridgeBound(host_stream_provider.provider(&self.stream_context)),
|
||||
.model_catalog = @import("gateway/host_model_catalog.zig").provider(&self.stream_context.transport),
|
||||
});
|
||||
providers.codex = .{
|
||||
.presentation = provider_catalog.find(.codex),
|
||||
.auth_strategy = .chatgpt,
|
||||
.agent_stream = host_openai_codex.provider(&self.codex_context),
|
||||
.agent_stream = bridgeBound(host_openai_codex.provider(&self.codex_context)),
|
||||
.model_catalog = host_openai_codex.modelCatalogProvider(&self.codex_context),
|
||||
};
|
||||
providers.grok = .{
|
||||
.presentation = provider_catalog.find(.grok),
|
||||
.auth_strategy = .grok,
|
||||
.agent_stream = host_xai_grok.provider(&self.grok_context),
|
||||
.agent_stream = bridgeBound(host_xai_grok.provider(&self.grok_context)),
|
||||
.model_catalog = host_xai_grok.modelCatalogProvider(&self.grok_context),
|
||||
};
|
||||
if (self.custom_context) |*context| {
|
||||
providers.configured_fn = host_chat_completions.bundle;
|
||||
providers.configured_fn = configuredBridgeBundle;
|
||||
providers.configured_context = context;
|
||||
}
|
||||
acp_server.runWithTransport(
|
||||
|
||||
@@ -5,14 +5,10 @@ const session_child_store = @import("../../core/session/session_child_store.zig"
|
||||
const io_mod = @import("../../core/shared/io.zig");
|
||||
const tool_dispatch = @import("../../core/tooling/tool_dispatch.zig");
|
||||
const compactor = @import("../../core/compactor/compactor.zig");
|
||||
const read_result_presentation = @import("../../core/tooling/read_tool_result_presentation.zig");
|
||||
|
||||
const Allocator = std.mem.Allocator;
|
||||
|
||||
const HandleNormalization = struct {
|
||||
trimmed: []const u8,
|
||||
suffix: []const u8,
|
||||
};
|
||||
|
||||
pub const Input = struct {
|
||||
handle: []u8,
|
||||
selector: union(enum) {
|
||||
@@ -130,20 +126,10 @@ fn inputDeinit(ptr: *anyopaque, alloc: Allocator) void {
|
||||
alloc.destroy(input);
|
||||
}
|
||||
|
||||
fn classifyHandleNormalization(handle: []const u8) HandleNormalization {
|
||||
const trimmed = std.mem.trim(u8, handle, " \t\r\n");
|
||||
const suffix = if ((std.mem.startsWith(u8, trimmed, "result-") or result_store.isImageHandle(trimmed)) and
|
||||
std.mem.findScalar(u8, trimmed, '.') == null)
|
||||
".txt"
|
||||
else
|
||||
"";
|
||||
return .{ .trimmed = trimmed, .suffix = suffix };
|
||||
}
|
||||
|
||||
pub fn validate(ctx: tool_dispatch.DispatchContext, erased: tool_dispatch.ToolInput) tool_dispatch.DispatchError!?[]u8 {
|
||||
const input = erased.as(Input);
|
||||
if (input.selector == .search) return null;
|
||||
const normalization = classifyHandleNormalization(input.handle);
|
||||
const normalization = read_result_presentation.classifyHandleNormalization(input.handle);
|
||||
if (normalization.trimmed.len == 0) return try ctx.allocator.dupe(u8, "read_tool_result field \"handle\" must not be empty");
|
||||
var record_buffer: compactor.RecordFileBuffer = undefined;
|
||||
if (compactor.recordFile(&record_buffer, normalization.trimmed)) |file| {
|
||||
@@ -279,6 +265,9 @@ pub fn isIrreversible(_: tool_dispatch.ToolInput) bool {
|
||||
return false;
|
||||
}
|
||||
|
||||
/// Labels each call by what it reads. See `read_tool_result_presentation.zig`.
|
||||
pub const presentation = read_result_presentation.presentation;
|
||||
|
||||
test "read_tool_result decodes range and query inputs" {
|
||||
const alloc = std.testing.allocator;
|
||||
const decoded_range = try decode(.{ .allocator = alloc }, "{\"handle\":\"h.txt\",\"start_byte\":2,\"byte_count\":9}");
|
||||
|
||||
@@ -6356,6 +6356,8 @@ printf '%s' ${JSON.stringify(trailingMarker)} > ${JSON.stringify(effectPath)}
|
||||
const pane = await tui.waitForPane((text) => hasEmptyComposer(text) && (text.includes("RETRIEVAL_TURN_COMPLETE") || text.includes("request failed:")), 30000);
|
||||
expect(pane).not.toContain("request failed:");
|
||||
expect(pane).toContain("RETRIEVAL_TURN_COMPLETE");
|
||||
// The read names the command whose saved output it searches.
|
||||
expect(pane).toContain("cat source.txt output for \"RETRIEVAL_MATCH\"");
|
||||
if (trigger === "manual") {
|
||||
await compactAndWait(tui, root, 20000);
|
||||
}
|
||||
@@ -6391,7 +6393,9 @@ printf '%s' ${JSON.stringify(trailingMarker)} > ${JSON.stringify(effectPath)}
|
||||
expect(readFileSync(join(toolResults, "compacted-T1.txt"), "utf8")).toContain(sourceHandle);
|
||||
const resumed = await runFx(["ask", "--json", "--resume-id", sessionId, "Recover the clipped tail from the saved retrieval without rerunning the command."], { cwd: root.workspace, env, timeoutMs: 30000 });
|
||||
expect(resumed.code).toBe(0);
|
||||
expect(resumed.stderr).toBe("Reading tool result\n");
|
||||
// Compaction folded the command into T1, so the read has no call
|
||||
// left to name.
|
||||
expect(resumed.stderr).toBe("Reading more command output\n");
|
||||
expect(JSON.parse(resumed.stdout).final_output).toBe("RETRIEVAL_RESTART_COMPLETE");
|
||||
expect(gateway.requests).toHaveLength(8);
|
||||
expect(readFileSync(join(root.workspace, "effects.txt"), "utf8")).toBe("once\n");
|
||||
|
||||
@@ -2881,7 +2881,12 @@ test.skipIf(!tmuxAvailable())("text typed while the app answers steers that turn
|
||||
await resumed.session.sendText("Continue after the steered turn.");
|
||||
await resumed.session.waitForText("AFTER_STEERING_RESUME", TIMEOUT);
|
||||
await quitApp(resumed);
|
||||
expect(gateway.requests.at(-1)!.body).toContain(steering);
|
||||
// The resumed session sends the steering as the live update the model
|
||||
// saw, not as a plain user message.
|
||||
const continued = gateway.requests.at(-1)!.body;
|
||||
expect(continued).toContain(steering);
|
||||
expect(continued).toContain("<user_steering>");
|
||||
expect(continued.indexOf("<user_steering>")).toBeLessThan(continued.indexOf(steering));
|
||||
expectWholeLog(fixture, id);
|
||||
} finally {
|
||||
hold.release?.();
|
||||
|
||||
@@ -3178,7 +3178,7 @@ describe.skipIf(!tmuxAvailable())("TUI gateway stream lifecycle", () => {
|
||||
const savedSession = readSavedSession();
|
||||
|
||||
expect(pane).toContain(finalText);
|
||||
expect(scrollback).toContain("Failed tool result");
|
||||
expect(scrollback).toContain("Failed more output");
|
||||
expect(continuation).toContain(expectedFailure);
|
||||
expect(savedSession).toContain('"status":"failure"');
|
||||
expect(savedSession).toContain(expectedFailure);
|
||||
@@ -5186,6 +5186,104 @@ describe.skipIf(!tmuxAvailable())("TUI gateway stream lifecycle", () => {
|
||||
TIMEOUT,
|
||||
);
|
||||
|
||||
test(
|
||||
"parallel saved-output reads name the command they read",
|
||||
async () => {
|
||||
root = realpathSync(mkdtempSync(join(tmpdir(), "fx-parallel-saved-reads-")));
|
||||
const home = join(root, "home");
|
||||
const workspace = join(root, "workspace");
|
||||
const stderrPath = join(root, "stderr.log");
|
||||
const tracePath = join(root, "trace.log");
|
||||
mkdirSync(join(home, ".fx"), { recursive: true });
|
||||
mkdirSync(workspace);
|
||||
writeFileSync(join(home, ".fx", "settings.json"), "{}");
|
||||
// Large enough that fx saves the command output behind a handle.
|
||||
const lines = Array.from({ length: 4000 }, (_, index) => `SAVED_LINE_${String(index).padStart(5, "0")}`);
|
||||
lines[2500] = "PARALLEL_READ_MATCH";
|
||||
writeFileSync(join(workspace, "source.txt"), `${lines.join("\n")}\n`);
|
||||
|
||||
const readGateway = startDynamicFakeGateway((body) => {
|
||||
const request = JSON.parse(body) as {
|
||||
prompt: Array<{ role: string; content: unknown }>;
|
||||
};
|
||||
const parts = request.prompt.flatMap((message) =>
|
||||
Array.isArray(message.content) ? message.content : [],
|
||||
) as Array<Record<string, unknown>>;
|
||||
const output = (callId: string): string | null => {
|
||||
const part = parts.find((candidate) =>
|
||||
candidate.type === "tool-result" && candidate.toolCallId === callId,
|
||||
);
|
||||
return part ? contentText(part.output) : null;
|
||||
};
|
||||
if (output("parallel_read_query") !== null && output("parallel_read_range") !== null) {
|
||||
return fakeGatewayFinalText("PARALLEL_READS_FINISHED");
|
||||
}
|
||||
const source = output("parallel_read_source");
|
||||
if (source === null) {
|
||||
return fakeGatewayToolCall("parallel_read_source", "shell", {
|
||||
request: { action: "run", command: "cat source.txt", profile: "clean", yield_time_ms: 30000 },
|
||||
});
|
||||
}
|
||||
const handle = (JSON.parse(source) as { full_output_handle: string | null }).full_output_handle;
|
||||
if (!handle) return new Response("missing saved output handle", { status: 500 });
|
||||
return fakeGatewaySse([
|
||||
{
|
||||
type: "tool-call",
|
||||
toolCallId: "parallel_read_query",
|
||||
toolName: "read_tool_result",
|
||||
input: { request: { handle, query: "PARALLEL_READ_MATCH" } },
|
||||
},
|
||||
{
|
||||
type: "tool-call",
|
||||
toolCallId: "parallel_read_range",
|
||||
toolName: "read_tool_result",
|
||||
input: { request: { handle, start_byte: 20000, byte_count: 1024 } },
|
||||
},
|
||||
{
|
||||
type: "finish",
|
||||
finishReason: { unified: "tool-calls", raw: "tool-calls" },
|
||||
},
|
||||
]);
|
||||
}, { classifierDecision: "clear" });
|
||||
gateway = readGateway;
|
||||
session = await TmuxSession.create({
|
||||
cwd: workspace,
|
||||
width: 110,
|
||||
height: 35,
|
||||
stderrPath,
|
||||
env: {
|
||||
HOME: home,
|
||||
AI_GATEWAY_API_KEY: "fake-parallel-saved-reads-key",
|
||||
VERCEL_OIDC_TOKEN: undefined,
|
||||
FX_DISABLE_KEYCHAIN: "1",
|
||||
FX_SOUND: "0",
|
||||
FX_AUTO_UPGRADE: "0",
|
||||
FX_PERMISSION_MODE: "auto",
|
||||
FX_GATEWAY_BASE_URL: readGateway.baseUrl,
|
||||
FX_GATEWAY_CHAT_URL: readGateway.chatUrl,
|
||||
FX_E2E_GATEWAY_CHAT_URL: readGateway.chatUrl,
|
||||
FX_MODEL: MODEL,
|
||||
FX_TRACE_LOG: tracePath,
|
||||
FX_TRACE_SCOPES: "tool",
|
||||
},
|
||||
});
|
||||
|
||||
await session.waitForComposer(TIMEOUT);
|
||||
await session.sendText("Run the fixture, then search and page its saved output.");
|
||||
await session.waitForText("PARALLEL_READS_FINISHED", TIMEOUT);
|
||||
const scrollback = await session.captureFullScrollback();
|
||||
|
||||
// Both reads ran as one parallel group and still name their source.
|
||||
expect(readFileSync(tracePath, "utf8")).toMatch(/parallel_tool_group_start[^\n]*count=2/);
|
||||
expect(scrollback).toContain("Searched cat source.txt output for \"PARALLEL_READ_MATCH\"");
|
||||
expect(scrollback).toContain("Read more output from cat source.txt");
|
||||
expect(readFileSync(stderrPath, "utf8")).toBe("");
|
||||
await session.sendText("/quit");
|
||||
expect(await session.waitForSessionEnd(TIMEOUT)).toBe(true);
|
||||
},
|
||||
TIMEOUT * 2,
|
||||
);
|
||||
|
||||
test(
|
||||
"separate provider messages render as separate paragraphs",
|
||||
async () => {
|
||||
|
||||
@@ -3397,7 +3397,13 @@ describe("request size budget", () => {
|
||||
|
||||
type SizeRequest = { body: string; images: string[]; tools: boolean; status: number };
|
||||
|
||||
function startSizeCappedGateway(rejection: { status: number; body: string }, images: Map<string, string>) {
|
||||
// Refuses every request over `cap`. By default each answer asks for the next
|
||||
// image with read_file; `describe` answers at once instead.
|
||||
function startSizeCappedGateway(
|
||||
rejection: { status: number; body: string },
|
||||
images: Map<string, string>,
|
||||
{ cap = SIZE_CAP_BYTES, answer = "read" }: { cap?: number; answer?: "read" | "describe" } = {},
|
||||
) {
|
||||
const byData = new Map([...images].map(([name, data]) => [data, name]));
|
||||
const requests: SizeRequest[] = [];
|
||||
let issued = 0;
|
||||
@@ -3415,12 +3421,13 @@ describe("request size budget", () => {
|
||||
const parsed = JSON.parse(body) as { tools?: unknown[] };
|
||||
const tools = Array.isArray(parsed.tools) && parsed.tools.length > 0;
|
||||
const carried = nativeFileParts(body).map((part) => byData.get(part.data.data) ?? "unknown");
|
||||
const status = body.length > SIZE_CAP_BYTES ? rejection.status : 200;
|
||||
const status = body.length > cap ? rejection.status : 200;
|
||||
requests.push({ body, images: carried, tools, status });
|
||||
if (status !== 200) {
|
||||
return new Response(rejection.body, { status, headers: { "content-type": "application/json" } });
|
||||
}
|
||||
if (!tools) return sseText("Summary: the user asked fx to read six images.");
|
||||
if (answer === "describe") return sseText("done: described the images");
|
||||
if (issued < SIZE_IMAGE_COUNT) {
|
||||
issued += 1;
|
||||
return sseToolCall("read_file", { path: `img0${issued}.png` }, `call_img_${issued}`);
|
||||
@@ -3498,4 +3505,37 @@ describe("request size budget", () => {
|
||||
}
|
||||
}, TIMEOUT * 3);
|
||||
}
|
||||
|
||||
test("a size limit two retries can't reach still leaves out images instead of compacting", async () => {
|
||||
const root = createIsolatedRoot();
|
||||
const images = writeSizeImages(root.workspace);
|
||||
// Six images of about 1 MB in one prompt against a 1.5 MB limit: each
|
||||
// refusal lowers fx's limit by a quarter, so the request fits only after
|
||||
// a third refusal leaves out all but the newest image.
|
||||
const cap = 1_500_000;
|
||||
const gateway = startSizeCappedGateway({ status: 413, body: payloadTooLargeBody }, images, { cap, answer: "describe" });
|
||||
try {
|
||||
const imageArgs = [...images.keys()].flatMap((name) => ["--image", join(root.workspace, name)]);
|
||||
const result = await runFx(["ask", "--json", "--auto", "--no-color", ...imageArgs, "Describe these images."], {
|
||||
cwd: root.workspace,
|
||||
env: gateway.env(root),
|
||||
timeoutMs: TIMEOUT * 2,
|
||||
});
|
||||
const json = parseFxJson(result);
|
||||
expect(json.output).toContain("done: described the images");
|
||||
|
||||
const rejected = gateway.requests.filter((request) => request.status !== 200);
|
||||
expect(rejected.length).toBeGreaterThan(2);
|
||||
// No summary request: the conversation was not compacted.
|
||||
expect(gateway.requests.every((request) => request.tools)).toBe(true);
|
||||
const accepted = gateway.requests.at(-1)!;
|
||||
expect(accepted.status).toBe(200);
|
||||
expect(accepted.body.length).toBeLessThanOrEqual(cap);
|
||||
expect(accepted.images).toEqual(["img06.png"]);
|
||||
expect(accepted.body).toContain(SIZE_NOTICE);
|
||||
} finally {
|
||||
gateway.stop();
|
||||
rmSync(root.root, { recursive: true, force: true });
|
||||
}
|
||||
}, TIMEOUT * 3);
|
||||
});
|
||||
+2
-2
@@ -10,12 +10,12 @@ TLA+ specifications for fx, checked with TLC, together with the scripts and Bun
|
||||
| `indexer/` | When a scanned root's listing may be reused, merging parallel grep results, the file picker's loading row, and listing snapshots (`src/core/indexer/`). | `indexer/check.sh` |
|
||||
| `mcp/` | The MCP-v2 engine (`src/mcp/`): wire format, requests, stdio, version detection, the tool catalog, both HTTP transports, input-required rounds, authorization, subscriptions, elicitation, and the engine. `rules.md` lists the MCP specification rules the models cite by ID. | `mcp/check.sh`, then the drivers in `drivers/mcp/` |
|
||||
| `mcp_host/` | fx's MCP host (`src/core/mcp_host/`): host lifecycle, saved credentials, logins shared between processes, sign-in, and elicitation routing. | `mcp_host/check.sh`, `bun test drivers/mcp_host/` |
|
||||
| `sdk/` | libfx durable sessions (`DurableSession.tla`), agent names and routing (`AgentRouting.tla`), tenants whose agents share one process (`TenantIsolation.tla`), steering (`DurableSteer.tla`), and the provider credential edge (`ProviderAuth.tla`). `check.sh` also writes the TLC state graphs the drivers walk. | `sdk/check.sh`, then the drivers |
|
||||
| `sdk/` | libfx durable sessions (`DurableSession.tla`), how a session object takes prompts in (`PromptIntake.tla`), images saved once by name (`ImageStore.tla`), agent names and routing (`AgentRouting.tla`), tenants whose agents share one process (`TenantIsolation.tla`), steering (`DurableSteer.tla`), and the provider credential edge (`ProviderAuth.tla`). `check.sh` also writes the TLC state graphs the drivers walk. | `sdk/check.sh`, then the drivers |
|
||||
| `session_manager/` | The session log, turn lifecycle, writer lock, snapshots, fork, subagents, catalog, host wiring, the side-file move, and v1 conversion. `rollout/` holds the models of the changes that moved fx onto the session manager. `system-design.md` is the design record the specs check and defines the `D1` to `D62` decision numbers that code comments cite. | `session_manager/check.sh`, `session_manager/rollout/run.py`, `bun test drivers/session_manager/` |
|
||||
| `shell_command/` | How an agent git command is admitted and run (`src/core/shell_command/safe_git.zig`), and one model turn through the shell tool, for the current design and a redesign. | `shell_command/check.sh`, `bun test drivers/shell_command/` |
|
||||
| `terminal/` | The in-memory shell snapshot owner (`src/core/terminal/shell_snapshot.zig`) and the lifecycle of hosted terminals. | `terminal/check.sh` |
|
||||
| `usage/` | `/usage` refreshes and the shared usage cache (`src/usage/`): `LeanUsage.tla` and `UsageModel.tla`. | `usage/check.sh`, then the drivers in `drivers/usage/` |
|
||||
| `drivers/` | The Bun drivers. They run the real fx binary or SDK code, record what happens, and have TLC check each run against its model. `drive.mjs`, `drive-agents.mjs`, `drive-tenants.mjs`, and `drive-steer.mjs` walk the SDK code along the `sdk/` state graphs. | See each folder's `check.sh` |
|
||||
| `drivers/` | The Bun drivers. They run the real fx binary or SDK code, record what happens, and have TLC check each run against its model. `drive.mjs`, `drive-intake.mjs`, `drive-images.mjs`, `drive-agents.mjs`, `drive-tenants.mjs`, and `drive-steer.mjs` walk the SDK code along the `sdk/` state graphs. | See each folder's `check.sh` |
|
||||
| `lib/` | `tlc.sh`, which finds TLC and Java and runs a table of configs with their expected verdicts, `tlc.py`, which asks `tlc.sh` for TLC and Java on behalf of the Python checkers, and `run-tlc.sh`, which runs TLC once for the drivers. | |
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -0,0 +1,407 @@
|
||||
#!/usr/bin/env bun
|
||||
// Drives real libfx along paths of the TLC state graph of ImageStore.tla,
|
||||
// and checks after every step that the code is in a state the model allows.
|
||||
//
|
||||
// bun tests/tla/drivers/drive-images.mjs [--walks N] [--seed S] [--backend native|wasm]
|
||||
//
|
||||
// Each walk keeps one session in a memory persistence with an image store,
|
||||
// first written by an engine without one, so its first image is saved
|
||||
// inline as an older libfx saved it. Then the walk opens an engine on it and
|
||||
// makes the prompts it chooses, each with image a or b; a turn's tool may
|
||||
// return image a or b as well, which the engine stores before the core gets
|
||||
// the result. Every image write
|
||||
// waits until the driver lets it land or fail; every model request waits
|
||||
// until the driver ends its turn. A crash leaves the engine behind at once:
|
||||
// its writes in flight never land, and nothing it does later reaches the
|
||||
// store. The driver reopens the session, resuming a turn the crash left
|
||||
// open, and may delete an image from the store. After each step the driver
|
||||
// waits until the code is quiet and moves to the model states that show the
|
||||
// same prompts, store, open state and held request. On every step, nothing
|
||||
// saved may name an image the store does not hold (until one is deleted),
|
||||
// only the old image, or a tool's image the store refused, may be saved
|
||||
// inline, and a model request must carry
|
||||
// exactly the history the model holds.
|
||||
//
|
||||
// `tests/tla/sdk/check.sh` writes graphs/images.dot from ImageStore-drive.cfg.
|
||||
import { strict as assert } from "node:assert";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { createFxEngine, createMemoryPersistence } from "../../../sdk/node.js";
|
||||
import { loadGraph, walkGraph } from "./graph.mjs";
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const option = (name, fallback) => {
|
||||
const index = args.indexOf(`--${name}`);
|
||||
return index >= 0 ? args[index + 1] : fallback;
|
||||
};
|
||||
const walks = Number(option("walks", 40));
|
||||
let seed = Number(option("seed", 1));
|
||||
const backend = option("backend", "native");
|
||||
if (!["native", "wasm"].includes(backend)) throw new Error("--backend is native or wasm");
|
||||
const verbose = args.includes("--verbose");
|
||||
const here = fileURLToPath(new URL(".", import.meta.url));
|
||||
const addon = resolve(here, "../../../zig-out/lib/libfx.node");
|
||||
const wasm = backend === "wasm" ? await readFile(resolve(here, "../../../zig-out/bin/fx-core.wasm")) : undefined;
|
||||
const quietMs = 150;
|
||||
const stepTimeoutMs = 10_000;
|
||||
// The model's window, which every engine the walk opens keeps too.
|
||||
const windowImages = Number(/^\s*Window = (\d+)$/m.exec(await readFile(resolve(here, "../sdk/ImageStore-drive.cfg"), "utf8"))[1]);
|
||||
const imageWindow = Symbol.for("libfx.imageWindow");
|
||||
const decoder = new TextDecoder();
|
||||
const sleep = (ms) => new Promise((done) => setTimeout(done, ms));
|
||||
const deferred = () => {
|
||||
let resolveIt;
|
||||
let rejectIt;
|
||||
const promise = new Promise((resolveP, rejectP) => { resolveIt = resolveP; rejectIt = rejectP; });
|
||||
promise.catch(() => {});
|
||||
return { promise, resolve: resolveIt, reject: rejectIt };
|
||||
};
|
||||
const random = () => {
|
||||
seed = (seed * 1103515245 + 12345) % 2147483648;
|
||||
return seed / 2147483648;
|
||||
};
|
||||
|
||||
// The model's images, as PNG bytes the kernel accepts, each its own.
|
||||
const png = Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jP0cAAAAASUVORK5CYII=", "base64");
|
||||
const labels = ["o", "a", "b"];
|
||||
const bytesOf = Object.fromEntries(labels.map((label) => [label, new Uint8Array(Buffer.concat([png, Buffer.from(`image-${label}-`.repeat(400))]))]));
|
||||
const base64Of = Object.fromEntries(labels.map((label) => [label, Buffer.from(bytesOf[label]).toString("base64")]));
|
||||
const nameOf = Object.fromEntries(await Promise.all(labels.map(async (label) => [label, Buffer.from(await crypto.subtle.digest("SHA-256", bytesOf[label])).toString("hex")])));
|
||||
const labelOfName = Object.fromEntries(labels.map((label) => [nameOf[label], label]));
|
||||
const labelOfBase64 = Object.fromEntries(labels.map((label) => [base64Of[label], label]));
|
||||
const labelOfBytes = (bytes) => labelOfBase64[Buffer.from(bytes).toString("base64")] ?? "?";
|
||||
|
||||
// What a session saved, decoded: the names its records and checkpoint give,
|
||||
// and the images they hold inline.
|
||||
function durableBytes(value) {
|
||||
if (typeof value === "string") return Buffer.from(value, "utf8");
|
||||
if (value?.encoding === "base64") return Buffer.from(value.data, "base64");
|
||||
return null;
|
||||
}
|
||||
function savedImages(store) {
|
||||
const named = new Set();
|
||||
const inline = [];
|
||||
const visit = (value, blobs) => {
|
||||
if (Array.isArray(value)) {
|
||||
for (const item of value) visit(item, blobs);
|
||||
return;
|
||||
}
|
||||
if (!value || typeof value !== "object") return;
|
||||
// One the session dropped keeps its name alone, which needs no bytes: an
|
||||
// image held inline takes the name of its bytes when it leaves.
|
||||
if (typeof value.inline_ref === "string" && value.dropped !== true) named.add(value.inline_ref);
|
||||
if (value.inline_data !== undefined) inline.push(durableBytes(value.inline_data));
|
||||
// A tool's image held inline: its base64 data.
|
||||
if (value.type === "image" && typeof value.data === "string" && typeof value.mimeType === "string") inline.push(Buffer.from(value.data, "base64"));
|
||||
if (Number.isInteger(value.inline_blob)) inline.push(blobs[value.inline_blob]);
|
||||
for (const item of Object.values(value)) visit(item, blobs);
|
||||
};
|
||||
for (const record of store.records) visit(JSON.parse(decoder.decode(record.data)), []);
|
||||
if (store.checkpoint) {
|
||||
// FXSN: magic, version, seq, turn, id length and id, an open turn from
|
||||
// version 2 on, then the FXCP checkpoint.
|
||||
const bytes = Buffer.from(store.checkpoint.data);
|
||||
const version = bytes[4];
|
||||
let at = 4 + 1 + 8 + 8;
|
||||
at += 1 + bytes[at];
|
||||
if (version >= 2) {
|
||||
const length = bytes.readUInt32LE(at);
|
||||
visit(JSON.parse(bytes.subarray(at + 4, at + 4 + length).toString("utf8")), []);
|
||||
at += 4 + length;
|
||||
}
|
||||
const checkpoint = bytes.subarray(at);
|
||||
const payload = checkpoint.subarray(4 + 2 + 2 + 4 + 32);
|
||||
if (checkpoint.readUInt16LE(4) === 1) {
|
||||
visit(JSON.parse(payload.toString("utf8")), []);
|
||||
} else {
|
||||
const jsonLength = payload.readUInt32LE(0);
|
||||
const json = JSON.parse(payload.subarray(4, 4 + jsonLength).toString("utf8"));
|
||||
const blobs = [];
|
||||
let offset = 4 + jsonLength;
|
||||
const count = payload.readUInt32LE(offset);
|
||||
offset += 4;
|
||||
for (let index = 0; index < count; index += 1) {
|
||||
const length = payload.readUInt32LE(offset);
|
||||
blobs.push(payload.subarray(offset + 4, offset + 4 + length));
|
||||
offset += 4 + length;
|
||||
}
|
||||
visit(json, blobs);
|
||||
}
|
||||
}
|
||||
return { named, inline };
|
||||
}
|
||||
|
||||
const vision = "images/vision";
|
||||
const usage = { inputTokens: { total: 1 }, outputTokens: { total: 1 } };
|
||||
// The model calls the turn's tool for image `label`.
|
||||
const callShot = (p, label) => new Response(
|
||||
[{ type: "tool-call", toolCallId: `call-${p}`, toolName: "shot", input: { label } }, { type: "finish", finishReason: { unified: "tool-calls", raw: "tool-calls" }, usage }, "[DONE]"]
|
||||
.map((frame) => `data: ${typeof frame === "string" ? frame : JSON.stringify(frame)}\n\n`).join(""),
|
||||
{ headers: { "content-type": "text/event-stream" } },
|
||||
);
|
||||
// The tool returns the image it is asked for.
|
||||
const shot = {
|
||||
name: "shot",
|
||||
description: "Returns an image",
|
||||
inputSchema: { type: "object", properties: { label: { type: "string" } }, required: ["label"] },
|
||||
execute: async ({ label }) => ({ type: "libfx.tool-result", text: `image ${label}`, images: [{ type: "image", data: base64Of[label], mimeType: "image/png" }] }),
|
||||
};
|
||||
const answer = () => new Response(
|
||||
[{ type: "text-delta", id: "a", delta: "seen" }, { type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage }, "[DONE]"]
|
||||
.map((frame) => `data: ${typeof frame === "string" ? frame : JSON.stringify(frame)}\n\n`).join(""),
|
||||
{ headers: { "content-type": "text/event-stream" } },
|
||||
);
|
||||
|
||||
// libfx's own key for a persistence's image store, which durable sessions
|
||||
// give the engines they run.
|
||||
const persistenceImages = Symbol.for("libfx.persistenceImages");
|
||||
|
||||
async function harness(n) {
|
||||
const store = createMemoryPersistence();
|
||||
// The image store: each image once, by name.
|
||||
const storedImages = new Map();
|
||||
const problems = [];
|
||||
// Held image writes by name, and the model request held now.
|
||||
const imageGates = new Map();
|
||||
let held = null;
|
||||
let answerNow = false;
|
||||
// Each engine's own fetch: a crashed engine reaches the model no more.
|
||||
const fetchFor = (current) => async (input, init) => {
|
||||
if (current && !current.alive) throw new Error("crashed");
|
||||
const request = input instanceof Request ? input : new Request(input, init);
|
||||
if (request.method === "GET") return Response.json({ object: "list", data: [{ id: vision, type: "language", tags: ["vision", "file-input"] }] });
|
||||
const prompt = JSON.parse(await request.text()).prompt;
|
||||
const users = prompt.filter((message) => message.role === "user");
|
||||
const text = users.map((message) => (Array.isArray(message.content) ? message.content : []).filter((part) => part.type === "text").map((part) => part.text).join("")).join("\n");
|
||||
const numbers = [...text.matchAll(/prompt-(\d+)/g)].map((match) => Number(match[1]));
|
||||
const images = users.flatMap((message) => (Array.isArray(message.content) ? message.content : []).filter((part) => part.type === "file"))
|
||||
.map((part) => labelOfBase64[part.data?.data] ?? "?");
|
||||
if (answerNow) return answer();
|
||||
const response = deferred();
|
||||
held = { p: numbers.at(-1) ?? 0, images, response, life: current };
|
||||
const signal = init?.signal ?? request.signal;
|
||||
signal?.addEventListener("abort", () => response.reject(signal.reason ?? new Error("aborted")), { once: true });
|
||||
return response.promise;
|
||||
};
|
||||
|
||||
// Each engine's view of the store, until it crashes.
|
||||
let life = null;
|
||||
const persistenceFor = (current) => ({
|
||||
load: () => (current.alive ? store.load() : Promise.reject(new Error("crashed"))),
|
||||
append: (input) => (current.alive ? store.append(input) : Promise.reject(new Error("crashed"))),
|
||||
saveCheckpoint: (input) => (current.alive ? store.saveCheckpoint(input) : Promise.reject(new Error("crashed"))),
|
||||
[persistenceImages]: {
|
||||
async put(name, bytes) {
|
||||
if (!current.alive) throw new Error("crashed");
|
||||
const gate = imageGates.get(name);
|
||||
if (gate) {
|
||||
imageGates.delete(name);
|
||||
const lands = await gate.promise;
|
||||
if (!lands) throw new Error("injected image write failure");
|
||||
// A write the crash caught in flight never lands.
|
||||
if (!current.alive) return new Promise(() => {});
|
||||
}
|
||||
if (!storedImages.has(name)) storedImages.set(name, bytes.slice());
|
||||
},
|
||||
get: async (name) => {
|
||||
if (!current.alive) throw new Error("crashed");
|
||||
return storedImages.get(name)?.slice();
|
||||
},
|
||||
},
|
||||
});
|
||||
const engineOptions = (persistence, current = null) => ({
|
||||
backend,
|
||||
nativeAddon: addon,
|
||||
...(wasm ? { wasm } : {}),
|
||||
apiKey: "images-key",
|
||||
model: vision,
|
||||
fetch: fetchFor(current),
|
||||
persistence,
|
||||
checkpointAfterBytes: 0,
|
||||
tools: [shot],
|
||||
[imageWindow]: { images: windowImages },
|
||||
});
|
||||
|
||||
// The session an older libfx saved: its image inline, in a record and in a
|
||||
// checkpoint.
|
||||
{
|
||||
answerNow = true;
|
||||
const old = await createFxEngine(engineOptions({ load: () => store.load(), append: (input) => store.append(input), saveCheckpoint: (input) => store.saveCheckpoint(input) }));
|
||||
const turn = old.prompt([{ type: "text", text: "old" }, { type: "image", data: bytesOf.o, mimeType: "image/png" }]);
|
||||
for await (const _ of turn) {}
|
||||
assert.equal((await turn.result).stopReason, "end_turn");
|
||||
await old.close();
|
||||
answerNow = false;
|
||||
}
|
||||
|
||||
let engine = null;
|
||||
let up = "down";
|
||||
let lost = false;
|
||||
let made = 0;
|
||||
const ph = Array(n).fill("unmade");
|
||||
const gates = {};
|
||||
// Each turn's tool image write, held until the driver lets it land or fail,
|
||||
// and the images the store refused, which the session holds inline.
|
||||
const toolGates = {};
|
||||
const refused = new Set();
|
||||
const watch = (p, turn) => {
|
||||
turn.result.then(
|
||||
(result) => { if (result.stopReason === "end_turn") ph[p - 1] = "ended"; },
|
||||
(error) => { if (error?.message === "injected image write failure") ph[p - 1] = "failed"; },
|
||||
);
|
||||
};
|
||||
|
||||
async function perform(act) {
|
||||
const [name, p, image] = act;
|
||||
switch (name) {
|
||||
case "Make": {
|
||||
made = p;
|
||||
ph[p - 1] = "putting";
|
||||
gates[p] = deferred();
|
||||
imageGates.set(nameOf[image], { promise: gates[p].promise });
|
||||
const turn = engine.prompt([{ type: "text", text: `prompt-${p}` }, { type: "image", data: bytesOf[image].slice(), mimeType: "image/png" }]);
|
||||
void turn.result.catch(() => {});
|
||||
watch(p, turn);
|
||||
return;
|
||||
}
|
||||
case "PutLand": gates[p].resolve(true); return;
|
||||
case "PutFail": gates[p].resolve(false); return;
|
||||
case "ToolCall": {
|
||||
if (!held || held.p !== p) throw new Error(`no model request is held for prompt ${p}`);
|
||||
toolGates[p] = { gate: deferred(), image };
|
||||
imageGates.set(nameOf[image], { promise: toolGates[p].gate.promise });
|
||||
const { response } = held;
|
||||
held = null;
|
||||
response.resolve(callShot(p, image));
|
||||
return;
|
||||
}
|
||||
case "ToolPutLand": toolGates[p].gate.resolve(true); return;
|
||||
case "ToolPutFail": {
|
||||
refused.add(toolGates[p].image);
|
||||
toolGates[p].gate.resolve(false);
|
||||
return;
|
||||
}
|
||||
case "TurnEnd": {
|
||||
if (!held || held.p !== p) throw new Error(`no model request is held for prompt ${p}`);
|
||||
const { response } = held;
|
||||
held = null;
|
||||
response.resolve(answer());
|
||||
return;
|
||||
}
|
||||
case "Crash": {
|
||||
life.alive = false;
|
||||
// The crash ends the engine's held request and image writes with it.
|
||||
if (held) {
|
||||
held.response.reject(new Error("crashed"));
|
||||
held = null;
|
||||
}
|
||||
for (let q = 1; q <= n; q += 1) if (ph[q - 1] === "putting") ph[q - 1] = "lost";
|
||||
for (const gate of Object.values(gates)) gate.resolve(true);
|
||||
for (const { gate } of Object.values(toolGates)) gate.resolve(true);
|
||||
const left = engine;
|
||||
engine = null;
|
||||
up = "down";
|
||||
void left.close().catch(() => {});
|
||||
return;
|
||||
}
|
||||
case "OpenBegin": {
|
||||
life = { alive: true };
|
||||
try {
|
||||
engine = await createFxEngine(engineOptions(persistenceFor(life), life));
|
||||
} catch (error) {
|
||||
if (error?.code !== "FX_JOURNAL_IMAGE_MISSING") throw error;
|
||||
up = "failed";
|
||||
return;
|
||||
}
|
||||
up = "open";
|
||||
// A turn the crash left open continues.
|
||||
const resumed = engine.resume();
|
||||
if (resumed) {
|
||||
const open = ph.findIndex((phase) => phase === "started") + 1;
|
||||
if (open === 0) problems.push("the engine resumed a turn the model has no started prompt for");
|
||||
void resumed.result.catch(() => {});
|
||||
watch(open, resumed);
|
||||
}
|
||||
return;
|
||||
}
|
||||
case "Lose": {
|
||||
lost = true;
|
||||
storedImages.delete(nameOf[p]);
|
||||
return;
|
||||
}
|
||||
default: throw new Error(`the driver cannot take ${name}`);
|
||||
}
|
||||
}
|
||||
|
||||
function observe() {
|
||||
// A prompt whose model request is held has started.
|
||||
if (held && held.life === life && ph[held.p - 1] !== "ended") ph[held.p - 1] = "started";
|
||||
const { named, inline } = savedImages(store);
|
||||
if (!lost) for (const name of named) if (!storedImages.has(name)) problems.push(`a record or checkpoint names ${labelOfName[name] ?? name}, which the store does not hold`);
|
||||
for (const bytes of inline) {
|
||||
const label = labelOfBytes(bytes);
|
||||
if (label !== "o" && !refused.has(label)) problems.push(`image ${label} is saved inline`);
|
||||
}
|
||||
return {
|
||||
made,
|
||||
ph: [...ph],
|
||||
up,
|
||||
store: [...storedImages.keys()].map((name) => labelOfName[name] ?? "?").sort(),
|
||||
held: held && held.life === life ? held.images : null,
|
||||
};
|
||||
}
|
||||
|
||||
async function settle() {
|
||||
let last = JSON.stringify(observe());
|
||||
let since = Date.now();
|
||||
const begun = Date.now();
|
||||
while (Date.now() - since < quietMs) {
|
||||
await sleep(15);
|
||||
const now = JSON.stringify(observe());
|
||||
if (now !== last) { last = now; since = Date.now(); }
|
||||
if (Date.now() - begun > stepTimeoutMs) break;
|
||||
}
|
||||
return observe();
|
||||
}
|
||||
|
||||
async function close() {
|
||||
if (held) held.response.resolve(answer());
|
||||
for (const gate of Object.values(gates)) gate.resolve(true);
|
||||
for (const { gate } of Object.values(toolGates)) gate.resolve(true);
|
||||
if (engine) await Promise.race([engine.close().catch(() => {}), sleep(3000)]);
|
||||
}
|
||||
|
||||
return { perform, settle, close, problems };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Walks.
|
||||
|
||||
const graph = await loadGraph(resolve(here, "../sdk/graphs", "images.dot"));
|
||||
const n = graph.states.get(graph.init).ph.length;
|
||||
const autoActs = new Set(["Send", "TurnStart", "ToolResult", "Checkpoint", "Stage", "OpenMissing", "OpenOk"]);
|
||||
// While a turn's tool stores its image or hands it on, no request is held.
|
||||
const toolRunning = (state) => state.tph.some((phase) => ["called", "put", "refused"].includes(phase));
|
||||
const view = (state) => JSON.stringify({
|
||||
made: state.made,
|
||||
ph: state.ph,
|
||||
up: state.up,
|
||||
store: [...state.store].sort(),
|
||||
// A model request carries the images the core keeps; a dropped one is a note.
|
||||
held: state.up === "open" && state.ph.includes("started") && !toolRunning(state) ? state.mem.filter((entry) => !entry.dropped).map((entry) => entry.i) : null,
|
||||
});
|
||||
const { steps, covered, totalEdges } = await walkGraph({
|
||||
graph,
|
||||
walks,
|
||||
random,
|
||||
verbose,
|
||||
autoActs,
|
||||
view,
|
||||
nameOf: (act) => act[0],
|
||||
sameAct: (a, b) => JSON.stringify(a) === JSON.stringify(b),
|
||||
label: (act) => `${act[0]}(${act.slice(1).join(",")})`,
|
||||
open: () => harness(n),
|
||||
});
|
||||
if (!process.exitCode) console.log(`ok images: ${walks} walks, ${steps} steps matched the model; ${covered} of ${totalEdges} edges covered (${backend})`);
|
||||
process.exit(process.exitCode ?? 0);
|
||||
@@ -0,0 +1,466 @@
|
||||
#!/usr/bin/env bun
|
||||
// Drives real libfx along paths of the TLC state graph of PromptIntake.tla,
|
||||
// and checks after every step that the code is in a state the model allows.
|
||||
//
|
||||
// bun tests/tla/drivers/drive-intake.mjs [--walks N] [--seed S] [--durability memory|world]
|
||||
// [--harness fx|scripted] [--backend native|wasm]
|
||||
//
|
||||
// Each walk opens one agent and one session object, which makes the prompts
|
||||
// the walk chooses: text, or a prompt with an image to prepare: base64 data
|
||||
// whose write to the session's image store waits until the driver lets it
|
||||
// land or fail. Every log write
|
||||
// of a prompt, or of a prompt's cancel, waits until the driver lets it land
|
||||
// or fail. With the fx harness each turn runs on a real fx engine, whose
|
||||
// model request the driver holds until it ends the turn; with the scripted
|
||||
// harness a turn is a stand-in the driver ends. After each step the driver
|
||||
// waits until the code is quiet, reads back what it shows, and moves to the
|
||||
// model states that show the same. Every stored prompt must name its image,
|
||||
// already in the image store with the caller's bytes; and every model
|
||||
// request must carry its prompt's image unchanged.
|
||||
//
|
||||
// `tests/tla/sdk/check.sh` writes graphs/intake.dot from PromptIntake-drive.cfg.
|
||||
import { mkdtemp, readFile, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join, resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { isDeepStrictEqual } from "node:util";
|
||||
import { createFxAgent, memory } from "../../../sdk/node.js";
|
||||
import { createDurableAgentFactory } from "../../../sdk/durable.js";
|
||||
import { loadGraph, walkGraph } from "./graph.mjs";
|
||||
|
||||
const args = process.argv.slice(2);
|
||||
const option = (name, fallback) => {
|
||||
const index = args.indexOf(`--${name}`);
|
||||
return index >= 0 ? args[index + 1] : fallback;
|
||||
};
|
||||
const walks = Number(option("walks", 40));
|
||||
let seed = Number(option("seed", 1));
|
||||
const durabilityKind = option("durability", "memory");
|
||||
const harnessKind = option("harness", "fx");
|
||||
const backend = option("backend", "native");
|
||||
if (!["memory", "world"].includes(durabilityKind)) throw new Error("--durability is memory or world");
|
||||
if (!["fx", "scripted"].includes(harnessKind)) throw new Error("--harness is fx or scripted");
|
||||
const verbose = args.includes("--verbose");
|
||||
const here = fileURLToPath(new URL(".", import.meta.url));
|
||||
const addon = resolve(here, "../../../zig-out/lib/libfx.node");
|
||||
const wasm = harnessKind === "fx" && backend === "wasm" ? await readFile(resolve(here, "../../../zig-out/bin/fx-core.wasm")) : undefined;
|
||||
const { appWorldAt, closeAppWorlds } = durabilityKind === "world" ? await import("../../../sdk/tests/app-world.mjs") : {};
|
||||
// How long the code must show nothing new before a step has settled. An fx
|
||||
// engine takes a moment to open, and a World's stream a moment to read.
|
||||
const quietMs = (harnessKind === "fx" ? 200 : 60) + (durabilityKind === "world" ? 350 : 0);
|
||||
const stepTimeoutMs = 10_000;
|
||||
const encoder = new TextEncoder();
|
||||
const decoder = new TextDecoder();
|
||||
const sleep = (ms) => new Promise((done) => setTimeout(done, ms));
|
||||
const until = async (check, ms = stepTimeoutMs) => {
|
||||
const started = Date.now();
|
||||
while (!check()) {
|
||||
if (Date.now() - started > ms) return false;
|
||||
await sleep(5);
|
||||
}
|
||||
return true;
|
||||
};
|
||||
const base64 = (bytes) => Buffer.from(bytes).toString("base64");
|
||||
const deferred = () => {
|
||||
let resolveIt;
|
||||
let rejectIt;
|
||||
const promise = new Promise((resolveP, rejectP) => { resolveIt = resolveP; rejectIt = rejectP; });
|
||||
promise.catch(() => {});
|
||||
return { promise, resolve: resolveIt, reject: rejectIt };
|
||||
};
|
||||
// A PNG the kernel accepts, followed by bytes that name the walk and prompt,
|
||||
// so each prompt's image is its own.
|
||||
const png = Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jP0cAAAAASUVORK5CYII=", "base64");
|
||||
const imageOf = (walk, i) => new Uint8Array(Buffer.concat([png, Buffer.from(`walk-${walk}-prompt-${i}`.padEnd(24, "."))]));
|
||||
|
||||
const nameOf = async (bytes) => Buffer.from(await crypto.subtle.digest("SHA-256", bytes)).toString("hex");
|
||||
// An image's stream on a World, as libfx names it.
|
||||
const imageStreamSuffix = (name) => `_user_${Buffer.from(`libfx-image-${name}`).toString("base64url")}`;
|
||||
|
||||
// A durability whose log writes of prompts and their cancels wait for the
|
||||
// driver, which lets each land or fail, as do the image writes it is told to
|
||||
// hold (`holdImage`). `images` has the bytes of every image write that
|
||||
// landed, by name. Works on memory() and on a World.
|
||||
function heldInputs(durability) {
|
||||
const held = new Map();
|
||||
const landed = [];
|
||||
const images = new Map();
|
||||
const heldImages = new Map();
|
||||
let open = false;
|
||||
const throughImage = async (name, bytes, write) => {
|
||||
const hold = heldImages.get(name);
|
||||
if (hold && !open) {
|
||||
heldImages.delete(name);
|
||||
if (!(await hold.promise)) throw Object.assign(new Error("injected image write failure"), { code: "FX_TEST_FAULT" });
|
||||
}
|
||||
const result = await write();
|
||||
images.set(name, Buffer.from(bytes).toString("base64"));
|
||||
return result;
|
||||
};
|
||||
const keyOf = (entry) => {
|
||||
if (entry?.k !== "input") return null;
|
||||
if (entry.type === "prompt") return `prompt:${entry.messageId}`;
|
||||
if (entry.type === "cancel" && typeof entry.target === "string") return `cancel:${entry.target}`;
|
||||
return null;
|
||||
};
|
||||
const through = async (entry, write) => {
|
||||
const key = keyOf(entry);
|
||||
if (key !== null && !open) {
|
||||
const lands = await new Promise((go) => held.set(key, { go }));
|
||||
held.delete(key);
|
||||
if (!lands) throw Object.assign(new Error("injected log write failure"), { code: "FX_TEST_FAULT" });
|
||||
}
|
||||
const result = await write();
|
||||
if (key !== null) landed.push(entry);
|
||||
return result;
|
||||
};
|
||||
const control = {
|
||||
held,
|
||||
landed,
|
||||
images,
|
||||
// The next write of the image named `name` waits for `settleImage`.
|
||||
holdImage(name) {
|
||||
const gate = {};
|
||||
gate.promise = new Promise((go) => { gate.go = go; });
|
||||
heldImages.set(name, gate);
|
||||
return gate;
|
||||
},
|
||||
async settle(key, lands) {
|
||||
if (!(await until(() => held.has(key)))) throw new Error(`no log write is held for ${key}`);
|
||||
held.get(key).go(lands);
|
||||
},
|
||||
openAll() {
|
||||
open = true;
|
||||
for (const item of [...held.values()]) item.go(true);
|
||||
for (const gate of [...heldImages.values()]) gate.go(true);
|
||||
},
|
||||
};
|
||||
if (typeof durability.create === "function") {
|
||||
const create = durability.create;
|
||||
const wrapped = new WeakMap();
|
||||
return [{
|
||||
...durability,
|
||||
create() {
|
||||
const backend = create();
|
||||
if (!wrapped.has(backend)) {
|
||||
wrapped.set(backend, {
|
||||
...backend,
|
||||
async session(id) {
|
||||
const log = await backend.session(id);
|
||||
return {
|
||||
...log,
|
||||
append: (entry) => through(entry, () => log.append(entry)),
|
||||
images: { put: (name, bytes) => throughImage(name, bytes, () => log.images.put(name, bytes)), get: (name) => log.images.get(name) },
|
||||
};
|
||||
},
|
||||
});
|
||||
}
|
||||
return wrapped.get(backend);
|
||||
},
|
||||
}, control];
|
||||
}
|
||||
const bound = (target, key) => {
|
||||
const value = Reflect.get(target, key);
|
||||
return typeof value === "function" ? value.bind(target) : value;
|
||||
};
|
||||
return [{
|
||||
...durability,
|
||||
async world() {
|
||||
const world = await durability.world();
|
||||
// An image write is a write of its stream; the driver knows its name.
|
||||
const streams = new Proxy(world.streams, {
|
||||
get(target, key) {
|
||||
if (key !== "write") return bound(target, key);
|
||||
return async (runId, stream, chunk) => {
|
||||
const name = [...pendingNames].find((candidate) => String(stream).endsWith(imageStreamSuffix(candidate)));
|
||||
if (name === undefined) return target.write(runId, stream, chunk);
|
||||
return throughImage(name, chunk, () => target.write(runId, stream, chunk));
|
||||
};
|
||||
},
|
||||
});
|
||||
const events = new Proxy(world.events, {
|
||||
get(target, key) {
|
||||
if (key !== "create") return bound(target, key);
|
||||
return async (runId, event, params) => {
|
||||
if (event?.eventType !== "step_created" || event.eventData?.stepName !== "fx.log") return target.create(runId, event, params);
|
||||
// What the World stores, read back as the log reads it.
|
||||
const entry = JSON.parse(decoder.decode(event.eventData.input));
|
||||
return through(entry, () => target.create(runId, event, params));
|
||||
};
|
||||
},
|
||||
});
|
||||
return new Proxy(world, { get: (target, key) => (key === "events" ? events : key === "streams" ? streams : bound(target, key)) });
|
||||
},
|
||||
}, control];
|
||||
}
|
||||
// The names of the walk's images, so a World's stream writes can be told
|
||||
// apart from its other streams'.
|
||||
const pendingNames = new Set();
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The code under test: one agent, one session object.
|
||||
|
||||
const vision = "intake/vision";
|
||||
const usage = { inputTokens: { total: 1 }, outputTokens: { total: 1 } };
|
||||
const answer = (text) => new Response(
|
||||
[{ type: "text-delta", id: "a", delta: text }, { type: "finish", finishReason: { unified: "stop", raw: "stop" }, usage }, "[DONE]"]
|
||||
.map((frame) => `data: ${typeof frame === "string" ? frame : JSON.stringify(frame)}\n\n`).join(""),
|
||||
{ headers: { "content-type": "text/event-stream" } },
|
||||
);
|
||||
const promptNumber = (text) => Number(/prompt-(\d+)/.exec(text ?? "")?.[1]);
|
||||
|
||||
async function harness(walk, n) {
|
||||
const dir = await mkdtemp(join(tmpdir(), "libfx-intake-"));
|
||||
const [durability, writes] = heldInputs(durabilityKind === "world" ? await appWorldAt(dir) : memory());
|
||||
const expected = {};
|
||||
const names = {};
|
||||
const kinds = {};
|
||||
const imageGates = {};
|
||||
const controllers = {};
|
||||
const turnsRunning = new Map();
|
||||
const started = new Set();
|
||||
const acc = {};
|
||||
const accError = {};
|
||||
const res = {};
|
||||
const early = {};
|
||||
const problems = [];
|
||||
let made = 0;
|
||||
let closed = false;
|
||||
let closing = null;
|
||||
// Each turn's image as the model or the stand-in turn received it; a text
|
||||
// prompt's turn receives none.
|
||||
const received = (i, image) => {
|
||||
if (kinds[i] === "text") {
|
||||
if (image !== undefined) problems.push(`text prompt ${i}'s turn received an image`);
|
||||
return;
|
||||
}
|
||||
if (image?.data !== expected[i] || image?.mimeType !== "image/png") problems.push(`prompt ${i}'s turn received ${JSON.stringify(image).slice(0, 120)}`);
|
||||
};
|
||||
const turnStarted = (i, end, abortWith) => {
|
||||
if (started.has(i)) problems.push(`prompt ${i}'s turn started twice`);
|
||||
started.add(i);
|
||||
turnsRunning.set(i, { end, abortWith });
|
||||
};
|
||||
let agent;
|
||||
if (harnessKind === "fx") {
|
||||
agent = createFxAgent({
|
||||
backend,
|
||||
nativeAddon: addon,
|
||||
...(wasm ? { wasm } : {}),
|
||||
apiKey: "intake-key",
|
||||
model: vision,
|
||||
durability,
|
||||
async fetch(input, init) {
|
||||
// The WebAssembly host passes a Request, the native one a URL and init.
|
||||
const request = input instanceof Request ? input : new Request(input, init);
|
||||
if (request.method === "GET") return Response.json({ object: "list", data: [{ id: vision, type: "language", tags: ["vision", "file-input"] }] });
|
||||
const prompt = JSON.parse(await request.text()).prompt;
|
||||
const last = prompt.filter((message) => message.role === "user").at(-1);
|
||||
const parts = Array.isArray(last?.content) ? last.content : [];
|
||||
const i = promptNumber(parts.filter((part) => part.type === "text").map((part) => part.text).join(""));
|
||||
const images = parts.filter((part) => part.type === "file");
|
||||
if (images.length !== (kinds[i] === "text" ? 0 : 1)) problems.push(`prompt ${i}'s model request carries ${images.length} images`);
|
||||
received(i, images[0] && { data: images[0].data?.data, mimeType: images[0].mediaType });
|
||||
const signal = init?.signal ?? request.signal;
|
||||
return new Promise((resolveResponse, rejectResponse) => {
|
||||
turnStarted(i, () => { turnsRunning.delete(i); resolveResponse(answer(`answered ${i}`)); });
|
||||
signal?.addEventListener("abort", () => { turnsRunning.delete(i); rejectResponse(signal.reason ?? new Error("aborted")); }, { once: true });
|
||||
});
|
||||
},
|
||||
});
|
||||
} else {
|
||||
agent = createDurableAgentFactory({
|
||||
harness: () => ({
|
||||
async open({ store }) {
|
||||
await store.load();
|
||||
let key = 0;
|
||||
const append = (record, marks) => store.append({ idempotencyKey: `intake:${Date.now()}:${key++}`, data: encoder.encode(JSON.stringify(record)), marks });
|
||||
return {
|
||||
prompt(input, { turnId }) {
|
||||
const i = promptNumber(input?.[0]?.text);
|
||||
received(i, input?.[1]);
|
||||
const ended = deferred();
|
||||
const result = (async () => {
|
||||
await append({ turnId, start: true }, [{ start: turnId }]);
|
||||
turnStarted(i, () => { turnsRunning.delete(i); ended.resolve("end_turn"); });
|
||||
const stopReason = await ended.promise;
|
||||
turnsRunning.delete(i);
|
||||
await append({ turnId, end: true }, [{ end: true }]);
|
||||
return { stopReason, usage: {} };
|
||||
})();
|
||||
return { result, steer: async () => {}, cancel: () => ended.resolve("cancelled"), async *[Symbol.asyncIterator]() { await result; } };
|
||||
},
|
||||
resume() { throw new Error("no walk leaves a turn open"); },
|
||||
get openTurn() { return null; },
|
||||
settled: async () => {},
|
||||
saveCheckpoint: async () => {},
|
||||
exportCheckpoint: async () => new Uint8Array(),
|
||||
close: async () => {},
|
||||
};
|
||||
},
|
||||
}),
|
||||
defaultDurability: async () => memory(),
|
||||
})({ durability });
|
||||
}
|
||||
const session = agent.session();
|
||||
|
||||
async function perform(act) {
|
||||
const i = act.p;
|
||||
switch (act.name) {
|
||||
case "Make": {
|
||||
const controller = new AbortController();
|
||||
if (act.early) controller.abort();
|
||||
const image = imageOf(walk, i);
|
||||
expected[i] = base64(image);
|
||||
names[i] = await nameOf(image);
|
||||
kinds[i] = act.kind;
|
||||
early[i] = act.early;
|
||||
const blocks = [{ type: "text", text: `prompt-${i}` }];
|
||||
// An image prompt's image write is held until the driver lets it go.
|
||||
if (act.kind === "image") {
|
||||
blocks.push({ type: "image", data: expected[i], mimeType: "image/png" });
|
||||
pendingNames.add(names[i]);
|
||||
imageGates[i] = writes.holdImage(names[i]);
|
||||
}
|
||||
const turn = session.prompt(blocks, { messageId: `p${i}`, signal: controller.signal });
|
||||
controllers[i] = controller;
|
||||
made = i;
|
||||
acc[i] = "pending";
|
||||
res[i] = "pending";
|
||||
turn.accepted.then(() => { acc[i] = "resolved"; }, (error) => { acc[i] = "rejected"; accError[i] = error; });
|
||||
turn.result.then(
|
||||
(result) => { res[i] = result.stopReason === "end_turn" ? "end" : result.stopReason === "cancelled" ? "cancelled" : `other:${result.stopReason}:${result.error?.message ?? ""}`; },
|
||||
() => { res[i] = "rejected"; },
|
||||
);
|
||||
return;
|
||||
}
|
||||
case "Prepared":
|
||||
imageGates[i].go(true);
|
||||
return;
|
||||
case "PrepareFail":
|
||||
imageGates[i].go(false);
|
||||
return;
|
||||
case "Store": return writes.settle(`prompt:p${i}`, true);
|
||||
case "StoreFail": return writes.settle(`prompt:p${i}`, false);
|
||||
case "Abort": controllers[i].abort(); return;
|
||||
case "CancelStore": return writes.settle(`cancel:p${i}`, true);
|
||||
case "Close": {
|
||||
closed = true;
|
||||
closing = agent.close();
|
||||
// A closed agent's session object takes no more prompts.
|
||||
try {
|
||||
session.prompt("after close");
|
||||
problems.push("prompt() after close did not throw");
|
||||
} catch (error) {
|
||||
if (!/is closed/.test(error.message)) problems.push(`prompt() after close threw ${error.message}`);
|
||||
}
|
||||
return;
|
||||
}
|
||||
case "End": {
|
||||
if (!(await until(() => turnsRunning.has(i)))) throw new Error(`prompt ${i} has no turn to end`);
|
||||
turnsRunning.get(i).end();
|
||||
return;
|
||||
}
|
||||
default: throw new Error(`the driver cannot take ${act.name}`);
|
||||
}
|
||||
}
|
||||
|
||||
function observe() {
|
||||
const stored = writes.landed.filter((entry) => entry.type === "prompt");
|
||||
const cancels = new Set(writes.landed.filter((entry) => entry.type === "cancel").map((entry) => entry.target));
|
||||
for (const entry of stored) {
|
||||
const i = promptNumber(entry.input?.[0]?.text);
|
||||
const image = entry.input?.[1];
|
||||
// What the log holds is JSON. An image prompt names its image, which
|
||||
// the image store already holds, with the caller's original bytes.
|
||||
if (!isDeepStrictEqual(JSON.parse(JSON.stringify(entry)), entry)) problems.push(`prompt ${i}'s entry is not JSON`);
|
||||
if (kinds[i] === "text") {
|
||||
if (entry.input?.length !== 1) problems.push(`text prompt ${i} stored ${JSON.stringify(entry.input).slice(0, 120)}`);
|
||||
continue;
|
||||
}
|
||||
if (image?.data?.sha256 !== names[i] || Object.keys(image.data).length !== 1 || image?.mimeType !== "image/png" || image?.type !== "image") problems.push(`prompt ${i} stored ${JSON.stringify(image).slice(0, 120)}`);
|
||||
if (writes.images.get(names[i]) !== expected[i]) problems.push(`prompt ${i} was stored before its image, or with other bytes`);
|
||||
}
|
||||
const order = stored.map((entry) => promptNumber(entry.input?.[0]?.text));
|
||||
const ph = [];
|
||||
for (let i = 1; i <= n; i += 1) {
|
||||
if (i > made) ph.push("unmade");
|
||||
else if (early[i]) ph.push("early");
|
||||
else if (order.includes(i)) ph.push("stored");
|
||||
else if (writes.held.has(`prompt:p${i}`)) ph.push("sending");
|
||||
else if (acc[i] === "rejected") ph.push(/is closed/.test(accError[i]?.message ?? "") ? "refused" : "failed");
|
||||
// Accepted, yet neither stored nor on its way: an abort withdrew it.
|
||||
else if (acc[i] === "resolved") ph.push("withdrawn");
|
||||
else ph.push("preparing");
|
||||
}
|
||||
const each = (fn) => Array.from({ length: n }, (_, index) => fn(index + 1));
|
||||
return {
|
||||
made,
|
||||
closed,
|
||||
log: order,
|
||||
ph,
|
||||
acc: each((i) => acc[i] ?? "pending"),
|
||||
res: each((i) => res[i] ?? "pending"),
|
||||
cancel: each((i) => (cancels.has(`p${i}`) ? "stored" : writes.held.has(`cancel:p${i}`) ? "held" : "none")),
|
||||
ran: each((i) => started.has(i)),
|
||||
};
|
||||
}
|
||||
|
||||
// Waits until the code shows nothing new for quietMs.
|
||||
async function settle() {
|
||||
let last = JSON.stringify(observe());
|
||||
let since = Date.now();
|
||||
const begun = Date.now();
|
||||
while (Date.now() - since < quietMs) {
|
||||
await sleep(15);
|
||||
const now = JSON.stringify(observe());
|
||||
if (now !== last) { last = now; since = Date.now(); }
|
||||
if (Date.now() - begun > stepTimeoutMs) break;
|
||||
}
|
||||
return observe();
|
||||
}
|
||||
|
||||
async function close() {
|
||||
writes.openAll();
|
||||
pendingNames.clear();
|
||||
for (const turn of [...turnsRunning.values()]) turn.end();
|
||||
await Promise.race([(closing ?? agent.close()).catch(() => {}), sleep(3000)]);
|
||||
await closeAppWorlds?.();
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
return { perform, settle, close, problems };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Walks.
|
||||
|
||||
const random = () => {
|
||||
seed = (seed * 1103515245 + 12345) % 2147483648;
|
||||
return seed / 2147483648;
|
||||
};
|
||||
|
||||
const graph = await loadGraph(resolve(here, "../sdk/graphs", "intake.dot"));
|
||||
const n = graph.states.get(graph.init).ph.length;
|
||||
const autoActs = new Set(["Released", "Start", "Skip", "CancelRun"]);
|
||||
const view = (state) => JSON.stringify({
|
||||
made: state.made, closed: state.closed, log: state.log, ph: state.ph, acc: state.acc, res: state.res, cancel: state.cancel, ran: state.ran,
|
||||
});
|
||||
const { steps, covered, totalEdges } = await walkGraph({
|
||||
graph,
|
||||
walks,
|
||||
random,
|
||||
verbose,
|
||||
autoActs,
|
||||
view,
|
||||
nameOf: (act) => act.name,
|
||||
sameAct: (a, b) => a.name === b.name && a.p === b.p && a.kind === b.kind && a.early === b.early,
|
||||
label: (act) => `${act.name}(${act.p}${act.name === "Make" ? `,${act.kind}${act.early ? ",early" : ""}` : ""})`,
|
||||
// Closing ends what a walk can do, so it comes late more often than not.
|
||||
choose: (pool) => (pool.length > 1 && random() < 0.75 ? pool.filter((item) => item.act.name !== "Close") : pool),
|
||||
open: (walk) => harness(walk, n),
|
||||
});
|
||||
if (!process.exitCode) console.log(`ok intake: ${walks} walks, ${steps} steps matched the model; ${covered} of ${totalEdges} edges covered (${harnessKind}, ${durabilityKind}${harnessKind === "fx" ? `, ${backend}` : ""})`);
|
||||
// A walk that ended with prompts stored after close leaves the memory
|
||||
// queue retrying with nobody listening; nothing more runs.
|
||||
process.exit(process.exitCode ?? 0);
|
||||
@@ -231,8 +231,9 @@ async function harness() {
|
||||
for (const [messageId, owner] of ownerOf) if (owner) stateOf.set(owner, ended.get(messageId) ?? "running");
|
||||
// The turns the model was sent, from the newest request: a message that
|
||||
// started a turn on the stream opens it, and every other message joins
|
||||
// the turn before it. A steer a resumed turn replays reaches the model as
|
||||
// a plain message, so the stream, not the text, says who owns a turn.
|
||||
// the turn before it. A steer that joined a turn reaches the model as a
|
||||
// live update, a resumed turn's too, but the stream, not the text, says
|
||||
// who owns a turn.
|
||||
const turns = [];
|
||||
const counts = new Map();
|
||||
for (const text of requests.at(-1)?.users ?? []) {
|
||||
|
||||
+100
-2
@@ -1,4 +1,5 @@
|
||||
// The state graphs tests/tla/sdk/check.sh has TLC write, as the drivers read them.
|
||||
// The state graphs tests/tla/sdk/check.sh has TLC write, as the drivers read
|
||||
// them, and the walk drive-intake.mjs and drive-images.mjs take along one.
|
||||
import { strict as assert } from "node:assert";
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
@@ -91,9 +92,106 @@ export async function loadGraph(path) {
|
||||
if (match) state[match[1]] = parseValue(match[2].trim());
|
||||
}
|
||||
states.set(id, state);
|
||||
if (state.act?.name === "Init") init = id;
|
||||
// `act` is a record with a name, or a tuple whose first item is one.
|
||||
if (state.act?.name === "Init" || (Array.isArray(state.act) && state.act[0] === "Init")) init = id;
|
||||
}
|
||||
}
|
||||
assert.ok(init, "the graph has an initial state");
|
||||
return { states, next, init };
|
||||
}
|
||||
|
||||
// Walks the code along `graph`, `walks` times. Each walk opens the code with
|
||||
// `open(walk)`, which returns `{ perform(act), settle(), close(), problems }`.
|
||||
// Each step is one the model allows from where the walk stands, chosen at
|
||||
// random among those that cross an edge no walk has crossed yet, if any;
|
||||
// `choose` may narrow that pool. After it the code must show, through
|
||||
// `view`, a state the model reaches once the code has taken every step in
|
||||
// `autoActs` it can on its own; `nameOf` gives a step's action name. A walk
|
||||
// the model does not allow, or whose code reports a problem, prints its path
|
||||
// and sets `process.exitCode`, and no more walks run. Returns the steps taken
|
||||
// and the edges covered.
|
||||
export async function walkGraph({ graph, walks, random, autoActs, nameOf, sameAct, label, view, open, choose = (pool) => pool, verbose = false }) {
|
||||
const actOf = (id) => graph.states.get(id).act;
|
||||
const successors = (id) => [...(graph.next.get(id) ?? [])].filter((to) => to !== id);
|
||||
// The states the code settles in after a step: every path of the steps it
|
||||
// takes on its own, to where it can take none.
|
||||
const settledFrom = (id, edges) => {
|
||||
const seen = new Set([id]);
|
||||
const queue = [id];
|
||||
const quiet = [];
|
||||
while (queue.length > 0) {
|
||||
const from = queue.shift();
|
||||
const autos = successors(from).filter((to) => autoActs.has(nameOf(actOf(to))));
|
||||
if (autos.length === 0) quiet.push(from);
|
||||
for (const to of autos) {
|
||||
edges.push(`${from}>${to}`);
|
||||
if (!seen.has(to)) { seen.add(to); queue.push(to); }
|
||||
}
|
||||
}
|
||||
return quiet;
|
||||
};
|
||||
const covered = new Set();
|
||||
const totalEdges = [...graph.next.values()].reduce((sum, targets) => sum + [...targets].length, 0);
|
||||
let steps = 0;
|
||||
for (let walk = 1; walk <= walks; walk += 1) {
|
||||
const code = await open(walk);
|
||||
const path = [];
|
||||
let current = new Set(settledFrom(graph.init, []));
|
||||
try {
|
||||
for (;;) {
|
||||
const options = [];
|
||||
for (const from of current) {
|
||||
for (const to of successors(from)) {
|
||||
const act = actOf(to);
|
||||
if (autoActs.has(nameOf(act))) continue;
|
||||
let item = options.find((entry) => sameAct(entry.act, act));
|
||||
if (!item) options.push(item = { act, fresh: false });
|
||||
if (!covered.has(`${from}>${to}`)) item.fresh = true;
|
||||
}
|
||||
}
|
||||
if (options.length === 0) break;
|
||||
const fresh = options.filter((item) => item.fresh);
|
||||
const pool = choose(fresh.length > 0 ? fresh : options);
|
||||
const { act } = pool[Math.floor(random() * pool.length)];
|
||||
path.push(label(act));
|
||||
if (verbose) console.log(`walk ${walk}: ${label(act)}`);
|
||||
await code.perform(act);
|
||||
const shown = await code.settle();
|
||||
if (code.problems.length > 0) throw new Error([...new Set(code.problems)].join("; "));
|
||||
const seen = JSON.stringify(shown);
|
||||
if (verbose) console.log(` shows ${seen}`);
|
||||
steps += 1;
|
||||
const next = new Set();
|
||||
const allowed = new Set();
|
||||
for (const from of current) {
|
||||
for (const to of successors(from).filter((id) => sameAct(actOf(id), act))) {
|
||||
const edges = [`${from}>${to}`];
|
||||
for (const end of settledFrom(to, edges)) {
|
||||
allowed.add(view(graph.states.get(end)));
|
||||
if (view(graph.states.get(end)) !== seen) continue;
|
||||
next.add(end);
|
||||
for (const edge of edges) covered.add(edge);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (next.size === 0) {
|
||||
console.error(`FAIL walk ${walk} after ${path.join(" ")}`);
|
||||
console.error(` the code shows ${seen}`);
|
||||
for (const item of allowed) console.error(` the model allows ${item}`);
|
||||
process.exitCode = 1;
|
||||
break;
|
||||
}
|
||||
current = next;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`FAIL walk ${walk} after ${path.join(" ")}: ${error.message}`);
|
||||
process.exitCode = 1;
|
||||
} finally {
|
||||
await code.close();
|
||||
}
|
||||
if (process.exitCode) break;
|
||||
if (walk % 25 === 0 || walk === walks) console.log(`${walk} walks, ${steps} steps, ${covered.size} of ${totalEdges} edges`);
|
||||
}
|
||||
assert.ok(steps > 0 || process.exitCode, "the walks took steps");
|
||||
return { steps, covered: covered.size, totalEdges };
|
||||
}
|
||||
@@ -16,8 +16,9 @@
|
||||
\* document lacks), and the document and the note are written at once,
|
||||
\* even by `get`. A document with the flag but no note, saved before the
|
||||
\* note existed, gets the note.
|
||||
\* Operations: get (config.resolve -> Runtime.savedCredential, for every
|
||||
\* HTTP server, whenever a host starts: `fx mcp list`, `fx ask`, the shell),
|
||||
\* Operations: get (Store.snapshot from Runtime.restoreCredentials, one read
|
||||
\* for every HTTP server, on the owner thread whenever a host starts:
|
||||
\* `fx mcp list`, `fx ask`, the shell),
|
||||
\* put (a `credential` event after a sign-in or refresh), remove (logout:
|
||||
\* runtime.zig `.logout`; verbs.zig remove and logout).
|
||||
\*
|
||||
@@ -100,8 +101,8 @@ ReadEffect ==
|
||||
ELSE IF ~marked THEN Mark
|
||||
ELSE Same
|
||||
|
||||
\* Store.get, from config.resolve when a host starts. An unreadable document
|
||||
\* gives no login and stays as it is.
|
||||
\* Store.snapshot, from Runtime.restoreCredentials when a host starts. An
|
||||
\* unreadable document gives no login and stays as it is.
|
||||
Get ==
|
||||
/\ ReadEffect
|
||||
/\ UNCHANGED <<v1, loggedOut, kept>>
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
\* Witness: an open that drops an image the store lost breaks OpenExact.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = TRUE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = FALSE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
@@ -0,0 +1,33 @@
|
||||
\* Witness: a window that never counts a tool image the store refused, held
|
||||
\* inline, lets the session hold more than Window images, breaking
|
||||
\* WindowBound.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = FALSE
|
||||
Tools = TRUE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
FailsOnlyOnLossInWindow
|
||||
WindowHeld
|
||||
WindowBound
|
||||
@@ -0,0 +1,24 @@
|
||||
\* Witness: a core that keeps every image, past its window, breaks WindowHeld.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 1
|
||||
LiveWindow = FALSE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
WindowHeld
|
||||
@@ -0,0 +1,28 @@
|
||||
\* Witness: a checkpoint naming an image the store never held breaks NoDanglingRef.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = TRUE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = FALSE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
@@ -0,0 +1,25 @@
|
||||
\* Witness: an open that stages every named image fails when the store lost
|
||||
\* one outside the window, and breaks FailsOnlyOnLossInWindow.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 1
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = FALSE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
FailsOnlyOnLossInWindow
|
||||
@@ -0,0 +1,29 @@
|
||||
\* Witness: a retried prompt's input that lands names an image the store may
|
||||
\* never have got, and breaks NoDanglingRef.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = TRUE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 1
|
||||
DropRepeats = FALSE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
@@ -0,0 +1,28 @@
|
||||
\* Witness: a prompt sent before its image is stored breaks NoDanglingRef.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = FALSE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = FALSE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
@@ -0,0 +1,31 @@
|
||||
\* Witness: a tool result handed to the core before its image is stored breaks
|
||||
\* NoDanglingRef.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = TRUE
|
||||
ToolStoreFirst = FALSE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
FailsOnlyOnLossInWindow
|
||||
WindowHeld
|
||||
@@ -0,0 +1,34 @@
|
||||
\* The graph drive-images.mjs walks: a plain engine with an image store, two
|
||||
\* prompts, an image an older libfx saved inline, a window of one image, so
|
||||
\* each prompt drops the image before it, and driver steps only once
|
||||
\* the code has settled. ImageStore.cfg checks the durable session.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 1
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = TRUE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION DriveSpec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
FailsOnlyOnLossInWindow
|
||||
WindowHeld
|
||||
WindowBound
|
||||
@@ -0,0 +1,31 @@
|
||||
\* A plain engine with an image store: prompts go straight to the core.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = FALSE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 0
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = TRUE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
FailsOnlyOnLossInWindow
|
||||
WindowHeld
|
||||
WindowBound
|
||||
@@ -0,0 +1,31 @@
|
||||
\* A durable session: prompts pass through the log, which names their images.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
Imgs = {"a", "b"}
|
||||
Old = TRUE
|
||||
Durable = TRUE
|
||||
MaxFaults = 1
|
||||
MaxCrashes = 2
|
||||
MaxLoss = 1
|
||||
StoreFirst = TRUE
|
||||
MarkOnlyStored = TRUE
|
||||
FailOnMissing = TRUE
|
||||
MaxRetries = 1
|
||||
DropRepeats = TRUE
|
||||
Window = 2
|
||||
LiveWindow = TRUE
|
||||
WindowAtOpen = TRUE
|
||||
CountInline = TRUE
|
||||
Tools = TRUE
|
||||
ToolStoreFirst = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
NoDanglingRef
|
||||
OneCopy
|
||||
OpenExact
|
||||
KnownStoredIsStored
|
||||
FailsOnlyOnLoss
|
||||
FailsOnlyOnLossInWindow
|
||||
WindowHeld
|
||||
WindowBound
|
||||
@@ -0,0 +1,528 @@
|
||||
------------------------------ MODULE ImageStore ------------------------------
|
||||
(***************************************************************************)
|
||||
(* How a libfx session saves its images: each image once, in the host's *)
|
||||
(* store, named by the hash of its bytes, while the session's records and *)
|
||||
(* checkpoints hold only the name. *)
|
||||
(* *)
|
||||
(* A prompt's image goes to the store first and the prompt is sent only *)
|
||||
(* once the write landed; with StoreFirst = FALSE it is sent while the *)
|
||||
(* write is in flight (ImageStore-SendBeforeStore.cfg). With Durable the *)
|
||||
(* prompt is stored in the session's log after its image, naming it, and *)
|
||||
(* the worker resolves the name before it runs the turn. *)
|
||||
(* *)
|
||||
(* A turn's first record names its image. A checkpoint writes the whole *)
|
||||
(* history again: it names the images the core knows are stored and keeps *)
|
||||
(* the rest inline. With Old the session starts with an image an older *)
|
||||
(* libfx saved inline, which the store never held; with MarkOnlyStored = *)
|
||||
(* FALSE a checkpoint names it anyway (ImageStore-NameUnstored.cfg). *)
|
||||
(* *)
|
||||
(* A durable caller may retry a prompt the log already holds, under its *)
|
||||
(* messageId, carrying any image. The retry stores nothing: the log keeps *)
|
||||
(* the input it holds and drops the retry's. With DropRepeats = FALSE the *)
|
||||
(* retry's input lands too, naming an image the store may never have got *)
|
||||
(* (ImageStore-RetryLands.cfg). *)
|
||||
(* *)
|
||||
(* The session keeps only its newest Window images. When a turn enters the *)
|
||||
(* history, the core drops each image older than the window: it keeps the *)
|
||||
(* name and gives up the bytes, and a checkpoint saves it by name alone. *)
|
||||
(* An open counts the history alone, as that turn's end did, and stages *)
|
||||
(* the named images inside the window and the open turn's. With LiveWindow *)
|
||||
(* = FALSE the core keeps every image (ImageStore-KeepAll.cfg); with *)
|
||||
(* WindowAtOpen = FALSE an open stages every named image, so one the store *)
|
||||
(* lost outside the window fails it (ImageStore-NeedAll.cfg). *)
|
||||
(* *)
|
||||
(* With Tools a turn's host tool may return an image. The engine stores it *)
|
||||
(* before it hands the result to the core, whose next record names it; a *)
|
||||
(* store that refuses it leaves it in the result, held inline. The window *)
|
||||
(* counts it as any image, and once it leaves it keeps the name of its *)
|
||||
(* bytes; with CountInline = FALSE the window never counts it, so the *)
|
||||
(* session holds more than Window images (ImageStore-InlineUncounted.cfg). *)
|
||||
(* With ToolStoreFirst = FALSE the result reaches the core while the write *)
|
||||
(* is in flight (ImageStore-ToolBeforeStore.cfg). A tool image joins the *)
|
||||
(* window with its turn, as the prompt's does, and an open stages every *)
|
||||
(* image of the open turn. *)
|
||||
(* *)
|
||||
(* The core may crash, losing what it held and the image writes in flight. *)
|
||||
(* It opens again in two phases: it names the images it needs, the host *)
|
||||
(* stages each from its store, then the core opens. An image the store *)
|
||||
(* lost fails the open; with FailOnMissing = FALSE the open drops it *)
|
||||
(* (ImageStore-DropMissing.cfg). *)
|
||||
(* *)
|
||||
(* `act` names each step and its arguments, so a driver can replay any *)
|
||||
(* path of the state graph against the real code. *)
|
||||
(***************************************************************************)
|
||||
EXTENDS Naturals, Sequences, FiniteSets
|
||||
|
||||
CONSTANTS
|
||||
N, \* prompts the caller makes, at most
|
||||
Imgs, \* images a prompt may carry; equal images share a name
|
||||
Old, \* whether the session starts with an image saved inline
|
||||
Durable, \* whether prompts pass through a durable session's log
|
||||
MaxFaults, \* bound on failed image writes
|
||||
MaxCrashes, \* bound on crashes of the core
|
||||
MaxLoss, \* bound on images the store loses
|
||||
StoreFirst, \* whether a prompt is sent only after its image is stored
|
||||
MarkOnlyStored, \* whether the core names only images it knows are stored
|
||||
FailOnMissing, \* whether an open that misses an image fails
|
||||
MaxRetries, \* bound on retries of prompts the log holds
|
||||
DropRepeats, \* whether the log drops a retried prompt's input
|
||||
Window, \* the newest images the session keeps
|
||||
LiveWindow, \* whether the core drops images older than the window
|
||||
WindowAtOpen, \* whether an open stages only the window's images
|
||||
CountInline, \* whether the window counts a tool image held inline
|
||||
Tools, \* whether a turn's tool may return an image
|
||||
ToolStoreFirst \* whether a tool's result reaches the core only after its image is stored
|
||||
|
||||
ASSUME N \in Nat /\ MaxFaults \in Nat /\ MaxCrashes \in Nat /\ MaxLoss \in Nat
|
||||
ASSUME Old \in BOOLEAN /\ Durable \in BOOLEAN /\ StoreFirst \in BOOLEAN
|
||||
ASSUME MarkOnlyStored \in BOOLEAN /\ FailOnMissing \in BOOLEAN
|
||||
ASSUME MaxRetries \in Nat /\ DropRepeats \in BOOLEAN
|
||||
ASSUME Window \in Nat \ {0} /\ LiveWindow \in BOOLEAN /\ WindowAtOpen \in BOOLEAN
|
||||
ASSUME CountInline \in BOOLEAN
|
||||
ASSUME Tools \in BOOLEAN /\ ToolStoreFirst \in BOOLEAN
|
||||
ASSUME "o" \notin Imgs
|
||||
|
||||
Prompts == 1..N
|
||||
None == "none"
|
||||
OldImg == "o"
|
||||
AllImgs == Imgs \cup {OldImg}
|
||||
Phases == {"unmade", "putting", "put", "queued", "sent", "started", "ended", "failed", "lost"}
|
||||
CoreStates == {"open", "down", "opening", "failed"}
|
||||
\* Where each turn's tool image is: none yet; called, its write in flight;
|
||||
\* put or refused by the store; given to the core; or lost to a crash.
|
||||
ToolPhases == {"none", "called", "put", "refused", "given", "lost"}
|
||||
|
||||
VARIABLES
|
||||
made, \* how many prompts the caller has made
|
||||
img, \* each prompt's image
|
||||
ph, \* where each prompt is
|
||||
putting, \* the prompts whose image write is in flight
|
||||
store, \* the images the host's store holds
|
||||
lostImgs, \* the images the store lost
|
||||
saved, \* the saved history: each turn's image, named, inline or dropped
|
||||
inputs, \* with Durable: the prompts the log holds, in the order they landed
|
||||
mem, \* the core's history while it runs: each image, whether it is known stored, and whether it was dropped
|
||||
up, \* the core: open, down, opening, or failed to open
|
||||
staged, \* the images staged for the open in progress
|
||||
dirty, \* whether records landed since the last checkpoint
|
||||
faults, \* failed image writes so far
|
||||
crashes, \* crashes so far
|
||||
retries, \* retries so far
|
||||
repeats, \* the images retried inputs that landed in the log name
|
||||
tool, \* the image each turn's tool returned
|
||||
tph, \* where each turn's tool image is
|
||||
tputting, \* the turns whose tool image write is in flight
|
||||
turnAt, \* where the open turn's images start in the saved history
|
||||
act \* the last step and its arguments
|
||||
|
||||
tools == <<tool, tph, tputting, turnAt>>
|
||||
vars == <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools, act>>
|
||||
|
||||
TypeOK ==
|
||||
/\ made \in 0..N
|
||||
/\ img \in [Prompts -> Imgs \cup {None}]
|
||||
/\ ph \in [Prompts -> Phases]
|
||||
/\ putting \subseteq Prompts
|
||||
/\ store \subseteq AllImgs
|
||||
/\ lostImgs \subseteq AllImgs
|
||||
/\ saved \in Seq([i : AllImgs, ref : BOOLEAN, dropped : BOOLEAN, named : BOOLEAN])
|
||||
/\ inputs \in Seq(Prompts)
|
||||
/\ mem \in Seq([i : AllImgs, stored : BOOLEAN, dropped : BOOLEAN, named : BOOLEAN])
|
||||
/\ up \in CoreStates
|
||||
/\ staged \subseteq AllImgs
|
||||
/\ dirty \in BOOLEAN
|
||||
/\ faults \in 0..MaxFaults
|
||||
/\ crashes \in 0..MaxCrashes
|
||||
/\ retries \in 0..MaxRetries
|
||||
/\ repeats \subseteq Imgs
|
||||
/\ tool \in [Prompts -> Imgs \cup {None}]
|
||||
/\ tph \in [Prompts -> ToolPhases]
|
||||
/\ tputting \subseteq Prompts
|
||||
/\ turnAt \in Nat
|
||||
|
||||
Init ==
|
||||
/\ made = 0
|
||||
/\ img = [p \in Prompts |-> None]
|
||||
/\ ph = [p \in Prompts |-> "unmade"]
|
||||
/\ putting = {}
|
||||
/\ store = {}
|
||||
/\ lostImgs = {}
|
||||
/\ saved = IF Old THEN << [i |-> OldImg, ref |-> FALSE, dropped |-> FALSE, named |-> TRUE] >> ELSE << >>
|
||||
/\ inputs = << >>
|
||||
/\ mem = << >>
|
||||
/\ up = "down"
|
||||
/\ staged = {}
|
||||
/\ dirty = FALSE
|
||||
/\ faults = 0
|
||||
/\ crashes = 0
|
||||
/\ retries = 0
|
||||
/\ repeats = {}
|
||||
/\ tool = [p \in Prompts |-> None]
|
||||
/\ tph = [p \in Prompts |-> "none"]
|
||||
/\ tputting = {}
|
||||
/\ turnAt = 0
|
||||
/\ act = << "Init" >>
|
||||
|
||||
\* The images the saved history names.
|
||||
Named == {saved[k].i : k \in {j \in 1..Len(saved) : saved[j].ref}}
|
||||
\* Whether image k of s is one the window keeps when it counts the first n:
|
||||
\* it counts every image, a tool image the store refused, held inline,
|
||||
\* too, and keeps every image past the first n. With CountInline = FALSE it
|
||||
\* counts only images with a name, which a refused tool image lacks.
|
||||
WindowKeeps(s, k, n) ==
|
||||
IF CountInline THEN Cardinality(k..n) <= Window
|
||||
ELSE ~s[k].named \/ Cardinality({j \in k..n : s[j].named}) <= Window
|
||||
\* A turn has started and not ended: its images are the last saved, from
|
||||
\* turnAt on, and it has not entered the history the window counts.
|
||||
OpenTurn == \E p \in Prompts : ph[p] = "started"
|
||||
Counted == IF OpenTurn THEN turnAt - 1 ELSE Len(saved)
|
||||
\* The named images an open stages: those inside the window, and the open
|
||||
\* turn's, which are past every counted one.
|
||||
Needed == {saved[k].i : k \in {j \in 1..Len(saved) : saved[j].ref /\ (WindowKeeps(saved, j, Counted) \/ ~WindowAtOpen)}}
|
||||
\* A core history whose images older than the window are dropped: each
|
||||
\* keeps its name, and the core no longer counts it as stored.
|
||||
Windowed(s) ==
|
||||
IF ~LiveWindow THEN s
|
||||
ELSE [k \in 1..Len(s) |-> IF WindowKeeps(s, k, Len(s)) THEN s[k] ELSE [i |-> s[k].i, stored |-> FALSE, dropped |-> TRUE, named |-> TRUE]]
|
||||
Images(s) == [k \in 1..Len(s) |-> s[k].i]
|
||||
Running == \E p \in Prompts : ph[p] \in {"sent", "started"}
|
||||
Queued(p) == \E k \in 1..Len(inputs) : inputs[k] = p
|
||||
\* The prompt the worker takes next: the first in the log it has not taken.
|
||||
NextInput(p) ==
|
||||
/\ ph[p] = "queued"
|
||||
/\ \E k \in 1..Len(inputs) :
|
||||
/\ inputs[k] = p
|
||||
/\ \A j \in 1..(k - 1) : ph[inputs[j]] # "queued"
|
||||
|
||||
Set(f, p, v) == [f EXCEPT ![p] = v]
|
||||
|
||||
----------------------------------------------------------------------------
|
||||
\* The caller
|
||||
|
||||
\* A prompt carrying image i. A plain engine takes prompts only while open.
|
||||
Make(p, i) ==
|
||||
/\ made + 1 = p
|
||||
/\ Durable \/ up = "open"
|
||||
/\ made' = p
|
||||
/\ img' = Set(img, p, i)
|
||||
/\ ph' = Set(ph, p, "putting")
|
||||
/\ putting' = putting \cup {p}
|
||||
/\ act' = << "Make", p, i >>
|
||||
/\ UNCHANGED <<store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The prompt's image write lands in the store.
|
||||
PutLand(p) ==
|
||||
/\ p \in putting
|
||||
/\ store' = store \cup {img[p]}
|
||||
/\ putting' = putting \ {p}
|
||||
/\ ph' = IF ph[p] = "putting" THEN Set(ph, p, "put") ELSE ph
|
||||
/\ act' = << "PutLand", p >>
|
||||
/\ UNCHANGED <<made, img, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The write fails. A prompt not yet sent fails with it; one sent early
|
||||
\* (StoreFirst = FALSE) goes on naming an image the store never got.
|
||||
PutFail(p) ==
|
||||
/\ p \in putting
|
||||
/\ faults < MaxFaults
|
||||
/\ faults' = faults + 1
|
||||
/\ putting' = putting \ {p}
|
||||
/\ ph' = IF ph[p] = "putting" THEN Set(ph, p, "failed") ELSE ph
|
||||
/\ act' = << "PutFail", p >>
|
||||
/\ UNCHANGED <<made, img, store, lostImgs, saved, inputs, mem, up, staged, dirty, crashes, retries, repeats, tools>>
|
||||
|
||||
----------------------------------------------------------------------------
|
||||
\* The host and the core
|
||||
|
||||
\* A plain engine sends a prompt once its image is stored.
|
||||
Send(p) ==
|
||||
/\ ~Durable
|
||||
/\ up = "open"
|
||||
/\ ~Running
|
||||
/\ IF StoreFirst THEN ph[p] = "put" ELSE ph[p] \in {"putting", "put"}
|
||||
/\ ph' = Set(ph, p, "sent")
|
||||
/\ act' = << "Send", p >>
|
||||
/\ UNCHANGED <<made, img, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* A durable prompt lands in the log after its image, naming it.
|
||||
Enqueue(p) ==
|
||||
/\ Durable
|
||||
/\ ph[p] = "put"
|
||||
/\ inputs' = Append(inputs, p)
|
||||
/\ ph' = Set(ph, p, "queued")
|
||||
/\ act' = << "Enqueue", p >>
|
||||
/\ UNCHANGED <<made, img, putting, store, lostImgs, saved, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The worker takes the next prompt and resolves its image's name. An
|
||||
\* image the store lost fails the turn.
|
||||
Take(p) ==
|
||||
/\ Durable
|
||||
/\ up = "open"
|
||||
/\ ~Running
|
||||
/\ NextInput(p)
|
||||
/\ ph' = Set(ph, p, IF img[p] \in store THEN "sent" ELSE "failed")
|
||||
/\ act' = << "Take", p >>
|
||||
/\ UNCHANGED <<made, img, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The turn's first record lands, naming the image the host stored.
|
||||
TurnStart(p) ==
|
||||
/\ ph[p] = "sent"
|
||||
/\ up = "open"
|
||||
/\ mem' = Append(mem, [i |-> img[p], stored |-> TRUE, dropped |-> FALSE, named |-> TRUE])
|
||||
/\ saved' = Append(saved, [i |-> img[p], ref |-> TRUE, dropped |-> FALSE, named |-> TRUE])
|
||||
/\ turnAt' = Len(saved) + 1
|
||||
/\ dirty' = TRUE
|
||||
/\ ph' = Set(ph, p, "started")
|
||||
/\ act' = << "TurnStart", p >>
|
||||
/\ UNCHANGED <<made, img, putting, store, lostImgs, inputs, up, staged, faults, crashes, retries, repeats, tool, tph, tputting>>
|
||||
|
||||
\* The model answers and the turn enters the history. Its image is the
|
||||
\* newest; the core drops any older than the window.
|
||||
TurnEnd(p) ==
|
||||
/\ ph[p] = "started"
|
||||
/\ tph[p] \in {"none", "given", "lost"}
|
||||
/\ up = "open"
|
||||
/\ mem' = Windowed(mem)
|
||||
/\ ph' = Set(ph, p, "ended")
|
||||
/\ dirty' = TRUE
|
||||
/\ act' = << "TurnEnd", p >>
|
||||
/\ UNCHANGED <<made, img, putting, store, lostImgs, saved, inputs, up, staged, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* A checkpoint between turns writes the history again from the core. A
|
||||
\* dropped image is saved by its name alone, neither named nor inline.
|
||||
Checkpoint ==
|
||||
/\ up = "open"
|
||||
/\ dirty
|
||||
/\ ~Running
|
||||
/\ saved' = [k \in 1..Len(mem) |->
|
||||
[i |-> mem[k].i,
|
||||
ref |-> ~mem[k].dropped /\ (mem[k].stored \/ ~MarkOnlyStored),
|
||||
dropped |-> mem[k].dropped,
|
||||
named |-> mem[k].named]]
|
||||
/\ dirty' = FALSE
|
||||
/\ act' = << "Checkpoint" >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, inputs, mem, up, staged, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The core crashes: it loses what it held and every write in flight. A
|
||||
\* prompt not yet in the log or not yet started is lost to a plain engine;
|
||||
\* a durable one stays in the log and runs again.
|
||||
Crash ==
|
||||
/\ crashes < MaxCrashes
|
||||
/\ up \in {"open", "opening"}
|
||||
/\ crashes' = crashes + 1
|
||||
/\ up' = "down"
|
||||
/\ mem' = << >>
|
||||
/\ staged' = {}
|
||||
/\ putting' = {}
|
||||
/\ ph' = [p \in Prompts |->
|
||||
CASE ph[p] \in {"putting", "put"} -> "lost"
|
||||
[] ph[p] = "sent" -> IF Durable THEN "queued" ELSE "lost"
|
||||
[] OTHER -> ph[p]]
|
||||
/\ tputting' = {}
|
||||
/\ tph' = [p \in Prompts |-> IF tph[p] \in {"called", "put", "refused"} THEN "lost" ELSE tph[p]]
|
||||
/\ act' = << "Crash" >>
|
||||
/\ UNCHANGED <<made, img, store, lostImgs, saved, inputs, dirty, faults, retries, repeats, tool, turnAt>>
|
||||
|
||||
\* Opening, first phase: the core reads the saved history and names the
|
||||
\* images it does not hold.
|
||||
OpenBegin ==
|
||||
/\ up = "down"
|
||||
/\ up' = "opening"
|
||||
/\ staged' = {}
|
||||
/\ act' = << "OpenBegin" >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* The host stages one image the open needs from its store.
|
||||
Stage(i) ==
|
||||
/\ up = "opening"
|
||||
/\ i \in Needed \ staged
|
||||
/\ i \in store
|
||||
/\ staged' = staged \cup {i}
|
||||
/\ act' = << "Stage", i >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, up, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* An image the open needs that the store does not hold fails the open.
|
||||
OpenMissing ==
|
||||
/\ up = "opening"
|
||||
/\ FailOnMissing
|
||||
/\ \E i \in Needed \ staged : i \notin store
|
||||
/\ up' = "failed"
|
||||
/\ act' = << "OpenMissing" >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* Opening, second phase: every image the open needs is staged, and the
|
||||
\* core holds the saved history again; a named image outside the window
|
||||
\* stays dropped, unread. With FailOnMissing = FALSE an image the store lost
|
||||
\* is left out.
|
||||
OpenOk ==
|
||||
/\ up = "opening"
|
||||
/\ \/ Needed \subseteq staged
|
||||
\/ ~FailOnMissing /\ \A i \in Needed \ staged : i \notin store
|
||||
/\ LET marked == [k \in 1..Len(saved) |->
|
||||
[saved[k] EXCEPT !.dropped = @ \/ (saved[k].ref /\ saved[k].i \notin staged /\ ~WindowKeeps(saved, k, Counted))]]
|
||||
Kept(s) == SelectSeq(s, LAMBDA e : e.dropped \/ ~e.ref \/ e.i \in staged)
|
||||
AsCore(s) == [k \in 1..Len(s) |-> [i |-> s[k].i, stored |-> s[k].ref /\ ~s[k].dropped, dropped |-> s[k].dropped, named |-> s[k].named]]
|
||||
IN mem' = Windowed(AsCore(Kept(SubSeq(marked, 1, Counted)))) \o AsCore(Kept(SubSeq(marked, Counted + 1, Len(saved))))
|
||||
/\ up' = "open"
|
||||
/\ staged' = {}
|
||||
/\ act' = << "OpenOk" >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* A retry of prompt p, which the log holds, under its messageId, carrying
|
||||
\* image i. It stores nothing; the log drops its input, or with DropRepeats
|
||||
\* = FALSE keeps it, naming i.
|
||||
Retry(p, i) ==
|
||||
/\ Durable
|
||||
/\ retries < MaxRetries
|
||||
/\ Queued(p)
|
||||
/\ retries' = retries + 1
|
||||
/\ repeats' = IF DropRepeats THEN repeats ELSE repeats \cup {i}
|
||||
/\ act' = << "Retry", p, i >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, tools>>
|
||||
|
||||
\* The model calls the turn's tool, which returns image i once. The engine
|
||||
\* writes it to the store before it hands the result on.
|
||||
ToolCall(p, i) ==
|
||||
/\ Tools
|
||||
/\ up = "open"
|
||||
/\ ph[p] = "started"
|
||||
/\ tph[p] = "none"
|
||||
/\ tool' = Set(tool, p, i)
|
||||
/\ tph' = Set(tph, p, "called")
|
||||
/\ tputting' = tputting \cup {p}
|
||||
/\ act' = << "ToolCall", p, i >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, turnAt>>
|
||||
|
||||
ToolPutLand(p) ==
|
||||
/\ p \in tputting
|
||||
/\ store' = store \cup {tool[p]}
|
||||
/\ tputting' = tputting \ {p}
|
||||
/\ tph' = IF tph[p] = "called" THEN Set(tph, p, "put") ELSE tph
|
||||
/\ act' = << "ToolPutLand", p >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, lostImgs, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tool, turnAt>>
|
||||
|
||||
\* The store refuses the image. The tool has run, so its result keeps it.
|
||||
ToolPutFail(p) ==
|
||||
/\ p \in tputting
|
||||
/\ faults < MaxFaults
|
||||
/\ faults' = faults + 1
|
||||
/\ tputting' = tputting \ {p}
|
||||
/\ tph' = IF tph[p] = "called" THEN Set(tph, p, "refused") ELSE tph
|
||||
/\ act' = << "ToolPutFail", p >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, saved, inputs, mem, up, staged, dirty, crashes, retries, repeats, tool, turnAt>>
|
||||
|
||||
\* The result reaches the core, and the turn's next record names the image
|
||||
\* the store holds; a refused one is held inline, without a name.
|
||||
ToolResult(p) ==
|
||||
/\ up = "open"
|
||||
/\ ph[p] = "started"
|
||||
/\ tph[p] \in IF ToolStoreFirst THEN {"put", "refused"} ELSE {"called", "put", "refused"}
|
||||
/\ LET named == tph[p] # "refused"
|
||||
IN /\ mem' = Append(mem, [i |-> tool[p], stored |-> named, dropped |-> FALSE, named |-> named])
|
||||
/\ saved' = Append(saved, [i |-> tool[p], ref |-> named, dropped |-> FALSE, named |-> named])
|
||||
/\ tph' = Set(tph, p, "given")
|
||||
/\ dirty' = TRUE
|
||||
/\ act' = << "ToolResult", p >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, store, lostImgs, inputs, up, staged, faults, crashes, retries, repeats, tool, tputting, turnAt>>
|
||||
|
||||
\* The store loses an image: corruption, or an operator deleting it.
|
||||
Lose(i) ==
|
||||
/\ Cardinality(lostImgs) < MaxLoss
|
||||
/\ i \in store
|
||||
/\ store' = store \ {i}
|
||||
/\ lostImgs' = lostImgs \cup {i}
|
||||
/\ act' = << "Lose", i >>
|
||||
/\ UNCHANGED <<made, img, ph, putting, saved, inputs, mem, up, staged, dirty, faults, crashes, retries, repeats, tools>>
|
||||
|
||||
\* Nothing is left to do: every prompt settled, or the open failed.
|
||||
Finished ==
|
||||
\/ up = "failed"
|
||||
\/ /\ up = "open"
|
||||
/\ made = N
|
||||
/\ putting = {}
|
||||
/\ tputting = {}
|
||||
/\ ~dirty
|
||||
/\ \A p \in Prompts : ph[p] \in {"ended", "failed", "lost"}
|
||||
|
||||
Done == Finished /\ UNCHANGED vars
|
||||
|
||||
Next ==
|
||||
\/ \E p \in Prompts, i \in Imgs : Make(p, i)
|
||||
\/ \E p \in Prompts : PutLand(p) \/ PutFail(p) \/ Send(p) \/ Enqueue(p) \/ Take(p) \/ TurnStart(p) \/ TurnEnd(p)
|
||||
\/ Checkpoint
|
||||
\/ Crash
|
||||
\/ OpenBegin
|
||||
\/ \E i \in AllImgs : Stage(i)
|
||||
\/ OpenMissing
|
||||
\/ OpenOk
|
||||
\/ \E i \in AllImgs : Lose(i)
|
||||
\/ \E p \in Prompts, i \in Imgs : Retry(p, i)
|
||||
\/ \E p \in Prompts, i \in Imgs : ToolCall(p, i)
|
||||
\/ \E p \in Prompts : ToolPutLand(p) \/ ToolPutFail(p) \/ ToolResult(p)
|
||||
\/ Done
|
||||
|
||||
Spec == Init /\ [][Next]_vars
|
||||
|
||||
\* The paths tests/tla/drivers/drive-images.mjs takes against a plain engine
|
||||
\* (Durable = FALSE). The engine runs one prompt at a time, so a prompt is
|
||||
\* made only when none is in flight. The code takes the steps in Auto on its
|
||||
\* own as soon as it can, so the driver moves only when none of them can.
|
||||
Idle == \A p \in Prompts : ph[p] \notin {"putting", "put", "sent", "started"}
|
||||
Auto ==
|
||||
\/ \E p \in Prompts : Send(p) \/ TurnStart(p) \/ ToolResult(p)
|
||||
\/ Checkpoint
|
||||
\/ \E i \in AllImgs : Stage(i)
|
||||
\/ OpenMissing
|
||||
\/ OpenOk
|
||||
Quiet == ~ENABLED Auto
|
||||
DriverStep ==
|
||||
\/ \E p \in Prompts, i \in Imgs : Idle /\ Make(p, i)
|
||||
\/ \E p \in Prompts : PutLand(p) \/ PutFail(p) \/ TurnEnd(p)
|
||||
\/ \E p \in Prompts, i \in Imgs : ToolCall(p, i)
|
||||
\/ \E p \in Prompts : ToolPutLand(p) \/ ToolPutFail(p)
|
||||
\/ Crash
|
||||
\/ OpenBegin
|
||||
\/ \E i \in AllImgs : Lose(i)
|
||||
DriveSpec == Init /\ [][(Quiet /\ DriverStep) \/ Auto \/ Done]_vars
|
||||
|
||||
----------------------------------------------------------------------------
|
||||
\* What must hold
|
||||
|
||||
\* Nothing saved names an image the store does not hold, unless the store
|
||||
\* lost it.
|
||||
NoDanglingRef ==
|
||||
lostImgs = {} =>
|
||||
/\ Named \subseteq store
|
||||
/\ Durable => \A k \in 1..Len(inputs) : img[inputs[k]] \in store
|
||||
/\ repeats \subseteq store
|
||||
|
||||
\* Only the image an older libfx saved inline is ever saved inline, but for
|
||||
\* a tool image the store refused, which has no name.
|
||||
OneCopy == \A k \in 1..Len(saved) : ~saved[k].ref /\ ~saved[k].dropped /\ saved[k].named => saved[k].i = OldImg
|
||||
|
||||
\* An open core holds exactly the saved history's images, in order.
|
||||
OpenExact == up = "open" => Images(mem) = Images(saved)
|
||||
|
||||
\* What the core counts as stored is stored, unless the store lost it.
|
||||
KnownStoredIsStored ==
|
||||
(lostImgs = {} /\ up = "open") => \A k \in 1..Len(mem) : mem[k].stored => mem[k].i \in store
|
||||
|
||||
\* An open fails only when the store lost an image.
|
||||
FailsOnlyOnLoss == up = "failed" => lostImgs # {}
|
||||
|
||||
\* An open fails only when the store lost an image the window keeps.
|
||||
FailsOnlyOnLossInWindow ==
|
||||
up = "failed" =>
|
||||
\E k \in 1..Len(saved) : saved[k].ref /\ WindowKeeps(saved, k, Counted) /\ saved[k].i \in lostImgs
|
||||
|
||||
\* An open core holds at most Window images between turns, counting each it
|
||||
\* holds inline, as a tool image the store refused.
|
||||
WindowBound ==
|
||||
(up = "open" /\ ~OpenTurn) => Cardinality({k \in 1..Len(mem) : ~mem[k].dropped}) <= Window
|
||||
|
||||
\* An open core keeps exactly its newest Window named images between turns:
|
||||
\* each older one is dropped, keeping its name. A running turn's images join
|
||||
\* the window once the turn enters the history.
|
||||
WindowHeld ==
|
||||
(up = "open" /\ ~OpenTurn) => \A k \in 1..Len(mem) : mem[k].dropped = ~WindowKeeps(mem, k, Len(mem))
|
||||
|
||||
=============================================================================
|
||||
@@ -0,0 +1,12 @@
|
||||
\* Expected to DEADLOCK: a failed prompt that never lets the prompts behind
|
||||
\* it go leaves them waiting forever, as long as no abort withdraws them.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
MaxFaults = 1
|
||||
Gate = TRUE
|
||||
CloseCheck = TRUE
|
||||
GateReleasesOnFailure = FALSE
|
||||
AbortWithdraws = FALSE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
@@ -0,0 +1,12 @@
|
||||
\* Expected to FAIL ClosedStopsPreparing: without the check a prompt
|
||||
\* still preparing its images when the agent closes is stored anyway.
|
||||
CONSTANTS
|
||||
N = 1
|
||||
MaxFaults = 0
|
||||
Gate = TRUE
|
||||
CloseCheck = FALSE
|
||||
GateReleasesOnFailure = TRUE
|
||||
AbortWithdraws = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
ClosedStopsPreparing
|
||||
@@ -0,0 +1,12 @@
|
||||
\* Expected to FAIL ImageOrder: without the gate a later prompt is stored
|
||||
\* before an earlier one still preparing its images.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
MaxFaults = 0
|
||||
Gate = FALSE
|
||||
CloseCheck = TRUE
|
||||
GateReleasesOnFailure = TRUE
|
||||
AbortWithdraws = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
ImageOrder
|
||||
@@ -0,0 +1,13 @@
|
||||
\* Expected to FAIL CancelOnlyWhenSent: a prompt that goes on after an
|
||||
\* abort before its send began is stored and then cancelled, and its cancel
|
||||
\* goes out before the prompt does.
|
||||
CONSTANTS
|
||||
N = 1
|
||||
MaxFaults = 0
|
||||
Gate = TRUE
|
||||
CloseCheck = TRUE
|
||||
GateReleasesOnFailure = TRUE
|
||||
AbortWithdraws = FALSE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
CancelOnlyWhenSent
|
||||
@@ -0,0 +1,21 @@
|
||||
\* The graph drive-intake.mjs walks: two prompts, driver steps only once the
|
||||
\* code has settled. PromptIntake.cfg checks three prompts on every path.
|
||||
CONSTANTS
|
||||
N = 2
|
||||
MaxFaults = 2
|
||||
Gate = TRUE
|
||||
CloseCheck = TRUE
|
||||
GateReleasesOnFailure = TRUE
|
||||
AbortWithdraws = TRUE
|
||||
SPECIFICATION DriveSpec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
ImageOrder
|
||||
StoredIffAccepted
|
||||
CancelOnlyWhenSent
|
||||
WithdrawnNeverStored
|
||||
NeverStoredTwice
|
||||
ClosedStopsPreparing
|
||||
CancelledOnlyOwn
|
||||
NoRunAfterRejected
|
||||
RunsStored
|
||||
@@ -0,0 +1,19 @@
|
||||
CONSTANTS
|
||||
N = 3
|
||||
MaxFaults = 2
|
||||
Gate = TRUE
|
||||
CloseCheck = TRUE
|
||||
GateReleasesOnFailure = TRUE
|
||||
AbortWithdraws = TRUE
|
||||
SPECIFICATION Spec
|
||||
INVARIANTS
|
||||
TypeOK
|
||||
ImageOrder
|
||||
StoredIffAccepted
|
||||
CancelOnlyWhenSent
|
||||
WithdrawnNeverStored
|
||||
NeverStoredTwice
|
||||
ClosedStopsPreparing
|
||||
CancelledOnlyOwn
|
||||
NoRunAfterRejected
|
||||
RunsStored
|
||||
@@ -0,0 +1,332 @@
|
||||
----------------------------- MODULE PromptIntake -----------------------------
|
||||
(***************************************************************************)
|
||||
(* How one durable session object takes prompts in, from `prompt()` to the *)
|
||||
(* session log, and how the worker runs what the log holds. *)
|
||||
(* *)
|
||||
(* A prompt is text, which needs nothing before it is stored, or carries *)
|
||||
(* images, which are prepared first: each image goes to the session's *)
|
||||
(* image store, which the log then names. A prompt made while an earlier *)
|
||||
(* image prompt from the same session object is not yet stored waits for *)
|
||||
(* it, then prepares its own images; with Gate = FALSE it does not wait *)
|
||||
(* (PromptIntake-NoGate.cfg). Once ready, a prompt whose agent closed *)
|
||||
(* meanwhile is refused; with CloseCheck = FALSE it is stored anyway *)
|
||||
(* (PromptIntake-NoCloseCheck.cfg). The wait ends when the earlier prompt *)
|
||||
(* is stored or fails; with GateReleasesOnFailure = FALSE a failure holds *)
|
||||
(* every prompt behind it forever (PromptIntake-GateHeldOnFailure.cfg, a *)
|
||||
(* deadlock). *)
|
||||
(* *)
|
||||
(* A prompt's signal may have aborted before the call, and then nothing is *)
|
||||
(* stored, or abort later. Until the prompt's send begins, an abort *)
|
||||
(* withdraws it: nothing is stored and no cancel goes out; with *)
|
||||
(* AbortWithdraws = FALSE the prompt goes on, and a cancel naming it goes *)
|
||||
(* out at once (PromptIntake-NoWithdraw.cfg). Once its send has begun, an *)
|
||||
(* abort sends a cancel naming the prompt, and the cancel and the prompt *)
|
||||
(* land in the log in either order. Preparing (an image write) or a log *)
|
||||
(* write may fail, up to MaxFaults times. *)
|
||||
(* *)
|
||||
(* The worker runs the log's prompts one at a time, in log order. It ends *)
|
||||
(* a prompt whose cancel landed without running it, and cancels a running *)
|
||||
(* one when its cancel lands. After the agent closes, a worker still *)
|
||||
(* running goes on until the log holds nothing more for it; no new one *)
|
||||
(* starts. *)
|
||||
(* *)
|
||||
(* `act` names each step and its prompt, so *)
|
||||
(* tests/tla/drivers/drive-intake.mjs can replay any path of the state graph *)
|
||||
(* against the real code. *)
|
||||
(***************************************************************************)
|
||||
EXTENDS Naturals, Sequences, FiniteSets
|
||||
|
||||
CONSTANTS
|
||||
N, \* prompts the caller makes, at most
|
||||
MaxFaults, \* bound on failed preparations and failed log writes
|
||||
Gate, \* whether a prompt waits for earlier image prompts
|
||||
CloseCheck, \* whether a prompt ready after close is refused
|
||||
GateReleasesOnFailure, \* whether a failed prompt lets later ones go
|
||||
AbortWithdraws \* whether an abort before the send withdraws the prompt
|
||||
|
||||
ASSUME N \in Nat /\ MaxFaults \in Nat
|
||||
ASSUME Gate \in BOOLEAN /\ CloseCheck \in BOOLEAN /\ GateReleasesOnFailure \in BOOLEAN /\ AbortWithdraws \in BOOLEAN
|
||||
|
||||
Prompts == 1..N
|
||||
Kinds == {"text", "image"}
|
||||
Phases == {"unmade", "early", "preparing", "sending", "stored", "failed", "refused", "withdrawn"}
|
||||
|
||||
VARIABLES
|
||||
made, \* how many prompts the caller has made
|
||||
kind, \* each prompt: text, or images to prepare
|
||||
early, \* whether its signal had aborted before the call
|
||||
before, \* the earlier image prompt it waits for, or 0
|
||||
ph, \* where it is on its way into the log
|
||||
acc, \* its `accepted`: pending, resolved or rejected
|
||||
res, \* its `result`: pending, end, cancelled or rejected
|
||||
aborted, \* whether its signal aborted after the call
|
||||
cancel, \* its cancel: none, held on its way, or stored
|
||||
log, \* the prompts the log holds, in the order they landed
|
||||
ran, \* whether the worker started its turn
|
||||
open, \* the prompt whose turn runs now, or 0
|
||||
done, \* the prompts whose turns ended
|
||||
active, \* whether a worker runs the session now
|
||||
closed, \* whether the agent closed
|
||||
faults, \* failures so far
|
||||
prepAtClose, \* the prompts still preparing when the agent closed
|
||||
act \* the step that led here
|
||||
|
||||
vars == <<made, kind, early, before, ph, acc, res, aborted, cancel, log, ran, open, done,
|
||||
active, closed, faults, prepAtClose, act>>
|
||||
|
||||
None == "none"
|
||||
Act(name, p) == [name |-> name, p |-> p, kind |-> None, early |-> FALSE]
|
||||
|
||||
TypeOK ==
|
||||
/\ made \in 0..N
|
||||
/\ kind \in [Prompts -> Kinds \cup {None}]
|
||||
/\ early \in [Prompts -> BOOLEAN]
|
||||
/\ before \in [Prompts -> 0..N]
|
||||
/\ ph \in [Prompts -> Phases]
|
||||
/\ acc \in [Prompts -> {"pending", "resolved", "rejected"}]
|
||||
/\ res \in [Prompts -> {"pending", "end", "cancelled", "rejected"}]
|
||||
/\ aborted \in [Prompts -> BOOLEAN]
|
||||
/\ cancel \in [Prompts -> {"none", "held", "stored"}]
|
||||
/\ log \in Seq(Prompts)
|
||||
/\ ran \in [Prompts -> BOOLEAN]
|
||||
/\ open \in 0..N
|
||||
/\ done \subseteq Prompts
|
||||
/\ active \in BOOLEAN
|
||||
/\ closed \in BOOLEAN
|
||||
/\ faults \in 0..MaxFaults
|
||||
/\ prepAtClose \subseteq Prompts
|
||||
/\ act \in [name: {"Init", "Make", "Released", "Prepared", "PrepareFail", "Store", "StoreFail",
|
||||
"Abort", "CancelStore", "Close", "Start", "Skip", "CancelRun", "End", "Finished"},
|
||||
p: 0..N, kind: Kinds \cup {None}, early: BOOLEAN]
|
||||
|
||||
Init ==
|
||||
/\ made = 0
|
||||
/\ kind = [i \in Prompts |-> None]
|
||||
/\ early = [i \in Prompts |-> FALSE]
|
||||
/\ before = [i \in Prompts |-> 0]
|
||||
/\ ph = [i \in Prompts |-> "unmade"]
|
||||
/\ acc = [i \in Prompts |-> "pending"]
|
||||
/\ res = [i \in Prompts |-> "pending"]
|
||||
/\ aborted = [i \in Prompts |-> FALSE]
|
||||
/\ cancel = [i \in Prompts |-> "none"]
|
||||
/\ log = <<>>
|
||||
/\ ran = [i \in Prompts |-> FALSE]
|
||||
/\ open = 0
|
||||
/\ done = {}
|
||||
/\ active = FALSE
|
||||
/\ closed = FALSE
|
||||
/\ faults = 0
|
||||
/\ prepAtClose = {}
|
||||
/\ act = Act("Init", 0)
|
||||
|
||||
Stored == {log[k] : k \in 1..Len(log)}
|
||||
Pending(d) == SelectSeq(log, LAMBDA i : i \notin d)
|
||||
|
||||
\* The latest image prompt not yet stored or failed: what a new prompt waits
|
||||
\* for. Every earlier image prompt is settled once it is, since it waited.
|
||||
Waiting == {j \in 1..made : kind[j] = "image" /\ ~early[j] /\ acc[j] = "pending"}
|
||||
GateNow == IF Gate /\ Waiting # {} THEN CHOOSE j \in Waiting : \A k \in Waiting : k <= j ELSE 0
|
||||
|
||||
\* Whether the prompt i waits for lets it go: it and every image prompt
|
||||
\* before it have settled, since one withdrawn while it waited settles
|
||||
\* before the prompts it waited for.
|
||||
Ready(i) ==
|
||||
IF before[i] = 0 THEN TRUE
|
||||
ELSE IF GateReleasesOnFailure
|
||||
THEN \A j \in 1..before[i] : (kind[j] = "image" /\ ~early[j]) => acc[j] # "pending"
|
||||
ELSE acc[before[i]] = "resolved"
|
||||
|
||||
\* A prompt that cannot be stored: both of its promises reject.
|
||||
Reject(i, phase) ==
|
||||
/\ ph' = [ph EXCEPT ![i] = phase]
|
||||
/\ acc' = [acc EXCEPT ![i] = "rejected"]
|
||||
/\ res' = [res EXCEPT ![i] = "rejected"]
|
||||
|
||||
Make(i, k, e) ==
|
||||
/\ ~closed
|
||||
/\ made = i - 1
|
||||
/\ made' = i
|
||||
/\ kind' = [kind EXCEPT ![i] = k]
|
||||
/\ early' = [early EXCEPT ![i] = e]
|
||||
/\ IF e
|
||||
THEN /\ ph' = [ph EXCEPT ![i] = "early"]
|
||||
/\ acc' = [acc EXCEPT ![i] = "resolved"]
|
||||
/\ res' = [res EXCEPT ![i] = "cancelled"]
|
||||
/\ UNCHANGED before
|
||||
ELSE /\ before' = [before EXCEPT ![i] = GateNow]
|
||||
/\ ph' = [ph EXCEPT ![i] = IF k = "text" /\ GateNow = 0 THEN "sending" ELSE "preparing"]
|
||||
/\ UNCHANGED <<acc, res>>
|
||||
/\ act' = [name |-> "Make", p |-> i, kind |-> k, early |-> e]
|
||||
/\ UNCHANGED <<aborted, cancel, log, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
\* A text prompt that waited goes on once the earlier prompt lets it.
|
||||
Released(i) ==
|
||||
/\ ph[i] = "preparing" /\ kind[i] = "text" /\ Ready(i)
|
||||
/\ IF closed /\ CloseCheck
|
||||
THEN Reject(i, "refused")
|
||||
ELSE /\ ph' = [ph EXCEPT ![i] = "sending"]
|
||||
/\ UNCHANGED <<acc, res>>
|
||||
/\ act' = Act("Released", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, aborted, cancel, log, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
Prepared(i) ==
|
||||
/\ ph[i] = "preparing" /\ kind[i] = "image" /\ Ready(i)
|
||||
/\ IF closed /\ CloseCheck
|
||||
THEN Reject(i, "refused")
|
||||
ELSE /\ ph' = [ph EXCEPT ![i] = "sending"]
|
||||
/\ UNCHANGED <<acc, res>>
|
||||
/\ act' = Act("Prepared", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, aborted, cancel, log, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
PrepareFail(i) ==
|
||||
/\ ph[i] = "preparing" /\ kind[i] = "image" /\ Ready(i)
|
||||
/\ faults < MaxFaults
|
||||
/\ Reject(i, "failed")
|
||||
/\ faults' = faults + 1
|
||||
/\ act' = Act("PrepareFail", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, aborted, cancel, log, ran, open, done, active, closed, prepAtClose>>
|
||||
|
||||
Store(i) ==
|
||||
/\ ph[i] = "sending"
|
||||
/\ log' = Append(log, i)
|
||||
/\ ph' = [ph EXCEPT ![i] = "stored"]
|
||||
/\ acc' = [acc EXCEPT ![i] = "resolved"]
|
||||
/\ act' = Act("Store", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, res, aborted, cancel, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
StoreFail(i) ==
|
||||
/\ ph[i] = "sending"
|
||||
/\ faults < MaxFaults
|
||||
/\ Reject(i, "failed")
|
||||
/\ faults' = faults + 1
|
||||
/\ act' = Act("StoreFail", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, aborted, cancel, log, ran, open, done, active, closed, prepAtClose>>
|
||||
|
||||
\* The signal aborts while the prompt's result is still pending. Before the
|
||||
\* prompt's send begins, it is withdrawn: never stored, its `accepted`
|
||||
\* resolved and its result cancelled, and no cancel goes out. Once the send
|
||||
\* has begun, or the prompt is stored, a cancel naming it goes out.
|
||||
Abort(i) ==
|
||||
/\ ph[i] \notin {"unmade", "early"}
|
||||
/\ ~aborted[i] /\ res[i] = "pending"
|
||||
/\ aborted' = [aborted EXCEPT ![i] = TRUE]
|
||||
/\ IF ph[i] = "preparing" /\ AbortWithdraws
|
||||
THEN /\ ph' = [ph EXCEPT ![i] = "withdrawn"]
|
||||
/\ acc' = [acc EXCEPT ![i] = "resolved"]
|
||||
/\ res' = [res EXCEPT ![i] = "cancelled"]
|
||||
/\ UNCHANGED cancel
|
||||
ELSE /\ cancel' = [cancel EXCEPT ![i] = "held"]
|
||||
/\ UNCHANGED <<ph, acc, res>>
|
||||
/\ act' = Act("Abort", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, log, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
CancelStore(i) ==
|
||||
/\ cancel[i] = "held"
|
||||
/\ cancel' = [cancel EXCEPT ![i] = "stored"]
|
||||
/\ act' = Act("CancelStore", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, res, aborted, log, ran, open, done, active, closed, faults, prepAtClose>>
|
||||
|
||||
Close ==
|
||||
/\ ~closed
|
||||
/\ closed' = TRUE
|
||||
/\ prepAtClose' = {i \in Prompts : ph[i] = "preparing"}
|
||||
/\ act' = Act("Close", 0)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, res, aborted, cancel, log, ran, open, done, active, faults>>
|
||||
|
||||
\* The worker. A new one starts only while the agent is open.
|
||||
CanRun == open = 0 /\ (~closed \/ active) /\ Pending(done) # <<>>
|
||||
|
||||
Start(i) ==
|
||||
/\ CanRun /\ Head(Pending(done)) = i /\ cancel[i] # "stored"
|
||||
/\ open' = i
|
||||
/\ ran' = [ran EXCEPT ![i] = TRUE]
|
||||
/\ active' = TRUE
|
||||
/\ act' = Act("Start", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, res, aborted, cancel, log, done, closed, faults, prepAtClose>>
|
||||
|
||||
\* A turn ends; the worker goes on while the log holds more for it.
|
||||
Ended(i, how) ==
|
||||
/\ done' = done \cup {i}
|
||||
/\ res' = [res EXCEPT ![i] = how]
|
||||
/\ active' = (Pending(done \cup {i}) # <<>>)
|
||||
|
||||
Skip(i) ==
|
||||
/\ CanRun /\ Head(Pending(done)) = i /\ cancel[i] = "stored"
|
||||
/\ Ended(i, "cancelled")
|
||||
/\ act' = Act("Skip", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, aborted, cancel, log, ran, open, closed, faults, prepAtClose>>
|
||||
|
||||
CancelRun(i) ==
|
||||
/\ open = i /\ i # 0 /\ cancel[i] = "stored"
|
||||
/\ open' = 0
|
||||
/\ Ended(i, "cancelled")
|
||||
/\ act' = Act("CancelRun", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, aborted, cancel, log, ran, closed, faults, prepAtClose>>
|
||||
|
||||
End(i) ==
|
||||
/\ open = i /\ i # 0 /\ cancel[i] # "stored"
|
||||
/\ open' = 0
|
||||
/\ Ended(i, "end")
|
||||
/\ act' = Act("End", i)
|
||||
/\ UNCHANGED <<made, kind, early, before, ph, acc, aborted, cancel, log, ran, closed, faults, prepAtClose>>
|
||||
|
||||
\* Every prompt made has been stored or refused: the run may stop here.
|
||||
Settled == \A i \in 1..made : acc[i] # "pending"
|
||||
Finished == Settled /\ UNCHANGED vars
|
||||
|
||||
Next ==
|
||||
\/ \E i \in Prompts, k \in Kinds, e \in BOOLEAN : Make(i, k, e)
|
||||
\/ \E i \in Prompts :
|
||||
\/ Released(i) \/ Prepared(i) \/ PrepareFail(i) \/ Store(i) \/ StoreFail(i)
|
||||
\/ Abort(i) \/ CancelStore(i) \/ Start(i) \/ Skip(i) \/ CancelRun(i) \/ End(i)
|
||||
\/ Close
|
||||
\/ Finished
|
||||
|
||||
Spec == Init /\ [][Next]_vars
|
||||
|
||||
\* The paths a driver can take. The code makes each of these steps on its
|
||||
\* own as soon as it can, and a driver acts only once the code has settled,
|
||||
\* so here the caller and the faults move only when none of them can.
|
||||
Auto == \E i \in Prompts : Released(i) \/ Start(i) \/ Skip(i) \/ CancelRun(i)
|
||||
Quiet == ~ENABLED Auto
|
||||
DriverStep ==
|
||||
\/ \E i \in Prompts, k \in Kinds, e \in BOOLEAN : Make(i, k, e)
|
||||
\/ \E i \in Prompts :
|
||||
Prepared(i) \/ PrepareFail(i) \/ Store(i) \/ StoreFail(i) \/ Abort(i) \/ CancelStore(i) \/ End(i)
|
||||
\/ Close
|
||||
DriveSpec == Init /\ [][(Quiet /\ DriverStep) \/ Auto \/ Finished]_vars
|
||||
|
||||
(***************************************************************************)
|
||||
(* Invariants. *)
|
||||
(***************************************************************************)
|
||||
|
||||
\* A prompt made after an image prompt is never stored before it.
|
||||
ImageOrder ==
|
||||
\A a, b \in 1..Len(log) : a < b => ~(log[b] < log[a] /\ kind[log[b]] = "image")
|
||||
|
||||
\* `accepted` resolves exactly when the prompt is stored, or for a prompt
|
||||
\* whose signal aborted before its send began, which is never stored.
|
||||
StoredIffAccepted ==
|
||||
\A i \in 1..made : (acc[i] = "resolved") <=> (i \in Stored \/ ph[i] \in {"early", "withdrawn"})
|
||||
|
||||
\* A cancel goes out only for a prompt whose send has begun: one that is
|
||||
\* on its way, stored, or whose store failed.
|
||||
CancelOnlyWhenSent == \A i \in Prompts : cancel[i] # "none" => ph[i] \in {"sending", "stored", "failed"}
|
||||
|
||||
\* A withdrawn prompt leaves nothing in the log.
|
||||
WithdrawnNeverStored == \A i \in Prompts : ph[i] = "withdrawn" => i \notin Stored /\ cancel[i] = "none"
|
||||
|
||||
NeverStoredTwice == \A a, b \in 1..Len(log) : a # b => log[a] # log[b]
|
||||
|
||||
\* A prompt still preparing when the agent closed is never stored.
|
||||
ClosedStopsPreparing == prepAtClose \cap Stored = {}
|
||||
|
||||
\* Only a prompt's own signal cancels it.
|
||||
CancelledOnlyOwn == \A i \in Prompts : res[i] = "cancelled" => (aborted[i] \/ early[i])
|
||||
|
||||
\* A prompt that could not be stored never runs.
|
||||
NoRunAfterRejected == \A i \in Prompts : ran[i] => acc[i] = "resolved"
|
||||
|
||||
\* One turn at a time, each for a prompt the log holds.
|
||||
RunsStored == open # 0 => (open \in Stored /\ open \notin done)
|
||||
=============================================================================
|
||||
+77
-3
@@ -1,7 +1,8 @@
|
||||
#!/bin/sh
|
||||
# Model-checks DurableSession.tla with TLC, confirms each witness is
|
||||
# violated, and writes the state graphs ../drivers/drive.mjs walks against
|
||||
# the code.
|
||||
# Model-checks DurableSession.tla, PromptIntake.tla, ImageStore.tla,
|
||||
# DurableSteer.tla and AgentRouting.tla with TLC, confirms each witness is
|
||||
# violated, and writes the state graphs the drivers in ../drivers/ walk
|
||||
# against the code.
|
||||
#
|
||||
# Needs Java 11+ and tla2tools.jar:
|
||||
# brew install openjdk
|
||||
@@ -14,6 +15,8 @@
|
||||
# bun ../drivers/drive.mjs write && bun ../drivers/drive.mjs cutoff
|
||||
# bun ../drivers/drive.mjs leased # the Heartbeat graph, on a World without liveness
|
||||
# bun ../drivers/drive.mjs queue # the QueueHeld graph, on a World whose queue shows it
|
||||
# bun ../drivers/drive-intake.mjs # prompts with images
|
||||
# bun ../drivers/drive-images.mjs # images stored once, across crashes and reopens
|
||||
# bun ../drivers/drive-agents.mjs # AgentRouting
|
||||
# bun ../drivers/drive-steer.mjs ideal && bun ../drivers/drive-steer.mjs faults # DurableSteer
|
||||
# bun ../drivers/drive-tenants.mjs # TenantIsolation
|
||||
@@ -98,6 +101,77 @@ for pair in NoInterrupt:NoSteerWaitsOnReply NoHandBack:NothingLost \
|
||||
fi
|
||||
done
|
||||
|
||||
# PromptIntake.tla: how a session object takes prompts in. Deadlock checking
|
||||
# stays on, since a prompt left waiting forever is a failure there; the
|
||||
# drive graph is the one drive-intake.mjs walks.
|
||||
for config in PromptIntake PromptIntake-drive; do
|
||||
echo "== $config"
|
||||
dump=""
|
||||
[ "$config" = PromptIntake-drive ] && dump="-dump dot,actionlabels graphs/intake"
|
||||
# shellcheck disable=SC2086 # $dump is two words or none
|
||||
if ! "$java" -XX:+UseParallelGC -cp "$jar" tlc2.TLC -workers auto -cleanup \
|
||||
-metadir "$work/$config" $dump -config "$config.cfg" PromptIntake.tla > "$work/$config.log" 2>&1; then
|
||||
cat "$work/$config.log"; status=1; continue
|
||||
fi
|
||||
grep -E 'distinct states|depth of' "$work/$config.log"
|
||||
done
|
||||
# NoGate breaks ImageOrder: a prompt that does not wait for an earlier one
|
||||
# still preparing its images is stored first. NoCloseCheck breaks
|
||||
# ClosedStopsPreparing: a prompt still preparing when the agent closes is
|
||||
# stored anyway. GateHeldOnFailure deadlocks: a failed prompt that never
|
||||
# lets the prompts behind it go leaves them waiting forever. NoWithdraw
|
||||
# breaks CancelOnlyWhenSent: a prompt that goes on after an abort before
|
||||
# its send began has its cancel go out before it does.
|
||||
for pair in NoGate:"Invariant ImageOrder is violated" NoCloseCheck:"Invariant ClosedStopsPreparing is violated" \
|
||||
GateHeldOnFailure:"Deadlock reached" NoWithdraw:"Invariant CancelOnlyWhenSent is violated"; do
|
||||
witness="${pair%%:*}" expected="${pair#*:}"
|
||||
if "$java" -cp "$jar" tlc2.TLC -workers auto -cleanup -metadir "$work/intake-$witness" \
|
||||
-config "PromptIntake-$witness.cfg" PromptIntake.tla 2>&1 | grep -q -F "$expected"; then
|
||||
echo "== witness PromptIntake-$witness reached"
|
||||
else
|
||||
echo "== witness PromptIntake-$witness NOT reached" >&2; status=1
|
||||
fi
|
||||
done
|
||||
|
||||
# ImageStore.tla: how a session saves each image once and names it. The
|
||||
# durable config sends prompts through the log; the engine config straight
|
||||
# to the core; the drive graph is the one drive-images.mjs walks. Deadlock
|
||||
# checking stays on: an open that can never finish is a failure.
|
||||
for config in ImageStore ImageStore-engine ImageStore-drive; do
|
||||
echo "== $config"
|
||||
dump=""
|
||||
[ "$config" = ImageStore-drive ] && dump="-dump dot,actionlabels graphs/images"
|
||||
# shellcheck disable=SC2086 # $dump is two words or none
|
||||
if ! "$java" -XX:+UseParallelGC -cp "$jar" tlc2.TLC -workers auto -cleanup \
|
||||
-metadir "$work/$config" $dump -config "$config.cfg" ImageStore.tla > "$work/$config.log" 2>&1; then
|
||||
cat "$work/$config.log"; status=1; continue
|
||||
fi
|
||||
grep -E 'distinct states|depth of' "$work/$config.log"
|
||||
done
|
||||
# SendBeforeStore breaks NoDanglingRef: a prompt sent while its image write
|
||||
# is in flight leaves a record naming an image the store does not hold.
|
||||
# NameUnstored breaks NoDanglingRef: a checkpoint names an image an older
|
||||
# libfx saved inline, which the store never held. DropMissing breaks
|
||||
# OpenExact: an open that drops an image the store lost runs on a history
|
||||
# that is not the one saved. RetryLands breaks NoDanglingRef: a retry stores
|
||||
# nothing, so a log that kept its input would name an image never stored.
|
||||
# KeepAll breaks WindowHeld: a core that keeps every image outgrows the
|
||||
# window an open can stage. NeedAll breaks FailsOnlyOnLossInWindow: an open
|
||||
# that stages every named image fails on one the window had dropped.
|
||||
# ToolBeforeStore breaks NoDanglingRef: a tool result handed to the core
|
||||
# while its image write is in flight leaves a record naming an image the
|
||||
# store does not hold. InlineUncounted breaks WindowBound: a window that
|
||||
# never counts a tool image the store refused holds more than Window images.
|
||||
for pair in SendBeforeStore:NoDanglingRef NameUnstored:NoDanglingRef DropMissing:OpenExact RetryLands:NoDanglingRef KeepAll:WindowHeld NeedAll:FailsOnlyOnLossInWindow ToolBeforeStore:NoDanglingRef InlineUncounted:WindowBound; do
|
||||
witness="${pair%%:*}" invariant="${pair#*:}"
|
||||
if "$java" -cp "$jar" tlc2.TLC -workers auto -cleanup -metadir "$work/images-$witness" \
|
||||
-config "ImageStore-$witness.cfg" ImageStore.tla 2>&1 | grep -q -F "Invariant $invariant is violated"; then
|
||||
echo "== witness ImageStore-$witness reached"
|
||||
else
|
||||
echo "== witness ImageStore-$witness NOT reached" >&2; status=1
|
||||
fi
|
||||
done
|
||||
|
||||
# AgentRouting.tla: which agent runs a session's turns when an app has
|
||||
# several. Its graph is what ../drivers/drive-agents.mjs walks, with process p2
|
||||
# lacking one agent; every sender stores its input before it queues it. Drains is checked under fairness, a simulation runs a larger
|
||||
|
||||
Reference in new issue
Block a user