mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-28 08:25:57 +08:00
feat(whiteboard): support rotation and custom SVG paths (#1965)
Native wb_draw_shape now takes an SVG path with a viewBox, plus rotation, transparent fill and optional stroke, with strict geometry validation. Removes the rectangle/circle/triangle presets and the shape field from the Native tool; stored shapes and Legacy preset playback stay compatible. Prompt guidance states coordinate, scaling, rotation and arc principles with one-line standard parts. Adds a whiteboard-shape evaluation harness. Closes #1853
This commit is contained in:
18 files changed
+1919
-59
No files matched your search
@@ -76,6 +76,7 @@ next-env.d.ts
|
||||
# docs
|
||||
/docs
|
||||
# Eval results
|
||||
eval/whiteboard-shapes/results/
|
||||
eval/whiteboard-layout/results/
|
||||
eval/outline-language/results/
|
||||
eval/orchestration/results/
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# Native whiteboard shape tools (#1853)
|
||||
|
||||
`wb_draw_shape` uses one SVG path interface for every shape. There is no
|
||||
`shape` discriminator or rectangle/circle/triangle preset mode. It accepts
|
||||
optional `rotate` (finite degrees,
|
||||
clockwise around the element center, default `0`), `fillColor` (default
|
||||
`#5b9bd5`, including `transparent`), and
|
||||
`stroke: { color, width, style? }` (positive width, style defaults to `solid`,
|
||||
also `dashed`/`dotted`). Omitted stroke preserves the existing no-outline behavior.
|
||||
|
||||
Every call requires both a nonblank SVG `path` **d attribute** and positive
|
||||
`viewBox: [width, height]`. It never accepts a complete SVG document. Position
|
||||
`x`/`y` is the upper-left corner of the **unrotated** box; element dimensions
|
||||
scale viewBox coordinates independently. Use a matching aspect ratio to
|
||||
preserve mathematical lengths. Plan a rotated shape's center first, then set
|
||||
`x = centerX - width / 2`, `y = centerY - height / 2`. Arc radii are expressed
|
||||
in viewBox units and scale with the element dimensions. Close filled shapes with `Z`; for open curves,
|
||||
request transparent fill and an explicit stroke.
|
||||
|
||||
Path coordinates are local: `(0,0)` is the unrotated element's upper-left.
|
||||
Do not put absolute whiteboard coordinates into the path or center it around
|
||||
zero without offsetting it into the viewBox.
|
||||
|
||||
| Shape | Path | viewBox |
|
||||
| --- | --- | --- |
|
||||
| Rectangle | `M0 0 H1 V1 H0 Z` | `[1,1]` |
|
||||
| Isosceles triangle | `M1 0 L2 1 L0 1 Z` | `[2,1]` |
|
||||
| Circle / ellipse | `M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z` | `[2,2]` |
|
||||
| Clockwise upper-right quarter circle | `M0 0 A1 1 0 0 1 1 1` | `[1,1]` |
|
||||
|
||||
For a circle, use equal element width/height; unequal dimensions give an
|
||||
ellipse. The quarter-circle example runs from the top-left to the bottom-right
|
||||
of its box. With equal width/height, its radius equals the element width,
|
||||
not half the width. Use transparent fill and an explicit stroke for this arc.
|
||||
|
||||
For a 3–4–5 triangle, use `path: "M0 0 L4 0 L0 3 Z"`, `viewBox: [4, 3]`,
|
||||
and `width:height = 4:3` to preserve its proportions.
|
||||
For a curve, use `path: "M0 1 C1 0 2 0 3 1"`, `viewBox: [3, 1]`,
|
||||
`fillColor: "transparent"`, `stroke: { color: "#333333", width: 2 }`.
|
||||
In **every actual call**, copy `nextMutation.expectedLastSeq` exactly from the
|
||||
latest `wb_read`. Read again between successful mutations. These examples
|
||||
omit the sequence deliberately; `null` is valid only when the read returns null.
|
||||
|
||||
## Validation and persistence
|
||||
|
||||
Both preparation and direct execution validate before appending or creating a
|
||||
session. `@openmaic/geometry` supplies the shared grammar, numeric, scaled-bounds,
|
||||
and resource checks from #1851. Input limits are 100,000 path code units,
|
||||
10,000 commands, and coordinate/dimension magnitudes up to 1,000,000.
|
||||
Malformed or excessive input returns an error the model can correct; direct
|
||||
execution returns `INVALID_WHITEBOARD_SHAPE` with an error field/code and
|
||||
character position where available. There is no rectangle fallback.
|
||||
|
||||
The tool constructs existing `PPTShapeElement` fields (`path`, `viewBox`,
|
||||
`rotate`, `fill`, `outline`) and appends the existing `element_added` event.
|
||||
Operation/element identity remains host-owned, with unchanged CAS, sequence,
|
||||
exact-replay and projection behavior. No DSL or runtime payload version change
|
||||
is needed. Unit tests reopen BrowserRuntimeStore, replay after a later write,
|
||||
retain old preset records, and verify rejected input leaves no sessions or
|
||||
projection events. Historical reads retain the renderer's separate, more
|
||||
permissive render budget; new producer limits do not invalidate old records.
|
||||
|
||||
Current whiteboard playback re-exports the SDK's `BaseShapeElement`. Tests
|
||||
compare both rendering entries for path, scale, angle, transparent fill and
|
||||
stroke. Existing worker tests bound malformed historical rendering.
|
||||
|
||||
## Legacy and cross-version compatibility
|
||||
|
||||
| Entry | Behavior |
|
||||
| --- | --- |
|
||||
| Native tool | Requires path/viewBox and persists full shape elements. Rejects the removed `shape` discriminator, Legacy `elementId`, or missing `expectedLastSeq`. |
|
||||
| Legacy child action tool (`classroom-actions.ts`) | Preset-only schema with `additionalProperties: false`; execution also explicitly rejects Native fields before emitting an action. |
|
||||
| Legacy JSON action parser (`call-agent.ts`) | Rejects custom mode, rotation, path/viewBox, stroke and Native sequence fields, returning an action warning. |
|
||||
| ActionEngine (including persisted playback/eval actions) | Rejects unsupported shape extensions and unknown shape names **before opening/creating a board**. Keeps supported preset playback unchanged. |
|
||||
| SDK/current whiteboard rendering | Reads existing shape fields, so no new storage schema is required. |
|
||||
|
||||
Do not convert Native calls into Legacy JSON actions. Native mutations emit
|
||||
projection hints, never draw actions that an older ActionEngine might
|
||||
misinterpret. Historical elements already store path/viewBox/rotation, so they
|
||||
remain readable without migration. New Native calls must omit `shape` and
|
||||
supply path/viewBox; older Native preset/custom arguments fail explicitly.
|
||||
Older Native strict schemas reject the new path-only arguments.
|
||||
Unpatched historical Legacy executors can discard rotation or turn custom
|
||||
shapes into rectangles; they cannot safely execute this new interface. Use the
|
||||
Native tool/projection path, or upgrade the executor to get the explicit
|
||||
rejections above. This change does not claim that old binaries negotiate new
|
||||
capabilities automatically.
|
||||
|
||||
## Live evaluation
|
||||
|
||||
Run with configured, authorized model IDs and credentials:
|
||||
|
||||
```sh
|
||||
node --env-file=.env.local --conditions=import --import tsx \
|
||||
eval/whiteboard-shapes/runner.ts \
|
||||
--models openai:gpt-5.6-sol,openai:gpt-5.6-luna
|
||||
```
|
||||
|
||||
The runner uses the production Native prompt, Pi agent and tool executors with
|
||||
an isolated BrowserRuntimeStore. Browser visibility is settled in-process;
|
||||
this is a tool/geometry eval, not an HTTP-route or full classroom UI test.
|
||||
The cases cover a 3–4–5 triangle and hypotenuse square, a pentagon, an open
|
||||
stroked curve, ordinary shapes drawn through paths, and error recovery. Recovery injects
|
||||
`M0 0 Z1 1` into the first otherwise valid custom call; the model receives the
|
||||
real error and must retry. This is an intentional fault, not a generation error.
|
||||
The additional `composition` case requests a 5–12–13 triangle and outward squares
|
||||
on all three sides. Its task gives only the triangle vertices, not square
|
||||
coordinates, paths, rotation angles or a construction recipe. Four fixed variants
|
||||
change scale, position and orientation; repetitions cycle through them. Scoring
|
||||
checks all four closed contours after viewBox scaling and rotation, with a 1 px
|
||||
vertex tolerance and correct edge order, exterior placement and canvas bounds.
|
||||
Separate elements and compound paths are both accepted. Missing, duplicate, extra,
|
||||
invisible or malformed contours fail. Hand-computed fixtures cover known correct
|
||||
and incorrect compositions independently of the reference-square generator.
|
||||
Filled scenarios require a valid static color with nonzero alpha and element
|
||||
opacity; SVG `none` and zero-alpha colors do not count as visible fills. The curve
|
||||
case checks one open cubic contour, its unrotated box and rotation, and a visible
|
||||
dark 2px stroke, rather than treating the presence of `C` in a path as sufficient.
|
||||
|
||||
For a small composition batch:
|
||||
|
||||
```sh
|
||||
node --env-file=.env.local --conditions=import --import tsx \
|
||||
eval/whiteboard-shapes/runner.ts --models deepseek:deepseek-flash \
|
||||
--cases composition --repetitions 12 --output /tmp/whiteboard-composition
|
||||
```
|
||||
|
||||
The runner uses production `runNativeChild`, with thinking enabled, a 4,096 output
|
||||
token limit, 5 provider transports and a 60-second timeout by default. Options
|
||||
`--thinking enabled|disabled`, `--max-provider-transports N` and `--timeout-ms N`
|
||||
make the budget explicit. A diagnostic run with `--max-provider-transports 12`
|
||||
can separate geometry failures from insufficient requests for several sequential
|
||||
read/mutation pairs; it is not a production-budget completion score. No production
|
||||
runtime limits are changed. `--cases all` runs every case (the default); comma-separated
|
||||
case IDs and `--repetitions N` select a subset and repeat it.
|
||||
|
||||
Results include the exact system/task prompts, Native stop reason, geometry
|
||||
assessment, persisted shapes and tool traces. A run passes only if the Native child
|
||||
completes and the persisted geometry is correct. Reports go to `results/report.json`
|
||||
and one JSON file per trial by default, all ignored by Git.
|
||||
Publish observed results, logs and screenshots in PR comments; do not commit
|
||||
run artifacts.
|
||||
@@ -0,0 +1,33 @@
|
||||
import { expectedComposition, type TriangleVertices } from './geometry-checks';
|
||||
|
||||
/** Four fixed placement/scale variants, generated before any model trials. */
|
||||
export function compositionScenario(repetition: number) {
|
||||
const variant = repetition % 4;
|
||||
const scale = [14, 16, 12, 15][variant];
|
||||
const angle = ([0, 23, -17, 90][variant] * Math.PI) / 180;
|
||||
const transform = ([x, y]: number[]): [number, number] => [
|
||||
Math.round((x * Math.cos(angle) - y * Math.sin(angle)) * 100) / 100,
|
||||
Math.round((x * Math.sin(angle) + y * Math.cos(angle)) * 100) / 100,
|
||||
];
|
||||
const rotated: TriangleVertices = [
|
||||
transform([0, 0]),
|
||||
transform([12 * scale, 0]),
|
||||
transform([0, 5 * scale]),
|
||||
];
|
||||
// Center the entire target figure on the canvas, including the outward squares.
|
||||
const points = expectedComposition(rotated).flat();
|
||||
const centerX = (Math.min(...points.map((p) => p[0])) + Math.max(...points.map((p) => p[0]))) / 2;
|
||||
const centerY = (Math.min(...points.map((p) => p[1])) + Math.max(...points.map((p) => p[1]))) / 2;
|
||||
const vertices = rotated.map(([x, y]) => [
|
||||
Math.round((x + 500 - centerX) * 100) / 100,
|
||||
Math.round((y + 281.25 - centerY) * 100) / 100,
|
||||
]) as TriangleVertices;
|
||||
return {
|
||||
variant,
|
||||
vertices,
|
||||
request:
|
||||
`请在白板上实际绘制一幅组合图：一个边长比为5–12–13的直角三角形ABC，` +
|
||||
`顶点A=(${vertices[0]}), B=(${vertices[1]}), C=(${vertices[2]})，` +
|
||||
'并在三条边上分别向三角形外侧作一个正方形。三角形和三个正方形都要有填充，完整放在画布内，不加文字、标注或其他图形。',
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,303 @@
|
||||
import type { PPTShapeElement } from '@openmaic/dsl';
|
||||
import { validateSvgPath } from '@openmaic/geometry';
|
||||
import { SVGPathData } from 'svg-pathdata';
|
||||
import tinycolor from 'tinycolor2';
|
||||
|
||||
const EPSILON = 1e-6;
|
||||
const distance = (a: number[], b: number[]) => Math.hypot(a[0] - b[0], a[1] - b[1]);
|
||||
const cross = (a: number[], b: number[], c: number[]) =>
|
||||
(b[0] - a[0]) * (c[1] - a[1]) - (b[1] - a[1]) * (c[0] - a[0]);
|
||||
|
||||
export function hasVisibleFill(s: PPTShapeElement): boolean {
|
||||
const color = tinycolor(s.fill);
|
||||
return color.isValid() && color.getAlpha() > 0 && (s.opacity ?? 1) > 0;
|
||||
}
|
||||
|
||||
export function isCurveScenarioShape(s: PPTShapeElement): boolean {
|
||||
const fill = tinycolor(s.fill);
|
||||
const stroke = tinycolor(s.outline?.color ?? '');
|
||||
const unfilled =
|
||||
s.fill.trim().toLowerCase() === 'none' || (fill.isValid() && fill.getAlpha() === 0);
|
||||
const angleDifference = Math.abs((s.rotate - 12.5) % 360);
|
||||
if (
|
||||
!unfilled ||
|
||||
!stroke.isValid() ||
|
||||
stroke.getAlpha() <= 0 ||
|
||||
!stroke.isDark() ||
|
||||
s.outline?.width !== 2 ||
|
||||
!((s.opacity ?? 1) > 0) ||
|
||||
![s.left - 100, s.top - 300, s.width - 240, s.height - 80].every(
|
||||
(difference) => Math.abs(difference) < EPSILON,
|
||||
) ||
|
||||
!(angleDifference < EPSILON || 360 - angleDifference < EPSILON) ||
|
||||
!s.viewBox.every((v) => Number.isFinite(v) && v > 0)
|
||||
)
|
||||
return false;
|
||||
const parsed = validateSvgPath(s.path);
|
||||
if (!parsed.ok || !parsed.bounds) return false;
|
||||
const commands = new SVGPathData(parsed.commands).toAbs().normalizeHVZ().normalizeST().commands;
|
||||
const first = commands[0];
|
||||
const last = commands[commands.length - 1];
|
||||
if (
|
||||
first?.type !== SVGPathData.MOVE_TO ||
|
||||
last?.type !== SVGPathData.CURVE_TO ||
|
||||
!commands.slice(1).every((c) => c.type === SVGPathData.CURVE_TO) ||
|
||||
distance([first.x, first.y], [last.x, last.y]) < EPSILON
|
||||
)
|
||||
return false;
|
||||
const sx = s.width / s.viewBox[0];
|
||||
const sy = s.height / s.viewBox[1];
|
||||
return (
|
||||
parsed.bounds.minX * sx >= -1 &&
|
||||
parsed.bounds.minY * sy >= -1 &&
|
||||
parsed.bounds.maxX * sx <= s.width + 1 &&
|
||||
parsed.bounds.maxY * sy <= s.height + 1
|
||||
);
|
||||
}
|
||||
|
||||
function polygonContours(s: PPTShapeElement): number[][][] {
|
||||
const result = validateSvgPath(s.path);
|
||||
if (!result.ok) return [];
|
||||
const commands = new SVGPathData(result.commands).toAbs().normalizeHVZ().commands;
|
||||
if (
|
||||
commands.some(
|
||||
(c) =>
|
||||
c.type !== SVGPathData.MOVE_TO &&
|
||||
c.type !== SVGPathData.LINE_TO &&
|
||||
c.type !== SVGPathData.CLOSE_PATH,
|
||||
)
|
||||
)
|
||||
return [];
|
||||
const angle = (s.rotate * Math.PI) / 180;
|
||||
const cx = s.left + s.width / 2,
|
||||
cy = s.top + s.height / 2;
|
||||
const contours: number[][][] = [];
|
||||
let points: number[][] = [];
|
||||
for (const c of commands) {
|
||||
if (c.type === SVGPathData.MOVE_TO && points.length) {
|
||||
const vertices = closedPolygonVertices(points);
|
||||
if (!vertices.length) return [];
|
||||
contours.push(vertices);
|
||||
points = [];
|
||||
}
|
||||
if ('x' in c && 'y' in c) {
|
||||
const x = (c.x * s.width) / s.viewBox[0] - s.width / 2;
|
||||
const y = (c.y * s.height) / s.viewBox[1] - s.height / 2;
|
||||
points.push([
|
||||
cx + x * Math.cos(angle) - y * Math.sin(angle),
|
||||
cy + x * Math.sin(angle) + y * Math.cos(angle),
|
||||
]);
|
||||
} else if (c.type === SVGPathData.CLOSE_PATH && points.length) {
|
||||
points.push(points[0]);
|
||||
}
|
||||
}
|
||||
const vertices = closedPolygonVertices(points);
|
||||
if (!vertices.length) return [];
|
||||
contours.push(vertices);
|
||||
return contours;
|
||||
}
|
||||
|
||||
function closedPolygonVertices(points: number[][]): number[][] {
|
||||
const vertices = points.filter((p, i) => i === 0 || distance(p, points[i - 1]) > EPSILON);
|
||||
if (vertices.length < 4 || vertices.some((p) => p.some((v) => !Number.isFinite(v)))) return [];
|
||||
const closed = distance(vertices[0], vertices[vertices.length - 1]) < EPSILON;
|
||||
if (!closed) return [];
|
||||
vertices.pop();
|
||||
// Collinear intermediate points do not add polygon corners. Keep reversals:
|
||||
// they must not hide a backtracking edge from the checks below.
|
||||
return vertices.filter((p, i) => {
|
||||
const before = vertices[(i + vertices.length - 1) % vertices.length];
|
||||
const after = vertices[(i + 1) % vertices.length];
|
||||
const dot = (p[0] - before[0]) * (after[0] - p[0]) + (p[1] - before[1]) * (after[1] - p[1]);
|
||||
return dot <= 0 || Math.abs(cross(before, p, after)) > EPSILON;
|
||||
});
|
||||
}
|
||||
|
||||
function polygonVertices(s: PPTShapeElement): number[][] {
|
||||
const contours = polygonContours(s);
|
||||
return contours.length === 1 ? contours[0] : [];
|
||||
}
|
||||
|
||||
function matchesVertices(actual: number[][], expected: number[][]): boolean {
|
||||
if (actual.length !== expected.length) return false;
|
||||
const indices = actual.map((p) => expected.findIndex((e) => distance(p, e) < 1));
|
||||
if (indices.includes(-1) || new Set(indices).size !== expected.length) return false;
|
||||
const step = (indices[1] - indices[0] + indices.length) % indices.length;
|
||||
return (
|
||||
(step === 1 || step === indices.length - 1) &&
|
||||
indices.every(
|
||||
(n, i) => (indices[(i + 1) % indices.length] - n + indices.length) % indices.length === step,
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
function matchesPolygon(s: PPTShapeElement, expected: number[][]): boolean {
|
||||
return matchesVertices(polygonVertices(s), expected);
|
||||
}
|
||||
|
||||
export type TriangleVertices = [[number, number], [number, number], [number, number]];
|
||||
|
||||
/** Independent expected vertices; never include these square coordinates in the model prompt. */
|
||||
export function expectedComposition(vertices: TriangleVertices): number[][][] {
|
||||
return [
|
||||
vertices,
|
||||
...vertices.map((a, i) => {
|
||||
const b = vertices[(i + 1) % 3];
|
||||
const opposite = vertices[(i + 2) % 3];
|
||||
const sign = cross(a, b, opposite) > 0 ? 1 : -1;
|
||||
const nx = sign * (b[1] - a[1]);
|
||||
const ny = sign * (a[0] - b[0]);
|
||||
return [a, b, [b[0] + nx, b[1] + ny], [a[0] + nx, a[1] + ny]];
|
||||
}),
|
||||
];
|
||||
}
|
||||
|
||||
export function assessComposition(shapes: PPTShapeElement[], vertices: TriangleVertices) {
|
||||
const expected = expectedComposition(vertices);
|
||||
const elementContours = shapes.map(polygonContours);
|
||||
const contours = elementContours.flat();
|
||||
const matches = contours.map((p) => expected.findIndex((e) => matchesVertices(p, e)));
|
||||
const withinCanvas = contours.every((p) =>
|
||||
p.every(([x, y]) => x >= -1 && y >= -1 && x <= 1001 && y <= 563.5),
|
||||
);
|
||||
const validElements =
|
||||
shapes.length > 0 && shapes.every((s, i) => hasVisibleFill(s) && elementContours[i].length > 0);
|
||||
const matchedContours = [...new Set(matches.filter((i) => i >= 0))];
|
||||
return {
|
||||
passed: validElements && withinCanvas && contours.length === 4 && matchedContours.length === 4,
|
||||
elementCount: shapes.length,
|
||||
contourCount: contours.length,
|
||||
matchedContours,
|
||||
unmatchedContours: matches.filter((i) => i < 0).length,
|
||||
validElements,
|
||||
withinCanvas,
|
||||
};
|
||||
}
|
||||
|
||||
export function isRightTriangle(s: PPTShapeElement): boolean {
|
||||
return matchesPolygon(s, [
|
||||
[100, 100],
|
||||
[340, 100],
|
||||
[100, 280],
|
||||
]);
|
||||
}
|
||||
|
||||
export function squareOnHypotenuse(s: PPTShapeElement): boolean {
|
||||
return matchesPolygon(s, [
|
||||
[340, 100],
|
||||
[100, 280],
|
||||
[280, 520],
|
||||
[520, 340],
|
||||
]);
|
||||
}
|
||||
|
||||
export function isPentagon(s: PPTShapeElement): boolean {
|
||||
const vertices = polygonVertices(s);
|
||||
if (vertices.length !== 5) return false;
|
||||
for (let i = 0; i < vertices.length; i++) {
|
||||
const a = vertices[i],
|
||||
b = vertices[(i + 1) % vertices.length];
|
||||
if (Math.abs(cross(a, b, vertices[(i + 2) % vertices.length])) < EPSILON) return false;
|
||||
for (let j = i + 1; j < vertices.length; j++) {
|
||||
if (j === i + 1 || (i === 0 && j === vertices.length - 1)) continue;
|
||||
const c = vertices[j],
|
||||
d = vertices[(j + 1) % vertices.length];
|
||||
const side = (n: number) => (n > EPSILON ? 1 : n < -EPSILON ? -1 : 0);
|
||||
const overlaps =
|
||||
Math.max(Math.min(a[0], b[0]), Math.min(c[0], d[0])) <=
|
||||
Math.min(Math.max(a[0], b[0]), Math.max(c[0], d[0])) + EPSILON &&
|
||||
Math.max(Math.min(a[1], b[1]), Math.min(c[1], d[1])) <=
|
||||
Math.min(Math.max(a[1], b[1]), Math.max(c[1], d[1])) + EPSILON;
|
||||
if (
|
||||
overlaps &&
|
||||
side(cross(a, b, c)) * side(cross(a, b, d)) <= 0 &&
|
||||
side(cross(c, d, a)) * side(cross(c, d, b)) <= 0
|
||||
)
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
export function isCircle(s: PPTShapeElement, cx: number, cy: number, radius: number): boolean {
|
||||
const result = validateSvgPath(s.path);
|
||||
if (!result.ok) return false;
|
||||
try {
|
||||
const commands = new SVGPathData(result.commands)
|
||||
.toAbs()
|
||||
.normalizeHVZ()
|
||||
.normalizeST()
|
||||
.qtToC()
|
||||
.aToC()
|
||||
.scale(s.width / s.viewBox[0], s.height / s.viewBox[1])
|
||||
.translate(s.left, s.top)
|
||||
.rotate((s.rotate * Math.PI) / 180, s.left + s.width / 2, s.top + s.height / 2).commands;
|
||||
const points: number[][] = [];
|
||||
let x = 0,
|
||||
y = 0,
|
||||
moves = 0;
|
||||
for (const c of commands) {
|
||||
if (c.type === SVGPathData.MOVE_TO) {
|
||||
points.push([c.x, c.y]);
|
||||
moves++;
|
||||
} else if (c.type === SVGPathData.LINE_TO || c.type === SVGPathData.CURVE_TO) {
|
||||
for (let i = 1; i <= 32; i++) {
|
||||
const t = i / 32,
|
||||
u = 1 - t;
|
||||
points.push(
|
||||
c.type === SVGPathData.LINE_TO
|
||||
? [x + t * (c.x - x), y + t * (c.y - y)]
|
||||
: [
|
||||
u ** 3 * x + 3 * u ** 2 * t * c.x1 + 3 * u * t ** 2 * c.x2 + t ** 3 * c.x,
|
||||
u ** 3 * y + 3 * u ** 2 * t * c.y1 + 3 * u * t ** 2 * c.y2 + t ** 3 * c.y,
|
||||
],
|
||||
);
|
||||
}
|
||||
} else return false;
|
||||
x = c.x;
|
||||
y = c.y;
|
||||
}
|
||||
if (moves !== 1 || points.length < 33) return false;
|
||||
const first = points[0],
|
||||
last = points[points.length - 1];
|
||||
if (Math.hypot(first[0] - last[0], first[1] - last[1]) > 1) return false;
|
||||
const area =
|
||||
Math.abs(
|
||||
points.reduce((sum, p, i) => {
|
||||
const next = points[(i + 1) % points.length];
|
||||
return sum + p[0] * next[1] - next[0] * p[1];
|
||||
}, 0),
|
||||
) / 2;
|
||||
return (
|
||||
Math.abs(area / (Math.PI * radius ** 2) - 1) < 0.02 &&
|
||||
points.every(([px, py]) => Math.abs(Math.hypot(px - cx, py - cy) - radius) < 1)
|
||||
);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export function areBasicShapes(shapes: PPTShapeElement[]): boolean {
|
||||
if (shapes.length !== 3 || shapes.some((s) => s.rotate !== 0 || !hasVisibleFill(s))) return false;
|
||||
return [80, 340, 600].every((x, i) => {
|
||||
const s = shapes.find(
|
||||
(v) => v.left === x && v.top === 100 && v.width === 120 && v.height === 120,
|
||||
);
|
||||
if (!s) return false;
|
||||
if (i === 1) return isCircle(s, x + 60, 160, 60);
|
||||
return matchesPolygon(
|
||||
s,
|
||||
i === 0
|
||||
? [
|
||||
[x, 100],
|
||||
[x + 120, 100],
|
||||
[x + 120, 220],
|
||||
[x, 220],
|
||||
]
|
||||
: [
|
||||
[x + 60, 100],
|
||||
[x + 120, 220],
|
||||
[x, 220],
|
||||
],
|
||||
);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,299 @@
|
||||
/** Live Native shape-tool eval; see README.md for configuration and limits. */
|
||||
import { mkdirSync, writeFileSync } from 'node:fs';
|
||||
import { parseArgs } from 'node:util';
|
||||
import { IDBFactory, IDBKeyRange } from 'fake-indexeddb';
|
||||
import type { PPTShapeElement } from '@openmaic/dsl';
|
||||
import { BrowserRuntimeStore } from '@openmaic/storage';
|
||||
import { createCallLlmStreamFn } from '@/lib/agent/runtime/stream-fn';
|
||||
import { runNativeChild } from '@/lib/agent/runtime/run-native-child';
|
||||
import { buildNativeChildPrompt, buildNativeChildTurnPrompt } from '@/lib/chat/pi/prompts';
|
||||
import { buildNativeWhiteboardTools } from '@/lib/chat/pi/tools/native-whiteboard';
|
||||
import { settleWhiteboardVisibility } from '@/lib/chat/pi/whiteboard-visibility';
|
||||
import type { AgentConfig } from '@/lib/orchestration/registry/types';
|
||||
import { APP_RUNTIME_PAYLOAD_VALIDATORS } from '@/lib/runtime/payload-validators';
|
||||
import type { StatelessChatRequest } from '@/lib/types/chat';
|
||||
import { createWhiteboardRuntimeService } from '@/lib/whiteboard/runtime/store';
|
||||
import { resolveModel } from '@/lib/server/resolve-model';
|
||||
import {
|
||||
areBasicShapes,
|
||||
assessComposition,
|
||||
hasVisibleFill,
|
||||
isCurveScenarioShape,
|
||||
isPentagon,
|
||||
isRightTriangle,
|
||||
squareOnHypotenuse,
|
||||
type TriangleVertices,
|
||||
} from './geometry-checks';
|
||||
import { compositionScenario } from './composition-scenarios';
|
||||
|
||||
interface Scenario {
|
||||
id: string;
|
||||
request: string;
|
||||
check: (shapes: PPTShapeElement[]) => boolean;
|
||||
injectError?: boolean;
|
||||
variant?: number;
|
||||
vertices?: TriangleVertices;
|
||||
}
|
||||
|
||||
interface RunOptions {
|
||||
thinking: 'enabled' | 'disabled';
|
||||
timeoutMs: number;
|
||||
maxProviderTransports: number;
|
||||
}
|
||||
|
||||
const teacher: AgentConfig = {
|
||||
id: 'shape-eval',
|
||||
name: 'Geometry teacher',
|
||||
role: 'teacher',
|
||||
persona: 'Draw mathematically accurate diagrams.',
|
||||
avatar: '',
|
||||
color: '#3366ff',
|
||||
allowedActions: ['wb_draw_shape'],
|
||||
priority: 10,
|
||||
createdAt: new Date(0),
|
||||
updatedAt: new Date(0),
|
||||
isDefault: true,
|
||||
};
|
||||
const body: StatelessChatRequest = {
|
||||
apiKey: '',
|
||||
messages: [],
|
||||
storeState: {
|
||||
stage: null,
|
||||
scenes: [],
|
||||
currentSceneId: null,
|
||||
mode: 'playback',
|
||||
whiteboardOpen: false,
|
||||
},
|
||||
config: { agentIds: [teacher.id] },
|
||||
};
|
||||
const cases: Scenario[] = [
|
||||
{
|
||||
id: 'pythagoras',
|
||||
request:
|
||||
'Draw exactly two shapes: a filled 3–4–5 right triangle with vertices A=(100,100), B=(340,100), C=(100,280), then an outward square of side 300 on its hypotenuse BC. Use a custom triangle path and a rotated rectangle for the square. Keep both inside the board.',
|
||||
check: (s: PPTShapeElement[]) =>
|
||||
s.length === 2 &&
|
||||
s.every(hasVisibleFill) &&
|
||||
s.some((v) => isRightTriangle(v)) &&
|
||||
s.some(squareOnHypotenuse),
|
||||
},
|
||||
{
|
||||
id: 'polygon',
|
||||
request:
|
||||
'Draw one filled irregular pentagon using a custom SVG path with 5 vertices. Place it in x=100,y=100,width=240,height=180. No other elements.',
|
||||
check: (s: PPTShapeElement[]) => s.length === 1 && hasVisibleFill(s[0]) && isPentagon(s[0]),
|
||||
},
|
||||
{
|
||||
id: 'curve',
|
||||
request:
|
||||
'Draw one open cubic Bezier curve, transparent fill, dark 2px stroke, with an unrotated box at x=100,y=300,width=240,height=80 and rotation 12.5 degrees around its center. Keep the path within that unrotated box. No other elements.',
|
||||
check: (s: PPTShapeElement[]) => s.length === 1 && isCurveScenarioShape(s[0]),
|
||||
},
|
||||
{
|
||||
id: 'basic-paths',
|
||||
request:
|
||||
'Draw exactly three filled shapes in a row using SVG paths: rectangle, complete circle, and top-centered isosceles triangle. Use x=80,340,600; y=100; width=height=120. No rotation or extra elements.',
|
||||
check: areBasicShapes,
|
||||
},
|
||||
{
|
||||
id: 'correction',
|
||||
request:
|
||||
'Draw a single filled custom 3–4–5 triangle at x=100,y=100,width=240,height=180. If the drawing tool returns a path error, fix the input and retry.',
|
||||
injectError: true,
|
||||
check: (s: PPTShapeElement[]) =>
|
||||
s.length === 1 && hasVisibleFill(s[0]) && isRightTriangle(s[0]),
|
||||
},
|
||||
];
|
||||
|
||||
async function runCase(modelString: string, scenario: Scenario, options: RunOptions) {
|
||||
const started = Date.now();
|
||||
const resolved = await resolveModel({ modelString });
|
||||
const store = new BrowserRuntimeStore({
|
||||
indexedDB: new IDBFactory(),
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
});
|
||||
const service = createWhiteboardRuntimeService({
|
||||
store,
|
||||
resolveLearnerKey: () => 'eval',
|
||||
withMaintenanceLock: (work) => work(),
|
||||
});
|
||||
const calls: Array<{ name: string; args: unknown; result?: unknown; error?: string }> = [];
|
||||
let injected = false;
|
||||
const tools = buildNativeWhiteboardTools({
|
||||
agent: teacher,
|
||||
messageId: `${modelString}:${scenario.id}`,
|
||||
service,
|
||||
stageId: 'shape-eval',
|
||||
learnerKey: 'eval',
|
||||
requestStartManualVisibilityRevision: 0,
|
||||
send: async (event) => {
|
||||
if (event.type === 'whiteboard' && event.data.kind === 'visibility_query')
|
||||
settleWhiteboardVisibility({
|
||||
queryId: event.data.queryId,
|
||||
stageId: 'shape-eval',
|
||||
learnerKey: 'eval',
|
||||
visibility: 'open',
|
||||
});
|
||||
},
|
||||
}).map((tool) => ({
|
||||
...tool,
|
||||
prepareArguments: (args: unknown) => {
|
||||
try {
|
||||
return tool.prepareArguments ? tool.prepareArguments(args) : args;
|
||||
} catch (error) {
|
||||
calls.push({
|
||||
name: tool.name,
|
||||
args,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
});
|
||||
throw error;
|
||||
}
|
||||
},
|
||||
execute: async (id: string, args: unknown, signal?: AbortSignal) => {
|
||||
const input =
|
||||
scenario.injectError && !injected && tool.name === 'wb_draw_shape'
|
||||
? ((injected = true), { ...(args as Record<string, unknown>), path: 'M0 0 Z1 1' })
|
||||
: args;
|
||||
const result = await tool.execute(id, input, signal);
|
||||
calls.push({ name: tool.name, args: input, result });
|
||||
return result;
|
||||
},
|
||||
}));
|
||||
const systemPrompt = buildNativeChildPrompt(
|
||||
body,
|
||||
teacher,
|
||||
[],
|
||||
tools.map((t) => t.name),
|
||||
);
|
||||
const native = await runNativeChild({
|
||||
systemPrompt,
|
||||
prompt: buildNativeChildTurnPrompt(scenario.request, teacher.role),
|
||||
streamFn: createCallLlmStreamFn({
|
||||
languageModel: resolved.model,
|
||||
source: 'pi-chat-native-child',
|
||||
thinkingConfig: { mode: options.thinking, enabled: options.thinking === 'enabled' },
|
||||
maxOutputTokens: 4096,
|
||||
}),
|
||||
tools,
|
||||
allowedToolNames: new Set(tools.map((t) => t.name)),
|
||||
timeoutMs: options.timeoutMs,
|
||||
maxProviderTransports: options.maxProviderTransports,
|
||||
});
|
||||
const elements = (await service.read('shape-eval')).whiteboard?.elements ?? [];
|
||||
const shapes = elements.filter((s): s is PPTShapeElement => s.type === 'shape');
|
||||
const failures = calls.filter((c) => c.error || (c.result as { isError?: boolean })?.isError);
|
||||
const geometryPassed = elements.length === shapes.length && scenario.check(shapes);
|
||||
return {
|
||||
model: modelString,
|
||||
scenario: scenario.id,
|
||||
variant: scenario.variant,
|
||||
vertices: scenario.vertices,
|
||||
request: scenario.request,
|
||||
systemPrompt,
|
||||
options,
|
||||
elapsedMs: Date.now() - started,
|
||||
passed: native.status === 'completed' && geometryPassed,
|
||||
geometryPassed,
|
||||
assessment: scenario.vertices ? assessComposition(shapes, scenario.vertices) : undefined,
|
||||
native,
|
||||
injected,
|
||||
failedCalls: failures.length,
|
||||
corrected: failures.length > 0 && geometryPassed,
|
||||
calls,
|
||||
shapes,
|
||||
};
|
||||
}
|
||||
|
||||
function positiveInteger(value: string, option: string): number {
|
||||
const n = Number(value);
|
||||
if (!Number.isSafeInteger(n) || n <= 0) throw new Error(`${option} must be a positive integer`);
|
||||
return n;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
models: { type: 'string' },
|
||||
output: { type: 'string', default: 'eval/whiteboard-shapes/results' },
|
||||
cases: { type: 'string', default: 'all' },
|
||||
repetitions: { type: 'string', default: '1' },
|
||||
thinking: { type: 'string', default: 'enabled' },
|
||||
'timeout-ms': { type: 'string', default: '60000' },
|
||||
'max-provider-transports': { type: 'string', default: '5' },
|
||||
},
|
||||
});
|
||||
const models = (values.models || process.env.EVAL_SHAPE_MODELS || '').split(',').filter(Boolean);
|
||||
if (!models.length)
|
||||
throw new Error('Pass --models provider:model,provider:model or set EVAL_SHAPE_MODELS');
|
||||
if (values.thinking !== 'enabled' && values.thinking !== 'disabled')
|
||||
throw new Error('--thinking must be enabled or disabled');
|
||||
const options: RunOptions = {
|
||||
thinking: values.thinking,
|
||||
timeoutMs: positiveInteger(values['timeout-ms']!, '--timeout-ms'),
|
||||
maxProviderTransports: positiveInteger(
|
||||
values['max-provider-transports']!,
|
||||
'--max-provider-transports',
|
||||
),
|
||||
};
|
||||
const repetitions = positiveInteger(values.repetitions!, '--repetitions');
|
||||
const selected =
|
||||
values.cases === 'all' ? [...cases.map((c) => c.id), 'composition'] : values.cases!.split(',');
|
||||
if (selected.some((id) => id !== 'composition' && !cases.some((c) => c.id === id)))
|
||||
throw new Error(`Unknown case in --cases ${values.cases}`);
|
||||
Object.defineProperty(globalThis, 'IDBKeyRange', { value: IDBKeyRange, configurable: true });
|
||||
const results = [];
|
||||
mkdirSync(values.output!, { recursive: true });
|
||||
for (const model of models)
|
||||
for (let repetition = 0; repetition < repetitions; repetition++)
|
||||
for (const id of selected) {
|
||||
const composition = compositionScenario(repetition);
|
||||
const scenario =
|
||||
id === 'composition'
|
||||
? {
|
||||
id,
|
||||
...composition,
|
||||
check: (shapes: PPTShapeElement[]) =>
|
||||
assessComposition(shapes, composition.vertices).passed,
|
||||
}
|
||||
: cases.find((c) => c.id === id)!;
|
||||
const result = await runCase(model, scenario, options);
|
||||
const trial = { ...result, repetition };
|
||||
results.push(trial);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
model,
|
||||
scenario: scenario.id,
|
||||
repetition,
|
||||
variant: scenario.variant,
|
||||
passed: result.passed,
|
||||
geometryPassed: result.geometryPassed,
|
||||
assessment: result.assessment,
|
||||
status: result.native.status,
|
||||
stopReason: result.native.stopReason,
|
||||
providerTransports: result.native.providerTransportCount,
|
||||
failedCalls: result.failedCalls,
|
||||
corrected: result.corrected,
|
||||
elapsedMs: result.elapsedMs,
|
||||
}),
|
||||
);
|
||||
writeFileSync(
|
||||
`${values.output}/${model.replace(/[^a-z0-9-]/gi, '_')}-${id}-${repetition}.json`,
|
||||
JSON.stringify(trial, null, 2),
|
||||
);
|
||||
writeFileSync(
|
||||
`${values.output}/report.json`,
|
||||
JSON.stringify(
|
||||
{ date: new Date().toISOString(), options, models, selected, repetitions, results },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
if (result.native.status === 'failed' && result.calls.length === 0)
|
||||
throw new Error(`Provider failed before any tool call: ${result.native.stopReason}`);
|
||||
}
|
||||
if (results.some((r) => !r.passed)) process.exitCode = 1;
|
||||
}
|
||||
main().catch((error) => {
|
||||
console.error(error instanceof Error ? error.message : String(error));
|
||||
process.exitCode = 1;
|
||||
});
|
||||
@@ -56,6 +56,7 @@ import {
|
||||
import katex from 'katex';
|
||||
import { safeKatexOptions } from '@openmaic/dsl';
|
||||
import { createLogger } from '@/lib/logger';
|
||||
import { legacyShapeCompatibilityError } from '@/lib/whiteboard/legacy-shape';
|
||||
|
||||
const log = createLogger('ActionEngine');
|
||||
|
||||
@@ -225,6 +226,11 @@ export class ActionEngine {
|
||||
}
|
||||
}
|
||||
|
||||
if (action.type === 'wb_draw_shape') {
|
||||
const error = legacyShapeCompatibilityError(action);
|
||||
if (error) throw new Error(error);
|
||||
}
|
||||
|
||||
// Auto-open whiteboard if a draw/clear/delete action is attempted while it's closed
|
||||
if (action.type.startsWith('wb_') && action.type !== 'wb_open' && action.type !== 'wb_close') {
|
||||
await this.ensureWhiteboardOpen(options);
|
||||
@@ -518,7 +524,7 @@ export class ActionEngine {
|
||||
id: action.elementId || '',
|
||||
type: 'shape',
|
||||
viewBox: [1000, 1000] as [number, number],
|
||||
path: SHAPE_PATHS[action.shape] ?? SHAPE_PATHS.rectangle,
|
||||
path: SHAPE_PATHS[action.shape],
|
||||
left: action.x,
|
||||
top: action.y,
|
||||
width: action.width,
|
||||
|
||||
@@ -258,6 +258,22 @@ export function buildNativeChildPrompt(
|
||||
'# Available Native tools',
|
||||
nativeToolInventory,
|
||||
...nativeWhiteboardGuidance,
|
||||
...(availableTools.includes('wb_draw_shape')
|
||||
? [
|
||||
'# Native whiteboard shapes',
|
||||
'- Geometry: every shape requires SVG path d data and viewBox:[w,h] with positive dimensions, never complete SVG markup. Element width/height scale the viewBox independently per axis; keep the viewBox ratio equal to width:height when proportions matter.',
|
||||
'- Path/viewBox coordinates are local: origin (0,0) is the unrotated element upper-left. Never put absolute whiteboard x/y into path data or center a path around zero without offsetting it into the viewBox.',
|
||||
'- x/y is the upper-left BEFORE rotation; rotate is a finite angle in degrees around the box center, positive clockwise (default 0). Work out the intended center first, then x=centerX-width/2 and y=centerY-height/2. Keep the rotated shape inside the canvas too.',
|
||||
'- Arc radii are in viewBox units and scale with element width/height; unequal axis scales turn circular arcs into elliptical arcs.',
|
||||
'- Close filled shapes with Z. Open curves use fillColor:"transparent" and stroke:{color,width,style?}; style defaults to solid (also dashed/dotted).',
|
||||
'- Examples below omit expectedLastSeq only for readability: in EVERY actual call copy it exactly from the latest wb_read; read again between successful mutations. Never copy a placeholder sequence.',
|
||||
'- Right triangle: path:"M0 0 L4 0 L0 3 Z", viewBox:[4,3]; use width:height=4:3 to preserve its proportions.',
|
||||
'- Open cubic curve: path:"M0 1 C1 0 2 0 3 1", viewBox:[3,1], fillColor:"transparent", stroke:{color:"#333333",width:2}.',
|
||||
'- Complete circle/ellipse: path:"M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z", viewBox:[2,2]. Equal element width/height gives a circle; different dimensions give an ellipse.',
|
||||
'- Clockwise upper-right quarter circle, top-left to bottom-right: path:"M0 0 A1 1 0 0 1 1 1", viewBox:[1,1]; for width=height its radius equals width.',
|
||||
'- On an invalid shape error, correct the reported field/path position or reduce path complexity and retry. Do not claim success until the mutation succeeds. Native shapes persist as PPTShapeElement records, never Legacy JSON actions.',
|
||||
]
|
||||
: []),
|
||||
'',
|
||||
'# Length & Style (CRITICAL)',
|
||||
buildLengthGuidelines(agent.role),
|
||||
|
||||
@@ -37,7 +37,7 @@ export function getActionDescriptions(allowedActions: string[]): string {
|
||||
wb_draw_text:
|
||||
'Add text to the whiteboard. Use for writing steps or key points. Use wb_draw_latex for mathematical equations and scientific notation. Parameters: { content: string, x: number, y: number, width?: number, height?: number, fontSize?: number, color?: string, elementId?: string }',
|
||||
wb_draw_shape:
|
||||
'Add a shape to the whiteboard. Use for diagrams and visual explanations. Parameters: { shape: "rectangle"|"circle"|"triangle", x: number, y: number, width: number, height: number, fillColor?: string, elementId?: string }',
|
||||
'Legacy preset-only shape drawing. Rotation, custom paths and strokes require the Native tool; these parameters are rejected here. Parameters: { shape: "rectangle"|"circle"|"triangle", x: number, y: number, width: number, height: number, fillColor?: string, elementId?: string }',
|
||||
wb_draw_chart:
|
||||
'Add a chart to the whiteboard. Use for data visualization (bar charts, line graphs, pie charts, etc.). Parameters: { chartType: "bar"|"column"|"line"|"pie"|"ring"|"area"|"radar"|"scatter", x: number, y: number, width: number, height: number, data: { labels: string[], legends: string[], series: number[][] }, themeColors?: string[], elementId?: string }',
|
||||
wb_draw_latex:
|
||||
|
||||
@@ -37,6 +37,7 @@ import {
|
||||
} from './classroom-actions';
|
||||
import { buildNativeSpotlightTool } from './native-spotlight';
|
||||
import { buildNativeWhiteboardTools } from './native-whiteboard';
|
||||
import { legacyShapeCompatibilityError } from '@/lib/whiteboard/legacy-shape';
|
||||
import type { WhiteboardRuntimeService } from '@/lib/whiteboard/runtime/store';
|
||||
import type { ElementReferenceEvidence } from '../element-reference';
|
||||
|
||||
@@ -76,7 +77,6 @@ type ActionWarning = {
|
||||
message: string;
|
||||
};
|
||||
|
||||
const SHAPE_TYPES = new Set(['rectangle', 'circle', 'triangle']);
|
||||
const CHART_TYPES = new Set(['bar', 'column', 'line', 'pie', 'ring', 'area', 'radar', 'scatter']);
|
||||
const LINE_STYLES = new Set(['solid', 'dashed']);
|
||||
const CODE_EDIT_OPERATIONS = new Set([
|
||||
@@ -294,9 +294,7 @@ function validateActionParams(action: ParsedAction): string | null {
|
||||
|
||||
if (action.actionName === 'wb_draw_shape') {
|
||||
return (
|
||||
(typeof params.shape === 'string' && SHAPE_TYPES.has(params.shape)
|
||||
? null
|
||||
: 'wb_draw_shape requires params.shape rectangle|circle|triangle') ??
|
||||
legacyShapeCompatibilityError(params) ??
|
||||
validateRequiredPosition(params, 'wb_draw_shape', ['x', 'y', 'width', 'height']) ??
|
||||
optionalString(params, 'fillColor', 'wb_draw_shape') ??
|
||||
validateOptionalElementId(params, 'wb_draw_shape')
|
||||
|
||||
@@ -6,6 +6,7 @@ import { getEffectiveActions } from '@/lib/chat/pi/tool-schemas';
|
||||
import type { WhiteboardActionRecord } from '@/lib/orchestration/types';
|
||||
import type { StatelessChatRequest } from '@/lib/types/chat';
|
||||
import type { SendEvent } from '../types';
|
||||
import { legacyShapeCompatibilityError } from '@/lib/whiteboard/legacy-shape';
|
||||
|
||||
const SpotlightParams = Type.Object({
|
||||
elementId: Type.String(),
|
||||
@@ -44,15 +45,22 @@ const WbDrawTextParams = Type.Object({
|
||||
});
|
||||
type WbDrawTextParams = Static<typeof WbDrawTextParams>;
|
||||
|
||||
const WbDrawShapeParams = Type.Object({
|
||||
shape: Type.Union([Type.Literal('rectangle'), Type.Literal('circle'), Type.Literal('triangle')]),
|
||||
x: Type.Number(),
|
||||
y: Type.Number(),
|
||||
width: Type.Number(),
|
||||
height: Type.Number(),
|
||||
fillColor: Type.Optional(Type.String()),
|
||||
elementId: Type.Optional(Type.String()),
|
||||
});
|
||||
const WbDrawShapeParams = Type.Object(
|
||||
{
|
||||
shape: Type.Union([
|
||||
Type.Literal('rectangle'),
|
||||
Type.Literal('circle'),
|
||||
Type.Literal('triangle'),
|
||||
]),
|
||||
x: Type.Number(),
|
||||
y: Type.Number(),
|
||||
width: Type.Number(),
|
||||
height: Type.Number(),
|
||||
fillColor: Type.Optional(Type.String()),
|
||||
elementId: Type.Optional(Type.String()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
);
|
||||
type WbDrawShapeParams = Static<typeof WbDrawShapeParams>;
|
||||
|
||||
const WbDrawChartParams = Type.Object({
|
||||
@@ -338,6 +346,15 @@ export function buildChildActionTools(opts: {
|
||||
parameters,
|
||||
executionMode: 'sequential',
|
||||
execute: async (_toolCallId, params) => {
|
||||
if (name === 'wb_draw_shape') {
|
||||
const error = legacyShapeCompatibilityError(params);
|
||||
if (error)
|
||||
return {
|
||||
content: [{ type: 'text', text: error }],
|
||||
details: { skipped: true, reason: 'unsupported_legacy_shape' },
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
if (
|
||||
name === 'spotlight' &&
|
||||
opts.authorizedSpotlightElementIds &&
|
||||
@@ -695,7 +712,7 @@ export function buildChildActionTools(opts: {
|
||||
makeActionTool(
|
||||
'wb_draw_shape',
|
||||
'Draw whiteboard shape',
|
||||
'Draw a rectangle, circle, or triangle on the whiteboard.',
|
||||
'Legacy preset-only shape drawing. Rotation, custom paths and strokes require the Native wb_draw_shape tool; never send those fields here.',
|
||||
WbDrawShapeParams,
|
||||
),
|
||||
);
|
||||
|
||||
@@ -16,6 +16,7 @@ import { Value } from 'typebox/value';
|
||||
import { createHash } from 'node:crypto';
|
||||
import katex from 'katex';
|
||||
import { safeKatexOptions } from '@openmaic/dsl';
|
||||
import { INPUT_PATH_LIMITS, validateShapeGeometry } from '@openmaic/geometry';
|
||||
|
||||
import type { AgentConfig } from '@/lib/orchestration/registry/types';
|
||||
import {
|
||||
@@ -61,16 +62,46 @@ const NativeWhiteboardDrawTextParams = Type.Object(
|
||||
const NativeWhiteboardDrawShapeParams = Type.Object(
|
||||
{
|
||||
expectedLastSeq: ExpectedLastSeq,
|
||||
shape: Type.Union([
|
||||
Type.Literal('rectangle'),
|
||||
Type.Literal('circle'),
|
||||
Type.Literal('triangle'),
|
||||
]),
|
||||
x: Type.Number(),
|
||||
y: Type.Number(),
|
||||
width: Type.Number({ exclusiveMinimum: 0 }),
|
||||
height: Type.Number({ exclusiveMinimum: 0 }),
|
||||
fillColor: Type.Optional(Type.String({ minLength: 1 })),
|
||||
rotate: Type.Optional(
|
||||
Type.Number({
|
||||
description:
|
||||
'Finite angle in degrees around the box center; defaults to 0. Positive is clockwise.',
|
||||
}),
|
||||
),
|
||||
fillColor: Type.Optional(
|
||||
Type.String({ minLength: 1, description: 'Use transparent for unfilled paths.' }),
|
||||
),
|
||||
stroke: Type.Optional(
|
||||
Type.Object(
|
||||
{
|
||||
color: Type.String({ minLength: 1 }),
|
||||
width: Type.Number({ exclusiveMinimum: 0, maximum: INPUT_PATH_LIMITS.maxCoordinate }),
|
||||
style: Type.Optional(
|
||||
Type.Union([Type.Literal('solid'), Type.Literal('dashed'), Type.Literal('dotted')]),
|
||||
),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
),
|
||||
path: Type.String({
|
||||
minLength: 1,
|
||||
maxLength: INPUT_PATH_LIMITS.maxLength,
|
||||
pattern: '\\S',
|
||||
description: 'SVG path d data only, beginning with M or m; never a complete SVG document.',
|
||||
}),
|
||||
// Homogeneous items works with providers that reject the older tuple items array.
|
||||
viewBox: Type.Unsafe<[number, number]>({
|
||||
type: 'array',
|
||||
items: Type.Number({ exclusiveMinimum: 0 }),
|
||||
minItems: 2,
|
||||
maxItems: 2,
|
||||
description:
|
||||
'Positive [width, height] in path coordinates. The element width/height scale these coordinates independently.',
|
||||
}),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
);
|
||||
@@ -265,12 +296,15 @@ const DRAW_TEXT_KEYS = new Set([
|
||||
]);
|
||||
const DRAW_SHAPE_KEYS = new Set([
|
||||
'expectedLastSeq',
|
||||
'shape',
|
||||
'x',
|
||||
'y',
|
||||
'width',
|
||||
'height',
|
||||
'fillColor',
|
||||
'rotate',
|
||||
'path',
|
||||
'viewBox',
|
||||
'stroke',
|
||||
]);
|
||||
const DRAW_CHART_KEYS = new Set([
|
||||
'expectedLastSeq',
|
||||
@@ -325,11 +359,52 @@ const EDIT_CODE_KEYS = new Set([
|
||||
'content',
|
||||
]);
|
||||
|
||||
const SHAPE_PATHS = {
|
||||
rectangle: 'M 0 0 L 1000 0 L 1000 1000 L 0 1000 Z',
|
||||
circle: 'M 500 0 A 500 500 0 1 1 499 0 Z',
|
||||
triangle: 'M 500 0 L 1000 1000 L 0 1000 Z',
|
||||
} as const;
|
||||
class WhiteboardShapeInputError extends Error {
|
||||
constructor(
|
||||
readonly geometryError: { code: string; field: string; message: string; position?: number },
|
||||
) {
|
||||
super(`wb_draw_shape ${geometryError.field}: ${geometryError.message} (${geometryError.code})`);
|
||||
}
|
||||
}
|
||||
|
||||
function prepareShapeArguments(args: unknown): NativeWhiteboardDrawShapeParams {
|
||||
let params: NativeWhiteboardDrawShapeParams;
|
||||
try {
|
||||
params = strictArguments<NativeWhiteboardDrawShapeParams>(
|
||||
NativeWhiteboardDrawShapeParams,
|
||||
args,
|
||||
DRAW_SHAPE_KEYS,
|
||||
);
|
||||
} catch {
|
||||
if (
|
||||
typeof args === 'object' &&
|
||||
args !== null &&
|
||||
'path' in args &&
|
||||
typeof args.path === 'string' &&
|
||||
args.path.length > INPUT_PATH_LIMITS.maxLength
|
||||
) {
|
||||
throw new WhiteboardShapeInputError({
|
||||
code: 'resource_limit',
|
||||
field: 'path',
|
||||
message: `Path exceeds ${INPUT_PATH_LIMITS.maxLength} code units; shorten the path.`,
|
||||
});
|
||||
}
|
||||
throw new WhiteboardShapeInputError({
|
||||
code: 'invalid_arguments',
|
||||
field: 'params',
|
||||
message:
|
||||
'Native whiteboard arguments must match the strict schema. Every shape requires a non-empty SVG path and positive [width, height] viewBox; omit the removed shape field. Copy expectedLastSeq from wb_read; x/y/rotate must be finite, width/height positive, and stroke requires color and positive width. No extra fields are allowed.',
|
||||
});
|
||||
}
|
||||
const geometry = validateShapeGeometry({
|
||||
path: params.path,
|
||||
viewBox: params.viewBox,
|
||||
width: params.width,
|
||||
height: params.height,
|
||||
});
|
||||
if (!geometry.ok) throw new WhiteboardShapeInputError(geometry.error);
|
||||
return params;
|
||||
}
|
||||
|
||||
const DEFAULT_CHART_THEME = ['#5b9bd5', '#ed7d31', '#a5a5a5', '#ffc000', '#4472c4'];
|
||||
|
||||
@@ -631,7 +706,12 @@ function elementMutationTool<TParams extends ElementMutationParams>(
|
||||
let element: PPTElement;
|
||||
try {
|
||||
element = config.createElement(params, invocationDigest);
|
||||
} catch {
|
||||
} catch (error) {
|
||||
if (error instanceof WhiteboardShapeInputError) {
|
||||
return failedResult('INVALID_WHITEBOARD_SHAPE', error.message, {
|
||||
error: error.geometryError,
|
||||
});
|
||||
}
|
||||
return failedResult(
|
||||
'WHITEBOARD_ELEMENT_BUILD_FAILED',
|
||||
'Whiteboard element construction failed; no mutation was accepted.',
|
||||
@@ -777,27 +857,29 @@ export function buildNativeWhiteboardTools(opts: NativeWhiteboardToolOptions): A
|
||||
name: 'wb_draw_shape',
|
||||
label: 'Draw whiteboard shape',
|
||||
description:
|
||||
'Append one rectangle, circle, or triangle to the authoritative learner whiteboard using nextMutation.expectedLastSeq from the latest wb_read result.',
|
||||
'Append an SVG path to the authoritative learner whiteboard. Copy nextMutation.expectedLastSeq from wb_read. rotate is a finite angle in degrees about the box center (default 0); x/y locate the unrotated upper-left corner. Every shape requires path d data and positive viewBox [w,h]; width/height scale path coordinates. Default fill is #5b9bd5 with no stroke; use fillColor:"transparent" and stroke:{color,width,style?} for open curves. Close filled polygons with Z. Invalid paths return a field, code and optional character position; correct the input and retry.',
|
||||
parameters: NativeWhiteboardDrawShapeParams,
|
||||
prepare: (args) =>
|
||||
strictArguments<NativeWhiteboardDrawShapeParams>(
|
||||
NativeWhiteboardDrawShapeParams,
|
||||
args,
|
||||
DRAW_SHAPE_KEYS,
|
||||
),
|
||||
createElement: (params, invocationDigest): PPTShapeElement => ({
|
||||
id: `native-wb-element:${invocationDigest}`,
|
||||
type: 'shape',
|
||||
viewBox: [1000, 1000],
|
||||
path: SHAPE_PATHS[params.shape],
|
||||
left: params.x,
|
||||
top: params.y,
|
||||
width: params.width,
|
||||
height: params.height,
|
||||
rotate: 0,
|
||||
fill: params.fillColor ?? '#5b9bd5',
|
||||
fixedRatio: false,
|
||||
}),
|
||||
prepare: prepareShapeArguments,
|
||||
createElement: (args, invocationDigest): PPTShapeElement => {
|
||||
// Also validate direct executor calls before append or session creation.
|
||||
const params = prepareShapeArguments(args);
|
||||
return {
|
||||
id: `native-wb-element:${invocationDigest}`,
|
||||
type: 'shape',
|
||||
viewBox: params.viewBox,
|
||||
path: params.path,
|
||||
left: params.x,
|
||||
top: params.y,
|
||||
width: params.width,
|
||||
height: params.height,
|
||||
rotate: params.rotate ?? 0,
|
||||
fill: params.fillColor ?? '#5b9bd5',
|
||||
...(params.stroke
|
||||
? { outline: { ...params.stroke, style: params.stroke.style ?? 'solid' } }
|
||||
: {}),
|
||||
fixedRatio: false,
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
/** Legacy actions cannot carry Native shape extensions. Validate before any side effects. */
|
||||
export function legacyShapeCompatibilityError(params: object): string | null {
|
||||
const shape = 'shape' in params ? params.shape : undefined;
|
||||
const extensions = ['rotate', 'path', 'viewBox', 'stroke', 'expectedLastSeq'];
|
||||
if (shape === 'custom' || extensions.some((key) => key in params)) {
|
||||
return 'Legacy wb_draw_shape does not support Native rotation/custom path/stroke parameters. Use the Native wb_draw_shape tool and RuntimeStore projection; do not send a Legacy action.';
|
||||
}
|
||||
if (shape !== 'rectangle' && shape !== 'circle' && shape !== 'triangle') {
|
||||
return 'Legacy wb_draw_shape requires shape rectangle|circle|triangle; unknown shapes cannot fall back to rectangles.';
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import type { StageStore } from '@/lib/api/stage-api';
|
||||
import type { Action } from '@/lib/types/action';
|
||||
import { ActionEngine } from '@/lib/action/engine';
|
||||
const mocks = vi.hoisted(() => ({ get: vi.fn(), addElement: vi.fn(), setWhiteboardOpen: vi.fn() }));
|
||||
vi.mock('@/lib/api/stage-api', () => ({
|
||||
createStageAPI: () => ({ whiteboard: { get: mocks.get, addElement: mocks.addElement } }),
|
||||
}));
|
||||
vi.mock('@/lib/store/canvas', () => ({
|
||||
useCanvasStore: {
|
||||
getState: () => ({ whiteboardOpen: false, setWhiteboardOpen: mocks.setWhiteboardOpen }),
|
||||
},
|
||||
}));
|
||||
|
||||
describe('Legacy ActionEngine shape compatibility', () => {
|
||||
it.each([
|
||||
{ shape: 'custom', path: 'M0 0 L4 0 L0 3 Z', viewBox: [4, 3] },
|
||||
{ shape: 'rectangle', rotate: 30 },
|
||||
{ shape: 'rectangle', stroke: { color: '#111', width: 2 } },
|
||||
{ shape: 'unknown' },
|
||||
{ shape: 'rectangle', expectedLastSeq: null },
|
||||
])('rejects unsupported shape $shape before opening or creating a board', async (params) => {
|
||||
const engine = new ActionEngine({} as StageStore);
|
||||
// Deliberately cross the wire-version boundary, bypassing the persisted Action type.
|
||||
const action = {
|
||||
id: 'legacy',
|
||||
type: 'wb_draw_shape',
|
||||
x: 10,
|
||||
y: 10,
|
||||
width: 100,
|
||||
height: 100,
|
||||
...params,
|
||||
} as unknown as Action;
|
||||
await expect(engine.execute(action, { silent: true })).rejects.toThrow('Legacy wb_draw_shape');
|
||||
expect(mocks.get).not.toHaveBeenCalled();
|
||||
expect(mocks.addElement).not.toHaveBeenCalled();
|
||||
expect(mocks.setWhiteboardOpen).not.toHaveBeenCalled();
|
||||
engine.dispose();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
import { createElement } from 'react';
|
||||
import { renderToStaticMarkup } from 'react-dom/server';
|
||||
import { parseHTML } from 'linkedom';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { PPTShapeElement } from '@openmaic/dsl';
|
||||
import { BaseShapeElement as WhiteboardShape } from '@/components/slide-renderer/components/element/ShapeElement/BaseShapeElement';
|
||||
import { BaseShapeElement as SDKShape } from '@openmaic/renderer/elements';
|
||||
|
||||
const base: PPTShapeElement = {
|
||||
id: 'custom',
|
||||
type: 'shape',
|
||||
left: 100,
|
||||
top: 100,
|
||||
width: 240,
|
||||
height: 180,
|
||||
rotate: -36.86989765,
|
||||
path: 'M 0 0 L 4 0 L 0 3 Z',
|
||||
viewBox: [4, 3],
|
||||
fill: '#5b9bd5',
|
||||
fixedRatio: false,
|
||||
};
|
||||
describe('current whiteboard and shared SDK shape rendering', () => {
|
||||
it.each([
|
||||
base,
|
||||
{ ...base, path: 'M0 0 L4 0 L4 2 L2 3 L0 2 Z' },
|
||||
{
|
||||
...base,
|
||||
path: 'M0 1 C1 0 2 0 3 1',
|
||||
viewBox: [3, 1] as [number, number],
|
||||
rotate: 12.5,
|
||||
fill: 'transparent',
|
||||
outline: { color: '#333333', width: 2, style: 'dashed' as const },
|
||||
},
|
||||
])('renders path $path with identical scaling, rotation, fill and stroke', (element) => {
|
||||
const current = renderToStaticMarkup(createElement(WhiteboardShape, { elementInfo: element }));
|
||||
expect(current).toBe(renderToStaticMarkup(createElement(SDKShape, { elementInfo: element })));
|
||||
const { document } = parseHTML(current);
|
||||
expect(document.querySelector('.rotate-wrapper')?.getAttribute('style')).toContain(
|
||||
`rotate(${element.rotate}deg)`,
|
||||
);
|
||||
const group = document.querySelector('svg g')!;
|
||||
expect(group.getAttribute('transform')).toContain(
|
||||
`scale(${element.width / element.viewBox[0]}, ${element.height / element.viewBox[1]})`,
|
||||
);
|
||||
const path = document.querySelector('svg path')!;
|
||||
expect(path.getAttribute('d')).toBe(element.path);
|
||||
expect(path.getAttribute('fill')).toBe(element.fill);
|
||||
expect(path.getAttribute('stroke-width')).toBe(String(element.outline?.width ?? 0));
|
||||
if (element.outline) {
|
||||
expect(path.getAttribute('stroke')).toBe(element.outline.color);
|
||||
expect(path.getAttribute('stroke-dasharray')).toBe('9 4');
|
||||
}
|
||||
expect(current).not.toContain('data-shape-geometry-error');
|
||||
});
|
||||
});
|
||||
@@ -1697,6 +1697,72 @@ describe('Pi call_agent JSON action output', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects Native shape extensions through both Legacy tool and JSON entry points', async () => {
|
||||
const { buildChildActionTools } = await import('@/lib/chat/pi/tools/classroom-actions');
|
||||
const events: StatelessEvent[] = [];
|
||||
const send = async (event: StatelessEvent) => {
|
||||
events.push(event);
|
||||
};
|
||||
const onActionDone = vi.fn();
|
||||
const params = [
|
||||
{ shape: 'rectangle', rotate: -30 },
|
||||
{ shape: 'custom', path: 'M0 0 L4 0 L0 3 Z', viewBox: [4, 3] },
|
||||
{ shape: 'circle', stroke: { color: '#111', width: 2 } },
|
||||
{ shape: 'triangle', expectedLastSeq: null },
|
||||
].map((extra) => ({ x: 10, y: 20, width: 100, height: 80, ...extra }));
|
||||
const draw = buildChildActionTools({
|
||||
body: makeBody(),
|
||||
agent: slideTeacher,
|
||||
messageId: 'legacy',
|
||||
send,
|
||||
onActionDone,
|
||||
maxActionsPerAgent: 8,
|
||||
enableWhiteboardTools: true,
|
||||
}).find((t) => t.name === 'wb_draw_shape')!;
|
||||
expect(draw.parameters).toMatchObject({ additionalProperties: false });
|
||||
for (const input of params)
|
||||
await expect(draw.execute('legacy-shape', input)).resolves.toMatchObject({
|
||||
isError: true,
|
||||
details: { reason: 'unsupported_legacy_shape' },
|
||||
});
|
||||
mockChildWithJsonOutput(
|
||||
JSON.stringify(params.map((params) => ({ type: 'action', name: 'wb_draw_shape', params }))),
|
||||
);
|
||||
const { buildCallAgentTool } = await import('@/lib/chat/pi/tools/call-agent');
|
||||
const tool = buildCallAgentTool({
|
||||
body: makeBody(),
|
||||
agentConfigs: [slideTeacher],
|
||||
send,
|
||||
languageModel: {} as never,
|
||||
onAgentDone: vi.fn(),
|
||||
onActionDone,
|
||||
thinkingConfig: { mode: 'disabled', enabled: false },
|
||||
abortSignal: new AbortController().signal,
|
||||
maxAgentTurns: 6,
|
||||
getAgentTurnCount: () => 0,
|
||||
getAgentResponses: () => [],
|
||||
getWhiteboardLedger: () => [],
|
||||
maxActionsPerAgent: 8,
|
||||
enableWhiteboardTools: true,
|
||||
});
|
||||
const result = await tool.execute('call', {
|
||||
agentId: slideTeacher.id,
|
||||
instruction: 'Draw these shapes.',
|
||||
});
|
||||
expect(result.details).toMatchObject({
|
||||
// The JSON dispatcher deduplicates equal warnings across the four rejected calls.
|
||||
actionWarnings: [
|
||||
expect.objectContaining({
|
||||
actionName: 'wb_draw_shape',
|
||||
reason: 'invalid_params',
|
||||
message: expect.stringContaining('Legacy wb_draw_shape does not support'),
|
||||
}),
|
||||
],
|
||||
});
|
||||
expect(events.filter((e) => e.type === 'action')).toEqual([]);
|
||||
expect(onActionDone).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('supports the full in-scope whiteboard action surface', async () => {
|
||||
mockChildWithJsonOutput(
|
||||
JSON.stringify([
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { AgentConfig } from '@/lib/orchestration/registry/types';
|
||||
import type { PPTShapeElement } from '@openmaic/dsl';
|
||||
import { BrowserRuntimeStore, RuntimeAppendConflictError } from '@openmaic/storage';
|
||||
import { IDBFactory, IDBKeyRange } from 'fake-indexeddb';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
@@ -239,7 +240,14 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
it('keeps expectedLastSeq required and host-owned identity across every additive schema', () => {
|
||||
const tools = build(service()).tools;
|
||||
const samples: Record<string, Record<string, unknown>> = {
|
||||
wb_draw_shape: { shape: 'rectangle', x: 1, y: 2, width: 30, height: 40 },
|
||||
wb_draw_shape: {
|
||||
path: 'M0 0 H1 V1 H0 Z',
|
||||
viewBox: [1, 1],
|
||||
x: 1,
|
||||
y: 2,
|
||||
width: 30,
|
||||
height: 40,
|
||||
},
|
||||
wb_draw_chart: {
|
||||
chartType: 'bar',
|
||||
x: 1,
|
||||
@@ -256,10 +264,15 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
|
||||
for (const [name, sample] of Object.entries(samples)) {
|
||||
const tool = tools.find((candidate) => candidate.name === name)!;
|
||||
expect(
|
||||
(tool.parameters as { properties?: { expectedLastSeq?: { description?: string } } })
|
||||
.properties?.expectedLastSeq?.description,
|
||||
).toContain('Copy nextMutation.expectedLastSeq exactly');
|
||||
const schema = tool.parameters as {
|
||||
properties?: { expectedLastSeq?: { description?: string } };
|
||||
anyOf?: Array<{ properties?: { expectedLastSeq?: { description?: string } } }>;
|
||||
};
|
||||
for (const variant of schema.anyOf ?? [schema]) {
|
||||
expect(variant.properties?.expectedLastSeq?.description).toContain(
|
||||
'Copy nextMutation.expectedLastSeq exactly',
|
||||
);
|
||||
}
|
||||
expect(() => tool.prepareArguments?.(sample)).toThrow('strict schema');
|
||||
expect(() =>
|
||||
tool.prepareArguments?.({ ...sample, expectedLastSeq: null, elementId: 'model-owned' }),
|
||||
@@ -346,11 +359,12 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('commits every remaining additive draw type through the existing element_added contract', async () => {
|
||||
it('preserves mixed elements across custom shape draws, rejection, reopen and replay', async () => {
|
||||
vi.stubGlobal('IDBKeyRange', IDBKeyRange);
|
||||
try {
|
||||
const indexedDB = new IDBFactory();
|
||||
const store = new BrowserRuntimeStore({
|
||||
indexedDB: new IDBFactory(),
|
||||
indexedDB,
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
});
|
||||
const runtime = createWhiteboardRuntimeService({
|
||||
@@ -363,7 +377,8 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
{
|
||||
name: 'wb_draw_shape',
|
||||
params: {
|
||||
shape: 'circle',
|
||||
path: 'M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z',
|
||||
viewBox: [2, 2],
|
||||
x: 40,
|
||||
y: 50,
|
||||
width: 120,
|
||||
@@ -447,8 +462,8 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
top: 50,
|
||||
width: 120,
|
||||
height: 120,
|
||||
viewBox: [1000, 1000],
|
||||
path: 'M 500 0 A 500 500 0 1 1 499 0 Z',
|
||||
viewBox: [2, 2],
|
||||
path: 'M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z',
|
||||
fill: '#123456',
|
||||
fixedRatio: false,
|
||||
}),
|
||||
@@ -531,6 +546,75 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
expect.stringMatching(/^native-wb-code-line:[0-9a-f]{64}:0$/u),
|
||||
expect.stringMatching(/^native-wb-code-line:[0-9a-f]{64}:1$/u),
|
||||
]);
|
||||
|
||||
const shape = tools.find((tool) => tool.name === 'wb_draw_shape')!;
|
||||
const custom = {
|
||||
expectedLastSeq: 5,
|
||||
path: 'M0 0 L4 0 L0 3 Z',
|
||||
viewBox: [4, 3],
|
||||
x: 50,
|
||||
y: 80,
|
||||
width: 120,
|
||||
height: 90,
|
||||
rotate: 12.5,
|
||||
fillColor: '#5b9bd5',
|
||||
};
|
||||
await expect(shape.execute('mixed-custom', custom)).resolves.toMatchObject({
|
||||
details: { committedSeq: 6, replayed: false },
|
||||
});
|
||||
await expect(
|
||||
tools
|
||||
.find((tool) => tool.name === 'wb_draw_text')!
|
||||
.execute('mixed-label', {
|
||||
expectedLastSeq: 6,
|
||||
content: 'Preserve the formula, line, chart, table and code.',
|
||||
x: 50,
|
||||
y: 190,
|
||||
}),
|
||||
).resolves.toMatchObject({ details: { committedSeq: 7 } });
|
||||
await expect(
|
||||
shape.execute('mixed-curve', {
|
||||
...custom,
|
||||
expectedLastSeq: 7,
|
||||
path: 'M0 1 C1 0 2 0 3 1',
|
||||
viewBox: [3, 1],
|
||||
fillColor: 'transparent',
|
||||
stroke: { color: '#333333', width: 2, style: 'dashed' },
|
||||
}),
|
||||
).resolves.toMatchObject({ details: { committedSeq: 8 } });
|
||||
|
||||
const mixed = await runtime.read('stage-1');
|
||||
expect(mixed.whiteboard?.elements.slice(0, draws.length)).toEqual(state.whiteboard?.elements);
|
||||
expect(mixed.whiteboard?.elements.map((element) => element.type)).toEqual([
|
||||
'shape',
|
||||
'chart',
|
||||
'latex',
|
||||
'table',
|
||||
'line',
|
||||
'code',
|
||||
'shape',
|
||||
'text',
|
||||
'shape',
|
||||
]);
|
||||
await expect(
|
||||
shape.execute('mixed-invalid', { ...custom, expectedLastSeq: 8, path: 'M0 0 Z1 1' }),
|
||||
).resolves.toMatchObject({ isError: true, details: { code: 'INVALID_WHITEBOARD_SHAPE' } });
|
||||
await expect(shape.execute('mixed-custom', custom)).resolves.toMatchObject({
|
||||
details: { committedSeq: 6, lastSeq: 8, replayed: true },
|
||||
});
|
||||
await expect(runtime.read('stage-1')).resolves.toEqual(mixed);
|
||||
|
||||
const reopened = createWhiteboardRuntimeService({
|
||||
store: new BrowserRuntimeStore({
|
||||
indexedDB,
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
}),
|
||||
resolveLearnerKey: () => 'learner-1',
|
||||
withMaintenanceLock: (work) => work(),
|
||||
});
|
||||
await expect(reopened.read('stage-1')).resolves.toEqual(mixed);
|
||||
const sessions = await store.listSessions('stage-1', 'learner-1');
|
||||
await expect(store.listRecords(sessions[0]!.id)).resolves.toHaveLength(9);
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
@@ -1735,3 +1819,402 @@ describe('Native RuntimeStore whiteboard tools', () => {
|
||||
expect(runtime.append).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
const customTriangle = {
|
||||
expectedLastSeq: null,
|
||||
path: 'M 0 0 L 4 0 L 0 3 Z',
|
||||
viewBox: [4, 3],
|
||||
x: 100,
|
||||
y: 100,
|
||||
width: 240,
|
||||
height: 180,
|
||||
};
|
||||
const invalidShapes = [
|
||||
{ ...customTriangle, path: undefined },
|
||||
{ ...customTriangle, viewBox: undefined },
|
||||
{ ...customTriangle, path: '' },
|
||||
{ ...customTriangle, path: ' ' },
|
||||
{ ...customTriangle, path: 'M0 0 Z1 1' },
|
||||
{ ...customTriangle, path: 'M0 0 L' },
|
||||
{ ...customTriangle, path: '<svg><path d="M0 0"/></svg>' },
|
||||
{ ...customTriangle, viewBox: [0, 3] },
|
||||
{ ...customTriangle, viewBox: [-1, 3] },
|
||||
{ ...customTriangle, viewBox: [4, Infinity] },
|
||||
{ ...customTriangle, viewBox: [4] },
|
||||
{ ...customTriangle, viewBox: [4, 3, 1] },
|
||||
{ ...customTriangle, path: 'M0 0 L1e999 0' },
|
||||
{ ...customTriangle, path: 'M0 0 L1000001 0' },
|
||||
{ ...customTriangle, path: 'M0 0' + ' l100000 0'.repeat(11) },
|
||||
{ ...customTriangle, path: 'M0 0 L10000 0', viewBox: [0.01, 3] },
|
||||
{ ...customTriangle, path: 'M0 0' + ' L1 1'.repeat(10001) },
|
||||
{ ...customTriangle, path: 'M0 0' + ' '.repeat(100001) },
|
||||
{ ...customTriangle, width: 0 },
|
||||
{ ...customTriangle, height: -1 },
|
||||
{ ...customTriangle, width: 1000001 },
|
||||
{ ...customTriangle, rotate: NaN },
|
||||
{ ...customTriangle, rotate: Infinity },
|
||||
{ ...customTriangle, rotate: '30' },
|
||||
{ ...customTriangle, stroke: { color: '#111', width: -1 } },
|
||||
{ ...customTriangle, stroke: { color: '#111', width: Infinity } },
|
||||
{ ...customTriangle, stroke: { color: '#111', width: 2, style: 'unknown' } },
|
||||
{ ...customTriangle, stroke: { color: '#111', width: 2, extra: true } },
|
||||
{ ...customTriangle, elementId: 'model-owned' },
|
||||
{ ...customTriangle, shape: 'rectangle' },
|
||||
{ ...customTriangle, shape: 'circle' },
|
||||
{ ...customTriangle, shape: 'triangle' },
|
||||
{ ...customTriangle, shape: 'custom' },
|
||||
{ ...customTriangle, shape: 'unknown' },
|
||||
];
|
||||
|
||||
describe('Native whiteboard rotation and custom geometry', () => {
|
||||
it('advertises one path-only schema and rejects removed shape discriminators', () => {
|
||||
const draw = build(service()).tools.find((tool) => tool.name === 'wb_draw_shape')!;
|
||||
expect(draw.parameters).toMatchObject({
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: expect.arrayContaining(['expectedLastSeq', 'path', 'viewBox']),
|
||||
});
|
||||
expect(draw.parameters).not.toHaveProperty('anyOf');
|
||||
expect(draw.parameters).not.toHaveProperty('properties.shape');
|
||||
for (const shape of ['rectangle', 'circle', 'triangle', 'custom']) {
|
||||
expect(() => draw.prepareArguments?.({ ...customTriangle, shape })).toThrow(
|
||||
'omit the removed shape field',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('publishes provider-compatible array items schemas', () => {
|
||||
const invalidPaths: string[] = [];
|
||||
function visit(value: unknown, path: string) {
|
||||
if (!value || typeof value !== 'object') return;
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (key === 'items' && Array.isArray(child)) invalidPaths.push(`${path}.items`);
|
||||
if (Array.isArray(child))
|
||||
child.forEach((item, index) => visit(item, `${path}.${key}[${index}]`));
|
||||
else visit(child, `${path}.${key}`);
|
||||
}
|
||||
}
|
||||
for (const tool of build(service()).tools) visit(tool.parameters, tool.name);
|
||||
expect(invalidPaths).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ name: 'rectangle', path: 'M0 0 H1 V1 H0 Z', viewBox: [1, 1] },
|
||||
{
|
||||
name: 'circle',
|
||||
path: 'M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z',
|
||||
viewBox: [2, 2],
|
||||
},
|
||||
{ name: 'isosceles triangle', path: 'M1 0 L2 1 L0 1 Z', viewBox: [2, 1] },
|
||||
])('draws a $name using path-only arguments and default styling', async ({ path, viewBox }) => {
|
||||
const runtime = service();
|
||||
const draw = build(runtime).tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
await draw.execute('basic-shape', {
|
||||
expectedLastSeq: null,
|
||||
path,
|
||||
viewBox,
|
||||
x: 1,
|
||||
y: 2,
|
||||
width: 30,
|
||||
height: 40,
|
||||
});
|
||||
expect(runtime.append).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
payload: expect.objectContaining({
|
||||
operation: {
|
||||
kind: 'element_added',
|
||||
element: expect.objectContaining({
|
||||
path,
|
||||
viewBox,
|
||||
rotate: 0,
|
||||
fill: '#5b9bd5',
|
||||
fixedRatio: false,
|
||||
}),
|
||||
},
|
||||
}),
|
||||
}),
|
||||
);
|
||||
const op = vi.mocked(runtime.append).mock.calls[0]![0].payload.operation;
|
||||
if (op.kind !== 'element_added' || op.element.type !== 'shape')
|
||||
throw new Error('missing shape');
|
||||
expect(op.element).not.toHaveProperty('outline');
|
||||
expect(op.element).not.toHaveProperty('shape');
|
||||
});
|
||||
|
||||
it.each([45, -36.86989765, 12.5])(
|
||||
'persists rotation %s without changing unrotated coordinates',
|
||||
async (rotate) => {
|
||||
const runtime = service();
|
||||
const draw = build(runtime).tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
const params = {
|
||||
expectedLastSeq: null,
|
||||
path: 'M0 0 H1 V1 H0 Z',
|
||||
viewBox: [1, 1],
|
||||
x: 520,
|
||||
y: 180,
|
||||
width: 150,
|
||||
height: 150,
|
||||
rotate,
|
||||
};
|
||||
expect(draw.prepareArguments?.(params)).toBe(params);
|
||||
await draw.execute('rotate', params);
|
||||
expect(runtime.append).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
payload: expect.objectContaining({
|
||||
operation: {
|
||||
kind: 'element_added',
|
||||
element: expect.objectContaining({
|
||||
left: 520,
|
||||
top: 180,
|
||||
width: 150,
|
||||
height: 150,
|
||||
rotate,
|
||||
}),
|
||||
},
|
||||
}),
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('retains historical preset geometry after path-only draws and storage reopen', async () => {
|
||||
vi.stubGlobal('IDBKeyRange', IDBKeyRange);
|
||||
try {
|
||||
const indexedDB = new IDBFactory();
|
||||
const createRuntime = () =>
|
||||
createWhiteboardRuntimeService({
|
||||
store: new BrowserRuntimeStore({
|
||||
indexedDB,
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
}),
|
||||
resolveLearnerKey: () => 'learner-1',
|
||||
withMaintenanceLock: (work) => work(),
|
||||
});
|
||||
const historical = [
|
||||
'M 0 0 L 1000 0 L 1000 1000 L 0 1000 Z',
|
||||
'M 500 0 A 500 500 0 1 1 499 0 Z',
|
||||
'M 500 0 L 1000 1000 L 0 1000 Z',
|
||||
].map(
|
||||
(path, index): PPTShapeElement => ({
|
||||
id: `historical-shape-${index}`,
|
||||
type: 'shape',
|
||||
path,
|
||||
viewBox: [1000, 1000],
|
||||
left: index * 100,
|
||||
top: 40,
|
||||
width: 80,
|
||||
height: 80,
|
||||
rotate: 0,
|
||||
fill: '#5b9bd5',
|
||||
fixedRatio: false,
|
||||
}),
|
||||
);
|
||||
const runtime = createRuntime();
|
||||
await runtime.append({
|
||||
stageId: 'stage-1',
|
||||
expectedLastSeq: null,
|
||||
payload: {
|
||||
payloadVersion: WHITEBOARD_RUNTIME_PAYLOAD_VERSION,
|
||||
operationId: 'historical-shape-import',
|
||||
operation: {
|
||||
kind: 'legacy_snapshot_imported',
|
||||
source: {
|
||||
kind: LEGACY_WHITEBOARD_SOURCE_KIND,
|
||||
fingerprint: `sha256:${'1'.repeat(64)}`,
|
||||
},
|
||||
whiteboard: {
|
||||
id: 'historical-board',
|
||||
viewportSize: 1000,
|
||||
viewportRatio: 0.5625,
|
||||
elements: historical,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const draw = build(runtime).tools.find((tool) => tool.name === 'wb_draw_shape')!;
|
||||
await expect(
|
||||
draw.execute('new-path', { ...customTriangle, expectedLastSeq: 0 }),
|
||||
).resolves.toMatchObject({ details: { committedSeq: 1 } });
|
||||
const reopened = await createRuntime().read('stage-1');
|
||||
expect(reopened.whiteboard?.elements.slice(0, 3)).toEqual(historical);
|
||||
expect(reopened.whiteboard?.elements).toHaveLength(4);
|
||||
expect(reopened.whiteboard?.elements[3]).toMatchObject({ path: customTriangle.path });
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ ...customTriangle, rotate: -12.5 },
|
||||
{
|
||||
...customTriangle,
|
||||
path: 'M0 0 L4 0 L4 2 L2 3 L0 2 Z',
|
||||
fillColor: 'transparent',
|
||||
stroke: { color: '#111111', width: 2, style: 'dashed' },
|
||||
},
|
||||
{
|
||||
...customTriangle,
|
||||
path: 'M0 1 C1 0 2 0 3 1',
|
||||
viewBox: [3, 1],
|
||||
fillColor: 'transparent',
|
||||
stroke: { color: '#333333', width: 2 },
|
||||
},
|
||||
{
|
||||
...customTriangle,
|
||||
path: 'M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z',
|
||||
viewBox: [2, 2],
|
||||
},
|
||||
{
|
||||
...customTriangle,
|
||||
path: 'M0 0 A1 1 0 0 1 1 1',
|
||||
viewBox: [1, 1],
|
||||
fillColor: 'transparent',
|
||||
stroke: { color: '#000000', width: 2 },
|
||||
},
|
||||
])('keeps custom path $path and optional stroke in PPTShapeElement', async (params) => {
|
||||
const runtime = service();
|
||||
const { tools, send } = build(runtime);
|
||||
const draw = tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
expect(draw.prepareArguments?.(params)).toBe(params);
|
||||
const result = await draw.execute('geometry', params);
|
||||
expect(result).toMatchObject({
|
||||
details: {
|
||||
committedSeq: 0,
|
||||
affected: {
|
||||
element: {
|
||||
type: 'shape',
|
||||
path: params.path,
|
||||
viewBox: params.viewBox,
|
||||
rotate: 'rotate' in params ? params.rotate : 0,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
if ('stroke' in params)
|
||||
expect(result.details).toMatchObject({
|
||||
affected: {
|
||||
element: {
|
||||
outline: {
|
||||
...params.stroke,
|
||||
style: 'style' in params.stroke ? params.stroke.style : 'solid',
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(send).toHaveBeenCalledExactlyOnceWith({
|
||||
type: 'whiteboard',
|
||||
data: { kind: 'projection', stageId: 'stage-1', lastSeq: 0 },
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects invalid inputs in preparation and direct execution without records, sessions or events', async () => {
|
||||
vi.stubGlobal('IDBKeyRange', IDBKeyRange);
|
||||
try {
|
||||
const store = new BrowserRuntimeStore({
|
||||
indexedDB: new IDBFactory(),
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
});
|
||||
const runtime = createWhiteboardRuntimeService({
|
||||
store,
|
||||
resolveLearnerKey: () => 'learner-1',
|
||||
withMaintenanceLock: (work) => work(),
|
||||
});
|
||||
const append = vi.fn(runtime.append);
|
||||
const { tools, send } = build({ ...runtime, append });
|
||||
const draw = tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
for (const params of invalidShapes) {
|
||||
expect(() => draw.prepareArguments?.(params)).toThrow(/wb_draw_shape/);
|
||||
await expect(draw.execute('invalid', params)).resolves.toMatchObject({
|
||||
isError: true,
|
||||
details: { code: 'INVALID_WHITEBOARD_SHAPE' },
|
||||
});
|
||||
}
|
||||
expect(append).not.toHaveBeenCalled();
|
||||
expect(send).not.toHaveBeenCalled();
|
||||
await expect(store.listSessions('stage-1', 'learner-1')).resolves.toEqual([]);
|
||||
await expect(runtime.read('stage-1')).resolves.toMatchObject({
|
||||
lastSeq: null,
|
||||
whiteboard: null,
|
||||
});
|
||||
const corrected = await draw.execute('corrected', customTriangle);
|
||||
expect(corrected).toMatchObject({ details: { committedSeq: 0 } });
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it('reports a useful grammar error and character position before append', async () => {
|
||||
const runtime = service();
|
||||
const draw = build(runtime).tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
await expect(
|
||||
draw.execute('bad-path', { ...customTriangle, path: 'M0 0 Z1 1' }),
|
||||
).resolves.toMatchObject({
|
||||
isError: true,
|
||||
details: { error: { code: 'syntax', field: 'path', position: expect.any(Number) } },
|
||||
content: [expect.objectContaining({ text: expect.stringContaining('wb_draw_shape path:') })],
|
||||
});
|
||||
expect(runtime.append).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('retains custom paths and angles after reopening storage, enforces CAS, and exactly replays once', async () => {
|
||||
vi.stubGlobal('IDBKeyRange', IDBKeyRange);
|
||||
const indexedDB = new IDBFactory();
|
||||
const createRuntime = () => {
|
||||
const store = new BrowserRuntimeStore({
|
||||
indexedDB,
|
||||
payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
|
||||
});
|
||||
const runtime = createWhiteboardRuntimeService({
|
||||
store,
|
||||
resolveLearnerKey: () => 'learner-1',
|
||||
withMaintenanceLock: (work) => work(),
|
||||
});
|
||||
return { store, runtime };
|
||||
};
|
||||
try {
|
||||
const { runtime } = createRuntime();
|
||||
const params = {
|
||||
...customTriangle,
|
||||
rotate: -36.86989765,
|
||||
fillColor: 'transparent',
|
||||
stroke: { color: '#111', width: 2 },
|
||||
};
|
||||
const draw = build(runtime).tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
await expect(draw.execute('same-shape', params)).resolves.toMatchObject({
|
||||
details: { committedSeq: 0, replayed: false },
|
||||
});
|
||||
const original = await runtime.read('stage-1');
|
||||
const reopened = createRuntime();
|
||||
await expect(reopened.runtime.read('stage-1')).resolves.toEqual(original);
|
||||
const { tools } = build(reopened.runtime);
|
||||
const replay = tools.find((t) => t.name === 'wb_draw_shape')!;
|
||||
await expect(replay.execute('same-shape', params)).resolves.toMatchObject({
|
||||
details: { committedSeq: 0, replayed: true },
|
||||
});
|
||||
await expect(replay.execute('stale-shape', params)).resolves.toMatchObject({
|
||||
isError: true,
|
||||
details: { code: 'STALE_STATE', actualLastSeq: 0 },
|
||||
});
|
||||
await expect(
|
||||
replay.execute('next-shape', {
|
||||
expectedLastSeq: 0,
|
||||
path: 'M1 0 L2 1 L0 1 Z',
|
||||
viewBox: [2, 1],
|
||||
x: 10,
|
||||
y: 20,
|
||||
width: 30,
|
||||
height: 40,
|
||||
}),
|
||||
).resolves.toMatchObject({ details: { committedSeq: 1 } });
|
||||
await expect(replay.execute('same-shape', params)).resolves.toMatchObject({
|
||||
details: { committedSeq: 0, lastSeq: 1, replayed: true },
|
||||
});
|
||||
const final = await reopened.runtime.read('stage-1');
|
||||
expect(final.whiteboard?.elements[0]).toEqual(original.whiteboard?.elements[0]);
|
||||
const sessions = await reopened.store.listSessions('stage-1', 'learner-1');
|
||||
await expect(reopened.store.listRecords(sessions[0]!.id)).resolves.toHaveLength(2);
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -66,6 +66,31 @@ function makeBody(overrides: Partial<StatelessChatRequest> = {}): StatelessChatR
|
||||
}
|
||||
|
||||
describe('Pi director prompt closure routing', () => {
|
||||
it('describes Native shape geometry only when its tool is available', () => {
|
||||
const withShapes = buildNativeChildPrompt(makeBody(), agents[0], [], ['wb_draw_shape']);
|
||||
expect(withShapes).toContain('viewBox:[w,h]');
|
||||
expect(withShapes).toContain('viewBox:[4,3]');
|
||||
expect(withShapes).toContain('M0 1 C1 0 2 0 3 1');
|
||||
expect(withShapes).toContain('fillColor:"transparent"');
|
||||
expect(withShapes).toContain('origin (0,0) is the unrotated element upper-left');
|
||||
expect(withShapes).toContain('M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z');
|
||||
expect(withShapes).toContain('M0 0 A1 1 0 0 1 1 1');
|
||||
expect(withShapes).toContain('keep the viewBox ratio equal to width:height');
|
||||
expect(withShapes).toContain('x=centerX-width/2 and y=centerY-height/2');
|
||||
expect(withShapes).toContain('Arc radii are in viewBox units');
|
||||
expect(withShapes).toContain('Close filled shapes with Z');
|
||||
expect(withShapes).not.toContain('hypotenuse');
|
||||
expect(withShapes).not.toContain('-36.86989765');
|
||||
expect(withShapes).not.toContain('shape:"');
|
||||
expect(withShapes).not.toContain('Presets:');
|
||||
expect(buildNativeChildPrompt(makeBody(), agents[0], [], ['wb_read'])).not.toContain(
|
||||
'# Native whiteboard shapes',
|
||||
);
|
||||
expect(buildChildPrompt(makeBody(), agents[0], [], [], ['wb_draw_shape'])).not.toContain(
|
||||
'Rotated square on the hypotenuse',
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps a navigable outline and requires scene evidence before delegation', () => {
|
||||
const body = makeBody({
|
||||
storeState: {
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
import type { PPTShapeElement } from '@openmaic/dsl';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
areBasicShapes,
|
||||
assessComposition,
|
||||
hasVisibleFill,
|
||||
isCircle,
|
||||
isCurveScenarioShape,
|
||||
isPentagon,
|
||||
isRightTriangle,
|
||||
squareOnHypotenuse,
|
||||
type TriangleVertices,
|
||||
} from '@/eval/whiteboard-shapes/geometry-checks';
|
||||
import { compositionScenario } from '@/eval/whiteboard-shapes/composition-scenarios';
|
||||
|
||||
function shape(overrides: Partial<PPTShapeElement> = {}): PPTShapeElement {
|
||||
return {
|
||||
id: 'eval-shape',
|
||||
type: 'shape',
|
||||
path: 'M0 0 H1 V1 H0 Z',
|
||||
viewBox: [1, 1],
|
||||
left: 80,
|
||||
top: 100,
|
||||
width: 120,
|
||||
height: 120,
|
||||
rotate: 0,
|
||||
fill: '#5b9bd5',
|
||||
fixedRatio: false,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
const triangle = shape({
|
||||
path: 'M0 0 L4 0 L0 3 Z',
|
||||
viewBox: [4, 3],
|
||||
left: 100,
|
||||
width: 240,
|
||||
height: 180,
|
||||
});
|
||||
const hypotenuseSquare = shape({
|
||||
left: 160,
|
||||
top: 160,
|
||||
width: 300,
|
||||
height: 300,
|
||||
rotate: -36.86989765,
|
||||
});
|
||||
const pentagon = shape({
|
||||
path: 'M0 0 L4 0 L4 2 L2 3 L0 2 Z',
|
||||
viewBox: [4, 3],
|
||||
width: 240,
|
||||
height: 180,
|
||||
});
|
||||
const circle = shape({
|
||||
path: 'M1 0 A1 1 0 1 1 1 2 A1 1 0 1 1 1 0 Z',
|
||||
viewBox: [2, 2],
|
||||
left: 340,
|
||||
});
|
||||
const curve = shape({
|
||||
path: 'M0 1 C1 0 2 0 3 1',
|
||||
viewBox: [3, 1],
|
||||
left: 100,
|
||||
top: 300,
|
||||
width: 240,
|
||||
height: 80,
|
||||
rotate: 12.5,
|
||||
fill: 'transparent',
|
||||
outline: { color: '#333333', width: 2, style: 'solid' },
|
||||
});
|
||||
|
||||
describe('whiteboard shape evaluation geometry', () => {
|
||||
it.each([
|
||||
'none',
|
||||
'transparent',
|
||||
'#0000',
|
||||
'#00000000',
|
||||
'rgba(12, 34, 56, 0)',
|
||||
'hsla(0, 0%, 0%, 0)',
|
||||
])('rejects an invisible SVG fill: %s', (fill) => {
|
||||
expect(hasVisibleFill(shape({ fill }))).toBe(false);
|
||||
const shapes = [shape({ fill }), circle, shape({ left: 600, path: 'M0.5 0 L1 1 L0 1 Z' })];
|
||||
expect(areBasicShapes(shapes)).toBe(false);
|
||||
});
|
||||
|
||||
it.each(['red', '#123', '#123456', '#12345680', 'rgba(12, 34, 56, 0.5)'])(
|
||||
'accepts visible static SVG colors: %s',
|
||||
(fill) => expect(hasVisibleFill(shape({ fill }))).toBe(true),
|
||||
);
|
||||
|
||||
it('rejects invalid colors and fully transparent elements', () => {
|
||||
expect(hasVisibleFill(shape({ fill: 'not-a-color' }))).toBe(false);
|
||||
expect(hasVisibleFill(shape({ opacity: 0 }))).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts the requested open cubic geometry with transparent fill and visible dark stroke', () => {
|
||||
expect(isCurveScenarioShape(curve)).toBe(true);
|
||||
expect(isCurveScenarioShape({ ...curve, fill: 'none' })).toBe(true);
|
||||
expect(isCurveScenarioShape({ ...curve, rotate: 372.5 })).toBe(true);
|
||||
});
|
||||
|
||||
it.each([
|
||||
{ path: 'M0 0 C1 1 2 1 3 0 Z' },
|
||||
{ path: 'M0 0 C1 1 2 1 0 0' },
|
||||
{ path: 'M0 1 L3 1' },
|
||||
{ path: 'M0 1 C1 0 2 0 3 1 M0 0 C1 0 2 0 3 0' },
|
||||
{ path: 'M0 1 C1 5 2 5 3 1' },
|
||||
{ left: 10000, top: 10000 },
|
||||
{ width: 1, height: 1 },
|
||||
{ rotate: -12.5 },
|
||||
{ fill: '#333333' },
|
||||
{ opacity: 0 },
|
||||
{ outline: { color: 'transparent', width: 2, style: 'solid' as const } },
|
||||
{ outline: { color: '#00000000', width: 2, style: 'solid' as const } },
|
||||
{ outline: { color: '#ffffff', width: 2, style: 'solid' as const } },
|
||||
{ outline: { color: '#333333', width: 1, style: 'solid' as const } },
|
||||
])('rejects a curve that violates the requested geometry or style: %j', (overrides) => {
|
||||
expect(isCurveScenarioShape({ ...curve, ...overrides })).toBe(false);
|
||||
});
|
||||
|
||||
it('scores the actual triangle after applying element rotation', () => {
|
||||
expect(isRightTriangle(triangle)).toBe(true);
|
||||
expect(isRightTriangle({ ...triangle, rotate: 90 })).toBe(false);
|
||||
expect(isRightTriangle({ ...triangle, path: 'M0 0 H4 V3 H0 Z' })).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts the actual outward hypotenuse square in either winding direction', () => {
|
||||
expect(squareOnHypotenuse(hypotenuseSquare)).toBe(true);
|
||||
expect(squareOnHypotenuse({ ...hypotenuseSquare, path: 'M0 0 V1 H1 V0 Z' })).toBe(true);
|
||||
expect(squareOnHypotenuse({ ...hypotenuseSquare, rotate: 36.86989765 })).toBe(false);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'M0 0 L1 0 L0 1 Z',
|
||||
'M0.5 0 A0.5 0.5 0 1 1 0.5 1 A0.5 0.5 0 1 1 0.5 0 Z',
|
||||
'M0 0 L1 1 L1 0 L0 1 Z',
|
||||
'M0 0 H0.5 V1 H0 Z',
|
||||
'M0 0 H1 V1 H0',
|
||||
])('rejects a non-square path in the correct hypotenuse box: %s', (path) => {
|
||||
expect(squareOnHypotenuse({ ...hypotenuseSquare, path })).toBe(false);
|
||||
});
|
||||
|
||||
it('counts real pentagon corners and tolerates repeated closing endpoints', () => {
|
||||
expect(isPentagon(pentagon)).toBe(true);
|
||||
expect(isPentagon({ ...pentagon, path: 'M0 0 L4 0 L4 2 L2 3 L0 2 L0 0 Z' })).toBe(true);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'M0 0 H4 V3 H0 Z',
|
||||
'M0 0 L4 0 L4 3 L0 3 L0 0 Z',
|
||||
'M0 0 L2 0 L4 0 L4 3 L0 3 Z',
|
||||
'M0 0 L4 2 L0 2 L4 0 L2 3 Z',
|
||||
'M0 0 L4 0 L0 3 Z M1 1 L2 1 L1 2 Z',
|
||||
])('rejects duplicate, collinear or intersecting pseudo-pentagons: %s', (path) => {
|
||||
expect(isPentagon({ ...pentagon, path })).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts complete circles and rejects displaced or non-circular paths', () => {
|
||||
expect(isCircle(circle, 400, 160, 60)).toBe(true);
|
||||
expect(isCircle({ ...circle, top: 40 }, 400, 160, 60)).toBe(false);
|
||||
expect(isCircle({ ...circle, path: 'M0 0 H2 V2 H0 Z' }, 400, 160, 60)).toBe(false);
|
||||
});
|
||||
|
||||
it('checks rectangle edge order instead of accepting any four matching corners', () => {
|
||||
const shapes = [shape(), circle, shape({ left: 600, path: 'M0.5 0 L1 1 L0 1 Z' })];
|
||||
expect(areBasicShapes(shapes)).toBe(true);
|
||||
expect(
|
||||
areBasicShapes([{ ...shapes[0], path: 'M0 0 L1 1 L1 0 L0 1 Z' }, ...shapes.slice(1)]),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('unseen 5–12–13 composition evaluation', () => {
|
||||
const vertices: TriangleVertices = [
|
||||
[300, 220],
|
||||
[492, 220],
|
||||
[300, 300],
|
||||
];
|
||||
// Hand-computed reference contours, independent of the expected-square generator.
|
||||
const paths = [
|
||||
'M300 220 L492 220 L300 300 Z',
|
||||
'M300 220 L492 220 L492 28 L300 28 Z',
|
||||
'M492 220 L300 300 L380 492 L572 412 Z',
|
||||
'M300 300 L300 220 L220 220 L220 300 Z',
|
||||
];
|
||||
const fromPath = (path: string) =>
|
||||
shape({
|
||||
path,
|
||||
viewBox: [1000, 562.5],
|
||||
left: 0,
|
||||
top: 0,
|
||||
width: 1000,
|
||||
height: 562.5,
|
||||
});
|
||||
const shapes = paths.map(fromPath);
|
||||
|
||||
it('accepts correct geometry as separate elements or one compound path, in either winding', () => {
|
||||
expect(assessComposition(shapes, vertices).passed).toBe(true);
|
||||
expect(assessComposition([fromPath(paths.join(' '))], vertices).passed).toBe(true);
|
||||
expect(assessComposition(shapes, [vertices[0], vertices[2], vertices[1]]).passed).toBe(true);
|
||||
});
|
||||
|
||||
it('scores actual transformed vertices rather than element boxes', () => {
|
||||
const rotated = shapes.map((s) => ({ ...s, rotate: 90 }));
|
||||
const rotatedVertices = vertices.map(([x, y]) => [
|
||||
500 - (y - 281.25),
|
||||
281.25 + x - 500,
|
||||
]) as TriangleVertices;
|
||||
expect(assessComposition(rotated, rotatedVertices).passed).toBe(true);
|
||||
expect(assessComposition(rotated, vertices).passed).toBe(false);
|
||||
});
|
||||
|
||||
it('rejects an inward square, crossed edges, and an unclosed contour', () => {
|
||||
for (const wrong of [
|
||||
'M300 220 L492 220 L492 412 L300 412 Z',
|
||||
'M492 220 L380 492 L300 300 L572 412 Z',
|
||||
'M492 220 L300 300 L380 492 L572 412',
|
||||
]) {
|
||||
const invalid = [shapes[0], shapes[1], fromPath(wrong), shapes[3]];
|
||||
expect(assessComposition(invalid, vertices).passed).toBe(false);
|
||||
expect(
|
||||
assessComposition([fromPath(invalid.map((s) => s.path).join(' '))], vertices).passed,
|
||||
).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects missing, duplicate, extra, or invisible geometry', () => {
|
||||
expect(assessComposition(shapes.slice(0, 3), vertices).passed).toBe(false);
|
||||
expect(assessComposition([shapes[0], shapes[1], shapes[2], shapes[2]], vertices).passed).toBe(
|
||||
false,
|
||||
);
|
||||
expect(assessComposition([...shapes, shape()], vertices).passed).toBe(false);
|
||||
expect(assessComposition([...shapes, fromPath('M0 0 L1 1')], vertices).passed).toBe(false);
|
||||
expect(
|
||||
assessComposition(
|
||||
shapes.map((s) => ({ ...s, fill: 'transparent' })),
|
||||
vertices,
|
||||
).passed,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it('rejects an otherwise correct composition outside the canvas', () => {
|
||||
const shifted = shapes.map((s) => ({ ...s, left: 1000 }));
|
||||
const shiftedVertices = vertices.map(([x, y]) => [x + 1000, y]) as TriangleVertices;
|
||||
expect(assessComposition(shifted, shiftedVertices).withinCanvas).toBe(false);
|
||||
expect(assessComposition(shifted, shiftedVertices).passed).toBe(false);
|
||||
});
|
||||
|
||||
it.each(['none', '#0000', '#00000000', 'rgba(12, 34, 56, 0)', 'hsla(0, 0%, 0%, 0)'])(
|
||||
'rejects geometrically correct compositions with invisible fill %s',
|
||||
(fill) => {
|
||||
expect(
|
||||
assessComposition(
|
||||
shapes.map((s) => ({ ...s, fill })),
|
||||
vertices,
|
||||
).passed,
|
||||
).toBe(false);
|
||||
expect(assessComposition([{ ...fromPath(paths.join(' ')), fill }], vertices).passed).toBe(
|
||||
false,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it('rejects geometrically correct but fully transparent elements', () => {
|
||||
expect(
|
||||
assessComposition(
|
||||
shapes.map((s) => ({ ...s, opacity: 0 })),
|
||||
vertices,
|
||||
).passed,
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it.each([0, 1, 2, 3])(
|
||||
'keeps the 5–12–13 ratio in variant %i without disclosing square placements',
|
||||
(variant) => {
|
||||
const scenario = compositionScenario(variant);
|
||||
const [a, b, c] = scenario.vertices;
|
||||
const length = (p: number[], q: number[]) => Math.hypot(p[0] - q[0], p[1] - q[1]);
|
||||
expect(length(a, b) / length(a, c)).toBeCloseTo(12 / 5, 3);
|
||||
expect(length(b, c) / length(a, c)).toBeCloseTo(13 / 5, 3);
|
||||
expect(scenario.request).not.toMatch(/viewBox|path:|rotate:|正方形.*\d/);
|
||||
},
|
||||
);
|
||||
});
|
||||
Reference in new issue
Block a user