mirror of
https://github.com/THU-MAIC/OpenMAIC.git
synced 2026-10-28 16:35:27 +08:00
@openmaic/generation@0.3.23
60
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 杨慎andcosarah
|
6e20534fe0 |
fix(svg): validate malformed slide shapes and recover their text (#1915)
Add @openmaic/geometry, a browser-independent validator for SVG path syntax, finite view boxes and transforms and bounded resources, with structured errors and shared shape-to-text recovery. Generation, rendering, both editors and PPTX export use it: invalid shapes with a label render as text, unlabelled ones are omitted, stored records are not rewritten, and recovered shapes cannot be resized. Imported PPTX labels keep their own insets. Bumps generation 0.3.22, renderer 0.1.16, editor 0.0.12 and geometry 0.1.0. Closes #1851 Co-authored-by: cosarah <cosarah@users.noreply.github.com> |
||
|
Yizuki_AmeandYizukiAme
|
86f38c3181 |
fix(renderer): bound inline math work without starving active slides (#1930)
Share a per-frame markup allowance (256k characters) across live prose roots, let thumbnails use only what the foreground leaves and finish in idle time, reuse deferred typesetting results, and keep snapshot rendering synchronous. Matrix-heavy slides no longer block the main thread in one long task; ordinary and dense slides still typeset before paint. Bumps @openmaic/renderer to 0.1.15. Fixes #1869 Co-authored-by: YizukiAme <YizukiAme@users.noreply.github.com> |
||
|
|
6760c2b764 |
refactor(chat): retire the legacy director-graph classroom chat (#1913)
* refactor(chat): move Pi's shared parser, action schemas and store-state type out of the legacy chat owners Pi classroom chat still imports three pieces from the legacy director-graph chat, which #1814 retires. Give each a Pi or neutral owner first, so the legacy files can later be deleted without touching Pi: - The structured-output parser (createParserState, parseStructuredChunk, finalizeParser, looksLikeStructuredFragment, ParseResult) moves verbatim from lib/orchestration/stateless-generate.ts to lib/chat/structured-output.ts. Only two comments change: they no longer name director-graph as the reader of the per-call ordered indices. This also stops the Pi route from loading director-graph and LangGraph through the parser's module. - lib/orchestration/tool-schemas.ts moves unchanged to lib/chat/pi/tool-schemas.ts. - AgentLoopStoreState moves unchanged from lib/chat/agent-loop.ts to lib/chat/store-state.ts; the Pi request path in use-chat-sessions uses it. The remaining legacy consumers import from the new locations until they are removed. No behavior change. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(chat): cover classroom model precedence on /api/chat/pi with the real resolver The user-route, lock and body precedence for classroom interaction is covered with the real resolveModel only through the legacy /api/chat route, which #1814 removes. Route-level Pi tests mock resolveModel. Add the same four cases against /api/chat/pi, asserting the model handed to the Pi runtime: - a chat-adapter user route wins over the body model; - a locked classroom slot wins over the user route; - with no matching route, the body model is used; - a locked classroom slot is kept despite unlisted body and route models. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(eval): run the whiteboard layout eval against Pi classroom chat The whiteboard eval drove the legacy /api/chat through runAgentLoop. Point it at /api/chat/pi, which #1814 keeps as the only classroom chat: - one Pi request per scripted user turn, with whiteboard tools enabled as the app does for in-class requests; - text and actions grouped by each agent's messageId and sealed into their own history message at agent_end, since one Pi request can hand the turn to several agents; - the previous turn's done.directorState carried into the next request within a scenario run, as the app does within a session; - an error event, a non-200 response, or a stream without done fails the scenario through the existing error path, which skips its remaining turns (previously an error event was swallowed and the turn still captured); - a native whiteboard event fails the run, because the harness cannot see the native child's server-side board. Not seeing one does not prove the server uses the legacy child; run the eval on a server configured for it. Scores are a new baseline and are not comparable with runs of the legacy runner. Scenarios, capture, scoring and reporting are unchanged. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(eval): settle queued actions and cancel the request when a whiteboard turn fails Review of the Pi whiteboard runner (#1913) found two lifecycle gaps on the failure paths it added: - A turn that failed (an SSE error, a native whiteboard event, or EOF without done) returned and disposed the scenario without waiting for its queued actions. The state manager drives shared store singletons, so a pending action could finish inside the next scenario and land in its screenshot. Every turn exit now settles the action queue first. A queued rejection is handled while the stream is still open and is logged at settle time, as on the success path, without replacing the turn's own error. - Stopping early only released the reader lock, so the request kept running. Each turn's request now has its own AbortController, aborted on exit, and an unfinished response body is cancelled before the lock is released. runScenario is exported with its CLI settings passed in, and main() runs only when the file is the entry point, so an offline Vitest suite can exercise the real function: failed turns settle before returning, the native early exit aborts and cancels, a concurrent action rejection is not unhandled, and the success path keeps one Pi request per turn, messageId grouping, directorState carry-over and score:null on scorer failure. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(eval): run the answering and answer-content evals on Pi; retire premature-end Both evals drove the legacy single-step director prompt and agent prompt, which #1814 retires. Run them on the real Pi director in process instead (eval/orchestration/pi-harness.ts): the Pi config helpers supply agent configs and budgets, whiteboard tools are on as for classroom requests, thinking is off, and the legacy child runtime is pinned. The harness skips the HTTP route, so route authorization and tool-capability checks are not exercised. - answering classifies by the first recorded call_agent attempt in the director tool trace: TEACHER, OTHER_AGENT (another or a non-existent agent), NO_DISPATCH (none recorded; wrong, kept in the denominator), or ERROR (a failed run, excluded as before). A later recorded attempt never rescues the first. - answer-content judges the full reply of the first recorded teacher attempt. A failed run, no recorded teacher attempt, or a first one that is skipped, marked isError or empty scores 0. The judge, opening window, threshold and all-samples denominator are unchanged. Pi's short teacher replies make the two metrics nearly coincide, and the score includes routing. - Both score only what the trace records. A schema-rejected attempt is not recorded, so a later one can score, and a legacy child's partial text after a provider failure is judged. Scores can be overstated. Real-chain tests (real Pi loop and agent SDK, scripted StreamFn) pin both limits. - The old prompt A/B variants are removed; results are a Pi baseline. - premature-end is retired: its single END decision has no Pi counterpart, and the close gate does not show a question was answered. - The three remaining eval scripts load with tsx --conditions=import, so ESM-only dependencies resolve. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(chat): always run classroom chat on Pi and fail builds that still opt out The classroom chat hook, the /api/chat/pi route and PlaybackChromeRoot no longer read NEXT_PUBLIC_PI_CHAT_ENABLED. The hook sends every turn through the existing Pi single-request path; the frontend no longer calls the legacy /api/chat loop (fetchStatelessChat and the runAgentLoop tail are removed). The Pi branch itself is unchanged apart from indentation. isPiChatEnabled and its readDefaultOnBoolean helper are removed. The independent courseware-reference gate stays default-off. next.config.ts now exports a phase function. In the production-build phase it fails when NEXT_PUBLIC_PI_CHAT_ENABLED is set to anything other than '', 'true' or '1' (exact match), which covers every value that used to select the legacy runtime. The error names the value and points to the Classroom model setting, the google native preset for Gemini, and the administrator for managed configurations. Other phases do not check the flag. The fixed env pin for the flag is removed. This is an intermediate step of #1814 PR 3: the legacy route, director graph and agent loop are still present and are removed in a follow-up commit, and Docker/Compose wiring and docs still describe the old rollback. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(chat): remove the legacy director-graph classroom chat backend The frontend has run every classroom turn on /api/chat/pi since the previous commit, so nothing reaches the legacy stack any more. Remove it: - app/api/chat/route.ts (no stub, redirect or 410; /api/chat/pi stays) - lib/orchestration/{stateless-generate,director-graph,ai-sdk-adapter, director-prompt,prompt-builder}.ts and lib/chat/agent-loop.ts - summarizers/{conversation-summary,message-converter}.ts - the director, agent-system and agent-system-wb-* templates, the speech-guidelines and whiteboard-reference snippets, and their prompt and snippet IDs Pi's default legacy child, the shared structured-output parser, the Pi tool schemas, the shared summarizers, the registry and the protocol types are unchanged. Dependency declarations are left for a follow-up commit. Tests: - Drop the legacy chat route from the model-authorization tests and the coverage matrix. The mixed compatibility cases keep their six and eight stage assertions, and the exhaustive resolver call-site scan is unchanged. The locked-classroom case now goes through /api/chat/pi. - Retire chat-route-user-routes (its precedence cases are rebuilt on the Pi route in route-model-precedence), the agent-loop async-state test, and the conversation-summary test. - Keep the #511 intent as a history input contract on Pi: toHistoryMessages keeps the human challenge as a user turn in order, and buildUserPrompt still reports it after a later agent reply. - Move the role, peer-context and language-directive prompt checks onto Pi's buildChildPrompt. Keep the PBL and placeholder-lint template tests and the getActionDescriptions LaTeX guard. - Add middleware cases for POST /api/chat (401 without access, pass-through otherwise) and a Playwright spec that expects a plain 404 from the running app. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(deps): drop the unused @langchain/langgraph dependency The legacy director graph was its only consumer and was removed in the previous commit. Remove the root declaration and let pnpm update the lockfile: @langchain/langgraph and @langchain/langgraph-checkpoint are no longer installed. @langchain/langgraph-sdk stays (an optional CopilotKit peer and a direct dependency of @ag-ui/langgraph). @langchain/core is kept: CopilotKit requires it as a peer and pnpm would install it anyway. partial-json and jsonrepair are kept for the Pi parser. The re-resolution also moved vite 8.0.0's postcss from 8.5.15 to 8.5.23 (both were already in the lockfile; Next's postcss is unchanged) and shortened four eslint-import snapshot keys without changing versions. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(docker): keep the Pi chat flag only as an input to the build check NEXT_PUBLIC_PI_CHAT_ENABLED no longer selects a chat runtime; next.config.ts only reads it to fail a production build that still opts out of Pi. Keep the builder ARG and the Compose build-arg forwarding so an explicit false still reaches that check, and drop the ENV line: build args are already visible to the builder's RUN steps, and the value has nothing to configure in the image. Remove the legacy-chat rollback comment and command from docker-compose.yml and replace the rollback section of .env.example with a short note that the legacy runtime is gone and which values now fail the build. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: describe classroom chat after the legacy runtime removal - Configuration docs (all six locales): drop the Pi rollback example, the advice to rebuild with the flag set to false and the Compose rollback command. Say instead that classroom chat always runs on Pi and needs a tool-calling Classroom model (google preset for Gemini), that the flag should be removed, which exact values a production build still accepts, and that the Compose service env_file supplies runtime values only. - READMEs: drop the LangGraph badge and describe lib/orchestration and the Pi director loop as they now are. - CHANGELOG: add the breaking change (model requirement, the flag, POST /api/chat answering 404 after the access gate) and the dependency removal. - lib/prompts/README.md: point the role and length guidelines at lib/chat/pi/prompts.ts, correct which tests check placeholders, use a missing snippet in the error example, and start the whiteboard eval server with the native child runtime off. - Remove the two comments that told CI and Playwright to leave the flag unset. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(chat): pin the server key on a user-route model; document answering eval limits - route-model-precedence: when the user route switches the classroom model to another provider without a key of its own, assert that the model is built with the server key rather than the body's key. - answering eval: state in the header comment and the generated report that a run which throws, is aborted or ends without done is ERROR even after it dispatched an agent, that ERROR samples stay out of the correct-rate denominator, and what the rate therefore does and does not measure. The scoring rules are unchanged. Refs #1814 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
Yizuki_Ameandwyuc
|
d725f19194 |
ci: cover every app PostgreSQL contract suite (#1867)
Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
|
2833c6c036 |
fix(render): bind render jobs to the requesting owner (#1884)
Each MP4 render job is bound to the immutable owner of the submitting request: the app proxy asserts the owner on submit, poll, cancel and download, and the render-service only finds jobs for that owner (foreign and unknown jobs get the same 404, with no cancellation or artifact lookup). Job operations live under a separate /owner-bound/render namespace, so an instance that predates owner binding answers 404 without running a job handler and mixed old/new pools fail closed; job responses must carry the owner-bound marker. Upgrade the render-service together with or before the app. Co-authored-by: duanjialing.777 <duanjialing.777@bytedance.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 5.5
|
636fab0d7e |
release: OpenMAIC 1.2.0-rc.1 (server-first) (#1794)
* ci: run CI for the provider-config integration branch
Development of the provider configuration RFC (#1701) lands on
integration/provider-config; build it on push and run CI for pull
requests into it, as with earlier integration branches.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* feat(config): capability slot registry and stage-to-slot mapping (#1726)
* feat(config): capability slot registry and stage-to-slot mapping
First P0 step of the provider configuration RFC (#1701, tracked in
#1725): define the capability slot forest and map every LLM stage key
to exactly one slot. Pure data with no callers yet, so there is no
behavior change.
Requirements are checked on the slot that declares them and are not
passed down to child slots, so agent.title can use a model without
tool calling even though agent requires it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): pin every stage destination and the capability roots
Review found that the station and root assertions derived their
expectations from the module under test: re-pointing a single-stage
station or dropping a capability root still passed. The tests now
compare against an independently written stage table and root list,
and pin agent.title as config-only. Also document that browserless
outlines still run on the generate-classroom model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): pin slotForStage and full slot lineages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): provider presets and the openmaic.yml schema (#1727)
* feat(config): provider presets and the openmaic.yml schema
Second P0 step of the provider configuration RFC (#1701, tracked in
#1725).
- lib/config/provider-presets.ts: one preset per built-in registry
entry, plus the token plans as multi-capability presets whose stage
recommendations become slot recommendations. Registry ids that
collide across capabilities get explicit preset ids.
- lib/server/model-config/openmaic-yml.ts: parse and validate the
operator's openmaic.yml (or the file named by OPENMAIC_CONFIG):
${VAR} interpolation, strict schema, and cross-checks that every
assignment names a declared provider whose preset offers the slot's
capability. Every problem is reported with its path.
- instrumentation.ts: an invalid file refuses to start. Without a file
nothing changes, and nothing resolves models through the file yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): harden openmaic.yml validation after review
- Look providers up by own key only, so "constructor:m" is not taken
for a declared provider.
- Refuse non-mapping objects YAML produces (an unquoted timestamp
becomes a Date that the schema would accept as an empty object), and
refuse a YAML alias that refers back to itself instead of overflowing
the stack.
- Accept lowercase variable names; refuse a "${" with no closing
brace, checked on the text as written, never on substituted secrets.
- Cross-check the entries that are valid on their own even when the
schema rejects others, so every problem is reported at once without
duplicate errors for declared-but-invalid providers.
- Name the three valid assignment shapes when a value has none of them.
- Test the boot path: an invalid file exits with code 1 before any
schedule starts; a valid one boots.
- Ignore a root openmaic.yml in git.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep secrets out of openmaic.yml diagnostics
Review round 2:
- Read only variables the environment itself has, as non-empty
strings: ${constructor} or an inherited value is not a variable.
- Never print a value that came from ${VAR}, and report YAML syntax
errors by reason and position without js-yaml's source excerpt,
which can quote a literal key.
- Keep the base URL requirement on presets: SearXNG from its registry
entry, and Azure OpenAI and self-hosted MinerU, which have no usable
default endpoint.
- Check slot names and valid model references even inside an entry
that fails the schema, and report a failed placeholder once rather
than again as an empty value.
- Boot test for the no-file case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* refactor(config): validate openmaic.yml in two phases
Rounds 1-3 of review kept finding edge cases in one feature: running
the cross-checks on the half-valid parts of a file the schema had
rejected, so that every problem showed at once. It hid problems inside
a rejected entry, lost a __proto__ slot key, and built misleading
provider errors from failed placeholders. That feature is gone.
Validation now runs in two phases. Phase 1 is the document itself:
placeholders, key names (checked on the document as written, since the
schema drops a __proto__ key), and the schema. Any problem there stops
before phase 2, the references between entries, which therefore only
ever sees a fully valid file. Each phase reports all of its problems,
one per path.
YAML syntax errors now report only their position: js-yaml's reason
can quote source text too (an alias or tag name), not just its excerpt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(api)!: generate-classroom takes requirement + uploaded materials; capabilities follow server config (#1728)
Narrows POST /api/generate-classroom to { requirement, materialIds? }; capabilities follow server configuration; materials go through the owner material library (upload, reference by id, delete); new GET /api/generate-classroom/capabilities; skill docs updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve a slot through the configuration layers (#1729)
* feat(config): resolve a slot through the configuration layers
Third P0 step of the provider configuration RFC (#1701, tracked in
#1725). resolveSlot walks from a slot to its capability root and
returns the first assignment it meets, consulting the deployment layer
(openmaic.yml, locked) before the workspace layer at every node. An
explicit null disables the subtree; nothing assigned up to the root is
unassigned, with no fallback to any vendor. The result carries the
provider, preset, registry entry, effective base URL, key, model, call
options, fallback, where it was resolved and whether it is locked, and
the requested slot's own requirements checked against the model
catalogue (met, unmet or unknown).
Pure and uncalled for now, so there is no behavior change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): lock only written slots and trust the catalogue only where it applies
Review round 1:
- locked now means the requested slot itself is written in the
deployment layer. Inheriting a deployment value does not lock a slot,
since the workspace may still assign it (source and resolvedAt still
report where the value came from).
- A custom OpenAI-compatible endpoint borrows the OpenAI registry for
transport only, so its preset no longer trusts that model catalogue:
requirements there resolve to unknown.
- The fallback is checked against the slot's requirements too, and the
result exposes fallbackRequirements so retries can refuse it.
- A malformed reference fails with a path-qualified SlotResolutionError
that does not echo the value, which is often a misplaced key.
- Requirement tests use named catalogue models instead of picking one
from the catalogue at run time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): order layers deployment-first and keep references out of errors
Review round 2:
- resolveSlot no longer depends on the order its caller passes the
layers in: deployment layers always come first, for assignments and
provider lookup alike.
- Resolution errors name the path and the preset, never a value taken
from the reference, so a key pasted into the provider position does
not end up in a log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): cover catalogue aliases and fallback capability errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(classroom)!: save server-generated classrooms through server persistence (#1730)
Server-side classroom generation saves the finished course create-only into the request owner's library with media in the owner's asset pool; jobs move to PostgreSQL with owner-scoped polling; /api/classroom is removed; legacy data/classrooms files are imported once in the background. Adds @openmaic/storage 0.36.0 PgAssetStore.releasePending.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): translate the legacy configuration into a deployment layer (#1731)
* feat(config): translate the legacy configuration into a deployment layer
Deployments without openmaic.yml keep working: provider variables,
server-providers.yml, DEFAULT_MODEL, MODEL_ROUTES and MODEL_FALLBACK are
translated into the openmaic.yml shape and serve as the deployment layer
for resolveSlot.
- Providers: each configured entry becomes a provider under its preset
id; force-disabled entries are left out; AliDocMind's key pair goes
into credentials.
- DEFAULT_MODEL goes on the llm root. Every other slot that serves stages
gets what those stages used before (their route, or else the default
model), written only where inheritance would give something else, so
an unrouted stage under a routed parent stays on the default model.
- Stages that now share a slot but had different routes, models whose
provider has no server configuration, and stages that used the
browser's model but would now inherit a server one become startup
notices, never failures.
- MODEL_FALLBACK attaches to every chat assignment without its own.
- When openmaic.yml exists it wins, with a notice if legacy variables
are also set.
Nothing reads the layer yet; routes switch over in P1 (#1725).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): follow the agent driver's rules and keep notices value-free
Review round 1:
- The agent never used DEFAULT_MODEL: it needs a valid maic-agent-driver
route and is off otherwise, so the translation writes agent: null where
it would inherit a server model (and a notice for a route that does
not work today). Unrouted conversation titles reuse the driver's model
with thinking off, as the title generator does.
- Notices name registry ids only; a credential pasted into a model
variable is never repeated.
- Provider entries and route options are checked against the new schema
and left out with a notice (field names only) instead of producing a
file that would not parse; driver-only options are dropped elsewhere.
- A route with its own fallback never falls back to MODEL_FALLBACK, even
when that fallback cannot carry over; the retry model is part of the
comparison with the inherited value.
- Force-off switches without a configured entry are reported, a legacy
configuration made only of them is detected, and translating one adds
a deprecation notice.
- loadDeploymentLayer reads the process environment and working
directory like the legacy loaders, instead of taking parameters it
could only partly honor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): compare routes by behavior, validate references, name only known ids
Review round 2:
- Stages sharing a slot are compared by what the route changes for them
(model, thinking, fallback), with bare ids read as openai models and a
route to DEFAULT_MODEL counted as no route; api and contextWindow are
inert outside the driver.
- A model reference carries over only when it names a provider from the
providers section and forms a valid reference.
- Section keys and force-off ids are named in notices only when they are
registry ids with a preset.
- Retries now follow the slot; where a call site picked its retry model
by another label (scene types under scene-content, browserless
generation), the change is reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): chat references need a chat entry; one notice for retries
Review round 3:
- A model reference carries over only when its provider was translated
from the providers section, not merely declared by another section
under the same id.
- Routes are compared by the retry model that takes effect, so leaving
out a fallback equals repeating MODEL_FALLBACK.
- The per-call-site retry notices kept missing cases (streaming and
calls outside server-managed routing never retry today). They are
replaced by one notice, given whenever a retry model is configured,
that retries now follow the slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(config)!: translate only providers and the default model (#1732)
* refactor(config)!: translate only providers and the default model
MODEL_ROUTES does not map one to one onto slots: several stages share a
slot, the agent driver and conversation titles have rules of their own,
and retries are picked by call-site labels. Emulating that took most of
the translation and still ended in notices that are easy to miss.
The translation now carries over only the unambiguous part: providers,
DEFAULT_MODEL as the llm root and MODEL_FALLBACK as its fallback (the
agent stays null, as it never used DEFAULT_MODEL). A deployment that
sets MODEL_ROUTES without openmaic.yml gets LegacyRoutesError asking it
to write the per-stage models as slots. loadDeploymentLayer is not wired
into startup yet; that happens when routes switch to resolveSlot (P1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): say what a dropped MODEL_FALLBACK loses
Without DEFAULT_MODEL there is no assignment to hold MODEL_FALLBACK, and
calls that retried on it (server providers picked in the browser) stop
retrying; the notice says so. thinkingSchema is module-private again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(persistence): workspace model configuration with keys encrypted at rest (#1733)
* feat(persistence): workspace model configuration with keys encrypted at rest
The store behind the web settings of RFC #1701 (#1725, P1): one row per
workspace (owner) in workspace_model_config, the same shape as
openmaic.yml without policy.
- Provider secrets (apiKey, credentials) are sealed per provider with
AES-256-GCM under OPENMAIC_SECRET_KEY, bound to the provider id, and
never stored in the config column. Without the variable, a secret is
created once in data/instance-secret.key (the Docker volume).
- A secret sealed under another instance secret is reported as
unreadable ("enter again"), and a save that brings no new secret for
that provider keeps it, so a misconfigured secret cannot destroy keys.
- Saves replace the document with a compare-and-swap on the revision,
after the owner's identity lock; two concurrent first saves cannot
both land (PostgreSQL contract test).
- A claim moves the anonymous configuration to the account unless the
account has its own (core participant, order 900).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(persistence): publish the instance secret atomically; force the races in tests
Review round 1:
- The generated secret is written and flushed under a private name,
then published with an exclusive link, so no process can read it half
written and exactly one of two starting processes creates it. A file
that is not a complete generated secret (empty, truncated) is refused
instead of deriving a key anyone could compute.
- The PostgreSQL test holds both first saves at their insert until both
have read the missing row, and holds the row lock for the update case
until both saves queue behind it (checked by backend pid, not by any
lock waiter in the database).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(persistence): verify the written secret; require full GCM tags
Review round 2:
- The generated secret is written in full, read back and checked before
it is published; the temporary file is removed on any failure.
- Sealed values open only with a 12-byte IV and a full 16-byte tag
(authTagLength), so a shortened tag cannot weaken authentication.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve slots at request time over deployment, workspace and defaults (#1734)
* feat(config): resolve slots at request time over deployment, workspace and defaults
The runtime half of RFC #1701 resolution (#1725, P1):
- deploymentConfig(): the deployment layer, loaded once per process.
The legacy translation now splits into the deployment's providers and
a separate default layer (DEFAULT_MODEL, MODEL_FALLBACK) that locks
nothing and ranks below the workspace, as DEFAULT_MODEL ranked below
the model a user picked.
- workspaceLayer(owner) reads the web settings; requestWorkspaceId(req)
names the request's owner (none for an owner minted by the request).
- lookupSlot walks deployment and workspace over the whole tree first;
the defaults are a second walk, so a default on a child never outranks
a workspace choice higher up.
- resolveStageModel builds the language model: the configured slot, else
what the request still names the old way (deprecated), else the
defaults, else a loud error; a slot turned off fails whatever the
request names. A workspace provider's endpoint is checked like a
caller-supplied one and gets the transport that refuses redirects.
- resolveSlot gains the default source, and reports which layer declared
the provider, its proxy and credentials.
Nothing calls it yet; call sites switch next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep request identity and workspace endpoints inside their bounds
Review round 1:
- A refused owner credential throws InvalidOwnerCredentialError (401)
instead of resolving with the deployment's models and keys.
- A retired request owner gets no workspace: canonicalizing it would
hand an old anonymous cookie the claiming account's settings. Only
background work, which names a stored owner, is forwarded.
- A workspace provider may not be Amazon Bedrock (which falls back to
the server's AWS credential chain) or set a proxy (which routes around
the checked, redirect-refusing transport); the deployment still may.
- Test seams for the deployment config and the workspace loader, and
tests for identity, forwarding and caching.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): read only the named owner's settings; refuse unmet requirements
Review round 2:
- workspaceLayer reads exactly the owner it is given. Forwarding through
a claim let a request whose owner was claimed between its check and the
read see the account's settings; background work passes the owner it
works for now instead.
- A model the catalogue says does not meet the slot's requirement is
refused before anything is built (SlotRequirementError), without
consulting the request.
- Tests: a database failure fails the call without falling back, a
workspace provider without a key never gets the deployment's, and the
transport each provider source gets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(identity): exempt the model settings lookup from cookie forwarding
requestWorkspaceId resolves the owner only to pick whose model settings a
generation call uses, and returns no response of its own, like the
vision prompt helper already listed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config)!: resolve every LLM call through its capability slot (#1735)
* feat(config)!: resolve every LLM call through its capability slot
The LLM call sites of RFC #1701 switch to slot resolution (#1725, P1):
- resolveModel resolves a stage through its slot for the request's or
job's workspace: the deployment and workspace configuration first; the
model and key a request names (x-model, x-model-routes, body fields)
only for a slot left unassigned, deprecated; then the defaults an
older deployment set with DEFAULT_MODEL. Routes that read headers get
this through resolveModelFromRequest; the chat routes pass their
workspace; background work (agent runs, titles, generation jobs)
passes the owner it works for now.
- Retries follow the slot: a slot-resolved model carries its slot's
fallback (lib/ai/model-fallbacks.ts), which callLLM and the outline
stream use; only a model from the request path still retries on
MODEL_FALLBACK. A fallback that cannot meet the slot is not used.
- The agent driver resolves the agent slot: tool calling required,
api defaults to openai-completions, thinking.effort still refused.
Conversation titles resolve agent.title with thinking off unless the
title slot sets it.
- /api/generate-classroom resolves each step through its slot for the
job's owner, its outline through course.outline.
- MODEL_ROUTES is gone: loadDeploymentLayer runs at startup and a server
that still sets it without openmaic.yml refuses to start. The
pbl-chat and maic-agent stage keys, which nothing resolved, are removed.
BREAKING CHANGE: MODEL_ROUTES is no longer read; write per-stage models
as slots in openmaic.yml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): a slot fallback needs no serverManaged stamp; follow claims per stage
Review round 1:
- callLLM arms a model's attached slot fallback whether or not the caller
passes serverManaged (the PBL agents pass none); the stamp still gates
MODEL_FALLBACK on the request path.
- /api/generate-classroom resolves the owner the job works for now at each
stage, so stages after a claim follow the moved settings.
Streaming calls (agent driver, classroom chat, PBL streams) still do not
retry on a fallback model, as before this change; that is the #1725 item
on retries at every call site.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve media and tool capabilities through their slots (#1737)
* feat(config): a provider-only reference names the provider's default model
Search and document providers mostly have no model to pick, so a slot
may name just the provider (`webSearch: tavily`); the target then has no
modelId and the adapter uses its default. Chat slots still need
providerId:modelId, checked in openmaic.yml and at resolution.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* feat(config): resolve media and tool capabilities through their slots
The capability routes and server paths of RFC #1701 (#1725, P1): text
to speech, speech recognition, images, video, web search and document
extraction resolve through their slots, like language models.
- lib/server/model-config/media.ts: the configured slot (deployment,
then workspace); else the provider a request names the old way
(deprecated); else the legacy defaults; else a loud error. A slot turned
off fails whatever the request names, and while the legacy
<CAP>_<VENDOR>_ENABLED=false switches are in effect a switched-off
provider stays off whoever assigns it. A workspace endpoint is checked
like a caller-supplied one and may not use a proxy.
- The legacy translation assigns the media roots the provider the server
picked when a request named none (first configured; web search by its
old priority; DEFAULT_IMAGE_PROVIDER for images), with the first pinned
model.
- Routes: /api/generate/{image,video,tts,voice}, /api/transcription,
/api/web-search, /api/parse-pdf and /api/extract-document. A TTS voice
applies only to the provider it was chosen for; voices register on the
tts slot's provider.
- Server paths: agent image and video generation, scene narration, the
voice catalog and registration, web search and fetch_url, material
extraction (the document slot's service, the asr slot for local
transcription), browserless classroom media, and the generation
capabilities reported by /api/health and the classroom capabilities
endpoint.
- A provider-only reference (`webSearch: tavily`) names the provider's
default model; chat slots still need providerId:modelId.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep slots authoritative on every media path
- Workspace providers reach media, search and document services only at
their preset's endpoints; a custom endpoint or proxy is deployment-only
(403 INVALID_URL).
- A configured or turned-off document slot decides the extraction service:
deprecated request fields may only pick a self-contained extractor, and
legacy operator credentials are no longer reached.
- Request paths resolve their own workspace without following a claim;
background extraction still does.
- A configured TTS slot keeps its own model; legacy pins apply only on the
deprecated and default paths.
- Local transcription and agent voice registration keep the connection's
network policy flags.
- The agent's web search forwards the whole resolved configuration.
- An unusable DEFAULT_IMAGE_PROVIDER leaves the image slot unassigned with a
notice instead of switching vendors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): public presets only for workspaces, plan default models
- A workspace preset whose default endpoint is on the server's own network
(self-hosted TTS, ASR, image) is deployment-only, and a workspace
provider's endpoint runs under the public-only policy.
- A provider-only reference to a token plan means the plan's own default
model for that capability.
- On the legacy default provider, the request's image, video and ASR model
still applies through its allowlist, and image/video without a model is
still MISSING_MODEL.
- An asr assignment a workspace may not use no longer blocks document
extraction, and a refused document service answers 403.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep legacy search and voice choices on their old paths
- Classroom search honours the provider and key a request names while the
webSearch slot is unassigned; only /api/web-search prefers the operator's
configured backend, as before.
- A TTS request's voice applies unless the request chose it for another
provider.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): legacy search model and capability discovery errors
- On the legacy default search provider, the request's search model still
applies through the server's pins.
- Capability discovery answers a refused credential (401) or a workspace
service it may not use (403) instead of 500.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): classroom submission refuses a workspace service with 403
A classroom submission whose materials would go to a document or speech
service the workspace may not use answers 403 INVALID_URL instead of 500, and
its material check resolves the request's own workspace without following a
claim.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): workspace image and video providers run public-only
A workspace-configured image or video provider now uses the strict public
transport for its requests and for the redirects its clip download follows,
whatever ALLOW_LOCAL_NETWORKS says; the operator's own providers and the
deprecated per-request path keep their policies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* Revert "fix(config): workspace image and video providers run public-only"
This reverts commit
|
||
|
wyucandClaude Opus 5.5
|
5312c2b4b4 |
feat(config): openmaic.yml, capability slots and server-side model configuration (RFC #1701) (#1765)
* ci: run CI for the provider-config integration branch
Development of the provider configuration RFC (#1701) lands on
integration/provider-config; build it on push and run CI for pull
requests into it, as with earlier integration branches.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* feat(config): capability slot registry and stage-to-slot mapping (#1726)
* feat(config): capability slot registry and stage-to-slot mapping
First P0 step of the provider configuration RFC (#1701, tracked in
#1725): define the capability slot forest and map every LLM stage key
to exactly one slot. Pure data with no callers yet, so there is no
behavior change.
Requirements are checked on the slot that declares them and are not
passed down to child slots, so agent.title can use a model without
tool calling even though agent requires it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): pin every stage destination and the capability roots
Review found that the station and root assertions derived their
expectations from the module under test: re-pointing a single-stage
station or dropping a capability root still passed. The tests now
compare against an independently written stage table and root list,
and pin agent.title as config-only. Also document that browserless
outlines still run on the generate-classroom model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): pin slotForStage and full slot lineages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): provider presets and the openmaic.yml schema (#1727)
* feat(config): provider presets and the openmaic.yml schema
Second P0 step of the provider configuration RFC (#1701, tracked in
#1725).
- lib/config/provider-presets.ts: one preset per built-in registry
entry, plus the token plans as multi-capability presets whose stage
recommendations become slot recommendations. Registry ids that
collide across capabilities get explicit preset ids.
- lib/server/model-config/openmaic-yml.ts: parse and validate the
operator's openmaic.yml (or the file named by OPENMAIC_CONFIG):
${VAR} interpolation, strict schema, and cross-checks that every
assignment names a declared provider whose preset offers the slot's
capability. Every problem is reported with its path.
- instrumentation.ts: an invalid file refuses to start. Without a file
nothing changes, and nothing resolves models through the file yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): harden openmaic.yml validation after review
- Look providers up by own key only, so "constructor:m" is not taken
for a declared provider.
- Refuse non-mapping objects YAML produces (an unquoted timestamp
becomes a Date that the schema would accept as an empty object), and
refuse a YAML alias that refers back to itself instead of overflowing
the stack.
- Accept lowercase variable names; refuse a "${" with no closing
brace, checked on the text as written, never on substituted secrets.
- Cross-check the entries that are valid on their own even when the
schema rejects others, so every problem is reported at once without
duplicate errors for declared-but-invalid providers.
- Name the three valid assignment shapes when a value has none of them.
- Test the boot path: an invalid file exits with code 1 before any
schedule starts; a valid one boots.
- Ignore a root openmaic.yml in git.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep secrets out of openmaic.yml diagnostics
Review round 2:
- Read only variables the environment itself has, as non-empty
strings: ${constructor} or an inherited value is not a variable.
- Never print a value that came from ${VAR}, and report YAML syntax
errors by reason and position without js-yaml's source excerpt,
which can quote a literal key.
- Keep the base URL requirement on presets: SearXNG from its registry
entry, and Azure OpenAI and self-hosted MinerU, which have no usable
default endpoint.
- Check slot names and valid model references even inside an entry
that fails the schema, and report a failed placeholder once rather
than again as an empty value.
- Boot test for the no-file case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* refactor(config): validate openmaic.yml in two phases
Rounds 1-3 of review kept finding edge cases in one feature: running
the cross-checks on the half-valid parts of a file the schema had
rejected, so that every problem showed at once. It hid problems inside
a rejected entry, lost a __proto__ slot key, and built misleading
provider errors from failed placeholders. That feature is gone.
Validation now runs in two phases. Phase 1 is the document itself:
placeholders, key names (checked on the document as written, since the
schema drops a __proto__ key), and the schema. Any problem there stops
before phase 2, the references between entries, which therefore only
ever sees a fully valid file. Each phase reports all of its problems,
one per path.
YAML syntax errors now report only their position: js-yaml's reason
can quote source text too (an alias or tag name), not just its excerpt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(api)!: generate-classroom takes requirement + uploaded materials; capabilities follow server config (#1728)
Narrows POST /api/generate-classroom to { requirement, materialIds? }; capabilities follow server configuration; materials go through the owner material library (upload, reference by id, delete); new GET /api/generate-classroom/capabilities; skill docs updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve a slot through the configuration layers (#1729)
* feat(config): resolve a slot through the configuration layers
Third P0 step of the provider configuration RFC (#1701, tracked in
#1725). resolveSlot walks from a slot to its capability root and
returns the first assignment it meets, consulting the deployment layer
(openmaic.yml, locked) before the workspace layer at every node. An
explicit null disables the subtree; nothing assigned up to the root is
unassigned, with no fallback to any vendor. The result carries the
provider, preset, registry entry, effective base URL, key, model, call
options, fallback, where it was resolved and whether it is locked, and
the requested slot's own requirements checked against the model
catalogue (met, unmet or unknown).
Pure and uncalled for now, so there is no behavior change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): lock only written slots and trust the catalogue only where it applies
Review round 1:
- locked now means the requested slot itself is written in the
deployment layer. Inheriting a deployment value does not lock a slot,
since the workspace may still assign it (source and resolvedAt still
report where the value came from).
- A custom OpenAI-compatible endpoint borrows the OpenAI registry for
transport only, so its preset no longer trusts that model catalogue:
requirements there resolve to unknown.
- The fallback is checked against the slot's requirements too, and the
result exposes fallbackRequirements so retries can refuse it.
- A malformed reference fails with a path-qualified SlotResolutionError
that does not echo the value, which is often a misplaced key.
- Requirement tests use named catalogue models instead of picking one
from the catalogue at run time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): order layers deployment-first and keep references out of errors
Review round 2:
- resolveSlot no longer depends on the order its caller passes the
layers in: deployment layers always come first, for assignments and
provider lookup alike.
- Resolution errors name the path and the preset, never a value taken
from the reference, so a key pasted into the provider position does
not end up in a log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(config): cover catalogue aliases and fallback capability errors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(classroom)!: save server-generated classrooms through server persistence (#1730)
Server-side classroom generation saves the finished course create-only into the request owner's library with media in the owner's asset pool; jobs move to PostgreSQL with owner-scoped polling; /api/classroom is removed; legacy data/classrooms files are imported once in the background. Adds @openmaic/storage 0.36.0 PgAssetStore.releasePending.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): translate the legacy configuration into a deployment layer (#1731)
* feat(config): translate the legacy configuration into a deployment layer
Deployments without openmaic.yml keep working: provider variables,
server-providers.yml, DEFAULT_MODEL, MODEL_ROUTES and MODEL_FALLBACK are
translated into the openmaic.yml shape and serve as the deployment layer
for resolveSlot.
- Providers: each configured entry becomes a provider under its preset
id; force-disabled entries are left out; AliDocMind's key pair goes
into credentials.
- DEFAULT_MODEL goes on the llm root. Every other slot that serves stages
gets what those stages used before (their route, or else the default
model), written only where inheritance would give something else, so
an unrouted stage under a routed parent stays on the default model.
- Stages that now share a slot but had different routes, models whose
provider has no server configuration, and stages that used the
browser's model but would now inherit a server one become startup
notices, never failures.
- MODEL_FALLBACK attaches to every chat assignment without its own.
- When openmaic.yml exists it wins, with a notice if legacy variables
are also set.
Nothing reads the layer yet; routes switch over in P1 (#1725).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): follow the agent driver's rules and keep notices value-free
Review round 1:
- The agent never used DEFAULT_MODEL: it needs a valid maic-agent-driver
route and is off otherwise, so the translation writes agent: null where
it would inherit a server model (and a notice for a route that does
not work today). Unrouted conversation titles reuse the driver's model
with thinking off, as the title generator does.
- Notices name registry ids only; a credential pasted into a model
variable is never repeated.
- Provider entries and route options are checked against the new schema
and left out with a notice (field names only) instead of producing a
file that would not parse; driver-only options are dropped elsewhere.
- A route with its own fallback never falls back to MODEL_FALLBACK, even
when that fallback cannot carry over; the retry model is part of the
comparison with the inherited value.
- Force-off switches without a configured entry are reported, a legacy
configuration made only of them is detected, and translating one adds
a deprecation notice.
- loadDeploymentLayer reads the process environment and working
directory like the legacy loaders, instead of taking parameters it
could only partly honor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): compare routes by behavior, validate references, name only known ids
Review round 2:
- Stages sharing a slot are compared by what the route changes for them
(model, thinking, fallback), with bare ids read as openai models and a
route to DEFAULT_MODEL counted as no route; api and contextWindow are
inert outside the driver.
- A model reference carries over only when it names a provider from the
providers section and forms a valid reference.
- Section keys and force-off ids are named in notices only when they are
registry ids with a preset.
- Retries now follow the slot; where a call site picked its retry model
by another label (scene types under scene-content, browserless
generation), the change is reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): chat references need a chat entry; one notice for retries
Review round 3:
- A model reference carries over only when its provider was translated
from the providers section, not merely declared by another section
under the same id.
- Routes are compared by the retry model that takes effect, so leaving
out a fallback equals repeating MODEL_FALLBACK.
- The per-call-site retry notices kept missing cases (streaming and
calls outside server-managed routing never retry today). They are
replaced by one notice, given whenever a retry model is configured,
that retries now follow the slot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(config)!: translate only providers and the default model (#1732)
* refactor(config)!: translate only providers and the default model
MODEL_ROUTES does not map one to one onto slots: several stages share a
slot, the agent driver and conversation titles have rules of their own,
and retries are picked by call-site labels. Emulating that took most of
the translation and still ended in notices that are easy to miss.
The translation now carries over only the unambiguous part: providers,
DEFAULT_MODEL as the llm root and MODEL_FALLBACK as its fallback (the
agent stays null, as it never used DEFAULT_MODEL). A deployment that
sets MODEL_ROUTES without openmaic.yml gets LegacyRoutesError asking it
to write the per-stage models as slots. loadDeploymentLayer is not wired
into startup yet; that happens when routes switch to resolveSlot (P1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): say what a dropped MODEL_FALLBACK loses
Without DEFAULT_MODEL there is no assignment to hold MODEL_FALLBACK, and
calls that retried on it (server providers picked in the browser) stop
retrying; the notice says so. thinkingSchema is module-private again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(persistence): workspace model configuration with keys encrypted at rest (#1733)
* feat(persistence): workspace model configuration with keys encrypted at rest
The store behind the web settings of RFC #1701 (#1725, P1): one row per
workspace (owner) in workspace_model_config, the same shape as
openmaic.yml without policy.
- Provider secrets (apiKey, credentials) are sealed per provider with
AES-256-GCM under OPENMAIC_SECRET_KEY, bound to the provider id, and
never stored in the config column. Without the variable, a secret is
created once in data/instance-secret.key (the Docker volume).
- A secret sealed under another instance secret is reported as
unreadable ("enter again"), and a save that brings no new secret for
that provider keeps it, so a misconfigured secret cannot destroy keys.
- Saves replace the document with a compare-and-swap on the revision,
after the owner's identity lock; two concurrent first saves cannot
both land (PostgreSQL contract test).
- A claim moves the anonymous configuration to the account unless the
account has its own (core participant, order 900).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(persistence): publish the instance secret atomically; force the races in tests
Review round 1:
- The generated secret is written and flushed under a private name,
then published with an exclusive link, so no process can read it half
written and exactly one of two starting processes creates it. A file
that is not a complete generated secret (empty, truncated) is refused
instead of deriving a key anyone could compute.
- The PostgreSQL test holds both first saves at their insert until both
have read the missing row, and holds the row lock for the update case
until both saves queue behind it (checked by backend pid, not by any
lock waiter in the database).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(persistence): verify the written secret; require full GCM tags
Review round 2:
- The generated secret is written in full, read back and checked before
it is published; the temporary file is removed on any failure.
- Sealed values open only with a 12-byte IV and a full 16-byte tag
(authTagLength), so a shortened tag cannot weaken authentication.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve slots at request time over deployment, workspace and defaults (#1734)
* feat(config): resolve slots at request time over deployment, workspace and defaults
The runtime half of RFC #1701 resolution (#1725, P1):
- deploymentConfig(): the deployment layer, loaded once per process.
The legacy translation now splits into the deployment's providers and
a separate default layer (DEFAULT_MODEL, MODEL_FALLBACK) that locks
nothing and ranks below the workspace, as DEFAULT_MODEL ranked below
the model a user picked.
- workspaceLayer(owner) reads the web settings; requestWorkspaceId(req)
names the request's owner (none for an owner minted by the request).
- lookupSlot walks deployment and workspace over the whole tree first;
the defaults are a second walk, so a default on a child never outranks
a workspace choice higher up.
- resolveStageModel builds the language model: the configured slot, else
what the request still names the old way (deprecated), else the
defaults, else a loud error; a slot turned off fails whatever the
request names. A workspace provider's endpoint is checked like a
caller-supplied one and gets the transport that refuses redirects.
- resolveSlot gains the default source, and reports which layer declared
the provider, its proxy and credentials.
Nothing calls it yet; call sites switch next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep request identity and workspace endpoints inside their bounds
Review round 1:
- A refused owner credential throws InvalidOwnerCredentialError (401)
instead of resolving with the deployment's models and keys.
- A retired request owner gets no workspace: canonicalizing it would
hand an old anonymous cookie the claiming account's settings. Only
background work, which names a stored owner, is forwarded.
- A workspace provider may not be Amazon Bedrock (which falls back to
the server's AWS credential chain) or set a proxy (which routes around
the checked, redirect-refusing transport); the deployment still may.
- Test seams for the deployment config and the workspace loader, and
tests for identity, forwarding and caching.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): read only the named owner's settings; refuse unmet requirements
Review round 2:
- workspaceLayer reads exactly the owner it is given. Forwarding through
a claim let a request whose owner was claimed between its check and the
read see the account's settings; background work passes the owner it
works for now instead.
- A model the catalogue says does not meet the slot's requirement is
refused before anything is built (SlotRequirementError), without
consulting the request.
- Tests: a database failure fails the call without falling back, a
workspace provider without a key never gets the deployment's, and the
transport each provider source gets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* test(identity): exempt the model settings lookup from cookie forwarding
requestWorkspaceId resolves the owner only to pick whose model settings a
generation call uses, and returns no response of its own, like the
vision prompt helper already listed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config)!: resolve every LLM call through its capability slot (#1735)
* feat(config)!: resolve every LLM call through its capability slot
The LLM call sites of RFC #1701 switch to slot resolution (#1725, P1):
- resolveModel resolves a stage through its slot for the request's or
job's workspace: the deployment and workspace configuration first; the
model and key a request names (x-model, x-model-routes, body fields)
only for a slot left unassigned, deprecated; then the defaults an
older deployment set with DEFAULT_MODEL. Routes that read headers get
this through resolveModelFromRequest; the chat routes pass their
workspace; background work (agent runs, titles, generation jobs)
passes the owner it works for now.
- Retries follow the slot: a slot-resolved model carries its slot's
fallback (lib/ai/model-fallbacks.ts), which callLLM and the outline
stream use; only a model from the request path still retries on
MODEL_FALLBACK. A fallback that cannot meet the slot is not used.
- The agent driver resolves the agent slot: tool calling required,
api defaults to openai-completions, thinking.effort still refused.
Conversation titles resolve agent.title with thinking off unless the
title slot sets it.
- /api/generate-classroom resolves each step through its slot for the
job's owner, its outline through course.outline.
- MODEL_ROUTES is gone: loadDeploymentLayer runs at startup and a server
that still sets it without openmaic.yml refuses to start. The
pbl-chat and maic-agent stage keys, which nothing resolved, are removed.
BREAKING CHANGE: MODEL_ROUTES is no longer read; write per-stage models
as slots in openmaic.yml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): a slot fallback needs no serverManaged stamp; follow claims per stage
Review round 1:
- callLLM arms a model's attached slot fallback whether or not the caller
passes serverManaged (the PBL agents pass none); the stamp still gates
MODEL_FALLBACK on the request path.
- /api/generate-classroom resolves the owner the job works for now at each
stage, so stages after a claim follow the moved settings.
Streaming calls (agent driver, classroom chat, PBL streams) still do not
retry on a fallback model, as before this change; that is the #1725 item
on retries at every call site.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(config): resolve media and tool capabilities through their slots (#1737)
* feat(config): a provider-only reference names the provider's default model
Search and document providers mostly have no model to pick, so a slot
may name just the provider (`webSearch: tavily`); the target then has no
modelId and the adapter uses its default. Chat slots still need
providerId:modelId, checked in openmaic.yml and at resolution.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* feat(config): resolve media and tool capabilities through their slots
The capability routes and server paths of RFC #1701 (#1725, P1): text
to speech, speech recognition, images, video, web search and document
extraction resolve through their slots, like language models.
- lib/server/model-config/media.ts: the configured slot (deployment,
then workspace); else the provider a request names the old way
(deprecated); else the legacy defaults; else a loud error. A slot turned
off fails whatever the request names, and while the legacy
<CAP>_<VENDOR>_ENABLED=false switches are in effect a switched-off
provider stays off whoever assigns it. A workspace endpoint is checked
like a caller-supplied one and may not use a proxy.
- The legacy translation assigns the media roots the provider the server
picked when a request named none (first configured; web search by its
old priority; DEFAULT_IMAGE_PROVIDER for images), with the first pinned
model.
- Routes: /api/generate/{image,video,tts,voice}, /api/transcription,
/api/web-search, /api/parse-pdf and /api/extract-document. A TTS voice
applies only to the provider it was chosen for; voices register on the
tts slot's provider.
- Server paths: agent image and video generation, scene narration, the
voice catalog and registration, web search and fetch_url, material
extraction (the document slot's service, the asr slot for local
transcription), browserless classroom media, and the generation
capabilities reported by /api/health and the classroom capabilities
endpoint.
- A provider-only reference (`webSearch: tavily`) names the provider's
default model; chat slots still need providerId:modelId.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep slots authoritative on every media path
- Workspace providers reach media, search and document services only at
their preset's endpoints; a custom endpoint or proxy is deployment-only
(403 INVALID_URL).
- A configured or turned-off document slot decides the extraction service:
deprecated request fields may only pick a self-contained extractor, and
legacy operator credentials are no longer reached.
- Request paths resolve their own workspace without following a claim;
background extraction still does.
- A configured TTS slot keeps its own model; legacy pins apply only on the
deprecated and default paths.
- Local transcription and agent voice registration keep the connection's
network policy flags.
- The agent's web search forwards the whole resolved configuration.
- An unusable DEFAULT_IMAGE_PROVIDER leaves the image slot unassigned with a
notice instead of switching vendors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): public presets only for workspaces, plan default models
- A workspace preset whose default endpoint is on the server's own network
(self-hosted TTS, ASR, image) is deployment-only, and a workspace
provider's endpoint runs under the public-only policy.
- A provider-only reference to a token plan means the plan's own default
model for that capability.
- On the legacy default provider, the request's image, video and ASR model
still applies through its allowlist, and image/video without a model is
still MISSING_MODEL.
- An asr assignment a workspace may not use no longer blocks document
extraction, and a refused document service answers 403.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): keep legacy search and voice choices on their old paths
- Classroom search honours the provider and key a request names while the
webSearch slot is unassigned; only /api/web-search prefers the operator's
configured backend, as before.
- A TTS request's voice applies unless the request chose it for another
provider.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): legacy search model and capability discovery errors
- On the legacy default search provider, the request's search model still
applies through the server's pins.
- Capability discovery answers a refused credential (401) or a workspace
service it may not use (403) instead of 500.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): classroom submission refuses a workspace service with 403
A classroom submission whose materials would go to a document or speech
service the workspace may not use answers 403 INVALID_URL instead of 500, and
its material check resolves the request's own workspace without following a
claim.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* fix(config): workspace image and video providers run public-only
A workspace-configured image or video provider now uses the strict public
transport for its requests and for the redirects its clip download follows,
whatever ALLOW_LOCAL_NETWORKS says; the operator's own providers and the
deprecated per-request path keep their policies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR
* Revert "fix(config): workspace image and video providers run public-only"
This reverts commit
|
||
|
wyucandClaude Opus 5.5
|
4320f986cf |
ci: list the live integration branches in the CI triggers (#1724)
* ci: stop triggering on the landed persistence integration branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * ci: list the live integration branches in the CI triggers Add integration/provider-config and integration/material-library to the push and pull_request branch lists, and drop the feat/maic-editor-v0 and feat/maic-editor-v1 entries, which no longer exist on the remote. Branches cut from main now inherit triggers for every open integration branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: leave integration/provider-config to its own branch It merges into main next, so main need not trigger on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EF6G824rLqUXidRrqrMskR --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fa47efe1b3 |
feat(persistence)!: server-backed persistence only, with a one-way legacy browser import (#1710)
* ci: run CI for the persistence-default integration branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(docker): start PostgreSQL by default and add a single-user owner * feat(docker): start PostgreSQL by default and add a single-user owner `docker compose up` now runs a server-backed app against a bundled PostgreSQL, started only once PostgreSQL is healthy and published on 127.0.0.1 only, with a new built-in singleUser owner auth method: every request resolves to one fixed owner that may publish, and no anonymous cookie is minted. Single-user mode (OWNER_SINGLE_USER=true, OWNER_SINGLE_USER_ID default "local") runs with or without ACCESS_CODE. Without one, the server logs one prominent startup warning that anyone who can reach it shares, edits and can delete the single library; it never inspects request peers or forwarding headers. It excludes PERSISTENCE_SHARED_OWNER_ID and follows the sharedTeam registration rules. Its principal gets a claim candidate, so a browser's earlier anonymous work is claimed into the single owner (OWNER_CLAIM_TRIGGER=auto in the Compose defaults). Compose defaults live in docker-compose.defaults.env, read before .env.local so they can be overridden. The app warns when its declared published address (OPENMAIC_PUBLISH_ADDRESS, passed by Compose) is not loopback while PostgreSQL uses the default password. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(docker): keep claims explicit and let .env.local set DATABASE_URL - Compose no longer defaults OWNER_CLAIM_TRIGGER to auto: an irreversible merge of every browser's anonymous library into the single owner must not happen on upgrade. The single-user principal still gets a claim candidate, so POST /api/identity/claim works; the docs explain it and warn about auto on a deployment several people used. - The bundled DATABASE_URL moves to docker-compose.defaults.env, read before .env.local, so an external database or a rotated password set there keeps working. The password is inserted unencoded, so it must be letters and digits (documented). - Upgrade docs no longer claim browser-stored courses are copied as they are opened: they stay in the browser, are not deleted, and are moved by the automatic browser-to-server migration shipped with this work. - Single-user mode without ACCESS_CODE now logs one warning instead of two; the docs note that the single owner id is permanent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence)!: server-backed persistence is the only persistence (#1707) * feat(persistence)!: make server-backed persistence the only persistence Courses, folders and folder membership, chat history and learner runtime, and generated media are now always stored on the server through the embedded /api/persistence endpoint. The browser keeps no durable data of its own. - Remove the build-time NEXT_PUBLIC_PERSISTENCE switch: the client bootstrap always configures the HTTP document, runtime and asset seams, and the Dockerfile, docker-compose.yml, the Pi route and the whiteboard runtime no longer read it. Unconfigured seams refuse to resolve instead of falling back to IndexedDB, and the learner key is always the server-derived one. - Remove the browser write paths: the browser document, runtime and asset stores as backends, the browser-mode branches of stage, folder, chat, media, narration and import storage, the Dexie backup export/import and the whole-database clear. The library lists through /api/stages, folders through /api/folders, and membership now goes through /api/folders/members. Regeneration always forks to a fresh asset. - Device-local state moves to its own IndexedDB database (lib/device-storage, maic-device-cache): the local media and narration cache and refused-bytes retention, staged PDF images, undo history, browser voice profiles (carried over once from the old database) and the auto-voice cache. "Clear Local Cache" clears exactly that and no longer claims to delete classrooms or chat history. - Keep the pre-server data readable for the one-way importer: the Dexie schema and read-only accessors for its tables and for the browser document, runtime and asset databases live in lib/legacy-browser-storage, which nothing on the regular paths reads and which has no write path. - Require a database: the server exits at boot without DATABASE_URL, with a message naming `pnpm db:up` and `docker compose up`. `pnpm db:up` / `pnpm db:down` start and stop the Compose postgres service alone, published on 127.0.0.1 through docker-compose.db.yml; .env.example carries the matching local DATABASE_URL. - E2E specs seed their courses through the persistence endpoint, each under a fresh id, instead of writing IndexedDB. - Guard tests pin the boot requirement, the absence of any NEXT_PUBLIC_PERSISTENCE read, and the legacy-storage boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the E2E suite against PostgreSQL The app refuses to start without DATABASE_URL, so the E2E job gets a postgres:16 service and a job-level DATABASE_URL. The unit, storage and render-service jobs are unchanged and need no database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: document always-on server persistence and the database requirement README (English and Chinese), the deployment guide in every locale, the extension cookbook and the changelog now describe PostgreSQL as required: `pnpm db:up` for local development, `docker compose up` for a deployment, and an external database for Vercel and other serverless hosts (the Deploy button prompts for DATABASE_URL). They list what stays in the browser, drop the NEXT_PUBLIC_PERSISTENCE instructions, and say that courses an earlier browser-only build stored stay in the browser until the one-way importer moves them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pbl): use the server-derived learner key for PBL v2 runtime PBL v2 synchronization, hydration and drain resolved the learner key from their default watermark KV store, which reads or mints a device key in localStorage instead of asking runtime configuration. The runtime store is the server one, which refuses any key but the owner's, so saving a course with a PBL v2 scene failed before the document write, and the minted key landed in the slot the one-way importer reads as the pre-server runtime partition. The learner key now comes from getLearnerKey(args.kv): the configured server-derived key, or an explicitly injected store. The default KV store still holds the drain watermarks and nothing else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): pin that clearing the cache keeps legacy databases - Seed the four pre-server databases (MAIC-Database, maic-documents, maic-runtime, maic-asset-pool) with real rows, run Clear Local Cache, and assert every database and row survives; assert the device database name differs from every legacy name. - The legacy-module import rule now also catches relative static, side-effect, dynamic and require imports at any depth. - New rule: the runtime learner key is never read from a KV store the caller made up, only from configuration or an injected store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(dev-db): run the development database as its own Compose project `pnpm db:up` / `db:down` shared the checkout's default Compose project with `docker compose up`, so they recreated or stopped a running stack's database. docker-compose.db.yml now declares its own project (`openmaic-dev-db`), which gives the development database its own container and data volume; it no longer touches the stack, and the two do not share data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: start the development database before `pnpm dev` CONTRIBUTING, the getting-started guide in every locale and the startup modes reference now run `pnpm db:up` and set DATABASE_URL before `pnpm dev`. README, the deployment guides, the cookbook and the changelog describe `pnpm db:up` as a separate development database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): sharpen the legacy-storage and learner-key guards - The Clear Local Cache test closes its seeding connections to maic-documents and maic-runtime, so a regression deleting either fails on the assertion that names the database instead of by timeout. - The legacy-module and Dexie import rules accept comments inside import() / require(), so a webpackIgnore hint no longer hides one. - The learner-key rule also flags an injected-looking name that is bound to a default local store in the same file; its remaining limit is documented, with the behavioural PBL test as the guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(dev-db): pin the development database's Compose project `pnpm db:up` / `db:down` now pass `-p openmaic-dev-db`, which takes precedence over COMPOSE_PROJECT_NAME from the shell or a .env file; the file's `name:` alone could be overridden and point the scripts at a running stack's database. The compose test pins the flag. The docs (all locales) and the file header say the development database is shared by every checkout on the machine, and how to run a separate one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): import legacy browser data into the server, once (#1708) * refactor(persistence): prepare the seams the legacy browser importer uses - Legacy module: read-only accessors for the auto-voice cache, the course ids the old media and narration tables name, the pre-server learner key, and asset bytes from the browser asset pool. - Narration adoption exports its row ownership rule so the importer applies the same one to rows of the old database. - The quiz legacy migration is split into a non-deleting core (importLegacyQuizSnapshot) and the regular path, which still deletes the keys it migrated. - The lazy document migration exports its snapshot canonicalizer. - Clear Local Cache keeps the importer's per-owner ledgers, as it keeps the pre-server learner key. - A library-changed window event lets an open library list courses that arrive in the background. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(persistence): import legacy browser data into the server, once On the first load after the upgrade, the client moves what earlier builds kept in this browser to the server, for the owner the server resolves: courses from the browser document store and the original tables (with chat, learner runtime, playback position, roster, folders and membership, pre-runtime quiz state), their media bytes through commitToPool and the existing write-back funnels, the bytes of server courses from earlier opt-in server builds that only the old tables hold, and device-only rows (failure records, refused bytes, auto-voice clips) into the device cache. It is automatic and silent, runs after the page is idle, never writes to a legacy store, and records every step in a per-owner ledger so it resumes after a crash and never imports twice. Runs are serialized across tabs with Web Locks. A course the owner already has stays authoritative; one whose id another owner holds is imported under a derived fresh id; one the owner deleted on the server stays deleted. Transient failures retry on a later load with backoff; permanent ones are recorded per item. The module is temporary and self-contained; its README lists the removal steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): pin that each owner keeps its own import ledger Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): run the legacy import against the app routes and a browser A PostgreSQL suite routes every request the importer and the app seams send to the real persistence, library and folder handlers, owner resolved from the anonymous cookie: a full import, a course another owner holds, a course the owner deleted, and an existing course whose media is filled in. An end-to-end spec seeds an old pre-server database in Chromium, loads the app, and checks that the course reaches the library with its narration uploaded, stays in the browser untouched, and is visible from a fresh browser context with the same owner cookie. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: describe the one-way import of legacy browser data The README (and its Chinese version), the deployment guide in every locale and the changelog now say what the importer does instead of announcing it: courses an earlier browser-only build kept in the browser move to the server automatically on the first load after the upgrade, and the browser copy stays untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): probe server assets through the shared asset-URL owner The importer asked the pool directly whether an id exists, which the asset-URL ownership boundary forbids outside use-asset-url. It now uses assetRefExists, the existing metadata-only probe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): guard the importer's pool probe like every other entry point Only a reference the pool could have issued is probed, as the lease guard requires; the importer is added to the list of guarded entry points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): import legacy browser data once per browser Review round 1 of the legacy browser importer. - Once per browser, not once per owner: the data belongs to whoever used the browser before the upgrade, so the first owner the import runs for claims it and any other owner gets nothing. The ledger is one key, maic:legacy-import:v2, recording the owner only as a SHA-256 digest (an anonymous owner id is a bearer credential) and a random salt. - Handoff: when the claiming owner is retired (403 OWNER_RETIRED), or the current owner holds courses or folders the importer created (a claim moved them), the current owner continues the unfinished items; courses already done are never imported again, so a claim neither duplicates nor resurrects them, and a half-copied course keeps its runtime. - Fresh ids come from the salt and the legacy id through SHA-256, not from the owner: unlinkable, unpredictable, and the same in every tab. - 401 pauses with backoff instead of closing the import; 403 FORBIDDEN_LEARNER stops the run and leaves the item pending. - Without Web Locks: ledger writes merge with the stored copy, and the library is listed again right before a course is placed, so a second tab does not import a course the first one just created. - Media the server refuses for good gets the app's failed-media record instead of a dangling reference; the legacy bytes stay. - A legacy whiteboard or PBL session is not created when the server already has an active one of that kind. - A folder that is gone leaves the course unfiled instead of failed, and a membership naming a folder the old database lacks counts as unfiled. - Narration rows from before the course column are adopted when exactly one legacy course names their key; unconvertible chat rows are noted; an unreadable document-store copy falls back to the original tables; PBL v2 documents go through the save-path strip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover the importer's handoff, tabs, refusals and review gaps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: say the legacy import runs once per browser The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say that the first owner to load the upgraded app in a browser claims its legacy data, what the handoff to an account does, that the ledger holds no owner id, and how refused media and the new pause and skip cases behave. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): note runtime the importer cannot find without the old learner key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(identity): let an owner confirm it absorbed a given owner through a claim GET /api/identity/merged-from?salt=<hex>&digest=<hex> answers whether a claim merged into the requesting owner an owner whose id hashes to SHA-256(salt, id). It reads only the requester's own owner_merges rows, never lists or names an owner, resolves the owner like every other owner-scoped route, and is uncacheable. The one-way import of pre-server browser data uses it to decide whether a different owner may continue an import the first owner started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): hand the legacy import to another owner only on a confirmed claim Review round 2 of the legacy browser importer. - The only way the browser's legacy data moves from the owner that claimed it to a different owner is the server confirming, through GET /api/identity/merged-from, that the current owner absorbed that owner through a claim. The inferred signals (an owner that never wrote, a library listing that holds imported courses) and the retired flag are gone; a 403 OWNER_RETIRED only stops the run and leaves items pending. - The first owner is bound only once the run's first authenticated request of its own (the library listing) succeeded, so a run that dies before that leaves the ledger unclaimed. - The owner is recorded as SHA-256 of the per-browser salt and the owner id; the server computes the same value from the salt the browser sends. - A "not absorbed" answer is reused for an hour instead of asking on every load, and the claiming owner stored first survives another tab's save unless that tab handed the import over; completion survives too. - Run-level failures (401, OWNER_RETIRED, FORBIDDEN_LEARNER, OWNER_BUSY) from any call stop the run and leave the course pending, including the "is the id taken" read and the library re-list, which could previously mark the course failed and close the import. - Refused media without a generation request (the user's own) gets a new non-retryable ASSET_REFUSED code, so it shows as failed without a Retry that could only fail; a skipped singleton runtime session leaves a note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): pin the importer ledger's owner merge and the singleton-skip note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: the legacy import moves to another owner only on a confirmed claim The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say that the import runs once per browser and moves to another owner only when that owner claimed the original one, confirmed by GET /api/identity/merged-from; that the owner is recorded as a salted digest; and how refused user media shows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): never let a transient failure settle a legacy import item Review round 3 of the legacy browser importer. - A read of the old browser stores that fails in storage (an aborted IndexedDB transaction, a closed database) now pauses the run and leaves the course pending. Only a record that cannot be migrated, parsed or validated is skipped. This covers the course read (which no longer falls back to the older table copy on a storage failure), and the quiz-scene and speech indexes, which used to read a failure as "no scenes" or "no holder" and drop quiz state or narration for good. - After a session-id collision, a failed read of the session propagates and is retried; only a successful read that shows the id taken skips it. - Without Web Locks, two tabs of different owners can both find the ledger unbound. Every ledger write keeps the owner stored first, and the run now re-reads the stored ledger after binding, before folders, before each course and before each course document is created, and stops if another owner holds it. - The "not yours" answer is reused for ten minutes instead of an hour, a stored wait further ahead than the longest the importer sets (a clock that ran ahead) counts as passed, and expired answers are pruned. - Generated media refused without an error code keeps its Retry; only media nothing can regenerate gets ASSET_REFUSED. A poster that fails transiently leaves the video pending instead of being dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover transient reads, racing owners, the not-yours cache and the claim client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(persistence): say what the ledger's salt protects, and the new pauses The salt defeats precomputed and cross-browser tables, not a targeted guess against one browser's ledger; the module README and the digest's comment now say so. The README also covers storage read failures, which pause instead of skipping, the two-owner race without Web Locks, and the ten-minute recheck. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(identity): bind a browser's legacy data to one owner on the server The one-way import of pre-server browser data now asks the server whose data it is, instead of deciding in the browser. - legacy_import_bindings (browser_id primary key, owner_id, timestamps), provisioned with owner_merges on fresh and upgraded databases. - POST /api/identity/legacy-import-binding {browserId}: one atomic insert (ON CONFLICT DO NOTHING) and a read-back; answers only whether the requesting owner holds the browser. Same-origin JSON, resolved and write-fenced like every owner write, 400 for a malformed id. - A claim participant re-keys the claimed owner's bindings to the account inside the claim transaction, so the account simply holds the browser. - Owner resolution refuses any request carrying X-OpenMAIC-Legacy-Import with 409 LEGACY_IMPORT_NOT_BOUND unless the owner it resolves to holds that browser: one central fence every owner-scoped route goes through. Requests without the header are unaffected. - GET /api/identity/merged-from is removed; the binding replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): import through fenced clients bound by the server Review round 4 of the legacy browser importer. - The ledger holds a random browser id and no owner information; the owner digest, the "not yours" cache and the client-side handoff are gone. A run first asks the server to bind the browser and imports only when the requesting owner holds it. - Every importer request goes through its own clients that carry the browser id, so the server refuses any write under an owner that does not hold the browser: after a cookie switch in another tab, from a page whose memoized owner is stale, or from a tab that lost the binding race. The runtime learner key is asked for during each run, never taken from the page. 409 LEGACY_IMPORT_NOT_BOUND stops the run with items pending. - The write-back funnels, commitToPool and the PBL save-path strip take an optional store, put or runtime, so the importer passes its fenced ones. - A failed read of one legacy course holds up only what it could affect: the quiz-scene and speech indexes leave the course out as unindexed, and only quiz state or narration it could own waits. Read failures are counted per course; after five failing runs over at least a day, a course whose document-store copy cannot be read falls back to the table copy, or is skipped with the reason when there is none. - Runtime ids are rewritten by replacing only their course segment, so a short course id cannot corrupt the prefix or the learner segment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover the server binding, its fence, bounded reads and short ids Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: the server binds a browser's legacy data to its first owner The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say: once per browser; the server binds the browser to the first owner; a claim carries the binding to the account; the importer's requests are refused for any other owner. They also describe the bounded pause for a course the old storage cannot read, and the removal steps for the server side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): keep a dropped asset request pending in the legacy import The asset client reports a fetch that got no answer (a dropped request, an aborted upload, the existence probe's deadline) as status 0 with HTTP_REQUEST_FAILED, which the importer read as a final refusal: the media was marked failed and the course could complete without it. Read that code as transient, keep the client's local validation failures final, and treat a 2xx/3xx answer a client could not use as transient too. Unit tests drive every client the importer uses with a rejected fetch and a non-JSON answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): ask a refused binding again only every ten minutes A browser whose legacy data another owner holds sent the binding request (a write) on every page load. Record a ten-minute recheck in the ledger (clamped like every other deferral), so a claim that moves the binding is still picked up, and open the old databases only once the browser is bound. The read budget now counts consecutive failing runs (a run that reads the course resets it) and counts a first failure dated in the future from the next one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): answer 503 with the cookie when the import fence cannot read The legacy import fence now reads the header name from the shared constant, and a database error while reading the binding answers the usual JSON error (503 PERSISTENCE_UNAVAILABLE) with the resolution's Set-Cookie values instead of escaping as a bare 500. Route tests cover that, a bind by an owner a claim retired (403 OWNER_RETIRED, no row), and one header name on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover dropped asset requests, the recheck and the read budget With the real asset client's error for an upload and an existence probe, the media stays pending and a later run imports it. A non-holder's five loads send one binding request and never open the old databases, and a claim still hands the import to the account after the delay. The read budget's run count and time span each hold on their own, a clock that ran ahead does not hold a course open, and a good read resets the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(persistence): complete the legacy import removal steps List every file the removal touches (the claim scenarios test, the claim participant table and list, every doc), keep the bindings table for one release after the code goes because of rolling deploys and give the DROP statement for that release. Add the claim's eighth participant to the README lists, fix the fresh-id wording in the changelog, and describe the recheck delay, the transient status-0 asset failures and the consecutive read budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): keep legacy quiz state through Clear Local Cache until the import completes Pre-runtime quiz drafts, answers, results and attempt ids exist only in localStorage, and the one-way importer copies them to the server with their course. Clear Local Cache deleted them even while the import was still pending (it waits for an idle page and can stay pending after a failure), so that quiz progress was lost for good. Clear Local Cache now keeps the four quiz key families unless the importer's ledger records the import as complete, read through a new `legacyImportIsComplete` helper in the ledger module. The ledger key moves there too, so the two modules do not import each other. Once the import is complete, the keys are cleared as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: show DATABASE_URL set in the local development setup The getting-started guide showed the required DATABASE_URL line commented out, so copying the block left the server unable to start. Each locale now shows `pnpm db:up` and, separately, the `.env.local` line to set. The `.env.example` header said every variable is optional; it now says that provider variables are, and that DATABASE_URL is required except under docker-compose.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(changelog): drop two upgrade notes the final code contradicts The Compose entry said its default DATABASE_URL overrides .env.local; the defaults file is read first, so a value in .env.local wins, as the same entry says further on. The runtime-session entry offered turning server persistence off as an upgrade path; that switch no longer exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(env): restore the ACCESS_CODE warning notes in .env.example A later change to the example file dropped the lines saying that an unset ACCESS_CODE fails open, that the server then logs a one-time startup warning and GET /api/health reports accessCodeConfigured: false. They are back, and say that single-user mode logs its own warning in place of the generic one, as the startup code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * ci: run every app PostgreSQL suite in the contract job The app-domain step named its suites one by one, and seven had been left off, among them the legacy importer's route suite. No other job gives the app suites a database, so they skipped everywhere. The step now lists every app `*.pg.test.ts`; all of them work in a schema or database of their own, or truncate only their own tables, and pass before and after the storage package suite in the same database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): establish the anonymous owner before the first API request (#1721) * fix(identity): establish the anonymous owner on the page response On a browser's first load the page sent several API requests without an owner cookie; each minted its own anonymous owner and set its own cookie, and the last answer to arrive won. Work done in between (the legacy import binding, a first course write) then belonged to an owner the browser no longer presented. The middleware now mints the anonymous cookie on document navigations that carry no valid one, with the route handlers' own value format and attributes (the cookie module is Edge-safe now: Web Crypto instead of node:crypto), and forwards it on the request so the page render resolves the same owner. API, RSC, prefetch and Server Action requests never mint there, and a valid cookie is never replaced. It mints only when neither OWNER_SINGLE_USER nor PERSISTENCE_SHARED_OWNER_ID is set; host registrations are invisible to an Edge middleware, so the README tells hosts where to skip it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(legacy-import): bind only an owner the browser already presents A binding request whose owner was minted by that very request answers 409 OWNER_NOT_ESTABLISHED with the minted cookie and binds nothing: other cookieless requests may still be minting owners of their own, and a binding to this one could be left with an owner nobody presents. The importer treats it as transient and retries on a later load. It also says plainly when a request after its own successful bind is refused as another owner (the owner cookie changed mid-run); items stay pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(legacy-import): cover a first visit whose first answers arrive out of order Against the real routes and PostgreSQL: a browser without a cookie loads a page, sends two requests at once, and the first answer reaches it only after the importer bound the browser; the import completes under the one owner the page established. A second case binds nothing for an owner the bind itself minted and imports on the next run. The E2E drives the same ordering in Chromium by holding the first owner-scoped answer until the binding answered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(legacy-import): drop an unused initial assignment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(changelog): note the page response now sets the anonymous owner cookie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): let hosts turn page-response minting off The middleware cannot see owner auth methods a host registers, so a host with anonymousFallback: false still got an anonymous cookie on a cookieless page load. Beside the host credential it is a claim candidate; with OWNER_CLAIM_TRIGGER=auto it is claimed and cleared on the next request, again after every such page load. OWNER_ANONYMOUS_PREMINT (default on; true/1/false/0, anything else fails startup) is read by the middleware before minting. At startup the server warns once when a registration turns the anonymous fallback off while pre-minting is still on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(identity): document OWNER_ANONYMOUS_PREMINT Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): keep the anonymous identity for 400 days and renew it in use The anonymous cookie had a fixed 30-day lifetime from its mint and was never renewed, so every anonymous visitor lost their whole library 30 days after the first visit however active they were; with persistence on the server only, that affects every anonymous deployment. It now lasts 400 days (the browser cap, one shared constant), and every route handler and Server Action response that resolves to a valid anonymous cookie re-sends the same value with a fresh Max-Age (best-effort where next/headers refuses writes). Page responses never renew, so pages stay cacheable, and a host principal never renews the anonymous cookie beside it. A response that clears the cookie (a claim, a retired owner) must not renew it, whatever order a route merged its headers in: withRequestOwner, and the owner-events retired answer, keep only the clearing value for a cookie a value clears. A renewal that lands after a claim cleared the cookie is pinned by a test: the next write with it, alone or beside the account, writes nothing, and its answer clears it again without renewing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(identity): 400-day renewed anonymous cookie; when to turn pre-minting off Hosts set OWNER_ANONYMOUS_PREMINT=false only when their registration sets anonymousFallback: false; hosts that keep anonymous visitors need pre-minting and skip it per request in middleware for requests their methods authenticate. The CHANGELOG notes the new lifetime and renewal, and that a lost cookie means a new owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): forward owner cookies from routes that resolve the owner directly The Pi chat and whiteboard-visibility routes and asset-id document extraction resolved the owner themselves and never sent its Set-Cookie values back, so an anonymous identity used only through them was never renewed (and a minted one never stored). attachOwnerCookies attaches a resolution's cookies to a response a route built itself, with the clear-wins rule, before a stream's body starts. Both Pi routes answer through it, success and error alike; resolveServerAsset hands the cookies back with every answer and the extraction route attaches them. A guard test scans for every module outside the identity seam that calls the owner resolution (or the asset helper) directly and requires it to forward the cookies; the two callers that cannot are listed with the reason. Behaviour tests cover the three paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: wyuc <dhq1204@yahoo.com> |
||
|
wyucandClaude Opus 5.5
|
a686481e73 |
feat(auth): owner identity seam — composable auth methods, owner-keyed persistence, host hooks, claims
* ci: run CI for the owner identity seam integration branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 1 — pluggable authenticator with today's behavior * feat(auth): add the owner identity seam with built-in authenticators Introduce lib/server/identity/: an OwnerAuthenticator contract (SubjectKind, OwnerPrincipal, AuthOutcome), a single-shot, boot-validated configureOwnerAuthenticator() registry, and the anonymousCookie and sharedTeam built-ins that reproduce today's owner resolution exactly (same anonymous_id cookie, owner id strings, statuses and env semantics). Every owner-scoped route handler and the workspace Server Action now resolve their owner through one memoized per-request resolution instead of the three previous copies (agent-runtime/owner.ts, with-owner.ts and the Server Action re-implementation). Publish and unpublish decide from the principal's course:publish role instead of an anon: id prefix. An invalid credential from an authenticator is answered with 401 on every surface and never falls back to an anonymous owner. A source scan keeps the anonymous owner cookie and anon: prefix checks out of code outside the identity module. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): describe the owner identity seam and host registration Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): tighten the identity boundary guard and Server Action cookies - Scan workspace package sources too, flag imports of the concrete built-in authenticators outside lib/server/identity, and police the slice/substring/indexOf/includes spellings of an anon: id-shape check. Stop re-exporting the built-ins from the host-facing index. - Refuse setCookies returned by authenticateFromContext, as the authenticate() fallback already did, and document that it must write cookies itself through next/headers. - Let the route-test owner stub carry explicit kind/roles, and cover the 403 forbidden branch of publish and unpublish for a signed-in owner without course:publish. - README: registration conflicts fail the boot; an invalid principal is rejected per request with a 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner * feat(auth): owner identity seam, phase 2 — runtime and assets keyed by the owner Runtime sessions and assets on /api/persistence now take their identity from the same memoized owner resolution as documents, instead of a client-chosen x-learner-key behind a development bearer token that shipped in the public bundle. - Runtime: the storage handler's learner key is principal.ownerId. A path or body naming another learner answers 403 FORBIDDEN_LEARNER and another learner's session answers 404, whatever x-learner-key says; the header is no longer read. The browser learns its key from the new GET /api/persistence/learner-key and sends no credential of its own. The Pi whiteboard routes derive the same key from the owner. PERSISTENCE_DEV_TOKEN, NEXT_PUBLIC_PERSISTENCE_TOKEN and PERSISTENCE_ALLOW_INSECURE_DEV_AUTH are gone from the server path and the client, and server-auth.ts is deleted. Learner merge stays refused. - Assets: allocations land in a per-owner partition (owner:<ownerId>), so quota is per owner and replace/delete reach only the owner's entries. Reads stay capability-by-id where a course viewer needs them: another owner's committed entry is readable while a live course references it. Legacy entries in the old shared partition stay readable by id, and may be replaced or deleted only by an owner who owns every course referencing them. Server-generated media is stored under the run owner's partition; asset-id extraction and vision image resolution use the same rule. - Tombstones: runtime of a deleted course reads as absent (404, empty lists) and takes no new sessions or records, through a guard in the app's runtime composition; no package schema change. The identity boundary scan now also fails on any source that reads x-learner-key or the retired development token variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(persistence): describe owner-keyed runtime and assets, drop the dev token Remove the development persistence token from the server-persistence recipe, .env.example, the Docker build arguments and the deployment docs; describe runtime learner keys, per-owner asset partitions and quota, the legacy shared-partition rule and the tombstone behavior; and record the breaking change for existing server-persistence runtime data under Unreleased. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(storage): scope asset references to principals, typed stage refusal, uniform create collision @openmaic/storage 0.32.0. - PgDocumentStore takes assetReferencePrincipals: with reference tracking on, a write references and commits only entries held by the listed principals. An id naming another principal's entry records no row and changes no lifecycle column, like an unknown id, and the write is never refused. Without the option any held entry is referenced, as before. AssetCollector takes a matching per-document-owner function so the one-time backfill scopes each document the same way. This keeps a document naming a leaked id from committing, exposing or pinning another owner's allocation. - RuntimeStageNotFoundError: a store may refuse createSession for a stage its host considers absent; the runtime handler answers 404 STAGE_NOT_FOUND, recognizing the error by class or by code. - A session create over a taken id answers 409 SESSION_ALREADY_EXISTS whoever holds it. It used to answer 403 for another learner's session and 409 for one's own, which told a caller whose session an id was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): close owner-seam boundary gaps in references, tombstones and legacy assets - Document writes reference and commit only the writer's own asset entries and legacy shared ones (owner-bound document store and the collector backfill), so another owner's course naming a pending id can no longer commit, expose or pin it. - The foreign-read rule additionally requires the referencing live course to belong to the entry's owner, as defense in depth for reference rows that did not come from a scoped write. - The tombstone refusal for new runtime sessions moved from a raw path comparison in the route into the guarded store's createSession (RuntimeStageNotFoundError, 404 STAGE_NOT_FOUND), so no encoded spelling of the path routes around it. - Replacing or deleting a legacy shared entry checks reference ownership and mutates in one transaction under the entry row lock, which a concurrent reference insert must wait for. - Tests build the uncommitted-but-referenced, tombstoned-with-rows and foreign-reference states directly so each half of the read rule is observable, and a PostgreSQL suite holds a competing reference open across a legacy delete. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(persistence): upgrade note for dev-token-gated deployments, owner-scoped references Tell operators whose only access gate was the development token to put the deployment behind an access code or gateway, register an owner authenticator, or turn server persistence off before upgrading, and describe that a course references only its owner's media. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(storage): per-owner reference principals and a typed taken-id conflict - PgDocumentStore's assetReferencePrincipals is now a function of the store's bound owner, evaluated per write, instead of a fixed list. A store re-bound with forOwner therefore scopes to the new owner rather than carrying the previous owner's principals onto its writes. It has the same shape as AssetCollector's backfill option, so one function serves both. - PgRuntimeStore and BrowserRuntimeStore raise RuntimeSessionExistsError for a taken session id, and the runtime handler classifies it as 409 SESSION_ALREADY_EXISTS even when its re-read cannot see the holder (for example a session a host-side wrapper hides). Recognized by class or by code. Still the unreleased 0.32.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): guard the Pi whiteboard runtime and answer hidden collisions with 409 - Every server-side runtime writer now uses the tombstone-guarded store: the Pi native whiteboard service gets it through the same helper as /api/persistence/runtime/*, so a deleted course takes no new whiteboard runtime either. - Creating a session whose id is held by a session a tombstone hides answers the uniform 409 SESSION_ALREADY_EXISTS instead of a 500, covered on PGlite and on real PostgreSQL. - The owner-bound document store passes its reference-principal function, so the scoping follows the bound owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 3 — accounts through an identity gateway (trustedProxyHeader) * feat(auth): owner identity seam, phase 3 — trusted-proxy-header authenticator Add the trustedProxyHeader built-in: real accounts through an identity gateway (oauth2-proxy, Authelia, a Keycloak-based proxy, an institutional reverse proxy) that signs the user in and forwards the verified user in a request header. OpenMAIC never handles passwords or tokens. - Selected with OWNER_AUTHENTICATOR=trusted-proxy and validated in instrumentation register(). TRUSTED_PROXY_SECRET (32-1024 printable ASCII characters) is required; header names default to x-openmaic-proxy-secret and x-forwarded-user, and an optional groups header with TRUSTED_PROXY_ADMIN_GROUPS grants admin. An unknown selector, a TRUSTED_PROXY_* variable without the selector, a shared owner id or a host-registered authenticator alongside it, and malformed, reserved or repeated header names fail the boot. - Trust boundary: Next.js exposes no TCP peer address to route handlers, middleware or Server Actions, and fills x-forwarded-for from the socket only when the client sent none, so no address allowlist is offered. The gateway secret is compared in constant time (hashed, timingSafeEqual). - Per request: a missing or wrong secret, a missing or blank user, a user header with a comma (duplicate lines arrive comma-joined), or a user whose proxy:<user> id fails the owner id guard is 401 INVALID_CREDENTIAL, never an anonymous owner. The guard failure is a 401 rather than a 500 because the value comes with the request. The user is trimmed and keeps its case. Every gateway user holds course:publish; kind user, assurance verified, channel proxy; no cookie. Route handlers and Server Actions share one code path. - The unset-ACCESS_CODE boot warning is skipped in this mode; the gateway is the access gate and ACCESS_CODE stays independent. - The identity boundary scan now also fails on gateway identity headers (x-forwarded-user/-groups/-email, x-auth-request-*, remote-user, the secret header) and TRUSTED_PROXY_* read outside lib/server/identity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): document accounts through an identity gateway Describe the trustedProxyHeader built-in in the README owner identity section (and its Chinese counterpart): configuration, request semantics, the trust boundary and why it is a shared secret, the interaction with ACCESS_CODE, boot-time validation, and an oauth2-proxy example that injects the user, groups and secret headers. Add the .env.example block, a SECURITY.md note and an Unreleased changelog entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): harden the trusted-proxy authenticator after review - Reserve the header names HTTP, Next.js and forwarding proxies set themselves: configuring the user, groups or secret header to rsc, next-action, forwarded, x-forwarded-for/-host/-proto/-port, x-real-ip, or anything starting with x-middleware-, x-invoke-, x-nextjs- or next-router- now fails the boot. - Log a one-time boot warning when TRUSTED_PROXY_ADMIN_GROUPS is set: the admin role is granted from the groups header, which is trusted on the strength of the shared secret alone, so the gateway must overwrite or strip it. - POST /api/chat/pi resolves the owner up front and answers 401 when the authenticator rejects the credential, like every other owner-resolving route, instead of continuing without an owner. The anonymous default always resolves, so it is unaffected. - The identity boundary scan also covers the Authentik, Azure App Service authentication, WebAuth, Cloudflare Access, Google IAP and AWS ALB OIDC header families. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): admin-groups warning and a corrected oauth2-proxy example - Warn next to TRUSTED_PROXY_ADMIN_GROUPS (README, README-zh, .env.example) that the groups header is trusted on the strength of the shared secret alone, and list the reserved header names. - The oauth2-proxy example now states it targets v7.14+ (nested claimSource/secretSource), uses the canonical bindAddress, sets preserveRequestValue: false on all three headers and insecureSkipNonce: false, forwards the subject with `claim: user`, and notes that the secret variable holds the literal secret. It adds the legacy-option migration caveat, a warning not to exempt app routes via skip-auth or trusted-ip options, and notes that the IdP must release the groups claim and group names must not contain commas. - Changelog: the new boot checks and the /api/chat/pi 401. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): owner identity seam, phase 4 — host extension hooks * feat(persistence): owner identity seam, phase 4 — host extension hooks Let a host add product behavior at four points without forking a route, registered once from instrumentation.ts register() like the owner authenticator and sealed on first use. Nothing changes when no hook is registered. - configurePersistenceHooks({ name, authorizeCreate, onCreate, library, beforeAssetAllocate }): - authorizeCreate / onCreate run once per created course, inside the create transaction, in the transaction that inserts the course's ownership row. Saves of a course the owner already holds, and a create that loses a race to a concurrent create of the same id, are updates and run no create hook. A refusal answers 403 CREATE_REFUSED; a throw rolls the course, its ownership row and the host's writes back together. The hooks receive the request principal, or only the owner id for a background agent run. - library chooses which stage ids GET /api/stages lists; the route builds the usual summaries in the provider's order and drops every id the read path would refuse (deleted or unclaimed). folderId is shown only on the principal's own courses. - beforeAssetAllocate runs inside the storage handler's asset authorization step for POST /assets, after the owner is resolved and before the body is read; a returned Response is sent instead, with nothing stored and no quota counted. - configureAssetByteStore({ name, create, signsReadUrls }) replaces the ASSET_S3_BUCKET switch for both the persistence provider and the asset collector. A host store must keep bytes outside the registry database. Under ASSET_BYTE_EGRESS=redirect a store that does not declare signsReadUrls stops the server at boot; the built-in layers keep their behavior. - @openmaic/storage 0.33.0: DocumentWriteRefusedError lets a store refuse a write as policy; the document HTTP handler answers 403 with its code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): harden the host extension hooks after review Correctness: - Registration reads every hook once and stores it bound to the object the host passed, so hooks defined on a class prototype or as non-enumerable properties are kept. A spread copy dropped them silently, including the authorizeCreate and beforeAssetAllocate gates. Same for configureAssetByteStore. The unknown-key check now applies to plain objects only, since class instances carry their own state fields. - The owner-bound document store carries each call's operation in its own async context (AsyncLocalStorage) instead of a field on the instance. One store is shared by every tool of an agent run; a concurrent call could overwrite the field before the transaction read it, so a create was gated as a read (no ownership row, no create hooks) or claimed another stage. create_stage also runs sequentially within a tool batch. Hardening: - isDocumentWriteRefusedError recognizes a refusal from another copy of the storage package by name and shape (not by code alone). The document handler and POST /api/stages use it. - beforeAssetAllocate also gates PUT /assets/{id}/content. The request carries operation ('create' | 'replace') and the decoded assetId from the handler's own routing. - Agent runs: a refusal on a background write carries a fixed message, and create_stage answers it with a stable tool result. The host's message never reaches the model. - A byte store declaring signsReadUrls without a signer degrades to direct bytes with a warning instead of failing reads, and the collector no longer requires signing. - DocumentActor is discriminated by source ('request' with the principal, 'background' without one). - Library ids the read path cannot address are dropped, and a provider answer is capped at 5000 ids. - The boot-time ownership backfill logs how many courses it adopted without hooks. - The server persistence provider no longer exposes an unscoped document store. - The writesOutsideRegistryDatabase trust boundary and the scope of upload admission (HTTP uploads, not server-generated media) are documented. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): refuse misspelled hooks on host classes too A class instance skipped the unknown-key check, so a misspelled hook such as authorizeCreat registered cleanly and never ran, leaving creation ungated. Both configurePersistenceHooks and configureAssetByteStore now reject any function-valued property of a class instance (own or on its prototype chain up to Object.prototype, constructor excluded) that is not a known key, and any unknown key of a plain object. The error names the key and suggests the known key within edit distance 2. Host classes keep their helpers private (#helper) or register a plain object; the README says so. Also: - isDocumentWriteRefusedError requires a string message from a cross-copy candidate, and documents that the name is the effective discriminator. - CHANGELOG: ServerPersistenceProvider no longer exposes an unscoped documentStore; use the owner-bound store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only * feat(persistence): owner identity seam, phase 5 — tenancy lives in stage_meta only Course ownership was recorded twice: in stage_meta, which already answers every access decision, and in document_stages.owner_id, which the storage package scoped by. Two tenancy records double what a later claim has to re-key, and a host that keeps ownership in its own table had to rewrite the package's SQL to strip the column. @openmaic/storage 0.34.0: - PgDocumentStore never reads or writes document_stages.owner_id. An owner-bound store scopes listings, writes, deletes, the freshness manifest and folder membership through the host's ownership relation, named with the new documentOwnership option ({ table, stageIdColumn, ownerIdColumn, tombstoneColumn, claimOnCreate }, or false for no document scoping). Identifiers are validated, never quoted into SQL from arbitrary input. - forOwner / ownerId without documentOwnership throws at construction, so a direct consumer cannot upgrade into a store that silently scopes nothing. An unbound store is tenant-agnostic. - claimOnCreate inserts the ownership row in the create transaction; a concurrent create of the same id by another owner rolls back. - folders: false lets a host whose document_stages has no folder_id use the store; folder ids are unique per owner, so membership goes through the ownership relation too. - AssetCollector reads each document's owner from documentOwnership for assetReferencePrincipals, and refuses the function without it. - Schema: fresh installs get no owner_id column and none of its indexes; an existing column is kept for one release, made nullable with its default dropped (each ALTER guarded by a catalog check, so a replay takes no lock), and its two indexes are dropped. document_stages_folder_idx replaces the owner/folder index. Application: - The owner-bound store binds the package to stage_meta (STAGE_META_OWNERSHIP) and lists through it in one query; the asset collector schedule passes the same relation. - The boot backfill copies column-only owners into stage_meta before any store is built, only when the legacy column exists, idempotently, and logs how many it adopted and how many disagree (stage_meta stands). Tests cover a fresh install and an upgrade from the previous schema on PGlite and PostgreSQL 16, a host table with neither ownership nor folder columns, folder isolation across owners sharing a folder id, and the concurrent-create race. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): serialize schema bootstrap; guard unscoped owner-bound stores Review follow-ups for the tenancy change. - Concurrent starts: two instances starting against one database raced on the schema DDL (a duplicate pg_class / pg_type entry, or "tuple concurrently updated"), failing one instance's first request. Every schema bootstrap the application runs -- the persistence provider's sequence (runtime, documents, stage_meta and its ownership backfill, owner materials, assets), the agent-session, session-material and user-skill stores, and the asset collector's -- now runs under one PostgreSQL advisory lock held on a dedicated connection and released in finally. It lives in the application because what needs serializing is the whole sequence, including tables the package does not own. A PostgreSQL test boots eight instances at once, four rounds each, on a fresh and on an upgraded database; without the lock it fails every run. - documentOwnership: false on an owner-bound PgDocumentStore now also requires allowCrossOwnerDocumentAccess: true, so binding an owner for folders or asset principals cannot silently expose every owner's documents. - Documented that an ownership relation must cascade with the document rows (a leftover row keeps the id reserved for its owner, which is also how a host keeps retired ids from being reused), with tests for both. Leftover rows are deliberately not made claimable: a host cannot be told apart from one that keeps them as retirement markers. - Upgrade notes: the one-time table locks on the first start, and the out-of-band CREATE INDEX CONCURRENTLY / DROP INDEX CONCURRENTLY path that leaves the start with nothing to build or drop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): leave the schema bootstrap lock out of the held-lock check The create-hooks suite asserts that its host lock is released with the create transaction by counting granted advisory locks database-wide. A suite booting a provider against the same database at that moment now holds the schema bootstrap lock, which is not the lock under test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in * feat(storage): owner merge primitives for claiming anonymous work (0.35.0) Add the per-store pieces a host needs to move one owner's data to another inside a single transaction of its own: - reassignDocumentFolders(tx, { fromOwnerId, toOwnerId, documentOwnership }) moves folders and filing: a same-named folder merges into the target's, a colliding id is renumbered, and filing is rewritten in one statement from the old ids. - PgAssetStore.reassignPrincipal(fromKey, toKey) moves a principal's entries under both principals' write locks, in key order; quota is not re-checked. - PgUserSkillStore.mergeOwner(from, to) moves skills, renaming a live handle the target already uses with the first free numeric suffix. - PgUserSkillStore takes a resolveFinalOwner hook, run as the create transaction's first statement, like the agent-session store's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(auth): owner identity seam, phase 6 — claiming anonymous work on sign-in A visitor who works anonymously and then signs in can move that work to their account in one transaction, and the anonymous id is retired afterwards. - Principal: pendingClaim { fromOwnerId, assurance }. The trusted-proxy built-in sets it when a gateway request also carries a valid anonymous cookie; the anonymous and shared-team built-ins never do. Authenticators may implement describeStoredOwner, canonicalize and clearPendingClaim; principalFromStoredOwner describes a stored id for work without a request. - Claims: claimOwner / claimPendingOwner and registerClaimParticipant (sealed on first claim). One transaction, both owners' identity locks taken exclusively in key order, participants in a fixed order (folders, courses, materials, agent sessions, skills, runtime, assets), recorded in the new owner_merges table. Idempotent per pair; refuses a non-anonymous source, an anonymous target, a source already claimed elsewhere, and chains. Same-named folders merge, colliding folder ids are renumbered, colliding skill handles get a suffix, and quotas are not applied to what moves. - Identity lock: every owner write (documents, folders, asset allocations, material registration, agent sessions, skills) takes the owner's advisory lock in shared mode first, so a write racing a claim is moved or refused, never orphaned or deadlocked. - Retired ids: request writes are refused with 403 OWNER_RETIRED; agent runs and media generation that started before the claim follow the id to the account (forwarding document store, canonicalized stage probes, forwarded generated assets and skills). - Trigger: POST /api/identity/claim (same-origin JSON only; clears the anonymous cookie), or OWNER_CLAIM_TRIGGER=auto on the first request carrying a pending claim. The runtime contract's learner merge now performs the same claim, allowed only for the request's own pending claim. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): count only the suite's own advisory locks after concurrent creates The check that a host's transaction-scoped lock is released counted every granted advisory lock in the database. Suites that run against the same database at the same time hold advisory locks of their own (a schema bootstrap, the owner identity locks a claim takes), which made it fail depending on timing. Count only locks held by this suite's connections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(auth): use a neutral example table for host claim participants Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(storage): owner merge fixes and store policy errors over HTTP (review round 1) - PgUserSkillStore.mergeOwner reserves every handle either owner holds before renaming: a claimed skill whose handle the target uses could be renamed into a handle another claimed skill still held, which violated the per-owner unique index and made that merge fail every time. - PgUserSkillStore.list orders ties by id. - PgRuntimeStore takes resolveFinalLearner(tx, learnerKey), run as the create transaction's first statement, so a host can fence session creates against its owner merges; reassignLearner(from, to) re-keys sessions without re-validating them, so one session a newer version wrote cannot block a merge. - Every HTTP handler (runtime, documents, assets) answers a DocumentWriteRefusedError as 403 with its code, and the new StorageBusyError (code, message, retryAfterSeconds) as 503 with Retry-After. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): claims review round 1 — fenced runtime creates, bounded locks, retired-cookie recovery - Runtime session creates take the owner's identity lock as the create transaction's first statement and refuse a retired owner, closing the window in which a create racing a claim was left under the retired id. - Lock waits are bounded: a claim waits OWNER_CLAIM_LOCK_WAIT_MS (default 5 s) for its identity locks, a write OWNER_WRITE_LOCK_WAIT_MS (default 30 s); running out, or losing a deadlock or serialization race, answers 503 OWNER_BUSY with Retry-After on the claim routes and on every fenced write. - Every 403 OWNER_RETIRED carries the Set-Cookie values that drop the retired anonymous credential (the anonymous built-in now implements clearPendingClaim), so a browser whose claim response was lost recovers. Writes by id to rows that moved (skill delete, session messages) answer OWNER_RETIRED instead of not-found or a bare 403. - OWNER_CLAIM_TRIGGER=auto no longer claims ahead of the explicit claim routes, which now report their own claim instead of a false refusal. - The owner-events stream tells a retired owner only that it moved, never the claiming account's id. - Creates of one course id take turns (a per-id advisory lock before the ownership probe), so a concurrent second create saves as an update instead of failing as reserved-document. - A claim's source must be described as anonymous by the authenticator; the write fences skip the retirement lookup for every other owner. The host canonicalize hook is removed: forwarding is core's owner_merges alone. - The folders participant locks the source's stage_meta rows first; runtime sessions are re-keyed without re-validation; the owner asset store fences every allocation and refuses to allocate without transactions; background skill reads and patches follow a mid-run claim; the forwarding document store forwards methods only. - Docs: when a pending claim can arise with the shipped authenticators, the anonymous cookie as a bearer credential, bounded waits, the exact scope of OWNER_RETIRED, and claims narrowed to what the tests show. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(auth): claims review round 2 — merge-record guard, boot-checked lock waits, stream credential drop - owner_merges records claims of anonymous owners only, since the write fences enforce retirement only for ids the authenticator describes as anonymous. Reading a row that retires any other owner now fails loudly instead of being followed but not enforced. The comment that pointed hosts at claimOwner for their own account merges is corrected: a host merging two signed-in accounts moves the rows itself and refuses the merged-away account in its authenticator. describeStoredOwner must classify ids stably. - OWNER_WRITE_LOCK_WAIT_MS and OWNER_CLAIM_LOCK_WAIT_MS are validated at boot with the same parser the lock paths use, so a malformed value stops the server instead of failing every write. - The owner-events stream answers a connect by an already retired identity with one owner_moved event and the Set-Cookie values that drop the retired credential, so a stale tab's reconnect loop ends after one round trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * chore(auth): final polish before landing the owner identity seam * chore(auth): final polish before landing the owner identity seam - Pi routes refuse a rejected credential with the seam's INVALID_CREDENTIAL body. - Drop the unused LegacyAssetMutations alias and resolveLockWaitMs re-export. - Scope the document_stages.owner_id deprecation wording: the store no longer reads or writes it; the boot backfill reads it and a claim mirrors ownership into it for rollback. - Document OWNER_CLAIM_TRIGGER and the two lock-wait variables in .env.example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(storage): 0.35.1 for the README wording fix Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(identity): one identity entry point with composable auth methods * feat(identity): compose owner auth methods behind one entry point Owner resolution now asks an ordered list of owner auth methods instead of one configured authenticator. Each method answers `authenticated`, `not-applicable` (no credential of its kind) or `invalid` (its credential is present but bad). Core keeps the first `authenticated` answer, refuses any `invalid` answer with 401 INVALID_CREDENTIAL without asking later methods, and when no method applies falls back to the anonymous cookie owner, or answers 401 when the host turned the fallback off. Route handlers and Server Actions share the same order and per-request memoization. - configureOwnerAuthentication({ methods, anonymousFallback? }) replaces configureOwnerAuthenticator: single-shot, sealed on first use, and validated at registration and at boot. - anonymousCookie becomes the built-in fallback method; sharedTeam becomes a method, selected by PERSISTENCE_SHARED_OWNER_ID as before when nothing is registered, and kept beside host methods only when the host lists sharedTeamAuthMethod() last. The variable set beside a registration that leaves it out fails the boot. - Core attaches pendingClaim itself: a host method's non-anonymous principal on a request that also carries a valid anonymous cookie. A method may not set one. Claim cookie clearing and OWNER_RETIRED recovery go through the anonymous method's clearCredential. - principalFromStoredOwner consults the anonymous method first, then the configured methods' describeStoredOwner in order. - Remove the built-in trusted-proxy header authenticator with OWNER_AUTHENTICATOR and every TRUSTED_PROXY_* variable. The boundary test now also keeps the resolution internals inside lib/server/identity/ and asserts no source reads the removed variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(identity): document owner auth methods and a gateway JWT recipe Rewrite the owner identity section around the ordered auth methods: the three answers and what resolution does with each, the anonymous fallback, registration and its boot checks, and the sharedTeam rules. Replace the removed built-in gateway authenticator's documentation with a short host-owned recipe that verifies a gateway-forwarded signed JWT (oauth2-proxy, Cloudflare Access, IAP) against the identity provider's keys with the jose library. Update the claim section for the core-owned claim candidate, and the Chinese README, SECURITY.md, CHANGELOG and the deployment docs in every locale to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(identity): tighten retired-owner cookies, retired config and the boundary - On 403 OWNER_RETIRED, core now calls clearCredential only on the anonymous fallback and on methods that declare `issuesAnonymousOwners: true`, so an account method's session cookie is never cleared there. A clearCredential that throws is logged and skipped instead of turning the refusal into a 500. - Setting any retired OWNER_AUTHENTICATOR / TRUSTED_PROXY_* variable fails the boot with a pointer to the auth-methods docs, instead of silently serving every request anonymously. Only the names are matched, in one module the boundary test checks never reads a value. - Add lib/server/identity/host/ for host auth methods. The boundary test now scans core identity files for gateway identity headers too, polices reads of an incoming Authorization header everywhere but the host directory, and keeps resolution internals out of host code. - Document on OwnerAuthMethodResult that a present but unusable credential must answer `invalid`, never `not-applicable`, and that a method that cannot decide throws. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(identity): fix the gateway JWT recipe's error split and add notes - The recipe now answers `invalid` only for token failures (expired, claim validation, malformed JWS/JWT, bad signature, no or several matching keys, disallowed or unsupported algorithm) and rethrows everything else: jose reports a key endpoint that is down, answers non-200 or returns unparsable data as a generic JOSEError, which the previous version turned into a 401 for every user. - The recipe file goes in lib/server/identity/host/, with an accurate description of what the boundary test enforces. - Document invalid versus not-applicable for malformed credentials, issuesAnonymousOwners, the boot failure on retired variables, and that the claim candidate is an unsigned bearer cookie a sibling subdomain can shadow (serve on a registrable domain of its own; cookie hardening is deferred). Mirror the notes in the Chinese README, SECURITY.md and the changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
LING DUAN
|
2a77a8a476 |
feat(chat): make Pi classroom runtime the default (#1628)
* feat(chat): make Pi classroom runtime the default * chore(chat): align docs and E2E with Pi default |
||
|
wyucandClaude Opus 5
|
6914e03478 |
fix(docker): build workspace packages in the builder stage, not during install (#1598)
* fix(docker): build workspace packages in the builder stage, not during install The root postinstall builds nine workspace packages inside `pnpm install`. In the Docker deps stage that step peaks above 1 GiB, almost entirely from the importer's rollup + terser pass, which exhausts small Docker VMs and hangs the build (a 1 GiB container fails with a JS heap OOM). - deps: `pnpm install --frozen-lockfile --ignore-scripts`, so the stage only resolves and links dependencies (peak ~1051 MiB -> ~315 MiB). - builder: run the same chain explicitly via the new `build:packages` script, with `--max-old-space-size=1024` so a runaway build fails with a clear heap error instead of taking down the VM; public/vendor is produced here now. - `postinstall` delegates to `build:packages`, so local installs are unchanged. - CI: build the deps + builder stages when an image input changes; the main Dockerfile was not built in CI before. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf * ci(docker): build the main image for any non-docs change The builder stage copies the whole build context, so application sources and config such as next.config.ts are image inputs too. Only skip the image build when every changed file is documentation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
wyucandClaude Opus 5
|
ed22afe869 |
ci: disable setup-node package-manager cache in publish jobs (#1584)
setup-node v5 turns on package-manager caching automatically when package.json declares `packageManager`, which requires pnpm on PATH. The `publish` and `mark` jobs only use npm on prebuilt tarballs and never install pnpm, so the first package publish after the Node 24 action migration failed in setup-node with "Unable to locate executable file: pnpm". Claude-Session: https://claude.ai/code/session_01DYifP8wM4XJQ3Hc2qsF6zf Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
LeoParkerOuandwyuc
|
042ad05965 |
ci: migrate publish actions to Node 24 (#1569)
* ci: migrate publish actions to Node 24 * ci: disable unused pnpm cache in ClawHub jobs --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
b5605ef979 |
feat(agent-runtime): workbench generate_image / generate_video write through the asset pool (#1007 part 6) (#1524)
The workbench tools store generated bytes in the asset pool under the shared principal and write the allocated id into the document, the same discipline as the classic chain since #1392; the runner's putScene creates the reference rows and commits the allocations. The video completion patch rewrites every placeholder slot and retires anything that would shadow the new id; immediate render is preserved by leasing the id at the render boundary. A store-full refusal fails the tool with a model-readable error and writes nothing. Legacy /api/classroom-media documents keep rendering. Closes #1522. |
||
|
wyuc
|
c605e9e6ef |
feat(persistence): turn on the server-owned asset lifecycle and release assets on course deletion (#1007 amendment, part 2) (#1473)
App wiring for the @openmaic/storage 0.31.0 lifecycle: reference tracking and document references are paired unconditionally and declared at startup, course deletion withdraws references inside the tombstone transaction, ASSET_PENDING_TTL_MS configures the pending window, and the dead client-side reclamation code is removed. |
||
|
Yanpeng Wang
|
a03edc43e4 |
feat(ci): add opt-in issue triage agent (#1439)
* feat(ci): add opt-in issue triage agent * feat(ci): support compatible triage Responses endpoints |
||
|
LING DUANandwyuc
|
ebf665f316 |
feat(playback): reference static GenUI components (#1281)
* feat(playback): reference static GenUI components * feat(playback): gate courseware references * fix(pi): bound Interactive source HTML before parsing * fix(pi): bound interactive reference HTML processing * fix(playback): close interactive reference review gaps --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
Yunare Maia
|
dde58944c1 |
chore(ci): migrate GitHub Actions off deprecated Node 20 runtime (#1343)
* chore(ci): migrate GitHub Actions off deprecated Node 20 runtime - actions/checkout v4 → v5 - actions/setup-node v4 → v5 - pnpm/action-setup v4 → v5 Node 20 reached EOL; v4 actions still use node20. Verified via driftcheck v0.1.8 (14 occurrences across 3 workflows, all node24). * fix(ci): bump cache@v4→v6 and upload-artifact@v4→v7 in e2e job Addresses review feedback from @YizukiAme on PR #1343. - actions/cache@v4 → @v6 (lines 243, 314) — drop-in replacement - actions/upload-artifact@v4 → @v7 (line 332) — Node 24 support landed in v6+, not v5; bump to v7 to ensure node20 runtime is truly eliminated. The publish workflows (publish-packages.yml, publish-openmaic-skill.yml) remain intentionally out of scope — tracked in #1345. * chore(ci): trigger checks after rebase |
||
|
Zhenghan Songandwyuc
|
ca3d785221 |
build(node): enforce direct dependency engine floor (#1337)
Why: - The root Node 20.9 contract predates direct runtime dependencies whose declared minimums now reach Node 22.19. - README, contributor, localized docs, and the OpenMAIC extension skill repeated the stale supported-version claim. What: - Raise the root Node minimum to 22.19 and align every operator, contributor, locale, and skill prerequisite. - Add a check that compares the root minimum with installed direct production dependency engine minimums. - Run the new contract check in CI after the frozen dependency install. Risk: - This changes the declared minimum only; no upper bound is added and Node 24 compatibility remains a separate concern. - The localized docs build was validated with the independent #1306 boundary fix from PR #1307, which is not included here. Tests: - RED on the base: engine check reported pi-agent-core, pi-ai, svg-pathdata, and undici floors - GREEN: root minimum 22.19 satisfies 35 engine-constrained direct dependencies - Node 20 lockfile-only install reports the root unsupported-engine warning - Node 22 frozen install and postinstall - Docs build with PR #1307 boundary: 34 pages and all locale postexport checks; docs types:check - Root Prettier, ESLint, TypeScript, i18n, package-version, and internal-dependency gates - Root pnpm test: 7137 passed, 81 skipped Live Docs: - GitHub issue #1304 tracks the Node contract; #1306 / PR #1307 tracks the separate docs-build prerequisite. Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyucandClaude Fable 5
|
04621578de |
release: OpenMAIC 1.0.0 — the agent workbench (#1228)
* feat(storage): add an agent-session store with PG backend and layered contracts (#1163) * feat(storage): add agent-session store with PG backend and layered contracts * test(storage): avoid BigInt literals for pre-ES2020 root typecheck * fix(storage): close agent-session store review findings * docs(storage): align hook ordering and contention-probe claims with the code * ci: run on the agent-workbench integration branch * chore(storage): bump to 0.5.0 for the agent-session store * fix(storage): carry replay compaction across page boundaries * feat(agent): add the driver model contract and stage route dialect (#1165) * feat(agent): add the driver model contract and stage route dialect * fix(agent): validate route context windows and clarify dialect precedence * feat(agent): adapt the agent-session store and runtime foundations (#1167) * feat(agent): adapt the agent-session store and runtime foundations * feat(agent): resolve request owner identity via an anonymous cookie * docs(agent): document the opt-in compaction default and harden edge cases * feat(agent): add the background session runner (#1169) * feat(agent): add the background session runner * feat(agent): wire the runner into startup behind feature flags * fix(agent): stop clean interruptions from consuming the attempt budget * fix(storage): charge the attempt budget for abandoned leases but not clean parks * docs(storage): document the attempt-charging contract and decouple its tests * feat(agent): add agent session and owner event streams (#1170) * feat(agent): add agent session and owner event streams * fix(agent): close the session-existence oracle and document the owner seam * feat(agent): add agent session lifecycle routes (#1171) * feat(agent): add agent session lifecycle routes * fix(agent): validate session-create input and preserve the owner cookie on errors * refactor(storage): drop the unused active-stage API from the agent-session contract (#1174) * refactor(storage): drop the unused active-stage API from the agent-session contract Tools address stages explicitly on every call, so the store keeps no mutable session-level stage pointer. Removes resolveActiveStage and setActiveStage from the store interface, their PG implementations, the active_stage_changed lifecycle event, the session_active_stage owner event variant, and the contract tests pinning them. The active_stage_id column and the DDL check constraint stay untouched for schema compatibility. * chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal * docs: document the agent runtime configuration surface (#1176) * fix(agent): repair orphaned and late tool results across interruption boundaries (#1180) * fix(agent): repair orphaned and late tool results across interruption boundaries A crash, shutdown, or provider failure can leave the durable transcript with tool calls that have no result, or with results ordered illegally for the provider. Three failure modes were fixed: - Orphaned tool calls: a run that died between an assistant tool-call frame and its result left a dangling call in the entry tree. Resume no longer synthesizes and persists receipts for it: interrupted results are a read-time provider view owned by a shared read-boundary repair, which returns the original array for a healthy transcript and never mutates the tree. - Late parallel results: a parallel tool can finish while pi unwinds an aborted assistant frame, leaving result(A), assistant(aborted), result(B) in durable order. Strict providers reject non-contiguous results, so the read-boundary repair moves existing results next to their owning assistant frame (in call order), omits incomplete unwind frames, and synthesizes receipts only for genuinely missing calls. - Interrupted calls at the write boundary: a call still in flight when the run winds down (shutdown, lease loss, cancellation, provider failure) had no receipt at all. The runner now tracks in-flight calls from their assistant frames and, before the terminal flush, appends an interrupted-result receipt for each still-orphaned call through the same attempt-fenced write chain, so a lease-stealing zombie never writes and the next claim sees a provider-safe transcript. * test(agent): pin the runner wiring for interruption-boundary tool repair * feat(agent): add neutral tool foundation libraries (#1184) * feat(agent): register a web_search tool on the session runner (#1185) * feat(storage): add a per-session URL trust gate (#1186) * feat(agent): add the skills system (#1189) * feat(agent): add the skills system (builtin directories and durable user skills) * fix(storage): serialize the user-skill quota check-and-insert per owner Two concurrent creates at the 50-skill boundary both counted 49 rows and both inserted (READ COMMITTED, no lock), overshooting the quota contract. The create transaction now takes a per-owner pg_advisory_xact_lock first, and the same-name idempotency check runs before the count check so an at-least-once retry of the create that committed as the owner's 50th row still returns its durable receipt instead of a quota error. The 23505 backstop is retained for writes that do not take the lock. * fix(agent): share unstorable-character validation and align skill lookup * feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190) * feat(agent): add session materials and a fetch_url tool behind the URL trust gate * fix(agent): harden session material fetching * feat(storage): add an ownership scope to stage documents (#1191) * feat(agent): add material read and search tools (#1192) * feat(agent): add stage read and patch tools (#1194) * feat(agent): add page generation and deck editing tools (#1198) * test(storage): keep the PG contract suite order-independent (#1200) * fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199) * fix(storage): revoke deleted session URL authority * fix(ssrf): reject private ISATAP endpoints in strict fetches * chore(storage): bump to 0.11.1 for the session-URL authority fix * feat(agent): add roster and voice registration tools (#1201) * feat(agent): add folder organisation tools (#1202) * feat(api): add stage and material HTTP routes (#1203) * feat(workbench): add the client data layer (#1204) * feat(workbench): add the client data layer * docs(workbench): write the ported comments in English * chore(edit): remove the in-editor agent panel (#1210) * chore(edit): remove the in-editor agent panel * style: apply prettier formatting * fix(agent): report the runtime as unusable without a database (#1207) * fix(agent): report the runtime as unusable without a database * style: apply prettier formatting * feat(agent): add image, video and pptx import tools (#1211) * feat(workbench): add the agent chat surface (#1205) * feat(workbench): add the agent chat surface * docs(workbench): write the ported comments in English * fix(workbench): label the folder and rename tools on the timeline * fix(workbench): label the roster and voice tools on the timeline The reconciliation test iterates every tool the runner registers and requires a display label of its own. The roster and voice-clone tools (list_voices, set_roster, clip_audio, register_voice) reached the integration base with the roster/voice-registration tools but never gained presentation rows, so they fell through to the default branch and rendered their wire names. Port their rows from the reference implementation (labels and i18n keys verbatim) and extend the reconciliation allowlist with ROSTER_TOOL_NAMES and VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product without a label. * feat(agent): add the material extraction lifecycle (#1212) * feat(storage): add material extraction lifecycle * feat(agent): execute queued material extraction * style: apply prettier formatting * style: satisfy prefer-const in the extraction runner * test: give material fixtures the extraction lifecycle fields The media-tools slice and the extraction lifecycle slice were each green in isolation but never compiled together: the lifecycle made derivedFrom and extraction required on AgentSessionMaterial while the media-tool fixtures predate them. * chore: remove stray task notes * fix(workbench): label the extraction lifecycle tools on the timeline * feat(workbench): add the workspace shell (#1206) * feat(workbench): add the workspace shell * docs(workbench): write the ported comments in English * i18n(workbench): align workspace keys across locales * fix(workbench): adopt the landed data layer and label the extraction tools - replace the sibling-slice seam stubs with the real data-layer modules - drop ambient declarations now shadowed by landed files - port timeline labels for the extraction lifecycle tools from the reference - align the new i18n keys across all locales * ci: retrigger * feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215) * fix(storage): restore capability-based stage access * fix(api): bind document access to request owner * fix(agent): restore three-state stage access on the tool layer Port probeStageAccess and the three-state StageAccess (owned / foreign / missing / tombstoned) and gate every stageId-bearing stage tool on an owned probe, mirroring the reference per tool: - move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage with the single not-yours message before touching the store. - The course/DSL toolset and the roster toolset are wrapped by withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every writer refuse a foreign stage with the same message and refusal shape. - Scene preview keeps its own probe and its own refusal text, and is registered beside the course toolset (never double-gated). - The runner injects one probe factory at the three call sites. Tests: the dsl cross-owner test premise (a foreign stage is readable by id) encoded an invented capability-read policy that the reference does not have at the tool layer; it now asserts foreign read/patch/grep are all refused while the owner still reads. Curriculum cross-owner assertions were already the reference's and now pass with the probes in place. * docs: correct per-file test counts in the fidelity report * test: fix type errors in stage-access fidelity test * test: adapt media-tool and gate suites to the owner-scoped store seam * feat(api): add owner-scoped course-folder HTTP routes Port the reference implementation's /api/folders family (list, create, rename, delete with ungroup/remove modes, and folder membership) onto the owner-bound document store, replacing its provider-based auth with the existing withRequestOwnerId / owner-scoped store seams. The storage package's folder store grows the pieces the routes need: DocumentFolder.order (schema column + max+1 assignment + ordering), renameFolder, deleteFolder(mode) with captured member ids, and setStageFolder(stageId, folderId | null) with idempotent un-filing. FolderNameError moves into folder-name-validation.ts (stage-storage re-exports it, keeping import sites intact). Every route gates on the configured agent runtime (plain 404 when off or unconfigured), keeps the reference's machine codes and envelopes, and is covered by gate tests plus a behavior suite. * feat(api): add stage-meta viewer surfaces for the classroom Port the reference implementation's viewer-facing stage state — can-edit / collected / published / generation-complete — on top of the stage-access base (stage_meta + tombstones). stage_meta gains published_at and generation_complete columns plus a stage_bookmarks table; the reference's deployment-specific origin/claimed_at columns are stripped. New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status, POST generation-complete / publish / unpublish (owner-only), POST /api/bookmarks. The resolver lives in lib/server/stage-access.ts. Wiring: a fetchStageMeta client with the reference's three-outcome contract, stage-store isOwner/isBookmarked/readOnly fields (upstream single-user defaults, no-op until the sidecar answers) plus setViewerAccess, the classroom apply path computing readOnly = !(isOwner || isBookmarked), the Stage editability gate, and a sidecar probe after each classroom load. A sidecar 'absent' answer keeps the editable default here because the classroom also serves local-only courses; server writes stay owner-enforced. * feat(api): port the reference material upload contract Rewrite POST /api/materials to the reference implementation's upload shape so the workbench uploader (uploadWorkbenchMaterial, which posts no session id and expects a flat 201 view) works unchanged: owner-scoped upload with mime normalization/validation (415), per-class size caps checked on the declared content-length and the streamed body (413), empty body (400), quota (429), sha256 reserve->store->finalize lifecycle with abandon on failure, flat { materialId, originalName, bytes, mime, extraction } 201, and an x-request-id echo. Adds the owner-scoped material library (owner_material table + quota + 24h lazy sweep, bytes in the host's asset registry as the neutral replacement for the reference's object-storage byte path) and the material cap configuration. The session-scoped GET list is left as-is; the reference's owner-material extraction worker is not ported (the branch's session-material extraction lifecycle already covers extraction). Gate tests now cover all 23 persistence routes across the three runtime env states; the materials behavior suite pins the new contract. * feat(media): add an optional local ffmpeg media extractor (#1213) Adds a local ffmpeg/ffprobe pipeline as a second media extraction provider behind the extractor registry, ported faithfully from the reference implementation: duration probing, keyframe-safe chunking, per-chunk ASR with timeout and deadline budgets, and timestamped transcript assembly. - Availability probing feeds the registry's candidate selection: the provider simply is not a candidate when ffmpeg/ffprobe are absent. - With neither ffmpeg nor a cloud provider configured, extraction fails with an actionable message naming both enablement paths. - Media materials route through the same extraction lifecycle and lease fence as documents; no parallel queue. - Tests inject the executable resolver so the missing-ffmpeg path is the default-tested one; the real pipeline test is skip-if-unavailable. - @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material lifecycle surface). * feat(storage): per-scene monotonic revisions via database triggers (#1214) * feat(storage): per-scene monotonic revisions via database triggers Restore the reference implementation's freshness granularity: a per-scene monotonic revision maintained by database triggers, so every writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without application cooperation. - Companion revision tables + trigger functions in the storage package's idempotent schema bootstrap, with the lock-order invariant, pg_notify wakeup and the suppression switch for batch writers. - ensureDocumentSchema gained a dollar-quote-aware statement splitter. - The freshness and manifest routes serve per-scene revisions. - Mutation-verified: dropping the triggers turns the revision tests red. - @openmaic/storage 0.13.0 -> 0.14.0. * fix: forward the freshness manifest through the owner-bound store * feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208) * feat(workbench): add the Pro entry points * feat(workbench): preserve Pro mode transition semantics * fix(workbench): drop ambient declarations shadowed by landed slices * fix(workbench): drop ambient declarations shadowed by the landed shell * feat: port workspace shell sibling modules Port the 16 leaf modules the Pro workspace shell imports but that were only ambient-declared, replacing the compile-time bridge with real implementations adapted from the sibling-slice reference: pure workbench helpers (session title, rail tab, course-chat bootstrap, created-course tabs, course-tabs memory, workspace navigation, pane navigation, pro-edit sizing, existing-course minting, first-message session), the neutral brand context and course-rename server API, the server-action session delete, the home discovery hook, the classroom pane host with its load-policy leaf, the theme toggle and floating-layer owner, plus the floating-layer-owner wiring the dialog/dropdown/tooltip portals stamp. Also add the workbench-shell locale copy for all 12 locales, port the reference tests for the ported modules, and drop types/workbench-sibling-slices.d.ts now that every declaration has a real implementation. * docs: keep ported comments in English and deployment-neutral * docs: announce 1.0.0 and refresh the feature overview (#1216) * docs: announce 1.0.0 and refresh the feature overview * docs: finalize 1.0.0 README after feature merge * fix(agent): control-plane routes answer 404, not 500, without a database The agent control-plane routes gated only on the runtime flag, so an enabled-but-unconfigured deployment (flag on, DATABASE_URL empty) answered 500 from a store that cannot connect. Gate them on the configured check instead, matching the stage/material routes: the whole surface is cleanly absent until both the flag and the database are present. The status probe keeps reporting both bits. * test: mock both runtime gate exports in the control-plane route suites * fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217) The generate_tts / scene-tts path checked the runner's AbortSignal between actions but never created the provider HTTP requests with it, so a session cancel left a hung synthesis fetch in flight until a restart repaired the tool result. Thread the signal end-to-end: TTSModelConfig carries an optional signal, generateTTS combines it with a per-request timeout (TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure, glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao, elevenlabs, lemonade) is created with that signal. A timeout now fails the tool call with TTSRequestTimeoutError (a clear retryable error) instead of wedging the session; a caller cancel propagates as the interruption so the runner settles the session as cancelled without a restart. Tests: hung-provider simulation rejects at the timeout with the retryable error; abort mid-flight aborts the captured request signal and surfaces the interrupted shape; removing the signal wiring makes the abort tests fail (red), restoring them turns green. * fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218) Finding 1: with server persistence on, listStages resolved to the generic GET /api/persistence/documents listing, which the capability model deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings owner-only). The home/workspace library now lists through the owner-scoped GET /api/stages surface (same anonymous-owner cookie the workbench uses) when server persistence is enabled; the server-side 403 is untouched. Finding 2: a run interrupted (session_interrupted) and repaired (session_resumed) that ends cancelled before agent_end stranded its pending classroom sightings, so the timeline's terminal card lost the course the answer produced. session_end (cancelled) now flushes the pending sightings into the same course card set agent_end paints, before the stopped caption. * chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219) * chore(classroom): remove the bookmark ('collected') concept entirely The stage-meta viewer port introduced a bookmark surface (stage_bookmarks table, POST /api/bookmarks, the isBookmarked sidecar field, and a readOnly rule that let a saved course stay editable). The product has no such concept, so remove it as a closure: - delete the /api/bookmarks route and the stage_bookmarks table plus its query helpers from the persistence bootstrap - drop isBookmarked from GET /api/stage-meta/[stageId] - simplify the classroom read-only rule to readOnly = !isOwner across the sidecar client, ownership signal, classroom load, stage store and the classroom page - keep publish/unpublish, generation-complete, isOwner and isPublic exactly as they were - update the gate and stage-meta route suites and the README mentions The workspace rail's Bookmark glyphs and comments describe the upstream saved-courses (favorites) section, which is driven by isOwner and renders no collect affordance; they are kept as unrelated homonyms. * chore(workbench): remove the saved-courses drawer UI The first pass removed the bookmark data model but kept the rail's "Saved courses" drawer, judging it a separate surface driven by `isOwner === false`. The home/workspace listing is owner-scoped, so that flag can never occur: `allSaved` is permanently empty and the drawer (plus the collapsed-rail Bookmark mini-button) is a dead affordance. Remove it: the SavedDrawer component and its mount, the savedOpen / savedSection state, the allSaved / matchedSaved derivations, the 'saved' variant of the course-list renderers, the mini Bookmark glyph, the drawer-only CSS, and the drawer's i18n keys from all 12 locales. The courses tab is now exactly one folders tree. The authored/favorites split in workspace-tree.ts goes with it; the tree module no longer reads `isOwner`. The discovery course type keeps the field — the shell still reads it for read-only gating. Upstream has no collect concept; the drawer could only ever render empty here. The reference implementation HAS this drawer (its favorites come from its account system), so this removal is a deliberate upstream product decision, not a fidelity bug. * fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221) * fix(workbench): restore the composer attach entry by gating it on the live runtime The AttachButton's rollout probe read a `materialsEnabled` field that this branch's /api/agent/runtime never answers (the materials routes gate on the runtime itself, like the stages), so the gate could never pass and the attach button never rendered — the Pro launch and chat composers showed only the @-mention and enhance glyphs. Substitute the field with the runtime's `enabled` value, which IS the upload action's precondition: POST /api/materials answers 404 whenever it is false, so the render condition now equals the action precondition (no dead button). The button's label (`proMode.attach`) is a user-visible string that becomes visible again; port the reference implementation's own translations verbatim into the 11 locales that still carried the Chinese copy. * feat(workbench): add the settings entry to the rail's bottom-left cluster The reference's rail foot carries a cluster of utilities (its saved-courses drawer, the language switcher, the display toggle). This branch removed the drawer — it could only ever render empty here — and the product decision is to fill that freed spot with the settings entry. Add a settings trigger to the foot cluster (expanded rail, beside the language and display toggles, and on the collapsed strip) and mount the model/provider SettingsDialog in the rail, wired to the trigger. It is the same dialog the classic home opens from its header pill; the workspace had no settings entry of its own, so nothing is duplicated within a surface. * test(workbench): pin the restored upload, attach, and settings entry points Covers the three restored entry points: - the courses-tab upload control: rendered beside the course name filter, wired to the discovery hook's ZIP import trigger, disabled while an import runs, and gated by the same condition as its action (the courses tab); - the composer attach control: an actual render of AttachButton under both probe answers (visible when the runtime says the upload path is live, hidden otherwise), its mounts in the launch and chat composers, the branch's runtime-field substitution in the probe, and the reference's own `proMode.attach` copy in all 12 locales; - the settings entry: the trigger in the rail's foot cluster (expanded and collapsed), beside the language and display toggles, opening the SettingsDialog the rail mounts. * chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223) A workbench build without the editor toggle has no way to edit a course: enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle. The workbench IS Pro mode, so its flag now implies the editor gate; the standalone flag remains for deployments that want the classroom editor without the workbench. Documents both flags in .env.example. * fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222) The Pro workbench chat did not stream: the session/owner SSE routes polled the durable event log on a 5s/30s clock with no wakeup, so message_update deltas (written at 150ms cadence) reached the browser in poll-sized blocks and the thinking strip only mounted after the whole reasoning text had accumulated. Port the reference's LISTEN/NOTIFY delta path: - storage: add in-transaction wake hooks (onSessionEventAppended, onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in the same transaction as the durable append; align readEventsAfterForReplay to rank the bounded page so the first delta after the cursor is always kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0. - app: port the process-wide event-notify bus (dedicated LISTEN client, self-check probe, reconnect backoff; notify through the storage transaction surface), wire the store hooks, subscribe both SSE routes before the initial read with the reference's initializing gate, and give the runner one {kind:'session'} subscription whose wake runs the cancel check and the message drain. Polls stay as the lossy-NOTIFY backstop. - lifecycle: start/stop the bus from instrumentation. Tests: storage hook + compaction contract; route wakeup latency; runner wakeup wiring with a fake agent; bus unit tests; PG contracts proving a real append wakes the routes and a live SSE route forwards a message_update on the wakeup, and that a rolled-back append never wakes. Also fix the pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against newer FK tables). * fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225) * fix(storage): refuse the non-transactional byte-write deadlock configuration A byte store whose plain write() runs on its own pooled connection cannot be invoked from inside a registry write transaction: after the transaction has claimed the blob-row lock, that write blocks on the lock the transaction just took while the transaction waits on the write - a self-deadlock PostgreSQL cannot detect (one side is idle in transaction). There is no lock-safe ordering for such a writer: bytes must be written after the row claim (writing before it lets the collector delete the bytes while the upsert waits), and any second-connection write after the claim is the deadlock. The configuration is therefore detected and refused: - AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the layer's plain byte operations cannot contend for the registry's row locks. - PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite) when the byte store has no writeWith and does not declare the flag, throwing a clear configuration error before any row is claimed. - The collector mirrors the guard on its delete path (deleteWith or a declared out-of-registry layer, else a configuration error). - The object store declares the flag (its out-of-transaction write remains legitimate); the in-registry PostgreSQL byte column provides writeWith / deleteWith instead. - Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so any future lock-contention variant fails loudly instead of hanging. Bumps @openmaic/storage to 0.18.0. * fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper The no-bucket case of lazyAssetByteStore returned a bare { write, read, delete } and dropped writeWith/readWith even though the underlying PgAssetByteStore has them. The registry's hasTransactionalWriter duck check then failed and put() fell back to the byte store's own pooled connection, which blocks forever on the blob-row lock the registry transaction just took when the bytes live in the same PostgreSQL - the production self-deadlock. The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned methods are forwarded eagerly (typed against the real signatures via PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no transactional writer exists there, the signed-URL method stays absent or lazy exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase so the registry may run the plain write inside its transaction. New tests pin the wrapper's transactional capability red-to-green and assert put()/resolve() route byte traffic through the transaction-pinned queryable. * fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget The classic home flow's Generate click drained in-flight ingests for the full 15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as a UX bound) and reuse the existing timeout fallback: sources that miss the cap proceed on the legacy byte path and each late-resolving id is released. * chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0 * fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226) * fix(agent): bound every tool call with a global timeout and settle it on cancel A tool await that neither resolves nor rejects wedges the session forever: the lease keeps heartbeating and the driver never reaches its next cancel checkpoint. Race every tool execution (in buildAgent) against a hard budget (OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for known long runners) and against the caller's AbortSignal, so even a signal-ignoring await cannot keep a cancelled session running. On timeout the call rejects with AgentToolTimeoutError; the agent loop turns the rejection into a structured error tool-result the agent can retry or proceed from, and the abort signal is delivered to the tool's in-flight work through a derived controller. Zombie-tool updates after settlement are dropped. * fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim The claim scan treated a session with cancel_requested_at set as a normal claim candidate: after a restart it re-leased the same session for attempt N+1 and resumed generating despite the pending cancel. claimNextSession now settles such candidates as cancelled under the claim lock (status cancelled, attempt reset, lease and cancel request cleared, terminal session_end event and owner projection) instead of leasing them, then keeps scanning. Bump @openmaic/storage to 0.18.0. * docs: takeaway-style 1.0.0 announcement with bilingual guide links The 1.0.0 head is now a short takeaway block — badge links to the official user guides (English and Chinese), five one-line highlights, and pointers into Features and the workbench setup section — instead of six dense paragraphs. The detailed provider-neutrality and freshness notes move into the Features workbench section, phrased database- neutrally (the announcement no longer names a specific database). Release date corrected to August 27. * fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229) * fix(workbench): wire workspace folder routes * fix(editor): restore reference workbench chrome * fix(workbench): persist composer materials and course refs * fix(workbench): preserve live reasoning frames * fix(persistence): back off failed streaming saves * chore(workbench): retire stale slice seams * test(editor): cover element pin layer * chore(storage): bump to 0.21.0 for the user-message ref/material fields * chore(editor): translate ported code comments to English * fix(agent): fence durable tool writes and consume cancel requests atomically (#1230) * fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231) * fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232) * fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233) * fix(workbench): send the opening session message exactly once with refs intact (#1234) * feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235) * fix(media): restore the reference classic media chain (#1236) * fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237) * fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238) * test(providers): reconcile the provider-config vendor-token debt count after the main merge The integration line's AK/SK fallback for the managed document provider adds occurrences that main's allowlist snapshot predates. Same mixed-composition debt category the group already documents; no new vendor behavior. * test(providers): reconcile vendor-token debt counts with the integration line The main-merge brought main's neutrality-guard snapshot next to integration features it predates (media-extractor fallback chain, local voice-profile deletion semantics, the enabled-TTS helper). Same debt categories the guard already documents; counts updated to the guard's own tally and two grouped entries added. No new vendor behavior. * fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239) * feat(skills): add Feynman and spiral curriculum methods (#1240) * feat(agent): port missing reference tools and skills (parity audit) (#1241) * feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242) * fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243) * feat(settings): skill management with real list, download, delete, and upload (#1244) * feat(settings): skill management section with real list, detail, and zip download * feat(skills): owner skill delete and upload across storage, API, and settings * fixup! feat(settings): skill management section with real list, detail, and zip download chore: neutralize a reference note in the settings header comment * fix(media): persist origin-independent classroom-media references from the agent runtime (#1245) * feat(editor): float the insert toolbar in the outer frame with collapse (#1246) The insert strip was bounded to the slide card, so it could only ever sit on top of slide content: the card's overflow clipped it and it could not be parked in the padding beside the slide. Move it into the studio frame the element picker's panel already roams (CanvasOverlayPortal + the frame selector), so both canvas overlays share one bounding container and their handles behave the same. While picking, the strip rises over the picker and goes inert, which is the z-order CANVAS_OVERLAY_Z already documents. Add a fold beside the grip: the chevron collapses the strip to that grip row and back, with the buttons unmounted rather than hidden. The fold is session-local state owned by EditShell, next to the drag offset, so a surface swap keeps it; nothing is persisted. Expanding a strip parked at the bottom edge re-clamps through the same bounds rule the keyboard move uses. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): align the chat timeline's left edge with the composer (#1247) * fix(agent): fence session claims while an ask_user question is outstanding (#1248) * fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249) * fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250) * docs(readme): surface the 1.0.0 user guide badges at the top (#1253) * fix(workbench): show newly created folders in the sidebar without reload (#1254) * docs(readme): add the release version prefix and drop the opt-in framing * fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255) The transcript and the composer each established their own column: their own `px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal padding values were never enough, because the two columns are centered inside different containing blocks — the transcript's is a scroll container, whose content box is narrower than the composer footer's by the scrollbar's width: transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2 composer box left = pad + (pane - 2*pad - measure) / 2 The padding cancels out of the difference and what remains is `-scrollbar/2` at every padding value, so the transcript sat half a scrollbar to the left of the composer and tuning the two paddings against each other could not move it. The column is now established once, by the nearest common ancestor of both (`chatColumn`), and the scroll viewport and the composer footer are siblings inside it that add no horizontal inset of their own. The cap carries the gutter on top of the 760px reading measure, so the text column keeps its width. The handed-over question row drops the padding that indented it past the agent's prose; framed rows keep their own inner padding, which is what a card's border sitting on the column edge means. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): lock pane-embedded classroom to edit mode (#1256) The workspace right pane painted the full learning chrome — speed control, play button, learner avatars, mic bar — for a course the agent had just created, then flipped to edit once the first scene landed. resolveStageChromeMode treated playback as the DEFAULT branch for a hosted classroom, so every shortfall fell into it: a course whose tab opens at stage_link time has no scenes yet, so currentSceneId is null and isHostedSceneEditable is false. A folded pane parked the playback root behind the fold and cross-faded it out over the pane on unfold, and a failed editor chunk dropped into playback permanently. Lock it at the pane instead of defaulting per entry path: - WorkbenchPanelProvider — the single element that mounts a classroom into the workspace — publishes editPinned (visible && !playback). Every entry path passes through it, so none of them decides. - The hosted resolution can no longer degrade to playback. Start Learning (workbenchLearning, new input, split out from pane visibility) is the one door; everything else resolves between the neutral loading shell and edit. - Stage's chrome dispatch is exhaustive on chromeMode, so the playback root is no longer the else-branch of a condition about the current scene. No flicker: chromeMode is resolved during render, and preloadEditor now answers synchronously (isEditorPreloaded) so a remount with the chunk already registered paints edit on the first frame. A failed import is no longer cached forever, so the lock cannot strand the pane. Standalone classrooms keep their stored mode unchanged. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
LING DUAN
|
146e164141 |
feat(whiteboard): add destructive runtime operations (#1173)
* feat(whiteboard): add destructive runtime operations * fix(whiteboard): preserve additive exact replay results * fix(whiteboard): support legacy code line ids |
||
|
wyuc
|
086e7626ea |
ci: cut wall-clock time and bound Playwright browser installs (#1146)
Cache Playwright browsers and Next compile artifacts, retry a timed-out Chromium download, build the production bundle before Playwright starts, and run Prettier/ESLint/tsc/i18n in parallel. Unit tests stay sequential. Skip Playwright apt deps on ubuntu-latest. Closes #1145 |
||
|
杨慎andwyuc
|
019fe6e6ac |
feat(video-export): add explicit CPU resource profiles (#1105)
* feat(video-export): add explicit CPU resource profiles * fix(video-export): preserve failure capture metrics * fix(video-export): report observed failure capture mode * docs(video-export): keep benchmark evidence out of service docs * fix(render-service): build from dated Debian snapshot --------- Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
杨慎
|
4ac70cceeb |
feat(video-export): capture self-contained static interactive HTML (#1086)
* feat(video-export): capture static interactive HTML * fix(video-export): harden interactive HTML capture diagnostics * fix(video-export): initialize runtime diagnostics in manifest * fix(video-export): close interactive capture review findings * fix(video-export): preserve packaged interactive resources * fix(video-export): address packaging review feedback * fix(video-export): harden interactive HTML packaging * fix(video-export): unify interactive asset packaging * fix(video-export): close remaining interactive packaging gaps |
||
|
xuyuanwei678
|
26d280c9b7 | fix(ci): stabilize Hyperframes lint command (#1097) | ||
|
xuyuanwei678
|
c38da84ef6 |
feat(renderer): decouple editor UI from the app (#1072)
* docs: define renderer editor task 1 integration * docs: plan renderer editor task 1 * feat: gate renderer slide editor * feat: adapt renderer editor intents * feat: connect renderer editor to slide surface * test: complete renderer intent shape fixture * fix: restore renderer editor action previews * fix: render editor alignment guides * perf: smooth renderer element dragging * feat: align renderer hidden element behavior * feat: add renderer canvas host commands * feat: add renderer canvas context interactions * test: cover renderer context menu interaction * fix: preserve renderer canvas interaction semantics * docs: design renderer text editing * docs: plan renderer text editing * feat(renderer): add ProseMirror text core * feat(renderer): add rich text editor controller * feat(renderer): render editable text in place * feat(renderer): enter text editing on click * feat: connect renderer rich text editing * feat: finalize renderer text edit lifecycle * fix: stabilize renderer text edit commits * fix(renderer): move elements from selected borders * docs: design renderer editing UI text toolbar * feat(renderer): publish editing UI contracts * feat(renderer): add controlled text format toolbar * style(renderer): format editing UI contracts * fix(renderer): address Task 2 review findings * fix(renderer): handle zero font sizes in toolbar * feat(renderer): add editing UI color picker * fix(renderer): address Task 3 color picker review * fix(renderer): commit native color input changes * fix(renderer): coordinate native color picker interaction * feat(renderer): anchor text toolbar to canvas elements * feat(renderer): compose editable canvas with text UI * feat(editor): use renderer text toolbar UI * fix(renderer): make editing UI tests type-safe * fix(renderer): harden editing UI integration * feat(renderer): add editable text UI * feat(renderer): add image and shape editing * feat(editor): extend renderer editing controls * feat(editor): add renderer chart insertion * docs: design renderer latex editor * docs: plan renderer latex editor * feat(renderer): add latex editor contract * feat(renderer): add latex editor dialog * feat(renderer): compose latex insert and edit UI * feat(editor): integrate renderer latex editing * fix(editor): consolidate latex toolbar actions * fix(renderer): show toolbar hover labels * test(renderer): cover localized latex tooltips * chore(renderer): hide latex sample palette * feat(renderer): add video editing controls * docs: define renderer audio editing * feat(renderer): render audio elements * feat(renderer): add audio editing UI * feat(renderer): add preview audio controls * feat(editor): add renderer element clipboard * feat(renderer): decouple editor UI from app * chore(renderer): release 0.0.6 * fix(renderer): clear editor CI gates * fix(editor): adapt renderer canvas to media resolution API * fix(i18n): complete French editor labels * fix(renderer): hide toolbar vertical overflow * chore(renderer): address review cleanup * chore(docs): remove planning artifacts from pr * feat(editor): add transactional core package * chore(editor): ignore local test cache * build(editor): publish core entrypoint * refactor(editor): route renderer changes through transactions * refactor(editor): split renderer editing into editor package * ci(editor): register editor package for release * docs(editor): document package dependencies * ci: rerun validation after editor release registration * test(editor): validate deletion against existing element * fix(editor): address transaction and package review findings * fix(editor): validate transactions and restore editor styles * fix(editor): protect required updates and react styles * fix(editor): validate required transaction fields * fix(editor): prevent insert popovers shifting canvas * refactor(editor): own renderer editing capabilities * fix(editor): harden editing transaction boundaries * refactor(editor): modularize editor UI integration * docs(editor): harden third-party integration * fix(editor): preserve video resize aspect ratio * fix(editor): discard text draft before deletion * fix(editor): preserve renderer picking and line presets * chore(editor): remove unused pick-layer import |
||
|
wyuc
|
3b4567076f |
refactor(generation): consume the package everywhere and delete lib/generation (#1090)
Part D of #1057 — the switch. All consumers import @openmaic/generation; lib/generation deleted (net -11,900 lines). PBL wiring preserved (single-call -> classified fallback -> injected loop), streaming outline route deduplicated onto buildOutlinePrompt, serverExternalPackages wired, and the change verified by a real-model four-kind end-to-end run against the production build with a rendered-classroom screenshot check. Closes #1057. |
||
|
wyuc
|
b64e787749 |
feat(generation): move scene generation and PBL single-call planning into the package (#1087)
Part C of #1057. Scene content/actions/widget generation, retry, action parsing, interactive post-processing, buildCompleteScene with injectable sceneId, and PBL single-call planning re-seated on AICallFn (pure core + kernel + package-relative planner assets; loop planner stays app-side behind the injected pblLoopFallback with provider/abort classification). Purely additive — app source untouched until Part D. Package 0.2.0. |
||
|
|
732c36364a |
fix(release): hand published versions to marker job (#1076)
Co-authored-by: Bryan Nathan <bryan@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
9556a035b1 |
feat(generation): move outline generation into the package (#1065)
Part B of #1057. Outline primitives, buildOutlinePrompt (single-source prompt construction with golden snapshots), logger injection, JSON repair, and the bare-Node second-consumer smoke script with CI coverage. Shim-first: no app-side behavior changes; duplicated prompt assets guarded by a CI parity check until Part D. |
||
|
wyuc
|
c0da724ec8 |
ci(skill): publish OpenMAIC skill to ClawHub (#1056)
* ci(skill): publish OpenMAIC skill to ClawHub * ci(skill): support manual ClawHub versions * ci(skill): validate manual ClawHub versions * ci(skill): normalize manual ClawHub versions * fix(skill): preserve build metadata in version checks * fix(skill): reject manual build metadata versions * refactor(skill): share ClawHub version validation * fix(skill): validate shared version checker inputs * test(skill): cover ClawHub version validation * test(skill): harden ClawHub version fixtures * test(skill): cover ClawHub checker edge cases * test(skill): cover ClawHub metadata contracts * ci(skill): harden ClawHub publish workflow * test(skill): cover ClawHub publish shell * test(skill): strengthen publish shell regressions * ci(skill): verify publish path on Bash 3.2 * ci(skill): harden ClawHub release guards * ci(skill): unify publish divergence handling * test(skill): bind publish divergence reasons * test(skill): bind stale tree inputs * test(skill): enforce stale guard ordering * test(skill): lock publish guard sequence * test(skill): enforce publish guard counts * test(skill): enforce publish job uniqueness |
||
|
wyuc
|
335180ec9c |
feat(generation): scaffold @openmaic/generation with pipeline types and packaged prompt assets (#1063)
Part A of #1057. Package skeleton (ESM, tsc, dsl/storage pattern), pipeline type contracts, prompt loader with package-relative asset resolution, 13 templates + 7 snippets byte-identical with golden tests, allowlist boundary lint, full publish/CI wiring, installed-tarball smoke coverage. |
||
|
|
de95350516 |
feat(video-export): deterministic Quiz/PBL cover cards (#985) (#995)
* feat(video-export): add deterministic Quiz/PBL cover cards Replace unsupported placeholders with authored-metadata cover cards that stay independent of learner progress, and pin dense-layout evidence for 720p/480p. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(video-export): add configurable localized cover CTA Replace the inert Start Quiz affordance with a destination-configurable CTA (NEXT_PUBLIC_VIDEO_EXPORT_CTA_DESTINATION), including Unicode/bidi display guards. Co-authored-by: Cursor <cursoragent@cursor.com> * test(ci): require Hyperframes video-export gates Lock CI/Docker contracts so cover materialization and Hyperframes sample lint stay required for export changes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ci): move HF_E2E_DIR onto e2e steps GitHub rejects runner.* in job-level env, which aborted the CI workflow with zero jobs. Keep the sample dir on the materialize/lint steps only. Co-authored-by: Cursor <cursoragent@cursor.com> * chore(video-export): tidy CI invoke and panel-box export Run the Hyperframes materialize step through pnpm exec vitest like the cover-card guardrail above it; the e2e job already pins Node 22. Stop re-exporting PBL_PANEL_DESIGN_BOX from the public barrel — only the layout browser test needs it, and it can import the emitter module directly. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> Co-authored-by: wyuc <wang-yc24@mails.tsinghua.edu.cn> |
||
|
wyuc
|
30dd236057 |
Publish validated package tarballs (#1040)
* Publish validated package tarballs * Format artifact verification scripts * Handle pnpm smoke-test arguments * Harden validated package publication * Make digest anchoring shell-portable * Upload release artifacts before tests * Anchor release integrity checks to GitHub SHA |
||
|
wyuc
|
90f5f3942b |
fix(release): dedupe the published dsl, close two release-path blind spots (#1019)
* fix(release): dedupe the published dsl, close two release-path blind spots
Three independent defects on the release path.
1. `@openmaic/storage`, `@openmaic/renderer` and `@openmaic/importer`
declared `"@openmaic/dsl": "workspace:*"`, which pnpm publishes as an
EXACT pin. The tarballs on the registry today say storage@0.1.1 needs
dsl 0.5.0, renderer@0.0.3 needs dsl 0.4.0 and importer@0.1.1 needs dsl
0.4.0, so installing storage together with renderer yields two copies
of the dsl. The dsl carries the schema, the validators and the version
constants, so a document produced against one copy can be validated by
the other's schema revision.
`workspace:^` publishes as `^0.5.0`. Under 0.x that admits 0.5.x, so a
dsl patch reaches dependents without republishing them while a dsl
minor still requires a deliberate dependent release. All three
packages take a patch version increase, both because
check-package-version-bumps.mjs requires one and because the corrected
range only reaches consumers through a new release.
The tarball smoke test now asserts the packed range, so an exact pin
cannot come back unnoticed.
2. The publish workflow's build-integrity guard ran `git diff --quiet --
packages/@openmaic`, which compares the worktree to the INDEX. The
steps before it run third-party package code, which needs no
credential to rewrite a tracked file and `git add` it, after which the
guard reported clean and the rewritten bytes were packed and
published. Both copies of the guard now compare against HEAD, the
commit actually being released.
3. `@openmaic/storage`'s PostgreSQL contract suites refuse to skip when
STORAGE_PG_CONTRACT_REQUIRED=1, but that refusal is a `throw` inside
the test modules, so it never fires if vitest stops collecting them.
Collection is decided by `packages/@openmaic/storage/vitest.config.ts`,
which is on the ignore list of publishable inputs, so excluding
`*.pg.test.ts` there turns both PostgreSQL backends off with a green
run and no version bump.
scripts/assert-pg-contract-suites.mjs reads vitest's json results and
requires both suite files to be present, passed and non-empty, keyed
on the file names rather than on a count. It runs in the publish
workflow's validate job and in the storage PostgreSQL contract
workflow.
Merging this publishes @openmaic/storage@0.1.2, @openmaic/renderer@0.0.4
and @openmaic/importer@0.1.2. That republish is the fix for (1), not a
side effect of it: the corrected dependency range exists only in a
tarball that has not been cut yet.
* fix(release): audit the contract database, pin the format version to a minor
Follow-up on cross-vendor review of the three release-path fixes.
1. The PostgreSQL assertion proved far less than it claimed. Vitest's
`assertionResults` are test CASES, not `expect()` calls, so one
`test('x', () => {})` per required filename satisfied it — and the
whole surface is reachable without a version bump, because `test/` is
on the same ignore list as `vitest.config.ts`. A `vi.mock('pg', ...)`
in the already-wired `test/setup.ts`, or a stubbed suite body, made it
print "ran against a real database" having checked nothing of the
sort.
The script now runs in two phases. Phase 1 keeps the file/status
checks. Phase 2 connects to PG_CONTRACT_URL from outside the vitest
process and requires all five tables the two backends own to exist and
to show inserts in pg_stat_user_tables. Nothing inside `test/` can
forge that. Cumulative insert counters rather than surviving rows,
because the suites clean up after themselves. The success message now
states only what was checked.
2. Validation packed with lifecycle scripts while publishing runs
`--ignore-scripts`, so committing `workspace:*` plus a `prepack` that
rewrites it to `workspace:^` passed the tarball assertion and then
published the exact pin. Both the smoke test and the workflow's
dry-run pack now pass `--config.ignore-scripts=true`, so validation
and publication have identical lifecycle semantics.
3. `workspace:^` unpins the serialized-format version that the exact pin
was silently holding. DSL_VERSION and RUNTIME_DSL_VERSION are
decoupled from the npm version by design, and storage compares them by
value across the package boundary. Under `^0.5.0` a dsl PATCH could
therefore hand a new format to an already-published storage: one
install resolves it and stamps the new version into document_stages,
another install pinned to the older patch reads that row and hard-
fails. Two installs of one published storage version, data-
incompatible.
check-package-version-bumps.mjs now requires a change to either
constant to carry at least a MINOR increase of the dsl package
version, since a caret does not cross a 0.x minor. The rule is
documented next to both constants. That doc edit is itself a
publishable change, so dsl takes a patch bump to 0.5.1.
4. The range and build-integrity guards ran only in the publish workflow,
so a pull request restoring `workspace:*` or leaving a stale generated
file failed at release time, after a version number had been spent.
ci.yml gains a cheap source-level range check
(scripts/check-internal-dependency-ranges.mjs) and the
`git diff --quiet HEAD` integrity check after its install step. The
tarball assertion stays where it is as the release-time proof of
actual published behaviour.
Merging this now publishes @openmaic/dsl@0.5.1 as well as
@openmaic/storage@0.1.2, @openmaic/renderer@0.0.4 and
@openmaic/importer@0.1.2.
* fix(release): fail the gates closed, and state what they actually prove
Confirmation round on the release-path fixes.
1. The format-version rule failed open if version.ts moved. It hard-coded
one source path and returned successfully when either revision lacked
it, so renaming the file, changing DSL_VERSION and bumping dsl by a
patch passed both checks. It now reports an error whenever the file is
absent at either revision while dsl has any publishable change. The
constant regex is anchored to the start of a line, so a commented-out
declaration cannot be read as the value, and a duplicate declaration is
an error rather than a coin flip.
2. "At least a MINOR" was wrong past 1.0.0: `^1.0.0` admits minors, so a
format change shipped as a minor would float into published dependents
exactly as the rule exists to prevent. The rule is now stated as what
it needs to be — an increase the dependents' caret range will NOT admit
— computed from the pre-change version: a minor while dsl is 0.x, a
major once it reaches 1.0.0. The error message and the docs in
version.ts say the same thing.
3. Internal declarations outside `dependencies` bypassed both range
checks. Keeping `dependencies` at `workspace:^` while adding an exact
`peerDependencies` or `optionalDependencies` entry published a second,
tighter constraint and passed. An owned package must now appear exactly
once, in `dependencies`; entries in the peer and optional fields are
rejected outright, in the source check and in the packed-manifest
assertion. The global count test is replaced by naming the three
dependents that must be present.
4. The format rule compared against the base tip, so an un-rebased branch
produced an inverted message. It now uses the merge base, matching the
`base...HEAD` form the rest of diff mode uses. Deliberately not applied
to the version comparison, where the tip is the correct reference.
5. Phase 2 had no baseline, so cumulative counters from an earlier run
satisfied it forever against a non-ephemeral database. It now captures
the counters before the vitest step and requires a positive delta.
Both workflows gained the capture step.
6. Phase 2 overclaimed. It proves rows were inserted into those five
tables during the run, not that the built PgDocumentStore and
PgRuntimeStore inserted them — `test/` is on the publishable-input
ignore list, so test code could write directly. The success message now
says exactly that, and the script documents the threat model: this
catches accidental silencing, which is the defect it was written for;
it is not a defence against someone who can merge changes to `test/`,
who can alter production code just as easily.
Also: scripts/openmaic-packages.mjs is now the single source for the
owned package list, consumed by all three checking scripts, and it
cross-checks itself against both the packages directory and
publish-packages.yml so a package added in one place and forgotten in
another fails rather than going quietly exempt. The workflow's own YAML
lists cannot import it, which is why they are cross-checked instead.
* fix(release): let the format-version source move without opening the gate
Failing closed on a missing version.ts is right, but on its own it made a
rename impossible to merge: the new path does not exist at the base
revision either, so the check refused both before and after the move.
DSL_VERSION_SOURCES is now an ordered candidate list, resolved per
revision against the first path that exists there. A rename lands by
prepending the new path in the same change, and the comparison still
happens across the move. Absent everywhere at either revision, while dsl
has any publishable change, remains an error.
* fix(release): report package-list drift instead of crashing on it
Every check in check-internal-dependency-ranges.mjs reads the shared
package list, so a list naming a package that does not exist threw an
ENOENT out of readManifest before any finding was printed. It failed
closed, but unreadably. The list is now validated first and fatally, with
its own headline.
* fix(release): anchor the caret to the base branch, and check the gates' own inputs
Final confirmation round.
1. The caret escape was computed from the merge base, which is the wrong
reference for it. With merge base dsl 0.5.1, an ordinary minor landing
on main taking the tip to 0.6.0, and HEAD at 0.6.1 with a changed
DSL_VERSION, the rule computed the escape from 0.5.1, got 0.6.0, and
accepted 0.6.1 — but dependents released against 0.6.0 publish
`^0.6.0`, which admits 0.6.1, so the new format reached them anyway.
0.6.1 is also the minimum the ordinary version check allows, making
that the default outcome rather than an unlucky one.
The escape is now computed from the HIGHEST dsl version reachable on
the base branch. Whether the format moved still comes from the merge
base, because that is a question about the branch. The comment on
mergeBaseWithHead now records that it is a no-op in both current CI
invocations (checkout uses the merge ref on pull_request, and the
before-SHA is an ancestor on push) rather than claiming a case that is
not exercised there.
2. The workflow cross-check matched raw text, so a commented-out trigger
path still satisfied it while silently disabling that package's
release. Full-line comments are now stripped first; the `on.push.paths`
list and every `for pkg in ...` loop are compared as exact sets, so an
unexpected entry fails as well as a missing one; and `--filter` usage
is checked more loosely, because individual steps legitimately filter
subsets. The limits of a textual check are stated where it lives.
3. `locate()` returned the first candidate format-version source that
existed, so during a half-finished rename the gate could compare a file
that is no longer the exported one. It now resolves every candidate per
revision and fails unless exactly one declares the constants.
4. INTERNAL_DEPENDENTS was checked in one direction only, so deleting an
entry exempted that package from the source check and from the packed
assertion that iterates the same map. It is now cross-checked both
ways, an owned package in another owned package's devDependencies is
rejected outright, and multiple owned dependencies no longer collapse
to whichever was declared last.
5. assertPackageListIsComplete() now also runs in release mode, which is
the gate that decides what gets published; and the PostgreSQL script
consumes flag values properly instead of treating the first non-flag
argument as the results path.
Also documents, in openmaic-packages.mjs, the threat model these gates
share: they catch mistakes, and anyone who can merge edits to scripts/
can edit production code directly, so deliberate subversion is out of
scope and cannot be closed by adding further checks.
|
||
|
wyuc
|
091989c48b |
feat(storage): promote KV HTTP contract + clients to main (Part A of #1000) (#1020)
* feat(storage): KV HTTP contract and clients (#1000 Part A) (#1002) * feat(storage): KV + Asset HTTP contracts and clients Adds the client half of the server backends for the two components in the storage RFC that still had a browser backend only. Asset contract: the ref is `sha256-<hex>` over the bytes, computed by one shared rule both backends call, and it *is* the address bytes are written to (`PUT /assets/{ref}`). The server re-hashes what it received and refuses a mismatch, so content addressing is enforced end to end rather than assumed, and uploads become idempotent. `resolve` returns a URL per call, either an absolute signed URL passed through untouched or a root-relative proxied path joined to the base origin — the document never stores a raw URL either way. KV contract: account scope only, with no scope path segment, query parameter or body field anywhere on the wire. `HttpAccountKV` is the only object that can reach it and none of its methods takes a scope, so a device value is not a request the transport can express; `HttpKVStore` routes `device` to a local backend it requires at construction. Tests assert the invariant both ways — `@ts-expect-error` on every scope-passing call (unused directives fail the build if a scope is ever added) and a fetch spy proving device traffic does not exist. Both clients pass the existing `runKVStoreContract` and `runStorageProviderContract` suites against an in-memory conformance server, the asset one in both the proxied and signed URL shapes. * fix(storage): close the review findings on the KV + asset HTTP clients Cross-review found the device-scope guarantee resting on its weakest layer, and several places where the contract promised more than the code or the conformance server could deliver. deviceStore: the two type-level layers only documented intent — the barrier that actually kept device values off the wire was the injected store, typed as `KVStore`, which `HttpAccountKV` satisfies structurally (its methods are the same methods minus an optional parameter). Injecting the transport as the device backend type-checked and put device values on the network. The device scope now requires a branded `LocalKVStore`, the transport declares itself not to be one, and a runtime check refuses what a cast smuggles past. resolve URLs: a media `src` carries cookies, never the client's headers hook, so the proxied shape is only sound for cookie-authenticated deployments — the contract now says so instead of claiming the content route sits behind "the same authentication", and the conformance reader uses a cookie rather than a header no browser could send. The client also validates what it resolves: http(s) schemes only, and no `//host` or `/\host` slipping through the relative branch to another origin. Identifiers: refs and keys arrive percent-decoded, so `..%2F..%2F` was a traversal waiting for the first backend to join it to a path. Both are now constrained on both sides, and the contract states the MUST that keeps a storage layer from being the first to look. Also: the JSON gate runs before anything reads the value (a `JSON.stringify` pre-flight ran caller code first and reclassified `{ toJSON: () => undefined }` as a delete); scope routing fails closed; an upload's Content-Type can no longer be relabelled by the headers hook; a mutation invalidates in-flight resolution; unparseable 2xx bodies become typed errors rather than native SyntaxErrors; signed tokens are validated rather than merely present; and the contracts settle the per-principal claim model, the media-type allowlist, the literal prefix rule, and the identifier charset that Part B would otherwise have had to guess at. * fix(storage): close the second review round on the KV + asset HTTP clients The device-scope guard was in one place and needed to be in three. The transport is exported and structurally a `KVStore`, so `kvPersistStorage(new HttpAccountKV(...), 'device')` and a plain `KVStore`-typed reference both compiled without a cast, dropped the scope argument on the floor, and put a device value on the wire. The transport now takes the scope it was pretending not to have — typed as `'account'`, refused at runtime for anything else, since bivariant method parameters mean the type alone cannot hold — and the persist adapter requires a `LocalKVStore` when paired with `'device'`. The resolved-URL checks were text tests where only parsing works. The URL parser strips ASCII tab, LF and CR before parsing, so `/<tab>/evil/x` walked past the `//` prefix check and loaded cross-origin — in the app-mounted shape, where a path-only base URL leaves that check as the only guard. The absolute branch let through `https://` with no host and `https:host/x`, which names one place standalone and another when a browser resolves it against a same-scheme page. Both branches are now decided by the parser: control characters refused outright, relative references validated against a probe origin, absolute ones required to parse as `scheme://host` with an http(s) scheme — and still returned byte-for-byte, so signatures survive. The media-type MUST had no implementation and contradicted itself: the conformance server stored and served `text/html` verbatim, and a metadata-less upload became `application/octet-stream`, which cannot be on a renderable allowlist. The server now implements the allowlist, serves everything else as an opaque attachment, and the contract scopes its "usable as an img src" promise to allowlisted types. The KV key domain was HTTP-only, which made "the same suite proves the backends equivalent" untrue: `a/b`, an empty key and a 600-byte key all passed in the browser and would have become unreachable on a server. An audit of the app's keys (learner key, storage generation, current-scene and migration markers, persist names) found none in violation, so the rules are lifted to the primitive and both backends enforce them, with the shared suite asserting it. Also: the claim model the last round wrote into the contract is now implemented and covered case by case; malformed percent-escapes are a 400 rather than a 500; the in-flight identity guard has a test that can fail; and the Content-Type conflict raises a typed error. * fix(storage): final review round — backend parity, URL probe, credentials Four small things the final pass found, plus the contract patches Part B would otherwise inherit. The `toJSON` delete semantics were documented backwards. `BrowserKVStore.set` was the trial-serialize the KV contract said no backend does, so `{ toJSON: () => undefined }` was a silent delete there and a rejection over HTTP — opposite answers to the same call. The browser backend now validates scope and key before anything can run caller code, recognizes the delete case by type, and refuses a value that still fails to serialize. Both cases are in the shared suite, so the two backends cannot drift apart again. The probe-origin check could not vouch for a URL naming the probe host itself: `//asset-url-probe.invalid/x` resolves onto the probe origin and passed, then resolves somewhere else entirely against a real base. Any authority-bearing form is refused outright, so the check no longer depends on the probe host being unguessable. Cross-origin cookie deployments had no way to send a cookie: `fetch` omits them cross-origin by default and the headers hook cannot help, since `Cookie` is a forbidden header name. Both clients accept `credentials` and pass it to `fetch`; the contract records the CORS obligations that come with it. Rather than restricting cookie deployments to same-origin, which the asset contract explicitly does not require. The conformance server's dot-segment check was reading a path the URL parser had already normalized, so `/assets/%2e%2e/x` arrived as `/x` and the documented 400 was really post-routing behaviour. It now validates the raw request target, and the injected transport preserves it instead of handing over a normalized one. Contract: the prefix rule names backslash (PostgreSQL's default LIKE escape, and the one an implementer skips because a legal key cannot contain it, though a prefix can), and acquiring a claim MUST leave the bytes present — a de-duplicating server that inserts only the claim can race reclamation of the last previous claim and leave a document resolving to a 404. Also: base URLs are checked for an http(s) scheme at construction, the ref domain moved next to the KV key domain and is enforced by both asset backends with shared suite coverage, the KV client refuses a hook-supplied Content-Type like the asset one, and CI runs the package typecheck that makes the `@ts-expect-error` guards mean something. * fix(storage): probe guard spelling, prefix validator, base url shape Three one-line-level holes from the closing review. The probe-host guard was spelled `startsWith('//')`, so the backslash forms `/\host/x` and `/\/host/x` reached the origin comparison, passed it by naming the probe host, and were stopped only by that host failing to resolve. It now refuses any second character that can open an authority — which is also what makes the comment true rather than aspirational: a path-absolute reference always inherits the base's origin, so with no authority expressible the comparison cannot be satisfied cross-origin, whatever the probe host is. The conformance server validated `?prefix` with the path-segment validator, rejecting `keys('.')` and `keys('..')` — legitimate prefixes of keys like `.hidden`, which the client rightly allows. A prefix travels in the query string and is never a path segment, so it now has its own validator: the key rules minus the non-empty and dot-segment requirements. The contract says the same, and the shared suite pins both backends to it. Base URL validation moved into one shared helper both clients call, since two copies of a security rule is how they drift. It now also refuses `https:host/path` — which parses standalone to that host but is resolved against the current document by `fetch`, sending requests and their credentials somewhere the operator never wrote — and a base carrying a query or fragment, which silently swallows the path concatenated onto it. * fix(storage): close two holes in the shared base-url check The path-only branch returned its input unexamined, so `//evil.example` and `/\evil.example` were accepted as paths. A browser reads both as an authority, which means every request built on such a base — and every credential the headers hook attaches — leaves the origin. A base is now required to be a single `/` followed by something that is not `/` or `\`, the same refusal the resolved-URL check already makes, for the same reason. Control characters are refused first: the parser strips or re-encodes them, and `/<tab>/evil.example` becomes `//evil.example` after it does. The query/fragment check read the parsed `search` and `hash`, which are empty strings for a base ending in a bare `?` or `#` — so exactly the inputs that swallow the concatenated path into a query or fragment were the ones that got through. It reads the raw string now, which also makes the post-parse check redundant rather than merely unreachable. * fix(storage): bypass HTTP caches on reads, and require a 404 for not-found Two mechanisms the clients were missing, on both sides. Reads are marked `cache: 'no-store'`. The asset resolve request answers two mutable questions — does this principal hold a claim, and which URL is valid now — so a cached answer returns an expired signed URL or replays a negatively cached 404 for an asset written since; the in-flight coalescing that would otherwise cover this lives in the client and cannot reach the HTTP cache layer. KV reads have the same problem for a different reason: `account` is by definition the scope another device may have just written, and nothing here can invalidate a cached copy when it does. The contracts now require `no-store` on those responses. Asset content is left cacheable — bytes at a content-addressed ref never change — and writes are unmarked, having nothing to read from a cache. The not-found mappings now require the status as well as the code. A proxy or gateway answering 401, 403 or 500 while echoing the body's error code was being read as "this does not exist", and `null` is indistinguishable from a real miss — so an outage reached the caller as data loss. * fix(storage): reject unusable base urls, and make the harness meet the contract Two configuration checks and two harness gaps. A base URL carrying userinfo built a client that threw a native TypeError on every operation, because Fetch refuses to construct a request from a URL with embedded credentials — one legible error where the deployment is configured beats a per-call failure with no explanation. A base with surrounding whitespace was worse than useless: `new URL` strips it, so validation passed, and then concatenation moved the space inside the URL where it is no longer trailing and no longer stripped, breaking every request. The conformance server now sends `Cache-Control: no-store` on the three reads the contracts require it on. The requirement landed last round with nothing asserting it, so the suite would have passed against exactly the implementation it exists to catch. And `DELETE /kv/entries/{key}` refuses a request body outright: the query and the header were already checked for a smuggled scope, leaving the body as the one channel through which a delete could still describe one. * fix(storage): allow a device-routing composite to persist device-scoped state cosarah found kvPersistStorage(httpKVStore, "device") wrongly rejected. An HttpKVStore is a full dual-scope composite: it routes device operations to the LocalKVStore it requires at construction and only reaches the network for account, so device values never leave the machine. The guard tested isLocalKVStore — "is the whole store local" — and so turned away a store that is safe for device precisely because it is not fully local. The guard now turns on the capability that actually matters: does the device scope stay local. A new DeviceSafeKVStore brand (servesDeviceScopeLocally: true) marks a store whose device writes never leave the device — held by a fully-local LocalKVStore (which extends it) and by HttpKVStore, and declared false by HttpAccountKV, the pure account transport with no local device backend. The persist device overload requires DeviceSafeKVStore; the account overload is unchanged. The security boundary the red team opened this whole line on is untouched: HttpAccountKV used for device is still doubly refused — its false brand is not assignable to the true DeviceSafeKVStore requires (type), and the runtime check rejects the cast that would erase it. The brand is structurally unforgeable, not a flag the transport could set; and the injected device backend of a composite still demands the stronger, fully-local LocalKVStore, so composites cannot nest into a router loop with no local floor. Regression test (cosarah's ask): kvPersistStorage(httpKVStore, "device") set/get/remove reach the injected device backend, land device-scoped there, and make zero HTTP requests. Mutation-checked: widening the brand so HttpAccountKV is accepted, or neutering the guard, turns the device-out reject test red. * refactor(storage): scope #1002 down to KV — remove the asset HTTP backend Per the maintainer decision, the AssetProvider server design moves to the global-resource-pool RFC (#1007), where AssetRef changes from content-addressed to an allocated global id and the contract model itself changes. Continuing to harden the content-addressed asset client here would be work on code that is being replaced, so the asset half of #1002 is withdrawn and the PR is pure KV. Removed (all introduced by #1002): - docs/asset-http-contract.md - src/asset/http.ts (HttpAssetProvider) and its index / package.json exports - src/asset/content-ref.ts (assertAssetRef / computeAssetRef / the ref shape) - test/http-asset-provider.test.ts and the asset half of the conformance server Reverted to the #858 baseline, untouched: - src/asset/browser.ts — back to opaque resolve/remove, no assertAssetRef guard. This directly removes cosarah's latest asset finding: an opaque ref such as `bucket/path` no longer throws, restoring the DSL AssetRef-opaque contract. - test/asset-contract.ts, test/asset-browser.test.ts — 0-diff from origin/main. Kept (all KV): kv-http-contract.md, src/kv/* (HttpKVStore / HttpAccountKV / the DeviceSafeKVStore brand), the KV tests, and the KV conformance server — renamed kv-conformance-server.ts and reduced to KV routing over the shared harness framework (body caps, no-store, scope-channel and dot-segment guards, auth hooks). base-url.ts stays; KV still uses it. * fix(storage): treat KV keys as opaque; document account-sync prerequisites cosarah found the key validation we added for traversal defense breaks legitimate callers: DSL identifiers (stageId/sceneId/learnerKey) are unconstrained strings — verified, the DSL only requires typeof === 'string' — and app callers compose keys like `editor-current-scene:<stageId>`, so a valid stage id `stage/one` now throws in BrowserKVStore before localStorage. Fix, matching the asset-opaque precedent: a KV key is an opaque string again. The `/` and `\` rejection is removed from assertKVKey / assertKVKeyPrefix and from the conformance server. Traversal is defended by encoding, not rejection — the HTTP client already percent-encodes the whole key into one path segment, so a separator never opens a new segment on the wire, and the server stores the decoded key as a plain map key (bound value), never a path. `a%2F..%2F..%2Fb` decodes to the single opaque key `a/../../b`, which addresses one entry and traverses nothing; a raw unencoded `..` path produces extra segments that match no route (404). The rules that remain are only what encoding cannot cover and the transport cannot carry (empty, whole-key `.`/`..`, NUL, lone surrogate, length) — none producible by `prefix + id`. Tests: the shared suite now round-trips keys with `/`, `\`, `%`, `:`, spaces across both backends; a `stage/one` key round-trips through the real conformance server; the traversal input is proven to store as one opaque key and the raw path to 404. Mutation-checked: re-adding the `/` rejection reddens exactly the round-trip cases. Docs (kv-http-contract.md): - "A key is an opaque string" replaces the identifier-shape section — keys may contain separators; traversal is an encoding + server-bound-value duty. - Account settings sync now documents its security prerequisite next to the account scope: a deployment MUST provide real per-user isolation of stored rows and encryption at rest before enabling sync; the reference server is not that posture. - kvPersistStorage(device) doc corrected from LocalKVStore to DeviceSafeKVStore (the device-safe brand accepts the HttpKVStore composite; HttpAccountKV stays double-rejected). * fix(storage): close the scope/key/prefix contract gaps with a coverage matrix Three completeness gaps, nailed by an executable matrix so an untested channel/shape can't slip through the next pass rather than patched one by one. 1. Scope via a GET body was not rejected. `assertNoScopeChannel` covered the query and header, but the bodyless GET routes never read their body, so `GET /kv/keys` with `{"scope":"device"}` returned 200 []. Bodyless methods (GET, DELETE) now reject any request body up front — that is the channel a scope would hide in — and only `set` reads a body, checking it for a scope field. The DELETE route's own body check folds into the shared one. 2. The 512-byte key bound wrongly rejected legitimate keys. A DSL id is an unconstrained string (verified: the DSL requires only typeof === 'string'), and `editor-current-scene:` + a 500-char stage id is 521 bytes, which BrowserKVStore rejected before localStorage. Keys are length-unconstrained; the bound is raised to a generous 8 KiB DoS ceiling and documented as a resource guard, not part of the key domain — not a derived/hashed key, which would break the literal-prefix semantics of keys(prefix). Both backends and the conformance server move in lockstep. 3. The prefix contract text contradicted itself: it claimed a legal key can never contain a backslash, while the key section (correctly) makes keys opaque, backslash included. Reworded — a prefix is opaque like a key — and the shared literal-prefix matrix now covers `\` (PostgreSQL's LIKE escape), which it had skipped, alongside `%`, `_`, and combinations. Coverage matrix (CI-executable): - scope-rejection: {get,set,remove,keys} × {query,header,body}. query/header run in-process; the body cells use a raw HTTP client (fetch cannot put a body on a GET — exactly the non-conforming client the finding described) against a loopback listener, skipped where the sandbox forbids binding. - key-validity: `/ \ % _ : .` space, BMP + astral Unicode, `%2F`, a 500-char composed key, and all-at-once — each round-trips (set/get, keys(prefix) match, remove) on both browser and HTTP backends via the shared suite. - prefix-charset: `%`, `_`, `\`, and combinations match literally, both backends. - Boundaries still rejected: whole-key `.`/`..`, NUL, lone surrogate, empty, and the DoS ceiling. Mutation-checked: removing the bodyless-body rejection reddens the three GET/ DELETE body cells; lowering the ceiling back to 512 reddens the long-key round-trip on both backends. Bumps @openmaic/storage to 0.1.2 (src/kv/* changed). * fix(storage): bound the KV key DoS ceiling by encoded size, not UTF-8 bytes Follow-up to the 8 KiB ceiling: measuring it in UTF-8 bytes ignored percent-encoding expansion and reopened a backend-parity break. A non-ASCII key is small in bytes but large on the wire — `€` is 3 UTF-8 bytes and `%E2%82%AC`, 9, encoded — so `'€'.repeat(2730)` (8190 bytes, under the old cap) encoded to ~24.6 KB, which the Node HTTP server rejects with 431 before routing while `BrowserKVStore`, which never builds a URL, accepted it. The ceiling is now measured on `encodeURIComponent(key).length` — the size the key occupies as a request target — at 4096, far above any real id (a 500-char id under `editor-current-scene:` encodes to ~523) and well below Node's default 16 KiB `maxHeaderSize` (empirically the 431 trigger on Node 22), so no key that passes validation can reach a 431. Both backends and the conformance server use the same encoded-size limit, so a key is accepted iff its encoded form fits, whichever backend it reaches — parity restored. `MAX_KV_KEY_BYTES` becomes `MAX_KV_KEY_ENCODED_LENGTH`. Tests, against a real node:http server for the 431 boundary: - the max-legal non-ASCII key (455×€, encoded 4095) round-trips over real HTTP with no 431, and the browser accepts the same key — parity on accept; - one code point over (456×€, encoded 4104) is rejected by both backends at validation, before any request — the exact parity case a byte bound missed (1368 bytes, would have passed, then 431'd over HTTP); - a raw client bypassing client validation with an over-ceiling (under 16 KiB) encoded key gets a clean 400, not a 431. Shared suite's key-domain matrix now bounds by encoded size on both backends. Mutation-checked: reverting the ceiling to UTF-8 bytes reddens the three encoded-length parity tests. Bumps @openmaic/storage to 0.1.3 (src/kv/* changed). * fix(storage): make the KV key domain truly opaque; keep transport limits out The recurring key-limit findings (512 → 8 KiB → 4096, each patch exposing the next) shared one root cause: transport-driven rules kept being added to the key domain — the browser primitive included — breaking the "opaque unconstrained string key" contract. This is the structural fix, per cosarah: "keep HTTP transport limits out of the browser primitive and preserve the existing string key domain." Approach (a), the minimal one: a KV key is any string, validated nowhere. The browser primitive is a Map/Storage that stores any key; the HTTP backend carries the key as a URL path segment and inherits that transport's limits, documented as HTTP-deployment concerns that never constrain the key domain. Every real key (a `prefix:id` from an unconstrained DSL string) round-trips on both backends; only pathological keys hit a transport limit, and none of those are keys a caller produces. (b) — keys in the request body for accept-parity — was unnecessary: verified that no real key needs it. Removed from the key domain (browser + HTTP + conformance server): - the encoded-size / length ceiling entirely — there is no key-length limit; - the NUL exclusion — NUL percent-encodes to %00, is preserved end to end, and is a legitimate Map/Storage key (a false "cannot represent" premise); - the `.`/`..`/empty/`/`/`\` rejections — all opaque, all round-trip. `assertKVKey` / `assertKVKeyPrefix` / `MAX_KV_KEY_*` are gone. Kept, at the transport layer only (HttpAccountKV), never reaching the primitive: - an unpaired UTF-16 surrogate genuinely has no percent-encoding, so the HTTP client surfaces a clear transport error (KEY_NOT_ENCODABLE) while the browser stores the same key — verified: encodeURIComponent throws on it. Scope completeness (P2): the "reject scope in every channel" guarantee had two holes — a scope path segment (`/kv/device/keys`) 404'd instead of 400, and only `x-scope` was checked among header spellings. The server now rejects a scope path segment and the whole prohibited header set (scope, x-scope, kv-scope, x-kv-scope). The coverage matrix is {get,set,remove,keys} × {path, query, each header spelling, body}. Coverage matrix reworked to verify arbitrary opaque keys (empty, NUL, control chars, separators, long, Unicode) round-trip on both backends — not to verify a limit. A real node:http test proves NUL and long keys round-trip end to end, and that a lone surrogate is a transport limit (browser accepts, HTTP client errors). Mutation-checked: re-adding a NUL rejection, removing the path-segment check, or dropping a header spelling each reddens the matching cells. Cleanup: package.json description updated (KV HTTP delivered, not "later"); network-server bind/skip/close lifecycle deduped into one helper. Bumps @openmaic/storage to 0.1.4 (src/kv/* changed). * fix(storage): reject whole-key "."/".." in the HTTP client — they alias entries Follow-up to making the key domain opaque: removing the `.`/`..` key-domain rejection was right (the browser stores them fine), but the HTTP client then let them reach `fetch`, where URL path normalization silently rewrites the target before it is sent. `encodeURIComponent('.')` is `.`, so `/kv/entries/.` collapses to `/kv/entries/` (the empty key) and `/kv/entries/..` walks up a level — so `get`/`set`/`remove` on a whole-key `.` or `..` silently read, overwrite, or delete a *different* entry, with no error. A data-correctness bug. Fix, same pattern as the unpaired-surrogate limit already handled: a whole-key `.`/`..` is something the URL-path transport structurally cannot carry, so the client's path-segment encoder refuses it with `KEY_NOT_ENCODABLE` before a request is built. Split the encoder in two: `encodeKeyPathSegment` (key → path segment, rejects whole-key `.`/`..`) and `encodeComponent` (the `keys()` prefix → query value, where `.`/`..` are legitimate and untouched). A key that merely *contains* a dot (`a.b`, `prefix:id`) is unaffected and round-trips. The browser primitive still stores `.`/`..` opaquely. The HTTP transport's limits are now complete — three cases, all fail-loud at the transport layer, all storable on the browser: (1) an unpaired surrogate (no percent-encoding), (2) a whole-key `.`/`..` (URL normalization aliases it), and (3) an over-long key (deployment request-target ceiling). The contract lists all three as HTTP-deployment limits, not key-domain constraints. Tests: whole-key `.`/`..` — browser stores and round-trips (distinct from the empty key); HTTP client throws `KEY_NOT_ENCODABLE` and builds no request; and a real-server test asserts the empty-key entry is never read, overwritten, or deleted by a `.`/`..` operation. A dot-containing key (`a.b`) round-trips. Mutation-checked: dropping the whole-key check reddens the alias tests. Bumps @openmaic/storage to 0.1.5 (src/kv/* changed). * fix(storage): reconcile the settings/profile persist consumer with the KV API The integration merge (#1002 KV into main) surfaced two things the merge itself could not resolve mechanically: - `lib/store/kv-persist.ts` (added on main by #1001) called `kvPersistStorage(kv, scope)` with a *variable* scope, which #1002's overloads deliberately reject — `'device'` must be paired with a device-safe store, so the overloads take the scope as a literal. Branch on the scope: `'account'` takes any `KVStore`; `'device'` narrows the backend to `DeviceSafeKVStore` via its brand (the adapter re-checks at runtime) and otherwise fails loud. The app wires only `'account'` today; the `'device'` arm keeps the generic helper honest. `ControllableKV` in the persist test wraps a local `BrowserKVStore`, so it is genuinely device-safe and now declares the brand. - `packages/@openmaic/storage` version bumped 0.1.5 → 0.1.6: the merge changed storage inputs (README + a pg-schema test carried from main) relative to the previous integration tip, so the version-bump guard requires an increase above #1002's 0.1.5. Still ahead of main's 0.1.1. Verified: root `tsc --noEmit` clean, the persist test green (40 passed), and the storage package suite green on Node 22. |
||
|
wyucandClaude Opus 5
|
cc95851dc5 |
chore(packages): publish storage and enforce version bumps (#998)
* chore(packages): publish storage and enforce version bumps * fix(packages): cover all publish inputs and tarballs * fix(packages): scope version guard to release inputs * fix(packages): fail closed on new package inputs * docs(packages): clarify version guard boundaries * fix(packages): harden release guard edge cases * fix(packages): align smoke peers and git inputs * docs(packages): make release smoke policy explicit * fix(packages): anchor version guard at repository root * fix(packages): report version guard setup errors * fix(packages): harden release validation * fix(ci): validate complete main pushes * test(storage): pin the exported PostgreSQL schemas DOCUMENT_PG_SCHEMA and RUNTIME_PG_SCHEMA are public API. A deployment that provisions these tables with its own migration tooling has to reproduce the DDL exactly for ensureDocumentSchema() / ensureSchema() to stay the intended no-op against an already-provisioned database. Nothing guarded that today: every statement is CREATE ... IF NOT EXISTS, so PostgreSQL silently accepts whatever table already exists under the name. A column type, a nullability, an index or a FK action can drift apart from a downstream migration without an error, and the first symptom is a store query failing in production or succeeding against the wrong types. Pin both constants verbatim, and assert every statement stays IF NOT EXISTS guarded so the ensure functions remain idempotent. The pin does not judge the DDL; it makes changing it impossible to do by accident. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): gate publishing on the version check The version check ran only in ci.yml. publish-packages.yml is a separate workflow, so it could reach `pnpm publish` on a main push, a matching tag, or a manual run with dry_run=false even when CI was red, skipped, or never ran. Run the validation inside the publish job, before publishing, in a new release mode that works for every trigger. A push range only exists for branch pushes, so release mode judges each package against the registry instead: a version that is not published yet is a release, and a version that is already published is accepted only when the package source has not moved since that release. `pnpm publish` skips an already-published version, so without that second half a drifted package releases as a silent no-op. The anchor for "since that release" is the @openmaic/<name>@<version> tag the job now writes after every successful publish, falling back to the push range for packages released before those tags existed. With neither anchor the release stops instead of guessing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): make the release gate prove only what it can Three independent reviews of the previous revision converged on one root cause: it used git tags and a push range as the record of which tree produced a published version. Neither is trustworthy. Tags are mutable and can be created or moved by hand, so "the tag resolves" was being read as "the tag is authentic"; pnpm records no gitHead; and a push range describes one push, not the origin of a release. Packages released before the scheme existed had no anchor at all, which made the tag and manual triggers fail closed from the first run with no way to self-heal, dry runs included. Drop the anchor machinery. Drift is prevented where it is provable, in diff mode at merge time, which sees every commit before it can be released. The pre-publish gate is now limited to claims a release can establish: every version it publishes is new and moves forward, an already-published version is reported and left alone, and any registry answer that is not a definitive 404 stops the release. Around it, close what the reviews found in the workflow: - real publishes only from a commit contained in main, so a tag or manual run cannot ship an unreviewed ref while holding NPM_TOKEN - one repository-wide concurrency group, so a main push and a tag push cannot each decide the same version is unpublished - publish and mark each package individually, so a partial failure stays retryable instead of leaving published packages unrecorded - fail if the build rewrites tracked package files, which would otherwise publish content that is not in the released commit - compare against every published version when ordering, and refuse to guess when the registry holds versions this check cannot order The schema pin also asserted the constants without asserting what the ensure functions execute, so those could diverge. Both are now run against a recording queryable and their exact statement sequence is asserted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): refuse to release from a tree behind the registry Simplifying the gate introduced a hole that a mutation check caught: a package whose local version is already published took the "already published, skip" path before any ordering check, so a tree rolled back to an older published version passed silently. That is not a harmless no-op, because pnpm publish rewrites each workspace dependency to the version in the tree, and a sibling released alongside it would be published declaring a dependency on the older package. Order every package against the registry first, then decide: at the highest published version it is a legitimate skip, below it the release stops. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): couple publishing to a green CI run for the same commit Second review round, on the reworked design. The headline defect was a hard blocker: `git fetch --depth=0` is not valid git (that is an actions/checkout convention), so under `set -e` every real publish aborted before the gate ever ran. Remove the depth argument; the checkout already has full history. The rest closes the gap both reviewers reached independently, that being on main is not the same as having passed the gate: - wait for this exact commit's CI conclusion and require success. ci.yml runs concurrently with this workflow on a push to main and blocks nothing, so an unpublished version whose source drifted could be released while its own version check was still running or already red. - require the commit on main's FIRST-PARENT history. Plain reachability also accepts every intermediate commit of every branch merged with a merge commit. A pull request that adds a bad tree and reverts it in the next commit leaves that tree an ancestor of main forever, and a tag pointing at it published it. - do not cancel CI runs on main. Each run validates only its own push range, so cancelling one drops the range that contained the change, and its replacement compares against the cancelled tip and sees nothing. - run the version check on every main push. A branch creation or force push reports an unusable `before`, which used to skip the gate entirely; fall back to the first parent instead. - publish with --ignore-scripts. Every package's prepublishOnly reruns the build, deleting and regenerating the dist that was just verified and smoke-tested, so npm could receive bytes nothing had checked. - run storage's PostgreSQL contract suites before publishing it. They skip themselves without a database, so the one backend that needs a real PostgreSQL was shipping unexercised. - order registry versions with real semver, prereleases included. Rejecting every non-x.y.z version meant one historical prerelease anywhere in one package's history would refuse every future release of every package. - warn when a release marker exists but points at another commit. Note what is NOT enforceable here: a workflow_dispatch or tag run executes the workflow definition from the selected ref, so a branch that edits this file can delete these checks. The boundary has to be a GitHub Environment holding NPM_TOKEN behind a main-only deployment branch rule. The job now declares that environment and the header states the required setup; the in-file checks are defence in depth. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): stop the first-parent check failing on SIGPIPE Third review round. The blocker this time: `git rev-list --first-parent | grep -qx` under `set -o pipefail`. grep exits at the first match, rev-list then dies of SIGPIPE, and the pipeline returns 141, so a commit that IS on main is read as "not on main" and every real publish is refused. It does not reproduce in a shallow clone, where main's history fits in the pipe buffer; with the full history this job checks out it always will. Count matches instead, which reads the whole stream. Also from this round: - identify the CI run by workflow file and triggering event rather than by a check-run display name. Names are not unique, so any other workflow or app publishing a check called "Lint, Typecheck & Unit Tests" could stand in for a red version gate. - fail the main-push version check when the push range is unusable instead of substituting HEAD^. A force push can replace many commits at once, so the previous commit is not the range that needs checking, and a green run here is what publishing depends on. - correct the stated limitation. It claimed the package-directory input model was exact for dsl and storage; it is not. Their dist is whatever the lockfile's TypeScript emits, and dsl's shipped JSON schema comes from the lockfile's ts-json-schema-generator, so a toolchain bump can change any of the four tarballs with no diff under the package directory. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(packages): make a merged version bump the only release input The tag trigger was incoherent. Its own documentation admitted that any @openmaic/* tag republishes the whole family and that which packages actually go out is decided by the manifests, so the tag name never participated in the decision. It was a second entry point to one decision function, it bypassed review entirely because a tag is not a diff and can be created by anyone with write access on any commit, and it cannot be guarded: environment deployment rules match GITHUB_REF, so refs/tags/* and a main-only rule are mutually exclusive. Drop it. A version bump that landed on main is now the only release input, and `@openmaic/<name>@<version>` tags are an output written after a package reaches the registry. Nothing is lost: a manual dispatch from main still republishes without a new commit, and that path is inside the protected environment. Split the workflow so the token has a boundary. `validate` holds everything that does not need NPM_TOKEN and runs from any ref, which keeps dry runs useful to contributors. `publish` is the only job declaring the `release` environment, so the token is never attached to a run that is merely validating. It rebuilds rather than sharing state, which is the right trade for a release. Also add the `actions: read` permission the CI-conclusion query needs; it was switched from the checks API to the Actions API without updating the scope, which would have failed with 403 on every real publish. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(contributing): explain when a package version must be bumped CONTRIBUTING said nothing about versioning or releases, while this branch adds a CI check that fails a contributor's PR with "publishable package inputs changed but version did not increase". A gate that rejects work without telling anyone the rule is the kind that ends up disabled. State the rule, the exact failure message, which files are exempt, and how to choose the number, with a specific warning for @openmaic/dsl: it is the contract the other packages validate against, so narrowing what an existing document may contain is breaking even when the diff is small. Also state what contributors do NOT do: publishing is automatic once a bump lands on main, and the release tag is a marker written afterwards rather than something to push. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): run renderer's tests before publishing renderer The pre-publish gate ran renderer's typecheck but never its test suite, so the one package whose behaviour is hardest to typecheck was the one shipping unexercised. It has 25 test files and 242 tests; they just were not in the filter list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(packages): keep the git credential away from package code Review finding: the publish job held `contents: write` while actions/checkout had persisted that credential, and it then ran `pnpm install` and the package builds. Any dependency or build script executed with a credentialed git remote in reach and a write scope to use it, which is a wider blast radius than publishing needs. Separate the two capabilities so no job holds both: - `validate` and `publish` run package code. Both now check out with `persist-credentials: false`, `validate` is pinned to `contents: read`, and `publish` drops write scope entirely, keeping only the `actions: read` it needs for the CI conclusion and `id-token: write` for provenance. - a new `mark` job is the only holder of `contents: write` and a git credential, and it installs nothing and builds nothing. `mark` also reconciles instead of only recording this run. A marker whose push failed previously could never be repaired before, because the release plan excludes versions already on the registry and the marker logic sat behind that plan. It now marks any registry version whose tag is missing, and leaves an existing tag alone rather than moving it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
wyuc
|
c923e50e4a | ci: retire the completed document-server-backend trunk from the PR whitelists (#981) | ||
|
wyucandCodex
|
5cef056d17 |
feat(storage): server-backed documents — HTTP contract, Postgres backend, one reference server for both stores (#979)
* ci: whitelist the document-server-backend integration trunk * feat(storage): DocumentStore HTTP contract + client + reference-server routes (1B PR1) (#975) Mirrors the runtime line's server-backed delivery for the document store: a documented /documents HTTP contract (validation/not-found/FUTURE_VERSION taxonomy; authorization stays a deployment concern via injected hooks with an any-authenticated default), an HttpDocumentStore that runs the same validators and migrate-on-read semantics as the browser backend before and after the wire, and document routes on the shared reference server — one server, both stores. The backend-agnostic document contract suite now runs against HTTP → reference server → browser store. Co-authored-by: Codex <codex@openai.com> * ci: whitelist the trunk in the storage PG contract workflow too * feat(storage): PgDocumentStore — Postgres document backend (1B PR2) (#976) Implements the full DocumentStore contract over document_stages / document_scenes / document_outlines, reusing the pure adapter split so every backend shares one aggregate semantics: saveDocument replaces the aggregate in one transaction under the stage row's FOR UPDATE lock (vanished scenes deleted, outline row added/removed — omission means deletion), incremental putStage/putScene/deleteScene take the same current-version gate as the browser backend, loads read FOR SHARE and migrate without writing back, listings never touch content, and deletes cascade by FK. The document contract suite is PG-gated the same way the runtime one is; local coverage comes from stubbed-queryable unit tests. Co-authored-by: Codex <codex@openai.com> * feat(document-store): sealed bootstrap injection seam for the document backend (1B PR3) (#977) configureDocumentStorage mirrors the runtime seam: module-level bootstrap injection sealed after first resolution, a factory that receives the app validators so a server-backed store cannot silently skip them, and reset-for-tests wiring. A deployment swaps the browser document backend for HttpDocumentStore without touching app code. Storage README roadmap updated — the document HTTP/PG backend items are done. Co-authored-by: Codex <codex@openai.com> * fix(storage): 1B final-review hardening — error classification, JSON-safety, body limits, parity coverage (#978) Unversioned legacy incremental writes classify as 400 instead of leaking a 500; document/scene reads assert JSON safety before serialization so a structured-clone value fails loud (500 NOT_JSON_SAFE naming the path) instead of silently converting; a shared streaming body reader bounds request bodies (default 32 MiB, 413 PAYLOAD_TOO_LARGE, option exposed through every handler factory); the putStage stale/future guards and corrupt-stamp listing tolerance move into the shared contract suite so all four backends prove them; both HTTP store errors retain structured details. Co-authored-by: Codex <codex@openai.com> * refactor(storage): typed document version/not-found errors replace prose classification (#980) DocumentVersionError (future | not-current, storedVersion) and DocumentNotFoundError now carry the classification the server previously regex-derived from message prose — instanceof matching mirrors the runtime sibling's RuntimeAppendConflictError pattern, so a wording change can no longer silently break status mapping. The prose fallback remains, after the typed checks, solely for third-party DocumentStore implementations. The HTTP client rematerializes 409s as DocumentVersionError for symmetry, and http.ts's private validator fields are renamed to match the public option names all three backends already share. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> |
||
|
|
d0d227424b |
feat(document): complete the document-persistence cutover — stage/scene/outline onto DocumentStore (#965)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * ci: whitelist the document-cutover integration trunk for PR checks * feat(document): widened-stage contracts + app document seam (#860 follow-up, PR1) (#961) Package: MaicDocument/DocumentStore/StageRow gain a TStage parameter (defaults keep existing users unchanged), BrowserDocumentStore accepts an injectable validateStage, and a version-guarded putStage closes the rename read-modify-write race — same current-version gate as putScene. App: lib/document/ seam — AppDocument types (currentSceneId is device state, not document data, and the app stage validator rejects it), a four-kind scene validator (slide/quiz delegate to the DSL; interactive/pbl validate envelope + discriminant binding only), and pure legacy canonicalizers (type rebound from content.type, singular whiteboard → whiteboards, outline envelope). No consumer switches off Dexie yet. Co-authored-by: Codex <codex@openai.com> * feat(document): stage-storage cutover + locked lazy migration (PR2) (#962) * feat(document): cut stage/scene persistence over to DocumentStore with locked lazy migration (PR2) stage-storage now persists through the lib/document seam: aggregate saveDocument with a mandatory outline envelope (omission means deletion in the contract; ordinary saves fall back to the stored envelope), lazy per-stage migration under a Web Lock (destination authoritative, one coherent legacy snapshot, save-verify-marker, legacy rows retained read-only; no-lock environments read legacy without migrating), currentSceneId in device KV, rename via version-guarded putStage, and a list that merges legacy-only stages until they migrate. stageOutlines' second write authority folds into the document service, making the final scene set and generationComplete one atomic commit. Co-authored-by: Codex <codex@openai.com> * fix(document-store): relocate the persistence seam out of lib/document lib/document/ is the server-side content-extraction domain (sharp, cloud SDKs); exporting the persistence seam from its barrel dragged node-only modules into the client bundle and broke the Turbopack build. The seam now lives in lib/document-store/ with its own barrel, and the extraction barrel is restored. Client-only guards become capability probes (indexedDB / localStorage presence) so injected node test environments keep working while a true server render still fails loud. * fix(document-store): no-lock write classification + lock-order inversion (PR2 fix round) Lock order is now per-stage document lock → global runtime epoch (a save never waits on a document lock while occupying the shared epoch, which deadlocked against fair-queued maintenance). Without Web Locks, mutation classifies instead of refusing wholesale: destination-backed and genuinely new documents accept the product's established lock-free/LWW risk, while a legacy-only document stays read-only so two authorities cannot fork — matching the chat-cutover-era acceptance suite, whose save assertions now verify through the public API instead of legacy Dexie rows. Co-authored-by: Codex <codex@openai.com> * style: prettier on document-store seam files --------- Co-authored-by: Codex <codex@openai.com> * feat(document): cut secondary consumers over to the document seam (PR3) (#963) ZIP classroom/video exports read stage metadata through the seam; classroom import builds one canonical aggregate under the per-stage lock and writes canonical 'whiteboards' (fixing the import-side singular-alias defect the manifest never had); backups become versioned document aggregates that finally include outline envelopes, while old {stages, scenes} backups are accepted through the legacy canonicalizers; clearDatabase now clears maic-documents and the seam's KV keys so clear-cache cannot strand documents; deleteStageWithRelatedData is deprecated in favor of deleteStageData. Four e2e seeders target maic-documents directly; classroom-interaction stays legacy-seeded to exercise real-UI lazy migration. Co-authored-by: Codex <codex@openai.com> * fix(document): final-review hardening — restore pre-images, honest markers, clear-race exclusion (3 P1 + 2 P2) (#964) - Backup rollback restores pre-existing destination documents (and imported current-scene KV values) from captured pre-images instead of deleting them - The skip-migration path only writes its marker after reconciling the legacy snapshot against the authoritative destination; a divergent snapshot logs and stays discoverable instead of being falsely certified - Migration's destination write enrolls in the global shared storage epoch (per-stage -> global order), so clearDatabase's exclusive epoch can no longer race a lazy migration into resurrecting a document - listStages drops legacy entries whose snapshot vanished mid-merge Co-authored-by: Codex <codex@openai.com> * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(document): backup covers no-lock legacy courses; clears fence stale saves (review P1s) (#968) Backup assembly appends read-only canonicalized aggregates for legacy-only stages the lazy migration could not persist (no-Web-Locks environments) — courses visible in the app can no longer vanish from an export. A device-scoped storage generation now fences document mutations: clears bump it inside their exclusive section, and any save or migration commit that entered before the clear re-reads it inside the shared epoch and fails loud instead of repopulating a wiped store. Co-authored-by: Codex <codex@openai.com> * fix(document): review round 2 — legacy-only chat backup coverage + full-width generation fence (#972) Backups now enumerate runtime chat sessions for every exported document, not just store-backed ones — a legacy-only course's chat history (the chat and document cutovers are independent seams) no longer drops out of the export; the runtime read injects an empty legacy chat store since legacy rows are collected by direct table reads, which also keeps the export working without Web Locks. The storage-generation fence now guards every mutating store method, with the exclusive-epoch delete cascades passing storageSharedLockHeld so the guard cannot self-deadlock against their own hold. --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
3eea9dc542 |
feat(runtime): complete the #869 learner-data cutover — quiz + playback onto RuntimeStore (#955)
* feat(runtime): persist quiz attempts in RuntimeStore * fix(runtime): coalesce quiz draft snapshots * fix(runtime): recover concurrent quiz attempts * fix(runtime): handle quiz completion races * fix(runtime): commit quiz review atomically * fix(runtime): dedupe concurrent quiz writes * fix(runtime): drop stale quiz drafts * fix: harden quiz runtime review recovery * fix: serialize quiz attempt identity * feat: read quiz state from runtime store * fix: persist quiz retries before resetting * fix: preserve authoritative quiz outcomes * fix: preserve legacy quiz retries during cutover * fix: reconcile legacy quiz snapshots safely * fix: drain rollover quiz write queues * fix: drain completed quiz retry queues * fix: reuse concurrent quiz retries * fix(quiz): preserve drafts across abrupt reloads * fix(quiz): recover empty retry sessions * fix(chat): abort stalled runtime state reads * fix(quiz): expose queued phases to readers * fix(quiz): retain concurrent writer tails * fix(quiz): canonicalize retry branches * fix(quiz): keep retry rollovers monotonic * fix(quiz): validate skipped retry siblings * fix(quiz): close read cutover races * test(classroom): cover legacy quiz summaries * fix(quiz): reset async consumers on scene changes * fix(quiz): close scene transition windows * test(pbl): cover launch freshness guards * fix(quiz): close cutover concurrency gaps * fix(quiz): harden retry and context freshness * fix(quiz): reject malformed legacy answers * fix(quiz): validate legacy answer values * test(quiz): cover legacy multi-answer migration * fix(merge): retire dead ChatRequestTemplate.storeState after quiz read cutover Main's three call sites built static storeState blocks that runAgentLoopFn never consumed (it always rebuilds fresh state via getStoreState); the quiz read cutover replaced that callback with the async two-phase RuntimeStore read, leaving the template field with zero consumers. Drop it. * feat(storage): conform HTTP/PG backends and reference server to RuntimeAppendOptions The quiz write path's expectedLastSeq / sessionTransition / RuntimeAppendConflictError semantics existed only in the browser backend; server-backed deployments would silently accept conflicting appends and leave completed sessions active. Forward the options over the wire, detect conflicts atomically under the PG transaction, map them to HTTP 409 RUNTIME_APPEND_CONFLICT, and rematerialize the typed error client-side so quiz retry logic works across every backend. Co-authored-by: Codex <codex@openai.com> * fix(chat): Pi single requests build storeState via the async runtime quiz read Pi bypasses runAgentLoop's per-iteration getStoreState and serializes the request template straight to /api/chat/pi, which rejects bodies without storeState. Extract the fresh-snapshot builder (async RuntimeStore quiz read with the scene-transition guard) and call it on the Pi path too. * ci: whitelist the runtime-data-cutover integration trunk for PR checks * feat(runtime): playback cutover — cursor in KV, discussion facts in RuntimeStore (#956) * feat(runtime): cut playback over to the runtime layer — cursor in KV, facts in RuntimeStore (#869) The fourth and last runtime family. Consumed-discussion facts become append-only 'playback' records folded into a set at read (at-least-once appends, no conflict machinery); the resume cursor is device-scoped last-write-wins KV per the amended #779/#869 split. sessionStorage keeps same-tab priority; KV takes over on fresh tabs/reloads. The dead Dexie playbackState machinery is retired, with a one-time lazy migration of any legacy row (cursor half + facts half) before deletion, and stage deletion now clears both the KV cursor and any unmigrated legacy row. Co-authored-by: Codex <codex@openai.com> * test(runtime): include playbackState in the stage-delete db mock --------- Co-authored-by: Codex <codex@openai.com> * fix(playback): persist discussion facts on every consumption path (#957) * fix(playback): persist discussion facts on every consumption path (final-review P0+P1s) - The engine now publishes a progress snapshot the moment a discussion is consumed (join / skip / unselected-agent auto-skip). onProgress otherwise fires before the discussion action executes and a discussion is the scene's last action, so the fact never reached persistence. - Reads fold records across ALL playback sessions in the learner partition (mergeLearner deliberately preserves same-kind sessions from both keys). - Legacy migration appends only not-yet-durable facts, so an interrupted migration resumes instead of dropping the tail. - recordConsumedDiscussion reports durability; the component drops failed ids from its observed set so a later progress tick retries (at-least-once). * test(e2e): live verification of the playback persistence chain Seeds a deterministic stage straight into the Dexie DB, starts the lecture via the canvas overlay, and asserts the full chain: discussion auto-skip appends a discussionConsumed record to maic-runtime, the device cursor lands in KV, and both survive a fresh browsing context (empty sessionStorage). * refactor(playback): consumed-discussion state is volatile by decision — cursor-only persistence (#959) Product ruling on #869's fourth family: playback learner state is front-end ephemeral UX, not learner data. A re-shown proactive card auto-skips, joined discussions' content already lives in chat runtime records, and no replay export / analytics consumer exists — so durable facts bought nothing over in-memory + same-tab sessionStorage. Drop lib/playback/runtime.ts and the RuntimeStore facts wiring; keep the device-scoped KV resume cursor (the half with real UX value), the engine's consumption-time progress snapshot (cursor freshness), and the legacy Dexie retirement (cursor half migrates, row deletes, consumed ids are dropped). * fix(review): P3 pair from cross-review — scene-id boundary + sessionTransition 400 (#966) * fix(review): scene-id boundary for quiz context + 4xx for malformed sessionTransition (P3 pair) Review findings on #955: didActiveSceneRemainUnchanged compared the active scene by object identity, so a store update reallocating the scene during the async quiz read dropped the learner's graded answers from that turn's request — the scene id is the real boundary. The records route now classifies a malformed sessionTransition as a validation failure instead of letting the store's throw surface as a 500. * fix(playback): superseded-engine cursor guard + migration write-window recheck Second-vendor review of the #959 shrink (requested after the cross-review noted it had single-vendor coverage) found: an engine orphaned by a scene switch during async lecture resume could pass the idle-only recheck, be resurrected, and publish its old scene's progress over the new scene's debounced cursor — the resume continuation now requires identity with the installed engine, and onProgress drops snapshots from superseded engines. The legacy cursor migration also rechecks KV immediately before its write so a concurrent tab's newer cursor cannot be overwritten and orphaned by the legacy-row delete. Co-authored-by: Codex <codex@openai.com> --------- Co-authored-by: Codex <codex@openai.com> * fix(review): approval follow-up P3 nits (#967) * release: v0.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(review): approval P3 nits — ISO gate on sessionTransition, dead mocks, corrupt-timestamp guard - The records route's sessionTransition guard now requires an ISO updatedAt (isIsoTimestamp), matching the sibling PATCH /status route - Dead vi.mock factories for the deleted playback-storage module dropped - A corrupt legacy playback timestamp falls back to 'now' instead of wedging migration into a permanent re-throw that disabled resume --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Codex <codex@openai.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
杨慎andClaude Opus 4.8
|
84b1907255 |
feat(video-export): service-backed MP4 render + in-app one-click export (#866) (#937)
* feat(video-export): service-backed MP4 render + in-app one-click export (#866)
Adds the last mile of classroom video export: turning the self-contained
Hyperframes project ZIP (#865) into an MP4 via an isolated render service,
one-click in-app.
- render-service/: standalone Node 22 + Chromium + FFmpeg container wrapping
@hyperframes/producer's library API. Async job model (POST /render -> 202
jobId, GET poll, GET download, DELETE cancel). Swappable JobStore /
ArtifactStore seams (in-memory + local-disk now; Redis/S3 + presigned-302
download later) so it scales horizontally without changing the HTTP contract.
Concurrency + per-user guards are config knobs.
- App integration: thin Next proxy routes under app/api/export-video/* (forward
only, no rendering) + capability probe. use-render-video.ts uploads the ZIP,
polls via runPolledTask, downloads the MP4; shared buildExportZip prefix with
the existing ZIP path. Export menu gains resolution/fps/quality selectors and
a progress bar; degrades to ZIP download when RENDER_SERVICE_URL is unset.
- docker-compose: render-service under an opt-in "video-export" profile.
- Entry is main.ts (not server.ts): the producer auto-starts its own server on
:9847 when the process entry path ends with /src/server.ts.
Verified end-to-end in the container: rendered a real 640s (10.7 min) classroom
ZIP to a valid H.264 720p + AAC MP4 (duration matches source) in ~9.6 min
(~0.9x realtime, 4-worker frame capture). Degrade path, queued-cancel + cleanup,
and per-user 429 guard all exercised. pnpm check / lint / tsc / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(video-export): global render progress store, percent+ETA UI, ring on export button (#866)
Addresses two UX issues found while driving the in-app MP4 export:
1. Progress display was raw and unfriendly (showed producer's English stage
strings like "Capturing frame 5130/19220") and had no time estimate. Now the
menu shows only "<percent>% · about <remaining> left". ETA is computed from a
recent-speed estimate (percent-per-ms over the last sample), EMA-smoothed —
which tracks the render's non-uniform pace (prep -> frame capture with a
4->1 worker drop -> encode) far better than a whole-run average, and never
shows a stale/rising ETA.
2. Switching scenes mid-render unmounted the export menu and lost the progress
(and reset the local "already rendering" ref, allowing a duplicate submit).
The whole render lifecycle now lives in a global store
(lib/store/video-render.ts), so progress survives menu close / scene switch
and duplicate submits are guarded by status. A persistent CircularProgress
ring on the export button shows live progress whether or not the menu is open.
Also fixes the progress scale: the producer reports progress as 0..100, but our
HTTP contract (and success path) is 0..1 — the service now normalizes it, so the
client no longer showed "2000%".
- lib/store/video-render.ts: new global store owning submit->poll->download,
recent-speed ETA, duplicate-submit guard.
- lib/video-export-app/use-render-video.ts: thin facade over the store.
- components/ui/circular-progress.tsx: lightweight SVG progress ring.
- components/stage/{header-controls,video-export-menu}.tsx: ring on the export
button; menu shows percent + ETA, subscribes to the store.
- render-service/src/render-manager.ts: normalize producer progress 0..100 -> 0..1.
- i18n: percent/ETA strings across all 8 locales (drops the stage-based string).
- render-service/package-lock.json: complete integrity hashes (reproducible npm ci).
Verified: ETA logic checked against the real segmented render curve (worker drop
raises ETA, encode speedup drives it to ~0); progress scale fix confirmed live
against the container (0.2 -> 20%). tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): persist render options in the store, not the menu component
Selecting 720p/24fps/draft, switching scenes, and reopening the export menu
showed the defaults again (1080p/30/standard). The selections lived in the
VideoExportMenu component's local state, which reset when the menu unmounted on
a scene switch — the running render still used the chosen options, but the UI
misrepresented them.
Move resolution/fps/quality into the global video-render store (with a
setOptions action). The menu now reads/writes the store, so selections survive
menu close / scene switch, and while a render runs the selectors reflect the
options that render is actually using. startRender() reads options from the
store instead of taking them as an argument.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): deployment correctness + resource/isolation controls (PR #937 review)
Addresses the blocking findings from wyuc's review. Output fidelity was fine;
these harden deployment and production resource/isolation boundaries.
#1 Compose advertised MP4 but couldn't render in prod:
- Capability now probes the service's /health (checkRenderServiceHealth), so a
configured-but-absent service reports disabled and the UI degrades to ZIP
instead of 502-ing.
- RENDER_SERVICE_URL is operator-supplied trusted config, so the proxy no longer
runs it through the SSRF guard — the one-command `docker compose --profile
video-export up` now works without globally weakening SSRF via
ALLOW_LOCAL_NETWORKS. resolveRenderServiceUrl() is now synchronous.
- Client degrades to ZIP on any failed submit (not only 501).
#2 Unbounded upload/queue (ZIP-bomb / DoS):
- unzip.ts bounds the archive via fflate's filter BEFORE decompression: entry
count, per-entry and total expanded size, and compression ratio.
- Proxy rejects oversized uploads (413) by Content-Length before forwarding.
- RenderManager enforces a global queue-depth cap (RENDER_MAX_QUEUE).
- All limits are env-tunable knobs in config.ts.
#3 Per-user guard was ineffective + admission ran after extraction:
- Identity is derived server-side (client IP) and forwarded as x-openmaic-client;
the service ignores any client-supplied userId, and the proxy strips it.
- Admission is split into reserve()/submit()/release(): the slot is reserved
BEFORE extraction, so a rejected caller never triggers a decompression.
Additional risks:
- Per-job wall-clock watchdog (RENDER_JOB_DEADLINE_MS) aborts + fails a hung
render so it can't hold a slot/scratch forever.
- Download proxy bounds only the time-to-headers, not the body stream, so large
MP4s over slow links no longer truncate.
- Client cancels the server job (DELETE) when a started render fails/times out.
- Compose puts render-service on an internal:true network (no host/internet
route), sandboxing the Chromium that runs the uploaded HTML; the export ZIP is
self-contained so no outbound is needed. README documents the standalone caveat.
Not closing #866: the smoke/golden-render CI acceptance criterion remains a
follow-up (see PR description).
Verified in-container: legal render 202; ZIP-bomb (entry-count + compression-
ratio) rejected 400 before any decompression; per-identity guard 429 with a
spoofed multipart userId ignored; reserve-before-extract leaves no scratch dir
on rejection; watchdog aborts an overrunning job and frees the slot. tsc / lint /
prettier / i18n pass; render-service tsc passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): real audio durations + burned-in subtitles (PR #937 review)
Two export-fidelity issues found in wyuc's deeper E2E:
A. Narration was scheduled from estimated durations, cutting audio off mid-
sentence and advancing the timeline early. The scheduler trusted
AudioFileRecord.duration (recorded only since #861), so the many existing
classrooms without it fell back to text-length estimates — measured 4.35s
average / 10.23s max underestimate across 47 clips. timeline-deps now probes
the real duration from each narration blob via an off-document <audio>
(symmetric to the existing video probe), preferring it over the stored
duration, then the estimate only when no audio asset exists. Everything
downstream (narration starts, scene/total duration, subtitle cues) re-derives
from the corrected value in the pure compiler — no compiler change needed.
B. The final MP4 had no subtitles (only H.264+AAC), and the ZIP's SRT/VTT used
the same estimated boundaries. The emitter now renders a burned-in subtitle
overlay: one caption box + a hidden div per cue, revealed/hidden by the paused
GSAP timeline at each cue's start/end (corrected timings from A), so Chromium's
frame capture bakes them in. The producer has no subtitle track of its own, so
burn-in is the v1 approach.
Verified: emitter unit tests + snapshot updated (subtitle overlay + toggle
statements, escaped text, hidden-by-default); 82 video-export tests pass incl.
the determinism red-line proxy. Rendered a synthetic subtitle project through the
container and confirmed by pixel analysis that captions appear only within their
cue window (2429 near-white px in the caption band at t=1.5s vs 0 at t=0.05s).
tsc / lint / prettier / i18n pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(video-export): subtitle layout + upload/admission hardening (PR #937 review)
Address the three P1 blockers plus actionable P2s from the
|
||
|
wyuc
|
34448beb6c |
feat(storage): server-backed runtime — pluggable backend seam, HTTP contract, Postgres backend, reference server (#939) (#946)
* ci: run PR checks against the runtime-server-backend integration branch (#939) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) (#940) * feat(storage): RuntimeStore HTTP contract + HttpRuntimeStore client (#939 Part B) - documented JSON HTTP contract for every RuntimeStore operation (server-assigned seq, learnerKey derived from auth — never trusted from the request, machine-readable error codes, idempotency notes) - HttpRuntimeStore client with injected fetch + auth headers hook; session reads (including createSession responses) migrate forward on the runtime line so an older server cannot leak stale envelopes - conformance test server bridging the contract onto the browser backend; full runRuntimeStoreContract green over HTTP plus error mapping cases * fix(storage): harden HTTP runtime backend after cross-review (#939 Part B) Cross-review fixes: - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently mangling them in transit; payload domain documented in the contract - appendRecord/listRecords responses validate via validateRuntimeRecord (server-assigned seq is no longer trusted verbatim) and listRecords sorts by seq like listSessions already sorted defensively - request headers no longer require an ambient Headers constructor when fetch is injected - conformance server classifies errors structurally (existence checks, DSL validators, version stamps) instead of regex-sniffing messages that interpolate caller-controlled ids - router preserves empty path segments so empty-key calls keep browser no-op semantics instead of shifting onto other routes - stray runtimeDslVersion/seq in request bodies are ignored (store- assigned wins), matching the browser reference; docs updated - loopback integration test exercises the real listening server - docs state the conformance server is test-only; auth-derived learnerKey is Part D's reference server * fix(storage): round-2 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard rewritten: additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; U+2028/U+2029 are accepted again (they round-trip through JSON per RFC 8259 — rejecting them lost legitimate pasted text) - conformance server classifies racing duplicate creates as 409 via a structural post-check, gates payloads and merge learner keys as 400 - segment() rejects '.'/'..' ids the URL layer would fold away - list responses must be arrays and mergeLearner's moved must be a finite number, else a typed MALFORMED_RESPONSE error * fix(storage): round-3 cross-review hardening for the HTTP backend (#939 Part B) - json-value guard: NUL is rejected in object keys too, and unpaired UTF-16 surrogates are rejected in string values and keys (jsonb refuses both; other JSON stacks corrupt lone surrogates to U+FFFD) - typed storage errors survive non-object 200 responses instead of degrading into TypeErrors on .id access - mergeLearner's moved must be a non-negative integer - conformance server classifies missing/non-object request bodies as 400 VALIDATION_FAILED - createSession/appendRecord gate the full init envelope, not only the payload (stray Date/undefined properties, NUL in ids) * fix(storage): round-4 convergence fixes for the HTTP backend (#939 Part B) - envelope JSON gate tolerates explicitly-undefined optional anchors (sceneId: undefined behaves like omission, matching the browser) - body-carried identifiers reject '.'/'..' so nothing persists that the URL layer cannot later address; conformance server mirrors it - mergeLearner keys pass the JSON-domain gate on client and server - json-value guard v4: rejects enumerable accessors (validation/serialize TOCTOU), Array subclasses and null-proto arrays, prototype-supplied array indices; sparse-array test now constructs a genuine hole * fix(storage): round-5 convergence fixes for the HTTP backend (#939 Part B) - undefined-stripping is limited to the DSL-declared optional anchors (sceneId/actionIndex/subAnchor); any other undefined member fails the JSON gate loud instead of being dropped silently - json-value guard: array and object prototype checks are realm-agnostic (chain-shape instead of identity), so ordinary values from another realm are accepted while subclasses stay rejected; shared isLosslessJsonString predicate exported for SQL key guards - conformance server merge route validates target-key addressability * fix(storage): close the toJSON prototype channel in the JSON guard (#939) toJSON is the single channel through which a prototype can alter JSON output — prototype properties never serialize and own accessors are already rejected — so refusing any value with a callable toJSON closes prototype influence on serialization entirely, including prototypes crafted to pass the realm-agnostic chain-shape check. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * feat(storage): PgRuntimeStore — Postgres runtime backend (#939 Part C) (#941) * feat(storage): PgRuntimeStore — Postgres runtime backend over an injected queryable (#939 Part C) - PgRuntimeStore implements RuntimeStore over a minimal injected Queryable (node-postgres and PGlite both satisfy it) — the package keeps zero runtime dependencies beyond @openmaic/dsl - runtime_sessions / runtime_records schema exported as RUNTIME_PG_SCHEMA with idempotent ensureSchema() - appends serialize per session via a session-row lock; MAX(seq)+1 and the insert share one transaction, UNIQUE(session_id, seq) plus bounded retry backstop non-cooperating writers - envelope semantics mirror the browser backend: version stamping, validation gates, migrate-on-read, fail-loud on future-stamped rows - full runRuntimeStoreContract green on PGlite plus PG-specific cases (concurrent-append seq atomicity, ensureSchema and mergeLearner idempotence) * fix(storage): require pinned transactions + JSON payload gate in PG backend (#939 Part C) Cross-review fixes: - withTransaction is now required; the BEGIN/COMMIT fallback is removed (on a pg Pool it spread BEGIN/body/COMMIT across different connections — no real transaction, leaked idle-in-transaction clients, broken mergeLearner atomicity; on a shared pinned client concurrent calls interleaved transactions) - single-statement deletes no longer wrap in a transaction hook call - appendRecord gates payloads through assertJsonValue: fail loud on values JSON cannot represent (Map, Date, NaN, nested undefined, NUL strings) instead of silently persisting something different - append retry also covers 40001/40P01; READ COMMITTED assumption documented at the retry loop - ensureSchema documented as create-only; redundant stage index dropped - deterministic interleaving test proves the 23505 retry path; real PostgreSQL contract lane added (postgres:16 service workflow, pg driver suite skipped locally without PG_CONTRACT_URL) * fix(storage): round-2 cross-review hardening for the PG backend (#939 Part C) - json-value guard rewritten (shared with the HTTP backend): additionally rejects -0, symbol-keyed and non-enumerable own properties, and non-index own properties on arrays; accepts U+2028/U+2029 - storage-pg-contract workflow now triggers for the runtime-server-backend integration branch and fails loud (STORAGE_PG_CONTRACT_REQUIRED=1) when PG_CONTRACT_URL is missing instead of silently skipping - loadSession distinguishes corrupt non-object rows from absent rows so getSession fails loud instead of reporting the session missing * fix(storage): round-3 cross-review hardening for the PG backend (#939 Part C) - json-value guard (shared): NUL rejected in object keys, unpaired UTF-16 surrogates rejected in values and keys - createSession/appendRecord gate the full persisted envelope through assertJsonValue, not only the payload — stray Date/undefined properties and NUL in ids fail loud instead of silently diverging from the returned record or leaking raw 22P05 - real-PG lane covers a genuine 23505 conflict from a second connection and recovery after an aborted transaction - retry-set asymmetry and mergeLearner's unbounded lock set documented * fix(storage): round-4 convergence fixes for the PG backend (#939 Part C) - NUL/lone-surrogate lookup and delete keys resolve to absent/no-op instead of leaking 22021 driver errors; setSessionStatus reports the session missing; mergeLearner from-key moves 0 - mergeLearner destination key passes the JSON-domain gate fail-loud - envelope JSON gate tolerates explicitly-undefined optional anchors, matching the browser backend - json-value guard v4 (shared) + tests for accessors, Array subclasses, prototype-supplied indices * fix(storage): round-5 convergence fixes for the PG backend (#939 Part C) - appendRecord pre-checks the session key like every other lookup path, so a NUL/lone-surrogate sessionId reports 'no session' instead of leaking a 22021 driver error - the queryable-key predicate now structurally reuses the shared isLosslessJsonString export instead of restating the string rule - undefined-stripping limited to the DSL-declared optional anchors - json-value guard: realm-agnostic prototype checks (shared) * fix(storage): close the toJSON prototype channel in the JSON guard (#939) Shared guard change with the HTTP backend branch; adds the crafted null-proto-prototype regression test. Both final-gate reviewers independently converged on this same channel. * fix(storage): probe toJSON via descriptor walk, not property read (#939) Reading value.toJSON would execute an inherited accessor, letting a stateful getter hide from validation and reappear at stringify time. The probe now walks own descriptors up the prototype chain without invoking any user code. Post-validation mutation of the caller's object graph is documented as out of scope — it is equally unpreventable for every other validated property. * fix(storage): toJSON probe mirrors JSON.stringify semantics exactly (#939) The own-most descriptor decides: absent is safe, a non-callable data value is an ordinary shadowing member and is safe, a callable data value is rejected, and an accessor is rejected because it cannot be inspected without invoking it. No daylight remains between the guard and the serializer on this property. * fix(storage): conform HTTP/PG backends to the post-#926 RuntimeStore interface (#943) The chat cutover (#926) added deleteAllRuntime() to the RuntimeStore contract while the HTTP and Postgres backends were developed in parallel against the pre-#926 interface. Implements the method on both backends (single-statement wipe on PG via the FK cascade; DELETE /runtime on the HTTP contract, documented as an operator-gated administrative endpoint), restoring a green typecheck and contract suite on the integration branch. * feat(runtime): injectable RuntimeStore backend + learner-key provider (#939 Part A) (#944) * wip(runtime): backend injection seam — pending gate verification * fix(runtime): cross-review hardening for the storage injection seam - stage-deletion IndexedDB probe applies only to the default browser backend; an injected store always receives deleteStageRuntime - explicit kv argument takes precedence over the configured learner-key provider, matching the store seam's explicit-beats-global rule - configured learner-key resolution is latched with in-flight dedup, mirroring the store singleton; identity changes require app-level handling - factory retry semantics documented; seal errors explain the module-level bootstrap requirement; isRuntimeStorageConfigured() probe and a test-only reset added - client-bootstrap-only contract documented (SSR/HMR caveats) * fix(runtime): snapshot configuration and make the test reset complete - configureRuntimeStorage copies the option fields so mutating the caller's object after configuring cannot swap the sealed backend or identity provider - resetRuntimeStorageForTests now clears every latched consumer cache (store singleton, learner-key in-flight promise) via a reset-hook registry, so a reset-then-reconfigure test actually gets the new backend * fix(runtime): reset also clears the default learner-key caches resetRuntimeStorageForTests left defaultInFlight/defaultKv latched, so a default-path test could leak its anonymous key or KV store into the next test despite the documented full-reset promise. * feat(storage): runtime reference server — auth-derived learnerKey (#939 Part D) (#945) * wip(storage): reference server — pending deleteAllRuntime route + gates * feat(storage): runtime reference server — auth-derived learnerKey enforcement (#939 Part D) - createRuntimeHttpHandler(store, options) wires the documented HTTP contract onto any injected RuntimeStore over node http - authenticate is required; every learner-scoped operation verifies the path/body learnerKey against the authenticated principal (403 FORBIDDEN_LEARNER) — the client-supplied value is never trusted - mergeLearner requires an explicit authorizeMerge grant and admin surfaces (stage cascade, DELETE /runtime) require authorizeAdmin; both default-deny - runnable reference entry demonstrates a pg Pool withTransaction and bearer-token authentication, marked as demo-only - contract suite green through HttpRuntimeStore -> listening reference handler -> PgRuntimeStore(pglite); security matrix tested (401/403 paths, admin default-deny); threat model documented * fix(storage): cross-review hardening for the reference server (#939 Part D) - principal learnerKey is optional: admin/merge-only credentials no longer fabricate learner identity; learner-scoped routes 403 without ownership - full session/record envelopes pass the JSON-domain gate at the handler, so NUL/lone-surrogate identifiers map to 400 instead of 500 - payload validation follows the injected store's validator map (options.payloadValidators) instead of imposing DSL defaults - reference factory accepts authenticate/authorizeMerge/authorizeAdmin/ payloadValidators overrides; docs no longer claim the factory binds to localhost; demo-impersonation warning hardened - 500 responses carry a generic message; details go to the server log - ownership checks precede version checks and unowned sessions read as 404, closing existence/version oracles; cross-learner denial matrix tested per route - merge/delete concurrency documented as linearizable-equivalent with the ownership re-check narrowed to the delete call * fix(storage): align reference-server semantics with the store contract - future-stamped sessions read and delete through unchanged; 409 FUTURE_VERSION applies only to guarded writes (status, append, merge) - check-then-write races reclassify structurally via a post-failure re-fetch: missing session 404, non-active session 400, never a message-sniffed or generic 500 * fix(storage): close the reference CLI's pg pool on startup failure ensureSchema opens connections inside createReferenceRuntimeServer, so a failed schema init or occupied port left the pool holding database resources until its idle timeout. * fix(storage): close the records-route existence oracle cosarah's review point on #946: the records list answered 200 [] for an absent session but 404 for another learner's, so the 404 leaked that an id exists. Absent and foreign sessions now answer identically (404 SESSION_NOT_FOUND) and the HTTP client maps that code back to an empty list, preserving the store contract's absent-lists-as-empty semantics. Contract doc records the server MAY/SHOULD and the client MUST. |
||
|
wyucand杨慎
|
6f7ffa8302 |
chore: tighten GitHub issue intake (#921)
* chore: tighten issue intake * fix: address issue intake review feedback * test: preserve scene selectors in pro mode --------- Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com> |
||
|
wyuc
|
1c5078842a |
feat(storage): RuntimeStore — sessions + append-only records, browser backend (#869 Part B) (#880)
* feat(storage): RuntimeStore contract — interface + backend-agnostic suite (#869) * feat(storage): BrowserRuntimeStore — IndexedDB backend for the runtime layer (#869) * test(storage): runtime version-skew coverage; run the storage suite in CI (#869) Backend-specific tests seed raw session rows (mirroring the document backend's reStampStage) to cover the version-skew surface the public API cannot express: a stale stamp fails loud while the runtime ladder is empty, a future stamp reads through unchanged but rejects appendRecord and setSessionStatus, and a stamp-stripped row hits the no-unversioned-epoch guard. Also covers the injected payloadValidators override (the contract case deferred to the backend test, as document-contract does for scene validators). The @openmaic/storage suite — including the pre-existing DocumentStore / KV / asset contracts — never ran in CI at all; add the missing step after the dsl one. README gains the RuntimeStore section. * fix(storage): harden RuntimeStore writes and listings per cross-review (#869) - mergeLearner now takes the same gates as every other write: rejects empty learner keys, self-merges return 0 without a transaction, future-stamped rows throw (mutating a newer client's data), and every re-keyed envelope is validated before put — any throw aborts the whole merge atomically instead of contaminating the target partition. - listSessions sorts by the instant a timestamp denotes (Date.parse), not by string order, which mis-ranks numeric zone offsets; and it tolerates corrupt rows by omission (the listDocuments precedent) so one poison row cannot make a partition unenumerable — direct reads stay fail-loud. - appendRecord validates the ACTUAL completed record (with the assigned seq) inside the transaction, on top of the pre-flight, and derives seq from a key cursor's primaryKey instead of deserializing the previous record's payload. - Docstrings now enumerate the corrupt-stamp throw paths and no longer claim every query is partition-scoped (mergeLearner is the one deliberate cross-stage sweep). - The stale-row test comment no longer promises an impossible one-line flip; the IDBKeyRange shim moves to test/setup.ts alongside the other guarded shims. * fix(storage): mergeLearner migrates stale sessions before re-keying (#869) The re-key previously spread the RAW stored row, unlike setSessionStatus and appendRecord which migrate a stale session in place before mutating it. Once the runtime ladder gains a real step, that would either write an old runtimeDslVersion into the target partition or fail validation when a migration adds required fields. Build the updated row from migrateSession(row) instead (order: future-guard, migrate, re-key, validate, put). Observable today (empty ladder): a below-epoch stale row makes the merge abort with the ladder's no-migration-path error — pinned by a new backend test with nothing moved. * fix(storage): reads gate stored sessions through envelope validation (#869) Reads previously ran only version resolution, so a row with a valid runtimeDslVersion but corrupt other fields (a non-ISO createdAt, a bogus status) came back as-is — contradicting the documented corrupt-row semantics, which covered only version corruption. Extend the same policy to the whole envelope: getSession validates the migrated row and fails loud as a stored-row integrity error; listSessions omits rows whose envelope validation fails, exactly like version-resolution failures. setSessionStatus and appendRecord need no change: the status update validates the full post-update envelope before writing (catching corrupt non-status fields), and appendRecord's own writes are already validated. |
||
|
|
83fdecf399 |
feat(dsl): runtime envelope — RuntimeSession + RuntimeRecord + core-kind skeletons (#869 Part A) (#870)
* feat(dsl): runtime envelope — RuntimeSession + RuntimeRecord + core-kind skeletons (#869)
* feat(dsl): runtime envelope validators (#869)
* test(dsl): runtime envelope rides the version ladder; document the runtime contract (#869)
* fix(dsl): harden runtime envelope validators and tighten contract linkage (#869)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): isQuizAttemptSkeleton requires a plain-record answers map (#869)
* fix(dsl): review round — ISO 8601 timestamps, dedicated runtime migration ladder, payload excludes undefined (#869)
* fix(dsl): mechanical version-line separation + ISO day-overflow check (#869)
- Runtime sessions now stamp their own envelope field (runtimeDslVersion,
RuntimeVersioned) instead of sharing dslVersion: the document and runtime
ladders read disjoint fields, so misrouted data is inert rather than
silently reinterpreted once the two lines diverge. Adds
runtimeDslVersionOf + needsRuntimeMigration as the runtime-line
counterparts of dslVersionOf + needsMigration.
- isIsoTimestamp: Date.parse normalizes day-of-month overflow on V8
(2026-02-30 parses, rolls to Mar 2) instead of rejecting it; add a pure
calendar round-trip so the verdict is correct and engine-independent.
- Report an empty-string timestamp once (required-field table), not twice.
- Scope migrateRuntime's contract to RuntimeSession; records are not
independently versioned units.
- Run the @openmaic/dsl test suite in PR CI (it previously ran only in the
publish workflow).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): isIsoTimestamp validates calendar components without Date.parse (#869)
* test(dsl): cover month-00 / day-00 in isIsoTimestamp negatives (#869)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): cross-line guard makes misrouted data inert across migration ladders (#869)
* fix(dsl): mirror the cross-line guard in the migration predicates (#869)
runLadder's cross-line guard returns a misrouted aggregate untouched, but
needsLadder still reported it as needing migration (its own stamp absent
reads as 0.0.0), so a correctly paired
`while (needsMigration(x)) x = migrate(x)` loop on a runtime-line object
would never terminate. Give the predicates the same guard, restoring the
predicate/runner agreement invariant the docs promise.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): fail loud on ambiguous cross-line envelopes (#869)
The guard treated raw sibling-key presence as proof of misrouting, but the
same byte state also arises as this line's data carrying a stray foreign
stamp — e.g. a valid session that once passed through the document runner.
Silently returning it unchanged permanently orphaned that data from its own
line (predicate reports current forever, runner never lifts it).
The state is undecidable from the envelope, so stop guessing: both runners
and both predicates now throw the same cross-line error on
own-stamp-absent + sibling-stamp-present, mirroring the malformed-stamp
rule. validateRuntimeSession additionally rejects any present dslVersion
on a session at the door, so the ambiguous state cannot enter storage
through a validated write path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): enforce the cross-line rule in the shared envelope reader (#869)
dslVersionOf / runtimeDslVersionOf still read a sibling-stamp-only
envelope as unversioned while the runners and predicates threw on it, so
an authoritative version read (e.g. storage's putScene guard messages)
could report 0.0.0 with a load-and-save remediation for a row the load
path refuses to migrate. Move the cross-line throw into versionOf itself:
the plain readers, the needs*Migration predicates, and the migration
runners now give one answer for one envelope, and the duplicated guard
sites in runLadder/needsLadder collapse into the shared reader.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): the runtime line has no unversioned epoch — sessions are born stamped (#869)
An unstamped object read as 0.0.0 let the runtime ladder lift arbitrary
document-shaped data; once a real session transform lands that walk
corrupts or throws. The runtime envelope is brand new — nothing
legitimately predates it and stores stamp runtimeDslVersion at write —
so an unstamped object on the runtime line is always a bug (a misrouted
legacy document or an unstamped producer write), not legacy data:
- RUNTIME_DSL_MIGRATIONS drops its legacy-lift entry (empty until the
first real shape change appends a step from the pinned initial version).
- migrateRuntime / needsRuntimeMigration / runtimeDslVersionOf throw on
an unstamped object (versionOf gains a per-line legacy version, null
meaning no epoch). The document line keeps its legacy lift — real
pre-versioning documents exist.
- RuntimeSession requires runtimeDslVersion (type and validator), so an
unstamped session cannot pass a validated write path and the
type/validator contract stays aligned.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(dsl): pin the future runtime ladder's first step to the initial version (#869)
A non-empty RUNTIME_DSL_MIGRATIONS must start at
INITIAL_RUNTIME_DSL_VERSION — the runtime line has no unversioned epoch,
so a ladder starting earlier (a copy-pasted 0.0.0 legacy lift) would
reintroduce the lift-arbitrary-unstamped-data hole. The contiguity test
now asserts the pinned origin as well.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(dsl): restore the empty runtime ladder — drop swept-in local fault injection (#869)
|
||
|
xuyuanwei678
|
261fddb062 |
fix(packages): add openmaic repository metadata (#813)
* fix(packages): add openmaic repository metadata * ci(packages): publish on openmaic manifest changes |
||
|
wyuc
|
7cb129183d |
chore(packages): publish the @openmaic/* SDK family to npm (#778) (#780)
* chore(packages): publish the @openmaic/* SDK family to npm (#778) Prepares the @openmaic/{dsl,renderer,importer} family for its first npm publish, and moves the SDK packages onto the @openmaic scope. Why the scope move: the @maic org name is unavailable on npm (an unscoped `maic` package already holds the name), so @maic/* is not claimable. @openmaic matches the project name, the scope is free, and the repo already ships an @openmaic/docs package — so the SDK family now lines up with that convention. - rename @maic/{dsl,renderer,importer} -> @openmaic/* across packages, the workspace glob, the package dir, and all import sites; lockfile regenerated - renderer: add publishConfig (public, registry.npmjs.org) — was missing, so a scoped publish would default to the wrong registry / restricted access - importer: add a files allowlist (dist, README, LICENSE) and drop the fragile .npmignore blacklist that shipped src; add an exports map so ESM consumers resolve dist/index.js instead of falling back to the .cjs main - all three: add a prepublishOnly build (+ test/typecheck) guard so a publish can never ship a stale or empty dist - add a tag-triggered publish workflow with npm provenance, pinned by name to the three @openmaic packages so the vendored forks (mathml2omml, pptxgenjs) are never published Refs #778, #720 (Phase 1). * fix(packages): address cross-review on the @openmaic publish prep Cross-review (Claude /code-review + codex) on this PR surfaced: - renderer's advertised CJS entry was broken: it keeps @openmaic/dsl external and imports a runtime enum from it, but dsl is ESM-only (no `require` condition), so `require('@openmaic/renderer')` would throw ERR_PACKAGE_PATH_NOT_EXPORTED. Make renderer ESM-only: drop the `.cjs` rollup output, `main` now points at the ESM build, and the `require` conditions are removed from `exports`. (importer is unaffected — it bundles dsl, so its CJS build still works.) - prepublishOnly re-ran the test suite during `pnpm -r publish`, so a flaky test after dsl had already published gave a non-atomic partial release. Reduce prepublishOnly to a build-only guard (never ship stale/empty dist) and move the real test/typecheck gate into the workflow, before any publish. - document that an @openmaic/* tag publishes the whole family via `pnpm -r` (pnpm skips already-published versions); the tag is a release marker, not a per-package gate. Verified: dsl + renderer + importer build; renderer emits ESM only (0 .cjs), all exports entries resolve; `npm pack` ships dist + README + LICENSE with no src leak; frozen-lockfile passes. Refs #778. * style: reflow @openmaic/dsl type imports past print-width after rename The @maic -> @openmaic rename lengthened two single-line type imports past prettier's 100-col width; prettier --check flagged them. Pure formatting. Refs #778. * docs(importer): mark @openmaic/importer browser-only (cr-loop accepted limitation) codex cross-review flagged that the published @openmaic/importer throws `XMLHttpRequest is not a constructor` when loaded in a pure Node process — its rollup build is browser-targeted (`nodeResolve({browser:true})` + a browser pdf.js build). The app only consumes it client-side ('use client'), so this is by design. Document it as an accepted limitation: prominent browser-only note in the README and a `browser` field in the manifest. Refs #778. |
||
|
wyuc
|
1d1ce80e04 |
feat(maic-agent): editor-agent line (v0) — Pro-mode "Edit with AI" (#777)
Promote the editor-agent line to main: read-then-act editor agent (read_scene_content, regenerate_scene, regenerate_scene_actions, edit_interactive_html), a routable maic-agent model stage, the agent sidebar UI, interactive-scene runtime-error capture, and the timeline/script editor. Cross-reviewed and smoke-verified; see #777. |