4 Commits

Include renames

Author	SHA1	Message	Date
3 people	fa47efe1b3	feat(persistence)!: server-backed persistence only, with a one-way legacy browser import (#1710) ... * ci: run CI for the persistence-default integration branch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(docker): start PostgreSQL by default and add a single-user owner * feat(docker): start PostgreSQL by default and add a single-user owner `docker compose up` now runs a server-backed app against a bundled PostgreSQL, started only once PostgreSQL is healthy and published on 127.0.0.1 only, with a new built-in singleUser owner auth method: every request resolves to one fixed owner that may publish, and no anonymous cookie is minted. Single-user mode (OWNER_SINGLE_USER=true, OWNER_SINGLE_USER_ID default "local") runs with or without ACCESS_CODE. Without one, the server logs one prominent startup warning that anyone who can reach it shares, edits and can delete the single library; it never inspects request peers or forwarding headers. It excludes PERSISTENCE_SHARED_OWNER_ID and follows the sharedTeam registration rules. Its principal gets a claim candidate, so a browser's earlier anonymous work is claimed into the single owner (OWNER_CLAIM_TRIGGER=auto in the Compose defaults). Compose defaults live in docker-compose.defaults.env, read before .env.local so they can be overridden. The app warns when its declared published address (OPENMAIC_PUBLISH_ADDRESS, passed by Compose) is not loopback while PostgreSQL uses the default password. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(docker): keep claims explicit and let .env.local set DATABASE_URL - Compose no longer defaults OWNER_CLAIM_TRIGGER to auto: an irreversible merge of every browser's anonymous library into the single owner must not happen on upgrade. The single-user principal still gets a claim candidate, so POST /api/identity/claim works; the docs explain it and warn about auto on a deployment several people used. - The bundled DATABASE_URL moves to docker-compose.defaults.env, read before .env.local, so an external database or a rotated password set there keeps working. The password is inserted unencoded, so it must be letters and digits (documented). - Upgrade docs no longer claim browser-stored courses are copied as they are opened: they stay in the browser, are not deleted, and are moved by the automatic browser-to-server migration shipped with this work. - Single-user mode without ACCESS_CODE now logs one warning instead of two; the docs note that the single owner id is permanent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence)!: server-backed persistence is the only persistence (#1707) * feat(persistence)!: make server-backed persistence the only persistence Courses, folders and folder membership, chat history and learner runtime, and generated media are now always stored on the server through the embedded /api/persistence endpoint. The browser keeps no durable data of its own. - Remove the build-time NEXT_PUBLIC_PERSISTENCE switch: the client bootstrap always configures the HTTP document, runtime and asset seams, and the Dockerfile, docker-compose.yml, the Pi route and the whiteboard runtime no longer read it. Unconfigured seams refuse to resolve instead of falling back to IndexedDB, and the learner key is always the server-derived one. - Remove the browser write paths: the browser document, runtime and asset stores as backends, the browser-mode branches of stage, folder, chat, media, narration and import storage, the Dexie backup export/import and the whole-database clear. The library lists through /api/stages, folders through /api/folders, and membership now goes through /api/folders/members. Regeneration always forks to a fresh asset. - Device-local state moves to its own IndexedDB database (lib/device-storage, maic-device-cache): the local media and narration cache and refused-bytes retention, staged PDF images, undo history, browser voice profiles (carried over once from the old database) and the auto-voice cache. "Clear Local Cache" clears exactly that and no longer claims to delete classrooms or chat history. - Keep the pre-server data readable for the one-way importer: the Dexie schema and read-only accessors for its tables and for the browser document, runtime and asset databases live in lib/legacy-browser-storage, which nothing on the regular paths reads and which has no write path. - Require a database: the server exits at boot without DATABASE_URL, with a message naming `pnpm db:up` and `docker compose up`. `pnpm db:up` / `pnpm db:down` start and stop the Compose postgres service alone, published on 127.0.0.1 through docker-compose.db.yml; .env.example carries the matching local DATABASE_URL. - E2E specs seed their courses through the persistence endpoint, each under a fresh id, instead of writing IndexedDB. - Guard tests pin the boot requirement, the absence of any NEXT_PUBLIC_PERSISTENCE read, and the legacy-storage boundary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the E2E suite against PostgreSQL The app refuses to start without DATABASE_URL, so the E2E job gets a postgres:16 service and a job-level DATABASE_URL. The unit, storage and render-service jobs are unchanged and need no database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: document always-on server persistence and the database requirement README (English and Chinese), the deployment guide in every locale, the extension cookbook and the changelog now describe PostgreSQL as required: `pnpm db:up` for local development, `docker compose up` for a deployment, and an external database for Vercel and other serverless hosts (the Deploy button prompts for DATABASE_URL). They list what stays in the browser, drop the NEXT_PUBLIC_PERSISTENCE instructions, and say that courses an earlier browser-only build stored stay in the browser until the one-way importer moves them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(pbl): use the server-derived learner key for PBL v2 runtime PBL v2 synchronization, hydration and drain resolved the learner key from their default watermark KV store, which reads or mints a device key in localStorage instead of asking runtime configuration. The runtime store is the server one, which refuses any key but the owner's, so saving a course with a PBL v2 scene failed before the document write, and the minted key landed in the slot the one-way importer reads as the pre-server runtime partition. The learner key now comes from getLearnerKey(args.kv): the configured server-derived key, or an explicitly injected store. The default KV store still holds the drain watermarks and nothing else. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): pin that clearing the cache keeps legacy databases - Seed the four pre-server databases (MAIC-Database, maic-documents, maic-runtime, maic-asset-pool) with real rows, run Clear Local Cache, and assert every database and row survives; assert the device database name differs from every legacy name. - The legacy-module import rule now also catches relative static, side-effect, dynamic and require imports at any depth. - New rule: the runtime learner key is never read from a KV store the caller made up, only from configuration or an injected store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(dev-db): run the development database as its own Compose project `pnpm db:up` / `db:down` shared the checkout's default Compose project with `docker compose up`, so they recreated or stopped a running stack's database. docker-compose.db.yml now declares its own project (`openmaic-dev-db`), which gives the development database its own container and data volume; it no longer touches the stack, and the two do not share data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: start the development database before `pnpm dev` CONTRIBUTING, the getting-started guide in every locale and the startup modes reference now run `pnpm db:up` and set DATABASE_URL before `pnpm dev`. README, the deployment guides, the cookbook and the changelog describe `pnpm db:up` as a separate development database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(persistence): sharpen the legacy-storage and learner-key guards - The Clear Local Cache test closes its seeding connections to maic-documents and maic-runtime, so a regression deleting either fails on the assertion that names the database instead of by timeout. - The legacy-module and Dexie import rules accept comments inside import() / require(), so a webpackIgnore hint no longer hides one. - The learner-key rule also flags an injected-looking name that is bound to a default local store in the same file; its remaining limit is documented, with the behavioural PBL test as the guard. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * build(dev-db): pin the development database's Compose project `pnpm db:up` / `db:down` now pass `-p openmaic-dev-db`, which takes precedence over COMPOSE_PROJECT_NAME from the shell or a .env file; the file's `name:` alone could be overridden and point the scripts at a running stack's database. The compose test pins the flag. The docs (all locales) and the file header say the development database is shared by every checkout on the machine, and how to run a separate one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * feat(persistence): import legacy browser data into the server, once (#1708) * refactor(persistence): prepare the seams the legacy browser importer uses - Legacy module: read-only accessors for the auto-voice cache, the course ids the old media and narration tables name, the pre-server learner key, and asset bytes from the browser asset pool. - Narration adoption exports its row ownership rule so the importer applies the same one to rows of the old database. - The quiz legacy migration is split into a non-deleting core (importLegacyQuizSnapshot) and the regular path, which still deletes the keys it migrated. - The lazy document migration exports its snapshot canonicalizer. - Clear Local Cache keeps the importer's per-owner ledgers, as it keeps the pre-server learner key. - A library-changed window event lets an open library list courses that arrive in the background. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(persistence): import legacy browser data into the server, once On the first load after the upgrade, the client moves what earlier builds kept in this browser to the server, for the owner the server resolves: courses from the browser document store and the original tables (with chat, learner runtime, playback position, roster, folders and membership, pre-runtime quiz state), their media bytes through commitToPool and the existing write-back funnels, the bytes of server courses from earlier opt-in server builds that only the old tables hold, and device-only rows (failure records, refused bytes, auto-voice clips) into the device cache. It is automatic and silent, runs after the page is idle, never writes to a legacy store, and records every step in a per-owner ledger so it resumes after a crash and never imports twice. Runs are serialized across tabs with Web Locks. A course the owner already has stays authoritative; one whose id another owner holds is imported under a derived fresh id; one the owner deleted on the server stays deleted. Transient failures retry on a later load with backoff; permanent ones are recorded per item. The module is temporary and self-contained; its README lists the removal steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): pin that each owner keeps its own import ledger Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): run the legacy import against the app routes and a browser A PostgreSQL suite routes every request the importer and the app seams send to the real persistence, library and folder handlers, owner resolved from the anonymous cookie: a full import, a course another owner holds, a course the owner deleted, and an existing course whose media is filled in. An end-to-end spec seeds an old pre-server database in Chromium, loads the app, and checks that the course reaches the library with its narration uploaded, stays in the browser untouched, and is visible from a fresh browser context with the same owner cookie. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: describe the one-way import of legacy browser data The README (and its Chinese version), the deployment guide in every locale and the changelog now say what the importer does instead of announcing it: courses an earlier browser-only build kept in the browser move to the server automatically on the first load after the upgrade, and the browser copy stays untouched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): probe server assets through the shared asset-URL owner The importer asked the pool directly whether an id exists, which the asset-URL ownership boundary forbids outside use-asset-url. It now uses assetRefExists, the existing metadata-only probe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): guard the importer's pool probe like every other entry point Only a reference the pool could have issued is probed, as the lease guard requires; the importer is added to the list of guarded entry points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): import legacy browser data once per browser Review round 1 of the legacy browser importer. - Once per browser, not once per owner: the data belongs to whoever used the browser before the upgrade, so the first owner the import runs for claims it and any other owner gets nothing. The ledger is one key, maic:legacy-import:v2, recording the owner only as a SHA-256 digest (an anonymous owner id is a bearer credential) and a random salt. - Handoff: when the claiming owner is retired (403 OWNER_RETIRED), or the current owner holds courses or folders the importer created (a claim moved them), the current owner continues the unfinished items; courses already done are never imported again, so a claim neither duplicates nor resurrects them, and a half-copied course keeps its runtime. - Fresh ids come from the salt and the legacy id through SHA-256, not from the owner: unlinkable, unpredictable, and the same in every tab. - 401 pauses with backoff instead of closing the import; 403 FORBIDDEN_LEARNER stops the run and leaves the item pending. - Without Web Locks: ledger writes merge with the stored copy, and the library is listed again right before a course is placed, so a second tab does not import a course the first one just created. - Media the server refuses for good gets the app's failed-media record instead of a dangling reference; the legacy bytes stay. - A legacy whiteboard or PBL session is not created when the server already has an active one of that kind. - A folder that is gone leaves the course unfiled instead of failed, and a membership naming a folder the old database lacks counts as unfiled. - Narration rows from before the course column are adopted when exactly one legacy course names their key; unconvertible chat rows are noted; an unreadable document-store copy falls back to the original tables; PBL v2 documents go through the save-path strip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover the importer's handoff, tabs, refusals and review gaps Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: say the legacy import runs once per browser The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say that the first owner to load the upgraded app in a browser claims its legacy data, what the handoff to an account does, that the ledger holds no owner id, and how refused media and the new pause and skip cases behave. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): note runtime the importer cannot find without the old learner key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(identity): let an owner confirm it absorbed a given owner through a claim GET /api/identity/merged-from?salt=<hex>&digest=<hex> answers whether a claim merged into the requesting owner an owner whose id hashes to SHA-256(salt, id). It reads only the requester's own owner_merges rows, never lists or names an owner, resolves the owner like every other owner-scoped route, and is uncacheable. The one-way import of pre-server browser data uses it to decide whether a different owner may continue an import the first owner started. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): hand the legacy import to another owner only on a confirmed claim Review round 2 of the legacy browser importer. - The only way the browser's legacy data moves from the owner that claimed it to a different owner is the server confirming, through GET /api/identity/merged-from, that the current owner absorbed that owner through a claim. The inferred signals (an owner that never wrote, a library listing that holds imported courses) and the retired flag are gone; a 403 OWNER_RETIRED only stops the run and leaves items pending. - The first owner is bound only once the run's first authenticated request of its own (the library listing) succeeded, so a run that dies before that leaves the ledger unclaimed. - The owner is recorded as SHA-256 of the per-browser salt and the owner id; the server computes the same value from the salt the browser sends. - A "not absorbed" answer is reused for an hour instead of asking on every load, and the claiming owner stored first survives another tab's save unless that tab handed the import over; completion survives too. - Run-level failures (401, OWNER_RETIRED, FORBIDDEN_LEARNER, OWNER_BUSY) from any call stop the run and leave the course pending, including the "is the id taken" read and the library re-list, which could previously mark the course failed and close the import. - Refused media without a generation request (the user's own) gets a new non-retryable ASSET_REFUSED code, so it shows as failed without a Retry that could only fail; a skipped singleton runtime session leaves a note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): pin the importer ledger's owner merge and the singleton-skip note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: the legacy import moves to another owner only on a confirmed claim The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say that the import runs once per browser and moves to another owner only when that owner claimed the original one, confirmed by GET /api/identity/merged-from; that the owner is recorded as a salted digest; and how refused user media shows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): never let a transient failure settle a legacy import item Review round 3 of the legacy browser importer. - A read of the old browser stores that fails in storage (an aborted IndexedDB transaction, a closed database) now pauses the run and leaves the course pending. Only a record that cannot be migrated, parsed or validated is skipped. This covers the course read (which no longer falls back to the older table copy on a storage failure), and the quiz-scene and speech indexes, which used to read a failure as "no scenes" or "no holder" and drop quiz state or narration for good. - After a session-id collision, a failed read of the session propagates and is retried; only a successful read that shows the id taken skips it. - Without Web Locks, two tabs of different owners can both find the ledger unbound. Every ledger write keeps the owner stored first, and the run now re-reads the stored ledger after binding, before folders, before each course and before each course document is created, and stops if another owner holds it. - The "not yours" answer is reused for ten minutes instead of an hour, a stored wait further ahead than the longest the importer sets (a clock that ran ahead) counts as passed, and expired answers are pruned. - Generated media refused without an error code keeps its Retry; only media nothing can regenerate gets ASSET_REFUSED. A poster that fails transiently leaves the video pending instead of being dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover transient reads, racing owners, the not-yours cache and the claim client Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(persistence): say what the ledger's salt protects, and the new pauses The salt defeats precomputed and cross-browser tables, not a targeted guess against one browser's ledger; the module README and the digest's comment now say so. The README also covers storage read failures, which pause instead of skipping, the two-owner race without Web Locks, and the ten-minute recheck. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * feat(identity): bind a browser's legacy data to one owner on the server The one-way import of pre-server browser data now asks the server whose data it is, instead of deciding in the browser. - legacy_import_bindings (browser_id primary key, owner_id, timestamps), provisioned with owner_merges on fresh and upgraded databases. - POST /api/identity/legacy-import-binding {browserId}: one atomic insert (ON CONFLICT DO NOTHING) and a read-back; answers only whether the requesting owner holds the browser. Same-origin JSON, resolved and write-fenced like every owner write, 400 for a malformed id. - A claim participant re-keys the claimed owner's bindings to the account inside the claim transaction, so the account simply holds the browser. - Owner resolution refuses any request carrying X-OpenMAIC-Legacy-Import with 409 LEGACY_IMPORT_NOT_BOUND unless the owner it resolves to holds that browser: one central fence every owner-scoped route goes through. Requests without the header are unaffected. - GET /api/identity/merged-from is removed; the binding replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): import through fenced clients bound by the server Review round 4 of the legacy browser importer. - The ledger holds a random browser id and no owner information; the owner digest, the "not yours" cache and the client-side handoff are gone. A run first asks the server to bind the browser and imports only when the requesting owner holds it. - Every importer request goes through its own clients that carry the browser id, so the server refuses any write under an owner that does not hold the browser: after a cookie switch in another tab, from a page whose memoized owner is stale, or from a tab that lost the binding race. The runtime learner key is asked for during each run, never taken from the page. 409 LEGACY_IMPORT_NOT_BOUND stops the run with items pending. - The write-back funnels, commitToPool and the PBL save-path strip take an optional store, put or runtime, so the importer passes its fenced ones. - A failed read of one legacy course holds up only what it could affect: the quiz-scene and speech indexes leave the course out as unindexed, and only quiz state or narration it could own waits. Read failures are counted per course; after five failing runs over at least a day, a course whose document-store copy cannot be read falls back to the table copy, or is skipped with the reason when there is none. - Runtime ids are rewritten by replacing only their course segment, so a short course id cannot corrupt the prefix or the learner segment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover the server binding, its fence, bounded reads and short ids Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: the server binds a browser's legacy data to its first owner The README (and its Chinese version), the deployment guide in every locale, the changelog and the module README now say: once per browser; the server binds the browser to the first owner; a claim carries the binding to the account; the importer's requests are refused for any other owner. They also describe the bounded pause for a course the old storage cannot read, and the removal steps for the server side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): keep a dropped asset request pending in the legacy import The asset client reports a fetch that got no answer (a dropped request, an aborted upload, the existence probe's deadline) as status 0 with HTTP_REQUEST_FAILED, which the importer read as a final refusal: the media was marked failed and the course could complete without it. Read that code as transient, keep the client's local validation failures final, and treat a 2xx/3xx answer a client could not use as transient too. Unit tests drive every client the importer uses with a rejected fetch and a non-JSON answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(persistence): ask a refused binding again only every ten minutes A browser whose legacy data another owner holds sent the binding request (a write) on every page load. Record a ten-minute recheck in the ledger (clamped like every other deferral), so a claim that moves the binding is still picked up, and open the old databases only once the browser is bound. The read budget now counts consecutive failing runs (a run that reads the course resets it) and counts a first failure dated in the future from the next one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): answer 503 with the cookie when the import fence cannot read The legacy import fence now reads the header name from the shared constant, and a database error while reading the binding answers the usual JSON error (503 PERSISTENCE_UNAVAILABLE) with the resolution's Set-Cookie values instead of escaping as a bare 500. Route tests cover that, a bind by an owner a claim retired (403 OWNER_RETIRED, no row), and one header name on both sides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(persistence): cover dropped asset requests, the recheck and the read budget With the real asset client's error for an upload and an existence probe, the media stays pending and a later run imports it. A non-holder's five loads send one binding request and never open the old databases, and a claim still hands the import to the account after the delay. The read budget's run count and time span each hold on their own, a clock that ran ahead does not hold a course open, and a good read resets the count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(persistence): complete the legacy import removal steps List every file the removal touches (the claim scenarios test, the claim participant table and list, every doc), keep the bindings table for one release after the code goes because of rolling deploys and give the DROP statement for that release. Add the claim's eighth participant to the README lists, fix the fresh-id wording in the changelog, and describe the recheck delay, the transient status-0 asset failures and the consecutive read budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * fix(persistence): keep legacy quiz state through Clear Local Cache until the import completes Pre-runtime quiz drafts, answers, results and attempt ids exist only in localStorage, and the one-way importer copies them to the server with their course. Clear Local Cache deleted them even while the import was still pending (it waits for an idle page and can stay pending after a failure), so that quiz progress was lost for good. Clear Local Cache now keeps the four quiz key families unless the importer's ledger records the import as complete, read through a new `legacyImportIsComplete` helper in the ledger module. The ledger key moves there too, so the two modules do not import each other. Once the import is complete, the keys are cleared as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs: show DATABASE_URL set in the local development setup The getting-started guide showed the required DATABASE_URL line commented out, so copying the block left the server unable to start. Each locale now shows `pnpm db:up` and, separately, the `.env.local` line to set. The `.env.example` header said every variable is optional; it now says that provider variables are, and that DATABASE_URL is required except under docker-compose.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(changelog): drop two upgrade notes the final code contradicts The Compose entry said its default DATABASE_URL overrides .env.local; the defaults file is read first, so a value in .env.local wins, as the same entry says further on. The runtime-session entry offered turning server persistence off as an upgrade path; that switch no longer exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(env): restore the ACCESS_CODE warning notes in .env.example A later change to the example file dropped the lines saying that an unset ACCESS_CODE fails open, that the server then logs a one-time startup warning and GET /api/health reports accessCodeConfigured: false. They are back, and say that single-user mode logs its own warning in place of the generic one, as the startup code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * ci: run every app PostgreSQL suite in the contract job The app-domain step named its suites one by one, and seven had been left off, among them the legacy importer's route suite. No other job gives the app suites a database, so they skipped everywhere. The step now lists every app `*.pg.test.ts`; all of them work in a schema or database of their own, or truncate only their own tables, and pass before and after the storage package suite in the same database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): establish the anonymous owner before the first API request (#1721) * fix(identity): establish the anonymous owner on the page response On a browser's first load the page sent several API requests without an owner cookie; each minted its own anonymous owner and set its own cookie, and the last answer to arrive won. Work done in between (the legacy import binding, a first course write) then belonged to an owner the browser no longer presented. The middleware now mints the anonymous cookie on document navigations that carry no valid one, with the route handlers' own value format and attributes (the cookie module is Edge-safe now: Web Crypto instead of node:crypto), and forwards it on the request so the page render resolves the same owner. API, RSC, prefetch and Server Action requests never mint there, and a valid cookie is never replaced. It mints only when neither OWNER_SINGLE_USER nor PERSISTENCE_SHARED_OWNER_ID is set; host registrations are invisible to an Edge middleware, so the README tells hosts where to skip it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(legacy-import): bind only an owner the browser already presents A binding request whose owner was minted by that very request answers 409 OWNER_NOT_ESTABLISHED with the minted cookie and binds nothing: other cookieless requests may still be minting owners of their own, and a binding to this one could be left with an owner nobody presents. The importer treats it as transient and retries on a later load. It also says plainly when a request after its own successful bind is refused as another owner (the owner cookie changed mid-run); items stay pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(legacy-import): cover a first visit whose first answers arrive out of order Against the real routes and PostgreSQL: a browser without a cookie loads a page, sends two requests at once, and the first answer reaches it only after the importer bound the browser; the import completes under the one owner the page established. A second case binds nothing for an owner the bind itself minted and imports on the next run. The E2E drives the same ordering in Chromium by holding the first owner-scoped answer until the binding answered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * test(legacy-import): drop an unused initial assignment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(changelog): note the page response now sets the anonymous owner cookie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): let hosts turn page-response minting off The middleware cannot see owner auth methods a host registers, so a host with anonymousFallback: false still got an anonymous cookie on a cookieless page load. Beside the host credential it is a claim candidate; with OWNER_CLAIM_TRIGGER=auto it is claimed and cleared on the next request, again after every such page load. OWNER_ANONYMOUS_PREMINT (default on; true/1/false/0, anything else fails startup) is read by the middleware before minting. At startup the server warns once when a registration turns the anonymous fallback off while pre-minting is still on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(identity): document OWNER_ANONYMOUS_PREMINT Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): keep the anonymous identity for 400 days and renew it in use The anonymous cookie had a fixed 30-day lifetime from its mint and was never renewed, so every anonymous visitor lost their whole library 30 days after the first visit however active they were; with persistence on the server only, that affects every anonymous deployment. It now lasts 400 days (the browser cap, one shared constant), and every route handler and Server Action response that resolves to a valid anonymous cookie re-sends the same value with a fresh Max-Age (best-effort where next/headers refuses writes). Page responses never renew, so pages stay cacheable, and a host principal never renews the anonymous cookie beside it. A response that clears the cookie (a claim, a retired owner) must not renew it, whatever order a route merged its headers in: withRequestOwner, and the owner-events retired answer, keep only the clearing value for a cookie a value clears. A renewal that lands after a claim cleared the cookie is pinned by a test: the next write with it, alone or beside the account, writes nothing, and its answer clears it again without renewing it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * docs(identity): 400-day renewed anonymous cookie; when to turn pre-minting off Hosts set OWNER_ANONYMOUS_PREMINT=false only when their registration sets anonymousFallback: false; hosts that keep anonymous visitors need pre-minting and skip it per request in middleware for requests their methods authenticate. The CHANGELOG notes the new lifetime and renewal, and that a lost cookie means a new owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz * fix(identity): forward owner cookies from routes that resolve the owner directly The Pi chat and whiteboard-visibility routes and asset-id document extraction resolved the owner themselves and never sent its Set-Cookie values back, so an anonymous identity used only through them was never renewed (and a minted one never stored). attachOwnerCookies attaches a resolution's cookies to a response a route built itself, with the clear-wins rule, before a stream's body starts. Both Pi routes answer through it, success and error alike; resolveServerAsset hands the cookies back with every answer and the extraction route attaches them. A guard test scans for every module outside the identity seam that calls the owner resolution (or the asset helper) directly and requires it to forward the cookies; the two callers that cannot are listed with the reason. Behaviour tests cover the three paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRME5T4oEaQ4EBUzrDXMKz --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: wyuc <dhq1204@yahoo.com>	2026-09-29 17:51:03 +08:00
wyucandClaude Opus 4.8	ee79762ef9	fix(access-code): expire tokens server-side and throttle verification behind a trusted proxy (#1513) ... When ACCESS_CODE is set, verification tokens (timestamp.HMAC) never expired because neither verifier checked the timestamp, and POST /api/access-code/verify had no attempt throttling. - Enforce a 7-day token lifetime in a shared, Edge-safe module used by both the Node verifier and the middleware Web-Crypto verifier, and reject non-canonical signatures in both. - Rate-limit verification only when the client identity is trusted (TRUST_PROXY_HEADERS=true): a per-client sliding window (10 failures / 60s) whose attempt is reserved atomically at check time, returning 429 with Retry-After. Without a trusted proxy the app cannot attribute requests to a client, so no shared throttle is applied — a long random ACCESS_CODE is the protection, and a warning is logged when it is short. - Store bounded, copied identity keys so a large forwarding header cannot retain memory. - Document the behavior in .env.example, README, and configuration docs. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-09-15 17:47:14 +08:00
wyucandClaude Fable 5	04621578de	release: OpenMAIC 1.0.0 — the agent workbench (#1228) ... * feat(storage): add an agent-session store with PG backend and layered contracts (#1163) * feat(storage): add agent-session store with PG backend and layered contracts * test(storage): avoid BigInt literals for pre-ES2020 root typecheck * fix(storage): close agent-session store review findings * docs(storage): align hook ordering and contention-probe claims with the code * ci: run on the agent-workbench integration branch * chore(storage): bump to 0.5.0 for the agent-session store * fix(storage): carry replay compaction across page boundaries * feat(agent): add the driver model contract and stage route dialect (#1165) * feat(agent): add the driver model contract and stage route dialect * fix(agent): validate route context windows and clarify dialect precedence * feat(agent): adapt the agent-session store and runtime foundations (#1167) * feat(agent): adapt the agent-session store and runtime foundations * feat(agent): resolve request owner identity via an anonymous cookie * docs(agent): document the opt-in compaction default and harden edge cases * feat(agent): add the background session runner (#1169) * feat(agent): add the background session runner * feat(agent): wire the runner into startup behind feature flags * fix(agent): stop clean interruptions from consuming the attempt budget * fix(storage): charge the attempt budget for abandoned leases but not clean parks * docs(storage): document the attempt-charging contract and decouple its tests * feat(agent): add agent session and owner event streams (#1170) * feat(agent): add agent session and owner event streams * fix(agent): close the session-existence oracle and document the owner seam * feat(agent): add agent session lifecycle routes (#1171) * feat(agent): add agent session lifecycle routes * fix(agent): validate session-create input and preserve the owner cookie on errors * refactor(storage): drop the unused active-stage API from the agent-session contract (#1174) * refactor(storage): drop the unused active-stage API from the agent-session contract Tools address stages explicitly on every call, so the store keeps no mutable session-level stage pointer. Removes resolveActiveStage and setActiveStage from the store interface, their PG implementations, the active_stage_changed lifecycle event, the session_active_stage owner event variant, and the contract tests pinning them. The active_stage_id column and the DDL check constraint stay untouched for schema compatibility. * chore(storage): bump @openmaic/storage to 0.7.0 for the contract removal * docs: document the agent runtime configuration surface (#1176) * fix(agent): repair orphaned and late tool results across interruption boundaries (#1180) * fix(agent): repair orphaned and late tool results across interruption boundaries A crash, shutdown, or provider failure can leave the durable transcript with tool calls that have no result, or with results ordered illegally for the provider. Three failure modes were fixed: - Orphaned tool calls: a run that died between an assistant tool-call frame and its result left a dangling call in the entry tree. Resume no longer synthesizes and persists receipts for it: interrupted results are a read-time provider view owned by a shared read-boundary repair, which returns the original array for a healthy transcript and never mutates the tree. - Late parallel results: a parallel tool can finish while pi unwinds an aborted assistant frame, leaving result(A), assistant(aborted), result(B) in durable order. Strict providers reject non-contiguous results, so the read-boundary repair moves existing results next to their owning assistant frame (in call order), omits incomplete unwind frames, and synthesizes receipts only for genuinely missing calls. - Interrupted calls at the write boundary: a call still in flight when the run winds down (shutdown, lease loss, cancellation, provider failure) had no receipt at all. The runner now tracks in-flight calls from their assistant frames and, before the terminal flush, appends an interrupted-result receipt for each still-orphaned call through the same attempt-fenced write chain, so a lease-stealing zombie never writes and the next claim sees a provider-safe transcript. * test(agent): pin the runner wiring for interruption-boundary tool repair * feat(agent): add neutral tool foundation libraries (#1184) * feat(agent): register a web_search tool on the session runner (#1185) * feat(storage): add a per-session URL trust gate (#1186) * feat(agent): add the skills system (#1189) * feat(agent): add the skills system (builtin directories and durable user skills) * fix(storage): serialize the user-skill quota check-and-insert per owner Two concurrent creates at the 50-skill boundary both counted 49 rows and both inserted (READ COMMITTED, no lock), overshooting the quota contract. The create transaction now takes a per-owner pg_advisory_xact_lock first, and the same-name idempotency check runs before the count check so an at-least-once retry of the create that committed as the owner's 50th row still returns its durable receipt instead of a quota error. The 23505 backstop is retained for writes that do not take the lock. * fix(agent): share unstorable-character validation and align skill lookup * feat(agent): add session materials and a fetch_url tool behind the URL trust gate (#1190) * feat(agent): add session materials and a fetch_url tool behind the URL trust gate * fix(agent): harden session material fetching * feat(storage): add an ownership scope to stage documents (#1191) * feat(agent): add material read and search tools (#1192) * feat(agent): add stage read and patch tools (#1194) * feat(agent): add page generation and deck editing tools (#1198) * test(storage): keep the PG contract suite order-independent (#1200) * fix(agent): revoke deleted-session URL authority and reject private ISATAP endpoints (#1199) * fix(storage): revoke deleted session URL authority * fix(ssrf): reject private ISATAP endpoints in strict fetches * chore(storage): bump to 0.11.1 for the session-URL authority fix * feat(agent): add roster and voice registration tools (#1201) * feat(agent): add folder organisation tools (#1202) * feat(api): add stage and material HTTP routes (#1203) * feat(workbench): add the client data layer (#1204) * feat(workbench): add the client data layer * docs(workbench): write the ported comments in English * chore(edit): remove the in-editor agent panel (#1210) * chore(edit): remove the in-editor agent panel * style: apply prettier formatting * fix(agent): report the runtime as unusable without a database (#1207) * fix(agent): report the runtime as unusable without a database * style: apply prettier formatting * feat(agent): add image, video and pptx import tools (#1211) * feat(workbench): add the agent chat surface (#1205) * feat(workbench): add the agent chat surface * docs(workbench): write the ported comments in English * fix(workbench): label the folder and rename tools on the timeline * fix(workbench): label the roster and voice tools on the timeline The reconciliation test iterates every tool the runner registers and requires a display label of its own. The roster and voice-clone tools (list_voices, set_roster, clip_audio, register_voice) reached the integration base with the roster/voice-registration tools but never gained presentation rows, so they fell through to the default branch and rendered their wire names. Port their rows from the reference implementation (labels and i18n keys verbatim) and extend the reconciliation allowlist with ROSTER_TOOL_NAMES and VOICE_CLONE_TOOL_NAMES, so a future tool cannot enter the product without a label. * feat(agent): add the material extraction lifecycle (#1212) * feat(storage): add material extraction lifecycle * feat(agent): execute queued material extraction * style: apply prettier formatting * style: satisfy prefer-const in the extraction runner * test: give material fixtures the extraction lifecycle fields The media-tools slice and the extraction lifecycle slice were each green in isolation but never compiled together: the lifecycle made derivedFrom and extraction required on AgentSessionMaterial while the media-tool fixtures predate them. * chore: remove stray task notes * fix(workbench): label the extraction lifecycle tools on the timeline * feat(workbench): add the workspace shell (#1206) * feat(workbench): add the workspace shell * docs(workbench): write the ported comments in English * i18n(workbench): align workspace keys across locales * fix(workbench): adopt the landed data layer and label the extraction tools - replace the sibling-slice seam stubs with the real data-layer modules - drop ambient declarations now shadowed by landed files - port timeline labels for the extraction lifecycle tools from the reference - align the new i18n keys across all locales * ci: retrigger * feat(api): folder routes, stage-meta viewer surfaces, and the material upload contract (#1215) * fix(storage): restore capability-based stage access * fix(api): bind document access to request owner * fix(agent): restore three-state stage access on the tool layer Port probeStageAccess and the three-state StageAccess (owned / foreign / missing / tombstoned) and gate every stageId-bearing stage tool on an owned probe, mirroring the reference per tool: - move_to_folder, rename_stage, read_stage_outline refuse a non-owned stage with the single not-yours message before touching the store. - The course/DSL toolset and the roster toolset are wrapped by withOwnerStageAuthorization: read_stage, patch_stage, grep_stage and every writer refuse a foreign stage with the same message and refusal shape. - Scene preview keeps its own probe and its own refusal text, and is registered beside the course toolset (never double-gated). - The runner injects one probe factory at the three call sites. Tests: the dsl cross-owner test premise (a foreign stage is readable by id) encoded an invented capability-read policy that the reference does not have at the tool layer; it now asserts foreign read/patch/grep are all refused while the owner still reads. Curriculum cross-owner assertions were already the reference's and now pass with the probes in place. * docs: correct per-file test counts in the fidelity report * test: fix type errors in stage-access fidelity test * test: adapt media-tool and gate suites to the owner-scoped store seam * feat(api): add owner-scoped course-folder HTTP routes Port the reference implementation's /api/folders family (list, create, rename, delete with ungroup/remove modes, and folder membership) onto the owner-bound document store, replacing its provider-based auth with the existing withRequestOwnerId / owner-scoped store seams. The storage package's folder store grows the pieces the routes need: DocumentFolder.order (schema column + max+1 assignment + ordering), renameFolder, deleteFolder(mode) with captured member ids, and setStageFolder(stageId, folderId | null) with idempotent un-filing. FolderNameError moves into folder-name-validation.ts (stage-storage re-exports it, keeping import sites intact). Every route gates on the configured agent runtime (plain 404 when off or unconfigured), keeps the reference's machine codes and envelopes, and is covered by gate tests plus a behavior suite. * feat(api): add stage-meta viewer surfaces for the classroom Port the reference implementation's viewer-facing stage state — can-edit / collected / published / generation-complete — on top of the stage-access base (stage_meta + tombstones). stage_meta gains published_at and generation_complete columns plus a stage_bookmarks table; the reference's deployment-specific origin/claimed_at columns are stripped. New gated routes: GET /api/stage-meta/[stageId] (per-viewer facts, 404 for absent/tombstoned, never returns the owner id), GET /api/stages/[id]/status, POST generation-complete / publish / unpublish (owner-only), POST /api/bookmarks. The resolver lives in lib/server/stage-access.ts. Wiring: a fetchStageMeta client with the reference's three-outcome contract, stage-store isOwner/isBookmarked/readOnly fields (upstream single-user defaults, no-op until the sidecar answers) plus setViewerAccess, the classroom apply path computing readOnly = !(isOwner || isBookmarked), the Stage editability gate, and a sidecar probe after each classroom load. A sidecar 'absent' answer keeps the editable default here because the classroom also serves local-only courses; server writes stay owner-enforced. * feat(api): port the reference material upload contract Rewrite POST /api/materials to the reference implementation's upload shape so the workbench uploader (uploadWorkbenchMaterial, which posts no session id and expects a flat 201 view) works unchanged: owner-scoped upload with mime normalization/validation (415), per-class size caps checked on the declared content-length and the streamed body (413), empty body (400), quota (429), sha256 reserve->store->finalize lifecycle with abandon on failure, flat { materialId, originalName, bytes, mime, extraction } 201, and an x-request-id echo. Adds the owner-scoped material library (owner_material table + quota + 24h lazy sweep, bytes in the host's asset registry as the neutral replacement for the reference's object-storage byte path) and the material cap configuration. The session-scoped GET list is left as-is; the reference's owner-material extraction worker is not ported (the branch's session-material extraction lifecycle already covers extraction). Gate tests now cover all 23 persistence routes across the three runtime env states; the materials behavior suite pins the new contract. * feat(media): add an optional local ffmpeg media extractor (#1213) Adds a local ffmpeg/ffprobe pipeline as a second media extraction provider behind the extractor registry, ported faithfully from the reference implementation: duration probing, keyframe-safe chunking, per-chunk ASR with timeout and deadline budgets, and timestamped transcript assembly. - Availability probing feeds the registry's candidate selection: the provider simply is not a candidate when ffmpeg/ffprobe are absent. - With neither ffmpeg nor a cloud provider configured, extraction fails with an actionable message naming both enablement paths. - Media materials route through the same extraction lifecycle and lease fence as documents; no parallel queue. - Tests inject the executable resolver so the missing-ffmpeg path is the default-tested one; the real pipeline test is skip-if-unavailable. - @openmaic/storage 0.13.0 -> 0.14.0 (media routing in the material lifecycle surface). * feat(storage): per-scene monotonic revisions via database triggers (#1214) * feat(storage): per-scene monotonic revisions via database triggers Restore the reference implementation's freshness granularity: a per-scene monotonic revision maintained by database triggers, so every writer (HTTP routes, agent tools, jobs, manual SQL) bumps it without application cooperation. - Companion revision tables + trigger functions in the storage package's idempotent schema bootstrap, with the lock-order invariant, pg_notify wakeup and the suppression switch for batch writers. - ensureDocumentSchema gained a dollar-quote-aware statement splitter. - The freshness and manifest routes serve per-scene revisions. - Mutation-verified: dropping the triggers turns the revision tests red. - @openmaic/storage 0.13.0 -> 0.14.0. * fix: forward the freshness manifest through the owner-bound store * feat(workbench): add the Pro entry points and preserve the mode-transition semantics (#1208) * feat(workbench): add the Pro entry points * feat(workbench): preserve Pro mode transition semantics * fix(workbench): drop ambient declarations shadowed by landed slices * fix(workbench): drop ambient declarations shadowed by the landed shell * feat: port workspace shell sibling modules Port the 16 leaf modules the Pro workspace shell imports but that were only ambient-declared, replacing the compile-time bridge with real implementations adapted from the sibling-slice reference: pure workbench helpers (session title, rail tab, course-chat bootstrap, created-course tabs, course-tabs memory, workspace navigation, pane navigation, pro-edit sizing, existing-course minting, first-message session), the neutral brand context and course-rename server API, the server-action session delete, the home discovery hook, the classroom pane host with its load-policy leaf, the theme toggle and floating-layer owner, plus the floating-layer-owner wiring the dialog/dropdown/tooltip portals stamp. Also add the workbench-shell locale copy for all 12 locales, port the reference tests for the ported modules, and drop types/workbench-sibling-slices.d.ts now that every declaration has a real implementation. * docs: keep ported comments in English and deployment-neutral * docs: announce 1.0.0 and refresh the feature overview (#1216) * docs: announce 1.0.0 and refresh the feature overview * docs: finalize 1.0.0 README after feature merge * fix(agent): control-plane routes answer 404, not 500, without a database The agent control-plane routes gated only on the runtime flag, so an enabled-but-unconfigured deployment (flag on, DATABASE_URL empty) answered 500 from a store that cannot connect. Gate them on the configured check instead, matching the stage/material routes: the whole surface is cleanly absent until both the flag and the database are present. The status probe keeps reporting both bits. * test: mock both runtime gate exports in the control-plane route suites * fix(agent): abort in-flight TTS on cancel and bound each provider request with a timeout (#1217) The generate_tts / scene-tts path checked the runner's AbortSignal between actions but never created the provider HTTP requests with it, so a session cancel left a hung synthesis fetch in flight until a restart repaired the tool result. Thread the signal end-to-end: TTSModelConfig carries an optional signal, generateTTS combines it with a per-request timeout (TTS_REQUEST_TIMEOUT_MS, default 30s, ported from the reference runtime's TTS bounds) via AbortSignal.any, and every provider fetch (openai, azure, glm, qwen incl. voice-clone + audio download, voxcpm, minimax, doubao, elevenlabs, lemonade) is created with that signal. A timeout now fails the tool call with TTSRequestTimeoutError (a clear retryable error) instead of wedging the session; a caller cancel propagates as the interruption so the runner settles the session as cancelled without a restart. Tests: hung-provider simulation rejects at the timeout with the retryable error; abort mid-flight aborts the captured request signal and surfaces the interrupted shape; removing the signal wiring makes the abort tests fail (red), restoring them turns green. * fix(workbench): PG-mode home listing via owner stages; keep the interrupted terminal course card (#1218) Finding 1: with server persistence on, listStages resolved to the generic GET /api/persistence/documents listing, which the capability model deliberately answers 403 FORBIDDEN_DOCUMENTS for (reads by id, listings owner-only). The home/workspace library now lists through the owner-scoped GET /api/stages surface (same anonymous-owner cookie the workbench uses) when server persistence is enabled; the server-side 403 is untouched. Finding 2: a run interrupted (session_interrupted) and repaired (session_resumed) that ends cancelled before agent_end stranded its pending classroom sightings, so the timeline's terminal card lost the course the answer produced. session_end (cancelled) now flushes the pending sightings into the same course card set agent_end paints, before the stopped caption. * chore(workbench): remove the bookmark concept and the saved-courses drawer (#1219) * chore(classroom): remove the bookmark ('collected') concept entirely The stage-meta viewer port introduced a bookmark surface (stage_bookmarks table, POST /api/bookmarks, the isBookmarked sidecar field, and a readOnly rule that let a saved course stay editable). The product has no such concept, so remove it as a closure: - delete the /api/bookmarks route and the stage_bookmarks table plus its query helpers from the persistence bootstrap - drop isBookmarked from GET /api/stage-meta/[stageId] - simplify the classroom read-only rule to readOnly = !isOwner across the sidecar client, ownership signal, classroom load, stage store and the classroom page - keep publish/unpublish, generation-complete, isOwner and isPublic exactly as they were - update the gate and stage-meta route suites and the README mentions The workspace rail's Bookmark glyphs and comments describe the upstream saved-courses (favorites) section, which is driven by isOwner and renders no collect affordance; they are kept as unrelated homonyms. * chore(workbench): remove the saved-courses drawer UI The first pass removed the bookmark data model but kept the rail's "Saved courses" drawer, judging it a separate surface driven by `isOwner === false`. The home/workspace listing is owner-scoped, so that flag can never occur: `allSaved` is permanently empty and the drawer (plus the collapsed-rail Bookmark mini-button) is a dead affordance. Remove it: the SavedDrawer component and its mount, the savedOpen / savedSection state, the allSaved / matchedSaved derivations, the 'saved' variant of the course-list renderers, the mini Bookmark glyph, the drawer-only CSS, and the drawer's i18n keys from all 12 locales. The courses tab is now exactly one folders tree. The authored/favorites split in workspace-tree.ts goes with it; the tree module no longer reads `isOwner`. The discovery course type keeps the field — the shell still reads it for read-only gating. Upstream has no collect concept; the drawer could only ever render empty here. The reference implementation HAS this drawer (its favorites come from its account system), so this removal is a deliberate upstream product decision, not a fidelity bug. * fix(workbench): restore the attach entry, add the rail settings entry, pin all three entry points (#1221) * fix(workbench): restore the composer attach entry by gating it on the live runtime The AttachButton's rollout probe read a `materialsEnabled` field that this branch's /api/agent/runtime never answers (the materials routes gate on the runtime itself, like the stages), so the gate could never pass and the attach button never rendered — the Pro launch and chat composers showed only the @-mention and enhance glyphs. Substitute the field with the runtime's `enabled` value, which IS the upload action's precondition: POST /api/materials answers 404 whenever it is false, so the render condition now equals the action precondition (no dead button). The button's label (`proMode.attach`) is a user-visible string that becomes visible again; port the reference implementation's own translations verbatim into the 11 locales that still carried the Chinese copy. * feat(workbench): add the settings entry to the rail's bottom-left cluster The reference's rail foot carries a cluster of utilities (its saved-courses drawer, the language switcher, the display toggle). This branch removed the drawer — it could only ever render empty here — and the product decision is to fill that freed spot with the settings entry. Add a settings trigger to the foot cluster (expanded rail, beside the language and display toggles, and on the collapsed strip) and mount the model/provider SettingsDialog in the rail, wired to the trigger. It is the same dialog the classic home opens from its header pill; the workspace had no settings entry of its own, so nothing is duplicated within a surface. * test(workbench): pin the restored upload, attach, and settings entry points Covers the three restored entry points: - the courses-tab upload control: rendered beside the course name filter, wired to the discovery hook's ZIP import trigger, disabled while an import runs, and gated by the same condition as its action (the courses tab); - the composer attach control: an actual render of AttachButton under both probe answers (visible when the runtime says the upload path is live, hidden otherwise), its mounts in the launch and chat composers, the branch's runtime-field substitution in the probe, and the reference's own `proMode.attach` copy in all 12 locales; - the settings entry: the trigger in the rail's foot cluster (expanded and collapsed), beside the language and display toggles, opening the SettingsDialog the rail mounts. * chore(config): the Pro workbench flag implies the MAIC Editor gate (#1223) A workbench build without the editor toggle has no way to edit a course: enabling NEXT_PUBLIC_PRO_WORKBENCH_ENABLED while forgetting NEXT_PUBLIC_MAIC_EDITOR_ENABLED produced exactly that split-brain bundle. The workbench IS Pro mode, so its flag now implies the editor gate; the standalone flag remains for deployments that want the classroom editor without the workbench. Documents both flags in .env.example. * fix(agent): wake SSE tails and the runner on durable deltas (streaming fidelity) (#1222) The Pro workbench chat did not stream: the session/owner SSE routes polled the durable event log on a 5s/30s clock with no wakeup, so message_update deltas (written at 150ms cadence) reached the browser in poll-sized blocks and the thinking strip only mounted after the whole reasoning text had accumulated. Port the reference's LISTEN/NOTIFY delta path: - storage: add in-transaction wake hooks (onSessionEventAppended, onOwnerEventAppended, onCancelRequested) so a host queues pg_notify in the same transaction as the durable append; align readEventsAfterForReplay to rank the bounded page so the first delta after the cursor is always kept (the live tail can never starve). Bump @openmaic/storage to 0.18.0. - app: port the process-wide event-notify bus (dedicated LISTEN client, self-check probe, reconnect backoff; notify through the storage transaction surface), wire the store hooks, subscribe both SSE routes before the initial read with the reference's initializing gate, and give the runner one {kind:'session'} subscription whose wake runs the cancel check and the message drain. Polls stay as the lossy-NOTIFY backstop. - lifecycle: start/stop the bus from instrumentation. Tests: storage hook + compaction contract; route wakeup latency; runner wakeup wiring with a fake agent; bus unit tests; PG contracts proving a real append wakes the routes and a live SSE route forwards a message_update on the wakeup, and that a rolled-back append never wakes. Also fix the pre-existing park-attempt-budget PG test TRUNCATE (missing CASCADE against newer FK tables). * fix(storage): asset writes self-deadlocked against pooled PostgreSQL (#1225) * fix(storage): refuse the non-transactional byte-write deadlock configuration A byte store whose plain write() runs on its own pooled connection cannot be invoked from inside a registry write transaction: after the transaction has claimed the blob-row lock, that write blocks on the lock the transaction just took while the transaction waits on the write - a self-deadlock PostgreSQL cannot detect (one side is idle in transaction). There is no lock-safe ordering for such a writer: bytes must be written after the row claim (writing before it lets the collector delete the bytes while the upsert waits), and any second-connection write after the claim is the deadlock. The configuration is therefore detected and refused: - AssetByteStore gains writesOutsideRegistryDatabase?: true, declaring that the layer's plain byte operations cannot contend for the registry's row locks. - PgAssetStore refuses put()/replace() up front (and defends coordinatedWrite) when the byte store has no writeWith and does not declare the flag, throwing a clear configuration error before any row is claimed. - The collector mirrors the guard on its delete path (deleteWith or a declared out-of-registry layer, else a configuration error). - The object store declares the flag (its out-of-transaction write remains legitimate); the in-registry PostgreSQL byte column provides writeWith / deleteWith instead. - Write transactions (put/replace/remove) set SET LOCAL lock_timeout = 30s so any future lock-contention variant fails loudly instead of hanging. Bumps @openmaic/storage to 0.18.0. * fix(persistence): forward the transactional byte methods through the lazy asset byte-store wrapper The no-bucket case of lazyAssetByteStore returned a bare { write, read, delete } and dropped writeWith/readWith even though the underlying PgAssetByteStore has them. The registry's hasTransactionalWriter duck check then failed and put() fell back to the byte store's own pooled connection, which blocks forever on the blob-row lock the registry transaction just took when the bytes live in the same PostgreSQL - the production self-deadlock. The no-bucket layer is statically PgAssetByteStore, so its transaction-pinned methods are forwarded eagerly (typed against the real signatures via PgForwardedByteStore). The bucket case keeps its lazy-probing semantics: no transactional writer exists there, the signed-URL method stays absent or lazy exactly as documented, and the wrapper now declares writesOutsideRegistryDatabase so the registry may run the plain write inside its transaction. New tests pin the wrapper's transactional capability red-to-green and assert put()/resolve() route byte traffic through the transaction-pinned queryable. * fix(home): cap the generate-prep ingest drain at 3s so Generate never waits the full server budget The classic home flow's Generate click drained in-flight ingests for the full 15s server budget. Cap the wait at GENERATE_DRAIN_CAP_MS (3000ms, documented as a UX bound) and reuse the existing timeout fallback: sources that miss the cap proceed on the legacy byte path and each late-resolving id is released. * chore(storage): bump to 0.19.0 over the concurrently landed 0.18.0 * fix(agent): bound every tool call with a timeout; never resurrect a cancelled session (#1226) * fix(agent): bound every tool call with a global timeout and settle it on cancel A tool await that neither resolves nor rejects wedges the session forever: the lease keeps heartbeating and the driver never reaches its next cancel checkpoint. Race every tool execution (in buildAgent) against a hard budget (OPENMAIC_AGENT_TOOL_TIMEOUT_MS, default 10 min, per-tool overrides for known long runners) and against the caller's AbortSignal, so even a signal-ignoring await cannot keep a cancelled session running. On timeout the call rejects with AgentToolTimeoutError; the agent loop turns the rejection into a structured error tool-result the agent can retry or proceed from, and the abort signal is delivered to the tool's in-flight work through a derived controller. Zombie-tool updates after settlement are dropped. * fix(storage): never re-lease a cancel-requested session; settle it as cancelled on claim The claim scan treated a session with cancel_requested_at set as a normal claim candidate: after a restart it re-leased the same session for attempt N+1 and resumed generating despite the pending cancel. claimNextSession now settles such candidates as cancelled under the claim lock (status cancelled, attempt reset, lease and cancel request cleared, terminal session_end event and owner projection) instead of leasing them, then keeps scanning. Bump @openmaic/storage to 0.18.0. * docs: takeaway-style 1.0.0 announcement with bilingual guide links The 1.0.0 head is now a short takeaway block — badge links to the official user guides (English and Chinese), five one-line highlights, and pointers into Features and the workbench setup section — instead of six dense paragraphs. The detailed provider-neutrality and freshness notes move into the Features workbench section, phrased database- neutrally (the announcement no longer names a specific database). Release date corrected to August 27. * fix(workbench): restore editor chrome, mode transition, streaming, materials, mentions, folders (#1229) * fix(workbench): wire workspace folder routes * fix(editor): restore reference workbench chrome * fix(workbench): persist composer materials and course refs * fix(workbench): preserve live reasoning frames * fix(persistence): back off failed streaming saves * chore(workbench): retire stale slice seams * test(editor): cover element pin layer * chore(storage): bump to 0.21.0 for the user-message ref/material fields * chore(editor): translate ported code comments to English * fix(agent): fence durable tool writes and consume cancel requests atomically (#1230) * fix(agent): enforce provider force-off in agent tools and scrub vendor identity from tool results (#1231) * fix(materials): serialize per-owner quota reservations and make crashed uploads reclaimable (#1232) * fix(editor): resolve dock-bar i18n keys, remove dock height drag, wire element referencing (#1233) * fix(workbench): send the opening session message exactly once with refs intact (#1234) * feat(editor): port timeline TTS preview single-flight and voice-all state latching (#1235) * fix(media): restore the reference classic media chain (#1236) * fix(import): adapt imported PPTX canvas size so decks render without overflow (#1237) * fix(editor): complete element referencing — renderer DOM contract and GenUI picking aligned with the reference (#1238) * test(providers): reconcile the provider-config vendor-token debt count after the main merge The integration line's AK/SK fallback for the managed document provider adds occurrences that main's allowlist snapshot predates. Same mixed-composition debt category the group already documents; no new vendor behavior. * test(providers): reconcile vendor-token debt counts with the integration line The main-merge brought main's neutrality-guard snapshot next to integration features it predates (media-extractor fallback chain, local voice-profile deletion semantics, the enabled-TTS helper). Same debt categories the guard already documents; counts updated to the guard's own tally and two grouped entries added. No new vendor behavior. * fix(agent): carry reasoning through the completions dialect so the thinking strip renders (#1239) * feat(skills): add Feynman and spiral curriculum methods (#1240) * feat(agent): port missing reference tools and skills (parity audit) (#1241) * feat(media): retire asset-registry wiring; media and materials follow the reference byte model (#1242) * fix(classroom): center adapted canvases in the stage and send back navigation home during generation (#1243) * feat(settings): skill management with real list, download, delete, and upload (#1244) * feat(settings): skill management section with real list, detail, and zip download * feat(skills): owner skill delete and upload across storage, API, and settings * fixup! feat(settings): skill management section with real list, detail, and zip download chore: neutralize a reference note in the settings header comment * fix(media): persist origin-independent classroom-media references from the agent runtime (#1245) * feat(editor): float the insert toolbar in the outer frame with collapse (#1246) The insert strip was bounded to the slide card, so it could only ever sit on top of slide content: the card's overflow clipped it and it could not be parked in the padding beside the slide. Move it into the studio frame the element picker's panel already roams (CanvasOverlayPortal + the frame selector), so both canvas overlays share one bounding container and their handles behave the same. While picking, the strip rises over the picker and goes inert, which is the z-order CANVAS_OVERLAY_Z already documents. Add a fold beside the grip: the chevron collapses the strip to that grip row and back, with the buttons unmounted rather than hidden. The fold is session-local state owned by EditShell, next to the drag offset, so a surface swap keeps it; nothing is persisted. Expanding a strip parked at the bottom edge re-clamps through the same bounds rule the keyboard move uses. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): align the chat timeline's left edge with the composer (#1247) * fix(agent): fence session claims while an ask_user question is outstanding (#1248) * fix(agent): settle-time rescue tracks real delivery instead of a count offset (#1249) * fix(persistence): migrate owner_material to oss_key and drop legacy asset_id (#1250) * docs(readme): surface the 1.0.0 user guide badges at the top (#1253) * fix(workbench): show newly created folders in the sidebar without reload (#1254) * docs(readme): add the release version prefix and drop the opt-in framing * fix(workbench): single-source the chat gutter so timeline and composer share a left edge (#1255) The transcript and the composer each established their own column: their own `px-*` gutter and their own `mx-auto w-full max-w-*` centering wrapper. Equal padding values were never enough, because the two columns are centered inside different containing blocks — the transcript's is a scroll container, whose content box is narrower than the composer footer's by the scrollbar's width: transcript text left = pad + (pane - 2*pad - scrollbar - measure) / 2 composer box left = pad + (pane - 2*pad - measure) / 2 The padding cancels out of the difference and what remains is `-scrollbar/2` at every padding value, so the transcript sat half a scrollbar to the left of the composer and tuning the two paddings against each other could not move it. The column is now established once, by the nearest common ancestor of both (`chatColumn`), and the scroll viewport and the composer footer are siblings inside it that add no horizontal inset of their own. The cap carries the gutter on top of the 760px reading measure, so the text column keeps its width. The handed-over question row drops the padding that indented it past the agent's prose; framed rows keep their own inner padding, which is what a card's border sitting on the column edge means. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(workbench): lock pane-embedded classroom to edit mode (#1256) The workspace right pane painted the full learning chrome — speed control, play button, learner avatars, mic bar — for a course the agent had just created, then flipped to edit once the first scene landed. resolveStageChromeMode treated playback as the DEFAULT branch for a hosted classroom, so every shortfall fell into it: a course whose tab opens at stage_link time has no scenes yet, so currentSceneId is null and isHostedSceneEditable is false. A folded pane parked the playback root behind the fold and cross-faded it out over the pane on unfold, and a failed editor chunk dropped into playback permanently. Lock it at the pane instead of defaulting per entry path: - WorkbenchPanelProvider — the single element that mounts a classroom into the workspace — publishes editPinned (visible && !playback). Every entry path passes through it, so none of them decides. - The hosted resolution can no longer degrade to playback. Start Learning (workbenchLearning, new input, split out from pane visibility) is the one door; everything else resolves between the neutral loading shell and edit. - Stage's chrome dispatch is exhaustive on chromeMode, so the playback root is no longer the else-branch of a condition about the current scene. No flicker: chromeMode is resolved during render, and preloadEditor now answers synchronously (isEditorPreloaded) so a remount with the chunk already registered paints edit on the first frame. A failed import is no longer cached forever, so the lock cannot strand the pane. Standalone classrooms keep their stored mode unchanged. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-27 19:26:25 +08:00
3 people	c071e02d83	feat: ACCESS_CODE site-level authentication (#407) (#411) ... * feat(access-code): add status API endpoint Returns whether ACCESS_CODE is enabled and whether the current user is authenticated via cookie, to support the frontend access gate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(access-code): add verify API endpoint Creates POST /api/access-code/verify that validates ACCESS_CODE env var and sets an HttpOnly 7-day cookie on success. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(access-code): add Next.js middleware to gate API routes Intercepts all requests when ACCESS_CODE env var is set. API routes return 401 without the openmaic_access cookie; page requests pass through so the frontend can render the access-code modal. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(i18n): add accessCode translations for all 4 locales Add accessCode top-level key to en-US, zh-CN, ja-JP, and ru-RU locale files with title, placeholder, submit, and error strings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: add ACCESS_CODE documentation to .env.example Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add AccessCodeGuard component and integrate into root layout Wraps page children with an access-code gate that checks /api/access-code/status on mount and renders AccessCodeModal when authentication is required. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(access-code): add AccessCodeModal component Adds the modal dialog component that prompts users for an access code, using existing Dialog/Button/Input UI components and i18n translation keys. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(access-code): fix lint error and format files - Fix useEffect setState pattern in access-code-guard.tsx - Format middleware.ts and layout.tsx with Prettier Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style(access-code): redesign modal as full-screen gate Replace generic Radix Dialog with a custom full-screen gate: - Mesh gradient background with noise texture - Frosted glass card with backdrop-blur - Staggered motion animations on mount - Inline submit button inside input field - Success state animation before dismissal - Proper dark mode support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(access-code): address code review findings Critical: - Middleware now validates HMAC-signed cookie tokens, not just existence - Verify endpoint uses timingSafeEqual for constant-time comparison - Cookie value is now `timestamp.HMAC(ACCESS_CODE, timestamp)` instead of UUID Important: - Middleware 401 response includes `errorCode` matching apiError shape - Status endpoint uses apiSuccess wrapper for consistency - Remove unused `accessCode.submit` i18n key from all 4 locales Suggestions: - Guard renders children during loading (overlay on top, not blocking) - Network error defaults to enabled=true (safer than silently disabling) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(access-code): use Web Crypto API in middleware for Edge Runtime Node.js crypto module is not available in Next.js Edge Runtime. Replace createHmac/timingSafeEqual with crypto.subtle HMAC and manual constant-length XOR comparison. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: 杨慎 <117187635+cosarah@users.noreply.github.com>	2026-04-12 22:14:14 +08:00