mirrors/maka
0
0 Fork 0
mirror of https://github.com/apache/maka.git synced 2026-10-28 17:04:39 +08:00
Code Activity
Files
main
maka/docs/archive/runtime-kernel.md
T
AstroHan c633ea5b25 chore: add ASF source headers and a release header audit gate (#3397) 
* chore: add an ASF source-header policy and audit it in the release gate

The first Apache Maka (Incubating) source release has to apply the ASF header
policy and enforce it, and nothing in the repository decided which files the
policy covers or checked that they comply.

`scripts/asf-license-headers.mjs` is that decision. It asserts two properties
of the tree it is pointed at, and nothing else. Every entry is classified
exactly once: covered by a comment syntax and required to carry the ASF header,
or matched by one reviewed exclusion rule that records why the header does not
belong there. And in every covered file the license text occurs exactly once,
as the canonical rendering at the top. A file that is neither covered nor
excluded fails the audit, so a new file type or an unexpected path cannot reach
a release candidate until someone writes down which of the two it is.

Both properties are stated about the artifact rather than about what the writer
emits. An audit that asks "does this file start with the bytes I would write?"
can only confirm its own writer: it treats a header rendering it does not
recognize as absent, writes a second license block above it, and then accepts
the result, because the block it just wrote is at the top of the file. So
detection is deliberately loose and acceptance deliberately strict. Detection
flattens comment punctuation and line wrapping away and finds the license text
written with `//`, in a JSDoc block, with `#`, inside an HTML comment, at any
indentation, with an `https` license URL, or rewrapped into a paragraph.
Acceptance takes only the exact canonical rendering. `write` is correspondingly
dumb: it inserts a header where there is none and refuses every other state,
because reconciling someone else's rendering is a decision, not a rewrite.

Nothing is dropped from the tree under audit. An extracted candidate is walked
whole, with no directory skipped by name, because a file inside a `dist/` or
`release/` directory in a source release is not a tree to audit around — it is
unclassified, and the gate says so. An entry that is neither a directory nor a
regular file is reported rather than passed over.

What the enumeration does decide is which tree a checkout stands in for: the
files a source release would carry, which means subtracting what `export-ignore`
prunes. `.gitattributes` owns that fact, so the audit asks Git for it rather
than restating it in a rule. A copy would answer for the original in the one
case that matters — an extracted candidate containing such a path would be
excused by the very reason that says it cannot be there. Asking Git also has to
walk the ancestors, since `export-ignore` on a directory prunes the subtree
while `git check-attr` reports one path at a time and does not inherit. The two
modes now audit the same set of files.

An exclusion rule names the files it excludes or states a structural property
that holds for anything it matches. A bare directory prefix does neither: it
lends its justification to whatever is added to that directory next. The rules
cover the ASF release documents themselves, verbatim upstream license texts,
the notices Maka publishes about a bundled third-party runtime, third-party
work kept under its own license, generated files that a `check:` script
byte-compares against a fresh run, payloads republished verbatim as prompt text
or as a pull request body, byte-significant fixtures, formats with no comment
syntax, binaries, and version-control metadata.

ASF policy treats the part of a file that the copyright owner did not
contribute as third-party work, so the files Maka adapted from opencode and
from the Vercel AI SDK are excluded rather than given a whole-file header.
Whether a heavily modified file should carry a combined header is a PPMC
decision. Their attribution belongs to LICENSE and the NOTICE audit.

The audit cannot determine provenance — that is a fact about people, not a
property of a path — so it refuses to let an unreviewed claim of one pass. A
covered file carrying an SPDX identifier, an explicit copyright line, or an
adaptation notice fails unless it is recorded in `reviewedProvenance` with the
decision that put an ASF header on it anyway. The markers were chosen by
measuring them against the whole tree: those three hit three files between
them, while `derived from` hits seventy and `opencode` hits forty-five, because
Maka supports it as a provider. A net that noisy gets answered with a list of
paths added to quiet it. This is a net and not a proof: it catches a file that
states its origin, not prose that alludes to one.

`scripts/protocol-epoch-check.mjs` gains one exemption. It requires an epoch
bump whenever a file under the protocol directory changes, a conservative proxy
for "the protocol changed" that is right to be conservative: a needless bump
costs a number, a missed one ships two incompatible protocols under one. But
inserting a license header provably does not change the protocol, and bumping
for it would tell every peer the wire is incompatible over a comment. The guard
now exempts a change that is exactly `applyHeader` applied to the previous
content — the same authority that writes the headers, so the exemption covers
the canonical insertion and nothing that merely resembles it. A protocol file
that gained a header and a real edit fails that comparison and still needs an
epoch.

`Prepare ASF source candidate` runs the audit inside the extracted archive
before anything installs or builds into that tree, so the release gate reads
the exact bytes a voter downloads instead of a checkout that resembles them. CI
runs it unconditionally beside the other install-free checks, because a header
gate that only sees the affected surfaces is not a gate.

Apache RAT was the alternative. It would put a JVM and a downloaded jar into
the audit path that reviewers are expected to reproduce, and its glob
exclusions fail open: a new unexpected file inside an excluded directory is
approved silently. This audit needs no toolchain beyond the Node the archive
already requires, and an unexpected file fails it.

The headers themselves are not in this commit. Until the sweep lands, the audit
reports every existing source file as missing one.

Refs #3271

Generated-by: Claude Code

* chore: apply the ASF license header to every covered source file

`npm run write:asf-headers` produced this commit, and `npm run
check:asf-headers` now passes on the checkout and on the extracted source
archive. Every file in the header sweep is exactly its previous content with
the standard ASF header inserted at the top, below an interpreter shebang, an
HTML doctype, or Markdown front matter where one of those has to open the
file. The two files described below, `DISCLAIMER-WIP` and
`.git-blame-ignore-revs`, are the only hand edits in this commit; nothing else
was touched by hand. `.asf.yaml`, which already carried the header, is
untouched.

`DISCLAIMER-WIP` no longer discloses that source files lack Apache license
headers, because they no longer do. The remaining disclosures about `NOTICE`
and about the software grant and ICLAs are unchanged and still open.

`.git-blame-ignore-revs` gets an entry for this sweep. Following the
convention the file documents, the placeholder records the landed hash in a
follow-up right after the squash-merge, since a hash that never reaches main
makes `git blame` fail outright.

Merging this will require open pull requests to rebase. That is the cost of
doing the sweep once; splitting it per directory would multiply the rebases
instead of avoiding them.

Refs #3271

Generated-by: Claude Code
2026-08-22 21:48:21 +08:00

10 KiB
Raw Permalink Blame History

Runtime Kernel Extraction

Archived on 2026-07-13. This is a point-in-time extraction record, not current architecture authority. Start with ARCHITECTURE.md.

This document explains the runtime-kernel work in this change set: what changed, why it was needed, what stayed stable, and how it was verified.

Summary

Maka already had the pieces of a local desktop coding agent: sessions, model streams, tool calls, permission prompts, abort handling, usage telemetry, bot and gateway entry points, and persisted session messages. The problem was that the core execution responsibilities were still concentrated in a few large runtime paths, especially AiSdkBackend and SessionManager.sendMessage().

This change set keeps the product surfaces stable but introduces clearer internal runtime boundaries:

SessionManager
  -> AgentRun
      -> AiSdkBackend
          -> ModelAdapter
          -> ToolRuntime
      -> RunTrace
      -> AgentRunStore

The intent is not to rewrite the runtime or replace the Vercel AI SDK. The goal is to make the existing runtime easier to reason about, easier to recover after interruption, and easier to extend with future backends or workflow integrations.

What Changed

ToolRuntime

ToolRuntime is now the internal boundary for the lifecycle of model-requested tools. The extracted runtime owns the work that used to be interleaved inside the AI SDK backend:

  • validate tool input before execution
  • evaluate permission policy
  • wait for parked permission decisions with timeout behavior
  • pause and resume the stream watchdog around tool execution
  • propagate abort signals into tools
  • classify tool failures
  • record tool telemetry and artifacts
  • emit tool and permission trace events

AiSdkBackend still bridges model stream behavior, but it no longer needs to own every detail of tool execution.

ModelAdapter

ModelAdapter is the provider-facing stream and error normalization layer. It keeps AI SDK-specific stream chunks, provider setup, usage normalization, and provider error mapping out of the higher-level backend orchestration shell.

This makes the boundary explicit:

provider / AI SDK details
  -> ModelAdapter
  -> Maka runtime events and usage records

That separation matters because future providers should not need to duplicate permission, tool, run, or session-state behavior.

RunTrace

RunTrace is an internal best-effort trace path for runtime events. It records milestones such as:

  • turn started
  • model resolved / stream started / stream completed / stream failed
  • tool started / completed / failed
  • permission requested / decided / failed
  • usage recorded
  • abort requested

Trace recorder failures are intentionally non-fatal. A failed trace write must not alter model or tool execution.

AgentRun Types and Store

The core package now defines internal AgentRun contracts:

  • AgentRunHeader
  • AgentRunEvent
  • AgentRunStatus
  • AgentRunStore

The storage package adds a file-backed run store:

sessions/<sessionId>/runs/<runId>/run.json
sessions/<sessionId>/runs/<runId>/events.jsonl

The store provides:

  • atomic run.json writes
  • append-only run event JSONL
  • same-run write serialization
  • session run listing
  • corrupt committed event-line recovery via event_corrupt
  • malformed unterminated tail tolerance

This ledger is separate from the existing session message JSONL so runtime diagnostics and recovery state do not pollute user-visible conversation history.

AgentRun Execution

SessionManager.sendMessage() now delegates the heavy turn lifecycle to internal AgentRun.execute().

AgentRun owns:

  • generating and recording run identity
  • appending the user message
  • writing initial turn state
  • locking the connection snapshot for the run
  • ensuring the active backend exists
  • driving backend stream events
  • projecting session status changes
  • writing turn completion, failure, abort, or permission-wait states
  • recording run started/completed/failed/cancelled status
  • routing RunTrace events into the durable run ledger
  • preserving abort source diagnostics such as renderer.stop_button

SessionManager remains the public runtime API and continues to own session CRUD, backend registry orchestration, active run lookup, and legacy recovery entry points.

Active Run Registry

The old active stream counters and turn-id side maps were replaced with:

activeRuns: Map<string, AgentRun>
turnToRunId: Map<string, string>

This makes overlapping run behavior explicit. It also avoids masking active stream accounting bugs with defensive counter clamping.

Startup Recovery

recoverInterruptedSessions() now prefers the AgentRun ledger when run rows are available. It scans persisted run headers and events, classifies stale non-terminal runs, repairs them, and then converges the existing session/turn projection.

Recovered cases include:

  • stale created or running runs
  • runs whose last event is run_started or model_stream_started
  • stale tool tails after tool_started
  • stale permission waits after permission_requested
  • model_stream_completed without a terminal run event
  • corrupt run event lines represented as event_corrupt

Legacy sessions without run ledger rows still use the prior message and turn-state recovery path.

What Stayed Stable

This work intentionally does not change:

  • window.maka.* preload API
  • Electron IPC channel names
  • renderer-visible SessionEvent behavior
  • user-visible permission modes
  • session message JSONL compatibility
  • builtin tool names or public tool behavior
  • model provider settings UI
  • bot and OpenGateway public entry points
  • Rive workflow integration semantics

The runtime kernel is internal. Public behavior should remain compatible while the internals become more explicit and recoverable.

Why This Was Needed

Before this work, a single turn execution was spread across session management, backend stream handling, tool wrapping, permission policy, telemetry, and abort logic. That made several questions difficult to answer after a failure:

  • Did the model stream start?
  • Which backend/model/connection did the turn use?
  • Was the run waiting for permission?
  • Which tool was running when the app exited?
  • Did the user press stop?
  • Was usage recorded?
  • Did a stale session status reflect a real active run or a crashed process?

The AgentRun ledger gives the runtime a durable fact record for those answers. ToolRuntime and ModelAdapter then reduce the amount of model/tool/provider logic that has to be understood at once.

Recovery Model

The new recovery path is conservative. It does not replay model streams or tools. Instead, it repairs stale state into deterministic terminal states so the app does not reopen with sessions permanently stuck in running or waiting_for_user.

When a stale non-terminal run is recovered, the runtime:

  1. Updates the AgentRun header to terminal state.
  2. Appends a durable recovery event such as run_failed or run_completed.
  3. Writes the existing session turn_state projection.
  4. Updates the session header out of active running/waiting states.

The failure class for app-restart recovery is recorded as app_restarted. Diagnostics are limited to small reason-code fields such as recovered, failureClass, recoveryReason, lastEventType, and eventCorrupt; raw user text and raw event payloads are not copied into recovery diagnostics.

Files Added

  • packages/runtime/src/tool-runtime.ts
  • packages/runtime/src/model-adapter.ts
  • packages/runtime/src/run-trace.ts
  • packages/core/src/agent-run.ts
  • packages/storage/src/agent-run-store.ts
  • packages/runtime/src/agent-run.ts
  • packages/runtime/src/agent-run-recovery.ts

Tests Added or Expanded

The runtime test suite now covers:

  • ToolRuntime permission allow/block/prompt/timeout paths
  • watchdog pause/resume behavior around tools
  • tool abort and failure classification behavior
  • tool telemetry and artifact recording behavior
  • ModelAdapter stream, usage, and provider error normalization
  • RunTrace recording and best-effort failure behavior
  • overlapping AgentRuns
  • backend build failure after user-message append
  • permission handoff without stuck run state
  • stop button abort source preservation
  • late complete after stop not overwriting aborted state
  • durable trace redaction
  • AgentRunStore create/read/update/list behavior
  • same-run AgentRun event append serialization
  • corrupt AgentRun event JSONL recovery
  • startup recovery from stale AgentRun ledger states
  • legacy startup recovery fallback
  • terminal AgentRun recovery idempotency

Verification

The changes were verified with:

npm --workspace @maka/core run typecheck
npm --workspace @maka/storage run test
npm --workspace @maka/runtime run typecheck
npm --workspace @maka/runtime run test
npm --workspace @maka/desktop run build:main
git diff --check

The final runtime suite included 315 passing tests after the AgentRun recovery work.

Follow-Up Work

This PR establishes the internal runtime-kernel shape, but it does not finish every possible cleanup. Good follow-up slices are:

  • merge usage and cost accounting more tightly with the run ledger
  • expose an internal run-inspection/debug read model
  • reduce the remaining SessionManager hook surface used by AgentRun
  • consider Gateway/Bot policy unification through the same run policy layer
  • consider Rive workflow/run mapping only after there is a product need
View Git Blame Copy Permalink