mirror of
https://github.com/openclaw/openclaw.git
synced 2026-10-26 10:13:27 +08:00
Related: #163788 ## What Problem This Solves Fixes commands hanging after output during forced Node shutdown without retaining the blanket Maglev and concurrent-Sparkplug workaround. ## User Impact CLI commands and the Gateway finish owned work and cleanup before exiting naturally, with Node's default compiler settings restored. Exit status, Unix signal termination, output, explicit compiler overrides, and system-CA trust are preserved. The Gateway still joins native skills-watcher retirement. This can retain synchronous FSEvents shutdown latency on macOS; the separate upstream certificate-loader stall is not fixed. Native macOS/Windows qualification is not claimed. ## Why This Change Was Made Existing lifecycle owners retain admitted work, native workers, child transports, updater recovery, and watchdogs through actual completion. The boundary guard prevents new forced normal exits. Cleanup dependencies are retained before a command can replace its package, without eagerly importing them into CLI bootstrap. No new configuration, dependency, or schema is introduced. ## Evidence - Earlier full pnpm build passed on composed tree faaafbe6b1f47e707f8392243898525990f3a452 (the terminal repair plus pinned CI base97e81ce), including the canonical CLI-bootstrap import gate, in 4m35.7s. Explicit/requested exits restore terminal state after cleanup; ordinary completion stays reset-free. The build retains a Control UI startup-JS size warning; no budget was raised or waived. - The built packaged CLI completed a real-PTY model round trip and exited naturally with status 0. Eight dual-TTY failure probes and twelve piped/dual-TTY telemetry probes passed. These used the built executable and existing helpers, not a claimed canonical Vitest smoke run. - The composed memory helper parses with one run/execute binding and passes 20 publication tests. The final runtime, one-shot, and cleanup suites passed 40 tests, including red proof of premature watcher loading. Broker regressions: 84 passed; real-process negative controls cover IPC disconnect and Unix signal propagation after output settlement. - The SIGTERM capture fixture retains its 20-second parent deadline and persistence assertions: red was 99 instead of 143; repaired case passed in 708ms (11.19s wrapper). The latest message/update fixture matrix passed 25/25 (179.52s): synthetic pending resources release only after observed stall diagnostics and join completion; ordered exit listeners remain checked. These later repairs change test/support files only, not production code. - Latest pinned-main integration preserved QA error reporting without forced exit: 204 QA tests, 10 reconciliation tests, and 6 real service-child relay tests passed. The reconciliation case failed before removing an obsolete admission close. Scoped types, lint, and three-pass review are complete; this integration did not repeat the full application build. - CI repair qualification: 96 tests across seven files passed (196.15s); final QA integration: 34 tests across three files passed (43.28s). Scoped types, type-aware lint, formatting, and mock-export checking passed. Fresh independent P0–P2 reviews found no actionable issues. - Browser startup recovery: all three saved/chat/toast cases passed (35.743s). Output and local-chat shutdown regressions were exercised with the rebuilt CLI. Fixture repairs preserve native completion, privacy, output, and fallback-model assertions. - Historical pre-integration proof: 115 affected test files; 137 built-executable cases with two platform skips; package-integrity checks; Linux Node 24.19.0 compiler activity and real Gateway SIGTERM-to-natural-exit in 175ms; published 2026.9.9 driver update with readiness and durable/incognito state checks. [Historical verification](https://github.com/openclaw/openclaw/actions/runs/38073218249). Compiler traces are functional witnesses, not throughput benchmarks or universal deadlock proof. Current-head CI run 38106144930 is red: 12 failing test jobs are attributed to the tested baseline (294b2e3a), with complete job and source-bound evidence for native admission. They cover browser catalog/focus and a stale Details selector, channel mocks, Codex import/steering, and Claude discovery fixtures. None is claimed fixed or green by this PR. The original Gateway read-admission and node-wire timeout observations remain unreproduced, not declared fixed. Earlier local smoke preparation was refused by managed-artifact separation; no guard was bypassed. Requested and directed by @steipete. ## Work sessions - [Original discussion with @steipete](https://team.openclaw.ai/chat/roboclaw/dashboard/fcef23f8-a69b-4b60-9869-3bd40ac48588) - [Implementation and verification](https://team.openclaw.ai/chat/roboclaw/subagent/c0adc270-c23a-4139-b3b7-00852b7db66f) ## Worked on by - @steipete <!-- openclaw-publication:09df66dc-cd63-4093-b782-154eeacdaa8f --> --- [View the OpenClaw team session](https://team.openclaw.ai/chat/roboclaw/subagent/c0adc270-c23a-4139-b3b7-00852b7db66f) Co-authored-by: steipete <58493+steipete@users.noreply.github.com>
70 lines
3.2 KiB
JavaScript
70 lines
3.2 KiB
JavaScript
// Shared stop policy; v2026.9.5 restart-health.constants.ts fixes the replacement window.
|
|
export const GATEWAY_RESTART_REPLACEMENT_TIMEOUT_MS = 60_000;
|
|
const GATEWAY_SHUTDOWN_DRAIN_TIMEOUT_MS = 315_000;
|
|
export const GATEWAY_SHUTDOWN_RESERVE_MS = 10_000;
|
|
export const GATEWAY_SUPERVISOR_EXIT_MARGIN_MS = 5_000;
|
|
export const GATEWAY_SHUTDOWN_TIMEOUT_MS =
|
|
GATEWAY_SHUTDOWN_DRAIN_TIMEOUT_MS + GATEWAY_SHUTDOWN_RESERVE_MS;
|
|
export const GATEWAY_SERVICE_STOP_TIMEOUT_MS =
|
|
GATEWAY_SHUTDOWN_TIMEOUT_MS + GATEWAY_SUPERVISOR_EXIT_MARGIN_MS;
|
|
|
|
export const LAUNCH_AGENT_EXIT_TIMEOUT_SECONDS = GATEWAY_SERVICE_STOP_TIMEOUT_MS / 1_000;
|
|
|
|
// Keep a positive shutdown budget when a supervisor's deadline is under 20s.
|
|
const GATEWAY_SUPERVISOR_EXIT_MARGIN_SHARE = 0.25;
|
|
|
|
// Preserve a legacy job's 5s drain where possible. Under 20s, trade some cleanup
|
|
// reserve for drain; on very short jobs, split the remaining budget in half.
|
|
// Inspection time is debited before this allocation, so a slow probe can also
|
|
// reduce the drain. The measured upgrade tradeoff is documented in restart-recovery.
|
|
const GATEWAY_SHUTDOWN_DRAIN_FLOOR_MS = 5_000;
|
|
const GATEWAY_SHUTDOWN_DRAIN_FLOOR_SHARE = 0.5;
|
|
|
|
/** The exit margin to hold back from a supervisor-enforced stop deadline. */
|
|
export const resolveSupervisorExitMarginMs = (stopTimeoutMs) =>
|
|
Math.min(
|
|
GATEWAY_SUPERVISOR_EXIT_MARGIN_MS,
|
|
Math.floor(Math.max(0, stopTimeoutMs) * GATEWAY_SUPERVISOR_EXIT_MARGIN_SHARE),
|
|
);
|
|
|
|
/** The post-drain reserve to hold back from a resolved shutdown budget. */
|
|
export const resolveShutdownReserveMs = (shutdownTimeoutMs) => {
|
|
const budgetMs = Math.max(0, shutdownTimeoutMs);
|
|
const drainFloorMs = Math.min(
|
|
GATEWAY_SHUTDOWN_DRAIN_FLOOR_MS,
|
|
Math.floor(budgetMs * GATEWAY_SHUTDOWN_DRAIN_FLOOR_SHARE),
|
|
);
|
|
return Math.min(GATEWAY_SHUTDOWN_RESERVE_MS, budgetMs - drainFloorMs);
|
|
};
|
|
|
|
// Escalation graces the Node recovery launcher applies to a stopping child. Kept
|
|
// here rather than in the launcher so both use the same escalation policy.
|
|
const RESPAWN_SIGNAL_EXIT_GRACE_MS = 1_000;
|
|
export const RESPAWN_SIGNAL_FORCE_KILL_GRACE_MS = 1_000;
|
|
const RESPAWN_SIGNAL_HARD_EXIT_GRACE_MS = 1_000;
|
|
|
|
// Every runRespawnedChild call site stamps one of these. The compile-cache
|
|
// marker is also used by a different respawner, but that one refuses foreground
|
|
// Gateway runs on darwin. Keep this list with the launcher deadline it authorizes.
|
|
const RESPAWN_LAUNCHER_MARKER_ENV_VARS = [
|
|
"OPENCLAW_NODE_UPDATE_RESPAWNED",
|
|
"OPENCLAW_COMPILE_CACHE_DISABLED_RESPAWNED",
|
|
"OPENCLAW_PACKAGED_COMPILE_CACHE_RESPAWNED",
|
|
];
|
|
|
|
/** Whether a `runRespawnedChild` parent started this process. */
|
|
export const isRespawnedByLauncher = (env) =>
|
|
RESPAWN_LAUNCHER_MARKER_ENV_VARS.some((name) => env[name] === "1");
|
|
|
|
// New launchers use the shared service policy. A serving Gateway separately
|
|
// retains the shorter deadline of an unidentified, already-running old launcher.
|
|
export const resolveLauncherStopTimeoutMs = ({ platform, foreground }) => {
|
|
const signalExitGraceMs =
|
|
platform !== "win32" && foreground
|
|
? GATEWAY_SERVICE_STOP_TIMEOUT_MS -
|
|
RESPAWN_SIGNAL_FORCE_KILL_GRACE_MS -
|
|
RESPAWN_SIGNAL_HARD_EXIT_GRACE_MS
|
|
: RESPAWN_SIGNAL_EXIT_GRACE_MS;
|
|
return signalExitGraceMs + RESPAWN_SIGNAL_FORCE_KILL_GRACE_MS;
|
|
};
|