7 Commits
Author SHA1 Message Date
Andras Bacsai 1a2a5362ad fix(security): secure remote command secrets and clone volume data 2026-10-27 09:47:18 +01:00
Andras BacsaiandClaude Opus 5.5 cf9f254507 fix(server-transfer): close security holes in import, claim, and archives
Import:
- Check every uuid in a bundle against a strict id format before any
  write, and quote the SSH mux socket and key paths in local shell
  commands. A bundle uuid could run commands in the Coolify container.
- Validate bundle fqdns, git source URLs, paths, networks, cron
  expressions, and database ports; cap the number of items.
- Do not confirm that another team has a uuid.
- Refuse a project icon path with a uuid that is not a plain id.

Claim, finish, and migrate:
- Pass checklist step keys to wire:click with Js::from (stored XSS).
- Link only to https target server URLs.
- Validate target_instance_url in /transfer/complete; never prefill an
  unverified URL for a transfer back; scope the transfer-back token to
  write+deploy for 1 hour.
- Release on an owner marker only when it is newer than the own claim.
- Do not touch apps that only use the server as an additional
  destination when an archive is released or deleted.
- Give Sentinel a new token on claim.
- Add a checklist step to remove the previous instance's SSH key.
- Block a transfer while backups or scheduled tasks run.
- Authorize refreshMigration and PrivateKey checkConnection, clear
  tokens from Livewire state, hide migration log/error from members.

Archives:
- Refuse ip, user, port, private key, and Cloudflare tunnel changes.
- Throttle Enable management and require a free server slot for it and
  for a transfer back.
- Skip cloud provider calls.
- Re-read force_disabled before each SSH command, so running jobs stop.
- Probe the import check target at the resolved address only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 21:02:32 +02:00
Andras BacsaiandClaude Opus 5.5 7c14c0098e feat(server-transfer): run instance migration as a background job (#12180)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:03:45 +02:00
Andras BacsaiandClaude Opus 5.5 69c809f4a2 fix(server): stop marking servers offline after one failed check
A single failed SSH attempt or any unrelated exception in
ServerConnectionCheckJob marked the server unreachable and unusable, so
deployments failed with "Server is not functional" while the server was
online.

- Retry a failed SSH attempt once after 3 seconds within the same check.
- Mark the server unreachable only after UNREACHABLE_THRESHOLD (2)
  consecutive failed checks; notifications use the same threshold.
- Keep the stored state when the job fails for a reason other than SSH or
  Docker (database, cache, bugs).
- Raise the job timeout to 60 seconds and the overlap lock above it. The
  old 15 second timeout killed checks before they wrote a result.
- Run the Docker and Compose version commands once with a 15 second
  timeout. A failed Compose check no longer marks the server unusable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 09:30:19 +02:00
Andras Bacsai d28bc91ff5 fix(scheduler): read fresh server state in each job and log scheduling decisions
Horizon workers kept Server::findCached() results between jobs, so scheduled
database backups used stale reachability and subscription state. A backup that
was skipped with the stale state counted as done for that day. Flush the
Server identity map before each queue job.

Add debug logs for the full path of a scheduled run, so a multi-node cloud
instance shows where a run is lost:
- manager start delay, gap since the previous start, host, pid, memory, and
  a warning when the start is later than the catch-up window
- skip logs with the server state that was used and the current database state
- occurrence enqueued, claimed, rejected, completed and failed, with host and
  the delay since the due time; a warning when the previous run is still open
- count of occurrences still open after 15 minutes on each manager run
- database backup runs that end without a backup
- server connection state changes with the checking node and the SSH error

Also keep publishing pending occurrences when one of them fails.
2026-09-28 16:15:00 +02:00
Kevin Dench 07d6c3aeeb fix(server): reset unreachable_count after successful connection check (#11417) 2026-08-26 14:53:42 +02:00
Andras Bacsai c8a332a3bc fix(servers): isolate cloud status checks from SSH checks
Track provider state independently, skip SSH work for placeholder IPs, and clean up failed cloud server provisioning.
2026-07-11 22:52:10 +02:00