feat(jobs): add queue delay resilience to scheduled job execution

Implement dedup key-based cron tracking to make scheduled jobs resilient to queue
delays. Even if a job is delayed by minutes, it will catch the missed cron window
by tracking previousRunDate in cache instead of relying on isDue() alone.

- Add dedupKey parameter to shouldRunNow() in ScheduledJobManager
  - When provided, uses getPreviousRunDate() + cache tracking for resilience
  - Falls back to isDue() for docker cleanups without dedup key
  - Prevents double-dispatch within same cron window

- Optimize ServerConnectionCheckJob dispatch
  - Skip SSH checks if Sentinel is healthy (enabled and live)
  - Reduces redundant checks when Sentinel heartbeat proves connectivity

- Remove hourly Sentinel update checks
  - Consolidate to daily CheckAndStartSentinelJob dispatch
  - Crash recovery handled by sentinelOutOfSync → ServerCheckJob flow

- Add logging for skipped database backups with context (backup_id, database_id, status)

- Refactor skip reason methods to accept server parameter, avoiding redundant queries

- Add comprehensive test suite for scheduling with various delay scenarios and timezones
This commit is contained in:
Andras Bacsai committed 2026-02-28 15:06:25 +01:00
1 parent f68793ed69
commit a0c177f6f2
5 files changed
+322 -106

No files matched your search

+6
View File
@@ -111,6 +111,12 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue
$status = str(data_get($this->database, 'status'));
if (! $status->startsWith('running') && $this->database->id !== 0) {
Log::info('DatabaseBackupJob skipped: database not running', [
'backup_id' => $this->backup->id,
'database_id' => $this->database->id,
'status' => (string) $status,
]);
return;
}
if (data_get($this->backup, 'database_type') === \App\Models\ServiceDatabase::class) {