Where your queue jobs go to die under load

The support ticket said "I never got my invoice email." I checked the logs. The job ran. It succeeded. failed_jobs was empty. Everything said the email went out. It did not.

That's the worst kind of queue bug: not a loud exception, but a job that quietly never finished the work it claimed to. Under normal traffic you never see it. Under load, jobs start vanishing, and the system keeps insisting everything is fine.

Here's how jobs actually disappear, and how I stop them.

A job doesn't "fail." It gets killed.

The mental model most people have is that a job either succeeds or throws an exception that lands in failed_jobs. But there's a third path, and it's the one that hurts: the worker process gets killed mid-job, before it can record anything at all.

Horizon kills a worker for two reasons, and both are silent by default:

• Memory. The worker exceeds memory (default 128 MB) and Horizon restarts it. Whatever it was running just stops.
• Timeout. The job runs longer than timeout and gets SIGKILL'd. No catch, no finally, no failed() hook. Gone.

At demo scale your jobs are small and fast, so neither ever triggers. At real scale, a report job loads 40,000 rows into memory, or an external API gets slow and your HTTP call hangs for 90 seconds. The worker dies, and because it was killed rather than throwing, nothing lands in failed_jobs.

That curve is the tell. Zero failures in testing, then a knee where memory pressure and slow dependencies start killing workers faster than they finish. The jobs aren't failing — they're being executed halfway and dropped.

The timeout chain has to be ordered

This is the single most common misconfiguration I fix. There are three timers, and if they're in the wrong order, retries eat each other.

• retry_after (in config/queue.php) — how long before the queue assumes a job died and hands it to another worker.
• timeout (the worker flag / Horizon supervisor) — how long before the process is killed.
• The job's own $timeout property, if set.

The rule: timeout must be shorter than retry_after. If retry_after is 90 and timeout is 120, the queue releases the job for a second worker at 90 seconds while the first worker is still running it. Now the same job runs twice, concurrently. That's how a "send email" becomes two emails, or a charge becomes two charges.

// config/horizon.php — one supervisor
'supervisor-1' => [
    'connection' => 'redis',
    'queue' => ['default'],
    'balance' => 'auto',
    'maxProcesses' => 10,
    'memory' => 256,        // MB before restart — raise it for report jobs
    'timeout' => 60,        // MUST be < retry_after
    'tries' => 3,
    'backoff' => [10, 30, 60], // wait 10s, then 30s, then 60s between retries
],

backoff as an array gives you exponential-ish retries: don't hammer a struggling downstream service three times in three seconds — space the attempts out so it can recover.

Tell the job when to stop trying

tries => 3 is fine until a job is retried three times over an hour and the underlying action is no longer valid — the order was cancelled, the user was deleted. retryUntil is better than a raw count for anything time-sensitive:

public function retryUntil(): \DateTime
{
    return now()->addMinutes(10);
}

Now the job retries as many times as it can inside a 10-minute window, then stops. For a payment webhook or a notification, "keep trying for 10 minutes, then give up and alert" is almost always what you actually want, not "try exactly 3 times whenever."

The two configs that make failures visible

Everything above is prevention. This is the part that turns a silent drop into a loud, catchable failure:

// worker killed at timeout
// no failed() hook fires
// failed_jobs stays empty
// support ticket is your only signal

public int $tries = 3;
public int $timeout = 55;

public function failed(\Throwable $e): void
{
    Log::error('Invoice job failed', [
        'user' => $this->user->id,
        'error' => $e->getMessage(),
    ]);
    // alert, mark record, compensate
}

Two things make the difference. First, set $timeout on the job below the worker timeout so the job throws a TimeoutExceededException you can catch, instead of getting SIGKILL'd with no trace. Second, actually implement failed() — it's the only place you get to react when all retries are exhausted.

The fix for vanishing jobs is almost never "add more retries." It's making the failure loud enough to see: order the timeout chain, cap memory sensibly, set a job-level timeout under the worker's, and implement failed(). Do that, and a dropped job stops being a support ticket three days later and becomes an alert you get in the same minute it happens.