Job has been attempted too many times or run too long

Webnet picture Webnet · Oct 31, 2018 · Viewed 28.7k times · Source

I have a job that works flawless locally, but in production I run into issues where it doesn't work. I've encompassed the entire handle() with a try/catch and am not seeing anything logged to Bugsnag, despite many other exceptions elsewhere from being deployed.

public function handle() {
    try {

        // do stuff

    } catch (\Exception $e) {
        Bugsnag::notifyException($e);

        throw $e;
    }
}

According to Laravel Horizon this queue job runs for 0.0026001930236816406 seconds and I never see it work and never see any other errors in the failed_jobs table as it relates to this job.

config/queue.php

    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => 'default',
        'retry_after' => (60 * 10), // 10 minutes
        'block_for' => null,
    ],

config/horizon.php

'environments' => [
    'production' => [
        'supervisor'        => [
            'connection'    => 'redis',
            'queue'         => [
                'default',
            ],
            'balance'       => 'auto',
            'processes'     => 10,
            'tries'         => 3,

            // 10 seconds under the queue's retry_after to avoid overlap
            'timeout'       => (60 * 10) - 10, // Just under 10 mins
        ],

If something is causing this job to retry over and over, how can I find out how? I'm at a loss.

Investigation thus far

  • My expectation is I should be able to run the query:
SELECT DISTINCT exception, COUNT(id) as errors
FROM failed_jobs 
WHERE payload LIKE '%[TAG-JOB-HAS]%' 
GROUP BY exception;

To see more than this error message:

Job has been attempted too many times or run too long

but that's all I see.

  • Laravel Horizon's dashboard shows the job in question is running for < 1 second, so I know it's not actually timing out.

Answer

Anil Kumar picture Anil Kumar · Nov 6, 2018

Try to catch the exception in the failed method given by laravel

/**
* The job failed to process.
*
* @param  Exception  $exception
* @return void
*/
public function failed(Exception $exception)
{
    // Send user notification of failure, etc...
}

and check whether your default queue driver in local is sync then its expected behavior.