Queued Job Failing

Question

I have a rather large Craft CMS queued job that copies multiple entries which is repeatedly failing.

The error I get is: The process "'/usr/local/bin/php' './craft' 'queue/exec' '621803' '300' '1' '9' '--verbose=1' '--interactive=' '--color='" exceeded the timeout of 300 seconds.

Is there anything I can do to ensure that it runs?

Hi Sean, thanks for providing that error message. I updated my answer in light of that with what I believe is your actual issue (I did keep most of the original answer around in case some of it might help someone else out at some point). — Mats Mikkel Rummelhoff, Jan 10 '23 at 22:59

Mats Mikkel Rummelhoff · Answer 1 · 2023-01-10T23:09:46.123

Your job is likely hitting the queue TTR (time to reserve), which by default is set to 300 seconds.

Assuming this is a custom job, you can actually set a different (i.e. much higher) TTR for this specific job by making your custom job class implement the RetryableJobInterface, and then add a getTtr() method to it, like this:

<?php
namespace foomodule\jobs;
use craft\queue\BaseJob;
use yii\queue\RetryableJobInterface;
class FooJob extends BaseJob implements RetryableJobInterface {
public function getTtr()
{
    // Give this job a TTR of 1 hour (3600 seconds)
    return 3600;
}

...


}

Alternatively, you can override Craft's queue config to set a higher TTR for all jobs (including core ones). The below should go into your site's config/app.php file:

return [
    'components' => [
        'queue' => [
            'ttr' => 3600, // Let all jobs run for an hour if need be
        ],
    ],
    ...
];

Aside from going over the time to reserve (which should be a relatively rare occurrence), one of the most common reasons for a queue job failing is exceeding PHP resource constraints – i.e. timeouts or memory restrictions.

If that's the case, the first order of business should be to set up a dedicated queue runner, which runs the Craft queue using the PHP CLI. The CLI typically doesn't suffer the same resource constraints as PHP-FPM does (you'll still need to mind the time to reserve though, as outlined above).

Moving to a dedicated queue runner involves setting Craft's runQueueAutomatically config setting to false, which prevents Craft from running the queue automatically over HTTP (the default behaviour). Then, you'll need to set up some sort of alternative queue runner.

Ideally, the queue runner should be an always-running deamon. If you use a server provisioning software like Ploi or Laravel Forge, those make it fairly straight-forward to create and manage such deamons. The command you'll want the deamon to run is craft queue/listen, i.e. something like php /path/to/project/craft queue/listen.

Another option is adding a cron job that runs the queue every minute, e.g. something like * * * * * /path/to/project/craft queue/run.

Finally, there's the Async Queue plugin, which could be helpful if neither of the above methods are feasible.

I'd also advise reading this article, which goes into detail on all of the above.

If neither the TTR nor hitting resource constraints is the reason for why a job is failing, this section in the official docs on dealing with failed jobs might be useful (you might also want to update the question if you have more details about what exactly is "failing" with your jobs).

Queued Job Failing

1 Answers1