What is the proper way to setup and use php-resque?

Question

I am trying to use php-resque to queue and execute ffmpeg conversions on my server. I understand broadly how it should work, but I am having some trouble with the details and can not find any tutorials. Specifically, I don't understand where I should place my job classes, and how to give the classes to my workers and start my workers. The read me only says "Getting your application underway also includes telling the worker your job classes, by means of either an autoloader or including them."

Hopefully someone can outline the overall structure of using php-resque.

Answer 1

You can put your job classes where you want. It'll depend on your application structure.

How to create a job class

For example, let's suppose the class VideoConversion, used for the ffmpeg conversion.

class VideoConversion {

    public function perform() {
        // The code for video conversion here
    }

}

In your main application, before using php-resque, let's say you have something like that

public function uploadVideo() {
    // Upload and move the video to a temp folder
    // Convert the video
}

And you want to enqueue the 'convert video' part. Let's just queue it to the convert queue:

public function uploadVideo() {
    // Upload and move the video to a temp folder
    // Let's suppose you need to convert a 'source video' to a 'destination video'
    Resque::enqueue('convert', 'VideoConversion', array('origine-video.avi', 'destination-video.avi'));
}

When queuing the job, we passed the path to the source and destination video to the VideoConversion class. You can pass other argument, it'll depend on how your VideoConversion class is written.

A worker will then poll the convert queue, and execute the VideoConversion job. What the worker will do is to instantiate the VideoConversion class, and execute the perform() method.

The job arguments (array('origine-video.avi', 'destination-video.avi')), third argument when queueing the job with Resque::enqueue, will be available inside the perform() method via $this->args.

# VideoConversion.php
class VideoConversion
{
    public function perform() {
    // $this->args == array('origine-video.avi', 'destination-video.avi');
    // Convert the video
}

Find your job classes

The VideoConversion class can be put anywhere, but you have to tell your workers where to find it. There's multiple ways to do that

Put you jobs classes in the include_path

In your .htaccess or the apache config, add the directory containing all your job classes to the include path. Your workers will automatically find them.

Main issue with this method is that all your jobs classes must be in the same folder, and that all your job classes are available everywhere.

Tell each worker where to find your job classes

When starting the worker, use the APP_INCLUDE argument to point to the job classes 'autoloader'.

APP_INCLUDE=/path/to/autoloader.php QUEUE=convert php resque.php

The above command will start a new worker, polling the queue named convert. We're also passing the file /path/to/autoloader.php to the worker. (see here to learn to start a worker)

Technically, the worker will include that file with include '/path/to/autoloader.php';.

You can then tell the workers how to find your job classes:

Use basic include

In the '/path/to/autoloader.php':

include /path/to/VideoConversion.php
include /path/to/anotherClass.php
...

Use an autoloader

Use php autoloader to load your job classes.

Use set_include_path()

set_include_path('path/to/job');

That way, your jobs are in the include_path just for this worker.

Closing thought

APP_INCLUDE is binded to the worker you're starting. If you're starting another worker, use APP_INCLUDE again. You can use a different file for each worker.

You can also design your job classes to execute more than one job. There's a tutorial explaining how to do that. It covers from the basic of a queue system to how to use and implement it.

If it's still not enough, take a look at resque documentation. php-resque API is exactly the same. Only difference is that Resque job classes are written in Ruby, whereas php-resque's one are in php.