How to Build a PHP Queue System

Question

I had to build a PHP Queue System, and found this brilliant article ~~http://squirrelshaterobots.com/programming/php/building-a-queue-server-in-php-part-1-understanding-the-project~~ and I used it to create a PHP queue system, its very easy to set-up and use.

Below is the code for queue.php, run from shell (puTTy or somesuch).

<?PHP 

//. set this constant to false if we ever need to debug
//. the application in a terminal.
define('QUEUESERVER_FORK', true);

//////// fork into a background process ////////
if(QUEUESERVER_FORK){    
    $pid = pcntl_fork(); 
    if($pid === -1) die('error: unable to fork.');    
    else if($pid) exit(0);        
    posix_setsid();    
    sleep(1);        
    ob_start();
}

$queue = array();

//////// setup our named pipe ////////
$pipefile = '/tmp/queueserver-input';

if(file_exists($pipefile))    
    if(!unlink($pipefile))         
        die('unable to remove stale file');

umask(0);


if(!posix_mkfifo($pipefile, 0666))    
    die('unable to create named pipe');

$pipe = fopen($pipefile,'r+');

if(!$pipe) die('unable to open the named pipe');

stream_set_blocking($pipe, false);

//////// process the queue ////////
while(1){    

    while($input = trim(fgets($pipe))){        
        stream_set_blocking($pipe, false);        
        $queue[] = $input;    
    }    

    $job = current($queue);    
    $jobkey = key($queue);    

    if($job){        
        echo 'processing job ', $job, PHP_EOL;                
        process($job);                
        next($queue);        
        unset($job, $queue[$jobkey]);            
    }else{        
        echo 'no jobs to do - waiting...', PHP_EOL;        
        stream_set_blocking($pipe, true);    
    }        

    if(QUEUESERVER_FORK) ob_clean();

}

?>

The hardest part was getting the pcntl functions to work on my server.

My question is "How do i get the job to start automatically when/if the server has to restart?"

As noted in comments, edited broken link and pointed to excellent web archive for posterity.

@MartinBean How do i get the job to start automatically when/if the server has to restart? — Folding Circles, Jan 04 '13 at 00:14
you can modify server starting script to do this, or you can add script which will report state of script/server and second script for restarting a job and then use them both from local or remote host to monitor server/restart job (with cronjob) — llamerr, Jan 04 '13 at 00:17
@Calos Keep in mind to run a PHP script using pcntl_*() functions PHP has to be run as CGI, it will not allow commandline usage if run as Apache module. — kittycat, Jan 04 '13 at 02:27
@Calos Do you happen to have the brilliant article of the page saved? The website seems to be down and it was exactly what I personally needed :) — Webeng, May 11 '16 at 16:34
It WAS a brilliant article - I bookmarked it for good reason. I think [this](http://web.archive.org/web/*/http://squirrelshaterobots.com/programming/php/building-a-queue-server-in-php-part-1-understanding-the-project/) is our best option now - January 2016 works. — TacoV, Aug 23 '16 at 09:10

score 11 · Accepted Answer · answered Jan 04 '13 at 00:49

My question is "How do i get the job to start automatically when/if the server has to restart?"

By adding it to the list of things started when the server starts. Unfortunately the instructions for doing so vary wildly by operating system and OS version. You probably want to use something slightly more cross-platform. I've had a great deal of luck with supervisor, which you can probably find in the package repos on your OS of choice.

That said, you are going down the route of madness. The thing you're doing has been done before, better, by awesome people. Check out the Gearman work queue system and the accompnaying PECL extension. It happens that supervisor is pretty handy for keeping your Gearman workers alive as well.

How to Build a PHP Queue System

1 Answers1

Linked