Laravel queued jobs processed immediately even with a delay

Question

I'm currently developing my personal application that is about private servers (for example, Minecraft servers) and since querying the server takes some time, I decided to implement queued jobs. However, they are not working properly, and they run immediately when called even though they are delayed, causing a massive latency in a page request.

Here's my HomeController's index() which calls the job to update every server with a 30 seconds delay:

public function index()
{
    $servers = Server::all();

    foreach($servers as $server)
    {
        // Job Dispatch
        $job = (new UpdateServer($server->id))->delay(30);
        $this->dispatch($job);
    }
    return view('serverlist.index', compact('servers'));
}

The job class that updates the servers is the following:

class UpdateServer extends Job implements SelfHandling, ShouldQueue
{
    use InteractsWithQueue, SerializesModels;
    protected $id;

    public function __construct($id)
    {
        $this->id = $id;
    }

    public function handle(){
        $server = Server::findOrFail($this->id);

        // Preparing the packet
        $test = new RAGBuffer();
        $test->addChar('255');
        $test->addChar('1');
        $test->addShort(1 | 8);

        // Finding the server
        $serverGame = new RAGServer($server->server_ip);

        // Get server information
        $status = $serverGame->sendPacket($test);

        $server->onlinePlayers = $status->getOnline();
        $server->peakPlayers = $status->getPeak();
        $server->maxPlayers = $status->getMax();

        if (!$server->save()) {
            // Error occurred
        }
    }
}

Whenever the HomeController's index() is run, there's a massive delay in the page request. I followed the tutorial at Laravel's Official Webpage, and I tried to find answers, but I didn't find anything.

So, what am I doing wrong? Why isn't the job getting delayed 30 seconds and then doing this in background in my server?

Also: The handle() is doing what it is supposed to. It queries the server, sends packets, and updates my database with the correct information.

Answer 1

You have to set up the queue driver you want to use in your project's root dir's .env file.

By default, the queue driver is sync which does exactly what you are describing, executing queues immediately.

You can choose of some different queue drivers, such as beanstalked or redis (which would be my choice). There's an excellent freebie on laracasts.com about setting up a beanstalked queue.

To view all available queue driver options in Laravel, have a look here.

Here's a .env example

APP_ENV=local
APP_DEBUG=true
APP_KEY=SomeRandomString

DB_HOST=localhost
DB_DATABASE=homestead
DB_USERNAME=homestead
DB_PASSWORD=secret

CACHE_DRIVER=file
SESSION_DRIVER=file
QUEUE_DRIVER=sync      // <-- Put the desired driver here

MAIL_DRIVER=smtp
MAIL_HOST=mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null

Answer 2

This was driving me crazy for ages before I realised that Laravel 5.7 renamed QUEUE_DRIVER to QUEUE_CONNECTION in the .env files

Answer 3

For someone who has made the changes from previous answers and still didn't work, check the default value of the queue file like this: dd(Config::get('queue.default'))

For me it didn't change until flushing the config cache:

php artisan config:clear

Answer 4

To test locally you could set the driver to

QUEUE_DRIVER=database

And run php artisan queue:table.

And then php artisan migrate, so you would get your queue saved into the database, so you visually could see what`s going on.

And to run your queues, simply run php artisan queue:listen ... and leave it running as you do with artisan serve.

Answer 5

If you are running on php artisan serve, restart this and run php artisan serve again. This worked for me after hours trying to wonder what it was. :)

Answer 6

Ensure that

'default' => env('QUEUE_DRIVER', 'database'), 

in file config/queue.php

And

QUEUE_DRIVER=database 

in the .env file to ensure the database driver is used.

Answer 7

If you are running tests against the queue service through phpunit, make sure that

<env name="QUEUE_DRIVER" value="X"/>

in phpunit.xml doesn't override your desired queue driver.

Answer 8

It's because the delay function takes an absolute date in the future

UpdateServer::dispatch($server->id)->delay(now()->addSeconds(30))

Answer 9

In my case, I had to implement ShouldQueue and use the Queueable trait:

class CustomNotification extends Notification implements ShouldQueue{
    use Queueable;
...

Answer 10

Even if you have configured everything properly this can still happen. We had this problem with Laravel 5.4 where we created a lot of jobs, some delayed, and added them to the queue via Queue:bulk($jobs). This call and also Queue::push($job) completely ignore the delay and cause the job to be processed immediately.

If you want your job to be put on the queue as you configured it you must call dispatch($job).

Answer 11

This is the complete steps to create user API and store its history in jobs table. In Jobs class:

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use App\Repositories\Eloquent\ApiRepo as ApiRepo;
use Log;
use App\Models\User;

class UserProcess implements ShouldQueue
{
     use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

     /**
     * The number of times the job may be attempted and override the queue tries.
     *
     * @var int
     */
    public $tries = 3;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public $user;

    public function __construct(User $user)
    {
        $this->user = $user; 
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {       
        try {
            // make api call
           Log::info("inside handle".$this->user->id);
            $apiRepo = new ApiRepo;
            $response = $apiRepo->getUserDetails($this->user->id);
            Log::info("Response".$response);
        } catch (\Throwable $exception) {
            if ($this->attempts() > 3) {
                // hard fail after 3 attempts
                throw $exception;
            }
            // requeue this job to be executes
            // in 3 minutes (180 seconds) from now
            $this->release(180);
            return;
        }
    }

}

In Controller Class:

namespace App\Http\Controllers\API;

use App\Http\Controllers\Controller;
use Illuminate\Http\Request;
use Validator;
use App\Models\User;
use App\Jobs\UserProcess;
use App\Models\UserHistory;
use Carbon\Carbon;

class UserController extends Controller
{
    public function create(Request $request)
    {
        $rules = [
            'first_name' => 'required|string|max:100',
            'last_name' => 'required|string|max:100',
            'email' => 'required|string|email|unique:users,email',
            'phone_number' => 'required|string|max:10',
            'address' => 'string',
            'date_of_birth' => 'string|date_format:Y-m-d|before:today',
            'is_vaccinated' => 'string|in:YES,NO',
            'vaccine_name' => 'string|required_if:is_vaccinated,==,YES|in:COVAXIN,COVISHIELD'
        ];

        $validator = Validator::make(array_map('trim', ($request->all())),$rules);

        if($validator->fails()){
            return response()->json($validator->errors());       
        }else{
            $user = new User;
            $user->first_name = $request->first_name;
            $user->last_name = $request->last_name;
            $user->email = $request->email;
            $user->phone_number = $request->phone_number;
            $user->address = $request->address;
            $user->date_of_birth = $request->date_of_birth;
            $user->is_vaccinated = $request->is_vaccinated;
            $user->vaccine_name = $request->vaccine_name;
            $user->save();

            $token = $user->createToken('auth_token')->plainTextToken;
            if($user->save()){
                $job = (new UserProcess($user))->delay(Carbon::now()->addMinutes(1));
                $this->dispatch($job);
            
                return response()
                ->json(['data' => $user,'status' => '200','message' => 'User Added Successfully','access_token' => $token, 'token_type' => 'Bearer']);
            }else{
                return response()
                ->json(['data' => $user,'status' => '409','message' => 'Something went wrong!']);
            }
        }
    } 
}

In ApiRepo Class:

namespace App\Repositories\Eloquent;

use App\Repositories\ApiInterface;
use Illuminate\Http\Request;
use App\Http\Requests;
use Illuminate\Http\Response;
use App\Http\Controllers\Controller;
use App\Models\User;
use App\Models\UserHistory;
use Log;

class ApiRepo implements ApiInterface {

     public function getUserDetails($userid) {
        Log::info('User ID - '.@$userid);
        $user_history = new UserHistory();
        $save_user = User::find($userid);
        Log::info('User Json Data - '.@$save_user);
        $user_history->user_id = $userid;
        $user_history->first_name = $save_user->first_name;
        $user_history->last_name = $save_user->last_name;
        $user_history->email = $save_user->email ;
        $user_history->phone_number = $save_user->phone_number;
        $user_history->address = $save_user->address;
        $user_history->date_of_birth = $save_user->date_of_birth;
        $user_history->is_vaccinated = $save_user->is_vaccinated;
        $user_history->vaccine_name = $save_user->vaccine_name;
        $user_history->save();
        if($user_history->save()){
            Log::info('User history Saved!');
        }
     }
}