Code Monkey home page Code Monkey logo

laravel-queue-debouncer's Introduction

Laravel Queue Debouncer

Easy queue job debouncing

Requirements

  • Laravel >= 6.0
  • An async queue driver

Note: v2.0 requires at least Laravel 6. For Laravel 5.5 ..< 6.0 support, check out v1.0.2

Installation

composer require mpbarlow/laravel-queue-debouncer

Background

This package allows any queue job or chain in your Laravel application to be debounced, meaning that no matter how many times it’s dispatched within the specified timeout window, it will only run once.

For example, imagine you dispatch a job to rebuild a cache every time a record is updated, but the job is resource intensive. Debouncing the job with a five minute wait would ensure that the cache is only rebuilt once, five minutes after you finish making changes.

Usage

Debounced jobs can largely be treated like any other dispatched job. The debouncer takes two arguments, the actual $job you want to run, and the $wait period.

As with a regular dispatch, $job can be either a class implementing Illuminate\Foundation\Bus\Dispatchable, a chain, or a closure. $wait accepts any argument that the delay method on a dispatch accepts (i.e. a DateTimeInterface or a number of seconds).

The debouncer returns an instance of Illuminate\Foundation\Bus\PendingDispatch, meaning the debouncing process itself may be assigned to a different queue or otherwise manipulated.

Calling the debouncer

There are several ways to use the debouncer:

Dependency Injection

use App\Jobs\MyJob;
use Mpbarlow\LaravelQueueDebouncer\Debouncer;

class MyController
{
    public function doTheThing(Debouncer $debouncer)
    {
        $debouncer->debounce(new MyJob(), 30);

        // The class is also invokable:
        $debouncer(new MyJob(), 30);
    }
}

Facade

use App\Jobs\MyJob;
use Mpbarlow\LaravelQueueDebouncer\Facade\Debouncer;

Debouncer::debounce(new MyJob, 30);

Helper function

use App\Jobs\MyJob;

debounce(new MyJob(), 30);

Trait

use Illuminate\Bus\Queueable;
use Illuminate\Foundation\Bus\Dispatchable;
use Mpbarlow\LaravelQueueDebouncer\Traits\Debounceable;

class MyJob {
    use Debounceable, Dispatchable, Queueable;
}

MyJob::debounce('foo', 'bar', 'baz', 30);

When monitoring the queue, you will see an entry for the package’s internal dispatcher each time the debounced job is queued, but the job itself will only run once, when the final wait time has expired.

For example, assuming the following code was ran at exactly 9am:

class MyJob
{
    use Dispatchable;

    public function handle()
    {
        echoHello!\n”;
    }
}

$job = new MyJob();

debounce($job, now()->addSeconds(5));
sleep(3);

debounce($job, now()->addSeconds(5));
sleep(3);

debounce($job, now()->addSeconds(5));

you should expect the following activity on your queue:

[2020-03-11 09:00:05][vHmqrBYeLtK3Lbiq5TsTZxBo2igaCZHC] Processing: Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:05][vHmqrBYeLtK3Lbiq5TsTZxBo2igaCZHC] Processed:  Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:08][LXdzLvilh5qhew7akNDnibCjaXksG81X] Processing: Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:08][LXdzLvilh5qhew7akNDnibCjaXksG81X] Processed:  Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:11][MnPIqk5fCwXjiVzuwPjkkOdPPBn0xR4d] Processing: Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:11][MnPIqk5fCwXjiVzuwPjkkOdPPBn0xR4d] Processed:  Closure (DispatcherFactory.php:28)
[2020-03-11 09:00:11][I2hvBoCB71qZQeD4umn5dd90zJUCAlJ5] Processing: App\Jobs\MyJob
Hello!
[2020-03-11 09:00:11][I2hvBoCB71qZQeD4umn5dd90zJUCAlJ5] Processed:  App\Jobs\MyJob

Customising Behaviour

This package provides a few hooks to customise things to your needs. To override the default behaviour, you should publish the config file:

php artisan vendor:publish --provider="Mpbarlow\LaravelQueueDebouncer\ServiceProvider"

This will copy queue_debouncer.php to your config directory.

Cache key provider

To identify the job being debounced, the package generates a unique key in the cache for each job type.

Two cache key providers are included:

  • Mpbarlow\LaravelQueueDebouncer\Support\CacheKeyProvider (default): uses the config's cache_prefix value with either: the fully-qualified class name for class-based jobs; or a SHA1 hash of the closure for closure jobs.
  • Mpbarlow\LaravelQueueDebouncer\Support\SerializingCacheKeyProvider: uses the config's cache_prefix value with a SHA1 hash of the serialized job. If you want to debounce jobs based on factors beyond their class name (for example, some internal state), this is the provider to use. This is also required if you need to debounce chains, as the default provider will debounce all chains dispatched by your application as if they are the same job, regardless of what jobs are contained within.

Alternatively, you can provide your own class or closure to cover any other behaviour you might need:

If you provide a class, it should implement Mpbarlow\LaravelQueueDebouncer\Contracts\CacheKeyProvider. Please note that your class is responsible for fetching and prepending the cache prefix should you still desire this behaviour.

Class-based providers may be globally registered as the default provider by changing the cache_key_provider value in the config. Alternatively, you may "hot-swap" the provider using the usingUniqueIdentifierProvider method:

$debouncer
    ->usingCacheKeyProvider(new CustomCacheKeyProvider())
    ->debounce($job, 10);

If you provide a closure, it may only be be hot-swapped:

$debouncer
    ->usingCacheKeyProvider(fn () => 'my custom key')
    ->debounce($job, 10);

Closure providers automatically have their value prefixed by the configured cache_prefix. To override this behaviour, implement a class-based provider that accepts a closure in its constructor, then calls it in its getKey method.

Unique identifier provider

Each time a debounced job is dispatched, a unique identifier is stored against the cache key for the job. When the wait time expires, if that identifier matches the value of the current job, the debouncer knows that no more recent instances of the job have been dispatched, and therefore it is safe to dispatch it.

The default implementation produces a UUID v4 for each dispatch. If you need to override this you may do so in the same manner as cache key providers, globally registering a class under the unique_identifier_provider key in the config, or hot-swapping using the usingUniqueIdentifierProvider method:

$debouncer
    ->usingUniqueIdentifierProvider(new CustomUniqueIdentifierProvider())
    ->debounce($job, 10);

$debouncer
    ->usingUniqueIdentifierProvider(fn () => 'my custom identifier')
    ->debounce($job, 10);

Class-based providers should implement Mpbarlow\LaravelQueueDebouncer\Contracts\UniqueIdentifierProvider.

Licence

This package is open-source software provided under the The MIT License.

laravel-queue-debouncer's People

Contributors

a-bashtannik avatar mpbarlow avatar onlime avatar roblesterjr04 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

laravel-queue-debouncer's Issues

Job Chaining

Thanks for your package, i've been trying to implement my own solution but this is so much easier and well thought out.
Quick question: How would you go about job chaining?

        $job = MyJob::withChain([
             new MySecondJob('a')
        ]);

Define queue on which closure is dispatched

Hi! I would like to create a dedicated queue for the debounce closures. We're debouncing a lot of jobs and it's messing with the main queue. I've gone through the code and haven't seen an option to define the queue name on which the closure is pushed.

What do you think of this? Could probably be an optional argument to the debounce helper / trait as well as a config setting.

Feature: use arguments to uniquely debounce jobs

A possible solution to this problem was described in #2 but that's only relevant for an older version of this package.

I think having the ability to debounce the same job, but with different arguments would be very useful and is the feature I'm looking for.

In my use case, we dispatch the same job, to update elasticsearch documents corresponding to the set of IDs passed as the job arguments. Eg. the argument might be [123, 456] or [345], or [456]. We also might end up queueing jobs with the same parameters repeatedly, and might end up with 5 instances of [456].

I want to be able to debounce the job based on the complete set of supplied parameters. So multiple [456] should be debounced into only one run after the delay expires. Same with multiple [123, 456] jobs.

I do not need the ability to debounce within the arguments, just to treat each unique combination of args as its own debounced job.

Deprecated API: Opis\Closure\SerializableClosure implements the Serializable interface

[2024-05-16 07:24:44] production.WARNING: Opis\Closure\SerializableClosure implements the Serializable interface, 
which is deprecated. Implement __serialize() and __unserialize() instead (or in addition, if support for old PHP versions is 
necessary) in /var/www/api.crmest.com/vendor/opis/closure/src/SerializableClosure.php on line 18

There is no updates on opis/closure package for 2 years.

Feature: debounce a same job but with different arguments

We only debounce jobs by same job, but if we have a same job but with different process data, debouncers fails.

For example "DebounceableSendUserNotification". We have the same job but for two different users. And when second user require the job, he stop all anothers debounceable jobs.

Serializing problem

When debouncing one of my jobs, I get this error:

   "message": "serialize(): &quot;date&quot; returned as member variable from __sleep() but does not exist",
    "exception": "ErrorException",
    "file": "/var/www/html/laravel/vendor/opis/closure/functions.php",
    "line": 20,
    "trace": [
        {
            "file": "/var/www/html/laravel/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php",
            "line": 259,
            "function": "handleError",
            "class": "Illuminate\\Foundation\\Bootstrap\\HandleExceptions",
            "type": "->"
        }

Happens with both CacheKeyProviders and when debouncing chains or single jobs.

Testing Assertions

Hello, thanks for this great package! Just wanted to ask how you recommend testing this, i.e. how do you think is the best way to assert that the job has been debounced :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.