Debounced jobs

[nandi95]

I had to implement a feature where a notification is sent out to the owner of the resource when the resource was commented on. The owner recieved an email for every comment. With a of burst activity (people replying to each other in quick succession) this would mean a lot of emails. This had to be debounced (every time someone replies restart the delay again).

I was bit suprised this wasn't already built in so I came up with the following solution.

This requires the database driver for the queue (but probably can be implemented with redis too)

// debounce the notification
$messagesCount = 1;

// if job exists remove
if (Cache::has($resource->getCacheKey() . '.message.added')) {
    $cacheValue = Cache::get($resource->getCacheKey() . '.message.added');

    DB::table('jobs')
        ->where('id', '=', $cacheValue['job_id'])
        ->delete();
    $messagesCount += (int) $cacheValue['messages_count'];
}

Cache::put(
    $resource->getCacheKey() . '.message.added',
    [
        /*
         * Returns the id.
         * @see Illuminate\Queue\DatabaseQueue::pushToDatabase
         */
        'job_id' => NotifyOfConversationMessageAdded::dispatch($resource, $messagesCount)->delay(now()->addMinutes(5)),
        /*
         * Number of messages that the notification is about.
         */
        'messages_count' => $messagesCount
    ],
    now()->addMinutes(5)
);

$resource->getCacheKey() is just a helper returning $this->getTable() . '.' . $this->getKey()

I could maybe make a PR for this but don't wanna spend the time before knowing it won't be in vain.

[gbradley]

I think this effect is achievable by using a combination of

• a delay
• using unique jobs to ensure only a single instance of NotifyOfConversationMessageAdded per resource is queued
• inside the job, check if its been delay minutes since the last comment; if so proceed with job, otherwise push back on the queue with a further delay

Granted it would be much nicer if this could be extracted away with a simple debounce() method or similar.

[nandi95]

inside the job, check if its been delay minutes since the last comment

That way we don't know then how many comments needs to be dealt with. All comments in the last {delay} minutes? All comments in the last {delay} + 3 minutes as if was released back to the queue? More?

[nandi95]

I think this should work.

The initial dispatch

// debounce the notification
$messagesCount = 1;

if (Cache::has($share->getCacheKey() . '.message.added')) {
    $messagesCount += (int) Cache::get($share->getCacheKey() . '.message.added');
}

Cache::put($share->getCacheKey() . '.message.added', $messagesCount, 300);

// it's a unique job so won't dispatch again
NotifyOfConversationMessageAdded::dispatch(
    $share->withoutRelations(),
    $shareLink->withoutRelations(),
    $messagesCount
)->delay(301);

The start of the unique job (released ($uniqueFor) after 5 minutes as well)

if (Cache::has($this->share->getCacheKey() . '.message.added')) {
    $this->delete();
    $messagesCount = (int) Cache::get($this->share->getCacheKey() . '.message.added');

    Cache::put($this->share->getCacheKey() . '.message.added', $messagesCount, 300);
    self::dispatch($this->share, $this->shareLink, $messagesCount)->delay(301);

    return;
}

[davutkmbr]

I created a simple solution using job middleware that resolves this issue. It may require customization based on your needs, but this version has worked for me.

https://gist.github.com/davutkmbr/43d442a39e2735274687cecdb70c2e74

[bentleyo]

@davutkmbr nice solution! I was thinking of solving the problem in a very similar way. The middleware makes this very clean.