RedisBatchRepository - a redis driver for batch jobs instead of database table job_batches

[cyppe]

Hi!

Work in progress of a driver for storing batch jobs in Redis including atomic locks.

It's fully compatible with Horizon and pruning of jobs (php artisan queue:prune-batches --hours=1 --unfinished=1 --cancelled=1)

Im not a package developer so don't have the experience to make one. But I think this is something really uiseful as I had massive performance issues when running 50+ workers on batches where table lock could add ~4 seconds delay on each job and blocking workers from fetching next job.

This this implementation I cut time in a 1000 job batch with 50 workers from 100+ seconds to 15 seconds.

Again - still work in progess. But please take a look and share some feedback. And if anyone want to build a package of it and also improve it it would be awesome! Just ping me so I can follow along.

My work so far is in this Gist: https://gist.github.com/cyppe/3a099fb97273f5ac08c6bb215792b538

[cyppe]

Created a package so it's easier to use. https://github.com/cyppe/laravel-batch-jobs-redis-driver

[borjajimnz]

Hello @cyppe just tested this driver and works perfectly, thanks for sharing. cc @themsaid

[ercsctt]

This is awesome, would love to see this supported by default in the framework.