Delayed::Threaded

Allows to start DJ in the same process using Thread.new { ... }

Extracted from JRuby-Rack-Worker: an effort to run Ruby workers in threads with Java (JRuby-Rack) deployments (previously known as Delayed::JRubyWorker).

NOTE: JRuby only, for now (PRs welcome)!

Installation

gem 'delayed-threaded'

and bundle or install it yourself as gem install delayed-threaded.

Usage

def start_worker
  options = { :quiet => true }
  if read_ahead = ENV['READ_AHEAD'] # DEFAULT_READ_AHEAD = 5
    options[:read_ahead] = read_ahead.to_i
  end
  if sleep_delay = ENV['SLEEP_DELAY'] # DEFAULT_SLEEP_DELAY = 5
    options[:sleep_delay] = sleep_delay.to_f
  end

  # some options are set to work per-thread (as a thread-local).
  worker = Delayed::Threaded::Worker.new(options)
  worker.start
rescue Exception => e
  msg = "FATAL #{e.inspect}"
  if backtrace = e.backtrace
    msg << ":\n  #{backtrace.join("\n  ")}"
  end
  STDERR.puts(msg)
end

# while other options are global and do not make sense to be set per-thread
Delayed::Worker.queues = (ENV['QUEUES'] || ENV['QUEUE'] || '').split(',')
Delayed::Worker.min_priority = ENV['MIN_PRIORITY'] if ENV['MIN_PRIORITY']
Delayed::Worker.max_priority = ENV['MAX_PRIORITY'] if ENV['MAX_PRIORITY']

Thread.new { start_worker }

ActiveRecord

There's an optional integration with the ActiveRecord backend, to clear the connections after work (as the worker sleeps), setup as a plugin using :

require 'delayed/active_record/release_connection_plugin.rb'

and a somehow fail-"safer" lock clearing mechanism in case of backend errors

# Replace DJ's default ClearLocks plugin with a fail-safe version 
# e.g. when a connection goes down in the middle of job processing, 
# `Delayed::Job.clear_locks!` causes a double fault to propagate.
require 'delayed/fail_safe/clear_locks_plugin.rb'

NOTE: delayed_job might not be the right tool for a fail-safe worker job.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install.

Copyright

Copyright (c) 2018 Karol Bucek. See LICENSE (http://en.wikipedia.org/wiki/MIT_License) for details.