Skip to content

Commit

Permalink
Merge pull request #3 from doximity/jcw/bring_in_progress_from_easykiq
Browse files Browse the repository at this point in the history
Bring in progress from easykiq and updates from standardized wiki
  • Loading branch information
jcwilk authored Apr 20, 2022
2 parents 1eedee1 + 132d0b4 commit 532e407
Show file tree
Hide file tree
Showing 46 changed files with 225 additions and 58 deletions.
26 changes: 13 additions & 13 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ orbs:
gem: doximity/gem-publisher@0

executors:
ruby-latest:
ruby-2-6:
resource_class: small
docker:
- image: circleci/ruby:2.6
- image: cimg/ruby:2.6
environment:
BUNDLE_VERSION: "~> 2.1"
BUNDLE_VERSION: "~> 1.17"

# yaml anchor filters
master_only: &master_only
Expand All @@ -33,7 +33,7 @@ version_tags_only: &version_tags_only

jobs:
build:
executor: ruby-latest
executor: ruby-2-6
steps:
- checkout
- run:
Expand All @@ -45,7 +45,10 @@ jobs:
- v1-bundle-{{ checksum "Gemfile.lock" }}-
- run:
name: Install Ruby Dependencies
command: bundle check --path=vendor/bundle || bundle install --local --frozen --path=vendor/bundle --jobs=4 --retry=3
command: |
bundle config set --local path 'vendor/bundle'
bundle config set --local frozen 'true'
bundle install --local --jobs=4 --retry=3
- save_cache:
key: v1-bundle-{{ checksum "Gemfile.lock" }}-
paths:
Expand All @@ -57,11 +60,8 @@ jobs:
name: Store test results
path: tmp/test-results
- run:
name: Build documentation
command: bundle exec rake ci:doc
- store_artifacts:
name: Saves documentation
path: doc
name: Run StandardRB
command: bundle exec standardrb
- persist_to_workspace:
root: .
paths:
Expand All @@ -76,7 +76,7 @@ workflows:
<<: *master_only
- gem/build:
<<: *master_only
executor: ruby-latest
executor: ruby-2-6
name: gem-build
requires:
- build
Expand All @@ -87,7 +87,7 @@ workflows:
<<: *pr_only
- gem/build:
<<: *pr_only
executor: ruby-latest
executor: ruby-2-6
name: gem-build
requires:
- build
Expand All @@ -111,7 +111,7 @@ workflows:
<<: *version_tags_only
- gem/build:
<<: *version_tags_only
executor: ruby-latest
executor: ruby-2-6
name: gem-build
requires:
- build
Expand Down
9 changes: 4 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
.rspec_status
/.ruby-version
/*.gem
/.bundle/
/.yardoc
/_yardoc/
/coverage/
/doc/
/pkg/
/spec/reports/
/tmp/

# rspec failure tracking
.rspec_status
/vendor/bundle/
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.0.3]
* Misc minimal fixes to get the gem building and releasable
[#2](https://github.com/doximity/rake-ui/pull/3)
* Copy over library code from prior sources with maintained history
[#1](https://github.com/doximity/simplekiq/pull/1)

## [0.0.2]
* Scrubbed version from rubygems, do not use
## [0.0.1]
* Scrubbed version from rubygems, do not use
5 changes: 4 additions & 1 deletion CONTRIBUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,7 @@
* Fixed bug with batch statuses in callbacks for empty batches

### Tiffany Troha
* Added support for specifying `sidekiq_options` for the child job in `Simplekiq::BatchingJob`
* Added support for specifying `sidekiq_options` for the child job in `Simplekiq::BatchingJob`

### [Daniel Pepper](https://github.com/dpep)
* On request, graciously took down his unused `simplekiq` placeholder from rubygems so we could continue using the name :raised_hands:
81 changes: 81 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
PATH
remote: .
specs:
simplekiq (0.0.3)
sidekiq (~> 5.2.9)

GEM
remote: https://rubygems.org/
specs:
ast (2.4.2)
coderay (1.1.3)
connection_pool (2.2.5)
diff-lcs (1.5.0)
method_source (1.0.0)
parallel (1.22.1)
parser (3.1.1.0)
ast (~> 2.4.1)
pry (0.13.1)
coderay (~> 1.1)
method_source (~> 1.0)
rack (2.2.3)
rack-protection (2.2.0)
rack
rainbow (3.1.1)
rake (12.3.3)
redis (4.5.1)
regexp_parser (2.2.1)
rexml (3.2.5)
rspec (3.10.0)
rspec-core (~> 3.10.0)
rspec-expectations (~> 3.10.0)
rspec-mocks (~> 3.10.0)
rspec-core (3.10.1)
rspec-support (~> 3.10.0)
rspec-expectations (3.10.1)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.10.0)
rspec-mocks (3.10.2)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.10.0)
rspec-support (3.10.3)
rspec_junit_formatter (0.5.1)
rspec-core (>= 2, < 4, != 2.12.0)
rubocop (1.26.1)
parallel (~> 1.10)
parser (>= 3.1.0.0)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0)
rexml
rubocop-ast (>= 1.16.0, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 1.4.0, < 3.0)
rubocop-ast (1.16.0)
parser (>= 3.1.1.0)
rubocop-performance (1.13.3)
rubocop (>= 1.7.0, < 2.0)
rubocop-ast (>= 0.4.0)
ruby-progressbar (1.11.0)
sidekiq (5.2.10)
connection_pool (~> 2.2, >= 2.2.2)
rack (~> 2.0)
rack-protection (>= 1.5.0)
redis (~> 4.5, < 4.6.0)
standard (1.9.1)
rubocop (= 1.26.1)
rubocop-performance (= 1.13.3)
unicode-display_width (2.1.0)

PLATFORMS
ruby

DEPENDENCIES
pry
rake (~> 12.0)
rspec (~> 3.2)
rspec_junit_formatter
simplekiq!
standard

BUNDLED WITH
2.1.4
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,13 @@ Any time that you find yourself needing to string together a long chain of jobs,
Add this line to your application's Gemfile:

```ruby
gem 'simplekiq'
gem "simplekiq"
```

Note that this gem requires you be a Sidekiq Pro paid subscriber to be able to use it, so after following the installation docs for getting the private gem configured with your system, ensure you have `sidekiq-pro` at version `~> 5.0.0` or higher and that it's being required:

```ruby
gem "sidekiq-pro", "~> 5.0.0"
```

And then execute:
Expand Down
7 changes: 5 additions & 2 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
require "bundler/gem_tasks"
# frozen_string_literal: true

require "rspec/core/rake_task"

FileList["tasks/*.rake"].each { |task| load task }

RSpec::Core::RakeTask.new(:spec)

task :default => :spec
task default: :spec
4 changes: 3 additions & 1 deletion lib/simplekiq.rb
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# frozen_string_literal: true

require "sidekiq"
require "sidekiq-ent"

# NB: You must explicitly require sidekiq-ent in your app!
# require "sidekiq-ent"

require "simplekiq/orchestration_executor"
require "simplekiq/orchestration"
Expand Down
12 changes: 6 additions & 6 deletions lib/simplekiq/batching_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,18 +110,18 @@ def handle_batches(args)
else
# Empty batches with no jobs will never invoke callbacks, so handle
# that case by immediately manually invoking :complete & :success.
on_complete(nil, { "args" => args }) if respond_to?(:on_complete)
on_success(nil, { "args" => args }) if respond_to?(:on_success)
on_complete(nil, {"args" => args}) if respond_to?(:on_complete)
on_success(nil, {"args" => args}) if respond_to?(:on_success)
end
end

def flush_batches(args)
batch_job_class = self.class.const_get(BATCH_CLASS_NAME)
sidekiq_batch.description ||= "Simplekiq Batch Jobs for #{self.class.name}, args: #{args}"

sidekiq_batch.on(:death, self.class, "args" => args) if respond_to?(:on_death)
sidekiq_batch.on(:complete, self.class, "args" => args) if respond_to?(:on_complete)
sidekiq_batch.on(:success, self.class, "args" => args) if respond_to?(:on_success)
sidekiq_batch.on("death", self.class, "args" => args) if respond_to?(:on_death)
sidekiq_batch.on("complete", self.class, "args" => args) if respond_to?(:on_complete)
sidekiq_batch.on("success", self.class, "args" => args) if respond_to?(:on_success)

sidekiq_batch.jobs do
batches.each do |job_args|
Expand All @@ -131,7 +131,7 @@ def flush_batches(args)
end

def queue_batch(*args)
self.batches << args
batches << args
end

def batch_description=(description)
Expand Down
6 changes: 3 additions & 3 deletions lib/simplekiq/orchestration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@ def execute(parent_batch)
end

def serialized_workflow
serialized_workflow = serial_workflow.map do |step|
@serialized_workflow ||= serial_workflow.map do |step|
case step[0]
when Array
step.map do |(job, *args)|
{ "klass" => job.name, "args" => args }
{"klass" => job.name, "args" => args}
end
when Class
job, *args = step
{ "klass" => job.name, "args" => args }
{"klass" => job.name, "args" => args}
end
end
end
Expand Down
2 changes: 2 additions & 0 deletions lib/simplekiq/orchestration_executor.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ def self.execute(workflow:, parent_batch:)
end

def run_step(parent_batch, workflow, step)
return if workflow.empty?

nest_under(parent_batch) do
*jobs = workflow.at(step)
sidekiq_batch = Sidekiq::Batch.new
Expand Down
2 changes: 1 addition & 1 deletion lib/simplekiq/version.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module Simplekiq
VERSION = "0.0.1"
VERSION = "0.0.3"
end
32 changes: 19 additions & 13 deletions simplekiq.gemspec
Original file line number Diff line number Diff line change
@@ -1,32 +1,38 @@
require_relative 'lib/simplekiq/version'
require_relative "lib/simplekiq/version"

Gem::Specification.new do |spec|
spec.name = "simplekiq"
spec.version = Simplekiq::VERSION
spec.authors = ["Jack Noble", "John Wilkinson"]
spec.email = ["jcwilkinson@doximity.com"]
spec.summary = %q{Sidekiq-based workflow orchestration library}
spec.description = %q{Provides tools for representing long chains of parallel and serial jobs in a flat, simple way.}
spec.homepage = "https://github.com/doximity/simplekiq"
spec.license = "MIT"
spec.required_ruby_version = Gem::Requirement.new(">= 2.3.0")
spec.name = "simplekiq"
spec.version = Simplekiq::VERSION
spec.authors = ["Jack Noble", "John Wilkinson"]
spec.email = ["jcwilkinson@doximity.com"]
spec.summary = "Sidekiq-based workflow orchestration library"
spec.description = "Provides tools for representing long chains of parallel and serial jobs in a flat, simple way."
spec.homepage = "https://github.com/doximity/simplekiq"
spec.license = "APACHE-2.0"
spec.required_ruby_version = Gem::Requirement.new(">= 2.5.0")

spec.metadata["homepage_uri"] = spec.homepage
spec.metadata["source_code_uri"] = spec.homepage
# TODO: spec.metadata["changelog_uri"] = "TODO: Put your gem's CHANGELOG.md URL here."

# Specify which files should be added to the gem when it is released.
# The `git ls-files -z` loads the files in the RubyGem that have been added into git.
spec.files = Dir.chdir(File.expand_path('..', __FILE__)) do
spec.files = Dir.chdir(File.expand_path("..", __FILE__)) do
`git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
end
spec.executables = []
spec.executables = []
spec.require_paths = ["lib"]

spec.add_development_dependency "rspec", "~> 3.2"
spec.add_development_dependency "rake", "~> 12.0"
spec.add_development_dependency "rspec_junit_formatter"
spec.add_development_dependency "pry"
spec.add_development_dependency "standard"

spec.add_dependency "sidekiq", "~> 5.2.9"
spec.add_dependency "sidekiq-pro", "~> 5.0.0"

# Can't define this explicitly because it would be inappropriate to vendor this
# pay-to-use library into the gem and it is not available on rubygems and
# this otherwise interferes with our publishing process.
# spec.add_dependency "sidekiq-pro", "~> 5.0.0"
end
Loading

0 comments on commit 532e407

Please sign in to comment.