Skip to content

AWS EB: Set up Sidekiq for Rails app on AWS Elastic Beanstalk

Jump to bottom
E. Lynette Rayle edited this page Mar 6, 2022 · 1 revision

Use this process to set up Sidekiq messaging queue for a Rails app on an AWS Elastic Beanstalk deployment.

References:

Assumptions

  • Your app has already been setup to run on AWS Elastic Beanstalk.
  • Examples show setting up Sidekiq 5.2.5
  • Procedure was tested with Ruby 2.5.5 and Rails 5.1.7

Prerequisites

  • Redis is required for Sidekiq.
    • Add Elastic Beanstalk ENV variable REDIS_HOST and set it to the host where Redis is installed.  Sysops may have set this already in the process of setting up the Redis host.

In your Rails application

Add sidekiq gem

Edit Gemfile and add...

gem 'sidekiq', '~> 5.2'

Update dependencies

bundle install

Set application to use sidekiq for job queue

Edit config/environments/development.rb, integration.rb, staging.rb, production.rb and add (or edit) line...

config.active_job.queue_adapter = :sidekiq

Edit config/environments/test.rb and set it to run jobs at the time they are called.  Jobs will not run asynchronously.

config.active_job.queue_adapter = :inline

Add initializers and configurations

Add config/initializers/sidekiq.rb

config = YAML.load(ERB.new(IO.read(Rails.root + 'config' + 'redis.yml')).result)[Rails.env].with_indifferent_access

redis_conn = { url: "redis://#{config[:host]}:#{config[:port]}/" }

Sidekiq.configure_server do |s|
  s.redis = redis_conn
end

Sidekiq.configure_client do |s|
  s.redis = redis_conn
end

Add config/sidekiq.yml.  This shows adding the default queue.  You may want to have more queues depending on your application needs.

---
:queues:
  - default

Add config/redis.yml

default:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 0
development:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 1
test:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 2
integration:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 3
staging:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 4
production:
  host: <%= ENV['REDIS_HOST'] || "localhost" %>
  port: 6379
  db: 5

Add route for sidekiq

Edit config/routes.rb and add the following near the top where you mount other engines...

  require 'sidekiq/web'
  mount Sidekiq::Web => '/sidekiq'

Add script to start/restart Sidekiq

Create .ebextensions/sidekiq.config.  This script will create two files on the AWS machine:  1. /opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq.sh starts up sidekiq on the AWS machine after each deploy,  2. /opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq.sh shuts down sidekiq on the AWS machine just before a deploy

# Sidekiq interaction and startup script
commands:
  create_post_dir:
    command: "mkdir -p /opt/elasticbeanstalk/hooks/appdeploy/post"
    ignoreErrors: true
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars

      EB_APP_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
      EB_APP_PID_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_pid_dir)
      EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
      EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
      EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)

      . $EB_SUPPORT_DIR/envvars
      . $EB_SCRIPT_DIR/use-app-ruby.sh

      SIDEKIQ_PID=$EB_APP_PID_DIR/sidekiq.pid
      SIDEKIQ_CONFIG=$EB_APP_DEPLOY_DIR/config/sidekiq.yml
      SIDEKIQ_LOG=$EB_APP_DEPLOY_DIR/log/sidekiq.log

      cd $EB_APP_DEPLOY_DIR

      if [ -f $SIDEKIQ_PID ]
      then
        su -s /bin/bash -c "kill -TERM `cat $SIDEKIQ_PID`" $EB_APP_USER
        su -s /bin/bash -c "rm -rf $SIDEKIQ_PID" $EB_APP_USER
      fi

      . /opt/elasticbeanstalk/support/envvars.d/sysenv

      sleep 10

      su -s /bin/bash -c "bundle exec sidekiq \
        -e $RACK_ENV \
        -P $SIDEKIQ_PID \
        -C $SIDEKIQ_CONFIG \
        -L $SIDEKIQ_LOG \
        -d" $EB_APP_USER

  "/opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars

      EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config container -k app_user)
      EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k script_dir)
      EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k support_dir)

      . $EB_SUPPORT_DIR/envvars
      . $EB_SCRIPT_DIR/use-app-ruby.sh

      SIDEKIQ_PID=$EB_APP_PID_DIR/sidekiq.pid
      if [ -f $SIDEKIQ_PID ]
      then
        su -s /bin/bash -c "kill -USR1 `cat $SIDEKIQ_PID`" $EB_APP_USER
      fi

Add script to give Sidekiq permission to files and directories it needs to access

Create .ebextensions/sidekiq_permissions.config.  This script will create one file on the AWS machine: 1. /opt/elasticbeanstalk/hooks/appdeploy/post/51_chmod_logs.sh sets permissions on log files to allow sidekiq to access them

# Set permissions for files accessed by Sidekiq
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/51_chmod_logs.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars

      EB_APP_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_dir)
      APP_LOG_DIR=$EB_APP_DEPLOY_DIR/log

      SIDEKIQ_LOG=$APP_LOG_DIR/sidekiq.log

      case $RACK_ENV in
        "integration")
          RAILS_LOG=$APP_LOG_DIR/integration.log
          ;;
        "staging")
          RAILS_LOG=$APP_LOG_DIR/staging.log
          ;;
        "production")
          RAILS_LOG=$APP_LOG_DIR/production.log
          ;;
      esac

      touch $SIDEKIQ_LOG
      chmod 0664 $SIDEKIQ_LOG
      touch $RAILS_LOG
      chmod 0664 $RAILS_LOG

 

Add instructions in README for starting sidekiq

Edit README.md and add...

Start Sidekiq in development for processing background jobs (Needs to run at the root of the rails app.)

    $ bundle exec sidekiq -d -L log/sidekiq.log -e development

Debugging Problems

Common Debugging Steps

Check if sidekiq is running...

ssh into the AWS machine and execute...

ps aux | grep sidekiq

if sidekiq is running, you will see something like...

webapp   17372  0.8 14.8 1908540 1210232 ?     Sl   Mar06   7:00 sidekiq 5.2.8 current [0 of 5 busy]

From this, we see that the PID for sidekiq is 17372

Restart sidekiq

This section is not well tested.  If you determine anything to be inaccurate or have a better way to handle this, either leave a comment below, or edit the page and make a comment as to what you think is correct, or send the info to elr37 and I will update the page.

To stop (on AWS)...

  • stop sidekiq process
    • PREREQ get PID for sidekiq using steps above for Check if sidekiq is running...
    • SUBSTITUTE actual PID for PID (e.g. 17372)
$ sudo kill _PID_
  • delete file holding the sidekiq pid
    • SUBSTITUTE the actual location of sidekiq.pid if it is different than what is shown here.
$ find / -name sidekiq.pid 2>/dev/null
/var/app/containerfiles/pids/sidekiq.pid
$ sudo rm /var/app/containerfiles/pids/sidekiq.pid

To start (on AWS)...

  • ASSUMES sidekiq is not running already
  • SUBSTITUTE the actual environment, i.e. staging or production, for integration, shown here)
bundle exec sidekiq -d -L log/sidekiq.log -e integration -P /var/app/containerfiles/pids/sidekiq.pid -C config/sidekiq.yml

To stop (on DEV machine)...

  • PREREQ get PID for sidekiq using steps above for Check if sidekiq is running...
  • SUBSTITUTE actual PID for PID (e.g. 17372)
$ sudo kill _PID_

To start (on DEV machine)...

bundle exec sidekiq -d -L log/sidekiq.log -e development

Jobs enqueue but don't process

In my case, sidekiq was not actually running.  The symptoms were that I could navigate to the sidekiq UI in my app (e.g. app_URL/sidekiq).  I could see jobs added to the queue, but they never ran.

When I checked in the terminal whether sidekiq was running, it was not.  Once I started sidekiq, the queued jobs began to process.

See info about checking whether sidekiq is running and how to start it under [#common-debugging-steps] above.

 

_app_URL_/sidekiq returns 502 Bad Gateway

Contact sysops.  As a result, some aspect of the machine was made bigger.  Not sure exactly what they did.

 

 

Clone this wiki locally