Setup Sidekiq on Ubuntu with upstart

The Github page for SideKiq is here. We will use to save the scheduled work and Upstart to make sure the uptime of our SideKiq service. How to program a worker in ruby is not covered here. Check the Github page for example.

Add this line to your Gemfile and install the gem.

gem sidekiq

Config SideKiq in sidekiq.yml, define the each queue and its priority.

---
:concurrency: 5
:pidfile: tmp/pids/sidekiq.pid
staging:
  :concurrency: 10
production:
  :concurrency: 20
:queues:
  - default
  - [queue_one, 3]
  - [queue_two, 1]
  - [queue_three, 1]

Add an initializer sidekiq.rb and define the url of your redis instance.

Sidekiq.configure_server do |config|
  config.redis = { url: 'redis://redis.example.com:7372/12' }
end

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://redis.example.com:7372/12' }
end

Create sidekiq.conf in /etc/init directory with the following content. Take note of the username and app_dir.

setuid ubuntu
setgid ubuntu
env HOME=/home/ubuntu

respawn
respawn limit 3 30

# TERM is sent by sidekiqctl when stopping sidekiq. Without declaring these as
# normal exit codes, it just respawns.
normal exit 0 TERM

# Older versions of Upstart might not support the reload command and need
# this commented out.
reload signal USR1

instance $index

script
# this script runs in /bin/sh by default
# respawn as bash so we can source in rbenv
exec /bin/bash <<'EOT'
  # Logs out to /var/log/upstart/sidekiq.log by default
  export HOME=/home/ubuntu
  echo "$HOME"
  export PATH="$HOME/.rbenv/bin:$PATH"
  eval "$(rbenv init -)"
  export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"
  echo "$PATH"
  cd ~/app_dir
  exec bundle exec sidekiq -i ${index} -e production
EOT
end script

Create sidekiq-workers.conf in /etc/init directory with the following content. Take note of the NUM_WORKERS.

start on runlevel [2345]
stop on runlevel [06]

# Set this to the number of Sidekiq processes you want
# to run on this machine
env NUM_WORKERS=1

pre-start script
  for i in `seq 1 ${NUM_WORKERS}`
  do
    start sidekiq index=$i
  done
end script

post-stop script
  for i in `seq 1 ${NUM_WORKERS}`
  do
    stop sidekiq index=$i
  done
end script

Create /tmp/pids directory if it does not exists.

Start the service.

sudo service sidekiq-worker start

To test your setup, kill your SideKiq process and you should see a new process is spawned.

sudo kill -9 pid
ps aux | grep sidekiq

If anything goes wrong, the log can be found at:

cd /var/log/upstart/sidekiq.log