The Neuronen Schmiede concerns itself with topics related to software development. One focus are robust web applications.

Use Sidekiq With Hanami

Permalink

Hanami, formerly known as Lotus, is a Ruby web framework like Ruby on Rails. I'm really fond of it due to it's small footprint and architecture which guides you to well designed applications. Another piece of software I have high opinions of is Sidekiq, a simple and efficient job processing solution for Ruby. Now let me show you how to use them together.

Setting up Sidekiq within Hanami

The first step is adding Sidekiq to your Gemfile and running bundle install. I also tend to generate binstubs for the gems I need to use outside the application code. To do this for Sidekiq run bundle binstubs sidekiq.

Gemfile

gem 'sidekiq', '~> 4.0.2'

Once Sidekiq is installed it's time to configure it. Mainly I want to specify which Redis URL it uses. For example when you are running both Production and Staging versions of your application on the same server and both access the same Redis instance you want them to not interfere with each other.

Instead of using the namespace option I will use different Redis databases. (A default Redis instance comes with 16 of these.) The database is specified in the Redis URL which in turn is an environment variable. You can read more about this topic in an article by Mike Perham, the creator of Sidekiq.

To configure Sidekiq we create config/sidekiq.rb and setup both server and client. In addition we require our newly created file in config/environment.rb so it is actually available within Hanami.

config/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch("REDIS_URL") }
end

Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch("REDIS_URL") }
end

config/environment.rb

# ...
require_relative '../lib/your-application-name'
require_relative '../apps/web/application'
require_relative './sidekiq'
# ...

Lastly add the REDIS_URL environment variable to your dotenv environment files and you are good to go. Remember to use different Redis databases for your environments. In this example I use the database 0 as you can see at the end of the URL.

.env.

REDIS_URL=redis://localhost:6379/0

Adding Sidekiq workers to a Hanami application

If you are using Hanami you are probably already aware of the two important directories lib/ and apps/. The first one houses your application core and the second contains your various delivery mechanisms like your API or web interface. Sidekiq workers are like entities at the core of your application. They are independent of the delivery mechanism and can be used in various places.

Therefore I suggest we store our workers within the lib/<your-application-name/workers directory. Let's add an example worker that does nothing but sleep.

lib/<your-application-name/workers/sleep_worker.rb

class SleepWorker
  include Sidekiq::Worker

  def perform(workload)
    sleep workload
  end
end

Within our application we can now schedule background jobs as shown in the Sidekiq manual. Just call SleepWorker.perform_async(10) and you are done. At this point we are able to schedule jobs, but that's it. Sidekiq stores them happily in Redis but never looks at them again. To change this we have to run Sidekiq along our web server.

Running Sidekiq and Hanami

In development I use Foreman to manage all the different pieces of an application. In our case that would be a web server and Sidekiq. The Procfile contains an entry for each piece so everything can be started and stopped with a single command. You can see we are using the -r flag to tell Sidekiq to require our applications Hanami environment file. This makes sure Sidekiq is correctly configured and the application core is loaded.

You can start everything with foreman start and watch how your workers get busy.

Procfile

app: bin/hanami server --host 127.0.0.1
sidekiq: bin/sidekiq -e development -r ./config/environment.rb

Mounting the Sidekiq web dashboard in Hanami

Sidekiq comes with Sidekiq::Web a nice web interface that allows a certain amount of introspection. In order to run it two things are necessary. First add Sinatra to your Gemfile and do not require it, Sidekiq itself will only require the parts it needs.

Gemfile

gem 'sinatra', '~> 1.4.6', require: false

In the second step you mount Sidekiq::Web to make it available. At the moment config.ru seems to be the only place where it works1.

config.ru

require './config/environment'
require 'sidekiq/web'

map '/admin/sidekiq' do
  use Sidekiq::Web
end

run Hanami::Container.new

Once you updated your config.ru you can view the Sidekiq web dashboard at localhost:2300/admin/sidekiq/. Be aware that the Sidekiq web dashboard is not protected with any kind of authentication. Anybody can visit the URL and mess with your jobs!

So please make sure to use somekind of authentication like Rack::Auth::Basic in production.

  1. Using Hanamis mount method results in a mounted Sidekiq web interface where all URL paths miss a critical segment. Which leads to wrong links and missing assets like stylesheets and scripts. This happens because the SCRIPT_NAME environment variable is not set and Sidekiq::Web uses it to construct paths. I'm still investigating the best way to solve this.