Sidekiq

What Is Sidekiq?

Web requests should ideally return quickly. Tasks like sending emails, processing images, generating reports, or calling third-party APIs can slow down a response if performed synchronously. Sidekiq solves this by allowing such tasks to be pushed into a background queue and processed separately, outside the normal request cycle.

Sidekiq uses Redis as its backing store to hold job queues, and runs jobs using a pool of threads, allowing it to process many jobs concurrently with a relatively small memory footprint compared to process-based background job processors.

Why Is Sidekiq Useful?

Without background job processing, slow or non-essential tasks block the main request, leading to:

• Slow page load times or API response times
• Poor user experience during operations like file uploads, emails, or notifications
• Increased risk of request timeouts
• Difficulty scaling tasks independently from web traffic

Sidekiq helps by:

• Moving slow or non-critical tasks out of the request cycle
• Allowing jobs to be retried automatically on failure
• Supporting scheduled and recurring jobs
• Scaling background processing independently of web servers
• Providing a built-in Web UI for monitoring queues, retries, and failures

It's especially useful for tasks like sending emails, processing uploads, calling external APIs, generating PDFs/reports, and syncing data.

How Does Sidekiq Work?

Sidekiq jobs are defined as plain Ruby classes that include the Sidekiq::Job module (previously Sidekiq::Worker in older versions). When a job is enqueued, its arguments are serialized and pushed onto a Redis queue. Separate Sidekiq worker processes continuously pull jobs off the queue and execute them using a thread pool.

Key components:

• Client – Enqueues jobs (usually your Rails app).
• Redis – Stores queued jobs.
• Sidekiq process – Pulls jobs from Redis and executes them using multiple threads.
• Web UI – A dashboard (sidekiq/web) for monitoring queues, scheduled jobs, retries, and failures.

Examples

Scenario 1: Defining a Job

class HardWorker 
  include Sidekiq::Job 
 
  def perform(user_id) 
    user = User.find(user_id) 
    UserMailer.welcome_email(user).deliver_now 
  end 
end 

Scenario 2: Enqueuing a Job

Instead of running this synchronously in a controller:

UserMailer.welcome_email(@user).deliver_now 

You enqueue it as a background job:

HardWorker.perform_async(@user.id) 

The controller returns immediately, and the email is sent asynchronously by a Sidekiq worker process.

Scenario 3: Scheduling a Job for Later

HardWorker.perform_in(1.hour, @user.id) 

This schedules the job to run after a delay, instead of immediately.

Scenario 4: Automatic Retries on Failure

class ReportGenerator 
  include Sidekiq::Job 
  sidekiq_options retry: 5 
 
  def perform(report_id) 
    report = Report.find(report_id) 
    report.generate! 
  end 
end 

If the job raises an error, Sidekiq automatically retries it with exponential backoff, up to the configured retry limit.

Where to Use Sidekiq?

• Sending emails and notifications
• Processing image or video uploads
• Calling third-party APIs (payments, SMS, webhooks)
• Generating reports, PDFs, or exports
• Data syncing and ETL-style background tasks
• Any operation that doesn't need to block the user's response

A Note of Caution

Background jobs should be idempotent where possible, since Sidekiq's retry mechanism may execute a job more than once (e.g., if a job fails partway through). Job arguments should also be kept simple (IDs rather than full objects), since arguments are serialized to JSON and stored in Redis, passing ActiveRecord objects directly can lead to stale or bloated data.

In Summary

Sidekiq is a background job processing library for Rails that uses Redis and multithreading to run asynchronous tasks efficiently. By moving slow or non-critical work out of the request cycle, it improves application responsiveness, enables retries and scheduling, and helps applications scale background processing independently of the web layer.