I saw a question yesterday about async jobs in Rails and thought I’d expand my thoughts here.

Quick History

First, some background. Our team has led maintenance of Delayed Job, the biggest background job gem in the Rails world since 2008ish. We didn’t write it, that honor goes to Tobi at Shopify, back when they were a much smaller organization. For reasons that probably had to do with Tobi’s job taking up more time, we ended up being the group publishing gems and maintaining it.

Delayed Job is still great. It has been battle-tested and remains solid. It can handle any ActiveRecord database and also MongoDB via Mongoid. It can even work outside of a Rails app.

There’s nothing wrong with it, and I don’t think anyone needs to move away from it. It works. It’ll continue to work.

However, we haven’t added it to new projects in years. Why? Better options came up that fit us better. I’ll explain.

Sidekiq

Sidekiq came out, backed by Redis, and a dedicated maintainer in Mike Perham. Mike’s done a fantastic job. We’ve been able to use the Pro and Enterprise versions over the years and they are very mature and well-built. It can handle anything you throw at it.

Sidekiq is still a favorite. Especially for heavy users, who will have millions of jobs and need reliability and/or support. I don’t think any of the teams we worked with ever needed support, which is testament to the product.

We also build a lot of small apps and would often deploy to Heroku. Heroku made it easy to use Sidekiq by having multiple add-ons with free Redis plans.

Redis has been solid for us, but it is one more moving piece that needs to be managed, and often comes with added cost. That makes sense for some teams, but not all. Sometimes we’ve used Redis for other things too (caching and rack-attack come to mind) but not every app needs that.

Free plans started to go away, so Sidekiq wasn’t as easy of a sell for these small projects. We started looking elsewhere.

Active Job

Somewhere along the line, Active Job was added to Rails. Active Job isn’t a job runner, but a standard API to different job queues. It is a good idea, and Delayed Job and Sidekiq both added support. We use Active Job’s API for everything these days.

Back to the Database

A big benefit of Delayed Job was that it used your app’s database. That both made it easier to run locally, and easier to deploy.

When we first worked on Delayed Job, multiple database support was important. We had clients already using MySQL, and MongoDB was in our arsenal for a while. PostgreSQL was always our favorite (before it was cool) and these days it is rare we need to touch another DB (at least for an app’s primary).

I long thought about what a re-imagining of Delayed Job that targeted PostgreSQL would look like. It would ditch all the other database support, only support Active Job, and do some other magic. Fortunately for all of us, someone else had an even better version of this idea and actually built it.

GoodJob

GoodJob is that better re-imagining. It only cares about PostgreSQL and takes advantage of many of its unique features. It is also Active Job only, so a simple API.

GoodJob also builds in many things you’d need Sidekiq Pro or Enterprise (or a Delayed Job plugin) to use: web UI, cron¹ jobs, batches, unique jobs, etc.

We’ve moved many projects to GoodJob over the past couple years, and it is our go-to for new projects. It is actively developed and continually improving.

What about Solid Queue?

Solid Queue came out at the end of 2023 to be a new default choice for Rails apps. It uses your database, and focuses on Active Job. It can use any database Rails can support, but doesn’t get some of the fancy PostgreSQL features we get in GoodJob.

We haven’t spent real time looking at Solid Queue yet. From the outside, it looks to be a good default for most people. A modern version of Delayed Job, without some of the pre-Active Job stuff that 99% of people won’t need.

Other job queues

Resque has a long history and is still going strong. I haven’t used it in a while, so can’t really comment intelligently.

Sucker Punch got some use in our office, to avoid using a separate job worker. It works well, but can lose jobs, which is a non-starter for many of our projects.

There are many more, some listed in the Active Job docs. I don’t have enough experience with any to comment.

Moving between Queue systems

I should note that Active Job makes it easy to switch queue systems. Having moved a Sidekiq project to GoodJob recently, you do have to pay attention to a few things, like concurrency and error handling, but it is a fairly quick process. So try something new, stick to the Active Job API, and you can always switch later.

Summary

• Delayed Job is still great and will keep working. Don’t change if you don’t need to.
• Sidekiq is great, especially Pro and Enterprise. No notes.
• GoodJob is our new primary pick. PostgreSQL-only which is a feature, not a bug, for us. Works great, and has great features.
• Solid Queue looks like it’ll be a good default for most people.
• There are others that might be great. I only know so much.
• Active Job makes it pretty easy to move between.

¹ One of these days I’ll write up how we monitor GoodJob cron using Dead Man’s Snitch.