Collective Idea

Collective Idea Logo

Zach Moazeni

Full Text Searching with Solr and Sunspot

By Zach Moazeni on March 08, 2011 in rails, solr, and sunspot

This post is part of our Exploring Solr and Sunspot series.

Full text searching can be a tricky subject. Luckily there are a number of great tools out there that are much better than doing "... content like '%cars%' ...". One excellent tool that we have used on numerous projects is Solr along with the Ruby library Sunspot.

What is Solr?

Solr is not technically a full text search engine itself, but instead is an HTTP layer wrapped around the Lucene engine. But for our purposes we can think of Solr as a search engine with a web API. The first thing you’ll notice with Solr is that it’s built in java, but don’t let that turn you off. One of my favorite things about Solr is that you can download it and start running out of the box without having to install or compile anything.

If you are courageous you can access the Solr API directly, fortunately there is a Ruby library Sunspot that will do all the heavy lifting for us.

What is Sunspot and how does it work?

When working with Rails, all you need to worry about is sunspot_rails. You won’t need to download or install anything with solr because the gem includes the solr libraries as well as rake tasks to start and stop the server.

What sunspot_rails does is hook into the ActiveRecord lifecycle callbacks and send updates to the solr server behind the scenes. Then you can perform a search which will first query the solr server then load the ActiveRecord instances for you, making the search feel transparent.

Having fun with Sunspot

Enough explaining, lets get our hands dirty. First start by adding the following gems to your bundler Gemfile:

# add this to your bundler Gemfile
gem 'sunspot_rails'

# install it by running
> bundle install

Solr runs as a separate process so you will need to start up the Solr server before you begin developing.

# to start the server
> rake sunspot:solr:start

# to stop the server
> rake sunspot:solr:stop

When you start the server you may notice files being copied into RAILS_ROOT/solr, that’s because Solr/Lucene stores the search indexes in the file system. That way if you restart the server, you’ll retain the search indexes.

Let’s create a scaffold for a Book:

> rails generate scaffold book title:string isbn:string
> rake db:migrate

We’re going to make a few small tweaks to the Book model to make it searchable and to offer a search feature. First add the following the app/models/book.rb

class Book < ActiveRecord::Base
  validates_presence_of :title, :isbn

  searchable do
    text :title
  end
end

What this does is define the fields sunspot will watch and communicate to solr. Lets also add a search action to the codebase. Add the following to config/routes.rb

resources :books do
  collection do
    get :search
  end
end

Add the following to app/controllers/books_controller.rb

# GET /books/search
# GET /books/search.xml
def search
  @books = Book.search do
    keywords params[:query]
  end.results

  respond_to do |format|
    format.html { render :action => "index" }
    format.xml  { render :xml => @books }
  end
end

And add the following to app/views/books/index.html.erb

<%= form_tag search_books_path, :method => :get do %>
<p>
  <%= text_field_tag :query, params[:query] %> <%= submit_tag "Search!" %>
</p>
<% end %>

Now add a few books and perform a search! For instance:

Searching for 'Great' and viewing results

Cool huh?

Updating indexed fields and re-indexing

Another great feature of Sunspot/Solr is changing the fields you’re indexing. In our case we have both a title and an isbn, so let’s search off of any of them. Update app/models/book.rb

class Book < ActiveRecord::Base
  validates_presence_of :title, :isbn

  searchable do
    text :title, :isbn
  end
end

And from this point on, Sunspot will include the isbn with the search indexes. But that doesn’t help with our existing data. Not to fear, sunspot has a handy rake task that will let you re-index your existing data again:

> rake sunspot:reindex # this will go through all your searchable models and re-index them with solr

Wrapping up

That was pretty slick and pretty easy eh?

You may have noticed that by default Sunspot/Solr doesn’t search within words. For instance I can search for “Great” but not “Gre”. That’s due to the quick default indexer. In a later blog post I’ll describe how to tweak the Solr config to allow you to search within words.

For those who want to play with a working codebase, I’ve uploaded a working demo onto github. There are a whole slew of fun things you can do with sunspot. I’d recommend checking out the site and the wiki for advanced usage.

By Zach Moazeni on March 08, 2011 in rails, solr, and sunspot

19 Comments

  1. Tatum

    Tatum March 08, 2011

    Thanks for sharing!

  2. alain

    alain March 11, 2011 http://www.canon5dtips.com

    No need for SunSpot-rails anymore. Just sunspot !

  3. Zach Moazeni

    Zach Moazeni March 11, 2011 http://collectiveidea.com

    @alain What do you mean? The gems are separated with the rails specific hooks in sunspot_rails.

    https://github.com/outoftime/sunspot/wiki/adding-sunspot-search-to-rails-in-5-minutes-or-less

    You can certainly use sunspot without sunspot_rails for non-Rails projects.

  4. Daniel Morrison

    Daniel Morrison March 19, 2011 http://collectiveidea.com

    alain & Zach: it does look like sunspot_rails has been merged into the main sunspot repo, but the sunspot_rails gem still exists.

  5. Jim

    Jim April 02, 2011

    Thanks, this is exactly what I was looking for.

  6. diebels727

    diebels727 May 05, 2011

    @zach, do you have any thoughts on searching “through” a model a-la relationships?  For example, searching through ‘Comments’ on a Post?

    As best I can tell, in the above case, some magic wizardry must be done on the Post to enable this because of the fact that Solr is document oriented.

  7. Zach Moazeni

    Zach Moazeni May 05, 2011 http://collectiveidea.com

    @diebels727 that’s a great question. I’ll put together a blog post on that instead of shoving it here in the comments. But check out the lambda syntax inside of the searchable block.

    text :body {|instance| … }

  8. Francisco Guzmán

    Francisco Guzmán January 23, 2012 http://twitter.com/panchew

    Does anyone know how to teach solr to return results from ‘#term’?
    I need to have different set results for ‘term’ and ‘#term’

  9. Peter

    Peter February 17, 2012

    I dont think this works ! I have tried it and i keep getting this error ActiveRecord::RecordNotFound in BooksController#show

  10. Austin

    Austin March 17, 2012

    So I followed pretty much exactly (just changing variable names and such) and get the following error: undefined method `searchable’ for #

    Then, I refresh the page again and the following error occurs:
    undefined method `key?’ for nil:NilClass

    In my terminal server I get:
    Started GET “/” for 127.0.0.1 at 2012-03-17 17:48:58 -0400

    NoMethodError (undefined method `key?’ for nil:NilClass):
      actionpack (3.2.1) lib/action_controller/metal/hide_actions.rb:36:in `visible_action?’….

    Any suggestions?

  11. Zach Moazeni

    Zach Moazeni March 19, 2012 http://collectiveidea.com

    @Peter and @Austin – check out the codebase I pushed up to https://github.com/collectiveidea/playing-with-sunspot – I just tested it out and it still works. I also updated the readme with more detailed instructions on how to get it running.

  12. idarfan

    idarfan April 09, 2012

    Dear Zach Moazeni:

    I had follow all about your examples. well done.
    it works like charm, but I need this program run with nginx .

    then I get the error message:

    Errno::ECONNREFUSED in KojenadultsController#search

    Connection refused – connect(2)

    this program is on my github, could you help me how to make it run with nginx? thx

    git://github.com/idarfan/kojenadults.git

  13. Paola

    Paola May 20, 2012

    I can’t charge this application on my browser. I have a problem, this exception is thrown for me: NoMethodError in BooksController#index. 
    I have the exactly code that you post.. Except that I use Json intead of xml for the render.. Please Help me

  14. Zach Moazeni

    Zach Moazeni May 21, 2012 http://collectiveidea.com

    @Paola: you should check out the example codebase on https://github.com/collectiveidea/playing-with-sunspot that may help tell the difference.

    @idarfan: I’m not sure about solr running behind nginx. Doesn’t seem necessary.

  15. сосна мебель

    сосна мебель June 11, 2012 http://sosnacity.ru

    Je suis toujours en ligne pour enquêter conseils qui peuvent m’être utile. Merci collectiveidea.com

  16. Mel

    Mel August 21, 2012

    Do you know how to update the index instead of just reindexing every time?

  17. limonka

    limonka August 24, 2012

    @Mel: You have to pass :auto_index for searchable in your model and set it to true.

  18. Mazen

    Mazen September 09, 2012

    Heyyy I’m building a website by RoR ..and i use Sunspot and solr Search Engine …i needed to put autocomplete functionality but i coudn’t .. tried to use the gem sunspot_autocomplete …but i got many errors …is this gem updated and i can use it normally or it’s out dated ? thanks a lot man 

  19. Tiago

    Tiago February 26, 2014

    If I search pro “Grea” the result is blank, how to solve this??

Post a Comment

Contact Us

Find us on Google Maps
Collective Idea
44 East 8th Street, Suite 410
Holland, Michigan 49423 USA 42.790334-86.105251

Follow us on the Interwebs

We are currently available for medium and long term projects. Please get in touch if we can be of service.