Ruby on Rails

Getting started with Ruby on Rails and Bonsai Elasticsearch is fast and easy. In this guide, we will cover the steps and the bare minimum amount of code needed to support basic search with Elasticsearch. Users looking for more details and advanced usage should consult the resources at the end of this page. 

Throughout this guide, you will see some code examples. These code examples are drawn from a very simple Ruby on Rails application, and are designed to offer some real-world, working code that new users will find useful. The complete demo app can be found in this GitHub repo.

Warning

The official Elasticsearch Ruby client is not supported on Bonsai after version 7.13. This is due to a change introduced in the 7.14 release of the gem. This change prevents the Ruby client from communicating with open-sourced versions of Elasticsearch 7.x, as well as any version of OpenSearch. The table below indicates compatibility:

If you are receiving a Elasticsearch::UnsupportedProductError, then you'll need to ensure you're using a supported version of the Elasticsearch Ruby client.

Note

In this example, we are going to connect to Elasticsearch using the official Elasticsearch gems. There are other gems for this, namely SearchKick, which is covered in another set of documentation.

Step 1: Spin up a Bonsai Cluster

Make sure that there is a Bonsai Elasticsearch cluster ready for your app to interact with. This needs to be set up first so you know which version of the gems you need to install; Bonsai supports a large number of Elasticsearch versions, and the gems need to correspond to the version of Elasticsearch you’re running.

Bonsai clusters can be created in a few different ways, and the documentation for each path varies. If you need help creating your cluster, check out the link that pertains to your situation:

• If you’ve signed up with us at bonsai.io, you will want to follow the directions here.
• Heroku users should follow these directions.

The Cluster URL

When you have successfully created your cluster, it will be given a semi-random URL called the Elasticsearch Access URL. You can find this in the Cluster Dashboard, in the Credentials tab:

Heroku users will also have a BONSAI_URL environment variable created when Bonsai is added to the application. This variable will contain the fully-qualified URL to the cluster.

Step 2: Confirm the Version of Elasticsearch Your Cluster is On

When you have a Bonsai Elasticsearch cluster, there are a few ways to check the version that it is running. These are outlined below:

Option 1: Via the Cluster Dashboard Details

The easiest is to simply get it from the Cluster Dashboard. When you view your cluster overview in Bonsai, you will see some details which include the version of Elasticsearch the cluster is running:

Option 2: Interactive Console

You can also use the Interactive Console. In the Cluster Dashboard, click on the Console tab. It will load a default view, which includes the version of Elasticsearch. The version of Elasticsearch is called “number” in the JSON response:

Option 3: Using a Browser or curl

You can copy/paste your cluster URL into a browser or into a tool like curl. Either way, you will get a response like so:

curl https://abcd123:efg456@my-cluster-123456.us-west-2.bonsaisearch.net:443
{
  "name" : "ip-172-31-14-16",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "jVJrINr5R5GVVXHGcRhMdA",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

The version of Elasticsearch is called “number” in the JSON response.

Step 3: Add the Gems

There are a few gems you will need in order to make all of this work. Add the following to your Gemfile outside of any blocks:

gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: 'master'
gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: 'master'
gem 'bonsai-elasticsearch-rails', github: 'omc/bonsai-elasticsearch-rails', branch: 'master'

This will install the gems for the latest major version of Elasticsearch. If you have an older version of Elasticsearch, then you should follow this table:

For example, if your version of Elasticsearch is 6.x, then you would use something like this:

gem 'elasticsearch-model', github: 'elastic/elasticsearch-rails', branch: '6.x'
gem 'elasticsearch-rails', github: 'elastic/elasticsearch-rails', branch: '6.x'
gem 'bonsai-elasticsearch-rails', github: 'omc/bonsai-elasticsearch-rails', branch: '6.x'

Make sure the branch you choose corresponds to the version of Elasticsearch that your Bonsai cluster is running.

What Do These Gems Do, Anyway?

• elasticsearch-model. This gem is the only one that is actually required. It does the actual search integration for Ruby on Rails applications.
• elasticsearch-rails. This optional gem adds some nice tools, such as rake tasks and ActiveSupport instrumentation.
• bonsai-elasticsearch-rails. This optional gem saves you the step of setting up an initializer. By default, the Elasticsearch client will attempt to create a connection to localhost:9200, which is not where your Bonsai cluster is located. This gem simply reads the BONSAI_URL environment variable which contains the cluster URL, and uses it to override the defaults.

Once the gems have been added to your Gemfile, run bundle install to install them.

Step 4: Add Elasticsearch to Your Models

Any model that you will want to be searchable with Elasticsearch will need to be configured. You will need to require the `elasticsearch/model` library and include the necessary modules in the model.

For example, this demo app has a User model that looks something like this:

require 'elasticsearch/model'

class User < ApplicationRecord
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks
  settings index: { number_of_shards: 1 }
end

Your model will need to have these lines included, at a minimum.

What Do These Lines Do?

• require 'elasticsearch/model' loads the Elasticsearch model library, if it has not already been loaded. This isn’t strictly necessary, provided the file is loaded somewhere at runtime.
• include Elasticsearch::Model tells the app that this model will be searchable by Elasticsearch. This is mandatory for the model to be searchable with Elasticsearch.
• include Elasticsearch::Model::Callbacks is an optional line, but injects Elasticsearch into the ActiveRecord lifecycle. So when an ActiveRecord object is created, updated or destroyed, it will also be created/updated/destroyed in Elasticsearch.
• settings index: { number_of_shards: 1 } This is also optional, but strongly recommended. By default, Elasticsearch will create an index for the model, with 5 primary shards and 1 replica. This will actually create 10 shards, and is ludicrously over-provisioned for most apps. This line simply overrides the default and specifies 1 primary shard(a replica will also be created by default).

You can optionally avoid (or increase) replicas by amending the settings like this:

require 'elasticsearch/model'

class User < ApplicationRecord
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks
  settings index: {
    number_of_shards:   1, # 1 primary shard; probably fine for most users
    number_of_replicas: 0  # 0 replicas; fine for dev, not for production.
  }
end

If you have more questions about shards and how many is enough, check out our Shard Primer and our documentation on Capacity Planning.

Step 5: Create a Search Route

You will need to set up a route to handle searching. There are a few different strategies for this, but we generally recommend using a dedicated controller for it, unrelated to the model(s) being indexed and searched. This is a more flexible approach, and keeps concerns separated. You can always render object-specific partials if your results involve multiple models.

Our Example

In our example Rails app, we have one model, User, with a handful of attributes. It looks something like this:

require 'elasticsearch/model'

class User < ApplicationRecord
  include Elasticsearch::Model
  include Elasticsearch::Model::Callbacks
  settings index: { number_of_shards: 1 }
end

To implement search, we created a file called app/controllers/search_controller.rb and added this code:

class SearchController< ApplicationController
  def run
    # This will search all models that have `include Elasticsearch::Model`
    @results = Elasticsearch::Model.search(params[:q]).records
  end
end

We then created a route in the config/routes.rb file:

post '/search', to: 'search#run'

Next, we need to have some Views to render the data we get back from Elasticsearch. The run controller action will be rendered by creating a file called app/views/search/run.html.erb and adding:

  
Search Results
  <% if @results.present? %>
    <%= render partial: 'search_result', collection: @results, as: :result %>
  <% else %>
    
Nothing here, chief!
  <% end %>

This way if there are no results to show, we simply put a banner indicating as such. If there are results to display, we will iterate over the collection (assigning each one to a local variable called result), and passing it off to a partial. Create a file called app/views/search/_search_result.html.erb and add:

  

    <%= link_to "#{result.first_name} #{result.last_name} <#{result.email}>", user_path(result) %>

    <%= result.company %>

    <%= result.company_description %>

    

  

This partial simply renders a search result using some of the data of the matching ActiveRecord objects.

At this point, the User model is configured for searching in Elasticsearch, and has routes for sending a query to Elasticsearch. The next step is to render a form so that a user can actually use this feature. This is possible with a basic form_with helper in a Rails view.

In this demo app, we added this to the navigation bar:

<%= form_with(url: "/search", method: "post", class: 'form-inline my-2 my-lg-0', local: true) do %>
    <%= text_field_tag(:q, nil, class: "form-control mr-sm-2", placeholder: "Search") %>
    <%= button_tag("Search", class: "btn btn-outline-info my-2 my-sm-0", name: nil) %>
<% end %>

This code renders a form that looks like this:

Please note that these classes use Bootstrap, which may not be in use with your application. The ERB scaffold should be easily adapted to your purposes.

We’re close to finishing up. We just need to tell the app where the Bonsai cluster is located, then push our data into that cluster.

Step 6: Tell Elasticsearch Where Your Cluster is Located

By default, the gems will try to connect to a cluster running on localhost:9200. This is a problem because your Bonsai cluster is not running on a localhost. We need to make sure Elasticsearch is pointed to the correct URL.

If you are using the bonsai-elasticsearch-rails gem, then all you need to do is ensure that there is an environment variable called BONSAI_URL set in your application environment that points at your Bonsai cluster URL.

Heroku users will have this already, and can skip to the next step. Other users will need to make sure this environment variable is manually set in their application environment. If you have access to the host, you can run this command in your command line:

# Substitute with your cluster URL, obviously:
export BONSAI_URL="https://abcd123:efg456@my-cluster-123456.us-west-2.bonsaisearch.net:443"

Writing an Initializer

You will only need to write an initializer if:

• You are not using the bonsai-elasticsearch-rails gem for some reason, OR
• You are not able to set the BONSAI_URL environment variable in your application environment

If you need to do this, then you can create a file called config/initializers/elasticsearch.rb . Inside this file, you will want to put something like this:

# Assuming you can set the BONSAI_URL variable:
Elasticsearch::Model.client = Elasticsearch::Client.new url: ENV['BONSAI_URL']

If you’re one of the few who can’t set the BONSAI_URL variable, then you’ll need to do something like this:

# Use your personal URL, not this made-up one:
Elasticsearch::Model.client = Elasticsearch::Client.new url: "https://abcd123:efg456@my-cluster-123456.us-west-2.bonsaisearch.net:443"

If you’re wondering why we prefer to use an environment variable instead of the URL, it’s simply a best practice. The cluster URL is considered sensitive information, in that anyone with the fully-qualified URL is going to have full read/write access to the cluster.

So if you have it in an initializer and check it into source control, that creates an attack vector. Many people have been burned by committing sensitive URLs, keys, passwords, etc to git, and it’s best to avoid it.

Additionally, if you ever need to change your cluster URL, updating the initializer will require another pass through CI and a deployment. Whereas you could otherwise just change the environment variable and restart Rails. Environment variables are simply the better way to go.

Step 7: Push Data into Elasticsearch

Assuming you have a database populated with data, the next step is to get that data into Elasticsearch.

Something to keep in mind here: Bonsai does not support lazy index creation, and even Elastic does not recommend using this feature. You’ll need to create the indices manually for each model that you want to search before you try and populate the cluster with data.

If you have the elasticsearch-rails gem, you can use one of the built-in Rake tasks. To do that, create a file called lib/tasks/elasticsearch.rake and simply put this inside:

require 'elasticsearch/rails/tasks/import'

Now you will be able to use a Rake task that will automatically create(or recreate) the index:

bundle exec rake environment elasticsearch:import:model CLASS='User'

You’ll need to run that Rake task for each model you have designated as destined for Elasticsearch. Note that if you’re relying on the BONSAI_URL variable to configure the Rails client, this variable will need to be present in the environment running the Rake task. Otherwise it will populate your local Elasticsearch instance(or raise some exceptions).

If you don’t use the Rake task, then you can also create and populate the index from within a Rails console with:

# Will return nil if the index already exists:
User.__elasticsearch__.create_index!

# Will delete the index if it already exists, then recreate:
User.__elasticsearch__.create_index! force: true

# Import the data into the Elasticsearch index:
User.import

Step 8: Put it All Together

At this point you should have all of the pieces you need to search your data using Elasticsearch. In our demo app, we have this simple list of users:

This search box is rendered by a form that will pass the query to the SearchController#run action, via the route set up in config/routes.rb :

This query will reach the SearchController#run action, where it will be passed to Elasticsearch. Elasticsearch will search all of the indices it has (just users at this point), and return any hits to a class variable called @results.

The SearchController will then render the appropriate views. Each result will be rendered by the partial app/views/search/_search_result.html.erb. It looks something like this:

Congratulations! You have implemented Elasticsearch in Rails!

Final Thoughts

This documentation demonstrated how to quickly get Elasticsearch added to a basic Rails application. We added the gems, configured the model, set up the search route, and created the views and partials needed to render the results. Then we set up the connection to Elasticsearch and pushed the data into the cluster. Finally, we were able to search that data through our app.

Hopefully this was enough to get you up and running with Elasticsearch on Rails. This documentation is not exhaustive. However, there are a number of other use cases not discussed here. There are other additional changes and customizations that can be implemented to make search more accurate and resilient. 

You can find information on these additional subjects below. And if you have any ideas or requests for additional content, please don’t hesitate to let us know!

Additional Resources

• Shard Primer
• Capacity Planning