Edukate Blog

Phone screenshot of a mobile search on Google.

Why we chose Algolia over Elasticsearch

As website search functionality evolves and complex cross-model searching becomes more popular, the need for an elegant and efficient solution to query databases is a necessity.

For Edukate’s https://www.edukate.com/scalability/financial wellness content library, which includes videos, articles, company-specific content, and other financial resources, a high-performance and natural-feeling search engine is important to our user experience.

Choosing Elasticsearch

When Edukate first looked for a search engine, we chose Elasticsearch because of its out-of-the-box document searching capabilities.

Because we use Ruby on Rails, we used the wrapper gem, Chewy, to get up and running with Elasticsearch quickly.

But as our content library grew with new resources and types of content being created and indexed all the time, we began to notice issues between searching the indices for relevant data and the performance of our application.

Given the nature of our index-heavy environment, this was the first indication that we needed a more robust search solution. To plan for future growth, we started looking for a new search engine.

Ranking-driven results

Based on our content delivery needs, we recognized the need for our search engine to not only interpret a complex content-ranking formula, but also have robust searching capabilities to deliver the most relevant content to our users.

We prioritized the need for a stronger ranking system and discovered Algolia.

Algolia has been leading the search space for a number of years and is about 200x faster than Elasticsearch.

The discrepancy in performance is because of Elasticsearch’s inability to automatically distribute data across multiple data centers to reduce latency.

While it’s possible to implement, their documentation specifies that it may cause slowdowns when returning results.

As we tested Algolia side-by-side, the difference in performance for our needs was clear. Our initial tests showed Algolia returned results between in as little as 1ms to 11ms, whereas Elasticsearch returned results in about 150ms to 300ms.

Below, you can see how long it then takes to deliver those results to the front-end portion of the application.

Algolia:

Elasticsearch:

The code difference for indexing a model is fairly different with all of the rulings and targets:

Elasticsearch:

define_type Video, delete_if: -> { deleted_at } do
settings analysis: {
   analyzer: {
     ngram_analyzer: {
        tokenizer: 'ngram_tokenizer'
     },
     tokenizer: {
      ngram_tokenizer: {
       type: 'nGram',
       min_gram: '2',
       max_gram: '4',
       token_chars: [ 'letter', 'digit' ]
      }
    }
   }
 }
 field :title
 field :description
end

As you can see, Elasticsearch lets us search by video title and description, but we don’t have a clear ranking for how to decide between two videos that are very similar.

Additionally, setting the tokenizer in Elasticsearch isn’t very intuitive, as you have to set all of the analyzer’s settings every time you conduct a search.

Algolia:

algoliasearch auto_index: true do
  attributes :title, :description
  searchableAttributes [‘title’, ‘description’]
  customRanking [‘desc(popularity)’]
  minWordSizefor1Typo 4
  minWordSizefor2Typos 8
  hitsPerPage 5
end

Using Algolia, we can search by attributes :title, :description and use a custom ranking formula when two videos are very similar using a much simpler tokenizer.

We’re very excited about the future of Algolia on our platform. In the near future, we’re looking to try out autocomplete, creating more personalized searches, and further adapting the ranking formula to our content strategy.

Algolia was simple to launch and was fairly simple to set up. And because the Algolia team built a Heroku plugin, we can keep our devops all in one place!

Evan Glazer is a software engineer and self-starter that prides himself in being able to pick up new languages quickly, efficiently, and effectively. At Edukate, he uses Ember and Ruby on Rails and works with Natural Language Processing and Machine Learning.

Leave a Reply

Your email address will not be published. Required fields are marked *

  • Edukate integrates with your current employer benefits provider.

  • Most of our programs can be rolled out in less than 30 days.

Schedule a Demo