How to Build a Search Engine with Algolia
This tutorial will give you a step by step walkthrough to setup an Algolia search engine for your data.
Algolia invites Codementor readers to register through this link https://www.algolia.com/redeem/codementor15 to get a voucher of 250$ on any Algolia plan (Free starter plan for 5 months).
Building a search engine for your website used to be a tedious process. You had to perfect long XML files and worry about the infrastructure. But now you can search your data in real-time with Algolia without having to worry about any of these issues.
This tutorial will give you a step by step walkthrough to setup a search engine for your any kind of data.
By the end of this tutorial, you’ll be able to set up a search page similar to this one:
http://demos.algolia.com/instant-search-demo/
1. Register for an account
Go to http://algolia.com and register for an account. You have 14 days free trial to evaluate Algolia.
Select the data center with the least latency or the closest to your location.
2. Creating your first index
If you don’t have a real set of data, you can use Algolia’s sample records (click on the use sample user profile records button). If you have a real set of data, you can import them as JSON or CSV.
3. Your Algolia index
The Algolia indices page shows you the data belonging to your index and allows you to search it in real-time. For example, you can type Lina in the search input to retrieve all the profiles that contain the word Lina in them.
4. Configuring ranking for your index
Locate the upper tab under your index and select Ranking to set up the ranking factors for your search. Algolia offers many ranking factors, I’m going to briefly mention the most important factors. They do have a good documentation and a decent support if you need to customize your index further more.
I suggest you keep the default values for the following fields, unless you face an issue with them later on:
- Min chars to accept 1 typo
- Min chars to accept 2 typos
- Allow typos on numeric tokens
- Ignore plural
- Disable typo-tolerance on
- Query words interpretation
- Ranking
Attributes to index: Lets you select the attributes you want algolia to search. For instance, you want to search the email address but never search the home address field.
Custom Ranking: Allows you to specify the popularity of your data. Say the number of followers defines how important a person is, making it more relevant when sorting search results.
Optional words: Whenever you’re searching text or sentences rather than keywords, optional words are the most important configuration. It will instruct the search engine to skip the defined optional words (also known as Stop words). Optional words are a list of commonly used words (such as “a”, “the”, etc.) that are deemed irrelevant because they occur frequently and matching those words would not make be helpful when aiming to return relevant data. Here’s a list of stop words for the English language, you can also find the list for other languages.
Synonyms: Synonyms list would be helpful to define equivalent keywords. For instance: “any”, “whatever” and “whatsoever” will be treated as if they were the same word.
Remove words if no results: In most cases, this setting is very useful as Algolia tries to re-run your search during the same API call by removing 1 word at a time. Use lastWords if you want to remove words from the end of the search query, or firstWords if you want to remove words from the beginning of the search query.
5. Configuring display for your index
I suggest you keep the default values for the following fields, unless you face an issue with them later on:
- Max Values Per Facet
- Unretrievable attributes
- Attributes to snippet
- Replace synonyms in highlight
- Highlight prefix tag
- Highlight postfix tag
- Attribute for distinct
- Distinct
Attributes for faceting: Faceting is a very useful feature offered by Algolia. It lets you categorize your data according to a certain field. For instance, you can categorize the data by company, language or country.
Hits per page: Like many other configuration options, the “hits per page” can be overriden at query-time. Meaning that when you send the request to Algolia using their API, you can provide a new value that will override the one saved in your index settings. This setting is used to specify the number of results to show per page.
Attributes to retrieve: Defines the attributes you want to retrieve, by default it returns all attributes.
Attributes to highlight: Defines the attributes that will be highlighted when a match is found. Very similar to the emphasis (bold) that happens in Google search when a page title contains a word you searched for. By default, it highlights all the attributes that have been indexed. Also by default, the highlighting will happen between <em> and </em> tags. You can change this via the highlight prefix tag and highlight postfix tag.
6. Adding/Updating data from the backend
In most scenarios, you’re going to need to add new documents to your index frm the backend. You might also need to update previous documents or remove them. Algolia provides excellent server side libraries for the following languages/frameworks; PHP, Ruby, Rails, Node, Python, Objective C, Java, C#, Go, etc.
These packages are of a very high standard in the open source community since they integrate seamlessly with each language’s package managers (composer for php, gem for ruby, etc.) and since they are unit tested.
Because the code is fairly straightforward, I’m going to show a quick demo using their PHP client.
Assuming that you have installed their library using composer, as recommended on their PHP documentation page, here’s a sample code for insert, update, delete and search methods:
require 'vendor/autoload.php'; // if you don't use Composer: require 'path/to/algoliasearch.php';
$client = new \AlgoliaSearch\Client("API KEY", "API SECRET");
$index = $algolia->initIndex( ‘index_name’ );
//Insert example
$result = $index->saveObject([
"first_name" => $first_name,
"last_name" => $last_name,
"company" => $company,
"objectID" => $user_id
]);
//update example (creates new document if objectID was not found
$result = $index->partialUpdateObject(array("first_name" => "Jimmie", "objectID" => "359"));
//delete example:
$index->deleteObject("359");
//search
$results_array = $index->search(‘how can I find the nearest restaurant?‘', array("hitsPerPage" => 5));
//notice how we are overriding the hitsPerPage at query time7. Searching your data in real-time
Algolia provides client side libraries for Javascript, Android and iOS. With these libraries we can search our index in real-time. I’m going to show how to do that using their Javascript client library:
You can install their client library from their Javascript Documentation page, or using bower:
//if you're using bower:
bower install algoliasearch
<script src="bower_components/algoliasearch/dist/algoliasearch.js"></script>
//or you can just include the latest version of the JS client hosted on jsDelivr CDN (compatibility guaranteed)
<script src="//cdn.jsdelivr.net/algoliasearch/3/algoliasearch.min.js"></script>
<script>
var client = algoliasearch('ApplicationID', 'apiKey');
var index = client.initIndex('YourIndexName');
var params = { hitsPerPage: 20 };
Be careful not to ever reveal your API secret key as it will be visible to users and they will be able to alter with your index. Using the JavaScript client, you should only perform read-only operations such as the Search functionality.
And then this is a sample code that searches your index.
index.search('How can I find the nearest restaurant?', params)
.then(function searchSuccess(content) {
for (var i = 0; i < content.hits.length; ++i) {
console.log(content.hits[i]);
}
})
.catch(function searchFailure(err) {
console.error(err);
});Other important features that you might consider
Algolia has a very handy Analytics module that shows the top queries that your users are searching for, alongside the top queries that are not returning any result. The analytics module will let you dig deep into your data according to your user’s search behavior.
Here’s a sample screenshot from the Algolia Analytics dashboard:
There’s also another great feature called DSN which stands for Distributed Search Network, that allows to replicate search instances around the world in the same way as CDNs do with static resources. With 12 data centers, results delivery times are under 50ms in the world’s top markets. DSN seamlessly synchronizes your data across all data centers. Visit their page to see more information on the location of the Data centers, average response time and code setup.
Conclusion
Whenever I want to consider a software as a service, I make sure to check how serious the company is. So far, Algolia has received about USD3 Million in funding, which makes it obvious how serious the company is. This also gives us a clear understanding of how interesting their pipeline would be. Also, from several personal experiences, they have a decent support. They wouldn’t mind jumping on a Skype call with you to help you with your setup.
These factors alongside the ones discussed in the article, such as how easy it is to setup, manage and search your index and how helpful the analytics module is, encourages us to use Algolia whenever we want to build a real-time search engine.
Don’t forget that you get 250$ free if you signup through the link at the beginning of the article.
PHP Expert Codementor Jad Joubran is the Co-founder and CTO of eTobb.com, an online platform connecting patients and doctors. He has managed a team of 5 developers and 2 designers and 6+ years of experience with PHP. He has also been selected as the Top 20 Lebanese Entrepreneurs in Lebanon by Executive Magazine in 2013. (Beirut, Lebanon and Silicon Valley, United States)
