Home » Uncategorized

How to build a search engine: Part 4

This post is the fourth part of the multi-part series on how to build a search engine –

In case you are in a hurry you can find the full code for the project at my Github Page

Just a sneak peek into how the final output is going to look like –

how to build a search engine - screenshot_es

This is the last part on building an end-end search engine. In this part we will take a look at how to go about building the front end. This will be an AngularJS application and will consist of some HTML and Javascript. Straight t it then –

The UI components –

  1. A simple textbox with autocomplete features
  2. A table that can be sorted and filtered

All codes are readily available on Github along with the data itself. Here we will just do a walkthrough of what we are doing to make it all happen.



<div class="header-limiter">  <div class="input-group">  <input id="txt_search" type="textbox" popover-trigger="focus" placeholder="Enter search query here..." class="form-control ui-widget" ng-model="search_query" ng-model-onblur focus-on="focusMe" ng-keyup="$event.keyCode == 13 ? search_click() : null" typeahead="item for item in autocomplete($viewValue) | limitTo:15 " typeahead-on-select="search_query=$model" autofocus>  <span class="input-group-btn">  <button class="btn btn-default" type="button" id="btn_search" ng-click='search_click()'>Search</button>  </span>  </div> 


This will do the model binding so that we can grab this component in the JS module

typeahead=”item for item in autocomplete($viewValue) | limitTo:15 “

For every character typed into this textbox, a query will be sent to elasticsearch for autocomplete suggestions. More clarity on this when we study the autocomplete function in the JS module


The code for the table part is as below –

<table class="table table-bordered table-striped">             <thead>                 <tr>              
<div><input class="form-control" ng-model="f._id"></div>
<a href="#" ng-click="sortType = '_id'; sortReverse = !sortReverse"> URI
<span ng-show="sortType == '_id' && !sortReverse" class="fa fa-caret-down"></span> <span ng-show="sortType == '_id' && sortReverse" class="fa fa-caret-up"></span> </a>
<div><input class="form-control" ng-model="f.fields.MNEMONIC"></div>
<a href="#" ng-click="sortType = 'fields.name'; sortReverse = !sortReverse"> Name
<span ng-show="sortType == 'fields.name' && !sortReverse" class="fa fa-caret-down"></span>
<span ng-show="sortType == 'fields.name' && sortReverse" class="fa fa-caret-up"></span> </a>
<div><input class="form-control" ng-model="f._score"></div>
<a href="#" ng-click="sortType = '_score'; sortReverse = !sortReverse"> Score
<span ng-show="sortType == '_score' && !sortReverse" class="fa fa-caret-down"></span> <span ng-show="sortType == '_score' && sortReverse" class="fa fa-caret-up"></span> </a>
<tr ng-repeat="result in search_res | orderBy:sortType:sortReverse | filter:f" ng-show="search_query">
<td>{{ result._id }}
<td>{{ result.fields.name }}</td>
<td>{{ result._score }}</td>

Each td of the thead part is configured to have an input textbox for filtering and and an ngClick for sorting.

The tbody part contains the ngRepeat which would help us push the search results onto it from the JS module

The JS Module

The bulk operations are happening inside the controller so that would be the point of focus.
The function for autocomplete is –

$scope.autocomplete = function(val) {       var keywords = [];       keywords.push(val);   
return client.suggest({
index: 'wiki_search',
size: 5,
body: {
"index_type_suggest" : {
"text" : val,
"completion" : {
"field" : "search_query"
}).then(function (response) {
for (var i in response['index_type_suggest'][0]['options']) {
keywords.push(response['index_type_suggest'][0]['options'][i]['text']); }
return keywords; });

Using the elastic.js module, it makes a call to client.suggest() and this function call is made on every character typed. The output is a list of suggestions which become an integral part of the search-as-you-type feature.

The filter function for the table is below –

$scope.filter_by = function(field) {                             //console.log(field);                              //console.log($scope.g[field]);                     if    
($scope.g[field] === '') { delete
$scope.f['__' + field];
$scope.f['__' + field] = true;
$scope.search_res.forEach(function(v) {
v['__' + field] = v[field] br/>
$scope.g[field]; })

The last part will be the call for the search. It basically calls the elastic.js module’s client.search() function along with a bunch of parameters that makes a request to elasticsearch to get back the response. The body for the search call is below –

{        "fields" : ["name"],            "query": {          
"bool": {
"should": [
{ {
"multi_match": {
"query": search_query,
"fields": [ "text","name"],
"type": "best_fields",
"tie_breaker": 0.5
"minimum_should_match" : 1

This tells elasticsearch to match the query string to the text and name columns in our dataset. I will hold back on the details of how elasticsearch treats this request as that will make this too long. May be a separate write up on that.

Deploying the front end

The final stage is a simple operation. Get all the front end stuff on my Github Page once you have got a grip of what happens. Once that is done, place the FrontEnd folder in <tomcat home>/webapps. Open a command prompt window, navigate to <tomcat home>/bin and run command startup.bat. That should start tomcat. Now all you need to do is  go to


to access the UI. It has been tailor-made to talk to your local elasticsearch. Make sure all systems are running and Viola! start some searches and tell me how it worked out for you!

Some Searches

The last part is having some fun with our custom fuzzy search application –
actor governor bodybuilding” returns Arnold Schwarzeneggar as the top result
england manchester united real madrid free-kicks” returns David Beckham as the top result
fast furious wwe wrestler” returns Dwayne Johnson as the top result

That was all in this multi-series on how to build a search engine. Good enough to get started right? It was an amazing experience penning down this series. See you on the next bit. Have fun!

Originally posted here