Back to top

Big Data Analytics in Recruiting


  • Text processing adds new dimensions to analytics applications
  • Techniques and best practices are well-established in the enterprise search industry
  • Looking at data from different perspectives provides additional insight.



Recruitment, from placing unskilled temporary workers, through to headhunting the next CEO for an organization, is a competitive, multi-billion dollar business. Many recruitment firms focus on key performance indicators such as:

  • Fill Rates: The percentage of job opportunities dealt with by their firm, that are actually filled by them, and contribute to revenue.
  • Recruiter Productivity: Not as easy to measure, but an indication how many opportunities individual recruiters are able to respond to or deal with, in the time available.

In combination, increased productivity multiplied by improved conversion rates, directly impact the bottom-line profits.



At a detailed level, there is much that recruitment organizations can do to improve the productivity of professional recruiters. 

"Searching" is a significant part of a professional recruiter's day, and it follows that improved search effectiveness leads directly to improved productivity. As with any search application, key factors include:

  • Ensuring search access to all potentially useful information sources.
  • Normalizing information sources, so that relevancy algorithms have a platform on which to perform.
  • Tuning relevancy to suit the use case.

We've been involved with a number of projects in the recruitment sector, where customers are pushing the boundaries of search effectiveness. For example, some are using the concept of "Search & Match." The most common search scenario for a recruiter is being presented with a new job opportunity to fill, and then searching for candidates who fit the bill.

Search & Match uses the entire job description as the search query and statistically maps it (using factors such as job titles, skills, location, and salary range) to suggest appropriate candidates from the CV pool to the recruiter, in a carefully ranked order.

Once recruiting companies have a comprehensive, relevancy tuned system in place for search, they can explore analysis applications, based initially on the same content sets under index.



At the macro level, recruitment companies can find strategy guidance through the analysis of inbound job descriptions and candidates' resumes.

The top-line numbers of CVs submitted and job vacancies gathered are routinely monitored, so that overall trends can be identified, and responded to. For example, recruiters experience predictable seasonal peaks and troughs in job-seeking activity, January being the busiest month of the year for many countries and industry sectors.

However, some of the most useful insights are only found by drilling down into the information.

In the structured world of transaction logs and data warehouses this is a known science, practiced for decades by (for example) the retail and financial services sectors. In the recruitment industry, the key data for analysis are job descriptions and resumes - but both are naturally unstructured in nature.


Structures can be added to job descriptions and resumes using text analytics (also known as "CV parsing"). It matters a lot how you go about this because analytical results are only actionable if the user has confidence in the underlying data. CV parsing adds structure, but that structure needs to be accurate and normalized. Transparency is provided by enabling analysts to easily check back to the source content, and understand how and why the structure underlying the analysis was created.

There are numerous dimensions that text analytics can use to add structure to job descriptions and resumes. Some of the more obvious are job titles, industries, locations, experience, skills, pay grades, and qualifications.

Through the provision of multiple dimensions, analyses can be conducted over specific combinations of (for example) geographic location and role. This can reveal "hot spots" which the recruiter may wish to target, or misalignments between vacancies and potential employee skills in particular states or cities.

Leading recruiters tend to keep historical data (although some are better at this than others), enabling trends to be plotted over longer periods of time.


Search Technologies believes that a holistic view of search and analytics is the best way forward for recruitment firms. Too often, resumes are isolated in specific repositories and difficult to search and share. An important first step is to ensure that all inbound resumes and job opportunities, regardless of the source, are captured and made easily (and very quickly) searchable. This is a classic enterprise search scenario, and it directly supports the KPIs of recruitment firms.

Creating a great search experience will almost always involve the provision of search navigation options, and for that, text analytics (CV parsing) is used to create the necessary metadata structure. 

Then based on this same platform, an additional step can add analytical functions. Large recruitment companies have a lot of content, which when analyzed and cross-referenced at a word level (there are billions of words in millions of documents), is very much a "Big Data" challenge.

Recruitment firms who take the initiative and fully leverage their content, are likely to create competitive advantages for themselves.


The technologies needed are readily available in the open source (SolrElasticsearch, and Hadoop for example), and professional, reasonably-priced support for production systems is available from providers such as Elasticsearch and Cloudera.

For many recruitment companies, the missing ingredient is implementation expertise, and that is where Search Technologies can help.