When Search Goes Cognitive - Why Should You Care?
As a vendor-agnostic search and analytics consulting company, we like to provide useful resources that introduce our audience to a broad range of innovative advancements in our space. One of the growing trends is “Cognitive Search.”
Today, search is no longer just keyword matching; it has evolved to become “cognitive” – the ability to deliver the most relevant answers to natural-language questions. In this post, our guest blogger, Laurent Fanichet from Sinequa, will be discussing how cognitive search is used and its impact on modern business use cases.
What's Cognitive Search?
We are still at an early stage of an evolving market, where many players try to position their offering as “cognitive.” The Cognitive Computing Consortium is trying to come to a consensual definition, such that potential user organizations get a more objective idea of what to expect from cognitive computing. Here is a short definition: a cognitive information system is capable of extracting relevant information from big and diverse data sets for users in their work context.
As we move into the era of this “cognitive computing,” new search solutions combine powerful indexing technology with advanced Natural Language Processing (NLP) capabilities and machine learning algorithms in order to build an increasingly deep corpus of knowledge from which to feed relevant information and 360° views to users in real-time. This is what leading analyst firms call “Cognitive Search” or “Insight Engines.” These cognitively-enabled platforms interact with users in a more natural fashion, learn/progress as they gain more experience with data and user behavior, and proactively establish links between related data from various sources, both internal and external.
In a recent brief, Forrester defines Cognitive Search as:
“Indexing, natural language processing, and machine-learning technologies combined to create an increasingly relevant corpus of knowledge from all sources of unstructured and structured data that use naturalistic or concealed query interfaces to deliver knowledge to people via text, speech, visualizations, and/or sensory feedback.”
How Does Cognitive Search Work to Deliver Relevant Knowledge?
It extracts valuable information from large volumes of complex and diverse data sources. It is crucial to tap into all available enterprise data whether internal or external, both structured and unstructured, to provide deeper insights to users in order for them to make better business decisions. Cognitive search provides this connection to provide comprehensive insights.
It provides contextually and relevant information. Finding relevant knowledge across all available enterprise data requires cognitive systems using Natural Language Processing (NLP) capable of “understanding” what unstructured data from texts (documents, emails, social media blogs, engineering reports, market research…), and rich-media content (videos, call center recordings..), is about. Machine learning algorithms help refine the insight gained from data. Trade and company dictionaries and ontologies help with synonyms and with relationships between different terms and concepts. That means a lot of intelligence and horsepower “under the hood” of a system providing “relevant knowledge” or insight.
It leverages machine learning capabilities to continuously improve the results relevancy. Machine learning algorithms provide added value by continuously refining and enhancing the search results in an effort to provide the best relevancy to users. The following machine learning algorithms are amongst the most popular ones:
- Classification by example – a supervised learning algorithm used to extract rules (create a model) to predict labels for new data given a training set composed of pre-labeled data. For example, in bioinformatics, we can classify proteins according to their structures and/or sequences. In medicine, classification can be used to predict the type of a tumor to determine if it’s harmful or not. Marketers can also use classification by example algorithms to help them predict if customers will respond to a promotional campaign by analyzing how they reacted to similar campaigns in the past.
- Clustering – an unsupervised learning algorithm whereby we aim to group subsets of documents by similarity. When we don’t necessarily want to run a search query on the whole index, clustering is used. The idea is to limit our search to a specific group of documents in each cluster. Unlike classification, the groups are not known beforehand, making this an unsupervised task. Clustering is often used for exploratory analysis. For example, marketing professionals can use clustering to discover different groups in their customer/prospect database and use these insights to develop targeted marketing campaigns. In the case of pharmaceutical research, we can cluster R&D project reports based on similar drugs, diseases, molecules and/or side effects cited in these reports.
- Regression – a supervised algorithm that predicts continuous numeric values from data by learning the relationship between input and output variables. For example, in the financial world, regression is used to predict stock prices according to the influence of factors like economic growth, trends or demographics. Regression can also be used to create applications that predict traffic flow conditions depending on the weather.
- Similarity – not a machine learning algorithm but simply a heavy computing process that helps build a matrix synthesizing the interaction of each sample of data with another one. This process often serves as a basis for the algorithms cited above, and can be used to identify similarities between people in a given group. For example, pharmaceutical R&D can rely on similarity applications to constitute worldwide teams of experts for a research project based on their skills and their footprints in previous research reports and/or scientific publications.
- Recommendation – one of the various use cases consists of merging several basic algorithms to create a recommendation engine proposing contents that might be of interest to users. This is called “content-based recommendation,” which offers personalized recommendations to users by matching their interest with the description and attributes of documents.
Thanks to new technology advancements, cognitive search brings to data-driven organizations a new generation of search enabling them to go far beyond the traditional search box, empowering its users to get immediate and relevant knowledge at the right time on the right device.
To learn more about Cognitive Search & Analytics in action, please feel free to read the IDC AstraZeneca Case Study.
About the author: Laurent Fanichet is the Vice President of Marketing at Sinequa. Based in New York, Laurent is responsible for driving the overall marketing strategy for the company including field and product marketing, marketing communications, as well as brand awareness. Sinequa, a Search Technologies' partner, is a leader in cognitive search and analytics.