Big Data and Data Warehouse Search Solutions
Solve Big Data Problems by Adding Search to a Data Warehouse
BIG DATA SOLUTIONS START WITH GOOD SEARCH
Search Technologies has been a leader in implementing search-based applications for quite some time with a “who’s who” list of customers. We have always been very pragmatic and have stayed away from the hype behind buzzwords like “cloud” and “big data.” Nevertheless, our work with customers and market forces have pulled us in that direction.
Whereas business intelligence and predictive analytics solutions used to be the domain of relational database and data warehouse projects with eight-figure price tags, we are now seeing the pairing of NoSQL database and search technologies to address some very interesting use cases. The problems addressed range from fraud detection, to social media data mining, to better predictions of government elections, and many more. And open source technologies have also made these solutions much more affordable.
THE DEMOCRATIZATION OF BIG DATA
There is a reason that search has become so readily accepted as user paradigm for big data applications. Unlike 20 years ago when corporate Intranets first emerged, we now use search every day in the form of Google or Bing. We now have, at our fingertips, the whole of all of mankind’s learnings.
Now people want to point that search bar behind the firewall and they want increasingly more data sources to be searched and richer search application experiences. We call this phenomenon the democratization of big data.
UNLOCKING DATA WAREHOUSES
While much is written about leveraging new data sources like social media and untapped log files, the reality is that much of the rich data needed for interesting business use cases reside in existing large data warehouses – the result of projects initiated long before “big data” was coined. The challenge is then unlocking this data and fitting it into more modern big data architectures.
- From a technical perspective, this means developing methodologies, processes, and technologies to arrange data from warehouses into a schema-less form (JSON or XML).
- Additional expertise is required on how to index the derived data to fully enable interactive analysis with powerful user interfaces.
Search Technologies provides expert consulting and implementation services, helping customers build killer analysis applications. Our expertise combines our know-how in both open source search (Elasticsearch, Solr) and big data technologies (Hadoop, Cloudera, Cassandra, etc.). In addition to extensive expertise, we also provide software tools for capturing both warehouse data and other content sources, and for the transformation of data into a schema-less format, ready for indexing.
Search Technologies can help you build custom search applications onto existing or new data warehouses with the following characteristics:
- A professionally implemented search engine is fast, flexible, and familiar in the way that it operates. Everyone knows search.
- A good search engine combined with an analytics user interface that allows anyone to interactively explore a dataset, and derive insight from it.
- A solution typically using trusted open source software such as Hadoop, Cassandra, Elasticsearch and Solr (or commercial versions of these platforms)
Big Data Use Cases We’ve Helped Customers Implement:
- Medical and Insurance Fraud Detection
- Search and Match for Recruiters and HR Staff
- Corporate Wide Search / Enterprise Search
- Log Analytics for Risk Management, E-Commerce Personalization, and IT Operations
- Precision Medicine
- Precision Agriculture
- Cyber Security / Threat Detection
- Recommendation Engines for Media & Publishing Companies
- Research & Analytics Platforms
For a deep dive into the subject, read our blog, Big Data for Search and Search for Big Data.
Contact us to see how big data solutions can work for you and get more details about custom applications we can help you implement.