Log Analytics for Improving Search and Personalization in Mobile Apps
Search for Mobile Apps - A Blog Series
Welcome to the third article in our Search for Mobile App series!
To recap, our second article discussed tips on identifying user intent based on their queries and defining tags and action for those queries that can control user interface (UI) display and how your app reacts. We also went over the popular type-ahead/query suggestion feature that takes what a user types and presents a curated list of suggestions.
So, now that you’re a pro at query tagging and crafting smart query suggestions, let’s discuss logging and analytics from a mobile perspective.
Logging? Do I need an axe?
You will if you want to cut through the forest of ambiguity when it comes to user behavior in general and user behavior that is specific to mobile apps. As discussed in Part 1 of this blog series, mobile app usability needs extra consideration. Screens are smaller than on a desktop computer, as are keyboards and text. Less space, mobile environments, and possible slower load times for the UI all lead to changes in user expectations and behavioral differences from desktop to mobile.
In order to fully understand your users, you need a system to capture their actions as they navigate through and use your app. Doing so captures queries, click-throughs, submitted or abandoned cart contents, entry points, notification actions, and other behaviors. A deep dive into this data can spot patterns from the past, which can point to trends for the future. This helps your search team to optimize the search engine, the content team to understand the data in the app, and the marketing team to better connect with your users. Other benefits of user behavior analysis are touched on in the bullet points below.
To gather all relevant data in the log in order to run a successful analysis, the following types of data must be captured:
- The application (App, Desktop, or Mobile Web) being used for search
Of course, these logs are coming from your app; you know that now. But your company might find it beneficial to merge all logs coming from all your applications in order to do global analysis and comparisons. You’ll need some sort of indicator to tell you whether any given log entry is from your mobile app versus your desktop site.
- The date and time (down to the second) when the search was executed
Users can be quick when navigating through an app, so you’ll want to be able to catch, for example, if they tapped by accident and caught their mistake (possibly a second or two on a page) or if they were reviewing their results (as indicated by more time spent between query submittals).
- The user ID of who executed the search
This will allow you to track a certain user’s actions across a session or a specific time frame. Did your user who just bought a phone off your site use your app to search for accessories?
- The text exactly as entered by the user
Using the exact query string gives insight into the user’s thought process. Do your users refer to a smart phone device as a phone, a cell, or a mobile, or do they just refer to it as the model, like Galaxy? Do your users vary their searches depending on what device they’re using?
- Search parameters
These should include user-specified parameters, such as applied filters, and should include device-specified parameters, such as target URL, geolocation or device type. You’ll want to capture that action to spot patterns and trends.
- Whether or not the query was selected from a query suggestions recommendation
This will help confirm and fine-tune your query suggestions.
- The starting result number requested
This tells you if the user used the pagination feature of the app to reach more results; this might indicate your relevancy needs to be tuned so that users don’t have to deal with swiping around their screen to see relevant results and answers.
You should already be capturing this data, but if not, don’t worry. Start now and begin building your data store. You want as much data as you can get to run a good analysis of your users’ behavior. Even a month’s worth of data can show trends and patterns in searching and results quality.
What data do I need to uncover user behavior patterns and trends?
Speaking of patterns and trends, these are a small sample of what you want to identify in your data:
- Top queries
These indicate what your users are most interested in finding or the top questions they have that they need answered. Query patterns, such as structure of the query (single term or question format), time frame of the query (holidays, weekends, certain days of the week), and popular trends can all be gleaned from your top queries.
- Zero-results queries
These are queries that produce no results. This could be because the term is not found anywhere in your index. This might be legitimate, or it might indicate you need to be a more thorough job on indexing your data. Zero-results queries might also be typos. If a zero-results query was only run once during a year of data, then it’s probably not something you need to address. But, if that query was run hundreds of times (and might actually be one of your top queries), consider implementing a term dictionary for spell-checking.
- Linguistic improvements
Review the top queries and those that produce no results and identify any areas of improvement that implementing synonyms, stemming, or lemmatization would help. For example, if your users are searching on "off road" and not finding any results, and your app is for a company that sells parts for vehicles that have 4x4 driving capabilities, you need to implement a synonym for that query to link "off road" to related terms, like "4x4," "4 wheel drive," "off-road," and "offroad," to name a few.
- Query tagging opportunities
Match your users’ query terms with patterns to identify their intent, define tags to capture intent, and allow query processing and the app UI to act on the tags to provide customized result behaviors. For a detailed example of query tagging, you can refer back to Part 2 of this blog series.
What entices your users to click? Are they clicking on ads or banners? Do they want to see more information on a result? Are they adding things to the cart? Are they completing the purchase?
Now that I have the data, what do I do with it?
Log analytics is your answer. It’s where all the data from your logs come together in harmony, or rather, in nicely sorted and organized columns of interesting information. Logs are what you feed your analytics tool; analytics are the computations run by the tool to analyze the data in the logs; the output of the tool is what your search analysts use to figure out what your users are thinking and doing inside your app.
Log analytics tools frequently use software development kits (SDKs) to capture information from mobile apps. Depending on the type of app, iOs or Android, different configurations must be applied for the analytics tool to capture the actions and events occurring in your app. Or, consider using a flexible, open source log analytics solution, such as the Elastic Stack (Elasticsearch, Logstash, Beats, and Kibana) that offers a powerful, fully featured visualization framework.
Exactly which tool to use and how to configure it is outside the scope of this article; here’s how we used an open source log analytics tool for an e-commerce customer’s logging needs for the technical details. For a deep-dive into the open source log analytics architecture built on the Elastic Stack, get our free white paper.