Analyzing Drop in User Engagement: 8 Scenarios and How to Handle Them

A statistical anomaly is any significant deviation from normal or projected values; for instance, having 500 or 15 installs instead of 100 per day on average.

Deviations, whether upward or downward, have both positive and negative implications. 500 installs per day instead of 100 may be due to either the app being put on the store’s top list, or mobile fraud.

This Guide to Analyzing Drop in User Engagement describes 8 scenarios of why user activity drops based on our assumptions and how myTracker can effectively address them.


How to track anomalies

Keeping track of anomalies helps quickly identify and solve technical problems in the app.

So, if a new version release results in only 5 authorization events per day for 100 installs, it is possible that new users have problems logging in to the app on a certain OS.

You can find such anomalies by using both the app’s internal analytics and dedicated solutions for monitoring user activity, for example, myTracker: our Report Builder will help you build reports on the metrics and in the form you like.

Positive anomalies

An anomaly may be positive if it results from online or offline marketing activities.

  • An anomaly may be positive if it results from online or offline marketing activities.
  • In the case of online activities, traffic source statistics could explain the rising metrics. For offline activities, you may need to compare the metrics against the timing of your TV ads.
  • With paid promotion channels, anomalies are quite straightforward: they occur when your partner buys more ads for the app or the website.

Negative anomalies

Users are most often worried about negative anomalies, because they are much more dangerous for the project as a whole and may lead to significant financial losses.

myTracker’s Report Builder enables users to test assumptions behind anomalies and build reports on relevant metrics to localize problems and engage the developer only when their causes become clear.

In our Guide to Analyzing User Engagement Drop, we will consider 8 user activity drop scenarios: 

  1. No, or a sharp drop in payments
  2. Unverified payments exceed verified ones
  3. Increase in interrupted sessions
  4. Differences in custom events between external/internal systems
  5. Different number of app installs in external/internal systems
  6. Different number of events in the partner’s system and myTracker
  7. Different Retention/Rolling Retention in external systems
  8. A sharp rise or drop in app installs

For each scenario, we will propose assumptions behind such a drop, as well as solutions from myTracker to confirm or reject each assumption.