At Hipmob we use both Google Analytics and Mixpanel. GA for general traffic stats, and Mixpanel for deep(er) business and user analytics and messaging. Both were unwieldy at first, but we’ve invested in both (more in Mixpanel) and have a good sense of how to use them.
As a relative data noob, the way I use analytics goes: question pops in head > I search our data > find some answer > make some adjustment based on the answer > return a few weeks later to see the result. The main problem with this is that it’s tough to know what to track in the first place - oftentimes I find that I haven’t really been tracking some important thing.
To solve this, I’ve started reading several blogs - Segment.io, Kissmetrics, Hubspot are 3 examples. Sometimes this helps. More often than not though, the lesson learned is not generalize-able enough for me to assess how I can use it, or they’re stats are orders of magnitude bigger than ours, so our ability to test and come to analogous conclusions is far constrained. I chalk it up to learning lessons I’ll use in the future. Given that my answers are only as good as my questions, I’m in a bit of a bind as I suspect my questions aren’t that good.
At heart, my problem is not knowing what to ask in the first place. I suspect I’m not alone in this. We (all) get so much data that sometimes I see a problem or opportunity too late. To me, a problem or opportunity is any datapoint that varies significantly relative to the average variation in its dataset. For example, I’d like to know anytime a stat I track behaves in a surprising way. For example, if it increases more than 2 standard deviations from its weekly trailing average. Any behavior like this is worth paying attention to.
As your business grows and becomes more complex, the number of things you should watch grows much faster than your ability to pay attention to it. A system that tells you when there’s a surprise in your data would help you spot problems before they blew up, and take advantage of nascent opportunities.
I’m writing this in the hope that someone smarter or more experienced has encountered this problem - I’d love to know how you solved it. My proposal is twofold:
1.Weekly report - a weekly email displaying the top 10 movers across all my data. The biggest changes in terms of % from last week to this week, be they visitors, signups, keywords, conversions, anything that we measure.
Weekly Alert Email
2.Instant Alerts - an email sent anytime a datapoint breaches a preset threshold eg. 2 standard deviations the weekly trailing average. You’d be able to adjust the threshold and mute any particularly volatile stats but this would tell you anything that’s surprising even if it’s not something you initially knew was important.
To start, you’d want to watch your core analytics, but eventually you could apply this approach to everything in your business: changes in customer interaction, logs, etc. The main idea behind this is to have your data telling you when something is happening, rather than forcing you to look. Know something that does this already? Please email me at email@example.com
Hacker News Discussion Here
Edit: Found a partial answer to this from Stephane Hamel over at Google+
If you go to “Intelligence Events” on the left panel (3rd down from the top) you’ll be able to create alerts across your entire data stack, sending you messages when your stats breach particular thresholds. Here’s a screenshot