One thing I have found can work very successfully is a hybrid team of web analysts and statisticians. When you combine the business and website knowledge that the analyst has with the “mad stats skills” that the statistician brings, you can create some truly powerful work.

There are a lot of different things that a web analytics team can leverage a statistician’s help for. This is by no means an exhaustive list, merely a place to get started.

1. Significance Testing

So you’ve run an A/B or Multivariate test. While your testing tool will likely also advise of you the statistical significance of your results, a statistician can dive deeper into this, and help you to measure significance outside of your tool. Perhaps you noticed shifts in site areas that weren’t one of your test success measures – a statistician can help you decide if these are merely interesting, or statistically significant.

Or perhaps you’ve tested in more of a time-series fashion. A statistician can try to tease out whether the change had an impact, or whether changes are due to seasonality. (This relates closely to the idea of an Impact Analysis.)

2. Impact Analysis

You make a site change, and you notice an increase in visits to a site area, or some key metric. You’re tempted to attribute this entire shift to the site change. (“Woo hoo! We’re up 5%!”) However, what about changes in marketing spend? Seasonality of your site traffic? Social initiatives? Are you taking those into account before reaching your conclusion?

A statistician’s analysis can attempt to tease out those additional variables to estimate the impact of the actual site change, vs. these confounding variables.

This same approach can be used to measure the impact of industry events or company changes (outside of the website) – anything, really. The benefit here is a better understand of the actual impact of events or initiatives, but a nice perk should be presenting your findings to the business and not having to freeze like a deer in headlights if someone says, “Yes but we spent another million dollars in paid search last week – did you factor that in?”

3. Standard reporting automation

Statisticians can use tools such as SAS to fetch data from FTP, combine and compute it, and deliver outputs to your system of choice (for example, Excel, if that’s somewhere you’re comfortable working.) This can allow you to schedule FTP delivery of SiteCatalyst reports, Discover reports, ad server reports (etc) – basically data from multiple sources – have SAS do the work of fetching multiple data sets, combine them and output to Excel.

That, however, doesn’t mean you need to deliver a huge scary data sheet to the business. On top of the data, you can build  a more user-friendly view (preferably formula-driven, so that you’re not manually updating!) in Excel to present the data.

This allows you to take a lot of the manual part (copy-paste, copy-paste) of standard reporting out the equation, and focus your time on explaining the shifts you might be seeing in the report. e.g. Perhaps traffic to a specific content area is down – start digging in. What traffic sources are driving it? Are there particular pages experiencing a more dramatic shift?

In addition, once the business sees the value of this work (the time it frees up for analysts to actually analyse!) it may actually help  argue for further automation and investment in further tools. So make sure you provide those insights, and use this work to prove why you shouldn’t spend your time copy-pasting.

4. Forecasting

Statisticians can build forecasting models to predict your site traffic, sales, ad impression volume – pretty much anything. You can go short-range, or long-range. Perhaps a simple “forecast through end of month” will suffice to start, or maybe you want to start forecasting three or six or twelve months in advance.

So why would you do this? Well, good analysts know that data needs context. That’s why we have KPIs, or compare month over month, year over year – to understand whether “2.6%” is “good”. Comparing to a forecast can be another way to get context for your data. If you’re diverging from your forecast, you can start digging in to see why. This divergence might be good – perhaps you saw a better than expected responses to your marketing initiatives. But on the flipside, you might also need to frantically search for why you’re suddenly down -10% compared to forecast …

Even a through-end-of-month forecast can be helpful here. An EOM forecast will tell you where you’ll likely end the month, based on current performance – even though you’re only on day 9. This will allow you to course correct throughout the month, rather than waiting till end of month to realise you didn’t match your forecast.

If your business sets site goals, forecasts can be the first step. First, forecast where your business will be for the next twelve months without any major initiatives. Simply assume the status quo. Then, look at the initiatives you want to add on top of that, and assess how much of an impact they may have. Forecast + specific initiatives = your goal. A statistician can also help you look back over time at previous initiatives and analyse their impact, to make sure that you’re not overstating how big an impact something new may have. (How many times have you heard “This is a game changer!” and found it barely moved the needle?)

There are still things you need to keep in mind when forecasting, but even starting small can bring value to your business.

Group Hug!

Still, analysts and statisticians may sometimes face some hurdles. Analysts need to learn the language of statisticians, and statisticians need to either learn the business, or be guided  by the analysts. A statistician exploring data with no understanding of the business, the website, or what any of it means normally doesn’t reveal great insights. On the flip side, the analyst really needs to start learning and at least dabbling in the world of statistics, and be able to translate complex concepts for the business users you support.

However, a cohesive team that learns to work together and leverage each other’s strengths can do amazing things.

Don’t have access to a statistician? Students often need real-life data for school projects. Consider seeking one out! (Who knows – you might find yourself a great future employee.)