As analysts, we need to work with complexity, while simplifying for end-users, yet avoid oversimplifying. Naturally, this is easier said that done …
Simplifying for others: This is incredibly important. If you can’t explain the problem and findings to someone in 25 seconds or less, you 1) will likely lose their attention, and 2) possibly don’t understand it well enough yourself to explain it yet. That’s our job. We work with the details and bring others in on the conclusions.
Oversimplifying: The balance required is to simplify the final conclusions without oversimplifying the problem, the data, or your analysis. The struggle, however, is that our brains are hard wired to simplify information.
Think about the amount of stimuli your brain receives every day. For example, you are crossing the street. I am outside. I see a long stretch of gray. That is a road. There is a red thing coming towards me. I hear a noise. The red thing is making the noise. That red thing is a car. Cars can hit me. It is going at 45mph. I am stationary. It will reach my location in 4 seconds. I will take 10 seconds to cross the street. I should not walk yet. And of course, I’m completely understating all that goes through our brains for even simple tasks. If our brains didn’t find a way to make sense of a high volume of inputs, we simply wouldn’t function.
Acknowledging Complexity: The challenge for analysts is to try to simplify the answer, without oversimplifying the questions along the way. If you make erroneous assumptions because it (over)simplifies your analysis, you could end up drawing the wrong conclusions. You will probably make your analysis easier, but render it less valuable.
We need to acknowledge, work with (and enjoy) complexity. (And we had better get used to it, because the digital measurement space is not getting simpler.) However, we need to avoid oversimplifying more than is necessary to sift signal from noise. We need to question what we know, evaluate what we assume, and separate fact from opinion. And if in doubt, invite someone else question you or poke holes in your analysis. Chances are, they’ll spot something you didn’t.
I won’t lie. I’m not shy, I talk a lot, and I enjoy contributing to conversations and debates. However, sometimes (read: often) I need to give myself a slap upside the head to shut up and listen.
Why? I already know everything I’m going to say. There’s nothing I can talk about that I don’t already know. For the time my mouth is engaged, my brain learns nothing.
On the other hand, if I shut up and listen, I might learn something new.
eMetrics 2010 was a great experience. Lots of smart, interesting speakers, and great food for thought. Here are some of my favourite points, quotes, etc:
Jim Sterne: Social Media – Time to Rethink Your Marketing Metrics
“Yes, I like data, but I’m all about customer centricity.”
Purpose of social media is to:
Gain awareness
Change attitudes
Consider your influence
Check on your competition
Take action and
Generate value.
Larry Freed from ForeSee Results:Managing Forward: Moving from Measuring the Past to Managing the Future
Measure what matters most – your customers. Need to measure what constitutes success from their view, not yours.
The consumer is in charge. Because of the internet, it is easy to shop around amongst competitors. You no longer have to go from store to store but can check another store’s prices from a mobile device. Power therefore now rests with a cross-channel consumer.
Satisfaction drives conversion, loyalty, retention and word of mouth. Satisfy your customers and be fiscally responsible to survive and thrive in the years ahead.
Knowledge is power. Integration of metrics yields big dividends. Turn data into information, information into intelligence.
Social Media Metrics Framework Faceoff
Possible objectives of social:
Fostering a dialogue
Promoting advocacy
Facilitating support
Spurring innovation
Generating awareness
Driving revenue
Measuring social media is still new, and it is challenging to figure out the right KPIs. Sentiment analysis is not yet at a point where it can be automated, as sentiment comes from a reader’s reaction. (And note, people’s reactions are not all the same!)
Hardest thing about social is getting a VP/Director/etc to focus on why they want to be in social. What are the goals? Put channel last (e.g. don’t worry about whether to be on Facebook or Twitter) and think first about why you want to be in social at all.
PR and Analytics often using different tools to measure social media success, and often don’t mix or share information. However, ideally they should integrate and share knowledge.
Joe Megibow, Expedia
“I like to build stuff and blow sh*t up”
Expedia achieved success via data integration and uniting teams. Originally had Tealeaf analysts, datawarehouse analysis, BI analysts (etc) in different countries, with no global standards of definition and measurement. There were multiple (conflicting) data systems, and while decisions were made by numbers, accuracy is a question, and they weren’t being made on facts.
[No wonder the web analytics industry doesn’t yet have standard definitions and measurement. Expedia didn’t even have them within one company!]
Now a united team, with consistent definitions and integrated data systems.
Joe’s six lessions:
If you’re not working with your peers, you’re competing with them. Make sure people aren’t coming to different people for the same questions and having them compete against each other!
Learn from finance. CFO has “the truth”, and seasoned analysts who work with it. Find your source of truth.
Don’t just count, DO! Go from data collection to data action. Analysts can actually push projects through!
Sign up for results. Are you willing to bet your job, your team?
Manage expectations. If something will take a month, be clear about that. Business leaders just want plans and forecasts they can count on.
Start small, and communicate, communicate, communicate. Find little business wins, and proactively deliver. Earn the right to do more.
The moral of the story? Get stuff done. Take ownership, get results, get more ownership.
Vendor two-minute presentations
I think this is all that really has to be shared!
Adam Greco, Salesforce.com
Web data and CRM data is like peanut butter and chocolate – better together. Web analytics data is good. Integrated with other systems is what drives real value.
Add offline success metrics to online variables.
Segment web analytics data by CRM fields
Target web promotions using web and CRM data.
PS. Awesome success via integrating golf handicap from CRM system into ad promotion on site!
Add easy to understand value of a website visitor via scoring. (E.g. reaching Page X is worth 1 point, Page Y is 2 points, etc.) Every visitor therefore has a score/value of their visit.
To justify your analytics budget, you need to show them the money. Tie your success to revenue. It’s what CEOs understand.
Stephane Hamel: Measuring Your Organisation’s Web Analytics Maturity
Only way to success: Creativity in continuous improvement and attention to details. Generate innovative ideas and manifest them through to reality. Requires original thinking and producing. Analytics should use creativity. Analytics without creativity is just theory and pure mathematics.
Analytics = how a business arrives at an optimal and realistic decision, based on existing data.
Need people, process and technology.
Good thoughts from the session:
No data is enough if someone doesn’t want to believe. (Seth Godin)
Creativity is always constrained in some way. (Stephane’s daughter!)
Constraints make us push our limits (Peter Gabriel)
If data blocks your creativity, either your idea sucks, or you’re not being creative enough. (Jim Sterne)
Anyone can make the simple complicated. The challenge is to make the hard stuff easy. Analysis requires breaking a complex topic into smaller parts, to gain a better understanding of it.
Web analytics may be new, but we can (and should) learn from existing disciplines, rather than recreate the wheel.
The maturity model: Essential elements of effective processes which describes an evolutionary improvement from ad hoc, immature processes to disciplined, mature processes with improved quality and effectiveness.
Example:
Defined by level of maturity on a variety of axes. Aim is to develop but also remain even (e.g. no sense being mature in technology and having poorly developed analyst resources.) Growing one level of maturity is equal to one year of development.
Take homes:
Get down to earth
Fix your issues
Gain experience
Maintain balance (no point being well developed in one area, and low in another.)
Bob Page, eBay
eBay has an executive team that believes you can optimise the business with data. (And FYI, eBay has a ridiculous amount of data.)
They operate via distributed teams, organised datawarehouses, virtual data marts, but guided by common “North Star” metrics – those that are most important to the business.
Have developed an analytics community – like a Facebook for their many analysts. Allows information sharing, knowledge, and building relationships.
Structure of eBay analytics:
Dual hub & spoke model
Centralised technical team under the CTO
Centralised business analysts under the CFO
Distributed product analysts
The validation of groups’ findings helps keep a “separation of church and state” – keeps the businesses and teams honest when held accountable to another team.
Web analysts need to speak with a common vocabulary to and with finance.
On testing:
Exploration and testing are core pillars of an analytics driven organisation.
If all of your tests succeed, you’re not pushing hard enough. You need to do silly things, and fail.
Ensighten Tag Management
Interesting new kid on the block regarding tag management. “Just because we’ve always done it that way, doesn’t mean it’s the right way.”
KPIs are your “Oh, sh*t!” metrics. If it doesn’t matter if they move up or down, that’s not a KPI. If they never change, that’s also not a KPI.
KPIs must also be actionable.
To make KPIs meaningful, they should be tied to people’s bonuses. Those are key for people.
There can (and should) be layers of KPIs: executives will have just a top-line few, middle will have more, analysts will have a lot. Being closer to the product and daily decisions will mean you should be looking at more detail.
Take the lead as an analyst in defining KPIs, and get buy-in. (If you ask for input, you may end up with 10,000 “Key” Performance Indicators.)
Agree on an expiration/review date, to make sure they get revisited from time to time.
Some think of KPIs as your dashboard, but John Lovett proposes thinking of them as your low fuel light, because you have to take action.
High level KPIs are great, but you must segment to make sure that they’re not hiding a lower-level trend. (E.g. One segment up, one down.)
If you’re not sure if a KPI is useful, or if anyone is looking at it? Go “metric radio silent” (or use fake data!!) to see if anyone notices!
Analysts, take the initiative, but collaborate. And don’t groan when a change in KPIs is needed …
Global Analytics at IBM
Analytics managers: Set a clear goal, give the team the right resources, and watch them soar. (Margaret Escobar from IBM)
IBM global web analytics team is a centralised and paid internal service. Internal clients contract with analytics at the beginning of the year as to the number of resources needed, the skill level, etc.
IBM focuses on:
People (need to communicate, especially in a global team)
Process (trying to establish “reusable assets” aka templates, as well as sharing methodology and focusing on documentation)
Tools
Trying to balance:
Consistency, with customisation
Demand, with overloading the team
Learning and exploring, but still getting the work done.
IBM product management: Need to embed metrics and analytics into your program develop, and decide what qualifies as success before you start.
Other notable
“I received the oddest compliment today: ‘You make analytics and fun no longer seem mutually exclusive.’ But later (different person): ‘I’m not sure you do anything smart …’ ” – Lee Isensee
“Analysts take a complex topic, then breaks it up to gain a better understanding. Don’t throw data at people. Tell the story.” – Michelle Rutan
Question: “Is there a tool that will integrate data from various sources?” Answer: “You’re the tool.”
“Counting is not analytics. Seems obvious, but what did you do last week?” – @sutterbomb
“Analysts need to get better at the why, not the what.” Pat LaPointe, Michael Dunn session.
I’m headed home from eMetrics (in fact, I’m writing this from cruising altitude. Love in-flight wi-fi!) It was my first time at an eMetrics event, and I have to say – it not only met but exceeded my hopes and expectations. (And for the record, I was very excited to go, so my expectations were set high!)
I’m going to write a more in-depth post at a later date, but wanted to share a few thoughts. If these don’t suffice, feel free to check out the Twitter stream on the conference, which I will fully admit I dominated.
However, I wanted to share a few thoughts and favourite quotes of the day.
“I like to build stuff, and blow sh*t up.” (Joe Megibow, Expedia)
“KPIs are your ‘Oh, sh*t!’ metrics.” (Angie Brown) [Apparently it was word of the day …]
Kill unnecessary reports by trying to go “metrics radio silent” and see who notices (Lee Isensee, Unica)
The highlights for me were Joe Megibow, Stephane Hamel and Bob Page (with Adam Greco rounding out the list) and definitely the networking, conversations, and smart people I got to spend three days with. I feel very fortunate to have found such an amazing industry to be a part of, and can’t wait to get more involved via the Analysis Exchange, Web Analytics Association, and hopefully more conference presentations!
I think most conferences end up having a focus. This definitely seems the year of social and the holy grail of multi-channel analytics and complete view of the customer. (Note: we’re now getting demanding. It’s not enough to tie all online. We need traditional and POS and phone data – oh my!)
I also presented about analytics at Kelley Blue Book (the presentation is available here) which was a great first presentation experience. Someone very silly put me in the biiiiig ballroom, but happily, people showed up, and even had questions later!
I’ll share more once I have a chance to compile notes and tweets.
eMetrics DC 2010: “Marketing Metrics: The Publisher Perspective”
When your business model is advertising, your focus is different. Michele talks about the tools, techniques and unique KPI’s at Kelley Blue Book in this session about audience measurement, web analytics and data integration to forecast site traffic, on-site advertising inventory and revenue. Michele goes deep about statistical rigor, automated reporting and managing an internal advertising data warehouse.
I am an analyst and a certified Les Mills group fitness instructor for BodyPump (weight training), RPM (indoor cycling), BodyCombat (mixed martial arts based group fitness) and BodyJam (dance based group fitness.)
While analyst and group fitness instructor seem very different, there’s actually a lot that analysts can learn from instructors.
When we are trained as instructors, we spend a lot of time thinking about how different people learn, and how to teach to all of them.
Visual learners need to see it to understand. In group fitness, these participants need you to demonstrate a move, not explain it. In analytics, this may mean visually displaying data, using diagrams, graphs and flow charts instead of data tables – and perhaps even hitting up the whiteboard from time to time.
Auditory learners need to hear it. In group fitness, they rely on verbal cues from the instructor. In analytics, you may have a thousand beautiful visual displays or PowerPoint slides, but it’s your commentary and explanation that will help these people understand.
Kinesthetic learners need to feel it to understand, to experience what you’re talking about. In group fitness, you can show them and tell them, but what they need is to feel the difference between “the right way” and “the wrong way” (for example, “Oh, now I can feel how muscle x engages when I turn my heel!”) This is the same group that tend to need repetition to perfect what they’re doing. In analytics, these are often the people that need to be led through your logic. It’s not enough to show them your findings, and to display the final results. They need to see the steps along the way that you used to answer your questions.
Now here’s where it gets trickier. When you are presenting to a group, they won’t all be the same type of learner. Which means that a good group fitness instructor and a good analyst needs to explain the same thing in different ways to ensure that everyone understands. For an analyst, this may mean using visual displays of information on your slides, talking through the explanation, and giving a step-by-step example to put everyone on the same page.
Keep in mind that you too have your own learning style. Your analysis and presentation style will likely match your learning style. (If you are a visual learner, a visual presentation will come easy to you.) It may take a more conscious effort to make sure you incorporate the learning styles you do not share. However, by tailoring your message to ensure you hit all learning styles, you stand the best chance of getting everyone to the same understanding.
Guru. Ninja. Rockstar. Expert. These descriptions are all over the place. (*cough* Twitter bios *cough cough*)
Done learning. This is what I hear.
Here’s the deal. Calling yourself an expert sounds like you think you’ve got nothing left to learn. How can you be an expert in web analytics, or social media? These fields have been around for all of about forty-five seconds. (And they’ve changed twenty-seven times since then!)
My $0.015: Don’t ever call yourself an expert, a guru, a rockstar. (And don’t just replace it with samurai or swami. You get my point.) Someone else may call you that, but let’s be honest, even then you should shrug it off.
The most appealing trait is a desire to learn, improve, to continue honing your skills. Focus on that. Let your work and development prove yourself. Not a self-appointed noun.
As a self-confessed geek, when I hear about a new tool and there’s a free option I can play with, I naturally implement it right away. After hearing the Beyond Web Analytics podcast discussed ClickTale, I took the opportunity to implement it on my blog for a little “new analytics tool fun”.
After a month playing around with this tool, here are some of my impressions:
Unique value
It is not merely repeating with what your Google Analytics, Omniture, WebTrends, etc tracking tells you. There is a definitely unique value. Think of the standard web analytics tools as telling you visitor’s behaviour between pages (e.g. visitors go from page X to page Y, exit at page Z.) ClickTale tells you what they do within a page (how far down do they scroll? Do they start filling in your form? What do they click? Where does their mouse move?)
Recordings of a site visit
One of the original offerings of the ClickTale was simply recordings of your visitors’ behaviour on your site. You literally watch users scroll up and down, click a link, go to another page, etc. It’s like sitting over their shoulder, or user testing without the option to ask them why they did something. However, no one can possibly watch all the videos, especially on a large site. The real value of the tool is what ClickTale added next: the aggregation of all of those videos into heat maps.
Aggregated heat maps
You can view an aggregated heat map of:
Mouse moves
Clicks
Scroll reach
Attention (via mouse attention)
The scroll reach is pretty interesting, especially on a blog, since normally the main page is a six-mile-long history of previous blog posts, and it’s interesting to see how far down people scroll.
ClickTale uses mouse over attention to estimate eye tracking (due to the correlation between the two) but are pretty clear that they don’t intend this to be a perfect replacement of eye tracking, merely an affordable way to get close to that kind of information.
Example heat maps:
So as you’ll see, the value of the heat maps is:
Not having to troll through multiple videos for insights. That’s just not possible with a site with millions of visitors.
In-page information that definitely complements what is provided by your standard web analytics tool.
For a blog specifically, the heatmaps can also be a way to see what of your content people are reading. If a visitor clicks to read a specific post, obviously you know they took this action even in a standard web analytics tool. However, where they read a blog post on the main page, ClickTale can fill in the gaps of what they read via scrolling and attention. This is a great insight missing from a standard tool.
A concern for frequently updated sites (such as blogs) might be the impacts of a site changing throughout the day, via adding new posts. Never fear, that has been taken care of: you can choose which version of a page you want to view the heatmaps for, if there are multiple versions throughout a timeframe.
Some other nifty features:
Form analytics
I only saw a demo of this, as my site isn’t really form-heavy, but to be honest, this thing rocks. Who starts filling in a field then stops? Who has to refill in a form? How much time do they take to fill in each field? What is form engagement vs form submission? This information is much richer than a “X number saw the page that had the form on it, then Y% saw the thank you page.” It’s pretty awesome. Check out this demo of it on ClickTale’s site: http://www.clicktale.com/product/form_analytics
Page and Site Analytics
ClickTale will also tell you which of your pages are the:
Most engaging
Most clicked
Most errored
Least scrolling
Slowest loading
The engagement time is pretty sweet also. In a world of tabbed browsing, a visitors may come to your site, read your post, but not close the page. (“Tick, tick, tick” goes Google Analytics time on site.) I myself tend to have multiple browser tabs open with links I’ve clicked from Twitter. ClickTale measures the time they actually spend engaged with the page, via mouse moves etc. It’s a richer metric than time on site.
Conclusion
I’ve only mentioned some of the functionality of ClickTale that I enjoyed. There are also options to search, find and watch videos matching certain criteria (e.g. videos from visitors coming in through search, or seeing a certain page – great for watching playback of site errors.) There is also an option for Omniture integration, which I didn’t try (as my blog doesn’t use Omniture) but is nice knowing it’s there for enterprise use.
All in all, my conclusion: ClickTale doesn’t replace a standard web analytics tool (nor do they claim it does, or should) but it’s a great supplement to give you more in-depth information about what people do on a page. I believe a clear competitor for this product is TeaLeaf, which I have seen a demo of, but not been in a position to use. The main thing that sways me towards ClickTale, even on an enterprise basis, is the price tag. TeaLeaf definitely seems a more costly solution. Now, it’s completely possible that TeaLeaf’s price tag is justified by the functionality; I haven’t used it so I can’t speak to the differences. But unfortunately, the reason I haven’t used TeaLeaf is because I couldn’t get past the price tag …
The best part?
Very easy to implement: Two lines of code (I could implement it. ‘Nuff said.) And seriously – all of this with only two lines of code. No special click tracking or form tracking. It’s as easy as implementing Google Analytics.
FREE! They have a free version that you can use on small volume sites. Plenty for us web analytics geeks to play around with! Now, there’s some functionality you miss out on with the free version, but it’s plenty to get you into the tool and allow you to evaluate whether the insights may be worth a small investment.
Disclaimer
I was fortunate enough to be given the opportunity to use ClickTale on my blog at an enterprise level access (thank you, ClickTale and Shmuli Goldberg!) Some of the features may only be available via paid subscriptions, and not in the free version, but the free version is definitely of value and worth checking out.
However, also keep in mind I used it on my blog, not on a larger corporate website. There may be some functionality that I’d like to have on a larger site that I didn’t notice was missing, just because of the size of the site I was looking at. I don’t claim this tool is the magic cure for any of your analytics ills, but it’s definitely worth looking at, to see if it might help answer some of the questions left open by your standard analytics tool.
Like many web analysts, I pretty much “fell” into this field. Coming from Law and Psychology degrees, it certainly wasn’t a clear path. At the time, I happened to be an assistant, and the web analytics team happened to need “temporary” help. Five years later, here I am: incredibly grateful to have fallen into a field that is fascinating, in high demand, and full of smart, engaged people.
Here’s a great example: the Analysis Exchange. This program brings together “students” of web analytics, mentors to support them, and non-profit organisations needing analytics assistance. (Note: I say “student” because you don’t need to be an actual student.) There are currently 1,000 members, and no open projects. As many needs as there are out there, the analytics community is rushing to fill them.
The fascinating thing about the Analysis Exchange is the skill level of some (probably most!) of the “students”. My first project (and currently only project – see above, no open projects!) went a little like this: Wow, my student is smart. I hope there’s even anything I can help him with! (He later said there was. I hope he wasn’t humouring me…!)
In fact, as Analysis Exchange mentor Jason Thompson commented on Twitter recently, many of the people we look up to in the analytics community are offering their services as students! When I myself first signed up, I was torn between student and mentor. Yes, I currently manage a team of analysts. However, would I be knowledgeable enough in a completely different setting and business model? Not to mention, any good analyst can always use more hands-on experience! So I won’t lie – going in as a student definitely crossed my mind. The basis for my eventual decision to be a mentor was that I wanted to leave the full-on student experience for those interested in web analytics, who may not have access to data or real life examples.
So here’s how our community works. You have a group of people who are in high demand, flat-out at their “real jobs”, with a real life outside of work. Despite this, they volunteer their time for 1) their own continued education and development and 2) to assist others in their growth. Most of these people never really think they’re ready to be a mentor, because no one considers themselves to be an “expert”. We think there’s always more to know, more to learn.
Web Analytics is developing fast. But with these people in our community, there is no worry that we won’t keep up. In fact, we’ll keep pushing, and moving our field even further forward.
PS. You know you’re addicted to Twitter when you have to consciously write “the web analytics community” rather than the “#measure community”.
