Monthly Archives: July 2010

OMMA Metrics #2: The full story

Posted on by
1

For those who were not as fortunate to attend OMMA Metrics in San Francisco, here are my key takeaways from each of the sessions. For those who did, I would love to hear yours.

Yes, it’s long. Feel free to skim what is of interest to you. There is no pop quiz!

Note: below are two fun, random facts that I enjoyed learning!

Evolving Analytics: Measuring and Analyzing the Digital Ecosystem at Lightspeed
Judah Phillips, Sr Director Global Site Analytics, Monster Worldwide

  • Everything in analytics is evolving: the skills needed, the size of teams within a company, the importance in an organisation and exposure to executive management, technology and tools and the scope of what we’re analysing (eg social, mobile, video, and tying traditional media back to the site.)
  • To compete on analytics requirements investment in people and technology.

Digital Measurement: A Retrospective and Predictions for the Future
Eric T. Peterson, Web Analytics Demystified

  • 50:50 rule: invest half your analytics budget on technology, half on people.
  • At scale, a centralised analytics group is all that ever works. Analytics should be in the center of marketing, operations, management, etc, but have “superusers” within each of the other departments. Decentralized analytics does not work.
  • We need to develop faith and trust from stakeholders by having more answers than questions.
  • Analysts are a service organisation. We have forgotten this! We need to serve to provide business intelligence: deliver incredible value to drive revenue; this will build trust.
  • We need to move beyond first generation tools and reporting and need to generate insights and recommendations.
  • Commit to create a measurable impact! “If you give me a testing tool, I will delivery a 5% lift in X.” At worst? If you fail, they’ll fire you and you’ll move on to another position (probably with a salary bump!)

Measuring Social Media: From Listening to Engagement to Value Generation
with Jascha Kaykas-Wolff (Involver), Anil Batra (POP), Jonathan Corbin (Wunderman), Taddy Hall (Meteor Solutions), Rand Schulman (Eightfold Logic and Schulman + Thorogood Group)

  • Think about 1) Reach, 2) Engagement, 3) Impact of social
  • To strategically enter into social, need to identify your objective, pick the appropriate channel (e.g. are the users you’re trying to reach on Facebook? Twitter? YouTube? etc), then find the right KPIs that take into account objective and channel.
  • Scalable measurement and monetisation is what is currently missing from social media.
  • Companies need to identify who the influences are, and who they influence.
  • Integrate social with other channels, and understand it in the context of all your marketing.
  • Don’t be afraid to do something different!
  • To measure success, ensure you take a baseline.

Analysing Across Multiple Channels: What Works and What Doesn’t for Multichannel Measurement
Akin Arikan (Unica), Roger Barnette (SearchIgnite), Casey Carey (Webtrends), Kevin Cavanaugh (Allant Group), Terry Cohen (Digitas), Andy Fisher (Starcom MediaVest Group)

  • Some channels are more involved with certain areas of the lifecycle. E.g. Mass media to attract attention, online to engage consumers and persuade, offline to grow and retain.
  • There are forty years of multi channel experience, but digital breaks all those rules. How do you mix it in?
  • Maturity of multiple channel measurement is mixed – some companies are doing a lot, but many are not, for a variety of reasons (e.g. silo nature of the organisation, perhaps using multiple agencies, etc.)
  • Financial services is ahead of the multi channel game, because they have statisticians, data, tools, etc.
  • Traditional media measurement has 30-40 years experience. In comparison, the techniques in digital are laughable. Digital needs to learn from this. However, in the digital space we embrace change, are fearless, and figure out how to benefit from the change. Need to combine these two: increase mathematical rigor in digital, and embrace change in traditional media. [Aka “Andy’s grumpy rant”]

A Measurement Manifesto
Josh Chasin (comScore)

  • Future of digital is not in selling clicks and click throughs.
  • Digital has a seeming ability to measure everything, but in some ways this hurts us. We’ll never be the most simple medium. The landscape is not simple, and it’s not getting simpler. However, we have opportunity for doing great and ground breaking things with metrics.
  • Strengths of digital: portable, affinity (consumers cluster around content of interest, and even create that content!)
  • Targetability makes audiences small. Affinity makes audiences relevant.
  • 20th century was the generation of the shouting brand. 21st century will be the listening brand.
  • Digital order of operations: Ready, aim, fire, measure, aim, fire, measure, aim, fire, measure …
  • We need to measure: audience size, ad effectiveness (across platforms), attribution, engagement, voice of the customer and brand robustness.

Engagement the Mobile Experience: Effective Mobile Measurement Strategies
Raj Aggarwal (Localytics) , June Dershewitz (Semphonic), Joy Liuzzo (InsightExpress), Evan Neufeld (Ground Truth), Virgil Waters (Acceleration), Jamie Wells (Microsoft Mobile Advertising)

  • Mobile often has different methods of data collection, as the common javascript tags may not work.
  • Some mobile metrics are the same as site analytics, but some are different. Mobile web is similar to desktop web, but applications can be unique.
  • Benefits of mobile analytics: you can have a more accurate reach metric, since there is a device ID, and people rarely share devices. Location is more granular and valuable.
  • Mobile is unstable right now – we are trying to figure out mobile analytics in a shifting environment. E.g. What will succeed: mobile web or apps? Will one succeed the other? Will there even remain a distinction between them?
  • Tools for mobile analytics: 1) Traditional web analytics tools and 2) Niche vendors (or a combination of both.) The benefit of traditional tools is the integration with your site analytics. The benefit of niche tools may be higher-value, mobile-specific data.
  • Third party measurement and Apple: feeling from the panel is that Apple will be forced to play by the market, and likely change its policies over time.

Metrics and Measurement at eBay
Bob Page (eBay)

  • eBay has a huge range (and volume!) of data (e.g. marketing, finance, customer service, user behaviour, web analytics, etc.)
  • There is no silver bullet. No one product will solve all your needs.
  • They have a huge datawarehouse that contains virtual data marts for different groups (e.g. marketing vs. finance) rather than silos.
  • They also have an internal web analytics community, building a type of “Facebook for analysts”: an internal social network where analysts can subscribe to each other’s feeds, look at the latest videos, discuss issues in forums, share PPTs etc.
  • Have a centralised technical team under the CTO, who is responsible for infrastructure, support etc.
  • Centralised business analytics team under the CFO, responsible for common, standard “north star” metrics.
  • Distributed product analysts in each business.
  • Note: size of the technical teams to support this is similar to the size of the core analysts.

Managing Analytics: An Executive’s Perspective on What Works, What Doesn’t, Best Practices and Lessons Learned
Judah Phillips (Monster Worldwide), Matt Booher (Bridge Worldwide), Yaakov Kimfeld (MediaVest), Dylan Lewis (Intuit), Jodi McDermott (comScore), David L. Smith (Mediasmith)

  • Executive sponsorship of analytics is changing: “We used to have a megaphone, now we have a seat at the table.”
  • Centralisation enables standardisation, and helps with the evolution of analytics.
  • Where Analytics lives in the organisation: sometimes with the CFO, sometimes within marketing – differs within different companies. Analytics needs to own the technology and the data, though technical teams may actually implement.
  • Challenge: Lack of standards, lack of an organisational body.
  • Challenge: Executive distrust in the data or its validity. Jodi at comScore spoke of 4-6 months of having to explain data capture and constantly evangelizing before executives would place faith in data over gut.
  • Challenge: Hiring/recruiting. Companies want to find everything in one person: a technologist, a marketer, a statistician. Region can make hiring even more difficult. General sense is to find the right individuals/hire for instinct. You can always teach people the finer points of being an analyst (e.g. how to use a particular tool.)
  • Project management: Ticket-type system, scrum process. But no matter the project management used, requires ruthless prioritisation

Online Measurement: The Good, the Bad and the Complicated
Joe Laszlo (IAB)

  • The good: Online measurement is competitive, we have many vendors options. Vendors have integrity and are continually innovating. We can measure nearly anything.
  • The bad: Contradictory metrics from vendor to vendor, and changes in methodology can render dramatic fluctuations in measurement.
  • Online is managing to capture direct-response dollars, but not branding dollars. This is because brand marketers want to understand what their spend did for brand awareness, purchase intent, etc. What they get is “engagement”: view throughs, time spent, etc. There is a disconnect between what measures of success digital offers them and what they want.
  • Traditional media measurement allows calculation of reach and frequency. Also has years of experience of what matters, and has well-accepted metrics.
  • Lack of online measurement standards makes accurate data comparisons impossible. This can not be solved by any individual company, therefore the IAB is tackling through a cross-industry task force.

Modeling Attribution: Practitioner Perspectives on the Media Mix
Cesar Brea (Force Five Partners), Gary Angel (Semphonic), Jason Harper (Organic), Drew Lipner (InsightExpress), Manu Mathew (VisualIQ), Kelly Olson (Red Bricks Media.)

  • Attribution: What campaign/medium is responsible for the sale?
  • But there are more questions now: Is it better for someone to touch campaign a and b? What about b first then a? It’s not just the attribution, but do the two campaigns contribute together, in what order, or are two overkill, etc? E.g. Evidence that display with search adds value to search: someone searches after seeing a banner ad.
  • Can get a lot of benefit from evaluating click attribution, but even more from impressions optimisation.

Understanding the Multi-Screen Consumer: What’s on their Screens, What’s on their Minds
Alison Lange-Engel (Microsoft Advertising)

  • We now access online content via a variety of screens: computer, mobile device, TV, gaming consoles. We are always on and always connected.
  • Microsoft Advertising conducted survey to answer these questions.
  • The most active segment is 24-35 year olds.
  • Consumers are rapidly adopting technology and want control of the experiences.
  • Online gamers are the “game changers”. They do more of everything, all the time, social influencers. They spend the most time blogging, viewing, texting links. They view their game console as a communication device.
  • A linear funnel is not relevant anymore, as all screens impact purchase and allow an impactful story to be told.
  • Computers and smartphones are the key points of purchase.
  • The younger segments are accepting of advertising across multiple screens, actually want information and entertainment. They find ads helpful when they are targeted to their preferences and interests. They want a consistent experience across screens, and like the ability to access content across multiple screens – it actually improves their opinion of the content provider.
  • The key to success is: Consistent messaging + connected to other mediums + relevant = engagement and results.
  • Full report at advertising.microsoft.com/multiscreen.

And fun facts for the day:

  • The birthplace of web analytics is Hawaii!
  • Web Analytics is still small. All the web analytics companies sold for less than DoubleClick!

For those who did not get to attend this event, I highly recommend checking it out next year. It was interesting, informative, with great choice of speakers and a nice mix of presentation vs. panel discussions. Learning has never been so fun!

OMMA Metrics #1: Fun with word clouds

Posted on by
1

I’m back from a fantastic experience at MediaPost’s OMMA Metrics, organised by Judah Phillips.

I am putting together a full write up on the takeaways from the amazing sessions, as well a review of the event as a whole (all good, I promise!) However, because I’m a nerd and wanted to take in as much info as possible, I took copious notes at each of the sessions (5,001 words in total over the one-day conference – and no, I’m not kidding) and put it together into a delightful word cloud using Wordle.

Enjoy, and stay tuned!

[Click to view larger image]

How to grow your web analytics skills (within your current role)

Posted on by
1

If you are an analyst looking to further develop your skills, what can you do (within your current role) to further grow and develop? Here are a few of my thoughts, though I am certain there are many others.

In no particular order …

1. Interact with others in the industry

  • Join Twitter, follow your web analytics peers. Twitter can be an amazing educational resource if you use it for something other than “I ate a ham sandwich today.” You get to hear about the challenges that analysts working with different business models or analytics tools face, what is going on in the industry, what the vendors are saying and perhaps new functionality they’re releasing.
  • But more importantly than reading what others say on Twitter: contribute. Voicing your views will force you to think them through. And everyone disagreeing with you (it will happen one day!) will be a great learning experience to see those other viewpoints.
  • Go to Web Analytics Wednesdays
  • Take the time to go to lunch/happy hour/etc with your peers within your company and “geek out”. While you may work in the same company, your responsibilities and experiences may still differ, and you can learn from the experiences, thoughts and views of others.

2. Take advantage of free learning opportunities

  • Attend free webinars. There are so many out there (you’ll find out about them through Twitter, blogs etc) and they can be a great resource
  • Attend free trainings (yes, they do exist. I can’t tell you how many emails I get from MicroStrategy about free one-day trainings.)

3. Attend conferences

  • This one can be tougher if your employer doesn’t support this. However, make an argument for why it is of benefit to the business. Trust me, the vendors give you plenty of information about how to sell their conference to your company!
  • If you can swing the cost, you do have the option to pay for it without your company’s support (or “financial assistance”) …!

4. Volunteer

  • Join the Analysis Exchange, a program that brings web analytics students, mentors and non-profit organisations together, to give more web analytics experience to the student and analytics assistance to the organisation.
  • Know a friend/family member/co-worker with their own site? Blog? Small business site? Volunteer your time to help them set up a free web analytics solution, and take time out of your schedule to analyse their site on a regular basis. Don’t know anyone? Why not start your own site? It doesn’t have to be big. It also doesn’t have to be about web analytics. But it will certainly give you a taste of analysing a different type of site, as well as some of the challenges of getting traffic!
  • Volunteer to work on things outside the scope of your standard role within your company. Is there a project out there that you think analytics could help with, but no one is asking for help? Volunteer it!

5. Read
6. Read
7. And then read some more

  • There are a lot of great books out there. Start with one. (A hint: If this sounds completely dull to you, and you can’t imagine anything worse than reading about analytics in your spare time, really take a look at whether you are in the right field …)
  • Read both corporate blogs (e.g. web analytics vendors: Omniture, Google Analytics, etc) and those of your peers
  • Ask your peers for their recommendations of books, blogs, journals, magazines, articles, etc
  • But don’t stop just at web analytics books. Start reading about related fields. Product development. Design. Usability. Marketing. Social media. Statistics. Even cognitive psychology!

8. Keep your eyes open to what employers are hiring for

  • Sure, maybe you’re happy where you are at your current company. Maybe you don’t feel you’ve extracted all the learnings you can from your current role. (That’s a great position to be in!) But keep your eyes open for what positions are out there.
  • Why? Seeing what employers want will allow you to keep a mental checklist of what skills you need to improve on, prior to your next promotion or job change. Better yet, think about what you want your next move to be, and monitor the companies that are hiring for that type of role. What are the requirements and responsibilities they have for it? This ensures you’re working towards filling those requirements in the future. You can’t grow into a position if you don’t even understand what it involves!

I would love to hear others’ thoughts on this. Please comment if you can think of any further advice.

What time of day do web analysts tweet?

Posted on by
4

Thanks to @menggoh (and Virgin America‘s wi-fi) I have been thoroughly entertained on a red eye between LA and Boston with Twitter analytics from The Archivist. Which means naturally, I was curious: based on when the web analytics community is active and tweeting (based on tweets to the #measure hashtag) when is a good time of day for me to post new blog posts?

The cool thing about The Archivist is that they give you tons of fancy charts about who the top users our, our daily tweet volume over the past few months, topics discussed etc. (Seriously, check it out – so much fun!) However, what I couldn’t find was a breakdown of total tweets by time of day. The even cooler thing about The Archivist is that they allow you to download an Excel file of the data. (Insert cry of geek joy here!) So naturally, I took the data and DIY’ed it.

So as it turns out, the peak of #measure tweets is between 6-7 pm. We’re primarily an evening/night owl community (insert HootSuite joke, anyone? Get it? Owl? Hoot? My apologies – I blame the red eye flight…) with much lower activity in the morning. Possibly we work on being a peaceful community by waiting for the coffee to set in before communicating too much!

Here is total tweet volume to the #measure hashtag for 5/11 through 7/2/10.
[Ideally, I’d like to overlap total tweet volume to see whether we follow the overall Twitter trends. I’ll update when I have further data.]

*Note: Time of day is based on United States EST

So what does this tell me? I theorize that I probably don’t want to post my tweets at 6pm, right when a huge volume of tweets are flooding in (too easy to be missed in the midst.) However, perhaps noon/1pm would be a good time, so my tweet is recently posted as we start getting more active on Twitter for the day.

Now, I’m off to test some time-of-day theories …

[This adventure in geekdom was proudly brought to you by The Archivist and Virgin America‘s wi-fi service.]

How often should you revisit your KPIs?

Posted on by
1

I have been thinking lately about the right intervals for revisiting and changing your site and business KPIs. I won’t bore you with all the back and forth, but merely share a few thoughts.

The stability of your business plays a role. If you are an established business, with established goals of your site, revisiting your KPIs too often suggests to me that you didn’t have the right ones in the first place. If, however, you are a newer, (somewhat) flying-by-the-seat-of-your-pants business, perhaps perfecting your approach, I can understand a much more fluid approach to what constitutes “success” and a need to more continuously evolve your KPIs.

My overall thoughts are that KPIs can’t and shouldn’t change every month, even for the latter business example. (Would it be unprofessional of me to say, “duh”?) You should be consistently measuring against the same yardstick.  However, I do think it’s good practice to take a look every three to six months and make sure your KPIs are 1) useful and 2) complete. Do you actually need all of them? Or, on the flip side, is there something new that should be included? Perhaps there are new capabilities you have developed that would allow a new KPI to be measured? After all, a year can allow for a lot of development in the analytics industry. Take advantage of new measurement options.

Recently, I have been involved in the re-evaluation of our KPIs. At the beginning of this effort, the website product team and the analytics team were involved in brainstorming new ways to evaluate the success of the site. Once we decided 1) what we should measure and 2) what we could measure, figured out the overlap of the two and selected from those, analytics began publishing the information. Now, a few months down the track, we’re at a point where our product managers are somewhat comfortable with the information, and the time has come to revisit. (After all, they can’t give feedback on something they don’t even understand or use yet. You have to give the information some time to allow for informed feedback.) Are they using the information? What’s missing? What’s overkill? I expect to do this every 3-12 months from here on out. Just like our site, I expect our measurement of it to be iteratively developed over time.

Parting hint: If you’re not sure if something is helpful or not, try removing it for a month. If no one complains, you have your answer. (But I never do this. Never. No, Really.)