Settling for Packaged Metrics and Reports Can Be a Dangerous Game

If you were a kid once, and I suppose we all were, you may have learned that no single manufacturer of bikes got every part right. So, in order to improve the quality of the finished unit, you have to purchase and install superior components which lighten the bike, make it operate more efficiently, offer comfort or convenience to the skillful rider. In the end, you might have upgrades and customization over 30-35% of your bike. The same idea applies across the board for just about everything.

So, why then, settle for the factory state of your web analytics tool?

Looking around for people to discuss many of their calculated metrics can be difficult. Joseph and Eric and their ongoing banter on “Engagement” certainly merits its share of attention. This was the primary topic recently at the eMetrics Marketing Optimization Summit in San Francisco and a tremendous panel discussion between the aforementioned as well as Gary Angel from Semphonic and moderated by Robbin Steif from Lunametrics. Aside from the ubiquitous topic of Engagement, there is so little available on what to do to hammer stats together for better understanding. People seem to guard their special configurations very closely. A few resources, however, are making their way into the world.

Dustin Wallace is at least one resource whom appears to want to share. Dustin is, according to what information is available on his posts, a relatively new blogger working in analytics at Sun Microsystems. 2 of his 5 posts available discuss manipulating packaged metrics for a better understanding of performance indications. In these, he discusses a formula to help draw comparisons over particular time frames and goes in closer on Bounce Rate (he breaks these up into ‘Exit Rate’, ‘Soft Bounce’ and ‘Hard Bounce’ and lays out there craft and execution in Omniture SiteCatalyst).

Another guy, Vijay Bathula, is doing analytics for Hewlett-Packard, publishing a blog discussing ‘advanced web metrics‘ and trying to develop some interesting concepts. A particularly attractive metric which Vijay brings to light is what he calls ‘Time2Click’. He states:

“Time2Click is the metric that tells the average amount of time that was taken in order to click a link on a web page. This metric helps web masters and marketers on how much time at an average, visitors are taking for making a decision to click a link or button on the web page. This metric is simple to calculate and great use to optimize the call-to-action buttons, positioning, anchor text in the links and much more without using any other expensive Split-Testing or multivariate testing tools.“

This promotes a clever angle for thinking about testing and a great sense of what I’m trying to relay by posting here. Formal metrics a great for the purpose of executive reporting, however, when being used by the analyst, especially one involved in heavily process-oriented practice, thinking outside of the box means going beyond what comes in the box.

An Invitation to a Discussion and Pending Project to Build Collaborative Set of eCommerce Calculated Metrics

My work recently, and in the context of this highly charged discussion of engagement, has focused on trying to build a series of powerful calculated metrics to try to get a better understanding of how people are interacting with eCommerce websites. It has brought me to the point where I’m willing to assert that, with regard to online retailers, a series of operations and statistics can be gathered and placed into major commercial reporting interfaces to ad value to the suite as well as promote a more complete functional model of interaction.

Among these I have decided to research, study, and pursue defining the following:

• Appraisal - This is the behavior of seeking information on product queue for a potential purchase. These are broad strokes in navigation based on general term and phrase usage, non-transactional focus and
• Acceleration - The point where the brain begins to move from a general information processing state to a more focused channel. During acceleration, a subject should (ideally) only move toward action and curtail further lateral navigation.
• Impulse - Having collected and been presented with one or several points where a call-to-action or option to execute an objective occurs, the confidence and slightly adrenal motivation carries a subject through an action and transition to a state of risk assessment.
• Commitment - Acting on the information and the excited state of having executed a checkpoint in a system of obligation; this measurement should seek to imply reduced regressive states when continuing through to additional actionable areas and streamlined return to transactional navigation due to acquired trust and familiarity.
• Conviction - The completion of the desired final action. Conviction should be measured by the degree to which a subject does or does not participate in building trust by gathering information as to policies, security, examining financial options, and other potentially pertinent information associated with going forward with a purchase.
• Affirmation - Presumed to be existing in both a natural and provoked state, affirmation is post-transactional and is the return to the site with an abbreviated or non-existent appraisal process. The degree to which the initial experience was positive and powerful should, hypothetically shorten time to accelerate and kindle the impulse state.

I have spent a fair amount of time looking into how these particular points work into the process of making a decision to invest money in return for a product with a perceived value. My resources in being able to do this, at this time, are limited. For that reason, I would love to hear from anyone who is willing to allow me or our team to look at site metrics and frequencies to identify these points in their business model and help build support for a larger, more universal application of these proposed transcendant metrics.

Anyone who is interested, please comment or write to me directly using the contact information provided. Any company who submits for the ability to aggregate and scruitinize data will receive a complementary copy of the publication as well as promise that any sensitive data used in the studies will be protected by the appropriate legal instrument.

This post is something akin to a rant. It is about the web analytics process and the ability to make it more functionally efficient. Its something we can all relate to:

It crossed my mind recently that there is so many facets of the analytics programs at maturity that there must be some means to internally optimize the process. I’m not talking about software or the many hundreds of small start-up companies or analysts throwing their hats in the ring, or any feature which they can add to our decision machines. I’m talking about grassroots analysis to grow business success through decision empowerment and process efficiency.

Quite simply, the steps to getting a data process start with the willingness to accept certain truths about the processes:

Knowing the objective of a website, and how that goal ties into the business model is essential in understanding where to plant an water web analytics. The connection between these can be elusive. Often, this is an aspect which comes to late in implementation or is overshadowed by a perceived need to begin producing reports. Unfortunately, this is good for the ego of the champion for analytics, but a detriment to properly installing functional analysis at the core of the human resource ecosystem.

Fueling a websites design and development machine requires making powerful assertions which can win support alone, or banish dissent with pure statistical science. Coming up with ideas is easy. Coming up with great ideas can be challenging. Coming up with great ideas and getting administration, colleagues, and the nay-sayer to go along with it can be impossible. There is a reason for this. It has to do with appeals. Appeals are the way in which information is communicated to reach the part of the receivers brain which helps them build a concept for themselves and the group. Everyone has a way they WANT to hear or see your information presented and how it affects their world. In that respect, you can either be a communications genius, or, build the case for math…which appeals to everyone. (I will write an entire post on appeals and how to use them some other time. This is a fascinating area which I think has very high relevance to analytics) Math is the key to making the process sprout. It is the foundation, the proof, and the method of delivery.

If making math work for the business routine of web analytics is the goal, we should break down the subroutine. This seems to be the main point of struggle for recent practice adopters and parties interested in participating in analytics. Each component is arguably a necessary step to producing useful reports which detail insights and suggest action, based on math. Here is HOW, piece by piece, you create action from analytics.

Just looking at reports day to day, any analyst should start to recognize patterns. The human mind is amazing with regard to this. We see lines and columns with comparable trends and past performance which either fit or do not fit. Here, you begin to ask questions. Remember this part, because this is the first major function of any analytical process. You find things that intrigue you, and seek to explain them by using the data.

With continued observation and investigation, you might begin to see correlations. Some might be as simple as every weekday visits decrease consistently by 60% between 12-2pm EST. You might be in the business of selling medical supplies. Where you might already know it is a lunch related subsiding, why does this happen? Why is it that consistent? What do these people DO during that TIME which prevents them from visiting and buying between 12-2pm?

If you’ve ever worked with a pharmacist or in a doctor’s office, or a hospital, you’ve likely experienced a fairly standard phenomena of their lunch habits. For 2 hours everyday, administrative work in the medical world is reduced to a skeleton crew. This impacts the amount of work which can be done outside of necessities, which, sadly, does not include purchasing catheters or bidets from BioRelief.com. What this does, though, is provides a certain correlation for, or explanation to the dip.

High value correlations should be based on the number of times when the desired action occurs, the consistent recurrence of that action, and how many different places this same action can be observed. These collected actions are not, in themselves, the goal, but rather, checkpoints which may or may not positively indicate some motive to achieve, or explain diversion from, the goal. The correlations give us an idea of how to cater our information to and find more avenues to improving the experience for the user.

Correlations lead to understanding and understanding should lead to hypothetical assertions. The place for making bold statements is in this stage. Take the pieces of correlated information yielded from curiosities and chasing rabbits and start to use them in building profiles or saying things like, to use the example above: “Shutting down our Search Marketing campaigns between 12-2 pm daily will save a total of $95 in fruitless clicks per day, which, when calculated is an estimated $25,000 in savings per year.”

Assertions should be elaborated on; then tested for validation and range. The world of online testing does not exist to give us something to talk about. The discussion is, instead, the result of seeing value in testing. A/B testing, using a control group versus variables, can be useful in all aspects of campaign or site optimization. Multivariate testing is an extremely efficient component for providing statistical validation of elements, as well as uncovering additional unintended correlations for further discussion. (Element design, additional perspectives, and other assertions from the design and development owners should help provide resources and add value to multivariate testing). Usability testing can place a single session under such scrutiny as to not only validate an assertion, but provide an immediate solution. (and, by the way, another great way to get people involved in the process).

Test results need to be scrutinized by analysts AND potential dissenters prior to reporting. Have colleagues and co-workers pick apart the outcomes. Get more input. This has the amazing effect of not only involving common adversaries in the project, but giving them an sense of where you are coming from. It helps them understand you. In doing to, you break down uncertainty and increase communication. (also a subject which I discussed at this venue). The business will begin to build around research based decisions.

By the end of this round of web analysis, you should have answers to report to the group, participation from people whom have been involved in the process. and the data and statistics to back up your position on any arguments you encounter. This will win favor, promote action, place value or at least validity behind statements, and, hopefully, spawn more questions, deeper analysis and better understanding.