In the next edition of our newsletter, GrokDotCom, John Quarto-vonTivadar and I co-wrote a piece discussing the comedy of errors that ensues when having your report jockey advise you on business decisions. Here's the article, hot off the press:
The Great Debate
“When All You Have is a Reporter, Every Analysis Looks like A Nail”
Analysis doesn’t happen in a vacuum. And as we’ve said a thousand times: you can torture the numbers to confess, uh, I mean, rationalize just about anything.
Matt Belkin of Omniture blogged recently about the differences between Visits and Unique Visitors, as they relate to measuring reach and as they factor into the Conversion Rate formula. That is, Matt argues that Conversions per Visit is more important than Conversions per Unique Visitor. On occasion, we see our own clients make this sort of slip-up, so it’s worth examining the arguments on their merits.
“[Measured] Visits...always. Sure, call me crazy - but my logic is actually quite simple.”
Simple? A finer description of this approach would be simplistic. Perhaps that’s the root of the problem. Matt is clearly an intelligent guy. But from a software vendor stand-point, it’s very easy to fall into the trap of seeing the Reporting software itself as the end, rather than as a means to an end. From a business owner’s perspective (and let’s face it, from a customer’s perspective), what she really needs is Analysis. If she could, the business owner would interview every one of those successful (and failed) customers to find out what the company did right and did wrong. Each of those customers is a Unique Visitor and it’s the totality of the unique visitor experience that caused (or inhibited) a conversion.
Reporting is often times simple; Analysis is rarely so. Reporting requires condensation into sound-bites; Analysis requires segmentation into knowledge-bites. There’s rigor involved in real analysis, wherein the software is the tool, not the talent.
Let’s look at Matt’s three reasons for preferring Visits over Visitors. As we’ll see, his blanket statements are superficially true – at least upon first glance, but that a deeper understanding illustrates just how little reasoning supports them.
Reason #1: Visits are more accurate than Unique Visitors.
Technically, this is correct, but only if your definition of
“accurate” is to simply “account for all data”. That’s like arguing we can
reduce the crime rate simply by making Theft or Assault a legal activity. Will
that make the citizenry safer from Theft or Assault? Or does it only remove
those factors from the measured crime statistic?
From a data confidence standpoint, measured Visits are more likely to measure actual visits because measured visits are no more and no less than the actual visits we could measure – hence their name. However, there’s more to this story.
Measured Visits is no more “accurate” at measuring reach than is Unique Visitors. In fact, they’re far less so. How can visits possibly tell us about Reach, in anything other a relative sense? Reach is about people, not sessions. The value of Visits in quantifying relative Reach is identical to that of Unique Visitors: the two are both apples-to-apples comparisons.
In short, measured visits accounts more accurately for actual visits, but actually provides you, the business owner, with less answers to the questions you're actually asking. Why is tracking Visits of less value? Matt’s Reason #2 explains:
Reason #2: Every Visit represents an opportunity to
persuade or convert a visitor to a customer.
Yes, absolutely this is correct, but not without some stipulations.
Do you understand the concepts of Macro vs. Micro
conversions? Have you planned your
scenarios accordingly, taking into account the three critical questions to
planning any persuasive system? Are
you willing to optimize your scenarios for each visitor segment, summing these
pipelines to have an Optimal Conversion Rate, rather than an Average Conversion
In the world of averages, every session does not represent an opportunity to persuade or convert a visitor to a customer. What every session does represent is an opportunity to persuade a visitor to decide to take an action (a micro-conversion) which brings her further into her buying process, and brings her closer to the end goal; the macro-conversion we’re looking to optimize (i.e. our revenue generating conversions).
Take for example, a visitor beginning their search for a new
car online. Does Cars.com truly have an
opportunity to persuade visitors to convert themselves during a single
session? Imagine a scenario where our
online car buyer visits once, researching various styles and brands. She comes back a few days later, narrows her
search to a few targeted makes and models, and downloads their fact
sheets. She returns the next day and
makes her purchase. That’s three visits in the course of a week, and in Matt’s
software-only approach, that’s a 33% conversion rate. How much time and effort should we spend trying
to guess ways to optimize the 33% CR we’d report if we used Measured Visits as
opposed to Unique Visitors?
The Conversion Rate in this scenario is the maximum it could
be -- there was one visitor, and she could only provide one sale. By focusing on Unique Visitors we keep our
focus on the thing that matters most: a single, unique,
Matt continues with:
Reason #3: Measuring visits is based on fairly
established industry standards
True, and that’s exactly why it has less value to you. Are you industry standard? “What do you want to be when you grow up, Tommy? “ “Oh, just average.” Is that the sort of business you’re running, to be just like everyone else? Or is your business gunning to be an Astronaut or a Fireman, or an Olympic swimmer?
It’s disappointing to return back to the world of averages. I recall a time when Amazon was the “industry standard” for online checkout processes. Many an e-tailer went belly up trying to copy their method, with similar lack of regard for the notion that maybe, just maybe, Amazon only worries about doing what’s right for Amazon.
And what industry is
setting these standards anyway? It’s the Software Vendor’s industry, not your
business’ industry, because their reporting tools report certain statistics
more easily than others Yes, they’re doing what’s right for them. And there’s
nothing wrong with that; we just have to recognize it as a limitation of the
reporting tools, and stop thinking that some magical black box can substitute
for rational planning and analysis.
Matt goes on to talk about how Unique Visitors are a subjective measure, one the industry can’t agree on in terms of length. He says this as if it’s a bad thing.
Our clients often ask us for benchmarks so they can measure the
size of their, uh, foot. We categorically refuse. What’s a
benchmark? When your system for
conversion actively persuades 97% of your visitors to go elsewhere, benchmarks have
little value beyond making you feel like a winner. I’d suggest a better way to make you feel
like more of a winner: gobs and gobs of revenue.
The only benchmarks you should worry about are internal benchmarks. Bottom Line, how much money am I making? Am I making all that I should, given the resources I’m allocating to the process? Can I make more?
Matt closes with this gem:
“The longer your unique visitor
timeframe, the more you effectively overstate success”
This is the purest evidence yet that you do not want to subscribe to these theories if your job, your company, or your family’s income is on the line. Instead, knowledge of your customers, and more specifically, their buying process is the real key. And, for the record, the buying process is non-linear, and is not the same across products and companies. One cannot guarantee a buying decision can be made in a single sitting, for every sale, across all the products on the face of the planet, simply because the software vendors only know how to measure one way.
We help define the complexity of our client's sales in 4-dimensions. Not all sales are equally as complex. There’s the issue of compacted and non-compacted information, as it relates to the various handles of information your customers grab onto, and their angle of approach. The length of time you define for your Unique Visitors should be long enough to encapsulate their buying process, but short enough to take into account when they fail to convert at the macro-level. It’s tremendously more important that some simple Reporting software.
If that last paragraph found you lost in a sea of vocabulary, a life-preserver in the form of our new book Waiting for Your Cat to Bark awaits you (but not before June 13th). You won’t be drowning for long. In the meantime, trust me, there’s nothing simple about truly analyzing your visitor behavior… unless of course you start with the conclusion and work backwards.