In the next edition of our newsletter, GrokDotCom, John Quarto-vonTivadar and I co-wrote a piece discussing the comedy of errors that ensues when having your report jockey advise you on business decisions. Here's the article, hot off the press:
The Great Debate
or,
“When All You Have is a Reporter, Every Analysis Looks
like A Nail”
Analysis doesn’t happen in a vacuum. And as we’ve said a thousand times: you can torture
the numbers to confess, uh, I mean, rationalize just about anything.
Matt Belkin of Omniture blogged recently about the
differences between Visits and Unique Visitors, as they relate to measuring
reach and as they factor into the Conversion Rate formula. That is, Matt argues
that Conversions per Visit is more important than Conversions per Unique Visitor. On occasion, we see our own clients make this
sort of slip-up, so it’s worth examining the arguments on their merits.
Matt writes:
“[Measured] Visits...always.
Sure, call me crazy - but my logic is actually quite simple.”
Simple? A
finer description of this approach would be simplistic. Perhaps
that’s the root of the problem. Matt is clearly an intelligent guy. But from
a software vendor stand-point, it’s very easy to fall into the trap of seeing
the Reporting software itself as the end, rather than as a means to an end.
From a business owner’s perspective (and let’s face it, from a customer’s
perspective), what she really needs is Analysis. If she could, the business
owner would interview every one of those successful (and failed) customers to
find out what the company did right and did wrong. Each of those customers is a
Unique Visitor and it’s the totality of the unique visitor experience that
caused (or inhibited) a conversion.
Reporting is often times simple; Analysis is rarely so. Reporting
requires condensation into sound-bites; Analysis requires segmentation into
knowledge-bites. There’s rigor involved in real analysis, wherein the software
is the tool, not the talent.
Let’s look at Matt’s three reasons for preferring Visits
over Visitors. As we’ll see, his blanket
statements are superficially true – at least upon first glance, but that a
deeper understanding illustrates just how little reasoning supports them.
Reason #1: Visits are more accurate than Unique
Visitors.
Technically, this is correct, but only if your definition of
“accurate” is to simply “account for all data”. That’s like arguing we can
reduce the crime rate simply by making Theft or Assault a legal activity. Will
that make the citizenry safer from Theft or Assault? Or does it only remove
those factors from the measured crime statistic?
From a data confidence standpoint, measured Visits are more
likely to measure actual visits because measured visits are no more and no less
than the actual visits we could measure – hence their name. However, there’s
more to this story.
Measured Visits is no more “accurate” at measuring reach
than is Unique Visitors. In fact, they’re far less so. How can visits possibly tell us about Reach,
in anything other a relative sense? Reach is about people, not sessions. The value
of Visits in quantifying relative Reach is identical to that of Unique
Visitors: the two are both apples-to-apples comparisons.
In short, measured visits accounts more accurately for actual visits, but actually provides you, the business owner, with less answers to the questions you're actually asking. Why is tracking Visits of less value? Matt’s Reason #2
explains:
Reason #2: Every Visit represents an opportunity to
persuade or convert a visitor to a customer.
Yes, absolutely this is correct, but not without some
stipulations.
Do you understand the concepts of Macro vs. Micro
conversions? Have you planned your
scenarios accordingly, taking into account the three critical questions to
planning any persuasive system? Are
you willing to optimize your scenarios for each visitor segment, summing these
pipelines to have an Optimal Conversion Rate, rather than an Average Conversion
Rate?
In the world of averages, every session does not represent
an opportunity to persuade or convert a visitor to a customer. What every session does represent is
an opportunity to persuade a visitor to decide to take an action (a micro-conversion)
which brings her further into her buying process, and brings her closer
to the end goal; the macro-conversion we’re looking to optimize (i.e. our
revenue generating conversions).
Take for example, a visitor beginning their search for a new
car online. Does Cars.com truly have an
opportunity to persuade visitors to convert themselves during a single
session? Imagine a scenario where our
online car buyer visits once, researching various styles and brands. She comes back a few days later, narrows her
search to a few targeted makes and models, and downloads their fact
sheets. She returns the next day and
makes her purchase. That’s three visits in the course of a week, and in Matt’s
software-only approach, that’s a 33% conversion rate. How much time and effort should we spend trying
to guess ways to optimize the 33% CR we’d report if we used Measured Visits as
opposed to Unique Visitors?
The Conversion Rate in this scenario is the maximum it could
be -- there was one visitor, and she could only provide one sale. By focusing on Unique Visitors we keep our
focus on the thing that matters most: a single, unique,
prepared-to-part-with-money customer.
Matt continues with:
Reason #3: Measuring visits is based on fairly
established industry standards
True, and that’s exactly why it has less value to
you. Are you industry standard? “What do you want to be when you grow up,
Tommy? “ “Oh, just average.” Is that the
sort of business you’re running, to be just like everyone else? Or is your
business gunning to be an Astronaut or a Fireman, or an Olympic swimmer?
It’s disappointing to return back to the world of averages. I recall a time when Amazon was the “industry
standard” for online checkout processes. Many an e-tailer went belly up trying to copy their method, with similar
lack of regard for the notion that maybe, just maybe, Amazon only worries about
doing what’s right for Amazon.
And what industry is
setting these standards anyway? It’s the Software Vendor’s industry, not your
business’ industry, because their reporting tools report certain statistics
more easily than others Yes, they’re doing what’s right for them. And there’s
nothing wrong with that; we just have to recognize it as a limitation of the
reporting tools, and stop thinking that some magical black box can substitute
for rational planning and analysis.
Matt goes on to talk about how Unique Visitors are a
subjective measure, one the industry can’t agree on in terms of length. He says this as if it’s a bad thing.
Our clients often ask us for benchmarks so they can measure the
size of their, uh, foot. We categorically refuse. What’s a
benchmark? When your system for
conversion actively persuades 97% of your visitors to go elsewhere, benchmarks have
little value beyond making you feel like a winner. I’d suggest a better way to make you feel
like more of a winner: gobs and gobs of revenue.
The only benchmarks you should worry about are internal
benchmarks. Bottom Line, how much money am I making? Am I making all that I should, given the
resources I’m allocating to the process? Can I make more?
Matt closes with this gem:
“The longer your unique visitor
timeframe, the more you effectively overstate success”
This is the purest evidence yet that you do not want to
subscribe to these theories if your job, your company, or your family’s income
is on the line. Instead, knowledge of your customers, and more specifically,
their buying process is the real key. And, for the record, the buying process
is non-linear, and is not the same across products and companies. One cannot guarantee a buying decision can be
made in a single sitting, for every sale, across all the products on the face
of the planet, simply because the
software vendors only know how to measure one way.
We help define the complexity of our client's sales in 4-dimensions. Not all sales are equally as complex. There’s the issue of compacted and
non-compacted information, as it relates to the various handles of information
your customers grab onto, and their angle of approach. The length of time you define for your
Unique Visitors should be long enough to encapsulate their buying process, but
short enough to take into account when they fail to convert at the macro-level.
It’s tremendously more important that some simple Reporting software.
If that last paragraph found you lost in a sea of
vocabulary, a life-preserver in the
form of our new book Waiting for Your
Cat to Bark awaits you (but not before June 13th). You won’t be drowning for long. In the meantime, trust me, there’s nothing simple about truly analyzing
your visitor behavior… unless of course you start with the conclusion and work
backwards.
