Ever heard the phrase "If you've got two watches, you never know what time it is."? I hadn't until last year at Shop.org (Vegas baby! VEGAS!). A guy who wandered into our booth used it to cleverly describe a problem I was blabbing about. I was explaining how I am often pulled into conversations with clients who are questioning the validity of the data we provide in our reporting.
"My <insert reporting package name here> numbers are different than your numbers!" is typically how the conversation starts. The rest of the conversation typically takes one of two paths….
Path #1: Apples and Oranges
Let's take clicks for example. Our original reporting goal was to do everything we could to make sure "Clicks" was representative of what the client would see if they logged into the respective marketing program's portal (Search, Comparison Shopping Engine, Email, etc.). Said another way, "Clicks" is defined on our side as the number of clicks the respective program is going to charge you for, assuming a CPC model.
I can assure you that "Clicks" within <insert reporting package name here> is not defined the same way. The definition is typically the total amount of inbound clicks that came from a specific lead source, typically tracked by a URL query argument. This number will include test clicks, fraudulent clicks, and other clicks the marketing program most likely will not ultimately charge for.
So, when I hear "Your click counts are too low in your reporting...", I smell Apples and Oranges. This situation is not limited to clicks. There are all kinds of metrics that have this issue when "comparing" two or more reporting systems.
Moral of the story?
1. Users shouldn’t assume "Clicks" = "Clicks".
2. Programs should do what they can to provide metrics matching all possible definitions.
On our side, we started to provide two different types of “click” values.
Raw Clicks: The number of leads that Channel Intelligence logged prior to any exclusions, such as click fraud rules or traffic from certain IP addresses. This number is designed to reflect the same number you would expect to see within any web site traffic reporting system you may employ.
Clicks: Clicks is intended to represent, as close as possible, the number of clicks you would expect to see within the respective marketing program's reporting portal. This number is achieved by filtering out raw clicks from sources known to also be filtered out of the respective marketing program's own reporting/billing, using actual clicks/cost data acquired from the marketing program directly, or a combination of both. Data acquired from the respective program will always supersede estimations made by Channel Intelligence. This level of combination is referred to as Blended Data within the reporting system.
This has helped tremendously. Clients can now line up their package’s version of "Clicks" to our "Raw Clicks" since they are designed to be the same thing. With those numbers being close to dead on (which in the reporting world means "equal"), this establishes the confidence necessary to not only get behind, but get excited about our version of "Clicks", which their package typically doesn’t have.
Path #2: Blinders are for Horses
Another common issue is that the sales data we associate with specific marketing programs is higher than <insert reporting package name here>. "My system says 129 orders and yours says 152" is a common concern I used to hear a lot.
Welcome to the "Blinders" problem.
What is occurring in this situation is that their internal reporting package is monitoring not only the CI managed programs, but every other marketing program driving traffic and sales. If CI is only watching traffic from the CI managed programs, the lack of peripheral vision results in the association of an order with a CI managed program when in actuality, a separate non-CI program led the purchaser back to the site in a subsequent click resulting in the sale.
What I've just described is a common problem with every marketing program out there that is only able (or willing) to watch their own program(s) for reporting purposes.
Moral of the Story?
The only way to accurately report on any ONE thing is to watch EVERYTHING.
So, a while back we started monitoring all marketing programs for our clients even though we only report numbers on the specific programs we manage. Cool thing is this has evolved into much more than we originally thought. This level of data collection allows us to not only provide accurate metrics for the programs we provide, but show the relationship these programs have with other marketing programs, CI managed or otherwise. Which programs are often working together to close sales? Which programs are influencing sales, but not getting credit because of a last in wins setup. And on and on. Pretty cool stuff, especially when put into the context of specific products, categories, keywords, etc.
So, in conclusion, you may be asking yourself “Why have multiple watches at all Dave? One is all I need right?”.
Great question. The way I have always answered this is that no one reporting package provides everything to everyone. Some are wonderfully flexible yet barely usable. Some are easy to use but limited in scope. I could go on. My point is that with more and more service providers offering analytics that are typically very specific to the service provided, it won’t be uncommon for a clients to have many “watches” to look at.
My advice is to make sure the minute hands are not being compared to hour or second hands, each watch tracks all 24 hours of the day, and every now and then synch them up. ;)
