A Stupendously Misleading Blog Post

I woke up this morning and caught this tweet from Rand Fishkin:

The tweet links to a blog post titled HubSpot’s Stupendously Misleading Reporting. Huh? I use HubSpot’s dashboards every day and never once thought they were misleading. Curious.

Here’s the criticism from the post author:

In Hubspot’s “Visits” reporting, the overall trend looks very positive – slow solid stead growth. We call this “up and to the right” and its the trend we strive for with SEO engagements – more traffic yields more calls, yields more business.

Look a little closer – Hubspot’s up and to the right is cumulative. Each month they are simply adding the cumulative total of all the previous months – so mathematically it can display nothing but up and to the right.

To summarize: UP RIGHT GOOD! DOWN RIGHT BAD! Yeesh, I’d hate to break the news about bounce rate. 😂

OK, seriously though, the thing that bothered me most about the blog post was that the author is accusing HubSpot of being intentionally misleading.

Don’t trust your vendors to tell you how good your vendors are – because – surprise surprise, the are incented to self report bigger and better numbers – a freakshow self-interested Napeoleon complex designed to hide their own failings.

I think that’s entirely wrong in this particular case and I’m going to explain why.

First, the default view for this report is this month vs. last month, but the author doesn’t mention that because it destroys the argument entirely.

HubSpot Marketing Dashboard

Traffic so far this month vs. last month. Is this confusing anyone?

Second, it’s a waterfall chart. The report is explicitly designed to show you how you are pacing vs. the previous period.

Success isn’t indicated by the direction of the line. It’s indicated by whether the colored line is trending above the grey line. If that’s not clear enough, there’s even even a little red/green arrow to help you puzzle it out:

Red down arrow

It’d be really hard to see that arrow and retort “BUT THE LINE IS UP AND TO THE RIGHT!”

You can also track your visits/contacts growth rate against a stated goal. Kind of like tracking the mileage on your car against the mileage limit on your lease.

When you are benchmarking, it’s way more helpful to see the cumulative data and the slope of the line against your target, in my opinion.

Also bothersome is that it looks like the author might have removed the explainer text which makes it explicitly clear that this is a cumulative and comparative view. The screenshot in the post is missing this highlighted bit:

Lastly, the de-facto traffic report in HubSpot is the Sources report, which isn’t designed as a comparative report and will show the traffic ups and downs just as the author wants.

In fact, HubSpot has dozens of reports and analytics widgets. This is literally the only one that is designed by default to show you cumulative data. So it’s very hard to argue that there’s a grand conspiracy by HubSpot to make people think their business is growing when they aren’t. And that makes the post seriously misleading.

Side note: I’m not a HubSpot apologist. I’m a big fan of HubSpot, but I’ve probably written more critically about the product than anyone.