Research on the relationship between web page latency and web analytics KPIs such as time on site, bounce rate, and conversion, including experimental data from Google, Microsoft Bing, Shopzilla, and Strangeloop
Performance Impact<br />How Web speed affects online business KPIs<br />
Today’s Hosts<br />Hooman Beheshti, <br />VP Product, Strangeloop<br />Alistair Croll,<br />Analyst, Bitcurrent<br />Author of O’Reilly’s Complete Web Monitoring<br />
10 s<br />1 s<br />100 ms<br />10 ms<br />Zzz<br />!<br />
http://www.flickr.com/photos/spunter/393793587<br />http://www.flickr.com/photos/laurenclose/2217307446<br />
Everything is interwoven.<br />
We’re getting better<br />
Impact of page load time on average daily searches per user<br />
Impact of additional delay on business metrics<br />
Shopzilla had another angle<br /><ul><li>Big, high-traffic site
100M impressions a day
8,000 searches a second
20-29M unique visitors a month
16 month re-engineering
Page load from 6 seconds to 1.2
Uptime from 99.65% to 99.97%
10% of previous hardware needs</li></ul>http://en.oreilly.com/velocity2009/public/schedule/detail/7709<br />
5-12% increase in revenue<br />
Transactional<br />SaaS<br />Buy something<br />(Amazon)<br />Use an app<br />(Salesforce)<br />Media<br />Collaborative<br />Click an ad<br />(Google News)<br />Create content<br />(Wikipedia)<br />
Tying web latency to business outcomes<br />
KPIs<br />http://www.flickr.com/photos/spunter/393793587<br />http://www.flickr.com/photos/laurenclose/2217307446<br />
ATTENTION<br />ENGAGEMENT<br />CONVERSION<br />NEWVISITORS<br />SEARCHES<br />TWEETS<br />MENTIONS<br />ADS SEEN<br />CONVERSIONRATE<br />GROWTH<br />TIMEONSITE<br />PAGESPERVISIT<br />NUMBEROF VISITS<br />x<br />ORDERVALUE<br />LOSS<br />BOUNCERATE<br />
It’s time for an experiment<br />
Strangeloop<br />Visitor<br />Webserver<br />Decide whetherto optimize<br />Normalcontent<br />Accelerated<br />Receivepage<br />Optimize?<br />Insert<br />segment<br />marker<br />Processscripts<br />Sendanalytics<br />Unaccelerated<br />Googleanalytics<br />
What we learned<br />
Traffic levels<br />
Bounce rate<br />
% New visitors<br />
Average time on site<br />
Pages per visit<br />
Conversion rate & order value<br />
Justifying an investment in performance<br />(<br />)<br />Currentdaily orders<br />Increased<br />conversions<br />Increasedorder value<br />*<br />+<br />ROI(days)<br />=<br />Cost of performanceenhancement<br />
Justifying an investment in performance<br />(<br />)<br />*<br />+<br />0.1607<br />0.0551<br />$10,000<br />$2,158<br />23.17days<br />=<br />=<br />$50,000<br />$50,000<br /><ul><li>Caveats
Your mileage will vary
This is just how to think about it</li></li></ul><li>Conclusions<br /><ul><li>Links between performance and business KPIs are undeniable
By talking the same language, IT and marketing can finally agree on what to do about it
Changing from “X times faster” to “$Y more money” makes the business care
More research is needed</li></li></ul><li>What we need next<br /># ofvisits<br />Optimized<br />0<br />10,000<br />Visitorlatency<br />Different visitors experienced different performance levels.<br />
What we need next<br /># ofvisits<br />21.58%better<br />0<br />10,000<br />Visitorlatency<br />Right now we have a single experiment, and a single resulting business impact.<br />
What we need next<br />Best 5%<br />Worst 5%<br /># ofvisits<br />Optimized<br />0<br />10,000<br />Visitorlatency<br />Visitors who were optimized fall into a range – the 5th to 95th percentile<br />
What we need next<br />24%<br />18%<br />14%<br />Gajillions<br />12%<br />9.5%<br />$ perday<br />0<br />0<br />10,000<br />Visitorlatency<br />If we have several experiments, we can understand the relationship better.<br />
What we need next<br />Gajillions<br />$ perday<br />0<br />0<br />10,000<br />Visitorlatency<br />Every web business has a curve like this hidden inside it.<br />
Questions? (Submit your questions using the GoToWebinar question tool)<br />
More information<br />Visit: www.watchingwebsites.com<br />Visit: www.bitcurrent.com<br />Twitter: @acroll<br />
Once upon a time, performance was a dark art. We struggled to deliver “good enough” without really knowing why.
We managed by anecdote. We were sure faster was better, but we couldn’t tie it to specific business outcomes.
The notion that speed is good for users isn’t new. The concept of “Flow” – a state of heightened engagement that we experience when we’re truly focused on something – was first proposed by mihalycsikszentmihalyi
It turns out that attention and engagement drop off predictably. At ten milliseconds, we actually believe something is physically accessible – think clicking a button and seeing it change color. At 100 milliseconds, we can have a conversation with someone without noticing the delay (remember old transatlantic calls?) At a second, we’re still engaged, but aware of the delay. At ten seconds, we get bored and tune out, because other things come into our minds.
How much was fast enough? It was anybody’s guess.
And guess they did.This is Zona’s formula for patience, the basis for the “eight second rule.” Unfortunately, things like tenacity, importance, and natural patience aren’t concrete enough for the no-nonsense folks that run web applications.
IT operators and marketers are completely different people. What convinces an IT person to fix performance doesn’t convince a marketer. They want to know how it will impact the business fundamentals.
By now, we know that everything matters. Usability, page latency, visitor mindset, and even sentiment on social media platforms all contribute to the business results you get from a site.
Fortunately, we’re getting better at linking performance to business outcomes.
One example of this is performance experimentation that Google’s done. Google’s a perfect lab. Not only do they have a lot of traffic, they also have computing resources to do back-end analysis of large data sets. Plus, they’re not afraid of experimentation – in fact, they insist on it. So they tried different levels of performance and watched what happened to visitors.
The results, which they presented at Velocity in May, were fascinating. There was a direct impact between delay and the number of searches a user did each day – and to make matters worse, the numbers often didn’t improve even when the delay was removed. You may think 0.7% drop isn’t significant, but for Google this represents a tremendous amount of revenue.
Microsoft’s Bing site is a good lab, too. They looked at key metrics, or KPIs, of their search site.
They showed that as performance got worse, all key metrics did, too. Not just the number of searches, but also the revenue (earned when someone clicks) and refinement of searches.
Shopzilla overhauled their entire site, dramatically reducing page load time, hardware requirements, and downtime.
They saw a significant increase in revenues
The site improvement increased the number of Google clicks that turned into actual visits
It also affected search engine scores. By improving load time, search engines (in this case Google UK) “learned” that this was a good destination. That’s right – Google actually penalizes sites that are slow by giving them a lower page ranking.
While this shows us metrics for large sites focused on sales and ad clicks, it doesn’t tell us about fundamentals.There are four fundamental site models, each of which has different business goals. An e-commerce site focused on transactions wants to convert visitors to buyers. A SaaS site wants to make subscribers renew. A media site wants to serve relevant ads and maximize searches or views. And so on.
If we want to convince marketing, we need to measure business metrics.
By tying performance and availability to Key Performance Indicators – KPIs – business and operations can finally have a conversation.
Whether those KPIs are shopping cart abandonment
Or visitor “bounce rate” (the number of visitors that leave immediately)
Or just traffic.
So what KPIs would we like to learn about? This is what web analytics folks work by, whether they’re running a media site, a SaaS platform, a transactional application, or a collaborative social network. It’s what the business cares about.
Strangeloop agreed to set up an experiment using their technology which would help measure this.
First, traffic. Despite splitting visitors to be optimized and unoptimized evenly, we had many more optimized sessions captured by the analytics. This may be a result of slower-loading pages failing to execute the analytics script, or abandoning the visit before the page had time to load.
Unoptimized visitors are roughly 1% more likely to leave the site immediately, without proceeding to other pages.
Strangely, the unoptimized visitors consisted of more new visitors than the optimized ones did. This seems counter-intuitive and warrants further study.
Optimized visitors spent more time on the site
And looked at more pages during their visit – if you’re a media property, this means more impressions for your advertisers.
On a second e-commerce site running roughly the same experiment, conversions were 16 percent higher and orders were 5.5% higher.