Speed Kills: How Much does a Slow Web Site Cost?
In my last post, I wrote about the cost of tech debt, using a case study of skyrocketing hardware costs. Here's another, subtler effect of poor performance: impatient customers don't stick around when they experience slowdowns. However, choosing to prioritize speed can be hard to justify when the cost isn't quantified.
Here's a quick experiment using one of my favorite data science toolkits: SciPy and Jupyter. By downloading the page load times from Google Analytics, and comparing it to the conversion rate (how often people buy stuff), it's possible to place an actual dollar value on page speed, unique to your audience.
For the site in the case study, the average page load time is 8.6 seconds and the opportunity cost is $285,000 annually for each extra second of page load time!
Constructing the chart
To create the chart, each data point from Google Analytics (a one hour interval) is charted on a scatterplot. In order to visualize the large volume of data, points are grouped into hexagonal areas, and brighter areas contain more points.
A curve fit to ?⋅?(?⋅?) reveals the overall pattern. The slope of this curve predicts how much revenue is gained or lost by one second of page load time.
If this site could trim two seconds off page load time, the expected return would be over half a million dollars per year. For most executives I've worked with, this is a totally non-intuitive result. That is why I believe it's crucial to use evidence-based, economic logic when making prioritization decisions.
Imperfect information
There are no doubt imperfections in the model. For example, we do not know how many customers might return later and still complete their purchase, or how many would never return, or even become brand detractors and influence their social connections to shop elsewhere. However, the goal is to improve decisions, not make them perfect. If the starting point is no information, then the bar is very low.
Zombie Feynman (via xkcd) sums this up perfectly:
Ideas are tested by experiment. . . . Everything else is bookkeeping.
Try it out
If you want to experiment with your own data, download the Jupyter notebook. 
To export the data from Google Analytics, first create custom report. Set the dimensions to "hour index" and the metrics to "avg. page load time" and "goal conversions". Then export the report as a CSV, and load the data into the Jupyter notebook.
Comments
Nice visualization and analysis. You sort of address this in the "Imperfect Information" section, but just be ever mindful of the "correlation does not imply causation" argument. Some of the increase in revenue may be because a more established company which has a larger revenue has the funds to always maintain the best hardware. The relationship exists regardless, but we don't know what causes what. Like you said, understanding this relationship is better than the "no information" starting point. Nice work.
I'll share my favorite correlation does not imply causation graphic:
https://upload.wikimedia.org/wikipedia/commons/d/de/PiratesVsTemp%28en%…
Mon, 12/11/2017 - 21:09
Thanks for the feedback! I'd like to clarify that all this data comes from a single web site, so the factors you mentioned (e.g. hardware) are relatively constant. Each sample is a 1-hour slice of this site's traffic.
I do agree this data alone doesn't prove causation. However, there is a pretty clear method of action: users have a finite amount of patience, and abandon the site when that patience is exhausted. Fortunately, others have done A/B testing where the page load time was varied independently (by adding artificial delays) and got similar results, which I think is pretty strong evidence for causation.
A reverse causation would be harder to explain. Could a temporary increase in conversion rate cause the site to run faster? Or could a third factor be causing both observations?
Thu, 12/07/2017 - 18:54
You have a major technical flow as you confuse between correlation to causality.
Probably desirable websites can afford having a fast load page, and also have the user experience understanding - because they know their business. HOWEVER, if you don't have a desirable website, or you have a bad business - then investing in making your page load fast, will gain in zero income improvement.
Tue, 12/12/2017 - 00:07
I think this analysis was based on a particular site. So assuming all other conditions are the same, the conclusion is for a specific site, the faster your website, the more likely consumer will buy stuff, which is definitely valid.
However, other conditions are not the same. For example, sales might be different on promotion period or holidays. Another interesting thinking is, because of more consumers flood into your website, it gets slower, which should show a reverse trend.
Thu, 12/07/2017 - 15:53