Slides from the webinar with Susan Weinschenk on how to measure the ROI of User Experience.

1. Webinar:
How to Measure the ROI of User Experience
Susan Weinschenk, Weinschenk Institute, LLC
Alfonso de la Nuez, UserZoom
#uzwebinar

2. Susan Weinschenk Alfonso de la Nuez
Founder and Principal co-Founder and co-CEO
Weinschenk Institute, LLC UserZoom
@thebrainlady @delanuez23
@userzoom

3. Quick Housekeeping
• Chat box is available if you have any questions
• There will be time for Q&A at the end
• We will be recording the webinar for future viewing
• All attendees will receive a copy of the slides/recording
• Twitter hashtag: #uzwebinar

4. Sample ROI Calculation
100 visitors a day come put an item in the shopping cart of your e-commerce
site
But 30% don’t complete because of a user experience problem with the cart
You are losing 30 sales a day due to a UX issue
@ $10 a purchase = $300 a day lost
300 x 365 = $109,500 a year lost
It will take $20,000 and 2 weeks to fix the UX issue
In 10 weeks you will have made up the $20k you spent
In 2.5 months you will have paid for the fix with the increased revenue: ROI
in 2.5 months

5. What We’ll Cover
different ways to calculate ROI
how to deal with estimates and assumptions for key indicators
what key indicators you should use
how to actually measure to compare with estimate
how to estimate cost of the UX work itself
which should you do: fix later or get it right from the start?
the role of tools (card sorting, user testing, etc) in your ROI calculations
fixes rather than UX from beginning

6. My First ROI Calculation (circa 1994)
Software for bank tellers
5000 users
Interface was hard to learn and hard to use -- poor user experience
Assumptions: waste 5 seconds per transaction
100 transactions per person per day
5000 people x 100 transactions x 5 seconds = 2.5 M seconds = 41,666
minutes = 694 hours x $7/hour = $4,858 a day x 252 days = $1,224,216

7. That’s Not All!
Take training from 2 weeks to 2 days, saves $1000 in training costs per new
person
10% turnover = 500 people a year x $1000 = $500,000 per year in training
PLUS help desk -- reduce calls by half, save 300 calls per month x $10 per call
= $3,000 a month = $36,000 per year
$1,224,216 + $500,000 + $36,000 = $1,760,216 in first year
Cost to fix: $500,000
Recoup investment in 3.5 months
ROI = 3.5 months

8. To Calculate Your ROI
1. Decide on your key indicators
2. Estimate lost opportunity or cost of having a poor user experience
3. Estimate the cost to research and fix a poor experience or cost to
create a new one to start with -- cost of doing it right
4. Do your calculations
5. Turn your money into length of time to recoup the investment

9. There is no magic ROI formula

10. What should we measure?
time on task?
task success?
satisfaction?
efficiency?
errors?

11. There is no one “key performance
indicator”

13. ROI Calculation Examples
•Errors:
•(# of Errors) X (Avg. Repair Time) x (Employee Cost) X (# of
employees) = Cost Savings
•(2 errors/week) X (60 minutes) X ($30/hour) X (100 employees) =
$6,000 per week = $300,000 per year

14. ROI Calculation Examples
•Cost of Development and Maintenance:
•(# of changes) X (Avg. hours per change) X (cost of developer) X
(4, if change is late!) = Cost Savings
•(20 changes) X (8 hours each) X ($40/hour) = $6,400 if fixed early,
or $25,600 if changed late

15. ROI Calculation Examples
•Productivity:
•(Time Saved) X( Employee Cost) X (# of employees) = Cost
Savings
•(1 hour a week) X ($30/hour) X (1000 employees) = $30,000 per
week = $1,500,000 a year

16. http://spectrum.ieee.org/comp
uting/software/why-software-
fails

17. This year,
The problem only gets worse as IT grows ubiquitous.
organizations and governments will spend an
estimated $1 trillion on IT hardware, software,
and services worldwide. Of the IT projects that are initiated,
from 5 to 15 percent will be abandoned before or shortly after delivery as
hopelessly inadequate. Many others will arrive late and over budget or require
massive reworking. Few IT projects, in other words, truly succeed.

19. software specialists spend about
In fact, studies have shown that
40 to 50 percent of their time on avoidable rework
rather than on what they call value-added work, which is basically work
that's done right the first time.

20. Why Projects Fail
•Unrealistic or unarticulated project goals
•Inaccurate estimates of needed resources
•Badly defined requirements
•Poor reporting of the project's status
•Unmanaged risks
•Poor communication among customers, developers, and users
•Use of immature technology
•Inability to handle the project's complexity
•Sloppy development practices
•Poor project management
•Stakeholder politics
•Commercial pressures

21. Designing the User Experience
• User Research
• Storyboarding, Designing, Prototyping, Visual Design
• User Testing & Iteration

22. Is It Worth It?
• Cost to design it right the first time
• Added cost than if you just designed it the way you normally do
• Cost to fix the user experience problems after the fact
• Opportunities lost if the user experience isn’t right

23. Watch Out
• Definition of “done”
• Definition of “success”

24. When You Design The User Experience
– Saves user time
– Saves developer time
– Increases customer satisfaction
– Increases use of the technology
– Reduces training costs
– Reduces calls to the help desk
– Increases self-service

25. Using ROI To Plan Your UX Work
Project Fix or From $ amount to be Cost of UX to ROI
Scratch? gained or saved fix or design
A Fix
B From Scratch
C From Scratch
D Fix
E Fix

26. Medicine Net Website Re-Design

27. The Role of Testing & Research Tools in ROI
• Use to benchmark for “before” and “after” comparisons
• Data from which to base your assumptions & estimates

28. Advantages of Remote Tools
• Less expensive than in-person,
moderated data collection
• Collect data from more people
for less cost -- enough to run
statistic significance
• Collect data from diverse
geographies for less cost
• Reduce the cost per project of
doing user research and user
testing during design
• Get results fast

29. intellectuals solve problems;
geniuses prevent them
~albert einstein

30. Email: susan@theteamw.com
Twitter: @thebrainlady
Phone: 847-909-5946
Website: www.theteamw.com
Facebook: Weinschenk Institute
Susan Weinschenk, Ph.D.
Weinschenk Institute, LLC