4. Data-driven UX: Measuring UX at every phase
of the UCD process
“The good thing about data? It speaks for itself.”
Define Ideate Prototype DeliverUnderstand
• Measure and
understand the
various parts of
the ecosystem
• Cross-reference
and validate
insights
• Protect the
researchers against
their own biases
• Inform decision
• Strategize the
roadmap
• Benchmark solution UX
• Monitor UX health
• Identify areas and group
of adoption risk
• Predict user adoption
• Measure user adoption
Define Ideate Prototype DeliverUnderstand
5. Distribution tower
Responsible for planning and delivering materials
and personnel to the crews on the wellsite
Objective:
Rethink the distribution tower to unlock
potentialities
7. From user interviews and
observations to data
• 5 user roles identified
• Users have a capital support role for field operation: if
they fail, operations cannot run
• Major issues experienced by users are:
• Communication failures
• Poor handover between teams
• Software systems unfit to user needs
• Creation of shadow systems
Understand
8. Logging data is a key
part of the process
toward data-driven UX
It allows:
• Coding insights and discover invisible
patterns
• Preserve the integrity of the qualitative
data against researcher biases
(empathy bias, memory bias,
confirmation bias…)
Understand
10. What do the numbers tell us?
• Communication seem to fail for most user roles, and particularly for the user role 5 What is happening
exactly?
• Handover is a major issue to user roles 1 and 5 Why? How?
• Current systems seem to fail all user roles, and all user roles have created their own shadow systems
0
1
2
3
4
5
6
7
8
9
COMMUNICATION HANDOVER SYSTEM FAILURE SHADOW SYSTEM
NUMBEROFISSUES
User role 1
User role 2
User role 3
User role 4
User role 5
Understand
11. What does that tell us about the
solution?
Systems are unfit for purpose for all users
• Exploration of shadow system reveals that current modules for personnel
planning and materials planning are unfit for purpose and need to be
redesigned
Communication and handover
• Communication and handover between user roles are a big issue and raised
more questions
Define Ideate
15. 0 2 4 6 8 10 12 14
User role 2
User role 4
User role 5
User role 3
User role 1
Number of topic exchanged between user role 5 and
their main contacts
Communication mapping survey
User role 5 only pushes information to contact 1 User role 1 is dependent on user role 5 on a lot
of topics
0% 20% 40% 60% 80% 100%
User role 2
User role 4
User role 3
User role 5
User role 1
Communication direction between user role 5 and their
main contacts
To
From
Define Ideate
16. What does that tell us about the
solution?
Communication and handover
• Reorganize seating so that user role 1 and user role 5 are now sitting next to
each other: force communication and increase quality of information
handover
Systems and shadow systems
• Exploration of shadow system reveals that current modules for personnel
planning and materials planning are unfit for purpose and need to be
redesigned
Define Ideate
17. How can data
help the
stakeholders
strategize the
roadmap?
0%
20%
40%
60%
80%
100%
User roles affected Number of users
Materials planning Personnel planning
Define Ideate
20. What type of issues for users?
0%
2%
4%
6%
8%
10%
12%
14%
16%
History Materials
planning
Personnel
planning
Data History Materials
planning
Personnel
planning
Data History Materials
planning
Personnel
planning
LIKE Unmet need Usability
Percentageofissues
Prototype Deliver
22. Usability testing issues tracking
0
5
10
15
20
25
30
35
40
FURTHER VERSION IN QUEUE NEED BUSINESS DECISION TRAINING
Unmet need
Usability
Prototype Deliver
24. Tracking usability testing fixes across time
Evolution of “unmet needs” issues Evolution of “usability issues”
0
5
10
15
20
25
30
35
40
FURTHER RELEASE IN QUEUE NEED BUSINESS
DECISION
TRAINING
RELEASE 1 RELEASE 2 RELEASE 3
0
5
10
15
20
25
30
35
40
FURTHER RELEASE IN QUEUE NEED BUSINESS
DECISION
TRAINING
RELEASE 1 RELEASE 2 RELEASE 3
26. 0
20
40
60
80
100
User group 1 User group 2
Legacy system
Release 1
Benchmarking UX and predict adoption
Track the UX health of your solution
across groups
Track the UX health of your solution
over time
0
20
40
60
80
100
Legacy system Release 1 Release 2 Release 3
*
28. The major
obstacles to
data-driven UX
Limited access to users
Team/stakeholders need feedback right away
Limited amount of time: UX Unicorn working on
several project
Strong insights from few interviews
Insufficient training in data analysis