Carol Smith presented this topic at the Heart of Agile conference in Pittsburgh, PA in April 2017.
UX Slows Agile down! Do you hear that a lot? Carol shared best practices and how to dispel this myth in this session. The presentation included discussions of how to successfully embedding the UX team and the pros and cons of Agile projects. Carol will introduce methods for the UX team to break down and include their work in the backlog so it can get prioritized. Finally, Carol will discuss several successful ways to integrate usability testing across iterations.
2. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Arguments Against UX/Design
• User Experience (UX) work is slow!
• Design is indecisive - make up your mind!
• Dev is always wrong!
3. IBM Watson / #HOAPITT / @carologic / April 28, 2017
"The biggest waste of all
is building something
no one wants"
- @ericries #LeanStartupMI via @MelBugai
4. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Shared Values between UX and Agile
• Users and interactions
– Not individual customers
• Working software
• User collaboration
– Not customer specific
– Usability testing with users
• Responding to change
5. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Design at IBM Watson
6. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Goals of UX
• Everyone understands users
• Solve problems with simplest solution
• Encourage collaborations
• Think ahead
7. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Increased understanding of users
• Who are they?
• Needs and goals?
• Problems they are trying to solve?
• Latent needs?
• Language/acronyms used?
8. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Goal is not to hit a nail
Image: https://commons.wikimedia.org/wiki/File:Hammer-lever_diagram.jpg
9. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Goal is to build…
10. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Users goals are not…
• Logging in
• Clicking next
• Uploading docs
• Filling out a form
• Solve their problem – as simply as possible
11. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Encourage Collaborations
• How can we approach this problem together?
12. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Think ahead to solve problems
• Big work is coming?
• Change as product matures?
• What next in product lifecycle?
Bart Image: https://en.wikipedia.org/wiki/Bart_Simpson
13. IBM Watson / #HOAPITT / @carologic / April 28, 2017
No BDUF!*
• Just enough
• Solve the problem
• Future vision
– Concept Cars
– Know the North Star
*Big Design Up Front
15. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Imbedding UX
16. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Squads include UX folk
• Researcher and interaction designer on squad
– Visual designer can be shared
• Larger products
– Many squads
– Focus on user interactions
– Consult on other teams as needed (e.g. review API flow)
17. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Designers are part of the team
• Standups/Scrums
• Planning/grooming, review, retro, demo
• Estimating work
• Take turns with Agile roles
• Design doing QA of design work
Image: http://intland.com/blog/project-management-en/tips-and-tricks-
to-make-the-most-of-your-retrospectives/
18. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Collaborative Activities
19. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Co-Design
• Developers, product owner/manager and designers work
together on early sketching, task analysis, etc.
• Pro’s
– All info needed – tech, business, user
– Shared understanding – saves time
– Solve problem together
• Con’s
– Time consuming; reduces time to build
20. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Usability Testing
• Team members actively observe
– Working software or prototypes
– Goal to have less usability issues in software
– Can watch remotely
• Pro’s
– Gain empathy for users; understand struggles and what works
• Cons
– Time; Not all sessions are great (but most are!)
21. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Visiting Customers
• Observe customers and see how they work
• Pro’s
– Major potential increase in empathy
• Con’s
– Major time investment
– Time away from coding
22. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Keep Trying
https://www.flickr.com/photos/fireatwillrva/5823369990
23. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Parallel-Track
Workflows
a.k.a. Staggered Sprints
24. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Agile Design Timing: Parallel Tracks
• Developer track:
– Focus is on production code
– Whole team (including design)
• Interaction designers track
– Focus is on user contact (Dev less involved)
Slide from Tutorial: Making UX Agile and Lean at UXPA 2016 by John Schrag,
Carol Smith and Thyra Rauch http://uxpa2016.org/event/making-ux-agile-and-lean
25. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Iteration 1
Slide from Tutorial: Making UX Agile and Lean at UXPA 2016 by John Schrag,
Carol Smith and Thyra Rauch http://uxpa2016.org/event/making-ux-agile-and-lean
26. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Iteration 2
Slide from Tutorial: Making UX Agile and Lean at UXPA 2016 by John Schrag,
Carol Smith and Thyra Rauch http://uxpa2016.org/event/making-ux-agile-and-lean
27. IBM Watson / #HOAPITT / @carologic / April 28, 2017
And so on…
Slide from Tutorial: Making UX Agile and Lean at UXPA 2016 by John Schrag,
Carol Smith and Thyra Rauch http://uxpa2016.org/event/making-ux-agile-and-lean
28. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Constant Communication
• Between the two tracks - essential for success
• Not just hand-offs
Slide from Tutorial: Making UX Agile and Lean at UXPA 2016 by John Schrag,
Carol Smith and Thyra Rauch http://uxpa2016.org/event/making-ux-agile-and-lean
29. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Pros and Cons - Parallel/Staggered work
• Pros
– Gives UX a head start
– Lot of backend work needs to be done anyway
– Establish common vision
• Who for?
• Common “elevator statement”
• Cons
– Can lead to BDUF and Waterfall
30. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Balancing Act
• Understand users, context, etc.
• Create personas, mental models, etc.
• Prepare for story mapping
and other sessions thoroughly
Strive for
UX Best
Practices
• Engineers need designs
to develop
• Research cannot continue forever
Meet
Production
Needs
Agile requires leaner methods
Time box and focus on outcomes
31. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Building #UX Stories
and
Breaking down work
32. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Story Card Progress
• Is UX Needed?
– What is the story?
– What do we know?
– What do we need to know?
• Lean UX to answer questions
– Interviews, observations, prototype testing, etc.
32
Kanban
Do Review Done
34. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Conversations to understand
• Relationships between stories
• Structure and hierarchy of related stories
• Functionality that is being built
• Conversations to share knowledge
• Clarify assumptions about what is to be built
35. IBM Watson / #HOAPITT / @carologic / April 28, 2017
User Story Mapping
• Organize user stories
into a map that
communicates experience
36. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Represents Complexity and Size of Work
time
Refine the map and test for
completeness with the
entire team - developers,
designers, BA’s etc.
37. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Encourage Information Radiators
• User information
• Customer journeys
• Photos from research
• Usability metrics
Photo: https://medium.com/@FBResearch/beyond-bullet-points-four-
creative-ways-to-share-research-c10fa047f025#.cdauyngei
38. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Enable design stories
• “Problems we will find while usability testing”
• “Research to inform personas”
• “Design debt”
39. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Integrating UX Activities
40. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Light Design - Lean UX
• Iterative
• Repeatable
• Incremental
• Get to 80% confidence
41. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Usability Testing is…
• Representative users
• Doing real tasks
• Being observed
• Using prototypes
or live products
• Not User Acceptance Testing
http://creativecommons.org/licenses/by-sa/2.0/
http://www.flickr.com/photos/raphaelquinet/513351385/sizes/l/in/photostream/
http://www.flickr.com/photos/raphaelquinet/
42. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Continuous Learning Benefits
• Save time & money
• Minimize human cost (users & team)
• Better UX means increased
– Usefulness
– Efficiency
– Productivity
43. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Small scope is good
• 5-7 participants uncover
~80% of major usability problems
• Don’t try to be statistically
significant
• Diminishing returns
http://www.useit.com/alertbox/20000319.html
Jakob Nielsen’s Alertbox. Why You Only Need to Test with 5 Users. March 19, 2000.
Number of Test Participants
Participants are representative of specific user group
and are doing representative tasks
More participants takes more time, costs more money
- means significantly diminishing return.
44. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Include people with disabilities
• “We are all only temporarily able-bodied.
Accessibility is good for us all.”
• Know the law, build for the spirit of accessibility
– U.S. Section 508 - https://www.section508.gov/content/learn
– WCAG 2.0 - https://www.w3.org/WAI/intro/wcag.php
-@mollydotcom at #stirtrek 2011 via @carologic #a11y
45. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Many ways to test
• Online software
– UserTesting.com (un-moderated)
– Web Meeting (moderated)
• In Person (moderated)
• Moderated is better to get to “why”
• Unmoderated is great for quick answers
46. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Regular Testing
(Yes, this is an old idea; a great one!)
47. IBM Watson / #HOAPITT / @carologic / April 28, 2017
“Teams should stretch
to get work into that day’s test
and use the cadence to drive
productivity.”
Jeff Gothelf
http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/
48. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Test whatever is ready…
• Concepts, rough ideas, brainstorming
• Work in Progress
• Prototypes - make to learn
– Clickable (Axure, Balsamiq – code is optional)
• Websites, mobile, products, services
• Competing designs (A/B testing)
• Comparative studies
• Conduct user research
49. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Be an Active Observer
• Offer to greet participants (in person)
• Help operate software/recording equipment
• Take notes
• Share your notes
50. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Board/Flip Chart Notes - look for Patterns
Issue P1 P2 P3
Search Used Yes No No
Widget 1 Used N/A Used –
unsure
about
Task 1 Notes 3 – easy 2 – needed
help
3 – easy
Task 2 Notes 2 – needed
help
2 – easy 2 – needed
help
Task 3 Notes 2 – needed
help
3 – easy Ran out of
time
Task 4 Notes 2 – needed
help
3 – easy Ran out of
time
51. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Make Useful & Usable
Recommendations - Quickly
52. IBM Watson / #HOAPITT / @carologic / April 28, 2017
True Statements
• All interfaces have usability problems
• Limited resources to fix them
• More problems than resources
• Less serious problems distract
• Intense focus on fixing most serious problems first
Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide
to Finding and Fixing Usability Problems. By Steve Krug
53. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Don’t support tree killers
Image of stack of paper:
https://commons.wikimedia.org/wiki/File:Stack_of_Copy_Paper.jpg
X Marks the Spot https://openclipart.org/detail/193050/x-marks-the-spot
54. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Make users visible
• Everyone understand user
needs
• Solve problems in the
simplest ways
• Encourage collaborations
• Think ahead
55. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Contact Carol Smith
in/CarolJSmith
@Carologic
slideshare.net/carologic
56. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Resources, References, etc.
57. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Recommended Readings
5
7
58. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Towards Building Teams
• Hire great people
• Balance skills and experience
• Enable to work to their strengths
• Encourage to grow, attend webinars, conferences, etc.
• Make time for 1:1’s
59. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Tool Considerations
• In-person or remote?
• Lab or on-site?
• Prototype limitations (can it be online?, is it a document or a clickable site?)
• Number of observers, number of participants?
• Number of facilitators?
• Logging and video editing needs (time on task, highlight video creation)?
• Surveys before or after?
• Eye tracking?
60. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Agile Frustrations via Spotify
Scaling Agile at Spotify via Slideshare of Vlad Mysla
http://www.slideshare.net/vmysla/scrum-at-spotify?qid=2345c3ad-7e68-4383-9673-
9e715ff47a75&v=default&b=&from_search=14
squad member
61. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Internet resources
• Usability.gov
• W3C Web Accessibility Initiative
– http://www.w3.org/WAI/
• Accessibility Standards in US (Section 508)
– http://www.access-board.gov/sec508/508standards.htm
• Jakob Nielsen
– http://www.useit.com
• UPA – professional UX association
– http://www.upainternational.org/
62. IBM Watson / #HOAPITT / @carologic / April 28, 2017
Reference List
• Albert, Bill, Tom Tullis, and Donna Tedesco. Beyond the Usability Lab.
• Gothelf , Jeff. http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-agile/
• Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in Diverse Situations. Tutorial,
2003 UPA Conference.
• Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability
Problems.
• Patton, Jeff (with Peter Economy). User Story Mapping, Discover the whole story, build the right product.
• Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct
Effective Tests. John Wiley & Sons, Inc.
63. Adapting Usability Investigations for Agile User-Centered Design by
Desirée Sy
Journal of Usability Studies, Volume 2, Issue 3 (the most-cited paper
in JUS)
http://www.uxpajournal.org/