A case study using high fidelity wireframes to test a mobile website from User Focus 2014 in Washington DC

1. Bringing Your Wireframe to LifeA case study using a high-fidelity wireframe to test a mobile website
Brandon Kopp
User Focus 2014
October 17th, 2014

2. The Consumer Expenditure Diary Survey
Captures all expenditures made by a household, with an emphasis on hard-to- remember expenses (e.g., groceries)
Requires a high level of detail
Currently done at the household level for two, 1-week periods
BLS is exploring the possibility of each household member (15+) entering their own expenditures for one week
2

3. The Consumer Expenditure Survey Mobile Diary
Already paper and prototype, desktop Internet versions of the diary
Allow respondents to enter information about their purchases on-the-go
Set out to create:
A mobile-optimized website, Notan App
Works across OS’s and devices
Simple and user-friendly
3

4. Timeline
4
Draft Design Complete
Test Version Release

5. Outline
I will focus on the wireframe testing
The range of options for early stage usability testing
How the high-fidelity wireframe was constructed
The testing procedures
The results and recommendations
Problems we did not discover with the wireframe (Test Version usability results)
Strengths/Weaknesses of the high-fidelity wireframe approach
5

6. Initial Design Concept
6

7. Low-Fidelity Wireframes
7
CE Diary
Username
Password
Login
Burden
Accessibility
Privacy
Security

8. A Range of Options
8
CE Diary
Username
Password
Login
Burden
Accessibility
Privacy
Security
CE Diary
Login
Burden Statement
Accessibility
Privacy
Security
OMB ID Number : 1220-0141
2/28/2015
Data usage rates may apply
Username:
Password:
CE Diary
Login
Burden Statement
Accessibility
Privacy
Security
OMB ID Number : 1220-0141
2/28/2015
Data usage rates may apply
Username:
Password: PrototypeLow-Fidelity WireframesHigh-FidelityWireframes
Allow participant to enter data and navigate through the diary without paper shuffling
Used Microsoft Powerpoint, Excel, and Visual Basic for Applications (VBA)

9. BUILDING THE HIGH- FIDELITY WIREFRAME
9

10. Visual Basic for Applications
Tools & Functions
 Text Fields
 Drop-Down Menus
 Radio Buttons (Select One)
 Check Boxes (Select All That Apply)
 Action Buttons
10
Reset
Option 1 Option 2 Option 3
Option A Option B Option C
Enter Description
Select Category
Option 1 Option 2 Option 3
Option A Option B Option C

11. Visual Basic for Applications
In the Background
11
Private Sub Option1_Click()
Option2.BackColor = &H8000000F
Option2.ForeColor = &H0&
Option3.BackColor = &H8000000F
Option3.ForeColor = &H0&
Option1.BackColor = &HFF0000
Option1.ForeColor = &HFFFFFF
End Sub
Turn Option 2 & 3 “Off”
Turn Option 1 “On”
Store Data
Reformat Numbers/Text
Password Authentication
Data Entry
Display Data
Test
Option 1
Option 2
Option 3
$0.00

12. CE Diary
Login
Burden Statement
Accessibility
Privacy
Security
OMB ID Number : 1220-0141
2/28/2015
Data usage rates may apply
12
Username:
Password:
CE Diary
Login
Burden Statement
Accessibility
Privacy
Security
OMB ID Number : 1220-0141
2/28/2015
Data usage rates may apply
Username:
Password:
respond.census.gov/ced
1

13. 13
Username:
Password:
Login

14. 14
High-Fidelity WireframeScreenshots
Login
Home Screen
Initial Setup

15. 15
High-Fidelity Wireframe Data Entry Screens
General Data Entry
Food and Drink Away From Home
Food and Drink For Home Consumption
Clothing, Shoes Jewelry, & Accessories
All OtherProducts & Services

16. WIREFRAME TESTING
16

17. Methods
10 Participants
3 with “A Moderate Amount” of smartphone experience
7 with “A Lot” of experience
13 tasks that covered the basic operations a respondent would be expected to perform
Logging in and performing setup
Entering expenditure data
Editing/deleting data
17

18. Example Tasks
Self-Directed Tasks (X 2)
“Think back to the last food purchase you made. Please add that item to the diary as if the purchase was made on [DATE].”
Scenario-Based Tasks (X 8)
“Now, imagine that today is [DATE]… You’re at the mall shopping and buy a pair of jeans for yourself for $61.95. Please add that item to the diary.”
Receipt-Based Tasks (X 3)
“You just went to the grocery store to pick up a few items –enter the items you purchased using this receipt.”
18

19. Results
There were only a few navigation issues
Returning to the Expense Summary Screen
Finding the Help/Information Menu
Most issues involved data requirements and not data entry
Entry of vendor name for item description
Entering multiple expenses together
Misclassification of items
19

20. Recommendations
Provide more instructions in future testing
Change some of the navigation buttons
Allow users to sort items on Home Screen
20
Old
New
Information Screen
Summary Screen
i
INFO

21. PROTOTYPE TESTING
21

22. Methods
22
Took place five months after wireframe testing
Used functional prototype instrument
29 participants used their own smartphone
Used methodology similar to wireframe testing

23. Results
Problems notdiscovered in wireframe testing
Several participants had difficulty just navigating to the site
Logging into the site was also a problem
Virtual keyboard issues
Similar to wireframe testing
Even with instructions, participants had difficulty entering data in a way that meets diary requirements
–This is also a problem in the paper and web diaries
23

24. CONCLUSIONS
24

25. High-Fidelity WireframesStrengths & Weaknesses
Strengths
More true-to-life than paper
Simulates actual functions of the Diary
Gives stakeholders something concrete to react to
Better appreciate what developers have to contend with
Weaknesses
There is a learning curve
You are limited (at first) by what you know is possible
Took about a week’s worth of time
P’s used the keyboard and mouse
You have to program ways for people to screw up
25

26. High-Fidelity WireframesConclusions
Overall, it was worth the time and effort
The testing did not detect some important usability issues, but these would have been missed on paper as well
The wireframe helped stakeholders visualize the final product and make more informed design choices
We developed the capability for using high- fidelity wireframes in future tests
We are exploring other tools that might improve on this process (speed & flexibility)
26

27. High-Fidelity Wireframes Other Projects
27

28. Contact Information
Brandon KoppResearch PsychologistOffice of Survey Methods Researchwww.bls.gov/osmr202-691-7514kopp.brandon@bls.gov