How to do UX in Government Without Losing Your Freaking Mind
1. How to do UX in
government without
losing your freakin
mind
UXPA-DC 2014
Jonathan Rubin
U.S. General Services
Administration
@jonathan_rubin #userfocus2014
11. DigitalGov User Experience Program
● Find us on Digitalgov.gov
● Help agencies succeed in UX via support + education
● Remove barriers to UX use
● Manage Federal UX community (525+ members)
● Also:
o Usability Starter Kit
o 25+ Usability Case Studies
o API Usability Evaluations
o Federal Usability Volunteer List
12.
13. Policy Challenges
o Paperwork Reduction Act (PRA)
o Terms of Service
o IT Procurement (quick review!)
o Section 508 / Accessibility
Secrets of Good Gov UX
o Sell it (well)
o Shut up and do it
o Viva UX Team
o Get crafty (resources!)
1
2
16. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
17. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
18. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
19. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media
20. 1. Paperwork Reduction Act
● Clearance needed to collect info from public
● For structured questions + open ended
● Survey, tests, focus groups, interviews, forms
● 10+ citizens per YEAR per item
● Does NOT apply to Feds, most Social Media
● Regulates incentives
21.
22. You create: Supporting Statements
● Justify what you’re doing and why
● Target audience
● Cost (if publishing in Fed Register)
● Screenshots (if applicable)
● List of questions (if applicable)
● # of estimated hours + respondents (Burden Hours)
23. Customer Service Generic Clearance
● AKA Umbrella Clearance
● You should already have this! If not: Get it!
o Search your agency on Reginfo.gov + collection
o Talk to PRA Officer
● Worth it if you are doing lots of collections
● Can’t publish results + no statistical info
● With it you can Fast Track
25. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
Includes:
Questions / screenshot
Supporting Statements
Law / Authority
26. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
27. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days
28. Survey
Focus
Group
Usability
Test
Forms
Regular
PRA
Process
OK. You’re
good to go
29. Usability
Test
Usability
Test
Usability
Test
Usability
Test
Regular
PRA
Process
for initial
package.
Allows
grouping
like items
together, if
it makes
sense.
Generic
“Umbrella”
Clearance
UT 1
UT 2
UT 3
UT 4
30.
31. Option 1. Get PRA Generic Clearance
Agency
develops PRA
plan
OMB / OIRA Notice in
Federal
Register
(60 days)
Notice in
Federal
Register
(30 days)
OMB Reviews
(30 days)
You’re good!
Get # to put
on your
survey
Agency gets
comments
OMB gets
comments
Time needed: 120+ days
32. Option 2. Use existing PRA Generic Clearance,
then get Fast Track
Agency
develops PRA
plan
OMB Reviews
(5 days)
Time needed: 1-3 weeks
You’re good!
Get # to put
on your
survey
34. Agency Burden hours
(8,000)
Requested 20 hours
(20 people x 1 hour each)
Requested 500 hours
(5000 people x 6
mins each)
March -
Usability
Test 1 (10
people )
April -
Usability
Test 2 (10
people)
Survey
(5000
people)
Request #1:
Usability Testing
Request #2:
Survey
38. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
39. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
40. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”
41. More PRA Tips
● Be nice to your PRA officer
● Don’t let burden hours expire
● Share PRA docs with other offices
● Digitalgov.gov > “Usability Starter Kit”
● Find examples from Reginfo.gov
42.
43.
44. Incentives and the PRA
● Their view: Trying to save the gov money
● OMB ceiling is
o $40 an hour general public
o $50-$75 for focus groups
o More $ needs justification in support statements
o Work with PRA officer early + often to increase $
● Contractors can get around this
● Besides money: Goodwill, swag, gift cards
47. One way to get amended TOS signed
Find a tool you
like
Have your
lawyers talk to
their lawyers
Come up with
Amended
Terms of
Service
Use tool /
Dominate
Universe
Review the
terms
48. TOS on DigitalGov.gov
● GSA can help
negotiate if there is
community need
● What is TOS?
● Template for
amending TOS
● Examples of 70+
signed TOS
● Flowchart!
50. Problems with UX in IT contracts
lots of people much smarter than me are working on IT procurement reform - just looking at UX in IT contracting.
some issues:
1. No UX law (unlike 508)
2. One-size fits all contracts don’t apply well to IT
3. COs + PMs not familiar/don’t care about UX
4. CO and PM may not work together from starT
5. Waterfall :(
6. Biz requirements… what about user req?
7. No UX in SOW > Proposals don’t include UX
8. Unclear about contractor UX qualifications
9. Lessons learned not shared
51.
52. Things that might help
things that might help
1. Don’t bury lede in RFP! Clearly state problem, what u want
2. Include UX definitions and glossary
3. Have a UX person look at contract first
4. Connet wireframes to usability testing
5. UX = deliverable / labor category, UAT 508… - right near 508
6. But be broad! Don’t constrain too much
7. Testing as milestone (UAT?), block out test time
8. Do your own user research BEFORE contract... (or get help)
9. Look at SOW Templates in Digitalgov.gov and usability.gov
10. Past performance - screenshots, not text
66. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them
67. Measuring UX performance
● Attend ALL UX activities
● Stress SHORT reports
● Create Baseline UX metrics, monitor
● Monitor ACSI / customer exp. increase
● Create KPIs and measure against them
● Refer to usability best practices
68.
69. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
70. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)
71. Techfar - Agile IT Guidance on FAR
● Techfar = Part of WH Digital Services Playbook
● Promotes iterative development (Agile)
o Improved manageability + budgetary feasibility
o Reduction of risk
o Frequent delivery of usable capabilities
o Increased flexibility
o Greater visibility into contractor performance
73. More Procurement resources
● Possible Contract Vehicles:
o 541 - Advertising and Marketing
541-3 - Web Design
541-4A - Market Research and Analysis
o GSA Schedule 70 - IT Services
● UX education at DigitalGov.gov + calendar
● Check out UX SOWs on DG.gov + Usability.gov
● FBO.gov
76. What accessible coding looks like
● Alternative text for all graphics and all non-text media
77. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
78. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
79. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
80. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language
81. What accessible coding looks like
● Alternative text for all graphics and all non-text media
● All form fields + controls labeled for asst. technologies
● Not mouse driven, keyboard can explore all content
● Contains header formatting (skip links) for easy reading
● Written in Plain Language
● Pop-ups are keyboard accessible
● Etc.
88. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
89. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
90. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
91. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
92. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
93. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)
94. More 508 resources
● Add into contract, start early!
● Talk to gov Accessibility Community of Practice
o Guest speakers, brown bags, etc.
● Checklists (HHS). You need more, but it’s a start
● Talk to vendors, esp. “I know accessibility” ones
● Hold their feet to the fire on 508 / check with
legal
● Use / Follow #a11y hashtag
● Role-Based Accessibility in Government (PPT)
● Find accessibility testers + 508 template
95. Policy Challenges
o PRA
o Terms of Service
o Procurement (quick!)
o Section 508
Secrets of Gov UX
o Sell it (well)
o Shut up and do do it
o Viva UX Team!
o Get crafty (resources!)
1
2
96. Sell it
sell it - right
1. Always resistance to new ideas, persistence! be a pain in the ass!
2. use real quotes
3. Explain UX but don’t be a jerk (+ use lingo)
4. Share + Create Usability Case Studies
5. #uxgov hashtag
6. Keep asking “What do the Users think?”
7. Share user failures (esp. big ones)
8. Build on a big mistake
9. Prioritize: What’s the most important thing?
10. Precedent: Digital Gov Strategy
97. Get out of the building!
shut up and get out there!
solo or 1-2 devs or others
1. the strategy is delivery
2. Deliver something and point to it!
3. Get real feedback (no hearsay!)
4. Get early / bad things in front of users ASAP
5. get it into strategic plan or development plan
6. Do easy techniques
o User Interviews
o Prototype tests
o Hallway tests
o User Free Exploration
98.
99. Be crafty - get resources
get crafty - find resources
1. Enlist passionate people on operations side
2. Start with fractions
3. Hallway Testing (also sells!)
4. UX Community Listserve
5. Free usability tools + check with CIO / Comms / procurement to see what tools people are using
6. Create your own agency UX community
7. But best of all….
100. Open Opportunities
● GSA program for inter-agency volunteering
● Digitalgov.gov, search “Open Opportunities”
● Create finite tasks or ongoing 20%er
● Me: 100% success in 24 hours or less
● Ideas
o Facilitator
o Script Writing
o Educator
o Speakers, Etc.
http://office.microsoft.com/en-001/powerpoint-help/what-is-presenter-view-HA010360578.aspx?CTT=5&origin=HA010067383
or search for connec tto rpinter and extend
start - connec to projrector - extend
white house new US digital services group, and they launched guides that put the important stuff first!
https://playbook.cio.gov/
Gov.uk Design - https://www.gov.uk/design-principles
CFPB design manual - http://cfpb.github.io/design-manual/
US Digital Service - https://playbook.cio.gov/
18F - https://18f.gsa.gov/
Digital Analytics Program - http://www.digitalgov.gov/services/dap/
bellcurve, what we want most at the top here….
here’s what it looks like in 2014
here’s what it looks like in 2014
and yet, all these global success stories mean jack if you’re stuck and can’t get people to care about or implement UX.
so there are good timers, but it’s not easy street yet.
spoke to a lot of smart peope, and here’s what I have for you today
They give you a number, that goes on printed forms. like your taxes.
one important way to get permission to collect data is...
WH OIRA - Office of Information and Regulatory Affairs
public gets to comment on this
here’s another way of looking at it
do it for a one-shot study, it takes 120 days
let’s say you want to do a lot of similar things.
you can get generic clearance which can let you run other requests under this request and
get cleared FASTER than 120 days
which brings us to…...
while it may not be perfect, can actually be attainable with a little planning.
the agencies who hve good UX are doing this. you can too
so we talked about this way….
5 days or AUTO renewal
so you have an option - take the shorter route! some people do the long route every time!
so you request a number of hours, and then use the hours to conduct user research
so if this is your development cycle, yuo can space out your requests with when you need user research
chat early and often!!
sometimes PRA officers are in CIOs office, sometimes in office of maangement
https://tosdr.org/
people DON’T read TOS. I don’t.
your Lawyers CARE about this
Apply for Free AND paid tools, often sign up before you know it.
applies to all the cool ux tools - uxpin, balsalmiq, usertesting.com, whatever
TOS protects companies from you.
gov caares it because of Legal protections against scary things - ads, fly for court, autorenewals, pay unlimited damages
You can get your agency in trouble for your dumbness (oops)
smaller companies can be easier to work with
huge percent of federal IT is contracted out
however, searches for usabiltiy, ux, user experience in fedbizopps less than 1% of IT projects had thse terms in the solicitations
and guess what - systems get made that are substandard
Granted, it is tough to evaluate UX when you don't have the final product and just have a paper proposal; however, concept PowerPoint slides can be included in proposals.
Additionally, demos of existing products could be offered to customers for "oral presentations".
no ux in, bad ux coming out
Option 1: Prime conducts test
Risky - conflict of interest
“those who build should not do the testing themselves” - maybe it will be biased
If they test, be at all their tests
Option 2: Subcontracted out
Risky - there must be transparency!
all the conversations are between the primary and the subcontractor
If they test, be at all their tests
Option 3: Hire independent, 3rd party ux contractor
get objective info
agency staff may not feel empowered to question boss, or each other
IT compnaies often say they do UX testing as well, but they don’t have the expertise
Option 4: agency conducts own test
Ideal: representative users
we show them what to look for and mentor them so that you can do it yourselfs
now, education and justification is a big hurdle in getting ux into the contract.
some support of contract UX from digital services playbook
https://playbook.cio.gov/
Federal Acquisition Regulation - FAR
http://www.zdnet.com/mission-impossible-white-house-looks-for-new-cio-7000034368/
https://github.com/WhiteHouse/playbook/blob/gh-pages/_includes/techfar-online.md
agile manifesto
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Federal Acquisition Regulation - FAR
http://www.zdnet.com/mission-impossible-white-house-looks-for-new-cio-7000034368/
https://github.com/WhiteHouse/playbook/blob/gh-pages/_includes/techfar-online.md
that’s kind of like saying….
you do user research because we make the mistake of thinking users are like us.
they’re not.
left- blind person using a screen reader.
right - paralyzed person using head wand to navigate a touch screen
imagine the effort required to navigate this way.
inattention to accessibliy compromises quality and the user experience. period.
People with Disabilities = 10%+ of your users!
last 6 months two big lawsuits
especially important around jobs, benefits and contracting
http://lflegal.com/2014/07/july-legal-update/
spoke to a lot of smart people, and here’s what I have for you today
sell it - right
Always resistance to new ideas, persistence! be a pain in the ass!
use real quotes
Explain UX but don’t be a jerk (+ use lingo)
Share + Create Usability Case Studies
#uxgov hashtag
Keep asking “What do the Users think?”
Share user failures (esp. big ones)
Build on a big mistake
Prioritize: What’s the most important thing?
Precedent: Digital Gov Strategy
sell it - right
Always resistance to new ideas, persistence! be a pain in the ass!
use real quotes
Explain UX but don’t be a jerk (+ use lingo)
Share + Create Usability Case Studies
#uxgov hashtag
Keep asking “What do the Users think?”
Share user failures (esp. big ones)
Build on a big mistake
Prioritize: What’s the most important thing?
Precedent: Digital Gov Strategy
You can’t do this alone
RIP- UX hero. VIva UX team
No magic bullet - put in the time (¼ time)
Be Obi Wan - give up your secrets
Meet everyone, cross-team relationships
Key: Analytics people
IT and CIO’s office
Devs
Go as high as you can
Join CX and UX
Have execs try site and top tasks
Staff tests with ANYONE you can
sell it - right
Always resistance to new ideas, persistence! be a pain in the ass!
use real quotes
Explain UX but don’t be a jerk (+ use lingo)
Share + Create Usability Case Studies
#uxgov hashtag
Keep asking “What do the Users think?”
Share user failures (esp. big ones)
Build on a big mistake
Prioritize: What’s the most important thing?
Precedent: Digital Gov Strategy
great way to get free resources!
i know it can be really, really hard to do this… but keep trying. citizens depend on you