My keynote presentation to guide the discussion at the Dayton are event for World Usability Day, November 11, 2010.

1. 1
Closing the gap between
research and practice
World Usability Day - November 11, 2010 - Dayton (area)
Keith Instone
keith <year> @ instone.org
@keithinstone
linkedin.com/in/keithinstone
facebook.com/keithinstone
instone.org
“development”“design”
“user
research”
“scientific
research”

2. 2
#UXRPI

3. 3
Welcome to World Usability Day 2010!
Theme: Communication

4. 4
2009

5. 5
2008

6. 6
2007

7. 7
2006

8. 8
2005

9. 9
2005 web site

10. 10
About me: Professional background
• BGSU > Computer science > Research associate > HCI
> Hypertext > Web usability > Information architecture >
User experience (practitioner)

11. 11
About me: IBM
• IBM > Transformation > CIO > Workforce & Web >
Enterprise Solutions > User experience

12. 12
Last time in this building

13. 13
Get on with it!
http://www.youtube.com/watch?v=l1YmS_VDvMY

14. 14
My current “community service” focus…
Researcher –
Practitioner
Interaction
with purposeful over-use of “interaction”
…instead of UXnet, conference organizing, local groups, …

15. 15
(My) goals for this session
• I am by no means an expert
– I suspect many people in this room have been
addressing this for a lot longer than me
• Facilitator – get you talking about it, then hopefully doing
something about it
• And I am hear to learn from you!

16. 16Did you do your homework?
http://instone.org/wud-uxrpi-prework
If you are a user experience practitioner, what types of
challenges do you face often that you wish you had a
"scientific" answer to?
• Have you tried to find answers in the research literature?
• What roadblocks did you encounter when looking for
answers?
• What successes have you had in taking research
findings and improving your practice?

17. 17Did you do your homework?
http://instone.org/wud-uxrpi-prework
If you are a researcher, what is the value in engaging with
practitioners? What is in it for you?
• Do you have any examples of success stories, where
your research got better because of interactions you had
with practitioners?

18. 18Did you do your homework?
http://instone.org/wud-uxrpi-prework
What should students of HCI, interaction design and other
user experience disciplines be taught about research to
better prepare themselves for the practitioner world?

19. 19
Trend? Fad? Good time to revisit?
• Start: How do we make the CHI 2010 conference better
for user experience practitioners?
• CHI conference bug / feature: the HCI research part
• Answer: Organize a workshop CHI 2010 workshop
• At same time: IA Summit session
• Since then: IUE “panel”, Norman in interactions
• Demarcating UX (Dagstahl)
• How do they all fit together? Dunno. So I volunteered to
give a talk to force me to figure it out.

20. 20
CHI 2010 workshop (SIG)
• This workshop explores whether problems exist between
HCI researchers and practitioners who are consumers of
research
– Yes!
• Articulate factors that may render the research literature
inaccessible or irrelevant to practitioners and to suggest
potential improvements and approaches
– Some
• Learn from researchers how their research could benefit
from practitioner input
– Not so sure yet

21. 21
Workshop questions
• How can the usefulness of research papers be
improved to suit varied audiences?
• How should research be disseminated to different
audiences, including practitioners?
• What are the barriers that discourage practitioners from
adopting research findings?
• How can collaboration between the two sub-
communities be increased for future CHI conferences?
• What should students of HCI and interaction design be
taught about research, to prepare them for the
practitioner world?

22. 22
Snippets from CHI position papers
(or, what people who are smarter than me say)
• Arnie Lund
– Biz sharing with research: “This is too secret for MSR to see!”
– Research collaborating with biz: “Who would get credit for idea?”
• John Karat
– Research papers are for the research community to evaluate
(not really for practitioners anyway)
• Kath Straub
– HFI’s Research Update Newsletter: Yes, practitioners
understand that recent research can speak to relevant issues
• Nigel Bevan
– Trying: UPA Usability Body of Knowledge
– Works: Research-Based Web Design & Usability Guidelines
(produced with US Government funding)

23. 23
Research culture
“Publish or perish”
Answers narrow questions
Open sharing
Experimentation
Corporate culture
“Produce or perish”
Wants broad answers
Strategic advantage
Fear of failure
HCI research culture
Publish for researchers,
not for practitioners
Expanding field
Status within academia
UX practice culture
No time for “research”:
good enough
Rapidly evolving practice
Status within corporations
Immovable objects

24. 24
Research culture
“Publish or perish”
Answers narrow questions
Open sharing
Experimentation
Corporate culture
“Produce or perish”
Wants broad answers
Strategic advantage
Fear of failure
HCI research culture
Publish for researchers,
not for practitioners
Expanding field
Status within academia
UX practice culture
No time for “research”:
good enough
Rapidly evolving practice
Status within corporations
Education
Hard to teach students when practice evolving quickly
Amateur practitioners
Economic model for training
Need more research into how to practice
Knowledge
No shared knowledge base
Hard to organize research for practical use
Multi/inter-disciplinary research & practice
Communication
Little shared language
Speed-of-operation differences
Finding each other
Fragmented professional organizations
Mapping “research answers” and “practical questions”
Bridges: the challenges
WUD 2010 theme!

25. 25

26. 26

27. 27
Information architecture: RPI challenges
• Research and practice in IA are fractured
– With most resources devoted to practice
– One-way ride from university to practice
– You cannot live in both worlds
• Need an IA research agenda…
• …Or, there is no such thing as IA research by itself:
there is HCI, IS, or LIS research that IAs care about
• Many people still confuse “project research” with
“scientific research”
• Can find existing research, hard to make sense of it

28. 28
If Don Norman is talking about it, …
• “The research-practice gap: The Need for Translational
Developers” - interactions, July/August 2010
• See also Kolko (Sep-Oct) “On academic knowledge production”

29. 29
A few “problem statement” quotes from Don
• Immense gap between research and practice
– Gap is fundamental
– Knowledge and skill sets required of each group differ
• Science: hypotheses, conclusions, and evidence
• Practices (of most professions): scientific links are
tenuous at best
– Reliance upon “best practice”

30. 30
Pasteur: fundamental
research aimed at solving
important applied
problems
Researchers most often
play in the fun quadrant,
finding lovely problems to
work on without regard for
whether anyone cares
outside of their fellow
research in-group
Inventors such as Thomas
Edison fit the quadrant of
searching for relevant
knowledge to solve an applied
problem, but without any
attempt to expand our general
understanding of phenomena
A third quadrant is filled with
tinkerers who produce
inventions that neither add
to fundamental
understanding nor have any
use
Applicability
Aimed at
some
practical
problem
Pure science.
No application
in mind
Knowledge
Search for new understanding Apply existing knowledge

31. 31
A new kind of practitioner (new discipline):
the translational developer
– Translate between the abstractions of research and the practicalities of practice
– Translating research findings into the language of practical development and
business
– Translating the needs of business into issues that researchers can address
Abstractions of research
Research findings
Issues to address with
research
Practicalities of practice
Language of development
and business
Needs of business
“stop pretending that researchers and practitioners speak the same language”
(Nov / Dec recap)

32. 32
Mark Newman comments
• Translational skills
– What are the translation skills, who has them, how to
motivate them? Lucrative publishing/consulting career
a la Nielsen?
• Translational research (at NIH)
– Big pots of money DO motivate researchers to shift
their focus. Would it work for HCI?

33. 33
Susan Weinschenk
What Makes Them
Click
Applying Psychology
to Understand How
People Think, Work,
and Relate

34. 34
Danielle Cooley (a practitioner)
To contrast with this,
Mark Ackerman (U of M
SI) drew:
“one head, two bodies”
(researcher body, teacher
body)

35. 35
Derek Poppink

36. 36
Why this is important (Susan Weinschenk)
• Research is power in UX practice
– Too many "urban legends" that people base their
designs on
– Rather than say, "I think that" about a design question
you can say, "A study by xxx showed that..., and
therefore we should do ....."
• There is a lot of misunderstanding by practitioners about
research
– "I need research on XYZ. Why aren't you talking
about that?" (There isn't any research to talk about!)
– "That research is from 2003. It's too old". (If it's valid
research it doesn't necessarily lose truth with age).

37. 37
Why this is hard (Susan, Mark)
• Research papers are hard to read and interpret
– Often poorly written, not have results that are practical
– Results cannot be or should not be generalized
• It takes a special (weird?) person
– To be able to read and understand the papers and
even more special to LIKE doing it
• The best papers may be outside of your field: and each
has its own research style
– “Scientific" model is only one; qualitative social
science is another
– Even researchers have a hard time with this stuff!!
• Research may not give you answers
– Best research results in more questions than answers

38. 38
Demarcating User eXperience (workshop)
September 15-18
• UX is seen as a holistic concept covering all aspects of
experiencing a phenomenon, but we are facing the point
where UX has become a concept too broad to be useful
in practice.
• Practitioners have difficulties to understand the concept
and to improve UX in their work, and researchers rather
use some other term to make their research scope clear.
• Net problem: Do not even have a clear core concept to
bridge the gap between research & practice
• Goal: Write a “white paper” to define UX for a certain set
of cases that make its scope clear

39. 39
White paper so far…
http://www.dagstuhl.de/Materials/index.en.phtml?10373
• (35 definitions of “user experience”….)
• Describes the core concepts of user experience and
distinguishes UX from related concepts
• Rooted in use
• Subjective
• Dynamic (changes over time)
• Designing for a desired experience (and evaluation to
see how close we are getting)
• Elizabeth Buie: "If the practitioners can't use it [the white
paper], it doesn't work!"

40. 40
PAUSE

41. 41

42. 42
Ideas for solutions
• <fill in this page>

43. 43
CHI 2011
• Editing “benefits” statements for the program
– Add “practitioner take-aways” to the program
• Awards: best case study, impact for research paper
• Community leadership
– Design
– Engineering
– Management
– User Experience
– Health
– Games and entertainment
– Sustainability
– Child-computer interaction

44. 44
Research culture
?
Corporate culture
Improve UX status
HCI research culture
Grand challenges
UX practice culture
?
Education
Training for managers
Knowledge
SIGCHI collaborate on UPA BoK
Communication
Brokerage system to connect R & P with
common interests
Translators between R & P
Face-to-face meetings (to start building
long-term relationships)
Social media
Ideas / Solutions
(very incomplete)

45. 45

46. 46

47. 47
(New) Usability BoK
• Living reference that represents the collective knowledge
of the usability profession
• Derived from published literature, conference
proceedings, and the experiences of practitioners (guide)
• Some goals (related to UXRPI):
– Define the knowledge underlying the usability
profession (methods, knowledge, and skills)
– Facilitate professional development, curriculum
development
• Users: practitioners, educators & researchers, …

48. 48
Next steps?
• Comment on Norman’s interactions article
• Tweet something with #uxrpi
• Draw / post sketch to Flickr – tag uxrpi
• Volunteer @ usabilitybok.org
• More details / links at instone.org
• Pick an idea from today’s discussion: let’s try it!
• Contact me: keith <year> @ instone.org - @keithinstone