90 minute training for experienced practitioners in best practices for analyzing and modeling qualitative user research, including KJ Analysis, personas, and scenarios. Tips and tricks and techniques included. Presented at the STC Summit 2010 on 3 May 2010.

1. Understanding Users Through Ethnography and Modeling
Jim Jarrett
Senior User Experience Architect and Manager
MedAssurant™
STC Technical Communication Summit 2010
3 May 2010
5:00-6:15pm
Reunion C

2. Understanding Users Through Ethnography and Modeling
Great product design starts with a deep understanding of the work
that users do in the real world. Build your understanding through
observation, interviews, surveys, and artifact collection. Share your
understanding with KJ analysis, personas, and scenarios. Validate
and prioritize your understanding with follow-up surveys.
 Introduction and Conference Context 5 minutes
 Ethnography and Data Collection 20 minutes
 Modeling and Validation 30 minutes
 Tips, Tricks, Tools, and Resources 10 minutes
 Q&A 10 minutes

3. Who is Jim Jarrett?
Senior User Experience Architect and Manager
MedAssurant™
 15 years at Rockwell Automation
started as a multimedia developer in 1991
other companies: Chiron, Roadway, IDD
 Dozens of on-site observations in domains of medical
informatics, industrial automation, and medical
diagnostics.
 Largest single user research project:
30 sites on 3 continents in 7 countries
across 3 trips totaling 18 days on-site
 Design for Six Sigma Blackbelt at Rockwell
$2 million+ in hard savings over 2 years
$24 million+ in revenue growth in same period
 Multiple patents in UI for industrial automation
jim@jarrettinteractiondesign.com
www.JarrettInteractionDesign.com
LinkedIn.com/in/jarrettinteractiondesign
Twitter.com/JarrettUX
immersed in context

4. Ethnography and Modeling
Sketching
Expert Inspections
Usability Evaluation
Usability Institute
“Understanding Users Through Ethnography and Modeling”
Monday 5:00-6:15pm Reunion C
“Discovering Usability Defects Through Expert Inspections” with Rich Gunther
Tuesday 8:00-9:15am Landmark B
“Improving Product Design Through Usability Evaluation” with Scott Butler
Tuesday 9:45-11:00am Reunion F
“Sketching User Experiences with the Design Studio Method” with Brian Sullivan
Wednesday 8:00-9:15am Reunion B
Inception
Elaboration
Construction
Transition

5. “… truth will sooner come out of error
than from confusion.”
Sir Francis Bacon
“Better wrong than vague!”
“An articulated guess beats an unspoken
assumption.”
Fred Brooks

6. Define Ethnography
ethno folk, people, nation
graphy writing, study of
“The scientific description of nations or races
of men, with their customs, habits, and
points of difference.”
OED
“The study and systematic recording of
human cultures.”
Merriam-Webster
 Qualitative, language-based, empirical,
descriptive research.
 Techniques include observation,
interviews, surveys, logs and journals,
gathering of artifacts and images.
 Holistic and contextual.
“I prefer design by experts - by
people who know what they are
doing.”
Don Norman
“… pay attention to what users do,
not what they say… ”
Jakob Nielsen
“A computer shall not harm your
work or, through inaction, allow
your work to come to harm.”
Jeff Raskin
“What do users do and how do
they talk about it?”
Jared Spool

7. Market Research vs. User Research
Market Research
 Markets and segments
 Best ways to influence
 Target demographics
 Buying behavior
 Message definition
 Focus groups, surveys,
and interviews
 Competitive analysis
“Why do people buy?”
User Research
 Individuals and context
 Best ways to work
 User profiles
 Goals and motives
 Task definition
 Observation, surveys,
and interviews
 Domain analysis
“What do people do?”

8. Data Collection Process
1. Define the target of your research.
 What questions are you trying to answer?
 Who should you study?
2. Arrange visits.
 Get permissions and release forms.
 Schedule timing for your targets.
 Coordinate with sales and marketing.
3. Prepare your site teams.
4. Two sites at a time, two observers at a time.
 Introduce, observe, interview, wrap-up, thanks.
 Collect artifacts and notice context.
 90 minutes.
5. Process with project team.
 Immerse them in the experience and data.
6. Follow up.
 Thanks + additional questions.
7. Retarget.
8. More visits, but not too many!

9. Example Observation 1
“I hate this [expletive deleted] history screen! I have to jump all
around and read everything to find out the patient’s latest A1C
measurement and see if she’s visited the pedorthist recently.”
Option 1
“How can we improve the history
screen?”
Option 2
“I noticed you came to this screen a
couple times while you were talking
to the patient.
What were you looking for?
Can you walk me through your
thinking?”
Don’t ask users what they want, watch what they do.
Users are not designers, and you are not a user.

10. Example Observation 2
During the discussion with the patient, the nurse refers to a small spiral bound
note pad from time to time, sometimes making a quick notation. She also jots
some short notes on a piece of scrap paper. While you are paying attention to
those activities, you miss some of the conversation with the patient.
Option 1
“Can you repeat what you said to the
patient? I missed some of the
details.”
Option 2
“While you were talking with the
patient, you looked at your
notebook and sometimes wrote
something. Other times, you wrote
on the scrap paper.
What were you writing down?
Why in one place versus the
other?”
Synthesize a big picture with rich, textured detail.
Invest in more research, not post-processing.

11. Example Observation 3
BUT!
 Get permission. Don’t violate policies, regulations, or laws.
 Get a signed release form if necessary.
 Don’t let the camera get in the way of really observing the work.
bunch of reference
material printed from
the web
support, clinic, and
social worker numbers
calendar with both
personal and work
events
brand name vs generic
reference for common
drugs
SOPs and
comprehensive clinic
and phsycian contacts
today’s call list
monitor turned to
allow photo; avoid
HIPAA issues
ID badge worn at all
times
follow up notes from
last call
A picture is worth a
thousand memory slips.
Triggers rich memories
from visit, immediately
afterward when
processing the data as well
as long term.
Provides rich, visceral
context for team members
and stakeholders who
weren’t there.
Particularly memorable
ones can be used for
personas.

12. Data Collection Challenges
 Confidentiality, privacy, and trust.
HIPAA, intellectual property, security, legal, regulatory.
 Complex environments.
International sites, hazardous work, longitudinal or rare work.
 Redundant data.
Repetitive work, rare exceptions, awkward times.
 Too much data.
Many sites, trip logistics, team availability.
 Exhaustion.
This is exhilarating work, but can be grueling.

13. Define Modeling
“A systematic description of an object or
phenomenon that shares important
characteristics with the object or
phenomenon.”
American Heritage Science Dictionary
“A simplified or idealized description or
conception of a particular system, situation, or
process… that is put forward as a basis for
theoretical or empirical understanding… a
conceptual or mental representation of
something.”
OED
 Concrete and detailed representations of
observations.
 Purposefully lossy: generalized but retaining
meaningful details.
 Domain models include KJ Analysis, Personas,
Scenarios and a variety of others.
“models can frame the design
problem… help designers
understand the domain”
Jared Spool
“The most effective behavioral
models are distilled from
interview and observation data of
real users into an archetypal
description of how a particular
type of person behaves and what
their goals are.”
Kim Goodwin
“A mental model is a picture of
how your end users are
supported by what you are
creating.”
Indi Young

14. KJ Analysis
“Affinity on steroids.”
Outcomes:
 Shared understanding
 Relationships
 Priorities
Process steps:
1. Define the question to be asked of the data.
2. Reduce the data set to 30 or fewer items
through multiple rounds of simple voting.
3. Group related items.
4. Title the groups.
5. Group related groups.
6. Create headlines for the higher level groups.
7. Visually lay out the groups with all data items.
8. Show relationships between groups.
9. Vote for the most important title‐level groups.
10. Draw a conclusion from the diagram.
We don’t have a clear definition of
what a requirement is
No clear definition of what
the difference levels of
requirements are
There seems to be
a discrepancy on
how detailed the
rqmts need to be.
I can’t delineate
between BR’s and
MR’s.
It’s not clear
where a market
requirement stops
and an SR starts
I don’t understand what the customer
needs; we don’t understand what is
important to deliver
I like to be involved in the whole process
All roles should be
involved throughout the
entire process
VOCs are very valuable
Is seems like we are
spending a lot of time putting
the same data in multiple
places and coordinating to
ensure the data is in sync
The process shouldn’t impede the
progress of the project
We’re going to
need to do some
clarification of what is
design and what is a
requirements
We don’t have a
good definition of
what a
requirement is
It seems like different
RAs have a different view
of what a requirement
should be
With requirements at
a high level it is difficult
to come up with a
good estimate
Perhaps we should
have two different
levels of use cases
We don’t know what “good
enough” is
How do you define
success at the end
of each iteration
We have lost the
concept of critical
release
requirements
We aren’t validating what
we are building with the
customer
Need to get out of the
lab exercise of I know
what the answer is now
I need to go find a
customer to tell me they
have that problem
Need to know when
we come out the other
end, have we built the
feature that is necessary
for the customer.
You better think about
performance
requirements before you
start down the design
path
Need a better way
to bring customer
feedback in earlier in
the process
VOCs are very
valuable for
providing focus to
product issues
VOCS were very
valuable which gave
us a very good
process and means
to rank and prioritize
requirements
Get engineering and
test involved earlier on
before the rqmts are
finalized
The requirement
analyst needs to
own the rqmts all the
way through the
process.
It’s difficult to look at
requirements in
ReqPro so everyone
gives up
I like the ability to
add attributes (Req
Pro)
Requirements tools are
difficult to work with
A lot of things you test
for will not show up in
requirements for
instance failure modes
If you read only the
SRs it is difficult to
understand what the
hell it is supposed to
do.
Hundreds of hours have been spend on
requirements with very specific detail for
consistency. The gap we have is that how
are we going to do a consistent design
without that level of detail
If the requirements don’t
have the detail I need,
where do I get them?
When we took UI detail
out of the use case it
wasn’t clear what the
rqmt really meant
Where do we put this
extra UI information
It’s nice to have implementation details
as an example to understand what the
RA was thinking
UI Information is a good
way to communicate ideas
Anything so that people
aren’t waiting for other
people to get stuff done
We should not have
to have everything
written down and
signed off to hand it
over the wall
No good way ot
fimplementing and
managing features access
products
We need to find a
better way to illustrate
requirements on a
features basis
All requirements
need to be
coordinated across
products
The process shouldn’t
impede the progress of
the project
To be honest, it’s to
painful to do a change
I don’t know where to go to get the information I need
Theme: Gaining a better understanding of the
requirement processes and how customers should be
involved
Lessons Learned: Better defections is needed for the
levels of requirements and where to get requirement
details,

15. Anatomy of a KJ Diagram
Headline
Relationship
cause and effect or contradiction
Title
Raw statement
Title with most votesTitle with second most votes
Title with third most votes
Question to ask?
Summary answer.
Time/date
Place
Participants

16. Personas
“The aspect of a person's
character that is displayed to
or perceived by others.”
OED
Tells a story
Concrete and specific
Memorable
Day-in-the-life narrative
Goals, tasks, and other
behavior
Skills and demographics
“Personas are helpful in creating and
iterating a design, building consensus,
marketing the product, and even
prioritizing bug fixes.”
Kim Goodwin

17. Persona: Identity
Memorable name with role or
user type
Memorable quote that captures
the essence of persona’s goals
and attitude
Memorable photo that strongly
represents persona and role

18. Persona: Narrative
 Detailed and specific
 Concrete
 Summarize a typical work day.

19. Persona: Skills
 Identify skills that
differentiate the
different personas.
 Rate the persona’s skill
level for each.
 Present visually and
simply.

20. Persona: Details
 Extend, generalize, and add specifics to narrative.
 Enumerated and prioritized.
 More analytical and processed than narrative.

21. Persona Quality
P Primary research
Is the persona based on contextual interviews with real customers?
E Empathy
Does the persona evoke empathy by including a name, a photograph and a
product-relevant narrative?
R Realistic
Does the persona appear realistic to people who deal with customers day-to-
day?
S Singular
Is each persona unique, having little in common with other personas?
O Objectives
Does the persona include product-relevant high-level goals and include a
quotation stating the key goal?
N Number
Is the number of personas small enough for the design team to remember the
name of each one, with one of the personas identified as primary?
A Applicable
Can the development team use the persona as a practical tool to make design
decisions?
http://www.userfocus.co.uk/articles/personas.html

22. Scenarios
“An outline or model of an
expected or supposed sequence of
events.”
American Heritage Dictionary
Envision the future.
Personas drive the action and
point of view.
Begin with a triggering event,
describes a sequence of actions,
and the results of those actions.
Technology and solution-free.
A few key scenarios are better
than a comprehensive set.
“A scenario supplies the context-
of-use necessary to set the
stage on the subtlety and
nuance you'll need to get the
mix just right.”
Jared Spool

23. Scenario Example
First step is the trigger that
begins the scenario
Step by step, who does what,
when, and why
Call out questions or
observations each step brings
to mind

24. Scenario vs. Use Case vs. User Story
A scenario is a concrete,
sequential narrative about a
specific workflow.
A use case is a generalization
of a number of scenarios and
can generate multiple
scenarios.
A user story is a single
statement that captures who,
what, and why. Usually
names a use case or scenario,
or is a single step in one.
“An outline or model of an
expected or supposed sequence
of events.”
American Heritage Dictionary
“… a collection of possible
sequences of interactions
between the system under
discussion and its external actors,
related to a particular goal.”
Alistair Cockburn
“A description of desired
functionality told from the
perspective of the user or
customer.”
Mike Cohn

25. Other Models
Contextual Inquiry
 Flow Models
 Sequence Models
 Artifact Models
 Cultural Models
 Physical Models
Organizational Models
Process Models
Stakeholder Models
Concept Models
 Choose models that
communicate the most
important information to the
most important stakeholders.
 Keep the raw data around for
many purposes:
 Explain or justify the models.
 Future data mining.
 Explorable body of knowledge.

26. Validate the Models
 Validate KJ priorities with
surveys.
 Rate and rank.
 Much larger audience.
 Validate personas with
interviews.
 “Which do you identify with?”
 “What parts don’t work?”
 “Describe a day in your work life.”
 Iterate throughout the
lifecycle and across multiple
projects.
 Over-communicate the
models and advocate their
use.
surveymonkey
Yahoo! personas poster from article at UIE.com

27. Tools, Tips, and Tricks of the Trade
 Get really good at taking notes while
watching and talking.
 Don’t use a laptop.
 Don’t bother recording.
Do another visit instead.
 Exception: Livescribe Pulse Smartpen
 Take lots of pictures (if permitted).
 “Revolving Door” technique.
 Leah Rader and Beth Toland
 Card sorting.
 Information Architecture
 Controlled Vocabularies
 Journaling.

28. What To Do with All This Research
 Share it!
Accessible repository, posters, walkthroughs with stakeholders,
conversations with SMEs, users, and customers.
 Invent new products.
How could current work be transformed or eliminated?
 Frame all downstream work.
Use cases, UI designs, and test scenarios based on the real world.
 Validate the system.
Does it support real people doing real work? Does it help them achieve
their goals?
 Simplify the system.
Eliminate functions and workflows that don’t support the work.
 Discover new avenues.
What more research paths are opened up?

29. Resources
The Design of Design: Essays from a Computer Scientist
Frederick P. Brooks, Jr.
Contextual Design: Defining Customer-Centered Systems
Hugh Beyer, Karen Holtzblatt
Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design
Karen Holtzblatt, Jessamyn Burns Wendell, Shelley Wood
Commercializing Great Products with Design for Six Sigma, chapter 16 “KJ Analysis”
Randy C. Perry, David W. Bacon
Designing for the Digital Age: How to Create Human-Centered Products and Services
Kim Goodwin, Alan Cooper
The Persona Lifecycle: Keeping People in Mind Throughout Product Design
John Pruitt, Tamara Adlin
AgileProductDesign.com
Jeff Patton
Writing Effective Use Cases
Alistair Cockburn