An introduction to designing for touchscreens and interactive gestures.

1. Tap is the New Click
Dan Saffer, Kicker Studio

8. DRC 2009 // Dan Saffer, Kicker Studio

9. DRC 2009 // Dan Saffer, Kicker Studio

10. We're using bodies
evolved for hunting,
gathering, and gratuitous
violence for information-
age tasks like word
processing and
spreadsheet tweaking.
—David Liddle

11. We’re in the midst
of an interaction
design revolution.

12. How do we design
for interactive gestures?

13. What we’re going
to talk about
Sensors and touchscreen types
Kinesiology and physiology
Touch targets
Communicating
Choosing appropriate gestures
Case study: Canesta Entertainment Center

14. Gesture: any physical
movement that can be
sensed and responded to
by a digital system
without the aid of a
traditional input device
such as a mouse or stylus.

17. DRC 2009 // Dan Saffer, Kicker Studio

19. DRC 2009 // Dan Saffer, Kicker Studio

21. Two types of
interactive gestures
Touchscreen
aka TUI
Single and multi-touch (MT)
Free-form
Wide variety of forms

22. Why not to have
a gestural interface
Heavy data input
Relies heavily on the visual (for now)
Can be inappropriate for context
More physically demanding

23. Why have a gestural
interface?
More flexible
Less visible hardware
Hardware fits context better
More “natural”
More fun

24. The secret sauce: sensors

25. Common sensors
Pressure
Light
Proximity
Acoustic
Tilt
Motion
Orientation

27. Types of touchscreens
Resistive: pressing two layers together creates the
touch event
Surface wave: finger disrupts ultrasonic waves
Capacitive: finger conducts electricity
Infrared: finger breaks grid of infrared beams
Camera-based: looks for “blobs.” Rear- and front-
projectors

28. Kinesiology & physiology

29. The ergonomics
of human gestures
Avoid hyperextension or extreme stretches
Avoid repetition
Utilize relaxed, neutral positions
Avoid staying in a static position
No “Gorilla Arm”

30. Gorilla arm
Humans not designed to hold their arms in front of
their faces, making small gestures
Ok for short-term use, not so much for repeated,
long-term use
Fun Fact: Telegraph operators had “glass arm”
Sorry, Minority Report-style UIs

31. Gorilla arm

32. Gorilla arm

33. Stephan Pheasant’s (via
Rob Tannen) cardinal
rules of anthropometrics
Reach
Clearence
Posture
Strength

34. The more challenging and
complicated the gesture,
the fewer people who will
be able to perform it.

35. What about
accessibility?
No good, clear answer
Improving via addition of haptics (and hopefully,
eventually, speech)
Some touchscreen systems much better than
traditional WIMP systems
Special care when designing touch targets

38. 16-20mm
8-10mm
10-14mm

39. Fingers
Fingernails: Blessing
and curse
Fake fingernails: evil
Finger oil
Fingerprints
(Left) Handedness
Wrist support
Gloves
Inaccurate (when
compared to a cursor)
Attached to a hand aka
Screen Coverage

40. DRC 2009 // Dan Saffer, Kicker Studio

41. DRC 2009 // Dan Saffer, Kicker Studio

42. Avoid putting essential
features or information
like a label below an
interface element that
can be touched, as it may
become hidden by the
user’s own hand.

43. Touch events and targets

44. Touch target size
Remember Fitts’ Law! (Time it takes to get to a target
= distance to target / size of target)
As close to the user as possible to avoid users’
covering the screen with their hands
Space between the targets (when possible)
Create reasonably-sized targets: no smaller than
1cm in diameter/square (the size of finger pads)

45. Touch target size
comparisons
~25mm ~18mm ~13mm ~8mm 5mm

46. Two touch target tricks
Iceberg tips
Adaptive targets

47. Two touch target tricks
Iceberg tips
Adaptive targets

48. Traditional UI elements
to watch out for
Cursors
MouseOvers and hovers
Double-click
Right-click
Selected default buttons
Undo

49. Touchscreen patterns

50. Tap to open/activate

51. Tap to select

52. Drag to move object

53. Slide to scroll

54. Spin to scroll

55. Flick to nudge

56. Pinch to shrink &
Spread to enlarge

57. Ghost fingers

58. Freeform patterns

59. Proximity activates/
deactivates

60. Move body to activate

61. Point to select/activate

62. Wave to activate

63. Rotate to change state

64. Step to activate

65. Shake to change state

66. Prototyping gestures

67. Low-fidelity:
Paper prototype

68. Low-fidelity: The “man
behind the curtain”

69. Low-fidelity:
Environments

70. High-fidelity: Exact

71. High-fidelity:
Off-the-Shelf

72. High-fidelity:
Do It Yourself

73. Turning gestures
into code
Variables: what are you measuring?
Data: get the data in from the sensor
Computation: determine difference between data
Patterns: what do the sums mean?
Action: if a pattern is matched, do something

74. Documenting gestures

75. Dance notation

76. Annotated wireframes
still work

77. Architectural
wireframes
“Master UI” “Individual UI”
Run by presenter
Live Touchscreen
Projection Area
Used by show attendees
[floor]
[ showing typical arm’s reach for 6’ tall user ] [ showing typical arm’s reach for 6’ tall user ]
touchscreenoverview
[floor]

78. Keyframes

79. Gestural modules

80. Gestural modules

81. Storyboards

82. Swim lanes framework

83. Animation

84. Movies

85. Communicating
interactive gestures

86. Three zones
of engagement

87. Attraction

88. Observation

89. Interaction

90. Attraction affordance

91. Written instruction

92. Illustration

93. Demonstration

94. Symbolic

95. Determining the
appropriate gesture

96. Four part equation
1. The task that needs to be performed
2. The available sensors and input devices
3. The physiology of the human body
4. The context
This can be pretty straightforward
Or not

97. Context
Different behaviors in different locations
Avoiding accidental emotional weight
Cultural issues

98. Usability issues
Avoid unintentional triggers via everyday actions!
Wide variation in performing gestures: need
requisite variety
Pick one: select then action, or selecting does action
Gestures as command keys: Provide a normal means
of performing the action (buttons, etc.) but have
“advanced” gestures as shortcuts

99. Case study: Canesta
Entertainment Center

101. Case study: Canesta
Entertainment Center

102. Case study: Canesta
Entertainment Center

103. Case study: Canesta
Entertainment Center

104. The complexity of the
gesture should match
the complexity of the
task at hand.

105. The best designs are
those that “dissolve
into behavior.”
(Naoto Fukasawa)

106. The best, most natural
designs, then, are those
that match the behavior
of the system to the
gesture humans might
already do to enable
that behavior.

107. Thanks.
http://www.kickerstudio.com
http://www.designinggesturalinterfaces.com
dan@kickerstudio.com
odannyboy on Twitter