Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 101

Acting naturally: why design needs ecological psychology

2

Share

Download to read offline

Presented at UX Week 2017

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Acting naturally: why design needs ecological psychology

  1. 1. Marsha Haverty Acting naturally @mjane_h UX WEEK 2017 Why design needs ecological psychology
  2. 2. Useful Clear Accessible Usable GOOD Resilient As a field, we have a lot to say about the qualities a design should have to support a good experience. Yet, the raw materials of our designs are increasing greatly in variety.
  3. 3. VARIETY Haptic hologram model image: ykaradayi via Instructables How information manifests e have more variety in how the information manifests. n addition to more traditional mixes of text and images, we have data visuali ations, holograms that occupy our physical surroundings, alerts to state changes in the form of mechanical vibrations and sounds.
  4. 4. VARIETY How information manifests How actors engage Woman hailing drone: Mercedez-Benz ; Haptic hologram model image: ykaradayi via Instructables; Girl with Amazon Echo: Android Central e have more variety in how actors engage. hings li e gesturing, sculpting directly in physical surroundings, tal ing aloud.
  5. 5. VARIETY How information manifests How actors engage Autonomy & intelligence Offload Assist Learn e have more variety in the autonomy and intelligence of our designs. esigns that go out and do or learn things on behalf of actors and then report bac . eigns that accelerate actors activities, or go out and learn and bring that bac to the actor.
  6. 6. Mode switching across contexts Voice CommandsGesture Touch Mouse Text SoundsDigital Agents Visualizations Mechanical Vibrations Assistive Technology Machine Learning Simulated Objects e are experimenting with mode switching to ta e our actors on more complex ourneys across contexts. aybe the actor types in some text in one situation, then, later issues a voice command, or taps in a selection, within the same meaningful activity.
  7. 7. Mode switching across contexts Gesture Touch Mouse SoundsDigital Agents Visualizations Mechanical Vibrations Machine Learning Simulated Objects Voice Commands Text Assistive Technology aybe a simulated ob ect will issue a little sound to indicate it has learned a new capability, and the actor performs a little gesture to say, reat, show me later.
  8. 8. Mode switching across contexts Gesture Touch Mouse Visualizations Mechanical Vibrations SoundsDigital Agents Assistive Technology Machine Learning Simulated Objects Voice Commands Text And many other mode combinations that we are wor ing out across more complex activities.
  9. 9. Image: DMITRIS_K Big Think Can’t see why with usual principles USEFUL CLEAR ACCESSIBLE USABLE RESILIENT ome of these experiences feel quite natural, once our actors get the hang of them. And others feel li e novelties they don t really want to be doing for very long. And we can t really see why with our usual principles.
  10. 10. Image: DMITRIS_K Big Think …NATURAL Something deeper still… here s something deeper still about us as humans that ma es an experience feel natural or distinctly off.
  11. 11. New vs. Familiar Simulated vs. Physical Intelligent vs. Mundane Photo by permission: Hugh Hochman hat do we mean by an experience that feels natural t doesn t depend on whether the interaction is new or familiar, it doesn t matter whether the information is physically present or simulated, it doesn t matter whether the design has some agency and intelligence, or is mundane. As humans, we re really good at assembling our actions in new ways. here s something deeper still that ma es an experience feel natural or distinctly off.
  12. 12. The way we, as humans, rely on information in our surroundings to behaveEcological Psychology Photo by permission: Hugh Hochman he field of ecological psychology shows us that natural behavior is about the way we as humans rely on information in our surroundings to behave.
  13. 13. Human brain connections image: Barcroft Media via Daily Mail And that has to do with the way we thin about these beautiful brains that we wal around with.
  14. 14. Behavior Surroundings Traditional cognition raditional cognition says we ta e in information from our surroundings, then our brains do all the wor to process that information and derive meaning and decide actions, and that is output as behavior.
  15. 15. Embodied cognition mbodied cognition, the style that ecological psychology is organi ed around
  16. 16. Embodied cognition suggests that meaning emerges as we engage the information in our surroundings directly. nformation is not an input, but a vital participant in human behavior. e depend on it in order to behave.
  17. 17. We humans rely on information in our surroundings to behave Two fundamental things all human behavior is about Each relies on different properties of information Ecological Psychology Photo by permission: Hugh Hochman e humans rely on information in our surroundings in order to behave. hen information has the right properties we are relying on, the behavior feels natural. hen those properties are missing, the behavior feels off. rom some of the earliest thin ing in ecological psychology, and some very recent thin ing, we can see that there are two fundamental things all human behavior is about. And each relies on different properties of information to facilitate natural human behavior.
  18. 18. Designs have properties to facilitate user experiences that feel natural Two fundamental things all human behavior is about Each relies on different properties of information Ecological Psychology Photo by permission: Hugh Hochman e can use this as designers to create designs that have properties to facilitate user experiences that feel natural.
  19. 19. Two fundamental things all human behavior is about Each relies on different properties of information AFFORDANCES Photo by permission: Hugh Hochman n order to see these things, we have to be very clear about what we mean by affordances.
  20. 20. AFFORDANCES Button sketch: Demodern; piano photo: CC0 free use license “Perceived affordance for action” “Real affordance for action” n design, we we tal about something li e our humble wor horse, the button, we hear things a button has a perceived affordance for action as compared to a physical button that has a real affordance for action. e place all the emphasis on the distinction between perceived and real, yet say they both prompt action.
  21. 21. AFFORDANCES Button sketch: Demodern; piano photo: CC0 free use license “Perceived affordance for action” “Real affordance for action” cological psychology shows us that the two inds of actions are very different, and each relies on very different properties of information. n this distinction, we get to the two fundamental things all human behavior is about... ut, to get there, we need to ta e affordances bac ....
  22. 22. All the way bac o the s.
  23. 23. Photo Sverker Runeson James Gibson And ames ibson, the founder of ecological psychology. e coined the term affordances in .
  24. 24. Photo Sverker Runeson ibson captured his concept of affordances in the boo , An ecological approach to visual perception. he core of affordances to ibson is the ob ect offers what it does because of what it is.
  25. 25. Image: A Chair in a Room, Wolf & Wood Interactive, Ltd. o see what he means by this, we can loo at a chair in virtual reality.
  26. 26. As humans, we will naturally try to sit on a chair in virtual reality Image: A Chair in a Room, Wolf & Wood Interactive, Ltd.
  27. 27. Aluminum chair image by Emeco; coconut chair image by RoveConcepts; all other chair images free use Surfaces Edges Textures Invariant Structure Perceive affordance Engage affordance ur whole lives, we have become attuned to the properties of ob ects that are sittable to us. e pic up on the relationships in the way the surface of the ob ect is proportional to us, the relationship between the edges of the surface and the support legs, the relationship of the support legs whether , , or to the ground. ot only can we perceive the affordance in invariant structure, we can engage it because part of that information is about the substance and sturdiness of the chair. e rely on that information to help us control our physical action of sitting.
  28. 28. Back in virtual reality… Image: A Chair in a Room, Wolf & Wood Interactive, Ltd. Sittable-to-me PERCEIVE AFFORDANCE Control physical action of sitting ENGAGE AFFORDANCE ac in virtual reality perceive that this ob ect is sittable to me. Yet, when go about relying on this information to control my physical action of sitting, it s not there
  29. 29. "It is important to remember that simulated objects, such as furniture, that may be encountered while using the product do not exist in the real world, and injuries may result when interacting with those simulated objects as if they were real, for example, by attempting to sit down on a virtual chair" VR manufacturer user guide e can tell people not to sit....
  30. 30. “There's no physical furniture to break your fall when you try to sit in a virtual chair. Someone asked, Where did that bruise come from? I achieved presence.” VR game player ut game players report things li e....
  31. 31. Photo Cornell University ames ibson would tell us that s not really about presence or immersion. t s about this fundamental way that we as humans rely on information to behave. e humans are attuned as sitters. And sitters are gonna sit.
  32. 32. Affordances PerceivingAnticipating ActingAligning Invariant Structure Attunement Affordance LAYOUT & OBJECTS Reference: Turvey, Michael (1992). Affordances and prospective control: an outline of the ontology, Ecological Psychology (4)3 173-187 Let’s capture all of this in a diagram. The important point here is that we rely on the affordance both for perceiving meaningful action and to help us control our actions unfolding over time. Text
  33. 33. Affordances Continuously orienting physical actions PerceivingAnticipating ActingAligning Invariant Structure Attunement Affordance LAYOUT & OBJECTS Reference: Turvey, Michael (1992). Affordances and prospective control: an outline of the ontology, Ecological Psychology (4)3 173-187 Engaging affordances is about continuously orienting our physical actions. That’s what affordances are for. This places soem pretty hefty requirements on the properties of information that can become affordances.
  34. 34. Dense, persistent, lawful Visual Information Surfaces Edges Textures Free use image Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 hen we are wal ing on a trail, we rely on visual information to both perceive that the trail is wal able to us, and to go about the action of wal ing. he surface of the trail, the edges that help define trail not trail, the textures that let us now that is wal able, must be dense there is a lot of visual information to help us pic up the layout of our surroundings, and persistent
  35. 35. Free use image Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 Dense, persistent, lawful Visual Information Surfaces Edges Textures if the visual infomration is blotted out, we can no longer go about wal ing on the trail....
  36. 36. Dense, persistent, lawful Visual Information Surfaces Edges Textures Free use image Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 and lawful once we are attuned to the affordances that this trail is wal able to us, we need to be able to rely on that as we engage these affordances. he information is about what this layout is made of. e can rely on it.
  37. 37. Affordances PerceivingAnticipating ActingAligning DENSE PERSISTENT LAWFUL PERCEPTUAL INFORMATION Attunement Invariant Structure Affordance LAYOUT & OBJECTS Continuously orienting physical actions Reference: Turvey, Michael (1992). Affordances and prospective control: an outline of the ontology, Ecological Psychology (4)3 173-187 We see that the information to support engaging affordances has to be dense, persistent, and lawful. But it also has to be made of perceptual information. Language is not dense or persistent enough to rely on.
  38. 38. Straighten the glass! Turn down the water! Free use image e can micromanage a toddler trying to fill a glass without spilling all we li e, but the toddler can t rely on our words to control her action unfolding over time. he has to become attuned to perceptual information the alignment of the edges of the glass to the edges of the water flow, the width of the water flow related to the surface rising in the glass. And once she becomes attuned to these perceptual relationships, they will serve as affordances for her filling a glass without spilling
  39. 39. Macro photo of Kindle Reader e-ink; vocal tract anatomy Bruno Dubuc SPOON Meaning by convention Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 Concepts are different hat s because concepts are different. hen we read a statement about equality or hear someone tal about a spoon, the meaning is not about the underlying dynamics that created the information. he meaning of the concept equality is not about the properties of the in that created the letters in the text. he meaning of the concept spoon is not about the function of the larynx from the person that uttered it. he meaning of language is by convention. e agree on conventional meaning as part of our culture and that functions as reasonably stable. As humans, we rely on very different properties of language to engage with it than we do with affordances.
  40. 40. Perceive invariant structure? Control physical action over time? o what about our button oes this ob ect offer what it does because of what it is an our actors perceive this invariant structure Yes utton, clic able, got it an actors rely on this information to control their physical actions over time in the act of engaging the button
  41. 41. Image by Steelcase hen an actor is clic ing or tapping a button, she can physically be doing any number of things sitting, lounging, wal ing, etc. f course some physical action on her part is needed to get that cursor or her finger over that button to clic it. he angle and tra ectory of her arm, the position of her wrist and fingers can ta e on a whole range of tra ectories to do the meaningful action of engaging the button. he s relying on the visual information in the button to now it s clic able, but she s not relying on that information to control her physical actions over time as she clic s the button. he way she is physically orienting is not where the meaning lies.
  42. 42. Orienting physically *is* the meaning of the activity Tilt Brush example by Google on https://www.tiltbrush.com/ go watch this animation online to get the idea here. iltbrush.com ompare that to this this woman is s etching in a . he particular way she is orienting her hand, her wrist, her arm, her whole body the meaning of the activity of s etching directly in her surroundings. he first bit of light she creates then becomes affordances she relies on continuously to orient herself physically in order to continuously create her curves of light. hat s what a s etch is. very curve is with respect to the existing curves. he is relying on this information to control her particular action unfolding over time.
  43. 43. This object offers what it does by convention A concept infused with visual qualities Information visualization of the concept of submitting, saving, ending, …. his ob ect offers what it does not because of what it is, but by convention. t is a concept infused with visual qualities. t s an information visuali ation of the concept of submitting, or saving, or ending. epending on what that action means. t s a very different ind of action than engaging affordances. t relies on information in a different way.
  44. 44. ConceptsAFFORDANCES e ve ust gone to a lot of trouble to pull concepts bac out of our notion of affordances.
  45. 45. Concepts So picky!!! e re being so pic y now
  46. 46. Concepts Fundamental Human Behavior Sabrina Golonka Ecological Psychologist Image by Reality Reporter ut we need to do this in order to tal about fundamental human behavior. A present day ecological psychologist, abrina olon a, gives us this understanding.
  47. 47. Concepts Fundamental Human Behavior Controlling action unfolding continuously over time Selecting which action to engage Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 e re being so pic y to separate out concepts from affordances, even thought it s historically been useful for us to combine them, so that we can see the two fundmental things all human behavior is about electing which action among possible actions to engage, and controlling action unfolding continuously over time. ach of these rely on different properties of information.
  48. 48. Concepts Fundamental Human Behavior Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 ACTOR AS CONDUCTOR ACTOR AS SCULPTOR hen we support these behaviors in design, call selecting action with concepts Actor as Conductor. And controlling action with affordances, Actor as Sculptor.
  49. 49. ACTOR AS CONDUCTOR Selecting Selecting Selecting Selecting Gif of Alondra de la Narra: http://indieclassical.tumblr.com/post/138392136778/exercisesinhumiliation-alondra-de-la-parra-is hen we thin about using a website to find things, the actor creates a search this is selecting action. he system returns results and as the actor aligns with the meaning of the results, she may ad ust her search, selecting an action again. he system returns the new results and she aligns with the meaning of those. he may select a filter selecting another, each time she selects, she bases her next selecting on aligning with the meaning of the results of the system.
  50. 50. ACTOR AS CONDUCTOR Selecting Selecting Selecting Selecting Software image via Autodesk Fusion 360Gif of Alondra de la Narra: http://indieclassical.tumblr.com/post/138392136778/exercisesinhumiliation-alondra-de-la-parra-is imilar with software the actor is selecting actions on a property panel for a command. hen she selects areas of the design on which to apply the command, then she selects to execute the command selecting, selecting, selecting.
  51. 51. ACTOR AS CONDUCTOR Cascade of iterative selections … The action of selecting action Gif of Alondra de la Narra: http://indieclassical.tumblr.com/post/138392136778/exercisesinhumiliation-alondra-de-la-parra-is he ind of action involved in actor as conductor is a cascade of iterative selections. t s the action of selecting action. his is a weird ind of action we support
  52. 52. Tilt Brush example by Google on https://www.tiltbrush.com/ Orienting physically *is* the meaning of the activity ACTOR AS SCULPTOR Pottery gif by permission Cherrico Pottery LLC et s loo at actor as sculptor. As we ve seen actor as sculptor is about the particular way an actor is physically orienting over time the meaning of the activity. i e this woman creating a s etch.
  53. 53. ACTOR AS SCULPTOR Pottery gif by permission Cherrico Pottery LLC Relying on affordances to control the way physical actions unfold over time Actor as sculptor is relying on affordances to control the way physical actions unfold over time. his is a very different ind of action compared to actor as conductor a cascade of selecting actions .
  54. 54. Industrial design of a car supports both he industrial design of a car supports both, actor as conductor, and actor as sculptor. he visual information streaming through the windshield becomes affordances for actor as sculptor to eep the car on the road, and in the lane, and moving around other cars. he driver is pic ing up on invariant structure in the surfaces and edges of the road and the lane, and the angle relationships of the her car to the other cars. f a car grows symmetrically, she becomes attuned to that meaning the car is coming right at her she ad usts her actions to create asymmetric angle growth the car going off to her side . As drivers, we become highly attuned to this perceptual relationships and rely on that to drive the car. he speedometer supports actor as conductor a glance at her speed and she can use this information to select to slow down.
  55. 55. ACTOR AS CONDUCTOR Woman hailing drone: Mercedez-Benz ; Girl with Amazon Echo: Android Central his woman is gesturing to control a drone. he gesture is a particular physical motion, but the meaning of the gesture is not in the exact way the actor executes it. As long as she gets the gist of the gesture, the meaning is engaged. his girl is tal ing aloud to find something or as for something or buy something or answer a question. he is selecting actions. Also actor as conductor, ust with voice.
  56. 56. Image via Autodesk Pier 9 Residency Program ACTOR AS SCULPTOR his actor is sculpting in physical space with a hologram. he wand he is using is wired up to give haptic information that he can rely on to now where the surfaces of his design is in place. e can rely on that to continuously control the position of the wand in order to sculpt in .
  57. 57. ACTOR AS CONDUCTOR Image via Autodesk Revit Live eleporting in is conducting. he actor selecting where to go next. electing, selecting, selecting.
  58. 58. Image: Google Soli project ACTOR AS SCULPTOR his is an experiment using radar to very precisely trac an actors fingers. he actor can mime turning a dial, pressing a button, moving a slider. his forms a little micro bit of actor as sculptor because the actors fingers become perceptual information for each other to rely on to control action unfolding continuously over time.
  59. 59. Action Perception Actor as Sculptor Performing physically for a good continuation Aligning Anticipating DESIGN Attunement PERCEPTUAL INFORMATION Properties Invariant Structure Affordance DENSE PERSISTENT LAWFUL Reference for properties: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 We looked at the properties of information to support engaging affordances earlier: it must be dense, persistent, and lawful, and it must be made of perceptual information. When we are talking about designed information, we need to add one more property!
  60. 60. Dense, persistent, lawful Visual Information Surfaces Edges Textures Free use image Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 emember this e rely on visual information about affordances of paths when we are wal ing about our surroundings. hat information must be dense, persitent, and lawful. hen we design information that supports engaging ob ects and our surroundings, we need one additional property.
  61. 61. Haptic Information Dense (enough) Persistent (enough) Lawful Surfaces Edges his woman has low vision ability, and this vest is designed to generate haptic information on the surface of the vest to replace visual information about obstacles in her path as she approaches them. his haptic information is not nearly as dense as visual information. nly for ob ects close to her. t s not as persistent as constant visual information. ut it s dense enough, persistent enough and certainly lawful and she can rely on it to control her action of avoiding obstacles continuously over time. sitters. And sitters are gonna sit.
  62. 62. Sound Information Dense (enough) Persistent (enough) Lawful Incoming cars Train/no train Crossing light timers Conversations Physical obstacles Reference: Gaver, William W. (1994). What in the world do we hear? An ecological approach to auditory perception. Ecological Psychology 5(1): 1-29. Coherent he problem with using sound for this is she is already relying on lots of other sound information for the meaningful activity of wayfinding incoming cars, train no train, crossing light timers, conversations. he sounds of obstacles steps on the sounds of other very important aspects of wayfinding. o, for designed information, we need to add a principle the information must be coherent with other information participating in the activity.
  63. 63. Action Perception Actor as Sculptor Performing physically for a good continuation Aligning Anticipating Attunement PERCEPTUAL INFORMATION Invariant Structure Affordance DENSE PERSISTENT LAWFUL COHERENT DESIGN So we add Coherent to the properties of information to support actor as sculptor.
  64. 64. Action Perception Actor as Sculptor Performing physically for a good continuation Aligning Anticipating Attunement PERCEPTUAL INFORMATION Invariant Structure Affordance DENSE PERSISTENT LAWFUL COHERENT New Kinds! New Kinds! New Dynamics! DESIGN We can infuse our designed objects with new kinds of underlying dynamics. And that means we must also design in new kinds of invariant structure to become new kinds of affordances, potentially along with new kinds of actions actors use to engage them. Our actors need to rely on information in our designs in order to perceive action, but also to rely on it as the action is unfolding.
  65. 65. New dynamics – new affordances What does a transformational affordance look or sound or feel like? What does a permutational affordance look or sound or feel like? What dynamic sound structures can we design to become affordances? What dynamic touch structures can we design to become affordances? e ve been tal ing mostly about invariant structure in visual information to serve as affordances. hat dynamic sound structures can we design to become affordances hat dynamic touch structures can we design to become affordances hen we thin about new dynamics what does a transformational affordance loo or sound or feel li e hat does a transformational affordance loo or sound or feel li e
  66. 66. Action Perception Actor as Sculptor Performing physically for a good continuation Aligning Anticipating Attunement PERCEPTUAL INFORMATION Invariant Structure Affordance DENSE PERSISTENT LAWFUL COHERENT DESIGN New Dynamics! If we give our designed objects new underlying dynamics, but we don’t also have them radiate information about those dynamics that our actors can perceive and become attuned to, that’s unnatural behavior.
  67. 67. Actor as Conductor Natural Human Conversation ow let s loo at design properties for actor as conductor. eirdly, we can see this best by comparing it to a natural human conversation.
  68. 68. Natural Human Conversation Anticipating Improvising Speaking Listening Aligning Entraining Listening Aligning Entraining Anticipating Improvising Speaking Cultural Possibilities Cultural Possibilities Thibald Paul J. (2011). First-Order Languaging Dynamics and Second-Order Language: The Distributed Language View. Ecological Psychology 23(3): 210-245.Hodges, Bert (2014). Righting language: the view from ecological psychology. Language SciencesReferences: Ecological psychology suggests that when someone in a conversation speaks, they are anticipating what’s possible in the conversation by improvising wordings. The listener is aligning and entraining with the speaker. As the dialogue progresses, the participants in the conversation engage in creative tensions. They are both anticipating and aligning with the possibilities of the conversation. This points to their shared values as part of culture.
  69. 69. Rhythm Grammar Wording Cultural Possibilities Natural Human Conversation Listening Rhythm Grammar Wording Aligning Entraining Anticipating Improvising Speaking Cultural Possibilities Anticipating Listening Aligning Entraining Improvising Speaking Thibald Paul J. (2011). First-Order Languaging Dynamics and Second-Order Language: The Distributed Language View. Ecological Psychology 23(3): 210-245.Hodges, Bert (2014). Righting language: the view from ecological psychology. Language SciencesReferences: They are both improvising and entraining with the rhythm, grammar, wordings of each other.
  70. 70. Rhythm Grammar Wording Cultural Possibilities Natural Human Conversation Listening Rhythm Grammar Wording Aligning Entraining Anticipating Improvising Speaking Cultural Possibilities Anticipating Listening Aligning Entraining Improvising Speaking CONCEPTS Coordinating action for a good continuation Thibald Paul J. (2011). First-Order Languaging Dynamics and Second-Order Language: The Distributed Language View. Ecological Psychology 23(3): 210-245.Hodges, Bert (2014). Righting language: the view from ecological psychology. Language SciencesReferences: They do this using concepts, conceptual information. And the whole things is about coordinating action for a good continuation. Even just sitting around joking with friends can be a good continuation for your shared values and culture.
  71. 71. Listening Aligning Entraining Anticipating Improvising Speaking Cultural Possibilities Rhythm Grammar Wording Now we can see what we’ve done to this structure as UX designers…. When we don’t have two actors engaging in a conversation, but….
  72. 72. Actor as Conductor Listening Aligning Anticipating Entraining Improvising Speaking DESIGN Cultural Possibilities Rhythm Grammar Wording FunctionContent IA IxD ID VD An actor, and a design, the design is made of the elements of user experience that Jesse outlined for us 15 years ago (Elements of User Experience, Jesse James Garrett).
  73. 73. Actor as Conductor Performing Aligning Anticipating Entraining Improvising Performing Cultural Possibilities DESIGN Rhythm Grammar Wording FunctionContent IA IxD ID VD Instead of speaking and listening, we say the actor is performing with the system (that may include clicking, tapping, voice commands, gestures, whatever else we invent for actors to engage a system.)
  74. 74. Actor as Conductor Performing Aligning Anticipating Entraining Improvising Cultural Possibilities DESIGN Rhythm Grammar Wording FunctionContent IA IxD ID VD Instead of a balance of improvising and entraining…
  75. 75. Actor as Conductor Aligning Anticipating Improvising Cultural Possibilities Labels Grammar Interactions Entraining Performing DESIGN FunctionContent IA IxD ID VD The user must fully entrain with the particular labels of categories and navigation selections, the grammar and particular wordings of searches and voice commands, the gist of gestures, and so on, in order to engage the system.
  76. 76. Actor as Conductor Aligning Anticipating Improvising Labels Grammar Interactions Entraining Performance Possibilities Performing DESIGN FunctionContent IA IxD ID VD Instead of cultural possibilities, the actor is anticipating and aligning with performance possibilities as she engages the system.
  77. 77. Actor as Conductor Aligning Anticipating Improvising Labels Grammar Interactions Entraining Performance Possibilities Finding, understanding, creating artifacts Performing for a good continuation Performing DESIGN FunctionContent IA IxD ID VD Instead of the system of actor and design performing together to coordinate action for a good continuation, the actor is performing for a good continuation. This is about finding, understanding, creating artifacts using software and websites.
  78. 78. Actor as Conductor Aligning Anticipating Entraining Improvising Performance Possibilities Finding, understanding, creating artifacts Labels Grammar Interactions Performing for a good continuation Zoom in… Performing DESIGN FunctionContent IA IxD ID VD Some of the things we’ve been experimenting with lately are tinkering with these mechanics even more. Let’s zoom in and take a look at those.
  79. 79. Actor as Conductor Aligning Anticipating Entraining Improvising Machine Learning (User) Performance Possibilities Labels Grammar Interactions Natural Language Processing hings li e natural language processing and machine learning about the particular lexicon and phrasings of a specific user is really about ad usting the system s tolerance for imprecision for entraining. he actor can be a little less precise and still entrain with the system.
  80. 80. Tolerance for imprecision for entraining Entraining Improvising NARROW Tolerance Precise entrainment WIDE Tolerance Forgives improvising Standard searching, voice commanding Searching with spelling/wording forgiveness Natural language processing Machine Learning user wordings, grammar Entraining Improvising hen we thin about tolerance for imprecision for entraining, a system with a narrow tolerance requires very precise entrainment. omething li e standard searching and voice commanding require precise entrainment. hings li e searching with spelling wording forgiveness increases the tolerance for imprecision. hings li e natural language processing and machine learning of the user s particular wordings and grammar greatly increase the tolerance for imprecision and allow the user to improvise a bit.
  81. 81. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor Natural Language Processing Machine Learning (User) Tolerance for Imprecision o, we can say that we are ad usting the tolerance for imprecision for entraining.
  82. 82. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor Machine Learning (domain) Natural Language Processing Machine Learning (User) Tolerance for Imprecision achine learning of the domain, or sub ect matter of the design increases the possibilities for performance by bringing new or enriched information to the performance.
  83. 83. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor Machine Learning (domain) Natural Language Processing Machine Learning (User) Tolerance for Imprecision his creates a scenario where the user is continuously aligning as new possibilities and discoveries are brought by the system. A little less anticipating and a little more aligning, but the payoff is serendipity and discovery.
  84. 84. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor System Agency Natural Language Processing Machine Learning (User) Machine Learning (Domain) Tolerance for Imprecision ith digital agents, the actor offloads the performance of anticipating and aligning to the digital agent. ven so, the agent still has to interrupt the actor occasionally for a decision or a status update. At that point, the agent must provide the right information to let the actor re enter the tension of anticipating and aligning to perform for a good continuation with the agent.
  85. 85. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor Natural Language Processing Machine Learning (User) Machine Learning (Domain) System Agency Conversational UI Tolerance for Imprecision onversational is not really about mimic ing natural human conversation as we ve seen that structure , but about helping the actor pre align with the performance possibilities of the system by offering what the options may be.
  86. 86. Aligning Anticipating Entraining Improvising Performance Possibilities Labels Grammar Interactions Actor as Conductor Anticipating Natural Language Processing Machine Learning (User) Machine Learning (Domain) System Agency Conversational UI Tolerance for ImprecisionPossibility Space Dynamics hese inds of things change the dynamics of the possibility space. o, we can see two mechanics we are tin ering with for the structure of actor as conductor.
  87. 87. Actor as Conductor Aligning Anticipating Entraining Improvising Finding, understanding, creating artifacts Performing for a good continuation Performing CONCEPTS DESIGN FunctionContent IA IxD ID VD Language Perceptual Mappings Just like a natural human conversation takes place by concepts, so does actor as conductor. The information in our designs has historically been made of concepts. Concepts can show up in our designs as language, and we also infuse our concepts with perceptual information. Let’s look at those…
  88. 88. CONCEPTS Language Perceptual Mappings Labels Directive, Descriptive, Narrative Text Language Button Perceptual Mappings Information Visualizations Sound MappingsHaptic Mappings anguage in our designs manifests as labels for navigation and filtering, directive or descriptive paragraphs and so forth. e also infuse our concepts with perceptual mappings. hings li e shape, and color, and position, line quality, and textures are all inds of information visuali ations of concepts. echanical vibrations are haptic mappings to concepts. e also create mappings between sounds and concepts often alerts, or notifications of state changes . hat it feels li e to actors engaging both types of conceptual information, though, is very different.
  89. 89. Button Perceptual MappingsLanguage is viscous Awareness Linearity Requires attention Information Visualizations Sound MappingsHaptic Mappings anguage is more viscous, it requires awareness and attention and rolls linearly.
  90. 90. Perception flows easily Tacit Reflexive Awareness Linearity Requires attention VISCOSITY EASE OF FLOW OF CONCEPTS Picked up Language is viscous erceptual mappings flow easily because perceptual information is tacit and reflexive, we don t have to activity thin about it to pic it up. o, we can thin about the viscosity, or ease of flow of meaning in our designs, when we select to deliberately use language and or infuse some of our concepts with perceptual mappings. here is much more to be said about this lease see https www.slideshare.net m ane h what we mean by meaning new structural properties of information architecture ias qid c b b f c fded ef d v b from search and https www.slideshare.net m ane h meaning modes in design fluxible qid cca dd de b b ab a ced v b from search for details and design examples.
  91. 91. Actor as Conductor Aligning Anticipating Entraining Improvising Finding, understanding, creating artifacts Performing for a good continuation Performing CONCEPTS DESIGN FunctionContent IA IxD ID VD Tolerance for ImprecisionPossibility Space Dynamics Viscosity So, these are some properties we can adjust and dial in for our designs to support actor as conductor.
  92. 92. Selecting Action for a good continuation HUMAN BEHAVIOR* Controlling Action physical relationship with surroundings Actor-as-Conductor WITH DESIGN Actor-as-Sculptor Natural User Experience Affordances Perceptual InformationConcepts Language Perceptual Mappings INFORMATION Dense (enough) Persistent (enough) Lawful Coherent PROPERTIES Viscosity Imprecision Tolerance Possibility Space Dynamics PERFORMANCE Aligning Anticipating Entraining Improvising Conceptual Performance Aligning Anticipating Acting Perceiving Physical Performance * Reference: Golonka, Sabrina (2015): Laws and conventions in language-related behaviors, Ecological Psychology 27(3) p. 236-250 et s capture all of this in one slide about natural user experience design. hen the human behavior is about selecting action for a good continuation, when that is done with designed information, we call that actor as conductor. he performance of actor as conductor is a conceptual performance, maintaining creative tensions between anticipating and aligning with the possibilities of the performance, and entraining with the system with some improvising possible lately . he information to support actor as conductor is made of concepts both language and perceptual mappings concepts infused with perceptual qualities . e have several design dials to set for actor as conductor viscosity determine the ind and quality of attention that suits the situation , imprecision tolerance for entraining with the system, and set the possibility space dynamics to suit the situation. or the other human behavior, controlling action unfolding over time to maintain a physical relationship with our surroundings, when done with design that is actor as sculptor. his is a physical performance engaged by a tension between acting and perceiving, and a continuous tension between anticipating what can be done and aligning with what is done as the actor acts. his ta es place by affordances which are made only of perceptual information. n this case, design properties are laws that must be in place in order to support this behavior the information ma ing up the affordances must be dense enough , persistent enough , lawful, and coherent with the other inds of information participating in the activity.
  93. 93. Gesture Touch Mouse Visualizations Mechanical Vibrations SoundsDigital Agents Assistive Technology Machine Learning Simulated Objects Voice Commands Text Mode switching across contexts o, it s not ust mode switching that we need to thin about
  94. 94. Gesture Touch Mouse Visualizations Mechanical Vibrations SoundsDigital Agents Assistive Technology Machine Learning Simulated Objects Voice Commands Text Conducting-Sculpting handoffs t s handing off between conducting and sculpting within a given activity that we also need to consider.
  95. 95. Conducting-Sculpting handoffs An architect with a building in A software actor as conductor , brings a simplified version of the building into virtual reality so he can wal around it relying on the way the building ta es up placetime, as he controls his actions to s etch new towers actor as sculptor , then bring the s etch bac into software to detail and rationali e it as actor as conductor. hat do we need to do as designers to flow this handoff between conducting and sculpting and bac again esigned by amuel Arsenault rassard esigned by amuel Arsenault rassard
  96. 96. Conducting-Sculpting handoffs Fusion 360 loft animated gif by SolidSmack ven within the same environment, we need to thin about this handoff. n A software, mostly supporting actor as conductor, we are providing some opportunities to sculpt. n this case, the actor is pulling on the geometry of her design in order to form it with respect to the rest of the design. his is a very different action than conducting with a property panel and commands that are applied by selecting. oes it ma e sense to have the same physical mouse supporting this sculpting as conducting o we need to organi e different interaction modes for conducting vs. sculpting in the same environments
  97. 97. Useful Clear Accessible Usable GOOD Resilient Natural thin of this as another tool for the tool it for us to use as we wal around our designs, evaluating what ind of experiences they will facilitate.
  98. 98. Augment humans performing with design for a good continuation Photo by permission: Hugh Hochman As we continue to search for ways to augment humans performing with design for a good continuation. we should noc ourselves out experimenting with the possibilities, with the underlying mechanics of that.
  99. 99. Designs have information with the properties actors, as humans, are relying on to behave Photo by permission: Hugh Hochman ic ing up a hologram and deforming it to categori e it spatially an intelligent agent that does a little haptic bee li e waggle dance on a person s hand to give its status update these things can come to feel quite natural. As long as our designs have information concepts or affordances with the properties that actors, as humans, are relying on to behave.
  100. 100. Marsha Haverty Thank you @mjane_h UX WEEK 2017

×