SlideShare ist ein Scribd-Unternehmen logo
1 von 34
Downloaden Sie, um offline zu lesen
THE WORLD IS
THE SCREEN
Convey UX 2014 | Seattle WA | Andrew Hinton | @inkblurt
Understanding Information Environments
(Note for PDF: this is a shorter version of a talk I’ve done over the last year; some slides and
content are newer, but some have been removed. For other versions, see slideshare.net/
andrewhinton )
Andrew Hinton | @inkblurt
INFORMATION ENVIRONMENT
http://commons.wikimedia.org/wiki/File:Derbyshire_Landscape.jpg
This is an environment. It’s a natural setting, full of physical, naturally occurring surfaces and
objects. It’s the sort of environment that humans as a species spent most of its evolutionary
history in.
Notice there’s a stone wall here. It’s something made by humans that adds structure to the
environment. It’s an object that impedes motion, but it’s also an object with cultural meaning
-- it’s telling us something about this place. Maybe it’s marking off owned property, or it
says “this is where my sheep are contained.”
>>
What if we think of this as not only an environment, but an information environment? In what
ways is that true?
That’s largely what this talk is about.
Because networked technology is becoming so pervasive, so much a part of the fabric of our
surroundings, that we need to understand the fundamentals of how humans comprehend
their environment, regardless of the specific technology.
And understanding that means understanding the sorts of information that are in the world,
and how they work together to create places, contexts, and meaning.
http://commons.wikimedia.org/wiki/File:Derbyshire_Landscape.jpg
Andrew Hinton | @inkblurt
INFORMATION
ENVIRONMENT
3
INFORMATION
ENVIRONMENT
These are two information environments that involve office supplies.
What is the difference, really, between shopping for them in a brick and mortar retail store,
and shopping for them through an online retailer like Amazon?
If we think of the physical store as an information interface, how much information is
conveyed, and of what kind, through one interface versus another? How useful is it for the
activity of deciding between one printer or stapler and another? There are things the digital
environment affords us that the physical one does not: we can compare things more easily,
sort them, search them; we can see more of a selection; we can see what others have said
about the products.
Andrew Hinton | @inkblurt
Subway station + Food store
Here we have a mashup where a place is simultaneously a subway station and a virtual store,
with pictures of products on the walls where shoppers can scan them for delivery or pickup
using the store’s mobile app. It stretches what we assume about how an environment works,
and even what a place is.
Andrew Hinton | @inkblurt
Google Glass
Leap Motion Microsoft Kinect
We’re being surrounded by sensors and digital agents that are trying to read our actions, or
layers of digital information added to what we see around us.
left glass image wired.com / others from google
Leap motion from their site
Kinect from blogs.msdn.com
Andrew Hinton | @inkblurt
Now, even your refrigerator can spam you.
http://www.bbc.co.uk/news/technology-25780908 bestbuy.com
Our environment is full of even mundane things that are networked and active in ways we
used to only joke about. Like the smart refrigerator that was recently caught being used as a
spam bot.
Andrew Hinton | @inkblurt
Understanding the nature
of our materials.
We’ve gone for a long time in our work being comfortable with knowing that what we make is
confined to a specific screen -- a bounded interactive surface. But now *all* the surfaces
around us are potentially digital and networked. Walls and windows, coffee pots and shoes.
So we need to re-assess, what is the nature of our materials?
Potters have a deep understanding of clay; masons have a deep understanding of stone.
We need to deepen our understanding of information.
Andrew Hinton | @inkblurt
INFORMATION
Digital
Semantic
Physical
Information is a word we use a lot, but I don’t think that in our general practice we have a
common framework for what it is, and the role it plays in the environments we inhabit.
I tried finding a single, official definition, but there really isn’t one -- instead, there are many
different ways of looking at various things in our world that people call information.
The problem is, all of them are pretty useful and valuable. So rather than just pick one, I
decided to pick several and see how they work together and inform each other, so to speak.
These modes or ways we experience information are physical, semantic, and digital.
Andrew Hinton | @inkblurt
Physical Information
10100010
01001000
01110011
10100010
01001000
01110011
Digital systems transmitting
to & receiving from other
digital systems.
Digital
People communicating
with people.
Semantic
Animals (including
people) perceiving the
environment.
Physical
Physical information has to do with the ecological relationship between an animal and its
physical environment.
The green field we looked at earlier was mostly made up of physical information that
creatures use to know where they can go and what they can do in those surroundings.
The way I’m framing this comes from something called ecological psychology, which has been
adapted for a branch of study called embodied cognition.
Andrew Hinton | @inkblurt
Ecological Information
"Pick-up"
Agent
Perception
Action
Ambient,
structured
energy arrays
Embodied cognition:
Body & Environment, not just Brain.
Embodied cognition sees the environment and the body as part of a larger dynamic, in which
the brain is a participant, but not necessarily the center-point. It may even just be a
supporting player.
A big part of this is *action*: we don’t perceive the world by sitting still and taking in sensory
data; we perceive by moving and interacting with the environment.
Much of our daily activity is unconsciously nudged and controlled by our environment.
Andrew Hinton | @inkblurt
“... there’s no such thing as
disembodied cognition [for]
living creatures.
All of our cognition has to
be embodied ...
Brains evolved in the
service of action.”
- Louise Barrett
http://podcasts.ox.ac.uk/embodiment-taking-sociality-seriously-audio
Louise Barrett, who wrote a great book on embodied cognition called Beyond the Brain, puts
it nicely -- -- there’s no such thing as disembodied cognition for living creatures, all of our
cognition has to be embodied because brains evolved in the service of action.
We didn’t arrive on the planet as brains that grew bodies to supplement their disembodied
brain activity. Brains evolved to supplement the abilities of bodies in environments. They’re
the product of the structures of their surroundings.
Andrew Hinton | @inkblurt
James J Gibson - Ecological Psychology of Perception
Now seen as pioneer of embodied cognition.
Some embodied cognition theorists have heavily adopted the work of James J Gibson, who
was a scientist of something called “ecological psychology” in the mid 20th century -- a field
he was part of inventing.
Andrew Hinton | @inkblurt
Ecological Information
"Pick-up"
Agent
Perception
Action
Ambient,
structured
energy arrays
Gibson used the word “information” to mean the
elements we perceive in the environment that tell us
what it “affords.”
“INFORMATION”
Gibson purposefully used the word “information” for this body-environment relationship. He
argued that our bodies do something he called “information pickup” -- where the body’s
perceptual system detects structural cues in the environment that specify what actions the
body can take. And he argued that this happens directly, with little or no need for brain
processing.
Andrew Hinton | @inkblurt
The perceived functional
properties of objects, places
and events in relation to an
individual perceiver.
AFFORDANCE
Perception exists only insofar
as we perceive affordances.
Gibson also invented the concept of affordance.
For Gibson, affordance isn’t a thing you add to something.
It’s the result of a property in the environment that presents the opportunity for action to a
perceiving organism.
This is true in the built environment as well as the natural one; the structure of these stairs
interacts with energy in the environment that my perception picks up on, and my body
detects that the specific structure of the steps will take my body upward if climb them.
Affordance was and still is a radical idea, in the sense that Gibson meant it.
Affordance is the organizing principle behind *ALL* perception. We don’t perceive anything
unless it affords meaningful action for a given context.
Gibson even went on to discuss how higher-order combinations of affordances can be found
in the complexities of human culture. A mailbox, for example, has the affordance for sending
mail -- but only because we’ve learned a lot of stuff about the world around that box that
makes it a special sort of container connected to a broader cultural system.
Andrew Hinton | @inkblurt
We perceive environment as “nested,”
not in logical hierarchy.
We perceive elements in the
environment as invariant (persistent) or
variant (in flux). Invariance grounds our
experience (literally).
Invariance
Nestedness
Two of Gibson’s principles of
environmental perception.
Gibson created a marvelous system of principles and components that work as building
blocks for how we perceive our environment and its affordances. I can only barely touch on
them here, but I’ll mention a few important ones.
>> Invariance is the property of environmental structure that makes some of it persistent
over time, so that we can make sense of the whole. Terrestrial animals like us evolved on
stable ground, under a persistent sky. Substances and surfaces that persist allow us to learn
what properties they have, and what they afford us. This can be a useful idea in digital design
because in software, even though we can make everything fluid and variant, we need to
design invariant structures that ground the experience.
>> Nestedness: we perceive the environment as structures that relate to one another as
nested -- a stream is in a valley, which is between mountains, which are part of a range,
which are between earth and sky. There aren’t always definite boundaries, though, and this
isn’t a purely logical relationship -- there are overlaps and redundancies, and shifts in
meaning. A stick on the ground can be kindling or a club, or it can be part of a roof over our
heads in a shelter. It’s not just one defined thing in how it relates to the rest of the
environment.
This is a powerful idea when we look at something like faceted navigation, where the
meaning of a piece of content or an object shifts depending on what perspective a user is
bringing to it.
Andrew Hinton | @inkblurt
Surfaces
Objects
Layout
Place
Some of the components that
make up environments.
There are many components Gibson outlines, like surface, object, layout and place -- that
work together to form the environment.
There isn’t time to cover it all here -- but I think it’s a useful framework we can borrow from,
because all the environments we design should meet the expectations of people’s embodied
experience. Our bodies want and need environments to work in these ways.
Andrew Hinton | @inkblurt
Semantic Information
10100010
01001000
01110011
10100010
01001000
01110011
Digital systems transmitting
to & receiving from other
digital systems.
Digital
Animals (including
people) perceiving the
environment.
Physical
People communicating
with people.
Semantic
Of course, humans don’t just exist in raw physical surfaces and objects -- we add to the
environment. In addition to buildings and roads and such, we add language to the world.
The semantic mode, in short, is language. But I mean language in the broad sense of things
we put into the environment to communicate with people. This can be all sorts of stuff:
speech, gestures, text, iconography, even buildings have semantic qualities.
Andrew Hinton | @inkblurt
LANGUAGE IS ENVIRONMENT
STRUCTURE THAT ENABLES US TO MAKE MORE STRUCTURE
Language is “a form of mind-
transforming cognitive
scaffolding: a ... symbolic
edifice [playing a] critical role
in promoting thought and
reason.”
- Andy Clark Supersizing the Mind
http://en.wikipedia.org/wiki/File:CantileverScaffold.jpg
Language is environment. In another excellent book on embodied cognition, Andy Clark
frames language as a form of cognitive scaffolding; it persists in the environment, as new
structure that allows us to make other structures. We wouldn’t have most of what we think of
as civilization without the scaffolding of language.
Andrew Hinton | @inkblurt
Language is the primary human interface to
complex systems.
Language is both the infrastructure we use for creating complexity, as well as the primary
interface we have for understanding and inhabiting complex systems.
Andrew Hinton | @inkblurt
WE LIVE IN LANGUAGE STRUCTURES
Language structures are an architecture that we live within together, whether we know it or
not. Whether these structures are defined explicitly like in this early IBM management
diagram, or defined tacitly through the collective assumptions within a shared culture, the
way we talk and write about our shared environment is also a structural feature of that
environment.
Andrew Hinton | @inkblurt
Physical Affordance +
Semantic (or “Linguistic”) Affordance
Physical
Semantic
Humans aren’t just perceivers of physical structure. We depend as much or more on the
symbolic workings of language – semantic information – as a part of our environment for
working together and figuring out our world, and making stuff – as well as just getting
around.
In this picture of a staircase at City Lights Bookstore in SF, we see stairs that have intrinsic
structural information that our bodies perceive as affording the ability to walk upward.
But there’s this other thing – a label – which signifies what where we will be going when we go
up the stairs.
We might call this semantic affordance. It’s an abstracted, symbolic version of how we
perceive and comprehend physical structure.
In linguistics we also hear of signification -- which I won’t go into in detail here -- but there
are signifiers at work in both the stairs and the label. The important point I want to get across
here is that affordance is at work even at the high-order level of meaning we have with
language, not just in physical structures.
Andrew Hinton | @inkblurt
Physical
Physical
+ Semantic
Simulated-
Physical
+ Semantic
Mostly
Semantic
Affordance across a spectrum
of physical to semantic.
Affordance is at the root of how we use language as well as the simulated physicality of the
stuff we put into interfaces.
>> The stairs intrinsically contain all the information needed for your body to know what will
happen when you climb them.
>> But look at this button. When people started making machines and systems that were
more complex, we had physical affordances that did more than we could tell just by touching
or looking at them. Because I’ve pushed similar buttons before, I know that the physical
information here means I should be able to push in the red part and it will probably spring
back out -- that’s all that the physical information is telling me. In order to know what the
button is going to cause elsewhere, I need a label.
>> When Microsoft updated Windows to add the Start button, they were basically adding an
invariant structure to the operating system so that users would know that, no matter what,
they could always “start” a new action from here. And like lots of other software, they used a
simulated button object, borrowing from what we were used to with buttons like the one on
the left. The button isn’t physical information, really ... it’s using semantic information to
simulate a physical object. That’s what graphical user interfaces do.
>> In other parts of interfaces, and especially on the web with hypertext, labels may not have
these simulated object qualities; so we have to rely a lot on the context of a label to know if it
just affords reading or if it’s something we can activate.
This is the issue at the heart of flat versus skeuomorphic design -- it’s really not about
aesthetics, it’s about solving this issue of affordance.
Additionally, the web introduced broad usage of labels as paths -- tapping a hyperlink
doesn’t just activate some mechanism, it takes you on a journey to what feels to your
perception like a new place.
Andrew Hinton | @inkblurt
Which red x????
Looks like a
“confirm” action.
Simulated-Physical & Semantic Information
In Conflict
Here are some examples of where things perform both semantically and ecologically, in
confusing ways, in a UI.
In this email application, logically speaking, the red X’s are all very different -- but physically
they’re barely different at all; they require a lot of thought to disambiguate their semantic
meaning. When I was in a hurry, I would reach for the closest red X to do whatever I’m trying
to do - close the message, decline an invite, or delete it entirely. About half the time, I would
click the wrong one.
>>In an unsubscribe interface for fab.com, my wife discovered that she was apparently re-
subscribing without realizing it, because that big red button -- like a big berry you can’t help
but pick -- contextually feels like it’s a confirmation, not a cancellation/re-subscription
action.
Andrew Hinton | @inkblurt
Animals (including
people) perceiving the
environment.
Physical
People communicating
with people.
Semantic
Digital Information
10100010
01001000
01110011
10100010
01001000
01110011
Digital systems transmitting
to & receiving from other
digital systems.
Digital
So the examples we just looked at weren’t just any sort of semantic information, they were
semantic information driven by digital technology. And digital technology relies on digital
information.
Digital information is how the black boxes talk to the other black boxes. It’s the lifeblood of
information technology. The whole point of digital information is to strip human meaning out
of it, which makes it more efficient for machines to transmit, store, and retrieve.
This isn’t stuff we see face to face very much. Mainly we encounter its *effects* in the
environment.
Andrew Hinton | @inkblurt
Digital Information Mode Leaking into Semantic Environment
We see machines around us trying to get us to perceive what they are saying, or what they
want to hear from us. We see them murmuring to each other in weird, noisy machine-only
semantics that we do not comprehend either ecologically or semantically. The gas pump has
to have a sticker added to it that explains what “Enter Data” means.
>>The Twitter profile with the iPhone coordinates expresses my location not in a semantic
way (the name of a city, for instance) but in a Cartesian grid that I have no contextual
orientation for, either semantically or ecologically.
>>The Delta app has information that I, as a human, can read, but it gives priority to the
machines that I encounter in the workflow of the airport.
Andrew Hinton | @inkblurt
Digital information enables
pervasive semantic place-making.
I don’t mean to paint digital information as a villain. It isn’t.
The ability to transmit, store and retrieve information in this way is a miracle.
An platform I like a lot is Avocado - it lets a couple keep in touch and share a place together,
pervasively.
It has nice touches that key into embodied experience of semantic information, like sending a
hug by touching the screen to your heart.
Another nice touch: the couple shares the same password - making a word into a very real
link of co-ownership of the place, like having the same keys to your home.
This sort of pervasively available place would be impossible without digital information in the
background. But it also requires a lot of discipline with semantic information structure to
make the place coherent.
Andrew Hinton | @inkblurt
Google Plus on Desktop Google Plus on Mobile
Easily post content in the circle I’m in.
Confusingly requires re-establishing the
circle where I’m posting.
Broken “invariants” across contexts.
Google Plus has invariance in how it establishes place between mobile and desktop versions.
When I’m on desktop, and I’m in my “Family” Circle reading posts, when I post something
new it automatically goes into that circle, just like I’d expect if I were having a conversation in
a room. But on mobile, if I’m in my Family circle, a new post defaults to viewable by everyone;
I have to work through a half dozen interactions to make it post just to the circle I perceived I
was already “in.”
Andrew Hinton | @inkblurt
The ontology of “news” is confused here.
• News Feed inside News
Feed?
• If All Friends, is it no longer
News? or Recent?
• Isn’t News “recent” by
definition?
• What, then, is News Feed as
a place? How am I changing
the kind of place it is here?
Ontology is the basis that we use for creating any sort of taxonomy - or relationship between
semantic invariants.
In the mobile version of Facebook’s “news feed” there are several options for how you want to
see the feed. Their meanings don’t make much sense together, though -- it’s not clear how
these are nested.
Is news feed another news feed inside news feed?
What does all friends mean? If I choose that, am I not seeing news anymore, or the most
recent?
Isn’t news recent by definition?
What is news-feed as a place? How am I changing the mode of the place when I select
something else?
Andrew Hinton | @inkblurt
What is “card” in this environment?
Lowes launched a service called MyLowes -- that requires the registration of a card. But they
also have a “Lowe’s Card” that’s a consumer credit card.
Conversations at checkout can end up like a “who’s on first” routine -- “do you have your
Lowe’s card?” “My Lowe’s card? That’s what I’m paying with.” “No I mean your ‘my lowe’s’
card.” “This IS my lowe’s card!”
Andrew Hinton | @inkblurt
Shopping Simultaneously in a Store & Online
http://stevenglassman.de/2013/11/25/america-vs-deutschland/
In these omni-channel scenarios, the semantic information has to align with how it’s
presented both online and physically in the store’s structure. Knowing if a huge store has a
specific peanut butter or not is great; knowing exactly which aisle to get it from is even
better.
Andrew Hinton | @inkblurt
The
Thing
Ontology helps answer:
“What is a thing across multiple contexts?”
Presently our various fields are preoccupied with how to have content and functionality make
sense in various contexts.
Ontology is at the heart of this problem.
In many organizations and project teams, there’s an over-obsession with things like layout in
each of the instantiations of a thing, but not enough discussion about how to define the
nature of the thing in abstract. That requires an ontological perspective.
And, done properly, it forms the main structures of an information environment - the
invariant pillars, so to speak - that allow language to stitch together coherence across
channels.
Andrew Hinton | @inkblurt
LANGUAGE IS
INFRASTRUCTURE.
LABEL
LABEL
LABEL
LABEL
LABEL
RULES
Labels
Connections
Rules
All this stuff is essentially made out of language... it’s language working as infrastructure.
We essentially make things out of labels, connections and rules.
Too often, we assume the labels are something to add later - but in reality they’re the thing
we have to figure out first.
This is why issues like ontology and taxonomy are so important - they establish the
“invariant” features of the environments we make.
Andrew Hinton | @inkblurt
• What are the invariants for perception
& action? What grounds the
experience?
• How is it structurally nested in relation
to a perceiver? Is it resilient for
different perspectives & contexts?
• Is it clear what is a place and what is
an object?
• How are semantic and physical
information working together, or not?
• Are the rules behind the environment
clearly expressed in its information?
Some of the questions we can ask ...
So in sum, this is just scratching the surface, but there are some questions we can ask.
>>What are the invariants for perception & action? What grounds the experience?
>>How is it structurally nested in relation to a perceiver? Is it resilient for different
perspectives & contexts?
>>Is it clear what is a place and what is an object?
>>How are semantic and physical information working together, or not?
>>Are the rules behind the environment clearly expressed in its information?
We have a long way to go to build heuristics and frameworks for responsibly creating the
broader environment that our users inhabit. Hopefully this is a helpful start.
Andrew Hinton | @inkblurt
THANKS!
Andrew Hinton | @inkblurt | andrewhinton.com
andrew@understandinggroup.com

Weitere ähnliche Inhalte

Was ist angesagt?

Acting naturally: why design needs ecological psychology
Acting naturally: why design needs ecological psychologyActing naturally: why design needs ecological psychology
Acting naturally: why design needs ecological psychologyMarsha Haverty
 
Meaning Modes in Design - Fluxible 2016
Meaning Modes in Design - Fluxible 2016Meaning Modes in Design - Fluxible 2016
Meaning Modes in Design - Fluxible 2016Marsha Haverty
 
Designing for Internet of Things is shaping relationships
Designing  for Internet of Things is shaping relationshipsDesigning  for Internet of Things is shaping relationships
Designing for Internet of Things is shaping relationshipsMassimiliano Dibitonto
 
CHI2013 presentation
CHI2013 presentationCHI2013 presentation
CHI2013 presentationDag Svanæs
 
Concept Research compiled blog
Concept Research compiled blogConcept Research compiled blog
Concept Research compiled blogHulika
 
Blended Reality: superstructing reality, superstructing selves
Blended Reality:  superstructing reality, superstructing selvesBlended Reality:  superstructing reality, superstructing selves
Blended Reality: superstructing reality, superstructing selvesSean Ness
 
Microcontent_Evideo2008
Microcontent_Evideo2008Microcontent_Evideo2008
Microcontent_Evideo2008jurijmlotman
 
What we mean by meaning: new structural properties of information architectur...
What we mean by meaning: new structural properties of information architectur...What we mean by meaning: new structural properties of information architectur...
What we mean by meaning: new structural properties of information architectur...Marsha Haverty
 
Acting Naturally with Information - IA Summit 2017
Acting Naturally with Information - IA Summit 2017Acting Naturally with Information - IA Summit 2017
Acting Naturally with Information - IA Summit 2017Marsha Haverty
 
Share Festival Networked Objects Manufacturing 031508.Key
Share Festival Networked Objects Manufacturing 031508.KeyShare Festival Networked Objects Manufacturing 031508.Key
Share Festival Networked Objects Manufacturing 031508.KeyJulian Bleecker
 
VR in education
VR in educationVR in education
VR in educationOsku Torro
 
Where The Action Is In Psychology
Where The Action Is In PsychologyWhere The Action Is In Psychology
Where The Action Is In PsychologyJ S
 
Situating Design, Designing Situations
Situating Design, Designing SituationsSituating Design, Designing Situations
Situating Design, Designing Situationsmarkshepard
 
Storytelling in the Global Village: A glimpse at our Web 2.0 selves
Storytelling in the Global Village: A glimpse at our Web 2.0 selvesStorytelling in the Global Village: A glimpse at our Web 2.0 selves
Storytelling in the Global Village: A glimpse at our Web 2.0 selvesChris Lindgren
 
Place Making: A Theory Of Knowledge Work
Place Making:  A Theory Of Knowledge WorkPlace Making:  A Theory Of Knowledge Work
Place Making: A Theory Of Knowledge WorkTrond Arne Undheim
 

Was ist angesagt? (20)

UX as CoPs
UX as CoPsUX as CoPs
UX as CoPs
 
TheRiseOfLettingGo
TheRiseOfLettingGoTheRiseOfLettingGo
TheRiseOfLettingGo
 
Acting naturally: why design needs ecological psychology
Acting naturally: why design needs ecological psychologyActing naturally: why design needs ecological psychology
Acting naturally: why design needs ecological psychology
 
Meaning Modes in Design - Fluxible 2016
Meaning Modes in Design - Fluxible 2016Meaning Modes in Design - Fluxible 2016
Meaning Modes in Design - Fluxible 2016
 
Designing for Internet of Things is shaping relationships
Designing  for Internet of Things is shaping relationshipsDesigning  for Internet of Things is shaping relationships
Designing for Internet of Things is shaping relationships
 
CHI2013 presentation
CHI2013 presentationCHI2013 presentation
CHI2013 presentation
 
Concept Research compiled blog
Concept Research compiled blogConcept Research compiled blog
Concept Research compiled blog
 
Blended Reality: superstructing reality, superstructing selves
Blended Reality:  superstructing reality, superstructing selvesBlended Reality:  superstructing reality, superstructing selves
Blended Reality: superstructing reality, superstructing selves
 
Microcontent_Evideo2008
Microcontent_Evideo2008Microcontent_Evideo2008
Microcontent_Evideo2008
 
What we mean by meaning: new structural properties of information architectur...
What we mean by meaning: new structural properties of information architectur...What we mean by meaning: new structural properties of information architectur...
What we mean by meaning: new structural properties of information architectur...
 
Acting Naturally with Information - IA Summit 2017
Acting Naturally with Information - IA Summit 2017Acting Naturally with Information - IA Summit 2017
Acting Naturally with Information - IA Summit 2017
 
Share Festival Networked Objects Manufacturing 031508.Key
Share Festival Networked Objects Manufacturing 031508.KeyShare Festival Networked Objects Manufacturing 031508.Key
Share Festival Networked Objects Manufacturing 031508.Key
 
Interactive Realism
Interactive RealismInteractive Realism
Interactive Realism
 
Knowledge and Self
Knowledge and SelfKnowledge and Self
Knowledge and Self
 
VR in education
VR in educationVR in education
VR in education
 
Where The Action Is In Psychology
Where The Action Is In PsychologyWhere The Action Is In Psychology
Where The Action Is In Psychology
 
Situating Design, Designing Situations
Situating Design, Designing SituationsSituating Design, Designing Situations
Situating Design, Designing Situations
 
Storytelling in the Global Village: A glimpse at our Web 2.0 selves
Storytelling in the Global Village: A glimpse at our Web 2.0 selvesStorytelling in the Global Village: A glimpse at our Web 2.0 selves
Storytelling in the Global Village: A glimpse at our Web 2.0 selves
 
Reading Summary Turkle
Reading Summary TurkleReading Summary Turkle
Reading Summary Turkle
 
Place Making: A Theory Of Knowledge Work
Place Making:  A Theory Of Knowledge WorkPlace Making:  A Theory Of Knowledge Work
Place Making: A Theory Of Knowledge Work
 

Ähnlich wie The World is the Screen - Convey UX 2014

Taming Context in the Internet of Things
Taming Context in the Internet of ThingsTaming Context in the Internet of Things
Taming Context in the Internet of ThingsWebVisions
 
Give a hoot! Mapping (and caring for) the semantic environment
Give a hoot! Mapping (and caring for) the semantic environmentGive a hoot! Mapping (and caring for) the semantic environment
Give a hoot! Mapping (and caring for) the semantic environmentJorge Arango
 
Chimpanzees Exhibit Culture Through Social Interaction
Chimpanzees Exhibit Culture Through Social InteractionChimpanzees Exhibit Culture Through Social Interaction
Chimpanzees Exhibit Culture Through Social InteractionAnn Garcia
 
The Subtle Art of Persuasion
The Subtle Art of PersuasionThe Subtle Art of Persuasion
The Subtle Art of PersuasionJames Box
 
A Model for Information Environments - Reframe IA Workshop 2013
A Model for Information Environments - Reframe IA Workshop 2013A Model for Information Environments - Reframe IA Workshop 2013
A Model for Information Environments - Reframe IA Workshop 2013Andrew Hinton
 
Audio in a social Web of linked data
Audio in a social Web of linked dataAudio in a social Web of linked data
Audio in a social Web of linked dataEduserv Foundation
 
Open, social and linked - what do current Web trends tell us about the future...
Open, social and linked - what do current Web trends tell us about the future...Open, social and linked - what do current Web trends tell us about the future...
Open, social and linked - what do current Web trends tell us about the future...Andy Powell
 
Dan buzzo-ignite-presentation
Dan buzzo-ignite-presentationDan buzzo-ignite-presentation
Dan buzzo-ignite-presentationdanbuzzo
 
Dan buzzo-ignite-presentation
Dan buzzo-ignite-presentationDan buzzo-ignite-presentation
Dan buzzo-ignite-presentationdanbuzzo
 
A room for understanding
A room for understandingA room for understanding
A room for understandingSjors Timmer
 
The Foundations of Spatial Computing
The Foundations of Spatial ComputingThe Foundations of Spatial Computing
The Foundations of Spatial ComputingSjors Timmer
 
Causes Of Food Waste Essay
Causes Of Food Waste EssayCauses Of Food Waste Essay
Causes Of Food Waste EssayDawn Mora
 
China 2050 Student Project - Tongji University
China 2050 Student Project - Tongji UniversityChina 2050 Student Project - Tongji University
China 2050 Student Project - Tongji UniversityEmmanuel Flores Elías
 
Objects tell human stories — real things connect people to ideas
Objects tell human stories — real things connect people to ideasObjects tell human stories — real things connect people to ideas
Objects tell human stories — real things connect people to ideasARTOMATIC
 

Ähnlich wie The World is the Screen - Convey UX 2014 (20)

Taming Context in the Internet of Things
Taming Context in the Internet of ThingsTaming Context in the Internet of Things
Taming Context in the Internet of Things
 
Give a hoot! Mapping (and caring for) the semantic environment
Give a hoot! Mapping (and caring for) the semantic environmentGive a hoot! Mapping (and caring for) the semantic environment
Give a hoot! Mapping (and caring for) the semantic environment
 
Orienteering Objects
Orienteering ObjectsOrienteering Objects
Orienteering Objects
 
Chimpanzees Exhibit Culture Through Social Interaction
Chimpanzees Exhibit Culture Through Social InteractionChimpanzees Exhibit Culture Through Social Interaction
Chimpanzees Exhibit Culture Through Social Interaction
 
Verum Factum: making the world with language | Andrew Hinton #IIAS15
Verum Factum: making the world with language | Andrew Hinton #IIAS15Verum Factum: making the world with language | Andrew Hinton #IIAS15
Verum Factum: making the world with language | Andrew Hinton #IIAS15
 
Physicalism
PhysicalismPhysicalism
Physicalism
 
The Subtle Art of Persuasion
The Subtle Art of PersuasionThe Subtle Art of Persuasion
The Subtle Art of Persuasion
 
A Model for Information Environments - Reframe IA Workshop 2013
A Model for Information Environments - Reframe IA Workshop 2013A Model for Information Environments - Reframe IA Workshop 2013
A Model for Information Environments - Reframe IA Workshop 2013
 
Audio in a social Web of linked data
Audio in a social Web of linked dataAudio in a social Web of linked data
Audio in a social Web of linked data
 
Open, social and linked - what do current Web trends tell us about the future...
Open, social and linked - what do current Web trends tell us about the future...Open, social and linked - what do current Web trends tell us about the future...
Open, social and linked - what do current Web trends tell us about the future...
 
Social Approaches To Learning
Social Approaches To LearningSocial Approaches To Learning
Social Approaches To Learning
 
Eliciting affordances for Smart Objects in IoT Era
Eliciting affordances for Smart Objects in IoT EraEliciting affordances for Smart Objects in IoT Era
Eliciting affordances for Smart Objects in IoT Era
 
Writing With Technology Essay
Writing With Technology EssayWriting With Technology Essay
Writing With Technology Essay
 
Dan buzzo-ignite-presentation
Dan buzzo-ignite-presentationDan buzzo-ignite-presentation
Dan buzzo-ignite-presentation
 
Dan buzzo-ignite-presentation
Dan buzzo-ignite-presentationDan buzzo-ignite-presentation
Dan buzzo-ignite-presentation
 
A room for understanding
A room for understandingA room for understanding
A room for understanding
 
The Foundations of Spatial Computing
The Foundations of Spatial ComputingThe Foundations of Spatial Computing
The Foundations of Spatial Computing
 
Causes Of Food Waste Essay
Causes Of Food Waste EssayCauses Of Food Waste Essay
Causes Of Food Waste Essay
 
China 2050 Student Project - Tongji University
China 2050 Student Project - Tongji UniversityChina 2050 Student Project - Tongji University
China 2050 Student Project - Tongji University
 
Objects tell human stories — real things connect people to ideas
Objects tell human stories — real things connect people to ideasObjects tell human stories — real things connect people to ideas
Objects tell human stories — real things connect people to ideas
 

Mehr von Andrew Hinton

Practical Conceptual Modeling at UX Detroit Feb 2015
Practical Conceptual Modeling at UX Detroit Feb 2015Practical Conceptual Modeling at UX Detroit Feb 2015
Practical Conceptual Modeling at UX Detroit Feb 2015Andrew Hinton
 
Context Design (beta) CHI Atlanta Nov 2012
Context Design (beta) CHI Atlanta Nov 2012Context Design (beta) CHI Atlanta Nov 2012
Context Design (beta) CHI Atlanta Nov 2012Andrew Hinton
 
Users Don't Have Goals
Users Don't Have GoalsUsers Don't Have Goals
Users Don't Have GoalsAndrew Hinton
 
Communities of Practice Intro
Communities of Practice IntroCommunities of Practice Intro
Communities of Practice IntroAndrew Hinton
 
Context for WebVisions 2011
Context for WebVisions 2011Context for WebVisions 2011
Context for WebVisions 2011Andrew Hinton
 
UX and Business Analysts - Stop the Madness
UX and Business Analysts - Stop the MadnessUX and Business Analysts - Stop the Madness
UX and Business Analysts - Stop the MadnessAndrew Hinton
 
Beyond Findability: Context
Beyond Findability: ContextBeyond Findability: Context
Beyond Findability: ContextAndrew Hinton
 

Mehr von Andrew Hinton (12)

Practical Conceptual Modeling at UX Detroit Feb 2015
Practical Conceptual Modeling at UX Detroit Feb 2015Practical Conceptual Modeling at UX Detroit Feb 2015
Practical Conceptual Modeling at UX Detroit Feb 2015
 
Context Design (beta) CHI Atlanta Nov 2012
Context Design (beta) CHI Atlanta Nov 2012Context Design (beta) CHI Atlanta Nov 2012
Context Design (beta) CHI Atlanta Nov 2012
 
Happiness machines
Happiness machinesHappiness machines
Happiness machines
 
Users Don't Have Goals
Users Don't Have GoalsUsers Don't Have Goals
Users Don't Have Goals
 
Communities of Practice Intro
Communities of Practice IntroCommunities of Practice Intro
Communities of Practice Intro
 
Context for WebVisions 2011
Context for WebVisions 2011Context for WebVisions 2011
Context for WebVisions 2011
 
UX and Business Analysts - Stop the Madness
UX and Business Analysts - Stop the MadnessUX and Business Analysts - Stop the Madness
UX and Business Analysts - Stop the Madness
 
Architecture and IA
Architecture and IAArchitecture and IA
Architecture and IA
 
Linkosophy2 Ebai
Linkosophy2 EbaiLinkosophy2 Ebai
Linkosophy2 Ebai
 
Beyond Findability: Context
Beyond Findability: ContextBeyond Findability: Context
Beyond Findability: Context
 
TheContextProblem
TheContextProblemTheContextProblem
TheContextProblem
 
Linkosophy
LinkosophyLinkosophy
Linkosophy
 

Kürzlich hochgeladen

UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 

Kürzlich hochgeladen (19)

UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 

The World is the Screen - Convey UX 2014

  • 1. THE WORLD IS THE SCREEN Convey UX 2014 | Seattle WA | Andrew Hinton | @inkblurt Understanding Information Environments (Note for PDF: this is a shorter version of a talk I’ve done over the last year; some slides and content are newer, but some have been removed. For other versions, see slideshare.net/ andrewhinton )
  • 2. Andrew Hinton | @inkblurt INFORMATION ENVIRONMENT http://commons.wikimedia.org/wiki/File:Derbyshire_Landscape.jpg This is an environment. It’s a natural setting, full of physical, naturally occurring surfaces and objects. It’s the sort of environment that humans as a species spent most of its evolutionary history in. Notice there’s a stone wall here. It’s something made by humans that adds structure to the environment. It’s an object that impedes motion, but it’s also an object with cultural meaning -- it’s telling us something about this place. Maybe it’s marking off owned property, or it says “this is where my sheep are contained.” >> What if we think of this as not only an environment, but an information environment? In what ways is that true? That’s largely what this talk is about. Because networked technology is becoming so pervasive, so much a part of the fabric of our surroundings, that we need to understand the fundamentals of how humans comprehend their environment, regardless of the specific technology. And understanding that means understanding the sorts of information that are in the world, and how they work together to create places, contexts, and meaning. http://commons.wikimedia.org/wiki/File:Derbyshire_Landscape.jpg
  • 3. Andrew Hinton | @inkblurt INFORMATION ENVIRONMENT 3 INFORMATION ENVIRONMENT These are two information environments that involve office supplies. What is the difference, really, between shopping for them in a brick and mortar retail store, and shopping for them through an online retailer like Amazon? If we think of the physical store as an information interface, how much information is conveyed, and of what kind, through one interface versus another? How useful is it for the activity of deciding between one printer or stapler and another? There are things the digital environment affords us that the physical one does not: we can compare things more easily, sort them, search them; we can see more of a selection; we can see what others have said about the products.
  • 4. Andrew Hinton | @inkblurt Subway station + Food store Here we have a mashup where a place is simultaneously a subway station and a virtual store, with pictures of products on the walls where shoppers can scan them for delivery or pickup using the store’s mobile app. It stretches what we assume about how an environment works, and even what a place is.
  • 5. Andrew Hinton | @inkblurt Google Glass Leap Motion Microsoft Kinect We’re being surrounded by sensors and digital agents that are trying to read our actions, or layers of digital information added to what we see around us. left glass image wired.com / others from google Leap motion from their site Kinect from blogs.msdn.com
  • 6. Andrew Hinton | @inkblurt Now, even your refrigerator can spam you. http://www.bbc.co.uk/news/technology-25780908 bestbuy.com Our environment is full of even mundane things that are networked and active in ways we used to only joke about. Like the smart refrigerator that was recently caught being used as a spam bot.
  • 7. Andrew Hinton | @inkblurt Understanding the nature of our materials. We’ve gone for a long time in our work being comfortable with knowing that what we make is confined to a specific screen -- a bounded interactive surface. But now *all* the surfaces around us are potentially digital and networked. Walls and windows, coffee pots and shoes. So we need to re-assess, what is the nature of our materials? Potters have a deep understanding of clay; masons have a deep understanding of stone. We need to deepen our understanding of information.
  • 8. Andrew Hinton | @inkblurt INFORMATION Digital Semantic Physical Information is a word we use a lot, but I don’t think that in our general practice we have a common framework for what it is, and the role it plays in the environments we inhabit. I tried finding a single, official definition, but there really isn’t one -- instead, there are many different ways of looking at various things in our world that people call information. The problem is, all of them are pretty useful and valuable. So rather than just pick one, I decided to pick several and see how they work together and inform each other, so to speak. These modes or ways we experience information are physical, semantic, and digital.
  • 9. Andrew Hinton | @inkblurt Physical Information 10100010 01001000 01110011 10100010 01001000 01110011 Digital systems transmitting to & receiving from other digital systems. Digital People communicating with people. Semantic Animals (including people) perceiving the environment. Physical Physical information has to do with the ecological relationship between an animal and its physical environment. The green field we looked at earlier was mostly made up of physical information that creatures use to know where they can go and what they can do in those surroundings. The way I’m framing this comes from something called ecological psychology, which has been adapted for a branch of study called embodied cognition.
  • 10. Andrew Hinton | @inkblurt Ecological Information "Pick-up" Agent Perception Action Ambient, structured energy arrays Embodied cognition: Body & Environment, not just Brain. Embodied cognition sees the environment and the body as part of a larger dynamic, in which the brain is a participant, but not necessarily the center-point. It may even just be a supporting player. A big part of this is *action*: we don’t perceive the world by sitting still and taking in sensory data; we perceive by moving and interacting with the environment. Much of our daily activity is unconsciously nudged and controlled by our environment.
  • 11. Andrew Hinton | @inkblurt “... there’s no such thing as disembodied cognition [for] living creatures. All of our cognition has to be embodied ... Brains evolved in the service of action.” - Louise Barrett http://podcasts.ox.ac.uk/embodiment-taking-sociality-seriously-audio Louise Barrett, who wrote a great book on embodied cognition called Beyond the Brain, puts it nicely -- -- there’s no such thing as disembodied cognition for living creatures, all of our cognition has to be embodied because brains evolved in the service of action. We didn’t arrive on the planet as brains that grew bodies to supplement their disembodied brain activity. Brains evolved to supplement the abilities of bodies in environments. They’re the product of the structures of their surroundings.
  • 12. Andrew Hinton | @inkblurt James J Gibson - Ecological Psychology of Perception Now seen as pioneer of embodied cognition. Some embodied cognition theorists have heavily adopted the work of James J Gibson, who was a scientist of something called “ecological psychology” in the mid 20th century -- a field he was part of inventing.
  • 13. Andrew Hinton | @inkblurt Ecological Information "Pick-up" Agent Perception Action Ambient, structured energy arrays Gibson used the word “information” to mean the elements we perceive in the environment that tell us what it “affords.” “INFORMATION” Gibson purposefully used the word “information” for this body-environment relationship. He argued that our bodies do something he called “information pickup” -- where the body’s perceptual system detects structural cues in the environment that specify what actions the body can take. And he argued that this happens directly, with little or no need for brain processing.
  • 14. Andrew Hinton | @inkblurt The perceived functional properties of objects, places and events in relation to an individual perceiver. AFFORDANCE Perception exists only insofar as we perceive affordances. Gibson also invented the concept of affordance. For Gibson, affordance isn’t a thing you add to something. It’s the result of a property in the environment that presents the opportunity for action to a perceiving organism. This is true in the built environment as well as the natural one; the structure of these stairs interacts with energy in the environment that my perception picks up on, and my body detects that the specific structure of the steps will take my body upward if climb them. Affordance was and still is a radical idea, in the sense that Gibson meant it. Affordance is the organizing principle behind *ALL* perception. We don’t perceive anything unless it affords meaningful action for a given context. Gibson even went on to discuss how higher-order combinations of affordances can be found in the complexities of human culture. A mailbox, for example, has the affordance for sending mail -- but only because we’ve learned a lot of stuff about the world around that box that makes it a special sort of container connected to a broader cultural system.
  • 15. Andrew Hinton | @inkblurt We perceive environment as “nested,” not in logical hierarchy. We perceive elements in the environment as invariant (persistent) or variant (in flux). Invariance grounds our experience (literally). Invariance Nestedness Two of Gibson’s principles of environmental perception. Gibson created a marvelous system of principles and components that work as building blocks for how we perceive our environment and its affordances. I can only barely touch on them here, but I’ll mention a few important ones. >> Invariance is the property of environmental structure that makes some of it persistent over time, so that we can make sense of the whole. Terrestrial animals like us evolved on stable ground, under a persistent sky. Substances and surfaces that persist allow us to learn what properties they have, and what they afford us. This can be a useful idea in digital design because in software, even though we can make everything fluid and variant, we need to design invariant structures that ground the experience. >> Nestedness: we perceive the environment as structures that relate to one another as nested -- a stream is in a valley, which is between mountains, which are part of a range, which are between earth and sky. There aren’t always definite boundaries, though, and this isn’t a purely logical relationship -- there are overlaps and redundancies, and shifts in meaning. A stick on the ground can be kindling or a club, or it can be part of a roof over our heads in a shelter. It’s not just one defined thing in how it relates to the rest of the environment. This is a powerful idea when we look at something like faceted navigation, where the meaning of a piece of content or an object shifts depending on what perspective a user is bringing to it.
  • 16. Andrew Hinton | @inkblurt Surfaces Objects Layout Place Some of the components that make up environments. There are many components Gibson outlines, like surface, object, layout and place -- that work together to form the environment. There isn’t time to cover it all here -- but I think it’s a useful framework we can borrow from, because all the environments we design should meet the expectations of people’s embodied experience. Our bodies want and need environments to work in these ways.
  • 17. Andrew Hinton | @inkblurt Semantic Information 10100010 01001000 01110011 10100010 01001000 01110011 Digital systems transmitting to & receiving from other digital systems. Digital Animals (including people) perceiving the environment. Physical People communicating with people. Semantic Of course, humans don’t just exist in raw physical surfaces and objects -- we add to the environment. In addition to buildings and roads and such, we add language to the world. The semantic mode, in short, is language. But I mean language in the broad sense of things we put into the environment to communicate with people. This can be all sorts of stuff: speech, gestures, text, iconography, even buildings have semantic qualities.
  • 18. Andrew Hinton | @inkblurt LANGUAGE IS ENVIRONMENT STRUCTURE THAT ENABLES US TO MAKE MORE STRUCTURE Language is “a form of mind- transforming cognitive scaffolding: a ... symbolic edifice [playing a] critical role in promoting thought and reason.” - Andy Clark Supersizing the Mind http://en.wikipedia.org/wiki/File:CantileverScaffold.jpg Language is environment. In another excellent book on embodied cognition, Andy Clark frames language as a form of cognitive scaffolding; it persists in the environment, as new structure that allows us to make other structures. We wouldn’t have most of what we think of as civilization without the scaffolding of language.
  • 19. Andrew Hinton | @inkblurt Language is the primary human interface to complex systems. Language is both the infrastructure we use for creating complexity, as well as the primary interface we have for understanding and inhabiting complex systems.
  • 20. Andrew Hinton | @inkblurt WE LIVE IN LANGUAGE STRUCTURES Language structures are an architecture that we live within together, whether we know it or not. Whether these structures are defined explicitly like in this early IBM management diagram, or defined tacitly through the collective assumptions within a shared culture, the way we talk and write about our shared environment is also a structural feature of that environment.
  • 21. Andrew Hinton | @inkblurt Physical Affordance + Semantic (or “Linguistic”) Affordance Physical Semantic Humans aren’t just perceivers of physical structure. We depend as much or more on the symbolic workings of language – semantic information – as a part of our environment for working together and figuring out our world, and making stuff – as well as just getting around. In this picture of a staircase at City Lights Bookstore in SF, we see stairs that have intrinsic structural information that our bodies perceive as affording the ability to walk upward. But there’s this other thing – a label – which signifies what where we will be going when we go up the stairs. We might call this semantic affordance. It’s an abstracted, symbolic version of how we perceive and comprehend physical structure. In linguistics we also hear of signification -- which I won’t go into in detail here -- but there are signifiers at work in both the stairs and the label. The important point I want to get across here is that affordance is at work even at the high-order level of meaning we have with language, not just in physical structures.
  • 22. Andrew Hinton | @inkblurt Physical Physical + Semantic Simulated- Physical + Semantic Mostly Semantic Affordance across a spectrum of physical to semantic. Affordance is at the root of how we use language as well as the simulated physicality of the stuff we put into interfaces. >> The stairs intrinsically contain all the information needed for your body to know what will happen when you climb them. >> But look at this button. When people started making machines and systems that were more complex, we had physical affordances that did more than we could tell just by touching or looking at them. Because I’ve pushed similar buttons before, I know that the physical information here means I should be able to push in the red part and it will probably spring back out -- that’s all that the physical information is telling me. In order to know what the button is going to cause elsewhere, I need a label. >> When Microsoft updated Windows to add the Start button, they were basically adding an invariant structure to the operating system so that users would know that, no matter what, they could always “start” a new action from here. And like lots of other software, they used a simulated button object, borrowing from what we were used to with buttons like the one on the left. The button isn’t physical information, really ... it’s using semantic information to simulate a physical object. That’s what graphical user interfaces do. >> In other parts of interfaces, and especially on the web with hypertext, labels may not have these simulated object qualities; so we have to rely a lot on the context of a label to know if it just affords reading or if it’s something we can activate. This is the issue at the heart of flat versus skeuomorphic design -- it’s really not about aesthetics, it’s about solving this issue of affordance. Additionally, the web introduced broad usage of labels as paths -- tapping a hyperlink doesn’t just activate some mechanism, it takes you on a journey to what feels to your perception like a new place.
  • 23. Andrew Hinton | @inkblurt Which red x???? Looks like a “confirm” action. Simulated-Physical & Semantic Information In Conflict Here are some examples of where things perform both semantically and ecologically, in confusing ways, in a UI. In this email application, logically speaking, the red X’s are all very different -- but physically they’re barely different at all; they require a lot of thought to disambiguate their semantic meaning. When I was in a hurry, I would reach for the closest red X to do whatever I’m trying to do - close the message, decline an invite, or delete it entirely. About half the time, I would click the wrong one. >>In an unsubscribe interface for fab.com, my wife discovered that she was apparently re- subscribing without realizing it, because that big red button -- like a big berry you can’t help but pick -- contextually feels like it’s a confirmation, not a cancellation/re-subscription action.
  • 24. Andrew Hinton | @inkblurt Animals (including people) perceiving the environment. Physical People communicating with people. Semantic Digital Information 10100010 01001000 01110011 10100010 01001000 01110011 Digital systems transmitting to & receiving from other digital systems. Digital So the examples we just looked at weren’t just any sort of semantic information, they were semantic information driven by digital technology. And digital technology relies on digital information. Digital information is how the black boxes talk to the other black boxes. It’s the lifeblood of information technology. The whole point of digital information is to strip human meaning out of it, which makes it more efficient for machines to transmit, store, and retrieve. This isn’t stuff we see face to face very much. Mainly we encounter its *effects* in the environment.
  • 25. Andrew Hinton | @inkblurt Digital Information Mode Leaking into Semantic Environment We see machines around us trying to get us to perceive what they are saying, or what they want to hear from us. We see them murmuring to each other in weird, noisy machine-only semantics that we do not comprehend either ecologically or semantically. The gas pump has to have a sticker added to it that explains what “Enter Data” means. >>The Twitter profile with the iPhone coordinates expresses my location not in a semantic way (the name of a city, for instance) but in a Cartesian grid that I have no contextual orientation for, either semantically or ecologically. >>The Delta app has information that I, as a human, can read, but it gives priority to the machines that I encounter in the workflow of the airport.
  • 26. Andrew Hinton | @inkblurt Digital information enables pervasive semantic place-making. I don’t mean to paint digital information as a villain. It isn’t. The ability to transmit, store and retrieve information in this way is a miracle. An platform I like a lot is Avocado - it lets a couple keep in touch and share a place together, pervasively. It has nice touches that key into embodied experience of semantic information, like sending a hug by touching the screen to your heart. Another nice touch: the couple shares the same password - making a word into a very real link of co-ownership of the place, like having the same keys to your home. This sort of pervasively available place would be impossible without digital information in the background. But it also requires a lot of discipline with semantic information structure to make the place coherent.
  • 27. Andrew Hinton | @inkblurt Google Plus on Desktop Google Plus on Mobile Easily post content in the circle I’m in. Confusingly requires re-establishing the circle where I’m posting. Broken “invariants” across contexts. Google Plus has invariance in how it establishes place between mobile and desktop versions. When I’m on desktop, and I’m in my “Family” Circle reading posts, when I post something new it automatically goes into that circle, just like I’d expect if I were having a conversation in a room. But on mobile, if I’m in my Family circle, a new post defaults to viewable by everyone; I have to work through a half dozen interactions to make it post just to the circle I perceived I was already “in.”
  • 28. Andrew Hinton | @inkblurt The ontology of “news” is confused here. • News Feed inside News Feed? • If All Friends, is it no longer News? or Recent? • Isn’t News “recent” by definition? • What, then, is News Feed as a place? How am I changing the kind of place it is here? Ontology is the basis that we use for creating any sort of taxonomy - or relationship between semantic invariants. In the mobile version of Facebook’s “news feed” there are several options for how you want to see the feed. Their meanings don’t make much sense together, though -- it’s not clear how these are nested. Is news feed another news feed inside news feed? What does all friends mean? If I choose that, am I not seeing news anymore, or the most recent? Isn’t news recent by definition? What is news-feed as a place? How am I changing the mode of the place when I select something else?
  • 29. Andrew Hinton | @inkblurt What is “card” in this environment? Lowes launched a service called MyLowes -- that requires the registration of a card. But they also have a “Lowe’s Card” that’s a consumer credit card. Conversations at checkout can end up like a “who’s on first” routine -- “do you have your Lowe’s card?” “My Lowe’s card? That’s what I’m paying with.” “No I mean your ‘my lowe’s’ card.” “This IS my lowe’s card!”
  • 30. Andrew Hinton | @inkblurt Shopping Simultaneously in a Store & Online http://stevenglassman.de/2013/11/25/america-vs-deutschland/ In these omni-channel scenarios, the semantic information has to align with how it’s presented both online and physically in the store’s structure. Knowing if a huge store has a specific peanut butter or not is great; knowing exactly which aisle to get it from is even better.
  • 31. Andrew Hinton | @inkblurt The Thing Ontology helps answer: “What is a thing across multiple contexts?” Presently our various fields are preoccupied with how to have content and functionality make sense in various contexts. Ontology is at the heart of this problem. In many organizations and project teams, there’s an over-obsession with things like layout in each of the instantiations of a thing, but not enough discussion about how to define the nature of the thing in abstract. That requires an ontological perspective. And, done properly, it forms the main structures of an information environment - the invariant pillars, so to speak - that allow language to stitch together coherence across channels.
  • 32. Andrew Hinton | @inkblurt LANGUAGE IS INFRASTRUCTURE. LABEL LABEL LABEL LABEL LABEL RULES Labels Connections Rules All this stuff is essentially made out of language... it’s language working as infrastructure. We essentially make things out of labels, connections and rules. Too often, we assume the labels are something to add later - but in reality they’re the thing we have to figure out first. This is why issues like ontology and taxonomy are so important - they establish the “invariant” features of the environments we make.
  • 33. Andrew Hinton | @inkblurt • What are the invariants for perception & action? What grounds the experience? • How is it structurally nested in relation to a perceiver? Is it resilient for different perspectives & contexts? • Is it clear what is a place and what is an object? • How are semantic and physical information working together, or not? • Are the rules behind the environment clearly expressed in its information? Some of the questions we can ask ... So in sum, this is just scratching the surface, but there are some questions we can ask. >>What are the invariants for perception & action? What grounds the experience? >>How is it structurally nested in relation to a perceiver? Is it resilient for different perspectives & contexts? >>Is it clear what is a place and what is an object? >>How are semantic and physical information working together, or not? >>Are the rules behind the environment clearly expressed in its information? We have a long way to go to build heuristics and frameworks for responsibly creating the broader environment that our users inhabit. Hopefully this is a helpful start.
  • 34. Andrew Hinton | @inkblurt THANKS! Andrew Hinton | @inkblurt | andrewhinton.com andrew@understandinggroup.com