SlideShare ist ein Scribd-Unternehmen logo
1 von 87
Downloaden Sie, um offline zu lesen
Lecture 6: Collaborative AR
       Applications
         pp
          Mark Billinghurst
                     g
            HIT Lab NZ
      University of Canterbury
Collaboration: What’s that ?
  Definition
f.n.
f n a joint work ; between several peoples who
  generate the creation of a shared task
Collaborative activities
Collaboration: Work, Leisure




CSCW (Computer Supported Collaborative Work)
Groupware
Collaboration: Taxonomy
Time/Space [Ellis91]

          Time

                   bu e boa d
                   bulletin board   pos , email, a
                                    post, e a , fax
   asynchronous
                           Shared editor
                     meeting,            chat,
                  brainstorming,    teleconference
   synchronous       seminar

                                                      Space
                   colocated           distant
Today’s Technology
Video Conferencing
  lack of spatial cues
  limited participants
  2D collaboration
Collaborative Virtual Environments
  separation from real world
  reduced conversational cues
Beyond Video Conferencing
2D Interface onto 3D
  VRML
Projection Screen
  CAVE, WorkBench
Volumetric Display
              p y
  scanning laser
Virtual Reality
  natural spatial cues
Beyond Virtual Reality
Lessons from CSCW
  Seamless
  Enhance Reality
Immersive Virtual Reality
  separates from real world
  reduces conversational cues
Collaboration in the Future ?




Remote Conferencing




                      Face to face Conferencing
Central Claim
Augmented Reality techniques can be used to provide
spatial cues that significantly enhance face-to-face and
remote collaboration on three-dimensional tasks.
Construct3D [Kaufmann 2000]




Collaborative geometry education tool
               g       y
Different learning modes (teacher, student, exam)
Tangible interaction – personal interaction panel
Collaborative Augmented Reality
 Seamless Interaction
 Natural Communication

 Attributes:
 Att ib t
   Virtuality
   Augmentation
   Cooperation
   Independence
   Individuality
Seamless CSCW
Seam (Ishii et. al.)
   spatial, temporal, functional discontinuity

Types of Seams
   Functional
    - between different functional workspaces
   Cognitive
     g
    - between different work practices
Functional Seams
Cognitive Seams
Effect of Seams
Functional Seams:
  Mediated differs from F-to-F Conversation
   - Loss of Gaze Information
   - Degradation of Non-Verbal Cues

Cognitive Seams:
  Learning Curve Effects
  User Frustration
Unanswered Questions
Does seamlessness enhance performance?
What
Wh AR cues can enhance collaboration ?
                     h        ll b
How does AR collaboration differ ?
What technology is required ?
...
Collaborative AR Interfaces
Face to Face Collaboration
WebSpace, Shared Space, Table Top Demo, Interface
    p ,           p ,           p     ,
 Comparison, AR Interface Comparison
Remote Collaboration
SharedView, RTAS, Wearable Info Space, WearCom, AR
  Conferencing, BlockParty
Transitional Interfaces
T          lI     f
MagicBook
Hybrid Interfaces
AR PRISM, GI2VIS
Face to Face Collaboration
Communication Cues
   A wide variety of communication cues used.
                                           sua
                                         Visual
   Audio                            Gaze
                                                        Gesture
Speech                                                  Face Expression
Paralinguistic                                          Body Position
Paraverbals
Prosodics
Intonation




                                 Object Manipulation
                                 Writing/Drawing
                 Environmental   Spatial Relationship
                                 Object Presence
In computer supported collaboration, however, it is often
   hard for users to exchange non-verbal communication
                           g
   cues, even when they are co-located.
Face-to-face collaboration
  People surround a table
  It is easy to see each other

                           Communication S
                           C     i ti Space      Task S
                                                 T k Space

Computer supported
collaboration
  ll b ti
  People sit side by side
  It is hard to see each other
     i h dt            h th


                           Communication Space   Task Space
Shared Space - Table Top Demo
Goal
  create compelling collaborative AR
              p    g
  interface usable by novices
Exhibit content
  matching card game
  face to face collaboration
  physical interaction
Results
2,500 3,000
2 500 - 3 000 users
Observations
  no problems with the interface
        bl     ih h i       f
   - only needed basic instructions
  physical objects easy to manipulate
  spontaneous collaboration
Subjective survey (157 people)
  Users felt they could easily play with other people and
  interact with objects
                   j
Improvements
  reduce lag, improve image quality, better HMD
           g, p          g q      y,
AR Pad
Handheld AR Display
  LCD screen
  SpaceOrb
  Camera
  Peripheral awareness
  P    h l
Support for Collaboration




      Virtual Viewpoint Visualization
Face to Face Collaboration
Compare two person collaboration in:
  Face to Face, AR, Projection Display

Task
  Urban design logic puzzle
   - Arrange 9 building to satisfy 10 rules in 7 minutes

Subjects
  Within subjects study (counter-balanced)
  12 pairs of college students
Face to Face Condition




Moving Model Buildings
AR Condition




Cards with AR Models
SVGA AR Display (800x600)
Video see-through AR
               g
Projection Condition




Tracked Input Devices
Task Space Separation
Interface Conditions
                        FtF                 AR              Projection
User Viewpoint
         p       Independent         Private           Public
                 Easy to change      Independent       Common
                                     Easy to change    Difficult to change
                                     Limited FOV


Interaction      Two handed          Two handed        Mouse-based
                 Natural object      Tangible AR       One-handed
                 manipulation        techniques        Time-multiplexed
                 Space-multiplexed   Space-multiplexed
Hypothesis
Collaboration with AR technology will produce
 behaviors that are more like natural face-to-
 face collaboration than from using a screen-
                                       screen
                based interface.
Metrics
Subjective
  Evaluative survey after each condition
  Forced-choice survey after all conditions
  Post experiment interview

Objective
  j
  Communication measures
   - Video transcription
                   p
Measured Results
Performance
  AR collaboration slower than FtF + Projection
                                        j
Communication
  Pointing/Picking gesture behaviors same in AR as FtF
  Deictic speech patterns same in AR as FtF
   - Both significantly different than Projection condition
            g         y                   j

Subjective
  FtF easier to work together and understand
  Interaction in AR easier than Proj. and same as FtF
Deictic Expressions
            30%


            25%


            20%


            15%

            10%

            5%


            0%
                    FtF        Proj       AR



Significant difference – ANOVA, F(2,33) = 5.77, P < 0.01
No difference between FtF and AR
Ease of Interaction




Significant d ff
S f         difference
      Pick - F(2,69) = 37.8, P < 0.0001
      Move - F(2,69) = 28.4, P < 0.0001
Interview Comments
“AR’s biggest limit was lack of peripheral vision. The interaction was natural, it was
 just difficult to see. In the projection condition you could see everything but the
 interaction was tough”
 Face to Face
     Subjects focused on task space
       - gestures easy to see gaze difficult
                          see,

 Projection display
     Interaction difficult (8/14)
       - not mouse-like, invasion of space

 AR display – “working solo together”
     Lack of peripheral cues = “tunnel vision (10/14 people)
                                tunnel vision”
Face to Face Summary
  Collaboration is partly a Perceptual task
     AR reduces perceptual cues -> Impacts collaboration
     Tangible AR metaphor enhances ease of interaction

  Users felt that AR collaboration different from FtF
  But:
     measured speech and gesture behaviors in AR condition is more similar to
     FtF condition than in Projection display

Thus we need to design AR interfaces that don’t reduce perceptual
  cues, while k
         h l keeping ease of interaction
                           f
Collaborative AR




AR Tennis
  Virtual tennis court
  Two user game
  Audio + haptic feedback
  Bluetooth messaging
Research Questions




Does having an AR interface enhance the face
to face gaming experience?
        g    g p
AR Experience
Conditions
  A: Face to Face AR
  B: Face to Face non-AR
  C: Non Face to Face

Measures
  Survey questions
  Ranking
Survey Questions
1/ How easy was it to work with your partner?
2/ How easily did your partner work with you?
            y     y    p                 y
3/ How easy was it to be aware of what your partner was doing?
4/ How enjoyable was the game?

Answered on a Likert scale from 1 to 7
   1 = not very easy, 7 = very easy
Results


                                              Rankings




User felt AR condition was much easier to see what their
p
partner was doing (Q3)
                 g( )
Easier to work with partner in AR case (Q1)
Easier for partner to work together in AR case (Q2)
Face to Face AR ranked best
Observations and Conclusions
Observations
  Tangible input proved to be very intuitive
  Users preferred audio over haptic cues as it provided
  greater awareness
  Players adapted behavior to interface


Conclusions
  Collaborative AR enhances face to face awareness
Remote Collaboration
2008 - CNN
AR Conferencing
Moves conferencing from the desktop to the workspace
Features
Hardware
  SGI O2
  Virtual i-O HMD
  head mounted camera
Software
  live video
  shared whiteboard
  vision based registration/tracking
Pilot Study
How does AR conferencing differ ?
 Task
    discussing images
    12 pairs of subjects
  Conditions
    audio only ( )
             y (AC)
    video conferencing (VC)
    mixed reality conferencing (MR)
Sample Transcript
Transcript Analysis



Users speak most in Audio Only condition
MR fewest words/min and interruptions/min
More results needed
Presence and Communication
           Presence Rating (0-100)
100
 90
80
70
60
50
40
30
20
10                                             Could tell when Partner was Concentrating
 0                                        14
      AC               VC            MR
                                          12
                                          10
                                          8
                                          6
                                          4
                                          2
                                          0
                                                  AC               VC                MR
Subjective Comments
Paid more attention to pictures
Remote video provided peripheral cues
In AR condition
  Difficult to see everything
  Remote user distracting
  Communication asymmetries
A Wearable Conferencing Space
Features
  mobile video conferencing
  full size images
  spatial audio/visual cues
  interaction with real world
  dozens of users
  body-stabilized data
Initial Prototype
Internet Telephony
Spatial Audio/Visuals
See-through HMD
          g
Head Tracking
Wireless Internet
Wearable Computer
Static Images
Software Architecture
Multicast Groups
Position Broadcasting
P        B d
  10 kb/s per person
Audio Broadcasting
  172 kb/s per person
Local sound spatialization
  DirectSound3D
Graphics Interface
  DirectX/Direct3D
Pilot User Study
Can MR spatial cues aid comprehension?
Task
  recognize words in spoken phrases
Conditions
  Number of speakers
   - 1,3,5 simultaneous speakers
  Spatial/Non Spatial Audio
  Visual/Non visual cues
Spatial Sound
            Spatial vs. Non Spatial Performance

        5
        4
Score




        3                                         SS
        2                                         NS
        1
        0
                1            3           5
                       Num Speakers
Subjective Ratings
                   Ease of Understanding

               6
               5
          5)
 ating (1-5


               4
                                           SS
               3
                                           NS
               2
Ra




               1
               0
                    1        3         5
                        Num Speakers
3D Live: Virtual Viewpoint Generation
Virtual Viewpoint Generation




Interpolate between real camera views to
generate a virtual camera view
AR Conferencing




Superimpose video of remote person over real world
System Architecture
Tangible Manipulation




Using real paddle to move virtual user
AR Remote Conferencing



Progression
  2D to Spatial Cues to 3D
         S i lC
  Increasing realism (visual/audio cues)
Multiscale Collaboration
The MagicBook
Goal: A collaborative AR interface supporting
 transitions from reality to virtual reality
 Physical Components
    Real book

 Display Elements
    p y
    AR and VR content
 Interaction Metaphor
    Book pages hold virtual scenes
Milgram’s Reality-Virtuality continuum

                       Mixed Reality


   Real        Augmented           Augmented          Virtual
Environment    Reality (AR)       Virtuality (AV)   Environment




              Reality - Virtuality (RV) Continuum
Milgram’s Continuum (1994)
                      Mixed Reality (MR)


Reality        Augmented           Augmented            Virtuality
                                                                 y
(Tangible      Reality (AR)        Virtuality (AV)      (Virtual
Interfaces)                                             Reality)


  Central Hypothesis
           yp
       The next generation of interfaces will support transitions
       along the Reality-Virtuality continuum
Transitions
Interfaces of the future will need to support transitions
along the RV continuum
Augmented Reality is preferred for:
  co-located collaboration
Immersive Virtual Reality is preferred for:
  experiencing world immersively (egocentric)
  sharing views
  remote c llab rati n
  rem te collaboration
MagicBook Metaphor
Features
Seamless transition between Reality and Virtuality
  Reliance on real d
  R li           l decreases as virtual increases
                                 i    li
Supports egocentric and exocentric views
  User can pick appropriate view
Computer becomes invisible
  Consistent interface metaphors
  Virtual content seems real
Supports collaboration
Collaboration
Collaboration on multiple levels:
  Physical Object
    y        j
  AR Object
  Immersive Virtual Space
Egocentric + exocentric collaboration
  multiple multi-scale users
       p
Independent Views
  Privacy, role division, scalability
Technology
Reality
  No technology
Augmented Reality
  Camera – tracking
  Switch – fly in
Virtual Reality
  Compass – tracking
  Press pad – move
  Switch – fly out
Scientific Visualization
Education
Hybrid Interfaces
Hybrid User Inerfaces
Goal: To
G l T incorporate AR into normal meeting
                               l
 environment
  Physical Components
     Real props
  Display El
  Di l Elements
             t
     2D and 3D (AR) displays
  Interaction Metaphor
     Use multiple tools – each relevant for the task
Hybrid User Interfaces


    1                   2                 3                   4
   PERSONAL         TABLETOP        WHITEBOARD          MULTIGROUP




Private Display   Private Display   Private Display   Private Display
                  Group Display     Public Display    Group Display
                                                      Public Display
Bridging Space (1)
  Office of the Future
  [Raskar98]
     office environment
     augmented with embedded
     front projection
     3D video conferencing
Bridging Space (2)
Emmie    [Butz99]
  Shared virtual “ether“ metaphor
  Incorporate existing standard applications
Bridging S
              B id i Space (3)
Studierstube (V2.0) [Schmalstieg2000]
  Similar multi-display AR
  Mixed view applications
  Example: Storyboard design
MagicMeeting [Regenbrecht 2002]




Hybrid Interface for Face to Face Collaboration
  AR view, Projection Screen, Desktop, PDA
              j                     p
Tangible Interaction
  Physical manipulators
Tangible Interactions



            Cake Platter Rotation




 Real Props – Cli i Pl
 R lP         Clipping Plane, Obj t Li hti
                              Object Lighting
Conclusions
Lessons Learned
Face to face collaboration
  AR preferred over immersive VR
         f     d      i      i
  AR facilitates seamless/natural communication
Remote C ll b ti
R   t Collaboration
  AR spatial cues can enhance communication
  AR conferencing improves video conferencing
         f      i i          id     f     i
  Many possible confounding factors
Future Work
Wearable collaborative AR system
  opportunistic collaboration
  just in time training
Communication Asymmetries
  interface, expertise, roles
Usability Studies
        y
  multi-user AR systems
  communication tasks

Weitere ähnliche Inhalte

Was ist angesagt?

Augmented Reality pdf
Augmented Reality pdf Augmented Reality pdf
Augmented Reality pdf Qualcomm
 
Designing for AR
Designing for ARDesigning for AR
Designing for AROleg Frolov
 
COMP 4010 Lecture3: Human Perception
COMP 4010 Lecture3: Human PerceptionCOMP 4010 Lecture3: Human Perception
COMP 4010 Lecture3: Human PerceptionMark Billinghurst
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
 
Comp4010 Lecture7 Designing AR Systems
Comp4010 Lecture7 Designing AR SystemsComp4010 Lecture7 Designing AR Systems
Comp4010 Lecture7 Designing AR SystemsMark Billinghurst
 
Augmented Reality & Applications
Augmented Reality & ApplicationsAugmented Reality & Applications
Augmented Reality & ApplicationsJishnu Pradeep
 
Comp4010 Lecture4 AR Tracking and Interaction
Comp4010 Lecture4 AR Tracking and InteractionComp4010 Lecture4 AR Tracking and Interaction
Comp4010 Lecture4 AR Tracking and InteractionMark Billinghurst
 
Augmented Reality - 8th Mass Media
Augmented Reality - 8th Mass MediaAugmented Reality - 8th Mass Media
Augmented Reality - 8th Mass MediaQualcomm
 
Virtual reality in hci
Virtual reality in hciVirtual reality in hci
Virtual reality in hcijeet patalia
 
Immersive Virtual Reality
Immersive Virtual RealityImmersive Virtual Reality
Immersive Virtual RealitySebastien Kuntz
 
Comp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VRComp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VRMark Billinghurst
 
COMP 4010 Lecture9 AR Interaction
COMP 4010 Lecture9 AR InteractionCOMP 4010 Lecture9 AR Interaction
COMP 4010 Lecture9 AR InteractionMark Billinghurst
 

Was ist angesagt? (20)

Augmented reality
Augmented realityAugmented reality
Augmented reality
 
Augmented Reality pdf
Augmented Reality pdf Augmented Reality pdf
Augmented Reality pdf
 
Designing for AR
Designing for ARDesigning for AR
Designing for AR
 
COMP 4010 Lecture3: Human Perception
COMP 4010 Lecture3: Human PerceptionCOMP 4010 Lecture3: Human Perception
COMP 4010 Lecture3: Human Perception
 
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationTalk to Me: Using Virtual Avatars to Improve Remote Collaboration
Talk to Me: Using Virtual Avatars to Improve Remote Collaboration
 
ISS2022 Keynote
ISS2022 KeynoteISS2022 Keynote
ISS2022 Keynote
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
Comp4010 Lecture7 Designing AR Systems
Comp4010 Lecture7 Designing AR SystemsComp4010 Lecture7 Designing AR Systems
Comp4010 Lecture7 Designing AR Systems
 
Augmented Reality & Applications
Augmented Reality & ApplicationsAugmented Reality & Applications
Augmented Reality & Applications
 
Comp4010 Lecture4 AR Tracking and Interaction
Comp4010 Lecture4 AR Tracking and InteractionComp4010 Lecture4 AR Tracking and Interaction
Comp4010 Lecture4 AR Tracking and Interaction
 
Augmented Reality - 8th Mass Media
Augmented Reality - 8th Mass MediaAugmented Reality - 8th Mass Media
Augmented Reality - 8th Mass Media
 
Virtual reality in hci
Virtual reality in hciVirtual reality in hci
Virtual reality in hci
 
Virtual Reality
Virtual RealityVirtual Reality
Virtual Reality
 
Immersive Virtual Reality
Immersive Virtual RealityImmersive Virtual Reality
Immersive Virtual Reality
 
Comp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VRComp4010 Lecture8 Introduction to VR
Comp4010 Lecture8 Introduction to VR
 
Virtual reality
Virtual realityVirtual reality
Virtual reality
 
COMP 4010 Lecture9 AR Interaction
COMP 4010 Lecture9 AR InteractionCOMP 4010 Lecture9 AR Interaction
COMP 4010 Lecture9 AR Interaction
 
Augmented reality
Augmented realityAugmented reality
Augmented reality
 
Augmented Reality
Augmented RealityAugmented Reality
Augmented Reality
 
Augmented reality
Augmented realityAugmented reality
Augmented reality
 

Ähnlich wie COSC 426 Lect. 6: Collaborative AR

Tangible AR Interface
Tangible AR InterfaceTangible AR Interface
Tangible AR InterfaceJongHyoun
 
COSC 426 lect. 4: AR Interaction
COSC 426 lect. 4: AR InteractionCOSC 426 lect. 4: AR Interaction
COSC 426 lect. 4: AR InteractionMark Billinghurst
 
Building Usable AR Interfaces
Building Usable AR InterfacesBuilding Usable AR Interfaces
Building Usable AR InterfacesMark Billinghurst
 
426 lecture 7: Designing AR Interfaces
426 lecture 7: Designing AR Interfaces426 lecture 7: Designing AR Interfaces
426 lecture 7: Designing AR InterfacesMark Billinghurst
 
Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...haochuan
 
2013 Lecture 6: AR User Interface Design Guidelines
2013 Lecture 6: AR User Interface Design Guidelines2013 Lecture 6: AR User Interface Design Guidelines
2013 Lecture 6: AR User Interface Design GuidelinesMark Billinghurst
 
Natural Interaction for Augmented Reality Applications
Natural Interaction for Augmented Reality ApplicationsNatural Interaction for Augmented Reality Applications
Natural Interaction for Augmented Reality ApplicationsMark Billinghurst
 
426 lecture6b: AR Interaction
426 lecture6b: AR Interaction426 lecture6b: AR Interaction
426 lecture6b: AR InteractionMark Billinghurst
 
互動式介面設計
互動式介面設計互動式介面設計
互動式介面設計瑞彣 高
 
The Reality of Augmented Reality: Are we there yet?
The Reality of Augmented Reality: Are we there yet?The Reality of Augmented Reality: Are we there yet?
The Reality of Augmented Reality: Are we there yet?Mark Billinghurst
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseMark Billinghurst
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR PrototypingMark Billinghurst
 
What User Interface to Use for VR: 2D, 3D or Speech – A User Study
What User Interface to Use for VR: 2D, 3D or Speech – A User StudyWhat User Interface to Use for VR: 2D, 3D or Speech – A User Study
What User Interface to Use for VR: 2D, 3D or Speech – A User StudyMatthias Wölfel
 
Talking to the hand
Talking to the handTalking to the hand
Talking to the handChris Lee
 
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration software
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration softwareSeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration software
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration softwareProcessOne
 
Multimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMultimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMark Billinghurst
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsMark Billinghurst
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR SystemsMark Billinghurst
 

Ähnlich wie COSC 426 Lect. 6: Collaborative AR (20)

Tangible AR Interface
Tangible AR InterfaceTangible AR Interface
Tangible AR Interface
 
COSC 426 lect. 4: AR Interaction
COSC 426 lect. 4: AR InteractionCOSC 426 lect. 4: AR Interaction
COSC 426 lect. 4: AR Interaction
 
Can You See What I See?
Can You See What I See?Can You See What I See?
Can You See What I See?
 
Building Usable AR Interfaces
Building Usable AR InterfacesBuilding Usable AR Interfaces
Building Usable AR Interfaces
 
426 lecture 7: Designing AR Interfaces
426 lecture 7: Designing AR Interfaces426 lecture 7: Designing AR Interfaces
426 lecture 7: Designing AR Interfaces
 
Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...Kinect-taped communication: Using motion sensing to study gesture use and sim...
Kinect-taped communication: Using motion sensing to study gesture use and sim...
 
2013 Lecture 6: AR User Interface Design Guidelines
2013 Lecture 6: AR User Interface Design Guidelines2013 Lecture 6: AR User Interface Design Guidelines
2013 Lecture 6: AR User Interface Design Guidelines
 
Natural Interaction for Augmented Reality Applications
Natural Interaction for Augmented Reality ApplicationsNatural Interaction for Augmented Reality Applications
Natural Interaction for Augmented Reality Applications
 
426 lecture6b: AR Interaction
426 lecture6b: AR Interaction426 lecture6b: AR Interaction
426 lecture6b: AR Interaction
 
互動式介面設計
互動式介面設計互動式介面設計
互動式介面設計
 
The Reality of Augmented Reality: Are we there yet?
The Reality of Augmented Reality: Are we there yet?The Reality of Augmented Reality: Are we there yet?
The Reality of Augmented Reality: Are we there yet?
 
Empathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole MetaverseEmpathic Computing: Developing for the Whole Metaverse
Empathic Computing: Developing for the Whole Metaverse
 
2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping2022 COMP4010 Lecture5: AR Prototyping
2022 COMP4010 Lecture5: AR Prototyping
 
What User Interface to Use for VR: 2D, 3D or Speech – A User Study
What User Interface to Use for VR: 2D, 3D or Speech – A User StudyWhat User Interface to Use for VR: 2D, 3D or Speech – A User Study
What User Interface to Use for VR: 2D, 3D or Speech – A User Study
 
Hands and Speech in Space
Hands and Speech in SpaceHands and Speech in Space
Hands and Speech in Space
 
Talking to the hand
Talking to the handTalking to the hand
Talking to the hand
 
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration software
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration softwareSeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration software
SeaBeyond 2011 ProcessOne - Marek Foss: designing mobile collaboration software
 
Multimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed RealityMultimodal Multi-sensory Interaction for Mixed Reality
Multimodal Multi-sensory Interaction for Mixed Reality
 
Comp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research DirectionsComp4010 Lecture13 More Research Directions
Comp4010 Lecture13 More Research Directions
 
Novel Interfaces for AR Systems
Novel Interfaces for AR SystemsNovel Interfaces for AR Systems
Novel Interfaces for AR Systems
 

Mehr von Mark Billinghurst

Future Research Directions for Augmented Reality
Future Research Directions for Augmented RealityFuture Research Directions for Augmented Reality
Future Research Directions for Augmented RealityMark Billinghurst
 
Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesMark Billinghurst
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the MetaverseMark Billinghurst
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseMark Billinghurst
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseMark Billinghurst
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR SystemsMark Billinghurst
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR InteractionMark Billinghurst
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR TechnologyMark Billinghurst
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: PerceptionMark Billinghurst
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsMark Billinghurst
 
Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsMark Billinghurst
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsMark Billinghurst
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsMark Billinghurst
 
Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Mark Billinghurst
 
Comp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignComp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignMark Billinghurst
 
Comp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsComp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsMark Billinghurst
 
Advanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise ARAdvanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise ARMark Billinghurst
 
Comp4010 lecture6 Prototyping
Comp4010 lecture6 PrototypingComp4010 lecture6 Prototyping
Comp4010 lecture6 PrototypingMark Billinghurst
 

Mehr von Mark Billinghurst (19)

Future Research Directions for Augmented Reality
Future Research Directions for Augmented RealityFuture Research Directions for Augmented Reality
Future Research Directions for Augmented Reality
 
Evaluation Methods for Social XR Experiences
Evaluation Methods for Social XR ExperiencesEvaluation Methods for Social XR Experiences
Evaluation Methods for Social XR Experiences
 
Empathic Computing: Delivering the Potential of the Metaverse
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
 
Empathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the MetaverseEmpathic Computing: Capturing the Potential of the Metaverse
Empathic Computing: Capturing the Potential of the Metaverse
 
Empathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader MetaverseEmpathic Computing: Designing for the Broader Metaverse
Empathic Computing: Designing for the Broader Metaverse
 
2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems2022 COMP4010 Lecture 6: Designing AR Systems
2022 COMP4010 Lecture 6: Designing AR Systems
 
2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction2022 COMP4010 Lecture4: AR Interaction
2022 COMP4010 Lecture4: AR Interaction
 
2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology2022 COMP4010 Lecture3: AR Technology
2022 COMP4010 Lecture3: AR Technology
 
2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception2022 COMP4010 Lecture2: Perception
2022 COMP4010 Lecture2: Perception
 
Empathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive AnalyticsEmpathic Computing and Collaborative Immersive Analytics
Empathic Computing and Collaborative Immersive Analytics
 
Metaverse Learning
Metaverse LearningMetaverse Learning
Metaverse Learning
 
Comp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research DirectionsComp4010 Lecture12 Research Directions
Comp4010 Lecture12 Research Directions
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR Applications
 
Comp4010 lecture11 VR Applications
Comp4010 lecture11 VR ApplicationsComp4010 lecture11 VR Applications
Comp4010 lecture11 VR Applications
 
Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality Grand Challenges for Mixed Reality
Grand Challenges for Mixed Reality
 
Comp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface DesignComp4010 Lecture10 VR Interface Design
Comp4010 Lecture10 VR Interface Design
 
Comp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and SystemsComp4010 Lecture9 VR Input and Systems
Comp4010 Lecture9 VR Input and Systems
 
Advanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise ARAdvanced Methods for User Evaluation in Enterprise AR
Advanced Methods for User Evaluation in Enterprise AR
 
Comp4010 lecture6 Prototyping
Comp4010 lecture6 PrototypingComp4010 lecture6 Prototyping
Comp4010 lecture6 Prototyping
 

Kürzlich hochgeladen

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 

Kürzlich hochgeladen (20)

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 

COSC 426 Lect. 6: Collaborative AR

  • 1. Lecture 6: Collaborative AR Applications pp Mark Billinghurst g HIT Lab NZ University of Canterbury
  • 2. Collaboration: What’s that ? Definition f.n. f n a joint work ; between several peoples who generate the creation of a shared task
  • 3. Collaborative activities Collaboration: Work, Leisure CSCW (Computer Supported Collaborative Work) Groupware
  • 4. Collaboration: Taxonomy Time/Space [Ellis91] Time bu e boa d bulletin board pos , email, a post, e a , fax asynchronous Shared editor meeting, chat, brainstorming, teleconference synchronous seminar Space colocated distant
  • 5. Today’s Technology Video Conferencing lack of spatial cues limited participants 2D collaboration Collaborative Virtual Environments separation from real world reduced conversational cues
  • 6. Beyond Video Conferencing 2D Interface onto 3D VRML Projection Screen CAVE, WorkBench Volumetric Display p y scanning laser Virtual Reality natural spatial cues
  • 7. Beyond Virtual Reality Lessons from CSCW Seamless Enhance Reality Immersive Virtual Reality separates from real world reduces conversational cues
  • 8. Collaboration in the Future ? Remote Conferencing Face to face Conferencing
  • 9. Central Claim Augmented Reality techniques can be used to provide spatial cues that significantly enhance face-to-face and remote collaboration on three-dimensional tasks.
  • 10. Construct3D [Kaufmann 2000] Collaborative geometry education tool g y Different learning modes (teacher, student, exam) Tangible interaction – personal interaction panel
  • 11. Collaborative Augmented Reality Seamless Interaction Natural Communication Attributes: Att ib t Virtuality Augmentation Cooperation Independence Individuality
  • 12. Seamless CSCW Seam (Ishii et. al.) spatial, temporal, functional discontinuity Types of Seams Functional - between different functional workspaces Cognitive g - between different work practices
  • 15. Effect of Seams Functional Seams: Mediated differs from F-to-F Conversation - Loss of Gaze Information - Degradation of Non-Verbal Cues Cognitive Seams: Learning Curve Effects User Frustration
  • 16. Unanswered Questions Does seamlessness enhance performance? What Wh AR cues can enhance collaboration ? h ll b How does AR collaboration differ ? What technology is required ? ...
  • 17. Collaborative AR Interfaces Face to Face Collaboration WebSpace, Shared Space, Table Top Demo, Interface p , p , p , Comparison, AR Interface Comparison Remote Collaboration SharedView, RTAS, Wearable Info Space, WearCom, AR Conferencing, BlockParty Transitional Interfaces T lI f MagicBook Hybrid Interfaces AR PRISM, GI2VIS
  • 18. Face to Face Collaboration
  • 19. Communication Cues A wide variety of communication cues used. sua Visual Audio Gaze Gesture Speech Face Expression Paralinguistic Body Position Paraverbals Prosodics Intonation Object Manipulation Writing/Drawing Environmental Spatial Relationship Object Presence
  • 20. In computer supported collaboration, however, it is often hard for users to exchange non-verbal communication g cues, even when they are co-located.
  • 21. Face-to-face collaboration People surround a table It is easy to see each other Communication S C i ti Space Task S T k Space Computer supported collaboration ll b ti People sit side by side It is hard to see each other i h dt h th Communication Space Task Space
  • 22. Shared Space - Table Top Demo Goal create compelling collaborative AR p g interface usable by novices Exhibit content matching card game face to face collaboration physical interaction
  • 23. Results 2,500 3,000 2 500 - 3 000 users Observations no problems with the interface bl ih h i f - only needed basic instructions physical objects easy to manipulate spontaneous collaboration Subjective survey (157 people) Users felt they could easily play with other people and interact with objects j Improvements reduce lag, improve image quality, better HMD g, p g q y,
  • 24. AR Pad Handheld AR Display LCD screen SpaceOrb Camera Peripheral awareness P h l
  • 25. Support for Collaboration Virtual Viewpoint Visualization
  • 26. Face to Face Collaboration Compare two person collaboration in: Face to Face, AR, Projection Display Task Urban design logic puzzle - Arrange 9 building to satisfy 10 rules in 7 minutes Subjects Within subjects study (counter-balanced) 12 pairs of college students
  • 27. Face to Face Condition Moving Model Buildings
  • 28. AR Condition Cards with AR Models SVGA AR Display (800x600) Video see-through AR g
  • 31. Interface Conditions FtF AR Projection User Viewpoint p Independent Private Public Easy to change Independent Common Easy to change Difficult to change Limited FOV Interaction Two handed Two handed Mouse-based Natural object Tangible AR One-handed manipulation techniques Time-multiplexed Space-multiplexed Space-multiplexed
  • 32. Hypothesis Collaboration with AR technology will produce behaviors that are more like natural face-to- face collaboration than from using a screen- screen based interface.
  • 33. Metrics Subjective Evaluative survey after each condition Forced-choice survey after all conditions Post experiment interview Objective j Communication measures - Video transcription p
  • 34. Measured Results Performance AR collaboration slower than FtF + Projection j Communication Pointing/Picking gesture behaviors same in AR as FtF Deictic speech patterns same in AR as FtF - Both significantly different than Projection condition g y j Subjective FtF easier to work together and understand Interaction in AR easier than Proj. and same as FtF
  • 35. Deictic Expressions 30% 25% 20% 15% 10% 5% 0% FtF Proj AR Significant difference – ANOVA, F(2,33) = 5.77, P < 0.01 No difference between FtF and AR
  • 36. Ease of Interaction Significant d ff S f difference Pick - F(2,69) = 37.8, P < 0.0001 Move - F(2,69) = 28.4, P < 0.0001
  • 37. Interview Comments “AR’s biggest limit was lack of peripheral vision. The interaction was natural, it was just difficult to see. In the projection condition you could see everything but the interaction was tough” Face to Face Subjects focused on task space - gestures easy to see gaze difficult see, Projection display Interaction difficult (8/14) - not mouse-like, invasion of space AR display – “working solo together” Lack of peripheral cues = “tunnel vision (10/14 people) tunnel vision”
  • 38. Face to Face Summary Collaboration is partly a Perceptual task AR reduces perceptual cues -> Impacts collaboration Tangible AR metaphor enhances ease of interaction Users felt that AR collaboration different from FtF But: measured speech and gesture behaviors in AR condition is more similar to FtF condition than in Projection display Thus we need to design AR interfaces that don’t reduce perceptual cues, while k h l keeping ease of interaction f
  • 39. Collaborative AR AR Tennis Virtual tennis court Two user game Audio + haptic feedback Bluetooth messaging
  • 40. Research Questions Does having an AR interface enhance the face to face gaming experience? g g p
  • 41. AR Experience Conditions A: Face to Face AR B: Face to Face non-AR C: Non Face to Face Measures Survey questions Ranking
  • 42. Survey Questions 1/ How easy was it to work with your partner? 2/ How easily did your partner work with you? y y p y 3/ How easy was it to be aware of what your partner was doing? 4/ How enjoyable was the game? Answered on a Likert scale from 1 to 7 1 = not very easy, 7 = very easy
  • 43. Results Rankings User felt AR condition was much easier to see what their p partner was doing (Q3) g( ) Easier to work with partner in AR case (Q1) Easier for partner to work together in AR case (Q2) Face to Face AR ranked best
  • 44. Observations and Conclusions Observations Tangible input proved to be very intuitive Users preferred audio over haptic cues as it provided greater awareness Players adapted behavior to interface Conclusions Collaborative AR enhances face to face awareness
  • 47. AR Conferencing Moves conferencing from the desktop to the workspace
  • 48. Features Hardware SGI O2 Virtual i-O HMD head mounted camera Software live video shared whiteboard vision based registration/tracking
  • 49. Pilot Study How does AR conferencing differ ? Task discussing images 12 pairs of subjects Conditions audio only ( ) y (AC) video conferencing (VC) mixed reality conferencing (MR)
  • 51. Transcript Analysis Users speak most in Audio Only condition MR fewest words/min and interruptions/min More results needed
  • 52. Presence and Communication Presence Rating (0-100) 100 90 80 70 60 50 40 30 20 10 Could tell when Partner was Concentrating 0 14 AC VC MR 12 10 8 6 4 2 0 AC VC MR
  • 53. Subjective Comments Paid more attention to pictures Remote video provided peripheral cues In AR condition Difficult to see everything Remote user distracting Communication asymmetries
  • 54. A Wearable Conferencing Space Features mobile video conferencing full size images spatial audio/visual cues interaction with real world dozens of users body-stabilized data
  • 55. Initial Prototype Internet Telephony Spatial Audio/Visuals See-through HMD g Head Tracking Wireless Internet Wearable Computer Static Images
  • 56. Software Architecture Multicast Groups Position Broadcasting P B d 10 kb/s per person Audio Broadcasting 172 kb/s per person Local sound spatialization DirectSound3D Graphics Interface DirectX/Direct3D
  • 57. Pilot User Study Can MR spatial cues aid comprehension? Task recognize words in spoken phrases Conditions Number of speakers - 1,3,5 simultaneous speakers Spatial/Non Spatial Audio Visual/Non visual cues
  • 58. Spatial Sound Spatial vs. Non Spatial Performance 5 4 Score 3 SS 2 NS 1 0 1 3 5 Num Speakers
  • 59. Subjective Ratings Ease of Understanding 6 5 5) ating (1-5 4 SS 3 NS 2 Ra 1 0 1 3 5 Num Speakers
  • 60. 3D Live: Virtual Viewpoint Generation
  • 61. Virtual Viewpoint Generation Interpolate between real camera views to generate a virtual camera view
  • 62. AR Conferencing Superimpose video of remote person over real world
  • 64. Tangible Manipulation Using real paddle to move virtual user
  • 65. AR Remote Conferencing Progression 2D to Spatial Cues to 3D S i lC Increasing realism (visual/audio cues)
  • 67. The MagicBook Goal: A collaborative AR interface supporting transitions from reality to virtual reality Physical Components Real book Display Elements p y AR and VR content Interaction Metaphor Book pages hold virtual scenes
  • 68. Milgram’s Reality-Virtuality continuum Mixed Reality Real Augmented Augmented Virtual Environment Reality (AR) Virtuality (AV) Environment Reality - Virtuality (RV) Continuum
  • 69. Milgram’s Continuum (1994) Mixed Reality (MR) Reality Augmented Augmented Virtuality y (Tangible Reality (AR) Virtuality (AV) (Virtual Interfaces) Reality) Central Hypothesis yp The next generation of interfaces will support transitions along the Reality-Virtuality continuum
  • 70. Transitions Interfaces of the future will need to support transitions along the RV continuum Augmented Reality is preferred for: co-located collaboration Immersive Virtual Reality is preferred for: experiencing world immersively (egocentric) sharing views remote c llab rati n rem te collaboration
  • 72. Features Seamless transition between Reality and Virtuality Reliance on real d R li l decreases as virtual increases i li Supports egocentric and exocentric views User can pick appropriate view Computer becomes invisible Consistent interface metaphors Virtual content seems real Supports collaboration
  • 73. Collaboration Collaboration on multiple levels: Physical Object y j AR Object Immersive Virtual Space Egocentric + exocentric collaboration multiple multi-scale users p Independent Views Privacy, role division, scalability
  • 74. Technology Reality No technology Augmented Reality Camera – tracking Switch – fly in Virtual Reality Compass – tracking Press pad – move Switch – fly out
  • 78. Hybrid User Inerfaces Goal: To G l T incorporate AR into normal meeting l environment Physical Components Real props Display El Di l Elements t 2D and 3D (AR) displays Interaction Metaphor Use multiple tools – each relevant for the task
  • 79. Hybrid User Interfaces 1 2 3 4 PERSONAL TABLETOP WHITEBOARD MULTIGROUP Private Display Private Display Private Display Private Display Group Display Public Display Group Display Public Display
  • 80. Bridging Space (1) Office of the Future [Raskar98] office environment augmented with embedded front projection 3D video conferencing
  • 81. Bridging Space (2) Emmie [Butz99] Shared virtual “ether“ metaphor Incorporate existing standard applications
  • 82. Bridging S B id i Space (3) Studierstube (V2.0) [Schmalstieg2000] Similar multi-display AR Mixed view applications Example: Storyboard design
  • 83. MagicMeeting [Regenbrecht 2002] Hybrid Interface for Face to Face Collaboration AR view, Projection Screen, Desktop, PDA j p Tangible Interaction Physical manipulators
  • 84. Tangible Interactions Cake Platter Rotation Real Props – Cli i Pl R lP Clipping Plane, Obj t Li hti Object Lighting
  • 86. Lessons Learned Face to face collaboration AR preferred over immersive VR f d i i AR facilitates seamless/natural communication Remote C ll b ti R t Collaboration AR spatial cues can enhance communication AR conferencing improves video conferencing f i i id f i Many possible confounding factors
  • 87. Future Work Wearable collaborative AR system opportunistic collaboration just in time training Communication Asymmetries interface, expertise, roles Usability Studies y multi-user AR systems communication tasks