3. Augmented Reality Definition
Defining Characteristics [Azuma 97]
Combines Real and Virtual Images
- Both can be seen at the same time
Interactive in real-time
- Virtual content can be interacted with
Registered in 3D
- Virtual objects appear fixed in space
4. What is not Augmented Reality?
Location-based services
Barcode detection (QR-codes)
Augmenting still images
Special effects in movies
…
… but they can be combined with AR!
5. AR vs VR
Virtual Reality: Replaces Reality
Scene Generation: requires realistic images
Display Device: fully immersive, wide FOV
Tracking and Sensing: low accuracy is okay
Augmented Reality: Enhances Reality
Scene Generation: minimal rendering okay
Display Device: non-immersive, small FOV
Tracking and Sensing: high accuracy needed
6. Milgram’s Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays
IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
8. A Brief History of AR (1)
1960’s: Sutherland / Sproull’s
first HMD system was see-
through
9. History Summary
1960’s – 80’s: Early Experimentation
1980’s – 90’s: Basic Research
Tracking, displays
1995 – 2005: Tools/Applications
Interaction, usability, theory
2005 - : Commercial Applications
Games, Medical, Industry
11. 2008 - Browser Based AR
Flash + camera + 3D graphics
High impact
High marketing value
Large potential install base
1.6 Billion web users
Ease of development
Lots of developers, mature tools
Low cost of entry
Browser, web camera
12. 2005 - Mobile Phone AR
Mobile Phones
camera
processor
display
AR on Mobile Phones
Simple graphics
Optimized computer vision
Collaborative Interaction
13. 2009 - Outdoor Information Overlay
Mobile phone based
Tag real world locations
GPS + Compass input
Overlay graphics data on live video
Applications
Travel guide, Advertising, etc
Wikitude, Layar, Junaio, etc..
Android based, Public API released
14. AR Today
Key Technologies Available
- Robust tracking (Computer Vision, GPS/sensors)
- Display (Handheld, HMDs)
- Input Devices (Kinect, etc)
- Developer tools (Qualcomm, Metaio, ARTW)
Commercial Business Growing
- Gaming, GPS/Mobile, Online Advertisement
• >$5 Billion USD by 2016 (Markets andMarkets)
• >$1.5 Billion USD in Mobile AR by 2014 (Juniper Research)
17. Applications: medical
“X-ray vision” for surgeons
Aid visualization, minimally-invasive operations.
Training. MRI, CT data.
Ultrasound project, UNC Chapel Hill.
Courtesy
UNC
Chapel
Hill
18. Medical AR Trials
Sauer et al. 2000 at Siemens
Corporate Research, NJ
Stereo video see through
F. Sauer, Ali Khamene, S. Vogt: An Augmented Reality Navigation System with a
Single-Camera Tracker: System Design and Needle Biopsy Phantom Trial,
MICCAI 2002
25. Interactive Museum Experiences
BlackMagic
Virtual America’s Cup
410,000 people in six months
MagicPlanet
TeManawa science museum
Virtual Astronomy
Collaborative AR experience
ARVolcano
Interactive AR kiosk
Scienceworks museum, Melbourne
27. Museum Archeology
LifePlus (2002-2004)
Natural feature tracking
Virtual characters
Mobile AR system
Archeoguide (2000-2002)
Cultural heritage on-site guide
Hybrid tracking
Virtual overlay
28. Sales and Marketing
Connect with brands and branded objects
Location Based Experiences
Lynx Angels
Web based
Rayban glasses
Mobile
Ford Ka campaign
Print based
Red Bull Magazine
29. Summary
AR technology can be used to develop a wide
range of applications
Promising application areas include
Games
Education
Engineering
Medicine
Museums
Etc..
47. Interaction Design
“Designing interactive products to support people in their
everyday and working lives”
Preece, J., (2002). Interaction Design
Design of User Experience with Technology
Higher in the value chain than product design
49. Interaction Design involves answering three questions:
What do you do? - How do you affect the world?
What do you feel? – What do you sense of the world?
What do you know? – What do you learn?
50. Interaction Design is All About You
Users should be
involved throughout
the Design Process
Consider all the needs
of the user
55. Summary
In order to build AR applications you need to
focus on the user experience
Great user experience is based on
Low level AR component technology
Authoring tools
Application/Interaction design
User experience texting
60. AR Displays
e.g. window
reflections
Virtual Images
seen off windows
e.g. Reach-In
Projection CRT Display
using beamsplitter
Not Head-Mounted
e.g. Shared Space
Magic Book
Liquid Crystal
Displays LCDs
Head-Mounted
Display (HMD)
Primarily Indoor
Environments
e.g. WLVA
and IVRD
Cathode Ray Tube (CRT)
or Virtual Retinal Display (VRD)
Many Military Applications
& Assistive Technologies
Head-Mounted
Display (HMD)
e.g. Head-Up
Display (HUD)
Projection Display
Navigational Aids in Cars
Military Airborne Applications
Not Head Mounted
(e.g. vehicle mounted)
Primarily Outdoor
(Daylight) Environments
AR
Visual Displays
61. Display Technologies
Types (Bimber/Raskar 2003)
Head attached
• Head mounted display/projector
Body attached
• Handheld display/projector
Spatial
• Spatially aligned projector/monitor
64. Head Mounted Displays (HMD)
- Display and Optics mounted on Head
- May or may not fully occlude real world
- Provide full-color images
- Considerations
• Cumbersome to wear
• Brightness
• Low power consumption
• Resolution limited
• Cost is high?
65. Key Properties of HMD
Field of View
Human eye 95 degrees horizontal, 60/70 degrees vertical
Resolution
> 320x240 pixel
Refresh Rate
Focus
Fixed/manual
Power
Size
66. Types of Head Mounted Displays
The image cannot be
displayed. Your
computer may not have
enough memory to open
the image, or the image
may have been
corrupted. Restart your
computer, and then open
the file again. If the red x
still appears, you may
have to delete the image
and then insert it again.
The
ima
ge
can
not
be
dis
play
ed.
You
r
co
mp
uter
Occluded
See-thru
Multiplexed
68. See-thru AR Architecture
Head!
Tracker
Host!
Processor
Data Base!
Model
Rendering!
Engine
Frame!
Buffer
head position/orientation
to network
Display!
Driver
see-thru!
combiner
Virtual Image
superimposed!
over real world
object
real world
Image source
73. The Virtual Retinal Display
Image scanned onto retina
Commercialized through Microvision
Nomad System - www.mvis.com
74. Strengths of optical see-through AR
Simpler (cheaper)
Direct view of real world
Full resolution, no time delay (for real world)
Safety
Lower distortion
No eye displacement
but COASTAR video see-through avoids this
80. Strengths of Video See-Through AR
True occlusion
Virtual images can block view of real world
Digitized image of real world
Flexibility in composition
Matchable time delays
More registration, calibration strategies
Wide FOV is easier to support
81. Optical vs. Video AR Summary
Both have proponents
Video is more popular today?
Likely because lack of available optical products
Depends on application?
Manufacturing: optical is cheaper
Medical: video for calibration strategies
82. Eye multiplexed AR Architecture
Head!
Tracker
Host!
Processor
Data Base!
Model
Rendering!
Engine
Frame!
Buffer
head position/orientation
to network
Display!
Driver
Virtual Image
inset into!
real world scene
real world
Opaque!
Image source
87. Vuzix M-100
▪ Monocular multiplexed display ($1000)
■ 852 x 480 LCD display, 15 deg. FOV
■ 5 MP camera, HD video
■ GPS, gyro, accelerometer
88. Display Types
▪ Curved Mirror
▪ off-axis projection
▪ curved mirrors in front of eye
▪ high distortion, small eye-box
▪ Waveguide
▪ use internal reflection
▪ unobstructed view of world
▪ large eye-box
89. See-through thin displays
▪ Waveguide techniques for thin see-through displays
▪ Wider FOV, enable AR applications
▪ Social acceptability
Opinvent Ora