ISMAR 2008
7th IEEE/ACM International Symposium on Mixed and Augmented Reality, 2008
Proceedings
State of the Art Report
Trends in Augmented Reality Tracking, Interaction and Display
: A Review of Ten Years of ISMAR
Feng Zhou (Center for Human Factors and Ergonomics, Nanyang Technological University, Singapore)
Henry Been-Lirn Duh (Department of Electrical and Computer Engineering/Interactive and Digital Media Institute, National University of Singapore)
Mark Billinghurst (The HIT Lab NZ, University of Canterbury, New Zealand)
Tracking
1. Sensor-based tracking -> ubiquitous tracking and dynamic data fusion
2. Vision-based tracking: feature-based and model-based
1) feature-based tracking techniques:
- To find a correspondence between 2D image features and their 3D world frame coordinates.
- Then to find the camera pose from projecting the 3D coordinates of the feature into the observed 3D image coordinates and minimizing the distance to their corresponding 3D features.
2) model-based tracking techniques:
- To explicitly use a model of the features of tracked objects such as a CAD model or 2D template of the object based on the distinguishable features.
- A visual serving approach adapted from robotics to calculate camera pose from a range of model features (line, circles, cylinders and spheres)
- knowledge about the scene by predicting hidden movement of the object and reducing the effects of outlier data
3. Hybrid tracking
- closed-loop-type tracking based on computer vision techonologies
- motion prediction
- SFM (structure from motion)
- SLAM (simultaneous localization and mapping)
Interaction and User Interfaces
1. Tangible
2. Collaborative
3. Hybrid
Display
1. See-through HMDs
1) OST = optical see-through
: the user to see the real world with virtual objects superimposed on it by optical or video technologies
2) VST = video see-through
: to display graphical infromation directly on real objects or even daily surfaces in everyday life
2. Projection-based Displays
3. Handheld Displays
Limitations of AR
> tracking
1) complexity of the scene and the motion of target objects, including the degrees of freedom of individual objects and their represenation
=> correspondence analysis: Kalman filters, particle filters.
2) how to find distinguishable objects for "markers" outdoors
> interaction
ergonomics, human factors, usability, cognition, HCI (human-computer interaction)
> AR displays
- HMDs - limited FOV, image distortions,
- projector-based displays - lack mobility, self-occlusion
- handheld displays - tracking with markers to limit the work range
Trends and Future Directions
1. Tracking
1) RBPF (Rao-Blackwellized particle filters) -> automatic recognition systems
2) SLAM, ubiquitous tracking, sensor network -> free from prior knowledge
3) pervasive middleware <- information fusion algorithms
2. Interaction and User Interfaces
"Historically, human knowledge, experience and emotion are expressed and communicated in words and pictures. Given the advances in interface and data capturing technology, knowledge, experience and emotion might now be presented in the form of AR content."
3. AR Displays
: software framework for the development of Augmented Reality (AR) and Virtual Reality applications
Graz University of Technology (TU Graz)
Sharedspace project
The Human Interface Technology Laboratory (HITLab) at the University ofWashington and ATR Media Integration & Communication in Kyoto,Japan join forces at SIGGRAPH 99
The Invisible Train - A Handheld Augmented Reality Game
AR Tennis
camera based tracking on mobile phones in face-to-face collaborative Augmented Reality
Emmie - Environment Management for Multi-User Information Environments
VITA: visual interaction tool for archaeology
HMD = head-mounted displays
OST = optical see-through
VST = video see-through
ELMO: an Enhanced optical see-through display using an LCD panel for Mutual Occlusion
FOV
http://en.wikipedia.org/wiki/Field_of_view_(image_processing)
HMPD = head-mounted projective displays
The Touring Machine
MARS - Mobile Augmented Reality Systems
Klimt - the Open Source 3D Graphics Library for Mobile Devices
AR Kanji - The Kanji Teaching application
references
Ronald T. Azuma
http://www.cs.unc.edu/~azuma/
A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355 - 385. Earlier version appeared in Course Notes #9: Developing Advanced Virtual Reality Applications, ACM SIGGRAPH '95 (Los Angeles, CA, 6-11 August 1995), 20-1 to 20-38.
Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner,Simon Julier, Blair MacIntyre
Recent Advances in Augmented Reality.
IEEE Computer Graphics and Applications 21, 6 (Nov/Dec 2001),34-47.
Ivan E. Sutherland
The Ultimate Display, IFIP `65, pp. 506-508, 1965
Kato, H. Billinghurst, M. Poupyrev, I. Imamoto, K. Tachibana, K. Hiroshima City Univ.
Virtual object manipulation on a table-top AR environment
Sandor, C., Olwal, A., Bell, B., and Feiner, S. 2005.
Immersive Mixed-Reality Configuration of Hybrid User Interfaces.
In
Proceedings of the 4th IEEE/ACM international Symposium on Mixed and Augmented Reality(October 05 - 08, 2005). Symposium on Mixed and Augmented Reality. IEEEComputer Society, Washington, DC, 110-113. DOI=http://dx.doi.org/10.1109/ISMAR.2005.37
An optical see-through display for mutual occlusion with a real-time stereovision system
Kiyoshi Kiyokawa, Yoshinori Kurata and Hiroyuki Ohno
Computers & Graphics Volume 25, Issue 5, October 2001, Pages 765-779
Bimber, O., Fröhlich, B., Schmalstieg, D., and Encarnação, L. M. 2005.
The virtual showcase. In
ACM SIGGRAPH 2005 Courses
(Los Angeles, California, July 31 - August 04, 2005). J. Fujii, Ed.
SIGGRAPH '05. ACM, New York, NY, 3. DOI=
http://doi.acm.org/10.1145/1198555.1198713
Bimber, O., Wetzstein, G., Emmerling, A., and Nitschke, C. 2005.
Enabling View-Dependent Stereoscopic Projection in Real Environments.
In
Proceedings of the 4th IEEE/ACM international Symposium on Mixed and Augmented Reality
(October 05 - 08, 2005). Symposium on Mixed and Augmented Reality. IEEE
Computer Society, Washington, DC, 14-23. DOI=
http://dx.doi.org/10.1109/ISMAR.2005.27
Cotting, D., Naef, M., Gross, M., and Fuchs, H. 2004.
Embedding
Imperceptible Patterns into Projected Images for Simultaneous
Acquisition and Display. In
Proceedings of the 3rd IEEE/ACM international Symposium on Mixed and Augmented Reality
(November 02 - 05, 2004). Symposium on Mixed and Augmented Reality.
IEEE Computer Society, Washington, DC, 100-109. DOI=
http://dx.doi.org/10.1109/ISMAR.2004.30
Ehnes, J., Hirota, K., and Hirose, M. 2004.
Projected Augmentation - Augmented Reality using Rotatable Video Projectors. In
Proceedings of the 3rd IEEE/ACM international Symposium on Mixed and Augmented Reality
(November 02 - 05, 2004). Symposium on Mixed and Augmented Reality.
IEEE Computer Society, Washington, DC, 26-35. DOI=
http://dx.doi.org/10.1109/ISMAR.2004.47
Arango, M., Bahler, L., Bates, P., Cochinwala, M., Cohrs, D., Fish, R.,
Gopal, G., Griffeth, N., Herman, G. E., Hickey, T., Lee, K. C., Leland,
W. E., Lowery, C., Mak, V., Patterson, J., Ruston, L., Segal, M.,
Sekar, R. C., Vecchi, M. P., Weinrib, A., and Wuu, S. 1993.
The Touring
Machine system.
Commun. ACM 36, 1 (Jan. 1993), 69-77. DOI= http://doi.acm.org/10.1145/151233.151239
Gupta, S. and Jaynes, C. 2006.
The universal media book: tracking and augmenting moving surfaces with projected information. In
Proceedings of the 2006 Fifth IEEE and ACM international Symposium on Mixed and Augmented Reality (Ismar'06) - Volume 00
(October 22 - 25, 2006). Symposium on Mixed and Augmented Reality. IEEE
Computer Society, Washington, DC, 177-180. DOI=
http://dx.doi.org/10.1109/ISMAR.2006.297811
Klein, G. and Murray, D. 2007.
Parallel Tracking and Mapping for Small AR Workspaces. In
Proceedings of the 2007 6th IEEE and ACM international Symposium on Mixed and Augmented Reality - Volume 00
(November 13 - 16, 2007). Symposium on Mixed and Augmented Reality.
IEEE Computer Society, Washington, DC, 1-10. DOI=
http://dx.doi.org/10.1109/ISMAR.2007.4538852
Neubert, J., Pretlove, J., and Drummond, T. 2007.
Semi-Autonomous
Generation of Appearance-based Edge Models from Image Sequences. In
Proceedings of the 2007 6th IEEE and ACM international Symposium on Mixed and Augmented Reality - Volume 00
(November 13 - 16, 2007). Symposium on Mixed and Augmented Reality.
IEEE Computer Society, Washington, DC, 1-9. DOI=
http://dx.doi.org/10.1109/ISMAR.2007.4538830
모바일 혼합현실 기술.pdf