기존 SLAM에서 쓰이는 레이저 레인지 스캐너 등 range and bearing 센서 대신 공간에 대한 풍부한 정보를 주는 카메라를 쓰면, 1차원 (인식된 물체까지의 거리 정보, depth)을 잃게 되어 bearing-only SLAM이 된다.
EKF requires Gaussian representations for all the involved random variables that form the map (the robot pose and all landmark's positions). Moreover, their variances need to be small to be able to approximate all the non linear functions with their linearized forms.
두 입력 이미지 프레임 사이에 baseline을 구할 수 있을 만큼 충분한 시점 차가 존재해야 랜드마크의 위치를 결정할 수 있으므로, 이를 확보하기 위한 시간이 필요하게 된다.
Danica Kragic1 and Markus Vincze2 1 Centre for Autonomous Systems,
Computational Vision and Active Perception Lab, School of Computer
Science and Communication, KTH, Stockholm, 10044, Sweden, dani@kth.se 2 Vision for Robotics Lab, Automation and Control Institute, Technische Universitat Wien, Vienna, Austria, vincze@acin.tuwien.ac.at
SUGGESTED CITATION: Danica Kragic and Markus Vincze (2010) “Vision for Robotics”,
Foundations and Trends® in Robotics: Vol. 1: No. 1, pp 1–78.
http:/dx.doi.org/10.1561/2300000001
Abstract
Robot vision refers to the capability of a robot to
visually perceive the environment and use this information for
execution of various tasks. Visual feedback has been used extensively
for robot navigation and obstacle avoidance. In the recent years, there
are also examples that include interaction with people and manipulation
of objects. In this paper, we review some of the work that goes beyond
of using artificial landmarks and fiducial markers for the purpose of
implementing visionbased control in robots. We discuss different
application areas, both from the systems perspective and individual
problems such as object tracking and recognition.
1 Introduction 2
1.1 Scope and Outline 4
2 Historical Perspective 7
2.1 Early Start and Industrial Applications 7
2.2 Biological Influences and Affordances 9
2.3 Vision Systems 12
3 What Works 17
3.1 Object Tracking and Pose Estimation 18
3.2 Visual Servoing–Arms and Platforms 27
3.3 Reconstruction, Localization, Navigation, and Visual SLAM 32
3.4 Object Recognition 35
3.5 Action Recognition, Detecting, and Tracking Humans 42
3.6 Search and Attention 44
4 Open Challenges 48
4.1 Shape and Structure for Object Detection 49
4.2 Object Categorization 52
4.3 Semantics and Symbol Grounding: From Robot Task to Grasping and HRI 54
4.4 Competitions and Benchmarking 56
D-SLAM: A Decoupled Solution to Simultaneous Localization and Mapping
Z. Wang, S. Huang and G. Dissanayake
ARC Centre of Excellence for Autonomous Systems (CAS), Faculty of Engineering, University of Technology, Sydney, Australia International Journal of Robotics Research Volume 26 Issue 2 - Publication Date: 1 February 2007 (Special Issue on the Fifth International Conference on Field and Service Robotics, 2005) http://dx.doi.org/10.1177/0278364906075173
On the Structure and Solution of the Simultaneous Localisation and Map Building Problem. Paul Michael Newman.
1999. Ph. D. thesis, Australian Centre for Field Robotics - The University of Sydney
Randall C. Smith and Peter Cheeseman. 1986. On the representation and estimation of spatial uncertainly. Int. J. Rob. Res. 5, 4 (December 1986), 56-68.
DOI=10.1177/027836498600500404 http://dx.doi.org/10.1177/027836498600500404
// ground truth of num_landmarks landmarks in the world coordinate
double landmark[num_landmarks];// = { 200, 600, 400 }; // x-position
for( int n = 0; n < num_landmarks; n++ )
{
landmark[n] = width * uniform_random();
}
// set the initial state
double rob_pos = 25.0; // initial robot position
double rob_vel = 10.0; // initial robot velocity
// set the initial covariance of the state
double rob_pos_cov = 0.01; // covariance of noise to robot position
double rob_vel_cov = 0.01; // covariance of noise to robot velocity
double obs_cov = 900; // covarriance of noise to measurement of landmarks
double xGroundTruth, vGroundTruth;
xGroundTruth = rob_pos;
vGroundTruth = rob_vel;
// H matrix
int Hrow = num_landmarks;
int Hcol = num_landmarks + 2;
CvMat* H = cvCreateMat(Hrow, Hcol, CV_64FC1);
cvZero(H); // initialize H matrix
// set H matrix
for (int row = 0; row < Hrow; row++)
{
cvmSet(H, row, 0, -1.0);
cvmSet(H, row, row+2, 1.0);
}
displayMatrix(H, "H matrix");
// Q matrix ; covariance of noise to H ; uncertainty of control
CvMat* Q = cvCreateMat(Hrow, Hrow, CV_64FC1);
cvZero(Q); // initialize Q matrix
// set Q matrix
for (int row = 0; row < Q->rows; row++)
{
cvmSet(Q, row, row, obs_cov);
}
displayMatrix(Q, "Q matrix");
// G matrix // transition
int Grow = num_landmarks + 2;
int Gcol = Grow;
CvMat* G = cvCreateMat(Grow, Gcol, CV_64FC1);
cvZero(G); // initialize G matrix
// set G matrix
cvmSet(G, 0, 0, 1.0); // previous position
cvmSet(G, 0, 1, 1.0); // velocity
for (int row = 1; row < Grow; row++)
{
cvmSet(G, row, row, 1.0); // constance of velocity
}
displayMatrix(G, "G matrix");
// R matrix ; covariance of noise to H ; uncertainty of observation
CvMat* R = cvCreateMat(Grow, Grow, CV_64FC1); // 5x5
cvZero(R); // initialize R matrix
// set R matrix
cvmSet(R, 0, 0, rob_pos_cov);
cvmSet(R, 1, 1, rob_vel_cov);
displayMatrix(R, "R matrix");
CvMat* mu = cvCreateMat(num_dim, 1, CV_64FC1); // state vector to be estimated
CvMat* rob_ground = cvCreateMat(num_dim, 1, CV_64FC1); // state vector to be estimated
CvMat* mu_p = cvCreateMat(num_dim, 1, CV_64FC1); // state to be predicted
CvMat* u = cvCreateMat(1, 1, CV_64FC1); // control vector
cvmSet(u, 0, 0, 1.0); // set u(0,0) to 1.0, the constant velocity here
CvMat* sigma = cvCreateMat(num_dim, num_dim, CV_64FC1); // covariance to be updated
CvMat* sigma_p = cvCreateMat(num_dim, num_dim, CV_64FC1); // covariance to be updated
CvMat* z = cvCreateMat(num_landmarks, 1, CV_64FC1); // measurement vector
CvMat* K = cvCreateMat(num_dim, num_landmarks, CV_64FC1); // K matrix // Kalman gain
// initialize "mu" vector
cvmSet(mu, 0, 0, rob_pos + sqrt(rob_pos_cov)*gaussian_random()); // set mu(0,0) to "rob_pos"
cvmSet(mu, 1, 0, rob_vel + sqrt(rob_vel_cov)*gaussian_random()); // set mu(0,0) to "rob_vel"
for(int n = 0; n < num_landmarks; n++)
{
// cvmSet(mu, n+2, 0, landmark[n] + sqrt(obs_cov)*gaussian_random());
cvmSet(mu, n+2, 0, landmark[n]);
}
displayMatrix(mu, "mu vector");
// initialize "sigma" matrix <-- This is the most critical point in tuning
cvSetIdentity(sigma, cvRealScalar(obs_cov));
displayMatrix(sigma, "sigma matrix");
// set the initial covariance of the state uncertainty
double rob_px_cov = 0.01; // covariance of noise to robot's x-position
double rob_py_cov = 0.01; // covariance of noise to robot's y-position
double rob_vx_cov = 0.01; // covariance of noise to robot's x-velocity
double rob_vy_cov = 0.01; // covariance of noise to robot's y-velocity
// set the initial covariance of the measurement noise
double obs_x_cov = 900; // covarriance of noise to x-measurement of landmarks
double obs_y_cov = 900; // covarriance of noise to y-measurement of landmarks
// ground truth of state
double xGroundTruth = rob_x;
double yGroundTruth = rob_y;
double vxGroundTruth = rob_vx;
double vyGroundTruth = rob_vy;
// ground truth of num_landmarks landmarks in the world coordinate
double landmark[2*num_landmarks];
for( int n = 0; n < num_landmarks; n++ )
{
landmark[2*n] = width * uniform_random();
landmark[2*n+1] = height * uniform_random();
}
// H matrix, measurement matrix
CvMat* H = cvCreateMat(2*num_landmarks, num_dim, CV_64FC1);
cvZero(H); // initialize H matrix
// set H matrix
for (int r = 0; r < num_landmarks; r++)
{
cvmSet(H, 2*r, 0, -1.0); // robot's x-position
cvmSet(H, 2*r, 2*r+4, 1.0); // landmark's x-position
cvmSet(H, 2*r+1, 1, -1.0); // robot's y-position
cvmSet(H, 2*r+1, 2*r+5, 1.0); // landmarks's y-position
}
displayMatrix(H, "H matrix");
// Q matrix ; covariance of noise to H; uncertainty of control
CvMat* Q = cvCreateMat(2*num_landmarks, 2*num_landmarks, CV_64FC1);
cvZero(Q); // initialize Q matrix
// set Q matrix
for (int row = 0; row < Q->rows; row++)
{
cvmSet(Q, row, row, obs_x_cov);
}
displayMatrix(Q, "Q matrix");
// G matrix // transition
CvMat* G = cvCreateMat(num_dim, num_dim, CV_64FC1);
cvZero(G); // initialize G matrix
// set G matrix
cvmSet(G, 0, 0, 1.0); // previous x-position
cvmSet(G, 0, 2, 1.0); // x-velocity
cvmSet(G, 1, 1, 1.0); // previous y-position
cvmSet(G, 1, 3, 1.0); // y-velocity
for (int row = 2; row < G->rows; row++)
{
cvmSet(G, row, row, 1.0); // constance of velocity
}
displayMatrix(G, "G matrix");
// R matrix ; covariance of noise to G; uncertainty of observation
CvMat* R = cvCreateMat(num_dim, num_dim, CV_64FC1);
cvZero(R); // initialize R matrix
// set R matrix
cvmSet(R, 0, 0, rob_px_cov);
cvmSet(R, 1, 1, rob_py_cov);
cvmSet(R, 2, 2, rob_vx_cov);
cvmSet(R, 3, 3, rob_vy_cov);
displayMatrix(R, "R matrix");
CvMat* rob_ground = cvCreateMat(num_dim, 1, CV_64FC1); // ground truth of state
CvMat* mu = cvCreateMat(num_dim, 1, CV_64FC1); // state vector to be estimated
CvMat* mu_p = cvCreateMat(num_dim, 1, CV_64FC1); // state vector to be predicted
CvMat* sigma = cvCreateMat(num_dim, num_dim, CV_64FC1); // covariance to be updated
CvMat* sigma_p = cvCreateMat(num_dim, num_dim, CV_64FC1); // covariance to be updated
CvMat* z = cvCreateMat(2*num_landmarks, 1, CV_64FC1); // measurement vector
CvMat* K = cvCreateMat(num_dim, 2*num_landmarks, CV_64FC1); // K matrix // Kalman gain
// initialize "mu" vector
cvmSet(mu, 0, 0, rob_x); // set mu(0,0) to "rob_x"
cvmSet(mu, 1, 0, rob_y); // set mu(1,0) to "rob_y"
cvmSet(mu, 2, 0, rob_vx); // set mu(2,0) to "rob_vx"
cvmSet(mu, 3, 0, rob_vy); // set mu(3,0) to "rob_vy"
for (int n = 0; n < 2*num_landmarks; n++)
{
cvmSet(mu, n+4, 0, landmark[n]); // set mu(4,0) to "landmark[0]", ...
}
displayMatrix(mu, "mu vector");
/* // initialize "sigma" matrix
cvmSet(sigma, 0, 0, rob_px_cov);
cvmSet(sigma, 1, 1, rob_py_cov);
cvmSet(sigma, 2, 2, rob_vx_cov);
cvmSet(sigma, 3, 3, rob_vy_cov);
for (int r = 4; r < sigma->rows; r=r*2)
{
cvmSet(sigma, r, r, obs_x_cov);
cvmSet(sigma, r+1, r+1, obs_y_cov);
}
*/ // initialize "sigma" matrix <-- This is the most critical point in tuning
cvSetIdentity(sigma, cvRealScalar(obs_x_cov));
displayMatrix(sigma, "sigma matrix");
// matrices to be used in calculation
CvMat* Hx = cvCreateMat(H->rows, mu->cols, CV_64FC1);
CvMat* Gt = cvCreateMat(G->cols, G->rows, CV_64FC1);
cvTranspose(G, Gt); // transpose(G) -> Gt
CvMat* sigmaGt = cvCreateMat(sigma->rows, G->rows, CV_64FC1);
CvMat* GsigmaGt = cvCreateMat(G->rows, G->rows, CV_64FC1); // 10x10
// observation relative to robot's position
for( int n = 0; n < 2*num_landmarks; n++ )
{
cvmSet(obs, n, 0, cvmGet(mu,0,0) + cvmGet(z,n,0));
}
// update the state with Kalman gain (ref. L5, EKF algorithm, 42p)
cvMatMul(H, mu, Hmu); // H * mu -> Hmu
cvSub(z, Hmu, miss); // z - Hmu -> miss
cvMatMul(K, miss, adjust); // K * miss -> adjust
cvAdd(mu_p, adjust, mu); // mu_p + adjust -> mu
displayMatrix(mu, "mu vector");
Robot mechanism Byung-Ju Yi (Hanyang University, Korea)
한양대 휴먼로보틱스 연구실 이병주 교수님 bj@hanyang.ac.kr - Classification of robotic mechanism and Design consideration of robotic mechanism
- Design Issue and application examples of master slave robotic system
- Trend of robotic mechanism research
Actuator and Practical PID Control Youngjin Choi (Hanyang University, Korea)
한양대 휴먼로이드 연구실 최영진 교수님 cyj@hanyang.ac.kr - Operation Principle of DC/RC/Stepping Motors & Its Practice
- PID Control and Tuning
- Stability of PID Control and Application Examples
Coordination of Robots and Humans Kazuhiro Kosuge (Tohoku University, Japan)
일본 도호쿠 대학 시스템 로보틱스 연구실 고스게 카즈히로 교수님 - Robotics as systems integration
- Multiple Robots Coordination
- Human Robot Coordination and Interaction
Robot Control Rolf Johansson (Lund University, Sweden)
스웨덴 룬드 대학 로보틱스 연구실 Rolf.Johansson@control.lth.se - Robot motion and force control
- Stability of motion
- Robot obstacle avoidance
Lecture from Industry or Government
(S. -R. Oh, KIST)
Special Talk from Government
(Y. J. Weon, MKE)
Mobile Robot Navigation Jae-Bok Song (Korea University, Korea)
고려대 지능로봇 연구실 송재복 교수님 jbsong@korea.ac.kr - Mapping
- Localization
- SLAM
3D Perception for Robot Vision In Kyu Park (Inha University, Korea) 인하대 영상미디어 연구실 박인규 교수님 pik@inha.ac.kr - Camera Model and Calibration
- Shape from Stereo Views
- Shape from Multiple Views
Lecture from Industry or Government
(H. S. Kim, KITECH)
Software Framework for LEGO NXT Sanghoon Lee (Hanyang University, Korea)
한양대 로봇 연구실 이상훈 교수님 - Development Environments for LEGO NXT
- Programming Issues for LEGO NXT under RPF of OPRoS
- Programming Issues for LEGO NXT under Roboid Framework
Lecture from Industry or Government
(Robomation/Mobiletalk/Robotis)
Robot Intelligence : From Reactive AI to Semantic AI Il Hong Suh (Hanyang University, Korea)
한양대 로봇 지능/통신 연구실 서일홍 교수님 - Issues in Robot Intelligence
- Behavior Control: From Reactivity to Proactivity
- Use of Semantics for Robot Intelligence
AI-Robotics
Henrik I. Christensen (Georgia Tech., USA)
- Semantic Mapping - Physical Interaction with Robots
- Efficient object recognition for robots
Lecture from Industry or Government
(M. S. Kim, Director of CIR, 21C Frontier Program)
HRI Dongsoo Kwon (KAIST, Korea)
- Introduction to human-robot interaction
- Perception technologies of HRI
- Cognitive and emotional interaction
Robot Swarm for Environmental Monitoring Nak Young Chong (JAIST, Japan)
- Self-organizing Mobile Robot Swarms: Models
- Self-organizing Mobile Robot Swarms: Algorithms
- Self-organizing Mobile Robot Swarms: Implementation
[1] J. Civera, A.J. Davison, and J.M.M Montiel. Inverse depth
parametrization for monocular SLAM. IEEE Trans. on Robotics, 24(5),
2008.
[2] J. Civera, Andrew Davison, and J. Montiel. Inverse Depth to Depth
Conversion for Monocular SLAM. In IEEE Int. Conf. on Robotics and
Automation, pages 2778 –2783, April 2007.
[3] A. J. Davison. Real-time simultaneous localisation and mapping with
a single camera. In Int. Conf. on Computer Vision, volume 2, pages
1403–1410, Nice, October 2003.
[4] Andrew J. Davison. Active search for real-time vision. Int. Conf. on Computer Vision, 1:66–73, 2005.
[5] Andrew J. Davison, Ian D. Reid, Nicholas D. Molton, and Olivier
Stasse. MonoSLAM: Real-time single camera SLAM. Trans. on Pattern
Analysis and Machine Intelligence, 29(6):1052–1067, June 2007.
[6] Ethan Eade and Tom Drummond. Scalable monocular SLAM. IEEE Int.
Conf. on Computer Vision and Pattern Recognition, 1:469–476, 2006.
[7] Thomas Lemaire and Simon Lacroix. Monocular-vision based SLAM using
line segments. In IEEE Int. Conf. on Robotics and Automation, pages
2791–2796, Rome, Italy, 2007.
[8] Nicholas Molton, Andrew J. Davison, and Ian Reid. Locally planar
patch features for real-time structure from motion. In British Machine
Vision Conference, 2004.
[9] J. Montiel, J. Civera, and A. J. Davison. Unified inverse depth
parametrization for monocular SLAM. In Robotics: Science and Systems,
Philadelphia, USA, August 2006.
[10] L. M. Paz, P. Pini´es, J. Tard´os, and J. Neira. Large scale 6DOF
SLAM with stereo-in-hand. IEEE Trans. on Robotics, 24(5), 2008.
[11] J. Sol`a, Andr´e Monin, Michel Devy, and T. Vidal-Calleja. Fusing
monocular information in multi-camera SLAM. IEEE Trans. on Robotics,
24(5):958–968, 2008.
[12] Joan Sol`a. Towards Visual Localization, Mapping and Moving
Objects Tracking by a Mobile Robot: a Geometric and Probabilistic
Approach. PhD thesis, Institut National Polytechnique de Toulouse, 2007.
[13] Joan Sol`a, Andr´e Monin, and Michel Devy. BiCamSLAM: Two times
mono is more than stereo. In IEEE Int. Conf. on Robotics and
Automation, pages 4795–4800, Rome, Italy, April 2007.
[14] Joan Sol`a, Andr´e Monin, Michel Devy, and Thomas Lemaire.
Undelayed initialization in bearing only SLAM. In IEEE/RSJ Int. Conf.
on Intelligent Robots and Systems, pages 2499–2504, Edmonton, Canada,
2005.
[15] Joan Sol`a, Teresa Vidal-Calleja, and Michel Devy. Undelayed
initialization of line segments in monocular SLAM. In IEEE Int. Conf.
on Intelligent Robots and Systems, Saint Louis, USA, 2009. To appear.
slamtb.m
% SLAMTB An EKF-SLAM algorithm with simulator and graphics.
%
% This script performs multi-robot, multi-sensor, multi-landmark 6DOF
% EKF-SLAM with simulation and graphics capabilities.
%
% Please read slamToolbox.pdf in the root directory thoroughly before
% using this toolbox.
%
% - Beginners should not modify this file, just edit USERDATA.M and enter
% and/or modify the data you wish to simulate.
%
% - More advanced users should be able to create new landmark models, new
% initialization methods, and possibly extensions to multi-map SLAM. Good
% luck!
%
% - Expert users may want to add code for real-data experiments.
%
% See also USERDATA, USERDATAPNT, USERDATALIN.
%
% Also consult slamToolbox.pdf in the root directory.
% Created and maintained by
% Copyright 2008-2009 Joan Sola @ LAAS-CNRS.
% Programmers (for the whole toolbox):
% Copyright David Marquez and Jean-Marie Codol @ LAAS-CNRS
% Copyright Teresa Vidal-Calleja @ ACFR.
% See COPYING.TXT for wull copyright license.
%% OK we start here
% clear workspace and declare globals
clear
global Map %#ok<NUSED>
%% I. Specify user-defined options - EDIT USER DATA FILE userData.m
%% II. Initialize all data structures from user-defined data in userData.m
% SLAM data
[Rob,Sen,Raw,Lmk,Obs,Tim] = createSlamStructures(...
Robot,...
Sensor,... % all user data
Time,...
Opt);
% Simulation data
[SimRob,SimSen,SimLmk,SimOpt] = createSimStructures(...
Robot,...
Sensor,... % all user data
World,...
SimOpt);
% Graphics handles
[MapFig,SenFig] = createGraphicsStructures(...
Rob, Sen, Lmk, Obs,... % SLAM data
SimRob, SimSen, SimLmk,... % Simulator data
FigOpt); % User-defined graphic options
%% III. Init data logging
% TODO: Create source and/or destination files and paths for data input and
% logs.
% TODO: do something here to collect data for post-processing or
% plotting. Think about collecting data in files using fopen, fwrite,
% etc., instead of creating large Matlab variables for data logging.
% Clear user data - not needed anymore
clear Robot Sensor World Time % clear all user data
%% IV. Main loop
for currentFrame = Tim.firstFrame : Tim.lastFrame
% Robot motion
% FIXME: see how to include noise in a clever way.
Rob(rob).con.u = SimRob(rob).con.u + Rob(rob).con.uStd.*randn(6,1);
Rob(rob) = motion(Rob(rob),Tim);
% Process sensor observations
for sen = Rob(rob).sensors
%TODO: see how to pass only used Lmks and Obs.
% Observe knowm landmarks
[Rob(rob),Sen(sen),Lmk,Obs(sen,:)] = correctKnownLmks( ...
Rob(rob), ...
Sen(sen), ...
Raw(sen), ...
Lmk, ...
Obs(sen,:), ...
Opt) ;
% Figures for all sensors
for sen = [Sen.sen]
SenFig(sen) = drawSenFig(SenFig(sen), ...
Sen(sen), Raw(sen), Obs(sen,:), ...
FigOpt);
makeVideoFrame(SenFig(sen), ...
sprintf('sen%02d-%04d.png', sen, currentFrame),...
FigOpt, ExpOpt);
end
% Do draw all objects
drawnow;
end
% 4. DATA LOGGING
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% TODO: do something here to collect data for post-processing or
% plotting. Think about collecting data in files using fopen, fwrite,
% etc., instead of creating large Matlab variables for data logging.
end
%% V. Post-processing
% ========== End of function - Start GPL license ==========
% # START GPL LICENSE
%---------------------------------------------------------------------
%
% This file is part of SLAMTB, a SLAM toolbox for Matlab.
%
% SLAMTB is free software: you can redistribute it and/or modify
% it under the terms of the GNU General Public License as published by
% the Free Software Foundation, either version 3 of the License, or
% (at your option) any later version.
%
% SLAMTB is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
% GNU General Public License for more details.
%
% You should have received a copy of the GNU General Public License
% along with SLAMTB. If not, see <http://www.gnu.org/licenses/>.
%
%---------------------------------------------------------------------
% SLAMTB is Copyright 2007,2008,2009
% by Joan Sola, David Marquez and Jean Marie Codol @ LAAS-CNRS.
% See on top of this file for its particular copyright.
This paper appears in: Robotics and Automation, 2009. ICRA '09. IEEE International Conference on
Publication Date: 12-17 May 2009
On page(s): 375-380
ISSN: 1050-4729
ISBN: 978-1-4244-2788-8
INSPEC Accession Number: 10748966
Digital Object Identifier: 10.1109/ROBOT.2009.5152290
Current Version Published: 2009-07-06
parallel implementation of monoSLAM with a 3D object tracker
information to register objects to the map's frame
the recovered geometry
I. Introduction
approaches to handling movement in the environment
segmentation between static and moving features
outlying moving points
1) active search -> sparse maps
2) robust methods -> multifocal tensors
3-1) tracking known 3D objects in the scene
-2) determining whether they are moving
-3) using their convex hulls to mask out features
"Knowledge that they are occluded rather than unreliable avoids the need to invoke the somewhat cumbersome process of feature deletion, followed later perhaps by unnecessary reinitialization."
[15] H. Zhou and S. Sakane, “Localizing objects during robot SLAM in semi-dynamic environments,” in Proc of the 2008 IEEE/ASME Int Conf on Advanced Intelligent Mechatronics, 2008, pp. 595–601.
"[15] noted that movement is likely to associated with objects in the scene, and classified them according to the likelihood that they would move."
the use of 3D objects for reasoning about motion segmentation and occlusion
"(RAPiD makes the assumption that the pose change required between current and new estimates is sufficiently small, first, to allow a linearization of the solution and, second, to make trivial the problem of inter-image correspondence.) The correspondences used are between predicted point to measured image edge, allowing search in 1D rather than 2D within the image. This makes very sparing use of image data — typically only several hundred pixels per image are addressed."
(1)First it was found to be easily broken by occlusions and changing
lighting. Robust methods to mitigate this problem have been
investigated monocularly by Armstrong and Zisserman.
(2)Although this has a marked effect on tracking performance, the second
problem found is that the accuracy of the pose recovered in a single
camera was poor, with evident correlation between depth and rotation
about axes parallel to the image plane. Maitland and Harris had
already noted as much when recovering the pose of a pointing device
destined for neurosurgical application. They reported much improved accuracy using two cameras; but the object
was stationary, had an elaborate pattern drawn on it and was visible at
all times to both cameras. (3)The third difficulty, or rather uncertainty,
was that the convergence properties and dynamic performances of the
monocular and multicamera methods were largely unreported.
(3) : little solution
(2) => [21] "recovered pose using 3 iterations of the pose update cycle per image"
(1) => [21], [22] : search -> matching -> weighting
[22] M. Armstrong and A. Zisserman, “Robust object tracking,” in Proc 2nd Asian Conference on Computer Vision, 1995, vol. I. Springer, 1996, pp. 58–62.
RANSAC
[23] M. Fischler and R. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, June 1981.
Least median of squares as the underlying standard deviation is unknown
[24] P. J. Rousseeuw, “Least median of squares regression,” Journal of the American Statistical Association, vol. 79, no. 388, pp. 871–880, 1984.
III. MonoSLAM with Tracked Objects A. Information from SLAM to the object tracker
B. Information from the object tracker to SLAM
"The convex hull is uniformly dilated by an amount that corresponds to the projection of the typical change in pose."
Techniques for Generating Consistent Maps
• Scan matching
• EKF SLAM
• Fast-SLAM
• Probabilistic mapping with a single map and a posterior about poses Mapping + Localization
• Graph-SLAM, SEIFs
Approximations for SLAM
• Local submaps
[Leonard et al.99, Bosse et al. 02, Newman et al. 03]
• Sparse links (correlations)
[Lu & Milios 97, Guivant & Nebot 01]
• Sparse extended information filters
[Frese et al. 01, Thrun et al. 02]
• Thin junction tree filters
[Paskin 03]
• Rao-Blackwellisation (FastSLAM)
[Murphy 99, Montemerlo et al. 02, Eliazar et al. 03, Haehnel et al. 03]
EKF-SLAM Summary
•Quadratic in the number of landmarks: O(n2)
• Convergence results for the linear case.
• Can diverge if nonlinearities are large!
• Have been applied successfully in large-scale environments.
• Approximations reduce the computational complexity.
Denis Chekhlov, Andrew Gee, Andrew Calway, Walterio Mayol-Cuevas
Ninja on a Plane: Automatic Discovery of Physical Planes for Augmented Reality Using Visual SLAM
맥미니에서의 설치를 끝내고 (test log on mac) 테스트해 보면 성능이 좋지 않아 그대로 쓸 수는 없는 상태이다.
0. Video Input
The software requires a video camera with a wide-angle
lens, capable of 640x480x30Hz video capture and an appropriate driver
installation (which is supported by libCVD.)
1. Camera Parameters
CameraCalibrator를 실행시키면 calibrator_settings.cfg 파일을 읽어 온다.
여기에 gvars (PTAM 라이브러리를 지원하는 Gvars3 라이브러리) settings이 설정되어 있...어야 하는데 빈 채로 주어졌다.
CameraCalibrator를 실행시킨 결과로 연산된 카메라 파라미터는 camera.cfg 파일에 저장된다.
실행 후 열어 보면,
1) settings.cfg 파일 로드 GUI.LoadFile("settings.cfg");
2) 사용자 입력 parsing GUI.StartParserThread();
3) 클래스 system (system.h) 실행 s.Run();
atexit
Set function to be executed on exit
The function pointed by the function pointer argument is called when the program terminates normally.
try-if 구문
1) Deitel 823p "catch handler"
2) theuhm@naver:
"에러가 발생한 객체는 예외를 발생시킴과 동시에 try블럭 안의 모든 객체는 스코프를 벗어나 참조할 수 없게 되므로 예외를
처리하는 동안 try블럭 안에서 예외를 발생시켰을 수 있는 객체의 참조를 원천적으로 막아 더 안전하고 깔끔한 예외처리를 할 수
있는 환경을 만들어줍니다. 그리고 예외를 던질 때에 예외 객체의 클래스를 적절히 구성하면, 예외 객체에 예외를 처리하는 방법을
담아서 던질 수도 있습니다. 그렇게 구성하면 굉장히 깔끔한 코드를 얻을 수 있죠.
set
Sets are a kind of associative containers that stores unique elements, and in which the elements themselves are the keys. Sets are typically implemented as binary search trees.
1) PTAM에서 핵심적 기능을 하는 클래스들과 클래스 "System"을 선언 // Defines the System class // This stores the main functional classes of the system class ATANCamera; class Map; class MapMaker; class Tracker; class ARDriver; class MapViewer; class System;
내가 설치할 컴퓨터 사양:
Model Name: Mac mini
Model Identifier: Macmini3,1
Processor Name: Intel Core 2 Duo
Processor Speed: 2 GHz
Number Of Processors: 1
Total Number Of Cores: 2
L2 Cache: 3 MB
Memory: 1 GB
Bus Speed: 1.07 GHz
Boot ROM Version: MM31.0081.B00
그래픽 카드:
NVIDIA GeForce 9400
"Intel Core 2 Duo processors 2.4GHz+ are fine."이라고 했는데, 2.0이면 되지 않을까? 그래픽 카드는 동일한 것이니 문제 없고.
1. library dependency 확인
1. TooN - a header library for linear algebra
2. libCVD - a library for image handling, video capture and computer vision
3. Gvars3 - a run-time configuration/scripting library, this is a sub-project of libCVD.
%% cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/toon co TooN
실행 결과:
cvs checkout: warning: failed to open /Users/lym/.cvspass for reading: No such file or directory
cvs checkout: Updating TooN
U TooN/Authors
U TooN/COPYING
U TooN/Cholesky.h
U TooN/Doxyfile
U TooN/GPL.txt
U TooN/LU.h
U TooN/Lapack_Cholesky.h
U TooN/Makefile.in
U TooN/README
U TooN/SVD.h
U TooN/SymEigen.h
U TooN/TODO
U TooN/TooN.h
U TooN/configure
U TooN/configure.ac
U TooN/determinant.h
U TooN/gauss_jordan.h
U TooN/gaussian_elimination.h
U TooN/generated.h
U TooN/helpers.h
U TooN/irls.h
U TooN/lapack.h
U TooN/make_make_vector.awk
U TooN/make_typeof.awk
U TooN/se2.h
U TooN/se3.h
U TooN/sl.h
U TooN/so2.h
U TooN/so3.h
U TooN/wls.h
cvs checkout: Updating TooN/Documentation
cvs checkout: Updating TooN/benchmark
U TooN/benchmark/generate_solvers.m
U TooN/benchmark/solve_ax_equals_b.cc
U TooN/benchmark/solvers.cc
cvs checkout: Updating TooN/doc
U TooN/doc/COPYING_FDL
U TooN/doc/Makefile
U TooN/doc/documentation.h
U TooN/doc/linoperatorsdoc.h
cvs checkout: Updating TooN/internal
U TooN/internal/allocator.hh
U TooN/internal/builtin_typeof.h
U TooN/internal/comma.hh
U TooN/internal/config.hh
U TooN/internal/config.hh.in
U TooN/internal/dchecktest.hh
U TooN/internal/debug.hh
U TooN/internal/deprecated.hh
U TooN/internal/diagmatrix.h
U TooN/internal/make_vector.hh
U TooN/internal/matrix.hh
U TooN/internal/mbase.hh
U TooN/internal/objects.h
U TooN/internal/operators.hh
U TooN/internal/overfill_error.hh
U TooN/internal/reference.hh
U TooN/internal/size_mismatch.hh
U TooN/internal/slice_error.hh
U TooN/internal/typeof.hh
U TooN/internal/vbase.hh
U TooN/internal/vector.hh
cvs checkout: Updating TooN/optimization
U TooN/optimization/brent.h
U TooN/optimization/conjugate_gradient.h
U TooN/optimization/downhill_simplex.h
U TooN/optimization/golden_section.h
cvs checkout: Updating TooN/test
U TooN/test/SXX_test.cc
U TooN/test/as_foo.cc
U TooN/test/brent_test.cc
U TooN/test/cg_test.cc
U TooN/test/cg_view.gnuplot
U TooN/test/chol.cc
U TooN/test/diagslice.cc
U TooN/test/dynamic_test.cc
U TooN/test/gauss_jordan.cc
U TooN/test/gaussian_elimination_test.cc
U TooN/test/golden_test.cc
U TooN/test/identity_test.cc
U TooN/test/lutest.cc
U TooN/test/make_vector.cc
U TooN/test/makevector.cc
U TooN/test/mat_test.cc
U TooN/test/mat_test2.cc
U TooN/test/mmult_test.cc
U TooN/test/normalize_test.cc
U TooN/test/normalize_test2.cc
U TooN/test/scalars.cc
U TooN/test/simplex_test.cc
U TooN/test/simplex_view.gnuplot
U TooN/test/sl.cc
U TooN/test/svd_test.cc
U TooN/test/sym.cc
U TooN/test/test2.cc
U TooN/test/test3.cc
U TooN/test/test_foreign.cc
U TooN/test/un_project.cc
U TooN/test/vec_test.cc
생성된 TooN 폴더에 들어가서
%%% ./configure
실행 결과:
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for dgesvd_ in -llapack... yes
checking for decltype... no
checking for __typeof__... yes
checking for __attribute__((deprecated))... yes
You're on the development branch of TooN 2.0. Everything will probably work, but
the interface is a bit different from TooN-1.x
If you want TooN-1, then get it using:
cvs -z3 -d:pserver:anoncvs@cvs.savannah.nongnu.org:/cvsroot/toon co -r Maintenance_Branch_1_x TooN
or update what you currently have using:
cvs up -r Maintenance_Branch_1_x
or head over to:
http://mi.eng.cam.ac.uk/~er258/cvd/
Otherwise, please report any bugs you come across.
%% cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/toon co -D "Mon May 11 16:29:26 BST 2009" TooN
실행 결과:
cvs checkout: warning: failed to open /Users/lym/.cvspass for reading: No such file or directory
? TooN/Makefile
? TooN/config.log
? TooN/config.status
cvs checkout: Updating TooN
U TooN/Cholesky.h
U TooN/Doxyfile
U TooN/LU.h
U TooN/Makefile.in
U TooN/SVD.h
U TooN/SymEigen.h
U TooN/TooN.h
U TooN/configure
U TooN/configure.ac
cvs checkout: `TooN/determinant.h' is no longer in the repository
U TooN/gauss_jordan.h
U TooN/gaussian_elimination.h
U TooN/helpers.h
U TooN/irls.h
U TooN/se2.h
U TooN/se3.h
U TooN/sl.h
U TooN/so2.h
U TooN/so3.h
U TooN/util.h
U TooN/wls.h
cvs checkout: Updating TooN/Documentation
cvs checkout: Updating TooN/benchmark
U TooN/benchmark/generate_solvers.m
cvs checkout: Updating TooN/doc
U TooN/doc/documentation.h
U TooN/doc/matrixdoc.h
cvs checkout: Updating TooN/internal
U TooN/internal/allocator.hh
cvs checkout: `TooN/internal/comma.hh' is no longer in the repository
RCS file: /sources/toon/TooN/internal/config.hh,v
retrieving revision 1.12
retrieving revision 1.8
Merging differences between 1.12 and 1.8 into config.hh
TooN/internal/config.hh already contains the differences between 1.12 and 1.8
U TooN/internal/config.hh.in
cvs checkout: `TooN/internal/dchecktest.hh' is no longer in the repository
U TooN/internal/debug.hh
cvs checkout: `TooN/internal/deprecated.hh' is no longer in the repository
U TooN/internal/diagmatrix.h
U TooN/internal/matrix.hh
U TooN/internal/mbase.hh
U TooN/internal/objects.h
U TooN/internal/operators.hh
cvs checkout: `TooN/internal/overfill_error.hh' is no longer in the repository
U TooN/internal/reference.hh
U TooN/internal/slice_error.hh
U TooN/internal/vector.hh
cvs checkout: Updating TooN/optimization
U TooN/optimization/conjugate_gradient.h
U TooN/optimization/downhill_simplex.h
U TooN/optimization/golden_section.h
cvs checkout: Updating TooN/test
U TooN/test/identity_test.cc
cvs checkout: `TooN/test/simplex_test.cc' is no longer in the repository
cvs checkout: `TooN/test/simplex_view.gnuplot' is no longer in the repository
U TooN/test/vec_test.cc
%% cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/libcvd co -D "Mon May 11 16:29:26 BST 2009" libcvd
실행 결과:
cvs checkout: warning: failed to open /Users/lym/.cvspass for reading: No such file or directory
cvs checkout: Updating libcvd
U libcvd/Authors
U libcvd/Doxyfile
U libcvd/LICENSE
U libcvd/Makefile.in
U libcvd/TODO
U libcvd/config.guess
U libcvd/config.sub
U libcvd/configure
U libcvd/configure.in
U libcvd/generate_dependencies.bash
U libcvd/install-sh
U libcvd/subimage_test.cc
cvs checkout: Updating libcvd/build
cvs checkout: Updating libcvd/build/vc2005
U libcvd/build/vc2005/config.h
U libcvd/build/vc2005/libcvd.sln
U libcvd/build/vc2005/libcvd.vcproj
cvs checkout: Updating libcvd/build/vc2008
U libcvd/build/vc2008/libcvd.sln
U libcvd/build/vc2008/libcvd.vcproj
cvs checkout: Updating libcvd/cvd
U libcvd/cvd/abs.h
U libcvd/cvd/bresenham.h
U libcvd/cvd/brezenham.h
U libcvd/cvd/byte.h
U libcvd/cvd/camera.h
U libcvd/cvd/colourspace.h
U libcvd/cvd/colourspace_convert.h
U libcvd/cvd/colourspace_frame.h
U libcvd/cvd/colourspacebuffer.h
U libcvd/cvd/colourspaces.h
U libcvd/cvd/connected_components.h
U libcvd/cvd/convolution.h
U libcvd/cvd/cpu_hacks.h
U libcvd/cvd/cvd_image.h
U libcvd/cvd/cvd_timer.h
U libcvd/cvd/deinterlacebuffer.h
U libcvd/cvd/deinterlaceframe.h
U libcvd/cvd/diskbuffer2.h
U libcvd/cvd/diskbuffer2_frame.h
U libcvd/cvd/documentation.h
U libcvd/cvd/draw.h
U libcvd/cvd/eventobject.h
U libcvd/cvd/exceptions.h
U libcvd/cvd/fast_corner.h
U libcvd/cvd/gl_helpers.h
U libcvd/cvd/glwindow.h
U libcvd/cvd/haar.h
U libcvd/cvd/harris_corner.h
U libcvd/cvd/helpers.h
U libcvd/cvd/image.h
U libcvd/cvd/image_convert.h
U libcvd/cvd/image_convert_fwd.h
U libcvd/cvd/image_interpolate.h
U libcvd/cvd/image_io.h
U libcvd/cvd/image_ref.h
U libcvd/cvd/integral_image.h
U libcvd/cvd/interpolate.h
U libcvd/cvd/irls.h
U libcvd/cvd/la.h
U libcvd/cvd/localvideobuffer.h
U libcvd/cvd/localvideoframe.h
U libcvd/cvd/message_queue.h
U libcvd/cvd/nonmax_suppression.h
U libcvd/cvd/random.h
U libcvd/cvd/readaheadvideobuffer.h
U libcvd/cvd/rgb.h
U libcvd/cvd/rgb8.h
U libcvd/cvd/rgba.h
U libcvd/cvd/ringbuffer.h
U libcvd/cvd/runnable.h
U libcvd/cvd/runnable_batch.h
U libcvd/cvd/se2.h
U libcvd/cvd/se3.h
U libcvd/cvd/serverpushjpegbuffer.h
U libcvd/cvd/serverpushjpegframe.h
U libcvd/cvd/so2.h
U libcvd/cvd/so3.h
U libcvd/cvd/synchronized.h
U libcvd/cvd/tensor_voting.h
U libcvd/cvd/thread.h
U libcvd/cvd/timeddiskbuffer.h
U libcvd/cvd/timer.h
U libcvd/cvd/utility.h
U libcvd/cvd/vector_image_ref.h
U libcvd/cvd/videobuffer.h
U libcvd/cvd/videobufferflags.h
U libcvd/cvd/videodisplay.h
U libcvd/cvd/videofilebuffer.h
U libcvd/cvd/videofilebuffer_frame.h
U libcvd/cvd/videoframe.h
U libcvd/cvd/videosource.h
U libcvd/cvd/vision.h
U libcvd/cvd/wls.h
U libcvd/cvd/wls_c.h
U libcvd/cvd/wls_cholesky.h
U libcvd/cvd/yc.h
cvs checkout: Updating libcvd/cvd/IRIX
U libcvd/cvd/IRIX/O2buffer.h
U libcvd/cvd/IRIX/O2videoframe.h
U libcvd/cvd/IRIX/sgi-video.h
cvs checkout: Updating libcvd/cvd/Linux
U libcvd/cvd/Linux/capture_logic.cxx
U libcvd/cvd/Linux/dvbuffer.h
U libcvd/cvd/Linux/dvbuffer3.h
U libcvd/cvd/Linux/dvframe.h
U libcvd/cvd/Linux/v4l1buffer.h
U libcvd/cvd/Linux/v4l1frame.h
U libcvd/cvd/Linux/v4l2buffer.h
U libcvd/cvd/Linux/v4l2frame.h
U libcvd/cvd/Linux/v4lbuffer.h
U libcvd/cvd/Linux/v4lcontrol.h
cvs checkout: Updating libcvd/cvd/OSX
U libcvd/cvd/OSX/qtbuffer.h
U libcvd/cvd/OSX/qtframe.h
cvs checkout: Updating libcvd/cvd/internal
U libcvd/cvd/internal/aligned_mem.h
U libcvd/cvd/internal/assembly.h
U libcvd/cvd/internal/builtin_components.h
U libcvd/cvd/internal/convert_pixel_types.h
U libcvd/cvd/internal/disk_image.h
U libcvd/cvd/internal/gl_types.h
U libcvd/cvd/internal/image_ref_implementation.hh
U libcvd/cvd/internal/is_pod.h
U libcvd/cvd/internal/load_and_save.h
U libcvd/cvd/internal/name_CVD_rgb_types.h
U libcvd/cvd/internal/name_builtin_types.h
U libcvd/cvd/internal/pixel_operations.h
U libcvd/cvd/internal/pixel_traits.h
U libcvd/cvd/internal/rgb_components.h
U libcvd/cvd/internal/scalar_convert.h
U libcvd/cvd/internal/simple_vector.h
cvs checkout: Updating libcvd/cvd/internal/io
U libcvd/cvd/internal/io/bmp.h
U libcvd/cvd/internal/io/fits.h
U libcvd/cvd/internal/io/jpeg.h
U libcvd/cvd/internal/io/png.h
U libcvd/cvd/internal/io/pnm_grok.h
U libcvd/cvd/internal/io/save_postscript.h
U libcvd/cvd/internal/io/text.h
U libcvd/cvd/internal/io/tiff.h
cvs checkout: Updating libcvd/cvd/internal/pnm
cvs checkout: Updating libcvd/cvd/lock
cvs checkout: Updating libcvd/cvd/python
U libcvd/cvd/python/interface.h
U libcvd/cvd/python/selector.h
U libcvd/cvd/python/types.h
cvs checkout: Updating libcvd/cvd_src
U libcvd/cvd_src/bayer.cxx
U libcvd/cvd_src/brezenham.cc
U libcvd/cvd_src/colourspace_convert.cxx
U libcvd/cvd_src/connected_components.cc
U libcvd/cvd_src/convolution.cc
U libcvd/cvd_src/corner_10.h
U libcvd/cvd_src/corner_12.h
U libcvd/cvd_src/corner_9.h
U libcvd/cvd_src/cvd_timer.cc
U libcvd/cvd_src/deinterlacebuffer.cc
U libcvd/cvd_src/diskbuffer2.cc
U libcvd/cvd_src/draw.cc
U libcvd/cvd_src/draw_toon.cc
U libcvd/cvd_src/eventobject.cpp
U libcvd/cvd_src/exceptions.cc
U libcvd/cvd_src/fast_corner.cxx
U libcvd/cvd_src/fast_corner_9_nonmax.cxx
U libcvd/cvd_src/faster_corner_10.cxx
U libcvd/cvd_src/faster_corner_12.cxx
U libcvd/cvd_src/faster_corner_9.cxx
U libcvd/cvd_src/faster_corner_utilities.h
U libcvd/cvd_src/globlist.cxx
U libcvd/cvd_src/gltext.cpp
U libcvd/cvd_src/glwindow.cc
U libcvd/cvd_src/half_sample.cc
U libcvd/cvd_src/image_io.cc
U libcvd/cvd_src/mono.h
U libcvd/cvd_src/nonmax_suppression.cxx
U libcvd/cvd_src/sans.h
U libcvd/cvd_src/serif.h
U libcvd/cvd_src/slower_corner_10.cxx
U libcvd/cvd_src/slower_corner_11.cxx
U libcvd/cvd_src/slower_corner_12.cxx
U libcvd/cvd_src/slower_corner_7.cxx
U libcvd/cvd_src/slower_corner_8.cxx
U libcvd/cvd_src/slower_corner_9.cxx
U libcvd/cvd_src/synchronized.cpp
U libcvd/cvd_src/tensor_voting.cc
U libcvd/cvd_src/thread.cpp
U libcvd/cvd_src/timeddiskbuffer.cc
U libcvd/cvd_src/utility_helpers.h
U libcvd/cvd_src/videodisplay.cc
U libcvd/cvd_src/videofilebuffer.cc
U libcvd/cvd_src/videosource.cpp
U libcvd/cvd_src/yuv411_to_stuff.cxx
U libcvd/cvd_src/yuv420.cpp
U libcvd/cvd_src/yuv422.cpp
U libcvd/cvd_src/yuv422.h
cvs checkout: Updating libcvd/cvd_src/IRIX
U libcvd/cvd_src/IRIX/O2buffer.cxx
U libcvd/cvd_src/IRIX/sgi-video.cxx
cvs checkout: Updating libcvd/cvd_src/Linux
U libcvd/cvd_src/Linux/dvbuffer.cc
U libcvd/cvd_src/Linux/dvbuffer3_dc1394v1.cc
U libcvd/cvd_src/Linux/dvbuffer3_dc1394v2.cc
U libcvd/cvd_src/Linux/kernel-video1394.h
U libcvd/cvd_src/Linux/v4l1buffer.cc
U libcvd/cvd_src/Linux/v4l2buffer.cc
U libcvd/cvd_src/Linux/v4lbuffer.cc
U libcvd/cvd_src/Linux/v4lcontrol.cc
cvs checkout: Updating libcvd/cvd_src/OSX
U libcvd/cvd_src/OSX/qtbuffer.cpp
cvs checkout: Updating libcvd/cvd_src/SSE2
cvs checkout: Updating libcvd/cvd_src/Win32
U libcvd/cvd_src/Win32/glwindow.cpp
U libcvd/cvd_src/Win32/win32.cpp
U libcvd/cvd_src/Win32/win32.h
cvs checkout: Updating libcvd/cvd_src/fast
U libcvd/cvd_src/fast/fast_10_detect.cxx
U libcvd/cvd_src/fast/fast_10_score.cxx
U libcvd/cvd_src/fast/fast_11_detect.cxx
U libcvd/cvd_src/fast/fast_11_score.cxx
U libcvd/cvd_src/fast/fast_12_detect.cxx
U libcvd/cvd_src/fast/fast_12_score.cxx
U libcvd/cvd_src/fast/fast_7_detect.cxx
U libcvd/cvd_src/fast/fast_7_score.cxx
U libcvd/cvd_src/fast/fast_8_detect.cxx
U libcvd/cvd_src/fast/fast_8_score.cxx
U libcvd/cvd_src/fast/fast_9_detect.cxx
U libcvd/cvd_src/fast/fast_9_score.cxx
U libcvd/cvd_src/fast/prototypes.h
cvs checkout: Updating libcvd/cvd_src/i686
U libcvd/cvd_src/i686/byte_to_double_gradient.s
U libcvd/cvd_src/i686/byte_to_float_gradient.s
U libcvd/cvd_src/i686/byte_to_short_difference.s
U libcvd/cvd_src/i686/convert_rgb_to_y.cc
U libcvd/cvd_src/i686/convolve_float.s
U libcvd/cvd_src/i686/convolve_float4.s
U libcvd/cvd_src/i686/convolve_gaussian.cc
U libcvd/cvd_src/i686/float_add_mul_add.s
U libcvd/cvd_src/i686/float_add_mul_add_unaligned.s
U libcvd/cvd_src/i686/float_assign_mul.s
U libcvd/cvd_src/i686/float_difference.s
U libcvd/cvd_src/i686/float_innerproduct.s
U libcvd/cvd_src/i686/gradient.cc
U libcvd/cvd_src/i686/halfsample.s
U libcvd/cvd_src/i686/int_difference.s
U libcvd/cvd_src/i686/median_3x3.cc
U libcvd/cvd_src/i686/rgb_to_gray.s
U libcvd/cvd_src/i686/short_difference.s
U libcvd/cvd_src/i686/testconf
U libcvd/cvd_src/i686/utility_byte_differences.cc
U libcvd/cvd_src/i686/utility_double_int.cc
U libcvd/cvd_src/i686/utility_float.cc
U libcvd/cvd_src/i686/yuv411_to_stuff_MMX.C
U libcvd/cvd_src/i686/yuv411_to_stuff_MMX_64.C
U libcvd/cvd_src/i686/yuv420p_to_rgb.s
U libcvd/cvd_src/i686/yuv422_to_grey.s
U libcvd/cvd_src/i686/yuv422_to_rgb.s
U libcvd/cvd_src/i686/yuv422_wrapper.cc
cvs checkout: Updating libcvd/cvd_src/noarch
U libcvd/cvd_src/noarch/convert_rgb_to_y.cc
U libcvd/cvd_src/noarch/convolve_gaussian.cc
U libcvd/cvd_src/noarch/default_memalign.cpp
U libcvd/cvd_src/noarch/gradient.cc
U libcvd/cvd_src/noarch/median_3x3.cc
U libcvd/cvd_src/noarch/posix_memalign.cpp
U libcvd/cvd_src/noarch/utility_byte_differences.cc
U libcvd/cvd_src/noarch/utility_double_int.cc
U libcvd/cvd_src/noarch/utility_float.cc
U libcvd/cvd_src/noarch/yuv422_wrapper.cc
cvs checkout: Updating libcvd/cvd_src/nothread
U libcvd/cvd_src/nothread/runnable_batch.cc
cvs checkout: Updating libcvd/cvd_src/posix
cvs checkout: Updating libcvd/cvd_src/thread
U libcvd/cvd_src/thread/runnable_batch.cc
cvs checkout: Updating libcvd/doc
U libcvd/doc/cameracalib2cm.pdf
U libcvd/doc/tutorial.h
cvs checkout: Updating libcvd/make
U libcvd/make/compile_deps.awk
U libcvd/make/log_to_changelog.awk
U libcvd/make/march_flags
cvs checkout: Updating libcvd/pnm_src
U libcvd/pnm_src/bmp.cxx
U libcvd/pnm_src/fits.cc
U libcvd/pnm_src/jpeg.cxx
U libcvd/pnm_src/png.cc
U libcvd/pnm_src/pnm_grok.cxx
U libcvd/pnm_src/save_postscript.cxx
U libcvd/pnm_src/text.cxx
U libcvd/pnm_src/text_write.cc
U libcvd/pnm_src/tiff.cxx
U libcvd/pnm_src/tiffwrite.cc
cvs checkout: Updating libcvd/progs
U libcvd/progs/calibrate.cxx
U libcvd/progs/cvd_display_image.cxx
U libcvd/progs/cvd_image_viewer.cxx
U libcvd/progs/img_play.cxx
U libcvd/progs/img_play_bw.cxx
U libcvd/progs/img_play_deinterlace.cxx
U libcvd/progs/img_play_generic.cxx
U libcvd/progs/se3_exp.cxx
U libcvd/progs/se3_inv.cxx
U libcvd/progs/se3_ln.cxx
U libcvd/progs/se3_post_mul.cxx
U libcvd/progs/se3_pre_mul.cxx
U libcvd/progs/video_play.cc
U libcvd/progs/video_play_bw.cc
U libcvd/progs/video_play_source.cc
cvs checkout: Updating libcvd/python
U libcvd/python/setup.py
cvs checkout: Updating libcvd/python/CVD
U libcvd/python/CVD/cvd.cpp
cvs checkout: Updating libcvd/test
U libcvd/test/diskbuffer2.cxx
U libcvd/test/dvbuffer3_bayerrgb.cxx
U libcvd/test/dvbuffer3_mono.cxx
U libcvd/test/dvbuffer_controls.cxx
U libcvd/test/dvbuffer_mono.cxx
U libcvd/test/dvbuffer_rgb.cxx
U libcvd/test/dvbuffer_yuvrgb.cxx
U libcvd/test/fast_test.cxx
U libcvd/test/floodfill_test.cc
U libcvd/test/font.cpp
U libcvd/test/o2buffer.cxx
U libcvd/test/test_images.cxx
U libcvd/test/v4l1buffer_bayer.cxx
U libcvd/test/v4l1buffer_mono.cxx
U libcvd/test/v4l1buffer_rgb.cxx
U libcvd/test/v4l2buffer.cxx
U libcvd/test/v4lbuffer_bayerrgb.cxx
U libcvd/test/v4lbuffer_mono.cxx
U libcvd/test/videoprog.cxx
cvs checkout: Updating libcvd/test/fast_test_image
U libcvd/test/fast_test_image/noise.pgm
cvs checkout: Updating libcvd/test/images
U libcvd/test/images/1-byte-bin.pgm
U libcvd/test/images/1-byte-bin.ppm
U libcvd/test/images/1-byte-txt.pgm
U libcvd/test/images/1-byte-txt.ppm
U libcvd/test/images/2-byte-bin.pgm
U libcvd/test/images/2-byte-bin.ppm
U libcvd/test/images/2-byte-txt.pgm
U libcvd/test/images/2-byte-txt.ppm
U libcvd/test/images/colour.jpg
U libcvd/test/images/grey.jpg
cvs checkout: Updating libcvd/test/images/tiff
U libcvd/test/images/tiff/grey-bool-inverted.tiff
U libcvd/test/images/tiff/grey-uint16-normal.tiff
U libcvd/test/images/tiff/grey-uint8-normal.tiff
U libcvd/test/images/tiff/rgb-uint16.tiff
U libcvd/test/images/tiff/rgb-uint8.tiff
cvs checkout: Updating libcvd/util
%% cvs -z3 -d:pserver:anonymous@cvs.savannah.nongnu.org:/sources/libcvd co -D "Mon May 11 16:29:26 BST 2009" gvars3
실행 결과:
cvs checkout: warning: failed to open /Users/lym/.cvspass for reading: No such file or directory
cvs checkout: Updating gvars3
U gvars3/Authors
U gvars3/GVars2.h.historic
U gvars3/LICENSE
U gvars3/Makefile.in
U gvars3/config.guess
U gvars3/config.sub
U gvars3/configure
U gvars3/configure.ac
U gvars3/fltk2_test
U gvars3/fltk_test
U gvars3/install-sh
U gvars3/main.cc
cvs checkout: Updating gvars3/build
cvs checkout: Updating gvars3/build/vc2005
U gvars3/build/vc2005/gvars3-headless.vcproj
U gvars3/build/vc2005/gvars3.sln
U gvars3/build/vc2005/gvars3.vcproj
cvs checkout: Updating gvars3/build/vc2008
U gvars3/build/vc2008/gvars3-headless.vcproj
U gvars3/build/vc2008/gvars3.sln
U gvars3/build/vc2008/gvars3.vcproj
cvs checkout: Updating gvars3/gvars2_compat
cvs checkout: Updating gvars3/gvars3
U gvars3/gvars3/GStringUtil.h
U gvars3/gvars3/GUI.h
U gvars3/gvars3/GUI_Fltk.h
U gvars3/gvars3/GUI_Fltk2.h
U gvars3/gvars3/GUI_Motif.h
U gvars3/gvars3/GUI_Widgets.h
U gvars3/gvars3/GUI_non_readline.h
U gvars3/gvars3/GUI_readline.h
U gvars3/gvars3/config.h.in
U gvars3/gvars3/default.h
U gvars3/gvars3/gv3_implementation.hh
U gvars3/gvars3/gvars3.h
U gvars3/gvars3/instances.h
U gvars3/gvars3/serialize.h
U gvars3/gvars3/type_name.h
cvs checkout: Updating gvars3/src
U gvars3/src/GStringUtil.cc
U gvars3/src/GUI.cc
U gvars3/src/GUI_Fltk.cc
U gvars3/src/GUI_Fltk2.cc
U gvars3/src/GUI_Motif.cc
U gvars3/src/GUI_impl.h
U gvars3/src/GUI_impl_headless.cc
U gvars3/src/GUI_impl_noreadline.cc
U gvars3/src/GUI_impl_readline.cc
U gvars3/src/GUI_language.cc
U gvars3/src/GUI_non_readline.cc
U gvars3/src/GUI_none.cc
U gvars3/src/GUI_readline.cc
U gvars3/src/gvars2.cc
U gvars3/src/gvars3.cc
U gvars3/src/inst.cc
U gvars3/src/inst_headless.cc
U gvars3/src/serialize.cc
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for decltype... no
checking for typeof... yes
You're on the development branch of TooN 2.0. Everything will probably work, but
the interface is a bit different from TooN-1.x
If you want TooN-1, then get it using:
cvs -z3 -d:pserver:anoncvs@cvs.savannah.nongnu.org:/cvsroot/toon co -r Maintenance_Branch_1_x TooN
or update what you currently have using:
cvs up -r Maintenance_Branch_1_x
or head over to:
http://mi.eng.cam.ac.uk/~er258/cvd/
Otherwise, please report any bugs you come across.
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking for ranlib... ranlib
checking how to run the C++ preprocessor... g++ -E
checking if compiler flag -Wall works... yes
checking if compiler flag -Wextra works... yes
checking if compiler flag -pipe works... yes
checking if compiler flag -ggdb works... yes
checking if compiler flag -fPIC works... yes
checking build system type... i386-apple-darwin9.7.0
checking host system type... i386-apple-darwin9.7.0
checking for best optimize flags...
checking if compiler flag -O3 works... yes
checking CPU type... unknown
------------------------------------
Checking processor specific features
------------------------------------
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking whether byte ordering is bigendian... no
checking for MMX support... yes
checking for MMXEXT support... yes
checking for SSE support... yes
checking for SSE2 support... yes
checking for SSE3 support... yes
checking for void*... yes
checking size of void*... 4
checking for inline asm statement... yes
checking assembler supports .type pseudo-op... no
-----------------------------------------------
Checking for operating system specific features
-----------------------------------------------
checking dc1394/dc1394.h usability... no
checking dc1394/dc1394.h presence... no
checking for dc1394/dc1394.h... no
checking for main in -ldc1394... no
checking for /opt/local... yes
checking for /sw... no
configure: Adding /usr/X11R6/include to the build path.
checking Carbon and QuickTime framework... yes
-------------------------------
Checking for optional libraries
-------------------------------
checking for X... libraries /usr/X11/lib, headers /usr/X11/include
checking for glDrawPixels in -lGL... yes
checking GL/glu.h usability... yes
checking GL/glu.h presence... yes
checking for GL/glu.h... yes
checking for gluGetString in -lGLU... yes
checking for tr1::shared_ptr... yes
checking for TooN... yes
checking Old TooN... no
checking for dgesvd_ in -lacml... no
checking if Accelerate framework is needed for LAPACK...
checking for dgesvd_... yes
checking for working pthreads... yes
checking for pthread_yield... no
checking for pthread_yield_np... yes
checking png.h usability... yes
checking png.h presence... yes
checking for png.h... yes
checking for png_init_io in -lpng... yes
checking jpeglib.h usability... yes
checking jpeglib.h presence... yes
checking for jpeglib.h... yes
checking for jpeg_destroy_decompress in -ljpeg... yes
checking JPEG read buffer size... 1 (safe reading)
checking tiffio.h usability... yes
checking tiffio.h presence... yes
checking for tiffio.h... yes
checking for TIFFReadRGBAImage in -ltiff... yes
checking for TIFFReadRGBAImageOriented in -ltiff... yes
checking for doxygen... no
-----------------------------------
Checking for platform compatibility
-----------------------------------
checking glob.h usability... yes
checking glob.h presence... yes
checking for glob.h... yes
checking for glob... yes
checking for GLOB_BRACE and GLOB_TILDE in glob.h... yes
checking whether feenableexcept is declared... no
checking for posix_memalign... no
--------------------------------
Checking for extra build options
--------------------------------
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking iostream usability... yes
checking iostream presence... yes
checking for iostream... yes
checking build system type... i386-apple-darwin9.7.0
checking host system type... i386-apple-darwin9.7.0
configure: Adding /sw to the build path.
configure: Adding /opt/local to the build path.
checking if compiler flag -Wall works... yes
checking if compiler flag -Wextra works... yes
--------------------------
Checking for options
--------------------------
checking for TooN... yes
checking for TooN-2... yes
checking if compiler flag -pthread works... yes
checking for rl_done in -lreadline... yes
------------------------------------------------
Checking for widget libraries (provides GUI_...)
------------------------------------------------
configure: WARNING: No GUI functionality enabled
Options:
toon readline
내 (OS X의) 경우, PTAM/Build/OS X에 있는 모든 (두 개의) 파일 Makefile과 VideoSource_OSX.cc를 PTAM 폴더에 옮겼다.
3-2. video source 셋업
카메라에 맞는 video input file을 컴파일하도록 Makefile을 수정해 주어야 한다.
맥의 경우, (아마도 Logitech Quickcam Pro 5000 을 기준으로 하는) 하나의 소스 파일만이 존재하므로 그대로 두면 될 듯.
3-3. video source 추가
다른 비디오 소스들은 libCVD에 클래스로 만들어져 있다고 한다. 여기에 포함되어 있지 않은 경우에는 VideoSource_XYZ.cc 라는 식의 이름을 갖는 파일을 만들어서 넣어 주어야 한다.
3-4. compile
PTAM 폴더에 들어가서
%% make
실행 결과:
g++ -g -O3 main.cc -o main.o -c -I /MY_CUSTOM_INCLUDE_PATH/ -D_OSX -D_REENTRANT
g++ -g -O3 VideoSource_OSX.cc -o VideoSource_OSX.o -c -I /MY_CUSTOM_INCLUDE_PATH/ -D_OSX -D_REENTRANT
g++ -g -O3 GLWindow2.cc -o GLWindow2.o -c -I /MY_CUSTOM_INCLUDE_PATH/ -D_OSX -D_REENTRANT
In file included from OpenGL.h:20,
from GLWindow2.cc:1:
/usr/local/include/cvd/gl_helpers.h:38:19: error: GL/gl.h: No such file or directory
/usr/local/include/cvd/gl_helpers.h:39:20: error: GL/glu.h: No such file or directory
/usr/local/include/cvd/gl_helpers.h: In function 'void CVD::glPrintErrors()':
/usr/local/include/cvd/gl_helpers.h:569: error: 'gluGetString' was not declared in this scope
make: *** [GLWindow2.o] Error 1
PTAM이 OpenGL을 사용하고 있는데, OpenGL이 Mac에 기본으로 설치되어 있으므로 신경쓰지 않았던 부분이다. 물론 system의 public framework으로 들어가 있음을 확인할 수 있다. 그런데 UNIX 프로그램에서 접근할 수는 없는가? (인터넷에서 검색해 보아도 따로 설치할 수 있는 다운로드 링크나 방법을 찾을 수 없다.)
에러 메시지에 대한 정확한 진단 ->
philphys: 일단 OpenGL은 분명히 있을 건데 그 헤더파일과 라이브러리가 있는 곳을 지정해 주지 않아서 에러가 나는 것 같아. 보통 Makefile에 이게 지정되어 있어야 하는데 실행결과를 보니까 전혀 지정되어 있지 않네. 중간에 보면 -I /MY_CUSTOM_INCLUDE_PATH/ 라는 부분이 헤더 파일의 위치를 지정해 주는 부분이고 또 라이브러리는 뒤에 링크할 때 지정해 주게 되어 있는데 거기까지는 가지도 못 했네.
즉, "링커가 문제가 아니라, 컴파일러 옵션에 OpenGL의 헤더파일이 있는 디렉토리를 지정해 주어야 할 것 같다"고 한다.
문제의 Makefile을 들여다보고
# DO NOT DELETE THIS LINE -- make depend depends on it.
# Edit the lines below to point to any needed include and link paths
# Or to change the compiler's optimization flags CC = g++ -g -O3 COMPILEFLAGS = -I /MY_CUSTOM_INCLUDE_PATH/ -D_OSX -D_REENTRANT LINKFLAGS = -framework OpenGL -framework VecLib -L/MY_CUSTOM_LINK_PATH/ -lGVars3 -lcvd
(다음은 philphys 인용) 파란색 부분 - 각 소스코드를 컴파일한 다음 컴파일된 오브젝트 코드를 실행파일로 링크하는 부분. 여기서는 $(LINKFLAGS)에 링커 프로그램에 전달되는 옵션이 들어간다. 초록색 부분 - 컴파일할 오브젝트 코드의 리스트 분홍색 부분 - CC는 컴파일러 프로그램과 기본 옵션. COMPILEFLAGS는 컴파일러에 전달하는 옵션들, 여기에 헤더 파일의 위치를 정할 수 있다. LINKFLAGS는 컴파일된 오브젝트 코드를 실행파일로 링크하는 링커에 들어가는 옵션. 여기에 라이브러리의 위치와 사용할 라이브러리를 지정해 준다. 라이브러리의 위치는 -L 옵션으로, 구체적인 라이브러리 이름은 -l 옵션으로.
maetel: 사실 프레임웍 안에 gl.h 파일이 있는 위치는 다음과 같다.
/System/Library/Frameworks/OpenGL.framework/Versions/A/Headers
philphys: "근데 코드에서는 그 헤더를 "GL/gl.h"로 찾는다는 게 문제. 이건 프레임웍 방식이 아닌 고전적인 유닉스 Xwindows의 OpenGL 방식이다. 즉 방금 보인 gl.h, glu.h 등이 있는 /usr/X11R6/include 를 COMPILERFLAGS에 -I 옵션으로 넣어 줘야 하는 게 아닐까."
philphys: /usr/X11R6/include 밑에 GL 폴더가 있고 거기에 필요한 헤더파일들이 모두 들어 있다. 그래서 코드에선 "GL/gl.h" 하는 식으로 explicit하게 GL 폴더를 찾게 된다.
그러고 보면 아래와 같은 설명이 있었던 것이다.
Since the Linux code compiles directly against the nVidia driver's GL headers, use of a different GL driver may require some modifications to the code.
CameraCalibrator 파일을 실행시켜 카메라 캘리브레이션을 시도했더니 GUI 창이 뜨는데 연결된 웹캠(Logitech QuickCam Pro 4000)으로부터 입력을 받지 못 한다.
4-0. 증상
CameraCalibrator 실행파일을 열면, 다음과 같은 터미널 창이 새로 열린다.
Last login: Fri Aug 7 01:14:05 on ttys001
%% /Users/lym/PTAM/CameraCalibrator ; exit;
Welcome to CameraCalibrator
--------------------------------------
Parallel tracking and mapping for Small AR workspaces
Copyright (C) Isis Innovation Limited 2008
Parsing calibrator_settings.cfg ....
! GUI_impl::Loadfile: Failed to load script file "calibrator_settings.cfg".
VideoSource_OSX: Creating QTBuffer....
IMPORTANT
This will open a quicktime settings planel.
You should use this settings dialog to turn the camera's
sharpness to a minimum, or at least so small that no sharpening
artefacts appear! In-camera sharpening will seriously degrade the
performance of both the camera calibrator and the tracking system.
>
그리고 Video란 이름의 GUI 창이 열리는데, 이때 아무런 설정을 바꾸지 않고 그대로 OK를 누르면 위의 터미널 창에 다음과 같은 메시지가 이어지면서 자동 종료된다.
.. created QTBuffer of size [640 480]
2009-08-07 01:20:57.231 CameraCalibrator[40836:10b] ***_NSAutoreleaseNoPool(): Object 0xf70e2c0 of class NSThread autoreleasedwith no pool in place - just leaking
Stack: (0x96827f0f 0x96734442 0x9673a1b4 0xbc2db7 0xbc7e9a 0xbc69d30xbcacbd 0xbca130 0x964879c9 0x90f8dfb8 0x90e69618 0x90e699840x964879c9 0x90f9037c 0x90e7249c 0x90e69984 0x964879c9 0x90f8ec800x90e55e05 0x90e5acd5 0x90e5530f 0x964879c9 0x94179eb9 0x282b48 0xd9f40xd6a6 0x2f16b 0x2fea4 0x26b6)
! Code for converting from format "Raw RGB data"
not implemented yet, check VideoSource_OSX.cc.
logout
[Process completed]
그러므로 3-3의 문제 -- set up video source (비디오 소스 셋업) --로 돌아가야 한다.
즉, VideoSource_OSX.cc 파일을 수정해서 다시 컴파일한 후 실행해야 한다.
Other video source classes are available with libCVD. Finally, if a custom video source not supported by libCVD is required, the code for it will have to be put into some VideoSource_XYZ.cc file (the interface for this file is very simple.)
삽질...
터미널에서 calibrator_settings.cfg 파일을 로드하지 못 했다고 하기에, 그 파일을 찾아보았다.
%% find . -name "calibrator_settings.cfg" -print
./calibrator_settings.cfg
알고 보니 ./는 현재 디렉토리를 말하나 보다. PTAM 폴더 밑에 바로 있다. 왜 못 봤을까...
열어 보니 다음과 같다.
// This file is parsed by the CameraCalibrator executable
// Put any custom gvars settings you want in here
// For example: to increase the camera calibrator's blur parameter,
// uncomment the following line
// CameraCalibrator.BlurSigma=2.0
VideoSource::VideoSource()
{
cout << " VideoSource_OSX: Creating QTBuffer...." << endl;
cout << " IMPORTANT " << endl;
cout << " This will open a quicktime settings planel. " << endl
<< " You should use this settings dialog to turn the camera's " << endl
<< " sharpness to a minimum, or at least so small that no sharpening " << endl
<< " artefacts appear! In-camera sharpening will seriously degrade the " << endl
<< " performance of both the camera calibrator and the tracking system. " << endl;
QTBuffer<yuv422>* pvb;
try
{
pvb= new QTBuffer<yuv422>(ImageRef(640,480), 0, true);
}
catch (CVD::Exceptions::All a)
{
cerr << " Error creating QTBuffer; expection: " << a.what << endl;
exit(1);
}
mptr = pvb;
mirSize = pvb->size();
cout << " .. created QTBuffer of size " << mirSize << endl;
};
그러나 위에서 보듯 size가 맞지 않고 frame이 겹치는 등 넘어갈 수 없는 문제가 있다. (오른쪽 이미지는 같은 상황에서의 비교를 위해 맥용 드라이버 macam에서의 입력을 캡쳐한 것.)
터미널에서의 결과:
%% /Users/lym/PTAM/CameraCalibrator ; exit;
Welcome to CameraCalibrator
--------------------------------------
Parallel tracking and mapping for Small AR workspaces
Copyright (C) Isis Innovation Limited 2008
Parsing calibrator_settings.cfg ....
! GUI_impl::Loadfile: Failed to load script file "calibrator_settings.cfg".
VideoSource_OSX: Creating QTBuffer....
IMPORTANT
This will open a quicktime settings planel.
You should use this settings dialog to turn the camera's
sharpness to a minimum, or at least so small that no sharpening
artefacts appear! In-camera sharpening will seriously degrade the
performance of both the camera calibrator and the tracking system.
> .. created QTBuffer of size [640 480]
2009-08-12 23:10:38.309 CameraCalibrator[5110:10b] *** _NSAutoreleaseNoPool(): Object 0xf8054e0 of class NSThread autoreleased with no pool in place - just leaking
Stack: (0x96670f4f 0x9657d432 0x965831a4 0xc17db7 0xc1ce9a 0xc1b9d3 0xc1fcbd 0xc1f130 0x924b09c9 0x958e8fb8 0x957c4618 0x957c4984 0x924b09c9 0x958eb37c 0x957cd49c 0x957c4984 0x924b09c9 0x958e9c80 0x957b0e05 0x957b5cd5 0x957b030f 0x924b09c9 0x90bd4eb9 0x282b48 0xd9e4 0xd5a6 0x2f15b 0x2fe94 0x25b6)
4-1-2. YUV format 확인
Logitech QuickCam Pro 4000는 YUV420P라는 글을 어디선가 보고, 코드의 yuv422 부분을 yuv420p로 바꾸었으나 증상은 그대로이다.
skype forum:
There is another problem here with Xgl unfortunately, as it
isn'tsupporting the YUV420P (I420) overlay colour/pixel format either.
Soeven if the camera captures successfully (which looks okay on
thosesettings), then you still won't see anything.
왼쪽: macam에서 캡쳐한 이미지 / 오른쪽: PTAM CameraCalibrator에서 보여지는 이미지
Logitech QuickCam Pro 4000
Welcome to CameraCalibrator
--------------------------------------
Parallel tracking and mapping for Small AR workspaces
Copyright (C) Isis Innovation Limited 2008
Parsing calibrator_settings.cfg ....
VideoSource_OSX: Creating QTBuffer....
IMPORTANT
This will open a quicktime settings planel.
You should use this settings dialog to turn the camera's
sharpness to a minimum, or at least so small that no sharpening
artefacts appear! In-camera sharpening will seriously degrade the
performance of both the camera calibrator and the tracking system.
> .. created QTBuffer of size [640 480]
2009-08-13 04:02:50.464 CameraCalibrator[6251:10b] ***
_NSAutoreleaseNoPool(): Object 0x9df180 of class NSThread autoreleased
with no pool in place - just leaking
Stack: (0x96670f4f 0x9657d432 0x965831a4 0xbc2db7 0xbc7e9a 0xbc69d3
0xbcacbd 0xbca130 0x924b09c9 0x958e8fb8 0x957c4618 0x957c4984
0x924b09c9 0x958eb37c 0x957cd49c 0x957c4984 0x924b09c9 0x958e9c80
0x957b0e05 0x957b5cd5 0x957b030f 0x924b09c9 0x90bd4eb9 0x282b48 0xd414
0xcfd6 0x2f06b 0x2fda4)
4-2. Camera Calibrator 실행
Camera calib is [ 1.51994 2.03006 0.499577 0.536311 -0.0005 ]
Saving camera calib to camera.cfg...
.. saved.
5. PTAM 실행
Welcome to PTAM
---------------
Parallel tracking and mapping for Small AR workspaces
Copyright (C) Isis Innovation Limited 2008
Parsing settings.cfg ....
VideoSource_OSX: Creating QTBuffer....
IMPORTANT
This will open a quicktime settings planel.
You should use this settings dialog to turn the camera's
sharpness to a minimum, or at least so small that no sharpening
artefacts appear! In-camera sharpening will seriously degrade the
performance of both the camera calibrator and the tracking system.
> .. created QTBuffer of size [640 480]
2009-08-13 20:17:54.162 ptam[1374:10b] *** _NSAutoreleaseNoPool(): Object 0x8f5850 of class NSThread autoreleased with no pool in place - just leaking
Stack: (0x96670f4f 0x9657d432 0x965831a4 0xbb9db7 0xbbee9a 0xbbd9d3 0xbc1cbd 0xbc1130 0x924b09c9 0x958e8fb8 0x957c4618 0x957c4984 0x924b09c9 0x958eb37c 0x957cd49c 0x957c4984 0x924b09c9 0x958e9c80 0x957b0e05 0x957b5cd5 0x957b030f 0x924b09c9 0x90bd4eb9 0x282b48 0x6504 0x60a6 0x11af2 0x28da 0x2766)
ARDriver: Creating FBO... .. created FBO.
MapMaker: made initial map with 135 points.
MapMaker: made initial map with 227 points.
CameraCalibrator를 실행하면 뜨는 GUI 창에서 다음과 같이 기본 설정이 적용되고,
Source: iSight
Compression type: Component Video - CCIR-601 uyvy
rms error 0.3 이하로 깔끔하게 수렴된 카메라 파라미터의 계산값이 다음과 같은 식으로 나올 때,
Camera calib is [ 1.22033 1.62577 0.489375 0.641251 0.544352 ]
PTAM을 실행하면 위의 Logitech QuickCam 두 기종보다는 features를 보다 잘 잡는다. (아직은 많이 부족하지만...)
Welcome to PTAM
---------------
Parallel tracking and mapping for Small AR workspaces
Copyright (C) Isis Innovation Limited 2008
Parsing settings.cfg ....
VideoSource_OSX: Creating QTBuffer....
IMPORTANT
This will open a quicktime settings planel.
You should use this settings dialog to turn the camera's
sharpness to a minimum, or at least so small that no sharpening
artefacts appear! In-camera sharpening will seriously degrade the
performance of both the camera calibrator and the tracking system.
> .. created QTBuffer of size [640 480]
ARDriver: Creating FBO... .. created FBO.
MapMaker: made initial map with 242 points.
MapMaker: made initial map with 259 points.
MapMaker: made initial map with 323 points.
MapMaker: made initial map with 626 points.
In Proceedings
of the International Conference on Computer Vision, Rio de Janeiro, Brazil, 2007 demo 1 demo 2
• real-time, high-accuracy localisation and mapping during tracking
• real-time (re-)localisation when when tracking fails
• on-line learning of image patch appearance so that no prior training or map structure is required and features are added and removed during operation.
Lepetit's image patch classifier (feature appearance learning)
=> integrating the classifier more closely into the process of map-building
(by using classification results to aid in the selection of new points to add to the map)
> recovery from tracking failure: local vs. global
local - particle filter -> rich feature descriptor
global - proximity using previous key frames
- based on SceneLib (Extended Kalman Filter)
- rotational (and a degree of perspective) invariance via local patch warping
- assuming the patch is fronto-parallel when first seen http://freshmeat.net/projects/scenelib/
active search
innovation covariance
joint compatibility test
randomized lists key-point recognition algorithm
1. randomized: (2^D - 1) tests -> D tests
2. independent treatment of classes
3. binary leaf scores (2^D * C * N bits for all scores)
4. intensity offset
5. explicit noise handing
ref.
Davison, A. J. and Molton, N. D. 2007. MonoSLAM: Real-Time Single Camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29, 6 (Jun. 2007), 1052-1067. DOI= http://dx.doi.org/10.1109/TPAMI.2007.1049
Lepetit, V., Lagger, P., and Fua, P. 2005. Randomized Trees for Real-Time Keypoint Recognition. In Proceedings
of the 2005 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (Cvpr'05) - Volume 2 - Volume 02 (June 20 - 26, 2005). CVPR. IEEE Computer Society, Washington, DC, 775-781. DOI= http://dx.doi.org/10.1109/CVPR.2005.288
Mark Paskin
Java library with several SLAM variants, including Kalman filter, information filter, and thin junction tree forms. Includes a MATLAB interface. http://www.stanford.edu/~paskin/slam/
Radish (The Robotics Many and varied indoor datasets, including large-area Data Set Repository) data from the CSU Stanislaus Library, the Intel Research Lab in Seattle, the Edmonton Convention Centre, and more. http://radish.sourceforge.net/
IJRR (The International Journal of Robotics Research)
IJRR maintains a Web page for each article, often containing data and video of results. A good paper example is by Bosse et al. [3], which has data from Killian Court at MIT. http://www.ijrr.org/contents/23\_12/abstract/1113.html
ICARCV 2010 - The 11th International Conference on Control, Automation, Robotics and Vision http://www.icarcv.org/2010/
History
- 1986, probabilistic SLAM problem (IEEE Robotics and Automation Conference)
Peter Cheeseman, Jim Crowley, and Hugh Durrant-Whyte, Raja Chatila, Oliver Faugeras, Randal Smith
> estimation-theoretic methods, consistent mapping
- consistent probabilistic mapping
Smith and Cheesman [39] and Durrant-Whyte [17]
> statistical basis
"There must be a high degree of correlation between estimates of the location of different landmarks in a map"
- visual navigation & sonar-based navigation
Ayache and Faugeras [1], Crowley [9] and Chatila and Laumond [6]
> Kalman filter-type algorithms
"The estimations of the landmarks are all necessarily correlated with each other because of the common error in estimated vehicle location."
> joint state (of the vehicle pose and every landmark position) to be updated following each landmark observation & estimator (state vector)
- random-walk behavior with unbounded error growth (without knowledge of the convergence behavior of the map)
> single estimation problem: "The combined mapping and localization problem is convergent."
"The more the correlations between landmarks grew, the better the solution."
- 1995, coining of SLAM (a paper at the International Symposium on Robotics Research) or called CLM (concurrent mapping and localization)
Csorba [10], [11]. the Massachusetts Institute of Technology [29], Zaragoza [4], [5], the ACFR at Sydney [20], [45], and others [7], [13]
> computational efficiency, addressing in data association, loop closure
- 1999 ISRR, convergence between the Kalman-filter-based SLAM methods and the probabilistic localisation and mapping methods introduced by Thrun
- 2000 IEEE ICRA
> algorithmic complexity, data association, implementation
Formulation
SLAM = process by which a mobile robot can build a map of an environment and at the same time use this map to deduce its location
(In SLAM, both the trajectory of the platform and the location of all landmarks are estimated online without the need for any a priori knowledge of location.)
Probabilistic SLAM
The SLAM probability distribution = the joint posterior density of the landmark locations and vehicle state (at time k) given the recorded observations and control inputs up to and including time k together with the initial state of the vehicle
recursive solution
: observation model + motion (state transition) model == Bayes theorem ==> estimate of SLAM distribution
observation model -> prediction (; measurement update)
+ motion model -> correction (; time update)
+ Markov process
=> map building problem + localization problem
: joint posterior density of the landmark locations and vehicle state
As the map is built, the location accuracy of the robot measured relative to the map is bounded only by the quality of the map and relative measurement sensor.
: Robot relative location accuracy becomes equal to the localization accuracy achievable with a given map.
2)
a set of samples of a more general non-Gaussian probability distribution to describe vehicle motion
Rao-Blackwellized particle filter or FastSLAM algorithm
> definition
mapping: 환경을 인식가능한 정보로 변환하고
localization: 이로부터 자기 위치를 추정하는 것
> issues
- uncertainty <= sensor
- data association (데이터 조합): 차원이 높은 센서 정보로부터 2-3차원 정도의 정보를 추려내어 이를 지속적으로 - 대응시키는 것
- 관찰된 특징점 자료들을 효율적으로 관리하는 방법
> localization (위치인식)
: 그 위치가 미리 알려진 랜드마크를 관찰한 정보를 토대로 자신의 위치를 추정하는 것
: 초기치 x0와 k-1시점까지의 제어 입력, 관측벡터와 사전에 위치가 알려진 랜드마크를 통하여 매 k시점마다 로봇의 위치를 추정하는 것
- 로봇의 위치추정의 불확실성은 센서의 오차로부터 기인함.
> mapping (지도작성)
: 기준점과 상대좌표로 관찰된 결과를 누적하여 로봇이 위치한 환경을 모델링하는 것
: 위치와 관측정보 그리고 제어입력으로부터 랜드마크 집합을 추정하는 것
- 지도의 부정확성은 센서의 오차로부터 기인함.
> Simultaneous Localization and Mapping (SLAM, 동시간 위치인식 및 지도작성)
: 위치한 환경 내에서 로봇의 위치를 추정하는 것
: 랜드마크 관측벡터와 초기값 그리고 적용된 모든 제어입력이 주어진 상태에서 랜드마크의 위치와 k시점에서의 로봇 상태벡터 xk의 결합확률
- 재귀적 방법 + Bayes 정리
- observation model (관측 모델) + motion model (상태 공간 모델, 로봇의 움직임 모델)
- motion model은 상태 천이가 Markov 과정임을 의미함. (현재 상태는 오직 이전 상태와 입력 벡터로서 기술되고, 랜드마크 집합과 관측에 독립임.)
- prediction (time-update) + correction (measurement-update)
- 불확실성은 로봇 주행거리계와 센서 오차로부터 유발됨.
Klein, G. and Murray, D. 2007. Parallel Tracking and Mapping for Small AR Workspaces
In Proceedings of the 2007 6th IEEE and ACM international Symposium on Mixed and Augmented Reality - Volume 00
(November 13 - 16, 2007). Symposium on Mixed and Augmented Reality.
IEEE Computer Society, Washington, DC, 1-10. DOI=
http://dx.doi.org/10.1109/ISMAR.2007.4538852
1. parallel threads of tracking and mapping
2. mapping from smaller keyframes: batch techniques (Bundle Adjustment)
3. Initializing the map from 5-point Algorithm
4. Initializing new points with epipolar search
5. mapping thousands of points
affine warp
warping matrix <- (1) back-projecting unit pixel displacements in the source keyframe pyramid level onto the patch's plane and then (2) projecting these into the current (target) frame
This paper appears in: Robotics and Automation, 2003. Proceedings. ICRA '03. IEEE International Conference on
Publication Date: 14-19 Sept. 2003
Volume: 2
On page(s): 1985 - 1991 vol.2
Number of Pages: 3 vol.lii+4450
ISSN: 1050-4729
ISBN: 0-7803-7736-2
INSPEC Accession Number:7877180
Digital Object Identifier: 10.1109/ROBOT.2003.1241885
Current Version Published: 2003-11-10
the problem of building a map of an unknown environment from a sequence of noisy landmark measurements obtained from a moving robot + a robot localization problem => SLAM
autonomous robots operating in environments where precise maps and positioning are not available
Extended Kalman Filter (EKF)
: used for incrementally estimating the joint posterior distribution over robot pose and landmark positions
limitations of EKF
1) Quadratic complexity (: sensor updates require time quadratic in the total number of landmarks in the map)
=> limiting the number of landmarks to only a few hundred (whereas natural environment models frequently contain millions of features
2) Data association / correspondence (: mapping of observations to landmarks)
=> associating a small numver of observations with incorrect landmarks to cause the filter to diverge
FastSLAM decomposes the SLAM problem into a robot localization problem, and a collection of landmark estimation problems that are conditioned on the robot pose estimate.
FastSLAM factors the SLAM posterior into a localization problem and K independent landmark estimation problems conditioned on the robot pose estimate.
> a modified particle filter to estimate the posterior over robot paths
> each particle possessing K independent Kalman filters that estimate the landmark locations conditioned on the particle's path
=> an instance of the Rao-Blackwellized particle filter
Representing particles as binary trees of Kalman Filters
-> Incorporating observations into FastSLAM in O(MlogK) time (M, # of particles; K, # of landmarks)
Each particle represents a different robot pose hypothesis.
=> Data association can be considered separately for every particle.
=> 1) The noise of robot motion does not affect the accuracy of data association.
2) Incorrectly associated particls will receive lower probabilities and will be removed in future resampling steps.
On real-world data with ambiguous data association
Adding extra odometric noise
( Odometry is the use of data from the movement of actuators to estimate change in position over time. )
Estimating an accurate map without any odometry in the environment in which the Kalman Filter inevitably diverges.
How to incorporate negative information resulting in a measurable increase in the accuracy of the resulting map
ii) motion noise: robot pose uncertainty after incorporating a control
=> 1) adding a large amount of error to the robot's pose
2) causing a filter to diverge
conditional independece
: The problem of determining the landmark locations could be decoupled into K independent estimation problems, one for each landmark.
FastSLAM estimates the factored SLAM posterior using a modified particle filter, with K independent Kalman Filters for each particle to estimate the landmark positions conditioned on the hypothesized robot paths. The resulting algorithm is an instance of the Rao-Blackwellized particle filter.
This paper appears in: Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on
Publication Date: 13-16 Oct. 2003
On page(s): 1403-1410 vol.2
ISBN: 0-7695-1950-4
INSPEC Accession Number: 7971070
Digital Object Identifier: 10.1109/ICCV.2003.1238654
Current Version Published: 2008-04-03
This paper appears in: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on
Publication Date: 17-22 June 2006
Volume: 1, On page(s): 469- 476
ISSN: 1063-6919
ISBN: 0-7695-2597-0
Digital Object Identifier: 10.1109/CVPR.2006.263
Current Version Published: 2006-07-05
monocular SLAM
particle filter + top-down search => real-time, large number of landmarks
the first to apply this FastSLAM-type particle filter to single-camera SLAM
1. Introduction
SLAM = Simultaneous Localization and Mapping
: process of causally estimating both egomotion and structure in an online system
SLAM using visual data in computer vision
SFM (= structure from motion): reconstructing scene geometry
+ causal or recursive estimation techniques
perspective-projection cameras
filtering methods to allow indirect observation models
Kalman filtering framework
Extended Kalman filter = EKF (-> to linearize the observation and dynamics models of the system)
causal estimation with recursive algorithms (cp. estimation depending only on observations up to the current time)
=> online operation (cp. SFM on global nonlinear optimization)
Davision's SLAM with a single camera
> EKF estimation framework
> top-down Bayesian estimation approach searching for landmarks in image regions constrained by estimate > uncertainty (instead of performing extensive bottom-up image processing and feature matching)
> Bayesian partial-initialization scheme for incorporating new landmarks
- cannot scale to large environment
EKF = the Extended Kalman filter
- N*N covariace matrix for N landmarks
- updated with N*N computation cost
> SLAM system using a single camera as the only sensor
> frame-rate operation with many landmarks
> FastSLAM-style particle filter (the first use of such an approach in a monocular SLAM setting)
> top-down active search
> an efficient algorithm for discovering the depth of new landmarks that avoids linearization errors
> a novel method for using partially initialized landmarks to help constrain camera pose
FastSLAM
: based on the Rao-Blackwellized Particle Filter
2. Background
2.1 Scalable SLAM
> submap
bounded complexity -> bounded computation and space requirements
Montemerlo & Thrun
If the entire camera motion is known then the estimates of the positions of different landmarks become independent of each other.
This paper appears in: Robotics, IEEE Transactions on
Publication Date: Oct. 2008
Volume: 24, Issue: 5
On page(s): 932-945
ISSN: 1552-3098
INSPEC Accession Number: 10301459
Digital Object Identifier: 10.1109/TRO.2008.2003276
First Published: 2008-10-03
Current Version Published: 2008-10-31
Javier Civera, Departamento de Informática e Ingeniería de Sistemas, Universidad de Zaragoza
Andrew J. Davison, Reader in Robot Vision at the Department of Computing, Imperial College London
monocular simultaneous localization and mapping (SLAM)
representation of uncertainty
the standard extended Kalman filter (EKF)
direct parametrization of the inverse depth of features
feature initialization
camera motion estimates
6-D state vector --> converted to the Euclidean XYZ form
linearity index => automatic detection and conversion to maintain maximum efficiency
I. Introduction
monocular camera
: projective sensor measuring the beairng of image features
monocular (adj) 단안(單眼)(용)의, 외눈의
A stereo camera is a type of camera with two or more lenses. This allows the camera to simulate human binocular vision.
structure from motion = SFM
1) feature matching
2) global camera location & scene feature position estimates
sliding window processing
Sliding Window Protocol is a bi-directional data transmission protocol used in the data link layer (OSI model) as well as in TCP (transport layer of the OSI model). It is used to keep a record of the frame sequences sent and their respective acknowledgements received by both the users.
In robotics and computer vision, visual odometry is the process of determining the position and orientation of a robot by analyzing the associated camera images.
Odometry is the use of data from the movement of actuators to estimate change in position over time. Odometry is used by some robots, whether they be legged or wheeled, to estimate (not determine) their position relative to a starting location.
visual SLAM
probabilistic filtering approach
initializing uncertain depth estimates for distance features
Gaussian distributions implicit in the EKF
a new feature parametrization that is able to smoothly cope with initialization of features at all depth - even up to "infinity" - within the standard EKF framework: direct parametrization of inverse depth relative to the camera position from which a feature was first observed
A. Delayed and Undelayed Initialization
main map; main probabilistic state; main state vector
test for inclusion
delayed initialization
> treating newly detected features separately from the main map to reduce depth uncertainty before insertion into the full filter (with a standard XYZ representation)
- Features that retain low parallax over many frames (those very far from the camera or close to the motion epipole) are usually rejected completely because they never pass the test for inclusion
> (in 2-D and simulation) Initialization is delayed until the measurement equation is approximately Gaussian and the point can be safely triangulated.
> 3-D monocular vision with inertial sensing + auxiliary particle filter (in high frame rate sequence)
undelayed initialization
> While features with highly uncertain depths provide little information on camera translation, they are extremely useful as bearing references for orientation estimation.
: a multiple hypothesis scheme, initializing features at various depths and pruning those not reobserved in subsequent images
> Gaussian sum filter approximated by a federated information sharing method to keep the computational overhead low
-> to spread the Gaussian depth hypotheses along the ray according to inverse depth
Davision's particle method --> (Sola et al.) Gaussian sum filter --> (Civera at al.) new inverse depth scheme
A Gaussian sum is more efficient representation than particles (efficient enough that the separate Gaussians can call be put into the main state vector), but not as efficient as the single Gaussian representation that the inverse depth parametrization aalows.
B. Points at Infinity
efficient undelayed initialization + features at all depths (in outdoor scenes)
Point at infinity: a feature that exhibits no parallax during camera motion due to its extreme depth
-> not used for estimating camera translationm but for estimating rotation
The homogeneous coordinate systems of visual projective geometry used normally in SFM allow explicit representation of points at infinity(, and they have proven to play an important role during offline structure and motion estimation).
sequential SLAM system
Montiel and Davison: In special case where all features are known to be infinite -- in very-large-scale outdoor scenes or when the camera rotates on a tripod -- SLAM in pure angular coordinates turns the camera into a real-time visual compass.
Our probabilistic SLAM algorithm must be able to represent the uncertainty in depth of seemingly infinite features. Observing no parallax for a feature after 10 units of camera translation does tell us something about its depth -- it gives a reliable lower bound, which depends on the amount of motion made by the camera (if the feature had been closer than this, we would have observed parallax).
The explicit consideration of uncertainty in the locations of points has not been previously required in offline computer vision algorithms, but is very important in a more difficult online case.
C. Inverse Depth Representation
There is a unified and straightforward parametrization for feature locations that can handle both initialization and standard tracking of both close and very distant features within the standard EKF framework.
standard tracking
An explicit parametrization of the inverse depth of a feature along a semiinfinite ray from the position from which it was first viewed allows a Gaussian distribution to cover uncertainty in depth that spans a depth range from nearby to infinity, and permits seamless crossing over to finite depth estimates of features that have been apparently infinite for long periods of time.
linearity index + inverse depth parametrization
The projective nature of a camera means that the image measurement process is nearly linear in this inverse depth coordinate.
Inverse depth appears in the relation between image disparity and point depth in a stereo vision; it is interpreted as the parallax with respect to the plane at infinity. (Hartley and Zisserman)
Inverse depth is used to relate the motion field induced by scene points with the camera velocity in optical flow analysis.
modified polar coordinates
target motion analysis = TMA
EKF-based sequential depth estimation from camera-known motion
UndelayedBOSLAM.pdf
VideoSource_OSX.cc
