Merge pull request #12772 from xoox:calib-release-object
More accurate pinhole camera calibration with imperfect planar target (#12772) 43 commits: * Add derivatives with respect to object points Add an output parameter to calculate derivatives of image points with respect to 3D coordinates of object points. The output jacobian matrix is a 2Nx3N matrix where N is the number of points. This commit introduces incompatibility to old function signature. * Set zero for dpdo matrix before using dpdo is a sparse matrix with only non-zero value close to major diagonal. Set it to zero because only elements near major diagonal are computed. * Add jacobian columns to projectPoints() The output jacobian matrix of derivatives with respect to coordinates of 3D object points are added. This might break callers who assume the columns of jacobian matrix. * Adapt test code to updated project functions The test cases for projectPoints() and cvProjectPoints2() are updated to fit new function signatures. * Add accuracy test code for dpdo * Add badarg test for dpdo * Add new enum item for new calibration method CALIB_RELEASE_OBJECT is used to whether to release 3D coordinates of object points. The method was proposed in: K. H. Strobl and G. Hirzinger. "More Accurate Pinhole Camera Calibration with Imperfect Planar Target". In Proceedings of the IEEE International Conference on Computer Vision (ICCV 2011), 1st IEEE Workshop on Challenges and Opportunities in Robot Perception, Barcelona, Spain, pp. 1068-1075, November 2011. * Add releasing object method into internal function It's a simple extension of the standard calibration scheme. We choose to fix the first and last object point and a user-selected fixed point. * Add interfaces for extended calibration method * Refine document for calibrateCamera() When releasing object points, only the z coordinates of the objectPoints[0].back is fixed. * Add link to strobl2011iccv paper * Improve documentation for calibrateCamera() * Add implementations of wrapping calibrateCamera() * Add checking for params of new calibration method If input parameters are not qualified, then fall back to standard calibration method. * Add camera calibration method of releasing object The current implementation is equal to or better than https://github.com/xoox/calibrel * Update doc for CALIB_RELEASE_OBJECT CALIB_USE_QR or CALIB_USE_LU could be used for faster calibration with potentially less precise and less stable in some rare cases. * Add RELEASE_OBJECT calibration to tutorial code To select the calibration method of releasing object points, a command line parameter `-d=<number>` should be provided. * Update tutorial doc for camera_calibration If the method of releasing object points is merged into OpenCV. It will be expected to be firstly released in 4.1, I think. * Reduce epsilon for cornerSubPix() Epsilon of 0.1 is a bigger one. Preciser corner positions are required with calibration method of releasing object. * Refine camera calibration tutorial The hypothesis coordinates are used to indicate which distance must be measured between two specified object points. * Update sample calibration code method selection Similar to camera_calibration tutorial application, a command line argument `-dt=<number>` is used to select the calibration method. * Add guard to flags of cvCalibrateCamera2() cvCalibrateCamera2() doesn't accept CALIB_RELEASE_OBJECT unless overload interface is added in the future. * Simplify fallback when iFixedPoint is out of range * Refactor projectPoints() to keep compatibilities * Fix arg string "Bad rvecs header" * Read calibration flags from test data files Instead of being hard coded into source file, the calibration flags will be read from test data files. opencv_extra/testdata/cv/cameracalibration/calib?.dat must be sync with the test code. * Add new C interface of cvCalibrateCamera4() With this new added C interface, the extended calibration method with CALIB_RELEASE_OBJECT can be called by C API. * Add regression test of extended calibration method It has been tested with new added test data in xoox:calib-release-object branch of opencv_extra. * Fix assertion in test_cameracalibration.cpp The total number of refined 3D object coordinates is checked. * Add checker for iFixedPoint in cvCalibrateCamera4 If iFixedPoint is out of rational range, fall back to standard method. * Fix documentation for overloaded calibrateCamera() * Remove calibration flag of CALIB_RELEASE_OBJECT The method selection is based on the range of the index of fixed point. For minus values, standard calibration method will be chosen. Values in a rational range will make the object-releasing calibration method selected. * Use new interfaces instead of function overload Existing interfaces are preserved and new interfaces are added. Since most part of the code base are shared, calibrateCamera() is now a wrapper function of calibrateCameraRO(). * Fix exported name of calibrateCameraRO() * Update documentation for calibrateCameraRO() The circumstances where this method is mostly helpful are described. * Add note on the rigidity of the calibration target * Update documentation for calibrateCameraRO() It is clarified that iFixedPoint is used as a switch to select calibration method. If input data are not qualified, exceptions will be thrown instead of fallback scheme. * Clarify iFixedPoint as switch and remove fallback iFixedPoint is now used as a switch for calibration method selection. No fallback scheme is utilized anymore. If the input data are not qualified, exceptions will be thrown. * Add badarg test for object-releasing method * Fix document format of sample list List items of same level should be indented the same way. Otherwise they will be formatted as nested lists by Doxygen. * Add brief intro for objectPoints and imagePoints * Sync tutorial to sample calibration code * Update tutorial compatibility version to 4.0
This commit is contained in:
Wenfeng CAI
committed by
Alexander Alekhin
Alexander Alekhin
parent
ca3848d773
commit
31be03a805
@@ -15,13 +15,6 @@
|
||||
using namespace cv;
|
||||
using namespace std;
|
||||
|
||||
static void help()
|
||||
{
|
||||
cout << "This is a camera calibration sample." << endl
|
||||
<< "Usage: camera_calibration [configuration_file -- default ./default.xml]" << endl
|
||||
<< "Near the sample file you'll find the configuration file, which has detailed help of "
|
||||
"how to edit it. It may be any OpenCV supported file format XML/YAML." << endl;
|
||||
}
|
||||
class Settings
|
||||
{
|
||||
public:
|
||||
@@ -43,6 +36,7 @@ public:
|
||||
|
||||
<< "Write_DetectedFeaturePoints" << writePoints
|
||||
<< "Write_extrinsicParameters" << writeExtrinsics
|
||||
<< "Write_gridPoints" << writeGrid
|
||||
<< "Write_outputFileName" << outputFileName
|
||||
|
||||
<< "Show_UndistortedImage" << showUndistorsed
|
||||
@@ -62,6 +56,7 @@ public:
|
||||
node["Calibrate_FixAspectRatio"] >> aspectRatio;
|
||||
node["Write_DetectedFeaturePoints"] >> writePoints;
|
||||
node["Write_extrinsicParameters"] >> writeExtrinsics;
|
||||
node["Write_gridPoints"] >> writeGrid;
|
||||
node["Write_outputFileName"] >> outputFileName;
|
||||
node["Calibrate_AssumeZeroTangentialDistortion"] >> calibZeroTangentDist;
|
||||
node["Calibrate_FixPrincipalPointAtTheCenter"] >> calibFixPrincipalPoint;
|
||||
@@ -210,6 +205,7 @@ public:
|
||||
int delay; // In case of a video input
|
||||
bool writePoints; // Write detected feature points
|
||||
bool writeExtrinsics; // Write extrinsic parameters
|
||||
bool writeGrid; // Write refined 3D target grid points
|
||||
bool calibZeroTangentDist; // Assume zero tangential distortion
|
||||
bool calibFixPrincipalPoint; // Fix the principal point at the center
|
||||
bool flipVertical; // Flip the captured images around the horizontal axis
|
||||
@@ -248,19 +244,39 @@ static inline void read(const FileNode& node, Settings& x, const Settings& defau
|
||||
enum { DETECTION = 0, CAPTURING = 1, CALIBRATED = 2 };
|
||||
|
||||
bool runCalibrationAndSave(Settings& s, Size imageSize, Mat& cameraMatrix, Mat& distCoeffs,
|
||||
vector<vector<Point2f> > imagePoints );
|
||||
vector<vector<Point2f> > imagePoints, float grid_width, bool release_object);
|
||||
|
||||
int main(int argc, char* argv[])
|
||||
{
|
||||
help();
|
||||
const String keys
|
||||
= "{help h usage ? | | print this message }"
|
||||
"{@settings |default.xml| input setting file }"
|
||||
"{d | | actual distance between top-left and top-right corners of "
|
||||
"the calibration grid }"
|
||||
"{winSize | 11 | Half of search window for cornerSubPix }";
|
||||
CommandLineParser parser(argc, argv, keys);
|
||||
parser.about("This is a camera calibration sample.\n"
|
||||
"Usage: camera_calibration [configuration_file -- default ./default.xml]\n"
|
||||
"Near the sample file you'll find the configuration file, which has detailed help of "
|
||||
"how to edit it. It may be any OpenCV supported file format XML/YAML.");
|
||||
if (!parser.check()) {
|
||||
parser.printErrors();
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (parser.has("help")) {
|
||||
parser.printMessage();
|
||||
return 0;
|
||||
}
|
||||
|
||||
//! [file_read]
|
||||
Settings s;
|
||||
const string inputSettingsFile = argc > 1 ? argv[1] : "default.xml";
|
||||
const string inputSettingsFile = parser.get<string>(0);
|
||||
FileStorage fs(inputSettingsFile, FileStorage::READ); // Read the settings
|
||||
if (!fs.isOpened())
|
||||
{
|
||||
cout << "Could not open the configuration file: \"" << inputSettingsFile << "\"" << endl;
|
||||
parser.printMessage();
|
||||
return -1;
|
||||
}
|
||||
fs["Settings"] >> s;
|
||||
@@ -276,6 +292,15 @@ int main(int argc, char* argv[])
|
||||
return -1;
|
||||
}
|
||||
|
||||
int winSize = parser.get<int>("winSize");
|
||||
|
||||
float grid_width = s.squareSize * (s.boardSize.width - 1);
|
||||
bool release_object = false;
|
||||
if (parser.has("d")) {
|
||||
grid_width = parser.get<float>("d");
|
||||
release_object = true;
|
||||
}
|
||||
|
||||
vector<vector<Point2f> > imagePoints;
|
||||
Mat cameraMatrix, distCoeffs;
|
||||
Size imageSize;
|
||||
@@ -295,7 +320,8 @@ int main(int argc, char* argv[])
|
||||
//----- If no more image, or got enough, then stop calibration and show result -------------
|
||||
if( mode == CAPTURING && imagePoints.size() >= (size_t)s.nrFrames )
|
||||
{
|
||||
if( runCalibrationAndSave(s, imageSize, cameraMatrix, distCoeffs, imagePoints))
|
||||
if(runCalibrationAndSave(s, imageSize, cameraMatrix, distCoeffs, imagePoints, grid_width,
|
||||
release_object))
|
||||
mode = CALIBRATED;
|
||||
else
|
||||
mode = DETECTION;
|
||||
@@ -304,7 +330,8 @@ int main(int argc, char* argv[])
|
||||
{
|
||||
// if calibration threshold was not reached yet, calibrate now
|
||||
if( mode != CALIBRATED && !imagePoints.empty() )
|
||||
runCalibrationAndSave(s, imageSize, cameraMatrix, distCoeffs, imagePoints);
|
||||
runCalibrationAndSave(s, imageSize, cameraMatrix, distCoeffs, imagePoints, grid_width,
|
||||
release_object);
|
||||
break;
|
||||
}
|
||||
//! [get_input]
|
||||
@@ -348,8 +375,8 @@ int main(int argc, char* argv[])
|
||||
{
|
||||
Mat viewGray;
|
||||
cvtColor(view, viewGray, COLOR_BGR2GRAY);
|
||||
cornerSubPix( viewGray, pointBuf, Size(11,11),
|
||||
Size(-1,-1), TermCriteria( TermCriteria::EPS+TermCriteria::COUNT, 30, 0.1 ));
|
||||
cornerSubPix( viewGray, pointBuf, Size(winSize,winSize),
|
||||
Size(-1,-1), TermCriteria( TermCriteria::EPS+TermCriteria::COUNT, 30, 0.0001 ));
|
||||
}
|
||||
|
||||
if( mode == CAPTURING && // For camera only take new samples after delay time
|
||||
@@ -515,7 +542,8 @@ static void calcBoardCornerPositions(Size boardSize, float squareSize, vector<Po
|
||||
//! [board_corners]
|
||||
static bool runCalibration( Settings& s, Size& imageSize, Mat& cameraMatrix, Mat& distCoeffs,
|
||||
vector<vector<Point2f> > imagePoints, vector<Mat>& rvecs, vector<Mat>& tvecs,
|
||||
vector<float>& reprojErrs, double& totalAvgErr)
|
||||
vector<float>& reprojErrs, double& totalAvgErr, vector<Point3f>& newObjPoints,
|
||||
float grid_width, bool release_object)
|
||||
{
|
||||
//! [fixed_aspect]
|
||||
cameraMatrix = Mat::eye(3, 3, CV_64F);
|
||||
@@ -530,6 +558,8 @@ static bool runCalibration( Settings& s, Size& imageSize, Mat& cameraMatrix, Mat
|
||||
|
||||
vector<vector<Point3f> > objectPoints(1);
|
||||
calcBoardCornerPositions(s.boardSize, s.squareSize, objectPoints[0], s.calibrationPattern);
|
||||
objectPoints[0][s.boardSize.width - 1].x = objectPoints[0][0].x + grid_width;
|
||||
newObjPoints = objectPoints[0];
|
||||
|
||||
objectPoints.resize(imagePoints.size(),objectPoints[0]);
|
||||
|
||||
@@ -548,14 +578,28 @@ static bool runCalibration( Settings& s, Size& imageSize, Mat& cameraMatrix, Mat
|
||||
tvecs.push_back(_tvecs.row(i));
|
||||
}
|
||||
} else {
|
||||
rms = calibrateCamera(objectPoints, imagePoints, imageSize, cameraMatrix, distCoeffs, rvecs, tvecs,
|
||||
s.flag);
|
||||
int iFixedPoint = -1;
|
||||
if (release_object)
|
||||
iFixedPoint = s.boardSize.width - 1;
|
||||
rms = calibrateCameraRO(objectPoints, imagePoints, imageSize, iFixedPoint,
|
||||
cameraMatrix, distCoeffs, rvecs, tvecs, newObjPoints,
|
||||
s.flag | CALIB_USE_LU);
|
||||
}
|
||||
|
||||
if (release_object) {
|
||||
cout << "New board corners: " << endl;
|
||||
cout << newObjPoints[0] << endl;
|
||||
cout << newObjPoints[s.boardSize.width - 1] << endl;
|
||||
cout << newObjPoints[s.boardSize.width * (s.boardSize.height - 1)] << endl;
|
||||
cout << newObjPoints.back() << endl;
|
||||
}
|
||||
|
||||
cout << "Re-projection error reported by calibrateCamera: "<< rms << endl;
|
||||
|
||||
bool ok = checkRange(cameraMatrix) && checkRange(distCoeffs);
|
||||
|
||||
objectPoints.clear();
|
||||
objectPoints.resize(imagePoints.size(), newObjPoints);
|
||||
totalAvgErr = computeReprojectionErrors(objectPoints, imagePoints, rvecs, tvecs, cameraMatrix,
|
||||
distCoeffs, reprojErrs, s.useFisheye);
|
||||
|
||||
@@ -566,7 +610,7 @@ static bool runCalibration( Settings& s, Size& imageSize, Mat& cameraMatrix, Mat
|
||||
static void saveCameraParams( Settings& s, Size& imageSize, Mat& cameraMatrix, Mat& distCoeffs,
|
||||
const vector<Mat>& rvecs, const vector<Mat>& tvecs,
|
||||
const vector<float>& reprojErrs, const vector<vector<Point2f> >& imagePoints,
|
||||
double totalAvgErr )
|
||||
double totalAvgErr, const vector<Point3f>& newObjPoints )
|
||||
{
|
||||
FileStorage fs( s.outputFileName, FileStorage::WRITE );
|
||||
|
||||
@@ -673,24 +717,30 @@ static void saveCameraParams( Settings& s, Size& imageSize, Mat& cameraMatrix, M
|
||||
}
|
||||
fs << "image_points" << imagePtMat;
|
||||
}
|
||||
|
||||
if( s.writeGrid && !newObjPoints.empty() )
|
||||
{
|
||||
fs << "grid_points" << newObjPoints;
|
||||
}
|
||||
}
|
||||
|
||||
//! [run_and_save]
|
||||
bool runCalibrationAndSave(Settings& s, Size imageSize, Mat& cameraMatrix, Mat& distCoeffs,
|
||||
vector<vector<Point2f> > imagePoints)
|
||||
vector<vector<Point2f> > imagePoints, float grid_width, bool release_object)
|
||||
{
|
||||
vector<Mat> rvecs, tvecs;
|
||||
vector<float> reprojErrs;
|
||||
double totalAvgErr = 0;
|
||||
vector<Point3f> newObjPoints;
|
||||
|
||||
bool ok = runCalibration(s, imageSize, cameraMatrix, distCoeffs, imagePoints, rvecs, tvecs, reprojErrs,
|
||||
totalAvgErr);
|
||||
totalAvgErr, newObjPoints, grid_width, release_object);
|
||||
cout << (ok ? "Calibration succeeded" : "Calibration failed")
|
||||
<< ". avg re projection error = " << totalAvgErr << endl;
|
||||
|
||||
if (ok)
|
||||
saveCameraParams(s, imageSize, cameraMatrix, distCoeffs, rvecs, tvecs, reprojErrs, imagePoints,
|
||||
totalAvgErr);
|
||||
totalAvgErr, newObjPoints);
|
||||
return ok;
|
||||
}
|
||||
//! [run_and_save]
|
||||
|
||||
@@ -39,6 +39,8 @@
|
||||
<Write_DetectedFeaturePoints>1</Write_DetectedFeaturePoints>
|
||||
<!-- If true (non-zero) we write to the output file the extrinsic camera parameters.-->
|
||||
<Write_extrinsicParameters>1</Write_extrinsicParameters>
|
||||
<!-- If true (non-zero) we write to the output file the refined 3D target grid points.-->
|
||||
<Write_gridPoints>1</Write_gridPoints>
|
||||
<!-- If true (non-zero) we show after calibration the undistorted images.-->
|
||||
<Show_UndistortedImage>1</Show_UndistortedImage>
|
||||
<!-- If true (non-zero) will be used fisheye camera model.-->
|
||||
|
||||
Reference in New Issue
Block a user