calibration.md

0-calibration-setup

Prepares the environment, ensures that all necessary arguments are set. Outputs its findings to the logger. python3 0-calibration-setup.py

1-Intrinsics

This step of the pipeline calculates the intrinsics (camera matrix, distortion coefficients, etc.) for each camera, based on the calibration footage placed in 'data/raw_data/calibration'.

This step requires calibration footage, this is footage of a calibration pattern (checkerboard). You can generate a calibration pattern using the following website: https://markhedleyjones.com/projects/calibration-checkerboard-collection Verify calibration pattern dimensions and amend them in /code/utils/arguments.py if necessary.

To execute run: python3 1-intrinsics.py

Verification images are placed in 'data/0-calibration/visualisation/undistorted-calibration'. Intrinsics are saved to 'data/0-calibration/calibs'. Below are some example outputs from this step of the pipeline. If calibration is unsuccessful, the images will be distorted, especially at the edges. If this is the case, please ensure that the keypoint coverage is good and that the calibration footage is of sufficient quality.

Example keypoint coverage:

Good keypoint coverage is shown in the example below. Ensure that you have good coverage, with keypoints covering the entire image.

Example undistorted image:

The undistorted image should be free from distortion. Example undistorted image generated by 1-intrinsics.py, the lines are straight throughout the image suggesting that lens distortion has successfully been accounted for.

Example monotonicity plot:

The monotonicity plot should show green lines, especially towards the centre of the image. Shown in this figure is a circle of non-monotonic distortion at the peripherie (shown in red).

2-Extrinsics

Undistorts environment footage and places images in 'data/0-calibration/opensfm/images' for the opensfm pipeline. Runs the OpenSfM pipeline to calculate the extrinsics. To execute run: python3 2-extrinsics.py

Places calculated extrinsics in 'data/0-calibration/calibs'. Reconstruction is placed in 'data/0-calibration/visualisation/reconstruction.json'.

3-Reconstruction Viewer

Launches the opensfm reconstruction viewer with the generated dataset. This should be used for verification purposes. Some keypoints may exist outside of the scene. bash 3-reconstruction_viewer.sh

An example reconstruction can be found at 'images/reconstruction.json'.

A visualisation in the reconstruction viewer should look like the following:

To double-check the reconstruction it is recommended to use MeshLab to visualise 'undistorted/depthmaps/merged.ply'.

4-Annotation

Launches an annotation tool to calibrate the ground plane. A histogram showing the distribution of points by their z-coordinate is plotted. The lowest peak is expected to correspond with the ground plane. Updates extrinsics.json based on the annotation. bash 4-annotation_tool.sh

The notebook takes you through the steps to calibrate the ground plane. It then gives an example of the ground plane projection for verification.

ROI Selection:

Choose a region of interest (ROI) that contains the ground plane. Try to avoid selecting points that have large obstacles.

An example ROI is shown below:

Setting Scale:

Set the scale of the scene. Choose two points on the ground plane and supply the real world distance between them (in cm).

An example scale selection is shown below:

Aligning the Ground Plane:

The ground plane is automatically aligned. Verification of the alignment is shown below:

First we ensure that the ground plane lies in the peak of the z-distribution for the ground plane points. The ground plane z-value is shown in red.

Second we display the reconstruction with the ground plane points in black and a bounding box around the ROI on the ground plane shown in blue. Camera positions are shown in red and the normal to the ground plane is shown in red. The normal should be pointing upwards. Ensure that the ROI and the ground plane meet your expectations and that the scale is correct.

Third we display a histogram of all z-values. The ground plane z-value is shown in red. Ensure that the ground plane z-value is the lowest peak. In our tests the peaks were fairly sharp and the ground plane z-value was the lowest peak.

Example ground plane projection:

Finally, verify that the calibration and ground plane projection are correct. The ground plane projection should show the ground plane for all the cameras. Points on the ground should be aligned between views and it should be centered on the region of interest selected earlier.

Status

At the end of this step of the pipeline you should have intrinsics and extrinsics that give you a working ground plane projection.