Skip to content

Awesome-Image-Registration-Organization/2D-3D-matching

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 

Repository files navigation

2D-3D matching is an exciting yet challenging field, which aims to build the connection between 2D image pixels to 3D point clouds. It is the foundation for camera localization, image to pint cloud fusion and virtual reality.

This repository collects:

  1. Books & Academic Papers
  2. Online Courses and Videos
  3. Outlier Datasets
  4. Open-source and Commercial Libraries/Toolkits
  5. Key Conferences & Journals

More items will be added to the repository. Please feel free to suggest other key resources by opening an issue report, submitting a pull request, or dropping me an email @Huai Yu (huaiy@andrew.cmu.edu). Enjoy reading!


Table of Contents


1. Books & Tutorials

1.1. Books

Multiple view geometry in computer vision by Richard Hartley and Andrew Zisserman, 2004: Mathematic and geometric basis for 2D-2D and 2D-3D registration. A must-read for people in the field of registration. [E-book]

Computer Vision: A Modern Approach by Charu Aggarwal and Saket Sathe: Great intro book for ensemble learning in outlier analysis.

Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Engineering By Jean Gallier and Jocelyn Quaintance. The latest book from upenn about the algebra and optimization theory.

视觉SLAM十四讲 高翔 and 张涛, et. al. 视觉配准方向较易懂的入门教材。通俗讲述视觉匹配的物理模型, 数学几何基础,优化过程等。 新手必读。[github] [Videos]

Three-Dimensional Computer vision-A Geometric Viewpoint Classical 3D computer vision textbook.

An invitation to 3D vision a self-contained introduction to the geometry of three-dimensional (3- D) vision.

1.2. Tutorials

Tutorial Title Venue Year Materials
Long-Term Visual Localization under Changing CVPR 2019 [URL]
Image Matching: Local Features & Beyond CVPR 2019 [URL ]

2. Courses/Seminars/Videos

16-822: Geometry-based Methods in Vision: [Website]

Talk: 2017以来的2D to 3D, 吴毅红, VALSE 2018: [Website]


3. Toolbox & Datasets

3.1. General toolbox

[C++ and Python] OpenCV: OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

[C++] PCL: Point Cloud Library. The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing.

[C++] Ceres Solver: Ceres Solver is an open source C++ library for modeling and solving large, complicated optimization problems. It can be used to solve Non-linear Least Squares problems with bounds constraints and general unconstrained optimization problems.

[C++] Open3D: Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization.

3.2. Datasets

Indoor LiDAR-RGBD Scan Dataset: http://redwood-data.org/indoor_lidar_rgbd/index.html

ETH3D SLAM & Stereo Benchmarks: https://www.eth3d.net/

EuRoC MAV Dataset: https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets

ViViD : Vision for Visibility Dataset: https://sites.google.com/view/dgbicra2019-vivid>

Apolloscape: Scene Parsing: http://apolloscape.auto/scene.html

KITTI Visual Odometry dataset: http://www.cvlibs.net/datasets/kitti/eval_odometry.php

NCLT Dataset: http://robots.engin.umich.edu/nclt/

Oxford Robotcar Dataset: https://robotcar-dataset.robots.ox.ac.uk/


4. Papers

4.1. Overview & Survey Papers

Paper Title Venue Year Ref Materials
自定位技术在自动驾驶中的应用 中国计算机学会通讯 2018 [1] [Info]
A survey on visual-based localization: On the benefit of heterogeneous PR 2018 [2] [PDF]
Local and Global Methods for Registering 2D Image Sets and 3D Point Clouds Preprint 2015 [3] [PDF]

4.2. Geometric 2D-3D features matching

Paper Title Venue Year Ref Materials
Automatic 3D to 2D registration for the photorealistic rendering of urban scenes CVPR 2005 [4] [PDF]
A systematic approach for 2D-image to 3D-range registration in urban environments ICCV 2008 [5] [PDF]
Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes IJCV 2008 [6] [PDF]

4.3. Simultaneously camera pose estimation and 2D-3D correspondence estimation

Paper Title Venue Year Ref Materials
Softposit: Simultaneous pose and correspondence determination ECCV 2001 [7] [PDF]
Globally optimal 2D-3D registration from points or lines without correspondences ICCV 2015 [8] [PDF]
A family of globally optimal branch-and-bound algorithms for 2D–3D correspondence-free registration PR 2019 [9] [PDF]
Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation TPAMI 2018 [10] [PDF]
The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation CVPR 2019 [11] [PDF]

4.4. Airborne (UAV) image to terriseal LiDAr scanner point cloud matching

Paper Title Venue Year Ref Materials
Automatic registration of UAV-borne sequent images and LiDAR data ISPRS 2015 [12] [PDF]
Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features ISPRS 2018 [13] [PDF]
Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Sensors 2019 [14] [PDF]
An Accurate TLS and UAV Image Point Clouds Registration Method for Deformation Detection of Chaotic Hillside Areas RS 2019 [15] [PDF]

5. Key Conferences/Workshops/Journals

5.1. Conferences & Workshops

IEEE International Conference on Computer Vision and Pattern Recognition

IEEE International Conference on Computer Vision

European Conference on Computer Vision

IEEE International Conference on Robotics and Automation

International Conference on 3D Vision

Winter Conference on Applications of Computer Vision

5.2. Journals

IEEE Transactions on Pattern Analysis and Machine Intelligence

International Journal of Computer Vision

ISPRS Journal of Photogrammetry and Remote Sensing


References

[1]李宏东, 周定富, 刘流. 自定位技术在自动驾驶中的应用. 中国计算机学会通讯,14卷 8期, 2018
[2]Piasco N, Sidibé D, Demonceaux C, et al. A survey on visual-based localization: On the benefit of heterogeneous data. Pattern Recognition, 2018, 74: 90-109.
[3]Paudel D P. Local and global methods for registering 2D image sets and 3D point clouds. Dijon, 2015.
[4]Liu L, Stamos I. Automatic 3D to 2D registration for the photorealistic rendering of urban scene. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). IEEE, 2005, 2: 137-143.
[5]Liu L, Stamos I. A systematic approach for 2D-image to 3D-range registration in urban environments. IEEE 11th International Conference on Computer Vision. IEEE, 2007: 1-8.
[6]Stamos I, Liu L, Chen C, et al. Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes. International Journal of Computer Vision, 2008, 78(2-3): 237-260.
[7]David P, Dementhon D, Duraiswami R, et al. SoftPOSIT: Simultaneous pose and correspondence determination. International Journal of Computer Vision, 2004, 59(3): 259-284.
[8]Brown M, Windridge D, Guillemaut J Y. Globally optimal 2D-3D registration from points or lines without correspondences. IEEE International Conference on Computer Vision. 2015: 2111-2119.
[9]Brown M, Windridge D, Guillemaut J Y. A family of globally optimal branch-and-bound algorithms for 2D–3D correspondence-free registration. Pattern Recognition, 2019, 93: 36-54.
[10]Campbell D J, Petersson L, Kneip L, et al. Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation. IEEE transactions on pattern analysis and machine intelligence, 2018.
[11]Campbell D, Petersson L, Kneip L, et al. The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation. IEEE Conference on Computer Vision and Pattern Recognition. 2019: 11796-11806.
[12]Yang B, Chen C. Automatic registration of UAV-borne sequent images and LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101: 262-274.
[13]Li J, Yang B, Chen C, et al. Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features. ISPRS journal of photogrammetry and remote sensing, 2018, 136: 41-57.
[14]Peng S, Ma H, Zhang L. Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations. Sensors, 2019, 19(5): 1086.
[15]Zang Y, Yang B, Li J, et al. An Accurate TLS and UAV Image Point Clouds Registration Method for Deformation Detection of Chaotic Hillside Areas. Remote Sensing, 2019, 11(6): 647.

About

Registration of 2D images with 3D point clouds

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published