2D-3D matching is an exciting yet challenging field, which aims to build the connection between 2D image pixels to 3D point clouds. It is the foundation for camera localization, image to pint cloud fusion and virtual reality.
This repository collects:
- Books & Academic Papers
- Online Courses and Videos
- Outlier Datasets
- Open-source and Commercial Libraries/Toolkits
- Key Conferences & Journals
More items will be added to the repository. Please feel free to suggest other key resources by opening an issue report, submitting a pull request, or dropping me an email @Huai Yu (huaiy@andrew.cmu.edu). Enjoy reading!
- 1. Books & Tutorials
- 2. Courses/Seminars/Videos
- 3. Toolbox & Datasets
- 4. Papers
- 5. Key Conferences/Workshops/Journals
Multiple view geometry in computer vision by Richard Hartley and Andrew Zisserman, 2004: Mathematic and geometric basis for 2D-2D and 2D-3D registration. A must-read for people in the field of registration. [E-book]
Computer Vision: A Modern Approach by Charu Aggarwal and Saket Sathe: Great intro book for ensemble learning in outlier analysis.
Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Engineering By Jean Gallier and Jocelyn Quaintance. The latest book from upenn about the algebra and optimization theory.
视觉SLAM十四讲 高翔 and 张涛, et. al. 视觉配准方向较易懂的入门教材。通俗讲述视觉匹配的物理模型， 数学几何基础，优化过程等。 新手必读。[github] [Videos]
Three-Dimensional Computer vision-A Geometric Viewpoint Classical 3D computer vision textbook.
An invitation to 3D vision a self-contained introduction to the geometry of three-dimensional (3- D) vision.
| Tutorial Title | Venue | Year | Materials |
|---|---|---|---|
| Long-Term Visual Localization under Changing | CVPR | 2019 | [URL] |
| Image Matching: Local Features & Beyond | CVPR | 2019 | [URL ] |
16-822: Geometry-based Methods in Vision: [Website]
Talk: 2017以来的2D to 3D， 吴毅红， VALSE 2018: [Website]
[C++ and Python] OpenCV: OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
[C++] PCL: Point Cloud Library. The Point Cloud Library (PCL) is a standalone, large scale, open project for 2D/3D image and point cloud processing.
[C++] Ceres Solver: Ceres Solver is an open source C++ library for modeling and solving large, complicated optimization problems. It can be used to solve Non-linear Least Squares problems with bounds constraints and general unconstrained optimization problems.
[C++] Open3D: Open3D is an open-source library that supports rapid development of software that deals with 3D data. The Open3D frontend exposes a set of carefully selected data structures and algorithms in both C++ and Python. The backend is highly optimized and is set up for parallelization.
Indoor LiDAR-RGBD Scan Dataset: http://redwood-data.org/indoor_lidar_rgbd/index.html
ETH3D SLAM & Stereo Benchmarks: https://www.eth3d.net/
EuRoC MAV Dataset: https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets
ViViD : Vision for Visibility Dataset: https://sites.google.com/view/dgbicra2019-vivid>
Apolloscape: Scene Parsing: http://apolloscape.auto/scene.html
KITTI Visual Odometry dataset: http://www.cvlibs.net/datasets/kitti/eval_odometry.php
NCLT Dataset: http://robots.engin.umich.edu/nclt/
Oxford Robotcar Dataset: https://robotcar-dataset.robots.ox.ac.uk/
| Paper Title | Venue | Year | Ref | Materials |
|---|---|---|---|---|
| 自定位技术在自动驾驶中的应用 | 中国计算机学会通讯 | 2018 | [1] | [Info] |
| A survey on visual-based localization: On the benefit of heterogeneous | PR | 2018 | [2] | [PDF] |
| Local and Global Methods for Registering 2D Image Sets and 3D Point Clouds | Preprint | 2015 | [3] | [PDF] |
| Paper Title | Venue | Year | Ref | Materials |
|---|---|---|---|---|
| Automatic 3D to 2D registration for the photorealistic rendering of urban scenes | CVPR | 2005 | [4] | [PDF] |
| A systematic approach for 2D-image to 3D-range registration in urban environments | ICCV | 2008 | [5] | [PDF] |
| Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes | IJCV | 2008 | [6] | [PDF] |
| Paper Title | Venue | Year | Ref | Materials |
|---|---|---|---|---|
| Softposit: Simultaneous pose and correspondence determination | ECCV | 2001 | [7] | [PDF] |
| Globally optimal 2D-3D registration from points or lines without correspondences | ICCV | 2015 | [8] | [PDF] |
| A family of globally optimal branch-and-bound algorithms for 2D–3D correspondence-free registration | PR | 2019 | [9] | [PDF] |
| Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation | TPAMI | 2018 | [10] | [PDF] |
| The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation | CVPR | 2019 | [11] | [PDF] |
| Paper Title | Venue | Year | Ref | Materials |
|---|---|---|---|---|
| Automatic registration of UAV-borne sequent images and LiDAR data | ISPRS | 2015 | [12] | [PDF] |
| Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features | ISPRS | 2018 | [13] | [PDF] |
| Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes | Sensors | 2019 | [14] | [PDF] |
| An Accurate TLS and UAV Image Point Clouds Registration Method for Deformation Detection of Chaotic Hillside Areas | RS | 2019 | [15] | [PDF] |
IEEE International Conference on Computer Vision and Pattern Recognition
IEEE International Conference on Computer Vision
European Conference on Computer Vision
IEEE International Conference on Robotics and Automation
International Conference on 3D Vision
Winter Conference on Applications of Computer Vision
IEEE Transactions on Pattern Analysis and Machine Intelligence
International Journal of Computer Vision
ISPRS Journal of Photogrammetry and Remote Sensing
| [1] | 李宏东， 周定富， 刘流. 自定位技术在自动驾驶中的应用. 中国计算机学会通讯，14卷 8期， 2018 |
| [2] | Piasco N, Sidibé D, Demonceaux C, et al. A survey on visual-based localization: On the benefit of heterogeneous data. Pattern Recognition, 2018, 74: 90-109. |
| [3] | Paudel D P. Local and global methods for registering 2D image sets and 3D point clouds. Dijon, 2015. |
| [4] | Liu L, Stamos I. Automatic 3D to 2D registration for the photorealistic rendering of urban scene. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). IEEE, 2005, 2: 137-143. |
| [5] | Liu L, Stamos I. A systematic approach for 2D-image to 3D-range registration in urban environments. IEEE 11th International Conference on Computer Vision. IEEE, 2007: 1-8. |
| [6] | Stamos I, Liu L, Chen C, et al. Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes. International Journal of Computer Vision, 2008, 78(2-3): 237-260. |
| [7] | David P, Dementhon D, Duraiswami R, et al. SoftPOSIT: Simultaneous pose and correspondence determination. International Journal of Computer Vision, 2004, 59(3): 259-284. |
| [8] | Brown M, Windridge D, Guillemaut J Y. Globally optimal 2D-3D registration from points or lines without correspondences. IEEE International Conference on Computer Vision. 2015: 2111-2119. |
| [9] | Brown M, Windridge D, Guillemaut J Y. A family of globally optimal branch-and-bound algorithms for 2D–3D correspondence-free registration. Pattern Recognition, 2019, 93: 36-54. |
| [10] | Campbell D J, Petersson L, Kneip L, et al. Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation. IEEE transactions on pattern analysis and machine intelligence, 2018. |
| [11] | Campbell D, Petersson L, Kneip L, et al. The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation. IEEE Conference on Computer Vision and Pattern Recognition. 2019: 11796-11806. |
| [12] | Yang B, Chen C. Automatic registration of UAV-borne sequent images and LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101: 262-274. |
| [13] | Li J, Yang B, Chen C, et al. Automatic registration of panoramic image sequence and mobile laser scanning data using semantic features. ISPRS journal of photogrammetry and remote sensing, 2018, 136: 41-57. |
| [14] | Peng S, Ma H, Zhang L. Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations. Sensors, 2019, 19(5): 1086. |
| [15] | Zang Y, Yang B, Li J, et al. An Accurate TLS and UAV Image Point Clouds Registration Method for Deformation Detection of Chaotic Hillside Areas. Remote Sensing, 2019, 11(6): 647. |