3D-Reconstruction-with-Deep-Learning-Methods

The focus of this list is on open-source projects hosted on Github.

Projects released on Github

TITLE	KEYWORDS	URL	LICENSE	Awesomeness
High Quality Monocular Depth Estimation via Transfer Learning	TensorFlow, PyTorch	https://github.com/ialhashim/DenseDepth https://arxiv.org/abs/1812.11941	GPL-3.0
Multi-view stereo image-based 3D reconstruction		https://github.com/adahbingee/pais-mvs	nn
Hybrid Ensemble Approach For 3D Object Reconstruction from Multi-View Monocular RGB images		https://github.com/Ajithbalakrishnan/3D-Object-Reconstruction-from-Multi-View-Monocular-RGB-images	nn
Deep 3D Semantic Scene Extrapolation	hybrid CNN, GAN, TensorFlow	https://github.com/AliAbbasi/Deep-3D-Semantic-Scene-Extrapolation http://user.ceng.metu.edu.tr/~ys/pubs/extrap-tvcj18.pdf	nn
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans	TensorFlow	https://github.com/angeladai/ScanComplete	Apache-2.0
AtLoc: Attention Guided Camera Localization	PyTorch	https://github.com/BingCS/AtLoc https://arxiv.org/abs/1909.03557	BY-NC-SA 4.0
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation	TensorFlow, cuDNN	https://github.com/charlesq34/pointnet	MIT License
PyTorch Implementation of DeepVO	PyTorch, CNN	https://github.com/ChiWeiHsiao/DeepVO-pytorch	nn
Fully Convolutional Geometric Features: Fast and accurate 3D features for registration and correspondence.	PyTorch	https://github.com/chrischoy/FCGF	MIT License
Morphing and Sampling Network for Dense Point Cloud Completion (AAAI2020)	PyTorch	https://github.com/Colin97/MSN-Point-Cloud-Completion	Apache-2.0
Real-Time Self-Adaptive Deep Stereo	TensorFlow	https://github.com/CVLAB-Unibo/Real-time-self-adaptive-deep-stereo	Apache-2.0
Geometry meets semantics for semi-supervised monocular depth estimation - ACCV 2018	TensorFlow	https://github.com/CVLAB-Unibo/Semantic-Mono-Depth	MIT License
BlenderProc: A procedural blender pipeline to generate images for deep learning	Blender	https://github.com/DLR-RM/BlenderProc	GPL-3.0
SingleViewReconstruction: 3D Scene Reconstruction from a Single Viewport	TensorFlow	https://github.com/DLR-RM/SingleViewReconstruction	MIT License
NNCAP — Neural Network Complex Approach to Photogrammetry		https://github.com/Dok11/nn-dldm	nn
Pytorch Implementation of Deeper Depth Prediction with Fully Convolutional Residual Networks	PyTorch	https://github.com/dontLoveBugs/FCRN_pytorch	nn
Improved Adversarial Systems for 3D Object Generation and Reconstruction	GAN	https://github.com/EdwardSmith1884/3D-IWGAN	MIT License
Deep Learning for Visual-Inertial Odometry	PyTorch, CNN	https://github.com/ElliotHYLee/Deep_Visual_Inertial_Odometry	MIT License
Machine Vision	List	https://github.com/Ewenwan/MVision	nn
Mesh R-CNN, an academic publication, presented at ICCV 2019	PyTorch, R-CNN	https://github.com/facebookresearch/meshrcnn	BSD-3-Clause License
PyTorch3d is FAIR's library of reusable components for deep learning with 3D data.	PyTorch	https://github.com/facebookresearch/pytorch3d	BSD-3-Clause License
Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera	PyTorch	https://github.com/fangchangma/self-supervised-depth-completion	MIT License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image	PyTorch	https://github.com/fangchangma/sparse-to-dense	BSD License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image	PyTorch	https://github.com/fangchangma/sparse-to-dense.pytorch	nn
PackNet-SfM: 3D Packing for Self-Supervised Monocular Depth Estimation	PyTorch	https://github.com/FangGet/PackNet-SFM-PyTorch	GPL-3.0
InvSFM: Revealing Scenes by Inverting Structure from Motion Reconstructions [CVPR 2019]	TensorFlow	https://github.com/francescopittaluga/invsfm	MIT License
Deep Monocular Visual Odometry using PyTorch (Experimental)	PyTorch	https://github.com/fshamshirdar/DeepVO	nn
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation	PyTorch	https://github.com/fxia22/pointnet.pytorch	MIT License
Pix2Depth - Depth Map Estimation from Monocular Image	Keras	https://github.com/gautam678/Pix2Depth	GPL-3.0
3DRegNet: A Deep Neural Network for 3D Point Registration	TensorFlow	https://github.com/goncalo120/3DRegNet	MIT License
Neural 3D Mesh Renderer – Single-Image 3D Reconstruction using Neural Renderer		https://github.com/hiroharu-kato/mesh_reconstruction	MIT License
Real-time Scalable Dense Surfel Mapping		https://github.com/HKUST-Aerial-Robotics/DenseSurfelMapping	nn
MVDepthNet: real-time multiview depth estimation neural network	PyTorch	https://github.com/HKUST-Aerial-Robotics/MVDepthNet	nn
DeepMatchVO: Beyond Photometric Loss for Self-Supervised Ego-Motion Estimation		https://github.com/hlzz/DeepMatchVO	MIT License
MIRorR: Matchable Image Retrieval by Learning from Surface Reconstruction	TensorFlow, CNN	https://github.com/hlzz/mirror	MIT License
Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction	Caffe	https://github.com/Huangying-Zhan/Depth-VO-Feat	non-commercial
Deep Learning 3D vision papers	papers, list, CN	https://github.com/huayong/dl-vision-papers	nn
Open3D PointNet implementation with PyTorch	PyTorch, jupyter, Open3D	https://github.com/intel-isl/Open3D-PointNet	MIT License
Semantic-TSDF for Self-driving Static Scene Reconstruction	PyTorch	https://github.com/irsisyphus/semantic-tsdf	MIT License
Weakly supervised 3D Reconstruction with Adversarial Constraint		https://github.com/jgwak/McRecon	MIT License
Using Deep learning Technique for Stereo vision and 3D reconstruction	TensorFlow, CN	https://github.com/jiafeng5513/EvisionNet	nn
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video	PyTorch	https://github.com/JiawangBian/SC-SfMLearner-Release	GPL-3.0
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries (official implementation)	PyTorch	https://github.com/JunjH/Revisiting_Single_Depth_Estimation	nn
Visualization of Convolutional Neural Networks for Monocular Depth Estimation (official implementation)	CNN, PyTorch	https://github.com/JunjH/Visualizing-CNNs-for-monocular-depth-estimation	MIT License
DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks	PyTorch	https://github.com/krrish94/DeepVO	nn
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction	Tensorflow	https://github.com/laughtervv/DISN	nn
DeepTAM: Deep Tracking and Mapping		https://github.com/lmb-freiburg/deeptam	GPL-3.0
DeMoN: Depth and Motion Network	Tensorflow	https://github.com/lmb-freiburg/demon	GPL-3.0
PyTorch implementation of CloudWalk's recent work DenseBody	PyTorch	https://github.com/Lotayou/densebody_pytorch	GPL-3.0
Self-supervised learning for dense depth estimation in monocular endoscopy	Tensorflow, Torch	https://github.com/lppllppl920/EndoscopyDepthEstimation-Pytorch	non-commercial
ContextDesc: Local Descriptor Augmentation with Cross-Modality Context	Tensorflow	https://github.com/lzx551402/contextdesc	nn
GL3D (Geometric Learning with 3D Reconstruction): a large-scale database created for 3D reconstruction and geometry-related learning problems		https://github.com/lzx551402/GL3D	MIT License
Deeper Depth Prediction with Fully Convolutional Residual Networks (official implementation)	Tensorflow	https://github.com/MahmoudSelmy/DeeperDepthEstimation	nn
Fine-Tuning Vgg16 For Depth Estimation	Tensorflow	https://github.com/MahmoudSelmy/DepthEstimationVGG	nn
3D reconstruction with neural networks using Tensorflow. See link for Video		https://github.com/micmelesse/3D-reconstruction-with-Neural-Networks	nn
Learning Depth from Monocular Videos using Direct Methods	PyTorch	https://github.com/MightyChaos/LKVOLearner	BSD-3-Clause
PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place Recognition	Tensorflow	https://github.com/mikacuy/pointnetvlad	MIT License
Attempting to estimate topography of a region from image data		https://github.com/nbelakovski/topography_neural_net	nn
DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNs	Tensorflow	https://github.com/neycyanshi/DDRNet	MIT License
Monocular depth estimation from a single image	PyTorch	https://github.com/nianticlabs/monodepth2	Copyright © Niantic, Inc. 2018. Patent Pending - non-commercial use only
3D-RelNet: Joint Object and Relation Network for 3D prediction	Torch, jupyter	https://github.com/nileshkulkarni/relative3d	nn
PlaneRCNN detects and reconstructs piece-wise planar surfaces from a single RGB image	Torch, RCNN	https://github.com/NVlabs/planercnn	Copyright (c) 2018 NVIDIA Corp. All Rights Reserved. This work is licensed under the Creative Commons Attribution NonCommercial ShareAlike 4.0 License.
OctoMap - An Efficient Probabilistic 3D Mapping Framework Based on Octrees.		https://github.com/OctoMap/octomap	University of Freiburg, Copyright (C) 2009-2014, octomap: New BSD License, octovis and related libraries: GPL
Unsupervised Monocular Depth Estimation neural network MonoDepth in PyTorch (Unofficial implementation)	PyTorch	https://github.com/OniroAI/MonoDepth-PyTorch	nn
Learning to Sample: A learned sampling approach for point clouds		https://github.com/orendv/learning_to_sample	MIT License
DeepMVS: Learning Multi-View Stereopsis	CNN, PyTorch	https://github.com/phuang17/DeepMVS	BSD 2-clause
DeepV2D: Video to Depth with Differentiable Structure from Motion	Tensorflow	https://github.com/princeton-vl/DeepV2D	nn
High Quality Monocular Depth Estimation via Transfer Learning	Tensorflow	https://github.com/priya-dwivedi/Deep-Learning/tree/master/depth_estimation	nn (GPL-3.0 ?)
Deep Single-View 3D Object Reconstruction with Visual Hull Embedding	CNN, Tensorflow	https://github.com/qweas120/PSVH-3d-reconstruction	MIT License
ScanNet is an RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses,...		https://github.com/ScanNet/ScanNet	Can be used with the restriction to give credit and include original Copyright
Visual inspection of bridges is customarily used to identify and evaluate faults	CNN	https://github.com/Shaggyshak/CS543_project_Image-based-Localization-of-Bridge-Defects-with-AR-Visualization	nn
Semantic 3D Occupancy Mapping through Efficient High Order CRFs	CNN	https://github.com/shichaoy/semantic_3d_mapping	BSD-3-Clause
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene		https://github.com/shubhtuls/factored3d	nn
Motion R-CNN codebase (old)	RCNN	https://github.com/simonmeister/old-motion-rcnn	MIT License
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation	PyTorch	https://github.com/sshan-zhao/GASDA	nn
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera		https://github.com/StanfordVL/3DSceneGraph	MIT License
Minkowski Engine is an auto-diff convolutional neural network library for high-dimensional sparse tensors	PyTorch	https://github.com/stanfordvl/MinkowskiEngine	MIT License
Learning Single-View 3D Reconstruction with Limited Pose Supervision (Official implementation)	Tensorflow	https://github.com/stevenygd/3d-recon	MIT License
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera (Tensorflow version)	Tensorflow	https://github.com/timctho/VNect-tensorflow	Apache-2.0
3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Image		https://github.com/val-iisc/3d-lmnet	MIT License
Learning to Find Good Correspondences		https://github.com/vcg-uvic/learned-correspondence-release	For reserch and evaluation only. Commercial usage requires written approval
A Framework for the Volumetric Integration of Depth Images		https://github.com/victorprad/InfiniTAM	non-commercial
Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation	Tensorflow	https://github.com/walsvid/Pixel2MeshPlusPlus	BSD-3-Clause
Adversarial Semantic Scene Completion from a Single Depth Image (Official implementation)	Tensorflow	https://github.com/wangyida/gan-depth-semantic3d	nn
SurfelWarp: Efficient Non-Volumetric Dynamic Reconstruction		https://github.com/weigao95/surfelwarp	BSD-3-Clause
PCN: Point Completion Network	Tensorflow	https://github.com/wentaoyuan/pcn	MIT License
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction		https://github.com/Xharlie/DISN	nn
Real-time motion from structure	CNN	https://github.com/yan99033/CNN-SVO	nn
Dense 3D Object Reconstruction from a Single Depth View	Tensorflow	https://github.com/Yang7879/3D-RecGAN-extended	MIT License
Semi-supervised monocular depth map prediction	Tensorflow	https://github.com/Yevkuzn/semodepth	GPL-3.0
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration	Tensorflow	https://github.com/yewzijian/3DFeatNet	MIT License
Estimated Depth Map Helps Image Classification: Depth estimation with neural network, and learning on RGBD images		https://github.com/yihui-he/Estimated-Depth-Map-Helps-Image-Classification	MIT License
Fit 3DMM to front and side face images simultaneously.		https://github.com/Yinghao-Li/3DMM-fitting	nn
The Perfect Match: 3D Point Cloud Matching with Smoothed Densities	CNN, Tensorflow	https://github.com/zgojcic/3DSmoothNet	BSD-3-Clause
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution	Tenosorflow	https://github.com/zhou13/neurvps	MIT License
LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image (Torch implementation)	Torch	https://github.com/zouchuhang/LayoutNet	MIT License
NeRF: Neural Radiance Fields		https://github.com/bmild/nerf	MIT License	10
Local Light Field Fusion at SIGGRAPH 2019		https://github.com/fyusion/llff	GPL-3.0	10
neural-volumes-learning-dynamic-renderable-volumes-from-images		https://research.fb.com/publications/neural-volumes-learning-dynamic-renderable-volumes-from-images/ https://github.com/facebookresearch/neuralvolumes	BY-NC 4.0
Learning Less is More - 6D Camera Localization via 3D Surface Regression		https://github.com/vislearn/LessMore	BSD-3-Clause
Local features		https://github.com/vcg-uvic/lf-net-release
Pix2Vox		https://github.com/hzxie/Pix2Vox
PlanarReconstruction: Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding	pytorch	https://github.com/svip-lab/PlanarReconstruction
Depth estimation with deep Neural networks		https://medium.com/@omarbarakat1995/depth-estimation-with-deep-neural-networks-part-1-5fa6d2237d0d https://medium.com/datadriveninvestor/depth-estimation-with-deep-neural-networks-part-2-81ee374888eb https://github.com/MahmoudSelmy/DeeperDepthEstimation https://github.com/MahmoudSelmy/DepthEstimationVGG/blob/master/README.md
High Quality Monocular Depth Estimation via Transfer Learning		https://github.com/ialhashim/DenseDepth https://arxiv.org/abs/1812.11941
D3Feat		https://github.com/XuyangBai/D3Feat
Hierarchical Deep Stereo Matching on High Resolution Images	pytorch	https://github.com/gengshan-y/high-res-stereo	MIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimation	pytorch	https://github.com/Xt-Chen/SARPN	nn
Pytorch code to construct a 3D point cloud model from single RGB image.	pytorch	https://github.com/lkhphuc/pytorch-3d-point-cloud-generation	nn
Depth estimation from RGB images using fully convolutional neural networks	pytorch	https://github.com/karoly-hars/DE_resnet_unet_hyb	BSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding	torch, tensorflow	https://github.com/svip-lab/PlanarReconstruction	MIT
TriDepth: Triangular Patch-based Deep Depth Prediction	PyTorch	https://github.com/syinari0123/tridepth	MIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network	torch	https://github.com/imran3180/depth-map-prediction	nn
Hybrid CNN for Single Image Depth Estimation	torch	https://github.com/karoly-hars/DE_resnet_unet_hyb	BSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketches	torch	https://github.com/jiajunwu/marrnet	nn
Consistent Video Depth Estimation		https://roxanneluo.github.io/Consistent-Video-Depth-Estimation/	nn
HF-Net: Robust Hierarchical Localization at Large Scale	torch, tensorflow	https://github.com/ethz-asl/hfnet	MIT
Hierarchical Deep Stereo Matching on High Resolution Images	pytorch	https://github.com/gengshan-y/high-res-stereo	MIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimation	pytorch	https://github.com/Xt-Chen/SARPN	nn
Pytorch code to construct a 3D point cloud model from single RGB image.	pytorch	https://github.com/lkhphuc/pytorch-3d-point-cloud-generation	nn
Depth estimation from RGB images using fully convolutional neural networks	pytorch	https://github.com/karoly-hars/DE_resnet_unet_hyb	BSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding	torch, tensorflow	https://github.com/svip-lab/PlanarReconstruction	MIT
TriDepth: Triangular Patch-based Deep Depth Prediction	PyTorch	https://github.com/syinari0123/tridepth	MIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network	torch	https://github.com/imran3180/depth-map-prediction	nn
Hybrid CNN for Single Image Depth Estimation	torch	https://github.com/karoly-hars/DE_resnet_unet_hyb	BSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketches	torch	https://github.com/jiajunwu/marrnet	nn
Consistent Video Depth Estimation		https://roxanneluo.github.io/Consistent-Video-Depth-Estimation/	nn
HF-Net: Robust Hierarchical Localization at Large Scale	torch, tensorflow	https://github.com/ethz-asl/hfnet	MIT

Other Projects

TITLE	KEYWORDS	URL	LICENSE
3D-Scene-GAN: Three-dimensional Scene Reconstruction with Generative Adversarial Networks	paper	https://openreview.net/forum?id=SkNEsmJwf
Google: Deep Learning Depth Prediction	magazine article, GER	https://www.digitalproduction.com/2019/05/27/google-deep-learning-depth-prediction/
SLAM and Deep Leraning for 3D Indoor Scene Understanding	PhD thesis	https://www.doc.ic.ac.uk/~ajd/Publications/McCormac-J-2019-PhD-Thesis.pdf
Dense 3D Object Reconstruction from a Single Depth View	3D-RecGAN++	https://arxiv.org/abs/1802.00411
Moving Camera, Moving People: A Deep Learning Approach to Depth Prediction		https://ai.googleblog.com/2019/05/moving-camera-moving-people-deep.html
Depth Estimation from a Single RGB Image		http://campar.in.tum.de/Chair/ProjectDepthPrediction
Deep Fundamental Matrix Estimation		http://vladlen.info/papers/deep-fundamental.pdf
depth_estimation		https://towardsdatascience.com/depth-estimation-on-camera-images-using-densenets-ac454caa893
3D-Machine-Learning List		https://github.com/timzhang642/3D-Machine-Learning
DEEP LEARNING-BASED 3D OBJECT RECONSTRUCTION - A SURVEY - Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Era		https://arxiv.org/pdf/1906.06543.pdf

I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs (CVPR 2023) https://github.com/jingsenzhu/i2-sdf MIT

https://github.com/lioryariv/idr

https://github.com/autonomousvision/differentiable_volumetric_rendering

https://github.com/Dok11/surface-match-dataset

Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Era https://arxiv.org/pdf/1906.06543v3.pdf

Dense 3D Object Reconstructionfrom a Single Depth View https://arxiv.org/pdf/1802.00411v2.pdf

https://dagshub.com/OperationSavta/SavtaDepth https://colab.research.google.com/drive/1XU4DgQ217_hUMU1dllppeQNw3pTRlHy1?usp=sharing https://huggingface.co/spaces/kingabzpro/savtadepth MIT License

https://github.com/gradslam/gradslam pyTorch

https://github.com/ventusff/neurecon

https://github.com/theICTlab/3DUNDERWORLD-SLS-GPU_CPU