Kimera is a C++ library for real-time metric-semantic simultaneous localization and mapping, which uses camera images and inertial data to build a semantically annotated 3D mesh of the environment. Kimera is modular, ROS-enabled, and runs on a CPU.
Kimera comprises four modules:
- A fast and accurate Visual Inertial Odometry (VIO) pipeline (Kimera-VIO)
- A full SLAM implementation based on Robust Pose Graph Optimization (Kimera-RPGO)
- A per-frame and multi-frame 3D mesh generator (Kimera-Mesher)
- And a generator of semantically annotated 3D meshes (Kimera-Semantics)
Click on the following links to install Kimera's modules and get started! It is very easy to install!
If you found any of the above modules useful, we would really appreciate if you could cite our work:
- [1] A. Rosinol, T. Sattler, M. Pollefeys, L. Carlone. Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities. IEEE Int. Conf. on Robotics and Automation (ICRA), 2019. arXiv:1903.01067
@InProceedings{Rosinol19icra-incremental,
title = {Incremental visual-inertial 3d mesh generation with structural regularities},
author = {Rosinol, Antoni and Sattler, Torsten and Pollefeys, Marc and Carlone, Luca},
year = {2019},
booktitle = {2019 International Conference on Robotics and Automation (ICRA)},
pdf = {https://arxiv.org/pdf/1903.01067.pdf}
}- [2] A. Rosinol, M. Abate, Y. Chang, L. Carlone, Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping. IEEE Intl. Conf. on Robotics and Automation (ICRA), 2020. arXiv:1910.02490.
@InProceedings{Rosinol20icra-Kimera,
title = {Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping},
author = {Rosinol, Antoni and Abate, Marcus and Chang, Yun and Carlone, Luca},
year = {2020},
booktitle = {IEEE Intl. Conf. on Robotics and Automation (ICRA)},
url = {https://github.com/MIT-SPARK/Kimera},
pdf = {https://arxiv.org/pdf/1910.02490.pdf}
}- [3] A. Rosinol, A. Gupta, M. Abate, J. Shi, L. Carlone. 3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans. Robotics: Science and Systems (RSS), 2020. arXiv:2002.06289.
@InProceedings{Rosinol20rss-dynamicSceneGraphs,
title = {{3D} Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans},
author = {A. Rosinol and A. Gupta and M. Abate and J. Shi and L. Carlone},
year = {2020},
booktitle = {Robotics: Science and Systems (RSS)},
pdf = {https://arxiv.org/pdf/2002.06289.pdf}
}- [4] A. Rosinol, A. Gupta, M. Abate, J. Shi, L. Carlone. Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs. arXiv:2101.06894.
@InProceedings{Rosinol21arxiv-Kimera,
title = {{K}imera: from {SLAM} to Spatial Perception with {3D} Dynamic Scene Graphs},
author = {A. Rosinol, A. Violette, M. Abate, N. Hughes, Y. Chang, J. Shi, A. Gupta, L. Carlone},
year = {2021},
booktitle = {arxiv},
pdf = {https://arxiv.org/pdf/2101.06894.pdf}
}In addition to the real-life tests on the Euroc dataset, we use a photo-realistic Unity-based simulator to test Kimera. The simulator provides:
- RGB Stereo camera
- Depth camera
- Ground-truth 2D Semantic Segmentation
- IMU data
- Ground-Truth Odometry
- 2D Lidar
- TF (ground-truth odometry of robots, and agents)
- Static TF (ground-truth poses of static objects)
Using this simulator, we created several large visual-inertial datasets which feature scenes with and without dynamic agents (humans), as well as a large variety of environments (indoors and outdoors, small and large). These are ideal to test your Metric-Semantic SLAM and/or other Spatial-AI systems!
Kimera is partially funded by ARL DCIST, ONR RAIDER, MIT Lincoln Laboratory, and “la Caixa” Foundation (ID 100010434), LCF/BQ/AA18/11680088 (A. Rosinol).
