SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation

Paper | Project Page | Video

SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation
Hang Yin*, Xiuwei Xu* $^\dagger$, Zhenyu Wu, Jie Zhou, Jiwen Lu$^\ddagger$

* Equal contribution $\dagger$ Project leader $\ddagger$ Corresponding author

We propose a zero-shot object-goal navigation framework by constructing an online 3D scene graph to prompt LLMs. Our method can be directly applied to different kinds of scenes and categories without training. 中文解读.

News

• [2024/09/26]: SG-Nav is accepted to NeurIPS 2024!

Demo

Scene1:

Scene2:

Demos are a little bit large; please wait a moment to load them. Welcome to the home page for more complete demos and detailed introductions.

Method

Method Pipeline:

Getting Started

For environment setup and dataset preparation, please follow:

• Installation

For evaluation, please follow:

• Evaluation

Citation

@article{yin2024sgnav, 
      title={SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation}, 
      author={Hang Yin and Xiuwei Xu and Zhenyu Wu and Jie Zhou and Jiwen Lu},
      journal={arXiv preprint arXiv:2410.08189},
      year={2024}
}