ICG-Net

ICG-Net: A Unified Approach for Instance Centric Grasping

About The Project

This repo contains the implementation of ICG-Net. For the Benchmark and Checkpoints, please refer to the Benchmark Repository.

Getting Started

To get a local copy up and running follow these steps:

Prerequisites

Installation

Pytorch 2.2, Cuda 12.1

To install MinkowskiEngine with Pytorch >= 2.0 and Cuda >= 12.0, you will need to install our patched version of MinkowskiEngine.

sudo apt-get install libopenblas-dev

export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"
# Install conda environment
conda env create -f conda_cu121.yml
conda activate icg_cuda121

# Install Pytorch and Dependencies
pip install torch torchvision torchaudio torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.2.0+cu121.html


# Install Patched MinkowskiEngine
git clone git@github.com:renezurbruegg/MinkowskiEngine.git
cd MinkowskiEngine

# Link conda cuda version for minkowski compilation
export CUDA_HOME=${CONDA_PREFIX}/envs/icg_cuda121
# Link cuda libraries.
export LD_LIBRARY_PATH=${CONDA_PREFIX}/lib:${LD_LIBRARY_PATH}
# Link cuda headers. This might not be necessary for all systems.
sudo ln -s ${CONDA_PREFIX}/lib/libcudart.so.12 /usr/lib/libcudart.so
python setup.py install --force_cuda --blas=openblas --blas_include_dirs=${CONDA_PREFIX}/include --cuda_home=$CUDA_HOME
cd -

# Install third party requirements
cd icg_net/third_party/pointnet2
python setup.py install
cd -

# Install icg_net as pip package
pip install -e .

Pytorch 1.12 Cuda 11.3

sudo apt-get install libopenblas-dev

# Install conda environment
export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"
conda env create -f conda_cu113.yml
conda activate icg_cuda113

# Install Pytorch and Dependencies
pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip install torch_geometric
pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-1.12.0+cu113.html



# Install MinkowskiEngine
git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine"
cd MinkowskiEngine
git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228
python setup.py install --force_cuda --blas=openblas

# Install third party requirements
cd icg_net/third_party/pointnet2
python setup.py install
cd -

# Install icg_net as pip package
pip install -e .

Usage

from icg_net import ICGNetModule, get_model
from icg_net.typing import ModelPredOut


# Load model
model: ICGNetModule = get_model(
    "icg_benchmark/data/51--0.656/config.yaml",
    device="cuda" if torch.cuda.is_available() else "cpu",
)
pc, normals = # ... load pc

out: ModelPredOut = model(
    torch.from_numpy(np.asarray(o3dc.points)).float(),
    normals=torch.from_numpy(np.asarray(o3dc.normals)).float(),
    grasp_pts=grasp_pts,
    grasp_normals=grasp_normals,
    n_grasps=512,
    each_object=True,
    return_meshes=True,
    return_scene_grasps=True,
)

print(out)

(back to top)

License

Distributed under the BSD-2 License. See LICENSE.txt for more information.

(back to top)

Citing

If you use this code in your research, please cite the following paper:

@article{zurbrugg2024icgnet,
  title={ICGNet: A Unified Approach for Instance-Centric Grasping},
  author={Zurbr{\"u}gg, Ren{\'e} and Liu, Yifan and Engelmann, Francis and Kumar, Suryansh and Hutter, Marco and Patil, Vaishakh and Yu, Fisher},
  journal={arXiv preprint arXiv:2401.09939},
  year={2024}
}