Learning Deep Morphological Networks with Neural Architecture Search

This is a project to study the effect of morphological networks on image classification and edge detection based on Neural Architecture Optimization.

There are two sub-projects, one is for image classification, which is NAO-V2, and the other is for edge detection, which is NAO_Seg_v2:

1. NAO-V2: Official implementation of NAO-V2: [Understanding and Improving One-shot Neural Architecture Optimization]. Thanks to Neural Architecture Optimization, we proposed a new morphological network that performs better on CIFAR10 and CIFAR100 than traditional convolutional layers.

2. Released with the paper Richer Convolutional Features for Edge Detection. The purpose of this project is edge detection. On the basis of ResNet-101, a decoder is added(where we call the new DNN NAO-Multi-scale) in order to get a better edge detection effect and prove the superiority of the morphological network. The decoder is based on cells that NAO-V2 generates.

Citations

If you find our work useful in your research, please consider citing:

@article{hu2022learning,
  title={Learning deep morphological networks with neural architecture search},
  author={Hu, Yufei and Belkhir, Nacim and Angulo, Jesus and Yao, Angela and Franchi, Gianni},
  journal={Pattern Recognition},
  volume={131},
  pages={108893},
  year={2022},
  publisher={Elsevier}
}

Requirements

• Pytorch >= 1.0.0
• Dataset for edge detection(provided by Richer Convolutional Features for Edge Detection repo)
  • [http://mftp.mmcheng.net/liuyun/rcf/data/bsds_pascal_train_pair.lst]
  • [http://mftp.mmcheng.net/liuyun/rcf/data/HED-BSDS.tar.gz]
  • [http://mftp.mmcheng.net/liuyun/rcf/data/PASCAL.tar.gz]
• Eval method provided by hed, but please replace the file eval_edge.m where we just modified the path and we keep the same for other parameters for fairness.
• and other requirements... (cv2, numpy , etc.)

Usage for classification

• To search the CNN architecture with the morphological network, please refer firstly to:

bash ./NAO_V2/train_search_cifar10.sh

• After model searching, we need to train CIFAR 10, then refer to:

Dataset	Script	Time	Top1 Error Rate
CIFAR-10	./NAO_V2/train_NAONet_V2_36_cifar10.sh	2 days	2.63%

bash ./NAO_V2/train_NAONet_V2_36_cifar10.sh

Usage for edge detection

BSD500

• Firstly, download the dataset and then use NAO to search the CNN architecture with the morphological gradient network; refer to:

bash ./NAO_Seg_v2/train_search_BSD500.sh

• After model searching, we need to train BSD500, then refer to:

Dataset	Script	Time	ODS	OIS	AP	R50
BSD500	./NAO_Seg_v2/train_NAOUNet_Berkeley.sh	2 days	0.814±0.001	0.831±0.001	0.850±0.002	0.908±0.005

bash ./NAO_Seg_v2/train_NAOUNet_Berkeley.sh

• After training, we can get test predicted features maps by running:

bash ./NAO_Seg_v2/test_NAOUNet_Berkeley.sh

Evaluation for edge detection

We are the first to add ODS python code to filter great architecture and save the better model during the process of training, which helps to reduce the time of evaluation greatly. This is because we don't need to do the evaluation, which costs about 2 hours for each checkpoint. We need to evaluate the predicted feature maps of the best and last checkpoints.

• Firstly, we need to modify the path to the correct address where you can refer to eval_edge.m
• Then download the complete evaluation project based on Matlab provided by HED, and run:

(echo "data_dir = '../output/epoch-x-test'"; cat eval_edge.m)|matlab -nodisplay -nodesktop -nosplash

URL

• Pretrained model for edge detection
• Test features maps
• Arch examples

License

The codes and models in this repo are released under the GNU GPLv3 license.

Acknowledgment

Thanks to the contributors of NAO, RCF, HED.

@misc{luo2019neural,
      title={Neural Architecture Optimization}, 
      author={Renqian Luo and Fei Tian and Tao Qin and Enhong Chen and Tie-Yan Liu},
      year={2019},
      eprint={1808.07233},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

@article{Liu_2019,
   title={Richer Convolutional Features for Edge Detection},
   volume={41},
   ISSN={1939-3539},
   url={http://dx.doi.org/10.1109/TPAMI.2018.2878849},
   DOI={10.1109/tpami.2018.2878849},
   number={8},
   journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
   publisher={Institute of Electrical and Electronics Engineers (IEEE)},
   author={Liu, Yun and Cheng, Ming-Ming and Hu, Xiaowei and Bian, Jia-Wang and Zhang, Le and Bai, Xiang and Tang, Jinhui},
   year={2019},
   month={Aug},
   pages={1939–1946}
}

@inproceedings{xie2015holistically,
  title={Holistically-nested edge detection},
  author={Xie, Saining and Tu, Zhuowen},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  pages={1395--1403},
  year={2015}
}