Data298 MSDA Project 2022.05

Team 8: Mavis Wang, Coco Yu, Xiaocen Xie, Ililta Gebrihiwet

Smart Mirror Fashion AI

A smart-mirror-based fashion AI using CP-VTON-PLUS.

Network Integration

Input

* Reference Person (Video)
* Target Cloth Item (Image)

Output

* body keypoints tracking video
* cloth transferring try-on video

Dataset

• Processed train/test VTON
• Processed Custom Person Video
• Processed CP-VTON Clothing and Mask

Video Pre-processing Pipeline

Key Improvement

• Improved pose tracking
• Improved human parsing (fixed-neck)
• Improved body pose estimation
  

Results

Project Management

SMFAI Modules

• Video Processing and Tracking
• Garment Mask
• Human body Pose Shape Estimation
• CP-VTON GMM
• CP-VTON TOM

Tasks

• Roadmap
• Data collection
• Finalize test dataset - ref_person video
• Finalize test dataset - target cloth
• Individual module development, testing, and improvement
• Test Video Object Tracking
• SMFAI network pipeline testing
• Network development
• Network testing
• Network evaluation
• Models integration

Citation

@InProceedings{SJSU MSDA Program Project SMFAI,
	title={Smart Mirror-Based Fashion AI Based on Machine Learning},
	author={Mavis Wang, Coco Yu, Xiaocen Xie, Ililta Gebrihiwet},
	month = {May},
	year = {2022}
}
@InProceedings{Minar_CPP_2020_CVPR_Workshops,
	title={CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On},
	author={Minar, Matiur Rahman and Thai Thanh Tuan and Ahn, Heejune and Rosin, Paul and Lai, Yu-Kun},
	booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
	month = {June},
	year = {2020}
}

This project is highly influenced by Toward Characteristic-Preserving Image-based Virtual Try-On Network and reproduced based on the repo CP-VTON-Plus Network.