An object removal from image system using deep learning image segmentation and inpainting techniques.
Object removal from image involves two separate tasks, object detection and object removal.
The first task is handled by the user drawing a bounding box around an object of interest to be removed. We could then remove all pixels inside the bounding box, but this could lead to loss of valuable information from the pixels in the box that are not part of the object. Instead Mask-RCNN, a state of the art instance segmentation model is used to get the exact mask of the object.
Filling in the image is done using DeepFillv2, an image inpainting generative adversarial network which employs a gated convolution system.
The result is a complete image with the object removed.
The DeepFillv2 model needs pretrained weights from here provided by this repository which is a reimplementation of DeepFillv2 in Pytroch. Code for DeepFillv2 model was borrowed and slightly modified from there.
Make sure to put the weights pth file in src/models/.
To run on example image,
./src/main.py [path of image]
When drawing bounding box, press 'r' to clear bounding box and reset image. Once box is drawn press 'c' to continue.
*Drawing bouding boxes is sometimes slow.
The following are some results of the system. The user selected bounding box is shown along with the masked image and inpainted final result.
- python3
- torch
- torchvision
- cv2
- matplotlib
- numpy