The code is based(mostly all NN code) on the ultralytics/yolov3 and is freely available for redistribution under the GPL-3.0 license. The original repository is licensed under GPL-3.0, hence the license is GPL-3.0.
The neural network to detect human face touch in an image or video stream using Yolo V3. The idea comes from Keras creator Francois Chollet tweet.
Here is the network prediction on a clip of Slavoj Zizek
Source: https://www.youtube.com/embed/YvJocp4Ovzg
Network output on the first ten second clip of the original video - https://www.youtube.com/embed/n44WsmRiAvY
Prediction on set of 17 images
conda env create -f environment.yml. This is for Mac OSX.conda activate facetouch.
- Create a virtual environment and install requirements using conda and pip.
conda install cython numpypip install -r requirements.txt.
If you issue with installation, all you need is the following libraries,
pytorch, torchvision, matplotlib, pycocotools, numpy, opencv-python, opencv-python-headless.
TODO: Add the dockerfile
- Download the pretrained weights,
gdown https://drive.google.com/uc?id=16lYS4bcIdM2HdmyJBVDOvt3Trx6N3W2R. - Remove the webcam lid or sticker(if any),
python detect.py --cfg cfg/yolov3-1cls.cfg --weights best_dataset_v5.pt --source 0. - Note, while rendering the UI (QT) for webcam or audio, the depending on OS, you may encounter errors. Now you can leave the webcam on in the background and hear the sound "Don't touch the face" while you bring the hand close to your face.
- To run on set of images or videos, run the command,
python detect.py --cfg cfg/yolov3-1cls.cfg --weights best_dataset_v5.pt --source test_images. The output is stored inoutputdirectory.
- The documentation for transfer learning is available in the original repo wiki.
- Download the dataset using gdown
gdown https://drive.google.com/uc?id=1ziZ0Bw40KE4pfxezyjq7FMzOntDNNzuu. - Unzip the dataset,
unzip dataset_v5.zip. - Download the pretrained weights from ultralytics,
gdown https://drive.google.com/uc?id=16lYS4bcIdM2HdmyJBVDOvt3Trx6N3W2R. - Train the network,
python train.py --weights yolov3-spp.weights --data dataset_v5/coco1.data --cfg cfg/yolov3-1cls.cfg. - The train will run 300 epochs and store the results in the
weightsdirectory. Thebest.ptis theweightsis used for later inference.
-
To download the images for training, you can run the file
face_touch_download.pylikepython face_touch_download.py --keyword "chin leaning" --limit 500 --dest chin_leaning. The above command opens the firefox web browser, sends the keyword "chin leaning" to google, downloads first 500 images to the directorychin_leaning. You need to have Firefox Selenium plugin. -
To annotate the downloaded images, you can use LabelImg. Install
pip install labelimg. You can annotate the images likelabelimg <images> coco1.names. The annotation will be stored in the same directory. Do it for all images directories. -
Now there are images and annotation. Create the dataset using following command,
$python dataset.py --directories face_touch --directories chin_leaning --name dataset_v5. This will createdataset_v5directory and zip file withlabelsandimagesdirectories along with train and valid split.
$ls -lat dataset_v5
total 128
drwxr-xr-x 59 user staff 1888 Mar 15 22:02 ..
-rw-r--r-- 1 user staff 91 Mar 15 17:24 coco1.data
drwxr-xr-x 9 user staff 288 Mar 15 17:24 .
-rw-r--r-- 1 user staff 10 Mar 15 17:24 coco1.names
-rw-r--r-- 1 user staff 4819 Mar 15 17:24 test.txt
-rw-r--r-- 1 user staff 7051 Mar 15 17:24 val.txt
-rw-r--r-- 1 user staff 36893 Mar 15 17:24 train.txt
drwxr-xr-x 5 user staff 160 Mar 15 17:24 labels
drwxr-xr-x 5 user staff 160 Mar 15 17:24 images
$ls -lat dataset_v5/images/
total 0
drwxr-xr-x 9 user staff 288 Mar 15 17:24 ..
drwxr-xr-x 106 user staff 3392 Mar 15 17:24 test
drwxr-xr-x 5 user staff 160 Mar 15 17:24 .
drwxr-xr-x 158 user staff 5056 Mar 15 17:24 val
drwxr-xr-x 782 user staff 25024 Mar 15 17:24 train- The data used for training can be found in Google Drive in the name
dataset_v5.zip- https://drive.google.com/drive/u/0/folders/17-rLAQ9GLda7M5mvDiitINqSLBUj_wbm