Quantized_Ball_detection_BL.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Quantized Ball_detection_BL.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "8vAGvftxHu8K"
      ],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tobieabel/robokeeper/blob/master/Quantized_Ball_detection_BL.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ubUsE7qtMWfj",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "### **Inroduction**:\n",
        "\n",
        "\n",
        "to create a Subbuteo robokeeper which can detect a subbuteo ball of any colour.  This colab does transfer learning on the SSD_Quant_Mobilenet_V2 model (as this is the only one that can then run on tflite and Coral TPU) and can reac a loss of sub 2.0 on about 30,000 training steps. \n",
        "\n",
        "\n",
        "### This notebook has been built upon and evolved from this excellent [medium post](https://medium.com/@alaasinjab/detailed-tutorial-build-your-custom-real-time-object-detector-5ade1017fd2d)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yi0JMo0RNT2Y",
        "colab_type": "text"
      },
      "source": [
        "### This notebook was designed to be ran from top to bottom without the need to mount Google Drive"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "65t7YUhnzDCE",
        "colab_type": "text"
      },
      "source": [
        "## BAll Detection Using Tensorflow Object Detection API \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CWrRz3kXDksW",
        "colab_type": "text"
      },
      "source": [
        "Workspace structure\n",
        "\n",
        "```\n",
        "gun_detection/\n",
        "        ├─ data/\n",
        "        │    ├── images/\n",
        "        │    │      ├── Ball (1).jpg\n",
        "        │    │      ├── Ball (2).jpg\n",
        "        │    │      └── ...\n",
        "        │    ├── train_labels/\n",
        "        │    │      ├── Ball (1).xml\n",
        "        │    │      ├── Ball (2).xml\n",
        "        │    │      └── ...\n",
        "        │    ├── test_labels/\n",
        "        │    │      ├── Ball (10).xml\n",
        "        │    │      ├── Ball (20).xml\n",
        "        │    │      └── ...\n",
        "        │    ├── label_map.pbtxt\n",
        "        │    ├── test_labels.csv\n",
        "        │    ├── train_labels.csv\n",
        "        │    ├── test_labels.record\n",
        "        │    └── train_labels.record\n",
        "        └─ models/\n",
        "             ├─ research/\n",
        "             │      ├── fine_tuned_model/\n",
        "             │      │         ├── frozen_inference_graph.pb\n",
        "             │      │         └── ...\n",
        "             │      │         \n",
        "             │      ├── pretrained_model/\n",
        "             │      │         ├── frozen_inference_graph.pb\n",
        "             │      │         └── ...\n",
        "             │      │         \n",
        "             │      ├── object_detection/\n",
        "             │      │         ├── utils/\n",
        "             │      │         ├── samples/\n",
        "             │      │         │      ├── samples/ \n",
        "             │      │         │      │       ├── configs/             \n",
        "             │      │         │      │       │     ├── ssd_mobilenet_v2_coco.config\n",
        "             │      │         │      │       │     ├── rfcn_resnet101_pets.config\n",
        "             │      │         │      │       │     └── ...\n",
        "             │      │         │      │       └── ... \n",
        "             │      │         │      └── ...                                \n",
        "             │      │         ├── export_inference_graph.py\n",
        "             │      │         ├── model_main.py\n",
        "             │      │         └── ...\n",
        "             │      │         \n",
        "             │      ├── training/\n",
        "             │      │         ├── events.out.tfevents.xxxxx\n",
        "             │      │         └── ...               \n",
        "             │      └── ...\n",
        "             └── ...\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AMlXJ2yIV8e7",
        "colab_type": "text"
      },
      "source": [
        "## Choosing a pre training model\n",
        "The model used for this project is `ssd_quant_mobilenet_v2_coco`.\n",
        "Check other models from [here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md#coco-trained-models).\n",
        "\n",
        "Because the interestes of this project is to interfere on real time video, i am chosing a model that has a high inference speed `(ms)` with relativly high `mAP` on COCO, plus is quantized so can be turned into a tflite model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JKEXJzgkaVEw",
        "colab_type": "text"
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TfBWfefIaOE6",
        "colab_type": "text"
      },
      "source": [
        "We need an earlier version of numpy as latest version seems to throw a floating point error during training.  Remember to press reset button in the warning message"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8N30Q7ZBdsa7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "outputId": "a2b93c4c-58e2-4854-e2ea-0ad3e9d69a6b"
      },
      "source": [
        "pip install numpy==1.17.4"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting numpy==1.17.4\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/d2/ab/43e678759326f728de861edbef34b8e2ad1b1490505f20e0d1f0716c3bf4/numpy-1.17.4-cp36-cp36m-manylinux1_x86_64.whl (20.0MB)\n",
            "\u001b[K     |████████████████████████████████| 20.0MB 158kB/s \n",
            "\u001b[31mERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.\u001b[0m\n",
            "\u001b[31mERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: numpy\n",
            "  Found existing installation: numpy 1.18.5\n",
            "    Uninstalling numpy-1.18.5:\n",
            "      Successfully uninstalled numpy-1.18.5\n",
            "Successfully installed numpy-1.17.4\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "numpy"
                ]
              }
            }
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Se4T7HueMdnj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "outputId": "a7e7d690-e407-4cb5-ed3a-483164f3027c"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HnuFnB94duc5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%tensorflow_version 1.x"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XbfWEcU6aCBw",
        "colab_type": "text"
      },
      "source": [
        "Models to choose from (add in a quantised SSD model)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j3_Ns54i3HgO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "\n",
        "# Some models to train on\n",
        "MODELS_CONFIG = {\n",
        "    'ssd_mobilenet_v2': {\n",
        "        'model_name': 'ssd_mobilenet_v2_coco_2018_03_29',\n",
        "        'pipeline_file': 'ssd_mobilenet_v2_coco.config',\n",
        "    },\n",
        "    'ssd_quant_mobilenet_v2': {\n",
        "        'model_name': 'ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03',\n",
        "        'pipeline_file': 'ssd_mobilenet_v2_quantized_300x300_coco.config',\n",
        "    },\n",
        "    'faster_rcnn_inception_v2': {\n",
        "        'model_name': 'faster_rcnn_inception_v2_coco_2018_01_28',\n",
        "        'pipeline_file': 'faster_rcnn_inception_v2_pets.config',\n",
        "    },\n",
        "    'rfcn_resnet101': {\n",
        "        'model_name': 'rfcn_resnet101_coco_2018_01_28',\n",
        "        'pipeline_file': 'rfcn_resnet101_pets.config',\n",
        "    }\n",
        "}\n",
        "\n",
        "# Select a model in `MODELS_CONFIG`.\n",
        "# I chose ssd_quant_mobilenet_v2 for this project to work on tflite, you could choose any\n",
        "selected_model = 'ssd_quant_mobilenet_v2'\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sv3Zm042QGJy",
        "colab_type": "text"
      },
      "source": [
        "## Installing Required Packages "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "68StUELaQPS2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 663
        },
        "outputId": "6debd6ec-57ec-45b6-a5b6-cf8aa716f60e"
      },
      "source": [
        "!apt-get install -qq protobuf-compiler python-pil python-lxml python-tk\n",
        "\n",
        "!pip install -qq Cython contextlib2 pillow lxml matplotlib\n",
        "\n",
        "!pip install -qq pycocotools"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Selecting previously unselected package python-bs4.\n",
            "(Reading database ... 144465 files and directories currently installed.)\n",
            "Preparing to unpack .../0-python-bs4_4.6.0-1_all.deb ...\n",
            "Unpacking python-bs4 (4.6.0-1) ...\n",
            "Selecting previously unselected package python-pkg-resources.\n",
            "Preparing to unpack .../1-python-pkg-resources_39.0.1-2_all.deb ...\n",
            "Unpacking python-pkg-resources (39.0.1-2) ...\n",
            "Selecting previously unselected package python-chardet.\n",
            "Preparing to unpack .../2-python-chardet_3.0.4-1_all.deb ...\n",
            "Unpacking python-chardet (3.0.4-1) ...\n",
            "Selecting previously unselected package python-six.\n",
            "Preparing to unpack .../3-python-six_1.11.0-2_all.deb ...\n",
            "Unpacking python-six (1.11.0-2) ...\n",
            "Selecting previously unselected package python-webencodings.\n",
            "Preparing to unpack .../4-python-webencodings_0.5-2_all.deb ...\n",
            "Unpacking python-webencodings (0.5-2) ...\n",
            "Selecting previously unselected package python-html5lib.\n",
            "Preparing to unpack .../5-python-html5lib_0.999999999-1_all.deb ...\n",
            "Unpacking python-html5lib (0.999999999-1) ...\n",
            "Selecting previously unselected package python-lxml:amd64.\n",
            "Preparing to unpack .../6-python-lxml_4.2.1-1ubuntu0.1_amd64.deb ...\n",
            "Unpacking python-lxml:amd64 (4.2.1-1ubuntu0.1) ...\n",
            "Selecting previously unselected package python-olefile.\n",
            "Preparing to unpack .../7-python-olefile_0.45.1-1_all.deb ...\n",
            "Unpacking python-olefile (0.45.1-1) ...\n",
            "Selecting previously unselected package python-pil:amd64.\n",
            "Preparing to unpack .../8-python-pil_5.1.0-1ubuntu0.2_amd64.deb ...\n",
            "Unpacking python-pil:amd64 (5.1.0-1ubuntu0.2) ...\n",
            "Setting up python-pkg-resources (39.0.1-2) ...\n",
            "Setting up python-six (1.11.0-2) ...\n",
            "Setting up python-bs4 (4.6.0-1) ...\n",
            "Setting up python-lxml:amd64 (4.2.1-1ubuntu0.1) ...\n",
            "Setting up python-olefile (0.45.1-1) ...\n",
            "Setting up python-pil:amd64 (5.1.0-1ubuntu0.2) ...\n",
            "Setting up python-webencodings (0.5-2) ...\n",
            "Setting up python-chardet (3.0.4-1) ...\n",
            "Setting up python-html5lib (0.999999999-1) ...\n",
            "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ERyocH9U-o2Y",
        "colab_type": "text"
      },
      "source": [
        "## General imports\n",
        "Other Imports will be done after downloading some packages later."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CEVLeKXh-s23",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import division, print_function, absolute_import\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import csv\n",
        "import re\n",
        "import cv2 \n",
        "import os\n",
        "import glob\n",
        "import xml.etree.ElementTree as ET\n",
        "\n",
        "import io\n",
        "import tensorflow.compat.v1 as tf\n",
        "\n",
        "from PIL import Image\n",
        "from collections import namedtuple, OrderedDict\n",
        "\n",
        "import shutil\n",
        "import urllib.request\n",
        "import tarfile\n",
        "\n",
        "from google.colab import files"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y8QeHvX6gpmC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0ade2901-d44b-43f6-e78f-b5bad170e769"
      },
      "source": [
        "#we need tenorflow v 1.15.0, object detection API is removed from tf v 2.0+\n",
        "print(tf.__version__)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.15.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NyP-2RtwepAe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d24028cf-83c6-442f-e94d-0e2e71ddaea6"
      },
      "source": [
        "print(np.__version__)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.17.4\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sOcbTFEiPBKA",
        "colab_type": "text"
      },
      "source": [
        "## Downloading and Orgniazing Images and Annotations\n",
        "1. Downloading the images and annotations from the [source](https://sci2s.ugr.es/weapons-detection)  and unziping them\n",
        "2. Creating a directory `(data)` to save some data such as; images, annotation, csv, etc...\n",
        "3. Creating two directories; for the training and testing labels (not the images)\n",
        "4. Randomly splitting our labels into 80% training and 20% testing and moving the splits to their directories: `(train_labels)` & `(test_labels)` "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2QY-CyUQwyZr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#creates a directory for the whole project\n",
        "!mkdir Ball_detection"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CHPQQmhm7RLe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "5d4211b1-ef9a-4443-eb9b-d1cde8751cae"
      },
      "source": [
        "cd Ball_detection"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZLc6trpdCaYF",
        "colab_type": "text"
      },
      "source": [
        "Manually upload the ballxml.zip file to the ball detection folder and then uzip with below code and download the ball.zip file from G-drive.  downloading from Gdrive requires you to then move the zip files from '/root/.keras/datasets/Ball' to Ball_detection"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L0VusMeyKtBD",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "7205c14d-4076-4cae-e2f8-aa574981cdf8"
      },
      "source": [
        "import pathlib\n",
        "data_root = tf.keras.utils.get_file(origin='file:///content/drive/My Drive/Robotics/Ball.zip',\n",
        "                                         fname = 'Ball.zip',archive_format='zip',extract=False)\n",
        "data_root = pathlib.Path('/root/.keras/datasets/Ball.zip')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from file:///content/drive/My Drive/Robotics/Ball.zip\n",
            "443367424/443365641 [==============================] - 6s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j4hA9iKbPDYo",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "53132a34-f7e0-45f5-f816-225a6d0cb3a8"
      },
      "source": [
        "shutil.move('/root/.keras/datasets/Ball.zip', '/content/Ball_detection/Ball.zip')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic": {
              "type": "string"
            },
            "text/plain": [
              "'/content/Ball_detection/Ball.zip'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7QvDObP05ccx",
        "colab_type": "text"
      },
      "source": [
        "Manually upload the xml file into ball_detection folde before running this step"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tp62o3a07UbP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Training images and annotations\n",
        "\n",
        "#Source: https://sci2s.ugr.es/weapons-detection\n",
        "\n",
        "\n",
        "#download the images zip\n",
        "#!wget https://sci2s.ugr.es/sites/default/files/files/TematicWebSites/WeaponsDetection/BasesDeDatos/WeaponS.zip\n",
        "\n",
        "#unzip the image file\n",
        "!unzip -q Ball.zip\n",
        "\n",
        "#download the annotations zip\n",
        "#!wget https://sci2s.ugr.es/sites/default/files/files/TematicWebSites/WeaponsDetection/BasesDeDatos/WeaponS_bbox.zip\n",
        "\n",
        "#unzip the annotations file\n",
        "!unzip -q BallXML.zip"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L0czMMeR8GxW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# creating a directory to store the training and testing data\n",
        "!mkdir data\n",
        "\n",
        "# folders for the training and testing data.\n",
        "!mkdir data/images data/train_labels data/test_labels\n",
        "\n",
        "\n",
        "# combining the images and annotation in the training folder:\n",
        "# moves the images to data folder\n",
        "!mv Ball/* data/images\n",
        "\n",
        "# moves the annotations to data folder\n",
        "!mv BallXML/* data/train_labels"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vv8pmB2D80M7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Deleting the zipped and unzipped folders \n",
        "!rm -rf BallXML.zip  Ball.zip Ball/  BallXML/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gYk9MonCbX8u",
        "colab_type": "text"
      },
      "source": [
        "Fix the next step to shuffle the files to make it more random"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PUl-XRwPvj4j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "\n",
        "# lists the files inside 'annotations' in a random order (not really random, by their hash value instead)\n",
        "# Moves the first 250 labels to the testing dir: `test_labels`\n",
        "!ls data/train_labels/* | sort -R | head -250 | xargs -I{} mv {} data/test_labels"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pmvDu-rUHz96",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "64676de4-be79-4000-8f35-6f48eb650f8f"
      },
      "source": [
        "# 2071 \"images\"(xml) for training\n",
        "%ls -1 data/train_labels/ | wc -l"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2071\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K8y-1_t7wRJc",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b6b25e65-ba45-4440-ea0c-f396d915ab2c"
      },
      "source": [
        "# 250 \"images\"(xml) for testing\n",
        "%ls -1 data/test_labels/ | wc -l"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "250\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pOfuwfPrPSMz",
        "colab_type": "text"
      },
      "source": [
        "## Preprocessing Images and Labels\n",
        "1. Converting the annotations from xml files to two csv files for each `train_labels/` and `train_labels/`.\n",
        "2. Creating a pbtxt file that specifies the number of class (one class in this case)\n",
        "3. Checking if the annotations for each object are placed within the range of the image width and height."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-RO5BS-Ooqyz",
        "colab_type": "text"
      },
      "source": [
        "Amend this to pick up xml files from relevant path in gdrive and pbtext file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TBHBFpWyEIDI",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "b5c1d179-8600-4a82-cb45-af40bf3da37a"
      },
      "source": [
        "\n",
        "#adjusted from: https://github.com/datitran/raccoon_dataset\n",
        "\n",
        "#converts the annotations/labels into one csv file for each training and testing labels\n",
        "#creats label_map.pbtxt file\n",
        "\n",
        "%cd /content/Ball_detection/data\n",
        "\n",
        "\n",
        "# images extension\n",
        "images_extension = 'jpg'\n",
        "\n",
        "# takes the path of a directory that contains xml files and converts\n",
        "#  them to one csv file.\n",
        "\n",
        "# returns a csv file that contains: image name, width, height, class, xmin, ymin, xmax, ymax.\n",
        "# note: if the xml file contains more than one box/label, it will create more than one row for the same image. each row contains the info for an individual box. \n",
        "def xml_to_csv(path):\n",
        "  classes_names = []\n",
        "  xml_list = []\n",
        "\n",
        "  for xml_file in glob.glob(path + '/*.xml'):\n",
        "    tree = ET.parse(xml_file)\n",
        "    root = tree.getroot()\n",
        "    for member in root.findall('object'):\n",
        "      classes_names.append(member[0].text)\n",
        "      value = (root.find('filename').text, #+ '.' + images_extension,\n",
        "               int(root.find('size')[0].text),\n",
        "               int(root.find('size')[1].text),\n",
        "               member[0].text,\n",
        "               int(member[4][0].text),\n",
        "               int(member[4][1].text),\n",
        "               int(member[4][2].text),\n",
        "               int(member[4][3].text))\n",
        "      xml_list.append(value)\n",
        "  column_name = ['filename', 'width', 'height', 'class', 'xmin', 'ymin', 'xmax', 'ymax']\n",
        "  xml_df = pd.DataFrame(xml_list, columns=column_name) \n",
        "  classes_names = list(set(classes_names))\n",
        "  classes_names.sort()\n",
        "  return xml_df, classes_names\n",
        "\n",
        "# for both the train_labels and test_labels csv files, it runs the xml_to_csv() above.\n",
        "for label_path in ['train_labels', 'test_labels']:\n",
        "  image_path = os.path.join(os.getcwd(), label_path)\n",
        "  xml_df, classes = xml_to_csv(label_path)\n",
        "  xml_df.to_csv(f'{label_path}.csv', index=None)\n",
        "  print(f'Successfully converted {label_path} xml to csv.')\n",
        "\n",
        "# Creating the `label_map.pbtxt` file\n",
        "label_map_path = os.path.join(\"label_map.pbtxt\")\n",
        "\n",
        "pbtxt_content = \"\"\n",
        "\n",
        "#creats a pbtxt file the has the class names.\n",
        "for i, class_name in enumerate(classes):\n",
        "    # display_name is optional.\n",
        "    pbtxt_content = (\n",
        "        pbtxt_content\n",
        "        + \"item {{\\n    id: {0}\\n    name: '{1}'\\n    display_name: 'Ball'\\n }}\\n\\n\".format(i + 1, class_name)\n",
        "    )\n",
        "pbtxt_content = pbtxt_content.strip()\n",
        "with open(label_map_path, \"w\") as f:\n",
        "    f.write(pbtxt_content)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/data\n",
            "Successfully converted train_labels xml to csv.\n",
            "Successfully converted test_labels xml to csv.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rtfjZcD-CCdM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "73addcbb-179f-4ff2-f3bc-26be3ecbb98f"
      },
      "source": [
        "#checking the pbtxt file\n",
        "!cat label_map.pbtxt"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "item {\n",
            "    id: 1\n",
            "    name: 'Ball'\n",
            "    display_name: 'Ball'\n",
            " }"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yP8gohagKFXn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        },
        "outputId": "ee88132a-70c4-4b59-8f2e-21c3ad08294c"
      },
      "source": [
        "# they are there!\n",
        "%ls -l"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "total 304\n",
            "drwxr-xr-x 2 root root  81920 Jul 21 07:15 \u001b[0m\u001b[01;34mimages\u001b[0m/\n",
            "-rw-r--r-- 1 root root     61 Jul 21 07:16 label_map.pbtxt\n",
            "drwxr-xr-x 2 root root  12288 Jul 21 07:15 \u001b[01;34mtest_labels\u001b[0m/\n",
            "-rw-r--r-- 1 root root  13846 Jul 21 07:16 test_labels.csv\n",
            "drwxr-xr-x 2 root root  81920 Jul 21 07:15 \u001b[01;34mtrain_labels\u001b[0m/\n",
            "-rw-r--r-- 1 root root 113847 Jul 21 07:16 train_labels.csv\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L4p7J6mFLLZf",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        },
        "outputId": "afeb2c97-bdcd-45af-a2c5-ea688d0e9941"
      },
      "source": [
        "#checks if the images box position is placed within the image.\n",
        "\n",
        "#note: while this doesn't checks if the boxes/annotatoins are correctly\n",
        "# placed around the object, Tensorflow will through an error if this occured.\n",
        "%cd /content/Ball_detection/data\n",
        "# path to images\n",
        "images_path = 'images'\n",
        "\n",
        "#loops over both train_labels and test_labels csv files to do the check\n",
        "# returns the image name where an error is found \n",
        "# return the incorrect attributes; xmin, ymin, xmax, ymax.\n",
        "for CSV_FILE in ['train_labels.csv', 'test_labels.csv']:\n",
        "  with open(CSV_FILE, 'r') as fid:  \n",
        "      print('[*] Checking file:', CSV_FILE) \n",
        "      file = csv.reader(fid, delimiter=',')\n",
        "      first = True \n",
        "      cnt = 0\n",
        "      error_cnt = 0\n",
        "      error = False\n",
        "      for row in file:\n",
        "          if error == True:\n",
        "              error_cnt += 1\n",
        "              error = False         \n",
        "          if first == True:\n",
        "              first = False\n",
        "              continue     \n",
        "          cnt += 1      \n",
        "          name, width, height, xmin, ymin, xmax, ymax = row[0], int(row[1]), int(row[2]), int(row[4]), int(row[5]), int(row[6]), int(row[7])     \n",
        "          path = os.path.join(images_path, name)\n",
        "          img = cv2.imread(path)         \n",
        "          if type(img) == type(None):\n",
        "              error = True\n",
        "              print('Could not read image', img)\n",
        "              continue     \n",
        "          org_height, org_width = img.shape[:2]     \n",
        "          if org_width != width:\n",
        "              error = True\n",
        "              print('Width mismatch for image: ', name, width, '!=', org_width)     \n",
        "          if org_height != height:\n",
        "              error = True\n",
        "              print('Height mismatch for image: ', name, height, '!=', org_height) \n",
        "          if xmin > org_width:\n",
        "              error = True\n",
        "              print('XMIN > org_width for file', name)  \n",
        "          if xmax > org_width:\n",
        "              error = True\n",
        "              print('XMAX > org_width for file', name)\n",
        "          if ymin > org_height:\n",
        "              error = True\n",
        "              print('YMIN > org_height for file', name)\n",
        "          if ymax > org_height:\n",
        "              error = True\n",
        "              print('YMAX > org_height for file', name)\n",
        "          if error == True:\n",
        "              print('Error for file: %s' % name)\n",
        "              print()\n",
        "      print()\n",
        "      print('Checked %d files and realized %d errors' % (cnt, error_cnt))\n",
        "      print(\"-----\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/data\n",
            "[*] Checking file: train_labels.csv\n",
            "\n",
            "Checked 2488 files and realized 0 errors\n",
            "-----\n",
            "[*] Checking file: test_labels.csv\n",
            "\n",
            "Checked 303 files and realized 0 errors\n",
            "-----\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RMHPlbSXWjEu",
        "colab_type": "text"
      },
      "source": [
        "If there are any errors, you can delete them with the below code"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vD5luKTsMx7F",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 130
        },
        "outputId": "066a5368-0ec3-4397-9f7c-e0770009cd20"
      },
      "source": [
        "#if we have any incorrect box position, we could just remove it \n",
        "#removing the image \n",
        "rm images/'armas (2815).jpg'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "SyntaxError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-22-28b12fd022ad>\"\u001b[0;36m, line \u001b[0;32m3\u001b[0m\n\u001b[0;31m    rm images/'armas (2815).jpg'\u001b[0m\n\u001b[0m            ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ze4z9bW3ZjhC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#removing the entry for it in the csv for that image as well\n",
        "\n",
        "#because we did a random split for the data, we dont know if it ended up being in training or testing\n",
        "# we will remove the image from both.\n",
        "\n",
        "#training\n",
        "#reading the training csv\n",
        "df = pd.read_csv('/content/gun_detection/data/train_labels.csv')\n",
        "# removing armas (2815).jpg\n",
        "df = df[df['filename'] != 'armas (2815).jpg']\n",
        "#reseting the index\n",
        "df.reset_index(drop=True, inplace=True)\n",
        "#saving the df\n",
        "df.to_csv('/content/gun_detection/data/train_labels.csv')\n",
        "\n",
        "\n",
        "#testing\n",
        "#reading the testing csv\n",
        "df = pd.read_csv('/content/gun_detection/data/test_labels.csv')\n",
        "# removing armas (2815).jpg\n",
        "df = df[df['filename'] != 'armas (2815).jpg']\n",
        "#reseting the index\n",
        "df.reset_index(drop=True, inplace=True)\n",
        "#saving the df\n",
        "df.to_csv('/content/gun_detection/data/test_labels.csv')\n",
        "\n",
        "# Just for the memory\n",
        "df = None\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A_tyvKnBP6qD",
        "colab_type": "text"
      },
      "source": [
        "## Downloading and Preparing Tensorflow model\n",
        "1. Cloning [Tensorflow models](https://github.com/tensorflow/models.git) from the offical git repo. The repo contains the object detection API we are interseted in. \n",
        "2. Compiling the protos and adding folders to the os environment.\n",
        "3. Testing the model builder."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IIxz1GqJQA3f",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "f236bc52-31d1-4748-a3b5-94891785c795"
      },
      "source": [
        "# Downlaods Tenorflow\n",
        "%cd /content/Ball_detection/\n",
        "!git clone --q https://github.com/tensorflow/models.git"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tjcAhsxRQ5N1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "outputId": "2ee31501-7cca-4f66-b771-cc91d6760b89"
      },
      "source": [
        "%cd /content/Ball_detection/models/research\n",
        "#compiling the proto buffers (not important to understand for this project but you can learn more about them here: https://developers.google.com/protocol-buffers/)\n",
        "!protoc object_detection/protos/*.proto --python_out=.\n",
        "\n",
        "# exports the PYTHONPATH environment variable with the reasearch and slim folders' paths\n",
        "os.environ['PYTHONPATH'] += ':/content/Ball_detection/models/research/:/content/Ball_detection/models/research/slim/'"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/models/research\n",
            "object_detection/protos/input_reader.proto: warning: Import object_detection/protos/image_resizer.proto but not used.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "skGTvD-Z9361",
        "colab_type": "text"
      },
      "source": [
        "need to install tf_slim into colab\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3-8ElZUi9gM2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156
        },
        "outputId": "bcf33329-15f8-4b87-a327-3c76de76b13d"
      },
      "source": [
        "!pip install tf_slim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting tf_slim\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/02/97/b0f4a64df018ca018cc035d44f2ef08f91e2e8aa67271f6f19633a015ff7/tf_slim-1.1.0-py2.py3-none-any.whl (352kB)\n",
            "\r\u001b[K     |█                               | 10kB 25.5MB/s eta 0:00:01\r\u001b[K     |█▉                              | 20kB 2.1MB/s eta 0:00:01\r\u001b[K     |██▉                             | 30kB 2.8MB/s eta 0:00:01\r\u001b[K     |███▊                            | 40kB 3.1MB/s eta 0:00:01\r\u001b[K     |████▋                           | 51kB 2.6MB/s eta 0:00:01\r\u001b[K     |█████▋                          | 61kB 2.8MB/s eta 0:00:01\r\u001b[K     |██████▌                         | 71kB 3.1MB/s eta 0:00:01\r\u001b[K     |███████▌                        | 81kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████▍                       | 92kB 3.6MB/s eta 0:00:01\r\u001b[K     |█████████▎                      | 102kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████▎                     | 112kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████▏                    | 122kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████                    | 133kB 3.4MB/s eta 0:00:01\r\u001b[K     |█████████████                   | 143kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████████                  | 153kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████                 | 163kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████▉                | 174kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████▊               | 184kB 3.4MB/s eta 0:00:01\r\u001b[K     |█████████████████▊              | 194kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████████████▋             | 204kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████████▌            | 215kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████████▌           | 225kB 3.4MB/s eta 0:00:01\r\u001b[K     |█████████████████████▍          | 235kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████████████████▍         | 245kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████████████▎        | 256kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████████████▏       | 266kB 3.4MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▏      | 276kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████████████████████      | 286kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████████████████     | 296kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 307kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▉   | 317kB 3.4MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▉  | 327kB 3.4MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▊ | 337kB 3.4MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▋| 348kB 3.4MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 358kB 3.4MB/s \n",
            "\u001b[?25hRequirement already satisfied: absl-py>=0.2.2 in /usr/local/lib/python3.6/dist-packages (from tf_slim) (0.9.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from absl-py>=0.2.2->tf_slim) (1.12.0)\n",
            "Installing collected packages: tf-slim\n",
            "Successfully installed tf-slim-1.1.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3bMNsrwTSJi2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# testing the model builder\n",
        "!python3 object_detection/builders/model_builder_test.py"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t9C3L_r4Pi6m",
        "colab_type": "text"
      },
      "source": [
        "## Generating Tf record\n",
        "- Generating two TFRecords files for the training and testing CSVs.\n",
        "- Tensorflow accepts the data as tfrecords which is a binary file that run fast with low memory usage. Instead of loading the full data into memory, Tenorflow breaks the data into batches using these TFRecords automatically"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nK2unk-9LB_E",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "e6cc405c-739e-4bf7-875f-2776106f1d16"
      },
      "source": [
        "#adjusted from: https://github.com/datitran/raccoon_dataset\n",
        "\n",
        "# converts the csv files for training and testing data to two TFRecords files.\n",
        "# places the output in the same directory as the input\n",
        "\n",
        "\n",
        "from object_detection.utils import dataset_util\n",
        "%cd /content/Ball_detection/models/\n",
        "\n",
        "DATA_BASE_PATH = '/content/Ball_detection/data/'\n",
        "image_dir = DATA_BASE_PATH +'images/'\n",
        "\n",
        "def class_text_to_int(row_label):\n",
        "\t\tif row_label == 'Ball':\n",
        "\t\t\t\treturn 1\n",
        "\t\telse:\n",
        "\t\t\t\tNone\n",
        "\n",
        "\n",
        "def split(df, group):\n",
        "\t\tdata = namedtuple('data', ['filename', 'object'])\n",
        "\t\tgb = df.groupby(group)\n",
        "\t\treturn [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]\n",
        "\n",
        "def create_tf_example(group, path):\n",
        "\t\twith tf.io.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:\n",
        "\t\t\t\tencoded_jpg = fid.read()\n",
        "\t\tencoded_jpg_io = io.BytesIO(encoded_jpg)\n",
        "\t\timage = Image.open(encoded_jpg_io)\n",
        "\t\twidth, height = image.size\n",
        "\n",
        "\t\tfilename = group.filename.encode('utf8')\n",
        "\t\timage_format = b'jpg'\n",
        "\t\txmins = []\n",
        "\t\txmaxs = []\n",
        "\t\tymins = []\n",
        "\t\tymaxs = []\n",
        "\t\tclasses_text = []\n",
        "\t\tclasses = []\n",
        "\n",
        "\t\tfor index, row in group.object.iterrows():\n",
        "\t\t\t\txmins.append(row['xmin'] / width)\n",
        "\t\t\t\txmaxs.append(row['xmax'] / width)\n",
        "\t\t\t\tymins.append(row['ymin'] / height)\n",
        "\t\t\t\tymaxs.append(row['ymax'] / height)\n",
        "\t\t\t\tclasses_text.append(row['class'].encode('utf8'))\n",
        "\t\t\t\tclasses.append(class_text_to_int(row['class']))\n",
        "\n",
        "\t\ttf_example = tf.train.Example(features=tf.train.Features(feature={\n",
        "\t\t\t\t'image/height': dataset_util.int64_feature(height),\n",
        "\t\t\t\t'image/width': dataset_util.int64_feature(width),\n",
        "\t\t\t\t'image/filename': dataset_util.bytes_feature(filename),\n",
        "\t\t\t\t'image/source_id': dataset_util.bytes_feature(filename),\n",
        "\t\t\t\t'image/encoded': dataset_util.bytes_feature(encoded_jpg),\n",
        "\t\t\t\t'image/format': dataset_util.bytes_feature(image_format),\n",
        "\t\t\t\t'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),\n",
        "\t\t\t\t'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),\n",
        "\t\t\t\t'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),\n",
        "\t\t\t\t'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),\n",
        "\t\t\t\t'image/object/class/text': dataset_util.bytes_list_feature(classes_text),\n",
        "\t\t\t\t'image/object/class/label': dataset_util.int64_list_feature(classes),\n",
        "\t\t}))\n",
        "\t\treturn tf_example\n",
        "\n",
        "for csv in ['train_labels', 'test_labels']:\n",
        "  writer = tf.io.TFRecordWriter(DATA_BASE_PATH + csv + '.record')\n",
        "  path = os.path.join(image_dir)\n",
        "  examples = pd.read_csv(DATA_BASE_PATH + csv + '.csv')\n",
        "  grouped = split(examples, 'filename')\n",
        "  for group in grouped:\n",
        "      tf_example = create_tf_example(group, path)\n",
        "      writer.write(tf_example.SerializeToString())\n",
        "    \n",
        "  writer.close()\n",
        "  output_path = os.path.join(os.getcwd(), DATA_BASE_PATH + csv + '.record')\n",
        "  print('Successfully created the TFRecords: {}'.format(DATA_BASE_PATH +csv + '.record'))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/models\n",
            "Successfully created the TFRecords: /content/Ball_detection/data/train_labels.record\n",
            "Successfully created the TFRecords: /content/Ball_detection/data/test_labels.record\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i1zRJducWs-X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        },
        "outputId": "73a9e650-e88e-415a-d59c-b59ff9da3c38"
      },
      "source": [
        "# TFRecords are created\n",
        "%ls -lX /content/Ball_detection/data/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "total 477212\n",
            "drwxr-xr-x 2 root root     81920 Jul 21 07:15 \u001b[0m\u001b[01;34mimages\u001b[0m/\n",
            "drwxr-xr-x 2 root root     12288 Jul 21 07:15 \u001b[01;34mtest_labels\u001b[0m/\n",
            "drwxr-xr-x 2 root root     81920 Jul 21 07:15 \u001b[01;34mtrain_labels\u001b[0m/\n",
            "-rw-r--r-- 1 root root     13846 Jul 21 07:16 test_labels.csv\n",
            "-rw-r--r-- 1 root root    113847 Jul 21 07:16 train_labels.csv\n",
            "-rw-r--r-- 1 root root        61 Jul 21 07:16 label_map.pbtxt\n",
            "-rw-r--r-- 1 root root  53140779 Jul 21 07:17 test_labels.record\n",
            "-rw-r--r-- 1 root root 435210481 Jul 21 07:17 train_labels.record\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xMckMSJqFMyc",
        "colab_type": "text"
      },
      "source": [
        "## Downloading the Base Model\n",
        "1. Based on the model selecting at the top of this notebook, downloading the model selected and extracting its content.\n",
        "2. Creating a dir to save the model while training."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UvN9Cw65FQzB",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "378b2c24-0f1a-4ec9-fe2d-8a6a7ec9d55d"
      },
      "source": [
        "%cd /content/Ball_detection/models/research\n",
        "\n",
        "# Name of the object detection model to use.\n",
        "MODEL = MODELS_CONFIG[selected_model]['model_name']\n",
        "\n",
        "# Name of the pipline file in tensorflow object detection API.\n",
        "pipeline_file = MODELS_CONFIG[selected_model]['pipeline_file']\n",
        "\n",
        "#selecting the model\n",
        "MODEL_FILE = MODEL + '.tar.gz'\n",
        "\n",
        "#creating the downlaod link for the model selected\n",
        "DOWNLOAD_BASE = 'http://download.tensorflow.org/models/object_detection/'\n",
        "\n",
        "#the distination folder where the model will be saved\n",
        "fine_tune_dir = '/content/Ball_detection/models/research/pretrained_model'\n",
        "\n",
        "#checks if the model has already been downloaded\n",
        "if not (os.path.exists(MODEL_FILE)):\n",
        "    urllib.request.urlretrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE)\n",
        "\n",
        "#unzipping the file and extracting its content\n",
        "tar = tarfile.open(MODEL_FILE)\n",
        "tar.extractall()\n",
        "tar.close()\n",
        "\n",
        "# creating an output file to save the model while training\n",
        "os.remove(MODEL_FILE)\n",
        "if (os.path.exists(fine_tune_dir)):\n",
        "    shutil.rmtree(fine_tune_dir)\n",
        "os.rename(MODEL, fine_tune_dir)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/models/research\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pbjXKVMmFk47",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        },
        "outputId": "e7a4415f-0590-4608-83d0-8366d3366d1f"
      },
      "source": [
        "#checking the content of the pretrained model.\n",
        "# this is the directory of the \"fine_tune_checkpoint\" that is used in the config file.\n",
        "!echo {fine_tune_dir}\n",
        "!ls -alh {fine_tune_dir}"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/content/Ball_detection/models/research/pretrained_model\n",
            "total 204M\n",
            "drwx------  2 303230 5000 4.0K Jan  4  2019 .\n",
            "drwxr-xr-x 63 root   root 4.0K Jul 21 07:18 ..\n",
            "-rw-------  1 303230 5000  93M Jan  4  2019 model.ckpt.data-00000-of-00001\n",
            "-rw-------  1 303230 5000  68K Jan  4  2019 model.ckpt.index\n",
            "-rw-------  1 303230 5000  20M Jan  4  2019 model.ckpt.meta\n",
            "-rw-------  1 303230 5000 4.3K Jan  4  2019 pipeline.config\n",
            "-rw-------  1 303230 5000  24M Jan  4  2019 tflite_graph.pb\n",
            "-rw-------  1 303230 5000  68M Jan  4  2019 tflite_graph.pbtxt\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HnjQgJZiGAcA",
        "colab_type": "text"
      },
      "source": [
        "## Configuring the Training Pipeline\n",
        "1. Adding the path for the TFRecords files and pbtxt,batch_size,num_steps,num_classes to the configuration file.\n",
        "2. Adding some Image augmentation.\n",
        "3. Creating a directory to save the model at each checkpoint while training. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "az14XVo31Ujp",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "d6e78a86-41a0-4b50-8982-e38def743d9d"
      },
      "source": [
        "\n",
        "#the path to the folder containing all the sample config files\n",
        "CONFIG_BASE = \"/content/Ball_detection/models/research/object_detection/samples/configs/\"\n",
        "\n",
        "#path to the specified model's config file\n",
        "model_pipline = os.path.join(CONFIG_BASE, pipeline_file)\n",
        "model_pipline"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic": {
              "type": "string"
            },
            "text/plain": [
              "'/content/Ball_detection/models/research/object_detection/samples/configs/ssd_mobilenet_v2_quantized_300x300_coco.config'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VT3m6pbXpN_M",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "ae680765-bfed-4614-f3d7-d466009bd702"
      },
      "source": [
        "#check the sample config file that is provided by the tf model\n",
        "!cat /content/Ball_detection/models/research/object_detection/samples/configs/ssd_mobilenet_v2_quantized_300x300_coco.config"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "# Quantized trained SSD with Mobilenet v2 on MSCOCO Dataset.\n",
            "# Users should configure the fine_tune_checkpoint field in the train config as\n",
            "# well as the label_map_path and input_path fields in the train_input_reader and\n",
            "# eval_input_reader. Search for \"PATH_TO_BE_CONFIGURED\" to find the fields that\n",
            "# should be configured.\n",
            "\n",
            "model {\n",
            "  ssd {\n",
            "    num_classes: 90\n",
            "    box_coder {\n",
            "      faster_rcnn_box_coder {\n",
            "        y_scale: 10.0\n",
            "        x_scale: 10.0\n",
            "        height_scale: 5.0\n",
            "        width_scale: 5.0\n",
            "      }\n",
            "    }\n",
            "    matcher {\n",
            "      argmax_matcher {\n",
            "        matched_threshold: 0.5\n",
            "        unmatched_threshold: 0.5\n",
            "        ignore_thresholds: false\n",
            "        negatives_lower_than_unmatched: true\n",
            "        force_match_for_each_row: true\n",
            "      }\n",
            "    }\n",
            "    similarity_calculator {\n",
            "      iou_similarity {\n",
            "      }\n",
            "    }\n",
            "    anchor_generator {\n",
            "      ssd_anchor_generator {\n",
            "        num_layers: 6\n",
            "        min_scale: 0.2\n",
            "        max_scale: 0.95\n",
            "        aspect_ratios: 1.0\n",
            "        aspect_ratios: 2.0\n",
            "        aspect_ratios: 0.5\n",
            "        aspect_ratios: 3.0\n",
            "        aspect_ratios: 0.3333\n",
            "      }\n",
            "    }\n",
            "    image_resizer {\n",
            "      fixed_shape_resizer {\n",
            "        height: 300\n",
            "        width: 300\n",
            "      }\n",
            "    }\n",
            "    box_predictor {\n",
            "      convolutional_box_predictor {\n",
            "        min_depth: 0\n",
            "        max_depth: 0\n",
            "        num_layers_before_predictor: 0\n",
            "        use_dropout: false\n",
            "        dropout_keep_probability: 0.8\n",
            "        kernel_size: 1\n",
            "        box_code_size: 4\n",
            "        apply_sigmoid_to_scores: false\n",
            "        conv_hyperparams {\n",
            "          activation: RELU_6,\n",
            "          regularizer {\n",
            "            l2_regularizer {\n",
            "              weight: 0.00004\n",
            "            }\n",
            "          }\n",
            "          initializer {\n",
            "            truncated_normal_initializer {\n",
            "              stddev: 0.03\n",
            "              mean: 0.0\n",
            "            }\n",
            "          }\n",
            "          batch_norm {\n",
            "            train: true,\n",
            "            scale: true,\n",
            "            center: true,\n",
            "            decay: 0.9997,\n",
            "            epsilon: 0.001,\n",
            "          }\n",
            "        }\n",
            "      }\n",
            "    }\n",
            "    feature_extractor {\n",
            "      type: 'ssd_mobilenet_v2'\n",
            "      min_depth: 16\n",
            "      depth_multiplier: 1.0\n",
            "      conv_hyperparams {\n",
            "        activation: RELU_6,\n",
            "        regularizer {\n",
            "          l2_regularizer {\n",
            "            weight: 0.00004\n",
            "          }\n",
            "        }\n",
            "        initializer {\n",
            "          truncated_normal_initializer {\n",
            "            stddev: 0.03\n",
            "            mean: 0.0\n",
            "          }\n",
            "        }\n",
            "        batch_norm {\n",
            "          train: true,\n",
            "          scale: true,\n",
            "          center: true,\n",
            "          decay: 0.9997,\n",
            "          epsilon: 0.001,\n",
            "        }\n",
            "      }\n",
            "    }\n",
            "    loss {\n",
            "      classification_loss {\n",
            "        weighted_sigmoid {\n",
            "        }\n",
            "      }\n",
            "      localization_loss {\n",
            "        weighted_smooth_l1 {\n",
            "        }\n",
            "      }\n",
            "      hard_example_miner {\n",
            "        num_hard_examples: 3000\n",
            "        iou_threshold: 0.99\n",
            "        loss_type: CLASSIFICATION\n",
            "        max_negatives_per_positive: 3\n",
            "        min_negatives_per_image: 3\n",
            "      }\n",
            "      classification_weight: 1.0\n",
            "      localization_weight: 1.0\n",
            "    }\n",
            "    normalize_loss_by_num_matches: true\n",
            "    post_processing {\n",
            "      batch_non_max_suppression {\n",
            "        score_threshold: 1e-8\n",
            "        iou_threshold: 0.6\n",
            "        max_detections_per_class: 100\n",
            "        max_total_detections: 100\n",
            "      }\n",
            "      score_converter: SIGMOID\n",
            "    }\n",
            "  }\n",
            "}\n",
            "\n",
            "train_config: {\n",
            "  batch_size: 24\n",
            "  optimizer {\n",
            "    rms_prop_optimizer: {\n",
            "      learning_rate: {\n",
            "        exponential_decay_learning_rate {\n",
            "          initial_learning_rate: 0.004\n",
            "          decay_steps: 800720\n",
            "          decay_factor: 0.95\n",
            "        }\n",
            "      }\n",
            "      momentum_optimizer_value: 0.9\n",
            "      decay: 0.9\n",
            "      epsilon: 1.0\n",
            "    }\n",
            "  }\n",
            "  fine_tune_checkpoint: \"PATH_TO_BE_CONFIGURED/model.ckpt\"\n",
            "  fine_tune_checkpoint_type:  \"detection\"\n",
            "  # Note: The below line limits the training process to 200K steps, which we\n",
            "  # empirically found to be sufficient enough to train the pets dataset. This\n",
            "  # effectively bypasses the learning rate schedule (the learning rate will\n",
            "  # never decay). Remove the below line to train indefinitely.\n",
            "  num_steps: 200000\n",
            "  data_augmentation_options {\n",
            "    random_horizontal_flip {\n",
            "    }\n",
            "  }\n",
            "  data_augmentation_options {\n",
            "    ssd_random_crop {\n",
            "    }\n",
            "  }\n",
            "}\n",
            "\n",
            "train_input_reader: {\n",
            "  tf_record_input_reader {\n",
            "    input_path: \"PATH_TO_BE_CONFIGURED/mscoco_train.record-?????-of-00100\"\n",
            "  }\n",
            "  label_map_path: \"PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt\"\n",
            "}\n",
            "\n",
            "eval_config: {\n",
            "  num_examples: 8000\n",
            "  # Note: The below line limits the evaluation process to 10 evaluations.\n",
            "  # Remove the below line to evaluate indefinitely.\n",
            "  max_evals: 10\n",
            "}\n",
            "\n",
            "eval_input_reader: {\n",
            "  tf_record_input_reader {\n",
            "    input_path: \"PATH_TO_BE_CONFIGURED/mscoco_val.record-?????-of-00010\"\n",
            "  }\n",
            "  label_map_path: \"PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt\"\n",
            "  shuffle: false\n",
            "  num_readers: 1\n",
            "}\n",
            "\n",
            "graph_rewriter {\n",
            "  quantization {\n",
            "    delay: 48000\n",
            "    weight_bits: 8\n",
            "    activation_bits: 8\n",
            "  }\n",
            "}"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lIzQoOLnXYcI",
        "colab_type": "text"
      },
      "source": [
        "Edit the config file, especially the number of steps to run through (20,000 in this case)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Kfsl5CsDGY3-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "f9e2f8fb-0ede-483a-c36b-2631771e0b24"
      },
      "source": [
        "#editing the configuration file to add the path for the TFRecords files, pbtxt,batch_size,num_steps,num_classes.\n",
        "# any image augmentation, hyperparemeter tunning (drop out, batch normalization... etc) would be editted here\n",
        "\n",
        "%%writefile {model_pipline}\n",
        "model {\n",
        "  ssd {\n",
        "    num_classes: 1 # number of classes to be detected\n",
        "    box_coder {\n",
        "      faster_rcnn_box_coder {\n",
        "        y_scale: 10.0\n",
        "        x_scale: 10.0\n",
        "        height_scale: 5.0\n",
        "        width_scale: 5.0\n",
        "      }\n",
        "    }\n",
        "    matcher {\n",
        "      argmax_matcher {\n",
        "        matched_threshold: 0.5\n",
        "        unmatched_threshold: 0.5\n",
        "        ignore_thresholds: false\n",
        "        negatives_lower_than_unmatched: true\n",
        "        force_match_for_each_row: true\n",
        "      }\n",
        "    }\n",
        "    similarity_calculator {\n",
        "      iou_similarity {\n",
        "      }\n",
        "    }\n",
        "    anchor_generator {\n",
        "      ssd_anchor_generator {\n",
        "        num_layers: 6\n",
        "        min_scale: 0.2\n",
        "        max_scale: 0.95\n",
        "        aspect_ratios: 1.0\n",
        "        aspect_ratios: 2.0\n",
        "        aspect_ratios: 0.5\n",
        "        aspect_ratios: 3.0\n",
        "        aspect_ratios: 0.3333\n",
        "      }\n",
        "    }\n",
        "    # all images will be resized to the below W x H.\n",
        "    image_resizer { \n",
        "      fixed_shape_resizer {\n",
        "        height: 300\n",
        "        width: 300\n",
        "      }\n",
        "    }\n",
        "    box_predictor {\n",
        "      convolutional_box_predictor {\n",
        "        min_depth: 0\n",
        "        max_depth: 0\n",
        "        num_layers_before_predictor: 0\n",
        "        #use_dropout: false\n",
        "        use_dropout: true # to counter over fitting. you can also try tweaking its probability below\n",
        "        dropout_keep_probability: 0.8\n",
        "        kernel_size: 1\n",
        "        box_code_size: 4\n",
        "        apply_sigmoid_to_scores: false\n",
        "        conv_hyperparams {\n",
        "          activation: RELU_6,\n",
        "          regularizer {\n",
        "            l2_regularizer {\n",
        "            # weight: 0.00004\n",
        "            weight: 0.001 # higher regularizition to counter overfitting\n",
        "          }\n",
        "          }\n",
        "          initializer {\n",
        "            truncated_normal_initializer {\n",
        "              stddev: 0.03\n",
        "              mean: 0.0\n",
        "            }\n",
        "          }\n",
        "          batch_norm {\n",
        "            train: true,\n",
        "            scale: true,\n",
        "            center: true,\n",
        "            decay: 0.9997,\n",
        "            epsilon: 0.001,\n",
        "          }\n",
        "        }\n",
        "      }\n",
        "    }\n",
        "    feature_extractor {\n",
        "      type: 'ssd_mobilenet_v2'\n",
        "      min_depth: 16\n",
        "      depth_multiplier: 1.0\n",
        "      conv_hyperparams {\n",
        "        activation: RELU_6,\n",
        "        regularizer {\n",
        "          l2_regularizer {\n",
        "            # weight: 0.00004\n",
        "            weight: 0.001 # higher regularizition to counter overfitting\n",
        "          }\n",
        "        }\n",
        "        initializer {\n",
        "          truncated_normal_initializer {\n",
        "            stddev: 0.03\n",
        "            mean: 0.0\n",
        "          }\n",
        "        }\n",
        "        batch_norm {\n",
        "          train: true,\n",
        "          scale: true,\n",
        "          center: true,\n",
        "          decay: 0.9997,\n",
        "          epsilon: 0.001,\n",
        "        }\n",
        "      }\n",
        "    }\n",
        "    loss {\n",
        "      classification_loss {\n",
        "        weighted_sigmoid {\n",
        "        }\n",
        "      }\n",
        "      localization_loss {\n",
        "        weighted_smooth_l1 {\n",
        "        }\n",
        "      }\n",
        "      hard_example_miner {\n",
        "        num_hard_examples: 3000 \n",
        "        iou_threshold: 0.75\n",
        "        loss_type: CLASSIFICATION\n",
        "        max_negatives_per_positive: 3\n",
        "        min_negatives_per_image: 3\n",
        "      }\n",
        "      classification_weight: 1.0\n",
        "      localization_weight: 1.0\n",
        "    }\n",
        "    normalize_loss_by_num_matches: true\n",
        "    post_processing {\n",
        "      batch_non_max_suppression {\n",
        "        score_threshold: 1e-8\n",
        "        iou_threshold: 0.6\n",
        "        \n",
        "        #adjust this to the max number of objects per class. \n",
        "        # ex, in my case, i have one ball in most of the images.\n",
        "        # . there are some images with more than one up to 4.\n",
        "        max_detections_per_class: 4\n",
        "        # max number of detections among all classes. I have 1 class only so\n",
        "        max_total_detections: 4\n",
        "      }\n",
        "      score_converter: SIGMOID\n",
        "    }\n",
        "  }\n",
        "}\n",
        "\n",
        "train_config: {\n",
        "  batch_size: 6 # training batch size\n",
        "  optimizer {\n",
        "    rms_prop_optimizer: {\n",
        "      learning_rate: {\n",
        "        exponential_decay_learning_rate {\n",
        "          initial_learning_rate: 0.004\n",
        "          decay_steps: 800720\n",
        "          decay_factor: 0.95\n",
        "        }\n",
        "      }\n",
        "      momentum_optimizer_value: 0.9\n",
        "      decay: 0.9\n",
        "      epsilon: 1.0\n",
        "    }\n",
        "  }\n",
        "\n",
        "  #the path to the pretrained model. \n",
        "  fine_tune_checkpoint: \"/content/Ball_detection/models/research/pretrained_model/model.ckpt\"\n",
        "  fine_tune_checkpoint_type:  \"detection\"\n",
        "  # Note: The below line limits the training process to 200K steps, which we\n",
        "  # empirically found to be sufficient enough to train the pets dataset. This\n",
        "  # effectively bypasses the learning rate schedule (the learning rate will\n",
        "  # never decay). Remove the below line to train indefinitely.\n",
        "  num_steps: 20000 \n",
        "  \n",
        "\n",
        "  #data augmentaion is done here, you can remove or add more.\n",
        "  # They will help the model generalize but the training time will increase greatly by using more data augmentation.\n",
        "  # Check this link to add more image augmentation: https://github.com/tensorflow/models/blob/master/research/object_detection/protos/preprocessor.proto\n",
        "  \n",
        "  data_augmentation_options {\n",
        "    random_horizontal_flip {\n",
        "    }\n",
        "  }\n",
        "  data_augmentation_options {\n",
        "    random_adjust_contrast {\n",
        "    }\n",
        "  }\n",
        "  data_augmentation_options {\n",
        "    ssd_random_crop {\n",
        "    }\n",
        "  }\n",
        "  data_augmentation_options {\n",
        "    random_adjust_contrast{\n",
        "    }  \n",
        "  }  \n",
        "  data_augmentation_options {\n",
        "      random_adjust_brightness{\n",
        "    }\n",
        "  } \n",
        "  data_augmentation_options{\n",
        "      random_distort_color{\n",
        "          \n",
        "      }\n",
        "  } \n",
        "}\n",
        "train_input_reader: {\n",
        "  tf_record_input_reader {\n",
        "    #path to the training TFRecord\n",
        "    input_path: \"/content/Ball_detection/data/train_labels.record\"\n",
        "  }\n",
        "  #path to the label map \n",
        "  label_map_path: \"/content/Ball_detection/data/label_map.pbtxt\"\n",
        "}\n",
        "\n",
        "eval_config: {\n",
        "  # the number of images in your \"testing\" data )\n",
        "  num_examples: 250\n",
        "  # the number of images to disply in Tensorboard while training\n",
        "  num_visualizations: 20\n",
        "\n",
        "  # Note: The below line limits the evaluation process to 10 evaluations.\n",
        "  # Remove the below line to evaluate indefinitely.\n",
        "  #max_evals: 10\n",
        "}\n",
        "\n",
        "eval_input_reader: {\n",
        "  tf_record_input_reader {\n",
        "      \n",
        "    #path to the testing TFRecord\n",
        "    input_path: \"/content/Ball_detection/data/test_labels.record\"\n",
        "  }\n",
        "  #path to the label map \n",
        "  label_map_path: \"/content/Ball_detection/data/label_map.pbtxt\"\n",
        "  shuffle: false\n",
        "  num_readers: 1\n",
        "}\n",
        "graph_rewriter {\n",
        "  quantization {\n",
        "    delay: 48000\n",
        "    weight_bits: 8\n",
        "    activation_bits: 8\n",
        "  }\n",
        "}"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Overwriting /content/Ball_detection/models/research/object_detection/samples/configs/ssd_mobilenet_v2_quantized_300x300_coco.config\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EuXXZLVEG8sO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# where the model will be saved at each checkpoint while training \n",
        "model_dir = 'training/'\n",
        "\n",
        "# Optionally: remove content in output model directory to fresh start.\n",
        "!rm -rf {model_dir}\n",
        "os.makedirs(model_dir, exist_ok=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8vAGvftxHu8K",
        "colab_type": "text"
      },
      "source": [
        "## Tensorboard\n",
        "1. Downlaoding and unzipping Tensorboard\n",
        "2. creating a link to visualize multiple graph while training.\n",
        "\n",
        "\n",
        "notes: \n",
        "  1. Tensorboard will not log any files until the training starts. \n",
        "  2. a max of 20 connection per minute is allowed when using ngrok, you will not be able to access tensorboard while the model is logging."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z2ucxlc5HxHL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        },
        "outputId": "53d41879-e9da-4231-c3ad-c39ad01f0156"
      },
      "source": [
        "#downlaoding ngrok to be able to access tensorboard on google colab\n",
        "!wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip\n",
        "!unzip -o ngrok-stable-linux-amd64.zip"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2020-07-21 07:19:44--  https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip\n",
            "Resolving bin.equinox.io (bin.equinox.io)... 52.5.95.18, 34.234.9.43, 54.164.74.108, ...\n",
            "Connecting to bin.equinox.io (bin.equinox.io)|52.5.95.18|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 13773305 (13M) [application/octet-stream]\n",
            "Saving to: ‘ngrok-stable-linux-amd64.zip’\n",
            "\n",
            "ngrok-stable-linux- 100%[===================>]  13.13M  43.1MB/s    in 0.3s    \n",
            "\n",
            "2020-07-21 07:19:44 (43.1 MB/s) - ‘ngrok-stable-linux-amd64.zip’ saved [13773305/13773305]\n",
            "\n",
            "Archive:  ngrok-stable-linux-amd64.zip\n",
            "  inflating: ngrok                   \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-w9ufxr7IAdv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#the logs that are created while training \n",
        "LOG_DIR = model_dir\n",
        "get_ipython().system_raw(\n",
        "    'tensorboard --logdir {} --host 0.0.0.0 --port 6006 &'\n",
        "    .format(LOG_DIR)\n",
        ")\n",
        "get_ipython().system_raw('./ngrok http 6006 &')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "idsi9zyNIIsr",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d24cae71-21fc-4cbe-b202-3ff2db24d861"
      },
      "source": [
        "#The link to tensorboard.\n",
        "#works after the training starts.\n",
        "\n",
        "### note: if you didnt get a link as output, rerun this cell and the one above\n",
        "!curl -s http://localhost:4040/api/tunnels | python3 -c \\\n",
        "    \"import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])\""
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "https://d25fb4c1a294.ngrok.io\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IuJcAPZFIfu7",
        "colab_type": "text"
      },
      "source": [
        "## Training\n",
        "\n",
        "Finally training the model!\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vnKt6g0_IgOe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "3f57f84f-2b5d-44a3-f24d-83ee262586c1"
      },
      "source": [
        "\n",
        "!python3 /content/Ball_detection/models/research/object_detection/model_main.py \\\n",
        "    --pipeline_config_path={model_pipline}\\\n",
        "    --model_dir={model_dir} \\\n",
        "    --alsologtostderr \\"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Forced number of epochs for all eval validations to be 1.\n",
            "W0721 07:20:03.839242 139661037922176 model_lib.py:758] Forced number of epochs for all eval validations to be 1.\n",
            "INFO:tensorflow:Maybe overwriting train_steps: None\n",
            "I0721 07:20:03.839435 139661037922176 config_util.py:552] Maybe overwriting train_steps: None\n",
            "INFO:tensorflow:Maybe overwriting use_bfloat16: False\n",
            "I0721 07:20:03.839528 139661037922176 config_util.py:552] Maybe overwriting use_bfloat16: False\n",
            "INFO:tensorflow:Maybe overwriting sample_1_of_n_eval_examples: 1\n",
            "I0721 07:20:03.839608 139661037922176 config_util.py:552] Maybe overwriting sample_1_of_n_eval_examples: 1\n",
            "INFO:tensorflow:Maybe overwriting eval_num_epochs: 1\n",
            "I0721 07:20:03.839693 139661037922176 config_util.py:552] Maybe overwriting eval_num_epochs: 1\n",
            "WARNING:tensorflow:Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.\n",
            "W0721 07:20:03.839792 139661037922176 model_lib.py:774] Expected number of evaluation epochs is 1, but instead encountered `eval_on_train_input_config.num_epochs` = 0. Overwriting `num_epochs` to 1.\n",
            "INFO:tensorflow:create_estimator_and_inputs: use_tpu False, export_to_tpu None\n",
            "I0721 07:20:03.839902 139661037922176 model_lib.py:809] create_estimator_and_inputs: use_tpu False, export_to_tpu None\n",
            "INFO:tensorflow:Using config: {'_model_dir': 'training/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
            "graph_options {\n",
            "  rewrite_options {\n",
            "    meta_optimizer_iterations: ONE\n",
            "  }\n",
            "}\n",
            ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f04f57db160>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
            "I0721 07:20:03.840288 139661037922176 estimator.py:212] Using config: {'_model_dir': 'training/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true\n",
            "graph_options {\n",
            "  rewrite_options {\n",
            "    meta_optimizer_iterations: ONE\n",
            "  }\n",
            "}\n",
            ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f04f57db160>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n",
            "WARNING:tensorflow:Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x7f04f57e6048>) includes params argument, but params are not passed to Estimator.\n",
            "W0721 07:20:03.840972 139661037922176 model_fn.py:630] Estimator's model_fn (<function create_model_fn.<locals>.model_fn at 0x7f04f57e6048>) includes params argument, but params are not passed to Estimator.\n",
            "INFO:tensorflow:Not using Distribute Coordinator.\n",
            "I0721 07:20:03.841571 139661037922176 estimator_training.py:186] Not using Distribute Coordinator.\n",
            "INFO:tensorflow:Running training and evaluation locally (non-distributed).\n",
            "I0721 07:20:03.841744 139661037922176 training.py:612] Running training and evaluation locally (non-distributed).\n",
            "INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.\n",
            "I0721 07:20:03.842059 139661037922176 training.py:700] Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n",
            "W0721 07:20:03.854504 139661037922176 deprecation.py:323] From /tensorflow-1.15.2/python3.6/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n",
            "WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.\n",
            "W0721 07:20:03.883735 139661037922176 dataset_builder.py:83] num_readers has been reduced to 1 to match input file shards.\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/builders/dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.\n",
            "W0721 07:20:03.888992 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/builders/dataset_builder.py:100: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.experimental.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.experimental_determinstic`.\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/builders/dataset_builder.py:175: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.Dataset.map()\n",
            "W0721 07:20:03.909488 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/builders/dataset_builder.py:175: DatasetV1.map_with_legacy_function (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.data.Dataset.map()\n",
            "WARNING:tensorflow:Entity <bound method TfExampleDecoder.decode of <object_detection.data_decoders.tf_example_decoder.TfExampleDecoder object at 0x7f04f5cd2c88>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'\n",
            "W0721 07:20:03.941941 139661037922176 ag_logging.py:146] Entity <bound method TfExampleDecoder.decode of <object_detection.data_decoders.tf_example_decoder.TfExampleDecoder object at 0x7f04f5cd2c88>> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Num'\n",
            "WARNING:tensorflow:Entity <function train_input.<locals>.transform_and_pad_input_data_fn at 0x7f051b96d840> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4\n",
            "W0721 07:20:04.121959 139661037922176 ag_logging.py:146] Entity <function train_input.<locals>.transform_and_pad_input_data_fn at 0x7f051b96d840> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/inputs.py:80: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.\n",
            "W0721 07:20:04.127499 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/inputs.py:80: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/utils/ops.py:493: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "W0721 07:20:04.134078 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/utils/ops.py:493: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/core/preprocessor.py:199: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.\n",
            "W0721 07:20:04.231545 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/core/preprocessor.py:199: sample_distorted_bounding_box (from tensorflow.python.ops.image_ops_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "`seed2` arg is deprecated.Use sample_distorted_bounding_box_v2 instead.\n",
            "WARNING:tensorflow:From /content/Ball_detection/models/research/object_detection/inputs.py:261: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.cast` instead.\n",
            "W0721 07:20:05.095406 139661037922176 deprecation.py:323] From /content/Ball_detection/models/research/object_detection/inputs.py:261: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.cast` instead.\n",
            "INFO:tensorflow:Calling model_fn.\n",
            "I0721 07:20:05.545196 139661037922176 estimator.py:1148] Calling model_fn.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "W0721 07:20:05.644503 139661037922176 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.166505 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.203791 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.239128 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.273747 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.308700 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0721 07:20:08.343824 139661037922176 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RPN8liiQc7Ue",
        "colab_type": "text"
      },
      "source": [
        "## Exporting The Trained model\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "upwUdom0lTub",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "8ab97fb0-7236-48fa-fe8f-819e71a13f8a"
      },
      "source": [
        "\n",
        "\n",
        "#the location where the exported model will be saved in.\n",
        "output_directory = '/content/Ball_detection/models/research/fine_tuned_model'\n",
        "\n",
        "# goes through the model is the training/ dir and gets the last one.\n",
        "# you could choose a specfic one instead of the last\n",
        "lst = os.listdir(model_dir)\n",
        "lst = [l for l in lst if 'model.ckpt-' in l and '.meta' in l]\n",
        "steps=np.array([int(re.findall('\\d+', l)[0]) for l in lst])\n",
        "last_model = lst[steps.argmax()].replace('.meta', '')\n",
        "last_model_path = os.path.join(model_dir, last_model)\n",
        "print(last_model_path)\n",
        "\n",
        "#exports the model specifed and inference graph\n",
        "!python  /content/Ball_detection/models/research/object_detection/export_tflite_ssd_graph.py \\\n",
        "    --input_type=image_tensor \\\n",
        "    --pipeline_config_path={model_pipline} \\\n",
        "    --output_directory={output_directory} \\\n",
        "    --trained_checkpoint_prefix={last_model_path} \\\n",
        "    --add_postprocessing_op=true"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "training/model.ckpt-30000\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "W0701 15:57:07.095890 140389234575232 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tf_slim/layers/layers.py:1089: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.666884 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.712033 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.748459 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.785671 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.825858 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "INFO:tensorflow:depth of additional conv before box predictor: 0\n",
            "I0701 15:57:09.971951 140389234575232 convolutional_box_predictor.py:156] depth of additional conv before box predictor: 0\n",
            "2020-07-01 15:57:10.030724: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1\n",
            "2020-07-01 15:57:10.051975: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.052743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-07-01 15:57:10.053128: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-07-01 15:57:10.054966: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-07-01 15:57:10.068387: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-07-01 15:57:10.068791: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-07-01 15:57:10.070778: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-07-01 15:57:10.091902: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-07-01 15:57:10.109275: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-07-01 15:57:10.109455: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.110330: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.111012: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-07-01 15:57:10.128524: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz\n",
            "2020-07-01 15:57:10.128760: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2fb0680 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n",
            "2020-07-01 15:57:10.128795: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version\n",
            "2020-07-01 15:57:10.189673: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.190410: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2fb0840 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
            "2020-07-01 15:57:10.190439: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla K80, Compute Capability 3.7\n",
            "2020-07-01 15:57:10.190588: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.191195: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-07-01 15:57:10.191281: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-07-01 15:57:10.191333: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-07-01 15:57:10.191375: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-07-01 15:57:10.191409: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-07-01 15:57:10.191441: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-07-01 15:57:10.191473: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-07-01 15:57:10.191506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-07-01 15:57:10.191600: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.192357: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.193028: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-07-01 15:57:10.193115: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-07-01 15:57:10.194737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-07-01 15:57:10.194766: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-07-01 15:57:10.194780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-07-01 15:57:10.194893: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.195585: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:10.196286: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "2020-07-01 15:57:10.196361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10805 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/Conv/add_fold\n",
            "I0701 15:57:12.064565 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/Conv/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv/depthwise/add_fold\n",
            "I0701 15:57:12.065007 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_1/expand/add_fold\n",
            "I0701 15:57:12.065441 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_1/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_1/depthwise/add_fold\n",
            "I0701 15:57:12.065701 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_1/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_2/expand/add_fold\n",
            "I0701 15:57:12.066019 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_2/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_2/depthwise/add_fold\n",
            "I0701 15:57:12.066261 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_2/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_3/expand/add_fold\n",
            "I0701 15:57:12.066631 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_3/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_3/depthwise/add_fold\n",
            "I0701 15:57:12.066857 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_3/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_4/expand/add_fold\n",
            "I0701 15:57:12.067182 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_4/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_4/depthwise/add_fold\n",
            "I0701 15:57:12.067458 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_4/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_5/expand/add_fold\n",
            "I0701 15:57:12.067850 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_5/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_5/depthwise/add_fold\n",
            "I0701 15:57:12.068130 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_5/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_6/expand/add_fold\n",
            "I0701 15:57:12.068505 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_6/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_6/depthwise/add_fold\n",
            "I0701 15:57:12.068807 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_6/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_7/expand/add_fold\n",
            "I0701 15:57:12.069181 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_7/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_7/depthwise/add_fold\n",
            "I0701 15:57:12.069439 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_7/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_8/expand/add_fold\n",
            "I0701 15:57:12.069774 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_8/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_8/depthwise/add_fold\n",
            "I0701 15:57:12.070118 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_8/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_9/expand/add_fold\n",
            "I0701 15:57:12.070562 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_9/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_9/depthwise/add_fold\n",
            "I0701 15:57:12.070788 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_9/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_10/expand/add_fold\n",
            "I0701 15:57:12.071155 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_10/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_10/depthwise/add_fold\n",
            "I0701 15:57:12.071525 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_10/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_11/expand/add_fold\n",
            "I0701 15:57:12.071828 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_11/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_11/depthwise/add_fold\n",
            "I0701 15:57:12.072051 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_11/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_12/expand/add_fold\n",
            "I0701 15:57:12.072384 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_12/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_12/depthwise/add_fold\n",
            "I0701 15:57:12.072621 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_12/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_13/expand/add_fold\n",
            "I0701 15:57:12.072969 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_13/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_13/depthwise/add_fold\n",
            "I0701 15:57:12.073202 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_13/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_14/expand/add_fold\n",
            "I0701 15:57:12.073518 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_14/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_14/depthwise/add_fold\n",
            "I0701 15:57:12.073747 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_14/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_15/expand/add_fold\n",
            "I0701 15:57:12.074056 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_15/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_15/depthwise/add_fold\n",
            "I0701 15:57:12.074322 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_15/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_16/expand/add_fold\n",
            "I0701 15:57:12.074722 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_16/expand/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_16/depthwise/add_fold\n",
            "I0701 15:57:12.074939 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/expanded_conv_16/depthwise/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/Conv_1/add_fold\n",
            "I0701 15:57:12.075268 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/Conv_1/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_2_1x1_256/add_fold\n",
            "I0701 15:57:12.075486 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_2_1x1_256/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_2_3x3_s2_512/add_fold\n",
            "I0701 15:57:12.075722 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_2_3x3_s2_512/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_3_1x1_128/add_fold\n",
            "I0701 15:57:12.075950 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_3_1x1_128/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_3_3x3_s2_256/add_fold\n",
            "I0701 15:57:12.076230 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_3_3x3_s2_256/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_4_1x1_128/add_fold\n",
            "I0701 15:57:12.076457 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_4_1x1_128/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_4_3x3_s2_256/add_fold\n",
            "I0701 15:57:12.076676 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_4_3x3_s2_256/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/add_fold\n",
            "I0701 15:57:12.076951 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_1_Conv2d_5_1x1_64/add_fold\n",
            "INFO:tensorflow:Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/add_fold\n",
            "I0701 15:57:12.077198 140389234575232 quantize.py:299] Skipping quant after FeatureExtractor/MobilenetV2/layer_19_2_Conv2d_5_3x3_s2_128/add_fold\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/tools/freeze_graph.py:127: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "W0701 15:57:12.708224 140389234575232 deprecation.py:323] From /tensorflow-1.15.2/python3.6/tensorflow_core/python/tools/freeze_graph.py:127: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use standard file APIs to check for files with this prefix.\n",
            "2020-07-01 15:57:13.633344: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:13.634037: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: \n",
            "name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
            "pciBusID: 0000:00:04.0\n",
            "2020-07-01 15:57:13.634136: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1\n",
            "2020-07-01 15:57:13.634180: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10\n",
            "2020-07-01 15:57:13.634224: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10\n",
            "2020-07-01 15:57:13.634265: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10\n",
            "2020-07-01 15:57:13.634312: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10\n",
            "2020-07-01 15:57:13.634351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10\n",
            "2020-07-01 15:57:13.634388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7\n",
            "2020-07-01 15:57:13.634475: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:13.635166: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:13.635811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0\n",
            "2020-07-01 15:57:13.635857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
            "2020-07-01 15:57:13.635878: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 \n",
            "2020-07-01 15:57:13.635891: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N \n",
            "2020-07-01 15:57:13.636028: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:13.636742: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
            "2020-07-01 15:57:13.637426: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10805 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
            "INFO:tensorflow:Restoring parameters from training/model.ckpt-30000\n",
            "I0701 15:57:13.638896 140389234575232 saver.py:1284] Restoring parameters from training/model.ckpt-30000\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/tools/freeze_graph.py:233: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.compat.v1.graph_util.convert_variables_to_constants`\n",
            "W0701 15:57:15.035121 140389234575232 deprecation.py:323] From /tensorflow-1.15.2/python3.6/tensorflow_core/python/tools/freeze_graph.py:233: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.compat.v1.graph_util.convert_variables_to_constants`\n",
            "WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
            "W0701 15:57:15.035431 140389234575232 deprecation.py:323] From /tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
            "INFO:tensorflow:Froze 632 variables.\n",
            "I0701 15:57:15.602539 140389234575232 graph_util_impl.py:334] Froze 632 variables.\n",
            "INFO:tensorflow:Converted 632 variables to const ops.\n",
            "I0701 15:57:15.711300 140389234575232 graph_util_impl.py:394] Converted 632 variables to const ops.\n",
            "2020-07-01 15:57:15.855676: I tensorflow/tools/graph_transforms/transform_graph.cc:317] Applying strip_unused_nodes\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VzED7WBahuE_",
        "colab_type": "text"
      },
      "source": [
        "Below code should take the saved model and convert it to a tflite file.  Make sure you change the final folder to be called \"Saved_Model\" - DO NOT USE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SXlqdGgMN7De",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "saved_model_dir = \"/content/Ball_detection/models/research/training/export/Servo/Saved_Model\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4GEuoGORxiXv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)\n",
        "converter.allow_custom_ops = True\n",
        "converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]\n",
        "tflite_model = converter.convert()\n",
        "open(\"Quant_converted_model.tflite\", \"wb\").write(tflite_model)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EAYj56uNFkw8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "files.download('Quant_converted_model.tflite')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z8R2wbZFQUQD",
        "colab_type": "text"
      },
      "source": [
        "The below code should take the frozen graph and convert it to TFLite using comand line tool, but I need to work out what the inputs arrays and syntax is - DO NOT USE"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aKa4nNyGQb7v",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 150
        },
        "outputId": "f6a13821-b7b3-4914-be1d-cbe0496cf084"
      },
      "source": [
        "tflite_convert \\\n",
        "--graph_def_file=tflite/tflite_graph.pb \\\n",
        "--output_file=tflite/detect.tflite \\\n",
        "--output_format=TFLITE \\\n",
        "--input_shapes=1,300,300,3 \\\n",
        "--input_arrays=normalized_input_image_tensor \\\n",
        "--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'  \\\n",
        "--inference_type=QUANTIZED_UINT8 \\\n",
        "--mean_values=128 \\\n",
        "--std_dev_values=127 \\\n",
        "--change_concat_input_ranges=false \\\n",
        "--allow_custom_ops \\"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "SyntaxError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-62-020061c8d5df>\"\u001b[0;36m, line \u001b[0;32m1\u001b[0m\n\u001b[0;31m    tflite_convert --graph_def_file=tflite/tflite_graph.pb --output_file=tflite/detect.tflite --output_format=TFLITE --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'  --inference_type=QUANTIZED_UINT8 --mean_values=128 --std_dev_values=127 --change_concat_input_ranges=false --allow_custom_ops\u001b[0m\n\u001b[0m                                                                                                                                                                                                                                                                                                                                                                                                                                                                              ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m can't assign to operator\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2tWb4WArQeEM",
        "colab_type": "text"
      },
      "source": [
        "Alternative code to covert from frozen graph using TFliteConverter.  This is the one thats actually working on RaspberryPi!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NZdnDUDxTQX1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "graph_def_file = (output_directory + '/tflite_graph.pb')\n",
        "input_arrays = [\"normalized_input_image_tensor\"]\n",
        "output_arrays = [\"TFLite_Detection_PostProcess\",\"TFLite_Detection_PostProcess:1\",\"TFLite_Detection_PostProcess:2\",\"TFLite_Detection_PostProcess:3\"]\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-RJzJyyiTmqb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "converter = tf.lite.TFLiteConverter.from_frozen_graph(\n",
        "  graph_def_file, \n",
        "  input_arrays, \n",
        "  output_arrays, \n",
        "  input_shapes={'normalized_input_image_tensor':[1, 300, 300, 3]}\n",
        "  \n",
        "  )"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i58g8ikO1YW3",
        "colab_type": "text"
      },
      "source": [
        "Allow custom ops is the bit I needed to add to make it work, I suspect it will also be the first bit to go wrong when tensorflow object detection API is updated for TF2.  Inference type and quantized input stats are necessary for post training quantization so that this mo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P6NfAHoWYXbQ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "converter.allow_custom_ops = True\n",
        "converter.inference_type = tf.uint8 \n",
        "converter.quantized_input_stats = {input_arrays[0] : (128., 128.)}  # mean_value, std_dev\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g9rzBId4UiJ7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0bf1eefe-692a-407f-97f6-b6324a9aae3a"
      },
      "source": [
        "tflite_model = converter.convert()\n",
        "open(\"Quant_converted_model_graph.tflite\", \"wb\").write(tflite_model)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "4711040"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "05qd9r-SIZWH",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 578
        },
        "outputId": "048d5b56-5dee-4b02-bde3-b48a69926f9c"
      },
      "source": [
        "%ls"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[0m\u001b[01;34ma3c_blogpost\u001b[0m/                      \u001b[01;34mlm_commonsense\u001b[0m/\n",
            "\u001b[01;34madversarial_crypto\u001b[0m/                \u001b[01;34mlstm_object_detection\u001b[0m/\n",
            "\u001b[01;34madversarial_logit_pairing\u001b[0m/         \u001b[01;34mmarco\u001b[0m/\n",
            "\u001b[01;34madversarial_text\u001b[0m/                  \u001b[01;34mmaskgan\u001b[0m/\n",
            "\u001b[01;34madv_imagenet_models\u001b[0m/               \u001b[01;34mnamignizer\u001b[0m/\n",
            "\u001b[01;34mattention_ocr\u001b[0m/                     \u001b[01;34mneural_gpu\u001b[0m/\n",
            "\u001b[01;34maudioset\u001b[0m/                          \u001b[01;34mneural_programmer\u001b[0m/\n",
            "\u001b[01;34mautoaugment\u001b[0m/                       \u001b[01;34mnext_frame_prediction\u001b[0m/\n",
            "\u001b[01;34mautoencoder\u001b[0m/                       \u001b[01;34mnst_blogpost\u001b[0m/\n",
            "\u001b[01;34mbrain_coder\u001b[0m/                       \u001b[01;34mobject_detection\u001b[0m/\n",
            "\u001b[01;34mcognitive_mapping_and_planning\u001b[0m/    \u001b[01;34mpcl_rl\u001b[0m/\n",
            "\u001b[01;34mcognitive_planning\u001b[0m/                \u001b[01;34mpretrained_model\u001b[0m/\n",
            "\u001b[01;34mcompression\u001b[0m/                       \u001b[01;34mptn\u001b[0m/\n",
            "\u001b[01;34mcvt_text\u001b[0m/                          \u001b[01;34mqa_kg\u001b[0m/\n",
            "\u001b[01;34mdeep_contextual_bandits\u001b[0m/           Quant_converted_model_graph.tflite\n",
            "\u001b[01;34mdeeplab\u001b[0m/                           README.md\n",
            "\u001b[01;34mdeep_speech\u001b[0m/                       \u001b[01;34mreal_nvp\u001b[0m/\n",
            "\u001b[01;34mdelf\u001b[0m/                              \u001b[01;34mrebar\u001b[0m/\n",
            "\u001b[01;34mdomain_adaptation\u001b[0m/                 \u001b[01;34msentiment_analysis\u001b[0m/\n",
            "\u001b[01;34mefficient-hrl\u001b[0m/                     \u001b[01;34mseq2species\u001b[0m/\n",
            "\u001b[01;34mfeelvos\u001b[0m/                           setup.py\n",
            "\u001b[01;34mfine_tuned_model\u001b[0m/                  \u001b[01;34mskip_thoughts\u001b[0m/\n",
            "\u001b[01;34mfivo\u001b[0m/                              \u001b[01;34mslim\u001b[0m/\n",
            "\u001b[01;34mglobal_objectives\u001b[0m/                 \u001b[01;34msteve\u001b[0m/\n",
            "\u001b[01;34mim2txt\u001b[0m/                            \u001b[01;34mstreet\u001b[0m/\n",
            "\u001b[01;34minception\u001b[0m/                         \u001b[01;34mstruct2depth\u001b[0m/\n",
            "\u001b[01;34mkeypointnet\u001b[0m/                       \u001b[01;34mswivel\u001b[0m/\n",
            "\u001b[01;34mlearned_optimizer\u001b[0m/                 \u001b[01;34mtcn\u001b[0m/\n",
            "\u001b[01;34mlearning_to_remember_rare_events\u001b[0m/  \u001b[01;34mtextsum\u001b[0m/\n",
            "\u001b[01;34mlearning_unsupervised_learning\u001b[0m/    \u001b[01;34mtraining\u001b[0m/\n",
            "\u001b[01;34mlexnet_nc\u001b[0m/                         \u001b[01;34mtransformer\u001b[0m/\n",
            "\u001b[01;34mlfads\u001b[0m/                             \u001b[01;34mvid2depth\u001b[0m/\n",
            "\u001b[01;34mlm_1b\u001b[0m/                             \u001b[01;34mvideo_prediction\u001b[0m/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JBO3nsdre29K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "files.download('Quant_converted_model_graph.tflite')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cobFSAZNOmv9",
        "colab_type": "text"
      },
      "source": [
        "Compile the model for the EdgeTPU Coral Accelerator"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XmXOUsjIByYI",
        "colab_type": "text"
      },
      "source": [
        "Download the compiler"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "swZLIFNm_DOa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "df96aec2-6a26-47c5-b0c1-62f14be40472"
      },
      "source": [
        "! curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -\n",
        "\n",
        "! echo \"deb https://packages.cloud.google.com/apt coral-edgetpu-stable main\" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list\n",
        "\n",
        "! sudo apt-get update\n",
        "\n",
        "! sudo apt-get install edgetpu-compiler\t"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r100   653  100   653    0     0  17184      0 --:--:-- --:--:-- --:--:-- 17184\n",
            "OK\n",
            "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main\n",
            "Get:1 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ InRelease [3,626 B]\n",
            "Get:2 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ Packages [93.7 kB]\n",
            "Ign:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease\n",
            "Get:4 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]\n",
            "Ign:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease\n",
            "Hit:6 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release\n",
            "Get:7 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release [564 B]\n",
            "Hit:8 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease\n",
            "Get:9 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release.gpg [833 B]\n",
            "Get:11 https://packages.cloud.google.com/apt coral-edgetpu-stable InRelease [6,332 B]\n",
            "Hit:12 http://archive.ubuntu.com/ubuntu bionic InRelease\n",
            "Get:13 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Packages [40.0 kB]\n",
            "Get:14 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 Packages [1,277 B]\n",
            "Get:15 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]\n",
            "Get:16 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB]\n",
            "Get:17 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [863 kB]\n",
            "Get:18 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,842 kB]\n",
            "Get:19 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]\n",
            "Get:20 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [977 kB]\n",
            "Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [13.6 kB]\n",
            "Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,403 kB]\n",
            "Get:23 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [889 kB]\n",
            "Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [101 kB]\n",
            "Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,292 kB]\n",
            "Fetched 7,794 kB in 5s (1,482 kB/s)\n",
            "Reading package lists... Done\n",
            "Reading package lists... Done\n",
            "Building dependency tree       \n",
            "Reading state information... Done\n",
            "The following package was automatically installed and is no longer required:\n",
            "  libnvidia-common-440\n",
            "Use 'sudo apt autoremove' to remove it.\n",
            "The following additional packages will be installed:\n",
            "  libedgetpu1-std\n",
            "The following NEW packages will be installed:\n",
            "  edgetpu-compiler libedgetpu1-std\n",
            "0 upgraded, 2 newly installed, 0 to remove and 43 not upgraded.\n",
            "Need to get 4,998 kB of archives.\n",
            "After this operation, 18.2 MB of additional disk space will be used.\n",
            "Get:1 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 libedgetpu1-std amd64 14.0 [306 kB]\n",
            "Get:2 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 edgetpu-compiler amd64 14.0 [4,692 kB]\n",
            "Fetched 4,998 kB in 0s (22.2 MB/s)\n",
            "debconf: unable to initialize frontend: Dialog\n",
            "debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 2.)\n",
            "debconf: falling back to frontend: Readline\n",
            "debconf: unable to initialize frontend: Readline\n",
            "debconf: (This frontend requires a controlling tty.)\n",
            "debconf: falling back to frontend: Teletype\n",
            "dpkg-preconfigure: unable to re-open stdin: \n",
            "Selecting previously unselected package libedgetpu1-std:amd64.\n",
            "(Reading database ... 144764 files and directories currently installed.)\n",
            "Preparing to unpack .../libedgetpu1-std_14.0_amd64.deb ...\n",
            "Unpacking libedgetpu1-std:amd64 (14.0) ...\n",
            "Selecting previously unselected package edgetpu-compiler.\n",
            "Preparing to unpack .../edgetpu-compiler_14.0_amd64.deb ...\n",
            "Unpacking edgetpu-compiler (14.0) ...\n",
            "Setting up libedgetpu1-std:amd64 (14.0) ...\n",
            "Setting up edgetpu-compiler (14.0) ...\n",
            "Processing triggers for libc-bin (2.27-3ubuntu1) ...\n",
            "/sbin/ldconfig.real: /usr/local/lib/python3.6/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gbk2J1E9BNNf",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 360
        },
        "outputId": "0680e220-93f3-4b22-d05e-0cfc56d48a51"
      },
      "source": [
        "! edgetpu_compiler Quant_converted_model_graph.tflite"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Edge TPU Compiler version 2.1.302470888\n",
            "\n",
            "Model compiled successfully in 747 ms.\n",
            "\n",
            "Input model: Quant_converted_model_graph.tflite\n",
            "Input size: 4.49MiB\n",
            "Output model: Quant_converted_model_graph_edgetpu.tflite\n",
            "Output size: 5.15MiB\n",
            "On-chip memory used for caching model parameters: 5.05MiB\n",
            "On-chip memory remaining for caching model parameters: 2.58MiB\n",
            "Off-chip memory used for streaming uncached model parameters: 0.00B\n",
            "Number of Edge TPU subgraphs: 1\n",
            "Total number of operations: 99\n",
            "Operation log: Quant_converted_model_graph_edgetpu.log\n",
            "\n",
            "Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.\n",
            "Number of operations that will run on Edge TPU: 98\n",
            "Number of operations that will run on CPU: 1\n",
            "See the operation log file for individual operation details.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IqgWz1HxCreP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 595
        },
        "outputId": "691db1b4-2148-47d6-83cb-037b57c16086"
      },
      "source": [
        "%ls"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[0m\u001b[01;34ma3c_blogpost\u001b[0m/                      \u001b[01;34mlstm_object_detection\u001b[0m/\n",
            "\u001b[01;34madversarial_crypto\u001b[0m/                \u001b[01;34mmarco\u001b[0m/\n",
            "\u001b[01;34madversarial_logit_pairing\u001b[0m/         \u001b[01;34mmaskgan\u001b[0m/\n",
            "\u001b[01;34madversarial_text\u001b[0m/                  \u001b[01;34mnamignizer\u001b[0m/\n",
            "\u001b[01;34madv_imagenet_models\u001b[0m/               \u001b[01;34mneural_gpu\u001b[0m/\n",
            "\u001b[01;34mattention_ocr\u001b[0m/                     \u001b[01;34mneural_programmer\u001b[0m/\n",
            "\u001b[01;34maudioset\u001b[0m/                          \u001b[01;34mnext_frame_prediction\u001b[0m/\n",
            "\u001b[01;34mautoaugment\u001b[0m/                       \u001b[01;34mnst_blogpost\u001b[0m/\n",
            "\u001b[01;34mautoencoder\u001b[0m/                       \u001b[01;34mobject_detection\u001b[0m/\n",
            "\u001b[01;34mbrain_coder\u001b[0m/                       \u001b[01;34mpcl_rl\u001b[0m/\n",
            "\u001b[01;34mcognitive_mapping_and_planning\u001b[0m/    \u001b[01;34mpretrained_model\u001b[0m/\n",
            "\u001b[01;34mcognitive_planning\u001b[0m/                \u001b[01;34mptn\u001b[0m/\n",
            "\u001b[01;34mcompression\u001b[0m/                       \u001b[01;34mqa_kg\u001b[0m/\n",
            "\u001b[01;34mcvt_text\u001b[0m/                          Quant_converted_model_graph_edgetpu.log\n",
            "\u001b[01;34mdeep_contextual_bandits\u001b[0m/           Quant_converted_model_graph_edgetpu.tflite\n",
            "\u001b[01;34mdeeplab\u001b[0m/                           Quant_converted_model_graph.tflite\n",
            "\u001b[01;34mdeep_speech\u001b[0m/                       README.md\n",
            "\u001b[01;34mdelf\u001b[0m/                              \u001b[01;34mreal_nvp\u001b[0m/\n",
            "\u001b[01;34mdomain_adaptation\u001b[0m/                 \u001b[01;34mrebar\u001b[0m/\n",
            "\u001b[01;34mefficient-hrl\u001b[0m/                     \u001b[01;34msentiment_analysis\u001b[0m/\n",
            "\u001b[01;34mfeelvos\u001b[0m/                           \u001b[01;34mseq2species\u001b[0m/\n",
            "\u001b[01;34mfine_tuned_model\u001b[0m/                  setup.py\n",
            "\u001b[01;34mfivo\u001b[0m/                              \u001b[01;34mskip_thoughts\u001b[0m/\n",
            "\u001b[01;34mglobal_objectives\u001b[0m/                 \u001b[01;34mslim\u001b[0m/\n",
            "\u001b[01;34mim2txt\u001b[0m/                            \u001b[01;34msteve\u001b[0m/\n",
            "\u001b[01;34minception\u001b[0m/                         \u001b[01;34mstreet\u001b[0m/\n",
            "\u001b[01;34mkeypointnet\u001b[0m/                       \u001b[01;34mstruct2depth\u001b[0m/\n",
            "\u001b[01;34mlearned_optimizer\u001b[0m/                 \u001b[01;34mswivel\u001b[0m/\n",
            "\u001b[01;34mlearning_to_remember_rare_events\u001b[0m/  \u001b[01;34mtcn\u001b[0m/\n",
            "\u001b[01;34mlearning_unsupervised_learning\u001b[0m/    \u001b[01;34mtextsum\u001b[0m/\n",
            "\u001b[01;34mlexnet_nc\u001b[0m/                         \u001b[01;34mtraining\u001b[0m/\n",
            "\u001b[01;34mlfads\u001b[0m/                             \u001b[01;34mtransformer\u001b[0m/\n",
            "\u001b[01;34mlm_1b\u001b[0m/                             \u001b[01;34mvid2depth\u001b[0m/\n",
            "\u001b[01;34mlm_commonsense\u001b[0m/                    \u001b[01;34mvideo_prediction\u001b[0m/\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ffG5vNShKU_G",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ca3acbf5-8a07-4d07-e517-1748caf1e3b7"
      },
      "source": [
        "files.download('Quant_converted_model_graph_edgetpu.tflite')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "\n",
              "    async function download(id, filename, size) {\n",
              "      if (!google.colab.kernel.accessAllowed) {\n",
              "        return;\n",
              "      }\n",
              "      const div = document.createElement('div');\n",
              "      const label = document.createElement('label');\n",
              "      label.textContent = `Downloading \"${filename}\": `;\n",
              "      div.appendChild(label);\n",
              "      const progress = document.createElement('progress');\n",
              "      progress.max = size;\n",
              "      div.appendChild(progress);\n",
              "      document.body.appendChild(div);\n",
              "\n",
              "      const buffers = [];\n",
              "      let downloaded = 0;\n",
              "\n",
              "      const channel = await google.colab.kernel.comms.open(id);\n",
              "      // Send a message to notify the kernel that we're ready.\n",
              "      channel.send({})\n",
              "\n",
              "      for await (const message of channel.messages) {\n",
              "        // Send a message to notify the kernel that we're ready.\n",
              "        channel.send({})\n",
              "        if (message.buffers) {\n",
              "          for (const buffer of message.buffers) {\n",
              "            buffers.push(buffer);\n",
              "            downloaded += buffer.byteLength;\n",
              "            progress.value = downloaded;\n",
              "          }\n",
              "        }\n",
              "      }\n",
              "      const blob = new Blob(buffers, {type: 'application/binary'});\n",
              "      const a = document.createElement('a');\n",
              "      a.href = window.URL.createObjectURL(blob);\n",
              "      a.download = filename;\n",
              "      div.appendChild(a);\n",
              "      a.click();\n",
              "      div.remove();\n",
              "    }\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "download(\"download_81db6955-df7d-41e3-a338-ac8d0e6c10b6\", \"Quant_converted_model_graph_edgetpu.tflite\", 5403600)"
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    }
  ]
}