latin_america_summit.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "latin-america-summit.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AkSGVSxZDrx8",
        "colab_type": "text"
      },
      "source": [
        "Tips:\n",
        "\n",
        "* Enable a GPU in Colab before running this notebook. *Edit -> Notebook settings -> Hardware accelerator -> GPU.*\n",
        "\n",
        "* Should you need to reset your Colab environment to a clean state, use *Runtime -> Factory reset runtime*. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5v5mm4amQRrm",
        "colab_type": "text"
      },
      "source": [
        "# AI Latin America SumMIT: Training and Interpreting Neural Networks\n",
        "\n",
        "Welcome! This notebook contains tutorials and short exercises. If you're new to Deep Learning, this is a *lot* of material to explore in a short workshop. Of course, you're not expected to understand all these concepts in one day. Our goals are to dive in, have fun, and for you to gain experience training a neural network in practice. \n",
        "\n",
        "**Part 1: Training neural networks**\n",
        "\n",
        "* Train a neural network to classify handwritten digits. This is the \"hello world\" of computer vision, and a great place to begin if you're new to the subject.\n",
        "\n",
        "* Train a convolutional neural network to classify images of cats and dogs, using a real-world dataset.\n",
        "\n",
        "**Part 2: Interpreting neural networks**\n",
        "\n",
        "* Next, you'll explore interpretabilty. You will use [LIME](https://github.com/marcotcr/lime), a practical open-source tool that can highlight the regions of that image that led a network to classify an image as a cat.\n",
        "\n",
        "**Instructions**:\n",
        "\n",
        "* Each section below contains a tutorial, then one or more short code blocks for you to complete. You can find sections for you to complete by searching for \"TODO: your code here.\" \n",
        "\n",
        "* Solutions are given below each exercise in a comment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jciXPo_3C3Ct",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%tensorflow_version 2.x\n",
        "import tensorflow as tf\n",
        "print(\"You are using TensorFlow version\", tf.__version__)\n",
        "if len(tf.config.list_physical_devices('GPU')) > 0:\n",
        "  print(\"You have a GPU enabled.\")\n",
        "else:\n",
        "  print(\"Enable a GPU before running this notebook.\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J2esj4IQFIRN",
        "colab_type": "text"
      },
      "source": [
        "Note: Colab has a variety of GPU types available (each new  instance is assigned one randomly, depending on availability). To see which type of GPU you have, you can run ```!nvidia-smi``` in a code cell. Some are quite fast!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SKbWlnHYMbKB",
        "colab_type": "text"
      },
      "source": [
        "# Tutorial 1) MNIST\n",
        "\n",
        "Training an image classifier on the MNIST dataset of handwritten digits is considered the \"hello world\" of computer vision. In this tutorial, you will download the dataset, then train a linear model, a neural network, and a deep neural network to classify it. \n",
        "\n",
        "The code below is based on a longer tutorial on [tensorflow.org](https://www.tensorflow.org/tutorials/keras/classification), which we recommended for future reading."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M6w-q4DDdKRL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow import keras\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gk5WiZoldYxG",
        "colab_type": "text"
      },
      "source": [
        "## Download the MNIST dataset\n",
        "MNIST contains 70,000 grayscale images in 10 categories. The images are low resolution (28 by 28 pixels)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xF6B1mtqdQLq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mnist = keras.datasets.mnist\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cz7X2_VSdoTk",
        "colab_type": "text"
      },
      "source": [
        "There are 60,000 images in the training set:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c6ziZ_hgdVuI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(train_images.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gOCEVIxfdrkm",
        "colab_type": "text"
      },
      "source": [
        "And 10,000 in the testing set:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "161xulFtdtLi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(test_images.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G8tV2hZedwbf",
        "colab_type": "text"
      },
      "source": [
        "Each label is an integer between 0-9:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VNlj_WDqd07Y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(train_labels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GHel6Cn5d7fb",
        "colab_type": "text"
      },
      "source": [
        "## Preprocess the data\n",
        "The pixel values in the images range between 0 and 255. Let's convert them to between 0 and 1 before feeding them to the network. To do so, divide the values by 255. It's important that the training set and the testing set are preprocessed in the same way:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y_rPiwYVeF-X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_images = train_images / 255.0\n",
        "test_images = test_images / 255.0"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z7n2On4UeIHV",
        "colab_type": "text"
      },
      "source": [
        "Let's display the first 25 images from the training set, and display the label below each image.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W7lAScI-eNdR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "    plt.subplot(5,5,i+1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.grid(False)\n",
        "    plt.imshow(train_images[i], cmap=plt.cm.binary)\n",
        "    plt.xlabel(train_labels[i])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ilarSa6DeV0j",
        "colab_type": "text"
      },
      "source": [
        "## Build the model\n",
        "Building the neural network requires configuring the layers of the model, then compiling the model. We will start with a single Dense layer. \n",
        "\n",
        "### Set up the layers\n",
        "\n",
        "The basic building block of a neural network is the layer. Layers extract representations from the data fed into them. For example, the first layer in a network might learn to detect edges (combinations of pixels), and the next layer may learn to detect lines (combinations of edges). Most of deep learning consists of chaining together simple layers. Most layers, such as [tf.keras.layers.Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense), have parameters that are learned during training.\n",
        "\n",
        "### Create a multiclass logistic regression model\n",
        "\n",
        "This model contains a single Dense layer. It is not a neural network yet, it is equivilent to multiclass logistic regression. If you add another layer, it will become a neural network. You will do that in an exercise below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BSHV4al-e_eU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),\n",
        "    keras.layers.Dense(10, activation='softmax')\n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wiH9Th8IfPJ7",
        "colab_type": "text"
      },
      "source": [
        "The first layer in this network, [tf.keras.layers.Flatten](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten), transforms the format of the images from a two-dimensional array (of 28 by 28 pixels) to a one-dimensional array (of 28 * 28 = 784 pixels). Think of this layer as unstacking rows of pixels in the image and lining them up. This layer has no parameters to learn; it only reformats the data. This is necessary since Dense layers require arrays as input.\n",
        "\n",
        "After the pixels are flattened, this model consists of a single Dense layer. This is a densely connected, or fully connected, neural layer. The Dense layer has 10 neurons with softmax activation. This returns an array of 10 probability scores that sum to 1. \n",
        "\n",
        "After classifying an image, each neuron will contains a score that indicates the probability that the current image belongs to one of the 10 classes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PFAUK1K-gFvz",
        "colab_type": "text"
      },
      "source": [
        "## Compile the model\n",
        "\n",
        "Before the model is ready for training, it needs a few more settings. These are added during the model's compile step:\n",
        "\n",
        "*Loss function* — This measures how accurate the model is during training. You want to minimize this function to \"steer\" the model in the right direction.\n",
        "\n",
        "*Optimizer* — This is how the model is updated based on the data it sees and its loss function.\n",
        "\n",
        "*Metrics* — Used to monitor the training and testing steps. The following example uses accuracy, the fraction of the images that are correctly classified."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lKclqoN7gMTD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss='sparse_categorical_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kqV1ZI6ogUJo",
        "colab_type": "text"
      },
      "source": [
        "## Train the model\n",
        "Training the neural network model requires the following steps:\n",
        "\n",
        "1. Feed the training data to the model. In this example, the training data is in the ```train_images``` and ```train_labels``` arrays.\n",
        "\n",
        "1. The model learns to associate images and labels.\n",
        "\n",
        "1. You ask the model to make predictions about a test set—in this example, the ```test_images``` array.\n",
        "\n",
        "1. Verify that the predictions match the labels from the ```test_labels``` array.\n",
        "\n",
        "To begin training, call the ```model.fit``` method — so called because it \"fits\" the model to the training data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sd21x84Gef2S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.fit(train_images, train_labels, epochs=10)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FCttErqEgw4C",
        "colab_type": "text"
      },
      "source": [
        "As the model trains, the loss and accuracy metrics are displayed. This model reaches an accuracy of about 0.90 (or 90%) on the training data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "es55EpWvg1HF",
        "colab_type": "text"
      },
      "source": [
        "## Evaluate accuracy\n",
        "Next, compare how the model performs on the test dataset:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rFoghgfRg4B0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=2)\n",
        "print('\\nTest accuracy:', test_acc)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EZ5-Ej3Tg7cU",
        "colab_type": "text"
      },
      "source": [
        "It turns out that the accuracy on the test dataset is a little less than the accuracy on the training dataset. This gap between training accuracy and test accuracy represents overfitting. Overfitting is when a machine learning model performs worse on new, previously unseen inputs than on the training data. An overfitted model \"memorizes\" the training data—with less accuracy on testing data. \n",
        "\n",
        "You will learn more about how to mitigate overfitting in the next tutorial."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rheqL8L2hBxM",
        "colab_type": "text"
      },
      "source": [
        "## Make predictions\n",
        "With the model trained, you can use it to make predictions about some images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7wIrXBw2hDTC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "predictions = model.predict(test_images)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2DJkopTnhEuP",
        "colab_type": "text"
      },
      "source": [
        "Here, the model has predicted the label for each image in the testing set. Let's take a look at the first prediction:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RtCv-McahE6w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(predictions[0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tlebz80khI_O",
        "colab_type": "text"
      },
      "source": [
        "A prediction is an array of 10 numbers. They represent the model's \"confidence\" that the image corresponds to each of the 10 digits. You can see which label has the highest confidence value:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I7yzszkqhMRF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(tf.argmax(predictions[0]))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ts6YwkHIhTOy",
        "colab_type": "text"
      },
      "source": [
        "# Exercise 1) MNIST\n",
        "\n",
        "In the above tutorial, you used a single Dense layer to reach about 90% accuracy. Now, let's transform your model into a neural network to reach about 95% accuracy. You'll need to add a second Dense layer, then compile and train the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WEQfxITliuBn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),\n",
        "    # TODO: your code here\n",
        "    # Add a Dense layer with 128 neurons and relu activation.\n",
        "    keras.layers.Dense(10, activation='softmax')\n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sNXJ7A2Ti_F5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Next, compile your new model by uncommenting this code \n",
        "# model.compile(optimizer='adam',\n",
        "#              loss='sparse_categorical_crossentropy',\n",
        "#              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lo8FZFFMjGO2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Last, train your model by uncommenting this code\n",
        "# model.fit(train_images, train_labels, epochs=10)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7O04h2fCjPwN",
        "colab_type": "text"
      },
      "source": [
        "Questions to think about:\n",
        "- How does the accuracy of your neural network compare with your first model? \n",
        "- What is the effect of epochs on accuracy? On validation accuracy?\n",
        "- What is the effect of the number of neurons per layer?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RcK_hxuVO9Ja",
        "colab_type": "text"
      },
      "source": [
        "After completing this exercise, you will have trained a neural network. Deep learning is \"code-light, and concept-heavy\". That is, while there's only a few lines you need to write to train a DNN, you will have to spend a good deal of time at home reading and studying all the different concepts involved.\n",
        "\n",
        "Now, let's work with a real-world dataset of thousands of images, that we'll read from disk."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0muBaA0fjFJn",
        "colab_type": "text"
      },
      "source": [
        "## Solution\n",
        "\n",
        "This is a neural network:\n",
        "\n",
        "```\n",
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "```\n",
        "\n",
        "This is a deep neural network:\n",
        "\n",
        "```\n",
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "```\n",
        "\n",
        "This is a deeper neural network =D\n",
        "\n",
        "```\n",
        "model = keras.Sequential([\n",
        "    keras.layers.Flatten(input_shape=(28, 28)),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(128, activation='relu'),\n",
        "    keras.layers.Dense(10, activation='softmax')\n",
        "])\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BQjV6NPRaWYi",
        "colab_type": "text"
      },
      "source": [
        "# Game break 1) Teachable Machine\n",
        "Just for fun, let's take a break from this notebook for 5 mins and try the Teachable Machine\n",
        "\n",
        "https://teachablemachine.withgoogle.com/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V0B8TOWwCsbM",
        "colab_type": "text"
      },
      "source": [
        "# Tutorial 2) Cats and Dogs\n",
        "In this exercise, you will train a convolutional neural network to classify images of cats and dogs."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ob2E2ZIeSbDv",
        "colab_type": "text"
      },
      "source": [
        " ## Download and explore the dataset\n",
        "\n",
        "One nice thing about running this notebook in Colab, although you are downloading large files - you are doing so on Google Cloud Platform, rather than using your local WiFi connection."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6aPzoJnd_8QA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import os"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w7EocqtTHnnc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "origin = 'https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip'\n",
        "path_to_zip = tf.keras.utils.get_file('cats_and_dogs.zip', origin=origin, extract=True)\n",
        "path_to_folder = os.path.join(os.path.dirname(path_to_zip), 'cats_and_dogs_filtered')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IlL8BiAkLn1E",
        "colab_type": "text"
      },
      "source": [
        "The unzipped dataset has the following directory structure:\n",
        "\n",
        "<pre>\n",
        "<b>cats_and_dogs_filtered</b>\n",
        "|__ <b>train</b>\n",
        "    |______ <b>cats</b>: [cat.0.jpg, cat.1.jpg, cat.2.jpg ....]\n",
        "    |______ <b>dogs</b>: [dog.0.jpg, dog.1.jpg, dog.2.jpg ...]\n",
        "|__ <b>validation</b>\n",
        "    |______ <b>cats</b>: [cat.2000.jpg, cat.2001.jpg, cat.2002.jpg ....]\n",
        "    |______ <b>dogs</b>: [dog.2000.jpg, dog.2001.jpg, dog.2002.jpg ...]\n",
        "</pre>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YrgB9RwpShf8",
        "colab_type": "text"
      },
      "source": [
        "Create variables that point to each of these directories."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X4b5Fsw2Lj-P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_dir = os.path.join(path_to_folder, 'train')\n",
        "validation_dir = os.path.join(path_to_folder, 'validation')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fG7DjqVkLlCY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_cats_dir = os.path.join(train_dir, 'cats')\n",
        "train_dogs_dir = os.path.join(train_dir, 'dogs')\n",
        "validation_cats_dir = os.path.join(validation_dir, 'cats')\n",
        "validation_dogs_dir = os.path.join(validation_dir, 'dogs')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kj1dSkUlLs-m",
        "colab_type": "text"
      },
      "source": [
        "Count the number of images in each directory."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dfwja3aILv3N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_cats_tr = len(os.listdir(train_cats_dir))\n",
        "num_dogs_tr = len(os.listdir(train_dogs_dir))\n",
        "\n",
        "num_cats_val = len(os.listdir(validation_cats_dir))\n",
        "num_dogs_val = len(os.listdir(validation_dogs_dir))\n",
        "\n",
        "total_train = num_cats_tr + num_dogs_tr\n",
        "total_val = num_cats_val + num_dogs_val\n",
        "\n",
        "print('Total training cat images:', num_cats_tr)\n",
        "print('Total training dog images:', num_dogs_tr)\n",
        "print('Total validation cat images:', num_cats_val)\n",
        "print('Total validation dog images:', num_dogs_val)\n",
        "print('---')\n",
        "print(\"Total training images:\", total_train)\n",
        "print(\"Total validation images:\", total_val)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u6HxxIkqSsj6",
        "colab_type": "text"
      },
      "source": [
        "Tip: in addition to Python, you can run shell commands in Colab (for example, ```!ls $train_cats_dir```)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8bOJXrOqSuoW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!ls $train_cats_dir "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8e_9iotCAH8E",
        "colab_type": "text"
      },
      "source": [
        "Let's take a look at a couple images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qGVX4U4RTeF6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "_ = plt.imshow(plt.imread(os.path.join(train_cats_dir, \"cat.0.jpg\")))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4PM8Th5UAUOa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "_ = plt.imshow(plt.imread(os.path.join(train_cats_dir, \"cat.1.jpg\")))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P1A2CLadVnQa",
        "colab_type": "text"
      },
      "source": [
        "## Data preprocessing"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p5JfskKZL716",
        "colab_type": "text"
      },
      "source": [
        "We will now need a way to read images from disk, and to format them into appropriately pre-processed floating point arrays before using them to train our network. Specifically, we will need to:\n",
        "\n",
        "- Read the image off disk.\n",
        "- Decode contents of these images and convert them into RGB arrays.\n",
        "- Convert the pixels values from integer to floating point numbers.\n",
        "- Rescale the pixel from values between 0 and 255 to values between 0 and 1 (neural networks work better with small input values - under the hood, each input is multiplied by a weight - large inputs could result in overflow).\n",
        "\n",
        "All these tasks can be done with the `ImageDataGenerator` class provided by `tf.keras`. It can read images from disk and preprocess them into proper arrays."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DbKVMUaBA14w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.preprocessing.image import ImageDataGenerator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tVOP4BOcL57E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# We will resize all images to the same size when they are read of disk.\n",
        "IMG_HEIGHT = 150\n",
        "IMG_WIDTH = 150"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yEYTv3G6L9I2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Rescale the pixel values from 0-255 to 0-1\n",
        "train_image_generator = ImageDataGenerator(rescale=1./255)\n",
        "validation_image_generator = ImageDataGenerator(rescale=1./255)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bmM-kS68L-iY",
        "colab_type": "text"
      },
      "source": [
        "After defining the generators for training and validation images, the `flow_from_directory` method load images from the disk, applies rescaling, and resizes the images into the required dimensions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ff0NXGJEVPWl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "batch_size = 64 # Read a batch of 64 images at each step"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zenaWVaHL_jI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,\n",
        "                                                           directory=train_dir,\n",
        "                                                           shuffle=True,\n",
        "                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                           class_mode='binary')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BbgDJgcUMAeY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "val_data_gen = validation_image_generator.flow_from_directory(batch_size=batch_size,\n",
        "                                                              directory=validation_dir,\n",
        "                                                              target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                              class_mode='binary')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h77qZjn2MFaZ",
        "colab_type": "text"
      },
      "source": [
        "## Display a few images and their labels\n",
        "\n",
        "Next, we will extract a batch of images from the training generator, then plot several of them with `matplotlib`. The `next` function returns a batch from the dataset. The return value of `next` function is in form of `(x_train, y_train)` where x_train is training features and y_train, its labels."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aRkntEZ1MJev",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "image_batch, labels_batch = next(train_data_gen)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zFItkxNsp1EN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# (64, 150, 150, 3) means a list of 64 images, each of which is 150x150x3.\n",
        "# 3 refers to the R,G,B color channels.\n",
        "print(image_batch.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m6sb2tmtqEYE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# (64,) means a list of 64 numbers\n",
        "# each of these will either be 0, or 1\n",
        "print(labels_batch.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z4iqLgMjMMWs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# This function will plot images\n",
        "# in the form of a grid with 1 row and 5 columns\n",
        "def plot_images(images):\n",
        "  fig, axes = plt.subplots(1, 5, figsize=(10,10))\n",
        "  axes = axes.flatten()\n",
        "  for img, ax in zip(images, axes):\n",
        "      ax.imshow(img)\n",
        "      ax.axis('off')\n",
        "  plt.tight_layout()\n",
        "  plt.show() "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LQUpgA1bMOti",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plot_images(image_batch[:5])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vnj3jfpvTlih",
        "colab_type": "text"
      },
      "source": [
        "Let's see how we can retrieve the labels, and which class they correspond to."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fq1rm3jTTa9E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(labels_batch[:5])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VuMd_qHITcxf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(train_data_gen.class_indices)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yukXvCdWMSDj",
        "colab_type": "text"
      },
      "source": [
        "## Create the model\n",
        "Your model will consist of three convolutional blocks followed by max pooling. There's a fully connected layer with 256 units on top. This model will output class probabilities (between 0 and 1) based on the `sigmoid` activation function. If the output is closer to 1, the image will be classified as a dog, otherwise a cat."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DAvbO8Q2BEN4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.layers import Conv2D, Dense, Flatten, MaxPooling2D\n",
        "from tensorflow.keras.models import Sequential"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1dd4nHoIMGy-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = Sequential([\n",
        "    Conv2D(32, 3, padding='same', activation='relu', \n",
        "           input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),\n",
        "    MaxPooling2D(),\n",
        "    Conv2D(32, 3, padding='same', activation='relu'),\n",
        "    MaxPooling2D(),\n",
        "    Conv2D(64, 3, padding='same', activation='relu'),\n",
        "    MaxPooling2D(),\n",
        "    Flatten(),\n",
        "    Dense(256, activation='relu'),\n",
        "    Dense(1, activation='sigmoid')\n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PN8hv9ItMXNr",
        "colab_type": "text"
      },
      "source": [
        "Compile the model, and select the adam optimizer to be used for gradient descent, and binary cross entropy for our loss function (roughly, cross entropy is a way to measure the distance between the prediction we wanted the network to make, and the prediction it made)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sarE21oqMYgC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qB2GwR_EMZwm",
        "colab_type": "text"
      },
      "source": [
        "## Model summary\n",
        "\n",
        "View all the layers of the network using the model's `summary` method:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4mgFrgh-MbL1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UGviR2YtBq_H",
        "colab_type": "text"
      },
      "source": [
        "This model has about 5M parameters to learn. Notice, nearly all of them are in the Dense layer at the bottom!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gx6po3HfMcTu",
        "colab_type": "text"
      },
      "source": [
        "## Train the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UmEm3VVwMeRg",
        "colab_type": "text"
      },
      "source": [
        "Use the `fit` method to train the network. You will train the model for 15 epochs (an epoch is one \"sweep\" over the training set, where each image is used exactly once to perform a round of gradient descent, and update the models parameters). \n",
        "\n",
        "The longer you train the model, the more accurate it will become on the training set (but the more likely it is to overfit, or memorize the training images rather than learn patterns that enable it to perform well on the unseen data in the validation set). Overfitting is a challenge with neural networks and machine learning in general. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U4ggEXVPWcxm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "epochs = 15"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oe5gKJ6eMfNt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "history = model.fit(\n",
        "    train_data_gen,\n",
        "    steps_per_epoch=total_train // batch_size,\n",
        "    epochs=epochs,\n",
        "    validation_data=val_data_gen,\n",
        "    validation_steps=total_val // batch_size\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qKBr3y0bueNY",
        "colab_type": "text"
      },
      "source": [
        "After training, your model is likely 90% accurate on the training set, but probably only about 70% accurate on the validation set."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rIYFh3m9MpvG",
        "colab_type": "text"
      },
      "source": [
        "## Evaluate your model\n",
        "Let's create plots to help us see the difference. Accuracy on the validation data is important: it helps you estimate how well our model is likely to work on new, unseen data in the future. \n",
        "\n",
        "We will create two plots, one for accuracy, and another for loss. Roughly, loss (or error) is the inverse of accuracy (lower is better). Unlike accuracy, loss takes the confidence of a prediction into account (a confidently wrong predicitions has a higher loss than one that is only slightly wrong)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1ZuxB7iMrBG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs_range = range(epochs)\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.plot(epochs_range, acc, label='Training Accuracy')\n",
        "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
        "plt.legend(loc='lower right')\n",
        "plt.title('Training and Validation Accuracy')\n",
        "\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.plot(epochs_range, loss, label='Training Loss')\n",
        "plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
        "plt.legend(loc='upper right')\n",
        "plt.title('Training and Validation Loss')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xeBooB9-bpht",
        "colab_type": "text"
      },
      "source": [
        "When there are a small number of training examples, the model sometimes learns from noises or unwanted details from training examples, to an extent that it negatively impacts the performance of the model on new examples. This phenomenon causes overfitting. It means that the model will have a difficult time generalizing on a new dataset, or making accurate predictions on images of cats and dogs that weren't included in the training set.\n",
        "\n",
        "In the next section, you'll use two techniques to mitigate overfitting: data augmentation and dropout."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G3hfnxDST1we",
        "colab_type": "text"
      },
      "source": [
        "## Tutorial 2.5) Improving accuracy\n",
        "\n",
        "Tip: this part of the tutorial is optional (it will take a few mins to train the model). If you're running tight on workshop time, now is a good time to start exercise 2, and you can finish this part of the tutorial at home."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DFrRJObuMv8d",
        "colab_type": "text"
      },
      "source": [
        "## Data augmentation\n",
        "Overfitting generally occurs when there are a small number of training examples. One way to fix this problem is to augment the dataset so that it has a larger number of training examples. Data augmentation generates more training data from existing training samples by applying random transformations (for example, rotation) that yield believable-looking images. With data augmentation, the model will never see the exact same picture twice during training. This helps expose the model to more aspects of the data, and can lead to better generalization.\n",
        "\n",
        "You can implement this using the ImageDataGenerator. Specifiy different transformations to the dataset and it will take care of applying it during the training process."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oIWUljgzb5dw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "image_gen_train = ImageDataGenerator(\n",
        "                    rescale=1./255,\n",
        "                    rotation_range=45,\n",
        "                    width_shift_range=.15,\n",
        "                    height_shift_range=.15,\n",
        "                    horizontal_flip=True,\n",
        "                    zoom_range=0.5\n",
        "                    )"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pLl4-mjKb-vo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data_gen = image_gen_train.flow_from_directory(batch_size=batch_size,\n",
        "                                                     directory=train_dir,\n",
        "                                                     shuffle=True,\n",
        "                                                     target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                     class_mode='binary')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BtLNwy9_b_RF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Show how the same image appears with different data augmentation\n",
        "augmented_images = [train_data_gen[0][0][0] for i in range(5)]\n",
        "plot_images(augmented_images)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OsKX1NNpchmM",
        "colab_type": "text"
      },
      "source": [
        "We only apply data augmentation to the training examples."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZZjPRpJkchwU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "image_gen_val = ImageDataGenerator(rescale=1./255)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TQIyYxvAcknP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "val_data_gen = image_gen_val.flow_from_directory(batch_size=batch_size,\n",
        "                                                 directory=validation_dir,\n",
        "                                                 target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                 class_mode='binary')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-s-DCjFcm1N",
        "colab_type": "text"
      },
      "source": [
        "## Dropout"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K34kcZfzco5M",
        "colab_type": "text"
      },
      "source": [
        "Another technique to reduce overfitting is to introduce dropout to the network. Dropout is a form of regularization that makes it more difficult for the network to memorize rare details (instead, it is forced to learn more general patterns).\n",
        "\n",
        "When you apply dropout to a layer it randomly drops out (set to zero) a number of activations during training. Dropout takes a fractional number as its input value, in the form such as 0.1, 0.2, 0.4, etc. This means dropping out 10%, 20% or 40% of the output units randomly from the applied layer.\n",
        "\n",
        "When appling 0.1 dropout to a certain layer, it randomly deactivates 10% of the output units in each training epoch.\n",
        "\n",
        "Create a new network architecture using Dropout."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UW2NXkclDxo6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.layers import Dropout"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s3e7ir-HcnZ7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = Sequential([\n",
        "    Conv2D(16, 3, padding='same', activation='relu', \n",
        "           input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),\n",
        "    MaxPooling2D(),\n",
        "    Dropout(0.2),\n",
        "    Conv2D(32, 3, padding='same', activation='relu'),\n",
        "    MaxPooling2D(),\n",
        "    Conv2D(64, 3, padding='same', activation='relu'),\n",
        "    MaxPooling2D(),\n",
        "    Dropout(0.2),\n",
        "    Flatten(),\n",
        "    Dense(256, activation='relu'),\n",
        "    Dense(1, activation='sigmoid')\n",
        "])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AMlmQwrwcu4V",
        "colab_type": "text"
      },
      "source": [
        "After introducing dropout to the network, compile the model and view the layers summary."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L77Pt-WFcxYz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model.compile(optimizer='adam',\n",
        "              loss='binary_crossentropy',\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "model.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XEMGlbwWc254",
        "colab_type": "text"
      },
      "source": [
        "## Train your new model\n",
        "This model will need to be trained for longer (more epochs) to achieve high accuracy. As this will take some time, here we'll train for only 15 epochs so we can move on to part two of this notebook. If you like, you can continue training at home. See if your new model can achieve higher validation accuracy than your original one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-jLoyPTRc2Va",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "history = model.fit(\n",
        "    train_data_gen,\n",
        "    steps_per_epoch=total_train // batch_size,\n",
        "    epochs=epochs,\n",
        "    validation_data=val_data_gen,\n",
        "    validation_steps=total_val // batch_size\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J7ktPjfbdCoC",
        "colab_type": "text"
      },
      "source": [
        "## Evaluate your new model\n",
        "Create plots of accuracy and loss. If you compare them with your previous plots, the new model should show less overfitting than the original one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Ma9lDmidMWs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "acc = history.history['accuracy']\n",
        "val_acc = history.history['val_accuracy']\n",
        "\n",
        "loss = history.history['loss']\n",
        "val_loss = history.history['val_loss']\n",
        "\n",
        "epochs_range = range(epochs)\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.plot(epochs_range, acc, label='Training Accuracy')\n",
        "plt.plot(epochs_range, val_acc, label='Validation Accuracy')\n",
        "plt.legend(loc='lower right')\n",
        "plt.title('Training and Validation Accuracy')\n",
        "\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.plot(epochs_range, loss, label='Training Loss')\n",
        "plt.plot(epochs_range, val_loss, label='Validation Loss')\n",
        "plt.legend(loc='upper right')\n",
        "plt.title('Training and Validation Loss')\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zvojYaquePbo",
        "colab_type": "text"
      },
      "source": [
        "You can now train your new model for longer (by increasing the number of epochs) and reach higher validation accuracy. As this will take some time to train, we will leave this as an exercise for you to explore at home, and continue on to part two."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lUU_gVFWKbXp",
        "colab_type": "text"
      },
      "source": [
        "# Exercise 2) Flowers\n",
        "\n",
        "In this exercise, you implement a CNN and use it to classify five different types of flowers (sunflowers, tulips, and so on) from a real-world dataset of images. You will download the dataset, read and preprocess the images using ImageDataGenerator, then train and evaluate a model. \n",
        "\n",
        "Several sections of the code are written for you, and there are parts for you to complete, using the same pattern as tutorial 2."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7HTIqYg8UkAO",
        "colab_type": "text"
      },
      "source": [
        "### Download the dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FlFiGskKKhI0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "origin = 'https://storage.googleapis.com/tensorflow-blog/datasets/mini_flowers.zip'\n",
        "path_to_zip = tf.keras.utils.get_file('mini_flowers.zip', origin=origin, extract=True)\n",
        "path_to_folder = os.path.join(os.path.dirname(path_to_zip))\n",
        "\n",
        "train_dir = os.path.join(path_to_folder, \"train/\")\n",
        "tedst_dir = os.path.join(path_to_folder, \"test/\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nLLaKHvLUm5e",
        "colab_type": "text"
      },
      "source": [
        "### Read the images off disk"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ywq7Sd-sXor-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_image_generator = ImageDataGenerator(rescale=1./255)\n",
        "validation_image_generator = ImageDataGenerator(rescale=1./255)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T4Gv8X_LXsRJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data_gen = train_image_generator.flow_from_directory(batch_size=batch_size,\n",
        "                                                           directory=train_dir,\n",
        "                                                           shuffle=True,\n",
        "                                                           target_size=(IMG_HEIGHT, IMG_WIDTH),\n",
        "                                                           class_mode='categorical')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bGxsPRMuUqZ9",
        "colab_type": "text"
      },
      "source": [
        "### Plot images and their labels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hWxGwHvzX7LF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "image_batch, labels_batch = next(train_data_gen)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vkL2crA7X7ym",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "    plt.subplot(5,5,i+1)\n",
        "    plt.xticks([])\n",
        "    plt.yticks([])\n",
        "    plt.grid(False)\n",
        "    plt.imshow(image_batch[i])\n",
        "    plt.xlabel(str(labels_batch[i]))\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TeZMOVlAfpAh",
        "colab_type": "text"
      },
      "source": [
        "Notice the labels are in one-hot format. Let's add some code to display the class names."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UT8PNcMiZmWI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(train_data_gen.class_indices)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WVsC1ryVaUz9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class_names = {v:k for k,v in train_data_gen.class_indices.items()}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uWTIFZ9EZz24",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plt.figure(figsize=(10,10))\n",
        "for i in range(25):\n",
        "  plt.subplot(5,5,i+1)\n",
        "  plt.xticks([])\n",
        "  plt.yticks([])\n",
        "  plt.grid(False)\n",
        "  plt.imshow(image_batch[i])\n",
        "  plt.xlabel(class_names[tf.argmax(labels_batch[i]).numpy()])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r7liWGNJVdpv",
        "colab_type": "text"
      },
      "source": [
        "### Read the test images"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EfgmDWPWVR5S",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Above, you created a DataGenerator for the training set\n",
        "# Next, create one to read the testing images using similar code\n",
        "# from the tutorial above\n",
        "\n",
        "# Important: just like in your training data generator\n",
        "# use ```class_mode='categorical'```\n",
        "\n",
        "# TODO: your code here"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ejPEgXR9f7hH",
        "colab_type": "text"
      },
      "source": [
        "## Implement a CNN"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lTCVcwDuftS1",
        "colab_type": "text"
      },
      "source": [
        "Now, it's time to define your model. You should create a similar model to the CNN used in the first part of the tutorial above.\n",
        "\n",
        "The only difference is that the final Dense layer of your model (which classifies the data based on the features provided by the convolutional base) must use softmax activation and have five output classes:\n",
        "\n",
        "```model.add(Dense(5, activation='softmax'))```\n",
        "\n",
        "This is because we now have five different types of flowers, instead of just cats and dogs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yClBCWAIYbDv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# TODO: your code here\n",
        "# Define a CNN using code similar to the above "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qceXRe7vYksG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# After you have defined your model, compile it model by \n",
        "# uncommenting this code\n",
        "\n",
        "# Important: notice that the loss has changed to ```categorical_crossentropy```\n",
        "# This is necessary because the labels are in one-hot format, as we saw above\n",
        "\n",
        "# Tip: although these loss functions sound complicated, there are only \n",
        "# a handful for you to learn.\n",
        "\n",
        "#model.compile(optimizer='adam',\n",
        "#              loss='categorical_crossentropy',\n",
        "#              metrics=['accuracy'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6d_hAUd0VI-I",
        "colab_type": "text"
      },
      "source": [
        "Finally, train your model using ```model.fit```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kn1EbFRnYmwA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# TODO: your code here"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9G4sDsafVer",
        "colab_type": "text"
      },
      "source": [
        "## Solution\n",
        "\n",
        "```\n",
        "# Define a model\n",
        "model = Sequential()\n",
        "model.add(Conv2D(32, (3, 3), activation='relu', \n",
        "                        input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)))\n",
        "model.add(MaxPooling2D())\n",
        "model.add(Conv2D(32, (3, 3), activation='relu'))\n",
        "model.add(MaxPooling2D())\n",
        "model.add(Conv2D(32, (3, 3), activation='relu'))\n",
        "model.add(MaxPooling2D())\n",
        "\n",
        "model.add(Flatten())\n",
        "model.add(Dense(128, activation='relu'))\n",
        "model.add(Dense(5, activation='softmax'))\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R2Tk_WFuauxw",
        "colab_type": "text"
      },
      "source": [
        "# Game break 2) Quick, Draw!\n",
        "Let's take a break from this notebook for 5 mins and try Quick, Draw!\n",
        "\n",
        "https://quickdraw.withgoogle.com/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Ua74E7d6U4t",
        "colab_type": "text"
      },
      "source": [
        "# Tutorial 3) Use LIME to explain a model\n",
        "\n",
        "Above, you gained experience training CNNs to classify real-world data. Next, you'll use an open-source tool to interpret \"why\" they classified images as they did. Note that in this section of the notebook, we're using an existing tool. In later tutorials, you will see how to write code from scratch to begin exploring related concepts. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QQLElSbf_Vih",
        "colab_type": "text"
      },
      "source": [
        "## Download and display an image"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S_YgD2mR_aVo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Feel free to replace with your own URL\n",
        "!wget https://storage.googleapis.com/applied-dl/cat-and-dog.jpg -O my-image.jpg"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JWhsj6GJ_fuZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "img = plt.imread(\"my-image.jpg\")\n",
        "_ = plt.imshow(img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kBaTY8uV_34u",
        "colab_type": "text"
      },
      "source": [
        "## Classify it with a pretrained CNN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mbhM4kGG-5Af",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from tensorflow.keras.applications.mobilenet_v2 import MobileNetV2, decode_predictions, preprocess_input\n",
        "from tensorflow.keras.preprocessing import image"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EMY-3LfN-6kH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mobilenet = MobileNetV2(weights='imagenet')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dneWzx54_AyE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mobilenet.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WCIL7FmF_Bkf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def preprocess(img_path):\n",
        "  # Use MobileNet's built-in preprocessing function\n",
        "  # This will scale the pixel values to between -1,1\n",
        "  # and add it to a batch (a list of lists) \n",
        "  # so it can be classified by the model\n",
        "  img = image.load_img(img_path, target_size=(224, 224))\n",
        "  img = image.img_to_array(img)\n",
        "  img = tf.expand_dims(img, axis=0)\n",
        "  img = preprocess_input(img)\n",
        "  return img"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W48U4mOpBBiX",
        "colab_type": "text"
      },
      "source": [
        "Here, you can see the three most likely predictions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c67aXA5L_J2b",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preprocessed_img = preprocess(\"my-image.jpg\")\n",
        "preds = mobilenet.predict(preprocessed_img)\n",
        "print('Predicted:', decode_predictions(preds, top=3)[0])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zAvabHqsBOrA",
        "colab_type": "text"
      },
      "source": [
        "## Install LIME"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_Oby_WJBGbT",
        "colab_type": "text"
      },
      "source": [
        "Next, we will ask the question: \"Which parts of the image led the classifier to believe that the image was of a Bernese mountain dog?\" To do that, we will use [LIME](https://github.com/marcotcr/lime)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vtkub-q5rD_V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install lime"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K5pERiNPOhjj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import lime\n",
        "\n",
        "from lime import lime_image"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uMDXOyoerj0m",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "explainer = lime_image.LimeImageExplainer()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ItZ8S6serj3i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%time\n",
        "# Hide color is the color for a superpixel turned OFF. \n",
        "# Alternatively, if it is NONE, the superpixel will be replaced by the average of its pixels\n",
        "explanation = explainer.explain_instance(preprocessed_img[0].numpy(), \n",
        "                                         mobilenet.predict, \n",
        "                                         top_labels=5, \n",
        "                                         hide_color=0, \n",
        "                                         num_samples=1000)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6qJoRt5kRND7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from skimage.segmentation import mark_boundaries"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Op4TILfiRffr",
        "colab_type": "text"
      },
      "source": [
        "## See the explanations\n",
        "We can see the top 5 superpixels that are most positive towards the class with the rest of the image hidden."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H0SzAchgRco8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], positive_only=True, num_features=5, hide_rest=True)\n",
        "# Rescale pixels to be between 0-1 so they display properly\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ag4yoynVRkN3",
        "colab_type": "text"
      },
      "source": [
        "Or with the rest of the image present:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "W7eT_unsRmdz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], \n",
        "                                            positive_only=True, \n",
        "                                            num_features=5, \n",
        "                                            hide_rest=False)\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evrOdUluRom1",
        "colab_type": "text"
      },
      "source": [
        "We can also see the 'pros and cons' (pros in green, cons in red):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jBYRECzDRqCd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], \n",
        "                                            positive_only=False, \n",
        "                                            num_features=10, \n",
        "                                            hide_rest=False)\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xUhYSty1Rr3b",
        "colab_type": "text"
      },
      "source": [
        "Or the pros and cons that have weight at least 0.1:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pHneObuDRtdd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], \n",
        "                                            positive_only=False, \n",
        "                                            num_features=1000, \n",
        "                                            hide_rest=False, \n",
        "                                            min_weight=0.1)\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VHorsrSgRw2m",
        "colab_type": "text"
      },
      "source": [
        "Let's see the explanation for the second highest prediction:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FVfOQ8vKRxn6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[1],\n",
        "                                            positive_only=True, \n",
        "                                            num_features=5, \n",
        "                                            hide_rest=True)\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vbh2h8JWR91s",
        "colab_type": "text"
      },
      "source": [
        "Pros and cons:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wo-grs0qR-hR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "temp, mask = explanation.get_image_and_mask(explanation.top_labels[1],\n",
        "                                            positive_only=False, \n",
        "                                            num_features=10, \n",
        "                                            hide_rest=False)\n",
        "plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2fcMFdPiKoAr",
        "colab_type": "text"
      },
      "source": [
        "# Exercise 3) Try LIME on your own images"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5cywaiBFG_7L",
        "colab_type": "text"
      },
      "source": [
        "Let's try LIME on your own images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bz2bH48iKqT9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 1) Use wget to download an image from the web\n",
        "# !wget image_url -O my-image.jpg"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pQ03FBj8HeKj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 2) Read your image from disk as above, using plt.imread\n",
        "#    then display it, using plt.imshow\n",
        "# e.g., img = plt.imread..."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4O9eU3OxHqlK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 3) Create a preprocessed version of your image by \n",
        "#    following the pattern above\n",
        "# e.g., preprocessed_img = preprocess..."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nDfvE_eAH6gv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 4) Classify your preprocessed image with MobileNet\n",
        "# e.g., predictions = mobilenet..."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qwxe0-VFINmt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 5) Create an explanation using LIME\n",
        "# e.g., explanation = explainer.explain_instance..."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7qydrANuJC3R",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 6) Display the pros and cons.\n",
        "# e.g., temp, mask = explanation.get_image_and_mask(\n",
        "# then\n",
        "# plt.imshow(mark_boundaries(temp / 2 + 0.5, mask))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pQj5Y6-7Ck0S",
        "colab_type": "text"
      },
      "source": [
        "You can learn more about LIME (and how to use it for other types of models, including text classifiers) at the [homepage](https://github.com/marcotcr/lime). From here, if you'd like to learn how to write code from scratch to explore features neural networks learn to detect, we recommend checking out Deep Dream! https://www.tensorflow.org/tutorials/generative/deepdream"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JztBOYqZ6yTm",
        "colab_type": "text"
      },
      "source": [
        "# Learning more"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lfSulmXVQyJp",
        "colab_type": "text"
      },
      "source": [
        "To learn more about TensorFlow and Machine Learning in general, I recommend the book Hands-on Machine Learning with Scikit-Learn, Keras and TensorFlow ([GitHub](https://github.com/ageron/handson-ml2), [Website](https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/)). Be sure to get the 2nd edition (which teaches the latest version of TensorFlow).\n",
        "\n",
        "You can also stay in touch with the latest TensorFlow developments by:\n",
        "- Reading our blog http://blog.tensorflow.org/\n",
        "- Watching our YouTube channel https://www.youtube.com/tensorflow.\n",
        "- And following our Twitter account: https://twitter.com/tensorflow\n",
        "\n",
        "Thank you!"
      ]
    }
  ]
}