Neural_Nets_Project_Starter_Code-Final_GoutamDutta.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gesture Recognition\n",
    "In this group project, you are going to build a 3D Conv model that will be able to predict the 5 gestures correctly. Please import the following libraries to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import os\n",
    "from scipy.misc import imread, imresize\n",
    "import datetime\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set the random seed so that the results don't vary drastically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(30)\n",
    "import random as rn\n",
    "rn.seed(30)\n",
    "from keras import backend as K\n",
    "import tensorflow as tf\n",
    "tf.set_random_seed(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this block, you read the folder names for training and validation. You also set the `batch_size` here. Note that you set the batch size in such a way that you are able to use the GPU in full capacity. You keep increasing the batch size until the machine throws an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_doc = np.random.permutation(open('/storage/Demo_Gesture/Project_data/train.csv').readlines())\n",
    "val_doc = np.random.permutation(open('/storage/Demo_Gesture/Project_data/val.csv').readlines())\n",
    "batch_size = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generator\n",
    "This is one of the most important part of the code. The overall structure of the generator has been given. In the generator, you are going to preprocess the images as you have images of 2 different dimensions as well as create a batch of video frames. You have to experiment with `img_idx`, `y`,`z` and normalization such that you get high accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height\n",
    "\n",
    "def generator(source_path, folder_list, batch_size):\n",
    "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
    "    img_idx = [x for x in range(0,x)] #create a list of image numbers you want to use for a particular video\n",
    "    while True:\n",
    "        t = np.random.permutation(folder_list)\n",
    "        num_batches = len(folder_list)//batch_size # calculate the number of batches\n",
    "        for batch in range(num_batches): # we iterate over the number of batches\n",
    "            batch_data = np.zeros((batch_size,x,y,z,3)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
    "            batch_labels = np.zeros((batch_size,5)) # batch_labels is the one hot representation of the output\n",
    "            for folder in range(batch_size): # iterate over the batch_size\n",
    "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
    "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
    "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
    "                    \n",
    "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
    "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
    "                    \n",
    "                    temp = imresize(image,(120,120))\n",
    "                    temp = temp/127.5-1 #Normalize data\n",
    "                    \n",
    "                    batch_data[folder,idx,:,:,0] = (temp[:,:,0]) #normalise and feed in the image\n",
    "                    batch_data[folder,idx,:,:,1] = (temp[:,:,1]) #normalise and feed in the image\n",
    "                    batch_data[folder,idx,:,:,2] = (temp[:,:,2]) #normalise and feed in the image\n",
    "                    \n",
    "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
    "            yield batch_data, batch_labels #you yield the batch_data and the batch_labels, remember what does yield do\n",
    "\n",
    "        \n",
    "        # write the code for the remaining data points which are left after full batches\n",
    "        if (len(folder_list) != batch_size*num_batches):\n",
    "            print(\"Batch: \",num_batches+1,\"Index:\", batch_size)\n",
    "            batch_size = len(folder_list) - (batch_size*num_batches)\n",
    "            batch_data = np.zeros((batch_size,x,y,z,3)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
    "            batch_labels = np.zeros((batch_size,5)) # batch_labels is the one hot representation of the output\n",
    "            for folder in range(batch_size): # iterate over the batch_size\n",
    "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
    "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
    "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
    "                    \n",
    "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
    "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
    "                    temp = imresize(image,(120,120))\n",
    "                    temp = temp/127.5-1 #Normalize data\n",
    "                    \n",
    "                    batch_data[folder,idx,:,:,0] = (temp[:,:,0])\n",
    "                    batch_data[folder,idx,:,:,1] = (temp[:,:,1])\n",
    "                    batch_data[folder,idx,:,:,2] = (temp[:,:,2])\n",
    "                   \n",
    "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
    "            yield batch_data, batch_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note here that a video is represented above in the generator as (number of images, height, width, number of channels). Take this into consideration while creating the model architecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# training sequences = 663\n",
      "# validation sequences = 100\n",
      "# epochs = 10\n"
     ]
    }
   ],
   "source": [
    "curr_dt_time = datetime.datetime.now()\n",
    "train_path = '/storage/Demo_Gesture/Project_data/train'\n",
    "val_path = '/storage/Demo_Gesture/Project_data/val'\n",
    "num_train_sequences = len(train_doc)\n",
    "print('# training sequences =', num_train_sequences)\n",
    "num_val_sequences = len(val_doc)\n",
    "print('# validation sequences =', num_val_sequences)\n",
    "num_epochs = 10 # choose the number of epochs\n",
    "print ('# epochs =', num_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model\n",
    "Here you make the model using different functionalities that Keras provides. Remember to use `Conv3D` and `MaxPooling3D` and not `Conv2D` and `Maxpooling2D` for a 3D convolution model. You would want to use `TimeDistributed` while building a Conv2D + RNN model. Also remember that the last layer is the softmax. Design the network in such a way that the model is able to give good accuracy on the least number of parameters so that it can fit in the memory of the webcam."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential, Model\n",
    "from keras.layers import Dense, GRU, Flatten, TimeDistributed, Flatten, BatchNormalization, Activation, Dropout\n",
    "from keras.layers.convolutional import Conv3D, MaxPooling3D\n",
    "from keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n",
    "from keras import optimizers\n",
    "import keras\n",
    "\n",
    "#write your model here\n",
    "#model a\n",
    "model_a = Sequential()\n",
    "\n",
    "model_a.add(Conv3D(8, #number of filters \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=(30, 120, 120, 3),\n",
    "                 padding='same'))\n",
    "model_a.add(BatchNormalization())\n",
    "model_a.add(Activation('relu'))\n",
    "\n",
    "model_a.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_a.add(Conv3D(16, #Number of filters, \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_a.add(BatchNormalization())\n",
    "model_a.add(Activation('relu'))\n",
    "\n",
    "model_a.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_a.add(Conv3D(32, #Number of filters \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_a.add(BatchNormalization())\n",
    "model_a.add(Activation('relu'))\n",
    "\n",
    "model_a.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_a.add(Conv3D(64, #Number pf filters \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_a.add(BatchNormalization())\n",
    "model_a.add(Activation('relu'))\n",
    "\n",
    "model_a.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_a.add(Flatten())\n",
    "\n",
    "model_a.add(Dense(1000, activation='relu'))\n",
    "model_a.add(Dropout(0.5))\n",
    "\n",
    "model_a.add(Dense(500, activation='relu'))\n",
    "model_a.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_a.add(Dense(5, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have written the model, the next step is to `compile` the model. When you print the `summary` of the model, you'll see the total number of parameters you have to train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_1 (Conv3D)            (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_1 (Batch (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_1 (MaxPooling3 (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_2 (Conv3D)            (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_2 (Batch (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_2 (MaxPooling3 (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_3 (Conv3D)            (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_3 (Batch (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_3 (MaxPooling3 (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_4 (Conv3D)            (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_4 (Batch (None, 3, 15, 15, 64)     256       \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_4 (MaxPooling3 (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,749\n",
      "Trainable params: 3,667,509\n",
      "Non-trainable params: 240\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "optimiser = optimizers.Adam(lr=0.001) #write your optimizer\n",
    "model_a.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "print (model_a.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us create the `train_generator` and the `val_generator` which will be used in `.fit_generator`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = generator(train_path, train_doc, batch_size)\n",
    "val_generator = generator(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'model_init' + '_' + str(curr_dt_time).replace(' ','').replace(':','_') + '/'\n",
    "    \n",
    "if not os.path.exists(model_name):\n",
    "    os.mkdir(model_name)\n",
    "        \n",
    "filepath = model_name + 'model-{epoch:05d}-{loss:.5f}-{categorical_accuracy:.5f}-{val_loss:.5f}-{val_categorical_accuracy:.5f}.h5'\n",
    "\n",
    "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto', period=1)\n",
    "\n",
    "LR = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=2, cooldown=1, verbose=1) # write the REducelronplateau code here\n",
    "callbacks_list = [checkpoint, LR]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `steps_per_epoch` and `validation_steps` are used by `fit_generator` to decide the number of next() calls it need to make."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (num_train_sequences%batch_size) == 0:\n",
    "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
    "else:\n",
    "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
    "\n",
    "if (num_val_sequences%batch_size) == 0:\n",
    "    validation_steps = int(num_val_sequences/batch_size)\n",
    "else:\n",
    "    validation_steps = (num_val_sequences//batch_size) + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now fit the model. This will start training the model and with the help of the checkpoints, you'll be able to save the model at the end of each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:17: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:22: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 7.0193 - categorical_accuracy: 0.2985Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 6.9996 - categorical_accuracy: 0.3000"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:42: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:46: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 94s 1s/step - loss: 6.9210 - categorical_accuracy: 0.3005 - val_loss: 8.7402 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-6.97551-0.30015-8.74023-0.30000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 28s 416ms/step - loss: 4.8372 - categorical_accuracy: 0.3134 - val_loss: 8.8919 - val_categorical_accuracy: 0.2400\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-4.83719-0.31343-8.89195-0.24000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 30s 442ms/step - loss: 2.8366 - categorical_accuracy: 0.2836 - val_loss: 2.1485 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-2.83657-0.28358-2.14845-0.30000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 29s 438ms/step - loss: 1.8071 - categorical_accuracy: 0.2836 - val_loss: 1.3704 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-1.80707-0.28358-1.37041-0.44000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 31s 458ms/step - loss: 1.5281 - categorical_accuracy: 0.3333 - val_loss: 1.3751 - val_categorical_accuracy: 0.4300\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-1.52811-0.33333-1.37514-0.43000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 30s 451ms/step - loss: 1.5627 - categorical_accuracy: 0.3383 - val_loss: 1.3685 - val_categorical_accuracy: 0.4100\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-1.56269-0.33831-1.36845-0.41000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 29s 432ms/step - loss: 1.5006 - categorical_accuracy: 0.3184 - val_loss: 1.2981 - val_categorical_accuracy: 0.3900\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-1.50062-0.31841-1.29806-0.39000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 28s 415ms/step - loss: 1.4152 - categorical_accuracy: 0.3881 - val_loss: 1.3751 - val_categorical_accuracy: 0.2900\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-1.41518-0.38806-1.37514-0.29000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 29s 434ms/step - loss: 1.5383 - categorical_accuracy: 0.3682 - val_loss: 1.2616 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-1.53827-0.36816-1.26156-0.54000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 29s 434ms/step - loss: 1.4350 - categorical_accuracy: 0.3831 - val_loss: 1.3229 - val_categorical_accuracy: 0.4200\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-1.43497-0.38308-1.32285-0.42000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f533a113160>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_a.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "classes = 5 #left swipe, right swipe, thumbs up, thumbs down, stop\n",
    "channel = 3\n",
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height\n",
    "\n",
    "def generator_ex(source_path, folder_list, batch_size):\n",
    "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
    "    img_idx = [x for x in range(0,x)] #create a list of image numbers you want to use for a particular video\n",
    "    while True:\n",
    "        t = np.random.permutation(folder_list)\n",
    "        num_batches = len(folder_list)//batch_size # calculate the number of batches\n",
    "        for batch in range(num_batches): # we iterate over the number of batches\n",
    "            batch_data = np.zeros((batch_size,x,y,z,channel)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
    "            batch_labels = np.zeros((batch_size,classes)) # batch_labels is the one hot representation of the output\n",
    "            for folder in range(batch_size): # iterate over the batch_size\n",
    "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
    "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
    "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
    "                    \n",
    "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
    "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
    "                    \n",
    "                    temp = imresize(image,(y,z))\n",
    "                    #Converting to gray scale\n",
    "                    temp = temp.mean(axis=-1,keepdims=1) \n",
    "                    temp = temp/127.5-1 #Normalize data\n",
    "                    batch_data[folder,idx] = temp #normalise and feed in the image\n",
    "                    \n",
    "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
    "                \n",
    "            yield batch_data, batch_labels #you yield the batch_data and the batch_labels, remember what does yield do\n",
    "\n",
    "        \n",
    "        # write the code for the remaining data points which are left after full batches\n",
    "        if (len(folder_list) != batch_size*num_batches):\n",
    "            print(\"Batch: \",num_batches+1,\"Index:\", batch_size)\n",
    "            batch_size = len(folder_list) - (batch_size*num_batches)\n",
    "            batch_data = np.zeros((batch_size,x,y,z,channel)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
    "            batch_labels = np.zeros((batch_size,classes)) # batch_labels is the one hot representation of the output\n",
    "            for folder in range(batch_size): # iterate over the batch_size\n",
    "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
    "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
    "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
    "                    \n",
    "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
    "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
    "                    temp = imresize(image,(y,z))\n",
    "                    #Converting to gray scale\n",
    "                    temp = temp.mean(axis=-1,keepdims=1) \n",
    "                    temp = temp/127.5-1 #Normalize data\n",
    "                    \n",
    "                    batch_data[folder,idx] = temp\n",
    "                   \n",
    "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
    "            yield batch_data, batch_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:22: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:17: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_5 (Conv3D)            (None, 30, 120, 120, 32)  2624      \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 30, 120, 120, 32)  0         \n",
      "_________________________________________________________________\n",
      "conv3d_6 (Conv3D)            (None, 30, 120, 120, 32)  27680     \n",
      "_________________________________________________________________\n",
      "activation_6 (Activation)    (None, 30, 120, 120, 32)  0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_5 (MaxPooling3 (None, 10, 40, 40, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 10, 40, 40, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_7 (Conv3D)            (None, 10, 40, 40, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_7 (Activation)    (None, 10, 40, 40, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_8 (Conv3D)            (None, 10, 40, 40, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_8 (Activation)    (None, 10, 40, 40, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_6 (MaxPooling3 (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 50176)             0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 512)               25690624  \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 25,889,509\n",
      "Trainable params: 25,889,509\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.losses import categorical_crossentropy\n",
    "from keras.optimizers import Adam\n",
    "\n",
    "# Define model b\n",
    "model_b = Sequential()\n",
    "model_b.add(Conv3D(32, kernel_size=(3, 3, 3), input_shape=(x,y,z,channel), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(Conv3D(32, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(MaxPooling3D(pool_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Dropout(0.25))\n",
    "\n",
    "model_b.add(Conv3D(64, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(Conv3D(64, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(MaxPooling3D(pool_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Dropout(0.25))\n",
    "\n",
    "model_b.add(Flatten())\n",
    "model_b.add(Dense(512, activation='relu'))\n",
    "model_b.add(Dropout(0.5))\n",
    "model_b.add(Dense(classes, activation='softmax'))\n",
    "\n",
    "model_b.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_b.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:22: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:17: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:22: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:17: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 1.7241 - categorical_accuracy: 0.2031Batch:  67 Index: 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 96s 1s/step - loss: 1.7210 - categorical_accuracy: 0.2000 - val_loss: 1.6107 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-1.72199-0.20211-1.61072-0.21000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 28s 415ms/step - loss: 1.6102 - categorical_accuracy: 0.1692 - val_loss: 1.6114 - val_categorical_accuracy: 0.1600\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-1.61024-0.16915-1.61143-0.16000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 31s 456ms/step - loss: 1.6124 - categorical_accuracy: 0.1741 - val_loss: 1.6107 - val_categorical_accuracy: 0.1600\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-1.61241-0.17413-1.61069-0.16000.h5\n",
      "\n",
      "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.6090 - categorical_accuracy: 0.2338 - val_loss: 1.6102 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-1.60904-0.23383-1.61017-0.21000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 30s 453ms/step - loss: 1.6107 - categorical_accuracy: 0.2090 - val_loss: 1.6095 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-1.61070-0.20896-1.60950-0.21000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.6085 - categorical_accuracy: 0.2388 - val_loss: 1.6095 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-1.60849-0.23881-1.60950-0.21000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 29s 432ms/step - loss: 1.6106 - categorical_accuracy: 0.1841 - val_loss: 1.6094 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-1.61061-0.18408-1.60943-0.21000.h5\n",
      "\n",
      "Epoch 00007: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 30s 444ms/step - loss: 1.6104 - categorical_accuracy: 0.1841 - val_loss: 1.6092 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-1.61040-0.18408-1.60921-0.21000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 29s 434ms/step - loss: 1.6094 - categorical_accuracy: 0.1990 - val_loss: 1.6095 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-1.60943-0.19900-1.60951-0.21000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 30s 443ms/step - loss: 1.6104 - categorical_accuracy: 0.2189 - val_loss: 1.6093 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-1.61045-0.21891-1.60934-0.21000.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f53381a4710>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing x,y,z values Experiment [ 1]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 60 # image width\n",
    "z = 60 # image height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_9 (Conv3D)            (None, 30, 60, 60, 32)    2624      \n",
      "_________________________________________________________________\n",
      "activation_9 (Activation)    (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_10 (Conv3D)           (None, 30, 60, 60, 32)    27680     \n",
      "_________________________________________________________________\n",
      "activation_10 (Activation)   (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_7 (MaxPooling3 (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_11 (Conv3D)           (None, 10, 20, 20, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_11 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_12 (Conv3D)           (None, 10, 20, 20, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_12 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_8 (MaxPooling3 (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 12544)             0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 512)               6423040   \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 6,621,925\n",
      "Trainable params: 6,621,925\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Define model b\n",
    "model_b = Sequential()\n",
    "model_b.add(Conv3D(32, kernel_size=(3, 3, 3), input_shape=(x,y,z,channel), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(Conv3D(32, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(MaxPooling3D(pool_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Dropout(0.25))\n",
    "\n",
    "model_b.add(Conv3D(64, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(Conv3D(64, kernel_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Activation('relu'))\n",
    "model_b.add(MaxPooling3D(pool_size=(3, 3, 3), padding='same'))\n",
    "model_b.add(Dropout(0.25))\n",
    "\n",
    "model_b.add(Flatten())\n",
    "model_b.add(Dense(512, activation='relu'))\n",
    "model_b.add(Dropout(0.5))\n",
    "model_b.add(Dense(classes, activation='softmax'))\n",
    "\n",
    "model_b.compile(optimizer=keras.optimizers.Adam(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_b.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10Source path =  /storage/Demo_Gesture/Project_data/val \n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n",
      "; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 1.6713 - categorical_accuracy: 0.1892Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 1.6703 - categorical_accuracy: 0.1924"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 82s 1s/step - loss: 1.6694 - categorical_accuracy: 0.1896 - val_loss: 1.6092 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-1.66998-0.19155-1.60916-0.21000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 26s 392ms/step - loss: 1.6110 - categorical_accuracy: 0.1841 - val_loss: 1.6096 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-1.61096-0.18408-1.60955-0.21000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 25s 372ms/step - loss: 1.6083 - categorical_accuracy: 0.2139 - val_loss: 1.6099 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-1.60831-0.21393-1.60988-0.21000.h5\n",
      "\n",
      "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 28s 413ms/step - loss: 1.6081 - categorical_accuracy: 0.2139 - val_loss: 1.6097 - val_categorical_accuracy: 0.1800\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-1.60806-0.21393-1.60965-0.18000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 27s 399ms/step - loss: 1.6133 - categorical_accuracy: 0.1592 - val_loss: 1.6090 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-1.61327-0.15920-1.60897-0.21000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 26s 382ms/step - loss: 1.6085 - categorical_accuracy: 0.2239 - val_loss: 1.6091 - val_categorical_accuracy: 0.2100\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-1.60851-0.22388-1.60911-0.21000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 27s 397ms/step - loss: 1.6105 - categorical_accuracy: 0.2090 - val_loss: 1.6083 - val_categorical_accuracy: 0.2300\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-1.61051-0.20896-1.60833-0.23000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 25s 371ms/step - loss: 1.5953 - categorical_accuracy: 0.2090 - val_loss: 1.5948 - val_categorical_accuracy: 0.1900\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-1.59532-0.20896-1.59479-0.19000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 25s 375ms/step - loss: 1.6047 - categorical_accuracy: 0.2388 - val_loss: 1.5946 - val_categorical_accuracy: 0.2500\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-1.60472-0.23881-1.59465-0.25000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 28s 415ms/step - loss: 1.5895 - categorical_accuracy: 0.2985 - val_loss: 1.5525 - val_categorical_accuracy: 0.3700\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-1.58947-0.29851-1.55246-0.37000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f53388fe780>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing Batch size  to 20 Experiment [ 2]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, 20)\n",
    "val_generator = generator_ex(val_path, val_doc, 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 20\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "32/67 [=============>................] - ETA: 1:39 - loss: 1.3914 - categorical_accuracy: 0.3750Batch:  34 Index: 20\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 105s 2s/step - loss: 1.4096 - categorical_accuracy: 0.3624 - val_loss: 1.3098 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-1.39264-0.37664-1.30983-0.49000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.4170 - categorical_accuracy: 0.4030 - val_loss: 1.3088 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-1.41703-0.40299-1.30883-0.44000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 27s 410ms/step - loss: 1.2228 - categorical_accuracy: 0.4726 - val_loss: 1.1473 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-1.22279-0.47264-1.14727-0.56000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 29s 433ms/step - loss: 1.2401 - categorical_accuracy: 0.4726 - val_loss: 1.1427 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-1.24013-0.47264-1.14271-0.62000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 29s 434ms/step - loss: 1.0855 - categorical_accuracy: 0.5473 - val_loss: 1.0945 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-1.08548-0.54726-1.09451-0.57000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 30s 441ms/step - loss: 1.0306 - categorical_accuracy: 0.5821 - val_loss: 1.1647 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-1.03057-0.58209-1.16474-0.46000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 29s 436ms/step - loss: 1.1514 - categorical_accuracy: 0.5025 - val_loss: 1.1483 - val_categorical_accuracy: 0.5500\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-1.15138-0.50249-1.14833-0.55000.h5\n",
      "\n",
      "Epoch 00007: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 29s 428ms/step - loss: 0.9278 - categorical_accuracy: 0.6219 - val_loss: 0.9821 - val_categorical_accuracy: 0.6900\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-0.92778-0.62189-0.98213-0.69000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 30s 447ms/step - loss: 0.8960 - categorical_accuracy: 0.5821 - val_loss: 0.9967 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-0.89600-0.58209-0.99674-0.66000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 29s 430ms/step - loss: 0.8453 - categorical_accuracy: 0.6517 - val_loss: 0.9581 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-0.84532-0.65174-0.95806-0.70000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f53388fe908>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing Batch size  to 30 Experiment [ 3]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, 30)\n",
    "val_generator = generator_ex(val_path, val_doc, 30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 30\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 30\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 3/67 [>.............................] - ETA: 4:34 - loss: 0.6558 - categorical_accuracy: 0.7333Batch:  4 Index: 30\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21/67 [========>.....................] - ETA: 2:51 - loss: 0.7590 - categorical_accuracy: 0.6905Batch:  23 Index: 30\n",
      "67/67 [==============================] - 99s 1s/step - loss: 0.8208 - categorical_accuracy: 0.6639 - val_loss: 0.8600 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-0.77600-0.68176-0.86000-0.67500.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 26s 384ms/step - loss: 0.7662 - categorical_accuracy: 0.6716 - val_loss: 0.8829 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-0.76625-0.67164-0.88292-0.66000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 27s 410ms/step - loss: 0.6805 - categorical_accuracy: 0.6915 - val_loss: 0.9753 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-0.68052-0.69154-0.97528-0.70000.h5\n",
      "\n",
      "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 25s 374ms/step - loss: 0.6333 - categorical_accuracy: 0.7463 - val_loss: 0.9962 - val_categorical_accuracy: 0.6100\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-0.63329-0.74627-0.99624-0.61000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 24s 357ms/step - loss: 0.5449 - categorical_accuracy: 0.7811 - val_loss: 0.9286 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-0.54486-0.78109-0.92857-0.70000.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 27s 403ms/step - loss: 0.5112 - categorical_accuracy: 0.8010 - val_loss: 0.9311 - val_categorical_accuracy: 0.6300\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-0.51116-0.80100-0.93107-0.63000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 28s 413ms/step - loss: 0.4480 - categorical_accuracy: 0.8060 - val_loss: 0.8609 - val_categorical_accuracy: 0.7400\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-0.44798-0.80597-0.86091-0.74000.h5\n",
      "\n",
      "Epoch 00007: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 26s 382ms/step - loss: 0.4500 - categorical_accuracy: 0.7960 - val_loss: 0.7969 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-0.44997-0.79602-0.79691-0.72000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 27s 400ms/step - loss: 0.4150 - categorical_accuracy: 0.8507 - val_loss: 1.0680 - val_categorical_accuracy: 0.6400\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-0.41504-0.85075-1.06796-0.64000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 27s 398ms/step - loss: 0.4580 - categorical_accuracy: 0.8259 - val_loss: 0.8437 - val_categorical_accuracy: 0.7600\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-0.45802-0.82587-0.84370-0.76000.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f5331673c88>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing Batch size  to 40 Experiment [ 4]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, 40)\n",
    "val_generator = generator_ex(val_path, val_doc, 40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 40\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 2/67 [..............................] - ETA: 6:05 - loss: 0.4529 - categorical_accuracy: 0.8000Batch:  3 Index: 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15/67 [=====>........................] - ETA: 4:27 - loss: 0.4242 - categorical_accuracy: 0.8367Batch:  17 Index: 40\n",
      "44/67 [==================>...........] - ETA: 1:22 - loss: 0.4243 - categorical_accuracy: 0.8336Batch:  29 Index: 23\n",
      "67/67 [==============================] - 210s 3s/step - loss: 0.4252 - categorical_accuracy: 0.8347 - val_loss: 0.8867 - val_categorical_accuracy: 0.7083\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-0.42551-0.83478-0.88669-0.70833.h5\n",
      "Epoch 2/10\n",
      "12/67 [====>.........................] - ETA: 2:23 - loss: 0.3914 - categorical_accuracy: 0.8465Batch:  35 Index: 19\n",
      "67/67 [==============================] - 142s 2s/step - loss: 0.4225 - categorical_accuracy: 0.8405 - val_loss: 0.9108 - val_categorical_accuracy: 0.7050\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-0.42200-0.84034-0.91077-0.70500.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 144s 2s/step - loss: 0.4081 - categorical_accuracy: 0.8385 - val_loss: 0.8104 - val_categorical_accuracy: 0.7300\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-0.40813-0.83845-0.81042-0.73000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 139s 2s/step - loss: 0.3951 - categorical_accuracy: 0.8516 - val_loss: 0.9578 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-0.39507-0.85162-0.95785-0.70000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 137s 2s/step - loss: 0.4009 - categorical_accuracy: 0.8464 - val_loss: 0.9408 - val_categorical_accuracy: 0.6800\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-0.40090-0.84636-0.94081-0.68000.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 7.812500371073838e-06.\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 141s 2s/step - loss: 0.3842 - categorical_accuracy: 0.8569 - val_loss: 0.8575 - val_categorical_accuracy: 0.7400\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-0.38425-0.85689-0.85750-0.74000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 140s 2s/step - loss: 0.4013 - categorical_accuracy: 0.8446 - val_loss: 0.8970 - val_categorical_accuracy: 0.7050\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-0.40127-0.84460-0.89701-0.70500.h5\n",
      "\n",
      "Epoch 00007: ReduceLROnPlateau reducing learning rate to 3.906250185536919e-06.\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 136s 2s/step - loss: 0.3840 - categorical_accuracy: 0.8525 - val_loss: 0.8991 - val_categorical_accuracy: 0.7100\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-0.38398-0.85250-0.89908-0.71000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 136s 2s/step - loss: 0.3793 - categorical_accuracy: 0.8481 - val_loss: 0.8953 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-0.37927-0.84811-0.89533-0.71500.h5\n",
      "\n",
      "Epoch 00009: ReduceLROnPlateau reducing learning rate to 1.9531250927684596e-06.\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 136s 2s/step - loss: 0.3617 - categorical_accuracy: 0.8745 - val_loss: 0.9105 - val_categorical_accuracy: 0.6950\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-0.36166-0.87445-0.91051-0.69500.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f5331624470>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Change Optimizer to Adadelta Experiment [5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_9 (Conv3D)            (None, 30, 60, 60, 32)    2624      \n",
      "_________________________________________________________________\n",
      "activation_9 (Activation)    (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_10 (Conv3D)           (None, 30, 60, 60, 32)    27680     \n",
      "_________________________________________________________________\n",
      "activation_10 (Activation)   (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_7 (MaxPooling3 (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_11 (Conv3D)           (None, 10, 20, 20, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_11 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_12 (Conv3D)           (None, 10, 20, 20, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_12 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_8 (MaxPooling3 (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 12544)             0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 512)               6423040   \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 6,621,925\n",
      "Trainable params: 6,621,925\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    }
   ],
   "source": [
    "model_b.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_b.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, 40)\n",
    "val_generator = generator_ex(val_path, val_doc, 40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 40\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 40\n",
      " 2/67 [..............................] - ETA: 6:39 - loss: 0.4547 - categorical_accuracy: 0.8125Batch:  3 Index: 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15/67 [=====>........................] - ETA: 4:24 - loss: 0.5939 - categorical_accuracy: 0.7800Batch:  17 Index: 40\n",
      "44/67 [==================>...........] - ETA: 1:20 - loss: 0.5839 - categorical_accuracy: 0.7729Batch:  29 Index: 23\n",
      "67/67 [==============================] - 207s 3s/step - loss: 0.5317 - categorical_accuracy: 0.7966 - val_loss: 0.8815 - val_categorical_accuracy: 0.6625\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-0.54416-0.79304-0.88149-0.66250.h5\n",
      "Epoch 2/10\n",
      "12/67 [====>.........................] - ETA: 2:11 - loss: 0.4361 - categorical_accuracy: 0.8158Batch:  35 Index: 19\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.3386 - categorical_accuracy: 0.8775 - val_loss: 0.8501 - val_categorical_accuracy: 0.7100\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-0.34065-0.87639-0.85008-0.71000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 136s 2s/step - loss: 0.2589 - categorical_accuracy: 0.8999 - val_loss: 1.2603 - val_categorical_accuracy: 0.6350\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-0.25886-0.89991-1.26033-0.63500.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 137s 2s/step - loss: 0.2011 - categorical_accuracy: 0.9298 - val_loss: 1.0706 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-0.20113-0.92976-1.07065-0.71500.h5\n",
      "\n",
      "Epoch 00004: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 136s 2s/step - loss: 0.1003 - categorical_accuracy: 0.9666 - val_loss: 1.2878 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-0.10026-0.96664-1.28779-0.71500.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0660 - categorical_accuracy: 0.9710 - val_loss: 1.3563 - val_categorical_accuracy: 0.6950\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-0.06603-0.97103-1.35627-0.69500.h5\n",
      "\n",
      "Epoch 00006: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0419 - categorical_accuracy: 0.9868 - val_loss: 1.4602 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-0.04188-0.98683-1.46019-0.67500.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0450 - categorical_accuracy: 0.9877 - val_loss: 1.4257 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-0.04502-0.98771-1.42570-0.72000.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 0.125.\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 140s 2s/step - loss: 0.0382 - categorical_accuracy: 0.9912 - val_loss: 1.6116 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-0.03815-0.99122-1.61157-0.66000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0275 - categorical_accuracy: 0.9921 - val_loss: 1.6300 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-0.02751-0.99210-1.63002-0.67500.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 0.0625.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f533165de80>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Change epoch to 20 Experiment [6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# epochs = 20\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 20 # choose the number of epochs\n",
    "print ('# epochs =', num_epochs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_9 (Conv3D)            (None, 30, 60, 60, 32)    2624      \n",
      "_________________________________________________________________\n",
      "activation_9 (Activation)    (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_10 (Conv3D)           (None, 30, 60, 60, 32)    27680     \n",
      "_________________________________________________________________\n",
      "activation_10 (Activation)   (None, 30, 60, 60, 32)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_7 (MaxPooling3 (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 10, 20, 20, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_11 (Conv3D)           (None, 10, 20, 20, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_11 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_12 (Conv3D)           (None, 10, 20, 20, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_12 (Activation)   (None, 10, 20, 20, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_8 (MaxPooling3 (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 4, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 12544)             0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 512)               6423040   \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 6,621,925\n",
      "Trainable params: 6,621,925\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "model_b.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_b.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, 40)\n",
    "val_generator = generator_ex(val_path, val_doc, 40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 40\n",
      "; batch size = 40\n",
      " 2/67 [..............................] - ETA: 6:16 - loss: 0.0179 - categorical_accuracy: 1.0000Batch:  3 Index: 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15/67 [=====>........................] - ETA: 4:26 - loss: 0.1311 - categorical_accuracy: 0.9583Batch:  17 Index: 40\n",
      "44/67 [==================>...........] - ETA: 1:21 - loss: 0.1206 - categorical_accuracy: 0.9615Batch:  29 Index: 23\n",
      "67/67 [==============================] - 206s 3s/step - loss: 0.0958 - categorical_accuracy: 0.9708 - val_loss: 1.4780 - val_categorical_accuracy: 0.6833\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2617_35_26.106810/model-00001-0.10298-0.96812-1.47802-0.68333.h5\n",
      "Epoch 2/20\n",
      "12/67 [====>.........................] - ETA: 2:20 - loss: 0.0512 - categorical_accuracy: 0.9781Batch:  35 Index: 19\n",
      "67/67 [==============================] - 140s 2s/step - loss: 0.0816 - categorical_accuracy: 0.9716 - val_loss: 1.6118 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2617_35_26.106810/model-00002-0.08097-0.97167-1.61185-0.67500.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 140s 2s/step - loss: 0.0485 - categorical_accuracy: 0.9772 - val_loss: 1.7824 - val_categorical_accuracy: 0.6950\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2617_35_26.106810/model-00003-0.04847-0.97717-1.78243-0.69500.h5\n",
      "\n",
      "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 135s 2s/step - loss: 0.0411 - categorical_accuracy: 0.9877 - val_loss: 1.8472 - val_categorical_accuracy: 0.6850\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2617_35_26.106810/model-00004-0.04112-0.98771-1.84721-0.68500.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 135s 2s/step - loss: 0.0315 - categorical_accuracy: 0.9895 - val_loss: 1.7786 - val_categorical_accuracy: 0.7050\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2617_35_26.106810/model-00005-0.03148-0.98946-1.77860-0.70500.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 137s 2s/step - loss: 0.0133 - categorical_accuracy: 0.9982 - val_loss: 1.4542 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2617_35_26.106810/model-00006-0.01334-0.99824-1.45425-0.72000.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 135s 2s/step - loss: 0.0149 - categorical_accuracy: 0.9956 - val_loss: 1.9153 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2617_35_26.106810/model-00007-0.01494-0.99561-1.91534-0.71500.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 134s 2s/step - loss: 0.0109 - categorical_accuracy: 0.9947 - val_loss: 2.1422 - val_categorical_accuracy: 0.6700\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2617_35_26.106810/model-00008-0.01087-0.99473-2.14220-0.67000.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 0.125.\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0086 - categorical_accuracy: 0.9965 - val_loss: 1.6353 - val_categorical_accuracy: 0.7350\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2617_35_26.106810/model-00009-0.00859-0.99649-1.63530-0.73500.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 133s 2s/step - loss: 0.0082 - categorical_accuracy: 0.9982 - val_loss: 2.2985 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2617_35_26.106810/model-00010-0.00824-0.99824-2.29847-0.66000.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 0.0625.\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 138s 2s/step - loss: 0.0159 - categorical_accuracy: 0.9965 - val_loss: 2.1800 - val_categorical_accuracy: 0.6650\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2617_35_26.106810/model-00011-0.01591-0.99649-2.17999-0.66500.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 137s 2s/step - loss: 0.0067 - categorical_accuracy: 0.9982 - val_loss: 1.9259 - val_categorical_accuracy: 0.7100\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2617_35_26.106810/model-00012-0.00670-0.99824-1.92591-0.71000.h5\n",
      "\n",
      "Epoch 00012: ReduceLROnPlateau reducing learning rate to 0.03125.\n",
      "Epoch 13/20\n",
      "62/67 [==========================>...] - ETA: 10s - loss: 0.0072 - categorical_accuracy: 0.9981"
     ]
    }
   ],
   "source": [
    "model_b.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Till this point we achieved 99.39% accuracy in training data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model C Experiment [7]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_5 (Conv3D)            (None, 30, 120, 120, 32)  2624      \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 30, 120, 120, 32)  0         \n",
      "_________________________________________________________________\n",
      "conv3d_6 (Conv3D)            (None, 30, 120, 120, 32)  27680     \n",
      "_________________________________________________________________\n",
      "activation_6 (Activation)    (None, 30, 120, 120, 32)  0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_5 (MaxPooling3 (None, 10, 40, 40, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 10, 40, 40, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_7 (Conv3D)            (None, 10, 40, 40, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_7 (Activation)    (None, 10, 40, 40, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_8 (Conv3D)            (None, 10, 40, 40, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_8 (Activation)    (None, 10, 40, 40, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_6 (MaxPooling3 (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_9 (Conv3D)            (None, 4, 14, 14, 64)     110656    \n",
      "_________________________________________________________________\n",
      "activation_9 (Activation)    (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_10 (Conv3D)           (None, 4, 14, 14, 64)     110656    \n",
      "_________________________________________________________________\n",
      "activation_10 (Activation)   (None, 4, 14, 14, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_7 (MaxPooling3 (None, 2, 5, 5, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 2, 5, 5, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 3200)              0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 512)               1638912   \n",
      "_________________________________________________________________\n",
      "batch_normalization_5 (Batch (None, 512)               2048      \n",
      "_________________________________________________________________\n",
      "dropout_6 (Dropout)          (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 2,061,157\n",
      "Trainable params: 2,060,133\n",
      "Non-trainable params: 1,024\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model_c = Sequential()\n",
    "model_c.add(Conv3D(32, kernel_size=(3, 3, 3), input_shape=(x,y,z,channel), padding=\"same\"))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(Conv3D(32, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_c.add(Dropout(0.25))\n",
    "\n",
    "model_c.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_c.add(Dropout(0.25))\n",
    "\n",
    "model_c.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_c.add(Activation('relu'))\n",
    "model_c.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_c.add(Dropout(0.25))\n",
    "\n",
    "model_c.add(Flatten())\n",
    "model_c.add(Dense(512, activation='relu'))\n",
    "model_c.add(BatchNormalization())\n",
    "model_c.add(Dropout(0.5))\n",
    "model_c.add(Dense(classes, activation='softmax'))\n",
    "model_c.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_c.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Changing back batch size, images per frame, height and width of image\n",
    "batch_size = 10\n",
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120# image height\n",
    "num_epochs = 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 3s - loss: 1.8396 - categorical_accuracy: 0.3154Batch:  67 Index: 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 135s 2s/step - loss: 1.8753 - categorical_accuracy: 0.3105 - val_loss: 3.1390 - val_categorical_accuracy: 0.2900\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-1.85423-0.31373-3.13896-0.29000.h5\n",
      "Epoch 2/20\n",
      "67/67 [==============================] - 29s 427ms/step - loss: 1.9485 - categorical_accuracy: 0.2488 - val_loss: 6.3683 - val_categorical_accuracy: 0.1600\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-1.94852-0.24876-6.36835-0.16000.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 31s 470ms/step - loss: 1.9667 - categorical_accuracy: 0.2438 - val_loss: 2.8393 - val_categorical_accuracy: 0.2900\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-1.96668-0.24378-2.83926-0.29000.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 28s 425ms/step - loss: 1.7844 - categorical_accuracy: 0.3333 - val_loss: 3.2987 - val_categorical_accuracy: 0.2900\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-1.78444-0.33333-3.29870-0.29000.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 30s 451ms/step - loss: 1.9605 - categorical_accuracy: 0.2935 - val_loss: 1.7031 - val_categorical_accuracy: 0.3400\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.96054-0.29353-1.70314-0.34000.h5\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 30s 453ms/step - loss: 1.7932 - categorical_accuracy: 0.2637 - val_loss: 1.3392 - val_categorical_accuracy: 0.4800\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.79320-0.26368-1.33921-0.48000.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 29s 438ms/step - loss: 1.8009 - categorical_accuracy: 0.2736 - val_loss: 1.4190 - val_categorical_accuracy: 0.4700\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.80087-0.27363-1.41902-0.47000.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 30s 441ms/step - loss: 1.6576 - categorical_accuracy: 0.3483 - val_loss: 1.5525 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.65762-0.34826-1.55247-0.40000.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 31s 459ms/step - loss: 1.5115 - categorical_accuracy: 0.3582 - val_loss: 1.6254 - val_categorical_accuracy: 0.2800\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.51150-0.35821-1.62538-0.28000.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 30s 448ms/step - loss: 1.4296 - categorical_accuracy: 0.4030 - val_loss: 1.2409 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.42958-0.40299-1.24086-0.50000.h5\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 28s 425ms/step - loss: 1.4469 - categorical_accuracy: 0.4129 - val_loss: 1.2708 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2708_18_49.548506/model-00011-1.44693-0.41294-1.27080-0.49000.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 39s 581ms/step - loss: 1.5740 - categorical_accuracy: 0.3234 - val_loss: 1.3148 - val_categorical_accuracy: 0.4200\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2708_18_49.548506/model-00012-1.57396-0.32338-1.31482-0.42000.h5\n",
      "\n",
      "Epoch 00012: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.4502 - categorical_accuracy: 0.3930 - val_loss: 1.1910 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2708_18_49.548506/model-00013-1.45021-0.39303-1.19100-0.50000.h5\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 31s 460ms/step - loss: 1.4499 - categorical_accuracy: 0.3234 - val_loss: 1.1594 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2708_18_49.548506/model-00014-1.44993-0.32338-1.15939-0.54000.h5\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.4181 - categorical_accuracy: 0.3980 - val_loss: 1.1752 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2708_18_49.548506/model-00015-1.41810-0.39801-1.17523-0.58000.h5\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 1.4124 - categorical_accuracy: 0.3881 - val_loss: 1.2145 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2708_18_49.548506/model-00016-1.41241-0.38806-1.21452-0.54000.h5\n",
      "\n",
      "Epoch 00016: ReduceLROnPlateau reducing learning rate to 0.125.\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 30s 453ms/step - loss: 1.4067 - categorical_accuracy: 0.4229 - val_loss: 1.2553 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2708_18_49.548506/model-00017-1.40666-0.42289-1.25534-0.52000.h5\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 1.4206 - categorical_accuracy: 0.4030 - val_loss: 1.2947 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2708_18_49.548506/model-00018-1.42062-0.40299-1.29468-0.52000.h5\n",
      "\n",
      "Epoch 00018: ReduceLROnPlateau reducing learning rate to 0.0625.\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 28s 422ms/step - loss: 1.3907 - categorical_accuracy: 0.4527 - val_loss: 1.2560 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2708_18_49.548506/model-00019-1.39074-0.45274-1.25599-0.57000.h5\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 32s 479ms/step - loss: 1.4558 - categorical_accuracy: 0.3731 - val_loss: 1.3023 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2708_18_49.548506/model-00020-1.45577-0.37313-1.30232-0.50000.h5\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 0.03125.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f030cc59710>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_c.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model D Experiment [8]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_11 (Conv3D)           (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_6 (Batch (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_11 (Activation)   (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_8 (MaxPooling3 (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_12 (Conv3D)           (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_7 (Batch (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_12 (Activation)   (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_9 (MaxPooling3 (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_13 (Conv3D)           (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_8 (Batch (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_13 (Activation)   (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_10 (MaxPooling (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_14 (Conv3D)           (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_9 (Batch (None, 3, 15, 15, 64)     256       \n",
      "_________________________________________________________________\n",
      "activation_14 (Activation)   (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_11 (MaxPooling (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_3 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_7 (Dropout)          (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_8 (Dropout)          (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,749\n",
      "Trainable params: 3,667,509\n",
      "Non-trainable params: 240\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "input_shape=(x,y,z,channel)\n",
    "\n",
    "nb_filters = [8,16,32,64]\n",
    "nb_dense = [1000, 500, 5]\n",
    "# Define model\n",
    "model_d = Sequential()\n",
    "\n",
    "model_d.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_d.add(BatchNormalization())\n",
    "model_d.add(Activation('relu'))\n",
    "\n",
    "model_d.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_d.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_d.add(BatchNormalization())\n",
    "model_d.add(Activation('relu'))\n",
    "\n",
    "model_d.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_d.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_d.add(BatchNormalization())\n",
    "model_d.add(Activation('relu'))\n",
    "\n",
    "model_d.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_d.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_d.add(BatchNormalization())\n",
    "model_d.add(Activation('relu'))\n",
    "\n",
    "model_d.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_d.add(Flatten())\n",
    "\n",
    "model_d.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_d.add(Dropout(0.5))\n",
    "\n",
    "model_d.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_d.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_d.add(Dense(nb_dense[2], activation='softmax'))\n",
    "model_d.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_d.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "65/67 [============================>.] - ETA: 2s - loss: 5.0803 - categorical_accuracy: 0.2769Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 5.0497 - categorical_accuracy: 0.2773"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 92s 1s/step - loss: 5.0001 - categorical_accuracy: 0.2781 - val_loss: 3.9369 - val_categorical_accuracy: 0.2400\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-5.03451-0.27753-3.93688-0.24000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 2.2612 - categorical_accuracy: 0.2736 - val_loss: 2.0963 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-2.26116-0.27363-2.09633-0.30000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 1.7592 - categorical_accuracy: 0.3433 - val_loss: 1.1549 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-1.75917-0.34328-1.15488-0.49000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.6257 - categorical_accuracy: 0.2786 - val_loss: 1.5620 - val_categorical_accuracy: 0.3200\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-1.62569-0.27861-1.56197-0.32000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 29s 429ms/step - loss: 1.5879 - categorical_accuracy: 0.3134 - val_loss: 1.2710 - val_categorical_accuracy: 0.4200\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.58790-0.31343-1.27101-0.42000.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 29s 440ms/step - loss: 1.4687 - categorical_accuracy: 0.3333 - val_loss: 1.1319 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.46874-0.33333-1.13188-0.57000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 30s 448ms/step - loss: 1.2672 - categorical_accuracy: 0.4677 - val_loss: 1.1011 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.26723-0.46766-1.10112-0.52000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 29s 430ms/step - loss: 1.3060 - categorical_accuracy: 0.4229 - val_loss: 1.2269 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.30602-0.42289-1.22692-0.46000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 30s 451ms/step - loss: 1.3309 - categorical_accuracy: 0.4478 - val_loss: 0.9628 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.33093-0.44776-0.96279-0.65000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 29s 427ms/step - loss: 1.3538 - categorical_accuracy: 0.4179 - val_loss: 1.1506 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.35376-0.41791-1.15058-0.51000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f02e8638c88>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 10\n",
    "model_d.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Changing Model with images per frame, width and height of the images Experiment [9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 64 # image width\n",
    "z = 64 # image height \n",
    "\n",
    "classes = 5\n",
    "channel = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_15 (Conv3D)           (None, 30, 64, 64, 32)    896       \n",
      "_________________________________________________________________\n",
      "activation_15 (Activation)   (None, 30, 64, 64, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_16 (Conv3D)           (None, 30, 64, 64, 32)    27680     \n",
      "_________________________________________________________________\n",
      "activation_16 (Activation)   (None, 30, 64, 64, 32)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_12 (MaxPooling (None, 10, 22, 22, 32)    0         \n",
      "_________________________________________________________________\n",
      "dropout_9 (Dropout)          (None, 10, 22, 22, 32)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_17 (Conv3D)           (None, 10, 22, 22, 64)    55360     \n",
      "_________________________________________________________________\n",
      "activation_17 (Activation)   (None, 10, 22, 22, 64)    0         \n",
      "_________________________________________________________________\n",
      "conv3d_18 (Conv3D)           (None, 10, 22, 22, 64)    110656    \n",
      "_________________________________________________________________\n",
      "activation_18 (Activation)   (None, 10, 22, 22, 64)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_13 (MaxPooling (None, 4, 8, 8, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_10 (Dropout)         (None, 4, 8, 8, 64)       0         \n",
      "_________________________________________________________________\n",
      "conv3d_19 (Conv3D)           (None, 4, 8, 8, 64)       110656    \n",
      "_________________________________________________________________\n",
      "activation_19 (Activation)   (None, 4, 8, 8, 64)       0         \n",
      "_________________________________________________________________\n",
      "conv3d_20 (Conv3D)           (None, 4, 8, 8, 64)       110656    \n",
      "_________________________________________________________________\n",
      "activation_20 (Activation)   (None, 4, 8, 8, 64)       0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_14 (MaxPooling (None, 2, 3, 3, 64)       0         \n",
      "_________________________________________________________________\n",
      "dropout_11 (Dropout)         (None, 2, 3, 3, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_4 (Flatten)          (None, 1152)              0         \n",
      "_________________________________________________________________\n",
      "dense_9 (Dense)              (None, 512)               590336    \n",
      "_________________________________________________________________\n",
      "batch_normalization_10 (Batc (None, 512)               2048      \n",
      "_________________________________________________________________\n",
      "dropout_12 (Dropout)         (None, 512)               0         \n",
      "_________________________________________________________________\n",
      "dense_10 (Dense)             (None, 5)                 2565      \n",
      "=================================================================\n",
      "Total params: 1,010,853\n",
      "Trainable params: 1,009,829\n",
      "Non-trainable params: 1,024\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model_e = Sequential()\n",
    "model_e.add(Conv3D(32, kernel_size=(3, 3, 3), input_shape=(x,y,z,channel), padding=\"same\"))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(Conv3D(32, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_e.add(Dropout(0.25))\n",
    "\n",
    "model_e.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_e.add(Dropout(0.25))\n",
    "\n",
    "model_e.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(Conv3D(64, padding=\"same\", kernel_size=(3, 3, 3)))\n",
    "model_e.add(Activation('relu'))\n",
    "model_e.add(MaxPooling3D(pool_size=(3, 3, 3), padding=\"same\"))\n",
    "model_e.add(Dropout(0.25))\n",
    "\n",
    "model_e.add(Flatten())\n",
    "model_e.add(Dense(512, activation='relu'))\n",
    "model_e.add(BatchNormalization())\n",
    "model_e.add(Dropout(0.5))\n",
    "model_e.add(Dense(classes, activation='softmax'))\n",
    "model_e.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_e.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 5\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 5\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 40s 600ms/step - loss: 2.3581 - categorical_accuracy: 0.2149 - val_loss: 5.3811 - val_categorical_accuracy: 0.1600\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-2.35805-0.21493-5.38115-0.16000.h5\n",
      "Epoch 2/20\n",
      "64/67 [===========================>..] - ETA: 1s - loss: 2.2653 - categorical_accuracy: 0.2094Batch:  133 Index: 5\n",
      "65/67 [============================>.] - ETA: 1s - loss: 2.2635 - categorical_accuracy: 0.2092"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 41s 613ms/step - loss: 2.2555 - categorical_accuracy: 0.2129 - val_loss: 4.4151 - val_categorical_accuracy: 0.1600\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-2.25866-0.21148-4.41509-0.16000.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 24s 358ms/step - loss: 2.1614 - categorical_accuracy: 0.1990 - val_loss: 1.7849 - val_categorical_accuracy: 0.1800\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-2.16142-0.19900-1.78486-0.18000.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 25s 374ms/step - loss: 2.0766 - categorical_accuracy: 0.2090 - val_loss: 1.9560 - val_categorical_accuracy: 0.2000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-2.07660-0.20896-1.95596-0.20000.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 27s 396ms/step - loss: 1.9846 - categorical_accuracy: 0.2438 - val_loss: 1.6872 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.98465-0.24378-1.68724-0.30000.h5\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 25s 373ms/step - loss: 1.8498 - categorical_accuracy: 0.2736 - val_loss: 1.6253 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.84982-0.27363-1.62527-0.30000.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 26s 383ms/step - loss: 1.9456 - categorical_accuracy: 0.2139 - val_loss: 1.6699 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.94557-0.21393-1.66993-0.30000.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 25s 372ms/step - loss: 1.8121 - categorical_accuracy: 0.2587 - val_loss: 1.5906 - val_categorical_accuracy: 0.3200\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.81214-0.25871-1.59058-0.32000.h5\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 25s 367ms/step - loss: 1.7915 - categorical_accuracy: 0.2687 - val_loss: 1.5339 - val_categorical_accuracy: 0.3400\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.79150-0.26866-1.53389-0.34000.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 23s 350ms/step - loss: 1.9289 - categorical_accuracy: 0.2338 - val_loss: 1.6318 - val_categorical_accuracy: 0.3200\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.92894-0.23383-1.63184-0.32000.h5\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 26s 388ms/step - loss: 1.8876 - categorical_accuracy: 0.2388 - val_loss: 1.6486 - val_categorical_accuracy: 0.2400\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2708_18_49.548506/model-00011-1.88762-0.23881-1.64855-0.24000.h5\n",
      "\n",
      "Epoch 00011: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 25s 379ms/step - loss: 1.7927 - categorical_accuracy: 0.2438 - val_loss: 1.5476 - val_categorical_accuracy: 0.3200\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2708_18_49.548506/model-00012-1.79270-0.24378-1.54758-0.32000.h5\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 25s 378ms/step - loss: 1.6826 - categorical_accuracy: 0.2985 - val_loss: 1.5907 - val_categorical_accuracy: 0.2400\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2708_18_49.548506/model-00013-1.68260-0.29851-1.59070-0.24000.h5\n",
      "\n",
      "Epoch 00013: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 23s 342ms/step - loss: 1.7363 - categorical_accuracy: 0.2687 - val_loss: 1.4505 - val_categorical_accuracy: 0.3400\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2708_18_49.548506/model-00014-1.73630-0.26866-1.45046-0.34000.h5\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 26s 385ms/step - loss: 1.6958 - categorical_accuracy: 0.2786 - val_loss: 1.5338 - val_categorical_accuracy: 0.2600\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2708_18_49.548506/model-00015-1.69585-0.27861-1.53384-0.26000.h5\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 21s 319ms/step - loss: 1.8104 - categorical_accuracy: 0.2289 - val_loss: 1.4905 - val_categorical_accuracy: 0.3800\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2708_18_49.548506/model-00016-1.81043-0.22886-1.49055-0.38000.h5\n",
      "\n",
      "Epoch 00016: ReduceLROnPlateau reducing learning rate to 0.125.\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 26s 390ms/step - loss: 1.7611 - categorical_accuracy: 0.2338 - val_loss: 1.4579 - val_categorical_accuracy: 0.4200\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2708_18_49.548506/model-00017-1.76111-0.23383-1.45788-0.42000.h5\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 26s 386ms/step - loss: 1.7780 - categorical_accuracy: 0.2438 - val_loss: 1.5007 - val_categorical_accuracy: 0.3600\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2708_18_49.548506/model-00018-1.77796-0.24378-1.50067-0.36000.h5\n",
      "\n",
      "Epoch 00018: ReduceLROnPlateau reducing learning rate to 0.0625.\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 24s 358ms/step - loss: 1.7237 - categorical_accuracy: 0.2985 - val_loss: 1.4956 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2708_18_49.548506/model-00019-1.72369-0.29851-1.49560-0.40000.h5\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 24s 357ms/step - loss: 1.6338 - categorical_accuracy: 0.2836 - val_loss: 1.4575 - val_categorical_accuracy: 0.3800\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2708_18_49.548506/model-00020-1.63381-0.28358-1.45752-0.38000.h5\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 0.03125.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f030ab7ddd8>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 5\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 20\n",
    "model_e.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model F Experiment [10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_21 (Conv3D)           (None, 30, 120, 120, 8)   224       \n",
      "_________________________________________________________________\n",
      "batch_normalization_11 (Batc (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_21 (Activation)   (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_15 (MaxPooling (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_22 (Conv3D)           (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_12 (Batc (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_22 (Activation)   (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_16 (MaxPooling (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_23 (Conv3D)           (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_13 (Batc (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_23 (Activation)   (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_17 (MaxPooling (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_24 (Conv3D)           (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_14 (Batc (None, 3, 15, 15, 64)     256       \n",
      "_________________________________________________________________\n",
      "activation_24 (Activation)   (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_18 (MaxPooling (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_5 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_11 (Dense)             (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_13 (Dropout)         (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_12 (Dense)             (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_14 (Dropout)         (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_13 (Dense)             (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,317\n",
      "Trainable params: 3,667,077\n",
      "Non-trainable params: 240\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height \n",
    "\n",
    "classes = 5\n",
    "channel = 1\n",
    "nb_dense = [1000, 500, 5]\n",
    "\n",
    "input_shape=(x,y,z,channel)\n",
    "\n",
    "# Define model\n",
    "model_f = Sequential()\n",
    "\n",
    "model_f.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_f.add(BatchNormalization())\n",
    "model_f.add(Activation('relu'))\n",
    "\n",
    "model_f.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_f.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_f.add(BatchNormalization())\n",
    "model_f.add(Activation('relu'))\n",
    "\n",
    "model_f.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_f.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_f.add(BatchNormalization())\n",
    "model_f.add(Activation('relu'))\n",
    "\n",
    "model_f.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_f.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_f.add(BatchNormalization())\n",
    "model_f.add(Activation('relu'))\n",
    "\n",
    "model_f.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_f.add(Flatten())\n",
    "\n",
    "model_f.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_f.add(Dropout(0.5))\n",
    "\n",
    "model_f.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_f.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_f.add(Dense(nb_dense[2], activation='softmax'))\n",
    "model_f.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_f.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n",
      "\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 6.9154 - categorical_accuracy: 0.2569Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 6.8595 - categorical_accuracy: 0.2561"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 87s 1s/step - loss: 6.8264 - categorical_accuracy: 0.2572 - val_loss: 5.2584 - val_categorical_accuracy: 0.2900\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-6.84936-0.25641-5.25835-0.29000.h5\n",
      "Epoch 2/30\n",
      "67/67 [==============================] - 26s 395ms/step - loss: 3.4366 - categorical_accuracy: 0.2438 - val_loss: 1.3242 - val_categorical_accuracy: 0.4500\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-3.43656-0.24378-1.32420-0.45000.h5\n",
      "Epoch 3/30\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 2.0743 - categorical_accuracy: 0.3035 - val_loss: 1.4278 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-2.07433-0.30348-1.42776-0.30000.h5\n",
      "Epoch 4/30\n",
      "67/67 [==============================] - 27s 396ms/step - loss: 1.7408 - categorical_accuracy: 0.4030 - val_loss: 1.3609 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-1.74083-0.40299-1.36094-0.30000.h5\n",
      "\n",
      "Epoch 00004: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 5/30\n",
      "67/67 [==============================] - 28s 419ms/step - loss: 1.5159 - categorical_accuracy: 0.3532 - val_loss: 1.1767 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.51587-0.35323-1.17674-0.49000.h5\n",
      "Epoch 6/30\n",
      "67/67 [==============================] - 26s 394ms/step - loss: 1.5173 - categorical_accuracy: 0.2786 - val_loss: 1.2578 - val_categorical_accuracy: 0.4700\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.51735-0.27861-1.25778-0.47000.h5\n",
      "Epoch 7/30\n",
      "67/67 [==============================] - 28s 414ms/step - loss: 1.3819 - categorical_accuracy: 0.4179 - val_loss: 1.1252 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.38195-0.41791-1.12525-0.51000.h5\n",
      "Epoch 8/30\n",
      "67/67 [==============================] - 28s 424ms/step - loss: 1.3516 - categorical_accuracy: 0.4328 - val_loss: 1.0642 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.35158-0.43284-1.06418-0.56000.h5\n",
      "Epoch 9/30\n",
      "67/67 [==============================] - 27s 400ms/step - loss: 1.3530 - categorical_accuracy: 0.4279 - val_loss: 1.1649 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.35299-0.42786-1.16488-0.49000.h5\n",
      "Epoch 10/30\n",
      "67/67 [==============================] - 28s 425ms/step - loss: 1.2447 - categorical_accuracy: 0.4776 - val_loss: 1.0272 - val_categorical_accuracy: 0.5900\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.24474-0.47761-1.02723-0.59000.h5\n",
      "Epoch 11/30\n",
      "67/67 [==============================] - 27s 401ms/step - loss: 1.3280 - categorical_accuracy: 0.4080 - val_loss: 0.9982 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2708_18_49.548506/model-00011-1.32801-0.40796-0.99819-0.58000.h5\n",
      "Epoch 12/30\n",
      "67/67 [==============================] - 27s 408ms/step - loss: 1.1958 - categorical_accuracy: 0.4826 - val_loss: 0.9495 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2708_18_49.548506/model-00012-1.19576-0.48259-0.94954-0.56000.h5\n",
      "Epoch 13/30\n",
      "67/67 [==============================] - 28s 413ms/step - loss: 1.0640 - categorical_accuracy: 0.5274 - val_loss: 1.0935 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2708_18_49.548506/model-00013-1.06405-0.52736-1.09350-0.52000.h5\n",
      "Epoch 14/30\n",
      "67/67 [==============================] - 28s 423ms/step - loss: 1.1977 - categorical_accuracy: 0.5274 - val_loss: 0.9546 - val_categorical_accuracy: 0.5900\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2708_18_49.548506/model-00014-1.19767-0.52736-0.95457-0.59000.h5\n",
      "\n",
      "Epoch 00014: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 15/30\n",
      "67/67 [==============================] - 27s 405ms/step - loss: 1.0990 - categorical_accuracy: 0.5323 - val_loss: 0.9413 - val_categorical_accuracy: 0.5900\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2708_18_49.548506/model-00015-1.09897-0.53234-0.94133-0.59000.h5\n",
      "Epoch 16/30\n",
      "67/67 [==============================] - 29s 430ms/step - loss: 1.0212 - categorical_accuracy: 0.5871 - val_loss: 0.9344 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2708_18_49.548506/model-00016-1.02117-0.58706-0.93438-0.58000.h5\n",
      "Epoch 17/30\n",
      "67/67 [==============================] - 28s 425ms/step - loss: 1.0751 - categorical_accuracy: 0.5224 - val_loss: 1.1322 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2708_18_49.548506/model-00017-1.07506-0.52239-1.13221-0.51000.h5\n",
      "Epoch 18/30\n",
      "67/67 [==============================] - 26s 392ms/step - loss: 0.9826 - categorical_accuracy: 0.6418 - val_loss: 0.8359 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2708_18_49.548506/model-00018-0.98262-0.64179-0.83589-0.62000.h5\n",
      "Epoch 19/30\n",
      "67/67 [==============================] - 26s 393ms/step - loss: 0.9634 - categorical_accuracy: 0.5970 - val_loss: 0.8777 - val_categorical_accuracy: 0.6000\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2708_18_49.548506/model-00019-0.96344-0.59701-0.87769-0.60000.h5\n",
      "Epoch 20/30\n",
      "67/67 [==============================] - 28s 424ms/step - loss: 1.0045 - categorical_accuracy: 0.6119 - val_loss: 0.8059 - val_categorical_accuracy: 0.6900\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2708_18_49.548506/model-00020-1.00451-0.61194-0.80586-0.69000.h5\n",
      "Epoch 21/30\n",
      "67/67 [==============================] - 28s 420ms/step - loss: 0.9417 - categorical_accuracy: 0.6269 - val_loss: 0.8713 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00021: saving model to model_init_2018-12-2708_18_49.548506/model-00021-0.94173-0.62687-0.87132-0.62000.h5\n",
      "Epoch 22/30\n",
      "67/67 [==============================] - 27s 400ms/step - loss: 0.9567 - categorical_accuracy: 0.6070 - val_loss: 0.7613 - val_categorical_accuracy: 0.6300\n",
      "\n",
      "Epoch 00022: saving model to model_init_2018-12-2708_18_49.548506/model-00022-0.95673-0.60697-0.76129-0.63000.h5\n",
      "Epoch 23/30\n",
      "67/67 [==============================] - 28s 413ms/step - loss: 0.8940 - categorical_accuracy: 0.6318 - val_loss: 0.8218 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00023: saving model to model_init_2018-12-2708_18_49.548506/model-00023-0.89398-0.63184-0.82184-0.62000.h5\n",
      "Epoch 24/30\n",
      "67/67 [==============================] - 29s 427ms/step - loss: 0.9119 - categorical_accuracy: 0.6517 - val_loss: 0.8274 - val_categorical_accuracy: 0.6300\n",
      "\n",
      "Epoch 00024: saving model to model_init_2018-12-2708_18_49.548506/model-00024-0.91186-0.65174-0.82735-0.63000.h5\n",
      "\n",
      "Epoch 00024: ReduceLROnPlateau reducing learning rate to 0.125.\n",
      "Epoch 25/30\n",
      "67/67 [==============================] - 29s 428ms/step - loss: 0.8403 - categorical_accuracy: 0.6716 - val_loss: 0.8327 - val_categorical_accuracy: 0.6400\n",
      "\n",
      "Epoch 00025: saving model to model_init_2018-12-2708_18_49.548506/model-00025-0.84031-0.67164-0.83267-0.64000.h5\n",
      "Epoch 26/30\n",
      "67/67 [==============================] - 27s 396ms/step - loss: 0.7955 - categorical_accuracy: 0.6965 - val_loss: 0.7159 - val_categorical_accuracy: 0.6800\n",
      "\n",
      "Epoch 00026: saving model to model_init_2018-12-2708_18_49.548506/model-00026-0.79548-0.69652-0.71591-0.68000.h5\n",
      "Epoch 27/30\n",
      "67/67 [==============================] - 29s 427ms/step - loss: 0.7188 - categorical_accuracy: 0.7363 - val_loss: 0.6894 - val_categorical_accuracy: 0.7300\n",
      "\n",
      "Epoch 00027: saving model to model_init_2018-12-2708_18_49.548506/model-00027-0.71885-0.73632-0.68941-0.73000.h5\n",
      "Epoch 28/30\n",
      "67/67 [==============================] - 28s 413ms/step - loss: 0.7403 - categorical_accuracy: 0.6866 - val_loss: 0.6508 - val_categorical_accuracy: 0.7400\n",
      "\n",
      "Epoch 00028: saving model to model_init_2018-12-2708_18_49.548506/model-00028-0.74027-0.68657-0.65075-0.74000.h5\n",
      "Epoch 29/30\n",
      "67/67 [==============================] - 28s 415ms/step - loss: 0.6470 - categorical_accuracy: 0.7512 - val_loss: 0.6526 - val_categorical_accuracy: 0.7600\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 00029: saving model to model_init_2018-12-2708_18_49.548506/model-00029-0.64705-0.75124-0.65264-0.76000.h5\n",
      "Epoch 30/30\n",
      "67/67 [==============================] - 27s 396ms/step - loss: 0.7928 - categorical_accuracy: 0.6766 - val_loss: 0.6484 - val_categorical_accuracy: 0.7400\n",
      "\n",
      "Epoch 00030: saving model to model_init_2018-12-2708_18_49.548506/model-00030-0.79279-0.67662-0.64836-0.74000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f030974af98>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 30\n",
    "model_f.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model G Experiment [11]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_25 (Conv3D)           (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_15 (Batc (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_25 (Activation)   (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_19 (MaxPooling (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_26 (Conv3D)           (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_16 (Batc (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_26 (Activation)   (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_20 (MaxPooling (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_27 (Conv3D)           (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_17 (Batc (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_27 (Activation)   (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_21 (MaxPooling (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_28 (Conv3D)           (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_18 (Batc (None, 3, 15, 15, 64)     256       \n",
      "_________________________________________________________________\n",
      "activation_28 (Activation)   (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_22 (MaxPooling (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_6 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_14 (Dense)             (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_15 (Dropout)         (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_15 (Dense)             (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_16 (Dropout)         (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_16 (Dense)             (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,749\n",
      "Trainable params: 3,667,509\n",
      "Non-trainable params: 240\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height \n",
    "\n",
    "classes = 5\n",
    "channel = 3\n",
    "nb_dense = [1000, 500, 5]\n",
    "\n",
    "input_shape=(x,y,z,channel)\n",
    "\n",
    "# Define model\n",
    "model_g = Sequential()\n",
    "\n",
    "model_g.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_g.add(BatchNormalization())\n",
    "model_g.add(Activation('relu'))\n",
    "\n",
    "model_g.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_g.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_g.add(BatchNormalization())\n",
    "model_g.add(Activation('relu'))\n",
    "\n",
    "model_g.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_g.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_g.add(BatchNormalization())\n",
    "model_g.add(Activation('relu'))\n",
    "\n",
    "model_g.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_g.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_g.add(BatchNormalization())\n",
    "model_g.add(Activation('relu'))\n",
    "\n",
    "model_g.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_g.add(Flatten())\n",
    "\n",
    "model_g.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_g.add(Dropout(0.5))\n",
    "\n",
    "model_g.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_g.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_g.add(Dense(nb_dense[2], activation='softmax'))\n",
    "model_g.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_g.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n",
      "65/67 [============================>.] - ETA: 2s - loss: 4.9236 - categorical_accuracy: 0.2615Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 4.8691 - categorical_accuracy: 0.2652"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 93s 1s/step - loss: 4.8757 - categorical_accuracy: 0.2612 - val_loss: 1.8970 - val_categorical_accuracy: 0.3800\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-4.87111-0.26395-1.89698-0.38000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 28s 420ms/step - loss: 2.6849 - categorical_accuracy: 0.2836 - val_loss: 2.1399 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-2.68486-0.28358-2.13988-0.40000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 28s 424ms/step - loss: 1.9323 - categorical_accuracy: 0.3234 - val_loss: 1.4312 - val_categorical_accuracy: 0.3900\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-1.93227-0.32338-1.43115-0.39000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 1.9627 - categorical_accuracy: 0.2886 - val_loss: 1.3860 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-1.96267-0.28856-1.38605-0.40000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 28s 417ms/step - loss: 1.6315 - categorical_accuracy: 0.3483 - val_loss: 1.7017 - val_categorical_accuracy: 0.3100\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.63147-0.34826-1.70168-0.31000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 28s 419ms/step - loss: 1.6364 - categorical_accuracy: 0.2836 - val_loss: 1.4727 - val_categorical_accuracy: 0.3700\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.63637-0.28358-1.47269-0.37000.h5\n",
      "\n",
      "Epoch 00006: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 29s 433ms/step - loss: 1.4460 - categorical_accuracy: 0.3632 - val_loss: 1.4450 - val_categorical_accuracy: 0.3800\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.44597-0.36318-1.44499-0.38000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 30s 447ms/step - loss: 1.3116 - categorical_accuracy: 0.4378 - val_loss: 1.2210 - val_categorical_accuracy: 0.4500\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.31158-0.43781-1.22098-0.45000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 28s 416ms/step - loss: 1.3052 - categorical_accuracy: 0.3881 - val_loss: 1.1758 - val_categorical_accuracy: 0.4900\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.30522-0.38806-1.17578-0.49000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 28s 423ms/step - loss: 1.3696 - categorical_accuracy: 0.4080 - val_loss: 1.2841 - val_categorical_accuracy: 0.4500\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.36960-0.40796-1.28414-0.45000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f02d5c51588>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 10\n",
    "model_g.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model H Experiment [12]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_29 (Conv3D)           (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_19 (Batc (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_29 (Activation)   (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_23 (MaxPooling (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_30 (Conv3D)           (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_20 (Batc (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_30 (Activation)   (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_24 (MaxPooling (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_31 (Conv3D)           (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_21 (Batc (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_31 (Activation)   (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_25 (MaxPooling (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_32 (Conv3D)           (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_22 (Batc (None, 3, 15, 15, 64)     256       \n",
      "_________________________________________________________________\n",
      "activation_32 (Activation)   (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_26 (MaxPooling (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_7 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_17 (Dense)             (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_17 (Dropout)         (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_18 (Dense)             (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_18 (Dropout)         (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_19 (Dense)             (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,749\n",
      "Trainable params: 3,667,509\n",
      "Non-trainable params: 240\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height \n",
    "\n",
    "classes = 5\n",
    "channel = 3\n",
    "\n",
    "input_shape=(x,y,z,channel)\n",
    "\n",
    "# Define model\n",
    "model_h = Sequential()\n",
    "\n",
    "model_h.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_h.add(BatchNormalization())\n",
    "model_h.add(Activation('relu'))\n",
    "\n",
    "model_h.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_h.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_h.add(BatchNormalization())\n",
    "model_h.add(Activation('relu'))\n",
    "\n",
    "model_h.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_h.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_h.add(BatchNormalization())\n",
    "model_h.add(Activation('relu'))\n",
    "\n",
    "model_h.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_h.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_h.add(BatchNormalization())\n",
    "model_h.add(Activation('relu'))\n",
    "\n",
    "model_h.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_h.add(Flatten())\n",
    "\n",
    "model_h.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_h.add(Dropout(0.5))\n",
    "\n",
    "model_h.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_h.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_h.add(Dense(nb_dense[2], activation='softmax'))\n",
    "model_h.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_h.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n",
      "65/67 [============================>.] - ETA: 2s - loss: 5.3448 - categorical_accuracy: 0.2523Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 5.2818 - categorical_accuracy: 0.2576"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 90s 1s/step - loss: 5.2455 - categorical_accuracy: 0.2587 - val_loss: 1.8305 - val_categorical_accuracy: 0.3700\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2708_18_49.548506/model-00001-5.27065-0.25792-1.83051-0.37000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 2.3795 - categorical_accuracy: 0.2687 - val_loss: 1.9366 - val_categorical_accuracy: 0.2800\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2708_18_49.548506/model-00002-2.37949-0.26866-1.93659-0.28000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 29s 432ms/step - loss: 1.8573 - categorical_accuracy: 0.2289 - val_loss: 1.2823 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2708_18_49.548506/model-00003-1.85731-0.22886-1.28228-0.50000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 29s 429ms/step - loss: 1.5744 - categorical_accuracy: 0.3333 - val_loss: 1.2822 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2708_18_49.548506/model-00004-1.57439-0.33333-1.28219-0.46000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 27s 407ms/step - loss: 1.6230 - categorical_accuracy: 0.3433 - val_loss: 1.2829 - val_categorical_accuracy: 0.3900\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2708_18_49.548506/model-00005-1.62296-0.34328-1.28289-0.39000.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 28s 417ms/step - loss: 1.4459 - categorical_accuracy: 0.3930 - val_loss: 1.1707 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2708_18_49.548506/model-00006-1.44593-0.39303-1.17071-0.50000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 30s 444ms/step - loss: 1.4208 - categorical_accuracy: 0.3930 - val_loss: 1.2200 - val_categorical_accuracy: 0.4300\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2708_18_49.548506/model-00007-1.42081-0.39303-1.22004-0.43000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 29s 437ms/step - loss: 1.4187 - categorical_accuracy: 0.4378 - val_loss: 1.1907 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2708_18_49.548506/model-00008-1.41869-0.43781-1.19073-0.52000.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 0.25.\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 29s 428ms/step - loss: 1.3558 - categorical_accuracy: 0.3881 - val_loss: 1.0789 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2708_18_49.548506/model-00009-1.35583-0.38806-1.07891-0.65000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 29s 428ms/step - loss: 1.2422 - categorical_accuracy: 0.5174 - val_loss: 1.1642 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2708_18_49.548506/model-00010-1.24220-0.51741-1.16420-0.44000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f0308bdcd68>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 10\n",
    "model_h.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model I Experiment [13]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_41 (Conv3D)           (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_30 (Batc (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_41 (Activation)   (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_35 (MaxPooling (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_42 (Conv3D)           (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_31 (Batc (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_42 (Activation)   (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_36 (MaxPooling (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_43 (Conv3D)           (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_32 (Batc (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_43 (Activation)   (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_37 (MaxPooling (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_44 (Conv3D)           (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "activation_44 (Activation)   (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "dropout_24 (Dropout)         (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_38 (MaxPooling (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_10 (Flatten)         (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_26 (Dense)             (None, 1000)              3137000   \n",
      "_________________________________________________________________\n",
      "dropout_25 (Dropout)         (None, 1000)              0         \n",
      "_________________________________________________________________\n",
      "dense_27 (Dense)             (None, 500)               500500    \n",
      "_________________________________________________________________\n",
      "dropout_26 (Dropout)         (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_28 (Dense)             (None, 5)                 2505      \n",
      "=================================================================\n",
      "Total params: 3,667,493\n",
      "Trainable params: 3,667,381\n",
      "Non-trainable params: 112\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "x = 30 # number of frames\n",
    "y = 120 # image width\n",
    "z = 120 # image height \n",
    "\n",
    "classes = 5\n",
    "channel = 3\n",
    "\n",
    "input_shape=(x,y,z,channel)\n",
    "\n",
    "# Define model\n",
    "model_i = Sequential()\n",
    "\n",
    "model_i.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_i.add(BatchNormalization())\n",
    "model_i.add(Activation('relu'))\n",
    "\n",
    "model_i.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_i.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_i.add(BatchNormalization())\n",
    "model_i.add(Activation('relu'))\n",
    "\n",
    "model_i.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_i.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_i.add(BatchNormalization())\n",
    "model_i.add(Activation('relu'))\n",
    "\n",
    "model_i.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_i.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_i.add(Activation('relu'))\n",
    "model_i.add(Dropout(0.25))\n",
    "\n",
    "model_i.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_i.add(Flatten())\n",
    "\n",
    "model_i.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_i.add(Dropout(0.5))\n",
    "\n",
    "model_i.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_i.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_i.add(Dense(nb_dense[2], activation='softmax'))\n",
    "model_i.compile(optimizer=keras.optimizers.Adadelta(), loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "model_i.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 10\n",
      "Epoch 1/10\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 2.2121 - categorical_accuracy: 0.3246Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 2.2026 - categorical_accuracy: 0.3242"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 93s 1s/step - loss: 2.2024 - categorical_accuracy: 0.3195 - val_loss: 1.4507 - val_categorical_accuracy: 0.2800\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2709_21_34.860565/model-00001-2.20253-0.32278-1.45066-0.28000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 27s 401ms/step - loss: 1.5683 - categorical_accuracy: 0.3085 - val_loss: 1.8319 - val_categorical_accuracy: 0.2000\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2709_21_34.860565/model-00002-1.56827-0.30846-1.83194-0.20000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 30s 450ms/step - loss: 1.4762 - categorical_accuracy: 0.3383 - val_loss: 2.2867 - val_categorical_accuracy: 0.2500\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2709_21_34.860565/model-00003-1.47623-0.33831-2.28666-0.25000.h5\n",
      "\n",
      "Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.5.\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 27s 406ms/step - loss: 1.3299 - categorical_accuracy: 0.3980 - val_loss: 1.4109 - val_categorical_accuracy: 0.4200\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2709_21_34.860565/model-00004-1.32988-0.39801-1.41094-0.42000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 29s 426ms/step - loss: 1.2537 - categorical_accuracy: 0.4677 - val_loss: 1.1849 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2709_21_34.860565/model-00005-1.25365-0.46766-1.18492-0.46000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.2623 - categorical_accuracy: 0.4627 - val_loss: 1.0514 - val_categorical_accuracy: 0.6000\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2709_21_34.860565/model-00006-1.26231-0.46269-1.05137-0.60000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.1712 - categorical_accuracy: 0.4925 - val_loss: 1.0840 - val_categorical_accuracy: 0.5300\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2709_21_34.860565/model-00007-1.17115-0.49254-1.08395-0.53000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 29s 428ms/step - loss: 1.0876 - categorical_accuracy: 0.4826 - val_loss: 0.9662 - val_categorical_accuracy: 0.6400\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2709_21_34.860565/model-00008-1.08757-0.48259-0.96617-0.64000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 30s 451ms/step - loss: 1.1470 - categorical_accuracy: 0.5672 - val_loss: 0.9071 - val_categorical_accuracy: 0.6800\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2709_21_34.860565/model-00009-1.14702-0.56716-0.90711-0.68000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 29s 438ms/step - loss: 1.0127 - categorical_accuracy: 0.5323 - val_loss: 1.0271 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2709_21_34.860565/model-00010-1.01271-0.53234-1.02710-0.58000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f02d96d1940>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "num_epochs = 10\n",
    "model_i.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "nb_filters = [8,16,32,64]\n",
    "nb_dense = [256, 128, 5]\n",
    "\n",
    "# Input\n",
    "input_shape=(30,120,120,3)\n",
    "\n",
    "# Define model\n",
    "model_final = Sequential()\n",
    "\n",
    "model_final.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model_final.add(BatchNormalization())\n",
    "model_final.add(Activation('relu'))\n",
    "\n",
    "model_final.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_final.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model_final.add(BatchNormalization())\n",
    "model_final.add(Activation('relu'))\n",
    "\n",
    "model_final.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_final.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_final.add(BatchNormalization())\n",
    "model_final.add(Activation('relu'))\n",
    "\n",
    "model_final.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model_final.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model_final.add(Activation('relu'))\n",
    "model_final.add(Dropout(0.25))\n",
    "\n",
    "model_final.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model_final.add(Flatten())\n",
    "\n",
    "model_final.add(Dense(nb_dense[0], activation='relu'))\n",
    "model_final.add(Dropout(0.5))\n",
    "\n",
    "model_final.add(Dense(nb_dense[1], activation='relu'))\n",
    "model_final.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model_final.add(Dense(nb_dense[2], activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_5 (Conv3D)            (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_5 (Batch (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_5 (MaxPooling3 (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_6 (Conv3D)            (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_6 (Batch (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_6 (Activation)    (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_6 (MaxPooling3 (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_7 (Conv3D)            (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_7 (Batch (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_7 (Activation)    (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_7 (MaxPooling3 (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_8 (Conv3D)            (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "activation_8 (Activation)    (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_8 (MaxPooling3 (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_2 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_4 (Dense)              (None, 256)               803072    \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 5)                 645       \n",
      "=================================================================\n",
      "Total params: 864,101\n",
      "Trainable params: 863,989\n",
      "Non-trainable params: 112\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "optimiser = keras.optimizers.Adam() #write your optimizer\n",
    "model_final.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "print (model_final.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/valSource path =  /storage/Demo_Gesture/Project_data/train ; batch size =  ; batch size = 10\n",
      "10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 2s - loss: 1.5199 - categorical_accuracy: 0.3062Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 1.5198 - categorical_accuracy: 0.3045"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 95s 1s/step - loss: 1.5213 - categorical_accuracy: 0.3050 - val_loss: 1.5963 - val_categorical_accuracy: 0.2800\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2709_21_34.860565/model-00001-1.52027-0.30468-1.59634-0.28000.h5\n",
      "Epoch 2/20\n",
      "67/67 [==============================] - 27s 402ms/step - loss: 1.5324 - categorical_accuracy: 0.2687 - val_loss: 1.4435 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2709_21_34.860565/model-00002-1.53241-0.26866-1.44352-0.46000.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.5477 - categorical_accuracy: 0.3483 - val_loss: 1.3526 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2709_21_34.860565/model-00003-1.54766-0.34826-1.35262-0.44000.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 30s 443ms/step - loss: 1.4832 - categorical_accuracy: 0.2935 - val_loss: 1.5432 - val_categorical_accuracy: 0.3300\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2709_21_34.860565/model-00004-1.48322-0.29353-1.54321-0.33000.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.3592 - categorical_accuracy: 0.3930 - val_loss: 1.1995 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2709_21_34.860565/model-00005-1.35916-0.39303-1.19954-0.51000.h5\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 30s 441ms/step - loss: 1.4306 - categorical_accuracy: 0.3831 - val_loss: 1.2518 - val_categorical_accuracy: 0.4800\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2709_21_34.860565/model-00006-1.43062-0.38308-1.25183-0.48000.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 28s 423ms/step - loss: 1.4282 - categorical_accuracy: 0.3831 - val_loss: 1.4388 - val_categorical_accuracy: 0.2800\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2709_21_34.860565/model-00007-1.42819-0.38308-1.43884-0.28000.h5\n",
      "\n",
      "Epoch 00007: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 28s 414ms/step - loss: 1.3502 - categorical_accuracy: 0.4080 - val_loss: 1.2025 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2709_21_34.860565/model-00008-1.35024-0.40796-1.20250-0.54000.h5\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 28s 423ms/step - loss: 1.3292 - categorical_accuracy: 0.4527 - val_loss: 1.3174 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2709_21_34.860565/model-00009-1.32918-0.45274-1.31739-0.51000.h5\n",
      "\n",
      "Epoch 00009: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 30s 453ms/step - loss: 1.2583 - categorical_accuracy: 0.4428 - val_loss: 1.1597 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2709_21_34.860565/model-00010-1.25827-0.44279-1.15968-0.57000.h5\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 29s 426ms/step - loss: 1.1599 - categorical_accuracy: 0.5174 - val_loss: 1.0969 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2709_21_34.860565/model-00011-1.15992-0.51741-1.09695-0.56000.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 26s 395ms/step - loss: 1.2539 - categorical_accuracy: 0.4726 - val_loss: 1.1764 - val_categorical_accuracy: 0.5500\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2709_21_34.860565/model-00012-1.25386-0.47264-1.17643-0.55000.h5\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 30s 441ms/step - loss: 1.1396 - categorical_accuracy: 0.5323 - val_loss: 0.9895 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2709_21_34.860565/model-00013-1.13960-0.53234-0.98950-0.56000.h5\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 31s 457ms/step - loss: 1.1689 - categorical_accuracy: 0.4925 - val_loss: 1.0455 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2709_21_34.860565/model-00014-1.16892-0.49254-1.04550-0.58000.h5\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 29s 430ms/step - loss: 1.1754 - categorical_accuracy: 0.4726 - val_loss: 1.0635 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2709_21_34.860565/model-00015-1.17541-0.47264-1.06351-0.62000.h5\n",
      "\n",
      "Epoch 00015: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 29s 427ms/step - loss: 1.0500 - categorical_accuracy: 0.5871 - val_loss: 0.9580 - val_categorical_accuracy: 0.6000\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2709_21_34.860565/model-00016-1.04998-0.58706-0.95798-0.60000.h5\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 29s 438ms/step - loss: 1.0708 - categorical_accuracy: 0.5473 - val_loss: 0.9653 - val_categorical_accuracy: 0.6300\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2709_21_34.860565/model-00017-1.07076-0.54726-0.96525-0.63000.h5\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 28s 417ms/step - loss: 1.0375 - categorical_accuracy: 0.5323 - val_loss: 0.9205 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2709_21_34.860565/model-00018-1.03746-0.53234-0.92051-0.66000.h5\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 27s 398ms/step - loss: 1.0679 - categorical_accuracy: 0.5274 - val_loss: 0.9534 - val_categorical_accuracy: 0.6400\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2709_21_34.860565/model-00019-1.06786-0.52736-0.95342-0.64000.h5\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 29s 431ms/step - loss: 1.0108 - categorical_accuracy: 0.5721 - val_loss: 0.9330 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2709_21_34.860565/model-00020-1.01084-0.57214-0.93297-0.66000.h5\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f02d55448d0>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "num_epochs = 20\n",
    "model_final.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20Source path =  /storage/Demo_Gesture/Project_data/trainSource path =  /storage/Demo_Gesture/Project_data/val ; batch size = \n",
      " ; batch size = 10\n",
      "10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "65/67 [============================>.] - ETA: 3s - loss: 2.0843 - categorical_accuracy: 0.2092Batch:  67 Index: 10\n",
      "66/67 [============================>.] - ETA: 1s - loss: 2.0760 - categorical_accuracy: 0.2106"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67/67 [==============================] - 132s 2s/step - loss: 2.0679 - categorical_accuracy: 0.2075 - val_loss: 1.4500 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2809_14_05.984886/model-00001-2.07353-0.20965-1.45004-0.40000.h5\n",
      "Epoch 2/20\n",
      "67/67 [==============================] - 30s 454ms/step - loss: 1.6320 - categorical_accuracy: 0.2537 - val_loss: 1.5572 - val_categorical_accuracy: 0.3000\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2809_14_05.984886/model-00002-1.63199-0.25373-1.55723-0.30000.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 32s 471ms/step - loss: 1.5626 - categorical_accuracy: 0.2537 - val_loss: 1.4360 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2809_14_05.984886/model-00003-1.56261-0.25373-1.43603-0.44000.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 30s 449ms/step - loss: 1.5626 - categorical_accuracy: 0.3085 - val_loss: 1.4860 - val_categorical_accuracy: 0.3300\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2809_14_05.984886/model-00004-1.56257-0.30846-1.48600-0.33000.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 32s 470ms/step - loss: 1.5091 - categorical_accuracy: 0.3483 - val_loss: 1.4852 - val_categorical_accuracy: 0.3300\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2809_14_05.984886/model-00005-1.50912-0.34826-1.48522-0.33000.h5\n",
      "\n",
      "Epoch 00005: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 32s 476ms/step - loss: 1.4593 - categorical_accuracy: 0.4179 - val_loss: 1.2932 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2809_14_05.984886/model-00006-1.45927-0.41791-1.29323-0.50000.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 28s 421ms/step - loss: 1.3016 - categorical_accuracy: 0.4229 - val_loss: 1.2274 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2809_14_05.984886/model-00007-1.30162-0.42289-1.22737-0.54000.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 30s 445ms/step - loss: 1.2789 - categorical_accuracy: 0.4677 - val_loss: 1.2698 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2809_14_05.984886/model-00008-1.27890-0.46766-1.26977-0.52000.h5\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 32s 475ms/step - loss: 1.2862 - categorical_accuracy: 0.4627 - val_loss: 1.1820 - val_categorical_accuracy: 0.5600\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2809_14_05.984886/model-00009-1.28619-0.46269-1.18199-0.56000.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 31s 460ms/step - loss: 1.2354 - categorical_accuracy: 0.5124 - val_loss: 1.1585 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2809_14_05.984886/model-00010-1.23538-0.51244-1.15848-0.54000.h5\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 28s 424ms/step - loss: 1.2598 - categorical_accuracy: 0.4726 - val_loss: 1.1415 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2809_14_05.984886/model-00011-1.25978-0.47264-1.14146-0.58000.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 29s 436ms/step - loss: 1.0939 - categorical_accuracy: 0.5572 - val_loss: 1.0520 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2809_14_05.984886/model-00012-1.09386-0.55721-1.05201-0.54000.h5\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 30s 444ms/step - loss: 1.1421 - categorical_accuracy: 0.5473 - val_loss: 1.0302 - val_categorical_accuracy: 0.6200\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2809_14_05.984886/model-00013-1.14212-0.54726-1.03023-0.62000.h5\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 32s 471ms/step - loss: 1.2821 - categorical_accuracy: 0.4627 - val_loss: 1.1904 - val_categorical_accuracy: 0.5500\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2809_14_05.984886/model-00014-1.28206-0.46269-1.19038-0.55000.h5\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 31s 457ms/step - loss: 1.0114 - categorical_accuracy: 0.5970 - val_loss: 1.0153 - val_categorical_accuracy: 0.6300\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2809_14_05.984886/model-00015-1.01143-0.59701-1.01525-0.63000.h5\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 29s 432ms/step - loss: 1.0391 - categorical_accuracy: 0.5771 - val_loss: 1.0303 - val_categorical_accuracy: 0.5500\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2809_14_05.984886/model-00016-1.03912-0.57711-1.03034-0.55000.h5\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 31s 464ms/step - loss: 0.9844 - categorical_accuracy: 0.6070 - val_loss: 1.0619 - val_categorical_accuracy: 0.5200\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2809_14_05.984886/model-00017-0.98435-0.60697-1.06191-0.52000.h5\n",
      "\n",
      "Epoch 00017: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 29s 433ms/step - loss: 0.8989 - categorical_accuracy: 0.6418 - val_loss: 1.0069 - val_categorical_accuracy: 0.5400\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2809_14_05.984886/model-00018-0.89887-0.64179-1.00688-0.54000.h5\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 34s 514ms/step - loss: 0.7630 - categorical_accuracy: 0.6915 - val_loss: 0.8926 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2809_14_05.984886/model-00019-0.76301-0.69154-0.89263-0.65000.h5\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 36s 538ms/step - loss: 0.9781 - categorical_accuracy: 0.6318 - val_loss: 1.0669 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2809_14_05.984886/model-00020-0.97808-0.63184-1.06690-0.51000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f0e7f652320>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 32\n",
    "num_epochs = 20\n",
    "model_final.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 64\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch:  2 Index: 64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 2/67 [..............................] - ETA: 12:18 - loss: 0.9170 - categorical_accuracy: 0.6094Batch:  3 Index: 36\n",
      " 4/67 [>.............................] - ETA: 11:53 - loss: 0.8175 - categorical_accuracy: 0.6719Batch:  4 Index: 28\n",
      " 9/67 [===>..........................] - ETA: 10:26 - loss: 0.7591 - categorical_accuracy: 0.6979Batch:  11 Index: 64\n",
      "38/67 [================>.............] - ETA: 2:37 - loss: 0.7285 - categorical_accuracy: 0.7154Batch:  29 Index: 23\n",
      "67/67 [==============================] - 288s 4s/step - loss: 0.7238 - categorical_accuracy: 0.7199 - val_loss: 0.8679 - val_categorical_accuracy: 0.6424\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2809_14_05.984886/model-00001-0.73238-0.71452-0.86791-0.64241.h5\n",
      "Epoch 2/20\n",
      " 3/67 [>.............................] - ETA: 4:57 - loss: 0.6043 - categorical_accuracy: 0.7544Batch:  7 Index: 16\n",
      " 6/67 [=>............................] - ETA: 3:45 - loss: 0.6082 - categorical_accuracy: 0.7281Batch:  35 Index: 19\n",
      "67/67 [==============================] - 168s 3s/step - loss: 0.6806 - categorical_accuracy: 0.7329 - val_loss: 0.8305 - val_categorical_accuracy: 0.6800\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2809_14_05.984886/model-00002-0.67985-0.73287-0.83045-0.68000.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 156s 2s/step - loss: 0.5746 - categorical_accuracy: 0.7805 - val_loss: 0.9897 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2809_14_05.984886/model-00003-0.57463-0.78051-0.98971-0.67500.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 158s 2s/step - loss: 0.5510 - categorical_accuracy: 0.8016 - val_loss: 0.7652 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2809_14_05.984886/model-00004-0.55103-0.80158-0.76522-0.72500.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.4876 - categorical_accuracy: 0.8060 - val_loss: 0.9317 - val_categorical_accuracy: 0.5750\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2809_14_05.984886/model-00005-0.48763-0.80597-0.93166-0.57500.h5\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 188s 3s/step - loss: 0.3994 - categorical_accuracy: 0.8578 - val_loss: 0.6009 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2809_14_05.984886/model-00006-0.39943-0.85777-0.60085-0.72500.h5\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 187s 3s/step - loss: 0.3441 - categorical_accuracy: 0.8815 - val_loss: 0.6947 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2809_14_05.984886/model-00007-0.34407-0.88147-0.69469-0.80000.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 160s 2s/step - loss: 0.3093 - categorical_accuracy: 0.8920 - val_loss: 0.7053 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2809_14_05.984886/model-00008-0.30926-0.89201-0.70529-0.67500.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 158s 2s/step - loss: 0.2483 - categorical_accuracy: 0.9201 - val_loss: 0.7601 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2809_14_05.984886/model-00009-0.24828-0.92011-0.76013-0.67500.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.2282 - categorical_accuracy: 0.9245 - val_loss: 0.8542 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2809_14_05.984886/model-00010-0.22815-0.92450-0.85422-0.65000.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 160s 2s/step - loss: 0.2121 - categorical_accuracy: 0.9254 - val_loss: 0.8608 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2809_14_05.984886/model-00011-0.21214-0.92537-0.86081-0.67500.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 157s 2s/step - loss: 0.1790 - categorical_accuracy: 0.9438 - val_loss: 0.4916 - val_categorical_accuracy: 0.7750\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2809_14_05.984886/model-00012-0.17895-0.94381-0.49160-0.77500.h5\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 157s 2s/step - loss: 0.1917 - categorical_accuracy: 0.9377 - val_loss: 0.7943 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2809_14_05.984886/model-00013-0.19171-0.93766-0.79426-0.72500.h5\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 161s 2s/step - loss: 0.1585 - categorical_accuracy: 0.9508 - val_loss: 0.6877 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2809_14_05.984886/model-00014-0.15852-0.95083-0.68769-0.72500.h5\n",
      "\n",
      "Epoch 00014: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.1633 - categorical_accuracy: 0.9491 - val_loss: 0.8085 - val_categorical_accuracy: 0.6250\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2809_14_05.984886/model-00015-0.16334-0.94908-0.80847-0.62500.h5\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 156s 2s/step - loss: 0.1640 - categorical_accuracy: 0.9517 - val_loss: 0.8886 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2809_14_05.984886/model-00016-0.16403-0.95171-0.88864-0.72500.h5\n",
      "\n",
      "Epoch 00016: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.1524 - categorical_accuracy: 0.9631 - val_loss: 0.4283 - val_categorical_accuracy: 0.8250\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2809_14_05.984886/model-00017-0.15243-0.96313-0.42828-0.82500.h5\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 158s 2s/step - loss: 0.1506 - categorical_accuracy: 0.9570 - val_loss: 0.8015 - val_categorical_accuracy: 0.6750\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2809_14_05.984886/model-00018-0.15061-0.95698-0.80155-0.67500.h5\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 157s 2s/step - loss: 0.1380 - categorical_accuracy: 0.9587 - val_loss: 0.5924 - val_categorical_accuracy: 0.7750\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2809_14_05.984886/model-00019-0.13805-0.95874-0.59244-0.77500.h5\n",
      "\n",
      "Epoch 00019: ReduceLROnPlateau reducing learning rate to 7.812500371073838e-06.\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.1287 - categorical_accuracy: 0.9675 - val_loss: 0.8443 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2809_14_05.984886/model-00020-0.12869-0.96752-0.84429-0.75000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f0e64691f28>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 64\n",
    "num_epochs = 20\n",
    "\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "\n",
    "\n",
    "model_final.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source path =  /storage/Demo_Gesture/Project_data/val ; batch size = 40\n",
      "Epoch 1/20\n",
      "Source path =  /storage/Demo_Gesture/Project_data/train ; batch size = 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:19: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:24: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 2/67 [..............................] - ETA: 7:32 - loss: 0.1220 - categorical_accuracy: 0.9625Batch:  3 Index: 40\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:44: DeprecationWarning: `imread` is deprecated!\n",
      "`imread` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``imageio.imread`` instead.\n",
      "/usr/local/lib/python3.5/dist-packages/ipykernel_launcher.py:48: DeprecationWarning: `imresize` is deprecated!\n",
      "`imresize` is deprecated in SciPy 1.0.0, and will be removed in 1.2.0.\n",
      "Use ``skimage.transform.resize`` instead.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15/67 [=====>........................] - ETA: 5:10 - loss: 0.1416 - categorical_accuracy: 0.9617Batch:  17 Index: 40\n",
      "44/67 [==================>...........] - ETA: 1:35 - loss: 0.1355 - categorical_accuracy: 0.9696Batch:  29 Index: 23\n",
      "67/67 [==============================] - 246s 4s/step - loss: 0.1449 - categorical_accuracy: 0.9592 - val_loss: 0.7256 - val_categorical_accuracy: 0.7208\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-12-2809_14_05.984886/model-00001-0.14324-0.96058-0.72556-0.72083.h5\n",
      "Epoch 2/20\n",
      "12/67 [====>.........................] - ETA: 2:34 - loss: 0.1099 - categorical_accuracy: 0.9737Batch:  35 Index: 19\n",
      "67/67 [==============================] - 161s 2s/step - loss: 0.1451 - categorical_accuracy: 0.9611 - val_loss: 0.7605 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-12-2809_14_05.984886/model-00002-0.14431-0.96137-0.76048-0.71500.h5\n",
      "Epoch 3/20\n",
      "67/67 [==============================] - 162s 2s/step - loss: 0.1430 - categorical_accuracy: 0.9596 - val_loss: 0.7225 - val_categorical_accuracy: 0.7450\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-12-2809_14_05.984886/model-00003-0.14302-0.95961-0.72253-0.74500.h5\n",
      "Epoch 4/20\n",
      "67/67 [==============================] - 163s 2s/step - loss: 0.1428 - categorical_accuracy: 0.9587 - val_loss: 0.6893 - val_categorical_accuracy: 0.7350\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-12-2809_14_05.984886/model-00004-0.14285-0.95874-0.68931-0.73500.h5\n",
      "Epoch 5/20\n",
      "67/67 [==============================] - 161s 2s/step - loss: 0.1465 - categorical_accuracy: 0.9579 - val_loss: 0.7625 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-12-2809_14_05.984886/model-00005-0.14652-0.95786-0.76251-0.71500.h5\n",
      "Epoch 6/20\n",
      "67/67 [==============================] - 166s 2s/step - loss: 0.1392 - categorical_accuracy: 0.9587 - val_loss: 0.7364 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-12-2809_14_05.984886/model-00006-0.13924-0.95874-0.73642-0.72000.h5\n",
      "\n",
      "Epoch 00006: ReduceLROnPlateau reducing learning rate to 3.906250185536919e-06.\n",
      "Epoch 7/20\n",
      "67/67 [==============================] - 163s 2s/step - loss: 0.1329 - categorical_accuracy: 0.9614 - val_loss: 0.7666 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-12-2809_14_05.984886/model-00007-0.13287-0.96137-0.76660-0.72500.h5\n",
      "Epoch 8/20\n",
      "67/67 [==============================] - 165s 2s/step - loss: 0.1455 - categorical_accuracy: 0.9587 - val_loss: 0.6951 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-12-2809_14_05.984886/model-00008-0.14549-0.95874-0.69512-0.72500.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 1.9531250927684596e-06.\n",
      "Epoch 9/20\n",
      "67/67 [==============================] - 164s 2s/step - loss: 0.1401 - categorical_accuracy: 0.9675 - val_loss: 0.8202 - val_categorical_accuracy: 0.6850\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-12-2809_14_05.984886/model-00009-0.14014-0.96752-0.82018-0.68500.h5\n",
      "Epoch 10/20\n",
      "67/67 [==============================] - 167s 2s/step - loss: 0.1207 - categorical_accuracy: 0.9763 - val_loss: 0.7244 - val_categorical_accuracy: 0.7350\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-12-2809_14_05.984886/model-00010-0.12070-0.97629-0.72444-0.73500.h5\n",
      "\n",
      "Epoch 00010: ReduceLROnPlateau reducing learning rate to 9.765625463842298e-07.\n",
      "Epoch 11/20\n",
      "67/67 [==============================] - 163s 2s/step - loss: 0.1600 - categorical_accuracy: 0.9517 - val_loss: 0.7016 - val_categorical_accuracy: 0.7250\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-12-2809_14_05.984886/model-00011-0.16005-0.95171-0.70164-0.72500.h5\n",
      "Epoch 12/20\n",
      "67/67 [==============================] - 162s 2s/step - loss: 0.1440 - categorical_accuracy: 0.9570 - val_loss: 0.7350 - val_categorical_accuracy: 0.7350\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-12-2809_14_05.984886/model-00012-0.14397-0.95698-0.73495-0.73500.h5\n",
      "\n",
      "Epoch 00012: ReduceLROnPlateau reducing learning rate to 4.882812731921149e-07.\n",
      "Epoch 13/20\n",
      "67/67 [==============================] - 163s 2s/step - loss: 0.1317 - categorical_accuracy: 0.9605 - val_loss: 0.7422 - val_categorical_accuracy: 0.7100\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-12-2809_14_05.984886/model-00013-0.13172-0.96049-0.74220-0.71000.h5\n",
      "Epoch 14/20\n",
      "67/67 [==============================] - 162s 2s/step - loss: 0.1372 - categorical_accuracy: 0.9614 - val_loss: 0.7474 - val_categorical_accuracy: 0.7150\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-12-2809_14_05.984886/model-00014-0.13720-0.96137-0.74744-0.71500.h5\n",
      "\n",
      "Epoch 00014: ReduceLROnPlateau reducing learning rate to 2.4414063659605745e-07.\n",
      "Epoch 15/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.1320 - categorical_accuracy: 0.9579 - val_loss: 0.7237 - val_categorical_accuracy: 0.7400\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-12-2809_14_05.984886/model-00015-0.13202-0.95786-0.72374-0.74000.h5\n",
      "Epoch 16/20\n",
      "67/67 [==============================] - 161s 2s/step - loss: 0.1367 - categorical_accuracy: 0.9605 - val_loss: 0.7725 - val_categorical_accuracy: 0.6950\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-12-2809_14_05.984886/model-00016-0.13672-0.96049-0.77253-0.69500.h5\n",
      "\n",
      "Epoch 00016: ReduceLROnPlateau reducing learning rate to 1.2207031829802872e-07.\n",
      "Epoch 17/20\n",
      "67/67 [==============================] - 164s 2s/step - loss: 0.1331 - categorical_accuracy: 0.9614 - val_loss: 0.7171 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-12-2809_14_05.984886/model-00017-0.13311-0.96137-0.71714-0.72000.h5\n",
      "Epoch 18/20\n",
      "67/67 [==============================] - 160s 2s/step - loss: 0.1568 - categorical_accuracy: 0.9596 - val_loss: 0.7666 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-12-2809_14_05.984886/model-00018-0.15678-0.95961-0.76662-0.72000.h5\n",
      "\n",
      "Epoch 00018: ReduceLROnPlateau reducing learning rate to 6.103515914901436e-08.\n",
      "Epoch 19/20\n",
      "67/67 [==============================] - 162s 2s/step - loss: 0.1435 - categorical_accuracy: 0.9622 - val_loss: 0.7280 - val_categorical_accuracy: 0.7200\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-12-2809_14_05.984886/model-00019-0.14353-0.96225-0.72797-0.72000.h5\n",
      "Epoch 20/20\n",
      "67/67 [==============================] - 159s 2s/step - loss: 0.1333 - categorical_accuracy: 0.9614 - val_loss: 0.7114 - val_categorical_accuracy: 0.7350\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-12-2809_14_05.984886/model-00020-0.13327-0.96137-0.71142-0.73500.h5\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 3.051757957450718e-08.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7f0e64681470>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 40\n",
    "num_epochs = 20\n",
    "\n",
    "train_generator = generator_ex(train_path, train_doc, batch_size)\n",
    "val_generator = generator_ex(val_path, val_doc, batch_size)\n",
    "\n",
    "\n",
    "model_final.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}