Siamese_Network_on_MNIST.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Siamese Network on MNIST.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "QOs9ALvXvdVa",
        "-ZsXdgc3vjTy",
        "PSP_nyLiMSVl",
        "UbHCPncfNEdA",
        "zFhR0MfbNOTr",
        "llCeVaXCVpcW",
        "NiuZWv1ZuPU0",
        "A3Ve0vaL14vA",
        "rTo1-5vIC1wV",
        "RVxIVZV2IHO1",
        "XqHK752HUhnY",
        "fIZTe9lCr_px"
      ],
      "authorship_tag": "ABX9TyMGdH7WxOyLEb+OD6HXFKu3",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Chirag-Shilwant/Classification-using-Siamese-Network-on-MNIST-Dataset/blob/main/Siamese_Network_on_MNIST.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VNskEAYDkgD8"
      },
      "source": [
        "import numpy as np\n",
        "from keras.datasets import mnist\n",
        "import matplotlib.pyplot as plt\n",
        "from tensorflow.keras import layers, models\n",
        "import keras as ks\n",
        "import tensorflow as tf\n",
        "import pickle\n",
        "from tensorflow.keras.layers import Activation\n",
        "from tensorflow.keras.layers import Flatten\n",
        "from tensorflow.keras.layers import Dense\n",
        "from tensorflow.keras.layers import Reshape\n",
        "from tensorflow.keras.layers import Input\n",
        "from tensorflow.keras.models import Model\n",
        "from tensorflow.keras.layers import Dot\n",
        "from tensorflow.keras.layers import Lambda\n",
        "from tensorflow.keras import backend as K\n",
        "import keras\n",
        "import random\n",
        "import tensorflow_addons as tfa"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2BQOYeqoQ953",
        "outputId": "00607db0-1c4a-43a3-e5bc-4b42ff46faa9"
      },
      "source": [
        "!pip install -q -U tensorflow-addons"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[?25l\r\u001b[K     |▌                               | 10kB 19.3MB/s eta 0:00:01\r\u001b[K     |█                               | 20kB 14.6MB/s eta 0:00:01\r\u001b[K     |█▍                              | 30kB 10.9MB/s eta 0:00:01\r\u001b[K     |█▉                              | 40kB 8.6MB/s eta 0:00:01\r\u001b[K     |██▎                             | 51kB 4.4MB/s eta 0:00:01\r\u001b[K     |██▉                             | 61kB 4.9MB/s eta 0:00:01\r\u001b[K     |███▎                            | 71kB 5.1MB/s eta 0:00:01\r\u001b[K     |███▊                            | 81kB 5.5MB/s eta 0:00:01\r\u001b[K     |████▏                           | 92kB 5.4MB/s eta 0:00:01\r\u001b[K     |████▋                           | 102kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████▏                          | 112kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████▋                          | 122kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████                          | 133kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████▌                         | 143kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████                         | 153kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████▌                        | 163kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████                        | 174kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████▍                       | 184kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████▉                       | 194kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████▎                      | 204kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████▉                      | 215kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████▎                     | 225kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████▊                     | 235kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████▏                    | 245kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████▋                    | 256kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████▏                   | 266kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████▋                   | 276kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████                   | 286kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████▌                  | 296kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████                  | 307kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████▍                 | 317kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████                 | 327kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████▍                | 337kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████▉                | 348kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████▎               | 358kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████▊               | 368kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████▎              | 378kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████▊              | 389kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████▏             | 399kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████▋             | 409kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████             | 419kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████▋            | 430kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████            | 440kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████▌           | 450kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████           | 460kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████▍          | 471kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████          | 481kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████▍         | 491kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████▉         | 501kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████▎        | 512kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████▊        | 522kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████▎       | 532kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████▊       | 542kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▏      | 552kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████████▋      | 563kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████████      | 573kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▋     | 583kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████████     | 593kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████████▌    | 604kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████████    | 614kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▍   | 624kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████████▉   | 634kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▍  | 645kB 5.9MB/s eta 0:00:01\r\u001b[K     |█████████████████████████████▉  | 655kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▎ | 665kB 5.9MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▊ | 675kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▏| 686kB 5.9MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▊| 696kB 5.9MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 706kB 5.9MB/s \n",
            "\u001b[?25h"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QOs9ALvXvdVa"
      },
      "source": [
        "# **Loading the Dataset**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LpcxQVaMkgAT"
      },
      "source": [
        "(X_train, Y_train), (X_test, Y_test) = mnist.load_data()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iCMEPeCbkf58"
      },
      "source": [
        "# Normalizing\n",
        "X_train = X_train.astype('float32')\n",
        "X_train /= 255\n",
        "X_train = X_train.reshape((len(X_train), np.prod(X_train.shape[1:])))\n",
        "X_test = X_test.astype('float32')\n",
        "X_test /= 255\n",
        "X_test = X_test.reshape((len(X_test), np.prod(X_test.shape[1:])))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NKTc1axnkf9P",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a8329ffe-edc8-4eff-f7e3-ff9ecce48bbd"
      },
      "source": [
        "print(\"Shape of training data:\")\n",
        "print(X_train.shape)\n",
        "print(Y_train.shape)\n",
        "\n",
        "print(\"Shape of test data:\")\n",
        "print(X_test.shape)\n",
        "print(Y_test.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Shape of training data:\n",
            "(60000, 784)\n",
            "(60000,)\n",
            "Shape of test data:\n",
            "(10000, 784)\n",
            "(10000,)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ZsXdgc3vjTy"
      },
      "source": [
        "# **Making Pairs for One Shot Learning**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TAbKQUlOM8nr"
      },
      "source": [
        "class Pairs:\n",
        "  def makePairs(self, x, y):\n",
        "    num_classes = 10\n",
        "    digit_indices = [np.where(y == i)[0] for i in range(num_classes)]\n",
        "\n",
        "    pairs = list()\n",
        "    labels = list()\n",
        "\n",
        "    for idx1 in range(len(x)):\n",
        "        x1 = x[idx1]\n",
        "        label1 = y[idx1]\n",
        "        idx2 = random.choice(digit_indices[label1])\n",
        "        x2 = x[idx2]\n",
        "        \n",
        "        labels += list([1])\n",
        "        pairs += [[x1, x2]]\n",
        "    \n",
        "        label2 = random.randint(0, num_classes-1)\n",
        "        while label2 == label1:\n",
        "            label2 = random.randint(0, num_classes-1)\n",
        "\n",
        "        idx2 = random.choice(digit_indices[label2])\n",
        "        x2 = x[idx2]\n",
        "        \n",
        "        labels += list([0])\n",
        "        pairs += [[x1, x2]]\n",
        "        \n",
        "    return np.array(pairs), np.array(labels)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iE10bm2e06N-"
      },
      "source": [
        "p = Pairs()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JVfxWqalQCMp"
      },
      "source": [
        "pairs_train, labels_train = p.makePairs(X_train, Y_train)\n",
        "pairs_test, labels_test = p.makePairs(X_test, Y_test)\n",
        "labels_train = labels_train.astype('float32')\n",
        "labels_test = labels_test.astype('float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSP_nyLiMSVl"
      },
      "source": [
        "# **Siamese Network**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tvu75RuYRTKj"
      },
      "source": [
        "def euclideanDistance(v):\n",
        "    x, y = v\n",
        "    sum_square = K.sum(K.square(x - y), axis=1, keepdims=True)\n",
        "    return K.sqrt(K.maximum(sum_square, K.epsilon()))\n",
        "\n",
        "def eucl_dist_output_shape(shapes):\n",
        "    shape1, shape2 = shapes\n",
        "    return (shape1[0], 1)\n",
        "\n",
        "def contrastive_loss(y_original, y_pred):\n",
        "    sqaure_pred = K.square(y_pred)\n",
        "    margin = 1\n",
        "    margin_square = K.square(K.maximum(0, margin - y_pred))\n",
        "    return K.mean(y_original * sqaure_pred + (1 - y_original) * margin_square)\n",
        "\n",
        "def compute_accuracy(y_original, y_pred):\n",
        "    pred = y_pred.ravel() < 0.5\n",
        "    return np.mean(pred == y_original)\n",
        "\n",
        "def accuracy(y_original, y_pred):\n",
        "    return K.mean(K.equal(y_original, K.cast(y_pred < 0.5, y_original.dtype)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "njW6iN9B2plv"
      },
      "source": [
        "input = Input(shape=(784,))\n",
        "x = Flatten()(input)\n",
        "x = Dense(64, activation='relu')(x)\n",
        "x = Dense(128, activation='relu')(x)\n",
        "x = Dense(256, activation='relu')(x)\n",
        "x = Lambda(lambda  x: K.l2_normalize(x,axis=1))(x)\n",
        "x = Lambda(lambda  x: K.l2_normalize(x,axis=1))(x)\n",
        "dense = Model(input, x)\n",
        "\n",
        "\n",
        "input1 = Input(shape=(784,))\n",
        "input2 = Input(shape=(784,))\n",
        "dense1 = dense(input1)\n",
        "dense2 = dense(input2)\n",
        "\n",
        "distance = Lambda(euclideanDistance, output_shape=eucl_dist_output_shape)([dense1, dense2])\n",
        "model = Model([input1, input2], distance)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UbHCPncfNEdA"
      },
      "source": [
        "# **Contrastive loss**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "95k-QnamSYf7",
        "outputId": "156ed6a6-aeb5-43b1-827a-1ff3cb366333"
      },
      "source": [
        "model.compile(loss = contrastive_loss, optimizer=\"adam\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_60\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_92 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_93 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_59 (Functional)           (None, 256)          91584       input_92[0][0]                   \n",
            "                                                                 input_93[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "lambda_86 (Lambda)              (None, 1)            0           model_59[0][0]                   \n",
            "                                                                 model_59[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "66vLX5PkTOjd",
        "outputId": "f5b80563-623a-45aa-b34e-f691d8546eba"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 8ms/step - loss: 0.0933 - accuracy: 0.8839 - val_loss: 0.0379 - val_accuracy: 0.9595\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0277 - accuracy: 0.9707 - val_loss: 0.0292 - val_accuracy: 0.9643\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0179 - accuracy: 0.9808 - val_loss: 0.0250 - val_accuracy: 0.9693\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0132 - accuracy: 0.9855 - val_loss: 0.0242 - val_accuracy: 0.9708\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0105 - accuracy: 0.9891 - val_loss: 0.0247 - val_accuracy: 0.9692\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0085 - accuracy: 0.9912 - val_loss: 0.0236 - val_accuracy: 0.9710\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0072 - accuracy: 0.9928 - val_loss: 0.0247 - val_accuracy: 0.9696\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0070 - accuracy: 0.9928 - val_loss: 0.0241 - val_accuracy: 0.9715\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0060 - accuracy: 0.9939 - val_loss: 0.0256 - val_accuracy: 0.9692\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0055 - accuracy: 0.9946 - val_loss: 0.0227 - val_accuracy: 0.9729\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7XEJZNaDTORH"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sHkwfafchjCO",
        "outputId": "1ea409d0-2756-4a1b-9b78-6eaacf529970"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 97.28999999999999\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "GFNgacH8G9FB",
        "outputId": "7f409a4c-35ca-4baa-d3b4-32d266a7af6f"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5fn/8fc9k2SyJ5MFBcIqiCCERFlFcYFU3LHVr3td2lqtFpcuale/trW1v1apra3VurX6FaxVpEVFFJVVKigICMgWIMgSsu/L5P79MZOQhAkJJJOT5X5d11w5c84z59wzynzmOc9ZRFUxxhhjmnM5XYAxxpiuyQLCGGNMUBYQxhhjgrKAMMYYE5QFhDHGmKAsIIwxxgRlAWG6FRF5S0Ru7Oi2ThKRbBGZ7nQd0LVqMc4Lc7oA0/OJSGmjp9FAFeALPP+2qr7U1nWp6gWhaNsVichbwFmBpx5AgerA8xdV9bZjXN+DwDBVvb7DijQ9mgWECTlVja2fFpFs4Juq+m7zdiISpqq1nVlbV9Y44ETkeSBHVX/iXEWmt7FdTMYxInKOiOSIyH0ish94TkS8IvIfEckVkYLAdFqj13wgIt8MTN8kIstE5HeBtjtF5ILjbDtERJaISImIvCsiT4jIiy3U3ZYafyEiywPre0dEUhotv0FEdolInoj8+Dg/u4tFZK2IFIrIChFJb7TsPhHZG9j2FhGZJiIzgB8BV4lIqYisa8M2PCIyW0S+DDxmi4gnsCwl8L4LRSRfRJaKiKul7R/PezTOs4AwTjsRSAIGAbfi/3/yucDzgUAF8KejvH4isAVIAX4LPCMichxt/w/4L5AMPAjccJRttqXGa4GbgT5ABPB9ABEZBfwlsP5+ge2lcQxEJBN4Fvh24PV/BeYHvtBHAHcC41U1DjgfyFbVt4GHgbmqGquqY9uwqR8Dk4AMYCwwAajvwXwPyAFSgRPwh4+2tP1jeX+m67CAME6rA36uqlWqWqGqear6L1UtV9US4FfA2Ud5/S5VfVpVfcALQF/8X1htbisiA4HxwM9UtVpVlwHzW9pgG2t8TlW/UNUK4BX8X7IAVwD/UdUlqloF/DTwGRyLW4G/quoqVfWp6gv4x3Um4R/b8QCjRCRcVbNVdfsxrr/edcBDqnpQVXOB/+VwcNbg//wGqWqNqi5V/4XdOnL7xmEWEMZpuapaWf9ERKJF5K+BXTDFwBIgUUTcLbx+f/2EqpYHJmOPsW0/IL/RPIA9LRXcxhr3N5oub1RTv8brVtUyIK+lbbVgEPC9wO6dQhEpBAYA/VR1G3A3/l7QQRGZIyL9jnH99foBuxo93xWYB/D/gG3AOyKyQ0TuD7yfjty+cZgFhHFa88sJfw8YAUxU1XhgamB+S7uNOsI+IElEohvNG3CU9u2pcV/jdQe2mXxs5bIH+JWqJjZ6RKvqywCq+n+qeib+IFHgkcDrjvXSzV8G1lFvYGAeqlqiqt9T1aHApcC99WMNR9m+6WYsIExXE4d/n36hiCQBPw/1BlV1F7AaeFBEIkRkMnBJiGp8FbhYRM4UkQjgIY793+HTwG0iMlH8YkTkIhGJE5ERInJeYDC5MlBn/S6sA8Dg+sHkNngZ+ImIpAYG2X8GvAgNg+TDAmM4Rfh3LdW1sn3TzVhAmK5mNhAFHAI+At7upO1eB0zGv7vnl8Bc/Pv1gznuGlV1I3AH/kHxfUAB/sHeNlPV1cC38A+MF+Df1XNTYLEH+E2gtv34B8kfCCz7Z+Bvnoh80oZN/RJ/cH4GrAc+CcwDGA68C5QCK4E/q+r7rWzfdDNiNwwy5kgiMhfYrKoh78EY01VZD8IYQETGi8hJIuIKnDNwGTDP6bqMcZKdSW2M34nAa/gHjHOA21X1U2dLMsZZtovJGGNMULaLyRhjTFA9ZhdTSkqKDh482OkyjDGmW1mzZs0hVU0NtqzHBMTgwYNZvXq102UYY0y3IiK7Wlpmu5iMMcYEZQFhjDEmKAsIY4wxQfWYMQhjTNdVU1NDTk4OlZWVrTc2IREZGUlaWhrh4eFtfo0FhDEm5HJycoiLi2Pw4MG0fD8nEyqqSl5eHjk5OQwZMqTNr7NdTMaYkKusrCQ5OdnCwSEiQnJy8jH34CwgjDGdwsLBWcfz+ff6gKis3M2OHT+isrLFG4gZY0yvFNKAEJEZIrJFRLbV35Kw2XKPiMwNLF8lIoMbLUsXkZUislFE1otIZChq9PlK2L371+TnLwzF6o0xXUBeXh4ZGRlkZGRw4okn0r9//4bn1dXVR33t6tWrmTVrVqvbOOOMMzqk1g8++ICLL764Q9bVXiEbpA7cn/cJIAv/1TE/FpH5qvp5o2bfAApUdZiIXI3/1oRXiUgY/jtX3aCq60QkGf9N0jtcdPQoIiL6UlDwLv36fTMUmzDGOCw5OZm1a9cC8OCDDxIbG8v3v//9huW1tbWEhQX/Ohw3bhzjxo1rdRsrVqzomGK7kFD2ICYA21R1h6pWA3PwX2O/scuAFwLTrwLTArcw/ArwmaquA1DVPFX1haJIEcHrnU5h4Xuo2p0RjektbrrpJm677TYmTpzID3/4Q/773/8yefJkMjMzOeOMM9iyZQvQ9Bf9gw8+yC233MI555zD0KFDefzxxxvWFxsb29D+nHPO4YorruCUU07huuuuo/6q2W+++SannHIKp59+OrNmzWq1p5Cfn8/MmTNJT09n0qRJfPbZZwB8+OGHDT2gzMxMSkpK2LdvH1OnTiUjI4PRo0ezdOnSdn9GoTzMtT/+m6vXywEmttRGVWtFpAj/9fhPBlREFgKpwBxV/W3zDYjIrcCtAAMHDjzuQr3eLA4c+AelpeuIi8s87vUYY1q3devdlJau7dB1xsZmMHz47GN+XU5ODitWrMDtdlNcXMzSpUsJCwvj3Xff5Uc/+hH/+te/jnjN5s2bef/99ykpKWHEiBHcfvvtR5xb8Omnn7Jx40b69evHlClTWL58OePGjePb3/42S5YsYciQIVxzzTWt1vfzn/+czMxM5s2bx+LFi/n617/O2rVr+d3vfscTTzzBlClTKC0tJTIykqeeeorzzz+fH//4x/h8PsrLy4/582iuq54HEQacCYwHyoH3RGSNqr7XuJGqPgU8BTBu3LjjvrGF1zsdgIKCRRYQxvQiV155JW63G4CioiJuvPFGtm7diohQUxN8r/ZFF12Ex+PB4/HQp08fDhw4QFpaWpM2EyZMaJiXkZFBdnY2sbGxDB06tOE8hGuuuYannnrqqPUtW7asIaTOO+888vLyKC4uZsqUKdx7771cd911fPWrXyUtLY3x48dzyy23UFNTw8yZM8nIyGjXZwOhDYi9wIBGz9MC84K1yQmMOyTgv2l8DrBEVQ8BiMibwGnAe4SAx9OX6OhTKShYxMCBPwzFJowxAcfzSz9UYmJiGqZ/+tOfcu655/L666+TnZ3NOeecE/Q1Ho+nYdrtdlNbW3tcbdrj/vvv56KLLuLNN99kypQpLFy4kKlTp7JkyRIWLFjATTfdxL333svXv/71dm0nlGMQHwPDRWSIiEQAVwPzm7WZD9wYmL4CWKz+nXULgTEiEh0IjrOBzwmhpKQsCguX4vNVhHIzxpguqqioiP79+wPw/PPPd/j6R4wYwY4dO8jOzgZg7ty5rb7mrLPO4qWXXgL8YxspKSnEx8ezfft2xowZw3333cf48ePZvHkzu3bt4oQTTuBb3/oW3/zmN/nkk0/aXXPIAkJVa4E78X/ZbwJeUdWNIvKQiFwaaPYMkCwi24B7gfsDry0AHsUfMmuBT1R1QahqBf9uJtUqioqWh3Izxpgu6oc//CEPPPAAmZmZHf6LHyAqKoo///nPzJgxg9NPP524uDgSEhKO+poHH3yQNWvWkJ6ezv33388LL/iP6Zk9ezajR48mPT2d8PBwLrjgAj744APGjh1LZmYmc+fO5a677mp3zT3mntTjxo3T9twwqLa2lOXLk0hLu4eTTnqkAyszxmzatImRI0c6XYbjSktLiY2NRVW54447GD58OPfcc0+nbT/Yf4fA+G7Q43h7/ZnU9cLCYomPn0xBwbtOl2KM6aGefvppMjIyOPXUUykqKuLb3/620yUdVVc9iskRXu90srN/TnX1ISIiUpwuxxjTw9xzzz2d2mNoL+tBNOL1ZgFKYeFip0sxxhjHWUA0Ehc3Drc7gYKCRU6XYowxjrOAaMTlCsPrPZf8/EX0lMF7Y4w5XhYQzXi9WVRV7aKiYrvTpRhjjKMsIJrxj0Ngu5mM6UHac7lv8J+k1tLVWp9//nnuvPPOji65S7CjmJqJihqGxzOQgoJF9O9/u9PlGGM6QGuX+27NBx98QGxsbIfd86G7sB5EM/7Lf2dRULCYEF1h3BjTBaxZs4azzz6b008/nfPPP599+/YB8PjjjzNq1CjS09O5+uqryc7O5sknn+Sxxx4jIyPjqJfRzs7O5rzzziM9PZ1p06axe/duAP75z38yevRoxo4dy9SpUwHYuHEjEyZMICMjg/T0dLZu3Rr6N32MrAcRhNc7nf37n6GkZDXx8c2vUG6MaY+7376btfs79nLfGSdmMHtG2y8CqKp897vf5Y033iA1NZW5c+fy4x//mGeffZbf/OY37Ny5E4/HQ2FhIYmJidx2221t6nV897vf5cYbb+TGG2/k2WefZdasWcybN4+HHnqIhQsX0r9/fwoLCwF48sknueuuu7juuuuorq7G5+t6P0itBxGE1zsNwM6qNqaHqqqqYsOGDWRlZZGRkcEvf/lLcnJyAEhPT+e6667jxRdfbPEucy1ZuXIl1157LQA33HADy5YtA2DKlCncdNNNPP300w1BMHnyZB5++GEeeeQRdu3aRVRUVAe+w45hPYggIiJSiY3NJD9/EYMG/djpcozpUY7ll36oqCqnnnoqK1euPGLZggULWLJkCf/+97/51a9+xfr169u9vSeffJJVq1axYMECTj/9dNasWcO1117LxIkTWbBgARdeeCF//etfOe+889q9rY5kPYgWeL3TKS5eQW1tqdOlGGM6mMfjITc3tyEgampq2LhxI3V1dezZs4dzzz2XRx55hKKiIkpLS4mLi6OkpKTV9Z5xxhnMmTMHgJdeeomzzjoLgO3btzNx4kQeeughUlNT2bNnDzt27GDo0KHMmjWLyy67rOF2ol2JBUQLvN4sVGsoKmr/fV2NMV2Ly+Xi1Vdf5b777mPs2LFkZGSwYsUKfD4f119/PWPGjCEzM5NZs2aRmJjIJZdcwuuvv97qIPUf//hHnnvuOdLT0/nHP/7BH/7wBwB+8IMfMGbMGEaPHs0ZZ5zB2LFjeeWVVxg9ejQZGRls2LCh3Tf3CQW73HcLfL4Kli3z0r//dxg27NEOW68xvZFd7rtrsMt9dxC3O4qEhDPthDljTK9lAXEUSUlZlJVtoKpqv9OlGGNMp7OAOAqvdzpgh7sa0xF6yu7s7up4Pn8LiKOIjc0kLCzZAsKYdoqMjCQvL89CwiGqSl5eHpGRkcf0OjsP4ihEXHi90ygo8F/+W0ScLsmYbiktLY2cnBxyc3OdLqXXioyMJC0t7ZheYwHRCq93Orm5r1BevomYmFFOl2NMtxQeHs6QIUOcLsMcI9vF1IrDl/+23UzGmN7FAqIVUVGDiYoaZoe7GmN6HQuINvB6p1NY+AF1dTVOl2KMMZ3GAqINvN4sfL5SiotXOV2KMcZ0GguINkhMPBdw2W4mY0yvEtKAEJEZIrJFRLaJyP1BlntEZG5g+SoRGRyYP1hEKkRkbeDxZCjrbE14uJe4uHE2UG2M6VVCFhAi4gaeAC4ARgHXiEjz40S/ARSo6jDgMeCRRsu2q2pG4HFbqOpsK683i+LiVdTWFjldijHGdIpQ9iAmANtUdYeqVgNzgMuatbkMeCEw/SowTbro2Wj+y274KCz8wOlSjDGmU4QyIPoDexo9zwnMC9pGVWuBIiA5sGyIiHwqIh+KyFnBNiAit4rIahFZHeozNBMSJuNyRdtuJmNMr9FVB6n3AQNVNRO4F/g/EYlv3khVn1LVcao6LjU1NaQFuVweEhPPJj/fBqqNMb1DKANiLzCg0fO0wLygbUQkDEgA8lS1SlXzAFR1DbAdODmEtbaJ1zudiootVFbuab2xMcZ0c6EMiI+B4SIyREQigKuB+c3azAduDExfASxWVRWR1MAgNyIyFBgO7AhhrW1il90wxvQmIQuIwJjCncBCYBPwiqpuFJGHROTSQLNngGQR2YZ/V1L9obBTgc9EZC3+wevbVDU/VLW2VUzMaMLDT7DzIYwxvUJIr+aqqm8Cbzab97NG05XAlUFe9y/gX6Gs7XiICF7vdAoK3kG1DpGuOoRjjDHtZ99wxygpKYuamlzKytY7XYoxxoSUBcQxqr8NqR3NZIzp6SwgjpHH05/o6JE2UG2M6fEsII6D15tFUdESfL5Kp0sxxpiQsYA4Dl7vdOrqKiguXuF0KcYYEzIWEMchMfEcRMJsN5MxpkezgDgOYWFxxMdPsvMhjDE9mgXEcfJ6p1NSsoaaGsfP3zPGmJCwgDhO/stuKAUFi50uxRhjQsIC4jjFxY3H7Y6z3UzGmB7LAuI4uVzhJCaeawPVxpgeywKiHbzeLCord1BR4fiFZo0xpsNZQLRD/WU3bDeTMaYnsoBoh+joEXg8ababyRjTI1lAtIP/8t9ZFBS8h6rP6XKMMaZDWUC0k9c7ndraAkpKPnW6FGOM6VAWEO3k9U4DbBzCGNPzWEC0U0TECcTEpFtAGGN6HAuIDuC//PdyfL5yp0sxxpgOYwHRAZKSslCtpqhoqdOlGGNMh7GA6AAJCWchEmG3ITXG9CgWEB3A7Y4mIWGKnQ9hjOlRLCA6iNebRVnZOqqrDzhdijHGdAgLiA5y+LIbdvlvY0zPYAHRQeLiTiMszGuHuxpjegwLiA4i4iYx8TwKChahqk6XY4wx7WYB0YGSkrKoqsqhouILp0sxxph2C2lAiMgMEdkiIttE5P4gyz0iMjewfJWIDG62fKCIlIrI90NZZ0fx34YUO9zVGNMjhCwgRMQNPAFcAIwCrhGRUc2afQMoUNVhwGPAI82WPwq8FaoaO1pU1FAiI4fYOIQxpkcIZQ9iArBNVXeoajUwB7isWZvLgBcC068C00REAERkJrAT2BjCGjuc15tFYeH71NXVOl2KMca0SygDoj+wp9HznMC8oG1UtRYoApJFJBa4D/jfo21ARG4VkdUisjo3N7fDCm8PrzcLn6+EkpL/Ol2KMca0S1cdpH4QeExVS4/WSFWfUtVxqjouNTW1cyprhdd7LiC2m8kY0+2FMiD2AgMaPU8LzAvaRkTCgAQgD5gI/FZEsoG7gR+JyJ0hrLXDhIcnExd3ul12wxjT7YUyID4GhovIEBGJAK4G5jdrMx+4MTB9BbBY/c5S1cGqOhiYDTysqn8KYa0dyuudTnHxR9TWljhdijHGHLeQBURgTOFOYCGwCXhFVTeKyEMicmmg2TP4xxy2AfcCRxwK2x15vVmo1lJY+KHTpRhjzHELC+XKVfVN4M1m837WaLoSuLKVdTwYkuJCKD7+DFyuKAoKFpGScrHT5RhjzHHpqoPUnUZVeWXjK/jqfB22Trc7koSEs2yg2hjTrfX6gHh3x7tc9epVXPfadVT7qjtsvV5vFuXlm6iqaj4ub4wx3UOvD4isk7J4ZPojzN04l0tfvpSy6rIOWW9Skv+yG3Y0kzGmu+r1AQHwwyk/5G+X/I1FOxaR9Y8sCioK2r3OmJgxhIen2nWZjDHdlgVEwDdO+wb/vPKfrNm3hqnPT2Vfyb52rU/Ehdc7nYKCd+3y38aYbskCopGvjvwqC65dwM6CnUx5dgrb87e3a31ebxY1NQcoK9vQQRUaY0znsYBoZvrQ6Sy+cTFFVUWc+dyZfHbgs+Ne1+HbkNo4hDGm+7GACGJC/wksvXkpbnFz9vNns2LPiuNaT2TkAKKiRtjhrsaYbskCogWjUkex/JblpEanMv3v03l729vHtR6vdzqFhR9SV1fVwRUaY0xoWUAcxaDEQSy9eSkjUkZwycuXMGfDnGNeR1JSFnV15RQXfxSCCo0xJnQsIFpxQuwJfHDjB0xOm8y1/7qWv3z8l2N6fWLiOYDbDnc1xnQ7bQoIEYkREVdg+mQRuVREwkNbWteREJnAwusXctHJF/GdN7/DL5f8ss2HroaFJRAfP8HGIYwx3U5bexBLgEgR6Q+8A9wAPB+qorqiqPAoXvuf17g+/Xp++v5PuXfhvdRpXZte6/VmUVKympqa9p+AZ4wxnaWtASGqWg58Ffizql4JnBq6srqmcHc4L8x8gVkTZjF71WxueeMWattw72mvNwuoo7Dw/dAXaYwxHaTNASEik4HrgAWBee7QlNS1ucTF7Bmzeeich3hh3Qt87ZWvUVlbedTXxMdPxO2OtfMhjDHdSlsD4m7gAeD1wE1/hgK99uewiPDTs3/Kny74E/O3zGfGizMoripusb3LFU5i4jk2DmGM6VbaFBCq+qGqXqqqjwQGqw+p6qwQ19bl3THhDl766kss37Occ184l9yy3Bbber3TqajYRkVFducVaIwx7dDWo5j+T0TiRSQG2AB8LiI/CG1p3cO1Y67ljavf4PPczznzuTPZXbQ7aDv/OIRddsMY0320dRfTKFUtBmYCbwFD8B/JZIALh1/IohsWcaD0AFOencLmQ5uPaBMdPZKIiH62m8kY0220NSDCA+c9zATmq2oNYNewbuTMgWfy4U0fUuOr4cxnz2T1l6ubLBeRwOW/30PbeHisMcY4qa0B8VcgG4gBlojIIKDlUdleauyJY1l2yzLiPHGc+8K5LN65uMlyrzeL2to8SkvXOlShMca0XVsHqR9X1f6qeqH67QLODXFt3dKwpGEsv2U5gxIGccFLFzBv87yGZYcv/227mYwxXV9bB6kTRORREVkdePwef2/CBNEvrh9Lbl7CaX1P42uvfI3nPn0OAI/nRGJiRttAtTGmW2jrLqZngRLgfwKPYuC5UBXVEyRFJbHohkVMGzKNW+bfwu9X/B7w72YqLFyKz1fhcIXGGHN0bQ2Ik1T156q6I/D4X2BoKAvrCWIjYvn3Nf/mylFX8v1F3+dH7/2IxMRpqFZRVLTM6fKMMeaowtrYrkJEzlTVZQAiMgWwn8Bt4Anz8PLXXsYb6eXXy35NXvktXBUXRkHBuyQlZTldnjHGtKitAXEb8HcRSQg8LwBuDE1JPY/b5ebJi58kOTqZXy/7NTv7pfK/0Qs56aRHnC7NGGNa1NajmNap6lggHUhX1UzgvNZeJyIzRGSLiGwTkfuDLPeIyNzA8lUiMjgwf4KIrA081onI5cf0rrogEeHhaQ/z/7L+H4u+zGXWR+soKM12uixjjGnRMd1RTlWLA2dUA9x7tLYi4gaeAC4ARgHXiMioZs2+ARSo6jDgMaD+J/UGYJyqZgAzgL+KSFt7O13a98/4Pk985Sd8UgBZL36F/Ip8p0syxpig2nPLUWll+QRgW2BQuxqYA1zWrM1lwAuB6VeBaSIiqlquqvU3Woikh521ffukB3loTDSf5W7n7OfP5suSL50uyRhjjtCegGjtS7s/sKfR85zAvKBtAoFQBCQDiMhEEdkIrAduaxQYDUTk1vpzM3JzW76Salcj4uayk8/n95kpZBdmM+XZKWzL3+Z0WcYY08RRA0JESkSkOMijBOgXysJUdZWqngqMBx4QkcggbZ5S1XGqOi41NTWU5XQ4rzeLMbEHefN/nqekqoQznz2TdfvXOV2WMcY0OGpAqGqcqsYHecSpamtjAnuBAY2epwXmBW0TGGNIAPKa1bAJKAVGt/52uo/6y24MjTzA0puXEu4O58znzuQH7/yAXYW7HK7OGGPat4upNR8Dw0VkiIhEAFcD85u1mc/hw2WvABarqgZeEwYQuDDgKfgvFthjREUNw+MZREHBu4xMHcnyW5Zz0fCLeOyjxxj6+FCu/OeVrNizAtUeNfxijOlGQhYQgTGDO4GFwCbglcDtSh8SkUsDzZ4BkkVkG/6jouoPhT0TWCcia4HXge+o6qFQ1eoEESEpKYuCgsXU1dUyMGEgc66Yw867dvKDM37AezveY8qzU5j0zCReXv8yNb4ap0s2xvQy0lN+oY4bN05Xr17desMu5ODBuXz++dVkZq4kIWFSk2Vl1WX8fd3fmb1qNl/kfUH/uP7cOeFObj39VpKikhyq2BjT04jIGlUdF2xZKHcxmVYkJk4DJOjVXWMiYrh9/O1sumMTC65dwKjUUTzw3gOkPZrG7f+5Pehd64wxpiNZQDgoIiKF2NjMo94fwiUuLhx+Ie/c8A7rb1/PdWOu47m1zzHyiZFc+NKFvLP9HRunMMaEhAWEw7ze6RQXr6S2trTVtqP7jObpS59mzz17+MW5v+DT/Z9y/ovnM/ovo3l6zdNU1Nj1E40xHccCwmFebxaqNRQVLWnza1JjUvnJ1J+w6+5d/H3m3/G4Pdz6n1sZ8NgAfrL4J3ZmtjGmQ1hAOCwh4Uxcrsjjug1phDuCG8bewJpb1/DhTR8yddBUHl76MINmD+L6165nzZdrQlCxMaa3sIBwmNsdSULCme26DamIMHXQVF676jW2zdrGnePvZP6W+Yx7ehxnPXcWr216DV+drwOrNsb0BhYQXYDXm0VZ2Qaqqva1e11DvUN5bMZj5Nybw2PnP8be4r187ZWvMeyPw3h05aMUVRZ1QMXGmN7AAqILqL/sRnt6Ec3Fe+K5e9LdbP3uVl77n9cYmDCQ773zPdIeS+Out+5ie/72DtuWMaZnsoDoAmJjMwgPT+nQgKjndrm5fOTlfHjTh6z+1mouP+Vy/rL6Lwz/43BmzpnJh9kf2mGyxhyDOq0juzCbt7a+xbs73qXaV+10SSFjZ1J3ERs3Xk1R0RImT96LSGu32mifL0u+5C8f/4Un1zzJofJDZJyYwd0T7+bq0VfjCfOEdNvGdBfVvmq25W9jU+4mNh0KPHI3sSVvC+U15Q3t4j3xXDT8ImaeMpMLhl1AnCfOwaqP3dHOpLaA6CL273+RzZtvoG/fbzF8+J9xuUJ/A72KmgpeWv8Ssz+azcbcjZwQcwLfGf8dviibWKAAABc0SURBVD726/SP60+4OzzkNRjjtNLqUjYf2nxEEGzL34ZPDx/cMShhEKeknMLIlJGMTB3JyJSRFFYWMm/zPOZ/MZ9D5YeIcEcwfeh0Zo6YySUjLuHE2BMdfGdtYwHRDagqO3f+hN27HyYp6SJOPXUubndMp2373R3vMnvVbN7c+mbD/MTIRPrE9KFPTB9So1Ob/O0T04fUmMPTyVHJuF3uTqnXmOORW5bb8OXfOAj2FB++r1mYK4zhScMbAqA+DEYkjyAmouV/j746Hyv2rGDe5nm8vvl1dhbuRBAmD5jMzBEzmXnKTIYnD++Mt3nMLCC6kb17n2Tr1juIizuNMWMWEBHRp1O3v+XQFj7I/oCDZQfJLc/lYNnBJtOHyg9Rp3VHvE4QkqOTmwZIfajEHDnPG+XFJaEbAquqraK0upSS6hL/36qSoz4/2rJ4Tzxp8WkMiB9AWnzaEY/kqOSQ7xbsDHVax4HSA+wp3kNOcQ57igJ/iw//zS3LJSYihgRPAgmRCSRGJh6e9iSSEJkQfFmj6Qh3REjfw+6i3YdDIPB386HN5FUcvtVMTHiMvzfQLAhO8p7U7p6zqrLh4AbmbZ7HvC3z+GTfJwCcmnoqM0/xh8XpfU/vMv/PWEB0M4cO/ZvPP7+KiIi+pKe/TXR01/nl4avzUVBZ4A+NsiMDpPl0fkV+0PW4xU1KdEqTAGkeLuHu8Ja/3Gta+JIPzKupa9vl0V3iIjYilriIOP9fT1yT5zHhMRRXFzd8WX5Z8mWT3Q4AHrenISwGJAwgLe7IEEmNSQ1pILamTus4WHaw4Yu/8Zd+/by9JXuprWt6Z9/69zYgwR+OfaL7UF5TTlFVEUVVRRRWFlJU6Z8uqiyipLqk1VqiwqIagiQxMrHpdCvhkhiZSFxEHD71tWl8ICU6pUkA1P9Ni0/rtP8euwp38caWN5i3eR5Ldi3Bpz7S4tO4bMRlzDxlJmcPOtvR3bkWEN1QcfEq1q+/GFVlzJj/HHE58O6ixldDXkVei4HSJFjKcimqOvp5GpFhkS1+mcd54ogNb2F+RGzQeVFhUcf0S85X5+NA2QFyinOOeNR/2e4t3ntEQIW7wukf3/9wkATpjZwQc8Jx7aar0zpyy3KP+MJvHALBaopwRzSpZUD8gIYgqJ+XEp1yzJ9PcVXxEeFx1OlAuNRPV9ZWtrodl7ia9GQHJgwMGgQp0Slt/yA7QV55Hgu2LuD1za+zcNtCKmorSIxM5OKTL2bmiJmcP+x8YiNiO7UmC4huqrx8G599NoPq6i8ZNWoOKSmXtv6ibq6qtqohNHx1viZf7rERsYR1wuB9ezX+wg4WIPWPKl9Vk9e5xU2/uH5NQqP+i/rE2BPJr8g/YpdP/S//5odahrvCm/zyHxA/4HAQBOalRqd2md0cjVXVVjWERvPwqJ9WlJOTT2ZkykhGpIzo9C/VjlBeU86i7YuYt2Ue/97yb/Iq8vC4PWSdlMXlp1zOJSdfQmpMasjrsIDoxqqrD7J+/cWUlKxh+PA/0b//7U6XZDqAqpJXkXfUnkhOcU6T3SX16nsjR/vl7/QuLXNsautqWb57Oa9vfp15m+exq2gXLnExZcCUhnGLod6hIdm2BUQ35/OV8fnnV5OX9x8GDryfIUMe7pK//EzHUlUKKwvJKc5hX+k+kqKS/OMAMX3sy78HU1XWHVjnH+TePI91B9YBMKbPGC4/5XJmnjKTjBMzOuw7wAKiB6irq2Xr1jvZt++vnHDC9YwY8QwuV+iOBjHGdA07C3Y2HBG1bPcy6rSOgQkDGw6fPWvQWe3a9WoB0UOoKrt3/5qdO39MYuI0Ro/+F2FhCU6XZYzpJLllufzni/8wb8s83tn+DpW1lSRFJXHH+Dt46NyHjmudRwuIrj/iZxqICIMG/QiPJ40tW77Bp59OJT39TTye/k6XZozpBKkxqdyceTM3Z95MWXUZC7cvZN7meUSFRYVke9aD6Kby899h48avERbmJT39LWJiTnW6JGNMN3S0HoSNdHVTSUlfISNjKaq1fPLJFAoLP3S6JGNMD2MB0Y3FxWVw2mkr8Xj6sW7dVzh4cK7TJRljehALiG4uMnIQmZnLiI+fyOefX82ePY/a/R2MMR3CAqIHCA9PIj39HVJTr2T79u+xbds9qNo9qI0x7RPSgBCRGSKyRUS2icj9QZZ7RGRuYPkqERkcmJ8lImtEZH3g73mhrLMncLsjGTVqDmlpd7N37x/YuPEqfL4Kp8syxnRjIQsIEXEDTwAXAKOAa0RkVLNm3wAKVHUY8BjwSGD+IeASVR0D3Aj8I1R19iQiLoYNe4yTTnqUQ4deY926LGpqgl9N1RhjWhPKHsQEYJuq7lDVamAOcFmzNpcBLwSmXwWmiYio6qeq+mVg/kYgSkTsXphtNGDAPYwaNZeSko/59NMpVFRkO12SMaYbCmVA9Af2NHqeE5gXtI2q1gJFQHKzNl8DPlHVKkyb9elzJWPHLqK6ej+ffjqZkpJPnS7JGNPNdOlBahE5Ff9up2+3sPxWEVktIqtzc3M7t7huIDFxKpmZyxGJYO3aqeTnL3S6JGNMNxLKgNgLDGj0PC0wL2gbEQkDEoC8wPM04HXg66q6PdgGVPUpVR2nquNSU0N/3fTuKCZmFKedtpLIyJNYv/5i9u173umSjDHdRCgD4mNguIgMEZEI4GpgfrM28/EPQgNcASxWVRWRRGABcL+qLg9hjb2Cx9OPzMwlJCaew5YtN5Od/Qs7V8IY06qQBURgTOFOYCGwCXhFVTeKyEMiUn9rtGeAZBHZBtwL1B8KeycwDPiZiKwNPPqEqtbeICwsnjFjFnDCCV8nO/tnfPHFt6lrdv9hY4xpzC7W18uoKjt3/oTdux8mKekiTj11Lm53jNNlGWMcYhfrMw1EhKFDf8Xw4X8hP/8t1q49h+rqg06XZYzpgiwgeqn+/W9j9Oh5lJVt5JNPJlNevtXpkowxXYwFRC+WknIJGRnv4/MV8+mnZ1BU9JHTJRljuhALiF4uPn4imZkrcbsTWLfuPA4dan6gmTGmt7KAMERHD+O001YQEzOaDRsuZ8+e31NXZyeuG9PbWUAYACIi+pCR8T7JyRexffv3WblyIDt3/pyqqv1Ol2aMcYgFhGngdscwevQbpKcvIj5+Art2/YKPPhrIpk03UFxshxAb09uEOV2A6VpEhKSk6SQlTae8fBt79/6R/fuf48CBF4mPP4O0tLtISfkqLpf9r2NMT2c9CNOi6OhhDB/+ByZPzmHYsNlUVx/g88+vYtWqIeza9RtqavKcLtEYE0IWEKZVYWHxpKXdxcSJWxg9ej5RUSPYufMBVq4cwJYtt1JWttHpEo0xIWD7CUybibhJSbmElJRLKC3dwN69j3PgwD/Yt+9pEhOnkZZ2N8nJFyJivzuM6QnsX7I5LrGxoxkx4ikmTdrDkCEPU16+mQ0bLuG//x1BTs7j1NYWO12iMaadLCBMu0REpDBo0ANMmrSTUaPmEB6eyrZtd7FyZRpbt95NRUXQW3kYY7oBCwjTIVyucPr0uYrTTlvBaaf9l+TkS/nyyz+zatVw1q+/lIKC9+weFMZ0MxYQpsPFx49n1KgXmTRpF4MG/YTi4o9Yt246q1en8+WXT+PzVThdojGmDSwgTMh4PH0ZMuQhJk3azYgRzyESxhdf3MrKlWns2PEAlZU5TpdojDkKCwgTcm53JH373sTpp39CRsaHJCaeze7dv+WjjwazcePVFBWttN1PxnRBdpir6TQiQmLiVBITp1JRkc3evX9i376/kZs7l7i48aSl3UVq6pW4XBFOl2qMwXoQxiFRUYMZNux3TJ6cw/DhT1BbW8ymTdfz0UeDyc7+hd3lzpguwO5JbboE1Try899h794/kJ//NiIeUlIuJSnpQpKSzsfj6et0icb0SEe7J7XtYjJdgoiL5OQZJCfPoKxsM3v3/olDh14jN/efAMTEjCU5+QKSkmYQH38GLle4wxUb0/NZD8J0WapKWdln5Oe/TV7eWxQXL0e1Frc7Dq93OklJM0hKmkFk5ECnSzWm27IehOmWRITY2LHExo5l4MD7qK0tpqBgMfn5b5Gf/zaHDr0OQHT0qIawSEg4C7c70uHKjekZrAdhuiVVpbx8c0NYFBZ+iGo1Llc0iYnnNgRGdPQwp0s1pkuzHoTpcUSEmJiRxMSMZMCAe/H5yigs/LAhMLZtWwBAZORJDWMXiYnn4HbHOFy5Md2H9SBMj1Revo38/LcDvYvF1NVVIOIhMXFqo97FSETE6VKNcdTRehAWEKbH8/kqKSpa1tC7KC//HACPZ2BDWHi90wgLi3e4UmM6n2MBISIzgD8AbuBvqvqbZss9wN+B04E84CpVzRaRZOBVYDzwvKre2dq2LCBMW1VW7m7oXRQUvIvPV4JIGPHxUxoCIzZ2rPUuTK/gSECIiBv4AsgCcoCPgWtU9fNGbb4DpKvqbSJyNXC5ql4lIjFAJjAaGG0BYUKlrq6G4uIVDYFRWroWgIiIExvCIi5uIpGRA+1OeaZHcmqQegKwTVV3BIqYA1wGfN6ozWXAg4HpV4E/iYioahmwTETsEBQTUi5XOImJZ5OYeDZDh/6aqqp95OcvDBxG+wb79z8faBdFVNTJREef0ugxgujok23g2/RYoQyI/sCeRs9zgIkttVHVWhEpApKBQ23ZgIjcCtwKMHCgnSxl2s/j6UvfvjfRt+9NqPooKVlNaelnlJdvprx8MyUlH5Ob+wqgjV4zsFlw+B8RESfabirTrXXrw1xV9SngKfDvYnK4HNPDiLiJj59IfHzT3zU+XyUVFVsDobGlITz27XuGurqyhnZud1zQ4IiKOgmXy9PZb8eYYxbKgNgLDGj0PC0wL1ibHBEJAxLwD1Yb02W53ZHExo4hNnZMk/mqSlXVXioqDodGeflmCgvf58CBfzReA1FRQ4KGR3h4cue+GWOOIpQB8TEwXESG4A+Cq4Frm7WZD9wIrASuABZrTznu1vQ6IkJkZBqRkWl4vdOaLKutLaWi4osmweE/E3wRqlUN7cLDU4iKGnFEcERGDsbl6tYdftMNhez/uMCYwp3AQvyHuT6rqhtF5CFgtarOB54B/iEi24B8/CECgIhkA/FAhIjMBL7S+AgoY7qTsLBY4uJOIy7utCbzVX1UVu4+Ijjy8v7N/v3PNLQTiSAq6qTAQPnJTf5GRJxgYx0mJOxEOWO6qJqagiZjHP4eyBdUVGxr0utwu+OPCA3/3+F28p9plV2LyZhuKDzcS0LCJBISJjWZ7+917GkUGP6/xcUrOHjwZRofYRURcWLQXkdU1FAbKDetsoAwppsRcRMVNZioqMEkJX2lyTKfr5LKyu1NgqOi4gsOHZpPTU3j27i6iIwc3Cw4/Od1eDxpdlKgASwgjOlR3O5IYmJOJSbm1COW1dQUUlGxtVnPYwuFhUubHJ7rckUSFTU8aM8jPDzZxjt6EQsIY3qJ8PBEwsPHEx8/vsl8VaW6et8RvY6ysg3k5b2Bam1DW7c7Drc7BpEIXC4PLpcHEQ8uV0Sjaf/z4NOeZq89PH3ksubrPTztdkfjckV09kfY61hAGNPLiQgeTz88nn54vec0WVZXV0tlZXZDcFRW7qSuroK6uipUq6mrq2oy7fMVtLjMP10VvIjjEBFxIh7PICIjDz8aP7cB+vazgDDGtMjlCiM6ehjR0cNITr6w3etTVVRrWw2Rurrqo077fEVUVu6msnIXpaWfcOjQPFSrm2wrLCzxqAESHp5qu8taYQFhjOk0IoJIOC5XeIeuV7WO6uoDVFbuoqpqF5WVjR87KCx8H5+vpMlrXK4oPJ6BREYOPCI8IiMHERHRv9efnNi7370xpkcQceHx9MXj6QtMOmK5qlJbW9goQHY3CZNDh9Y1O8oLwI3H0z9o78P/fCBud1SnvD+nWEAYY3o8ESE83Et4uJe4uIygbXy+CqqqdjfpfdQHSGHhEqqqcoC6Jq8JC/MSFpZ4HA8vbndsl9/FZQFhjDGA2x0VuMfHiKDL6+pqqa7e2yRAqqv34/MVUVtbSG2t/zDi+mmfr7SVLbqOM1z8j84IGAsIY4xpA5crrGH3UlvU1dXg8xU3BEawR01NQZPn5eVbGqYbn5sSnJuwsATCwhJJSZnJsGG/b/+bbMYCwhhjQsDlCsflSj7uS7jX1dVQW1t01ICpf3g8A1pf4XGwgDDGmC7I5QonIiKFiIgU52pwbMvGGGO6NAsIY4wxQVlAGGOMCcoCwhhjTFAWEMYYY4KygDDGGBOUBYQxxpigLCCMMcYEJaraeqtuQERygV1O19FOKcAhp4voQuzzaMo+j8Pss2iqPZ/HIFVNDbagxwRETyAiq1V1nNN1dBX2eTRln8dh9lk0FarPw3YxGWOMCcoCwhhjTFAWEF3LU04X0MXY59GUfR6H2WfRVEg+DxuDMMYYE5T1IIwxxgRlAWGMMSYoC4guQEQGiMj7IvK5iGwUkbucrslpIuIWkU9F5D9O1+I0EUkUkVdFZLOIbBKRyU7X5CQRuSfw72SDiLwsIpFO19SZRORZETkoIhsazUsSkUUisjXw19sR27KA6Bpqge+p6ihgEnCHiIxyuCan3QVscrqILuIPwNuqegowll78uYhIf2AWME5VRwNu4Gpnq+p0zwMzms27H3hPVYcD7wWet5sFRBegqvtU9ZPAdAn+L4D+zlblHBFJAy4C/uZ0LU4TkQRgKvAMgKpWq2qhs1U5LgyIEpEwIBr40uF6OpWqLgHym82+DHghMP0CMLMjtmUB0cWIyGAgE1jlbCWOmg38EKhzupAuYAiQCzwX2OX2NxGJcboop6jqXuB3wG5gH1Ckqu84W1WXcIKq7gtM7wdO6IiVWkB0ISISC/wLuFtVi52uxwkicjFwUFXXOF1LFxEGnAb8RVUzgTI6aPdBdxTYt34Z/uDsB8SIyPXOVtW1qP/chQ45f8ECoosQkXD84fCSqr7mdD0OmgJcKiLZwBzgPBF50dmSHJUD5KhqfY/yVfyB0VtNB3aqaq6q1gCvAWc4XFNXcEBE+gIE/h7siJVaQHQBIiL49zFvUtVHna7HSar6gKqmqepg/IOPi1W11/5CVNX9wB4RGRGYNQ343MGSnLYbmCQi0YF/N9PoxYP2jcwHbgxM3wi80RErtYDoGqYAN+D/tbw28LjQ6aJMl/Fd4CUR+QzIAB52uB7HBHpSrwKfAOvxf4f1qstuiMjLwEpghIjkiMg3gN8AWSKyFX8v6zcdsi271IYxxphgrAdhjDEmKAsIY4wxQVlAGGOMCcoCwhhjTFAWEMYYY4KygDCmFSLia3T48VoR6bAzmUVkcOOrchrTlYQ5XYAx3UCFqmY4XYQxnc16EMYcJxHJFpHfish6EfmviAwLzB8sIotF5DMReU9EBgbmnyAir4vIusCj/hIRbhF5OnCPg3dEJCrQflbgHiGficgch96m6cUsIIxpXVSzXUxXNVpWpKpjgD/hvwotwB+BF1Q1HXgJeDww/3HgQ1Udi/96ShsD84cDT6jqqUAh8LXA/PuBzMB6bgvVmzOmJXYmtTGtEJFSVY0NMj8bOE9VdwQutrhfVZNF5BDQV1VrAvP3qWqKiOQCaapa1Wgdg4FFgRu9ICL3AeGq+ksReRsoBeYB81S1NMRv1ZgmrAdhTPtoC9PHoqrRtI/DY4MXAU/g7218HLhBjjGdxgLCmPa5qtHflYHpFRy+DeZ1wNLA9HvA7dBwz+2EllYqIi5ggKq+D9wHJABH9GKMCSX7RWJM66JEZG2j52+rav2hrt7AVVargGsC876L/w5wP8B/N7ibA/PvAp4KXH3Thz8s9hGcG3gxECICPG63GjWdzcYgjDlOgTGIcap6yOlajAkF28VkjDEmKOtBGGOMCcp6EMYYY4KygDDGGBOUBYQxxpigLCCMMcYEZQFhjDEmqP8PfA5BeayofykAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zFhR0MfbNOTr"
      },
      "source": [
        "# **Triplet Loss**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yiECQd7ENN7W",
        "outputId": "2a880fc1-2a69-44e1-accf-188cf2e5d72f"
      },
      "source": [
        "model.compile(loss = tfa.losses.TripletSemiHardLoss(), optimizer=\"adam\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_62\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_95 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_96 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_61 (Functional)           (None, 256)          91584       input_95[0][0]                   \n",
            "                                                                 input_96[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "lambda_89 (Lambda)              (None, 1)            0           model_61[0][0]                   \n",
            "                                                                 model_61[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sLgWoExqNNwG",
        "outputId": "2a7bda94-ab6f-466e-ba53-594fe37385a5"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 48s 50ms/step - loss: 0.9227 - accuracy: 0.7951 - val_loss: 0.7593 - val_accuracy: 0.9112\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.7313 - accuracy: 0.9213 - val_loss: 0.6899 - val_accuracy: 0.9357\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.6471 - accuracy: 0.9440 - val_loss: 0.6710 - val_accuracy: 0.9409\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.5861 - accuracy: 0.9525 - val_loss: 0.6153 - val_accuracy: 0.9477\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.5277 - accuracy: 0.9607 - val_loss: 0.6208 - val_accuracy: 0.9454\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.5000 - accuracy: 0.9649 - val_loss: 0.6372 - val_accuracy: 0.9476\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.4554 - accuracy: 0.9689 - val_loss: 0.6254 - val_accuracy: 0.9499\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.4296 - accuracy: 0.9718 - val_loss: 0.5963 - val_accuracy: 0.9554\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.3854 - accuracy: 0.9758 - val_loss: 0.5352 - val_accuracy: 0.9573\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 46s 49ms/step - loss: 0.3593 - accuracy: 0.9781 - val_loss: 0.5460 - val_accuracy: 0.9603\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KKgn4bjiONoa"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1iO9omw9ONgs",
        "outputId": "88bc9af7-4a82-4eb6-fa0f-2789568f4741"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 96.03\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "WwVNISn5PYpS",
        "outputId": "c177aea2-bf38-4989-83e4-e6a92f541f84"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVfrH8c+TnpCEhCQkkEAKnXSIFFmVoqtix64o6LqKDUXdtf12Ze1uU6wIrsIqa28o9oooXUJvISQQUkhCKqTn/P6YIQSkJGEmN2Ge9+uVV2bu3PJklPudc8+dc8QYg1JKKdflZnUBSimlrKVBoJRSLk6DQCmlXJwGgVJKuTgNAqWUcnEaBEop5eI0CFSHJCKfi8gkR69rJRHJEpHTra4DOlYtynoeVhegThwiUtnsqR9QAzTYn99kjJnX0n0ZY852xrodkYh8Dpxif+oNGKDW/vwNY8yUVu5vOtDXGDPRYUWqE5oGgXIYY4z//scikgXcYIz55tD1RMTDGFPfnrV1ZM2DTETmADnGmP+zriLlavTSkHI6ERktIjkicq+I5AOviUiwiHwqIoUiUmJ/HNVsmx9E5Ab748kiskhE/mlfd7uInN3GdWNFZKGIVIjINyLygoi8cYS6W1LjIyLys31/X4lIaLPXrxGRbBEpFpEH2/jenSsi6SJSKiK/iEhSs9fuFZFd9mNvFpFxInIW8ABwuYhUisjqFhzDW0SeEZFc+88zIuJtfy3U/neXisgeEflJRNyOdPy2/I3KehoEqr1EAN2AaOBGbP/vvWZ/3huoAp4/yvbDgc1AKPB34D8iIm1Y93/AMiAEmA5cc5RjtqTGq4DrgO6AF3APgIgMBl6y77+n/XhRtIKIpAKvAjfZt38ZmG8/cQ8AbgNOMsYEAGcCWcaYL4DHgbeNMf7GmOQWHOpBYASQAiQDw4D9LZK7gRwgDAjHFjLmSMdvzd+nOg4NAtVeGoGHjDE1xpgqY0yxMeZ9Y8w+Y0wF8Bhw2lG2zzbGzDbGNABzgR7YTkwtXldEegMnAX81xtQaYxYB8490wBbW+JoxZosxpgp4B9vJFOAS4FNjzEJjTA3wF/t70Bo3Ai8bY5YaYxqMMXOx9buMwNb34g0MFhFPY0yWMWZbK/e/39XAw8aY3caYQuBvHAjIOmzvX7Qxps4Y85OxDVDmyOMri2kQqPZSaIyp3v9ERPxE5GX7pZNyYCEQJCLuR9g+f/8DY8w++0P/Vq7bE9jTbBnAziMV3MIa85s93tespp7N922M2QsUH+lYRxAN3G2/LFMqIqVAL6CnMSYDuBNbq2a3iLwlIj1buf/9egLZzZ5n25cB/APIAL4SkUwRuc/+9zjy+MpiGgSqvRw6zO3dwABguDEmEDjVvvxIl3scIQ/oJiJ+zZb1Osr6x1NjXvN9248Z0rpy2Qk8ZowJavbjZ4x5E8AY8z9jzO+wBYYBnrJv19ohhXPt+9ivt30ZxpgKY8zdxpg44Hzgrv19AUc5vupkNAiUVQKwXXMvFZFuwEPOPqAxJhtYAUwXES8RGQmc56Qa3wPOFZHfiYgX8DCt//c2G5giIsPFpouInCMiASIyQETG2jt1q+117r/0VADE7O/UbYE3gf8TkTB7Z/dfgTegqbO6r72PpQzbJaHGYxxfdTIaBMoqzwC+QBGwBPiinY57NTAS22WaR4G3sV13P5w212iMWQ/ciq1zOg8owdbp2mLGmBXAH7F1UJdgu0Qz2f6yN/CkvbZ8bJ3V99tfe9f+u1hEfm3BoR7FFpBrgLXAr/ZlAP2Ab4BKYDHwojHm+2McX3UyohPTKFcmIm8Dm4wxTm+RKNVRaYtAuRQROUlE+oiIm/2e+wuAj6yuSykr6TeLlauJAD7A1nGbA9xsjFllbUlKWUsvDSmllIvTS0NKKeXiOt2lodDQUBMTE2N1GUop1amsXLmyyBgTdrjXOl0QxMTEsGLFCqvLUEqpTkVEso/0ml4aUkopF6dBoJRSLk6DQCmlXFyn6yNQSnVcdXV15OTkUF1dfeyVlVP4+PgQFRWFp6dni7fRIFBKOUxOTg4BAQHExMRw5HmDlLMYYyguLiYnJ4fY2NgWb6eXhpRSDlNdXU1ISIiGgEVEhJCQkFa3yDQIlFIOpSFgrba8/y4TBJWVa8jMvB8dUkMppQ7mMkFQWvojO3Y8SXHxEaeoVUp1csXFxaSkpJCSkkJERASRkZFNz2tra4+67YoVK5g6deoxj3HyySc7pNYffviBc8891yH7Ol4u01ncs+cUcnNnkpFxF926nYWbm7fVJSmlHCwkJIT09HQApk+fjr+/P/fcc0/T6/X19Xh4HP60l5aWRlpa2jGP8csvvzim2A7EZVoEbm6e9O07g+rqTHJynrG6HKVUO5k8eTJTpkxh+PDh/PnPf2bZsmWMHDmS1NRUTj75ZDZv3gwc/Al9+vTpXH/99YwePZq4uDieffbZpv35+/s3rT969GguueQSBg4cyNVXX9106fmzzz5j4MCBDB06lKlTpx7zk/+ePXu48MILSUpKYsSIEaxZswaAH3/8salFk5qaSkVFBXl5eZx66qmkpKSQkJDATz/9dNzvkcu0CAC6dTudkJALyM5+lPDwa/H27mF1SUqdsLZuvZPKynSH7tPfP4V+/Vr/QS4nJ4dffvkFd3d3ysvL+emnn/Dw8OCbb77hgQce4P333//NNps2beL777+noqKCAQMGcPPNN//m3vxVq1axfv16evbsyahRo/j5559JS0vjpptuYuHChcTGxnLllVces76HHnqI1NRUPvroI7777juuvfZa0tPT+ec//8kLL7zAqFGjqKysxMfHh1mzZnHmmWfy4IMP0tDQwL59+1r9fhzKZVoE+/Xp808aG2vJzNTpVZVyFZdeeinu7u4AlJWVcemll5KQkMC0adNYv379Ybc555xz8Pb2JjQ0lO7du1NQUPCbdYYNG0ZUVBRubm6kpKSQlZXFpk2biIuLa7qPvyVBsGjRIq655hoAxo4dS3FxMeXl5YwaNYq77rqLZ599ltLSUjw8PDjppJN47bXXmD59OmvXriUgIKCtb0sTl2oRAPj59SUqaho7dz5FZOQtBAYOs7okpU5Ibfnk7ixdunRpevyXv/yFMWPG8OGHH5KVlcXo0aMPu42394F+RHd3d+rr69u0zvG47777OOecc/jss88YNWoUX375JaeeeioLFy5kwYIFTJ48mbvuuotrr732uI7jci0CgOjoB/HyimDr1qkY02h1OUqpdlRWVkZkZCQAc+bMcfj+BwwYQGZmJllZWQC8/fbbx9zmlFNOYd68eYCt7yE0NJTAwEC2bdtGYmIi9957LyeddBKbNm0iOzub8PBw/vjHP3LDDTfw66+/HnfNLhkEHh4BxMU9SUXFUgoK/md1OUqpdvTnP/+Z+++/n9TUVId/ggfw9fXlxRdf5KyzzmLo0KEEBATQtWvXo24zffp0Vq5cSVJSEvfddx9z584F4JlnniEhIYGkpCQ8PT05++yz+eGHH0hOTiY1NZW3336bO+6447hr7nRzFqelpRlHTExjTCO//jqCmppdDBu2GQ8PfwdUp5Rr27hxI4MGDbK6DMtVVlbi7++PMYZbb72Vfv36MW3atHY7/uH+O4jISmPMYe+PdckWAYCIG337PkttbS47djxhdTlKqRPI7NmzSUlJIT4+nrKyMm666SarSzoql+ssbq5r1xGEh1/Dzp3/okePP+DrG2d1SUqpE8C0adPatQVwvFy2RbBfXNwTiHiwbdufrC5FKaUs4fJB4O0dSXT0AxQVfUBJyXdWl6OUUu3O5YMAICrqLnx8YsjIuIPGRsffRaCUUh2ZBgHg7u5Dnz7/Yu/edeTlzbK6HKWUalcaBHahoRcRFDSW7dv/Ql3dHqvLUUq1wfEMQw22L3MdaXTROXPmcNtttzm65A5Bg8BOROjb9xnq60vJynrI6nKUUm2wfxjq9PR0pkyZwrRp05qee3l5HXP7owXBiUyDoBl//0R69pzCrl0vUVm5zupylFIOsHLlSk477TSGDh3KmWeeSV5eHgDPPvssgwcPJikpiSuuuIKsrCxmzpzJ008/TUpKylGHd87KymLs2LEkJSUxbtw4duzYAcC7775LQkICycnJnHrqqQCsX7+eYcOGkZKSQlJSElu3bnX+H91KLv09gsOJjX2Y3bvfJCPjTpKTv9b5V5Vqozu/uJP0fMcOQ50SkcIzZ7V8MDtjDLfffjsff/wxYWFhvP322zz44IO8+uqrPPnkk2zfvh1vb29KS0sJCgpiypQpv5nM5nBuv/12Jk2axKRJk3j11VeZOnUqH330EQ8//DBffvklkZGRlJaWAjBz5kzuuOMOrr76ampra2loaDiu98AZtEVwCE/PEGJiHqa09Fud1lKpTq6mpoZ169ZxxhlnkJKSwqOPPkpOTg4ASUlJXH311bzxxhtHnLXsSBYvXsxVV10FwDXXXMOiRYsAGDVqFJMnT2b27NlNJ/yRI0fy+OOP89RTT5GdnY2vr68D/0LH0BbBYTSf1jI4+Ezc3X2sLkmpTqc1n9ydxRhDfHw8ixcv/s1rCxYsYOHChXzyySc89thjrF279riPN3PmTJYuXcqCBQsYOnQoK1eu5KqrrmL48OEsWLCA8ePH8/LLLzN27NjjPpYjaYvgMNzcPOjb9xmd1lKpTs7b25vCwsKmIKirq2P9+vU0Njayc+dOxowZw1NPPUVZWRmVlZUEBARQUVFxzP2efPLJvPXWWwDMmzePU045BYBt27YxfPhwHn74YcLCwti5cyeZmZnExcUxdepULrjggqZpKDsSpwaBiJwlIptFJENE7jvM671F5HsRWSUia0RkvDPraY3m01rW1ORaXY5Sqg3c3Nx47733uPfee0lOTiYlJYVffvmFhoYGJk6cSGJiIqmpqUydOpWgoCDOO+88Pvzww2N2Fj/33HO89tprJCUl8frrrzNjxgwA/vSnP5GYmEhCQgInn3wyycnJvPPOOyQkJJCSksK6deuOexIZZ3DaMNQi4g5sAc4AcoDlwJXGmA3N1pkFrDLGvCQig4HPjDExR9uvo4ahbomqqm0sWzaY7t2vZNCgOe1yTKU6Mx2GumPoSMNQDwMyjDGZxpha4C3ggkPWMUCg/XFXoEN99Pb17UOvXndRUDCX8vKlVpejlFJO4cwgiAR2NnueY1/W3HRgoojkAJ8Btx9uRyJyo4isEJEVhYWFzqj1iHr3fsA+reUdOq2lUuqEZHVn8ZXAHGNMFDAeeF1EflOTMWaWMSbNGJMWFhbWrgUePK3lvHY9tlKdUWeb9fBE05b335lBsAvo1ex5lH1Zc38A3gEwxiwGfIBQJ9bUJuHh1xAQMIzMzHupr6+0uhylOiwfHx+Ki4s1DCxijKG4uBgfn9bd8u7M7xEsB/qJSCy2ALgCuOqQdXYA44A5IjIIWxC077WfFrBNazmDVatGsmPH48TFPW51SUp1SFFRUeTk5NDel3DVAT4+PkRFRbVqG6cFgTGmXkRuA74E3IFXjTHrReRhYIUxZj5wNzBbRKZh6ziebDroR4mDp7W8Qae1VOowPD09iY2NtboM1UpOu33UWdrz9tFD1dTksnRpf7p1+z0JCR9YUoNSSrWFVbePnnC8vXsSHf0gRUUfUlLyrdXlKKWUQ2gQtFJU1DR8fGLJyLhTp7VUSp0QNAha6eBpLV+2uhyllDpuGgRtEBp6YbNpLYutLkcppY6LBkEbHJjWsoysrOlWl6OUUsdFg6CNbNNa3qzTWiqlOj0NguMQG/s3PDwCyci4Q79JqZTqtDQIjsOBaS2/o6joY6vLUUqpNtEgOE49e07Bzy+ebdvupqGh2upylFKq1VwmCDYUbmDq51Opqa9x6H7d3Dzo12+GfVrLpx26b6WUag8uEwRfb/ua55Y9x+i5o8mtcOz8N8HB4wgNvZDs7Md0WkulVKfjMkFwx4g7ePfSd1lbsJahs4byy85fHLr/Pn3+iTF1ZGbe79D9KqWUs7lMEABcMvgSltywhC6eXRg9ZzQvr3DcN4MPTGv5X53WUinVqbhUEAAkdE9g+R+XMy5uHFMWTOHGT250WL+BbVrLHmzdOlWntVRKdRouFwQAwb7BfHrlp9z/u/uZ/etsh/UbHJjWchkFBW84oFKllHI+lwwCAHc3dx4f97jD+w3Cwyfap7W8j/r6CgdUqpRSzuWyQbCfo/sNRNzo1+9Zamvz2LHjCQdVqZRSzuPyQQCO7zcIDBxOePi17Nz5L6qqtjmwUqWUcjwNAjtH9xvExT2BiCfbtt3jwCqVUsrxNAiacWS/wYFpLT9iz55vHFypUko5jgbBYVwy+BKW3rAUfy//4+o3sE1rGafTWiqlOjQNgiOI7x7PshuWcXrc6W3uN9g/reW+fevJzZ3ppEqVUur4aBAcRbBvMJ9c+QkP/O6BNvcbhIZeQFDQOLKy/qrTWiqlOiQNgmNwd3PnsXGP8d6l7zX1G/y84+cWb998Wsvt2x9yYqVKKdU2GgQtdPHgi5v6DcbMHcPMFTNbPCuZv38CPXveTG7uS1RWrnVypUop1ToaBK3QvN/g5gU3t6rfwDatZVed1lIp1eFoELRS836DV1a9wmlzTmNX+a5jbufpGUJs7COUln5PUdFH7VCpUkq1jAZBGzTvN1i3ex1ps9Na1G/Qo8dN+PnFs3XrLezdu7EdKlVKqWPTIDgOre03cHPzID7+bYwxpKefRmXl6nasVimlDk+D4DjFd49n+R+Xc0afM1rUb9ClSzypqQtxc/MmPX0M5eXL2rFapZT6LacGgYicJSKbRSRDRO47zOtPi0i6/WeLiJQ6sx5nCfIJYv4V83nwlAdb1G/g59eflJSf8PAIYvXq0ykt/akdq1VKqYM5LQhExB14ATgbGAxcKSKDm69jjJlmjEkxxqQAzwEfOKseZ3N3c+fRsY/y/mXvt6jfwNc3htTUn/Dy6smaNWfqeERKKcs4s0UwDMgwxmQaY2qBt4ALjrL+lcCbTqynXUwYNKHF/Qbe3pGkpv6Ir29f1q49l6KiT9u5WqWUcm4QRAI7mz3PsS/7DRGJBmKB747w+o0iskJEVhQWFjq8UEdrTb+Bl1c4KSk/4O+fyPr1F7F797vtXK1SytV1lM7iK4D3jDENh3vRGDPLGJNmjEkLCwtr59LapjX9Bp6e3UhO/oaAgOFs2HAF+fn/bedqlVKuzJlBsAvo1ex5lH3Z4VzBCXBZ6FCt6Tfw8OhKcvKXBAWNYdOmSeTmHt+UmUop1VLODILlQD8RiRURL2wn+/mHriQiA4FgYLETa7HUof0G03+YzuKdi6murz5oPXf3LiQmfkq3buewZcsUdu582qKKlVKuRJw57o2IjAeeAdyBV40xj4nIw8AKY8x8+zrTAR9jzG9uLz2ctLQ0s2LFCmeV7FSl1aVc++G1fLLlEwA83TxJiUhhRNSIpp/YoFiMqWPjxqspLHyP2NhHiY5+0OLKlVKdnYisNMakHfa1zjYAWmcOgv3yKvJYumspS3KWsCRnCctzl7Ovbh8AYX5hjIgawfDIYfRs/JHw+m8YGHc/sbGPISIWV66U6qw0CDq4+sZ61u1ex9KcpSzZZQuHTUWbABAgpgukRcTz+8F3MjJqJIPCBuEmHaWfXynVGWgQdEIlVSUs27WMxTmL+X7LHFYVZlNhn/Y40DuQYZHDGB453N56GE5Yl85xN5VSyhoaBJ2cMYbMzP/jl82Pk8NIdjQmsXTXMtYUrKHBfsdtn+A+B/U1JIUn4eXuZXHlSqmO4mhB4NHexajWExH69HkMDw9/tm9/gPNCe/LC+CVU1dexMm9lU1/Dd9u/Y97aeQD4ePgwpMcQRkQeCIeowCjtZ1BK/Ya2CDqZnJwZZGTcSbdu44mPfw93d9+m14wx5JTnNAXDkl1LWJm7kpoG27eaewb0tIVC5AiGRw1neORwvD28rfpTlFLtSC8NnWByc2ezZctNBAWNJiFhPh4e/kdct7ahltX5qw+6S2lbyTYAYoNimXXeLE6PO729SldKWUSD4ASUn/8GmzZNJjBwOElJn+Hh0bXF2xbuLeTH7B954NsH2LpnK5NTJvOv3/+Lbr7dnFixUspKRwsCvQexk4qImEh8/NtUVCwnPX0cdXXFLd42rEsYlwy+hNVTVnP/7+7n9dWvM+iFQby97u2jzrCmlDoxaRB0YmFhF5OQ8BF7964jPX00NTX5rdre19OXx8c9zsobV9IrsBdXvH8FF7x1ATnlOU6qWCnVEWkQdHIhIeNJSlpAVVUm6emnUV3d+pN4ckQyS25Ywr9+/y++yfyGwS8M5sXlL9JoGp1QsVKqo9EgOAEEB48jOfkramvzSU8/haqqzFbvw8PNg7tG3sW6W9YxPGo4t352K6e+diobCzc6oWKlVEeiQXCC6Np1FMnJ31JfX86qVaeyd++mNu0nLjiOryZ+xZwL5rChcAMpL6fwyI+PUNtQ6+CKlVIdhQbBCSQwMI2UlB8wpo709NOorFzTpv2ICJNSJrHx1o1cNPAi/vrDXxny8hCW5CxxcMVKqY5Ag+AE4++fSGrqQkQ8SU8fTXl522+1DfcP561L3mL+FfMpqynj5P+czJ1f3EllbaUDK1ZKWU2D4ATk5zeA1NSf8PDoyurV4ygrO/ysaC113oDzWH/Lem456RZmLJ1B/IvxfJHxhYOqVUpZTYPgBOXrG0tKyk94eUWwevXvKSn59rj2F+gdyPPjn2fRdYvo4tmFs+edzcQPJlK0r8hBFSulrKJBcALz8YkiNXUhvr59WLPmHIqLFxz3Pkf1HsWqm1bx11P/yjvr32HQC4OYt2aefhFNqU5Mg+AE5+UVTkrK9/j7J7Ju3UUUFr5/3Pv09vDmb2P+xq83/Uqf4D5M/HAi4/83nuzSbAdUrJRqbzrWkIuory9jzZpzKC9fzMCBc4mImOiQ/TY0NvDi8he5/9v7AXh83OPcetKtuLu5O2T/J7Ky6jLS89NtPwXprMpbRXZZNr0CexEXHEef4D706dan6Xd012g83T2tLlt1UjronAKgoWEva9eeT2np9/TvP5OePW902L6zS7O5ecHNfJ7xOcMjh/PK+a+Q0D3BYfvvzPYPD77/pL8qfxXp+elsL93etE54l3BSe6QSGxRLTnkO20q2kVmSSXV9ddM67uJO7669D4TDIUHh73XkUWiV0iBQTRoaqli//hL27PmMmJjp9O79AG5ujvmUaYzhzXVvcscXd1BWXcZ9v7uPB0950KXmPKhvrGdz0eaDTvjp+ekUV9kGBRSEfiH9SIlIISU8hdQeqaREpBDhH/GbfTWaRvIq8thWso1te7bZfjd7vKdqz0Hrd+/S/eBwaPa4e5fuOimRizvuIBCRLkCVMaZRRPoDA4HPjTF1ji312DQIjl9jYy2bN/+BgoI36NIlkf79Z9G16wiH7b9oXxHTvpzGG2veYGDoQF457xVG9R7lsP13FJW1lawpWHPQJ/11u9c1fYr3dvcmMTzxoBN+YvdEArwDHHL80urSAwFh/51Zksm2km3sLNuJ4cC/7S6eXejTrc+BS07NQiI6KBoPN52s8ETniCBYCZwCBAM/A8uBWmPM1Y4stCU0CBynqOhjtmy5ldraXCIjbyU29jE8PAIdtv8vMr5gyqdTyC7L5pa0W3ji9CcI9Hbc/ttTfmX+by7tbC3e2nSy7ebbjZSIFFIjbCf8lIgUBoYOtOwEW1NfQ1Zp1mFbE5klmU2z1oHtklN0UPRBAZHYPZExsWN03usTiCOC4FdjzBARuR3wNcb8XUTSjTEpji72WDQIHKu+vpzt2/+PXbuex8urJ/36PU9Y2IUO239lbSV/+e4vzFg6g8jASF465yXO7X+uw/bvaI2mkYw9GbYTft4q0gtsJ//8ygNDfMcExRx0wk+NSO1U80E3mkZyK3J/05rY/7ikugSAIJ8gLhx4IZcNvoxxceM0FDo5RwTBKuAW4GngD8aY9SKy1hiT6NhSj02DwDnKy5eyefON7N27htDQi+jX7zm8vSMdtv+lOUu54ZMbWLd7HZfHX86Ms2YQ7h/usP0fTUNjA2U1ZZRWl1JSVUJpdWnTT0m17Xnh3kLWFa5jdf5q9tbtBWwjssaHxR90wk+OSCbIJ6hd6rZKSVUJP+/8mXc3vMvHmz6mrKaMIJ8gLhp4EZfFX8a42HF691In5IggOA24G/jZGPOUiMQBdxpjpjq21GPTIHCexsY6cnKeJivrIUQ8iYt7gp49pyDimFtBaxtq+fvPf+eRhY/QxbML/z7z30xKnnTMT9LGGCpqKw6cvJudzPefyI/4vKqEitqKo+7fTdwI9glmcNjgg076g8MGu1RH9+HU1NfwdebXvLP+HT7e/DHlNeV08+3GhQMu5LL4yxgbO1ZDoZNw6F1DIuIG+Btjyh1RXGtpEDhfVdU2tmy5mZKSrwkIGM6AAbPx93dc429T0Sb++MkfWbRjEafHnc5p0acd9WReWl16zElyAr0DCfIJIsgniGCf4KbHR1oW7Hvgub+XP26i3608lpr6Gr7a9hXvbHiHjzd9TEVtBd18uzFh4AQujb+UMTFjNBQ6MEe0CP4HTAEasHUUBwIzjDH/cGShLaFB0D6MMeze/T8yMu6kvr6UXr3+RHT0X3B393XI/htNI7NWzuLeb+6lvKYcP0+/Np3Eg3yC6OrdVb/A1s6q66ttoWBvKVTWVhLiG8KEQRO4dPCljIkdo3cidTCOCIJ0Y0yKiFwNDAHuA1YaY5IcW+qxaRC0r7q6YrZtu4f8/Dn4+PShf/+ZdOt2usP2v3/CG+2I7Lyq6qr4ctuXvLvhXeZvnk9lbSWhfqFMGDiBy+Iv47SY0zQUOgBHBMF6IAX4H/C8MeZHEVltjEl2bKnHpkFgjZKS79iy5SaqqjIID7+GPn3+hZdXmNVlqQ6mqq6KLzK+aAqFvXV7CfMLY8IgWyicGn2qhoJFHBEEU4F7gdXAOUBv4A1jzCnH2O4sYAbgDrxijHnyMOtcBkwHDLDaGHPV0fapQWCdhoYqsrMfY+fOp3B370rfvv8mPPyaTnPbpGpfVXVVfJ7xOe9ueJdPNn/SFAoXD7q4KRT0kl77cXCaszMAABovSURBVMoQEyLiYYypP8rr7sAW4AwgB1vfwpXGmA3N1ukHvAOMNcaUiEh3Y8zuox1Xg8B6lZXr2LLlRsrLFxMUNI7+/Wfi59fX6rJUB7avbh+fb/2cdza8w6dbPmVf3T7Cu4Q3tRRO6X2KhoKTOaJF0BV4CDjVvuhH4GFjTNlRthkJTDfGnGl/fj+AMeaJZuv8HdhijHmlhX+LBkEHYUwjubkvk5l5H8bUEh39V3r1usdh4xapE9e+un18tvUz3ln/Dgu2LmgKhUsGX8Klgy/ld71/p6HgBI4IgveBdcBc+6JrgGRjzISjbHMJcJYx5gb782uA4caY25qt8xG2VsMobJePphtjfjMHoojcCNwI0Lt376HZ2TrufUdRU5PL1q1TKSp6ny5dEujff7ZDxy1SJ7a9tXttobDhHRZsWUBVfRUR/hFcMugSLo2/lFG9RmkoOIjD7ho61rJDXm9JEHwK1AGXAVHAQiDRGFN6pP1qi6BjKiqaz9att1JTs4uePW8hLu5xh45bpE58lbWVB7UUquurSeiewLwJ80gKb/cbFE84RwuCln6LpkpEftdsh6OAqmNsswvo1ex5lH1ZcznAfGNMnTFmO7bWQb8W1qQ6kNDQ8znppA1ERt5Obu6LLFs2iMLCD60uS3Ui/l7+XBZ/Ge9d9h6Ffyrkvxf+l6J9RZw0+yT+vfjfx/xSoWq7lgbBFOAFEckSkSzgeeCmY2yzHOgnIrEi4gVcAcw/ZJ2PgNEAIhIK9AcyW1iT6mA8PALo128GQ4YswdMzjPXrJ7Bu3UVUV+dYXZrqZPy9/Lkm+RrWTFnD2X3P5u6v7uaM188gp1z/X3KGFgWBMWb/dwaSgCRjTCow9hjb1AO3AV8CG4F37IPVPSwi59tX+xIoFpENwPfAn4wxxW38W1QHERg4jKFDlxMX9xR79nzJ8uWDycl5HmMarC5NdTJhXcL48PIPeeW8V1ias5Skl5J4d/27Vpd1wjme20d3GGN6O7ieY9I+gs6lqirTPm7RV/Zxi2bh76/Xe1XrbS3eysQPJ7Js1zKuTb6W585+rtPOb2EFR/QRHHa/x7GtchG+vnEkJX3BoEFvUF2dycqVQ8nMvJ+GhmN1MSl1sH4h/Vh03SIeOu0h3ljzBskzk1m0Y5HVZZ0QjicIOtdkx8oyIkJ4+NUMG7aR8PBr2LHjSZYvT2TPnm+sLk11Mp7unkwfPZ1F1y3CTdw4bc5pPPjtg9Q1tPusuSeUowaBiFSISPlhfiqAnu1UozpBeHqGMHDgqyQnf4uIG2vWnMHGjddQW3vUL5Mr9Rsje40k/aZ0JidP5vFFj3PyqyezuWiz1WV1WkcNAmNMgDEm8DA/AcYYHTlKtUlw8FjS0tYQHf1/7N79FkuX9mfnzqdpbKy1ujTViQR4B/CfC/7D+5e9T2ZJJqkvpzJzxUza2u/pynQ2DmUJd3cfYmMfIS1tLV27jmTbtrtYvjyJ4uLPrS5NdTITBk1g7c1rOSX6FG5ecDPnv3U+u/dqK7M1NAiUpbp0GUhi4mckJn4KNLJ27XjWrDmHffu0ma9armdATz6/+nNmnDWDr7d9TeJLiXy65VOry+o0NAiU5USEkJBzOOmkdfTp80/KyhaxfHkCGRl3U1d3xNFGlDqIm7gxdfhUVt64kgj/CM578zxuWXAL++r2WV1ah6dBoDoMNzcvevW6m+HDtxARMZmcnKdZtqw/ubmz9ctoqsXiu8ez7IZl3DPyHmaumMmQl4ewMnel1WV1aBoEqsPx8gpnwIDZDB26Aj+/AWzZciMrV6ZRWrrQ6tJUJ+Ht4c0/fv8Pvrn2G/bW7WXEf0bwxE9P0NCoHygOR4NAdVgBAUNISVnI4MFvUVdXTHr6aaxffznV1TusLk11EmNjx7JmyhomDJrAA989wOi5o8kqzbK6rA5Hg0B1aCJC9+6XM2zYJqKjH6K4eD7Llg1g+/bpNDTotV91bMG+wbx18Vu8ftHrrClYQ9JLSby++nW9zbQZDQLVKbi7+xEbO51hwzYTEnIB2dl/Y9myARQUvKX/oNUxiQgTkyayespqUiJSuPaja7ni/SsoqSqxurQOQYNAdSo+Pr2Jj3+LlJSFeHqGsXHjlaxadQoVFdoZqI4tJiiG7yd9z+NjH+eDjR+QNDOJ77Z/Z3VZltMgUJ1SUNApDB26nP79Z1NVtYWVK09i06YbqK0tsLo01cG5u7lz/yn3s+QPS+ji2YVx/x3HPV/dQ019jdWlWabNw1BbRYehVoeqry8jK+sRdu2agZubH9HRfyEqaipubl5Wl6Y6uH11+7jnq3t4acVLJIUnMW/CPBK6J1hdVpOa+hp2lu8kuzSbrNIsTu51MoPCBrVpX8c9Z3FHokGgjmTfvs1kZNzNnj0L8PXtR9++T9Ot23hEdMR0dXQLtizg+vnXU1ZdxlOnP8Xtw2/HTZx/waS6vpodZTvIKs1qOtlnlR14nFuRi2k20POMs2YwdfjUNh1Lg0C5lOLiz8nImEZV1Wa6dTuLPn3+TZcubfsUpVzH7r27uWH+DXyy5RPOiDuDORfOoWfA8Q2yXFVXRXaZ/QS//2RfduBxXmXeQeu7izu9uvYiJiiG6K7RxATFHPQ4KjAKT3fPNtWiQaBcTmNjHbt2PU9W1t9obNxLZORtREc/hKdnkNWlqQ7MGMOslbOY9uU0fD19mX3ebCYMmnDE9StrK8kuzT74ZN/s8aGD33m6eTad6GO6xhAddOBkHxMUQ8+Anni4OWdgZw0C5bJqa3ezfftfyMubjadnCLGxj9Kjxw2IuFtdmurANhdtZuKHE1mRu4LrUq7jooEXHfZkX7Sv6KDtvNy9mj69H/SJ3n7C7+HfA3c3a/7f0yBQLq+iIp2MjDsoK1tIly7J9Os3g6Cg06wuS3VgdQ11/O3Hv/HEoidoNI0A+Hj4HPGyTUxQDOH+4e3St9AWGgRKYWv2Fxa+x7Zt91BTs4OwsEuJi/s7vr4xVpemOrDNRZsprS4lJiiG7l26d9qbDzQIlGqmoaGKnTv/yY4dTwCGXr3+RO/e9+Lu3sXq0pRymqMFQcdswyjlRO7uvsTE/IVhwzYTGjqB7OxHWLp0ANnZT1BTs8vq8pRqdxoEymX5+PRi8OB5pKYuws+vH9u3P8Dixb1Zs2Y8u3e/R2Oj637TVLkWnYBeubyuXUeRkvI9+/ZlkJ8/h/z8OWzYcCkeHiGEh19Njx7X4++fbHWZSjmN9hEodQhjGigp+Ya8vFcpKvoIY2rx908lIuJ6wsOvwtOzm9UlKtVq2lmsVBvV1RVTUPAm+fmvUlm5ChEvQkMvpEeP6wkOPl2/j6A6DQ0CpRygoiKd/PzXKCh4g/r6PXh7RxEePokePa7D17eP1eUpdVQaBEo5UGNjDUVF88nPf409e74EGuna9TR69LiOsLBL9DZU1SFpECjlJNXVORQU/Jf8/NeoqsrA3T2A7t0vJyLiOgIDR3baLx+pE49l3yMQkbNEZLOIZIjIfYd5fbKIFIpIuv3nBmfWo5Sj+fhEER39AMOGbSElZSFhYRdTUPAmq1aNYtmyQezY8XdqavKOvSOlLOS0FoHYetG2AGcAOcBy4EpjzIZm60wG0owxt7V0v9oiUB1dfX0FhYXvkpf3KuXlPwPuhIScTUTE9YSEnKMT5ihLHK1F4MzvEQwDMowxmfYi3gIuADYcdSulOjkPjwB69LieHj2uZ9++zfbvJsyluPhTPD3DCA+fSETE9fj7d5yZsJRrc+aloUhgZ7PnOfZlh7pYRNaIyHsi0utwOxKRG0VkhYisKCwsdEatSjmFn98A4uKeYMSIHSQmLqBr11PZtet5VqxIZOXKYeza9RJ1daVWl6lcnNVDTHwCxBhjkoCvgbmHW8kYM8sYk2aMSQsLC2vXApVyBDc3D0JCxpOQ8B4jR+bSt+8zNDZWs3XrLSxe3IMNG65iz55vMPbhjpVqT84Mgl1A80/4UfZlTYwxxcaY/QO6vAIMdWI9SnUIXl6hREXdQVraaoYOXUFExPXs2fM5a9acwdKlfcjNfZnGxlqry1QuxJlBsBzoJyKxIuIFXAHMb76CiPRo9vR8YKMT61GqQxERAgKG0r//C4wcmcegQW/i5RXBli1TWLq0H7m5szQQVLtwWhAYY+qB24AvsZ3g3zHGrBeRh0XkfPtqU0VkvYisBqYCk51Vj1Idmbu7D+HhV5Ca+gtJSV/g5dWDLVtu0kBQ7UK/UKZUB2SMYc+eL8nKmk5FxVK8vaOJjn6QiIhJevupahOdmEapTkZECAk5iyFDFpOY+DleXuFs2XIjy5YNIDf3FRob66wuUZ1ANAiU6sAOBMISEhM/w9MzjC1b/siyZf01EJTDaBAo1QnYAuFshgxZSmLigoMCIS/vPxoI6rhoECjVidgCYbw9ED7F0zOUzZtvYNmyAeTlvaqBoNpEg0CpTsgWCOcwZMgyeyCEsHnzH1i2bCB5ea9pIKhW0SBQqhNrHggJCZ/g4RHM5s3XayCoVtEgUOoEICKEhp7L0KHLSUiYj4dHkD0QBpGXN4fGxnqrS1QdmAaBUicQWyCcx9ChK+yB0JXNm69j2bKB5OfP1UBQh6VBoNQJ6OBA+BgPj0A2bZqsgaAOS4NAqROYLRDOZ+jQlSQkfISHRwCbNk1m+fJB5Of/VwNBARoESrkEWyBcwNChvxIf/yFubl3YtGkSy5cPJj//dQ0EF6dBoJQLERHCwi4kLW1/IPixadO1GgguToNAKRck4tYsED7Azc3XHgjx5Oe/gTENVpeo2pEGgVIuzBYIF5GWtor4+Pdxc/Nh06ZrWLZsIDk5M6ivL7O6RNUONAiUUvZAmGAPhPfw9AwlI+NOfvklki1bbmHv3g1Wl6icSINAKdXEFggXM2TIYoYMWU5Y2CXk5b3K8uXxpKePo7DwI71sdALSIFBKHVZgYBqDBs1h5MidxMY+TlXVVtavv4glS+LYseMp6uqKrS5ROYgGgVLqqLy8woiOvp/hwzOJj38fX98+ZGbex+LFUWzadD0VFausLlEdJw+rC1BKdQ5ubh6EhU0gLGwClZXr2LXreQoKXic//zUCA08mMvJ2wsIm6FSanZC2CJRSrebvn8CAATMZOXIXffr8m9raAjZuvJIlS2LIyvobNTX5VpeoWkGDQCnVZp6eQfTqNY3hw7eQmLgAf/8UsrKms2RJbzZsuIqyssUYY6wuUx2DXhpSSh03ETdCQsYTEjKeffu2smvXC+Tnv8bu3W/i7z+EyMjb6d79CtzdfawuVR2GtgiUUg7l59ePfv2eYeTIXfTr9yKNjdVs3nwdixdHkZl5P9XVO6wuUR1Cg0Ap5RQeHv5ERt7MSSetIzn5W4KCTmXHjr+zZEks69ZNoKTke71s1EHopSGllFOJCMHBYwkOHkt1dTa7dr1EXt5sioo+xM8vnsjI2wgPn4iHh7/VpbosbREopdqNj080ffo8yciROQwY8Cpubl5s3XozixdHkZExjX37Mqwu0SVpECil2p27uy89elzH0KErSU39mZCQs9m163mWLevHmjXjKS7+HGMarS7TZeilIaWUZUSErl1PpmvXk6mp+Te5uS+Tl/cya9eOx9e3LxERkwkOPpOAgCGI6OdWZ5HO1lmTlpZmVqxYYXUZSiknaWyspbDwfXbtep7y8l8A8PAIITh4HN26/Z7g4DPw8eltcZWdj4isNMakHe41bREopToUNzcvwsOvJDz8SmprCygp+YY9e76ipORrCgvfAcDXt39TKAQFjcbDI9Diqjs3p7YIROQsYAbgDrxijHnyCOtdDLwHnGSMOerHfW0RKOWajDHs3buekpKvKSn5itLSH2lsrELEg8DAEQQH24IhICANNzf9jHuoo7UInBYEIuIObAHOAHKA5cCVxpgNh6wXACwAvIDbNAiUUi3R2FhDWdkvlJR8xZ49X1NZ+Stg8PAIIihobFOLwdc3zupSOwSrLg0NAzKMMZn2It4CLgAOneroEeAp4E9OrEUpdYJxc/MmOHgMwcFjiIt7gtraIkpLv226jFRU9AEAPj596NbtDIKDf09Q0Bg8PYMsrrzjcWYQRAI7mz3PAYY3X0FEhgC9jDELROSIQSAiNwI3AvTurZ1ESqnf8vIKpXv3y+ne/XKMMVRVbWkKhYKCN8jNnQm4ERg4nODgM+jW7fcEBAzDzc3T6tItZ9mFNLHdC/ZvYPKx1jXGzAJmge3SkHMrU0p1diKCn98A/PwGEBV1O42NdZSXL2m6jJSd/SjZ2Q/j7h5AUNCYZpeR+iEiVpff7pwZBLuAXs2eR9mX7RcAJAA/2N/4CGC+iJx/rH4CpZRqDTc3T4KCTiEo6BRiYx+hrq6E0tLv2LPH1vFcXDwfAG/v6KbLSMHB4/D07GZx5e3DmZ3FHtg6i8dhC4DlwFXGmPVHWP8H4B7tLFZKtbeqqm1Nl5FKSr6loaEcEAIC0ggOPoPQ0AsJCEjr1K0FSzqLjTH1InIb8CW220dfNcasF5GHgRXGmPnOOrZSSrWGr28fIiNvJjLyZhob66moWE5Jydfs2fMVO3Y8xY4dj+PnN4iIiEmEh0/E2zvS6pIdSr9ZrJRSR1FXV0ph4Tvk58+1f9PZjeDgM4iImERo6IW4u/taXWKLWPI9AmfRIFBKWWXfvq0UFPyX/Pz/UlOzA3f3QLp3v4zw8El07TqqQ1860iBQSikHMqaR0tIfyc+fS2HhezQ27sXHpw8REdcSHn4tvr4xVpf4GxoESinlJPX1lRQVvU9+/lxKS78HoGvX04iImExY2MV4eARYXKGNBoFSSrWD6ups8vNfp6BgLlVVGbi5+REWdjEREZMIChpj6VDaGgRKKdWOjDGUly8mP38uu3e/RUNDOd7evQgPv4aIiEn4+fVv95o0CJRSyiINDVUUFX1MQcFc9uz5CmgkMHCE/dLR5e029pEGgVJKdQA1NbkUFMwjP38u+/atR8Sb0NALiIiYRHDw7506fLYGgVJKdSDGGCorfyU/fy4FBf+jvr4YL68Iune/moiISfj7Jzr8mBoESinVQTU21lJc/BkFBXMpLv4UY+rx908lImIS3btfhZdXmEOOo0GglFKdQG1tIbt3v0l+/lwqK39FxINu3cYTETGJkJBzcXPzavO+NQiUUqqTqaxcR0HBXAoK3qC2Nh8Pj2706/cc4eFXtWl/Onm9Ukp1Mv7+Cfj7/4PY2CcoKfma/Py5+PjEOOVYGgRKKdWBubl5EBJyNiEhZzvvGE7bs1JKqU5Bg0AppVycBoFSSrk4DQKllHJxGgRKKeXiNAiUUsrFaRAopZSL0yBQSikX1+mGmBCRQiDb6jqOUyhQZHURHYi+Hwfoe3EwfT8OdjzvR7Qx5rAj2HW6IDgRiMiKI4354Yr0/ThA34uD6ftxMGe9H3ppSCmlXJwGgVJKuTgNAmvMsrqADkbfjwP0vTiYvh8Hc8r7oX0ESinl4rRFoJRSLk6DQCmlXJwGQTsSkV4i8r2IbBCR9SJyh9U1WU1E3EVklYh8anUtVhORIBF5T0Q2ichGERlpdU1WEpFp9n8n60TkTRHxsbqm9iIir4rIbhFZ12xZNxH5WkS22n8HO+p4GgTtqx642xgzGBgB3Coigy2uyWp3AButLqKDmAF8YYwZCCTjwu+LiEQCU4E0Y0wC4A5cYW1V7WoOcNYhy+4DvjXG9AO+tT93CA2CdmSMyTPG/Gp/XIHtH3qktVVZR0SigHOAV6yuxWoi0hU4FfgPgDGm1hhTam1VlvMAfEXEA/ADci2up90YYxYCew5ZfAEw1/54LnCho46nQWAREYkBUoGl1lZiqWeAPwONVhfSAcQChcBr9ktlr4hIF6uLsooxZhfwT2AHkAeUGWO+srYqy4UbY/Lsj/OBcEftWIPAAiLiD7wP3GmMKbe6HiuIyLnAbmPMSqtr6SA8gCHAS8aYVGAvDmz6dzb2698XYAvInkAXEZlobVUdh7Hd9++we/81CNqZiHhiC4F5xpgPrK7HQqOA80UkC3gLGCsib1hbkqVygBxjzP4W4nvYgsFVnQ5sN8YUGmPqgA+Aky2uyWoFItIDwP57t6N2rEHQjkREsF0D3miM+bfV9VjJGHO/MSbKGBODrRPwO2OMy37iM8bkAztFZIB90Thgg4UlWW0HMEJE/Oz/bsbhwp3ndvOBSfbHk4CPHbVjDYL2NQq4Btun33T7z3iri1Idxu3APBFZA6QAj1tcj2XsLaP3gF+BtdjOVS4z3ISIvAksBgaISI6I/AF4EjhDRLZiazE96bDj6RATSinl2rRFoJRSLk6DQCmlXJwGgVJKuTgNAqWUcnEaBEop5eI0CJSyE5GGZrf1pouIw77ZKyIxzUeSVKoj8bC6AKU6kCpjTIrVRSjV3rRFoNQxiEiWiPxdRNaKyDIR6WtfHiMi34nIGhH5VkR625eHi8iHIrLa/rN/aAR3EZltH2P/KxHxta8/1T5HxRoRecuiP1O5MA0CpQ7wPeTS0OXNXiszxiQCz2MbNRXgOWCuMSYJmAc8a1/+LPCjMSYZ23hB6+3L+wEvGGPigVLgYvvy+4BU+36mOOuPU+pI9JvFStmJSKUxxv8wy7OAscaYTPuggfnGmBARKQJ6GGPq7MvzjDGhIlIIRBljaprtIwb42j6pCCJyL+BpjHlURL4AKoGPgI+MMZVO/lOVOoi2CJRqGXOEx61R0+xxAwf66M4BXsDWelhun4hFqXajQaBUy1ze7Pdi++NfODB94tXAT/bH3wI3Q9OczF2PtFMRcQN6GWO+B+4FugK/aZUo5Uz6yUOpA3xFJL3Z8y+MMftvIQ22jwpaA1xpX3Y7thnF/oRtdrHr7MvvAGbZR4xswBYKeRyeO/CGPSwEeFanqFTtTfsIlDoGex9BmjGmyOpalHIGvTSklFIuTlsESinl4rRFoJRSLk6DQCmlXJwGgVJKuTgNAqWUcnEaBEop5eL+H3sZ9nESYJvQAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "llCeVaXCVpcW"
      },
      "source": [
        "# **Regularized Cross Entropy**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WEHQUgcAVn6w",
        "outputId": "f02c5695-d8ad-4714-d95a-40ff650be407"
      },
      "source": [
        "model.compile(loss = \"categorical_crossentropy\", optimizer=\"adam\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_64\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_98 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_99 (InputLayer)           [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_63 (Functional)           (None, 256)          91584       input_98[0][0]                   \n",
            "                                                                 input_99[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "lambda_92 (Lambda)              (None, 1)            0           model_63[0][0]                   \n",
            "                                                                 model_63[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rxIpVv9ymMee",
        "outputId": "e8a56c03-14bc-437f-f077-51e8d8831b79"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 8ms/step - loss: 5.9396e-08 - accuracy: 0.5536 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 5.9753e-08 - accuracy: 0.5495 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 5.9568e-08 - accuracy: 0.5524 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 6.0025e-08 - accuracy: 0.5482 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 5.9663e-08 - accuracy: 0.5512 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 5.9573e-08 - accuracy: 0.5505 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 5.9407e-08 - accuracy: 0.5534 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 5.9541e-08 - accuracy: 0.5516 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 6.0050e-08 - accuracy: 0.5487 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 5.9677e-08 - accuracy: 0.5491 - val_loss: 5.9605e-08 - val_accuracy: 0.5602\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xOvcZDI4mMKw"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1VCvFVEUmMBB",
        "outputId": "c6789e71-4782-4bac-af85-7f5cc6b5c876"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 55.97\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "CY2AcHmimptw",
        "outputId": "89a382db-4aca-41d9-8bb6-f90ef77963f6"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAda0lEQVR4nO3de3RV9Z338fdH7hKEFlJUqIKtAyqGRKIoKApMq1YdHWufqlRBaxGXD3hpFazTKWNtl7Y+U6VqEa3gVFRaLPaCRccqopXqgKKCl7FikFjQGMutXrj4ff44OxhiEhI4OyewP6+1srLv+3s25HzOb//22VsRgZmZZdcehS7AzMwKy0FgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yCwVknSHyWNzveyhSSpQtI/F7oOaF21WOG1LXQBtvuQtKHW6J7AR8CWZPzCiJjZ1G1FxIlpLNsaSfojcEwy2gEIYGMyfndEjGvm9iYDX4yIb+StSNutOQgsbyKiqGZYUgVwQUQ8Unc5SW0jYnNL1taa1Q4ySTOAyoj4t8JVZFnjU0OWOknHSaqUNFHSamC6pM9I+oOkKkl/T4Z711pnvqQLkuExkp6UdEOy7BuSTtzBZftKWiBpvaRHJN0i6e4G6m5KjT+Q9Odkew9L6lFr/jmSVkiqlnT1Dh67kyUtkbRG0lOSSmrNmyjprWTfr0oaKekE4LvA1yVtkPR8E/bRQdKNkv6W/NwoqUMyr0fyutdIek/SE5L2aGj/O/IarfAcBNZS9gY+C+wPjCX3f296Mr4f8AFwcyPrDwZeBXoAPwZ+IUk7sOw9wDNAd2AycE4j+2xKjWcD5wGfA9oD3wGQdDDw82T7+yb7600zSCoD7gQuTNa/Dfhd8sbdD/i/wOER0QU4HqiIiHnAj4BZEVEUEQObsKurgSOBUmAgcARQ0yL5NlAJFAM9yYVMNLT/5rw+az12ySCQdKekdyQtzdP2fixpmaSXJU1p5A3GdtzHwPcj4qOI+CAiqiPi/oh4PyLWAz8Ejm1k/RURcXtEbAHuAvYh98bU5GUl7QccDvx7RGyMiCeB3zW0wybWOD0i/jciPgB+Re7NFOAM4A8RsSAiPgK+lxyD5hgL3BYRT0fEloi4i1y/y5Hk+l46AAdLahcRFRHxejO3X2MUcE1EvBMRVcB/8ElAbiJ3/PaPiE0R8UTkblCWz/1bge2SQQDMAE7Ix4YkDQGGAiXAAHJvFI29IdmOqYqID2tGJO0p6bbk1Mk6YAHQTVKbBtZfXTMQEe8ng0XNXHZf4L1a0wBWNlRwE2tcXWv4/Vo17Vt72xHxD6C6oX01YH/g28lpmTWS1gCfB/aNiL8Cl5Jr1bwj6T5J+zZz+zX2BVbUGl+RTAP4CfBX4GFJyyVNSl5PPvdvBbZLBkFELADeqz1N0hckzZO0ODmP2b+pmwM6kmvWdwDaAW/ntWCD3HGu7dtAP2BwROwFDEump9kaWwV8VtKetaZ9vpHld6bGVbW3neyze/PKZSXww4joVutnz4i4FyAi7omIo8kFRgDXJ+s195bCf0u2UWO/ZBoRsT4ivh0RBwD/Alxe0xfQyP5tF7NLBkEDpgHjI2IQufO0tzZlpYhYCDxG7g93FfBQRLycWpVWowu5c+5rJH0W+H7aO4yIFcAiYLKk9pKOAk5JqcbZwMmSjpbUHriG5v+93Q6MkzRYOZ0lnSSpi6R+kkYknbofJnXWnHp6G+hT06nbBPcC/yapOOns/nfgbtjaWf3F5HTpWnKnhD7ezv5tF7NbBIGkImAI8GtJS8h1qu2TzDtd0tJ6fh5K5n8ROIhcR14vYISkY+rfk+XRjUAn4F3gL8C8FtrvKOAocqdprgVmkTvvXp8drjEilgEXk+ucXgX8nVyna5NFxCLgW+Q6qP9O7hTNmGR2B+C6pLbV5Dqrr0rm/Tr5XS3p2Sbs6lpyAfkC8CLwbDIN4EDgEWADsBC4NSIe287+bRejXfXBNJL6kOuMGyBpL+DViNhnB7ZzBdAxIn6QjP878GFE/Dif9VrrJGkW8EpEpN4iMWutdosWQUSsA96Q9DWApBndlMvmAN4EjpXUVlI7ch3FPjW0m5J0eNKftEdyzf2pwAOFrsuskHbJIJB0L7lmaj/lvqj0TXJN/m8mX6BZRu4PvClmA6+TaxI/DzwfEb9PoWxrHfYG5pM71TEFuCginitoRWYFtsueGjIzs/zYJVsEZmaWP7vcTed69OgRffr0KXQZZma7lMWLF78bEcX1zdvlgqBPnz4sWrSo0GWYme1SJK1oaJ5PDZmZZZyDwMws4xwEZmYZt8v1EZhZ67Vp0yYqKyv58MMPt7+wpaJjx4707t2bdu3aNXkdB4GZ5U1lZSVdunShT58++LEeLS8iqK6uprKykr59+zZ5PZ8aMrO8+fDDD+nevbtDoEAk0b1792a3yBwEZpZXDoHC2pHj7yAwM8s4B4GZ7Taqq6spLS2ltLSUvffem169em0d37hxY6PrLlq0iAkTJmx3H0OGDMlLrfPnz+fkk0/Oy7Z2ljuLzWy30b17d5YsWQLA5MmTKSoq4jvf+c7W+Zs3b6Zt2/rf9srLyykvL9/uPp566qn8FNuKuEVgZru1MWPGMG7cOAYPHsyVV17JM888w1FHHUVZWRlDhgzh1VdfBbb9hD558mTOP/98jjvuOA444ACmTJmydXtFRUVblz/uuOM444wz6N+/P6NGjaLmbs4PPvgg/fv3Z9CgQUyYMGG7n/zfe+89TjvtNEpKSjjyyCN54YUXAHj88ce3tmjKyspYv349q1atYtiwYZSWljJgwACeeOKJnT5GbhGYWSpee+1SNmxYktdtFhWVcuCBNzZ7vcrKSp566inatGnDunXreOKJJ2jbti2PPPII3/3ud7n//vs/tc4rr7zCY489xvr16+nXrx8XXXTRp67Nf+6551i2bBn77rsvQ4cO5c9//jPl5eVceOGFLFiwgL59+3LWWWdtt77vf//7lJWV8cADD/Doo49y7rnnsmTJEm644QZuueUWhg4dyoYNG+jYsSPTpk3j+OOP5+qrr2bLli28//77zT4edTkIzGy397WvfY02bdoAsHbtWkaPHs1rr72GJDZt2lTvOieddBIdOnSgQ4cOfO5zn+Ptt9+md+/e2yxzxBFHbJ1WWlpKRUUFRUVFHHDAAVuv4z/rrLOYNm1ao/U9+eSTW8NoxIgRVFdXs27dOoYOHcrll1/OqFGjOP300+nduzeHH344559/Pps2beK0006jtLR0p44NOAjMLCU78sk9LZ07d946/L3vfY/hw4czZ84cKioqOO644+pdp0OHDluH27Rpw+bNm3domZ0xadIkTjrpJB588EGGDh3KQw89xLBhw1iwYAFz585lzJgxXH755Zx77rk7tR/3EZhZpqxdu5ZevXoBMGPGjLxvv1+/fixfvpyKigoAZs2atd11jjnmGGbOnAnk+h569OjBXnvtxeuvv86hhx7KxIkTOfzww3nllVdYsWIFPXv25Fvf+hYXXHABzz777E7X7CAws0y58sorueqqqygrK8v7J3iATp06ceutt3LCCScwaNAgunTpQteuXRtdZ/LkySxevJiSkhImTZrEXXfdBcCNN97IgAEDKCkpoV27dpx44onMnz+fgQMHUlZWxqxZs7jkkkt2uuZd7pnF5eXl4QfTmLVOL7/8MgcddFChyyi4DRs2UFRURERw8cUXc+CBB3LZZZe12P7r+3eQtDgi6r0+1i0CM7M8u/322yktLeWQQw5h7dq1XHjhhYUuqVHuLDYzy7PLLrusRVsAO8stAjOzjHMQmJllnIPAzCzjHARmZhnnzmIz221UV1czcuRIAFavXk2bNm0oLi4G4JlnnqF9+/aNrj9//nzat29f762mZ8yYwaJFi7j55pvzX3iBpRoEkroBdwADgADOj4iFteaPAiYCAtYDF0XE82nWZGa7r+3dhnp75s+fT1FRUd6eObCrSPvU0E3AvIjoDwwEXq4z/w3g2Ig4FPgB0PidmczMmmnx4sUce+yxDBo0iOOPP55Vq1YBMGXKFA4++GBKSko488wzqaioYOrUqfz0pz+ltLS00ds7V1RUMGLECEpKShg5ciRvvvkmAL/+9a8ZMGAAAwcOZNiwYQAsW7aMI444gtLSUkpKSnjttdfSf9HNlFqLQFJXYBgwBiAiNgLbPCIoImo/4eEvwLa39jOzXdal8y5lyer83oa6dO9Sbjyh6TeziwjGjx/Pb3/7W4qLi5k1axZXX301d955J9dddx1vvPEGHTp0YM2aNXTr1o1x48Y1qRUxfvx4Ro8ezejRo7nzzjuZMGECDzzwANdccw0PPfQQvXr1Ys2aNQBMnTqVSy65hFGjRrFx40a2bNmyU8cgDWm2CPoCVcB0Sc9JukNS50aW/ybwxxTrMbOM+eijj1i6dClf+tKXKC0t5dprr6WyshKAkpISRo0axd13393gU8sasnDhQs4++2wAzjnnHJ588kkAhg4dypgxY7j99tu3vuEfddRR/OhHP+L6669nxYoVdOrUKY+vMD/S7CNoCxwGjI+IpyXdBEwCvld3QUnDyQXB0fVtSNJYYCzAfvvtl1rBZpY/zfnknpaI4JBDDmHhwoWfmjd37lwWLFjA73//e374wx/y4osv7vT+pk6dytNPP83cuXMZNGgQixcv5uyzz2bw4MHMnTuXr3zlK9x2222MGDFip/eVT2m2CCqByoh4OhmfTS4YtiGphFyH8qkRUV3fhiJiWkSUR0R5zRUAZmbb06FDB6qqqrYGwaZNm1i2bBkff/wxK1euZPjw4Vx//fWsXbuWDRs20KVLF9avX7/d7Q4ZMoT77rsPgJkzZ3LMMccA8PrrrzN48GCuueYaiouLWblyJcuXL+eAAw5gwoQJnHrqqVsfQ9mapBYEEbEaWCmpXzJpJPBS7WUk7Qf8BjgnIv43rVrMLJv22GMPZs+ezcSJExk4cCClpaU89dRTbNmyhW984xsceuihlJWVMWHCBLp168Ypp5zCnDlztttZ/LOf/Yzp06dTUlLCL3/5S2666SYArrjiCg499FAGDBjAkCFDGDhwIL/61a8YMGAApaWlLF26dKcfIpOGVG9DLamU3Kf99sBy4Dzg6wARMVXSHcBXgRXJKpsbuk1qDd+G2qz18m2oW4fm3oY61e8RRMQSoO6Op9aafwFwQZo1mJlZ43yLCTOzjHMQmFle7WpPPdzd7MjxdxCYWd507NiR6upqh0GBRATV1dV07NixWev5pnNmlje9e/emsrKSqqqqQpeSWR07dqR37+bdpMFBYGZ5065dO/r27VvoMqyZfGrIzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGpRoEkrpJmi3pFUkvSzqqzvz+khZK+kjSd9KsxczM6tc25e3fBMyLiDMktQf2rDP/PWACcFrKdZiZWQNSaxFI6goMA34BEBEbI2JN7WUi4p2I+B9gU1p1mJlZ49I8NdQXqAKmS3pO0h2SOu/IhiSNlbRI0qKqqqr8VmlmlnFpBkFb4DDg5xFRBvwDmLQjG4qIaRFRHhHlxcXF+azRzCzz0gyCSqAyIp5OxmeTCwYzM2tFUguCiFgNrJTUL5k0Engprf2ZmdmOSfuqofHAzOSKoeXAeZLGAUTEVEl7A4uAvYCPJV0KHBwR61Kuy8zMEqkGQUQsAcrrTJ5aa/5qoHeaNZiZWeP8zWIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxqQaBpG6SZkt6RdLLko6qM1+Spkj6q6QXJB2WZj1mZvZpbVPe/k3AvIg4Q1J7YM86808EDkx+BgM/T36bmVkLSa1FIKkrMAz4BUBEbIyINXUWOxX4r8j5C9BN0j5p1WRmZp+W5qmhvkAVMF3Sc5LukNS5zjK9gJW1xiuTaduQNFbSIkmLqqqq0qvYzCyD0gyCtsBhwM8jogz4BzBpRzYUEdMiojwiyouLi/NZo5lZ5qUZBJVAZUQ8nYzPJhcMtb0FfL7WeO9kmpmZtZDUgiAiVgMrJfVLJo0EXqqz2O+Ac5Orh44E1kbEqrRqMjOzT0v7qqHxwMzkiqHlwHmSxgFExFTgQeArwF+B94HzUq7HzMzqaFIQJJ28H0TEx5L+CegP/DEiNjW2XkQsAcrrTJ5aa34AFzevZDMzy6emnhpaAHSU1At4GDgHmJFWUWZm1nKaGgSKiPeB04FbI+JrwCHplWVmZi2lyUGQ3B5iFDA3mdYmnZLMzKwlNTUILgWuAuZExDJJBwCPpVeWmZm1lCZ1FkfE48DjAJL2AN6NiAlpFmZmZi2jSS0CSfdI2iu5emgp8JKkK9ItzczMWkJTv0dwcESskzQK+CO5W0UsBn6SWmV5Nua+Mpa++0ahyzAz22EDevRlxpnP5X27Te0jaCepHXAa8Lvk+wOR92rMzKzFNbVFcBtQATwPLJC0P7AuraLSkEaKmpntDpraWTwFmFJr0gpJw9MpyczMWlJTO4u7SvrPmmcCSPp/QN1nC5iZ2S6oqX0EdwLrgf+T/KwDpqdVlJmZtZym9hF8ISK+Wmv8PyQtSaMgMzNrWU1tEXwg6eiaEUlDgQ/SKcnMzFpSU1sE44D/Sh5ID/B3YHQ6JZmZWUtq6lVDzwMDJe2VjK+TdCnwQprFmZlZ+pr1qMqIWBcRNd8fuDyFeszMrIXtzDOLlbcqzMysYHYmCHyLCTOz3UCjfQSS1lP/G76ATqlUZGZmLarRIIiILi1ViJmZFcbOnBoyM7PdgIPAzCzjHARmZhnX1G8W7xBJFeRuVrcF2BwR5XXmf4bcDe2+AHwInB8RS9OsyczMtpVqECSGR8S7Dcz7LrAkIv5VUn/gFmBkC9RkZmaJQp8aOhh4FCAiXgH6SOpZ2JLMzLIl7SAI4GFJiyWNrWf+88DpAJKOAPYHetddSNLYmofiVFVVpVqwmVnWpB0ER0fEYcCJwMWShtWZfx3QLXm2wXjgOXL9CduIiGkRUR4R5cXFxSmXbGaWLan2EUTEW8nvdyTNAY4AFtSavw44D0CSgDeA5WnWZGZm20qtRSCps6QuNcPAl4GldZbpJql9MnoBsKDW3U3NzKwFpNki6AnMyX3Qpy1wT0TMkzQOICKmAgcBd0kKYBnwzRTrMTOzeqQWBBGxHBhYz/SptYYXAv+UVg1mZrZ9hb581MzMCsxBYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWca1TXPjkiqA9cAWYHNElNeZ3xW4G9gvqeWGiJieZk1mZratVIMgMTwi3m1g3sXASxFxiqRi4FVJMyNiYwvUZWZmFP7UUABdJAkoAt4DNhe2JDOzbEk7CAJ4WNJiSWPrmX8zcBDwN+BF4JKI+LjuQpLGSlokaVFVVVW6FZuZZUzaQXB0RBwGnAhcLGlYnfnHA0uAfYFS4GZJe9XdSERMi4jyiCgvLi5OuWQzs2xJNQgi4q3k9zvAHOCIOoucB/wmcv4KvAH0T7MmMzPbVmpBIKmzpC41w8CXgaV1FnsTGJks0xPoByxPqyYzM/u0NK8a6gnMyfUD0xa4JyLmSRoHEBFTgR8AMyS9CAiY2MgVRmZmloLUgiAilgMD65k+tdbw38i1FMzMrEAKffmomZkVmIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMs5BYGaWcQ4CM7OMcxCYmWWcg8DMLOMcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjGub5sYlVQDrgS3A5ogorzP/CmBUrVoOAooj4r006zIzs0+kGgSJ4RHxbn0zIuInwE8AJJ0CXOYQMDNrWa3p1NBZwL2FLsLMLGvSDoIAHpa0WNLYhhaStCdwAnB/A/PHSlokaVFVVVVKpZqZZVPaQXB0RBwGnAhcLGlYA8udAvy5odNCETEtIsojory4uDitWs3MMinVIIiIt5Lf7wBzgCMaWPRMfFrIzKwgUgsCSZ0ldakZBr4MLK1nua7AscBv06rFzMwaluZVQz2BOZJq9nNPRMyTNA4gIqYmy/0r8HBE/CPFWszMrAGpBUFELAcG1jN9ap3xGcCMtOowM7PGtabLR83MrAAcBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhnnIDAzyzgHgZlZxjkIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xQRha6hWSRVASsKXcdO6gG8W+giWhEfj235eHzCx2JbO3M89o+I4vpm7HJBsDuQtCgiygtdR2vh47EtH49P+FhsK63j4VNDZmYZ5yAwM8s4B0FhTCt0Aa2Mj8e2fDw+4WOxrVSOh/sIzMwyzi0CM7OMcxCYmWWcg6AFSfq8pMckvSRpmaRLCl1ToUlqI+k5SX8odC2FJqmbpNmSXpH0sqSjCl1TIUm6LPk7WSrpXkkdC11TS5J0p6R3JC2tNe2zkv5b0mvJ78/kY18Ogpa1Gfh2RBwMHAlcLOngAtdUaJcALxe6iFbiJmBeRPQHBpLh4yKpFzABKI+IAUAb4MzCVtXiZgAn1Jk2CfhTRBwI/CkZ32kOghYUEasi4tlkeD25P/Reha2qcCT1Bk4C7ih0LYUmqSswDPgFQERsjIg1ha2q4NoCnSS1BfYE/lbgelpURCwA3qsz+VTgrmT4LuC0fOzLQVAgkvoAZcDTha2koG4ErgQ+LnQhrUBfoAqYnpwqu0NS50IXVSgR8RZwA/AmsApYGxEPF7aqVqFnRKxKhlcDPfOxUQdBAUgqAu4HLo2IdYWupxAknQy8ExGLC11LK9EWOAz4eUSUAf8gT83+XVFy7vtUcgG5L9BZ0jcKW1XrErlr//Ny/b+DoIVJakcuBGZGxG8KXU8BDQX+RVIFcB8wQtLdhS2poCqByoioaSHOJhcMWfXPwBsRURURm4DfAEMKXFNr8LakfQCS3+/kY6MOghYkSeTOAb8cEf9Z6HoKKSKuiojeEdGHXCfgoxGR2U98EbEaWCmpXzJpJPBSAUsqtDeBIyXtmfzdjCTDnee1/A4YnQyPBn6bj406CFrWUOAccp9+lyQ/Xyl0UdZqjAdmSnoBKAV+VOB6CiZpGc0GngVeJPdelanbTUi6F1gI9JNUKembwHXAlyS9Rq7VdF1e9uVbTJiZZZtbBGZmGecgMDPLOAeBmVnGOQjMzDLOQWBmlnEOArOEpC21LutdIilv3+yV1Kf2XSTNWpO2hS7ArBX5ICJKC12EWUtzi8BsOyRVSPqxpBclPSPpi8n0PpIelfSCpD9J2i+Z3lPSHEnPJz81t0ZoI+n25B77D0vqlCw/IXlGxQuS7ivQy7QMcxCYfaJTnVNDX681b21EHArcTO6uqQA/A+6KiBJgJjAlmT4FeDwiBpK7X9CyZPqBwC0RcQiwBvhqMn0SUJZsZ1xaL86sIf5msVlC0oaIKKpnegUwIiKWJzcNXB0R3SW9C+wTEZuS6asiooekKqB3RHxUaxt9gP9OHiiCpIlAu4i4VtI8YAPwAPBARGxI+aWabcMtArOmiQaGm+OjWsNb+KSP7iTgFnKth/9JHsRi1mIcBGZN8/Vavxcmw0/xyeMTRwFPJMN/Ai6Crc9k7trQRiXtAXw+Ih4DJgJdgU+1SszS5E8eZp/oJGlJrfF5EVFzCelnkruCfgSclUwbT+6JYleQe7rYecn0S4Bpyd0it5ALhVXUrw1wdxIWAqb4EZXW0txHYLYdSR9BeUS8W+hazNLgU0NmZhnnFoGZWca5RWBmlnEOAjOzjHMQmJllnIPAzCzjHARmZhn3/wGZNxKicddttQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NiuZWv1ZuPU0"
      },
      "source": [
        "# **Comparison of Regularized Cross Entropy, Triplet Loss and Contrastive loss functions**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "emVP7Hiovobc"
      },
      "source": [
        "*   Test Accuracy using **Contrastive loss : 97.289%**\n",
        "*   Test Accuracy using **Triplet Loss : 96.030%**\n",
        "*   Test Accuracy using **Regularized Cross Entropy : 55.97%**\n",
        "\n",
        "Hence, **Contrastive loss is best** among all three loss functions."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A3Ve0vaL14vA"
      },
      "source": [
        "# **Siamese Network with various Optimisers**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SIp7rCqX2I0w"
      },
      "source": [
        "**RMSprop**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wi1GGT2X2Gn5",
        "outputId": "4e51f358-ab70-4c35-9e7a-87d3171af4a2"
      },
      "source": [
        "model.compile(loss = contrastive_loss, optimizer=\"RMSprop\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_80\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_122 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_123 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_79 (Functional)           (None, 256)          91584       input_122[0][0]                  \n",
            "                                                                 input_123[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "lambda_116 (Lambda)             (None, 1)            0           model_79[0][0]                   \n",
            "                                                                 model_79[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0eYWrU_82Gkg",
        "outputId": "5ecd400b-0ecb-4e78-b626-7c8b4600d271"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 8ms/step - loss: 0.0888 - accuracy: 0.8892 - val_loss: 0.0401 - val_accuracy: 0.9552\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0280 - accuracy: 0.9702 - val_loss: 0.0275 - val_accuracy: 0.9674\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0181 - accuracy: 0.9801 - val_loss: 0.0242 - val_accuracy: 0.9697\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0130 - accuracy: 0.9857 - val_loss: 0.0227 - val_accuracy: 0.9725\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0102 - accuracy: 0.9888 - val_loss: 0.0243 - val_accuracy: 0.9696\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0090 - accuracy: 0.9904 - val_loss: 0.0228 - val_accuracy: 0.9723\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0074 - accuracy: 0.9924 - val_loss: 0.0230 - val_accuracy: 0.9717\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0069 - accuracy: 0.9926 - val_loss: 0.0230 - val_accuracy: 0.9714\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0063 - accuracy: 0.9933 - val_loss: 0.0217 - val_accuracy: 0.9733\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0055 - accuracy: 0.9943 - val_loss: 0.0232 - val_accuracy: 0.9717\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K_j4LVDW13EH"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mtW7m7YO2SRB",
        "outputId": "ababe412-9459-4e9c-df44-67b54b0d582d"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 96.18345000000001\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "ZWAJyP762R1E",
        "outputId": "2b04eafe-374a-4482-8bf3-aedb966ebf28"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5fn/8fedmezrJARUgoR9S0Iom4BaRKii1qXaKqJirVX7dbeLVttq/bb9aWvr0tq61K2gX2nd677gWtAKikCAsAYJa8gO2Sf37485hCQMZB1Olvt1XXPlzFnvGSWfPOc55zmiqhhjjDHNhbldgDHGmK7JAsIYY0xQFhDGGGOCsoAwxhgTlAWEMcaYoCwgjDHGBGUBYboVEXlDROZ19rpuEpE8EZnpdh3QtWox7vO6XYDp+URkb6O3MUA14HfeX6mqT7d2X6o6OxTrdkUi8gZwgvM2ElCgxnm/QFWvauP+7gCGqupFnVak6dEsIEzIqWrc/mkRyQMuV9V3m68nIl5VrTuStXVljQNORJ4E8lX1F+5VZHobO8VkXCMi00UkX0RuFpGdwBMi4hORV0WkQESKnem0Rtt8ICKXO9OXisgnInKPs+5mEZndznUHichHIlIuIu+KyIMisuAQdbemxv8Vkf84+3tbRPo0Wn6xiGwRkUIRua2d390ZIrJcREpEZLGIZDVadrOIbHOOnSsiJ4vIqcCtwPkisldEvmrFMSJF5D4R2e687hORSGdZH+dzl4hIkYh8LCJhhzp+ez6jcZ8FhHHbUUAyMBC4gsD/k084748FKoG/HGb7yUAu0Af4PfCYiEg71n0G+C+QAtwBXHyYY7amxguB7wN9gQjgJwAiMhr4m7P/Y5zjpdEGIjIOeBy40tn+YeAV5xf6COAaYKKqxgOnAHmq+ibwO2Chqsap6thWHOo24DggGxgLTAL2t2B+DOQDqUA/AuGjhzp+Wz6f6TosIIzb6oHbVbVaVStVtVBVn1fVClUtB34LfPMw229R1UdV1Q88BRxN4BdWq9cVkWOBicCvVLVGVT8BXjnUAVtZ4xOquk5VK4F/EvglC3Ae8KqqfqSq1cAvne+gLa4AHlbVz1TVr6pPEejXOY5A304kMFpEwlU1T1U3tnH/+80F7lTV3apaAPyaA8FZS+D7G6iqtar6sQYGduvM4xuXWUAYtxWoatX+NyISIyIPO6dgyoCPgCQR8Rxi+537J1S1wpmMa+O6xwBFjeYBbD1Uwa2scWej6YpGNR3TeN+qug8oPNSxDmEg8GPn9E6JiJQAA4BjVHUDcAOBVtBuEXlWRI5p4/73OwbY0uj9FmcewB+ADcDbIrJJRG5xPk9nHt+4zALCuK35cMI/BkYAk1U1ATjRmX+o00adYQeQLCIxjeYNOMz6HalxR+N9O8dMaVu5bAV+q6pJjV4xqvp/AKr6jKoeTyBIFLjb2a6tQzdvd/ax37HOPFS1XFV/rKqDgTOBm/b3NRzm+KabsYAwXU08gXP6JSKSDNwe6gOq6hZgKXCHiESIyBTg2yGq8TngDBE5XkQigDtp+7/DR4GrRGSyBMSKyOkiEi8iI0RkhtOZXOXUuf8U1i4gfX9nciv8H/ALEUl1Otl/BSyAhk7yoU4fTimBU0v1LRzfdDMWEKaruQ+IBvYAnwJvHqHjzgWmEDjd8xtgIYHz+sG0u0ZVzQGuJtApvgMoJtDZ22qquhT4IYGO8WICp3oudRZHAnc5te0k0En+c2fZv5yfhSLyRSsO9RsCwbkCWAl84cwDGAa8C+wFlgB/VdX3Wzi+6WbEHhhkzMFEZCGwVlVD3oIxpquyFoQxgIhMFJEhIhLm3DNwFvCS23UZ4ya7k9qYgKOAFwh0GOcDP1LVL90tyRh32SkmY4wxQdkpJmOMMUH1mFNMffr00fT0dLfLMMaYbmXZsmV7VDU12LIeExDp6eksXbrU7TKMMaZbEZEth1pmp5iMMcYEZQFhjDEmKAsIY4wxQfWYPghjTNdVW1tLfn4+VVVVLa9sQiIqKoq0tDTCw8NbvU1IA8K5I/V+wAP8XVXvarY8EvgHMJ7AGDjnq2qesyyLwINQEggM9jWx8bDQxpjuIz8/n/j4eNLT0zn085xMqKgqhYWF5OfnM2jQoFZvF7JTTM7Y+A8Cs4HRwBznaVqN/QAoVtWhwL04wwKLiJfAqJFXqeoYYDqBB5QYY7qhqqoqUlJSLBxcIiKkpKS0uQUXyj6IScAGVd2kqjXAswTGt2nsLAJP9oLAMMgnO8MHfwtYoapfQcMTvPwhrNUYE2IWDu5qz/cfyoDoT9OncuU784Kuo6p1BMaVTwGGE3i+7Vsi8oWI/CzYAUTkChFZKiJLCwoK2lVkVdXXbNp0K1VVh3yAmDHG9Epd9SomL3A8gTH6jwfO2f+0qsZU9RFVnaCqE1JTg94I2CK/v5yvv/5/FBW91aGCjTFdV2FhIdnZ2WRnZ3PUUUfRv3//hvc1NTWH3Xbp0qVcd911LR5j6tSpnVLrBx98wBlnnNEp++qoUHZSb6PpYxvTnHnB1sl3+h0SCXRW5wMfqeoeABF5HfgG8F5nFxkTM5qIiGMoLn6HY465vLN3b4zpAlJSUli+fDkAd9xxB3FxcfzkJz9pWF5XV4fXG/zX4YQJE5gwYUKLx1i8eHHnFNuFhLIF8TkwTEQGOY9WvAB4pdk6rwDznOnzgEUaGF72LSDTeTi8F/gmsDoURYoIPt9MiovfQ9WejGhMb3HppZdy1VVXMXnyZH72s5/x3//+lylTpjBu3DimTp1Kbm4u0PQv+jvuuIPLLruM6dOnM3jwYB544IGG/cXFxTWsP336dM477zxGjhzJ3Llz2T9q9uuvv87IkSMZP3481113XYsthaKiIs4++2yysrI47rjjWLFiBQAffvhhQwto3LhxlJeXs2PHDk488USys7PJyMjg448/7vB3FLIWhKrWicg1BH7Ze4DHVTVHRO4ElqrqK8BjwHwR2QAUEQgRVLVYRP5EIGQUeF1VXwtVrT7fTHbt+gd79y4nPv4boTqMMQZYv/4G9u5d3qn7jIvLZtiw+9q8XX5+PosXL8bj8VBWVsbHH3+M1+vl3Xff5dZbb+X5558/aJu1a9fy/vvvU15ezogRI/jRj3500L0FX375JTk5ORxzzDFMmzaN//znP0yYMIErr7ySjz76iEGDBjFnzpwW67v99tsZN24cL730EosWLeKSSy5h+fLl3HPPPTz44INMmzaNvXv3EhUVxSOPPMIpp5zCbbfdht/vp6Kios3fR3MhvQ9CVV8HXm8271eNpquA7x5i2wU4D0gPNZ9vJgDFxe9YQBjTi3z3u9/F4/EAUFpayrx581i/fj0iQm1t8CvrTz/9dCIjI4mMjKRv377s2rWLtLS0JutMmjSpYV52djZ5eXnExcUxePDghvsQ5syZwyOPPHLY+j755JOGkJoxYwaFhYWUlZUxbdo0brrpJubOnct3vvMd0tLSmDhxIpdddhm1tbWcffbZZGdnd+i7AbuTGoDIyKOJjc2gqOgdjj32ZrfLMaZHa89f+qESGxvbMP3LX/6Sk046iRdffJG8vDymT58edJvIyMiGaY/HQ11dXbvW6YhbbrmF008/nddff51p06bx1ltvceKJJ/LRRx/x2muvcemll3LTTTdxySWXdOg4XfUqpiPO55tFaekn+P2VbpdijHFBaWkp/fsHrsR/8sknO33/I0aMYNOmTeTl5QGwcOHCFrc54YQTePrpp4FA30afPn1ISEhg48aNZGZmcvPNNzNx4kTWrl3Lli1b6NevHz/84Q+5/PLL+eKLLzpcswWEw+ebhWo1paWfuF2KMcYFP/vZz/j5z3/OuHHjOv0vfoDo6Gj++te/cuqppzJ+/Hji4+NJTEw87DZ33HEHy5YtIysri1tuuYWnngrcV3zfffeRkZFBVlYW4eHhzJ49mw8++ICxY8cybtw4Fi5cyPXXX9/hmnvMM6knTJigHXlgkN+/j08+8ZGWdgNDhvy+EyszxqxZs4ZRo0a5XYbr9u7dS1xcHKrK1VdfzbBhw7jxxhuP2PGD/XcQkWWqGvQ6XmtBODyeWBISplJc/I7bpRhjeqhHH32U7OxsxowZQ2lpKVdeeaXbJR2WdVI3kpw8i82bf0FNzW4iIvq6XY4xpoe58cYbj2iLoaOsBdGIzzcLgOLiRS5XYowx7rOAaCQ+fjxeb5KdZjLGGCwgmhDxkJQ0g+Lid+gpnffGGNNeFhDN+HyzqK7eSmXlOrdLMcYYV1lANJOcHOiHKCqy00zG9BQdGe4bAjepHWq01ieffJJrrrmms0vuEuwqpmaio4cQFTWI4uJ3SUvrmf/RjeltWhruuyUffPABcXFxnfbMh+7CWhBB+HwzKSl5n/r6zr+b0hjTNSxbtoxvfvObjB8/nlNOOYUdO3YA8MADDzB69GiysrK44IILyMvL46GHHuLee+8lOzv7sMNo5+XlMWPGDLKysjj55JP5+uuvAfjXv/5FRkYGY8eO5cQTTwQgJyeHSZMmkZ2dTVZWFuvXrw/9h24ja0EE4fPNYseORykv/y+Jib3rLwZjQu2GN29g+c7OHe47+6hs7ju19YMAqirXXnstL7/8MqmpqSxcuJDbbruNxx9/nLvuuovNmzcTGRlJSUkJSUlJXHXVVa1qdVx77bXMmzePefPm8fjjj3Pdddfx0ksvceedd/LWW2/Rv39/SkpKAHjooYe4/vrrmTt3LjU1Nfj9/g59B6FgLYggfL4ZgNjlrsb0UNXV1axatYpZs2aRnZ3Nb37zG/Lz8wHIyspi7ty5LFiw4JBPmTuUJUuWcOGFFwJw8cUX88kngbHdpk2bxqWXXsqjjz7aEARTpkzhd7/7HXfffTdbtmwhOjq6Ez9h57AWRBDh4SnEx4+nuPhd0tNvd7scY3qUtvylHyqqypgxY1iyZMlBy1577TU++ugj/v3vf/Pb3/6WlStXdvh4Dz30EJ999hmvvfYa48ePZ9myZVx44YVMnjyZ1157jdNOO42HH36YGTNmdPhYnclaEIfg882irOxT6urK3S7FGNPJIiMjKSgoaAiI2tpacnJyqK+vZ+vWrZx00kncfffdlJaWsnfvXuLj4ykvb/l3wdSpU3n22WcBePrppznhhBMA2LhxI5MnT+bOO+8kNTWVrVu3smnTJgYPHsx1113HWWed1fA40a7EAuIQfL6ZqNZRUvKB26UYYzpZWFgYzz33HDfffDNjx44lOzubxYsX4/f7ueiii8jMzGTcuHFcd911JCUl8e1vf5sXX3yxxU7qP//5zzzxxBNkZWUxf/587r//fgB++tOfkpmZSUZGBlOnTmXs2LH885//JCMjg+zsbFatWtXhh/uEgg33fQj19dV88omPo4++nGHDHmh5A2PMIdlw312DDffdScLCIklMPNE6qo0xvZYFxGEkJ8+iomItVVX5bpdijDFHnAXEYfh8MwEoLn7X5UqM6f56yuns7qo9378FxGHExmYSHt7XTjMZ00FRUVEUFhZaSLhEVSksLCQqKqpN29l9EIchEobPN5Pi4ndRrUfE8tSY9khLSyM/P5+CggK3S+m1oqKiSEtLa9M2FhAt8PlmsXv3M+zbt4q4uCy3yzGmWwoPD2fQoEFul2HayP4kbsGBfgg7zWSM6V0sIFoQFZVGTMxIez6EMabXCWlAiMipIpIrIhtE5JYgyyNFZKGz/DMRSXfmp4tIpYgsd14PhbLOlvh8sygt/Qi/v8rNMowx5ogKWUCIiAd4EJgNjAbmiMjoZqv9AChW1aHAvcDdjZZtVNVs53VVqOpsDZ9vFvX1lZSVBX+ilDHG9EShbEFMAjao6iZVrQGeBc5qts5ZwFPO9HPAySIiIaypXZKSpiPitfshjDG9SigDoj+wtdH7fGde0HVUtQ4oBVKcZYNE5EsR+VBETghhnS3yeuNJSDjOOqqNMb1KV+2k3gEcq6rjgJuAZ0QkoflKInKFiCwVkaWhvr7a55tJefkyamsLQ3ocY4zpKkIZENuAAY3epznzgq4jIl4gEShU1WpVLQRQ1WXARmB48wOo6iOqOkFVJ6SmpobgIxzg880ClOLiRSE9jjHGdBWhDIjPgWEiMkhEIoALgFearfMKMM+ZPg9YpKoqIqlOJzciMhgYBmwKYa0tio+fhMeTYKeZjDG9RsjupFbVOhG5BngL8ACPq2qOiNwJLFXVV4DHgPkisgEoIhAiACcCd4pILVAPXKWqRaGqtTXCwrwkJZ1kHdXGmF4jpENtqOrrwOvN5v2q0XQV8N0g2z0PPB/K2trD55tJYeHLVFZuJDp6iNvlGGNMSHXVTuouKTl5FoDdVW2M6RUsINogOno4kZEDrB/CGNMrWEC0gYjg882ipGQRqn63yzHGmJCygGgjn28WdXUllJcvc7sUY4wJKQuINvL5ZgA2/LcxpuezgGijiIi+xMVlW0e1MabHs4BoB59vFmVli6mr2+t2KcYYEzIWEO3g881CtZbS0o/dLsUYY0LGAqIdEhOPRyTS+iGMMT2aBUQ7eDzRJCYebwFhjOnRLCDaKTl5Fvv2raK6eofbpRhjTEhYQLRTYPhvbPA+Y0yPZQHRTnFx2Xi9KRYQxpgeywKinUTC8PlOprj4HVTV7XKMMabTWUB0gM83i5qaHVRUrHa7FGOM6XQWEB1gw38bY3oyC4gOiIoaSHT0MOuHMMb0SBYQHRQY/vsD6utr3C7FGGM6lQVEB/l8M6mv30dZ2adul2KMMZ3KAqKDkpJOAsLsrmpjTI9jAdFB4eFJJCRMso5qY0yPYwHRCXy+WZSXf05tbYnbpRhjTKexgOgEPt9MoJ6SkvfdLsUYYzqNBUQnSEg4jrCwWOuHMMb0KBYQnSAsLIKkpOkWEMaYHsUCopMkJ8+isnIDlZV5bpdijDGdwgKik9jw38aYnsYCopPExIwiIuIYO81kjOkxQhoQInKqiOSKyAYRuSXI8kgRWegs/0xE0pstP1ZE9orIT0JZZ2cQEXy+mRQXv4dqvdvlGGNMh4UsIETEAzwIzAZGA3NEZHSz1X4AFKvqUOBe4O5my/8EvBGqGvcrrCjslP34fLOoqytk794vO2V/xhjjplC2ICYBG1R1k6rWAM8CZzVb5yzgKWf6OeBkEREAETkb2AzkhLBGPvn6EwbeN5C3N77d4X0F7oewfghjTM8QyoDoD2xt9D7fmRd0HVWtA0qBFBGJA24Gfn24A4jIFSKyVESWFhQUtKvIbxz9DdKT0rn4xYvZuXdnu/axX2TkUcTGZtqwG8aYHqGrdlLfAdyrqnsPt5KqPqKqE1R1QmpqarsOFBMew8LzFlJeXc4lL15CfQf7D3y+mZSWfoLfX9mh/RhjjNtCGRDbgAGN3qc584KuIyJeIBEoBCYDvxeRPOAG4FYRuSZUhY7pO4b7T72fdza9wz2L7+nQvny+WahWU1r6cSdVZ4wx7ghlQHwODBORQSISAVwAvNJsnVeAec70ecAiDThBVdNVNR24D/idqv4lhLVy+Tcu53tjvsdti27j0/z2P9shKelERCLscldjTLcXsoBw+hSuAd4C1gD/VNUcEblTRM50VnuMQJ/DBuAm4KBLYY8UEeGRMx4hLSGNOc/PoaSqfSOzejyxJCZOtY5qY0y3J6rqdg2dYsKECbp06dIO7+ez/M84/onjOWfkOSw8byHORVVtsmXLb9m8+RdMnbqLiIi+Ha7JGGNCRUSWqeqEYMu6aie1ayanTea3M37Lv1b/i0e/eLRd+zgw7MZ7nVmaMcYcURYQQfxk6k/41pBvcf2b17Nq96o2bx8fPx6v12f9EMaYbs0CIogwCeMfZ/+DxMhEzn/ufCpqK9q0vYiHpKQZFBe/S085hWeM6X0sIA6hX1w/FnxnAWsK1nDDmze0efvk5FlUV2+lsnJdCKozxpjQs4A4jJmDZ3LL8bfw6BePsnDVwjZtu3/YDbur2hjTXVlAtODX03/NlLQpXPHqFWwq3tTq7aKjhxAVNcj6IYwx3ZYFRAvCPeE8c+4zhEkYc56fQ42/ptXb+nyzKCl5n/r62hBWaIwxoWEB0QrpSen8/dt/57/b/ssvFv2i1dv5fLPw+8spL/88hNUZY0xoWEC00rmjz+VHE37EHxb/gTc3vNmqbXy+kwCx00zGmG7JAqIN/vitP5LZN5NLXryEHeU7Wlw/PDyF+Pjx1lFtjOmWLCDaIDo8moXnLWRvzV4ufvHiVg0N7vPNoqzsU+rqyo5AhcYY03ksINpoVOoo/jz7z7y3+T3u/qT5E1IPFhh2w09JyQchr80YYzqTBUQ7XDbuMi7IuIBfvv9LFm9dfNh1ExOnEhYWbaO7GmO6nVYFhIjEikiYMz1cRM4UkfDQltZ1iQgPnf4QA5MGMuf5ORRXFh9y3bCwSBITT7SOamNMt9PaFsRHQJSI9AfeBi4GngxVUd1BYlQiz577LNvLt3P5vy8/7JhLycmzqKhYS1VV/hGs0BhjOqa1ASGqWgF8B/irqn4XGBO6srqHif0nctfJd/HCmhd4aOlDh1zvwPDf1oowxnQfrQ4IEZkCzAVec+Z5QlNS93LjlBuZPXQ2N751Iyt2rQi6TmxsJuHh/awfwhjTrbQ2IG4Afg686Dw2dDDwfujK6j7CJIwnz36S5Ohkzn/ufPbV7DtoHRHB55vpDP/d8qWxxhjTFbQqIFT1Q1U9U1Xvdjqr96jqdSGurdvoG9uXBd9ZQO6eXK57I/jX4vPNpLZ2N/v2rTzC1RljTPu09iqmZ0QkQURigVXAahH5aWhL615mDJrBbSfcxuPLH+eZlc8ctDw5OdAPYXdVG2O6i9aeYhqtqmXA2cAbwCACVzKZRm6ffjvTBkzjqlevYkPRhibLIiP7ExMzyjqqjTHdRmsDIty57+Fs4BVVrQXsWZrNeMO8PHPuM3jDvEGHBvf5ZlFa+jF+f5VLFRpjTOu1NiAeBvKAWOAjERkI2OBCQRybeCyPn/U4S7cv5db3bm2yzOebSX19JWVlh7/72hhjuoLWdlI/oKr9VfU0DdgCnBTi2rqts0eezdUTr+aPS/7I6+tfb5iflDQdEa+dZjLGdAut7aROFJE/ichS5/VHAq0Jcwj3fOsesvplMe+leWwv3w6A1xtPQsJx1lFtjOkWWnuK6XGgHPie8yoDnghVUT1BlDeKhectpKK2goteuAh/vR8I9EPs3fsFtbWFLldojDGH19qAGKKqt6vqJuf1a2BwKAvrCUb2GcmDpz3I+3nv8/8++X/A/mE3lOLiRe4WZ4wxLWhtQFSKyPH734jINKAyNCX1LPPGzmNu5lxu/+B2Pt7yMfHxE/F4EqwfwhjT5bU2IK4CHhSRPBHJA/4CXNnSRiJyqojkisgGEbklyPJIEVnoLP9MRNKd+ZNEZLnz+kpEzmn1J+piRIS/nf43BvsGc+ELF1JSXUZS0kkUF79z2BFgjTHGba29iukrVR0LZAFZqjoOmHG4bUTEAzwIzAZGA3NEZHSz1X4AFKvqUOBeYP8j2lYBE1Q1GzgVeFhEvK38TF1OfGQ8z577LLv27uKyly/D55tJVVUelZUb3S7NGGMOqU1PlFPVMueOaoCbWlh9ErDB6bOoAZ4Fzmq2zlnAU870c8DJIiKqWqGqdc78KHrATXnjjxnP72f9npdzX+afebsBbHRXY0yX1pFHjkoLy/sDWxu9z3fmBV3HCYRSIAVARCaLSA6wEriqUWAcKEDkiv2X3hYUFLTvUxxB10++njOGn8HPP7ibLdX9rB/CGNOldSQgQvpXvap+pqpjgInAz0UkKsg6j6jqBFWdkJqaGspyOoWI8MRZT9Anpg935FSyfc97qPrdLssYY4I6bECISLmIlAV5lQPHtLDvbcCARu/TnHlB13H6GBKBJjcIqOoaYC+Q0eKn6Qb6xPTh6e88zZbycv60tpTy8qVul2SMMUEdNiBUNV5VE4K84lW1pU7jz4FhIjJIRCKAC4BXmq3zCjDPmT4PWKSq6mzjBXDGfRpJYCyoHmF6+nRunXYTb+2Cxz7/g9vlGGNMUB05xXRYTp/BNcBbwBrgn87T6O4UkTOd1R4DUkRkA4FO7/2Xwh4PfCUiy4EXgf9R1T2hqtUNd8y4i3HJsfxiyUusL1zvdjnGGHMQ6SnX4k+YMEGXLu1ep2s+XnEVZ/z7YYb0yWbJDz4l0hvpdknGmF5GRJap6oRgy0LWgjAtG5N2Lj8bAV/uXM4t7x50H6ExxrjKAsJFiYnHc3xqJJeMyOa+z+7j37n/drskY4xpYAHhIo8nmqSkE7givYZxR43j4hcv5pZ3byFnd47bpRljjAWE23y+WdRWreaZs//G8ccezz2L7yHjbxmMe3gcf1ryJ3aU73C7RGNML2UB4TKfbyYASbqOVy98le0/3s4Dpz5AeFg4P377x6Tdm8YpC05h/lfz2Vuz1+VqjTG9iV3F5DLVehYv7kdy8mxGjfpHk2W5e3JZsGIBC1YuIK8kj5jwGM4ZeQ4XZ13MyYNPxhvWbccvNMZ0EYe7iskCogvIybmA0tKPmDJlGyIHD3Glqvxn639YsGIBC3MWUlJVwlFxRzEnYw4XZV3EuKPGBd3OGGNaYgHRxe3Y8Ri5uZczceIqYmPHHHbd6rpqXl//OvNXzOfVda9SW1/L6NTRXJR5EXOz5nJs4rFHqGpjTE9g90F0cfv7IYqKWh7dNdIbyTmjzuGF819g50928tDpD5Ecncyti25l4H0Dmf7kdB774jFKqkpCXbYxpoezFkQX8dlnw4mOHkZW1mvt2n5T8SaeWfkM81fMZ13hOiI9kZw54kwuyrqIU4eeSoQnopMrNsb0BHaKqRtYt+5qdu58iilTthIe7mv3flSVpduXMn/FfJ5d9SwFFQWkRKdw/pjzuXjsxUzuP9n6K4wxDSwguoHS0iUsX/5NoqOHk5X1BlFRA1reqAW1/lre3vg2C1Yu4KW1L1FVV8XQ5KEN/RVDk4d2QuXGmO7MAqKbKC5exKpV5+DxxJOV9SZxcZ33CIyy6jJeWPMC81fM5/3N76MoU9KmcFHWRZw/5nxSYlI67VjGmO7DAqIb2bv3K1asmI3fX0Fm5sskJX2z04+RX5bf0F+xapVAtikAABZySURBVPcqvGFeTht2GhdnXcwZw88gynvQw/t6pbr6OjYWbSQhMoGj4o6yU3OmR7KA6GaqqrawYsVsKis3MmrUfPr2/V7IjvXVzq9YsGIBz6x6hu3l20mMTOS0YaeR2TeTUamjGNVnFEOSh/Tom/JUlR17d7Bi1wpW7lrJyt2B15qCNVT7qwHwRfnI6JtBRt8MxqSOCfzsO4Y+MX1crt6YjrGA6IZqa4tYufJMysr+w5Ah9zJgwA0hPZ6/3s/7ee8zf8V8Fm1eRH5ZfsOy8LBwhqUMY3TqaEb1CYTGqNRRjEgZQXR4dEjr6mzl1eWs2r0qEAKNwqCosqhhnWPijyGzbyZZ/bIYkzqGsuoycgpyWLV7FTkFOU0uIe4X248xfceQkZrREBpjUseQGJXoxsczps0sILopv7+SNWsuYs+eF0hL+zFDhvwekSNz60p5dTlr96xldcFq1uxZE3gVrGFj8UbqtR4AQUhPSj8QHKkHwiMpKumI1HkodfV1rCtc1yQEVu5ayeaSzQ3rxEXEkdE3g6y+WWT2yySzbyaZ/TJJjk4+5H5Vle3l2xsCY39o5OzOYV/tvob10hLSAi2O1EBoZPTNYFSfUcRGxIb0cxvTVhYQ3Ziqnw0bbmDbtr/Qt+8FjBz5JGFh7j15rrqumvVF6wPBUbCmITxy9+Q2nI4BOCruKEb1GXVQeHT2ufz9v7CbtwhWF6ymxl8DgEc8DE8ZTla/rIYQyOybycCkgYR1UuDWaz1fl34dCIzdOawqCIRH49NUgjDIN6jpaarUMYzsM9KeJmhcYwHRzakqW7f+gU2bbiYp6SQyMl7E6+1apzD89X7ySvIOanGs2bOGsuqyhvUSIxMZlTqK0X1GN2lxpCelt/jLuq2nh/aHwcg+I13rePfX+9lYvDEQGk5rY9XuVeQW5lJXXwcEAmxo8tCD+jiGJg8l3BPe6TWpKnX1dVT7q6mqq6K6rrrJdFVdFdX+6qDTNf4aFG3YT8M+0Sb7b+/81m4T5Y1iaPJQhqcMJz0pvUf3kTVW469hQ9GGJn+crS5YzYnHnsj9s+9v1z4tIHqInTsXkJv7fWJiRpGV9QaRkf3dLqlF+zuA1xSsOSg8du3b1bBelDeKESkjmrQ4/PX+Tj091JXU+GtYX7i+SWjkFOSwoWhDwym88LBwRvYZ2XB6yhvmDf5Lvdkv82C/9JsHQONftt1deFg4Q5KHMDxlOMOThzM8ZTgj+oxgeMpw+sX265ZXn+2r2UduYe5BLfUNRRsa/rAAGJg4kFGpozhj2BlcPenqdh3LAqIHKSp6h5yc7+D1+sjKeqPFwf26sqLKogP9HI3+EeSV5DWss//0UGa/zCZh0Jmnh7qSytpK1u5Z2yQ0Vu1e1eQ7ifBEEOWNItITSaQ3ssXpNq/rjTzkdIQnosn3Lhz45dv4F3FH5rdmm701e1lfuJ51hesCr6J15O7JZUPRhianOuMj4g8EhhMe+1/xkfGt+48SQkWVRQf+3y9Yw+o9gX8LW0q3NKyzv4XZ/HTtyD4jO6VPywKihykv/5KVK0+jvr6KjIxXSEo6we2SOlVFbQW5e3IREVdPD3UlVXVVAAf9gjZN+ev9bC3b2hAcuXtyWVcUmN5SsqVJy+nouKObBMaIlECrY5BvUKeOXba/Fd38D6FgreiRfUY2XCk4OjVwGnZo8tCQjqVmAdEDVVbmsWLFqVRV5TFq1AL69j3P7ZKM6dKq6qrYWLSR3MLcAy2PwnXkFuayp2JPw3oe8TDIN6hJaOx/9Y/vf8hTVvv74Zq3BoL1wwW78m9g4kA8YZ6Qfw/NWUD0ULW1hc69EksYOvR+0tKudbskY7qlosqiJqesGodIZV1lw3ox4TEHAiN5OOGe8IZAyC3MbWjpwYEr+Rq3BkJxJV9HWUD0YIF7JS5kz56XGDDgpwwefNcRu1fCmJ6uXuvZXr79wOkqp79jXeE6Nhdvpl7rGZg08KCbSEf1GYUvuv2jMh9JhwuI3nFtWA/m8UQzZsxzrF9/LVu3/oHq6u2MHPk4YWH2/AdjOipMwkhLSCMtIY0Zg2Y0WVbjr8Ff7+92owm0hQVEDyDiYdiwB4mMTGPz5tuoqdlJRsYLeL0JbpdmTI8V4YmAI99lcESF9FyEiJwqIrkiskFEbgmyPFJEFjrLPxORdGf+LBFZJiIrnZ8zmm9rmhIRBg68lZEjn6S09EO+/PJEqqu3u12WMaYbC1lAiIgHeBCYDYwG5ojI6Gar/QAoVtWhwL3A3c78PcC3VTUTmAfMD1WdPc1RR80jM/NVKis38MUXU9i3b43bJRljuqlQtiAmARtUdZOq1gDPAmc1W+cs4Cln+jngZBERVf1SVff/+ZsDRIuIDVbTSsnJpzBu3IfU11fz5ZfTKC39j9slGWO6oVAGRH9ga6P3+c68oOuoah1QCjR/tNm5wBeqWt1sPiJyhYgsFZGlBQUFnVZ4TxAfP55vfGMx4eF9+OqrmRQUvOh2ScaYbqZLXw8pImMInHa6MthyVX1EVSeo6oTU1NQjW1w3EB09mHHjFhMbO5acnHPZtu1Bt0syxnQjoQyIbcCARu/TnHlB1xERL5AIFDrv04AXgUtUdWMI6+zRIiL6kJ29iJSUb7N+/TVs2vTzg0bPNMaYYEIZEJ8Dw0RkkIhEABcArzRb5xUCndAA5wGLVFVFJAl4DbhFVe0Eegd5PDGMGfM8Rx99BV9/fRdr186jvr7G7bKMMV1cyALC6VO4BngLWAP8U1VzROROETnTWe0xIEVENgA3Afsvhb0GGAr8SkSWO6++oaq1NwgL8zJ8+EOkp/8vu3bNZ+XKM6irK3e7LGNMF2ZDbfRCO3Y8QW7uD4mLyyIz83UiI49yuyRjjEsON9RGl+6kNqFx9NHfJzPz31RU5PLll1OoqMh1uyRjTBdkAdFLpaTMJjv7Q/z+Cr74YiqlpUvcLskY08VYQPRiCQkTnHslkvnqqxns2fOy2yUZY7oQC4heLjp6iHOvRBarVn2HbdsecrskY0wXYQFhiIhIJTt7EcnJs1m//kds2HATNTV2Z7oxvZ0FhAHA44klI+Mljj76SvLz72XJkgGsXft9ysu/cLs0Y4xLLCBMg7AwLyNGPMTEias5+ugfsHv3v1i2bDxffDGN3bsXUl9f63aJxpgjyALCHCQ2dhTDhz/IlCn5DBlyLzU1u1i9+gI+/TSdvLz/paZml9slGmOOAAsIc0jh4UkMGHADkyevIzPzVWJjM8nL+xVLlhzLmjWXUFb2udslGmNCyB45alokEkZKyumkpJxORUUu27b9hZ07n2TXrvkkJBxH//7Xkpp6nj0H25gexloQpk1iYkYwbNifmTJlG0OH3k9tbSFr1szl008Hkpf3a6qrd7pdojGmk1hAmHbxehNIS7uOSZPWkpn5BnFx48jLu4NPPz2W1asvoqzsM7dLNMZ0kJ1iMh0SOP10Kikpp1JRsY5t2x5k584n2L37aeLjJ9K//7X07fs9wsLsibHGdDfWgjCdJiZmOMOG3c+UKdsYNuwv1NWVsXbtJSxZciybN/+K6urtLe/EGNNlWECYTuf1xtO//9VMmrSarKy3SEiYxJYtv+HTTweyevUcSksX21PtjOkG7BSTCRmRMJKTv0Vy8reorNzItm0PsmPH4+ze/SxxceNJS7uW1NTz8Xii3C7VGBOEtSDMEREdPYShQ//ElCn5DBv2V+rrK1i79lI+/XQAmzb9gqqqfLdLNMY0YwFhjiivN47+/X/ExIk5jB37LgkJU/n669/x6afp5OR8j5KST+z0kzFdhJ1iMq4QEXy+k/H5TqaycjPbt/+VHTv+TkHBv4iLy3aufpqDxxPtdqnG9Fr2TGrTZfj9+9i162m2bfsz+/atIiwsmqSkk0hOnk1Kymyio4e4XaIxPc7hnkltAWG6HFWlpORD9ux5nsLCN6iq2ghAdPQwJyxOIzHxm9a5bUwnsIAw3VpFxXqKit6gqOgNSko+oL6+yloXxnQSCwjTY/j9lZSUfNAQGJWVGwCIjh5KcvJskpNnk5Q03foujGklCwjTY1VUbGjUunjfaV1ENbQukpNnExMz1O0yjemyLCBMrxBoXXzYqHWxHrDWhTGHYwFheqXKyo0UFjZuXVQ6rYvpjVoXw9wu0xhXWUCYXs/vr6S09KOGwKisXAdAVNQQUlIaty5iXK7UmCPLAsKYZiorN1FU9AaFhW9QUrKI+vpKRCJJSpreEBjR0cMQEbdLNSakXAsIETkVuB/wAH9X1buaLY8E/gGMBwqB81U1T0RSgOeAicCTqnpNS8eygDDt5fdXUVr6UUNgVFbmAhAVNZjk5NkkJk4lNjaDmJiR9lhV0+O4EhAi4gHWAbOAfOBzYI6qrm60zv8AWap6lYhcAJyjqueLSCwwDsgAMiwgzJFUWbm5oaO7uHgR9fUVAIh4iY4eTmxsRpNXdPRgAv+7G9P9HC4gQjkW0yRgg6pucop4FjgLWN1onbOAO5zp54C/iIio6j7gExGx6xPNERcdPYj+/f+H/v3/h/r6Gioq1rFv36qGV3n5MgoK/gUE/rgKC4smJmb0QcERGdnfTlGZbi2UAdEf2NrofT4w+VDrqGqdiJQCKcCe1hxARK4ArgA49thjO1qvMQcJC4sgLi6DuLiMJvP9/n3s27e6SXAUF7/Nrl1PNazj9SYdFBqxsRmEh6cc6Y9hTLt069FcVfUR4BEInGJyuRzTi3g8sSQkTCQhYWKT+bW1hezbl9MkOHbvfpa6upKGdSIijiI2NrNJaMTEjMbrjTvSH8OYwwplQGwDBjR6n+bMC7ZOvoh4gUQCndXGdEvh4SkkJZ1IUtKJDfNUlZqa7U1CY9++VWzf/hD19ZUN60VFDQoSHCOsY9y4JpQB8TkwTEQGEQiCC4ALm63zCjAPWAKcByzSnnLdrTEOESEysj+Rkf1JTj6lYb6qn8rKzc2CYyWFha8BfmfbAx3j0dHDiI4eTFTUYKKjhxAZeYx1jpuQCllAOH0K1wBvEbjM9XFVzRGRO4GlqvoK8BgwX0Q2AEUEQgQAEckDEoAIETkb+FbjK6CM6e5EPMTEDCUmZiipqWc3zK+vr27WMb6S8vKlFBQ8z/7gCGwfQVRUeqPQaPrT64134VOZnsRulDOmm6ivr6W6eiuVlZuoqtpEZeVG52dg2u8vbbJ+eHhq0OCw1odpzK3LXI0xnSgsLJzo6MAv+mBqa4sbAqNxcJSVfcru3f/EWh+mrSwgjOkhwsN9hIePJz5+/EHLDm59HGiFlJYuaUPrYxDh4ak2ZlUvYQFhTC/QntZHVdUmyso+O6j1EdhfFF5vCuHhKYSH93F+phx2ntebiEjYEfi0prNYQBhj2tD6yKO2dg91dYXU1h547d27wplXBNQf4igewsOTWwySpvOSCQsLD+lnN4dmAWGMOayWWh+NqdZTV1fqBMfBQdJ4XlVVHuXlS6mtLUS1+pD79HgSDhEkzd8nN0x7PHE2zEknsIAwxnQakTCnNeIDWjeUmqpSX1/RJERqawubhcuehp8VFbnU1hbi95cdpo5wvN7kIEHSvAWT3CRw7KbEpiwgjDGuEhE8nlg8nliiolo/plp9fS11dUVOcBQ1CZQD8wOvysoN1Nb+txWtlbgmwRI8VFIarZOM15vUYy8ZtoAwxnRLYWHhRET0IyKiX6u3adpaKTroFFjjYKmrK6Sq6mtnuphD962Ax5NIeLgPr/fAq/H7QJAEW5bYpcPFAsIY02u0t7US6FspCRIiRdTWFlNXd+BVW1tERcVqZ7r4sC0WEKePJTlosBx4nxxkWeivCrOAMMaYFgT6VpIJD09u87Z+f2Wj8Cimrq6o0XRxs2XF7Nu3rWFateZwVeH1JuL1+ujT5xyGDv1j+z/gIVhAGGNMCHk80Xg80URGHtOm7QKnwyoPCpPAqbGmwRIZOaDlHbaDBYQxxnRBgdNhMXg8MURG9nelBrut0RhjTFAWEMYYY4KygDDGGBOUBYQxxpigLCCMMcYEZQFhjDEmKAsIY4wxQVlAGGOMCUpU1e0aOoWIFABb3K6jg/oAe9wuogux76Mp+z4OsO+iqY58HwNVNTXYgh4TED2BiCxV1Qlu19FV2PfRlH0fB9h30VSovg87xWSMMSYoCwhjjDFBWUB0LY+4XUAXY99HU/Z9HGDfRVMh+T6sD8IYY0xQ1oIwxhgTlAWEMcaYoCwgugARGSAi74vIahHJEZHr3a7JbSLiEZEvReRVt2txm4gkichzIrJWRNaIyBS3a3KTiNzo/DtZJSL/JyJRbtd0JInI4yKyW0RWNZqXLCLviMh656evM45lAdE11AE/VtXRwHHA1SIy2uWa3HY9sMbtIrqI+4E3VXUkMJZe/L2ISH/gOmCCqmYAHuACd6s64p4ETm027xbgPVUdBrznvO8wC4guQFV3qOoXznQ5gV8A7jxjsAsQkTTgdODvbtfiNhFJBE4EHgNQ1RpVLXG3Ktd5gWgR8QIxwHaX6zmiVPUjoKjZ7LOAp5zpp4CzO+NYFhBdjIikA+OAz9ytxFX3AT8D6t0upAsYBBQATzin3P4uIrFuF+UWVd0G3AN8DewASlX1bXer6hL6qeoOZ3on0K8zdmoB0YWISBzwPHCDqpa5XY8bROQMYLeqLnO7li7CC3wD+JuqjgP20UmnD7oj59z6WQSC8xggVkQucreqrkUD9y50yv0LFhBdhIiEEwiHp1X1BbfrcdE04EwRyQOeBWaIyAJ3S3JVPpCvqvtblM8RCIzeaiawWVULVLUWeAGY6nJNXcEuETkawPm5uzN2agHRBYiIEDjHvEZV/+R2PW5S1Z+rapqqphPofFykqr32L0RV3QlsFZERzqyTgdUuluS2r4HjRCTG+XdzMr24076RV4B5zvQ84OXO2KkFRNcwDbiYwF/Ly53XaW4XZbqMa4GnRWQFkA38zuV6XOO0pJ4DvgBWEvgd1quG3RCR/wOWACNEJF9EfgDcBcwSkfUEWll3dcqxbKgNY4wxwVgLwhhjTFAWEMYYY4KygDDGGBOUBYQxxpigLCCMMcYEZQFhTAtExN/o8uPlItJpdzKLSHrjUTmN6Uq8bhdgTDdQqarZbhdhzJFmLQhj2klE8kTk9yKyUkT+KyJDnfnpIrJIRFaIyHsicqwzv5+IvCgiXzmv/UNEeETkUecZB2+LSLSz/nXOM0JWiMizLn1M04tZQBjTsuhmp5jOb7SsVFUzgb8QGIUW4M/AU6qaBTwNPODMfwD4UFXHEhhPKceZPwx4UFXHACXAuc78W4Bxzn6uCtWHM+ZQ7E5qY1ogIntVNS7I/DxghqpucgZb3KmqKSKyBzhaVWud+TtUtY+IFABpqlrdaB/pwDvOg14QkZuBcFX9jYi8CewFXgJeUtW9If6oxjRhLQhjOkYPMd0W1Y2m/RzoGzwdeJBAa+Nz5wE5xhwxFhDGdMz5jX4ucaYXc+AxmHOBj53p94AfQcMztxMPtVMRCQMGqOr7wM1AInBQK8aYULK/SIxpWbSILG/0/k1V3X+pq88ZZbUamOPMu5bAE+B+SuBpcN935l8PPOKMvuknEBY7CM4DLHBCRIAH7FGj5kizPghj2snpg5igqnvcrsWYULBTTMYYY4KyFoQxxpigrAVhjDEmKAsIY4wxQVlAGGOMCcoCwhhjTFAWEMYYY4L6/9iBeJt/QX3WAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8mDekMlz2TZQ"
      },
      "source": [
        "**Mini Batch Gradient Descent**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hRiZz64w2V39",
        "outputId": "e0bac651-4648-4f72-8bc7-4c1faf1af51d"
      },
      "source": [
        "model.compile(loss = contrastive_loss, optimizer=\"SGD\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_78\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_119 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_120 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_77 (Functional)           (None, 256)          91584       input_119[0][0]                  \n",
            "                                                                 input_120[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "lambda_113 (Lambda)             (None, 1)            0           model_77[0][0]                   \n",
            "                                                                 model_77[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W9kgVK6K2Vrl",
        "outputId": "d13e6d78-ce97-4bb5-d89d-652b7b44de3e"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 7ms/step - loss: 0.2147 - accuracy: 0.6786 - val_loss: 0.1529 - val_accuracy: 0.8081\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1472 - accuracy: 0.8129 - val_loss: 0.1327 - val_accuracy: 0.8345\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1298 - accuracy: 0.8383 - val_loss: 0.1222 - val_accuracy: 0.8502\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1207 - accuracy: 0.8514 - val_loss: 0.1151 - val_accuracy: 0.8589\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1140 - accuracy: 0.8613 - val_loss: 0.1099 - val_accuracy: 0.8660\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1080 - accuracy: 0.8696 - val_loss: 0.1056 - val_accuracy: 0.8718\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1041 - accuracy: 0.8751 - val_loss: 0.1019 - val_accuracy: 0.8765\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.1007 - accuracy: 0.8814 - val_loss: 0.0987 - val_accuracy: 0.8799\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0973 - accuracy: 0.8855 - val_loss: 0.0959 - val_accuracy: 0.8836\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0941 - accuracy: 0.8900 - val_loss: 0.0933 - val_accuracy: 0.8865\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e0GK6QLT2Vd5"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ldGp646Z2VPd",
        "outputId": "89fe5807-693b-480f-e6d9-985b91de2395"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 88.615\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "j3Ve_SkG2VFc",
        "outputId": "1bfb0af8-777a-4aa8-a3be-608dc5cc5f9e"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUVd7H8c8vmfReB0iA0HtIIPSHImWxoCKCooBi76Cuq+66hfVZXfdZd3V1XcWCujZQBEWxgaCoIBg6BJAWJBDSKwmp5/ljhhgwkDaTSfm9X6+8nLlz77m/O7vkm3PPveeKMQallFLqbG6uLkAppVTzpAGhlFKqRhoQSimlaqQBoZRSqkYaEEoppWqkAaGUUqpGGhCqRRGRT0Xkekev60oikiwiE11dBzSvWpTrWVxdgGr9RKSw2ltfoASosL+/zRjzVl3bMsZc5Ix1myMR+RQYbX/rBRig1P7+TWPM7fVsbwHQ3Rgz22FFqlZNA0I5nTHG//RrEUkGbjbGrD57PRGxGGPKm7K25qx6wInIa0CKMeb3rqtItTV6ikm5jIiME5EUEXlIRE4Ar4pIiIh8LCIZIpJjfx1dbZuvRORm++u5IvKtiDxpX/ewiFzUwHW7iMg6ESkQkdUi8pyIvHmOuutS4/+KyHf29r4QkfBqn88RkSMikiUijzTwu5siIttEJFdE1otIbLXPHhKRY/Z97xORCSJyIfA74GoRKRSR7XXYh5eIPC0ix+0/T4uIl/2zcPtx54pItoh8IyJu59p/Q45RuZ4GhHK1dkAo0Bm4Fdv/J1+1v+8EFAP/Ps/2w4B9QDjwf8ArIiINWPdtYBMQBiwA5pxnn3Wp8VrgBiAS8AQeABCRvsDz9vY72PcXTT2ISDywCLjNvv1CYIX9F3ov4G5giDEmAJgMJBtjPgMeB5YYY/yNMQPrsKtHgOFAHDAQGAqc7sH8GkgBIgArtvAx59p/fY5PNR8aEMrVKoE/GWNKjDHFxpgsY8z7xpgiY0wB8Bgw9jzbHzHGvGSMqQBeB9pj+4VV53VFpBMwBPijMabUGPMtsOJcO6xjja8aY340xhQD72L7JQswHfjYGLPOGFMC/MH+HdTHrcBCY8xGY0yFMeZ1bOM6w7GN7XgBfUXEwxiTbIw5WM/2T5sFPGqMSTfGZAB/5ufgLMP2/XU2xpQZY74xtondHLl/5WIaEMrVMowxp06/ERFfEVloPwWTD6wDgkXE/Rzbnzj9whhTZH/pX891OwDZ1ZYBHD1XwXWs8US110XVaupQvW1jzEkg61z7OofOwK/tp3dyRSQX6Ah0MMYcAO7F1gtKF5HFItKhnu2f1gE4Uu39EfsygL8DB4AvROSQiDxsPx5H7l+5mAaEcrWzpxP+NdALGGaMCQTG2Jef67SRI6QCoSLiW21Zx/Os35gaU6u3bd9nWP3K5SjwmDEmuNqPrzHmHQBjzNvGmP/BFiQG+Jt9u/pO3Xzc3sZpnezLMMYUGGN+bYzpClwG3H96rOE8+1ctjAaEam4CsJ3TzxWRUOBPzt6hMeYIkAgsEBFPERkBXOqkGpcCU0Tkf0TEE3iU+v87fAm4XUSGiY2fiFwiIgEi0ktExtsHk0/Z6zx9CisNiDk9mFwH7wC/F5EI+yD7H4E3oWqQvLt9DCcP26mlylr2r1oYDQjV3DwN+ACZwPfAZ02031nACGyne/4CLMF2Xr8mDa7RGLMbuAvboHgqkINtsLfOjDGJwC3YBsZzsJ3qmWv/2At4wl7bCWyD5L+1f/ae/b9ZIrKlDrv6C7bg3AHsBLbYlwH0AFYDhcAG4D/GmLW17F+1MKIPDFLql0RkCbDXGOP0HoxSzZX2IJQCRGSIiHQTETf7PQOXAx+4ui6lXEnvpFbKph2wDNuAcQpwhzFmq2tLUsq19BSTUkqpGukpJqWUUjVqNaeYwsPDTUxMjKvLUEqpFmXz5s2ZxpiImj5rNQERExNDYmKiq8tQSqkWRUSOnOszPcWklFKqRhoQSimlaqQBoZRSqkatZgxCKdV8lZWVkZKSwqlTp2pfWTmFt7c30dHReHh41HkbDQillNOlpKQQEBBATEwM536ek3IWYwxZWVmkpKTQpUuXOm+np5iUUk536tQpwsLCNBxcREQICwurdw9OA0Ip1SQ0HFyrId9/mw+IsrJskpP/TGHhTleXopRSzUqbDwgQjhx5nBMnXnV1IUopJ8nKyiIuLo64uDjatWtHVFRU1fvS0tLzbpuYmMi8efNq3cfIkSMdUutXX33FlClTHNJWY7X5QWoPjxDCwi4hPf0dunX7O+d+9LFSqqUKCwtj27ZtACxYsAB/f38eeOCBqs/Ly8uxWGr+dZiQkEBCQkKt+1i/fr1jim1GtAcBWK2zKS09QU7OGleXopRqInPnzuX2229n2LBhPPjgg2zatIkRI0YQHx/PyJEj2bdvH3DmX/QLFizgxhtvZNy4cXTt2pVnnnmmqj1/f/+q9ceNG8f06dPp3bs3s2bN4vSs2Z988gm9e/dm8ODBzJs3r9aeQnZ2NlOnTiU2Npbhw4ezY8cOAL7++uuqHlB8fDwFBQWkpqYyZswY4uLi6N+/P998802jv6M234MACA29GHf3INLS3iQ0dJKry1GqVdu//14KC7c5tE1//zh69Hi63tulpKSwfv163N3dyc/P55tvvsFisbB69Wp+97vf8f777/9im71797J27VoKCgro1asXd9xxxy/uLdi6dSu7d++mQ4cOjBo1iu+++46EhARuu+021q1bR5cuXbjmmmtqre9Pf/oT8fHxfPDBB6xZs4brrruObdu28eSTT/Lcc88xatQoCgsL8fb25sUXX2Ty5Mk88sgjVFRUUFRUVO/v42waEIC7uzeRkTNIT19MRcXzuLv7urokpVQTmDFjBu7uttPKeXl5XH/99ezfvx8RoaysrMZtLrnkEry8vPDy8iIyMpK0tDSio6PPWGfo0KFVy+Li4khOTsbf35+uXbtW3YdwzTXX8OKLL563vm+//bYqpMaPH09WVhb5+fmMGjWK+++/n1mzZjFt2jSio6MZMmQIN954I2VlZUydOpW4uLhGfTegAVHFap1NaurLZGauwGqd6epylGq1GvKXvrP4+flVvf7DH/7ABRdcwPLly0lOTmbcuHE1buPl5VX12t3dnfLy8gat0xgPP/wwl1xyCZ988gmjRo3i888/Z8yYMaxbt46VK1cyd+5c7r//fq677rpG7UfHIOyCgkbj5dWRtLQ3XV2KUsoF8vLyiIqKAuC1115zePu9evXi0KFDJCcnA7BkyZJatxk9ejRvvfUWYBvbCA8PJzAwkIMHDzJgwAAeeughhgwZwt69ezly5AhWq5VbbrmFm2++mS1btjS6Zg0IOxE3IiOvJSfnc0pLM1xdjlKqiT344IP89re/JT4+3uF/8QP4+Pjwn//8hwsvvJDBgwcTEBBAUFDQebdZsGABmzdvJjY2locffpjXX38dgKeffpr+/fsTGxuLh4cHF110EV999RUDBw4kPj6eJUuWMH/+/EbX3GqeSZ2QkGAa+8CgwsKdJCbG0qPHv4mKustBlSml9uzZQ58+fVxdhssVFhbi7++PMYa77rqLHj16cN999zXZ/mv630FENhtjaryOV3sQ1fj7D8DPL1ZPMymlnOKll14iLi6Ofv36kZeXx2233ebqks5LB6nPYrXO5tChBykuPoiPTzdXl6OUakXuu+++Ju0xNJZTexAicqGI7BORAyLycA2fjxGRLSJSLiLTz/rs/0Rkt4jsEZFnpIlm+oqMvAYQ0tLeaordKaVUs+W0gBDbnBXPARcBfYFrRKTvWav9BMwF3j5r25HAKCAW6A8MAcY6q9bqvL2jCQ4eR1raW7SW8RmllGoIZ/YghgIHjDGHjDGlwGLg8uorGGOSjTE7gMqztjWAN+AJeAEeQJoTaz2D1TqL4uIfKSho3KC3Ukq1ZM4MiCjgaLX3KfZltTLGbADWAqn2n8+NMXscXuE5hIdfiYiXDlYrpdq0ZnkVk4h0B/oA0dhCZbyIjK5hvVtFJFFEEjMyHHfvgodHMOHhl5KevpjKSsdfD62UalqNme4bbDepnWu21tdee427777b0SU3C84MiGNAx2rvo+3L6uIK4HtjTKExphD4FBhx9krGmBeNMQnGmISIiIhGF1xdZOQsysrSyclZ7dB2lVJN7/R039u2beP222/nvvvuq3rv6elZ6/bnC4jWzJkB8QPQQ0S6iIgnMBNYUcdtfwLGiohFRDywDVA32SkmgLCwi7BYQvQ0k1Kt1ObNmxk7diyDBw9m8uTJpKamAvDMM8/Qt29fYmNjmTlzJsnJybzwwgs89dRTxMXFnXca7eTkZMaPH09sbCwTJkzgp59+AuC9996jf//+DBw4kDFjxgCwe/duhg4dSlxcHLGxsezfv9/5B11PTrsPwhhTLiJ3A58D7sAiY8xuEXkUSDTGrBCRIcByIAS4VET+bIzpBywFxgM7sQ1Yf2aM+chZtdbEzc2LiIirSEt7g/LyQiwW/6bcvVKt1r2f3cu2E46d7juuXRxPX1j3SQCNMdxzzz18+OGHREREsGTJEh555BEWLVrEE088weHDh/Hy8iI3N5fg4GBuv/32XzxkqCb33HMP119/Pddffz2LFi1i3rx5fPDBBzz66KN8/vnnREVFkZubC8ALL7zA/PnzmTVrFqWlpVRUVDTqO3AGp94oZ4z5BPjkrGV/rPb6B2ynns7ergJw+S2GVussUlMXkpX1IVbrLFeXo5RykJKSEnbt2sWkSbbnv1RUVNC+fXsAYmNjmTVrFlOnTmXq1Kn1anfDhg0sW7YMgDlz5vDggw8CMGrUKObOnctVV13FtGnTABgxYgSPPfYYKSkpTJs2jR49ejjq8BxG76Q+j6CgUXh5dSYt7S0NCKUcpD5/6TuLMYZ+/fqxYcOGX3y2cuVK1q1bx0cffcRjjz3Gzp07G72/F154gY0bN7Jy5UoGDx7M5s2bufbaaxk2bBgrV67k4osvZuHChYwfP77R+3KkZnkVU3Mh4obVei3Z2V9QWtpkt2EopZzMy8uLjIyMqoAoKytj9+7dVFZWcvToUS644AL+9re/kZeXR2FhIQEBARQUFNTa7siRI1m8eDEAb731FqNH2y6+PHjwIMOGDePRRx8lIiKCo0ePcujQIbp27cq8efO4/PLLqx4n2pxoQNTCap0NVJCeXvvc7UqplsHNzY2lS5fy0EMPMXDgQOLi4li/fj0VFRXMnj2bAQMGEB8fz7x58wgODubSSy9l+fLltQ5SP/vss7z66qvExsbyxhtv8K9//QuA3/zmNwwYMID+/fszcuRIBg4cyLvvvkv//v2Ji4tj165djX64jzPodN91kJg4CBEPBg/e6JT2lWrtdLrv5kGn+3YCq3UWBQWbKCr60dWlKKVUk9GAqAOd4VUp1RZpQNSBl1cHgoPH6wyvSjWC/ttxrYZ8/xoQdWS1zubUqYPk5+s4hFL15e3tTVZWloaEixhjyMrKwtvbu17b6X0QdRQRMY39++8gPf0tgoKGu7ocpVqU6OhoUlJScOSkmqp+vL29iY7+xX3J56UBUUcWSyBhYZeRnr6Ybt3+iZubh6tLUqrF8PDwoEuXLq4uQ9WTnmKqB6t1NmVlmeTkfOHqUpRSyuk0IOohNHQyFkuoXs2klGoTNCDqwc3Nk8jIq8nM/IDy8tpvu1dKqZZMA6KerNbZVFYWk5m53NWlKKWUU2lA1FNg4Ai8vbvoaSalVKunAVFPIoLVOoucnNWUlJxwdTlKKeU0GhANEBk5C6gkPX2xq0tRSimn0YBoAD+/3vj7D9bnVSulWjUNiAayWmdTWLiZkyf3uroUpZRyCg2IBoqMnAm4kZ6ug9VKqdZJA6KBvLzaERIykbS0N3UCMqVUq6QB0Qi2GV6Tyc9f7+pSlFLK4TQgGiE8fCpubj56T4RSqlXSgGgEiyWA8PCppKcvobKy1NXlKKWUQ2lANJLVOovy8myysz93dSlKKeVQGhCNFBLyKzw8wvWeCKVUq6MB0Uhubh5ERs4kK2sF5eX5ri5HKaUcRgPCASIjZ1FZeYqMjGWuLkUppRxGA8IBAgOH4e3dTU8zKaVaFQ0IB7DN8Dqb3Nw1lJQcd3U5SinlEBoQDmK1zgIM6envuLoUpZRyCA0IB/H17UFAwFA9zaSUajU0IBzIap1FYeE2Tp7c7epSlFKq0TQgHCgy8mrAXafeUEq1Ck4NCBG5UET2icgBEXm4hs/HiMgWESkXkelnfdZJRL4QkT0ikiQiMc6s1RE8Pa2Ehv6KtLS3MKbS1eUopVSjOC0gRMQdeA64COgLXCMifc9a7SdgLvB2DU38F/i7MaYPMBRId1atjmS1zqKk5Cfy8r5zdSlKKdUozuxBDAUOGGMOGWNKgcXA5dVXMMYkG2N2AGf8uW0PEosxZpV9vUJjTJETa3UY2wyvfjpYrZRq8ZwZEFHA0WrvU+zL6qInkCsiy0Rkq4j83d4jOYOI3CoiiSKSmJGR4YCSG8/d3Y/w8KlkZLxHZWWJq8tRSqkGa66D1BZgNPAAMAToiu1U1BmMMS8aYxKMMQkRERFNW+F5WK2zKS/PISvrU1eXopRSDebMgDgGdKz2Ptq+rC5SgG3201PlwAfAIAfX5zQhIRPx8IjU00xKqRbNmQHxA9BDRLqIiCcwE1hRj22DReR0t2A8kOSEGp3Czc1in+H1Y8rKcl1djlJKNYjTAsL+l//dwOfAHuBdY8xuEXlURC4DEJEhIpICzAAWishu+7YV2E4vfSkiOwEBXnJWrc5gtc7GmBIyM993dSlKKdUgYoxxdQ0OkZCQYBITE11dRhVjDJs29cbLqwNxcWtdXY5SStVIRDYbYxJq+qy5DlK3eLYZXmeRm/s1p04drX0DpZRqZjQgnEhneFVKtWQaEE7k49ONwMDhOjeTUqpF0oBwMqt1NidP7qCwcIerS1FKqXrRgHCyiIirELFoL0Ip1eJoQDiZp2cEISGTSU9/W2d4VUq1KBoQTcBqnU1JSQq5uetcXYpSStWZBgRwqvyUU9sPD78Md3d/nXpDKdWitPmAOJRziL7P9eX9JOfd8ezu7kt4+DQyMpZSUeHcMFJKKUdp8wERFRCF1d/K3A/nkpThvOmerNbZVFTkkZ290mn7UEopR2rzAeFl8WLpjKX4efgxdfFU8k7lOWU/ISHj8fRsp1czKaVajDYfEABRgVG8N+M9DuceZs7yOVQ64WojEXciI68hK2slZWXZDm9fKaUcTQPCbnTn0fzzV//kox8/4n+//l+n7MM2w2spGRlLndK+Uko5kgZENXcPvZs5sXNY8PUCPv7xY4e37+8fj69vbz3NpJRqETQgqhERFk5ZSHy7eGYvm83+rP0Ob99qnU1e3jpOnTri0LaVUsrRNCDO4uPhw7Krl2FxszB1yVQKSwsd2n5k5LUApKW97dB2lVLK0TQgahATHMPi6YvZm7mXGz68AUc+VMnHpwuBgaNIS3vToe0qpZSjaUCcw8SuE/nrhL+yNGkpf1//d4e2bbXOpqgoicLC7Q5tVymlHEkD4jx+M/I3zOg7g99++VtWHVzlsHYjI2cgYiE9XQerlVLNlwbEeYgIiy5fRJ/wPsx8fybJuckOadfDI4zQ0ItJS3sbYyoc0qZSSjmaBkQt/D39WX71cioqK5i2ZBrFZcUOaddqnUVp6XFyc79ySHtKKeVoGhB10COsB29Oe5OtJ7Zy28e3OWRwOSzsUtzdA/SeCKVUs6UBUUdTek5hwdgFvLHjDf696d+Nbs/d3YeIiOn2GV4d0ytRSilH0oCohz+M/QOX9ryU+7+4n2+OfNPo9qzWWVRUFJCV9ZEDqlNKKcfSgKgHN3HjjSveoEtwF6a/N51j+cca1V5w8Dg8PTvoaSalVLOkAVFPQd5BLL96OSdLT3Llu1dSUl7S4LZE3LFaryU7+xPKyrIcWKVSSjWeBkQD9Ivsx2tTX2PjsY3M+3Reo9qKjJyFMeWkp7/noOqUUsoxNCAaaHrf6Tw06iFe3PIiL295ucHt+PsPxNe3nz6vWinV7NQpIETET0Tc7K97ishlIuLh3NKav8fGP8akrpO465O72JiysUFt2GZ4nUV+/ncUFx92cIVKKdVwde1BrAO8RSQK+AKYA7zmrKJaCnc3d9658h06BHTgynevJK0wrUHtWK22GV7T03WGV6VU81HXgBBjTBEwDfiPMWYG0M95ZbUcYb5hLLtqGVnFWVy19CrKKsrq3Ya3d2eCgsboDK9KqWalzgEhIiOAWcBK+zJ355TU8sS3j+elS19i3ZF1/GbVbxrUhtU6i6KivaSlveHg6pRSqmHqGhD3Ar8FlhtjdotIV2Ct88pqeWbHzmb+sPn8a+O/eHNH/QecrdY5BAePY+/euRw79rwTKlRKqfqpU0AYY742xlxmjPmbfbA60xhT6/WdInKhiOwTkQMi8nANn48RkS0iUi4i02v4PFBEUkSk8XNbNIG/T/o7YzuP5daPbmXbiW312tbd3YcBAz4hLGwK+/ffyZEjj+vpJqWUS9X1Kqa37b+s/YBdQJKInPdcioi4A88BFwF9gWtEpO9Zq/0EzAXONTr7v9gGyFsED3cPlkxfQqhPKFcsuYKsovrd/Obu7kO/fu9jtc7m8OFHOHToQQ0JpZTL1PUUU19jTD4wFfgU6ILtSqbzGQocMMYcMsaUAouBy6uvYIxJNsbsACrP3lhEBgNWbFdNtRhWfyvLrl7G8YLjXLvsWioq6/e8Bzc3D3r3fp0OHe7i6NEn2bfvFn1mhFLKJeoaEB72+x6mAiuMMWVAbX/aRgFHq71PsS+rlf001j+AB2pZ71YRSRSRxIyMjLo03SSGRg3luYuf44uDX/D7Nb+v9/YibvTo8SydO/+BEydeISlpJpWVDZ/SQymlGqKuAbEQSAb8gHUi0hnId1ZRwJ3AJ8aYlPOtZIx50RiTYIxJiIiIcGI59XfzoJu5ddCtPPHdE7yf9H69txcRunR5lG7d/klGxlJ27ryMioqTTqhUKaVqVtdB6meMMVHGmIuNzRHgglo2OwZ0rPY+2r6sLkYAd4tIMvAkcJ2IPFHHbZuNZy56huHRw5n74VySMpIa1EbHjvfRq9cr5OSsZvv2X1FWluPgKpVSqmZ1HaQOEpF/nj6dIyL/wNabOJ8fgB4i0kVEPIGZwIq67M8YM8sY08kYE4PtNNN/jTG/uAqqufOyeLF0xlL8PPyYungqeafyGtRO+/Y30q/fexQUJLJt2zhKSk44uFKllPqlup5iWgQUAFfZf/KBV8+3gTGmHLgb+BzYA7xrv4fiURG5DEBEhohICjADWCgiuxt2GM1XVGAU7814j8O5h5mzfA6V5hfj8XUSETGNAQM+prj4INu2jaa4ONmxhSql1FmkLpdRisg2Y0xcbctcKSEhwSQmJrq6jHN6duOzzPtsHn8e92f+OPaPDW4nL+97du68CDc3PwYO/AI/v7OvHFZKqboTkc3GmISaPqtrD6JYRP6nWoOjAH2Qcj3cPfRu5sTOYcFXC1j548raNziHoKDhxMWtAyrYunUM+fnNNxSVUi1bXQPiduA5EUm2Dxz/G7jNaVW1QiLCwikLiWsXx6xls9iftb/Bbfn7DyAu7hsslgC2b7+AnJyvHFeoUkrZ1fUqpu3GmIFALBBrjIkHxju1slbIx8OHZVcvw+Jm4YolV1BYWtjgtnx9uxMf/y1eXp3YseNCMjM/cmClSilVzyfKGWPy7XdUA9zvhHpavZjgGBZPX8yezD3ctOKmRk2l4eUVRXz8Ovz9Y9m16wpOnNCn0imlHKcxjxwVh1XRxkzsOpG/Tvgr7+5+lyfXP9motjw8whg48EuCg8ewd+8cUlJaxLyGSqkWoDEBobPINcJvRv6GGX1n8PCXD7Pq4KpGtWWxBNhngr2cAwfuITn5LzrJn1Kq0c4bECJSICL5NfwUAB2aqMZWSURYdPki+oT3Yeb7M0nOTW5Ue+7u3vTrtxSr9TqSk//AwYO/1pBQSjXKeQPCGBNgjAms4SfAGGNpqiJbK39Pf5ZfvZyKygqmLZlGcVnjrhx2c7PQu/erREXdQ0rKU+zbdxOVleUOqlYp1dY05hSTcoAeYT14c9qbbD2xlds+vq3Rf/WLuNG9+7/o3PlPnDjxKklJV+tMsEqpBtGAaAam9JzCgrELeGPHG0x5Zwop+eedxLZWtplgF9C9+9NkZi5j585LKS9v+CW1Sqm2SQOimfjj2D/y9OSnWXt4Lf3+04+Xt7zc6N5EdPR8evd+jZycL9mxYxJlZdkOqlYp1RZoQDQTIsL84fPZecdO4tvFc8tHtzD5zckcyT3SqHbbtbuefv3ep6BgC9u2jaWkJNVBFSulWjsNiGamW2g31ly/hucufo71R9fT//n+PP/D8w2eBRYgImIqsbGfUFx8mK1bR1NcfNiBFSulWisNiGbITdy4c8id7LpzFyOiR3DnJ3cy4b8TOJh9sMFthoRMIC7uS8rLs9m6dRQnT7a6mdWVUg6mAdGMxQTH8Pnsz3np0pfYkrqF2Bdi+df3/2pwbyIwcJh9JljsM8FucmS5SqlWRgOimRMRbh50M7vv3M24mHHc+/m9jHl1DPsy9zWoPX///sTHf4vFEsz27RPIyVnj4IqVUq2FBkQLER0YzcfXfMzrU19nd8Zu4hbG8ffv/k5FZUW92/Lx6Up8/Ld4e8ewY8fFZGZ+6ISKlVItnQZECyIiXDfwOpLuTGJyt8k8uPpBRi4aSVJGUr3b8vJqT1zc1/j7x7Fr15WcOPFfJ1SslGrJNCBaoPYB7Vl+9XLeufIdDmYfJH5hPI9/8zhlFWX1asfDI5SBA1cTHDyOvXuvJyXlGSdVrJRqiTQgWigRYWb/mSTdlcTlvS7nkTWPMOzlYWw/sb1e7Vgs/sTGriQ8/AoOHJhPcvKfdZI/pRSgAdHiRfpF8u6Md1k6YynHCo6R8FICC75aQGlFaZ3bcHPzom/fd2nXbi7JyQs4cGAeFRX6yHGl2joNiFbiyr5XknRnElf3u5o/f/1nEl5MYPPxzXXe3s3NQq9erxAdfR/Hjv2bTZt6cuLE65hG3KCnlEmwnUMAABo+SURBVGrZNCBakTDfMN6c9iYfzvyQzKJMhr08jN99+TtOlZ+q0/a2mWD/SVzcV3h6tmPv3rkkJg4iO7txDzRSSrVMGhCt0GW9LiPpriSuG3gdf/32rwxaOIiNKRvrvH1w8FgGDdpInz5vU1GRx44dv2L79gspLNzhxKqVUs2NBkQrFewdzKLLF/HprE8pKC1g5KKRPPDFA3V+KJGIG1brNQwdupdu3f5BQcEmEhPj2Lv3Rk6datx05EqplkEDopW7sPuF7L5zN7cMuoV/bPgHA18YyLc/fVvn7d3cvOjY8X6GDTtIdPT9pKW9xaZNPTl06BHKy/OdWLlSytU0INqAQK9AXpjyAqvnrKassowxr45h/qfzOVl6ss5teHiE0L37kwwdupfw8Kn89NPjbNzYnWPHnqOysn73XyilWgYNiDZkQtcJ7LxjJ3cNuYtnNj3DgOcHsPbw2nq14ePThb5932bQoB/w8+vH/v1388MP/cjIWKb3TyjVymhAtDH+nv48e/GzfD33a9zd3Bn/3/Hc8fEdFJQU1KudwMAEBg5cQ//+HyFiYffuK9m6dTR5eRucVLlSqqlpQLRRYzqPYfvt27l/+P0s3LyQ/s/354uDX9SrDREhPHwKCQk76NlzIcXFB9i6dSS7d8+gqOiAkypXSjUVDYg2zNfDl39M/gff3fgdvh6+TH5zMjd9eBO5p3Lr1Y6bm4UOHW5l2LADxMQsICvrU374oS/798+ntDTTSdUrpZxNA0IxouMItt62lYdHPcxr21+j97978/g3j5NVlFWvdiwWf2Ji/sSwYftp1+4Gjh37Nxs3duPIkSd06g6lWiBpLQOLCQkJJjEx0dVltHiJxxP53Ze/Y9WhVfhYfLhu4HXcO/xeeof3rndbJ08mcejQQ2RlfYyXV0e6dPkLVutsRPTvEqWaCxHZbIxJqOkzp/5LFZELRWSfiBwQkYdr+HyMiGwRkXIRmV5teZyIbBCR3SKyQ0Sudmad6mcJHRL4Ys4X7LxjJ9cOuJbXtr1Gn+f6cMnbl7D60Op6Xank59eXAQM+YuDAtXh4RLJ37/U6dYdSLYjTehAi4g78CEwCUoAfgGuMMUnV1okBAoEHgBXGmKX25T0BY4zZLyIdgM1AH2PMOU+Oaw/COdJPpvNC4gs898NzpJ9MZ0DkAO4dfi/XDrgWb4t3ndsxppL09CUcPvw7Tp1KJiRkMt26/R/+/rFOrF4pVRtX9SCGAgeMMYeMMaXAYuDy6isYY5KNMTuAyrOW/2iM2W9/fRxIByKcWKs6h0i/SP449o8cufcIiy5bBMBNK26i89Od+fNXfyb9ZHqd2jlz6o4nKSjYWDV1R0nJMWceglKqgZwZEFHA0WrvU+zL6kVEhgKewMEaPrtVRBJFJDEjI6PBharaeVu8uSH+Brbfvp3Vc1YzpMMQFny9gE5PdeKmD29iV/quOrVjm7rj12dM3bFxYw+dukOpZqhZjxaKSHvgDeAGU8ODCYwxLxpjEowxCRER2sFoCiLChK4T+Pjaj9lz1x5uiLuBd3a9w4DnB/CrN37Fp/s/pbIOz5Dw8AjVqTuUauacGRDHgI7V3kfbl9WJiAQCK4FHjDHfO7g25QC9w3vz/JTnOXrfUR4f/zi7M3Zz8dsX0+8//ViYuJCisqJa2/h56o5N+Pr2tU/d0Z+MjOU6dYdSLubMgPgB6CEiXUTEE5gJrKjLhvb1lwP/PT1wrZqvMN8wfjv6txyef5g3r3gTXw9fbl95O52e6sTv1/ye1ILUWtsIDBxCXNxa+vdfgYg7u3dPY8uWYaSmvkJ5eWETHIVS6mxOvQ9CRC4GngbcgUXGmMdE5FEg0RizQkSGYAuCEOAUcMIY009EZgOvArurNTfXGLPtXPvSq5iaD2MM3/z0DU99/xQf7v0Qi5uFmf1nct/w+4hvH1/r9pWV5Zw4sYiUlKcpKtqDm5sfkZEzad/+ZgIDhyEiTXAUSrUN57uKSW+UU051MPsgz2x8hle2vsLJspOMixnHfcPvY0rPKbjVcsOcMYb8/A2kpr5MevoSKiuL8PXtS/v2N2O1zsHTM7yJjkKp1ksDQrlc7qlcXt7yMs9uepaf8n6ie2h35g+bz9y4ufh7+te6fXl5PunpS0hNfYWCgo2IeBAePpX27W8mJGSi3p2tVANpQKhmo7yynGV7lvHU90/xfcr3BHsHc8ugW7hn6D10DOpYewNAYeFOUlNfIS3tDcrLs/Hy6kT79jfSrt0NeHt3cvIRKNW6aECoZmnD0Q089f1TvL/nfQRhRr8Z3Df8PoZGDa3T9pWVJWRmfkBq6ivk5KwChJCQX9G+/c2Eh1+Gm5uncw9AqVZAA0I1a0dyj/Dspmd5actL5JfkM7LjSO4bfh9Te0/F4mapUxvFxcmcOPEqJ04soqQkBQ+PcKzW62jf/ib8/Po6+QiUark0IFSLUFBSwKvbXuXp75/mcO5hOgR0YHqf6UzvO51RnUbVOqgNYEwF2dmrSE19maysDzGmnMDAEbRvfzMREVdhsdQ+3qFUW6IBoVqUisoKVuxbwevbX+ezA59RUlFCe//2TOszjel9pzO602jc3dxrbae0NJ20tDdITX2ZoqK9uLv7Exk5k3btbtLLZZWy04BQLVZBSQEr969kadJSPtn/CcXlxUT6RTKtty0sxsaMrfU0VM2Xy/azXy47Wy+XVW2aBoRqFQpLC/l0/6cs3bOUj3/8mKKyIsJ9w7mi9xVM7zudC2IuwMPd47xt/Hy57MsUFGxCxNN+uexNermsapM0IFSrU1RWxGcHPmNp0lI++vEjCksLCfEOYWrvqczoO4MJXSfg6X7+q5h+ebls52qXy9btklulWjoNCNWqnSo/xecHPmfpnqWs2LeC/JJ8gryCuLz35czoO4NJXSfhZfE65/Y1XS4bGjqZdu1u0stlVaunAaHajJLyElYfWs17Se/x4b4PyT2VS6BXIJf2vJTpfaczudtkfDx8zrl9cfFh++Wyr9ovl40gIuIqwsMvJzh4rIaFanU0IFSbVFpRyprDa3hv93t8sO8Dsouz8ff0Z0rPKUzvM52LelyEr4dvjdvaLpf9gtTUV8jO/oTKymLc3QMJDb2Q8PDLCA29CA+P0CY+IqUcTwNCtXllFWV8lfwVS5OWsmzvMjKLMvH18OWSHpcwve90Lu5x8TnnhKqoKCIn50uyslaQmfkRZWVpgDvBwaMJC7uM8PDL8PHp1rQHpJSDaEAoVU15ZTnrjqyzhcWeZaSdTMPH4sNFPS5iep/pTOk5hQCvgBq3NaaSgoIfyMxcQVbWCk6etD1q1de3H+HhlxEWdhmBgUP1aijVYmhAKHUOFZUVfPvTtyxNWsr7e94ntTAVL3cvLux+IdP7TufSnpcS5B10zu2Liw+RlfURmZkryM39GqjAw8NKWNgUwsMvIyRkIu7uNZ/GUqo50IBQqg4qTSXrj65nadJSliYt5VjBMTzdPRnbeSyTuk5iUrdJxFpjzznlR1lZDtnZn5KZuYLs7E+pqMjHzc2bkJBJhIVdRljYFLy82jXxUSl1fhoQStVTpalk07FNLE1ayqcHPiUpIwmAcN9wJnSZwKSuk5jYdSKdgzvXvH1lKbm56+zjFisoKTkCCIGBw6rGLXx9++p0H8rlNCCUaqTjBcf58tCXrDq0itWHVpNaaHvOdvfQ7lVhcUHMBYT4hPxiW2MMJ0/urBq3KCj4AQBv765V4xZBQf+Dm9v57wJXyhk0IJRyIGMMezL3sPrQalYdWsVXyV9RWFqIm7iR0CGBiV0mMrHrREZ2HFnjDXolJcfJyvqYzMwV5OSsxpgSLJZgQkMvtl9CeyEWy7nHPZRyJA0IpZyorKKMTcc2VfUuvk/5ngpTgY/FhzGdxzCxqy0wahq/KC8vJCdnFVlZK8jK+piyskxELAQHj6s6FeXtXfNpLKUcQQNCqSaUX5LPuiPrWHVwFasPr64av4jwjWBC1wlM7DKRSd0m0SnozMejGlNBfv73Vaeiior2AuDnN9Des5hMQMAQvZtbOZQGhFIudLzgOKsPra76OT1+0SO0BxO7TmRS10lc0OUCgr2Dz9iuqOjHqkto8/K+BSpxc/MlKGg0ISEXEBw8noCAQYjU/mwMpc5FA0KpZsIYQ1JGki0sDq/+xfjF6QHvEdEjzhi/KCvLJjf3K3Jz15KTs4aiIluvxN09iODgMQQHjyck5AL8/AboTXqqXjQglGqmyirK2HhsY1Xv4vT4ha+Hr238wj7gPcA64Izxi5KSE/bAWENu7lqKiw8AYLGEERw8jpCQ8QQHX4Cvb2+9lFadlwaEUi1Efkk+Xyd/XXWF1J7MPQBE+kUyutNoRnUcxciOI4lvH3/G8y5OnTpa1bvIzV1DSclRADw92xEcfEFVD8Pbu6sGhjqDBoRSLdSx/GN8efhLVh9azbc/fcvh3MMAeFu8GdJhSFVgjOg4gnBf26NTjTGcOnXIHha20LBNMAheXp2qehfBwePx9o522bGp5kEDQqlWIrUglfVH17P+6Hq+O/odW1K3UFZZBkCvsF6M7DiyKjR6hffCTdwwxlBUtJfc3DXk5KwlN3ct5eXZAPj4dCc42BYYISEX4OlpdeXhKRfQgFCqlSouKybxeGJVYKw/up6s4iwAQn1CGRE9oiowhkQNwdfDF2MqOXlyZ9XpqNzcdVRU5AO2WWlPXyEVHDxWn3nRBmhAKNVGGGP4MevHM3oZp8cxLG4W4tvFVwXGqE6j6BDQgcrKcgoLt1SdjsrL+5bKyiJA8PePs/cuxhMUNBqLJdC1B6gcTgNCqTYsuzibDUc3VAXGpmObKC4vBqBzUGdGdRrFyGhbYPSP7I8bleTnbyI3dy25uWvIy1uPMaWAOwEBCQQFjSIwcARBQSPw8opy7cGpRtOAUEpVKasoY9uJbVWB8d3R7zhecBwAf09/hkUNq+plDI8ejr+HJ/n5G+ynpL6ioCARY0oA8PLqSGDgiKrA8PeP1zu9WxgNCKXUORlj+CnvpzPGMbanbafSVCII/SP7VwXGyI4jiQmK5uTJbeTlbSA/3/Zz+rJaES8CAgZXBUZg4Ai8vDq4+AjV+WhAKKXqpaCkgE3HNlUFxoaUDeSX2AayQ31CSeiQwOD2g0nokEBChwQivdzIz/++KjAKCjbbT0vZLq2tHhj+/nHay2hGNCCUUo1SUVlBUkYS64+uJ/F4IptTN7MzfSflleWA7UFKCR0SSGifwOAOgxnUbgCBpFNQ8H21XkYKAG5u3gQEJFSdmrL1MvRJe67isoAQkQuBfwHuwMvGmCfO+nwM8DQQC8w0xiyt9tn1wO/tb/9ijHn9fPvSgFCqaZ0qP8WOtB1sPr6ZxOOJJKYmsjt9NxWmAgCrn5XBHQZXhUb/0A74m8PVehlbqnoZ3t4xZwSGv/9AfYBSE3FJQIhtiskfgUlACvADcI0xJqnaOjFAIPAAsOJ0QIhIKJAIJAAG2AwMNsbknGt/GhBKuV5xWTHb07ZX9TISjyeSlJFEpakEoL1/+6rQiG8XS+9AD3wqfiQ/fwN5eRsoLT0GgJubzxm9jKCgEXoTn5OcLyAsTtzvUOCAMeaQvYjFwOVAVUAYY5Ltn1Wete1kYJUxJtv++SrgQuAdJ9arlGokHw8fhkcPZ3j08KplJ0tPVoXG6eBY+eNKDLY/TqMCouxjGrcRG9GZ7n4leJUnkZe3npSUpzDm/wDw9u5yRmD4+fXHze2XT+xTjuPMgIgCjlZ7nwIMa8S2v7jgWkRuBW4F6NSp09kfK6WaAT9Pv6oroE4rLC1ka+rWql5G4vFEVuxbURUaHQM7ktAhgUHtLqFviD9dfQqwlO4gN3ct6elvAyDigZ9ffwICBuPvP4iAgEH4+cXi7u7jkuNsjZwZEE5njHkReBFsp5hcXI5Sqo78Pf0Z3Xk0ozuPrlqWX5LP1tStZ5yeWr53edXnMcExDG4/koGR3egd4EEnrzw8KvaRkbGM1NSX7Wu54+fX1x4Yg+2hMRCLxb+Jj7B1cGZAHAM6VnsfbV9W123HnbXtVw6pSinVLAV6BTI2ZixjY8ZWLcs9lVsVGompiWw+vpn397xf9XmkXySxkYPoE9aZHoE+dPIuwteSQnb2p6Slnb6uRfD17V3Vy7D1OOKwWIKa+AhbHmcOUluwDVJPwPYL/wfgWmPM7hrWfQ34+KxB6s3AIPsqW7ANUmefa386SK1U25BTnMPWE1vZkbaDnWk72ZG+g13puzhVfgoAN3GjZ1hP+oV3p0dQEF19K4j2zCCgYg/lZcer2vHx6Y6/v62XcTo82uLkhK68zPVibJexugOLjDGPicijQKIxZoWIDAGWAyHAKeCEMaaffdsbgd/Zm3rMGPPq+falAaFU21VRWcHBnINnhMaOtB0cyjlUtU6AZwD9InrRKziS7gHudPTKo4P7YTwqfh7u9PaOqTo9dTo0PD0jXXFITUZvlFNKtUkFJQXszth9RnDsTNtJzqmfr5jvGBhFn9BoegT60tm7mGiPY0S4HcVif8Krl1d0VVic7nF4erZvNU/m04BQSik7YwzHCo79orexN3Nv1Z3hnu6e9AzpSM+gYLr4VdLJM5Moy1FCPUEEPDys9l5GPP7+A/DzG4CPT0/c3FredT8aEEopVYvSilL2Zu61hUbajqrexrGCn6+tCfUOondIJN38LXTyzifKkkon30r8LLaJCv38+uDnN6Dqx99/AJ6eHZp1b0MDQimlGiirKIud6TvPCI5d6bsoKiuqWqe9XwhdAgLo5AtRnnlEeebR2Q9CPMDDI7QqLGzBEYufX/9mc+mtBoRSSjlQpankUM4hdqXvYk/GHvZk2n72Zu6lsLSwar0gTx+6BATS2beSDl55dPQupbMvtPMGX58u1XoasfbTVD2a/DSVBoRSSjUBYwwp+Sm2wKgWHHsy9pBRlFG1npe7hRh/fzr5VhLlWUBnP0NnX4j29SQkoK+9l/Fzr8OZg+IaEEop5WJZRVlVvYzq4ZGcm1y1jrsIUb4+dPKtoKN3CZ18obMvdA0MwRo88IzxDUedptKAUEqpZqqorIh9mft+0evYn7WfssqyqvUivDzo5FtBJ5/KquDoGdqJjiFxBAePplOnBxq0fw0IpZRqYcoqyjiUc+gXwbE3Yw+FZSer1guwuDHSGslnN6c2aD+umu5bKaVUA3m4e9ArvBe9wnsxtffUquU1jXOEeIc4pQYNCKWUakFEhI5BHekY1JFfdfuVU/fl5tTWlVJKtVgaEEoppWqkAaGUUqpGGhBKKaVqpAGhlFKqRhoQSimlaqQBoZRSqkYaEEoppWrUaqbaEJEM4Iir62ikcCDT1UU0I/p9nEm/j5/pd3GmxnwfnY0xETV90GoCojUQkcRzzYnSFun3cSb9Pn6m38WZnPV96CkmpZRSNdKAUEopVSMNiOblRVcX0Mzo93Em/T5+pt/FmZzyfegYhFJKqRppD0IppVSNNCCUUkrVSAOiGRCRjiKyVkSSRGS3iMx3dU2uJiLuIrJVRD52dS2uJiLBIrJURPaKyB4RGeHqmlxJRO6z/zvZJSLviIi3q2tqSiKySETSRWRXtWWhIrJKRPbb/+uQR8xpQDQP5cCvjTF9geHAXSLS18U1udp8YI+ri2gm/gV8ZozpDQykDX8vIhIFzAMSjDH9AXdgpmuranKvAReetexh4EtjTA/gS/v7RtOAaAaMManGmC321wXYfgFEubYq1xGRaOAS4GVX1+JqIhIEjAFeATDGlBpjcl1blctZAB8RsQC+wHEX19OkjDHrgOyzFl8OvG5//TowFQfQgGhmRCQGiAc2urYSl3oaeBCodHUhzUAXIAN41X7K7WUR8XN1Ua5ijDkGPAn8BKQCecaYL1xbVbNgNcak2l+fAKyOaFQDohkREX/gfeBeY0y+q+txBRGZAqQbYza7upZmwgIMAp43xsQDJ3HQ6YOWyH5u/XJswdkB8BOR2a6tqnkxtnsXHHL/ggZEMyEiHtjC4S1jzDJX1+NCo4DLRCQZWAyMF5E3XVuSS6UAKcaY0z3KpdgCo62aCBw2xmQYY8qAZcBIF9fUHKSJSHsA+3/THdGoBkQzICKC7RzzHmPMP11djysZY35rjIk2xsRgG3xcY4xps38hGmNOAEdFpJd90QQgyYUludpPwHAR8bX/u5lAGx60r2YFcL399fXAh45oVAOieRgFzMH21/I2+8/Fri5KNRv3AG+JyA4gDnjcxfW4jL0ntRTYAuzE9jusTU27ISLvABuAXiKSIiI3AU8Ak0RkP7Ze1hMO2ZdOtaGUUqom2oNQSilVIw0IpZRSNdKAUEopVSMNCKWUUjXSgFBKKVUjDQilaiEiFdUuP94mIg67k1lEYqrPyqlUc2JxdQFKtQDFxpg4VxehVFPTHoRSDSQiySLyfyKyU0Q2iUh3+/IYEVkjIjtE5EsR6WRfbhWR5SKy3f5zeooIdxF5yf6Mgy9ExMe+/jz7M0J2iMhiFx2masM0IJSqnc9Zp5iurvZZnjFmAPBvbLPQAjwLvG6MiQXeAp6xL38G+NoYMxDbfEq77ct7AM8ZY/oBucCV9uUPA/H2dm531sEpdS56J7VStRCRQmOMfw3Lk4HxxphD9skWTxhjwkQkE2hvjCmzL081xoSLSAYQbYwpqdZGDLDK/qAXROQhwMMY8xcR+QwoBD4APjDGFDr5UJU6g/YglGocc47X9VFS7XUFP48NXgI8h6238YP9ATlKNRkNCKUa5+pq/91gf72enx+DOQv4xv76S+AOqHrmdtC5GhURN6CjMWYt8BAQBPyiF6OUM+lfJErVzkdEtlV7/5kx5vSlriH2WVZLgGvsy+7B9gS432B7GtwN9uXzgRfts29WYAuLVGrmDrxpDxEBntFHjaqmpmMQSjWQfQwiwRiT6epalHIGPcWklFKqRtqDUEopVSPtQSillKqRBoRSSqkaaUAopZSqkQaEUkqpGmlAKKWUqtH/A9BvKGQsjEBkAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cv-hHybT2S4N"
      },
      "source": [
        "**Adam Optimizer**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KDmCFhKM2XQn",
        "outputId": "a89e85dc-513e-4b1e-b1d4-04a5a7a41cc9"
      },
      "source": [
        "model.compile(loss = contrastive_loss, optimizer=\"adam\", metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_74\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_113 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_114 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_73 (Functional)           (None, 256)          91584       input_113[0][0]                  \n",
            "                                                                 input_114[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "lambda_107 (Lambda)             (None, 1)            0           model_73[0][0]                   \n",
            "                                                                 model_73[1][0]                   \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tHNd-2fC_Txl",
        "outputId": "f5b80563-623a-45aa-b34e-f691d8546eba"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 8ms/step - loss: 0.0933 - accuracy: 0.8839 - val_loss: 0.0379 - val_accuracy: 0.9595\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0277 - accuracy: 0.9707 - val_loss: 0.0292 - val_accuracy: 0.9643\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0179 - accuracy: 0.9808 - val_loss: 0.0250 - val_accuracy: 0.9693\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0132 - accuracy: 0.9855 - val_loss: 0.0242 - val_accuracy: 0.9708\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0105 - accuracy: 0.9891 - val_loss: 0.0247 - val_accuracy: 0.9692\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0085 - accuracy: 0.9912 - val_loss: 0.0236 - val_accuracy: 0.9710\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0072 - accuracy: 0.9928 - val_loss: 0.0247 - val_accuracy: 0.9696\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0070 - accuracy: 0.9928 - val_loss: 0.0241 - val_accuracy: 0.9715\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0060 - accuracy: 0.9939 - val_loss: 0.0256 - val_accuracy: 0.9692\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0055 - accuracy: 0.9946 - val_loss: 0.0227 - val_accuracy: 0.9729\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Imwb91oc2W5v"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8NqyUolvCgWe",
        "outputId": "1ea409d0-2756-4a1b-9b78-6eaacf529970"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 97.28999999999999\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "4yalLvaq2Wjl",
        "outputId": "77c0b134-46f4-43d7-c008-f0697584e608"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVxVdf7H8deHyyqgIiKyqCAoaoiouLeYVtpqY06LZNmqTWl72TTTWFP9aqaadq3MaiZKG7csm2zPHM1CJTeUzYVNRJRNZP/+/uDqUKKicjnA/TwfDx/ce+5Z3vfMdD/3+/2e+z1ijEEppZTzcrE6gFJKKWtpIVBKKSenhUAppZycFgKllHJyWgiUUsrJaSFQSiknp4VAtUgi8h8RubGp17WSiOwSkQuszgEtK4uynqvVAVTbISKl9Z62AyqAGvvzacaYhMbuyxhzsSPWbYlE5D/AOfanHoABKu3P3zfGTD/F/c0GIo0x1zdZSNWmaSFQTcYY43PksYjsAm41xnz12/VExNUYU92c2Vqy+oVMRN4Fsowxf7IukXI22jWkHE5ERotIlog8LCJ7gXdExE9EPhWRfBE5aH8cWm+b70TkVvvjqSKyWkSes6+7U0QuPs11w0VklYiUiMhXIvKaiLx/nNyNyfhXEfmvfX9fiEjneq9PEZHdIlIgIo+e5rm7TESSRKRQRNaISEy91x4WkWz7sXeIyFgRGQ/8EbhGREpF5JdGHMNDRF4UkRz7vxdFxMP+Wmf7+y4UkQMi8oOIuBzv+KfzHpX1tBCo5tIV6AT0AG6n7v9779ifdwcOA6+eYPthwA6gM/A34G0RkdNY9wPgJ8AfmA1MOcExG5NxMnAT0AVwBx4AEJF+wBz7/oPtxwvlFIjIQGA+MM2+/RvAcvsHdxRwFzDEGOMLjAN2GWM+B54GFhpjfIwxAxpxqEeB4UAsMAAYChxpkdwPZAEBQCB1RcYc7/in8v5Uy6GFQDWXWuAvxpgKY8xhY0yBMWaxMabMGFMCPAWcd4Ltdxtj3jLG1ADvAUHUfTA1el0R6Q4MAR4zxlQaY1YDy493wEZmfMcYk2KMOQx8RN2HKcAk4FNjzCpjTAXwZ/s5OBW3A28YY9YZY2qMMe9RN+4ynLqxFw+gn4i4GWN2GWPST3H/R8QDTxhj9hlj8oHH+V+BrKLu/PUwxlQZY34wdROUNeXxlcW0EKjmkm+MKT/yRETaicgb9q6TYmAV0FFEbMfZfu+RB8aYMvtDn1NcNxg4UG8ZQObxAjcy4956j8vqZQquv29jzCGg4HjHOo4ewP32bplCESkEugHBxpg04B7qWjX7RGSBiASf4v6PCAZ213u+274M4O9AGvCFiGSIyCz7+2nK4yuLaSFQzeW309zeD0QBw4wx7YFz7cuP193TFHKBTiLSrt6ybidY/0wy5tbft/2Y/qcWl0zgKWNMx3r/2hljPgQwxnxgjDmbuoJhgGft253qlMI59n0c0d2+DGNMiTHmfmNMT+AK4L4jYwEnOL5qZbQQKKv4UtfnXiginYC/OPqAxpjdQCIwW0TcRWQEcLmDMi4CLhORs0XEHXiCU//v7S1guogMkzreInKpiPiKSJSIjLEP6pbbcx7pesoDwo4M6jbCh8CfRCTAPtj9GPA+HB2sjrSPsRRR1yVUe5Ljq1ZGC4GyyouAF7Af+BH4vJmOGw+MoK6b5klgIXX97g057YzGmK3AndQNTucCB6kbdG00Y0wicBt1A9QHqeuimWp/2QN4xp5tL3WD1Y/YX/u3/W+BiGxoxKGepK5AbgI2AxvsywB6AV8BpcBa4HVjzLcnOb5qZURvTKOcmYgsBLYbYxzeIlGqpdIWgXIqIjJERCJExMV+zf0EYJnVuZSykv6yWDmbrsAS6gZus4A7jDEbrY2klLW0a0gppZycdg0ppZSTa3VdQ507dzZhYWFWx1BKqVZl/fr1+40xAQ291uoKQVhYGImJiVbHUEqpVkVEdh/vNe0aUkopJ6eFQCmlnJwWAqWUcnKtboxAKdVyVVVVkZWVRXl5+clXVg7h6elJaGgobm5ujd5GC4FSqslkZWXh6+tLWFgYx79vkHIUYwwFBQVkZWURHh7e6O20a0gp1WTKy8vx9/fXImAREcHf3/+UW2RaCJRSTUqLgLVO5/w7TSE4dGgr6emz0Ck1lFLq15ymEBw8+BWZmc+yb99Cq6MopRykoKCA2NhYYmNj6dq1KyEhIUefV1ZWnnDbxMREZs6cedJjjBw5skmyfvfdd1x22WVNsq8z5TSDxSEhd5GXl0Ba2t106nQRbm6drI6klGpi/v7+JCUlATB79mx8fHx44IEHjr5eXV2Nq2vDH3txcXHExcWd9Bhr1qxpmrAtiNO0CERsREW9RVVVAenpD5x8A6VUmzB16lSmT5/OsGHDeOihh/jpp58YMWIEAwcOZOTIkezYsQP49Tf02bNnc/PNNzN69Gh69uzJyy+/fHR/Pj4+R9cfPXo0kyZNok+fPsTHxx/tev7ss8/o06cPgwcPZubMmSf95n/gwAGuvPJKYmJiGD58OJs2bQLg+++/P9qiGThwICUlJeTm5nLuuecSGxtLdHQ0P/zwwxmfI6dpEQD4+Ayge/cH2bPnGQIDr8fPb4zVkZRqs1JT76G0NKlJ9+njE0uvXi+e8nZZWVmsWbMGm81GcXExP/zwA66urnz11Vf88Y9/ZPHixcdss337dr799ltKSkqIiorijjvuOOba/I0bN7J161aCg4MZNWoU//3vf4mLi2PatGmsWrWK8PBwrrvuupPm+8tf/sLAgQNZtmwZ33zzDTfccANJSUk899xzvPbaa4waNYrS0lI8PT158803GTduHI8++ig1NTWUlZWd8vn4LadpERzRo8djeHpGkJIyjZqaw1bHUUo1g9///vfYbDYAioqK+P3vf090dDT33nsvW7dubXCbSy+9FA8PDzp37kyXLl3Iy8s7Zp2hQ4cSGhqKi4sLsbGx7Nq1i+3bt9OzZ8+j1/E3phCsXr2aKVOmADBmzBgKCgooLi5m1KhR3Hfffbz88ssUFhbi6urKkCFDeOedd5g9ezabN2/G19f3dE/LUU7VIgCw2byIinqDX365gN27/0rPnk9bHUmpNul0vrk7ire399HHf/7znzn//PNZunQpu3btYvTo0Q1u4+HhcfSxzWajurr6tNY5E7NmzeLSSy/ls88+Y9SoUaxcuZJzzz2XVatWsWLFCqZOncp9993HDTfccEbHcboWAYCf31i6dp1KZubfKS3dZHUcpVQzKioqIiQkBIB33323yfcfFRVFRkYGu3btAmDhwpNfqXjOOeeQkJAA1I09dO7cmfbt25Oenk7//v15+OGHGTJkCNu3b2f37t0EBgZy2223ceutt7Jhw4YzzuyUhQAgIuI5XF392LHjNoypsTqOUqqZPPTQQzzyyCMMHDiwyb/BA3h5efH6668zfvx4Bg8ejK+vLx06dDjhNrNnz2b9+vXExMQwa9Ys3nvvPQBefPFFoqOjiYmJwc3NjYsvvpjvvvuOAQMGMHDgQBYuXMjdd999xplb3T2L4+LiTFPdmCYv7wOSk+OJjHyJ0NCTXz+slDqx5ORk+vbta3UMy5WWluLj44MxhjvvvJNevXpx7733NtvxG/rfQUTWG2MavD7WaVsEAF26XEenTuPJyPgj5eV7rI6jlGoj3nrrLWJjYznrrLMoKipi2rRpVkc6IacuBCJCr15zAENKyh90+gmlVJO49957SUpKYtu2bSQkJNCuXTurI52QUxcCAC+vMMLDn+TAgRXk5//b6jhKKdXsnL4QAISEzMDHZzCpqTOpqjpodRyllGpWWggAFxdXoqLmUVW1n/T0B62Oo5RSzUoLgZ2vbyzdut3P3r1vc/Dgd1bHUUqpZqOFoJ6wsL/g6dmTlJTbqanRe64q1dqcyTTUUPdjruPNLvruu+9y1113NXXkFkELQT02Wzt6957L4cOp7N79pNVxlFKn6Mg01ElJSUyfPv3o1TtJSUm4u7ufdPsTFYK2TAvBb3TqdCGBgTeQmfkspaWbrY6jlDpD69ev57zzzmPw4MGMGzeO3NxcAF5++WX69etHTEwM1157Lbt27WLu3Ln84x//IDY29oTTO+/atYsxY8YQExPD2LFj2bOn7ndI//73v4mOjmbAgAGce+65AGzdupWhQ4cSGxtLTEwMqampjn/Tp8jpJp1rjIiI5zlw4DN27LiNQYP+i4jN6khKtTr3fH4PSXubdhrq2K6xvDi+8ZPZGWOYMWMGH3/8MQEBASxcuJBHH32U+fPn88wzz7Bz5048PDwoLCykY8eOTJ8+/Zib2TRkxowZ3Hjjjdx4443Mnz+fmTNnsmzZMp544glWrlxJSEgIhYWFAMydO5e7776b+Ph4KisrqalpeVPaaIugAe7unYmI+AclJevIzp5jdRyl1GmqqKhgy5YtXHjhhcTGxvLkk0+SlZUFQExMDPHx8bz//vvHvWvZ8axdu5bJkycDMGXKFFavXg3AqFGjmDp1Km+99dbRD/wRI0bw9NNP8+yzz7J79268vLya8B02DYe2CERkPPASYAPmGWOe+c3r3YH3gI72dWYZYz5zZKbGCgyMJy/vfXbufITOnSfg6dnN6khKtSqn8s3dUYwxnHXWWaxdu/aY11asWMGqVav45JNPeOqpp9i8+cy7gufOncu6detYsWIFgwcPZv369UyePJlhw4axYsUKLrnkEt544w3GjGlZN8VyWItA6vpTXgMuBvoB14lIv9+s9ifgI2PMQOBa4HVH5TlVIkLv3nMwppbU1Dt1+gmlWiEPDw/y8/OPFoKqqiq2bt1KbW0tmZmZnH/++Tz77LMUFRVRWlqKr68vJSUlJ93vyJEjWbBgAQAJCQmcc845AKSnpzNs2DCeeOIJAgICyMzMJCMjg549ezJz5kwmTJhw9DaULYkju4aGAmnGmAxjTCWwAJjwm3UM0N7+uAOQ48A8p8zLK5zw8CcoKPiE/PxFVsdRSp0iFxcXFi1axMMPP8yAAQOIjY1lzZo11NTUcP3119O/f38GDhzIzJkz6dixI5dffjlLly496WDxK6+8wjvvvENMTAz/+te/eOmllwB48MEH6d+/P9HR0YwcOZIBAwbw0UcfER0dTWxsLFu2bDnjm8g4gsOmoRaRScB4Y8yt9udTgGHGmLvqrRMEfAH4Ad7ABcaY9Q3s63bgdoDu3bsP3r17t0MyN6S2tpoNG4ZRUZHN0KHJuLn5NduxlWptdBrqlqG1TUN9HfCuMSYUuAT4l4gck8kY86YxJs4YExcQENCsAeumn3iLqqr9ZGQ83KzHVkqp5uDIQpAN1B9hDbUvq+8W4CMAY8xawBPo7MBMp8XXdxDdut1Lbu5bFBZ+b3UcpZRqUo4sBD8DvUQkXETcqRsMXv6bdfYAYwFEpC91hSDfgZlOW1jYbDw9w9mxY5pOP6HUCeiFFdY6nfPvsEJgjKkG7gJWAsnUXR20VUSeEJEr7KvdD9wmIr8AHwJTTQv9f5HN5m2ffmIHe/Y8bXUcpVokT09PCgoKtBhYxBhDQUEBnp6ep7SdU9+z+HQkJ09h376FxMVtxNv7LMtyKNUSVVVVkZWVRXm5tpqt4unpSWhoKG5ubr9afqLBYp1i4hRFRLxAQcF/2LHjNgYOXE0DY9tKOS03NzfCw8OtjqFOkdN8iu0v28+7Se+e8X7c3QOIjHyB4uK15OTMPfNgSillMacpBC+ve5mbPr6JB794kFpTe0b7Cgycgp/fhWRkzKK8PKuJEiqllDWcphD85by/cOeQO3lu7XNMXjyZiuqK095X3fQTczGmmtTUu3RgTCnVqjlNIbC52Hjl4lf42wV/Y+HWhYx7fxwHD5/+jeq9vHoSFvY4BQUfs3//kiZMqpRSzctpCgHUfZN/cNSDfDDxA9ZkruHsd85md+HpT1cRGnovPj6xpKbOoKqqsAmTKqVU83GqQnDEdf2vY+X1K8kuzmbE2yNO++YZddNPzKOyMo+MjFlNnFIppZqHUxYCgPPDz2f1zauxudg4551z+CL9i9Paj6/vYEJD7yE39w0KC48/W6FSSrVUTlsIAKK7RPPjLT/S068nl35wKe8lvXda+wkPfwIPjx6kpNxObe3pD0IrpZQVnLoQAIS0D2HV1FWc1+M8pn48lSdXPXnKVwEdmX6irGw7u3f/n4OSKqWUYzh9IQDo4NmBz+I/Y0rMFP787Z+Z9uk0qmurT2kf/v7j6dJlMnv2PM2hQ9sclFQppZqeFgI7d5s77135Ho+e8yhvbXiLCQsmUFpZekr7iIz8BzabLzt23IY5wx+tKaVUc9FCUI+I8OSYJ5l76Vw+T/uc8987n7zSvEZv7+7ehYiI5ykuXkNOzhsOTKqUUk1HC0EDpsVN4+NrP2Zb/jZGvD2ClIKURm/bteuNdOw4loyMWVRU/PY+PEop1fJoITiOy3pfxrc3fktpZSkj3x7Jmsw1jdruf9NPVJKaOsPBKZVS6sxpITiBoSFDWXvLWvy8/Bj7z7EsTV7aqO3atYskLGw2+/cvJT+/cdsopZRVtBCcRESnCNbcvIbYrrFc9dFVvPrTq43aLjT0Pry9B5CaeifV1UUOTqmUUqdPC0EjBHgH8PUNX3NF1BXM+M8MHvryoZNOZe3i4kZU1Fv26SceaaakSil16rQQNFI7t3Ysvnoxf4j7A39f83fil8SfdCrr9u2HEBo6k5ycORQV/beZkiql1KnRQnAKbC42Xr3kVZ694FkWbFnAuPfHUVh+4llHw8L+iodHd3bsuE2nn1BKtUhaCE6RiPDQqIdImJhQN5X1/LPZU7TnuOu7uvrQu/ccysqS2bPn2WZMqpRSjaOF4DRN7j+ZldevJKs466RTWfv7X0KXLteye/dTHDqU3IwplVLq5LQQnIEjU1m7iAvnvnMuX6Z/edx1IyNfxGbzJiXldp1+QinVomghOENHprIO9wvnkg8u4Z+//LPB9dzdA4mIeI6iotXk5r7VzCmVUur4tBA0gfpTWd+47EaeWvVUg1NZd+16Ex07jiE1dQa5ufMtSKqUUsfSQtBEjkxlfX3M9fzp2z8x/dPpx0xlLSKcddZiOnY8nx07biEt7X6MqbEosVJK1XG1OkBb4m5z559X/pPu7bvz9OqnyS7JZuGkhXi7ex9dx82tI/37ryA9/X6ysl6grGwb/fotwNW1g4XJlVLOTFsETUxEeGrsU8y5dA7/SfsPo98bfcxU1i4urvTq9RK9e7/BwYNfsWHDCMrK0ixKrJRydloIHGR63HSWXbOMrfu2MnL+yAansg4Ovp2YmC+prMxjw4ZhHDz4rQVJlVLOzqGFQETGi8gOEUkTkVkNvP4PEUmy/0sRkRP/TLeVuTzqcr6b+h0lFSWMfHskazPXHrOOn99oBg/+GXf3rmzadBHZ2XMtSKqUcmYOKwQiYgNeAy4G+gHXiUi/+usYY+41xsQaY2KBV4AljspjlaEhQ1lzyxr8vPwY888xLNu+7Jh1vLx6MmjQWvz8LiI19Q5SU2dQe4r3TFZKqdPlyBbBUCDNGJNhjKkEFgATTrD+dcCHDsxjmchOkay5eQ0DAgcwceFEHv/ucTKLMn+1jqtre/r3X05o6P1kZ7/K5s0XU1V10KLESiln4shCEALU/7TLsi87hoj0AMKBb47z+u0ikigiifn5+U0etDkEeAfwzY3fMLHvRGZ/P5vuL3Zn6FtDeWb1M0fHD0RsREY+R1TUfAoLv2fDhmGUle2wOLlSqq1rKYPF1wKLzHEuqjfGvGmMiTPGxAUEBDRztKbTzq0di65exI67dvDM2GcQER75+hGiXo0i+vVoHvv2MTbmbqRr16nExn5LdXUh69cP48CBL6yOrpRqw6ShX8A2yY5FRgCzjTHj7M8fATDG/F8D624E7jTGnPTGwHFxcSYxMbGp41omsyiTZduXsWT7ElbtXkWtqSWsYxgT+0zk0p4j8D74OIfLkomM/AchIXchIlZHVkq1QiKy3hgT1+BrDiwErkAKMBbIBn4GJhtjtv5mvT7A50C4aUSYtlYI6ss/lM8nKZ+wJHkJX2Z8SWVNJV29AzmnixdDfXYxvu+t9It6HRcXN6ujKqVaGUsKgf3AlwAvAjZgvjHmKRF5Akg0xiy3rzMb8DTGHHN5aUPaciGor7iimM9SP2NJ8hI+S/2MQ1WH8HGF87oGMnXY37kk6iraubWzOqZSqpWwrBA4grMUgvoOVx3my4wvSdjwHP/J+IGSamjn6sXFvS5hYt+JXNrrUjp46hQVSqnj00LQhuw/sJqE/17Od3mHWHvQl7yyA7i5uHFBzwv4XZ/fMaHPBLp4d7E6plKqhdFC0MaUl2eyZcsEiks2Utz+LlYXeLB0+1IyDmbgIi6c3f1sJvaZyO/6/o7uHbpbHVcp1QJoIWiDamoOkZx8I/v3L6Zr15vo1et1tuTvYOn2pSxJXsLmfZsBiAuOO1oU+nTuY3FqpZRVtBC0UcbUsmvX4+ze/QTt248iOnoJ7u513UKpBalHi8K67HUA9O3cl4l9JzKx70QGdh2ol6Iq5US0ELRx+/YtZPv2qbi5BdK//3J8fGJ+9XpWcRbLti9j6falfL/re2pMDT069ODy3pczPnI8o8NG/+qeCUqptkcLgRMoLk5ky5YJVFcX0a9fAp07Nzyt0/6y/Xyy4xOWbF/C1xlfc7j6MO42d87ufjbjIsYxLmIcMYEx2lpQqo3RQuAkKipy2LLlSkpKEgkPf4ru3Wed8AO9vLqc1XtWszJtJZ+nf86WfVsACPIJ4qKIixgXMY4LIy6kc7vOzfUWlFIOooXAidTUHGbHjpvZt28BXbrEExU1D5vNs1HbZhdn80X6F6xMX8mXGV9y4PABBGFw8GDGR4xnXOQ4hocOx9VF73CqVGujhcDJGGPYvfspdu36M76+w4iOXoaHR9dT2kdNbQ2JOYmsTF/JyvSV/Jj1I7WmlvYe7RkbPrauGylyHGEdwxzzJpRSTUoLgZPKz19CcvIU3Nw6ER39Mb6+g057X4XlhXyd8TWfp33OyvSVZBbXzTAe5R91tCiMDhut014o1UJpIXBiJSVJbNlyBVVV++nT55906TLpjPdpjGH7/u1HWwvf7fqO8upy3G3unNvj3KODztFdonXQWakWQguBk6uo2MvWrb+juPhHwsIep0ePPzfpB/ThqsP8sOcHVqbVFYat+XUTzAb7BnNRxEWMjxjPBT0vwL+df5Md83QZYyivLqewvJCiiqK6v+VFVNZUMrbnWG3RqDZLC4GipqaclJTbycv7FwEBV9OnzzvYbI750MsqzjpaFL7K+IqD5QcRhCEhQ462FoaFDjutQeea2hqKKoooKi865sP8yPNjXvvNOlW1VQ3uO8Q3hKfGPMWUAVNwkZZyzyalmoYWAgXUfRvOzPw7GRmz8PEZRHT0Mjw9Qx16zJraGn7O+fno2MJP2T9Ra2rp4NGBC3pewEURF9Heo/2xH+YNfMAXlhdSWll60mN6u3nT0bMjHTw70MGjw9HHHT3sfz07/mp5B48OFFcU89h3j5GYk8jArgN5/qLnOT/8fIeeG6WakxYC9Sv79y8nOTkesBER8TeCgm5Fmukb8IHDB/g642tWpq/k87TPyS7J/tXrri6ux3xQ/+r5bz7Af7tOe4/2uNlO78Y9taaWBVsW8MjXj7CnaA+X976cv134N52jSbUJWgjUMcrKUkhJmUZh4Xd06HA2vXu/ibd332bNYIwh7UAa1bXVRz/QvVy9LB9gPlx1mJfWvcTTPzxNWVUZ0wZPY/bo2QR4t977ZSulhUA1yBjD3r3vkp5+PzU1pXTv/gjduz/S6B+gtXX7Du3j8e8e5431b+Dt7s0fz/4jdw+/G09XPT+q9TlRIdARMScmIgQF3cTQodsJCLia3bufIDExlsLC762O1iJ08e7Ca5e+xuY7NnNej/OY9fUsol6N4oPNH1Braq2Op1ST0UKgcHfvQr9+7xMT8znGVJKUNJrt22+lquqA1dFahL4BfVl+3XK+vuFr/L38iV8Sz/B5w/lh9w9WR1OqSWghUEd16jSOIUO20K3bQ+zd+y4//dSXvLwFtLbuQ0cZEz6GxNsTeXfCu+SU5HDuu+cyceFEUgtSrY6m1BnRQqB+xWZrR0TEswwenIinZw+Sk69j8+ZLOHx4p9XRWgQXceHG2BtJmZHCX8//K1+kf0G/1/txz+f3UFBWYHU8pU6LFgLVIF/fWAYNWktk5EsUFv7Azz9Hs2fPc9TWVlsdrUVo59aOP537J9JmpnFT7E288tMrRL4SyfNrnqeiusLqeEqdEi0E6rhEbISGzmTo0G34+Y0lI+NBNmwYSnGxXrV1RFefrrx5+Zv8Mv0XhocO54EvH6Dva335aOtH2qWmWg0tBOqkPD27Ex39MWedtYjKyr1s2DCMtLR7qa4++a98nUV0l2j+E/8fVl6/Eh93H65ZdA2j5o9ibeZaq6MpdVKNKgQi4i32n56KSG8RuUJETu/nm6pVEhECAq5iyJBtBAffTlbWi/z881kUFKywOlqLclHERWyctpF5l89jZ+FORs4fyTWLriHjYIbV0ZQ6rsa2CFYBniISAnwBTAHedVQo1XK5uXWkd+85DBy4GpvNh82bL2Pr1qupqMi1OlqLYXOxccugW0idkcpj5z7GJzs+oe9rfXngiwc4ePig1fGUOkZjC4EYY8qAicDrxpjfA2c5LpZq6Tp0GEVc3EbCw59k//7l/PRTX3Jy3sToD62O8nH34fHzHyd1Rirx/eN5Ye0LRL4SycvrXqayptLqeEod1ehCICIjgHjgSF+AzTGRVGvh4uJOjx6PMmTIJnx9B5KSMo2NG8/l0KFtVkdrUULahzB/wnw2TNvAwK4Dufvzu4l+PZqlyUt1QFm1CI0tBPcAjwBLjTFbRaQn8O3JNhKR8SKyQ0TSRGTWcda5WkS2ichWEfmg8dFVS9GuXW8GDPiGqKj5lJUlk5gYy86dj1FTU251tBYltmssX075kk+v+xRXF1cmfjSR8949j5+zf7Y6mnJypzzpnH3Q2McYU3yS9WxACnAhkAX8DFxnjNlWb51ewEfAGGPMQRHpYmDE+pgAAByOSURBVIzZd6L96qRzLVtl5T7S0u5j374EvLx6ExX1Jh07nmd1rBanuraaeRvm8di3j5Ffls/k/pN5eszT9OjYw+poLcKhykMs37GcnJIcAn0C6erTlUDvur/+7fz1xkGn4YxnH7V/U58O1FD3gd4eeMkY8/cTbDMCmG2MGWd//giAMeb/6q3zNyDFGDOvsW9GC0HrcODASlJS7qC8fCddu95MRMTfcXPrZHWsFqe4ophnVj/DP378B8YYZg6byU2xN9E3oHmnBG8Jqmur+TL9SxI2J7B0+1LKqsoaXM8mNrp4dyHQJ/BocTj69zdFw8/LT4uGXVMUgiRjTKyIxAODgFnAemNMzAm2mQSMN8bcan8+BRhmjLmr3jrLqGs1jKJuzGG2MebzBvZ1O3A7QPfu3Qfv3r37pJmV9Wpqyti163EyM5/Hzc2fyMgX6dLlWsvvN9AS7Snaw6PfPErCpgQMhn4B/biq71VM6jeJ/l36t9lzZoxhXfY6EjYlsHDrQvLL8vHz9OP3/X5PfEw8MYEx7Du0j72le8krzav7e+g3f0vzyDuU1+AAvKuLK4HegY0qGh09O7bZ8wxNUwi2ArHAB8CrxpjvReQXY8yAE2zTmELwKVAFXA2EUneZan9jTOHx9qstgtanpCSJlJTbKSn5GT+/cfTuPQcvr3CrY7VI2cXZLN2+lEXbFvHDnh+oNbVEdopkUt9JXNXvKgYHDW4TH1YpBSkkbEogYXMC6QfT8bB5cEXUFcT3j2d85Hg8XD1OaX/GGArLC39VHE5UNKobmCrF3eZOF+8uvy4W9r/nhZ1HTOBxv/e2Ck1RCGYCDwO/AJcC3YH3jTHnnGCbxnQNzQXWGWPesT//GphljDnu6JkWgtbJmBqys19j585HMaaGsLAnCA29B5fTuIG9s8grzWPZ9mUsTl7MNzu/ocbU0KNDj6MthWGhw1pVt8fe0r0s2LKAhM0JJOYkIghjwscQ3z+eiX0n0sGzQ7PkqDW1HDx8sFFFY9+hfdSYGgBGdhvJHXF3MKnfpFZ5cyKH3KFMRFyNMcedgUxEXKnr9hkLZFM3tjDZGLO13jrjqRtAvlFEOgMbgVhjzHGncdRC0LqVl2eSmnonBQWf4OMTS+/eb9G+fYP/31T1FJQVsHzHchYlL+LL9C+pqq0ixDeEiX0nclXfqzi7+9nYXFreFd0lFSUsSV5CwuYEvt75NbWmlkFBg4jvH8+10dcS7BtsdcQTqjW17C3dy8ItC5m7fi4pBSn4e/lzU+xNTIubRmSnSKsjNlpTtAg6AH8BzrUv+h54whhTdJLtLgFepK7/f74x5ikReQJINMYsl7o27vPAeOoGop8yxiw40T61ELR+xhj2719CauoMKivzCAn5A2Fhf8XNraPV0VqFwvJCPk35lMXJi/k87XPKq8vp4t2F3/X5HZP6TWJ02GhcLWxpVdZUsjJtJQmbE1i+YzmHqw8T3jGc+P7xTO4/udUOhBtj+GbnN8xJnMOy7cuoMTVcFHER0wdP5/Koyy09543RFIVgMbAFeM++aAowwBgzsclSNpIWgrajurqIjIw/kpMzBze3zvTs+Sxdu96ItKLuDquVVpbyWepnLNq2iBWpKyirKsPfy58JUROY1G8SY3uOxd3m7vActaaWNZlrSNiUwEfbPuLA4QN0bteZq/tdTXxMPCNCR7SJsY0jckpyeHvD27y54U2yirMI8Q3htkG3ceugWwlpH2J1vAY12VVDJ1vWHLQQtD0lJRtITb2L4uK1tG8/gl69XsXXd5DVsVqdsqoyVqatZHHyYpbvWE5JZQkdPDpwedTlTOo7iYsiLsLLzatJj7l131YSNifwweYP2F20Gy9XL67scyXx/eO5KOIi3Gxte27K6tpqVqSsYO76uaxMW4mLuDChzwSmD57O2J5jW9QYTlMUgrXAg8aY1fbno4DnjDEjmjRpI2ghaJuMqWXv3n+SkfEQVVUFBAdPJzz8r/rbg9NUUV3BVxlfsSh5ER9v/5iD5QfxdvPmst6XcVXfq7ik1yV4u3uf1r6zi7P5cMuHJGxOIGlvEjaxcWHEhcT3j+fKPlfi4+7TxO+mdcg4mMEbiW8wP2k++8v2E9kpkumDpzM1dir+7fytjtckhWAA8E/gyLD+QeBGY8ymJkvZSFoI2raqqkJ27XqM7OzXcHPrRHj4/xEUdLN2F52Bqpoqvt31LYu3LWbp9qXkl+Xj5erFxb0u5qq+V3FZ78to79H+hPsoLC9k8bbFJGxO4Ltd32EwDA0ZSnz/eK456xoCfQKb6d20fBXVFSxOXsycxDms3rMaD5sHV591NXfE3cHw0OGWdZE12VVDItIewBhTLCL3GGNebKKMjaaFwDmUlv5CaupdFBWtxtd3KL16vaZXFzWBmtoaftjzA4u3LWZx8mJyS3Nxt7lzUcRFTOo7iSuirsDPyw+o+0D7LPUz3t/8PitSVlBRU0GvTr2ODvr28u9l8btp+TbnbWZu4lz+telflFSWMCBwAHfE3cHk/pPx9fBt1iyOunx0jzGm+xklOw1aCJyHMYa8vPdJT3+Qqqp9BAXdTs+eT+HmZn0zuy2oNbX8mPUji7YtYnHyYvYU7cHVxZWx4WMJ9g1m6falFJYXEugdyLXR1xLfP5644Lg2NejbXEorS/lg8wfMSZxD0t4kfN19uT7meu6Iu4P+gf2bJYOjCkGmMabbGSU7DVoInE91dRG7ds0mK+sVXF070LPn0wQF3UrdvIaqKRhjSMxJZHHyYhZtW0TeoTwm9p1IfP94xoSPafGXRrYWR6bUmJM4h4VbFlJRU8GobqOYHjfd4T9U0xaBahNKSzfbu4tW4esbZ+8uGmp1rDbnyGeCfvN3rAOHD/Bu0rvMTZxL6oFU/L38uXngzUwbPI2IThFNfrzTLgQiUgI0tIIAXsaYZv+aoIXAuRlj2LfvQ9LTH6Cyci9BQbcQHv407u4BVkdT6rTUmlq+2fkNcxPnHv2h2riIcUyPm85lvS9rstaYQ1oEVtFCoACqq4vZtetxsrJewtW1PeHhTxIcPE27i1SrllOSw7wN83hz/Ztkl2QT2j706A/VznQ6Di0Eqs06dGgrqakzKCz8Fh+fQfTq9SodOjT7z1uUalLVtdV8mvIpcxPnsjJ9JTaxMaHPBGaNmsWQkCGntc8TFQK9OFu1at7eZzFgwNf067eAysq9bNw4ku3bb6ay8oQ3ulOqRXN1ceXKPlfy+fWfkzYjjftG3Mf3u74n/WC6Q46nLQLVZlRXl7J791/JynoBm82HsLC/Ehw8Xae6Vm1CeXU5NrGd9rQd2iJQTsHV1YeIiGeJi9uMr28caWkzWL8+jqKi/1odTakz5unq6bC5m7QQqDbH27sPMTFf0K/fv6muLmDjxrNJTr6Ryso8q6Mp1SJpIVBtkojQpcskhg7dTvfuj7Bv34esW9ebrKyXqG3gNoVKOTMtBKpNs9m86dnzaYYM2UL79iNIS7uH9esHUVi4yupoSrUYWgiUU2jXrjcxMf/hrLOWUF1dRFLSeWzbdj0VFblWR1PKcloIlNMQEQICfsfQocn06PEn8vP/zU8/RZGZ+QK1tVVWx1PKMloIlNOx2doRHv5XhgzZSocO55Cefj+JibHs3fsvamoOWx1PqWanhUA5rXbtIunf/1Oioz/GmBq2b7+BtWtDSUu7n7KyFKvjKdVstBAopyYidO58BUOHJjNgwDf4+V1AdvbL/PRTFElJY9m379/U1lZaHVMph9KfXCpFXUHw8zsfP7/zqajYy96975CT8wbbtl2Nm1sgQUG3EBR0G15eYVZHVarJ6RQTSh2HMTUcOPAFOTlzKSj4FDB06nQxwcF34O9/sc50qlqVE00xoS0CpY5DxIa//8X4+19MefkecnPnkZs7jy1bLsfDoxtBQbcTFHQLHh5BVkdV6oxoi0CpU1BbW0VBwSfk5Mzl4MEvEXHF338CwcHT8fMbg4gOu6mWSVsESjURFxc3AgImEhAwkbKyNHJz3yQ3dz779y/GyyuSoKBpdO06FXf3zlZHVarRtEWg1BmqqSln//4l5OTMoahoNSLuBAT8nuDg6XToMErv/ataBG0RKOVANpsngYGTCQycTGnpFnJz32Dv3n+yb18C7dqdRXDwdLp2nYKraweroyrVIO3QVKoJ+fhE06vXK4wcmUNU1DxsNi/S0mawZk0wO3bcRknJeqsjKnUMhxYCERkvIjtEJE1EZjXw+lQRyReRJPu/Wx2ZR6nmYrN5ExR0C4MH/8ygQT8TGDiZvLwPWL8+jvXrh5Cb+zY1NYesjqkU4MAxAqm7yDoFuBDIAn4GrjPGbKu3zlQgzhhzV2P3q2MEqrWqri4iL+99srPnUFa2FZutPV273kBQ0DR8fKKtjqfaOKtuVTkUSDPGZBhjKoEFwAQHHk+pFs3VtQMhIXcyZMhmBg5cTefOV5CT8yaJif3ZuPEc8vISqKkptzqmckKOLAQhQGa951n2Zb91lYhsEpFFItKtoR2JyO0ikigiifn5+Y7IqlSzERE6dBhF377/YsSIbCIinqOyci/Jydezdm0o6ekPc/jwTqtjKidi9WDxJ0CYMSYG+BJ4r6GVjDFvGmPijDFxAQEBzRpQKUdyd+9Mt273M3ToDgYM+IqOHc8jM/N51q2LYNOmyygo+Axjaq2Oqdo4R14+mg3U/4Yfal92lDGmoN7TecDfHJhHqRZLxAU/v7H4+Y2lvDyL3Ny3yM19k82bL8XTM5zg4DsICroZNzd/q6OqNsiRLYKfgV4iEi4i7sC1wPL6K4hI/UlargCSHZhHqVbB0zOU8PDHGT58N/36LcTDoxsZGQ+xZk0IyclTKS7+yeqIqo1xWIvAGFMtIncBKwEbMN8Ys1VEngASjTHLgZkicgVQDRwApjoqj1KtjYuLO126XE2XLldTWrqFnJw55OX9k7y89/D1jSM4+A906XItNpuX1VFVK6dTTCjVilRXF9svQX2NsrJtuLr60bXrzQQHT6ddu0ir46kW7ESXj2ohUKoVMsZQVLSK7OzX2b9/CcZU4+c3jpCQO/H3v0TvlaCOoXMNKdXGiAgdO55Hx47nUVGRQ27uPHJy3mDLlivw8OhBcPB0goJuwd1dr7JTJ6ctAqXaiLp7JSwnO/t1Cgu/OToLakjInbRvP1xnQXVy2iJQygnU3SvhKgICruLQoWRycuawd+977NuXgI9PLMHBfyAwcDI2m7fVUVULY/UPypRSDuDt3ZdevV5mxIhseveeizE1pKTczpo1IaSm3kNZ2Q6rI6oWRLuGlHICxhiKi9eQnf0a+fmLMKYKP78LCA7+A/7+l+Piop0DbZ12DSnl5I7Mb9ShwygqK/9Bbu7b5OTMZevWiXh4hBIUNI2goFvx8OhqdVRlAW0RKOWkamurOXBgBdnZr3Pw4BeI1I0xBAf/gQ4dztbB5TZGWwRKqWO4uLjSufMEOneeQFlZCjk5c9m79x327VuAt3cM3brdT5cu1+Li4m51VOVgOlislKJdu95ERr5gH1x+C6hl+/Yb+fHHnuzZ83eqq4usjqgcSAuBUuoom60dwcG3Ehe3if79/0O7dn3IyHiItWu7kZb2AOXlmSffiWp1tBAopY4hIvj7jyc29isGD16Pv/9lZGW9yLp1PUlOnkJp6S9WR1RNSAuBUuqEfH0H0a/fBwwfnk5IyAz2719GYmIsv/xyEQcOfElru+BEHUsLgVKqUTw9exAZ+QLDh+8hPPz/OHRoC5s2XURi4kD27n2f2toqqyOq06SFQCl1Stzc/OjRYxbDh+8kKmo+xlSxffsU1q3rSWbm81RXF1sdUZ0iLQRKqdPi4uJBUNBNDBmyhf79V+Dl1Yv09AdYu7Yb6ekPUV6eZXVE1UhaCJRSZ6RuYPkSYmO/YdCgn/H3v4TMzBdYty6c5OQbKC3dZHVEdRJaCJRSTaZ9+zj69fuQYcPSCA6+k/z8JSQmDuCXX8Zz4MBXOrDcQmkhUEo1OS+vMHr1epERI/YQHv40hw79wqZNF7J+/SDy8hJ0YLmF0UKglHIYN7dO9OjxCMOH7yIq6m1qaytITr6edesiyMx8QQeWWwgtBEoph6sbWL7ZPrD8KZ6ePUlPv5+1a7uTnv4wFRXZVkd0aloIlFLNRsQFf/9LGTjwOwYN+olOncaRmfkcP/4YTnLyVEpLt1gd0SlpIVBKWaJ9+yGcddZC+8DydPLz/01iYn82bbqYgwe/0YHlZqSFQCllKS+vcPttNTMJD3+SkpKN/PLLWNavj2Pfvo8wptbqiG2eFgKlVItQN7D8KMOH76J377eorS1j27ZrSEwcQH7+Yi0IDqSFQCnVothsngQH38qQIVvp128BxlSzdeskEhMHsX//x9pl5ABaCJRSLZKIC126XMOQIVvo2/d9amvL2LLlStavj6OgYIUWhCakhUAp1aKJ2AgMjGfIkG1ERb1DdfVBNm++jA0bhnPgwEotCE1AC4FSqlVwcXElKGgqQ4fuICpqHpWVeWzaNJ6NG8/m4MGvtSCcAYcWAhEZLyI7RCRNRGadYL2rRMSISJwj8yilWj8XFzeCgm5h2LAUeveeS0XFHn755QKSkkZTWPi91fFaJYcVAhGxAa8BFwP9gOtEpF8D6/kCdwPrHJVFKdX2uLi4Exw8jWHD0ujV61UOH04lKWk0SUljKSxcbXW8VsWRLYKhQJoxJsMYUwksACY0sN5fgWeBcgdmUUq1US4uHoSE3MmwYelERr7IoUNbSUo6h19+GUdR0Y9Wx2sVHFkIQoDMes+z7MuOEpFBQDdjzIoT7UhEbheRRBFJzM/Pb/qkSqlWz2bzIjT0boYPzyAi4jlKSzewceMINm26lOLiRKvjtWiWDRaLiAvwAnD/ydY1xrxpjIkzxsQFBAQ4PpxSqtWy2drRrdv9DBu2k549n6G4+Ec2bBjC5s0TKCnZaHW8FsmRhSAb6Fbveah92RG+QDTwnYjsAoYDy3XAWCnVFFxdfeje/WGGD99JePiTFBWtYv36QWzZchWlpZutjteiOLIQ/Az0EpFwEXEHrgWWH3nRGFNkjOlsjAkzxoQBPwJXGGO0DaeUajKuru3p0eNRhg3bSVjYbA4e/IrExBi2br2GQ4e2WR2vRXBYITDGVAN3ASuBZOAjY8xWEXlCRK5w1HGVUqohbm4dCQv7C8OH76JHjz9x4MBn/PxzNNu2TaasbIfV8Swlre1HGHFxcSYxURsNSqkzU1m5n6ys58nKepna2nICA+Pp0eMx2rWLtDqaQ4jIemNMg13v+stipZRTcnfvTM+e/8fw4Tvp1u0+8vMX8dNPfdi+/RYOH95pdbxmpYVAKeXU3N27EBHxd4YNyyA0dAZ5eQn89FNvduyYRnn5HqvjNQstBEopBXh4dCUy8h8MH55OcPB09u59l3XrIklJ+QPl5butjudQWgiUUqoeD48QevV6hWHD0ggKuoXc3Hn8+GNPtmy5isLC79vk5HZaCJRSqgGent3o3XsOw4al0b37QxQWfkdS0mgSEweQkzOPmpoyqyM2GS0ESil1Ap6e3enZ8/8YMSKLqKh5gJCSchtr13YjPf3hNtFtpIVAKaUawWbzIijoFuLikoiN/Z6OHc8nM/O5o91GBw9+12q7jVytDqCUUq2JiNCx47l07Hgu5eV7yMmZQ07Om+zfvwRv7/6EhMwkMHAyNls7q6M2mrYIlFLqNB3bbeRi7zYKbVXdRloIlFLqDP2v22gjsbHf4+c3lszM5+3dRhNbfLeRdg0ppVQTabjb6C32719q7zaaQWBgfIvrNtIWgVJKOcD/uo0yiYp6m7puo9tbZLeRFgKllHKgum6jm+3dRqtaZLeRdg0ppVQzqOs2OoeOHc9pcd1G2iJQSqlmduJuo4c4fHhXs+bRQqCUUhZpuNvoBdati7B3G33bLN1G2jWklFIW+3W3UWa9H6ktxds72v4jNcd1G2mLQCmlWhBPz2707Pm0vdtoPmA72m2Ul/ehQ46pLQKllGqB6rqNbqJr16kUFa0mO/sVPD3DHXIsLQRKKdWC1e82chTtGlJKKSenhUAppZycFgKllHJyWgiUUsrJaSFQSiknp4VAKaWcnBYCpZRycloIlFLKyYnV82CfKhHJB1rOHR1OT2dgv9UhWhA9H/+j5+LX9Hz82pmcjx7GmICGXmh1haAtEJFEY0yc1TlaCj0f/6Pn4tf0fPyao86Hdg0ppZST00KglFJOTguBNd60OkALo+fjf/Rc/Jqej19zyPnQMQKllHJy2iJQSiknp4VAKaWcnBaCZiQi3UTkWxHZJiJbReRuqzNZTURsIrJRRD61OovVRKSjiCwSke0ikiwiI6zOZCURudf+38kWEflQRDytztRcRGS+iOwTkS31lnUSkS9FJNX+16+pjqeFoHlVA/cbY/oBw4E7RaSfxZmsdjeQbHWIFuIl4HNjTB9gAE58XkQkBJgJxBljogEbcK21qZrVu8D43yybBXxtjOkFfG1/3iS0EDQjY0yuMWaD/XEJdf+hh1ibyjoiEgpcCsyzOovVRKQDcC7wNoAxptIYU2htKsu5Al4i4gq0A3IsztNsjDGrgAO/WTwBeM/++D3gyqY6nhYCi4hIGDAQWGdtEku9CDwE1FodpAUIB/KBd+xdZfNExNvqUFYxxmQDzwF7gFygyBjzhbWpLBdojMm1P94LBDbVjrUQWEBEfIDFwD3GmGKr81hBRC4D9hlj1ludpYVwBQYBc4wxA4FDNGHTv7Wx939PoK5ABgPeInK9talaDlN33X+TXfuvhaCZiYgbdUUgwRizxOo8FhoFXCEiu4AFwBgRed/aSJbKArKMMUdaiIuoKwzO6gJgpzEm3xhTBSwBRlqcyWp5IhIEYP+7r6l2rIWgGYmIUNcHnGyMecHqPFYyxjxijAk1xoRRNwj4jTHGab/xGWP2ApkiEmVfNBbYZmEkq+0BhotIO/t/N2Nx4sFzu+XAjfbHNwIfN9WOtRA0r1HAFOq+/SbZ/11idSjVYswAEkRkExALPG1xHsvYW0aLgA3AZuo+q5xmugkR+RBYC0SJSJaI3AI8A1woIqnUtZieabLj6RQTSinl3LRFoJRSTk4LgVJKOTktBEop5eS0ECillJPTQqCUUk5OC4FSdiJSU++y3iQRabJf9opIWP2ZJJVqSVytDqBUC3LYGBNrdQilmpu2CJQ6CRHZJSJ/E5HNIvKTiETal4eJyDcisklEvhaR7vblgSKyVER+sf87MjWCTUTess+x/4WIeNnXn2m/R8UmEVlg0dtUTkwLgVL/4/WbrqFr6r1WZIzpD7xK3aypAK8A7xljYoAE4GX78peB740xA6ibL2irfXkv4DVjzFlAIXCVffksYKB9P9Md9eaUOh79ZbFSdiJSaozxaWD5LmCMMSbDPmngXmOMv4jsB4KMMVX25bnGmM4ikg+EGmMq6u0jDPjSflMRRORhwM0Y86SIfA6UAsuAZcaYUge/VaV+RVsESjWOOc7jU1FR73EN/xujuxR4jbrWw8/2G7Eo1Wy0ECjVONfU+7vW/ngN/7t9Yjzwg/3x18AdcPSezB2Ot1MRcQG6GWO+BR4GOgDHtEqUciT95qHU/3iJSFK9558bY45cQupnnxW0ArjOvmwGdXcUe5C6u4vdZF9+N/CmfcbIGuqKQi4NswHv24uFAC/rLSpVc9MxAqVOwj5GEGeM2W91FqUcQbuGlFLKyWmLQCmlnJy2CJRSyslpIVBKKSenhUAppZycFgKllHJyWgiUUsrJ/T8uchT0we+o2gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rTo1-5vIC1wV"
      },
      "source": [
        "# **Comparison of RMSProp, Mini Batch Gradient Descent and Adam Optimizer**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-xePivvuD_ZG"
      },
      "source": [
        "Accuracy of Siamese Network using different optimizers are:\n",
        "*  Test Accuracy using **RMSProp optimizer: 96.189%**\n",
        "*  Test Accuracy using **Mini Batch Gradient Descent optimizer: 88.615%**\n",
        "*  Test Accuracy using **Adam optimizer: 97.289%**\n",
        "\n",
        "\n",
        "As we can see that Adam and RMSProp optimizer gave around same accuracies with Adam giving slightly higher. The update rule for Adam is very similar to RMSProp, except we look at the cumulative history of gradients which speeds up the process in case of Adam.\n",
        "\n",
        "\n",
        "The **best optimizer I think is Adam Optimizer** and the reasons are :\n",
        "\n",
        "* Adam combines the best properties of RMSProp and AdaGrad to work well even with noisy or sparse datasets.\n",
        "* Adam is well suited for problems that are large in terms of data and/or parameters.\n",
        "* Adam is Computationally efficient and has Little memory requirements.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RVxIVZV2IHO1"
      },
      "source": [
        "# **Hyper Parameter Optimization on Siamese Network with Adam optimizer**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ILXCR7eKIGmT"
      },
      "source": [
        "opt = ks.optimizers.Adam(learning_rate=0.0005, beta_1=0.888, beta_2=0.8888, epsilon=1e-8)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bq9xuNyAIdl5",
        "outputId": "4164639b-1f5d-4462-b669-fba754f910f5"
      },
      "source": [
        "model.compile(loss = contrastive_loss, optimizer=opt, metrics=[accuracy])\n",
        "model.summary()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"model_102\"\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_155 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "input_156 (InputLayer)          [(None, 784)]        0                                            \n",
            "__________________________________________________________________________________________________\n",
            "model_101 (Functional)          (None, 256)          91584       input_155[0][0]                  \n",
            "                                                                 input_156[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "lambda_149 (Lambda)             (None, 1)            0           model_101[0][0]                  \n",
            "                                                                 model_101[1][0]                  \n",
            "==================================================================================================\n",
            "Total params: 91,584\n",
            "Trainable params: 91,584\n",
            "Non-trainable params: 0\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dGi6g5zDIdY9",
        "outputId": "9386f85c-8615-4878-c5fc-45e0fcbc1c6d"
      },
      "source": [
        "history = model.fit([pairs_train[:,0], pairs_train[:,1]], labels_train[:], batch_size=128, epochs=10, validation_data =([pairs_test[:, 0], pairs_test[:, 1]], labels_test))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/10\n",
            "938/938 [==============================] - 8s 8ms/step - loss: 0.1139 - accuracy: 0.8523 - val_loss: 0.0468 - val_accuracy: 0.9489\n",
            "Epoch 2/10\n",
            "938/938 [==============================] - 7s 8ms/step - loss: 0.0386 - accuracy: 0.9604 - val_loss: 0.0319 - val_accuracy: 0.9655\n",
            "Epoch 3/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0237 - accuracy: 0.9757 - val_loss: 0.0274 - val_accuracy: 0.9669\n",
            "Epoch 4/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0167 - accuracy: 0.9828 - val_loss: 0.0256 - val_accuracy: 0.9701\n",
            "Epoch 5/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0129 - accuracy: 0.9868 - val_loss: 0.0255 - val_accuracy: 0.9673\n",
            "Epoch 6/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0107 - accuracy: 0.9890 - val_loss: 0.0243 - val_accuracy: 0.9705\n",
            "Epoch 7/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0087 - accuracy: 0.9912 - val_loss: 0.0228 - val_accuracy: 0.9726\n",
            "Epoch 8/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0076 - accuracy: 0.9922 - val_loss: 0.0237 - val_accuracy: 0.9689\n",
            "Epoch 9/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0067 - accuracy: 0.9934 - val_loss: 0.0236 - val_accuracy: 0.9701\n",
            "Epoch 10/10\n",
            "938/938 [==============================] - 7s 7ms/step - loss: 0.0060 - accuracy: 0.9941 - val_loss: 0.0233 - val_accuracy: 0.9712\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5OX0_VxRIdNO"
      },
      "source": [
        "y_pred_te = model.predict([pairs_test[:, 0], pairs_test[:, 1]])\n",
        "te_acc = compute_accuracy(labels_test, y_pred_te)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HGi_XLdmIj6R",
        "outputId": "9ababef7-992d-400a-b9d1-6639086806ae"
      },
      "source": [
        "print(\"Accuracy on test set:\", te_acc*100)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Accuracy on test set: 97.59555\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "_OsDKAubIjx9",
        "outputId": "28e7621d-3532-40e8-9337-c4e06853f4ad"
      },
      "source": [
        "history_dict = history.history\n",
        "loss_values = history_dict['loss']\n",
        "val_loss_values = history_dict['val_loss']\n",
        "epochs = range(1, (len(history.history['val_accuracy']) + 1))\n",
        "plt.plot(epochs, loss_values, 'y', label='Training loss')\n",
        "plt.plot(epochs, val_loss_values, 'g', label='Test loss')\n",
        "plt.title('Training and Test loss')\n",
        "plt.xlabel('Epochs')\n",
        "plt.ylabel('Loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5dn/8c81k2SykmTCJouETRSBBImgoIggStSCWisoKnZT21q0dtG2T1vK0/bR5+fTqq2tonXFFiyKUgWx7hsugIAgooABooCQjezr9ftjDjEJE5KQmZxJcr1fr3nlLPc555ppnS/n3PecI6qKMcYY05TH7QKMMcZEJgsIY4wxQVlAGGOMCcoCwhhjTFAWEMYYY4KygDDGGBOUBYTpVERklYjMC3VbN4lIjoic43YdEFm1GPdFuV2A6fpEpKTBbDxQCdQ689ep6uOt3ZeqZoejbSQSkVXAmc6sD1CgyplfrKrXt3F/C4BhqnplyIo0XZoFhAk7VU08PC0iOcB3VPXFpu1EJEpVazqytkjWMOBE5GEgV1X/y72KTHdjl5iMa0RkiojkisgtIrIPeEhEUkXkWRE5ICIFzvSABtu8KiLfcaavEZE3ReQOp+1nIpJ9jG0Hi8jrIlIsIi+KyD0isriZultT43+LyFvO/l4QkZ4N1l8lIrtEJE9EfnmMn92FIrJBRApF5G0RGdNg3S0i8rlz7G0iMk1EZgC/AGaLSImIbGzFMXwicqeIfOG87hQRn7Oup/O+C0UkX0TeEBFPc8c/lvdo3GcBYdzWF/ADg4BrCfx/8iFn/nigHPjLUbafAGwDegL/C/xdROQY2v4DeA9IAxYAVx3lmK2p8Qrgm0BvIAb4CYCIjAT+5uy/n3O8AbSBiIwFHgSuc7a/D1jhfKGPAG4ATlXVJOA8IEdVnwf+ACxV1URVzWjFoX4JnAZkAhnAeODwGcyPgVygF9CHQPhoc8dvy/szkcMCwritDviNqlaqarmq5qnqk6papqrFwO+Bs46y/S5VvV9Va4FHgOMIfGG1uq2IHA+cCvxaVatU9U1gRXMHbGWND6nqJ6paDjxB4EsW4FLgWVV9XVUrgV85n0FbXAvcp6rvqmqtqj5CoF/nNAJ9Oz5gpIhEq2qOqu5o4/4PmwssVNUvVfUA8Fu+Cs5qAp/fIFWtVtU3NHBjt1Ae37jMAsK47YCqVhyeEZF4EbnPuQRzCHgdSBERbzPb7zs8oaplzmRiG9v2A/IbLAPY01zBraxxX4PpsgY19Wu4b1UtBfKaO1YzBgE/di7vFIpIITAQ6Keq24GbCJwFfSkiS0SkXxv3f1g/YFeD+V3OMoD/B2wHXhCRnSJyq/N+Qnl84zILCOO2prcT/jEwApigqj2Ayc7y5i4bhcJewC8i8Q2WDTxK+/bUuLfhvp1jprWtXPYAv1fVlAaveFX9J4Cq/kNVzyAQJArc7mzX1ls3f+Hs47DjnWWoarGq/lhVhwAzgZsP9zUc5fimk7GAMJEmicA1/UIR8QO/CfcBVXUXsBZYICIxInI68LUw1bgMuFBEzhCRGGAhbf/v8H7gehGZIAEJInKBiCSJyAgRmep0Jlc4dR6+hLUfSD/cmdwK/wT+S0R6OZ3svwYWQ30n+TCnD6eIwKWluhaObzoZCwgTae4E4oCDwDvA8x103LnA6QQu9/wOWErgun4wx1yjqm4BfkCgU3wvUECgs7fVVHUt8F0CHeMFBC71XOOs9gG3ObXtI9BJ/nNn3b+cv3kisr4Vh/odgeDcBHwIrHeWAQwHXgRKgDXAX1X1lRaObzoZsQcGGXMkEVkKfKyqYT+DMSZS2RmEMYCInCoiQ0XE4/xmYBbwtNt1GeMm+yW1MQF9gacIdBjnAt9T1Q/cLckYd9klJmOMMUHZJSZjjDFBdZlLTD179tT09HS3yzDGmE5l3bp1B1W1V7B1YQ0Ip7PvLsALPKCqtzVZ7wMeBcYRGF44W1VzRCQaeAA4xanxUVX9n6MdKz09nbVr14bhXRhjTNclIruaWxe2S0zObQfuAbKBkcDlzo3KGvo2UKCqw4A/8dUvLr8B+FR1NIHwuE5E0sNVqzHGmCOFsw9iPLBdVXeqahWwhMDQwYZmEbhpGgR+YTrN+WWmAgkiEkXgB0lVwKEw1mqMMaaJcAZEfxrf8CzXWRa0jfOgmCICwwyXAaUEfmm6G7hDVfPDWKsxxpgmIrWTejyBe7v0A1KBN0TkRVXd2bCRiFxL4NbHHH/88R1epDGmdaqrq8nNzaWioqLlxiYsYmNjGTBgANHR0a3eJpwB8TmN74g5wFkWrE2uczkpmUBn9RXA86paTeCWwW8BWUCjgFDVRcAigKysLPtBhzERKjc3l6SkJNLT02n+eU4mXFSVvLw8cnNzGTx4cKu3C+clpveB4RJ4lGMMMIcjH8KyApjnTF8KvOw8dGQ3MBVARBIIPAjl4zDWaowJo4qKCtLS0iwcXCIipKWltfkMLmwB4fQp3ACsBrYCT6jqFhFZKCIznWZ/B9JEZDtwM3Crs/weIFFEthAImodUdVO4ajXGhJ+Fg7uO5fMPax+Eqq4EVjZZ9usG0xUEhrQ23a4k2PJwqKjYzRdf/I3+/W/A52vah26MMd1Xt7/VRm1tMbt330Ze3sqWGxtjOqW8vDwyMzPJzMykb9++9O/fv36+qqrqqNuuXbuW+fPnt3iMiRMnhqTWV199lQsvvDAk+2qvSB3F1GHi40fi8w0kP38V/fp91+1yjDFhkJaWxoYNGwBYsGABiYmJ/OQnP6lfX1NTQ1RU8K/DrKwssrKyWjzG22+/HZpiI0i3P4MQEfz+bAoKXqSurtrtcowxHeSaa67h+uuvZ8KECfzsZz/jvffe4/TTT2fs2LFMnDiRbdu2AY3/Rb9gwQK+9a1vMWXKFIYMGcLdd99dv7/ExMT69lOmTOHSSy/lxBNPZO7cuRy+a/bKlSs58cQTGTduHPPnz2/xTCE/P5+LLrqIMWPGcNppp7FpU6Ar9rXXXqs/Axo7dizFxcXs3buXyZMnk5mZyahRo3jjjTfa/Rl1+zMIAL9/Bnv3LuLQobdJSTnL7XKM6dI+/fQmSko2hHSfiYmZDB9+Z5u3y83N5e2338br9XLo0CHeeOMNoqKiePHFF/nFL37Bk08+ecQ2H3/8Ma+88grFxcWMGDGC733ve0f8tuCDDz5gy5Yt9OvXj0mTJvHWW2+RlZXFddddx+uvv87gwYO5/PLLW6zvN7/5DWPHjuXpp5/m5Zdf5uqrr2bDhg3ccccd3HPPPUyaNImSkhJiY2NZtGgR5513Hr/85S+pra2lrKyszZ9HUxYQQGrqNESiyMtbZQFhTDfyjW98A6/XC0BRURHz5s3j008/RUSorg5+ReGCCy7A5/Ph8/no3bs3+/fvZ8CAAY3ajB8/vn5ZZmYmOTk5JCYmMmTIkPrfIVx++eUsWrToqPW9+eab9SE1depU8vLyOHToEJMmTeLmm29m7ty5XHLJJQwYMIBTTz2Vb33rW1RXV3PRRReRmZnZrs8GLCAAiIrqQXLyGeTnP8/Qobe1vIEx5pgdy7/0wyUhIaF++le/+hVnn302y5cvJycnhylTpgTdxufz1U97vV5qamqOqU173HrrrVxwwQWsXLmSSZMmsXr1aiZPnszrr7/Oc889xzXXXMPNN9/M1Vdf3a7jdPs+iMP8/hmUlm6ksvILt0sxxrigqKiI/v0DQ90ffvjhkO9/xIgR7Ny5k5ycHACWLl3a4jZnnnkmjz/+OBDo2+jZsyc9evRgx44djB49mltuuYVTTz2Vjz/+mF27dtGnTx+++93v8p3vfIf169e3u2YLCIffnw1Afv7zLldijHHDz372M37+858zduzYkP+LHyAuLo6//vWvzJgxg3HjxpGUlERycvJRt1mwYAHr1q1jzJgx3HrrrTzySODm13feeSejRo1izJgxREdHk52dzauvvkpGRgZjx45l6dKl3Hjjje2uucs8kzorK0vb88AgVWXNmgEkJ0/i5JOfCGFlxpitW7dy0kknuV2G60pKSkhMTERV+cEPfsDw4cP50Y9+1GHHD/a/g4isU9Wg43jtDMIRGO46g4KC/1BXF/p/PRhjzP33309mZiYnn3wyRUVFXHfddW6XdFTWSd2A35/Nvn0PcujQO6SknOF2OcaYLuZHP/pRh54xtJedQTSQmnoO4LV+CGOMwQKikejoFJKTJ5Kfv8rtUowxxnUWEE34/TMoKVlPZeU+t0sxxhhXWUA0cXi4a0HBCy5XYowx7rKAaCIxMZOYmL52mcmYLqQ9t/uGwI/Umrtb68MPP8wNN9wQ6pIjgo1iakJESE09j7y8f6Nai4jX7ZKMMe3U0u2+W/Lqq6+SmJgYsmc+dBZ2BhFEWlo2NTX5HDr0vtulGGPCZN26dZx11lmMGzeO8847j7179wJw9913M3LkSMaMGcOcOXPIycnh3nvv5U9/+hOZmZlHvY12Tk4OU6dOZcyYMUybNo3du3cD8K9//YtRo0aRkZHB5MmTAdiyZQvjx48nMzOTMWPG8Omnn4b/TbeRnUEEkZo6HfCQn7+K5OTT3C7HmC7lpudvYsO+0N7uO7NvJnfOaP1NAFWVH/7whzzzzDP06tWLpUuX8stf/pIHH3yQ2267jc8++wyfz0dhYSEpKSlcf/31rTrr+OEPf8i8efOYN28eDz74IPPnz+fpp59m4cKFrF69mv79+1NYWAjAvffey4033sjcuXOpqqqitra2XZ9BOIT1DEJEZojINhHZLiK3BlnvE5Glzvp3RSTdWT5XRDY0eNWJSPvvXdtK0dF+evSYYP0QxnRRlZWVbN68menTp5OZmcnvfvc7cnNzARgzZgxz585l8eLFzT5lrjlr1qzhiiuuAOCqq67izTffBGDSpElcc8013H///fVBcPrpp/OHP/yB22+/nV27dhEXFxfCdxgaYTuDkMDF+3uA6UAu8L6IrFDVjxo0+zZQoKrDRGQOcDswW1UfBx539jMaeFpVQ/tPjhb4/dnk5PyGqqoDxMT06shDG9OlteVf+uGiqpx88smsWbPmiHXPPfccr7/+Ov/+97/5/e9/z4cfftju49177728++67PPfcc4wbN45169ZxxRVXMGHCBJ577jnOP/987rvvPqZOndruY4VSOM8gxgPbVXWnqlYBS4BZTdrMAh5xppcB00REmrS53Nm2QwWGu6oNdzWmC/L5fBw4cKA+IKqrq9myZQt1dXXs2bOHs88+m9tvv52ioiJKSkpISkqiuLi4xf1OnDiRJUsCX1ePP/44Z555JgA7duxgwoQJLFy4kF69erFnzx527tzJkCFDmD9/PrNmzap/nGgkCWdA9Af2NJjPdZYFbaOqNUARkNakzWzgn8EOICLXishaEVl74MCBkBR9WFLSKURH9yIvzy4zGdPVeDweli1bxi233EJGRgaZmZm8/fbb1NbWcuWVVzJ69GjGjh3L/PnzSUlJ4Wtf+xrLly9vsZP6z3/+Mw899BBjxozhscce46677gLgpz/9KaNHj2bUqFFMnDiRjIwMnnjiCUaNGkVmZiabN29u98N9wiFst/sWkUuBGar6HWf+KmCCqt7QoM1mp02uM7/DaXPQmZ8APKCqo1s6Xntv9x3M1q1XkZ//PBMn7kfEBnwZc6zsdt+RIZJu9/05MLDB/ABnWdA2IhIFJAN5DdbPoZmzh47g92dTXX2Q4uJ1bpVgjDGuCWdAvA8MF5HBIhJD4Mt+RZM2K4B5zvSlwMvqnNJI4J/sl+FC/8NhqannAmKjmYwx3VLYAsLpU7gBWA1sBZ5Q1S0islBEZjrN/g6kich24Gag4VDYycAeVd0ZrhpbEhPTk6SkU+3238aEQFd5emVndSyff1h/KKeqK4GVTZb9usF0BfCNZrZ9FXD9V2p+fza7dv031dX5REf73S7HmE4pNjaWvLw80tLSOHKgogk3VSUvL4/Y2Ng2bWe/pG6B3z+DXbt+S37+C/TpM8ftcozplAYMGEBubi6hHm1oWi82NpYBAwa0aRsLiBb06HEqUVFp5Oc/bwFhzDGKjo5m8ODBbpdh2sjGbrZAxIvffy75+c+jWud2OcYY02EsIFrB759BdfV+Sko69G4fxhjjKguIVvD7zwOw4a7GmG7FAqIVYmL6kJg4zoa7GmO6FQuIVvL7Z1BUtIbq6kK3SzHGmA5hAdFKaWnZQC0FBf9xuxRjjOkQFhCtlJQ0gaioFLvMZIzpNiwgWsnjiSI1dboz3NVuGWCM6fosINrA78+mquoLSksj78EexhgTahYQbeD3zwCwy0zGmG7BAqINfL7jSEjIsKfMGWO6BQuINkpLy+bQobeoqTnkdinGGBNWFhBt5Pdno1pDQcFLbpdijDFhZQHRRj16nI7X28Nuu2GM6fIsINrI44kmNfUc8vNX2XBXY0yXZgFxDPz+bCorcykr+8jtUowxJmzCGhAiMkNEtonIdhG5Nch6n4gsdda/KyLpDdaNEZE1IrJFRD4UkbY9Ky+MDg93tdFMxpiuLGwBISJe4B4gGxgJXC4iI5s0+zZQoKrDgD8BtzvbRgGLgetV9WRgClAdrlrbKjZ2AAkJo6wfwhjTpYXzDGI8sF1Vd6pqFbAEmNWkzSzgEWd6GTBNAk80PxfYpKobAVQ1T1Vrw1hrm/n92RQVvUFNTYnbpRhjTFiEMyD6A3sazOc6y4K2UdUaoAhIA04AVERWi8h6EflZsAOIyLUislZE1nb0w9ADw12rKSx8uUOPa4wxHSVSO6mjgDOAuc7fi0VkWtNGqrpIVbNUNatXr14dWmBy8iS83kS7zGSM6bLCGRCfAwMbzA9wlgVt4/Q7JAN5BM42XlfVg6paBqwETgljrW3m8cSQkjLN7u5qjOmywhkQ7wPDRWSwiMQAc4AVTdqsAOY505cCL2vg23Y1MFpE4p3gOAuIuDGlaWnZVFTkUFa2ze1SjDEm5MIWEE6fwg0Evuy3Ak+o6hYRWSgiM51mfwfSRGQ7cDNwq7NtAfBHAiGzAVivqs+Fq9Zj9dXdXe0ykzGm65GucnkkKytL165d2+HHfe+9kfh8A8nIWN3hxzbGmPYSkXWqmhVsXaR2Uncafn82hYWvUVtb5nYpxhgTUhYQ7eT3z0C1ksLCV9wuxRhjQsoCop1SUibj8cTbU+aMMV2OBUQ7eTw+UlOn2n2ZjDFdjgVECPj9M6io2EFZ2adul2KMMSFjARECfn82gF1mMsZ0KRYQIRAXN4S4uBPs9xDGmC7FAiJE/P4ZFBa+Qm1tudulGGNMSFhAhIjfn01dXQWFha+5XYoxxoSEBUSIpKSchccTa/0QxpguwwIiRLzeOFJSplg/hDGmy7CACCG/P5vy8k8oL9/pdinGGNNuFhAhZMNdjTFdiQVECMXFDSM2dohdZjLGdAkWECEkIvj92RQUvExtbYXb5RhjTLtYQIRYWlo2dXVlFBW96XYpxhjTLhYQIZaSMgURn11mMsZ0ehYQIeb1JpCSMtkCwhjT6VlAhIHfn01Z2VYqKna5XYoxxhyzsAaEiMwQkW0isl1Ebg2y3iciS53174pIurM8XUTKRWSD87o3nHWGmg13NcZ0BWELCBHxAvcA2cBI4HIRGdmk2beBAlUdBvwJuL3Buh2qmum8rg9XneEQHz8Cn2+QPUTIGNOphfMMYjywXVV3qmoVsASY1aTNLOARZ3oZME1EJIw1dQgRIS0tm8LCl6irq3K7HGOMOSbhDIj+wJ4G87nOsqBtVLUGKALSnHWDReQDEXlNRM4MdgARuVZE1orI2gMHDoS2+nby+7OprS2hqOgtt0sxxphjEqmd1HuB41V1LHAz8A8R6dG0kaouUtUsVc3q1atXhxd5NCkpZyMSbaOZjDGdVjgD4nNgYIP5Ac6yoG1EJApIBvJUtVJV8wBUdR2wAzghjLWGXFRUEsnJZ1pHtTGm0wpnQLwPDBeRwSISA8wBVjRpswKY50xfCrysqioivZxObkRkCDAc6HS3SPX7sykt/ZCKily3SzHGmDYLW0A4fQo3AKuBrcATqrpFRBaKyEyn2d+BNBHZTuBS0uGhsJOBTSKygUDn9fWqmh+uWsPF758B2HBXY0znJKrqdg0hkZWVpWvXrnW7jEZUlXfeOZ6kpAmMGrXM7XKMMeYIIrJOVbOCrYvUTuou4au7u/6Hurpqt8sxxpg2sYAAVn26ijqtC8u+/f4Z1NYe4tChNWHZvzHGhEu3D4j/7PgP5//jfP77tf8Oy/5TU89BJMr6IYwxnU63D4hzhpzD1RlXs+C1BTz50ZMh339UVA969Jhkv4cwxnQ63T4gRIT7LryP0wacxtVPX82GfRtCfgy/fwYlJRuorNwb8n0bY0y4dPuAAIiNimX57OX44/zM/OdM9pfsD+n+09IO3911dUj3a4wx4WQB4eib2Jdn5jzDwbKDXPLEJVTWVIZs3wkJY4iJOc4uMxljOhULiAZOOe4UHr7oYd7e8zbfe+57hOo3IoHhrjMoKHiBurqakOzTGGPCrVUBISIJIuJxpk8QkZkiEh3e0txx2cmX8avJv+KhDQ9x17t3hWy/fn82NTWFFBe/F7J9GmNMOLX2DOJ1IFZE+gMvAFcBD4erKLctmLKAi0+8mB+/8GNWbw9Nv0Fq6nTAa5eZjDGdRmsDQlS1DLgE+KuqfgM4OXxlucsjHh69+FFG9R7F7GWz2XZwW7v3GR2dQo8ep9lT5owxnUarA0JETgfmAs85y7zhKSkyJMYksmLOCmK8McxcMpOC8oJ27zMtLZuSknVUVYV2lJQxxoRDawPiJuDnwHLnjqxDgFfCV1ZkGJQyiKdmP8VnBZ8x58k51LSzg9nvPzzc9YVQlGeMMWHVqoBQ1ddUdaaq3u50Vh9U1flhri0inHH8Gfztgr/xwo4X+OkLP23XvhITM4mO7m39EMaYTqG1o5j+ISI9RCQB2Ax8JCLt+7bsRL59yre5ccKN3PnunTz4wYPHvB8RD37/DPLzV6NaG8IKjTEm9Fp7iWmkqh4CLgJWAYMJjGTqNu449w6mD5nO9c9ez5u73zzm/QSGu+ZTXBxZz64wxpimWhsQ0c7vHi4CVqhqNdA1njTUSlGeKJZeupTBqYO5ZOkl7CrcdUz78funAx4bzWSMiXitDYj7gBwgAXhdRAYBh8JVVKRKjUtlxZwVVNVWMWvJLEqqStq8j+joNHr0GG/9EMaYiNfaTuq7VbW/qp6vAbuAs1vaTkRmiMg2EdkuIrcGWe8TkaXO+ndFJL3J+uNFpEREftLK9xN2I3qOYOmlS/nwyw+Z9/S8Y3rQkN+fTXHx+1RVHQxDhcYYExqt7aROFpE/isha5/V/BM4mjraNF7gHyAZGApeLyMgmzb4NFKjqMOBPwO1N1v+RQJ9HRDlv2HncMf0Ontr6FL999bdt3j4w3FUpKLDhrsaYyNXaS0wPAsXAZc7rEPBQC9uMB7ar6k5VrQKWALOatJkFPOJMLwOmiYgAiMhFwGfAllbW2KFuOu0mvpn5TRa+vpB/bflXm7ZNShpHdHRPu8xkjIlorQ2Ioar6G+fLfqeq/hYY0sI2/YE9DeZznWVB26hqDVAEpIlIInALcNR/novItYfPag4cONDKtxIaIsLfLvgbEwdOZN7T81i/d30btvWQmnqeM9w1PM/CNsaY9mptQJSLyBmHZ0RkElAenpIAWAD8SVWP2gusqotUNUtVs3r16hXGcoLzRfl46rKn6Bnfk1lLZrGvZF+rt01Ly6a6+gDFxa0PFmOM6UitDYjrgXtEJEdEcoC/ANe1sM3nwMAG8wOcZUHbiEgUkAzkAROA/3WOdRPwCxG5oZW1dqg+iX1YcfkK8svzuWRp6x80lJp6LiB2mckYE7FaO4ppo6pmAGOAMao6FpjawmbvA8NFZLCIxABzgBVN2qwA5jnTlwIvO6OkzlTVdFVNB+4E/qCqf2ndW+p4mX0zefSiR1mTu4brnr2uVQ8aionpRVJSFvn5z3dAhcYY03ZteqKcqh5yflENcHMLbWuAG4DVwFbgCedGfwtFZKbT7O8E+hy2O/s7YihsZ/H1kV9nwVkLeGTjI/xxzR9btY3fn82hQ+9QXZ0f5uqMMabt5Fgfqykie1R1YMstO0ZWVpauXevu7SvqtI7Zy2bz1NanePbyZ8kenn3U9kVFa/jgg4mMHLmE3r1nd1CVxhjzFRFZp6pZwda155nU3epWG63hEQ8Pz3qYMX3GMOfJOWw9sPWo7Xv0GE9UlN8uMxljItJRA0JEikXkUJBXMdCvg2rsVBJiEnhmzjPERsUyc8lM8subv3wk4sXvP5f8/OdtuKsxJuIcNSBUNUlVewR5JalqVEcV2dkcn3w8y2cvZ3fRbmYvm33UBw35/TOoqtpHScnGDqzQGGNa1p5LTOYoJg6cyH0X3seLO1/k5tXN9+f7/TMA7DKTMSbiWECE0TWZ13DzaTfz5/f+zP3r7g/aJiamD4mJp9jvIYwxEccCIsxun3475w09j++v/D6v73o9aBu/fwZFRW9TXV3YwdUZY0zzLCDCLMoTxZJLlzA0dShff+Lr5BTmHNEmcHfXWgoLX+rw+owxpjkWEB0gJTaFFZevoKauhpn/nElxZXGj9T16nIbXm2xPmTPGRBQLiA5yQtoJLL10KVsObOGq5Vc1etCQxxOF3z+d/PxVrbpNhzHGdAQLiA507tBz+eO5f+SZbc/w61d+3Whdz54XU1X1Bdu332QhYYyJCPZbhg42f8J8PvzyQ37/xu8Z1XsUc0bNAaB378spLl5Hbu4fUa1m+PC/IGL5bYxxjwVEBxMR/nrBX9mWt41vPvNNhvmHkdUvCxFh6NA78Hhi2L37NurqqhgxYpGFhDHGNfbt44IYbwxPXvYkvRN6c9GSi9hbvBcIhMfgwX9g0KBfsW/f3/n442+iWutytcaY7soCwiW9E3qzYs4KCisKuWjpRVTUVACHQ2Ih6ekL2b//UbZuvZq6o9yqwxhjwsUCwkUZfTN47OLHeO/z9/juv7/bqHM6Pf1XDBlyG19++Q+2br2CurpqFys1xnRHFhAuu/iki1k4ZSGLNy3m9rdubxQSxx9/C0OH/h8HDvyLjz6aTV1dlS5HH2gAABhjSURBVIuVGmO6GwuICPBfk/+Ly06+jJ+/9HNG/W0U//PG/7CrcBcAAwfezLBhd3Pw4HK2bPk6dXWte+a1Mca0lwVEBBARHr3oUe694F78cX5+8fIvSL8rnSkPT+GB9Q+Q2PMqhg//G3l5z7J580XU1pa7XbIxphs45keORppIeORoqHxW8Bn/+PAfPLbpMbblbSPGG8PXTvga5w/sy4Dye+iTdg6jRj2D1xvvdqnGmE7uaI8cDWtAiMgM4C7ACzygqrc1We8DHgXGAXnAbFXNEZHxwKLDzYAFqrr8aMfqSgFxmKqybu86Fm9azD83/5MvS78kxZfAmf5SLhk6lrmTXyM6OsntMo0xnZgrASEiXuATYDqQC7wPXK6qHzVo831gjKpeLyJzgItVdbaIxANVqlojIscBG4F+qtrseM+uGBAN1dTV8OLOF1m8aTFPbV1GeU0l/eJ9XDN2PldlfosTe57odonGmE7oaAERzj6I8cB2Vd2pqlXAEmBWkzazgEec6WXANBERVS1rEAaxQNe4DtYOUZ4oZgybweJLFvPlTw9yz7Qb6O+r4ra3/h8n3XMSp95/Kne9cxf7S/a7XaoxposIZ0D0B/Y0mM91lgVt4wRCEZAGICITRGQL8CFwfbCzBxG5VkTWisjaAwcOhOEtRKbEmES+f8afee7yp/jX6VH8aOQAauuquWn1TfT/Y3+yH8/m8U2PU1pV6napxphOLGJHManqu6p6MnAq8HMRiQ3SZpGqZqlqVq9evTq+SJf16nURk09ZzqzeX7JonIcN332LWybdwkcHPuLK5VfS544+XL38al7Y8QI19mtsY0wbhTMgPgcGNpgf4CwL2kZEooBkAp3V9VR1K1ACjApbpZ1Yz54XMmrUM5SWfkTV59fzmzNv5LMbP+O1a17jitFXsGLbCs5bfB4D/zSQm1ffzPq96+124saYVglnJ3UUgU7qaQSC4H3gClXd0qDND4DRDTqpL1HVy0RkMLDH6aQeBKwh0Jl9sLnjdfVO6pbk57/I5s0ziY0dTEbGS/h8fQGoqKlg5acrWbxpMc9+8izVddWc1PMkrhxzJXNHz2VQyiCXKzfGuMnNYa7nA3cSGOb6oKr+XkQWAmtVdYVz2egxYCyQD8xR1Z0ichVwK1AN1AELVfXpox2ruwcEQGHha2zadAE+3wAyM1/G5+vXaH1+eT7/2vIvFn+4mDd3vwnA5EGTuXL0lVw68lJS41LdKNsY4yLXAqIjWUAEFBa+yYcfZhMT05eMjJeJjR0YtF2wH+NdeMKFXDn6Ss4ffj6+KF8HV26McYMFRDdTVPQOmzadR3R0GhkZLxMXl95sW1Vl/d719T/G21+6nxhvDENThzKi5whGpAVeJ6SdwIieI+gZ37Pj3ogxJuwsILqhQ4fWsmnTdLzeHmRmvkJc3JAWtzn8Y7xXPnuFT/I/YdvBbWzP3051g1uN++P8XwVG2oj6EBnqH0ps1BEDzYwxEc4CopsqLv6AjRvPweOJIzPzFeLjh7d5HzV1NeQU5rDt4DY+yfuEbXnbAq+D29hbsre+nUc8DEoeVB8YDQOkf1J/RCSUb80YEyIWEN1YSckmNm48B5EoMjJeJiEhdLfkKK4s/io0Dm6rP+v4JO8TSqu/+pFefHT8V4HhhMbh+SSf3UvKGDdZQHRzpaUfsWHDVEDJyHiJxMTw/qREVfm8+PNAeBzc1uisI6cwB21w55TjEo8LBIb/hK/6PHqOID0lnShPVFjrNMZYQBigrGwbGzZMRbWKjIwXSUzMcKWOipoKduTvOOKsY1veNvLL8+vbRXuiSYtPwyMePOJBkPppj3gQaTIfxvUpvhT6JvYN+kqMSbTLZ6ZTs4AwAJSVbWfjxqnU1paSkfEfkpJOcbukRg6WHWx01pFfno+qUqd11FH31bTzUprMh2F9bV0tBRUF7C/ZT63WHlFzfHR8o8Dok9AnaJD0SehjQ4dNRLKAMPXKyz9jw4azqa0tYsyY1fToMd7tkjqFOq0jryyPfSX7jnjtL93faD6vPC/oPlJjU5s9E2n4SotLw+vxdvA7NN2VBYRppKJiFxs2TKW6+iBjxjxPcvLpbpfUpVTVVvFl6ZdBw6Tha2/JXsqqy47Y3iteeif0Dpx5JDpnJAl9GZg8kEHJgxiUMohByYOsg9+EhAWEOUJFRS4bN06lqmovo0evJCXlTLdL6pZKqkqOPCMpcc5IShsvb3pHXn+cv1FgNJpOGURaXJr1j5gWWUCYoCorv2DjxmlUVOxm9OhnSU092+2STDPqtI59JfvYVbiLXUW7yCnMqZ/eVbSLXYW7Gg0thkD/yNEC5LjE4yLqUlZlTSX55flBXwUVBY3mi6uKEQQRqR9g0KHTzrGjPFH4vD58UT5ivDH10839balNjDemfrqjwt0CwjSrqmo/GzZMo6JiB6NGrcDvn+52SeYYqCr55fn1YdHorzPdtG8kyhPFwB4Dmw2QgT0GtrljXVUprS5t9ou+uS/8/PL8oJfbDvOKl9S4VPxxfvxxfpJiApfXDg8mODzAINj04cEILU0fbR/N7a+mrobK2koqayqprK2kTuva/j9eM6I90a0OmSmDpvDTST89puMcLSBsoHk3FxPTh8zMV9i48Rw+/PBrjBq1nLS0bLfLMm0kIqTFp5EWn8YpxwUfnVZSVcLuot1BA+TFnS/yRfEXjX6jIgh9E/s2CpC+iX0pqSoJfKlXNPnSLw986Te8NUtTPq+v/kveH+dncMpgxh03rtGyYK+kmKROcbmspq6Gqtqq+sBozd+q2qrWtQ3SvrymnMKKwmYHRrSXnUEYAKqr89i48VxKSzczcuQSevW62O2STAerqq0i91Bus2cgu4t213/5J8YkHvlFHnv0L/nUuFTiouI6xRd9d2KXmEyrVFcXsmnTeRQXv0dKyjTS039NSspkt8syEaJO6ygoLyDJl0SMN8btckyIHC0gIvaZ1KbjRUenkJn5CkOH3kFp6WY2bDiLDz6YQkHBS/aYUoNHPKTFp1k4dCMWEKYRrzeegQN/zGmnfcawYXdRXv4pGzeewwcfnEF+/moLCmO6EQsIE5TXG8eAAfOZMGEHw4f/lcrKPWzaNIP16ydw8OCzFhTGdANhDQgRmSEi20Rku4jcGmS9T0SWOuvfFZF0Z/l0EVknIh86f6eGs07TPK83lv79v8eECds54YT7qa4+yObNX2PdunEcOLAcDeGwPmNMZAlbQIiIF7gHyAZGApeLyMgmzb4NFKjqMOBPwO3O8oPA11R1NDAPeCxcdZrW8Xhi6NfvO4wfv40RIx6itraYLVsuYe3aDL788gk0yI3sjDGdWzjPIMYD21V1p6pWAUuAWU3azAIecaaXAdNERFT1A1X9wlm+BYgTEbsVZgTweKI57rhrOPXUrZx00mJUa/joo9m8//5o9u9/nLomt4MwxnRe4QyI/sCeBvO5zrKgbVS1BigC0pq0+TqwXlUrmx5ARK4VkbUisvbAgQMhK9y0zOOJok+fuZx66mZGjlyKiJetW6/k/fdHsm/fI9Qd5cdSxpjOIaI7qUXkZAKXna4Ltl5VF6lqlqpm9erVq2OLMwCIeOnd+zKysjZy8slP4fUm8PHH1/DeeyP44osHqKurcrtEY8wxCmdAfA4MbDA/wFkWtI2IRAHJQJ4zPwBYDlytqjvCWKcJAREPvXpdzLhx6xk1agXR0T355JPv8u67w/n8879RV3fECaAxJsKFMyDeB4aLyGARiQHmACuatFlBoBMa4FLgZVVVEUkBngNuVdW3wlijCTERoWfPr3HKKe8yevQqfL7+fPrp93nnnaHk5t5NbW252yUaY1opbAHh9CncAKwGtgJPqOoWEVkoIjOdZn8H0kRkO3AzcHgo7A3AMODXIrLBefUOV60m9ESEtLQZjB37FhkZLxIXN4zt22/knXcGs2fP/1FbW9ryTowxrrJ7MZkOU1j4Gjk5/01h4UtER/dk4MCf0K/f94mKsiejGeMWuxeTiQgpKWeRmfkiY8e+RVJSFjt33so776STk/M7amqK3C7PGNOEBYTpcMnJExkzZhWnnPIuycmTyMn5FWvWDOKzz35DdXW+2+UZYxwWEMY1PXqMZ/ToFYwbt57U1Gns2rWQd95JZ+fOX1BVddDt8ozp9iwgjOuSksYyatSTZGVtwu/PZvfu23jnnXS2bbuOgwf/bR3axrjEOqlNxCkt3cru3f/DgQNPUVdXikgMKSln4fefT1paNnFxJ9hTyYwJEXuinOmU6uoqKSp6k7y8VeTnr6SsbCsAsbFD8PuzSUs7n5SUKXi98S5XakznZQFhuoTy8hzy81eRn7+KgoKXqKsrw+OJJSVlCn5/Nn7/+cTHD3O7TGM6FQsI0+XU1lZQVPQ6+fmryMtbSXn5JwDExQ1zLkWdT3LyWXi9sS5Xakxks4AwXV55+Y76S1GFha9QV1eBxxNHSspU0tICZxdxcYPdLtOYiGMBYbqV2tpyCgtfrT+7qKgI3OsxLm4EaWnn4/dnk5IyGY/HHjFijAWE6dbKyj4lP38leXmrKCx8FdVKPJ4EUlOnOZ3d2cTGDnK7TGNcYQFhjKO2tpSCgleczu6VVFTkABAfP7L+7CI5+Qw8nhh3CzWmg1hAGBOEqlJWto38/JXk56+isPA1VKvxehNJTT0Hvz8QGLGxA9wu1ZiwOVpARHV0McZEChEhIeFEEhJOZODAm6mpKaGw8GXy8laSn7+SgwefBiA2Np3ExEwSEjJITMwkMTGT2NhB9mM90+VZQBjjiIpKpGfPmfTsOdM5u/iIvLxVFBevpaRkAwcPPgMEzri93mQnLL4KjYSEkdbxbboUCwhjggicXZxMQsLJ9ctqa0spLd1MScmG+tfevQ9QV1fmbBNFfPxJ9YFxOECio9PcehvGtIsFhDGt5PUm0KPHBHr0mFC/TLWW8vIdDUJjIwUFL7F//2P1bXy+AY1CIyEhg7i4IYjYvTJNZLOAMKYdRLzEx59AfPwJ9O59Wf3yqqovKSnZ6LwC4ZGXtwqoBcDrTWzQp5HhBMcovN44l96JMUcKa0CIyAzgLsALPKCqtzVZ7wMeBcYBecBsVc0RkTRgGXAq8LCq3hDOOo0JtZiY3vj90/H7p9cvq62toKxsS6NLVPv3P8oXXxQ7LTzEx4844hJVTEwfd96E6fbCFhAi4gXuAaYDucD7IrJCVT9q0OzbQIGqDhOROcDtwGygAvgVMMp5GdPpeb2xJCWNIylpXP0y1ToqKnIahUZR0Vt8+eU/69vExPQlISGD+PgTiY8fTlxc4BUbezyB/8yMCY9wnkGMB7ar6k4AEVkCzAIaBsQsYIEzvQz4i4iIqpYCb4qI3ZrTdGkiHuLihhAXN4RevS6pX15dnd/g8tRGSks3snfvG/Ud4oFtY5xthxMXN6w+OOLjh+PzDbQ+DtNu4QyI/sCeBvO5wITm2qhqjYgUAWmAPW/SdGvR0X5SU88mNfXs+mWqSlXVXsrLP6Ws7FPKyw+/tlNQ8CJ1deX1bUV8xMUNrQ+PhmcePl9/Cw/TKp26k1pErgWuBTj++ONdrsaY8BIRfL5++Hz9SEk5q9E61ToqK79oEBqB4Ajch+p5VCvr23o8sQ3OOBqfecTE9LMfAJp64QyIz4GBDeYHOMuCtckVkSggmUBndauo6iJgEQRutdGuao3pxEQ8xMYOIDZ2QKOzDjgcHrlNzjy2U1b2MXl5z6FaVd/W44lvFB4NzzxiYvpaeHQz4QyI94HhIjKYQBDMAa5o0mYFMA9YA1wKvKxd5eZQxkSIQHgcT2zs8aSmTmu0TrWWioo9jc46yss/pbR0M3l5K1Ctrm/r9SYSGzsUn+84YmL6NvvyentYkHQRYQsIp0/hBmA1gWGuD6rqFhFZCKxV1RXA34HHRGQ7kE8gRAAQkRygBxAjIhcB5zYZAWWMaScRL3Fx6cTFpRMYcPiVuroaKit3N7pcVVGxg6qqfZSWbqaqah+qNUfs0+OJPWqAREf3aRAm9sS/SGZ3czXGHBPVOmpqCqiq2kdV1X7nb/BXdfWBoPvwepOPGiZfvXrZkN4wsbu5GmNCTsRDdHQa0dFpje5ZFUxdXTXV1QeaCZBAuJSUfEBV1T5qaw8F2YOH6OheTlj0ISamD9HRvYiO7unU0LPRKyrKj8djX2/tZZ+gMSbsPJ7o+hFYLamtLWvxjKSsbBs1NXnU1pY0u5+oqNQjgqO5QAmESqoN/23CAsIYE1G83nji4gYTFze4xba1tRXU1ORRXX3wqK/Kyj2UlGyguvoAdXUVzezNQ3S0v0FgBA+SxqGS3KU75C0gjDGdltcbi9fbH5+vf6u3qa0tazFQqqsPUlGxk+Li96muPtBoNFdjHqKiUpxXKlFRKURHpzaab/q34fpIf7StBYQxplvxeuPxegPDfltDVamtLWkmSPKoqSl0XgXU1BRQWvqFM114lLOVAI8n7qgB0jR8GrbxepPCfknMAsIYY45CRIiKSiIqKqlVl70aClwCK6wPjKZ/q6sbz1dVfUFZ2UfOfBGHn2AYnIeoqGSiolLo2fNihg37v3a9z2AsIIwxJkwCl8D64vP1bfO2gWHEh1oRMAX4fANb3uExsIAwxpgIFBhGnEJ0dAqQ7koNNqbLGGNMUBYQxhhjgrKAMMYYE5QFhDHGmKAsIIwxxgRlAWGMMSYoCwhjjDFBWUAYY4wJqss8MEhEDgC73K6jnXoCB90uIoLY59GYfR5fsc+isfZ8HoNUtVewFV0mILoCEVnb3JOduiP7PBqzz+Mr9lk0Fq7Pwy4xGWOMCcoCwhhjTFAWEJFlkdsFRBj7PBqzz+Mr9lk0FpbPw/ogjDHGBGVnEMYYY4KygDDGGBOUBUQEEJGBIvKKiHwkIltE5Ea3a3KbiHhF5AMRedbtWtwmIikiskxEPhaRrSJyuts1uUlEfuT8d7JZRP4pIrFu19SRRORBEflSRDY3WOYXkf+IyKfO39RQHMsCIjLUAD9W1ZHAacAPRGSkyzW57UZgq9tFRIi7gOdV9UQgg278uYhIf2A+kKWqowAvMMfdqjrcw8CMJstuBV5S1eHAS858u1lARABV3auq653pYgJfAP3drco9IjIAuAB4wO1a3CYiycBk4O8AqlqlqoXuVuW6KCBORKKAeOALl+vpUKr6OpDfZPEs4BFn+hHgolAcywIiwohIOjAWeNfdSlx1J/AzoM7tQiLAYOAA8JBzye0BEUlwuyi3qOrnwB3AbmAvUKSqL7hbVUToo6p7nel9QJ9Q7NQCIoKISCLwJHCTqh5yux43iMiFwJequs7tWiJEFHAK8DdVHQuUEqLLB52Rc219FoHg7AckiMiV7lYVWTTw24WQ/H7BAiJCiEg0gXB4XFWfcrseF00CZopIDrAEmCoii90tyVW5QK6qHj6jXEYgMLqrc4DPVPWAqlYDTwETXa4pEuwXkeMAnL9fhmKnFhARQESEwDXmrar6R7frcZOq/lxVB6hqOoHOx5dVtdv+C1FV9wF7RGSEs2ga8JGLJbltN3CaiMQ7/91Moxt32jewApjnTM8DngnFTi0gIsMk4CoC/1re4LzOd7soEzF+CDwuIpuATOAPLtfjGudMahmwHviQwHdYt7rthoj8E1gDjBCRXBH5NnAbMF1EPiVwlnVbSI5lt9owxhgTjJ1BGGOMCcoCwhhjTFAWEMYYY4KygDDGGBOUBYQxxpigLCCMaYGI1DYYfrxBREL2S2YRSW94V05jIkmU2wUY0wmUq2qm20UY09HsDMKYYyQiOSLyvyLyoYi8JyLDnOXpIvKyiGwSkZdE5HhneR8RWS4iG53X4VtEeEXkfucZBy+ISJzTfr7zjJBNIrLEpbdpujELCGNaFtfkEtPsBuuKVHU08BcCd6EF+DPwiKqOAR4H7naW3w28pqoZBO6ntMVZPhy4R1VPBgqBrzvLbwXGOvu5Plxvzpjm2C+pjWmBiJSoamKQ5TnAVFXd6dxscZ+qponIQeA4Va12lu9V1Z4icgAYoKqVDfaRDvzHedALInILEK2qvxOR54ES4GngaVUtCfNbNaYRO4Mwpn20mem2qGwwXctXfYMXAPcQONt433lAjjEdxgLCmPaZ3eDvGmf6bb56DOZc4A1n+iXge1D/zO3k5nYqIh5goKq+AtwCJANHnMUYE072LxJjWhYnIhsazD+vqoeHuqY6d1mtBC53lv2QwBPgfkrgaXDfdJbfCCxy7r5ZSyAs9hKcF1jshIgAd9ujRk1Hsz4IY46R0weRpaoH3a7FmHCwS0zGGGOCsjMIY4wxQdkZhDHGmKAsIIwxxgRlAWGMMSYoCwhjjDFBWUAYY4wJ6v8DVgqD8Nesxa8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XqHK752HUhnY"
      },
      "source": [
        "# **Accuracies before and after Hyper Parameter Optimization**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K8xvwG6NTPvF"
      },
      "source": [
        "For Siamese Network With Adam Optimizer,\n",
        "* Before Hyper Parameter Optimization, the **Accuracy is 97.289%**\n",
        "* After Hyper Parameter Optimization, the **Accuracy increased to 97.595%**\n",
        "\n",
        "Changes made in hyperparameters are as follows,\n",
        "\n",
        "**learning_rate=0.0005, beta_1=0.888, beta_2=0.8888, epsilon=1e-8**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fIZTe9lCr_px"
      },
      "source": [
        "# **Pros and Cons of Siamese Networks**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I9sgFuT7gLZt"
      },
      "source": [
        "**Pros of Siamese Network**\n",
        "1. Given a few images per class, Siamese Networks will recognise those images in the future with the help of One-shot learning.\n",
        "2. Siamese focuses on learning embeddings (in the deeper layer) that group together similar classes and concepts. As a result, semantic similarity can be learned.\n",
        "\n",
        "\n",
        "\n",
        "**Cons of Siamese Network**\n",
        "1. Siamese Networks need more training time than normal networks because they use quadratic pairs to learn from, and they are slower than normal classification learning.\n",
        "2. Since pairwise learning is used in training, the distance from each class will be output instead of the prediction probabilities.\n",
        "\n",
        "\n"
      ]
    }
  ]
}