DM2020-Lab2-Homework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Student Information\n",
    "Name:Iuan Kai Fang\n",
    "\n",
    "Student ID: 108003854\n",
    "\n",
    "GitHub ID: s108003854\n",
    "\n",
    "Kaggle name: Iuan Kai Fang\n",
    "\n",
    "Kaggle private scoreboard snapshot:\n",
    "\n",
    "[Snapshot](img/pic0.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. First: __This part is worth 30% of your grade.__ Do the **take home** exercises in the [DM2020-Lab2-Master Repo](https://github.com/fhcalderon87/DM2020-Lab2-Master). You may need to copy some cells from the Lab notebook to this notebook. \n",
    "\n",
    "\n",
    "2. Second: __This part is worth 30% of your grade.__ Participate in the in-class [Kaggle Competition](https://www.kaggle.com/c/dm2020-hw2-nthu/) regarding Emotion Recognition on Twitter. The scoring will be given according to your place in the Private Leaderboard ranking: \n",
    "    - **Bottom 40%**: Get 20% of the score (ie. 20% of 30% )\n",
    "\n",
    "    - **Top 41% - 100%**: Get (101-x)% of the score, where x is your ranking in the leaderboard (ie. (101-x)% of 30% )   \n",
    "    Submit your last submission __BEFORE the deadline (Dec. 5th 11:59 pm, Saturday)__. Make sure to take a screenshot of your position at the end of the competition and store it as '''pic0.png''' under the **img** folder of this repository and rerun the cell **Student Information**.\n",
    "    \n",
    "\n",
    "3. Third: __This part is worth 30% of your grade.__ A report of your work developping the model for the competition (You can use code and comment it). This report should include what your preprocessing steps, the feature engineering steps and an explanation of your model. You can also mention different things you tried and insights you gained. \n",
    "\n",
    "\n",
    "4. Fourth: __This part is worth 10% of your grade.__ It's hard for us to follow if your code is messy :'(, so please **tidy up your notebook** and **add minimal comments where needed**.\n",
    "\n",
    "\n",
    "You can submit your homework following these guidelines: [Git Intro & How to hand your homework](https://github.com/fhcalderon87/DM2020-Lab1-Master/blob/master/Git%20Intro%20%26%20How%20to%20hand%20your%20homework.ipynb), but make sure to fork the [DM2020-Lab2-Homework](https://github.com/fhcalderon87/DM2020-Lab2-Homework) repository this time! Also please __DON´T UPLOAD HUGE DOCUMENTS__, please use Git ignore for that.\n",
    "\n",
    "Make sure to commit and save your changes to your repository __BEFORE the deadline (Dec. 8th 11:59 pm, Tuesday)__. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Begin Assignment Here\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "## load a pickle file\n",
    "train_df = pd.read_pickle(\"train_df.pkl\")\n",
    "test_df = pd.read_pickle(\"test_df.pkl\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 1 (Take home): **  \n",
    "Plot word frequency for Top 30 words in both train and test dataset. (Hint: refer to DM lab 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIoAAAJtCAYAAABZgJv1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA88ElEQVR4nO3debhkZ1kv7N+TRFBACEgIQxIDmKCgECAMHvUTlCGgSJgkOIA5HAMKIkdlUs8BBDwBFTlRRNEwOTAIQiJzVAY5EE0CMYAQCHMiJJHIICgCeb4/1rtJ9c7uzu7u3bWqu+/7uvbVtd9Vu9bTtavWrvVb71DdHQAAAAA4YO4CAAAAAFgNgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEiSHDR3AVfl+te/fh955JFzlwEAAACwzzjnnHP+tbsPWd++8kHRkUcembPPPnvuMgAAAAD2GVX1iY3aDT0DAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMBwlUFRVb2gqi6pqvcttL28qs4dXx+vqnNH+5FV9R8L2/5w4WduX1XvraoLquqUqqo98j8CAAAAYJcctIn7vCjJ7yd5yVpDdz947XZV/U6Szy/c/yPdfcwGj/O8JD+b5B+SvD7JcUnesNMVAwAAALBHXGWPou5+e5LLNto2egX9eJKX7ugxqupGSa7d3Wd2d2cKnY7f6WoBAAAA2GN2d46iH0hycXd/eKHtplX1nqp6W1X9wGi7SZILF+5z4WgDAAAAYEVsZujZjjwk2/Ym+nSSI7r7s1V1+ySvqapb7eyDVtVJSU5KkiOOOGI3SwQAAABgM3a5R1FVHZTk/klevtbW3V/p7s+O2+ck+UiSo5NclOSwhR8/bLRtqLuf393HdvexhxxyyK6WCAAAAMBO2J2hZ3dL8sHu/saQsqo6pKoOHLdvluSoJB/t7k8n+UJV3XnMa/TQJKftxr4BAAAA2GJXGRRV1UuTvCvJLarqwqp6+Nh0Qq48ifX/l+S8qjo3ySuTPLK71ybC/vkkf5Lkgkw9jax4BgAAALBCalqEbHUde+yxffbZZydJLn3en81SwyE/91Oz7BcAAABgT6iqc7r72PXtu7vqGQAAAAD7CEERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAAZBEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAACGqwyKquoFVXVJVb1voe0pVXVRVZ07vu69sO1JVXVBVZ1fVfdcaD9utF1QVU/c+v8KAAAAALtjMz2KXpTkuA3af7e7jxlfr0+SqrplkhOS3Gr8zB9U1YFVdWCS5ya5V5JbJnnIuC8AAAAAK+Kgq7pDd7+9qo7c5OPdN8nLuvsrST5WVRckuePYdkF3fzRJqupl477/vPMlAwAAALAn7M4cRY+uqvPG0LTrjrabJPnUwn0uHG3bawcAAABgRexqUPS8JDdPckySTyf5na0qKEmq6qSqOruqzr700ku38qEBAAAA2I5dCoq6++Lu/np3X57kj3PF8LKLkhy+cNfDRtv22rf3+M/v7mO7+9hDDjlkV0oEAAAAYCftUlBUVTda+PZ+SdZWRDs9yQlVdfWqummSo5L8Y5KzkhxVVTetqqtlmvD69F0vGwAAAICtdpWTWVfVS5PcJcn1q+rCJE9OcpeqOiZJJ/l4kkckSXe/v6pekWmS6q8leVR3f308zqOTvCnJgUle0N3v3+r/DAAAAAC7bjOrnj1kg+ZTd3D/ZyR5xgbtr0/y+p2qDgAAAICl2Z1VzwAAAADYhwiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAAZBEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCTJQXMXsLe75A9PmWW/N3jkY2bZLwAAALDv0qMIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACTZRFBUVS+oqkuq6n0Lbb9VVR+sqvOq6tVVdfBoP7Kq/qOqzh1ff7jwM7evqvdW1QVVdUpV1R75HwEAAACwSzbTo+hFSY5b13ZGku/u7lsn+VCSJy1s+0h3HzO+HrnQ/rwkP5vkqPG1/jEBAAAAmNFVBkXd/fYkl61re3N3f218e2aSw3b0GFV1oyTX7u4zu7uTvCTJ8btUMQAAAAB7xFbMUfTfk7xh4fubVtV7quptVfUDo+0mSS5cuM+Fow0AAACAFXHQ7vxwVf1akq8l+fPR9OkkR3T3Z6vq9kleU1W32oXHPSnJSUlyxBFH7E6JAAAAAGzSLvcoqqqfSfKjSX5yDCdLd3+luz87bp+T5CNJjk5yUbYdnnbYaNtQdz+/u4/t7mMPOeSQXS0RAAAAgJ2wS0FRVR2X5PFJfqy7v7zQfkhVHThu3yzTpNUf7e5PJ/lCVd15rHb20CSn7Xb1AAAAAGyZqxx6VlUvTXKXJNevqguTPDnTKmdXT3LGWOX+zLHC2f+X5Deq6qtJLk/yyO5emwj75zOtoPYtmeY0WpzXCAAAAICZXWVQ1N0P2aD51O3c91VJXrWdbWcn+e6dqg4AAACApdmKVc8AAAAA2AcIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAAZBEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEiyyaCoql5QVZdU1fsW2q5XVWdU1YfHv9cd7VVVp1TVBVV1XlXdbuFnHjbu/+GqetjW/3cAAAAA2FWb7VH0oiTHrWt7YpK/7e6jkvzt+D5J7pXkqPF1UpLnJVOwlOTJSe6U5I5JnrwWLgEAAAAwv00FRd399iSXrWu+b5IXj9svTnL8QvtLenJmkoOr6kZJ7pnkjO6+rLv/LckZuXL4BAAAAMBMdmeOokO7+9Pj9meSHDpu3yTJpxbud+Fo2177lVTVSVV1dlWdfemll+5GiQAAAABs1pZMZt3dnaS34rHG4z2/u4/t7mMPOeSQrXpYAAAAAHZgd4Kii8eQsox/LxntFyU5fOF+h4227bUDAAAAsAJ2Jyg6PcnaymUPS3LaQvtDx+pnd07y+TFE7U1J7lFV1x2TWN9jtAEAAACwAg7azJ2q6qVJ7pLk+lV1YabVy05O8oqqeniSTyT58XH31ye5d5ILknw5yYlJ0t2XVdXTkpw17vcb3b1+gmwAAAAAZrKpoKi7H7KdTT+8wX07yaO28zgvSPKCTVcHAAAAwNJsyWTWAAAAAOz9BEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIEly0NwFsGf8y3N/aen7vPGjnr30fQIAAABbR48iAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAIaD5i6A/ccHn3vfWfb7nY86bZb9AgAAwN5GUMR+7V3P/9FZ9vu9J712lv0CAADAjhh6BgAAAEASPYpg5bzp1HvPst97Pvz1s+wXAACA1aFHEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADDsclBUVbeoqnMXvr5QVY+tqqdU1UUL7fde+JknVdUFVXV+Vd1za/4LAAAAAGyFg3b1B7v7/CTHJElVHZjkoiSvTnJikt/t7t9evH9V3TLJCUluleTGSf6mqo7u7q/vag0AAAAAbJ2tGnr2w0k+0t2f2MF97pvkZd39le7+WJILktxxi/YPAAAAwG7aqqDohCQvXfj+0VV1XlW9oKquO9pukuRTC/e5cLQBAAAAsAJ2Oyiqqqsl+bEkfzmanpfk5pmGpX06ye/swmOeVFVnV9XZl1566e6WCAAAAMAmbEWPonsleXd3X5wk3X1xd3+9uy9P8se5YnjZRUkOX/i5w0bblXT387v72O4+9pBDDtmCEgEAAAC4KlsRFD0kC8POqupGC9vul+R94/bpSU6oqqtX1U2THJXkH7dg/wAAAABsgV1e9SxJquqaSe6e5BELzc+qqmOSdJKPr23r7vdX1SuS/HOSryV5lBXPAAAAAFbHbgVF3f2lJN+2ru2nd3D/ZyR5xu7sEwAAAIA9Y6tWPQMAAABgLycoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAADDbq16BuwfXvnC42bZ7wNPfOMs+wUAANhf6VEEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAAZBEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMBw0dwEAu+qFL77H0vd54sPevPR9AgAALIseRQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQJLkoLkLANiXnPLn95xlv4/5yTfNsl8AAGDfIigC2Mc95RXzhFdP+XHhFQAA7G0MPQMAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAACG3Q6KqurjVfXeqjq3qs4ebderqjOq6sPj3+uO9qqqU6rqgqo6r6put7v7BwAAAGBrbFWPort29zHdfez4/olJ/ra7j0ryt+P7JLlXkqPG10lJnrdF+wcAAABgN+2poWf3TfLicfvFSY5faH9JT85McnBV3WgP1QAAAADATtiKoKiTvLmqzqmqk0bbod396XH7M0kOHbdvkuRTCz974WjbRlWdVFVnV9XZl1566RaUCAAAAMBVOWgLHuP7u/uiqrpBkjOq6oOLG7u7q6p35gG7+/lJnp8kxx577E79LAAAAAC7Zrd7FHX3RePfS5K8Oskdk1y8NqRs/HvJuPtFSQ5f+PHDRhsAAAAAM9utoKiqrllV37p2O8k9krwvyelJHjbu9rAkp43bpyd56Fj97M5JPr8wRA0AAACAGe3u0LNDk7y6qtYe6y+6+41VdVaSV1TVw5N8IsmPj/u/Psm9k1yQ5MtJTtzN/QMAAACwRXYrKOrujya5zQbtn03ywxu0d5JH7c4+AQAAANgztmLVMwAAAAD2AYIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIEly0NwFALD/OfHVx82y3xfe742z7BcAAPYWehQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGA4aO4CAGAV3Ou0R82y3zfc97mz7BcAADaiRxEAAAAASfQoAoCVdu9XP33p+3z9/X596fsEAGA1CIoAgJ3yI391yiz7fd39HzPLfgEA9ieGngEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAIaD5i4AAGB3/cir/mSW/b7uAf9jlv0CAOwpehQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAw0FzFwAAsC/60Vf++Sz7fe0Df3KW/QIA+wY9igAAAABIIigCAAAAYBAUAQAAAJDEHEUAAPuN+7zyr2bZ718/8P6z7BcA2Hl6FAEAAACQRFAEAAAAwGDoGQAAs7rvK9+49H2e9sDjlr5PANgbCIoAAGCd+73qHbPs99UP+P5Z9gsAaww9AwAAACCJHkUAALBXeNCrzptlv3/5gFvPsl8A5qFHEQAAAABJ9CgCAAB20WNe/alZ9nvK/Q6fZb8A+wM9igAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAYDJrAABgn/HcV188y34fdb9DZ9kvwFbTowgAAACAJLsRFFXV4VX1lqr656p6f1X94mh/SlVdVFXnjq97L/zMk6rqgqo6v6ruuRX/AQAAAAC2xu4MPftakl/u7ndX1bcmOaeqzhjbfre7f3vxzlV1yyQnJLlVkhsn+ZuqOrq7v74bNQAAAACwRXY5KOruTyf59Lj9xar6QJKb7OBH7pvkZd39lSQfq6oLktwxybt2tQYAAIC9wV+98l+Xvs/7P/D6S98nsPfbkjmKqurIJLdN8g+j6dFVdV5VvaCqrjvabpLkUws/dmF2HCwBAAAAsES7HRRV1bWSvCrJY7v7C0mel+TmSY7J1OPod3bhMU+qqrOr6uxLL710d0sEAAAAYBN2Kyiqqm/KFBL9eXf/VZJ098Xd/fXuvjzJH2caXpYkFyU5fOHHDxttV9Ldz+/uY7v72EMOOWR3SgQAAABgk3Z5jqKqqiSnJvlAdz97of1GY/6iJLlfkveN26cn+YuqenamyayPSvKPu7p/AAAAdt1b/nye0Rt3/cntdwZ4z59cssRKrnDb/3GDWfYLq2h3Vj37viQ/neS9VXXuaPvVJA+pqmOSdJKPJ3lEknT3+6vqFUn+OdOKaY+y4hkAAADA6tidVc/ekaQ22PT6HfzMM5I8Y1f3CQAAAMCesyWrngEAAACw99udoWcAAACwT/v4cz4zy36PfOwNt7vtM799wRIrucINf+U7drj94uecs6RKrnDoY2+/9H3u6wRFAAAAwD7p4lPeOst+D33MXWbZ71Yw9AwAAACAJIIiAAAAAAZDzwAAAACW5JLn/vUs+73Bo+6zqfvpUQQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgEBQBAAAAkERQBAAAAMAgKAIAAAAgiaAIAAAAgEFQBAAAAEASQREAAAAAg6AIAAAAgCSCIgAAAAAGQREAAAAASQRFAAAAAAyCIgAAAACSCIoAAAAAGARFAAAAACQRFAEAAAAwCIoAAAAASCIoAgAAAGAQFAEAAACQRFAEAAAAwCAoAgAAACCJoAgAAACAQVAEAAAAQBJBEQAAAACDoAgAAACAJIIiAAAAAAZBEQAAAABJBEUAAAAADIIiAAAAAJIIigAAAAAYBEUAAAAAJBEUAQAAADAIigAAAABIIigCAAAAYBAUAQAAAJBEUAQAAADAICgCAAAAIImgCAAAAIBBUAQAAABAEkERAAAAAIOgCAAAAIAkgiIAAAAABkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABgERQAAAAAkERQBAAAAMAiKAAAAAEgiKAIAAABgWHpQVFXHVdX5VXVBVT1x2fsHAAAAYGNLDYqq6sAkz01yryS3TPKQqrrlMmsAAAAAYGPL7lF0xyQXdPdHu/u/krwsyX2XXAMAAAAAG6juXt7Oqh6Y5Lju/h/j+59OcqfufvS6+52U5KTx7S2SnL8Fu79+kn/dgsfZSqtYU7Kadalpc9S0eatYl5o2R02bt4p1qWlz1LR5q1iXmjZHTZu3inWpaXPUtHmrWNe+XtO3d/ch6xsP2qIH31Ld/fwkz9/Kx6yqs7v72K18zN21ijUlq1mXmjZHTZu3inWpaXPUtHmrWJeaNkdNm7eKdalpc9S0eatYl5o2R02bt4p17a81LXvo2UVJDl/4/rDRBgAAAMDMlh0UnZXkqKq6aVVdLckJSU5fcg0AAAAAbGCpQ8+6+2tV9egkb0pyYJIXdPf7l7T7LR3KtkVWsaZkNetS0+aoafNWsS41bY6aNm8V61LT5qhp81axLjVtjpo2bxXrUtPmqGnzVrGu/bKmpU5mDQAAAMDqWvbQMwAAAABWlKAIAAAAgCSCIgAAAAAGQREAAAAASfbRoKiqDqqqR1TVG6vqvPH1hqp6ZFV90wrUd72qut7cdaypqoOr6jFV9eyqOmXta8Z6DqyqD861/51VVbPMhD+ep0dU1dOq6vvWbfv1mWq6RlU9vqoeV1XfXFU/U1WnV9Wzqupac9Q06jqgqg4Yt69WVbdbpfdgklTV7eauYSNV9aGZ93+zqnpBVT29qq5VVX9cVe+rqr+sqiPnrG3UtxLH8/Ea/+9V9bqq+qeqendVvayq7jJ3bYtW4bmC/UlV3XSDtjvMUQsAu298Ht7j51X75KpnVfXSJJ9L8uIkF47mw5I8LMn1uvvBM9R0RJJnJfnhUVsluXaSv0vyxO7++LJrWqjtnUnOTPLeJJevtXf3i2es6bQkv9Ddn5yrhkU7OLmpJP/U3Ycts54kqao/SXKNJP+Y5KeTvK27f2lse3d3Lz14qKpXJPlUkm9JcoskH0jy8iQ/luSG3f3TM9R0fJI/yvTafmSSX03y76O+n+vuv56hpvW/m0pyWpL7ZDouv3vZNSVJVX0xydofhRr/XiPJl5N0d197hprenuSlSa6T5KeSvDDJK5LcI8lPdvcPzVDTyh3Pq+qFST6R5G+SPDDJF5L8fZInJDmtu39vhpp+vbufPm7fMslrknxTpufrwd39D8uuadRynSRPSnJ8khtkes1fkuk9eHJ3f26OukZthya5yfj2ou6+eK5a1ozn67gs1JXkTTM/T/fM9PtbrOm07n7jXDUlSVXdf4Pmzyd5b3dfsux6kunzQJL7dPdF4/sfTPL73f09c9Qzavi+JOd295eq6qeS3C7J/+3uT8xV0/ZU1bW6+9/nriNJqurHuvv0Fajj4CQPTXJkkoPW2rv7MTOVlKo6J8kLkvxFd//bXHVsT1X9Znf/6tx1rKKqumOmz5hnjc8KxyX5YHe/fua6jk1yeJKvJ/lQd8/eiaGqvifJS5JcL9NnqUuTPKy737dH9rePBkUf6u6jd3bbHq7pXUmek+SV3f310XZgkgcleWx333nZNS3UNkuosCPj5PC2mUKQL621d/ePzVTP1zOdhNVCc4/vb9LdV5uhpvO6+9bj9kFJ/iDJ9ZM8JMmZ3X3bGWo6t7uPqapK8ukkN+ruHt//01q9S67pPUnulSm8+qckd+ju86vq25O8qruPnaGmyzOFs19ZaL7zaOs5wo9R1ylJDk7yuLUT1Kr6WHdf6Yr0Emt6z9pruao+2d1HbLRtyTWt3PF88Xgwvj+zu+9cVVfPdEL2XTPU9I2/LVX1ukwnp28YHwqf093/bdk1jVrelCnUe3F3f2a03TDTxaQf7u57zFDTMUn+MFMgetFoPixTEPnzM4bHD03y5CRvXlfX3ZM8tbtfMkNNz0lydKYPy4sXAx+a5MPd/YvLrmnNeJ1/b5K3jKa7JDknyU2T/EZ3/+kMNd0h0+eD+2QKZP5Pkh/t7k8tu5aFms5Lcpskt07yoiR/kuTHu/sH56ppe9b/3VnifteHjpXkuUl+Pkm6+6+WXdM3ClnNC8zfkeTEJA9Ocnami0pv7hlOdDcYlVGZLui+JJk3UNueqnpDd99rhv0+OdNn9IOSnJHkTpmOn3fPdEHiGTPU9INJfifT39/bJ/l/Sa6b5KtJfnrmY+c7k/xad79lfH+XJL+5pz5PHXTVd9krXVZVD8p0Enh5MnXLz/Qhfq6U+frd/fLFhnGC8bKqetpMNa3506r62SSvzcKJa3dfNl9J+V8z7nsjH810AnGlHk5VNdcB4xvhVHd/LclJ44D7d0lmG+Y16umqev3aH+jx/Wyp9MLJ4Ce7+/zR9om14WgzeFCSxyR5Vne/YdT2se6+60z1JJk+vFTV7ZO8tKpek+T3c0UPo7lcXlVHZwqwrlFVx3b32eND4YEz1bSKx/OvVtXNu/sjo8faf426vjLne2/Bjdde6939j1X1LTPWcmR3P3OxYRwjnllV/32mml6U5BHre1lV1Z0znfDcZo6ikvxaktuv7z1UVddN8g8ZJz5Ldu+NLvhV1cuTfCjJbEFRps/V37UQtB+a6Tm6U5K3J1l6UDSu0j8mU9j3n0nu1t2XLruOdb42PhfcN1OAfGpVPXyuYqrql7a3KfN9nnp5kjdl6u24dpHympkCv04yW1CU5JvXerCviu6+IMmvVdX/SvKjmXoXfX30tv2/Sz6nuV+St2V6z6397k7IFBrPZoPe7N/YlOSYJZay6IFj31dP8pkkh3X3F6rqtzP9jVl6UJTpQuA9uvvSMXT32d39fVV19ySnZurRPpdrroVESdLdb62qa+6pne2rQdEJSZ6Z5LlV9bnRdnCmhPKEmWo6p6r+INNwuLVg4fBMVzDfM1NNa/4ryW9l+kC4dkLRSW42W0XTVaY/W6Huo8/JlCZvNBTuWcst5RvOrqrjFrvad/dTq+qiJM+bsaZrdfe/d/c3Triq6uZJvjhTTamqA0ZovFjTgVkI25apu181ejU8bZyY/nLmD2SSJN19TlXdLcmjM33Q+eaZS3p8kr/OdNXy+CRPqqpbZ+p58bMz1bSKx/PHJXlLVX0l09/2E5Kkqg7JdBFgDjerqtMzfQg9rKqu0d1fHtvmnC/wE1X1+Ew9ihZP6H8mV/w+l+2a60OiJOnuM/fkh8BNqGx8bLo82/awXab/rKo7dPdZ69rvkCkImdPhve1wwUtG22VV9dVlFlJVf51tf3fXyDQM7tSqmq2H9vDFqnpSpl4WPzAu2sx5TPjNTJ+Dv7bBtrkuKP23JCcnOau7n5dMvQe6+8SZ6lm0iheYMz4bnJjk3kleleTPk3x/pguoxyyxlFsmeVqmIVS/0t3/UlVPnrPH1XBWps91Gx27D15uKd/wtXGh7ctV9ZHu/kKSdPd/jN73czhwIUz/ZJJvHzWdMXq0zumjIwxdu+jwU5k6M+wR++TQsySpqu9Kct9cefz6B2aq52pJHr5BTacnObW7v7K9n11CbR9Ncsfu/te5alivqp6e6UTn3ZmuCrxpju6j62r6zmzw+5vrNbW31ZRpvPEcXYDvkGl+iP9c135kku/v7j9bdk3r6rhtkmcn+e7uPmTOWkY9a7+/wzKdWFyQ5DUzv6bWjuc3zvQB5+JMx/P3zlTPSh7Px/P0gCQ3zHQiP+vxYHTfXnROd//7CGUe2N3Pnamu6yZ5Yqbf3w1G88WZfn8nz3GBYgxVuHmm3ieL4eNDk3ysux+97JpGXQ9L8r8zXRlfq+uITMMCntbdL5qhpttluhjyrbli6NnhmUKQR3X3bFftR4B8RJK/HE0PyFTj45K8dpm9Rjd4/22ju9+2rFrWG0M9fyJTCPL3Nc37dpc5hjKOet6ZaV7MK712qupT3X34DGWtjYb4hUwXSZ6Q5GXdPedF3CRJVT0qU0+Pz2XhAvOctdU0R9HnMvX4eNXi3+Gq+qvu3mj+sD1d0+2T/HaS1yV5dHcfuewa1tXzviT36+4Pb7Btltd5Vf1Dkrt295cXLuquzY33lp5nvtUXZHpd/12mOVYv6u5fqqprJHl3d3/nsmtaqO26SZ6aKQDtTHNRPnVPfW7ZJ4OiqnpCppDhZdl2TP0JmQ6yJ89V2yqqqjcnOX7hSu9KqKrK1L3vxCTHZprA9tTu/sgMtTw+04eal2XbORFme02paefVmJR87qte643X+reuXUmZsY4nZJrjamV+f+M1tVaT4/l2eJ62RlWd2N0vnGnf98rGwf/cE3peN8k9c+XJrGft8TvChsWJvz8zZz3JN47l98/0IT6Z5rZ41ZwXumo1J0hfqZqq6hZJLttoSF5VHboC9d0kye8mOXZFgqJVvMB8s+7eYz0rdtU4Jvx8ku/t7p+auZYHZrpwev4G247v7tfMUNPVN7q4VlXXzzTP6dIvCNa0QvrPZuoZ9k9JXtDdXx9D5m/QM066X1Xfs8znZF8Nij6U5Fbd/dV17VdL8v7uPmqGmg7KdAX6+Kzr5ZQp/Fhql+RFVfXqJLfKNDRvsQvp7JOtVdVtMgVFx2Wq785Jzujuxy+5jlV8TalpczWt4gpV18g0tKuT/F6myRcfkOSDmSY8nWWFlRX9/a1iTTdL8uuZjuHPzPQB/nszrfL3uJleU6v4PB2QaTjeA7KwckiSP+zuty67ns2omSauXXUreGK/ciuxJav1PNWOJ0j/ue5e+jDZdTVdmDEsdc6a2HmreIF5vPd+M9N8ePeqafWs7+3uU2cuba8w80WSA5Kkuy8fn1m+O8nHV+2i7iqoqr/PNJ/TCzOt8Pf5Pbm/ucbd7mmXZxqisN6NsjA7/5L9aabxsU/NNHb23uP2bZLMOvQl05LFz0jyzkwTra19zaaqfnF0I31Wpity39PdP5dp9vkHzFDSKr6m1LQ5L0/y6iQ37O6juvs7Rj2vydT7Yg4vSnJoppVwXpdpbo3fyvShea75pZLV/P2tYk0vyjTW/0uZVn45P9OqHW/MNFR2Dqv4PJ2aaWz/yZmC/teOtl+vql+YqaZU1Xnb+XpvpvflHDVdp6pOrqoPVNVlVfXZcfvkmpainkVVHVNVZyZ5a6ZQ9FlJ3lZVZ9b2J0bd0zU9NNOw9LtkGh57jSR3zTR32ENnqmnxeXpWVuB5ynSc+sXu/q7uvtv4+s4kjx3b5q7p7itS0+L774Or8v5bxZoWfCnJuVX1R1V1ytrXzDW9KNPk32t/Bz+U6XW1dFV1eFW9rKr+vqp+dfRQWdv2mjlq2oSnzrHTqjo+0yrJF9U0uf3fZ/o8fF5V3WeOmnakqt4w5/67+wcyzUt0RKa/eX9R0yTbe8S+2qPouEyr9Xw4246p/45MY0TfuL2f3YM1fag3WKXjqrYty0hwvzNTD4fzu/u/Zq7nqZm6+l2pe19Vfdey59xY0deUmjZX04e315tiR9v2cE3ndvcxo0vypzN1r+3x/T/1wjLnS65rFX9/q1jTe7r7tuP2Nj1QFrctuaZVfJ7OW3wtV9WZ3X3nqrp6knO7+7uWXdOo4+JMQ6nWD52qJO/s7o0Ctz1d05sy9XJ8cV+xSuMNM02w/UPdPcsqK1V1bra/GtsfdffSV2OrqvOT3Gl976EaK7HN8XlqRZ+nHf3tu2BcNNnvaxr73t7772GZVrxd+vtvFWtaqO1hG7X3jJM1V9VZ3X2HdX+fz+3uY2ao5YxMk2mfmWk0ye2T3Ke7PzvXZ4RR13nb25Tk6O6++jLrSabPTJkutH1LpmFed+ju86vq2zMN2z12hpp2tDrca7v7RsusZ8NCpgV5jk9ySpIvZKrtV7t7S1dD3CdXPevuN9a0nPIds2235LN6mll9DpdV1YMyvejXJuo6INMy2XOP8793kj9K8pFML7SbVtUjeixlPIfufnJVHVhVN87C67S7P7nskGjsd+VeU2ratFVcoSrJNPNjVb2+R2I/vp8tvV/F398q1pTk8lHTwUmuUVXHdvfZVfUdSQ6co6AVfZ6+WlU37+6PjA9e/zVq/cqcr/NMPZuu1d3nrt9QVW9dejWTI7v7mYsN4+Tw5Kqac5WjVVyNbRVXYlvF5+kNVfW6bDxB+tKD4xWuKdn++++ZNa1MqqZt65h79a6NfKmqvi3j2DBC2j06LGcHDunuPxy3f6GqfirJ26vqx7LxsWtZDs0OLpIsv5zJQhD6yR7zJ3X3J8Z58hxWcXW4JEldsbLfjyQ5I1MA+e5xvvyuJIKizRhhzJlz17HghEzdtp9bVZ8bbQdn6o5/wkw1rXl2krt29wVJUtNS5q9LMltQVFWPTvKUTCvRrA2b6CSz9LRIVvI1pabNeWimKzpPzQYrVM1U09lVda3u/vfu/sYHvvHe++JMNSVZyd/fKtb0+CR/nenYdHySJ40/3tfJNAHiLFbweXpckrdU1Vcyfd44IUmq6pBMYc0suvvhO9j2E8usZcEnapqQ/MU95rWpac6Nn8kVJ9RzWMUT+2ckeXdN86RcaSW2mWpaueepux9TG0+Q/tyeaYL0VaxpWMX33yrWlFHHUUn+T6bJfr95rb3nnWj7lzJ9rrt5Vf2/JIckeeBMtXxTVX1zj9V2u/vPquozmYbGzRUcJ6t5kSR1xWpni5+HD0xytZlK+kCmHqIbrg43Qz2Lfi/Jn2TqPfQfa43d/S9V9etbvbN9cujZqqorlnjeZjLrOXrILFrrrrnwfSX5x8W2GWq6IFPX8s/OVQPsSXXFMvTrw6sPtgPzyls4nt8401WnizMdz5e+QscqG8/TA5LcMFOwtraS16x/91bNGDb1xEyvqRuM5oszHRNO7hlXGNvOif2sq7HVCq7EtorPE5uziu+/VaxpobZ3JHlypoUc7pOph8MB3f2/56pp1HVQkltk+pt8fs+0UFBV/c9My6i/bV37bZM8q7v32Jwye5uqukOmldj+c137kUm+v7uXPo9vreDqcHMRFC1JTctOn5AVXLq4qp6XacLRV2TqtfOgJJ9M8jdJstXjHTdZ01uS3L27v7bsfbNvqRVcYWxcJfyJrNAy9GxeWYp+UzxPW6NmXI1mldUKrTC2impaGe5JmYKGQzP9Dbwk02q7J/cMK8StYk1XZRXff3PXVFXndPftq+q93f09i20z1HL/HW2f4xyGfdfc772NVNUbuvtee+SxBUXLUSu4dPFCDTt6wffi0JhlqapTM10VeF2SrywU8+xl18Lerapekamb9rdkek19INNKaD+WaSW0n56hppU9HnDV/P42x/O0NWrdhOlL3vfKndjXjpdY//nufvcMNa3i87RyE6SvYk1XZc733/bMXVNVvTPJ9yd5Zabf50WZXue3mKGWlTuHWVNVz0ry9CT/kWkI6q2T/M85esnsjfZkALKr5nrv1UwTbO+zcxStoLWli9ev4jXn0sVJku6ec6LM7fnk+Lpa5hujyr7h6O7+8TGk8tNJ7tbdPbpO/9NMNa3s8YBN8fvbHM/TJtWOV6M5dJm1rPOKTCeCd93gxP4VSeY4sX9Rtr/C2AuTLH2Fsazm87SKE6SvYk0r+f5bxZoW/GKSayR5TKZ5wX4o0wIhS9fdJ45Jjx/Y3a+Yo4YduEd3P76q7pfk40nun+TtSQRFw1UEIMcssZQrdrya771ZJtgWFC3PY5P8bVVtuHTxHAWNsbwPzzQZ6zbzJiU5da6xvUnS3U+da9/sm0Y4tCorjD02K3Y8YKc8Nn5/m/HYeJ42ayVXo8lqntiv4gpjq/g8reJkyKtYU7Ka779VrClJ0t1njZv/nml+oll19+XjdbVqQdHaefaPJPnL7v78dM2SBau4wtgqvvdmmWBbULQkvZpLF/9ppq7aT82286Q8LFPa/eB5yvrGqjiPT3KrbLuiwg/NVRN7rZVbYWxFjwdskt/f5niedspKrkaT1TyxX7kVxrKaz9ODM02G/LaqWj8Z8oPUtI1VfP+tXE0LQyyPzzTB9uxDLBf8TVX9SqapBb601tjdl81XUl5bVR/MNPTs58a5zX9exc/sb1ZxhbGVe+9lWgn8gO1s+4U9tVNzFO3HqupD3X30zm5bhpqWvX15kl9J8shM4dWl3f2EuWpi71VWGAPYaau68tKqrTC2qs/T9qzohKwrVxPbWuU5pqrqYxs0d3ffbOnFLKiq6yX5fHd/ffR4/Na15w4rjG2FPXnsFBTtx6rqzCS/k+RV3X35aDsg01WdX+ruO81Y29qKCud1961H21ndfYe5amLvZIUxgK3nxH5zVvF5mnsy5I2sYk1sq6rO396E1Tvatr+xEtvWWMVj5yrak8dOQdF+rKqOTPLMJHfNNAQtmcaDviXJE7t7o3R+KarqzO6+87h6cUqSf0nyyu6++Vw1sXey8hLA1ptx9ZeVW2FsR2Z8nnY0IevR3X31ZdaTrGZNbN7o7f832XiI5d27+24zlpeq+u4kt8y2U1a8ZIY61sKNGyT5b5l6YSXT+dY7u/tHl13T3kh4fIW5jp3mKNqPdffHq+opSd6TdZNZzxkSDU8fHwZ/OcnvJbl2kv85b0nspay8BLALVnT1l5VbYWxFn6dVnJB1FWti81Z1jqlU1ZOT3CVTUPT6JPdK8o5Mc5kt1dpq0iNYu2V3f3p8f6NMqzYyrOixcxXNcuwUFO3HquoJGcNvkqytIHJYkpdW1SxDcqrqmWMeom/p7s8n+XymBB521WNj5SWAXbGKJ/aruMLYKj5Pqzgh6yrWxCaNubaeML62Md57cw4TemCS2yR5T3efOHo6zb0M/eFrIdFwcabPn1xhFY+dq2iWY6ehZ/uxVRySU1XvTXLrJOd09+2WvX/2TWPuLSsvAeyEqjo1yQu7+x0bbPuL7v6JGWpaueEvq/g8wTLNPUyoqv6xu+9YVedkusD8xSQf6O7vnLGm309yVJKXjqYHJ7mgu/fYKlV7G8fO1SYo2o+NJRvv2d2fWNf+7UnePMekdFX1W0l+Nsm1knx5cVOm1QuuveyaAIDVsG6FsbWhCZ/JNPzlmTMvhw37rFWeY6qq/iDJr2YaKfHLSf49yblrw8BmrOv+SX5gfPv27n71nPXAzhAU7ceq6rgkv59kwyE53f3GGWq6end/papO6+77Lnv/AADAtqrq4uxgmFB333j5VV3ZWKzn2t29vWAL2ARzFO3HuvuNVXV0VmtIzruS3C7JF2baPwCwwqrqnkmOz5UX4lj6BS7Yj6zsHFNVdb8kf9fdnx+L9RxcVcd392tmqOWLmVZjvNKmGB3BXkSPIlZKVb0vyW8meVqSx63f3t1/tfSiAICVUFXPSXJ0ptWMLhzNhyV5aJIPd/cvzlQaMJOqOre7j1nX9p7uvu1MJcFeT48iVs0jk/xkkoOT3Gfdtk4iKAKA/de9u/vo9Y1V9fIkH0oiKIL9zwEbtM1+nltVt8m2cxQZDsdeY/Y3ECwas96/o6rO7u5T564HAFgp/1lVd+jus9a13yHJf85REDC7s6vq2UmeO75/VJJzZqwnVfWLmRboWbvI/edV9fzu/r0Zy4JNM/SMlVJVP9TdfzdWCbgSQ88AYP9VVbdL8rwk35orhp4dnuTzSR7V3bOeHALLV1XXTPK/k/zwaDojydO7+0sz1nReku9dq2HU+K7uvvVcNcHO0KOIVfODSf4uVx52lhh6BgD7te5+d5I7VdUNszCZdXd/ZsaygBmNMOYJc9exTiVZXBzo66MN9gp6FLGSquqm3f2xq2oDAPYvVXWdJMdl21XP3tTdn5utKGDpxrHgSZlWQbxBpovKlyQ5LcnJcx4TquqXkjwsyatH0/FJXtTdz5mrJtgZG038BavgVRu0vXLpVQAAK6OqHprk3UnukuQa4+uuSc4Z24D9xyuS/FuSu3T39br72zIdDz43ts2mu5+d5MQkl42vE4VE7E30KGKlVNV3JrlVkmcledzCpmsneVx332qWwgCA2VXV+UnutL6nQFVdN8k/bLQiGrBvqqrzu/sWO7ttGarqehs0f7G7v7r0YmAXmKOIVXOLJD+a5OBsO0/RFzOtHAAA7L8q0/CS9S6P+T9gf/OJqnp8khd398VJUlWHJvmZJJ+as7BMPR8Pz9TjqTKd23ymqi5O8rMm3mfVCYpYKd19WpLTqup7u/tdc9cDAKyUZyR5d1W9OVecCB6R5O5JnjZbVcAcHpzkiUneVlU3GG0XJzk9yYNmq2pyRpJXdvebkqSq7pHkAUlemOQPktxpxtrgKhl6xkqpqoOSPDzJ/ZLceDRflGlSulN11wSA/dsYZnbPXHky63+brypglVTVid39whn3/97u/p51bed1962r6tzuPmam0mBTBEWslKp6aaYJ6F6c5MLRfFimVQOu190Pnqk0AGBFjOEl3wiK1oadACRJVX2yu4+Ycf9vTvK3SV42mh6cqefjcUnO6u7bzVUbbIagiJVSVR/a3kSUO9oGAOz7quqYJH+Y5DqZLihVpgtKn0vy89397tmKA5aqqs7b3qYkR3f31ZdZzzYFVF0/yZOTfP9oekeS30jy+SRHdPcFc9UGm2GOIlbNZVX1oCSv6u7Lk6SqDsg0zliXcgDYv70oySO6+x8WG6vqzpnm/rjNHEUBszg00zDU9ecIleSdyy/nCt39r0l+YTubhUSsPEERq+aEJM9M8tyq+txoOzjJW8Y2AGD/dc31IVGSdPeZVXXNOQoCZvPaJNfq7nPXb6iqty69mmm/10nypCTHJ7lBplUaL8k03+rJ3f25OeqCnWXoGSunqr4ryX2z7SSVp3X3B+arCgCYW1WdkuTmSV6SK1Y9OzzJQ5N8rLsfPVdtAFX1piR/l+TF3f2Z0XbDJD+T5Ie6+x4zlgebJihipVTVEzL1HHpZpoAomeYeOCHJy7r75LlqAwDmV1X3ypUvKJ3e3a+fryqApKrO7+5b7Ow2WDWCIlZKVX0oya26+6vr2q+W5P3dfdQ8lQEAAGzfWO3sbzL1KLp4tB2aqUfR3bv7bjOWB5t2wNwFwDqXJ7nxBu03GtsAgP1UVV2nqk6uqg9U1WVV9dlx++SqOnju+oD93oOTfFuSt41j1GVJ3prkepkW54G9gh5FrJSqOi7J7yf5cK6Ye+CIJN+R5NHd/ca5agMA5mX+D2BvVVUndvcL564DNkNQxMqpqgOS3DHbzj1wVnd/fb6qAIC5mf8D2FtV1Se7+4i564DNOGjuAmC97r48yZlz1wEArJxPVNXjs/H8H5/a0Q8C7GlVdd72NiU5dJm1wO4QFAEAsLd4cJInZpr/4waj7eIkp8f8H8D8Dk1yzyT/tq69krxz+eXArjH0DACAvZ75P4C5VdWpSV7Y3e/YYNtfdPdPzFAW7DRBEQAAez3zfwDA1jD0DACAvYL5PwBgzxMUAQCwtzD/BwDsYYIiAAD2Fq9Ncq3uPnf9hqp669KrAYB9kDmKAAAAAEiSHDB3AQAAAACsBkERAAAAAEkERQAAAAAMgiIAAAAAkgiKAAAAABj+fxQbJPilPBB7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1440x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "# Answer here\n",
    "cv = CountVectorizer()\n",
    "cv_train=cv.fit_transform(train_df['text'])\n",
    "train_word_list = cv.get_feature_names();  \n",
    "train_count_list = cv_train.toarray().sum(axis=0)  \n",
    "sorted_index = np.argsort(-train_count_list)\n",
    "x_vect = np.asarray(cv.get_feature_names())\n",
    "\n",
    "g = sns.barplot(x=x_vect[sorted_index[:30]], \n",
    "            y=train_count_list[sorted_index[:30]])\n",
    "g.set_xticklabels(cv.get_feature_names()[:30], rotation = 90);\n",
    "g.figure.set_size_inches(20,10)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABIQAAAJvCAYAAAAKgQpNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA70klEQVR4nO3deZhlZ1ku/PuBHFBACUgThiQGEVBQxjAd9ZNBICiYAEHiUYkRjQOIiDKpn+gBPYAeRBRRjgGCchgMQxAZZZRPAiTMyBRBIDmQxIMgDqDA8/2xVpFKpbpT3VVda3fe3++6+uraa1X1uq9d1av2vtd637e6OwAAAACM4wpLBwAAAABgdymEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABnOZhVBVPbOqLqyq92/Y/vNV9aGq+kBVPWnd9sdU1blV9eGqusfBCA0AAADAgTtsC5/z7CR/mOQ5axuq6s5Jjk9yi+7+UlVde95+0yQnJblZkusl+euqunF3f2WngwMAAABwYC6zEOruN1fVMRs2/2ySJ3T3l+bPuXDefnyS58/bP15V5ya5XZK37usY17rWtfqYYzYeAgAAAIADdc455/xjd+/ZbN9W7hDazI2TfE9V/VaSLyb55e5+R5LrJzlr3eedN2/bp2OOOSZnn332AUYBAAAAYKOq+sTe9h1oIXRYkmsmuUOS2yZ5YVV9y36GOjXJqUly9NFHH2AMAAAAAPbXga4ydl6SF/fk7Um+muRaSc5PctS6zzty3nYp3f2M7j62u4/ds2fTu5cAAAAAOAgOtBB6aZI7J0lV3TjJlZL8Y5KXJTmpqq5cVTdIcqMkb9+BnAAAAADskMscMlZVz0typyTXqqrzkjw2yTOTPHNeiv4/kpzc3Z3kA1X1wiR/l+TLSR5shTEAAACA1VJTj7OsY489tk0qDQAAALBzquqc7j52s30HOmQMAAAAgEOUQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwRy2dIDNXPT0P1/kuHt+9kcXOS4AAADAbnKHEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAzmMguhqnpmVV1YVe/fZN8vVVVX1bXmx1VVT62qc6vqvVV164MRGgAAAIADt5U7hJ6d5LiNG6vqqCR3T/LJdZvvmeRG859Tkzx9+xEBAAAA2EmXWQh195uTfHaTXb+X5JFJet2245M8pydnJTm8qq67I0kBAAAA2BEHNIdQVR2f5Pzufs+GXddP8ql1j8+btwEAAACwIg7b3y+oqqsk+ZVMw8UOWFWdmmlYWY4++ujt/FMAAAAA7IcDuUPohklukOQ9VfUPSY5M8s6quk6S85Mcte5zj5y3XUp3P6O7j+3uY/fs2XMAMQAAAAA4EPtdCHX3+7r72t19THcfk2lY2K27+zNJXpbkgfNqY3dI8vnu/vTORgYAAABgO7ay7Pzzkrw1yU2q6ryqetA+Pv0VST6W5Nwk/yvJz+1ISgAAAAB2zGXOIdTdP3wZ+49Z93EnefD2YwEAAABwsBzQKmMAAAAAHLr2e5WxUV34x09d5LjX/pmHLnJcAAAA4PLLHUIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAIO5zEKoqp5ZVRdW1fvXbfudqvpQVb23ql5SVYev2/eYqjq3qj5cVfc4SLkBAAAAOEBbuUPo2UmO27DttUm+o7tvnuQjSR6TJFV10yQnJbnZ/DV/VFVX3LG0AAAAAGzbZRZC3f3mJJ/dsO013f3l+eFZSY6cPz4+yfO7+0vd/fEk5ya53Q7mBQAAAGCbdmIOoZ9I8sr54+sn+dS6fefN2y6lqk6tqrOr6uyLLrpoB2IAAAAAsBXbKoSq6leTfDnJc/f3a7v7Gd19bHcfu2fPnu3EAAAAAGA/HHagX1hVP57kXknu2t09bz4/yVHrPu3IeRsAAAAAK+KA7hCqquOSPDLJD3b3v63b9bIkJ1XVlavqBklulOTt248JAAAAwE65zDuEqup5Se6U5FpVdV6Sx2ZaVezKSV5bVUlyVnf/THd/oKpemOTvMg0le3B3f+VghQcAAABg/11mIdTdP7zJ5tP28fm/leS3thMKAAAAgINnJ1YZAwAAAOAQohACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABjMZRZCVfXMqrqwqt6/bts1q+q1VfXR+e9rzNurqp5aVedW1Xur6tYHMzwAAAAA+28rdwg9O8lxG7Y9OsnruvtGSV43P06Seya50fzn1CRP35mYAAAAAOyUyyyEuvvNST67YfPxSU6fPz49yQnrtj+nJ2clObyqrrtDWQEAAADYAQc6h9AR3f3p+ePPJDli/vj6ST617vPOm7cBAAAAsCK2Pal0d3eS3t+vq6pTq+rsqjr7oosu2m4MAAAAALboQAuhC9aGgs1/XzhvPz/JUes+78h526V09zO6+9juPnbPnj0HGAMAAACA/XWghdDLkpw8f3xykjPXbX/gvNrYHZJ8ft3QMgAAAABWwGGX9QlV9bwkd0pyrao6L8ljkzwhyQur6kFJPpHkh+ZPf0WS709ybpJ/S3LKQcgMAAAAwDZcZiHU3T+8l1133eRzO8mDtxsKAAAAgINn25NKAwAAAHBoUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAzmsKUDcOD+z9Mevshxr/fgJy9yXAAAAGBnuEMIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEctnQALn8+9LTjd/2Y3/bgM/e5/63PuNcuJbnYHU99+a4fEwAAALbCHUIAAAAAg1EIAQAAAAzGkDFYyKtP+/5FjnuPB71ikeMCAACwOtwhBAAAADAYhRAAAADAYLZVCFXVL1bVB6rq/VX1vKr6uqq6QVW9rarOraoXVNWVdiosAAAAANt3wIVQVV0/yUOTHNvd35HkiklOSvLEJL/X3d+a5J+SPGgnggIAAACwM7Y7ZOywJF9fVYcluUqSTye5S5Iz5v2nJzlhm8cAAAAAYAcdcCHU3ecn+d0kn8xUBH0+yTlJPtfdX54/7bwk199uSAAAAAB2znaGjF0jyfFJbpDkekmumuS4/fj6U6vq7Ko6+6KLLjrQGAAAAADsp+0MGfu+JB/v7ou6+z+TvDjJdyU5fB5CliRHJjl/sy/u7md097HdfeyePXu2EQMAAACA/bGdQuiTSe5QVVepqkpy1yR/l+QNSU6cP+fkJGduLyIAAAAAO2k7cwi9LdPk0e9M8r7533pGkkcleXhVnZvkm5KctgM5AQAAANghh132p+xddz82yWM3bP5Ykttt598FAAAA4ODZ7rLzAAAAABxiFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGC2tcoYcPlyxrOOW+S4J57yqkWOCwAAMCp3CAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMJjDlg4AsC/POv3uixz3lJNfs8hxAQAAdoM7hAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGMy2CqGqOryqzqiqD1XVB6vqjlV1zap6bVV9dP77GjsVFgAAAIDt2+4dQr+f5FXd/W1JbpHkg0keneR13X2jJK+bHwMAAACwIg64EKqqqyf5f5KcliTd/R/d/bkkxyc5ff6005OcsL2IAAAAAOyk7dwhdIMkFyV5VlW9q6r+tKqumuSI7v70/DmfSXLEZl9cVadW1dlVdfZFF120jRgAAAAA7I/tFEKHJbl1kqd3962S/Gs2DA/r7k7Sm31xdz+ju4/t7mP37NmzjRgAAAAA7I/tFELnJTmvu982Pz4jU0F0QVVdN0nmvy/cXkQAAAAAdtIBF0Ld/Zkkn6qqm8yb7prk75K8LMnJ87aTk5y5rYQAAAAA7KjDtvn1P5/kuVV1pSQfS3JKppLphVX1oCSfSPJD2zwGAAAAADtoW4VQd787ybGb7Lrrdv5dAAAAAA6e7cwhBAAAAMAhSCEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxmu8vOAwzpqc+9x64f86E/8updPyYAAHD55A4hAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwVhlDOBy4jdeuPsrn/3GD1n5DAAADkXuEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGMxhSwcA4PLrlJcct8hxn3WfVy1yXAAAOFS4QwgAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYzGFLBwCA3XTPMx+8yHFfefzTFjkuAABsxh1CAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgFEIAAAAAg1EIAQAAAAxGIQQAAAAwGIUQAAAAwGAUQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMJhtF0JVdcWqeldVvXx+fIOqeltVnVtVL6iqK20/JgAAAAA7ZSfuEPqFJB9c9/iJSX6vu781yT8ledAOHAMAAACAHbKtQqiqjkzyA0n+dH5cSe6S5Iz5U05PcsJ2jgEAAADAztruHUJPSfLIJF+dH39Tks9195fnx+cluf42jwEAAADADjrsQL+wqu6V5MLuPqeq7nQAX39qklOT5Oijjz7QGABwyPv+lzx+keO+4j6/tshxAQBY3nbuEPquJD9YVf+Q5PmZhor9fpLDq2qtaDoyyfmbfXF3P6O7j+3uY/fs2bONGAAAAADsjwMuhLr7Md19ZHcfk+SkJK/v7h9J8oYkJ86fdnKSM7edEgAAAIAdsxOrjG30qCQPr6pzM80pdNpBOAYAAAAAB+iA5xBar7vfmOSN88cfS3K7nfh3AQAAANh5B+MOIQAAAABWmEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGMxhSwcAAFbTD7z4qbt+zL+670N3/ZgAACNyhxAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBiFEAAAAMBgrDIGABwyfuBFf7rrx/yr+/3kPvff64zn7lKSS3r5iT+yyHEBgMsHdwgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMRiEEAAAAMBjLzgMAXM7c+4wXL3LcvzzxvoscFwDYf+4QAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwlp0HAOCgO/6MVy1y3DNPPG6f++/zorfsUpKLveR+373P/fd/0Xt3KcnF/uJ+N9/1YwKwLHcIAQAAAAxGIQQAAAAwGEPGAACAfXroSz61yHGfep+j9rrvaS+5YBeTXOzB9zlir/tefMY/7mKSi933xGstclzg0OYOIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwRy2dAAAAAAOnjc896JdP+adf2TPPve/608v3KUkF7vVT157148Jq8wdQgAAAACDUQgBAAAADEYhBAAAADAYhRAAAADAYBRCAAAAAINRCAEAAAAMxrLzAAAADO8fnvKZRY57zMOus9d9n/ndc3cxycWu88vfutd9FzzlnF1McrEjHnabfe6/4Klv3J0g6xzx0Dvtc/+FT/vL3QmyzrUffO8tf647hAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABnPAhVBVHVVVb6iqv6uqD1TVL8zbr1lVr62qj85/X2Pn4gIAAACwXdu5Q+jLSX6pu2+a5A5JHlxVN03y6CSv6+4bJXnd/BgAAACAFXHAhVB3f7q73zl//IUkH0xy/STHJzl9/rTTk5ywzYwAAAAA7KAdmUOoqo5Jcqskb0tyRHd/et71mSRH7MQxAAAAANgZ2y6EqupqSV6U5GHd/c/r93V3J+m9fN2pVXV2VZ190UUXbTcGAAAAAFu0rUKoqv5LpjLoud394nnzBVV13Xn/dZNcuNnXdvczuvvY7j52z54924kBAAAAwH7YzipjleS0JB/s7iev2/WyJCfPH5+c5MwDjwcAAADATjtsG1/7XUl+LMn7qurd87ZfSfKEJC+sqgcl+USSH9pWQgAAAAB21AEXQt39liS1l913PdB/FwAAAICDa0dWGQMAAADg0KEQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAKIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMAohAAAAgMEohAAAAAAGoxACAAAAGIxCCAAAAGAwCiEAAACAwSiEAAAAAAajEAIAAAAYjEIIAAAAYDAHrRCqquOq6sNVdW5VPfpgHQcAAACA/XNQCqGqumKSpyW5Z5KbJvnhqrrpwTgWAAAAAPvnYN0hdLsk53b3x7r7P5I8P8nxB+lYAAAAAOyH6u6d/0erTkxyXHf/5Pz4x5Lcvrsfsu5zTk1y6vzwJkk+vEOHv1aSf9yhf2unyLQ1q5gpWc1cMm2NTFu3irlk2hqZtm4Vc8m0NTJt3SrmkmlrZNq6Vcwl09bItHU7leubu3vPZjsO24F//IB09zOSPGOn/92qOru7j93pf3c7ZNqaVcyUrGYumbZGpq1bxVwybY1MW7eKuWTaGpm2bhVzybQ1Mm3dKuaSaWtk2rrdyHWwhoydn+SodY+PnLcBAAAAsLCDVQi9I8mNquoGVXWlJCcledlBOhYAAAAA++GgDBnr7i9X1UOSvDrJFZM8s7s/cDCOtYkdH4a2A2TamlXMlKxmLpm2RqatW8VcMm2NTFu3irlk2hqZtm4Vc8m0NTJt3SrmkmlrZNq6g57roEwqDQAAAMDqOlhDxgAAAABYUQohAAAAgMEohAAAAAAGoxA6iKrqmlV1zaVzAKujqm69dAb2n/M5AACXN4dsIVRVh1XVT1fVq6rqvfOfV1bVz1TVf1kw19FV9fyquijJ25K8vaounLcds1SuOdvhVfXQqnpyVT117c+Cea5YVR9a6vj7o6oWmXl+fo5+uqoeV1XftWHfry2RaT72VarqkVX1iKr6uqr68ap6WVU9qaqutmCuK1TVFeaPr1RVt17yTfx8/PV/bpPkZVV1q1UqhqrqIwsf/1uq6plV9fiqulpV/a+qen9V/cWS581VPJ/PP+M/UVV/VVXvqap3znnutESevVGe7VtVHbHuvHDE0nk49FXVDTbZdtslsnDgquoHl84ArIb5NfFBf191yK4yVlXPS/K5JKcnOW/efGSSk5Ncs7sfsFCutyZ5SpIzuvsr87YrJrl/kod19x2WyDXn+NskZyV5X5Kvrm3v7tMXzHRmkp/v7k8ulWFdlr29gakk7+nuI3czT5JU1Z8muUqStyf5sSRv6u6Hz/ve2d2LlApV9cIkn0ry9UlukuSDSV6Q5AeTXKe7f2yBTCck+ZNMP9s/k+RXkvzLnO9nu/svF8j01Uz/5760bvMd5m3d3XdZINMXkqyd+Gv++ypJ/m3O9I0LZHpzkucluXqSH03yrCQvTHL3JD+yxPM051q583lVPSvJJ5L8dZITk/xzkr9J8qgkZ3b3HyyQ6de6+/HzxzdN8tIk/yXTz9cDuvttu51pznL1JI9JckKSa2f6ub8wyZlJntDdn1sg0y2T/HGmn/Xz581HZno983Pd/c7dzjTnunqS45Jcf950fpJXL/Ecrct0j0zfu/WZzuzuVy2VaU1V3XeTzZ9P8r7uvnC38yTTa4Ik9+7u8+fH35vkD7v7O5fIM2f4riTv7u5/raofTXLrJL/f3Z9YKtNmqupq3f0vCxx3489RJXlakp9Lku5+8W5nukSYqsOTPDDJMUkOW9ve3Q9dKFKq6pwkz0zyv7v7n5bKsTdV9dvd/StL59hMVb2yu++50LFvl+k15jvm1wnHJflQd79iiTzrVdWxSY5K8pUkH+nuxW9WqKrvTPKcJNfMdF64KMnJ3f3+g3K8Q7gQ+kh333h/9x1sVfXR7r7R/u7bDUsWCHszvxG8VabC41/Xtnf3rl8hqaqvZHqjVes29/z4+t19pQUyvbe7bz5/fFiSP0pyrSQ/nOSs7r7Vbmeas7y7u29ZVZXk00mu2909P37PWuZdzvSuJPfMVFK9J8ltu/vDVfXNSV7U3ccukOl+SR6a6c3nK+dtH+/uS13J3cVMT01yeJJHdPcFK5LpXWs/y1X1ye4+erN9C+RaufP5+nPC/Pis7r5DVV050xuvb18g09d+t1TVX2V6E/rK+QXgU7r7v+52pjnLq5O8Psnp3f2Zedt1Ml04umt3332BTO9O8tMbS7KqukOSP+nuWyyQ6YFJHpvkNblkSXW3JL/Z3c9ZINNTktw40wvi9Rf9Hpjko939C7udab355/yOSd4wb7pTknOS3CDJf+/uP1sg020zvUa4d6bi5X8kuVd3f2q3s6zL9N4kt0hy8yTPTvKnSX6ou793qUyb2fh7ZxeP+59JXp2pqF577XlikjMyvXn+id3OtN6KXkj+1iSnJHlAkrMzXUB6TS/whrYuPcqiMl28fU6yTHG2j7vPK8nLu/u6u5knSarqsZlenx+W5LVJbp/p3Hm3TBcefmu3M825vjfJ/8x0QeY2Sf6/JNdI8p9Jfmzhc+ffJvnV7n7D/PhOSX77YL2eOuyyP2Vlfbaq7p/pjd5Xk+lW+kxXbpdsjM+pqj/KdOfS2g/SUZlegL5rsVSTP6uqn0ry8qy7Y6G7P7tcpPy/Cx57o49lepNwqbuVqmqpk8LXSqju/nKSU+cT6+uTLDY0a81cAr1i7Rfx/Hixlnndm75PdveH522fmM8NS+R50fym9HFV9RNJfikX352ziO5+aE1D155XVS9N8odLZ0ry1aq6caai6ipVdWx3nz2/8LvigrlW8Xz+n1V1w+7++/mF338kSXd/acn/e+tcb6387O63V9XXL5jlmO5+4voN8zniifP/xyVcdbM7prr7rKq66hKBkvxqkttsvBuoqq6RaajkrhdCSb5/swt7VfWCJB9JsmghlOn187evK9WPyPQ83T7Jm5PseiE0X3l/aKZi74tJvq+7L9rtHBt8eX5dcHymovi0qnrQEkGq6uF725XlXk/91yRPSPKO7n56Mr3x6+5TFsqz0det3ZW+Krr73CS/WlX/b5J7Zbpb6Cvz3bO/v8vvae6T5E2Z/s+tFXonZSqHl/KOTJlqk32H726UrzkxyS2TXDnJZ5Ic2d3/XFW/m+l3zCKFUKY7wO/e3RfNQ26f3N3fVVV3S3JaprvUl3LVtTIoSbr7jQfzNcKhXAidlOSJSZ5WVZ+btx2eqXE8aaFMyXT16kFJfjOXvM35ZZl+uJb0H0l+J9OLv7U3Dp3kWxZLNF01+vMVue3zKZma4c2Grz1pd6N8zdlVddz6W+S7+zer6vwkT18o01quq3X3v6y/glVVN0zyhaVCVdUV5oJ4faYrZl2xttvm29B/sapulalY+Ialsqzp7nOq6vuSPCTTC4evWzjSI5P8ZaYrkCckeUxV3TzTsJqfWjDXKp7PH5HkDVX1pUy/w09Kkqrak6nsX8K3VNXLMr0APbKqrtLd/zbvW2xOvySfqKpHZrpDaP0b9x/PxQXfbnvlfHfJc3LJkvGBSZYaClXZvBT+ajZ/U7EbvlhVt+3ud2zYfttMZcfSjlr7mZpdOG/77HzXx66pqr/MJb9/V8k0fO20qlrkjut1vlBVj8l018T3zBdnljon/Ham18Bf3mTfUheN3jG/+fz5qnpDpqG/q1Dsr1nFC8mZXx+ckuT7k7woyXOTfHemi6W33MUoN03yuEzDn365u/9PVT12yTuoMk3h8NPd/dGNOxa8uP3lnobd/1tV/X13/3OSdPe/1zS1wlKuuK40/2SSb06S7n7tfJfqkj42l55rFxd+NNONCwfFITtkLEmq6tuTHJ9Ljy//4HKpVldVfSzJ7br7H5fOsqaqHp/pDc07M7X8r17its91eb4tl/6ZetmSP1OrmCnZe65MY4KXuHX3tpnmb/jihu3HJPnu7v7z3c600Tyk7hvWfhkumGPte3dkpjcP5yZ56cI/52vn8+tlehN6Qabz+fuWyrSq5ufqfkmuk+lN+6LnhPm26/XO6e5/mcuXE7v7aQvlukaSR2f6ubr2vPmCTOepJyx1IaKq7pnNz+mLzKVQVScn+fVMV7nX3jAcnel2/sd197MXyHTrTBc9viEXDxk7KlPR8eDuXvIKfOY7B49O8hfzpvtlyvmITMMy7ryLWfY5/Kq737RbWTaah2j+t0x3wPxNVR2d5E4LDUP820xzVl7qZ6eqPtXdR+12pg0Zrp/k95Ic291LXqj9mqp6cKa7Nz6XdReSl8xX0xxCn8t0UeZF3f2ldfte3N2bze91sDPdJsnvJvmrJA/p7mN2O8O6LCdmei384U32ndDdL10g09uS3Lm7/23dhdu1ueve0MvNh/rMTD/Xr880B+r53f3wqrpKknd297ctkWvOdo1MFyO/e874N5mGcB+U1y2HbCFUVY/KVCQ8P5cc835Skud39xMWynVYpivKJ2RDUZXktO7e1StH61XVa5KcsO7K7UqY3yTfPVPbf2ymyWRP6+6/3+Ucj8z0wuX5ueScBYv9TK1iplXOtabmCcJX4CrWVTLdhdNJ/iDTmPf7JflQpnkmlpjE8lGZ5qBame/d/PO0lmmVzuffkuTX5kxPzPRi/Y6ZrsA9orv/YYFMK/lcHWqq6pTuftbSOVbF/OLzHrn0pNKL3r07Fwpfy7Q2LHhp8+uW+2Z6sZ5Mc0+8aOELWkfkks/VBfv6/N2wSpmq6iZJPrvZMLqqOmIVnq9Vs6IXkr+luw/anRIHaj4n/FySO3b3jy6dZzNL/d6rqiuvL+7Wbb9WpjlIF7nwV9Oq5D+V6U6v9yR5Znd/ZR7qfu1ecPL7qvrO3XxeDuVC6CNJbraxYKmqKyX5QC80eXOt6OpnSVJVL0lys0zD6tbf+rnYagFrquoWmQqh4zLlu0OS13b3I3cxw8r9TK1iplXNNV95fFKSu2b6P1hJvjFT8//ohd68r+JqbKv4vVu5TPPxN65+9uxM37/FVj9bxedqHgZycqay82srdST54+5+427n2YpabhLZtZXPjk9yRFZg5bN12VbmzfucZ+VWPluzSs9V7Xvlup/t7l2f72xDpvMyDyddMtMqqhVcCXG9VbyQPP/f++1M89Xds6YVq+7Y3UtPy7Hylvq9Nx/7CknS3V+dX698R5J/WPrC7aqqqr/JNOfSszKtqPf5g3m8Q3kOoa9mGlqwsb27btbNhL+A2/SlJ0I8L8lZ8wv5Jb10/rMyquoXMs2d8I+ZVp94RHf/53zi+GimuUV2yyr+TK1ipmQ1c70g0zxQP9KXXiL8+ZlKxt124+7+ofnK0aczTfLZVfWWTFcjlrCK37tVzJRMw/vWJvr8ue7+3Xn7aVX1kIUyreJzdVqmPE/INHnkP2e6vfnX5qtcf7BEqJpWONp0V6YyZgkvzFRS37kvufLZj8/7llj57JbZ5M17TfMz/lx3v3OBTJutfHbnJL9dVYusfDbnumU2KV+WfK4yFdV7W7nu2ZlW+Ro+04qWL2vngzv1pVdCXOR8sMG/Jnl3TfMbrcqF5GdnepP8q/Pjj2R6/bfrhVBVHZVpXqrrJ3llkt9Zu1hTVS/t7hMWyLRyv/eq6oQkf5JpAZGfSfIrSf4lyU2q6me7+y+XyLUvVfXK7r7nUsfv7u+pabGVUzItcPL2JM/q7tcejOMdyoXQw5K8rqo+mkuOef/WTEM0lrKqq5+lu0+fW9lvy/SL8MPd/R9LZkpyzST33Xhb3twg32uXszwsq/cztYqZktXMda3ufsH6DXMx9PyqetxCmdZyrNJqbA/L6n3vVjFTspqrnz0sq/dc3aYvXhXnLVV1Vnf/+nyH1bszDZdcwhGZhkFt/N1bSf529+Mk2fvKZ0+oqqVWFnp29v7m/VlZplBYxZXPktV8rlZx5bpVzLSK5csqroS43kuzYheSM73We2FNk5Wnu79cVV9ZKMszM01qfVam6ULeVFX37u7/m3mC4gWs4u+9x2Y6N359pouht+3uD1fVN2d6/hYphGqaq27TXdndyck31d0fqapfS3J2kqcmudV8gflXuvvFO3msQ7YQ6u5XzS/Ub5dL3k78jrW7Axayqqufpaq+P1ND+/eZfthvUFU/3fMSwUvo7sdW1RWr6npZ9/PY3Z/sXZ4gdRV/plYx0wrnWsUlwlduNbZV/N6tYqbZyq1+tqLP1X9W1Q27++/nF1j/MWf90oLFZzKtjHO17n73xh1V9cZdTzP5RK3eymer+OZ9FVc+S1bzuVrFletWMdMqli+reD74ml52tay9+deq+qbM54e5jD2ow2n2YU93//H88c9X1Y8meXNV/WA2P3/thlX8vbf2f21t2NqH522fWBtKtpB3ZFppd7PfKYfvbpRLqotX0vuBJK9Ncu/ufuf8fvmtSXa0EDpk5xBaZbWiq59V1YeS3Ku7z50f3zDJX/Wys6g/JMlvZFr5ZW24Q3f3zZfKxKFpvvvtQdl85bPTepMJ7XYp10qtxsb+KaufXaaqukumOye+lKnYP6m731ZVezINA97Nob8rrVZw5bOqemqSG2bzN+8f7+5dv/OsVnDlsznXyj1Xc66VWrluFTPVNB/OX2fz8uVu3f19C2RaufPBelV1oyT/I9Oku1+3tr2XXWXs1pnuOv2OJO9PsifTapZ7Gyp1MLN8INOdjF9ct+37Mg0rvWp3X3e3M62iqnpXpufpq1V1u+5++7z9ikne093fsVCu9ye5T3d/dJN9i648WFVvyjSVyhnd/e8b9v1Yd//Z5l95gMfzfmRn1YqufjZne0d333bd40ry9vXbFsh0bpLbz7dXwuVKrfhqbOxbWdFry+bi7H5JrpOp3F9787fohZBDSS248tmqvXmfM63qymcr91xx2Va9fNloyfPBugxvyTTc5/eS3DvTHQtX6O5fXzjXYZkW6qhM018ssoJzVf1ipuXJ37Rh+62SPKm777ZErlVTVbdN8r71xdm8/Zgk393df75QrhPnXB/eZN8J3f3S3U+1DIXQDqsVXAFmXYanZxrT+sJMtzLeP8knM10xyU6PR9xipjdkujLz5d0+NpcvtZpLvK/s+YDL5vu3NYqznVELrgCzqmqFVvNaVbWCK9etYqZ9WYXyZaNVOB9U1TndfZuqel93f+f6bQtkue++9i/xHobLrxU9Jxy0ia4P2TmEVtgqrgCz5usyXQ353vnxRZkm+Lp3pl/WS5xMP5bkjfNY8/UrGDx5gSwc2p6di5d4/6tMS7z/TqYl3p+eZNeXeM9qnw+4bL5/W/OT2bw4e3KSD2RafYys7AowK/fmvVZw5bM518o9V1nBletWNNO+/GamScF31SqeDzb40jzHy0fnKR7OT3K1hbLcex/7lnoPkySpqicleXySf880R9bNk/ziUne+HEoOZsmxTUudExaZ6NodQjusqo5L8oeZlky/1Aow3b3UZHorqaoeu9n27v7N3c7Coa2q3t3dt5yHQn46yXW7u+fH71liXirng0Ob79/WzPPT3aM3rBZZ0woir+numyyTbPVU1QXZxwow3X29BTK9OtOb99M3efN+l+7e9TfvVfXu7H01rz/p7iVW81rV5+rDe/s/tq99A2baV/ly4+6+8m7mSVbzfHCJENNQnw9mmmD3cZkK2id191kL5blCpvmCXrjE8fdm3evP+yS5V5KHJ3nzUuepVXMZJcfLl5praUXPCV/J3ie6vkN3f/3BOK47hHZYr+AKMPNY2wdlWiXnEhNdZ5psd5Gxt4nih503l0ArscT7Kp4P2Drfvy17WJLXVdWmxdlSoVbUKq4As7fVl55QVacslGkVV/NKVvO5+kSt3kpVq5hpFZfjXsXzwdd09zvmD/8l0/xBi5onJX5kprvMVsna++kfSPIX3f356Voks1VdzWsVzwkfzHQxZNOJrg/WQRVCB0F3fzXJIu35XvxZks9luv1t/cS2Jyf580xzrSyiplVoHpnkZrnkCgZ3WSoTh6yVW+I9WcnzAfvB9++yKc62rrsftI99/203s6yzim/eV3HZ8mQ1n6sHZJos+U1VtXGy5PvL9DUrV76s6Plg/dDIEzJNwL0KQyPX/HVV/XKSFyT517WN3f3Z5SLl5fOdsv+e5Gfn9zZfvIyvGckiJccWrNw5IdPK21fYy76fP1gHNWRsAFX1ke6+8f7u2w01LQP6giS/nORnMpVUF3X3o5bKxKGrLPEOsF9WdfWlVVzNa1Wfq71Z0YlRVy4Tl7SKQyPXZfv4Jpu7u79l18OsU1XXTPL57v7KfBfjN6w9d6OzmtfOOJjnToXQAKrqrCT/M8mL5qvda+Nw75/k4d19+wWzra1g8N61OV6q6h3dfdulMnFoKku8A+wob963bhWfq1VYqWqjVczEJa3iHFCryMpn27eK581VdTDPnQqhAVTVMUmemOTOmYaOJdOYzTckeXR3b9a274qqOqu77zBfjXhqkv+T5IzuvuFSmTg0lSXCAXbUUm/eV3Q1r31a8LlaxYlRVy4TWzffvf/X2Xxo5N26+/sWjJeq+o4kN80lp5p4zgI51oqMayf5r5nuqkqm91t/29332u1MhxoF8SUtde40h9AAuvsfquo3krwrGyaVXrIMmj1+fuH3S0n+IMk3JvnFZSNxiLJEOMB+uowXoEstfb2Sy5av6HO1ihOjrmImtm4V54BK8rXVie+UqRB6RZJ7JnlLpvnGdlV3nzJnek2Sm3b3p+fH103y7N3Os6pW9Ly5qhY5dyqEBlBVj8o8bCbJ2qodRyZ5XlUtMpSmqp44zxP09d39+SSfz9Sow4F6WKx0BLC/VvHN+yqu5pWs5nO1ihOjrmImtmieC+tR859LmP//LTnE58Qkt0jyru4+Zb5z6c8XzJMkR62VQbMLMr3+ZLKK581Vtci505CxAaziUJqqel+Smyc5p7tvvdvH5/JpnhvLSkcAW1RVpyV5Vne/ZZN9/3uJ1Y5WdcjKKj5XsJuWHuJTVW/v7ttV1TmZLiR/IckHu/vbFsz0h0lulOR586YHJDm3uw/aqlCHEufN1acQGsC8FOI9uvsTG7Z/c5LXLDE5XFX9TpKfSnK1JP+2flem1QK+cbczAQDL27Ca19qQgs9kGrLyxIWXmIbLtVWeA6qq/ijJr2Qa+fBLSf4lybvXhm8tmOu+Sb5nfvjm7n7JknlgfyiEBlBVxyX5wySbDqXp7lctkOnK3f2lqjqzu4/f7eMDAACXVFUXZB9DfLr7eruf6tLmRXO+sbv3VmABW2AOoQF096uq6sZZraE0b01y6yT/vNDxAYAVVVX3SHJCLr0Yxq5fxILBrOwcUFV1nySv7+7Pz4vmHF5VJ3T3SxfI8oVMKyBealeMduAQ4g4hFlFV70/y20kel+QRG/d394t3PRQAsLiqekqSG2daOei8efORSR6Y5KPd/QsLRQMWVFXv7u5bbtj2ru6+1UKR4JDnDiGW8jNJfiTJ4UnuvWFfJ1EIAcCYvr+7b7xxY1W9IMlHkiiEYExX2GTb4u9nq+oWueQcQoaxcchY/D8QY5pnmn9LVZ3d3actnQcAWBlfrKrbdvc7Nmy/bZIvLhEIWAlnV9WTkzxtfvzgJOcsmCdV9QuZFspZu5j93Kp6Rnf/wYKxYMsMGWMRVXWX7n79PCv/pRgyBgBjqqpbJ3l6km/IxUPGjkry+SQP7u5F3wACy6iqqyb59SR3nTe9Nsnju/tfF8z03iR3XMswZ3xrd998qUywP9whxFK+N8nrc+nhYokhYwAwrO5+Z5LbV9V1sm5S6e7+zIKxgIXNpcujls6xQSVZv0jPV+ZtcEhwhxCLqqobdPfHL2sbADCOqrp6kuNyyVXGXt3dn1ssFLCI+XzwmEwrD14708XjC5OcmeQJS54XqurhSU5O8pJ50wlJnt3dT1kqE+yPzSbmgt30ok22nbHrKQCAlVBVD0zyziR3SnKV+c+dk5wz7wPG8sIk/5TkTt19ze7+pkznhM/N+xbT3U9OckqSz85/TlEGcShxhxCLqKpvS3KzJE/KJZed/8Ykj+jumy0SDABYVFV9OMntN171r6prJHnbZiuQAZdfVfXh7r7J/u7bDVV1zU02f6G7/3PXw8ABMIcQS7lJknvl0svOfyHTTP0AwJgq05CQjb4ac3PAiD5RVY9Mcnp3X5AkVXVEkh9P8qklg2W6m/GoTHcwVab3Np+pqguS/JRJ8Fl1CiEW0d1nJjmzqu7Y3W9dOg8AsDJ+K8k7q+o1ufjN3tFJ7pbkcYulApbygCSPTvKmqrr2vO2CJC9Lcv/FUk1em+SM7n51klTV3ZPcL8mzkvxRktsvmA0ukyFjLKKqDkvyoCT3SXK9efP5mSaHO81tlgAwrnl42D1y6Uml/2m5VMCqqapTuvtZCx7/fd39nRu2vbe7b15V7+7uWy4UDbZEIcQiqup5mSaCOz3JefPmIzPN0n/N7n7AQtEAgBUwDwlZv+z8BUvmAVZPVX2yu49e8PivSfK6JM+fNz0g092MxyV5R3ffeqlssBUKIRZRVR/Z26SQ+9oHAFy+VdUtk/xxkqtnumhUmS4afS7Jz3X3OxcLB+y6qnrv3nYluXF3X3k381wiQNW1kjw2yXfPm96S5L8n+XySo7v73KWywVaYQ4ilfLaq7p/kRd391SSpqitkGgfsdnAAGNezk/x0d79t/caqukOmeTlusUQoYDFHZBpCuvE9QiX5292Pc7Hu/sckP7+X3cogVp5CiKWclOSJSZ5WVZ+btx2e5A3zPgBgTFfdWAYlSXefVVVXXSIQsKiXJ7lad797446qeuOup5mOe/Ukj0lyQpJrZ1oZ8cJM86E+obs/t0Qu2F+GjLGYqvr2JMfnkhNGntndH1wuFQCwpKp6apIbJnlOLl5l7KgkD0zy8e5+yFLZAJKkql6d5PVJTu/uz8zbrpPkx5PcpbvvvmA82DKFEIuoqkdluhPo+ZmKoGSaH+CkJM/v7icslQ0AWFZV3TOXvmj0su5+xXKpACZV9eHuvsn+7oNVoxBiEVX1kSQ327i8fFVdKckHuvtGyyQDAADYu3l1sb/OdIfQBfO2IzLdIXS37v6+BePBll1h6QAM66tJrrfJ9uvO+wCAAVXV1avqCVX1war6bFX93/njJ1TV4UvnA8i0vPw3JXnTfJ76bJI3JrlmpkVy4JDgDiEWUVXHJfnDJB/NxfMDHJ3kW5M8pLtftVQ2AGA55uYADmVVdUp3P2vpHLAVCiEWMy8zf7tccn6Ad3T3V5ZLBQAsydwcwKGsqj7Z3UcvnQO2wrLzLKa7v5rkrKVzAAAr5RNV9chsPjfHp/b1hQC7oareu7ddSY7YzSywHQohAABWyQOSPDrT3BzXnrddkORlMTcHsBqOSHKPJP+0YXsl+dvdjwMHxpAxAAAOCebmAFZBVZ2W5Fnd/ZZN9v3v7v5vC8SC/aYQAgDgkGBuDgDYOYaMAQCwMszNAQC7QyEEAMAqMTcHAOwChRAAAKvk5Umu1t3v3rijqt6462kA4HLKHEIAAAAAg7nC0gEAAAAA2F0KIQAAAIDBKIQAAAAABqMQAgAAABiMQggAAABgMP8/L2ONOO8OQSoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1440x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cv_test=cv.transform(test_df['text'])\n",
    "test_word_list = cv.get_feature_names();  \n",
    "test_count_list = cv_test.toarray().sum(axis=0)  \n",
    "sorted_index = np.argsort(-test_count_list)\n",
    "\n",
    "x_vect = np.asarray(cv.get_feature_names())\n",
    "\n",
    "g1 = sns.barplot(x=x_vect[sorted_index[:30]], \n",
    "            y=test_count_list[sorted_index[:30]])\n",
    "g1.set_xticklabels(cv.get_feature_names()[:30], rotation = 90);\n",
    "g1.figure.set_size_inches(20,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 2 (Take home): **  \n",
    "Generate an embedding using the TF-IDF vectorizer instead of th BOW one with 1000 features and show the feature names for features [100:110]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['awareness',\n",
       " 'away',\n",
       " 'awe',\n",
       " 'awesome',\n",
       " 'awful',\n",
       " 'b',\n",
       " 'baby',\n",
       " 'back',\n",
       " 'bad',\n",
       " 'ball']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Answer here\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "import nltk\n",
    "# Initialize tfidf vectorizer\n",
    "tfidf_vectorizer = TfidfVectorizer(max_features=1000,tokenizer=nltk.word_tokenize)\n",
    "\n",
    "# Create tfidf train and test variables\n",
    "X_train_tfidf = tfidf_vectorizer.fit_transform(train_df['text']).toarray() \n",
    "tfidf_vectorizer.get_feature_names()[100:110]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 3 (Take home): **  \n",
    "Can you interpret the results above? What do they mean?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Answer here\n",
    "#在anger的data中被預測為sadness有9筆, joy有5筆, fear有15筆, anger有55筆\n",
    "#在fear的data中被預測為sadness有14筆, joy有13筆, fear有70筆, anger有16筆\n",
    "#在joy的data中被預測為sadness有9筆, joy有54筆, fear有9筆, anger有9筆\n",
    "#在sadness的data中被預測為sadness有42筆, joy有7筆, fear有16筆, anger有4筆"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\kevin79577\\appdata\\local\\programs\\python\\python37\\lib\\site-packages\\sklearn\\feature_extraction\\text.py:484: UserWarning: The parameter 'token_pattern' will not be used since 'tokenizer' is not None'\n",
      "  warnings.warn(\"The parameter 'token_pattern' will not be used\"\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(3613, 500)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "import nltk\n",
    "\n",
    "# build analyzers (bag-of-words)\n",
    "BOW_500 = CountVectorizer(max_features=500, tokenizer=nltk.word_tokenize) \n",
    "\n",
    "# apply analyzer to training data\n",
    "BOW_500.fit(train_df['text'])\n",
    "\n",
    "train_data_BOW_features_500 = BOW_500.transform(train_df['text'])\n",
    "\n",
    "## check dimension\n",
    "train_data_BOW_features_500.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 4 (Take home): **  \n",
    "Build a model using a ```Naive Bayes``` model and train it. What are the testing results? \n",
    "\n",
    "*Reference*: https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training accuracy: 0.79\n",
      "testing accuracy: 0.69\n"
     ]
    }
   ],
   "source": [
    "# Answer here\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "clf = MultinomialNB()\n",
    "clf.fit(X_train, y_train)\n",
    "y_train_pred = clf.predict(X_train)\n",
    "y_test_pred = clf.predict(X_test)\n",
    "\n",
    "\n",
    "## accuracy\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "acc_train = accuracy_score(y_true=y_train, y_pred=y_train_pred)\n",
    "acc_test = accuracy_score(y_true=y_test, y_pred=y_test_pred)\n",
    "\n",
    "print('training accuracy: {}'.format(round(acc_train, 2)))\n",
    "print('testing accuracy: {}'.format(round(acc_test, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 5 (Take home): **  \n",
    "\n",
    "How do the results from the Naive Bayes model and the Decision Tree model compare? How do you interpret these differences? Use the theoretical background covered in class to try and explain these differences."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we review the training results of Decision Tree model , the training accuracy score of the decision tree model is 0.99, while the test data result is only 0.66. It seems that the decision tree model is over-fitting and has poor performance on the test data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 6 (Take home): **  \n",
    "\n",
    "Plot the Training and Validation Accuracy and Loss (different plots), just like the images below (Note: the pictures below are an example from a different model). How to interpret the graphs you got? How are they related to the concept of overfitting/underfitting covered in class?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train.shape:  (3613, 500)\n",
      "y_train.shape:  (3613,)\n",
      "X_test.shape:  (347, 500)\n",
      "y_test.shape:  (347,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "import keras\n",
    "\n",
    "\n",
    "# for a classificaiton problem, you need to provide both training & testing data\n",
    "X_train = BOW_500.transform(train_df['text'])\n",
    "y_train = train_df['emotion']\n",
    "\n",
    "X_test = BOW_500.transform(test_df['text'])\n",
    "y_test = test_df['emotion']\n",
    "\n",
    "## take a look at data dimension is a good habbit  :)\n",
    "print('X_train.shape: ', X_train.shape)\n",
    "print('y_train.shape: ', y_train.shape)\n",
    "print('X_test.shape: ', X_test.shape)\n",
    "print('y_test.shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "check label:  ['anger' 'fear' 'joy' 'sadness']\n",
      "\n",
      "## Before convert\n",
      "y_train[0:4]:\n",
      " 3258    sadness\n",
      "2358        joy\n",
      "1217       fear\n",
      "1904       fear\n",
      "Name: emotion, dtype: object\n",
      "\n",
      "y_train.shape:  (3613,)\n",
      "y_test.shape:  (347,)\n",
      "\n",
      "\n",
      "## After convert\n",
      "y_train[0:4]:\n",
      " [[0. 0. 0. 1.]\n",
      " [0. 0. 1. 0.]\n",
      " [0. 1. 0. 0.]\n",
      " [0. 1. 0. 0.]]\n",
      "\n",
      "y_train.shape:  (3613, 4)\n",
      "y_test.shape:  (347, 4)\n"
     ]
    }
   ],
   "source": [
    "## deal with label (string -> one-hot)\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "label_encoder = LabelEncoder()\n",
    "label_encoder.fit(y_train)\n",
    "print('check label: ', label_encoder.classes_)\n",
    "print('\\n## Before convert')\n",
    "print('y_train[0:4]:\\n', y_train[0:4])\n",
    "print('\\ny_train.shape: ', y_train.shape)\n",
    "print('y_test.shape: ', y_test.shape)\n",
    "\n",
    "def label_encode(le, labels):\n",
    "    enc = le.transform(labels)\n",
    "    return keras.utils.to_categorical(enc)\n",
    "\n",
    "def label_decode(le, one_hot_label):\n",
    "    dec = np.argmax(one_hot_label, axis=1)\n",
    "    return le.inverse_transform(dec)\n",
    "\n",
    "y_train = label_encode(label_encoder, y_train)\n",
    "y_test = label_encode(label_encoder, y_test)\n",
    "\n",
    "print('\\n\\n## After convert')\n",
    "print('y_train[0:4]:\\n', y_train[0:4])\n",
    "print('\\ny_train.shape: ', y_train.shape)\n",
    "print('y_test.shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_shape:  500\n",
      "output_shape:  4\n"
     ]
    }
   ],
   "source": [
    "# I/O check\n",
    "input_shape = X_train.shape[1]\n",
    "print('input_shape: ', input_shape)\n",
    "\n",
    "output_shape = len(label_encoder.classes_)\n",
    "print('output_shape: ', output_shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, 500)]             0         \n",
      "_________________________________________________________________\n",
      "dense (Dense)                (None, 64)                32064     \n",
      "_________________________________________________________________\n",
      "re_lu (ReLU)                 (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "re_lu_1 (ReLU)               (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 4)                 260       \n",
      "_________________________________________________________________\n",
      "softmax (Softmax)            (None, 4)                 0         \n",
      "=================================================================\n",
      "Total params: 36,484\n",
      "Trainable params: 36,484\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Model\n",
    "from keras.layers import Input, Dense\n",
    "from keras.layers import ReLU, Softmax\n",
    "\n",
    "# input layer\n",
    "model_input = Input(shape=(input_shape, ))  # 500\n",
    "X = model_input\n",
    "\n",
    "# 1st hidden layer\n",
    "X_W1 = Dense(units=64)(X)  # 64\n",
    "H1 = ReLU()(X_W1)\n",
    "\n",
    "# 2nd hidden layer\n",
    "H1_W2 = Dense(units=64)(H1)  # 64\n",
    "H2 = ReLU()(H1_W2)\n",
    "\n",
    "# output layer\n",
    "H2_W3 = Dense(units=output_shape)(H2)  # 4\n",
    "H3 = Softmax()(H2_W3)\n",
    "\n",
    "model_output = H3\n",
    "\n",
    "# create model\n",
    "model = Model(inputs=[model_input], outputs=[model_output])\n",
    "\n",
    "# loss function & optimizer\n",
    "model.compile(optimizer='adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# show model construction\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "113/113 [==============================] - 1s 3ms/step - loss: 1.3589 - accuracy: 0.3329 - val_loss: 1.2736 - val_accuracy: 0.4179\n",
      "Epoch 2/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 1.1081 - accuracy: 0.6066 - val_loss: 0.9402 - val_accuracy: 0.6657\n",
      "Epoch 3/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.6421 - accuracy: 0.7806 - val_loss: 0.7809 - val_accuracy: 0.7003\n",
      "Epoch 4/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.4224 - accuracy: 0.8583 - val_loss: 0.7824 - val_accuracy: 0.7061\n",
      "Epoch 5/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.3255 - accuracy: 0.8798 - val_loss: 0.7883 - val_accuracy: 0.7089\n",
      "Epoch 6/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.2723 - accuracy: 0.9091 - val_loss: 0.8721 - val_accuracy: 0.6916\n",
      "Epoch 7/25\n",
      "113/113 [==============================] - 0s 2ms/step - loss: 0.2202 - accuracy: 0.9226 - val_loss: 0.8851 - val_accuracy: 0.6830\n",
      "Epoch 8/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.1779 - accuracy: 0.9496 - val_loss: 0.9236 - val_accuracy: 0.6916\n",
      "Epoch 9/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.1605 - accuracy: 0.9486 - val_loss: 1.0439 - val_accuracy: 0.6916\n",
      "Epoch 10/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.1330 - accuracy: 0.9609 - val_loss: 1.0524 - val_accuracy: 0.6830\n",
      "Epoch 11/25\n",
      "113/113 [==============================] - 0s 2ms/step - loss: 0.1186 - accuracy: 0.9677 - val_loss: 1.1413 - val_accuracy: 0.6801\n",
      "Epoch 12/25\n",
      "113/113 [==============================] - 0s 2ms/step - loss: 0.1069 - accuracy: 0.9701 - val_loss: 1.1724 - val_accuracy: 0.6715\n",
      "Epoch 13/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.1076 - accuracy: 0.9624 - val_loss: 1.2522 - val_accuracy: 0.6686\n",
      "Epoch 14/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0879 - accuracy: 0.9713 - val_loss: 1.2555 - val_accuracy: 0.6715\n",
      "Epoch 15/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0813 - accuracy: 0.9767 - val_loss: 1.3011 - val_accuracy: 0.6686\n",
      "Epoch 16/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0972 - accuracy: 0.9697 - val_loss: 1.3503 - val_accuracy: 0.6628\n",
      "Epoch 17/25\n",
      "113/113 [==============================] - 0s 2ms/step - loss: 0.0719 - accuracy: 0.9741 - val_loss: 1.3563 - val_accuracy: 0.6830\n",
      "Epoch 18/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0714 - accuracy: 0.9749 - val_loss: 1.4124 - val_accuracy: 0.6744\n",
      "Epoch 19/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0730 - accuracy: 0.9761 - val_loss: 1.4227 - val_accuracy: 0.6542\n",
      "Epoch 20/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0687 - accuracy: 0.9745 - val_loss: 1.4231 - val_accuracy: 0.6801\n",
      "Epoch 21/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0683 - accuracy: 0.9779 - val_loss: 1.5134 - val_accuracy: 0.6686\n",
      "Epoch 22/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0601 - accuracy: 0.9798 - val_loss: 1.4912 - val_accuracy: 0.6715\n",
      "Epoch 23/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0655 - accuracy: 0.9802 - val_loss: 1.5588 - val_accuracy: 0.6657\n",
      "Epoch 24/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0606 - accuracy: 0.9794 - val_loss: 1.5382 - val_accuracy: 0.6657\n",
      "Epoch 25/25\n",
      "113/113 [==============================] - 0s 1ms/step - loss: 0.0597 - accuracy: 0.9780 - val_loss: 1.5648 - val_accuracy: 0.6599\n",
      "training finish\n"
     ]
    }
   ],
   "source": [
    "from keras.callbacks import CSVLogger\n",
    "\n",
    "csv_logger = CSVLogger('logs/training_log.csv')\n",
    "\n",
    "# training setting\n",
    "epochs = 25\n",
    "batch_size = 32\n",
    "\n",
    "# training!\n",
    "history = model.fit(X_train, y_train, \n",
    "                    epochs=epochs, \n",
    "                    batch_size=batch_size, \n",
    "                    callbacks=[csv_logger],\n",
    "                    validation_data = (X_test, y_test))\n",
    "print('training finish')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>epoch</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>loss</th>\n",
       "      <th>val_accuracy</th>\n",
       "      <th>val_loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.357598</td>\n",
       "      <td>1.332735</td>\n",
       "      <td>0.417867</td>\n",
       "      <td>1.273603</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.636867</td>\n",
       "      <td>1.012044</td>\n",
       "      <td>0.665706</td>\n",
       "      <td>0.940164</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.785497</td>\n",
       "      <td>0.603749</td>\n",
       "      <td>0.700288</td>\n",
       "      <td>0.780876</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.845281</td>\n",
       "      <td>0.433205</td>\n",
       "      <td>0.706052</td>\n",
       "      <td>0.782405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.871021</td>\n",
       "      <td>0.350591</td>\n",
       "      <td>0.708934</td>\n",
       "      <td>0.788291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0.899253</td>\n",
       "      <td>0.288830</td>\n",
       "      <td>0.691643</td>\n",
       "      <td>0.872090</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>0.914476</td>\n",
       "      <td>0.236440</td>\n",
       "      <td>0.682997</td>\n",
       "      <td>0.885066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>0.936895</td>\n",
       "      <td>0.197970</td>\n",
       "      <td>0.691643</td>\n",
       "      <td>0.923639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>0.947966</td>\n",
       "      <td>0.167950</td>\n",
       "      <td>0.691643</td>\n",
       "      <td>1.043917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>0.953501</td>\n",
       "      <td>0.148558</td>\n",
       "      <td>0.682997</td>\n",
       "      <td>1.052362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>0.962912</td>\n",
       "      <td>0.127842</td>\n",
       "      <td>0.680115</td>\n",
       "      <td>1.141340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>0.964019</td>\n",
       "      <td>0.118419</td>\n",
       "      <td>0.671470</td>\n",
       "      <td>1.172430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>0.964019</td>\n",
       "      <td>0.111086</td>\n",
       "      <td>0.668588</td>\n",
       "      <td>1.252189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>0.969554</td>\n",
       "      <td>0.097044</td>\n",
       "      <td>0.671470</td>\n",
       "      <td>1.255497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>0.970662</td>\n",
       "      <td>0.094391</td>\n",
       "      <td>0.668588</td>\n",
       "      <td>1.301141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>0.971215</td>\n",
       "      <td>0.097478</td>\n",
       "      <td>0.662824</td>\n",
       "      <td>1.350277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>0.972045</td>\n",
       "      <td>0.083921</td>\n",
       "      <td>0.682997</td>\n",
       "      <td>1.356267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>0.972876</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.674352</td>\n",
       "      <td>1.412435</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>0.974813</td>\n",
       "      <td>0.079464</td>\n",
       "      <td>0.654179</td>\n",
       "      <td>1.422704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>0.974260</td>\n",
       "      <td>0.072585</td>\n",
       "      <td>0.680115</td>\n",
       "      <td>1.423147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>0.975644</td>\n",
       "      <td>0.076258</td>\n",
       "      <td>0.668588</td>\n",
       "      <td>1.513445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>0.977027</td>\n",
       "      <td>0.069191</td>\n",
       "      <td>0.671470</td>\n",
       "      <td>1.491238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>0.975920</td>\n",
       "      <td>0.072916</td>\n",
       "      <td>0.665706</td>\n",
       "      <td>1.558761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>0.976474</td>\n",
       "      <td>0.069565</td>\n",
       "      <td>0.665706</td>\n",
       "      <td>1.538168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>24</td>\n",
       "      <td>0.976751</td>\n",
       "      <td>0.063853</td>\n",
       "      <td>0.659942</td>\n",
       "      <td>1.564751</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    epoch  accuracy      loss  val_accuracy  val_loss\n",
       "0       0  0.357598  1.332735      0.417867  1.273603\n",
       "1       1  0.636867  1.012044      0.665706  0.940164\n",
       "2       2  0.785497  0.603749      0.700288  0.780876\n",
       "3       3  0.845281  0.433205      0.706052  0.782405\n",
       "4       4  0.871021  0.350591      0.708934  0.788291\n",
       "5       5  0.899253  0.288830      0.691643  0.872090\n",
       "6       6  0.914476  0.236440      0.682997  0.885066\n",
       "7       7  0.936895  0.197970      0.691643  0.923639\n",
       "8       8  0.947966  0.167950      0.691643  1.043917\n",
       "9       9  0.953501  0.148558      0.682997  1.052362\n",
       "10     10  0.962912  0.127842      0.680115  1.141340\n",
       "11     11  0.964019  0.118419      0.671470  1.172430\n",
       "12     12  0.964019  0.111086      0.668588  1.252189\n",
       "13     13  0.969554  0.097044      0.671470  1.255497\n",
       "14     14  0.970662  0.094391      0.668588  1.301141\n",
       "15     15  0.971215  0.097478      0.662824  1.350277\n",
       "16     16  0.972045  0.083921      0.682997  1.356267\n",
       "17     17  0.972876  0.079587      0.674352  1.412435\n",
       "18     18  0.974813  0.079464      0.654179  1.422704\n",
       "19     19  0.974260  0.072585      0.680115  1.423147\n",
       "20     20  0.975644  0.076258      0.668588  1.513445\n",
       "21     21  0.977027  0.069191      0.671470  1.491238\n",
       "22     22  0.975920  0.072916      0.665706  1.558761\n",
       "23     23  0.976474  0.069565      0.665706  1.538168\n",
       "24     24  0.976751  0.063853      0.659942  1.564751"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's take a look at the training log\n",
    "training_log = pd.DataFrame()\n",
    "training_log = pd.read_csv(\"logs/training_log.csv\")\n",
    "training_log"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0v0lEQVR4nO3dd3xUZfb48c8hlNBbQJAWpKNSlth1BVkUG4gN0FWxwIpiXXdtu4qKX8vyUyxYcAUEdcFVUVTQBYVVVwUCAioBAkgJnYC0EEg5vz+eO8kkZJJJyOQmmfN+veY1d26bc2eSe+Yp97miqhhjjDEAVfwOwBhjTPlhScEYY0wOSwrGGGNyWFIwxhiTw5KCMcaYHJYUjDHG5LCkYI6ZiMwWkRtKe10TPURERaS933EYELtOITqJyIGgl7WAw0CW9/pPqvpO2Ud17ESkLbAWeF1VR/odjwmPiCjQQVXX+B1LtLOSQpRS1TqBB7ARuDRoXk5CEJGq/kVZItcDe4DBIlKjLN9YRGLK8v1KQ0WM2USWJQWTh4j0FpEUEblfRLYBk0SkoYh8KiI7RWSPN90yaJv5InKLNz1MRL4VkbHeur+KyIUlXLetiHwtIvtFZK6IjBeRtwuJXXBJ4W9ABnBpvuUDRWSpiOwTkbUi0t+b30hEJonIFi+Oj4Ljy7ePnGoOEZksIq+KyCwROQj0EZGLReRH7z02icjofNufLSLfichv3vJhInKKiGwPPkGLyOUisizEcU4WkddEZI732fxXRNoELe/sLdstIqtE5Op82+aJuYD91xeRN0Vkq4hsFpExgdi8eP8nIi+LyF4RWSkifYO2PV5EZnrvvUZEhgctixGRh7zPfr+ILBaRVkFv/QcRSfY+m/He92nKmCUFU5BmQCOgDTAC93cyyXvdGjgEvFzI9qcBq4A44FngzUL+wQtb911gIdAYGA1cV0TcZwMtgWnAe0BO24WInApMAf4CNAB+D6z3Fk/FVaGdCDQFni/ifYJdAzwJ1AW+BQ7iElMD4GJgpIhc5sXQBpgNvAQ0AXoAS1V1EZAKnB+03+u8eEO5FngC97ktBd7x3qM2MAf32TUFhgCviEjXQmLObzKQCbQHenpx3RK0/DRcFV0c8CjwoYg08pZNA1KA44Ergf8TkfO8ZfcCQ4GLgHrATUBa0H4vAU4BugFXAxcUcvwmUlTVHlH+wJ0c/+BN9waOALGFrN8D2BP0ej5wizc9DFgTtKwWoECz4qyLSz6ZQK2g5W8DbxcS1z+Bj7zpM3Clhabe69eB5wvYpjmQDTQsYNkw4Nt88xRo701PBqYU8dmOC7wv8CAwI8R69wPveNONcCfL5iHWnQxMC3pdB9ce1AoYDHyTb/3XgUfDiRk4Dte+VDNo3lBgXtBnsgWvPdKbtxCXxFp5cdQNWvYUMNmbXgUMDPG+Cpwd9Po94AG//zei8WElBVOQnaqaHnghIrVE5HUR2SAi+4CvgQYSuj56W2BCVQO/BOsUc93jgd1B8wA2hQpYRGoCV+H9YlbV73FtJdd4q7TC/brNr5X3PntC7bsIeWISkdNEZJ64qra9wK24X9SFxQAu4V3q/dK/Gndi3xrO+6rqAWA37jNrA5zmVcH8JiK/4UoVzULFnE8boBqwNWj713GljoDN6p25PRu89w58Z/vzLWvhTRd2/BD0t4BLiqH+ZkwEWVIwBcnfJe3PQCfgNFWth6t6AYhkne9WoJGI1Aqa1yrUysAgXJXEKyKyTVx7SAtyq5A2Ae0K2G6T9z4NClh2EFd6AUBEmhWwTv7P6l1gJtBKVesDr5H7OYWKAVXdDHwPXI771T21oPWC5HwWIlIHV7rY4r3Hf1W1QdCjjubtiVVYl8NNuJJCXND29VT1xKB1WuSrDmztvfcW3GdZN9+yzUH7LvD4TflhScGEoy6uHeE3r+740Ui/oapuABKB0SJSXUTOIF/DcT43ABOBk3HVWz2As4DuInIy8CZwo4j0FZEqItJCRDp7v8Zn45JJQxGpJiKBpLcMOFFEeohILK5doyh1cb+W0712jGuClr2Da0y9WkSqikhjEekRtHwK8FfvGD4s4n0u8hqtq+PaFn5Q1U3Ap0BHEbnOO5ZqXkN2lzBix/s8/gP8PxGp531W7UTk3KDVmgJ3evu+CugCzPLe/zvgKRGJFZFuwM24UhC46r0nRKSDON1EpHE4cZmyY0nBhGMcUBPYBfwAfF5G73strm0gFRgDTMf9is1DRFoAfYFxqrot6LHYi/UGVV0I3IhrRN4L/BdXVQLul3kGsBLYAdwNoKqrgceBuUAyBTfK5ncb8LiI7AcewdWN4+1vI66R9c+46p6lQPegbWd4Mc3IV21WkHdxyXk30Av4o/ce+3ENw0Nwv9y3Ac8Axemeez1QHViB6977Pq7tJWAB0AH39/AkcKWqpnrLhgLx3nvPwLVlzPWWPYf7PP4D7MMl6prFiMuUAbt4zVQYIjIdWKmqES+p+EVE1uIuHpxbyDqTgRRV/VuZBZb73sNwHQXOLuv3NmXDSgqm3PKqPdp5VRj9gYHARz6HFTEicgWuvv8rv2Mx0auiXa1qokszXN16Y1zf95Gq+qO/IUWGiMwHugLXqWq2z+GYKGbVR8YYY3JY9ZExxpgcEas+EpGJuMvWd6jqSQUsF+AFXG+MNGCYqi4par9xcXEaHx9fytEaY0zltnjx4l2q2qSo9SLZpjAZNz5OqPFbLsR1a+uAG0vlVe+5UPHx8SQmJpZSiMYYEx1EZEM460Ws+khVv8b1oQ5lIG4MFlXVH3DDJjQvZH1jjDER5mebQgvyjsGSQu4YKXmIyAgRSRSRxJ07d5ZJcMYYE40qREOzqk5Q1QRVTWjSpMgqMWOMMSXkZ1LYTN4BzlqSO3CWMcYYH/iZFGYC13sDY50O7C1iqGBjjDERFskuqf/C3bAlTkRScIN3VQNQ1deAWbjuqGtwXVJvjFQsxhhjwhOxpKCqQ4tYrsDtkXp/Y4wxxWdjHxljKqWMDEhPh0OHivdIT4eqVSE2FmrUyPsoaF6NGlCtGmRlQWZm0c+B6awsUIXs7NxH/tf55517Lpx01KXApcuSgjGmSFlZ7iSbkQFHjuROB79OS4MDB2D//tDPwdMHD0KVKhAT407CoZ6Dp6Hgk3hBJ/esLH8/s0h49VVLCsZUWgcPwvbt7qR2+HDeR0HzAvODf20W9ks0+Dn/yby409nHMG6rCNSp4x516+Y+N/Xu+hwcZ+BkHup4VKFmzdxHnTrQpEneeeE+YmMLnl+jhnu/UN9DQfMyMgpPbAU9x8S4pBh4iOR9XdC8OmVw12pLCsZE0J49sHYtrFnjHoHptWth6zH2tQucWIr6dR0T46o3qld3z4HpWrWOnheYDn5d0DoFzatV6+gTf5067kRbpUJcEZUr8BnWru13JGXPkoKJCpmZ4f8SD0wXd1T5w4dh/fq8J//d+QZ6Of54aN8e+vd3z8cf736xFqf+unr13F+RxpQ2Swqm0snOhhUr4Ntv4Ztv3PPGjWXz3lWqQJs20K4dXH21O/G3a+eeTzjB/Zo2pjyzpGAqvCNHYPHi3ATwv//l/kJv1gzOOQduvjn3l3eoX+DBy6pXd9UuxVG1KrRo4bY1pqKypGAqnH374Pvvc5PAggWu2gegY0cYNMglgrPPdr/OrZrFmPBZUjDl0oEDeRtlgxtqN21y9f0xMdCzJ4wc6RLA2Wfn9mgxxpSMJQXjm+xs+OknV/+fv3fO9u15142Lc/Xyv/89dOgAZ54Jp59eNl30jIkmlhRMmdqxA/7zH/jiC/e8Y0fushYt3In/4ovzNtC2awf16/sXszHRxJKCiaiMDPjuO5cEvvgClnh34Y6Lg/PPhwsugF69oG1b65ljTHlgScGUunXrcpPAV1+5IQ1iYlyVz5gxro9+z54V74ImY6KBJQVTKg4ehOefhylTIDnZzWvTBq65xpUGzjvPqoCMqQgsKZhjkp0Nb78NDz0EmzfDH/4Ao0a5RNCxo3UHNaaisaRgSuzrr+Hee92FYwkJMG2a6xZqjKm4rFbXFNvatXDFFW5s9+3bYepUdwGZJQRjKj5LCiZsv/0G990HXbrA55/D44/DqlXwxz9ao7ExlYVVH5kiZWTAhAnw6KNuTKFhw1wvouOP9zsyY0xps993JiRV+Owz6NbNNR6ffLJrP5g40RKCMZWVJQVzlOxs+PJL14PokkvcXag++shdc9Czp9/RGWMiyaqPTI4NG2DyZJg0yU03auSuPbjtNhsO2phoYUkhyh065EoBEye60gFA377w9NNw2WXu/gLGmOgR0aQgIv2BF4AY4J+q+nS+5W2AiUATYDfwR1VNiWRMxrUVLFniEsG777peRW3auIbkG26A+Hi/IzTG+CViSUFEYoDxQD8gBVgkIjNVdUXQamOBKar6loicBzwFXBepmKJdaqq7+njiRFi+3N1h7Ior4KaboE8f61ZqjIlsSeFUYI2qrgMQkWnAQCA4KXQF7vWm5wEfRTCeqLVtG/z5z/D+++7WlQkJ8MorMGQINGzod3TGmPIkkkmhBbAp6HUKcFq+dZYBl+OqmAYBdUWksaqmBq8kIiOAEQCtW7eOWMCV0aefupLA/v3uDmU33eS6mBpjTEH8rjC4DzhXRH4EzgU2A1n5V1LVCaqaoKoJTZo0KesYK6S0NNdr6NJL3TUFixfDuHGWEIwxhYtkSWEz0CrodUtvXg5V3YIrKSAidYArVPW3CMYUFX780Q1ZvXKlG5ZizBjXfmCMMUWJZElhEdBBRNqKSHVgCDAzeAURiRORQAwP4noimRLKzoaxY+G002DfPpgzB/7xD0sIxpjwRSwpqGomMAr4AkgC3lPVX0TkcREZ4K3WG1glIquB44AnIxVPZbd5s7u95V/+4q5CXr7c3dvAGGOKQ1TV7xiKJSEhQRMTE/0Oo1z54AMYPhwOH4YXX3SNyXZzG2NMMBFZrKoJRa3nd0OzOQYHDsDNN8OVV0L79rB0qXttCcEYU1KWFCqohQvd4HSTJsHDD8P//gcdOvgdlTGmorOkUMEcOeJubnPmmW56/nzXu6haNb8jM8ZUBjYgXgWyaJGrHvrpJ9fldPx4aNDA76iMMZWJlRQqgLQ016vo9NPd+EUffwzvvGMJwRhT+qykUM7Nm+d6Fq1dC3/6EzzzDNSv73dUxpjKykoK5dRvv8GIEXDeea430bx58NprlhCMMZFlSaEcmjkTTjwR3nzTVRstWwa9e/sdlTEmGlhSKEe2b4fBg2HgQIiLgwUL4NlnoVYtvyMzxkQLSwrlgCpMnQpdu7pbY44ZA4mJ7r4HxhhTlqyh2Wc7drhbYH7+OZxxhqsy6tLF76iMMdHKSgo+UoUbb3SNyC++CN98YwnBGOMvKyn4aMoUmDXL3fzmjjv8jsYYY6yk4JvNm+Guu+Dssy0hGGPKD0sKPlB1F6IdOQITJ0IV+xaMMeWEVR/54O234bPP4PnnbWRTY0z5Yr9Ry9iWLXDnnXDWWVZtZIwpfywplKFAtVF6uqs2ionxOyJjjMnLqo/K0DvvwKefwnPPQceOfkdjjDFHs5JCGdm61VUbnXmmezbGmPLIkkIZUIVbb4VDh6zayBhTvln1URl491038unYsdCpk9/RGGNMaFZSiLBt21wvozPOgLvv9jsaY4wpXESTgoj0F5FVIrJGRB4oYHlrEZknIj+KyHIRuSiS8ZS1QLVRWhpMmmTVRsaY8i9iSUFEYoDxwIVAV2CoiHTNt9rfgPdUtScwBHglUvH4Ydo0dz/lMWOs2sgYUzFEsqRwKrBGVdep6hFgGjAw3zoK1POm6wNbIhhPmdq+HUaNgtNPh3vu8TsaY4wJTySTQgtgU9DrFG9esNHAH0UkBZgFFHiNr4iMEJFEEUncuXNnJGItVaowciQcPGjVRsaYisXvhuahwGRVbQlcBEwVkaNiUtUJqpqgqglNmjQp8yCLa/p0mDEDnngCOnf2OxpjjAlfJJPCZqBV0OuW3rxgNwPvAajq90AsEBfBmCIuUG102mlw771+R2OMMcUTyaSwCOggIm1FpDquIXlmvnU2An0BRKQLLimU//qhQtx+Oxw4YNVGxpiKKWJJQVUzgVHAF0ASrpfRLyLyuIgM8Fb7MzBcRJYB/wKGqapGKqZIW7kSPvgAHn7YbqtpjKmYInpFs6rOwjUgB897JGh6BXBWJGMoS3PmuOfrrvM3DmOMKSm/G5orlblzoV07iI/3OxJjjCkZSwqlJCMD5s2Dfv38jsQYY0rOkkIpWbQI9u+HP/zB70iMMabkLCmUkrlzQQT69PE7EmOMKTlLCqVkzhxISIBGjfyOxBhjSs6SQinYvx9++MGqjowxFZ8lhVLw3/9CZqY1MhtjKj5LCqVg7lyoWdPdf9kYYyoySwqlYM4c+P3voUYNvyMxxphjY0nhGG3ZAitWWHuCMaZysKRwjObOdc+WFIwxlYElhWM0dy40aQLduvkdiTHGHDtLCsdA1SWFvn2hin2SxphKwE5lx2DFCti61bqiGmMqD0sKxyAwVLa1JxhjKgtLCsdg7lzo2BFat/Y7EmOMKR1FJgURuVRELHnkk5EB8+dbKcEYU7mEc7IfDCSLyLMi0jnSAVUUP/wABw9ae4IxpnIpMimo6h+BnsBaYLKIfC8iI0SkbsSjK8fmznU9jnr39jsSY4wpPWFVC6nqPuB9YBrQHBgELBGROyIYW7k2Zw6ccgo0aOB3JMYYU3rCaVMYICIzgPlANeBUVb0Q6A78ObLhlU9798LChVZ1ZIypfKqGsc4VwPOq+nXwTFVNE5GbIxNW+TZ/PmRlWSOzMabyCScpjAa2Bl6ISE3gOFVdr6pfFrahiPQHXgBigH+q6tP5lj8PBG5gWQtoqqoNwo7eJ3PnQq1acMYZpbTDAwcgNRV273bPwdP5nwPTrVvDqFEwdCjExpZSIMaYaBdOUvg3EHyngCxv3imFbSQiMcB4oB+QAiwSkZmquiKwjqreE7T+HbgG7XJvzhw491yoXr0YG+3fD8nJuY/Vq3OnU1NDb1enDjRu7O7z2bgxtGrlpr/9Fm66Ce6/H269FUaOhObNj/nYjDHRLZykUFVVjwReqOoREQnndHgqsEZV1wGIyDRgILAixPpDgUfD2K+vNm2CVavgT38KscLatbB06dEn/u3b867XsiV06ABXXAEnnABxce6kH5wAGjYMfZMGVZg3D8aNgzFj4Omn4eqr4e673c2ijTGmBMJJCjtFZICqzgQQkYHArjC2awFsCnqdApxW0Ioi0gZoC3wVxn59VehQ2ePGwb33uhM2QLNm7sR/ySXuOfBo187VPx0LETjvPPdYswZeegkmTYJ33nG3gLvrLrj8cqgazldsjDFOOGeMW4F3RORlQHAn+utLOY4hwPuqmlXQQhEZAYwAaO3zmBJz58Jxx8FJJwXNzMpyyeDFF2HQIPj736F9e6hbRpdytG8PL7wATzzhEsOLL8Lgwa6q6fbbYfhwV/owxpgiiAZ+1Ra1okgdAFU9EOb6ZwCjVfUC7/WD3vZPFbDuj8DtqvpdUftNSEjQxMTEsGIubdnZrtq+Xz94+21vZloaXHstfPQR3HMP/OMfEBPjS3w5srLgs89cyWXePHcD6euvh8sug1NPLf0EoQopKe4y7y1bir99hw5w2mmuyswYExEislhVi6xbDqtuQUQuBk4EYkUEAFV9vIjNFgEdRKQtsBlXGrimgH13BhoC34cTi59+/hl27AiqOtqxAy69FBYtcr/O7ygn1/LFxMCAAe6xfLkrRUyeDK+/7pZ36uROwqef7h4nn1y8aqaDB2HxYpcEAo+tW4verijt27t4ArF161bM1nxjzLEq8kwgIq/huov2Af4JXAksLGo7Vc0UkVHAF7guqRNV9RcReRxIDLRR4JLFNA23yOKjPO0JK1fCRRfBtm0wYwYMHOhrbCF16wZvvulKDYmJuSfxzz+HKVPcOrVqQa9euUnitNOgRQu3LDvbNZj/8AMsWOCef/rJlUbAtY+cd17udu3aufaOcGVlwS+/5O5/7tzcYlhsbG5cgUTRsmXx9m+MKZYiq49EZLmqdgt6rgPMVtVzyibEvPysPrrwQli/HpJe/9pVxVSrBp984qpkKhpV2LAhN0ksWABLlsARr6NZy5auV9Ty5fDbb25evXru5Bw4QZ96qrsXaWnHtWlT3iS0eDEcPuyWH3+8SxTNmuXtqVXQs5Uyiu/wYZg40f0t/O537rs++WT3t24qtHCrj8JJCgtV9VQR+QG4HEgFflHV9qUTavH4lRQOH3bnmfHn/Ith84ZB27Ywe7Z7riwOH3bdaQMn43XroHv33CTQubM/9x09cgSWLcuNa9ky2LnTXcSXkRF6uzp1cpNE69a5x5GQUHadACqKI0dcFeOYMS4p16sH+/a5ZTVrus8suMoxUJIsK9nZrvrzyy/dRUIXXeRKpSZspZkU/g68BPTFXYymwBuq+khpBFpcfiWF+fOUL857mqd4yP1Rfvih9ejxm6q7GjzU1d/Bz6tXuwe4xHbiiXlPcl26ROeNtjMyYOpU13Nt/Xr3WTz+uKsj3bgxb7tRcEmyRYvcz+70012p4li7WYeyejXccgt8840rme7c6eZ37AgXX+wSxDnnhL6mxwDhJwVUNeQDN2DemUGvawD1C9sm0o9evXppmcvI0IU9hquCHrnqGtX09LKPwRy71FTV2bNVR49W7d9ftWFDVZdaVOvWVe3bV/Xhh1VnzlTdvt3vaCMrI0P1rbdU27Vzx5+QoDprlmp2duht0tNVFyxQfeEF1aFDVU84Iffzi4lRHTBAdcmS0o3xmWdUY2NVGzRQnTjRxZec7GK44ALVGjXc+9epo3rZZaoTJqhu2lR6MVQiuLbcIs+x4ZQUflTVcjP8RJmXFPbvd1cKf/45k1s+zLCNT1hDZ2Wh6q42D25XWbYstxG9du2jrzIPPBc0r149N4RuQWNYhRrHKiMjdLtIqLaSZs1KflFiVhZMnw6PPeZ+gffs6aYvuaRkf9c7drjP7Ztv4I03XPvToEEwerTr5FBSP/3khnFJTHTtd+PHu/ak/A4ehK++glmzXDfsTd71st26uRLExRe79rHijCu2d697r+ALTjt2dM9t21bYtqrSrD4ai+su+qEWtXIZKNOksHkzXHIJ+tNP/Cn7NZr//RYee6xs3tr4JC3NNWwvXOi+/1Ankezs8PcZExP6xF+tmttfQSeo9PSC91e1qusEEHzSCjxatSr4OpnsbHj/fXeyTkpyJ83HHnO95krrR87eva6X23PPufaIq66CRx91VXXhOnIEnnwS/u//3DAvL7/s9hNOjKqwYoVLDrNmufHBAgm+IPnHFQtM163rvvvAEDV79uRuU6UKxMcfnTBCfe6FadgQmjYtsx+ZpZkU9gO1gUwgHXdVs6pqvdIItLjKNCmcfz58/z3f3v0+54y5gG++gbPPLpu3NuVYdrY76eU/ie/bB/XrH/0rv169kv3jHzp0dDLatcv1GguMq7VmjUtkATVquAbY4JNWjRowdqy70KZrV5cMLr88cm0oe/a4xDBunPslP3iwSw6di7ib78KFrnTwyy/ugtBx49yYYCX122+uYXrv3qNP/MXpnZaamnccs+DH/v0ljw9cAioouXfo4GItxYRRakmhvCmzpKDq/nCGDOE2eZWpU93/pPXMM+WKqruKvKDRd9euze3K26mTKyVcdVXZXXGfmuqS0UsvuQR3zTXwyCPuhBcsLc0ljeeec0MGvP66q/Yp71Rd9dnq1e47KM65VNUl+ODva/36vCXQBg2Orr466yxo06ZE4ZZmSeH3Bc3XfDfdKStllhS2bXN/oC+8QIeX7qRzZ3dJgjEVRlaWG35k+3Z3bYdfw6/s2OGGfxk/3lUPXXedGx/shBPg66/h5ptdiWfECHj2WVfaikZHjsCvvxZcItm40SWSV191Q+WXQGkOc/GXoOlY3JDYi4HzShRZRZGUBMC2hl1Ys8bdz8aYCiUmxv2qLOEvy1LTtKlLCn/+MzzzDLz2mrtq/Zxz3NhcJ5zgGov79Cl6X5VZ9equRNep09HL0tNdya9p04iHUWSloqpeGvToB5wE7ClquwrPSwrztnUB7H7MxhyzZs3g+efdyW3kSNfGcc897qr5aE8IRYmNdQ32pT2CQAFK0q8tBehS2oGUO0lJULcuHye2oHlzd22TMaYUHH+8G0DyxRf9jsQUIJwB8V7CXcUMrmTRA1gSwZjKh6QktEsXvvxKuPBCuzTBGBMdwikpBLfqZgL/UtX/RSie8iMpid29+rFroVUdGWOiRzhJ4X0gXb27oolIjIjUUtW0IraruPbuhS1b+Lm7qzPq29fneIwxpoyEc/XKl0DNoNc1gbmRCaecCDQyb+9C164FX11vjDGVUThJIVaDbsHpTUdoOMRywksKX23pQq9ePsdijDFlKJykcFBEfhd4ISK9gEORC6kcSEpCq1fnu21tj7r40hhjKrNw2hTuBv4tIltw4x41AwZHMijfJSVxuE1HspKr0t6XWwkZY4w/ikwKqrpIRDoDgcvsVqlqIbe7qgSSkkg97neQfPQwLcYYU5kVWX0kIrcDtVX1Z1X9GagjIrdFPjSfpKfDr7+yoZbreWQlBWNMNAmnTWG4qv4WeKGqe4DhEYvIb6tXQ3Y2K7QLcXFuoEJjjIkW4SSFGJHc63lFJAaomLceCofX82jBvi5WSjDGRJ1wksLnwHQR6SsifYF/AbMjG5aPkpJAhP9u7WhJwRgTdcJJCvcDXwG3eo+fyHsxW0gi0l9EVonIGhF5IMQ6V4vIChH5RUTeDTfwiElKIju+LWs217RGZmNM1Amn91G2iCwA2gFXA3HAB0Vt51UzjQf64UZWXSQiM1V1RdA6HYAHgbNUdY+IRH6w8KIkJXGwdRf0V2tkNsZEn5BJQUQ6AkO9xy5gOoCqhjvw+anAGlVd5+1vGjAQWBG0znBgvNd4jaruKO4BlKqsLFi9mm0XXgBYd1RjTPQprPpoJe7uapeo6tmq+hKQVYx9twA2Bb1O8eYF6wh0FJH/icgPItK/GPsvfb/+CocPs666dUc1xkSnwpLC5cBWYJ6IvOE1Mpf2XQWqAh2A3rgSyRsi0iD/SiIyQkQSRSRx586dpRxCEK/n0bIjXWjcGBo2jNxbGWNMeRQyKajqR6o6BOgMzMMNd9FURF4VkfPD2PdmoFXQ65bevGApwExVzVDVX4HVuCSRP5YJqpqgqglNInk7Oi8p/G+3dUc1xkSncO7RfFBV31XVS3En9h9xPZKKsgjoICJtRaQ6MASYmW+dj3ClBEQkDledtC7s6EtbUhI0a8ayDQ0sKRhjolI4XVJzqOoe71d7kbedUdVMYBTwBZAEvKeqv4jI4yIywFvtCyBVRFbgSiN/UdXU4h1CKUpKIqtzVzZutEZmY0x0CmeU1BJT1VnArHzzHgmaVuBe7+EvVUhKYu8l16FqjczGmOhUrJJCpbZ1K+zbR0od1/PISgrGmGhkSSHAa2ReVcW6oxpjopclhYAV7pq6xWldaNQIGjXyOR5jjPGBJYWApCSoX5/Fm5tZKcEYE7UsKQQkJUGXLqxZK5YUjDFRy5JCQFISWR27WHdUY0xUs6QAsGcPbN9OatMuZGdbI7MxJnpZUoCcnke/1rTuqMaY6GZJAXKSwi9Z1h3VGBPdLCmASwo1arBkdzwNG0Ljxn4HZIwx/rCkAC4pdOrE6rUxVkowxkQ1SwqQ2x11jVUdGWOimyWFQ4dg/XoyO3ZhwwZrZDbGRDdLCqtWgSo7Gll3VGOMsaTg9TxKrmrdUY0xxpJCUhJUqcLy9I6AlRSMMdHNkkJSEpxwAqvW16BBA+uOaoyJbpYUvJ5HycmulCDid0DGGOOf6E4KmZmwejV07WrdUY0xhmhPCuvWQUYGmR26sH69NTIbY0x0JwWv59GW+tYd1RhjwJKCe9LOgJUUjDHGkkKLFqzcUg+wkoIxxkQ0KYhIfxFZJSJrROSBApYPE5GdIrLUe9wSyXiOsmJFTs+j+vUhLq5M390YY8qdiCUFEYkBxgMXAl2BoSLStYBVp6tqD+/xz0jFcxRVWLkyz0B41h3VGBPtIllSOBVYo6rrVPUIMA0YGMH3K56UFDhwwEZHNcaYIJFMCi2ATUGvU7x5+V0hIstF5H0RaVXQjkRkhIgkikjizp07Syc6r5HZuqMaY0wuvxuaPwHiVbUbMAd4q6CVVHWCqiaoakKTJk1K5529pLCxdheysqykYIwxENmksBkI/uXf0puXQ1VTVfWw9/KfQK8IxpNXUhI0bMjK3U0BKykYYwxENiksAjqISFsRqQ4MAWYGryAizYNeDgCSIhhPXoG7ra11rctWUjDGmAgmBVXNBEYBX+BO9u+p6i8i8riIDPBWu1NEfhGRZcCdwLBIxXOUoIHw6tWD0qqVMsaYiqxqJHeuqrOAWfnmPRI0/SDwYCRjKFBqKuzc6UoKc607qjHGBPjd0OwPr5HZuqMaY0xeUZ0UMtp34ddfrZHZGGMCojcp1KzJBtpYd1RjjAkSvUmhUyeS17rDt5KCMcY40ZsUvPYEsJKCMcYERF9SOHgQNmzI6Y5aty40bep3UMYYUz5EX1JYtco9B92X2bqjGmOME31JwbqjGmNMSNGZFGJiyIxvb91RjTEmn+hMCu3bs2FrdTIzraRgjDHBojMpeI3MYCUFY4wJFl1JISMDkpOtO6oxxoQQ0QHxyp01ayAz05UUEqFOHTjuOL+DMqb8y8jIICUlhfT0dL9DMUWIjY2lZcuWVKtWrUTbR1dSCO55NM26oxoTrpSUFOrWrUt8fDxi/zTllqqSmppKSkoKbdu2LdE+oqv6KJAUOne27qjGFEN6ejqNGze2hFDOiQiNGzc+phJd9CWFVq3IjK3DunXWyGxMcVhCqBiO9XuKvqTQpQsbN2LdUY0xpgDRkxSys2HlSuuOakwFlJqaSo8ePejRowfNmjWjRYsWOa+PHDlS6LaJiYnceeedZRRpxRc9Dc2bNkFamnVHNaYCaty4MUuXLgVg9OjR1KlTh/vuuy9neWZmJlWrFnw6S0hIICEhoSzCLLbC4vZL+YomkoJ6HiV/CLVrQ7Nm/oZkTEV0993gnZ9LTY8eMG5c8bYZNmwYsbGx/Pjjj5x11lkMGTKEu+66i/T0dGrWrMmkSZPo1KkT8+fPZ+zYsXz66aeMHj2ajRs3sm7dOjZu3Mjdd99dYCli5MiRLFq0iEOHDnHllVfy2GOPAbBo0SLuuusuDh48SI0aNfjyyy+pVasW999/P59//jlVqlRh+PDh3HHHHcTHx5OYmEhcXByJiYncd999zJ8/n9GjR7N27VrWrVtH69ateeqpp7juuus4ePAgAC+//DJnnnkmAM888wxvv/02VapU4cILL2T48OFcddVVLFmyBIDk5GQGDx6c87o0RGVSsNFRjakcUlJS+O6774iJiWHfvn188803VK1alblz5/LQQw/xwQcfHLXNypUrmTdvHvv376dTp06MHDnyqD79Tz75JI0aNSIrK4u+ffuyfPlyOnfuzODBg5k+fTqnnHIK+/bto2bNmkyYMIH169ezdOlSqlatyu7du4uMe8WKFXz77bfUrFmTtLQ05syZQ2xsLMnJyQwdOpTExERmz57Nxx9/zIIFC6hVqxa7d++mUaNG1K9fn6VLl9KjRw8mTZrEjTfeWGqfJ0RTUujdG/7xD2jShDVr4KST/A7ImIqpuL/oI+mqq64iJiYGgL1793LDDTeQnJyMiJCRkVHgNhdffDE1atSgRo0aNG3alO3bt9OyZcs867z33ntMmDCBzMxMtm7dyooVKxARmjdvzimnnAJAvXr1AJg7dy633nprTjVQo0aNiox7wIAB1KxZE3AXBo4aNYqlS5cSExPD6tWrc/Z74403UqtWrTz7veWWW5g0aRLPPfcc06dPZ+HChcX6zIoSPQ3NPXvCffeRmYl1RzWmkqhdu3bO9N///nf69OnDzz//zCeffBKyr36NGjVypmNiYsjMzMyz/Ndff2Xs2LF8+eWXLF++nIsvvrhE/f6rVq1KdnY2wFHbB8f9/PPPc9xxx7Fs2TISExOLbDi/4oormD17Np9++im9evWicePGxY6tMBFNCiLSX0RWicgaEXmgkPWuEBEVkYi3Bm3a5IZAskZmYyqXvXv30qJFCwAmT55c4v3s27eP2rVrU79+fbZv387s2bMB6NSpE1u3bmXRokUA7N+/n8zMTPr168frr7+ek1wC1Ufx8fEsXrwYoMBqrOC4mzdvTpUqVZg6dSpZWVkA9OvXj0mTJpGWlpZnv7GxsVxwwQWMHDmy1KuOIIJJQURigPHAhUBXYKiIdC1gvbrAXcCCSMUSzLqjGlM5/fWvf+XBBx+kZ8+eR/36L47u3bvTs2dPOnfuzDXXXMNZZ50FQPXq1Zk+fTp33HEH3bt3p1+/fqSnp3PLLbfQunVrunXrRvfu3Xn33XcBePTRR7nrrrtISEjIqeIqyG233cZbb71F9+7dWblyZU4pon///gwYMICEhAR69OjB2LFjc7a59tprqVKlCueff36JjzMUUdVS3ymAiJwBjFbVC7zXDwKo6lP51hsHzAH+AtynqomF7TchIUETEwtdpVCvvAK33w6bN8Pxx5d4N8ZElaSkJLp06eJ3GMYzduxY9u7dyxNPPFHg8oK+LxFZrKpF1sZEsqG5BbAp6HUKcFrwCiLyO6CVqn4mIn8JtSMRGQGMAGjduvUxBZWcDLVqQfPmx7QbY4zxxaBBg1i7di1fffVVRPbvW+8jEakCPAcMK2pdVZ0ATABXUjiW97XuqMaYimzGjBkR3X8kG5o3A62CXrf05gXUBU4C5ovIeuB0YGakG5uTk62R2RhjQolkUlgEdBCRtiJSHRgCzAwsVNW9qhqnqvGqGg/8AAwoqk3hWGRlWXdUY4wpTMSSgqpmAqOAL4Ak4D1V/UVEHheRAZF638JYd1RjjClcRNsUVHUWMCvfvEdCrNs7krGAdUc1xpiiRM8VzWCjoxpTQfXp04cvvvgiz7xx48YxcuTIkNv07t2bY+m+Hq2iKikkJ0PNmnZ9gjEVzdChQ5k2bVqeedOmTWPo0KE+RVS0Y7mAzk/RMyAe1h3VmFLhw9jZV155JX/72984cuQI1atXZ/369WzZsoVzzjkn5DDXoTz++ON88sknHDp0iDPPPJPXX38dEWHNmjXceuut7Ny5k5iYGP7973/Trl27o4avfvrpp+nduzdjx44lISGBXbt2kZCQwPr165k8eTIffvghBw4cICsri88++4yBAweyZ88eMjIyGDNmDAMHDgRgypQpjB07FhGhW7duvPLKK3Tr1o3Vq1dTrVo19u3bR/fu3XNel5WoSgrJyWAXZRpT8TRq1IhTTz2V2bNnM3DgQKZNm8bVV1+NiBQ4zHW3bt1C7mvUqFE88ohr2rzuuuv49NNPufTSS7n22mt54IEHGDRoEOnp6WRnZxc4fHVRlixZwvLly2nUqBGZmZnMmDGDevXqsWvXLk4//XQGDBjAihUrGDNmDN999x1xcXHs3r2bunXr0rt3bz777DMuu+wypk2bxuWXX16mCQGiKCkEuqMO8KXfkzGViE9jZweqkAJJ4c033wQKHua6sKQwb948nn32WdLS0ti9ezcnnngivXv3ZvPmzQwaNAhwg85B6OGrC9OvX7+c9VSVhx56iK+//poqVaqwefNmtm/fzldffcVVV11FXFxcnv3ecsstPPvss1x22WVMmjSJN954o4SfVslFTVJISYEjR6yR2ZiKauDAgdxzzz0sWbKEtLQ0evXqlTPM9aJFi2jYsCHDhg0rdJjr9PR0brvtNhITE2nVqhWjR4+O6LDY77zzDjt37mTx4sVUq1aN+Pj4Qt/vrLPOYv369cyfP5+srCxO8uHGL1HT0GzdUY2p2OrUqUOfPn246aabchqYQw1zHUrghBwXF8eBAwd4//33Aahbty4tW7bko48+AuDw4cOkpaWFHL46eFjswD4KsnfvXpo2bUq1atWYN28eGzZsAOC8887j3//+N6mpqXn2C3D99ddzzTXXRGRY7HBETVKw7qjGVHxDhw5l2bJlOUkh1DDXoTRo0IDhw4dz0kknccEFF+TcRQ1g6tSpvPjii3Tr1o0zzzyTbdu2hRy++r777uPVV1+lZ8+e7Nq1K+T7XXvttSQmJnLyySczZcoUOnfuDMCJJ57Iww8/zLnnnkv37t25995782yzZ88e33pWRWzo7Egp6dDZH38MkybBhx9ClahJhcaUDhs6u+y8//77fPzxx0ydOrXE+yivQ2eXKwMHuocxxpRXd9xxB7Nnz2bWrFlFrxwhUZMUjDGmvHvppZf8DiF62hSMMcemolU1R6tj/Z4sKRhjihQbG0tqaqolhnJOVUlNTc25zqIkrPrIGFOkli1bkpKSws6dO/0OxRQhNjaWli1blnh7SwrGmCJVq1aNtm3b+h2GKQNWfWSMMSaHJQVjjDE5LCkYY4zJUeGuaBaRncCGEm4eB4S+Jr3yi+bjj+Zjh+g+fjt2p42qNilqgwqXFI6FiCSGc5l3ZRXNxx/Nxw7Rffx27MU7dqs+MsYYk8OSgjHGmBzRlhQm+B2Az6L5+KP52CG6j9+OvRiiqk3BGGNM4aKtpGCMMaYQlhSMMcbkiJqkICL9RWSViKwRkQf8jqcsich6EflJRJaKSPFvW1fBiMhEEdkhIj8HzWskInNEJNl7buhnjJES4thHi8hm7/tfKiIX+RljpIhIKxGZJyIrROQXEbnLmx8t332o4y/W9x8VbQoiEgOsBvoBKcAiYKiqrvA1sDIiIuuBBFWNigt4ROT3wAFgiqqe5M17Ftitqk97Pwoaqur9fsYZCSGOfTRwQFXH+hlbpIlIc6C5qi4RkbrAYuAyYBjR8d2HOv6rKcb3Hy0lhVOBNaq6TlWPANMAuzlnJaWqXwO7880eCLzlTb+F+2epdEIce1RQ1a2qusSb3g8kAS2Inu8+1PEXS7QkhRbApqDXKZTgw6rAFPiPiCwWkRF+B+OT41R1qze9DTjOz2B8MEpElnvVS5Wy+iSYiMQDPYEFROF3n+/4oRjff7QkhWh3tqr+DrgQuN2rYoha6upMK3+9aa5XgXZAD2Ar8P98jSbCRKQO8AFwt6ruC14WDd99AcdfrO8/WpLCZqBV0OuW3ryooKqbvecdwAxcdVq02e7VuQbqXnf4HE+ZUdXtqpqlqtnAG1Ti719EquFOiO+o6ofe7Kj57gs6/uJ+/9GSFBYBHUSkrYhUB4YAM32OqUyISG2v0QkRqQ2cD/xc+FaV0kzgBm/6BuBjH2MpU4ETomcQlfT7FxEB3gSSVPW5oEVR8d2HOv7ifv9R0fsIwOuGNQ6IASaq6pP+RlQ2ROQEXOkA3O1X363sxy4i/wJ644YN3g48CnwEvAe0xg29frWqVroG2RDH3htXdaDAeuBPQXXslYaInA18A/wEZHuzH8LVq0fDdx/q+IdSjO8/apKCMcaYokVL9ZExxpgwWFIwxhiTw5KCMcaYHJYUjDHG5LCkYIwxJoclBWM8IpIVNJLk0tIcTVdE4oNHLjWmvKrqdwDGlCOHVLWH30EY4ycrKRhTBO9+FM9696RYKCLtvfnxIvKVN9DYlyLS2pt/nIjMEJFl3uNMb1cxIvKGN9b9f0Skprf+nd4Y+MtFZJpPh2kMYEnBmGA181UfDQ5atldVTwZexl0ZD/AS8JaqdgPeAV705r8I/FdVuwO/A37x5ncAxqvqicBvwBXe/AeAnt5+bo3MoRkTHrui2RiPiBxQ1ToFzF8PnKeq67wBx7apamMR2YW7qUmGN3+rqsaJyE6gpaoeDtpHPDBHVTt4r+8HqqnqGBH5HHdjnI+Aj1T1QIQP1ZiQrKRgTHg0xHRxHA6aziK3Te9iYDyuVLFIRKytz/jGkoIx4Rkc9Py9N/0dbsRdgGtxg5EBfAmMBHcrWBGpH2qnIlIFaKWq84D7gfrAUaUVY8qK/SIxJldNEVka9PpzVQ10S20oIstxv/aHevPuACaJyF+AncCN3vy7gAkicjOuRDASd3OTgsQAb3uJQ4AXVfW3UjoeY4rN2hSMKYLXppCgqrv8jsWYSLPqI2OMMTmspGCMMSaHlRSMMcbksKRgjDEmhyUFY4wxOSwpGGOMyWFJwRhjTI7/D/VafEB7s+d7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1FUlEQVR4nO3dd3wUdf748debJBB6R5EiiIAComgElcQDUQ5C00NRDhWsZzkVRYXzRDmVH96JyCGoxyl66vfACqKAWJEmJUFAEAtHDb1IJwLJ+/fHZ0I2IWUD2Ux29/18POaxs7OzM+/Jwr53PlVUFWOMMSZLGb8DMMYYU7pYYjDGGJODJQZjjDE5WGIwxhiTgyUGY4wxOVhiMMYYk4MlBuMrEZkhIv2Le1/jDxEZICJz/Y7DnJpYvwMw4UdEDgQ8rQD8BmR4z/+kqv8X7LFUtWso9i0KEekAvK2q9UNxfGPCjSUGU2SqWilrXUTWAber6he59xORWFU9VpKxRSv7W5viZEVJptiISAcRSRORwSKyFXhdRKqLyCciskNEfvXW6we8Z5aI3O6tDxCRuSIy0tt3rYh0Pcl9G4vIbBHZLyJfiMg4EXn7JK7pXO+8e0RkpYj0DHgtWUR+8M6xSUQe9rbX8q5zj4jsFpE5IpLn/zURURG5X0TWiMhOEXkucF8RuVVEVnnXOFNEzsz13ntF5Bfgl3yOf4mIzPdiWebdHQX+PUeIyCIR2SciH4lIjYDXe3rXvMfb99yA1xqIyIfe57pLRMbmOm+en4sJD5YYTHE7HagBnAncifs39rr3vCFwGBib77uhHfATUAv4B/CaiMhJ7PtfYBFQExgG3FTUCxGROOBj4DOgDnAf8H8i0tzb5TVc0VlloBXwlbd9EJAG1AZOAx4DChp75hogAbgQ6AXc6p2/l/feP3jHmgNMzPXeq3F/hxZ5xF8PmAY8g/tMHgY+EJHaAbvd7J2vLnAMGOO9t5l3roHeuacDH4tIWRGJAT4B1gONgHrApIBjFuUzNKWRqtpiy0kvwDrgSm+9A3AEiC9g/wuAXwOez8IVRQEMAFYHvFYB94V6elH2xSWgY0CFgNffxtUj5BVTByAtj+1JwFagTMC2icAwb30D8CegSq73PQV8BJwdxN9PgS4Bz+8BvvTWZwC3BbxWBjgEnBnw3isKOPZg4K1c22YC/QP+ns8GvNbC+/xigKHAu7nOvcn7W10K7ABi8zhngZ+hLeGx2B2DKW47VDU964mIVBCRf4nIehHZB8wGqnm/OvOyNWtFVQ95q5WKuO8ZwO6AbQAbi3gdeMfZqKqZAdvW434hA/QGkoH1IvKNiFzqbX8OWA185hURDSnkPIGxrffOC+4u659eUc4eYDcgAefP/d7czgSuy3q/d4xE3N1BfueOw/3SP8N7DoD3N9jonbsBsF7zr9MoymdoSiFLDKa45S4yGQQ0B9qpahXgcm97KIsWtgA1RKRCwLYGJ3GczUCDXPUDDXG/nFHVxaraC1fMNAV419u+X1UHqepZQE/gIRHpVMB5AmNr6J0X3Bfxn1S1WsBSXlXnB+xfUBHVRtwdQ+D7K6rqswWc+yiw04shsD5DvH03ecdtKCLWeCVCWWIwoVYZV6+wx6vYfDLUJ1TV9UAKMMwrE78U6FHY+0QkPnDB1VEcAh4VkTiv4rYHMMk7bj8RqaqqR4F9QKZ3nO4icrb3ZboX15Q3M69zeh4RV0nfAHgAeMfb/grwFxFp6R23qohcV4Q/xdtADxH5vYjEeNfVQQIq/4EbRaSFl0SfAt5X1QxckusmIp28upZBuGbJ872/yxbgWRGp6B23fRHiMqWcJQYTaqOB8rhfoQuAT0vovP1wZeG7cJWv7+C+2PJTD5fAApcGuETQFRf/S8DNqvqj956bgHVeEdld3jkBmgJfAAeAb4GXVPXrAs79EZAKLMVVFr8GoKqTgb/jEtE+YIUXS1BUdSOuMvsxXJ3ARuARcv6/fwt4A1f8Ew/c7733J+BG4EXv2nsAPVT1iJc4egBn4+pZ0oDrg43LlH6iahP1mMgnIu8AP6pqyO9YikJEFGiqqqt9OPcsXIX8qyV9blO62R2DiUgicrGINBGRMiLSBffLeYrPYRkTFqzyyESq04EPcf0Y0oC7VfU7f0MyJjxYUZIxxpgcrCjJGGNMDmFXlFSrVi1t1KiR32EYY0xYSU1N3amqtQvfMwwTQ6NGjUhJSfE7DGOMCSsisr7wvZyQFSWJyAQR2S4iKwrYp4OILPVGcPwmVLEYY4wJXijrGN4AuuT3oohUw3UY6qmqLYGi9Og0xhgTIiFLDKo6GzfoV37+CHyoqhu8/beHKhZjjDHB87OOoRkQ5/W+rAz8U1XfPJkDHT16lLS0NNLT0wvf2eQrPj6e+vXrExcX53coxhgf+ZkYYoGLgE64sXS+FZEFqvpz7h1F5E7cpC80bNjwhAOlpaVRuXJlGjVqhM0HcnJUlV27dpGWlkbjxo39DscY4yM/+zGkATNV9aCq7sSN039+Xjuq6nhVTVDVhNq1T2xtlZ6eTs2aNS0pnAIRoWbNmnbXZYzxNTF8BCSKSKw35G87YNXJHsySwqmzv6ExBkJYlCQiE3HTANYSkTTcOPxxAKr6iqquEpFPgeW4sepfVdV8m7YaY0xUUoXly2HmTEhIgCuuCPkpQ5YYVLVvEPs8h5sGMazt2rWLTp3cBF1bt24lJiaGrCKvRYsWUbZs2Xzfm5KSwptvvsmYMWOCPl9WJ79atWqdWuDGmFM3axaMHw9Nm0K7dtC2LZzq/83t2+Hzz10y+Owz2LbNbf/LX8I7MUSTmjVrsnTpUgCGDRtGpUqVePjhh4+/fuzYMWJj8/5TJyQkkJCQUBJhGmOK02+/weOPw/PPQ9Wq8M47kOlN1HfWWdlJol07aNMG4uPzP9aRIzBvnksCM2fCd95AwLVqwVVXQefObjnjjPyPUYwsMYTIgAEDiI+P57vvvqN9+/bccMMNPPDAA6Snp1O+fHlef/11mjdvzqxZsxg5ciSffPIJw4YNY8OGDaxZs4YNGzYwcOBA7r///gLPM2rUKCZMmADA7bffzsCBAzl48CB9+vQhLS2NjIwMhg4dyvXXX8+QIUOYOnUqsbGxdO7cmZEjR5bEn8KYyLNiBfTr54p4/vQnlxxUITUVFi6ERYtgzhyYONHtHxcH55+fnSjatQOR7ETw9ddw8CDExsJll8Hw4S4RXHghlCn5quCISwwDB4L3473YXHABjB5d9PelpaUxf/58YmJi2LdvH3PmzCE2NpYvvviCxx57jA8++OCE9/z44498/fXX7N+/n+bNm3P33Xfn268gNTWV119/nYULF6KqtGvXjt/97nesWbOGM844g2nTpgGwd+9edu3axeTJk/nxxx8REfbs2VP0CzIm2mVmwpgxMGSIu0v4+GPo3j379d/9zi1ZNm92SWLhQre8+Sa89FLOYzZpAv37u0TQsSNUqVIy11KAiEsMpcl1111HTEwM4L6c+/fvzy+//IKIcPTo0Tzf061bN8qVK0e5cuWoU6cO27Zto379+nnuO3fuXK655hoqVqwIwB/+8AfmzJlDly5dGDRoEIMHD6Z79+4kJSVx7Ngx4uPjue222+jevTvdA/8xG2MKl5YGAwbAl19Cjx7w6qtQp07B7znjDLj6arcAZGTAjz+6JHH0KFx5pUsMpUzEJYaT+WUfKllf2ABDhw6lY8eOTJ48mXXr1tGhQ4c831OuXLnj6zExMRw7dqzI523WrBlLlixh+vTpPP7443Tq1IknnniCRYsW8eWXX/L+++8zduxYvvrqqyIf25io9O67cNddrl5h/Hi4/XZXFFRUMTHQsqVbSjGbqKeE7N27l3r16gHwxhtvFMsxk5KSmDJlCocOHeLgwYNMnjyZpKQkNm/eTIUKFbjxxht55JFHWLJkCQcOHGDv3r0kJyfzwgsvsGzZsmKJwZhSSxVeeAFuuMH9Yly82P1KL4q9e+Hmm+H6612ro6VL4Y47Ti4phJGIu2MorR599FH69+/PM888Q7du3YrlmBdeeCEDBgygbdu2gKt8btOmDTNnzuSRRx6hTJkyxMXF8fLLL7N//3569epFeno6qsqoUaOKJQZjSqXMTHjgARg7FmrXdi2GACpUcBXA7du75dJLoVq1vI8xZw7cdJMrQnrySfjrX10lchQIuzmfExISNPdEPatWreLcc8/1KaLIYn9LE/aOHYNbb4W33oJBg+C552DLFtccdN48mDvX/fLPyHC//Fu2zE4U7dtD/fouEfz9767Z6dtvwyWX+H1Vp0xEUlU1qLbxdsdgjIkc6enQty9MmQJPP+1+5Yu4SuDrrnMLwIEDrrVQVrKYOBH+9S/3WvnycPiwKzIaNQoqVfLtcvxiicEYExkOHHCtf7780jUpve++/PetVMn1IM7qRZyRAStXuiTx3Xeu1VGPHiUSdmlkicEYE/5274Zu3dxdwH/+4yqMiyImBlq3douxxGCMCXNbt7rOYT/9BO+/D9dc43dEYc8SgzEmfK1f7zqJbd4M06a5dXPKLDEYY8LTjz+6AeYOHIAvvnBNT02xsA5uxaBjx47MnDkzx7bRo0dz99135/ueDh06kLvZbUHbjTEBliyBpCQ3KumsWZYUipklhmLQt29fJk2alGPbpEmT6Nu30CkpjDFFNWeOG2yuQgXXJ+H8PGcENqfAEkMxuPbaa5k2bRpHjhwBYN26dWzevJmkpCTuvvtuEhISaNmyJU8++WSRjjtx4kTOO+88WrVqxeDBgwHIyMhgwIABtGrVivPOO48XXngBgDFjxtCiRQtat27NDTfcULwXaExp8emn8PvfQ926Lik0bep3RBEp8uoYfBh3u0aNGrRt25YZM2bQq1cvJk2aRJ8+fRARhg8fTo0aNcjIyKBTp04sX76c1kE0idu8eTODBw8mNTWV6tWr07lzZ6ZMmUKDBg3YtGkTK1a4WVCzhs9+9tlnWbt2LeXKlbMhtU3pcegQbNrklrS0nOtbtrhB6VTdkpmZvZ7ftnXroFUrN4dBYSObmpMWyjmfJwDdge2q2qqA/S4GvgVuUNX3QxVPqGUVJ2Ulhtdeew2Ad999l/Hjx3Ps2DG2bNnCDz/8EFRiWLx4MR06dDg+RWi/fv2YPXs2Q4cOZc2aNdx3331069aNzp07A9C6dWv69evH1VdfzdVZQ/waU1JmzXKTzeT+8s/rR0qVKm7YiTPOcDOUibilTJns9cAlcHtyMvztb/mPb2SKRSjvGN4AxgJv5reDiMQAfwc+K7az+jTudq9evXjwwQdZsmQJhw4d4qKLLmLt2rWMHDmSxYsXU716dQYMGEB6evopnad69eosW7aMmTNn8sorr/Duu+8yYcIEpk2bxuzZs/n4448ZPnw433//fb7TiRpTbNasgYcego8+cl/cp58O9erB2We7CWvq1XNL/frZ61E4xES4Cdk3h6rOFpFGhex2H/ABcHGo4igplSpVomPHjtx6663HK5337dtHxYoVqVq1Ktu2bWPGjBn5zsOQW9u2bbn//vvZuXMn1atXZ+LEidx3333s3LmTsmXL0rt3b5o3b86NN95IZmYmGzdupGPHjiQmJjJp0iQOHDhANftVZULlwAEYMQJGjnQjjo4Y4YpxC5rX2IQN335Sikg94BqgI4UkBhG5E7gToGHDhqEP7iT17duXa6655ngLpfPPP582bdpwzjnn0KBBA9q3bx/0serWrcuzzz5Lx44dUVW6detGr169WLZsGbfccguZ3qTjI0aMICMjgxtvvJG9e/eiqtx///2WFExoqMJ//wuPPuo6ld10Ezz7bIlNUm9KRkiH3fbuGD7Jq45BRN4DnlfVBSLyhrdfoXUMNux2aNnf0uRryRK4/3430NxFF8GLL1r/gTASLsNuJwCTxM2EVAtIFpFjqjrFx5iMMblt3+6Gr37tNTfpzWuvubmPy1hr90jlW2JQ1cZZ6wF3DFP8iscYk8vRozBuHAwbBgcPwoMPwhNPQNWqfkdmQiyUzVUnAh2AWiKSBjwJxAGo6ivFfT5VRSJ8HtZQC7fZ/EyIZGa6sYcGDoRVq1yHstGj4Zxz/I7MlJBQtkoKejwIVR1wKueKj49n165d1KxZ05LDSVJVdu3aRby1KoksqpCaChs3uj4FgcvevSdu27MH9u1z72vSBKZOhe7dXVNUEzUioqF7/fr1SUtLY8eOHX6HEtbi4+OpX7++32GY4vLVVzB0KMyff+JrVau6TmJZS6NGOZ83bAj9+kG5ciUYsCktIiIxxMXF0bhx40L3O3zYTedqTESbM8fVBcya5TqUjRsHl12W/aVfubKbscyYfERNs4L33oMaNWDDBr8jMSZEFixwM5ldfrmbq2DMGFi9Gu65x433lXVXYEnBFCJqEsN550F6upvkyZiIkpLi5ju+9FI3gOTIkfC//8F991lPZHNSoiYxNG/u6tI+/tjvSIwpJsuWwdVXw8UXu7uFESPc2EWDBrm5Cow5SVGTGESgRw9XH3fwoN/RGHMKVq6E665zxUOzZsFTT8HatTBkiA1QZ4pF1CQGcInht9/g88/9jsSYk7B8OfTt68pFZ850LY7WrnWPVar4HZ2JIFGVGJKSXCs9K04yYWXePNeX4Pzz4ZNP3AB2a9e6O4Xq1f2OzkSgiGiuGqy4OOjSxVVAZ2baUC+mFFN1dwUjRsDs2W5Cm6efhnvvtWRgQi56vhozMmDRInp0V7Ztg8WL/Q7ImDxkZLi21RddBF27usrk0aPdlJaPP25JwZSI6EkMb74J7drRvfFKypRxd+TGlBpHjsCECdCiBfTp41pITJjgmp0+8ABUrOh3hCaKRE9i8OZGrjpvOu3bWz2DKSUOHnR3BE2awG23uQTw3nvwww9wyy1QtqzfEZooFD11DPXqucq7GTPo0eNRHn3U9YIuxRPCmXCSVSewdasbe+XwYdejMms9r+eHD7ump7t2ufmRX33V/YCxAeuMz6InMYArsx05kl7/2MujVOWTT9xoAcackkOH3K99b0rXHETcAF2BS3x89nrnzvDnP7uxjIwpJaIrMSQnw7PP0nT9F5x9dm8+/tgSgzlFmzZBr15u2ssRI+CGG3J+8cfF2R2ACTvRlRguvRSqVkVmTKdHj96MGwcHDlhnUXOSFi1yQ1Ls3w8ffeR6UBoTAaKn8hkgNtbNRjV9Oj26K0eOWC9oc5ImTnT1AuXKwbffWlIwESVkiUFEJojIdhFZkc/r/URkuYh8LyLzReT8UMWSQ3IybN1KYqWlVK1qzVZNEWVmuv4Ef/wjtG3r7hpatfI7KmOKVSjvGN4AuhTw+lrgd6p6HvA0MD6EsWTr4kKK+3x6jl7QxhTqwAHo3RuGD4fbb3e3m7Vr+x2VMcUuZIlBVWcDuwt4fb6q/uo9XQCUzJySp50GCQmuOKkH1gvaBGf9emjf3s2BPHo0jB9vfQxMxCotdQy3ATPye1FE7hSRFBFJKZZ5nZOTYcECktvtIibGOruZQsyb5+Y8WL8epk93PZGtpZGJYL4nBhHpiEsMg/PbR1XHq2qCqibULo5b9+RkyMyk+uLPrBe0Kdjrr0PHjm5Y3gULXOMFYyKcr4lBRFoDrwK9VHVXiZ04IcGNVukVJy1f7n4MGnNcRoabCe3WW90cygsXwjnn+B2VMSXCt8QgIg2BD4GbVPXnEj15TIyrhP70U3okZwDWOsl4Nm2CsWNdT+RRo9ww1zNmQI0afkdmTIkJZXPVicC3QHMRSROR20TkLhG5y9vlCaAm8JKILBWRlFDFkqfkZNi5k+b7U2ja1BJDVFuzBkaOdB0g69eH++5zndZefdUlibg4vyM0pkSFrOezqvYt5PXbgdtDdf5Cde7sZuqZPp0ePdoxdqz1go4qq1bBBx+4ZelSt61NG3jmGfjDH+Dcc30Nzxg/+V757JuaNeGSS2D6dLp3x3pBRzpV+O471znt3HPdvAdDh7rxjEaOdPMeLFkCf/2rJQUT9aI3MYArTkpJIbHpNpsLOpK9956b7+DCC91Ad3XruiKitDSYP99VMp91lt9RGlNqWGIA4r6aSdeu1gs6Ir30Elx/vas8fvVVN1/CV1+5SuV69fyOzphSKboTwwUXwOmnH2+2un27G/rGRABVeOoplwC6d4c5c9ycCTaEhTGFiu7EIOIm75k5k65XHbNe0JEiMxMGDoQnn4Sbb4YPP3R1CcaYoER3YgBXnLRnD9V/WkBiojVbDXtHj0L//jBmjEsOr7/uhls3xgTNEsNVV7kOb9YLOvwdPuyamr79tmt2OmqUa5JsjCkS+19TtSokJh5vtgp21xCW9uxx4xhNmwYvv+yandpAd8acFEsM4IqTli2jeaVNNG1q9QxhZ9s26NDBDXI3cSLcdVehbzHG5M8SAxxvtsqMGfToAV9/7UZEMGFg3Tp3x/fLLy6jX3+93xEZE/YsMQC0bAkNGhyvZ7Be0GFixQo3ec6uXfDFFzYktjHFxBIDuLLo5GT4/HPaX3yEatWsOKnUW7DADYetCrNnuwHwjDHFwhJDluRkOHCAuIVzrRd0aff553Dlla4387x50KqV3xEZE1GsgXeWK65wc/hOn06PHlcwcaLrBX3JJX4HFgVUYd8+1/V827acj3lt27MHzj8fZs50c3gbY4qVJYYslSrB734H06fT5a8jj/eCtsQQQunp0Ls3fPkl/PZb3vvUrOm+/OvUccNi16nj6oPuuAOqVSvRcI2JFpYYAiUnw4MPUn3PWhITG/PxxzB8uN9BRShVuOcemD7djWfUuHF2Ash6rFXLJskxxgeWGAJ5icE1W72Hhx92vaDPPNPvwCLQSy+54SqeeAL+9je/ozHGBAjl1J4TRGS7iKzI53URkTEislpElovIhaGKJWhNm7px+73+DGCtk0Ji9mw3jlGPHm6gO2NMqRLKVklvAF0KeL0r0NRb7gReDmEswclqtvrllzRrmE6zZpYYit3GjXDddS4Bv/WWjWVkTCkUsv+Vqjob2F3ALr2AN9VZAFQTkbqhiidoXbu6wdi++YYuXdww/keP+h1UhMiqbD58GKZMceNUGWNKHT9/rtUDNgY8T/O2+atDB4iPh+nTSUpy32FLlvgdVARQhbvvhsWL3ein55zjd0TGmHyExX28iNwpIikikrJjx47Qnqx8edenYfp0EhPdprlzQ3vKqDBuHLzxhqtT6NnT72iMMQXwMzFsAhoEPK/vbTuBqo5X1QRVTahdElMzJifD6tWcvv8Xzj7bEsMpmz3btfbq2dO1QjLGlGp+JoapwM1e66RLgL2qusXHeLJ17eoevbuGuXNdSYg5CRs3wrXXusrmN9+0ymZjwkAom6tOBL4FmotImojcJiJ3iUjWYPnTgTXAauDfwD2hiqXIzjrLlYF7iWHnTvjpJ7+DCkNZM6qlp1tlszFhJGQd3FS1byGvK3BvqM5/ypKTYexYLv/7QaAic+ZYfWmRZFU2p6TARx/ZH8+YMGL39flJToYjRzh7w1fUqWP1DEU2diz85z9W2WxMGLLEkJ/ERKhUCZmRXc9ggvTNN1bZbEwYs8SQn3LloHNneP99Ol20hzVrYPNmv4MKAxs2uJ7NZ59tPZuNCVP2v7Ygjz8Ou3fTe+njgN01FCp3ZXOVKn5HZIw5CZYYCtKmDdxzD3U+eJnL4pcwZ47fAZVSO3fCyJFu8pzUVOvZbEyYs8RQmKefRmrX5tWy9zBvjs31eZwqzJoFfftCvXrwyCNuHoXJk62y2ZgwZ4mhMNWqwXPPce6+hSQsn8DevX4H5LOdO+H5590dQceO8OmnrlnqihVuxMGrr/Y7QmPMKbLEEIwbb+TX85L4fzqElJm7/I6m5Km6lkZ//KO7O3j4Yahd2zVH3bwZRo+Gli39jtIYU0wsMQRDhHL/Hkc19lBlxF/8jqbk7NwJo0bBuee6UWdnzIC77oLvv3c18Tff7AYdNMZEFJvaM0gV2p3H/532AH2XvgALb4N27fwOqXjs2wdr1sD//nfi4/r1kJEBl13m7g6uvRYqVPA7YmNMiFliKIIV1w5j67hJnH73PZRZvAhiYvwOKXirVsGCBScmgJ07c+5Xs6Yb8K5tW+jXD/r0gVat/InZGOOLoBKDiFQEDqtqpog0A84BZqhqVM1tdvEVlXlw3Cje+e4G+Ne/4J7SM+5fgd54A+64A44dc8msYUP35d+7txswsEkT93jWWTbQnTEG0SDGkxaRVCAJqA7MAxYDR1S1X2jDO1FCQoKmpKSU9GkB2L4dTjtNWdfkSs7ctcQNuVqnji+xBEUVhg2Dp56Cq65yk+U0agRxcX5HZowpYSKSqqoJwewbbOWzqOoh4A/AS6p6HRB1zVDq1IFmzYRnG4yDgwdh8GC/Q8rfkSPQv79LCrfcAtOmQdOmlhSMMYUKOjGIyKVAP2Caty2MCtiLT2IivLv8HPShQa6IZt48v0M60Z490KWLG6vo6afhtdcsIRhjghZsYhgI/AWYrKorReQs4OuQRVWKJSXB7t2wqvfj0KCBq2c4dszvsLKtXw/t27vmpG+95cZ7EvE7KmNMGAkqMajqN6raU1X/LiJlgJ2qen+IYyuVEhPd45wlFV3HruXLXdl9aZCaCpdcAps2wcyZcOONfkdkjAlDQSUGEfmviFTxWietAH4QkUdCG1rp1KSJGxJo7lzgmmtckc3QobDF5+mqP/kELr/cDRc+f74brsIYY05CsEVJLVR1H3A1MANoDNxU2JtEpIuI/CQiq0VkSB6vNxSRr0XkOxFZLiLJRQneDyKuOGnOHO/Jiy+6it6HH/YvqJdegl69XA/lBQugRQv/YjHGhL1gE0OciMThEsNUr/9Cge1cRSQGGAd0BVoAfUUk9zfW48C7qtoGuAF4qQix+yYx0RXlb9yIm5Bm8GD473/h6xKudsnMdKOa3nsvdOvmxjM6/fSSjcEYE3GCTQz/AtYBFYHZInImsK+Q97QFVqvqGlU9AkwCeuXaR4Gs2VyqAmExR1pSkns8PnHPkCHQuLH7gj5aQn3+Dh+G66938yDce68b7rpixZI5tzEmogVb+TxGVeuparI664HCCrHrARsDnqd52wINA24UkTRgOnBfXgcSkTtFJEVEUnbs2BFMyCHVujVUqhSQGMqXhzFj3LATo0eH5qS//eZ62P38sysuuvJK+OADNwT2iy+G1/AcxphSLdghMaoCTwKXe5u+AZ4CTnV2gr7AG6r6vNdP4i0RaaWqOWbEUdXxwHhwPZ9P8ZynLDYWLr0011Sf3bu7CWr+9jfXkSw21jVjzW85ejR7/cgRN5jdnj1u2bv3xPX09JxBxMfDe++5YS2MMaYYBTuI3gRca6Q+3vObgNdxPaHzswloEPC8vrct0G1AFwBV/VZE4oFawPYg4/JNUhI8+aT7zq5Wzdv4z3+6AeeuuaboByxXzh2oWjU3XlG1anDmmdnrgdurVXPzHzRqVAxXYowxOQWbGJqoauBP07+JyNJC3rMYaCoijXEJ4Qbgj7n22QB0At4QkXOBeMD/sqIgJCa6oYjmz4fkrLZUjRrBL7+4munY2OCWuDj3WK6cn5djjDHHBZsYDotIoqrOBRCR9sDhgt6gqsdE5M/ATNzwGRO8XtNPASmqOhUYBPxbRB7EVUQP0GBG9SsF2rVz3+dz5gQkBoC6dd1ijDFhKtjEcBfwplfXAPAr0L+wN6nqdFylcuC2JwLWfwDaBxlDqVKhAlx0Ua56BmOMiQDBtkpapqrnA62B1l6/gytCGlkYSEyERYtOrBc2xphwVqQ5n1V1n9cDGuChEMQTVhITXYMin6aHMMaYkChSYsgl6ofsbO8VgllxkjEmkpxKYgiLSuJQql0bzjnHEoMxJrIUWPksIvvJOwEIUD4kEYWZpCTXzywzE8qcSpo1xphSosCvMlWtrKpV8lgqq2qwLZoiWmKi6+S2cqXfkRhjTPGw37in6PjEPXP8jcMYY4qLJYZT1LgxnHGG1TMYYyKHJYZTJOLuGiwxGGMihSWGYpCY6CbtWb/e70iMMebUWWIoBidM3GOMMWHMEkMxOO88qFLFKqCNMZHBEkMxiImByy6zOwZjTGSwxFBMEhNdX4bdu/2OxBhjTo0lhmKS1Z9h3jx/4zDGmFNliaGYtG3rJmOz4iRjTLizxFBMypeHhARLDMaY8BfSxCAiXUTkJxFZLSJD8tmnj4j8ICIrReS/oYwn1BITYfFiOFzgpKfGGFO6hSwxiEgMMA7oCrQA+opIi1z7NAX+ArRX1ZbAwFDFUxKSkuDoUZccjDEmXIXyjqEtsFpV16jqEWAS0CvXPncA41T1VwBV3R7CeELussvc42ef+RuHMcacilAmhnrAxoDnad62QM2AZiIyT0QWiEiXvA4kIneKSIqIpOzYsSNE4Z66mjWhZ08YMwa2bfM7GmOMOTl+Vz7HAk2BDkBf4N8iUi33Tqo6XlUTVDWhdu3aJRthET33nKtjeOIJvyMxxpiTE8rEsAloEPC8vrctUBowVVWPqupa4GdcoghbzZrBvffCq6/C99/7HY0xxhRdKBPDYqCpiDQWkbLADcDUXPtMwd0tICK1cEVLa0IYU4l44gmoWhUGDQKN+pmxjTHhJmSJQVWPAX8GZgKrgHdVdaWIPCUiPb3dZgK7ROQH4GvgEVXdFaqYSkqNGvDkk/D55zBjht/RGGNM0YiG2U/ahIQETUlJ8TuMQh05Aq1auQH2li93vaKNMcYvIpKqqgnB7Ot35XPEKlsWRo6EH3+E8eP9jsYYY4JniSGEevSAjh1dsdKvv/odjTHGBMcSQwiJwKhRbiju4cP9jsYYY4JjiSHELrgAbrnFdXpbvdrvaIwxpnCWGErAM8+4OofBg/2OxBhjCmeJoQTUrQtDhsCHH8I33/gdjTHGFMwSQwkZNAgaNICHHoLMTL+jMcaY/FliKCHly8OIEbBkCbz1lt/RGGNM/iwxlKC+fd0UoI89BgcP+h2NMcbkzRJDCSpTxjVf3bzZjcJqjDGlkSWGEta+PfTpA//4B2zKPdasMcaUApYYfPDss5CR4YqUjDGmtLHE4IPGjeHBB+HNNyEMxgM0xkQZSww+eewxqF3bNV8NswFujTERzhKDT6pUgaefhjlzYPJkv6Mxxphslhh8dNtt0LIlPPIIHDrkdzTGGONYYvBRbCz885+wdi106ABbt/odkTHGhDgxiEgXEflJRFaLyJAC9ustIioiQc0uFEk6dYIpU2DlSmjXDlas8DsiY0y0C1liEJEYYBzQFWgB9BWRFnnsVxl4AFgYqlhKu549XV3D0aOun8Pnn/sdkTEmmoXyjqEtsFpV16jqEWAS0CuP/Z4G/g6khzCWUu/CC2HhQmjUCLp2hX//2++IjDHRKpSJoR6wMeB5mrftOBG5EGigqtNCGEfYaNAA5s6Fzp3hzjvd/A02EqsxpqT5VvksImWAUcCgIPa9U0RSRCRlx44doQ/OR5Urw9SpcM89btiMPn2sxZIxpmSFMjFsAhoEPK/vbctSGWgFzBKRdcAlwNS8KqBVdbyqJqhqQu3atUMYcukQGwtjx8ILL7jJfTp2hG3b/I7KGBMtQpkYFgNNRaSxiJQFbgCmZr2oqntVtZaqNlLVRsACoKeq2iARgAgMHOg6v61Y4Vos/fCD31EZY6JByBKDqh4D/gzMBFYB76rqShF5SkR6huq8kaZXLzcd6G+/waWXwhdf+B2RMSbSiYbZQD0JCQmaEoUjz23YAN27w6pV8PLLcPvtfkdkjAknIpKqqkH1FbOez2GiYUPXYqlTJ7jjDjeHdHpUN/A1xoSKJYYwUqUKfPIJ3HuvmwmuTRuXLIwxpjhZYggzWS2WPv0UDh+GpCTXtHXfPr8jM8ZECksMYer3v3etlR58EP71L2jRwvV/MMaYU2WJIYxVquSKlL79FmrUcC2Y+vSxUVqNMafGEkMEaNsWUlNh+HB313DuuTBhgs0MZ4w5OZYYIkRcnJsudNkyOO88NwnQlVfC//7nd2TGmHBjiSHCNG8Os2bBK69ASopLEs89B8eO+R2ZMSZcWGKIQGXKwJ/+5IbQ6NwZHn3UFTctWuR3ZMaYcGCJIYLVq+fGWnr/fdiyxY23dP31VrxkjCmYJYYIJwK9e8NPP8HQoa6D3DnnwP33Q4SPYG6MOUmWGKJElSrw1FOwerWrmH7pJWjSBJ55Bg4e9Ds6Y0xpYokhytSt6yqmV6xw4y4NHQpNm7qpRK2C2hgDlhii1jnnuPqHuXOhcWM3lWjr1q4fhPV/MCa6WWKIcu3bu+QwebKbX7pXL7j8cliwwO/IjDF+scRgEIGrr3bFS6+84uohLr3UVVrPn293EMZEG0sM5rjYWNf/4ZdfXEX155+7O4rWreHFF+HXX/2O0BhTEiwxmBNUquQqpTdvdpXS5cu75q1nnAH9+9tdhDGRLqSJQUS6iMhPIrJaRIbk8fpDIvKDiCwXkS9F5MxQxmOKplIlN4XookWwZAkMGODqItq3d0NtjBljdxHGRKKQJQYRiQHGAV2BFkBfEWmRa7fvgARVbQ28D/wjVPGYU9OmjZtrevNmePVVqFgRHngg+y5i3jy7izAmUoTyjqEtsFpV16jqEWAS0CtwB1X9WlUPeU8XAPVDGI8pBpUquQ5yCxfCd9/BLbe4u4jERGjVys0PsXat31EaY05FKBNDPWBjwPM0b1t+bgNm5PWCiNwpIikikrLDxnEoNS64wPWg3rIFXnvNJY1Bg+Css9xrw4a55GF3EsaEl1JR+SwiNwIJwHN5va6q41U1QVUTateuXbLBmUJVrAi33uruIlavhuefd0NwPP00XHghNGrkip2+/tp6VxsTDkKZGDYBDQKe1/e25SAiVwJ/BXqq6m8hjMeUgCZN4KGHYPZsN8XohAnu7mH8eLjiCjjtNFcn8eGHNkaTMaWVaIju80UkFvgZ6IRLCIuBP6rqyoB92uAqnbuo6i/BHDchIUFTUlJCELEJpYMH4bPPYMoU+Phj15opPh6uusqN2ZSQ4BJIxYp+R2pMZBKRVFVNCGrfUCUGL5BkYDQQA0xQ1eEi8hSQoqpTReQL4Dxgi/eWDaras6BjWmIIf8eOuWE4pkyBjz6Cdevc9jJl3HzVF13kEsVFF7lkUaGCj8EaEyFKTWIIBUsMkWfLFkhNdVORZi3btrnXypSBli1zJovzz3ed7owxwbPEYMKaqusvkTtZZDVIi4lxw3S0a5e9NG/ukogxJm+WGEzEUYW0NJcsFi92vbEXLYJ9+9zrVavCxRfnTBZ16vgbszGliSUGExUyM+HHH10z2azl++8hI8O93qhRdpJo08bNgV23rutvYUy0scRgotbBg25cp8BksXFjzn0qVXIJInA544wTt1Wr5oYkNyYSFCUxxIY6GGNKUsWKkJTklixbtsDKle5x82b3mLWkprrHvPpUxMVBzZpuqVUr52Ne22rVsmRiIoMlBhPxsu4ACrJ/f86EsWWLq+zeuRN27XKPP/3kBgvctSv/HtzVqrk5tJs2hWbNcj5WrVrsl2ZMSFhiMAaoXNktzZoVvq+qq/TOShhZj9u3w//+5yY6mjcPJk7MOU5U7drZSSIrYZx5puvoV65c3ktMTOHxZGbCb79BejocPuweA5cjR1xRWaNGULbsSf+JTBSxxGBMEYm4X/9Vq7oBA/OTnu4Sxc8/u2Txyy9ufeZMeOON4M4VG3tislDN/tI/fNh98QejTBmXiM4++8TlrLNcgjIGLDEYEzLx8a5zXsuWJ762f78bcDAtzf3aD2ZJT3ePZcq4Y2ct5csX/Dwuzp1n9ers5Z13YPfu7HhEoH797ERx5pluIMSKFV1lfUGPdhcSeSwxGOODypVdE9o2bfyLYfdud0cTmDBWr3ZDlRRldPu4uOwEEVjxXti6iEs+NWq4pWbNvNcDn1eq5JLj4cNw6FDOJb9tMTHZx8g6Ts2arj4omKK6wqi6OqfY2MhpeGCJwZgolfWle/HFJ76Wng4HDrjWWsE8HjgAR49mvz+wbiW/9cxMV1ezezds2ABLl7r1khp1VwSqV8+ZNLISR2Zm4Ukn8DVVd7ysu7Xy5Qte4uNdkssqksxaqlU7cVt8fMknHEsMxpgTZBVD1apV8udOT3ej7+7e7Sr2d+/OXt+/332xVqjglvzWA58fO5b9/qzjZa0HPt+61TVr3r3b/frP65jVquV9jrJls+9kshoAZK1nLbt25Xx+4IC7nsK6ksXFZSeMu+92w9qHmiUGY0ypEh8fXBPjoqhRw9WdlDaZmS457N3rlj17stfz2nb66SUTlyUGY4zxSZky2UVGpYmNR2mMMSYHSwzGGGNysMRgjDEmh5AmBhHpIiI/ichqERmSx+vlROQd7/WFItIolPEYY4wpXMgSg4jEAOOArkALoK+ItMi1223Ar6p6NvAC8PdQxWOMMSY4obxjaAusVtU1qnoEmAT0yrVPL+A/3vr7QCeRSOk7aIwx4SmUiaEeEDhFSpq3Lc99VPUYsBeomftAInKniKSISMqOovTVN8YYU2RhUfmsquNVNUFVE2rXru13OMYYE9FC2cFtE9Ag4Hl9b1te+6SJSCxQFdhV0EFTU1N3isj6k4ypFrDzJN8bCaL5+qP52iG6r9+u3Tkz2DeFMjEsBpqKSGNcArgB+GOufaYC/YFvgWuBr7SQSahV9aRvGUQkJdg5TyNRNF9/NF87RPf127UX/dpDlhhU9ZiI/BmYCcQAE1R1pYg8BaSo6lTgNeAtEVkN7MYlD2OMMT4K6VhJqjodmJ5r2xMB6+nAdaGMwRhjTNGEReVzMRrvdwA+i+brj+Zrh+i+frv2IpJCivSNMcZEmWi7YzDGGFMISwzGGGNyiJrEUNiAfpFMRNaJyPcislREUvyOJ9REZIKIbBeRFQHbaojI5yLyi/dY3c8YQyWfax8mIpu8z3+piCT7GWOoiEgDEflaRH4QkZUi8oC3PVo++/yuv8iff1TUMXgD+v0MXIUbmmMx0FdVf/A1sBIiIuuABFWNik4+InI5cAB4U1Vbedv+AexW1We9HwbVVXWwn3GGQj7XPgw4oKoj/Ywt1ESkLlBXVZeISGUgFbgaGEB0fPb5XX8fivj5R8sdQzAD+pkIoaqzcf1iAgUO2Pgf3H+YiJPPtUcFVd2iqku89f3AKtx4bNHy2ed3/UUWLYkhmAH9IpkCn4lIqojc6XcwPjlNVbd461uB0/wMxgd/FpHlXlFTRBalBPLmdmkDLCQKP/tc1w9F/PyjJTFEu0RVvRA3N8a9XnFD1PKGXYn8MtRsLwNNgAuALcDzvkYTYiJSCfgAGKiq+wJfi4bPPo/rL/LnHy2JIZgB/SKWqm7yHrcDk3FFa9Fmm1cGm1UWu93neEqMqm5T1QxVzQT+TQR//iISh/tS/D9V/dDbHDWffV7XfzKff7QkhuMD+olIWdyYTFN9jqlEiEhFryIKEakIdAZWFPyuiJQ1YCPe40c+xlKisr4UPdcQoZ+/N8nXa8AqVR0V8FJUfPb5Xf/JfP5R0SoJwGuiNZrsAf2G+xtRyRCRs3B3CeDGxvpvpF+7iEwEOuCGHN4GPAlMAd4FGgLrgT6qGnGVtPlcewdcMYIC64A/BZS5RwwRSQTmAN8Dmd7mx3Dl7NHw2ed3/X0p4ucfNYnBGGNMcKKlKMkYY0yQLDEYY4zJwRKDMcaYHCwxGGOMycESgzHGmBwsMRjjEZGMgBEolxbnKLwi0ihwxFNjSrOQzvlsTJg5rKoX+B2EMX6zOwZjCuHNZ/EPb06LRSJytre9kYh85Q1O9qWINPS2nyYik0Vkmbdc5h0qRkT+7Y2V/5mIlPf2v98bQ3+5iEzy6TKNOc4SgzHZyucqSro+4LW9qnoeMBbXgx7gReA/qtoa+D9gjLd9DPCNqp4PXAis9LY3BcapaktgD9Db2z4EaOMd567QXJoxwbOez8Z4ROSAqlbKY/s64ApVXeMNUrZVVWuKyE7cxChHve1bVLWWiOwA6qvqbwHHaAR8rqpNveeDgThVfUZEPsVNrjMFmKKqB0J8qcYUyO4YjAmO5rNeFL8FrGeQXcfXDRiHu7tYLCJW92d8ZYnBmOBcH/D4rbc+HzdSL0A/3ABmAF8Cd4ObVlZEquZ3UBEpAzRQ1a+BwUBV4IS7FmNKkv0yMSZbeRFZGvD8U1XNarJaXUSW43719/W23Qe8LiKPADuAW7ztDwDjReQ23J3B3bgJUvISA7ztJQ8BxqjqnmK6HmNOitUxGFMIr44hQVV3+h2LMSXBipKMMcbkYHcMxhhjcrA7BmOMMTlYYjDGGJODJQZjjDE5WGIwxhiTgyUGY4wxOfx/fNk/JExAvcUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epoch = training_log.iloc[:,0]\n",
    "acc = training_log.iloc[:,1]\n",
    "loss = training_log.iloc[:,2]\n",
    "val_acc = training_log.iloc[:,3]\n",
    "val_loss = training_log.iloc[:,4]\n",
    "\n",
    "plt.plot(acc,'b-',label=\"Train accuracy\")\n",
    "plt.plot(val_acc,'r-',label=\"Val accuracy\")\n",
    "plt.title(u'Training Accuracy per epoch')\n",
    "plt.xlabel(u'Epochs')\n",
    "plt.ylabel(u'Accuracy')\n",
    "\n",
    "plt.legend()\n",
    "plt.show()\n",
    "\n",
    "plt.plot(loss,'b-',label=\"Train loss\")\n",
    "plt.plot(val_loss,'r-',label=\"Val loss\")\n",
    "\n",
    "plt.title(u'Training Loss per epoch')\n",
    "plt.xlabel(u'Epochs')\n",
    "plt.ylabel(u'Loss')\n",
    "\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "### ** >>> Exercise 7 (Take home): **  \n",
    "\n",
    "Now, we have the word vectors, but our input data is a sequence of words (or say sentence). \n",
    "How can we utilize these \"word\" vectors to represent the sentence data and train our model?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Answer here\n",
    "    May be we can cut every vacabularies of sentenses in the beginning  \n",
    "ex: most_similar(happy, topn=10) \n",
    "    \"Today is my birthday\" is the sentence we want to seperate every words of that, afterward \n",
    "    Today = A    we abstrat A for (today,0.50033) \n",
    "    is = B       we abstrat B for (is,0.41033) \n",
    "    my = C       we abstrat C for (my,0.62033) \n",
    "    birthday = D we abstrat D for (birthday,0.95033) \n",
    "then we got the closest score D which reach the score 10 (happy) highestly \n",
    "and we could regard the emotion of this sentence as \"happy\"  \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### ** >>> Exercise 8 (Take home): **  \n",
    "\n",
    "Generate a t-SNE visualization to show the 15 words most related to the words \"angry\", \"happy\", \"sad\", \"fear\" (60 words total).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Answer here\n",
    "word_list = ['happy', 'angry', 'data', 'mining']\n",
    "\n",
    "topn = 15\n",
    "happy_words = ['happy'] + [word_ for word_, sim_ in w2v_google_model.most_similar('happy', topn=topn)]\n",
    "angry_words = ['angry'] + [word_ for word_, sim_ in w2v_google_model.most_similar('angry', topn=topn)]        \n",
    "sad_words = ['sad'] + [word_ for word_, sim_ in w2v_google_model.most_similar('sad', topn=topn)]        \n",
    "fear_words = ['fear'] + [word_ for word_, sim_ in w2v_google_model.most_similar('fear', topn=topn)]        \n",
    "\n",
    "print('happy_words: ', happy_words)\n",
    "print('angry_words: ', angry_words)\n",
    "print('sad_words: ', sad_words)\n",
    "print('fear_words: ', fear_words)\n",
    "\n",
    "target_words = happy_words + angry_words + sad_words + fear_words\n",
    "print('\\ntarget words: ')\n",
    "print(target_words)\n",
    "\n",
    "print('\\ncolor list:')\n",
    "cn = topn + 1\n",
    "color = ['b'] * cn + ['g'] * cn + ['r'] * cn + ['y'] * cn\n",
    "print(color)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn.manifold import TSNE\n",
    "\n",
    "## w2v model\n",
    "model = w2v_google_model\n",
    "\n",
    "## prepare training word vectors\n",
    "size = 200\n",
    "target_size = len(target_words)\n",
    "all_word = list(model.vocab.keys())\n",
    "word_train = target_words + all_word[:size]\n",
    "X_train = model[word_train]\n",
    "\n",
    "## t-SNE model\n",
    "tsne = TSNE(n_components=2, metric='cosine', random_state=28)\n",
    "\n",
    "## training\n",
    "X_tsne = tsne.fit_transform(X_train)\n",
    "\n",
    "## plot the result\n",
    "plt.figure(figsize=(7.5, 7.5), dpi=115)\n",
    "plt.scatter(X_tsne[:target_size, 0], X_tsne[:target_size, 1], c=color)\n",
    "for label, x, y in zip(target_words, X_tsne[:target_size, 0], X_tsne[:target_size, 1]):\n",
    "    plt.annotate(label, xy=(x,y), xytext=(0,0),  textcoords='offset points')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Third Part\n",
    "We apply three kinds of process\n",
    "\n",
    "\n",
    "\n",
    "1. TFIDF + LinearSVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "identification_csv = pd.read_csv(\"../input/dm2020-hw2-nthu/data_identification.csv\")\n",
    "emotion_csv = pd.read_csv(\"../input/dm2020-hw2-nthu/emotion.csv\")\n",
    "\n",
    "values = []\n",
    "with open('../input/dm2020-hw2-nthu/tweets_DM.json', 'r') as tweet_data:\n",
    "    for line in tweet_data.readlines():\n",
    "        k = json.loads(line)\n",
    "        values.append(k['_source']['tweet'])  ## 轉成json陣列 以方便可以直接轉成dataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data = pd.DataFrame(values) \n",
    "print(raw_data)\n",
    "raw_data = raw_data.sort_values(by=['tweet_id']) ## 依照tweet_id 做排序\n",
    "identification_csv = identification_csv.sort_values(by=['tweet_id'])\n",
    "ident_dict = identification_csv.set_index('tweet_id')['identification'].to_dict()## 將dataframe轉成dict 為了取出value的陣列\n",
    "raw_data['identification'] = ident_dict.values() ## 直接加入陣列在新的column\n",
    "raw_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## 依照'identification'的label做篩選 找出trainingData 和 testData\n",
    "df_train = raw_data.loc[lambda raw_data: raw_data['identification'] == 'train',:]\n",
    "df_test = raw_data.loc[lambda raw_data: raw_data['identification'] == 'test',:]\n",
    "\n",
    "## 把emotion在df_train中加入新的column\n",
    "df_train = df_train.sort_values(by=['tweet_id'])\n",
    "emotion_csv = emotion_csv.sort_values(by=['tweet_id'])\n",
    "emotion_dict = emotion_csv.set_index('tweet_id')['emotion'].to_dict()\n",
    "df_train['emotion'] = emotion_dict.values()\n",
    "print(df_train.shape)\n",
    "df_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## save as pickle file\n",
    "df_train.to_pickle(\"/kaggle/working/train_df.pkl\") \n",
    "df_test.to_pickle(\"/kaggle/working/test_df.pkl\") \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# load a pickle file\n",
    "train_df = pd.read_pickle(\"../input/tt-pickle-file/train_df.pkl\")[:-5000]\n",
    "test_df =  pd.read_pickle(\"../input/tt-pickle-file/train_df.pkl\")[-5000:]\n",
    "final_test_df = pd.read_pickle(\"../input/tt-pickle-file/test_df.pkl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.svm import LinearSVC\n",
    "import nltk\n",
    "# Initialize tfidf vectorizer\n",
    "tfidf_vectorizer = TfidfVectorizer(tokenizer=nltk.word_tokenize,stop_words='english', min_df=100)\n",
    "\n",
    "# Create tfidf train and test variables\n",
    "X_train_tfidf = tfidf_vectorizer.fit_transform(train_df['text']) \n",
    "y_train = train_df['emotion']\n",
    "X_test_tfidf = tfidf_vectorizer.transform(test_df['text'])\n",
    "y_test = test_df['emotion']\n",
    "\n",
    "# Create a LinearSVC model\n",
    "tfidf_svc = LinearSVC()\n",
    "\n",
    "# Train model\n",
    "tfidf_svc.fit(X_train_tfidf, y_train)\n",
    "\n",
    "# Run predict on tfidf test data to get predictions\n",
    "tfidf_svc_pred = tfidf_svc.predict(X_test_tfidf)\n",
    "\n",
    "# Calculate accuracy using the metrics module\n",
    "tfidf_svc_score = tfidf_svc.score(X_test_tfidf, y_test)\n",
    "\n",
    "print(\"LinearSVC Score:   %0.3f\" % tfidf_svc_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_X_test_tfidf = tfidf_vectorizer.transform(final_test_df['text'])\n",
    "final_tfidf_svc_pred = tfidf_svc.predict(final_X_test_tfidf)\n",
    "\n",
    "final_test_df['emotion'] = final_tfidf_svc_pred\n",
    "csvData = final_test_df[['tweet_id','emotion']].to_csv('/kaggle/working/predict_01.csv',header=['id','emotion'],index=False) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Third Part\n",
    "2. bow + DT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "identification_csv = pd.read_csv(\"../input/data-provided/data_identification.csv\")\n",
    "emotion_csv = pd.read_csv(\"../input/data-provided/emotion.csv\")\n",
    "with open('../input/tweets/tweets_DM.json', 'r') as tweet_data:\n",
    "    values = [json.loads(line)['_source']['tweet'] for line in tweet_data.readlines() ]\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data = pd.DataFrame(values)\n",
    "raw_data = raw_data.sort_values(by=['tweet_id'])\n",
    "identification_csv = identification_csv.sort_values(by=['tweet_id'])\n",
    "ident_dict = identification_csv.set_index('tweet_id')['identification'].to_dict()\n",
    "raw_data['identification'] = ident_dict.values()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = raw_data.loc[lambda raw_data: raw_data['identification'] == 'train',:]\n",
    "df_test = raw_data.loc[lambda raw_data: raw_data['identification'] == 'test',:]\n",
    "\n",
    "df_train = df_train.sort_values(by=['tweet_id'])\n",
    "emotion_csv = emotion_csv.sort_values(by=['tweet_id'])\n",
    "emotion_dict = emotion_csv.set_index('tweet_id')['emotion'].to_dict()\n",
    "df_train['emotion'] = emotion_dict.values()\n",
    "print(df_train.shape)\n",
    "df_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## save as pickle file\n",
    "df_train.to_pickle(\"/kaggle/working/train_df.pkl\") \n",
    "df_test.to_pickle(\"/kaggle/working/test_df.pkl\") \n",
    "df_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# load a pickle file\n",
    "train_df = pd.read_pickle(\"../input/pickle-data/train_df.pkl\")[:-1000]\n",
    "test_df =  pd.read_pickle(\"../input/pickle-data/train_df.pkl\")[-1000:]\n",
    "final_test_df = pd.read_pickle(\"../input/pickle-data/test_df.pkl\")\n",
    "final_test_df.shape\n",
    "final_test_df.head()\n",
    "print(train_df.shape)\n",
    "train_df.dropna(inplace=True)\n",
    "print(train_df.shape)\n",
    "print(test_df.shape)\n",
    "test_df.dropna(inplace=True)\n",
    "print(test_df.shape)\n",
    "train_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### generate feature_Bow ###\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "import nltk\n",
    "\n",
    "# build analyzers (bag-of-words)\n",
    "BOW_500 = CountVectorizer(max_features=1000, tokenizer=nltk.word_tokenize) \n",
    "\n",
    "# apply analyzer to training data\n",
    "BOW_500.fit(train_df['text'][:20000])\n",
    "\n",
    "train_data_BOW_features_500 = BOW_500.transform(train_df['text'][:20000])\n",
    "\n",
    "## check dimension\n",
    "train_data_BOW_features_500.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### generate X_train X_test ###\n",
    "\n",
    "# for a classificaiton problem, you need to provide both training & testing data\n",
    "X_train = BOW_500.transform(train_df['text'])\n",
    "y_train = train_df['emotion']\n",
    "\n",
    "X_test = BOW_500.transform(test_df['text'])\n",
    "y_test = test_df['emotion']\n",
    "\n",
    "## take a look at data dimension is a good habbit  :)\n",
    "print('X_train.shape: ', X_train.shape)\n",
    "print('y_train.shape: ', y_train.shape)\n",
    "print('X_test.shape: ', X_test.shape)\n",
    "print('y_test.shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### build Desicion Model ###\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "DT_model = DecisionTreeClassifier(random_state=0)\n",
    "\n",
    "## training!\n",
    "DT_model = DT_model.fit(X_train, y_train)\n",
    "\n",
    "## predict!\n",
    "y_train_pred = DT_model.predict(X_train)\n",
    "y_test_pred = DT_model.predict(X_test)\n",
    "\n",
    "## so we get the pred result\n",
    "y_test_pred[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### accuracy ###\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "acc_train = accuracy_score(y_true=y_train, y_pred=y_train_pred)\n",
    "acc_test = accuracy_score(y_true=y_test, y_pred=y_test_pred)\n",
    "\n",
    "print('training accuracy: {}'.format(round(acc_train, 2)))\n",
    "print('testing accuracy: {}'.format(round(acc_test, 2)))\n",
    "\n",
    "\n",
    "## precision, recall, f1-score,\n",
    "from sklearn.metrics import classification_report\n",
    "\n",
    "print(classification_report(y_true=y_test, y_pred=y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = BOW_500.transform(final_test_df['text'])\n",
    "final_train_pred = DT_model.predict(X_train)\n",
    "\n",
    "final_test_df['emotion'] = final_train_pred\n",
    "csvData = final_test_df[['tweet_id','emotion']].to_csv('/kaggle/working/predict.csv',header=['id','emotion'],index=False) \n",
    "final_train_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Third Part\n",
    "3. TFIDF + DNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# load a pickle file\n",
    "train_df = pd.read_pickle(\"../input/pickle-data/train_df.pkl\")[:-1000]\n",
    "test_df =  pd.read_pickle(\"../input/pickle-data/train_df.pkl\")[-1000:]\n",
    "final_test_df = pd.read_pickle(\"../input/pickle-data/test_df.pkl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "# Initialize tfidf vectorizer\n",
    "tfidf_vectorizer = TfidfVectorizer(tokenizer=nltk.word_tokenize,stop_words='english', min_df=100)\n",
    "\n",
    "# Create tfidf train and test variables\n",
    "X_train_tfidf = tfidf_vectorizer.fit_transform(train_df['text']).toarray() \n",
    "y_train = train_df['emotion']\n",
    "X_test_tfidf = tfidf_vectorizer.transform(test_df['text']).toarray() \n",
    "y_test = test_df['emotion']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## deal with label (string -> one-hot)\n",
    "\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "import keras\n",
    "label_encoder = LabelEncoder()\n",
    "label_encoder.fit(y_train)\n",
    "print('check label: ', label_encoder.classes_)\n",
    "print('\\n## Before convert')\n",
    "print('y_train[0:]:\\n', y_train[0:])\n",
    "print('\\ny_train.shape: ', y_train.shape)\n",
    "print('y_test.shape: ', y_test.shape)\n",
    "\n",
    "def label_encode(le, labels):\n",
    "    enc = le.transform(labels)\n",
    "    return keras.utils.to_categorical(enc)\n",
    "\n",
    "def label_decode(le, one_hot_label):\n",
    "    dec = np.argmax(one_hot_label, axis=1)\n",
    "    return le.inverse_transform(dec)\n",
    "\n",
    "y_train = label_encode(label_encoder, y_train)\n",
    "y_test = label_encode(label_encoder, y_test)\n",
    "\n",
    "print('\\n\\n## After convert')\n",
    "print('y_train[0:]:\\n', y_train[0:])\n",
    "print('\\ny_train.shape: ', y_train.shape)\n",
    "print('y_test.shape: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# I/O check\n",
    "input_shape = X_train_tfidf.shape[1]\n",
    "print('input_shape: ', input_shape)\n",
    "\n",
    "output_shape = len(label_encoder.classes_)\n",
    "print('output_shape: ', output_shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Model\n",
    "from keras.layers import Input, Dense\n",
    "from keras.layers import ReLU, Softmax\n",
    "\n",
    "# input layer\n",
    "model_input = Input(shape=(input_shape, ))  # 500\n",
    "X = model_input\n",
    "\n",
    "# 1st hidden layer\n",
    "X_W1 = Dense(units=1024)(X)  # 64\n",
    "H1 = ReLU()(X_W1)\n",
    "\n",
    "# 2nd hidden layer\n",
    "H1_W2 = Dense(units=1024)(H1)  # 64\n",
    "H2 = ReLU()(H1_W2)\n",
    "\n",
    "# output layer\n",
    "H2_W3 = Dense(units=output_shape)(H2)  # 4\n",
    "H3 = Softmax()(H2_W3)\n",
    "\n",
    "model_output = H3\n",
    "\n",
    "# create model\n",
    "model = Model(inputs=[model_input], outputs=[model_output])\n",
    "\n",
    "# loss function & optimizer\n",
    "model.compile(optimizer='adam',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# show model construction\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.callbacks import CSVLogger\n",
    "csv_logger = CSVLogger('/kaggle/working/training_log.csv')\n",
    "\n",
    "# training setting\n",
    "epochs = 10\n",
    "batch_size = 128\n",
    "\n",
    "# training!\n",
    "history = model.fit(X_train_tfidf, y_train, \n",
    "                    epochs=epochs, \n",
    "                    batch_size=batch_size, \n",
    "                    callbacks=[csv_logger],\n",
    "                    validation_data = (X_test_tfidf, y_test))\n",
    "print('training finish')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "pred_result = model.predict(X_test, batch_size=128)\n",
    "pred_result[:5]\n",
    "pred_result = label_decode(label_encoder, pred_result)\n",
    "\n",
    "from sklearn.metrics import accuracy_score\n",
    "print('testing accuracy: {}'.format(round(accuracy_score(label_decode(label_encoder, y_test), pred_result), 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_X_test = tfidf_vectorizer.transform(final_test_df['text']).toarray() \n",
    "final_pred_result = model.predict(final_X_test, batch_size=128)\n",
    "final_pred_result = label_decode(label_encoder, final_pred_result)\n",
    "final_test_df['emotion'] = final_pred_result\n",
    "csvData = final_test_df[['tweet_id','emotion']].to_csv('/kaggle/working/predict.csv',header=['id','emotion'],index=False) "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}