Classification_Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color='red'> Project 2 : Buzz Prediction on Twitter\n",
    "\n",
    "Project Description:\n",
    "- There are two different datasets for Regression and Classification tasks. Right-most column in both the datasets is a dependent variable i.e. buzz.\n",
    "- Data description files are also provided for both the datasets.\n",
    "- Deciding which dataset is for which task is part of the project.\n",
    "- Read data into Jupyter notebook, use pandas to import data into a data frame.\n",
    "- Preprocess data: Explore data, check for missing data and apply data scaling. Justify the type of scaling used.\n",
    "\n",
    "Regression Task:\n",
    "- Apply all the regression models you've learned so far. If your model has a scaling parameter(s) use Grid Search to find the best scaling parameter. Use plots and graphs to help you get a better glimpse of the results. \n",
    "- Then use cross-validation to find average training and testing score. \n",
    "- Your submission should have at least the following regression models: KNN regressor, linear regression, Ridge, Lasso, polynomial regression, SVM both simple and with kernels. \n",
    "- Finally, find the best regressor for this dataset and train your model on the entire dataset using the best parameters and predict buzz for the test_set.\n",
    "\n",
    "Classification Task:\n",
    "- Decide about a good evaluation strategy and justify your choice.\n",
    "- Find best parameters for the following classification models: KNN classification, Logistic Regression, Linear Support Vector Machine, Kernelized Support Vector Machine, Decision Tree. \n",
    "- Which model gives the best results?\n",
    "\n",
    "Deliverables:\n",
    "- Submit IPython notebook. Use markdown to provide inline comments for this project.\n",
    "- Rename notebook with your group number and submit only one notebook. Before submitting, make sure everything runs as expected. To check that, restart the kernel (in the menubar, select Kernel > Restart) and then run all cells (in the menubar, select Cell > Run All).\n",
    "- Visualization encouraged.\n",
    "\n",
    "Questions regarding the project:\n",
    "- We have created a discussion board under Projects folder on e-learning. Create threads over there and post your queries related to project there.\n",
    "- There is a high possibility that your classmate has also faced the same problem and knows the solution. So this is an effort to encourage collaborative learning, reducing mails for frequently asked queries and also making all the information available to everyone.\n",
    "- Please check existing threads for your query before creating a new one. It goes without saying that do not share your code or complete solutions there.\n",
    "- We will also answer queries there. We will not be answering any project related queries through the mail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Instructions given in class: (11/07)\n",
    "Steps to be followed:\n",
    "1. read csv files\n",
    "2. explore dataset\n",
    "3. evaluation \n",
    "4. split \n",
    "5. scale\n",
    "6. model\n",
    "-GridSearchCV - best parameters\n",
    "-train, test score\n",
    "\n",
    "Proj 3 (If you need head start):\n",
    "7. ensemble\n",
    "8. dimension reduction - repeat step 4 to 7\n",
    "9. Deep learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Dataset For Classification Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "Dataset_2 = pd.read_csv('Twitter-Absolute-Sigma-500.data', header = None)   #Reading Data For classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 140707 entries, 0 to 140706\n",
      "Data columns (total 78 columns):\n",
      "0     140707 non-null int64\n",
      "1     140707 non-null int64\n",
      "2     140707 non-null int64\n",
      "3     140707 non-null int64\n",
      "4     140707 non-null int64\n",
      "5     140707 non-null int64\n",
      "6     140707 non-null int64\n",
      "7     140707 non-null int64\n",
      "8     140707 non-null int64\n",
      "9     140707 non-null int64\n",
      "10    140707 non-null int64\n",
      "11    140707 non-null int64\n",
      "12    140707 non-null int64\n",
      "13    140707 non-null int64\n",
      "14    140707 non-null float64\n",
      "15    140707 non-null float64\n",
      "16    140707 non-null float64\n",
      "17    140707 non-null float64\n",
      "18    140707 non-null float64\n",
      "19    140707 non-null float64\n",
      "20    140707 non-null float64\n",
      "21    140707 non-null float64\n",
      "22    140707 non-null float64\n",
      "23    140707 non-null float64\n",
      "24    140707 non-null float64\n",
      "25    140707 non-null float64\n",
      "26    140707 non-null float64\n",
      "27    140707 non-null float64\n",
      "28    140707 non-null int64\n",
      "29    140707 non-null int64\n",
      "30    140707 non-null int64\n",
      "31    140707 non-null int64\n",
      "32    140707 non-null int64\n",
      "33    140707 non-null int64\n",
      "34    140707 non-null int64\n",
      "35    140707 non-null float64\n",
      "36    140707 non-null float64\n",
      "37    140707 non-null float64\n",
      "38    140707 non-null float64\n",
      "39    140707 non-null float64\n",
      "40    140707 non-null float64\n",
      "41    140707 non-null float64\n",
      "42    140707 non-null float64\n",
      "43    140707 non-null float64\n",
      "44    140707 non-null float64\n",
      "45    140707 non-null float64\n",
      "46    140707 non-null float64\n",
      "47    140707 non-null float64\n",
      "48    140707 non-null float64\n",
      "49    140707 non-null float64\n",
      "50    140707 non-null float64\n",
      "51    140707 non-null float64\n",
      "52    140707 non-null float64\n",
      "53    140707 non-null float64\n",
      "54    140707 non-null float64\n",
      "55    140707 non-null float64\n",
      "56    140707 non-null int64\n",
      "57    140707 non-null int64\n",
      "58    140707 non-null int64\n",
      "59    140707 non-null int64\n",
      "60    140707 non-null int64\n",
      "61    140707 non-null int64\n",
      "62    140707 non-null int64\n",
      "63    140707 non-null float64\n",
      "64    140707 non-null float64\n",
      "65    140707 non-null float64\n",
      "66    140707 non-null float64\n",
      "67    140707 non-null float64\n",
      "68    140707 non-null float64\n",
      "69    140707 non-null float64\n",
      "70    140707 non-null int64\n",
      "71    140707 non-null int64\n",
      "72    140707 non-null int64\n",
      "73    140707 non-null int64\n",
      "74    140707 non-null int64\n",
      "75    140707 non-null int64\n",
      "76    140707 non-null int64\n",
      "77    140707 non-null float64\n",
      "dtypes: float64(43), int64(35)\n",
      "memory usage: 83.7 MB\n"
     ]
    }
   ],
   "source": [
    "Dataset_2.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>68</th>\n",
       "      <th>69</th>\n",
       "      <th>70</th>\n",
       "      <th>71</th>\n",
       "      <th>72</th>\n",
       "      <th>73</th>\n",
       "      <th>74</th>\n",
       "      <th>75</th>\n",
       "      <th>76</th>\n",
       "      <th>77</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>889</td>\n",
       "      <td>939</td>\n",
       "      <td>960</td>\n",
       "      <td>805</td>\n",
       "      <td>805</td>\n",
       "      <td>1143</td>\n",
       "      <td>1121</td>\n",
       "      <td>549</td>\n",
       "      <td>613</td>\n",
       "      <td>587</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>889</td>\n",
       "      <td>939</td>\n",
       "      <td>960</td>\n",
       "      <td>805</td>\n",
       "      <td>805</td>\n",
       "      <td>1143</td>\n",
       "      <td>1121</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>542</td>\n",
       "      <td>473</td>\n",
       "      <td>504</td>\n",
       "      <td>626</td>\n",
       "      <td>647</td>\n",
       "      <td>795</td>\n",
       "      <td>832</td>\n",
       "      <td>366</td>\n",
       "      <td>288</td>\n",
       "      <td>318</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>542</td>\n",
       "      <td>473</td>\n",
       "      <td>504</td>\n",
       "      <td>626</td>\n",
       "      <td>647</td>\n",
       "      <td>795</td>\n",
       "      <td>832</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>92</td>\n",
       "      <td>99</td>\n",
       "      <td>196</td>\n",
       "      <td>100</td>\n",
       "      <td>184</td>\n",
       "      <td>79</td>\n",
       "      <td>162</td>\n",
       "      <td>66</td>\n",
       "      <td>59</td>\n",
       "      <td>118</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>92</td>\n",
       "      <td>99</td>\n",
       "      <td>196</td>\n",
       "      <td>100</td>\n",
       "      <td>184</td>\n",
       "      <td>79</td>\n",
       "      <td>162</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>90</td>\n",
       "      <td>87</td>\n",
       "      <td>92</td>\n",
       "      <td>344</td>\n",
       "      <td>184</td>\n",
       "      <td>848</td>\n",
       "      <td>184</td>\n",
       "      <td>83</td>\n",
       "      <td>78</td>\n",
       "      <td>76</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>90</td>\n",
       "      <td>87</td>\n",
       "      <td>92</td>\n",
       "      <td>344</td>\n",
       "      <td>184</td>\n",
       "      <td>848</td>\n",
       "      <td>184</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>169</td>\n",
       "      <td>98</td>\n",
       "      <td>101</td>\n",
       "      <td>90</td>\n",
       "      <td>96</td>\n",
       "      <td>95</td>\n",
       "      <td>185</td>\n",
       "      <td>141</td>\n",
       "      <td>68</td>\n",
       "      <td>85</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>169</td>\n",
       "      <td>98</td>\n",
       "      <td>101</td>\n",
       "      <td>90</td>\n",
       "      <td>96</td>\n",
       "      <td>95</td>\n",
       "      <td>185</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 78 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    0    1    2    3    4     5     6    7    8    9  ...    68   69   70  \\\n",
       "0  889  939  960  805  805  1143  1121  549  613  587 ...   1.0  1.0  889   \n",
       "1  542  473  504  626  647   795   832  366  288  318 ...   1.0  1.0  542   \n",
       "2   92   99  196  100  184    79   162   66   59  118 ...   1.0  1.0   92   \n",
       "3   90   87   92  344  184   848   184   83   78   76 ...   1.0  1.0   90   \n",
       "4  169   98  101   90   96    95   185  141   68   85 ...   1.0  1.0  169   \n",
       "\n",
       "    71   72   73   74    75    76   77  \n",
       "0  939  960  805  805  1143  1121  1.0  \n",
       "1  473  504  626  647   795   832  1.0  \n",
       "2   99  196  100  184    79   162  0.0  \n",
       "3   87   92  344  184   848   184  1.0  \n",
       "4   98  101   90   96    95   185  1.0  \n",
       "\n",
       "[5 rows x 78 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Dataset_2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Dataset.columns\n",
    "Dataset_2[77].unique().astype(int)   #Dependent variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Dataset_2.isnull().sum().sum()    #No Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>68</th>\n",
       "      <th>69</th>\n",
       "      <th>70</th>\n",
       "      <th>71</th>\n",
       "      <th>72</th>\n",
       "      <th>73</th>\n",
       "      <th>74</th>\n",
       "      <th>75</th>\n",
       "      <th>76</th>\n",
       "      <th>77</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "      <td>140707.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>172.279823</td>\n",
       "      <td>155.150625</td>\n",
       "      <td>165.464476</td>\n",
       "      <td>176.820549</td>\n",
       "      <td>186.937700</td>\n",
       "      <td>216.209208</td>\n",
       "      <td>243.866510</td>\n",
       "      <td>87.050154</td>\n",
       "      <td>78.639236</td>\n",
       "      <td>84.269574</td>\n",
       "      <td>...</td>\n",
       "      <td>1.113444</td>\n",
       "      <td>1.196131</td>\n",
       "      <td>172.838807</td>\n",
       "      <td>155.630878</td>\n",
       "      <td>165.938674</td>\n",
       "      <td>177.314810</td>\n",
       "      <td>187.463794</td>\n",
       "      <td>216.776294</td>\n",
       "      <td>244.479194</td>\n",
       "      <td>0.197396</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>509.872276</td>\n",
       "      <td>471.573236</td>\n",
       "      <td>495.360236</td>\n",
       "      <td>528.351277</td>\n",
       "      <td>560.331281</td>\n",
       "      <td>632.188378</td>\n",
       "      <td>707.402192</td>\n",
       "      <td>234.731748</td>\n",
       "      <td>218.448179</td>\n",
       "      <td>233.536510</td>\n",
       "      <td>...</td>\n",
       "      <td>1.374287</td>\n",
       "      <td>1.826150</td>\n",
       "      <td>510.937549</td>\n",
       "      <td>472.462733</td>\n",
       "      <td>496.233557</td>\n",
       "      <td>529.286514</td>\n",
       "      <td>561.309487</td>\n",
       "      <td>633.203935</td>\n",
       "      <td>708.436795</td>\n",
       "      <td>0.398035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>22.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>125.000000</td>\n",
       "      <td>112.000000</td>\n",
       "      <td>119.000000</td>\n",
       "      <td>126.000000</td>\n",
       "      <td>133.000000</td>\n",
       "      <td>161.000000</td>\n",
       "      <td>186.000000</td>\n",
       "      <td>70.000000</td>\n",
       "      <td>64.000000</td>\n",
       "      <td>67.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.100000</td>\n",
       "      <td>1.119048</td>\n",
       "      <td>126.000000</td>\n",
       "      <td>113.000000</td>\n",
       "      <td>119.000000</td>\n",
       "      <td>127.000000</td>\n",
       "      <td>134.000000</td>\n",
       "      <td>162.000000</td>\n",
       "      <td>187.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>24210.000000</td>\n",
       "      <td>22899.000000</td>\n",
       "      <td>20495.000000</td>\n",
       "      <td>27007.000000</td>\n",
       "      <td>30957.000000</td>\n",
       "      <td>28603.000000</td>\n",
       "      <td>37505.000000</td>\n",
       "      <td>15105.000000</td>\n",
       "      <td>15730.000000</td>\n",
       "      <td>16389.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>185.666672</td>\n",
       "      <td>295.000000</td>\n",
       "      <td>24301.000000</td>\n",
       "      <td>22980.000000</td>\n",
       "      <td>20495.000000</td>\n",
       "      <td>27071.000000</td>\n",
       "      <td>31028.000000</td>\n",
       "      <td>28697.000000</td>\n",
       "      <td>37505.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 78 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  0              1              2              3   \\\n",
       "count  140707.000000  140707.000000  140707.000000  140707.000000   \n",
       "mean      172.279823     155.150625     165.464476     176.820549   \n",
       "std       509.872276     471.573236     495.360236     528.351277   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%         3.000000       2.000000       3.000000       3.000000   \n",
       "50%        22.000000      19.000000      20.000000      22.000000   \n",
       "75%       125.000000     112.000000     119.000000     126.000000   \n",
       "max     24210.000000   22899.000000   20495.000000   27007.000000   \n",
       "\n",
       "                  4              5              6              7   \\\n",
       "count  140707.000000  140707.000000  140707.000000  140707.000000   \n",
       "mean      186.937700     216.209208     243.866510      87.050154   \n",
       "std       560.331281     632.188378     707.402192     234.731748   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%         3.000000       4.000000       5.000000       2.000000   \n",
       "50%        23.000000      28.000000      33.000000      13.000000   \n",
       "75%       133.000000     161.000000     186.000000      70.000000   \n",
       "max     30957.000000   28603.000000   37505.000000   15105.000000   \n",
       "\n",
       "                  8              9       ...                   68  \\\n",
       "count  140707.000000  140707.000000      ...        140707.000000   \n",
       "mean       78.639236      84.269574      ...             1.113444   \n",
       "std       218.448179     233.536510      ...             1.374287   \n",
       "min         0.000000       0.000000      ...             0.000000   \n",
       "25%         2.000000       2.000000      ...             1.000000   \n",
       "50%        11.000000      13.000000      ...             1.000000   \n",
       "75%        64.000000      67.000000      ...             1.100000   \n",
       "max     15730.000000   16389.000000      ...           185.666672   \n",
       "\n",
       "                  69             70             71             72  \\\n",
       "count  140707.000000  140707.000000  140707.000000  140707.000000   \n",
       "mean        1.196131     172.838807     155.630878     165.938674   \n",
       "std         1.826150     510.937549     472.462733     496.233557   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%         1.000000       3.000000       2.000000       3.000000   \n",
       "50%         1.000000      22.000000      19.000000      21.000000   \n",
       "75%         1.119048     126.000000     113.000000     119.000000   \n",
       "max       295.000000   24301.000000   22980.000000   20495.000000   \n",
       "\n",
       "                  73             74             75             76  \\\n",
       "count  140707.000000  140707.000000  140707.000000  140707.000000   \n",
       "mean      177.314810     187.463794     216.776294     244.479194   \n",
       "std       529.286514     561.309487     633.203935     708.436795   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%         3.000000       3.000000       4.000000       6.000000   \n",
       "50%        22.000000      23.000000      28.000000      33.000000   \n",
       "75%       127.000000     134.000000     162.000000     187.000000   \n",
       "max     27071.000000   31028.000000   28697.000000   37505.000000   \n",
       "\n",
       "                  77  \n",
       "count  140707.000000  \n",
       "mean        0.197396  \n",
       "std         0.398035  \n",
       "min         0.000000  \n",
       "25%         0.000000  \n",
       "50%         0.000000  \n",
       "75%         0.000000  \n",
       "max         1.000000  \n",
       "\n",
       "[8 rows x 78 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Dataset_2.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "corr = Dataset_2.corr()    #calculate correlation matrix\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2415357c5f8>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATcAAAD/CAYAAACU7pFfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJztnXm8VVX1wL8LcMxZc0gsnC3NUBHtlxYOKWmimZZDhZoRaWmWKWS/n1r5y7TSJvtJikOKgoqKhCiKolioCKgYIoPMo8yDgOj6/bH3OXffyznvnnfPu++9e9/6fj7nc/c9ezh7n/fOvmudvdZeoqoYhmHUG+1augOGYRjVwCY3wzDqEpvcDMOoS2xyMwyjLrHJzTCMusQmN8Mw6hKb3AzDqEtyTW4i0l1EJovIVBHp01SdMgzDyItUasQrIu2Bd4AvA3OAV4FzVfU/Tdc9wzCMysgjuXUFpqrqdFXdADwInN403TIMw8hHhxx19wRmB9/nAEeVqWO+XoZRXaSlO9BayCO5Jd3ETSYvEeklImNFZGy/fv1yXM4wDCM7eSS3OcBewfeOwLzSQqraD4hmNZPcDMNoFvJIbq8C+4vI3iKyOXAOMKRcpSnHnJzjkoZhGNmoeLUUQEROAW4F2gP9VfWGMlVMcjOM6mLv3Dy5JrcKsMnNMKqLTW6eZvdQiNTSKcecbCqqYRhVwyQ3w6gvTHLz5JbcRKS9iIwXkaFN0SHDMIymoCnU0suBSVkLJ6mlpqIahtHU5F0t7QjcA9wA/ERVv1qmiqmlhlFdTC315JXcbgWuAj5qgr4YhmE0GRVPbiLyVWCRqr5WplyR+1U5tdTUU8MwmoI8Wx79Bvg2sBHYEtgOGKyq32qgmqmlhlFdTC31VCy5qWpfVe2oqp1wrlcjy0xsQPkFhVByMynOMIxKsW3GDcOoS8yI1zDqC1NLPa3C/SqLWmoqqmEYjcEkN8OoL0xy8+SNfnWFiLwlIhNF5AER2bKpOmYYhpGHPHZuewKXAV1U9RDcnm7nlKuXVy01Vy3DMLKQx85tT2AM8DlgJfAY8CdVfbqBaqaWGkZ1MbXUk8fObS7wO2AWMB9YUWZiMwzDaDbyqKU74uKU7g18AviYiGxixJvH/aqcWmoqqmEYaeRRS88Guqvqd/337wBHq+olDVQztdQwqouppZ48q6WzgKNFZGsREeAEMuzrVi3JzSQ4wzBC8rxzexl4GBgHvOnbsqjLhmG0CsyI1zDqC1NLPTXnfmW7iRiGkQXbFcQwjLrE1FLDqC9MLfWUldxEpL+ILBKRicG5m0XkbRF5Q0QeFZEdsl6wudVSU1ENo21SVnITkS8Cq4F7vQ8pInISbufdjSLyWwBVvTrD9UxyM4zqYpKbp6zkpqovAEtLzj2tqhv91zFAxyr0zTAMo2KaYkHhIuDJtMzmcr+y3UQMwwjJtKAgIp2AoZFaGpy/BugCnKnZViZMLTWM6mJqqadDpRVFpCfwVeCEjBObYRhGs1GRWioi3YGrgR6qurYxdVuDWmrqqWHUP1lWSx8AugG7AAuBa4G+wBbAEl9sjKr2znA9k/AMo7qYWurJslp6rqruoaqb+SDMd6rqfqq6l6p29keWiQ1ofZKbSXGGUZ+Y+5VhGHWJuV8ZRn1haqmnIvcrf/5HIjLZh/a7KesFW7NaaiqqYdQPlbpfHQdcA5yqqutFZFdVXZTheia5GUZ1McnNU5H7FfAD4EZVXe/LZJnYDMMwmo1KFxQOAI4VkZdFZJSIHJm1Yq2opaaiGkZtU5H7lX//NhK4HDgSGAjsk+SpICK9gF4At99++xG9evVqqr4bhrEpppZ6KnW/mgMM9pPZKyLyEc7Id3FpQVXtRyFwjL1zMwyjWahULX0MOB5ARA4ANgfey1Kx1tRS203EMGqTSt2v/gH0BzoDG4ArVXVkhuuZ5GYY1cXUUo8Z8RpGfWGTm6fuQ/s1pVpq6qlh1A4muRlGfWGSmyeL+9VeIvKciEzyrlaX+/M7icgIEZniP3fMcsF6kdxMijOM1k0WU5CNwE9VdZyIbAu8JiIjgAuAZ1X1RhHpA/TBbWBpGEYNM+WYkxM1rP1HP1VTUmGj1VIReRz4iz+6qep8EdkDeF5VDyxT3dRSw6guuSegKcd+JXlye/HJmprcGrWg4D0VDgNeBnZT1fkA/nPXLG3Uo1pqKqpRV4gkHzVGZslNRLYBRgE3qOpgEVmuqjsE+ctUdZP3buZ+ZRjNSu5ZaOpxpyVOCvs990RNzXCZ3K9EZDPgEeB+VR3sTy8UkT0CtTRxZxBzvzKMGqMGpbQksqyWCnAnMElV/xBkDQF6+nRP4PEsF1wx+An3OWQYK4YMc+nHhrLisaFxOi4bpqN6g5+I08seeIRlDzwCwPKHCpdfPujRTdLLBz1aSA8cXMhPS/v2iuqF7QZloz4su38Qy+4fVP4mGEYrRtq3SzxqjSzuV8cALwJvAh/50z/HvXcbBHwSmAWcraql+76VYpKbYVSX3GLXtO5fT3xO9x3+SE2JdGXVUlUdTfoNO6Fpu2MYRovTVtTSpmb1yBcAWPHEcFY8MTw+F52PPgFWjXiukH56ZPwZpVcOf5aVw58FYPkjQzYpG5UBWPnkCFY+OcKlhz1dyA/S4fUi1XfViOfi80nthm2EYxr51lRGvjW1zN0wjNaHtGuXeNQa5n5lGPVFbrFreo9zE5/TfYY8UFMiXcXuV0H+lSKiIrJLlgvGL+ofGRJLW+GL+KKFgYRFguUDB8cv85cNeIhlAx5yaf9SH2Bp//vidPyyPyi79B8DC/nBAkDYRnS9pf3vi9uL6pfWi9pbevcAlt49ADCHe6N2qZcFhYrdr1T1PyKyF/Bl3IKCYRj1gNTeRJZExe5XqjpCRB4GfoUzA+miquV24zW11DCqS27VccbZPROf004P3dNg2yLSHfgj0B64Q1VvLMn/CXAxTmBaDFykqjOD/O2AScCjqvrDXIMgh/uViPQA5qrq641po5ydW2g/Ftq5RSpsqM6GKmqanVukMob2aqH6maSKFl0vqLf0ngcK+WXs3Gw3EaNmkXbJR0NVRNoDfwW+AnwGOFdEPlNSbDxOCDoUeBgoDeb+K5wXVJOQeXLz7lePAD/GzbzXAP+ToV4vERkrImP79etXrrhhGC2MdGifeJShKzBVVaer6gbgQeD0sICqPqeqa/3XMUDH+JoiRwC7AU/TRGQN7bcZMBR4SlX/ICKfBZ4Foo52BOYBXVV1QQNNmVpqGNUlt1o681u9Ep/TT93XL7VtETkL6K6qF/vv3waOSlMvReQvwAJV/bWItMOFCv02zna2S7OopUnuV6r6pqruqqqdVLUTLtTf4WUmNqCgahappSXpiFBNjOsF7ldFrlFpq6w+XbQiG7pZJbhqQYoaHLRbtOJaoVpqKqrRKmnXLvEItTB/hLtgJE18iZOkiHwL6ALc7E9dAgxT1dlNOYyK3a9UdVhQZga2oGAYrYHcktusCy9JfE4/eddtDUlunweuU9WT/fe+AKr6m5JyJwJ/Br6kqov8ufuBY3Hzyza4UKG3qWqfPOPI634VlemUpxOGYbQiKjMFeRXYX0T2BuYC5wDnFTUrchhwO059jXcRUtXzgzIX4ASlXBMbtID7VezK9OwoVj07Kj4Xnw9droJ05O4Ull057OnY9anIHSohHbpfRS5S0fk4HbplJbltpZSN2gvdr6Ye34Opx/dw6W6nxmUt8LPR2qnEiFdVNwI/BJ7CmXMMUtW3ROSX3rICnBq6DfCQiEwQkSEpzTXNOMz9yjDqitxq6ezeVyQ+p3v93y1tw/1KRDqLyBg/A48Vka7V765hGNWmXhzns/Q4cr/6NHA0cKk3zrsJuF5VO+Ps3UoN8hJZ8+9X3OeYsawZMzY+F56Py4bpKP+ll1nz0ssArH7xX6x+8V/u/OgxhbIJ6TWjxxSlI1aPeqmiekVt+D6Eu5uEqmbkbwrFKuqcS68EYF6f65jX5zoAZixZjmG0KG0thkJcoRD96kqgv6oOFJFzgdNU9byGa5taahhVJvcsNOeyqxOf045/+m1NzXB5ol/9GLhZRGYDvwP6ZmmjnqNfhWVD+7hwj7oiGzyfH9ruhXvGRdKqYTQnbUktBYrdr1R1JfAD4ApV3Qu4Amfom1TP3K8Mo5ZoJ8lHjVGR+5U/twLYQVXVezGsUNXtyjRlaqlhVJfcs9C8n/134nP6iZt/VVMzXJ7oV/OAL/n08cCULBdsK2ppUUSvwK4u3A49yRUttO0L1VKzgzOajRT3q1ojT/Srlbi9mzoA64BLVPW1Mtczyc0wqktu6Wp+3+sTn9M9fnNtTUlued2vjmja7hiG0eK0L7u9UU3Q7LJmW1RLw9XScDeReOPOoGwWtdRUVKOqtFU7t5yYWmoY1SX3LLTguhsTn9Pdr+tTUzNclgWFLUXkFRF53btfXe/P3y8ik0Vkooj09yuqhmHUOnViCpJFLV0PHK+qnwM6A91F5GjgfuAg4LPAVrjAD2Vp62ppaMQbbXJZqVpqu4kY1UDat088ao1GqaUisjUwGviBqr4cnL8C2EVVrynThKmlhlFdcotYC2+8JfE53a3PFXmjX30RuBU4FDhHVR8O8j4J3AHshZsnTlHVGTmGkW1BQUTai8gEYBEwomRi2wy39/nwtPqGYdQOIpJ4lKmTJfrVLOACYACbci9ws9+goyturslFpslNVT/0u390BLqKyCFB9m3AC6r6YlLdUvertqiWhqpmUVyIhFgQlaqlpp4aTUYFof3IFv1qhqq+QcFe1l3OTYIdVHWEL7c6iJJV+TAq2BXkWmCNqv7Opw8DzlTVj8pUBVNLDaPa5FZLF/3+L4nP6a4//WGTRL8SkbuBoZFaKiJn4N7ZbwD2Bp4B+qjqh3nGkWW19OMisoNPbwWcCLwtIhcDJwPnZpzYgLazoBDuChLu9FEUeSthV5Bwi/RKJTeT4ow8pC0oNFX0qwQ64ALEXAkcCeyDU19zUdZDAdgDuMfr1O1we6MPFZGNwEzg314fH6yqv8zbIcMwWpiU92uq2g9I29pnDm4xICKKZZyFOcB4VZ3uLi+P4TbGTdxpKCtmxGsY9UVutfS9v/498Tnd5dLvNaSWdgDewQVVnouLhnWeqr6VUPZuitXS9sA44ERVXSwidwFjVfWvecZh7lfNoJaGqmZk2waFxYU0FTbcyrwpxmQYmahgQSFL9CsROVJE5gBnA7eLyFu+7oc4lfRZEXkTN0H/PfcwTHIzjLoit+S25Pa7Ep/Tnb9/YU25KeRxvxIRuUFE3vGRsS6rfncNw6g67dsnHzVGHverC3AvEA/yhncPZrlgW1FLwxXQ1c+PjssmblbZBO5XjembYTRIW9wVJHS/Av6Me2E4tRHXM7XUMKpL7llo6d0DEp/TnS44r6ZmuDzuV/sC3/T2Lk+KyP7V7KhhGM1EnUhuedyvtgDWqWoX3MpG/6S6bdb96onhceyElUOfistG6mdUprRsuIPI2lcKu7Y3Zd9MRTUaQjq0TzxqjYrdr3DuEt1VdYYPIrNcVbcvU93UUsOoLrlFrGUPPJL4nO547tdrSnyr2P0KeAwX9QpcFKx3qtVJwzCaD2kniUetkUUt3QN4TkTewFkdj1DVocCNwNe90d1vsM0qi9XScLPKYLV02f2D4nQY+i9SS4tWS8eMrUrf0uoZBlA379zMiNcw6ovcs9DyR4YkPqc7fL1HTc1w5n5VJcktdKkq2mY82M8tcZvx0P2qSnZuWeoZbRdp3y7xqDWy7ApiGEZbogajyydhaqlh1Be5VceVQ59KfE63++rJ9amWekPe8SIy1H/fW0ReFpEpIjJQRDbP0k5bVEvTthmP8pc98EisorYWtdRU1DZMnYT2yyy5ichPgC7Adqr6VREZhNug8kER+T/gdVX9W5lmTHIzjOqSX3Ib/myy5Nb9hLzRr7bABYI5AlgCfNPbyW6Gi3x1OO5V2b2q+pu848jqftURONV3AG+0ezwQhea6Bzgjb2cMw2h5KllQyBj96rvAMlXdD7gF+K0/fzawhap+FjfxfV9EOuUdR1a19FbgKgpRa3bGeSRs9N/nAHtmaajNqKUPPR4HYA5VzXKbVa4c9nScv+alOIJii6mlYVmjjdCuXfLRMGWjX/nv9/j0w8AJXlBS4GN+N9+tcIFiVuYdRlm1VES+iguQeomIdMPtmHkh8G8/AyMiewHD/MxbWr8X0Avg9ttvP6JXr16lRQzDaDpyq6Wrnx+dOCls0+2YXNGvRGSiLzPHf58GHAWsAP6B26J8a+AKH68hF1lMQb4A9BCRU4Atge1wktwOItLBS2+pwSBKgkrYOzfDaO2keCOEgoqnXzAJZYl+lVamK/Ah8AlgR+BFEXkmChhTKWVlTVXtq6odVbUTcA4wUlXPB54DzvLFegKPZ7lgm1FLBw6Ow/gVxVAI3K+i/DAo88onR8T5q0e9VJW+NcV9M+qXtNB+qtpPVbsERyhdZYl+FZfxKuj2wFLgPGC4qn6gqouAl3CLl/nG0cjNKrsBV/rV0n1wevVOwHjgW6q6vkwTJrkZRnXJrZau+fcric/pxz7fNVf0KxG5FPisqvYWkXNwwdy/ISJXAwcBF+HU0leBc3x0+opplCmyqj6vql/16emq2lVV91PVszNMbECys/iKIcNYMWRYnI7LhumELblDiSfcvjspHb7gL7I1S0s3ol60SLDs/kGxZBZKPCPenBKXLScdTT2+R2LZon3gonsV3Iui/DDttzoPFy2K6vm/QdhuWC/cLr3ovgbjN+qMKkW/wsUh3VlEpgI/Afr4838FtgEm4ia2u/JObGDuV4ZhlFDp9kaqOgwYVnLuf4L0OpzZR2m91Unn82LuV4ZRX+RWS9eOHZ/4nG7d5bCaclOo2P0qOP9nEVmdtZ1oh4yVw56ObbpWPz863vMs3EFj1bOjCmnvwrRqxHOF9NMjYxuySHUKy0ZlwL3Uj17sh7ZkYTqsF+/S8eyouB9hfrhIEC0CrBz6VLyl+DMTp/LMRBc7pzEv7ad2OzXOX3rPA3E6vC/xluRp9y0cf8K9Wj3qpXixIhx/uIARtZdl/KHabdQ+0qFD4lFrNOad2+U4XTpGRLoAOzRpjwzDaFna0maV3v3qHuAG4Cd+tbQ98AxuGXeKqm6T4XqmlhpGdck9C62bOCnxOd3ykE/X1AxXqfsVuJWRIao6vzEXjFfsglW4ZQMeYtmAh4ryN0lHq6LBCmm4m0bo1rS0/31xOs4PrhHamhWlE1yjlva/L24vzI/aCttYevcAlt49AGgaW7JQRY1WbMP7Eq7OFuWHaW9LF96r0pXTpDGFK8TxvfdthWNuaHxGjdJWJLcU96tewCCgm6puFJHVaZKbuV8ZRrOSX3J7+51kye2gA2pqhssyuf0G+DawkYL71Xp/rPPFPglMj3xNG8DUUsOoLrknoPVTpiU+p1vsv29NTW6Vul/tqKq7q2onf35thokNSDHiLTHMLS0LycaoRSpTqIqFBraR2laizsZlA1WrnBHv0n8MTL5GGSPeStXSOZdeucn4w/tSdN+CcYT3LWkcqUa8oYoaxXcIjXgT1N0sYzJqjMp2BWl1VOx+VXI+VS0twSQ3w6guuaWrDdNnJD6nm+/TqaYkNzPiNYz6Iv/kNnN28uT2qb1qanJrdlkzUS0tSUeEq6V5fUuLVmTT/EkTViSL1OCg3aTdPcJrNIVaOr/v9ZuMHyjyw03yyS1S7cv4lkZGx5Csooa7m6T5ltpuIvVFvYT2M8nNMOqL3NLVB/MWJD6nm31i9/qU3BKiX50gIuNEZIKIjBaRTAsKkSvP6pEvJLv4BC5XSS5FofvVyidHFFyfQneohHSq+1WCG1VqvbRreOkndL+aenyPeIeP0F6tMVLOtEXLNhk/UOQOFd6X0vywn6Vlo3Rj3N2S7k801nhMXzylwTGZFFcD1En0qzzuV38DzlfVzsAA4BdN2THDMFoGkXaJR/l60l1EJovIVBHpk5C/hQ8DOtWHBe0U5PX15yeLSJP8AuZxv5oMfEdVXxaRvsC2qvrzMk2ZWmoY1SW3iLVq1arE53TbbbdtaLPK9rjNKr+M23H3VeBcVf1PUOYS4NBgs8qvqeo3fZSsB3DbjX8C59Z5gKp+mGccedyvLgaGicgcnJHvjUkVS1n7ymvxZ1q6tCzAmjFj3ee/X4mDFa8ZPYY1o8fE5+OyCenSenF+mA6iTSXWC/PDtG8j3G0jVMvC3T1CFXXOD38GuIWDaPFg+uKCKhqqcNH4w/uyZszY+HzqfUsaR5Z60f0Oyhbd1+C+hf2M3M+mdjs1Hms0zmisEaHabdQ8eaJfnQ48qKrrVfVdYKpvLxcVuV95yW0w8Fsvuf0MODCKfFNS39yvDKP5aCnJLU/0q+uAMap6nz9/J/Ckqj5MDiqKfiUi/wQOUtVIfBkIDE+qbNGvDKM+qGL0qyx1G01F7lc4MXJ7ETnAF/syJXu9pfHBnLnxZ1q6tGxavQ3TZ7Bh+gyXnjk7LrthxqxN0htmzCpKN1Q2bK+onr/WJmWjPgT9mXn+95h5/veAYlUzyb0s3LgzVPeijSg3GV/Ut5mzi9KNKRunZ83JXi9hzABTjzstTs9e5mLppu30EtrShZtjhoGrjdZJFaNfZanbaPJEv/oa8Evce7hlwEUZ4gya5GYY1aWl1NI80a8OxllcRAsKzwL7511QMCNew6gvck9uS9euS3xOd9p6ywbb9q+ubgXaA/1V9QYR+SUwVlWHiMiWuMjyh+EktnMigUhErsGF9tsI/FhVn8w7jmb3qfhgwUL3OW8BH8xbEJ+Lz/vPTdJR2bBeqKIG6tUHc+dvks6k+ob1orLh9RLaLSobXGPhyjUsXLkGgHlXXxuXTVplTVv1DdXEqA+pfQvzw3Q0/rnzC+mSe5hYL+l+J4wZYPHq9+P0rAsvAYpXjle/+K84P0yHKvj6KdMAd6/C+2U0P6rJR/l6OkxVD1DVfVX1Bn/uf1R1iE+v82FA9/NhQacHdW/w9Q5siokNTHIzjHojt+S2ePX7ic/px7fZqqbcFDKFtBGRGcAq4ENgo6p2EZGbgdOADcA04EJVXV6tjhqG0Tx89NFH5QvVAI1RS49T1c6q2sV/HwEcoqqH4l4k9s3SSNoqXLwi2YhVv/XT3mX9tHfd+WD1LlJxwvPhSmZUpzRdtBoalZ0yLW4vKT9sIywb7sIR+mFO73FunI4MXpf+Y2C8EWZRqL1AhUtaqS0af9qqb8L4U+93wviK6iWMGYo3/4x9awMj3iW337XJmKFk5dSvDBeFOQx8Vo3m48OPNPGoNbK6X80Auqjqeyn5XwPOUtXzyzRVe3fIMGqL3Krj3GXJq6V77pi+WtoaySq5KfC0iLzmDflKuQjI9BIwlBjy2muFklIorSWlQymnKD9FiovLZqkX9CFKh9tzh5LbkjvujdORZBJGpgp393j/9YmF8TdCqspSNpTikuqFEnE5iTfcBy+SPN+77U7eu+1OoMQ9K4hMFu4ysvbVcQAsvPEWFt54CwCLb/lrnG+7iTQfH+pHiUetkXVy+4KqHg58BbhURL4YZfgl3I3A/UkVRaSXiIwVkbH9+vVLKmIYRiviI9XEo9Zo9GqpiFwHrFbV34lIT6A3cIKqrs1QvfbukGHUFrlVx2mLliU+p/vuumN9qaUi8jER2TZKAycBE0WkO3A10CPjxAak2E9lsddKKLth1pzYvi3Ndi1JnS3n4hWm02zpilRmfz7sT9ouJEkbXs657GrmXHY1UKKKhrZ7CfZ4RbZrjRhHUTrBJi6sl+Ueh7Z70SJI6FIWqp8zv9O7UC/Y6WTd2+8AxW5b4Uaa4c4qpqJWl3pRS7PsCrIPEDlEdgAGeMvjqcAWwBKfN0ZVeye1EWCSm2FUl9zS1eQF7yU+pwfuvktNSW5l7dy8FfHnEs5n2lbcMIzaohbNPpJodverVatWxZ9p6dKyafU2LlzMxoWLXXpxwUolOhemw7KhW1dS2bC9omuklU24xrpJk1k3aTIA778Z+w4XuVetHTve5Y97nffHvQ7AgusKe36+s3BJnA7HF49/8XuFfobjT7gXpWWjdHiPE+tlGP+6iYUNYd6f8KYbZ+BSFq2EQvGGmLN7XxGn569YDTibuMguLgyCPePsnoV637sMgJnnXczM8zbZQtDIiaomHrWGuV8ZRn2RW3V8Y3Zy9KtD96qt6FcVu1/58z8CfogzBfmnql5VpX4ahtFMfNjW3a9E5DjcppWHqurBwO+yNNKUammeeg2VrbReWDZc1S1S/QJ1Lqne1IVLC/nBBo5N2bemvm/hymm58YfpGUsKrsiRce/S/vfF6dBtK3TxijYBDV21pn3lLKZ95SyM/FTDzk1EdhKRESIyxX/umFKupy8zxZuaRefPFZE3ReQNERkuIruUvWal7lciMgi3zfAzWQbnMbXUMKpLbtVx7LtzE5/TLnvvWXHbInITsFRVb/Rh/3ZU1atLyuwEjAW64OaK14AjcFrjPOAzqvqeb2utql7X0DXzuF8dABzr4w+OEpEjszSUFoi4XADfpq7XUNlK64Vllw96NN5SPHSpCiWQKH/5Q4/HLkyhTVhahKnWdt9C96sosPPygYNZPnDwJvnhluOhzV+00BI69Yd2d0muYaG9XrjwEUqHRuP58KOPEo+chJGv7gHOSChzMjBCVZeq6jLc5hzdcRO2AB/z0bK2I8M25JneueHcr+aJyK7ACBF529fdETgaOBIYJCL7aIkoWBL9iuMyXtAwjJbho+qYguymqvMBVHW+n0tK2ROYHXyfA+ypqh+IyA+AN4E1wBTg0nIXrNj9CjgRuFFVn/fnpwFHq+ri9NqmlhpGlcmtlr70zszE5/SYAzt9n/ToV4jIM8DuCVWvAe5R1R2CsstUtei9mw8RuoWq/tp//29gLfAnXHS9XsB04M/AgqhcGhW7XwGPAcf78wcAmwOJWyKFtBm1NNjPLYpyBcXRr6L8tL3fQremVq2WhtGt/FhDtbwo3++UAsULJrFamuIml7TTSZFammK7Z65ajSdNLS0T/QpVPVFVD0k4HgcWisgeAP5zUcKl06JgdfbtT/Oa4SDgv8qNI8s7t92A0SLyOvAKzuTE/b4/AAAVl0lEQVRjONAf2McHWn0Q6FmqkhqGUXtUaVeQIUC0+tkTeDyhzFPASSKyo19NPcmfmwt8RkQ+7stlCiVqRryGUV/kVktHvjU18Tk9/uD98qyW7oyTuD4JzALOVtWlItIF6B1Eqr8I+LmvdoOq3uXP9wYuBz4AZgIXqOoSGqDZ3a/aoloarSBC8Tbb0SpiPaml0U4eS+95IN7Jo1FqabhaGuxYkrR1fJbV0rQ+G+lUY5txVV2iqieo6v7+c6k/Pzaa2Pz3/j461n7RxObP/5+qflpVD1XV08pNbGCSm2HUGzXlIlVNMkluIrKDiDwsIm+LyCQR+XxWi2PDMIyWIKta+kdguKoehNv+aBLQB3hWVfcHnvXfy9Im1dLAiDcybAVYdv+gTcrWuloajTVcLY02n4RGrpYGammlRrzlxmTUL1k2q9wOeB0oMtAVkclAN2+QtwfwvKoeWOZ6ppYaRnUxtdSTRXLbB1gM3CUi40XkDm/vVmRxDCRZHBuGYbQIWSa3DsDhwN9U9TCc+0MmFRQ2jX7VZtTS0F80UDWLfEt9fhjar+bVUq9qFvmWBvnh+MuqpSm+pXFYwiC+Q6Vqqamn9UsWtXR3XHyETv77sbjJbT9MLTWM1oappZ6ykpuqLgBmi0g0cZ0A/IdsFseb0GYkt+CFerjTR7SIAIXFhaKyPmIU1OauIJHkGUaxKnI5SwjgDAUptcjOLcX9qikXFNLGZNQ+WXcF+RFwv4hsjnNcvRA3MQ4Ske/iLY6r00XDMIzGY0a8hlFfmFrqMferFlRLo5fraWVXj3qpKn1rcrU0UDtjtfT+QQU7vsC2L6ksFFTwNLU0UkXDdGN3Ban0f8GoTUxyM4z6wiQ3T7NLboZhGM1Bxb6lQd6VIqJZotFAwaUmjI5Umi4tm1YvtHMKVZikdKjCpJWN2grPh9cI84vS3i4rtNGa3fuKOOjw9MXL4rKh+9HKoU+5z+HPxipaGLQ5XC2tdExpZbPei7Bs0piBoqhTs5etBIpV7RVDhhXGnLIaHO2cEqqM839R2Gh1ardT4/Sim/4EFNsHLrnjXpbccS9QiKSV9V40VLY0bdQOWaNf3QO8qKp3+BXTrVV1uYjsBdwBHAQcEUbHSsHUUsOoLqaWerJsM74d8EXgTgBV3aCqUcDJW4CrsEnLMIxWRsW+pSLSA5irqq83VLnU/SoytPxgwUI+WLAQKDbADIMWJ6XDskXqbOCqU06dLVcWAhUmi8rs2wvVnYUr17Bw5RoA5v/3DXHZIrXTp9f8+5VCeszYOD9U/RKvF9zDxqjzqffCt9VQvdI+ACxe/X6cnv29ywBY/eK/WP3iv9yYAvWzKB24l0WbUc7rcx3z+lwHFK9YTuv+9cL1bvmru8bIF+LNMcPV2TCYc977Fqbn972e+X2vx6gNsrhfdQHG4ML7vSwifwQ24KS5k1R1RVLQ5hRMwjOM6mJqqSeL5DYHmKOq0c/swzhH+r2B1/3E1hEY5/1QDcMwWpxKfUvHqequqtrJO9TPAQ73ZRuk0tW7pLKhwWeowhXtIJGwkllUtkx6w/QZhV0ownx/3TAdll3x2NA4zF24WeWMs3vG6aX/GAhk2xUksW9p408qWzL+OB2ugIZjKnffgntcZJjrV0Onnng6U088HSBexYRiI+ZotRgKBsvhammoiiYZ2IZxGmZ840JmfONC15/AaLhRf9My6bS+Ga2TrKulnXGrorFvqQ93H+XPwNRSw2gNmFrqyWTnpqoTfBDWQ1X1jHBi8/mdMkxsQIrkNnd+4WVv+II7TCdJbll+gQPJJpR4Giobnk+tF6ajPgT9WTn0qVgyKYp+5SUNgOmnnQMUbzMeulyte+vt5H5WII1mktwac9+CdBh0OpI8Q7uzUOoKJbdQon1/nFuXWnTTn2I7tsW3/i3OL+dGFdoVzur5g8R+5pXiw74t+v1fEvtmtB7MQ8EwjLrEfEsNo74wtdSTJ7RfZxEZIyITvB1b1yxtJdmrNYmdWzkbtFD1bYRtU6pNWIKtXHiNta+OY+2r44Bi27ZQFYvSc3/6C+b+9BcArJs4qdBuuKCSwXatwb6l2bkl2LaljSntGqFtXmTHFrqUhYsksy68JE6vfeW1OL1+8lSgeJPLyIYNitX5JLV05nkXM/O8i4vyU8fUiPsWpkNbuvDvGLp7mYraeqjY/QoYBNyiqk+KyCnAVararUxTJrkZRnUxyc1TdifewP3qAnDuV8AGEVFgO19se2BelfpoGIbRaPKE9vsxcLOIzAZ+B/TNcsFoI8FVq1alpkvLVqNeQ2UrrReWXT/t3XhTxXWTJsdlo1VBKKig6yZOitPRahzAhFkFNakp+9bU923d2+8UxvfmW4BTOSO18/0Jbxbyg/Scy66O0zOWOHflot09vB0gwMxv9SrUu/RKd+47vZn5nd5AsYr67tfOzz2mpPSS2++KXbuW3j2g0PdzLir07ZKfunOB3Z3RMlTqfrUSJ62NUtVHROQbQC9VPTGhfi+gF8Dtt99+RK9evUqLGIbRdJha6skT2u8YYAdVVRERYIWqbpfeEmDv3Ayj2tjk5skT2m8e8CV/7nhgSpYLLl27Lv5MS5eWDdNL1qxjyRqXbs1qadqe/knpcPxvzC6s0oUGva1ZLQ1XFhtT7+35hdXwSM1b2v++ePUxdNsKAzvPuuiHQIkqeua3effMb8fn844pUS3tdzdL+t3t0sHOI6G716zv/miTvk3vcS7Te5yL0bxU7H4FHAz8EbcosQ64RFVfS23EYZKbYVQXk9w8FbtfqepoVT1CVT+nqkdlmNiAthP9asXgJ+ItxUN7rTAocZSf5mQf2se15uhXoftVNNbQpSzMD9OhW1pkK1fklhfY4CW5hoX2amnSWFP+L4TXC20wkzZ4KA0SHUrpRvNg7leGYdQl5n5lGPWFqaWeLDEUDvQuVtGxUkR+LCI3e3esN0TkURHZIcsF24xaOmRYHPVpxRPD47Lhi/E4PyhbpJYGbk2tWi0NxhfthBKq5WH0q7BsqK5HNnFFammwUJG0S0dLqqVFbmsVqqXmqlVdsqyWTlbVzqraGTgCWAs8CowADlHVQ4F3yGjEaxiG0Rw0Si0VkZOAa1X1CyXnvwacparnJ9eMMbXUMKqLqaWexi4onAM8kHD+IuDJLA20GbU0WAEN1a9ws8YoP1UtrZXV0kDtXP38aDfOYHePotXSQC0NV0sT1dJwtTRh08lWuVrqdxZprFpqKmrTk1ly87uBzAMOVtWFwflrgC7AmZrQmLlfGUazYpKbpzGT2+nApap6UnCuJ9AbOEFV12ZoxtRSw6guNrl5GqOWnkugkopId+BqoEfGiQ0wtTQ04o3cdmp+tTRUSxOMeMPoWBWrpa1stbRaamnYNyMfWd2vtgZmA/uo6gp/biqwBbDEFxujqr3LNGWSm2FUF5PcPGbEaxj1hU1unmZ3v2qLammoahb5lvr85YMejVW3WldLI1Uzzbc0NGIua8Rbi2pphUa85fpmNB6T3AyjvjDJzVOx+5XP+5GITBaRt0TkpiwXbDOSW+B+tOrpkXHZ8OV6KNlE0k0YKapm7NzCnT685BlKo9F9SCsLBSl1w6w5caDo1ux+1dySm0lxjadsgBhVnQx0BhCR9sBc4FEROQ44HThUVdeLyK5V7alhGEYjqNj9SkQGAf1U9ZlGXM/UUsOoLqaWevK4Xx0AHCsiL4vIKBE5MksDbV4tDbakjtXSsGyoqr30clX61txqadFOKGlqqVfBa2VXkErV0miL/Kbqm5FO5snNu1/1AB7ypzoAOwJHAz8DBvlAMaX1evmI9GP79evXBF02DMMoT8XuVyIyHLhRVZ/336cBR6vq4vRWTC01jCpjaqmnYvcr4DFc1CtE5ABc8Jj3yjXSVtTSxbf+jcW3/g0o3gkkDLq86Pd/2aTs0nsKtzh0VWrNamnU93Csi276UzzWaJylZcOgyyuHPQ24AM9RkOcN02fE+VHQaigEfg4DX5eqqHnHlJReP2Ua66dMc30LdikJg26ve+vtOD8q09SrpaaiZiOP+9XmQH/cSuoG4EpVHZneCmCSm2FUG5PcPFmjX61V1Z2jic2f26Cq31LVQ1T18AwTG4CIyPdxf4DUo1yZvPmtpQ27Rv31sxVcw4hQ1WY9gLF5y+TNby1t2DXqr5+t5Rp2qIX2MwyjPrHJzTCMuqQlJrcsxm7lyuTNby1t2DWat422dI02T3PvCmIYhtEsmFpqGEZdYpObYRh1Sdktj/IiIgfhtkbaE2fEOw8YoqqTfH5XQFX1VRH5DNAdeFtVh6W1aRiGUY6qvnMTkatxblsPAnP86Y643UUexAWY+Qpukh0BHAU8D5wIPKWqN4jIUcAkVV0pIlsBfYDDgf8A/6uqK0RkX+BrwF7ARmAK8IAGRsetERHZVVUXtXQ/6oXWcD9FZGdVXVK+pFF1qmlEB7wDbJZwfnPcBPQm0B7YGlgJbOfztwLe8Om3gA4+3Q+4FTgGuBYYDFyGmxh/AfwLuA24ATf5dWtpQ0Lf7yeBnUqOnYEZuJ1VdgLG+THs20A7HYDvA8OBN4DXfdu9gc38fbwKt0vLlsAFwBDgJmAb4IfALr6t/YAXgOXAy8Bn/fl2wEXAP337r+F+iLr5/Pa+D78CvlDSv1/gNi+Nvm/mzw0B/hfY2p/fB+e692vfr78DE3E7znQqdy/9Z677CWwP3Ai8jYvgtgSY5M/t4MvsDvwN+Ktv/zr/PzsI2MOXje5nF2A6MBWYCXzJj+2XuP/hFcBiYAxwQdCP7YDfAP8Azivp421A95I+3+n/9gOA3Vr6f7s1H9WW3N4GTlbVmSXnPwU8DaxV1cP8ufFR2n+foKqdRWSSqn7anxunqoeHZXAPW2dV/dD7wA5T1W4i8kngcVU9TES2A/ripMYnVXVA0MZtODV5uP++PfAH4EjcA3eFqi4UkS7AzbidiPviHs6uuAm8F+muLwIMBXbD/dOHdMRJtOrLPQJ8A1iA26RgoKrOC/r6AG4yuodiSbgn7oEWnA/wVsCBuId1EHAa7kE9XFUP9m39E7hDVR8VkW7ADeo2Ib3L9/MZ4Czcj86LuBi1jwOfw02irwDfBkap6k+ivw9A9DcSkd/jJoW7gDOAnVX1OyLygh/f9sC3fP4g4CTgfODKhu6lqu4hIh/luZ8i8hQwErhHVRf4c7v7e3miqn7Z73zzT+BjwHnA/b6d03HaxT6q+llf9zngKnWvVw7ATT5zgUf9vfyGb+dB3KQ7V1V/LiKP4H7ox+B+VD7ATXLrE+7nHX4sfwfOBL6kqmek3CujmjMn7v3ZVJx00c8fw/257jiJIfo1b1fyCzXOpx8CLvTpu4AuPn0A8Crul3QLf25H4LWgnYn+8xHcr+wZOCnikaDOuOha/vsdOIniU8AVwGP+/Cs4Ffpc3ARylj9/AvBv4EPcw/JcwvE+7oEdjpeQfN13g3TYh2Nxv9oLfP1e/vzkBu71O8AEnxZfV4Lvb4T1gVdL6r8Rfgbnx/jPLXCT5RtBXgf/Nx3s88cD44P8CXjJPeqDT4dlZpVcb3y5e+nL5bqfZe7l5Az9nICT+iKtYkxJ/pvA6yXnXo3+13HvlYn+ZkGZa4CXcD8Kpf+bpWUnpI3BDq2+b6n/Qx4NfB0nCRwNtPd5W6TU2YWCmrQ9cDcwDTcZfoAT/0fhpIjL/YPbz/+zRRPhx4EXmuofKMMDORHYP2U8s/1nR9xk/QdgW2B6UGZcQr32uB+Bu/z3McDZFP8QtAO+6e/NhOB8/5K2Xsep63fj1MKfAz8GPglciJOIwKmh+/r04dE99N//Ez2UJW1f6+/nFP+3+Zr/e08q7UNwjQNw0vF7FH6w9vN/y7L3Mu/9xGkOVxGodjjp+mrgmbC/Pv3rkrbeAH7k2zkep7LeCnwRuB6nZv4LOMaXPw33HjmqH02gk8K/pz/XE6fKzsRJoj8BfurvrYR9qPbzW8tHi3cgc0fdP+/ngCMoedcAHIybOA9KqZv7HwgnnZ2Em1xmAmf4818CxvrrH5hy/TNKvp+Gm6gWBOcezHAPOgEDgUU4Se0dnx4I7I2TOrdJqLcvMNqnL8BNhO8Bq/ALM8D2Pv94YJZv+13gKH/+47h3d/cRvAcKrnEx7ofnbj95RMduPn934FmfPgGY7P8ux+Ak6Sl+LGc05l5Wej9xUv5vcT+Iy4Clvj+/BXbyZX6Zcj/3Ax726W7+/o/HSWvDcJLhZsChOIl/OTAaOCC4l5f59E04Nbj0Gt39Pbm25Ph4cD/vbennsjUfLd6BZhlkE/wD4SbWp3Aq9kHAH/0/7VvAf/kyB/kHd5vS65Tm496LHZKWn1Tfp4/Cvevb2U8MVwKnBPldgSN9+jO4SftUCipqmH8wbkI/peR6n09o45Ske+vLNPiQZXkIce8l26XkHeP7cFID9Y/FvctKLFPahr+P0YS+NW4iG4qb3LYPyoSLXNcDT0RlSvI3aSPhGkX1/fnLgL0aGFeD+XakH23e/UpELlTVuyrNj8rgJMtLcb/+nYHLVfVxnz8OJ9E0lH8XbjUzMV9VDxeRayk2nemKU89PxE28HWjAtCYhv6i+OtObctc4qnT4wHG4d2RpHB/lq2oPERnSUBlgd1Xt6sf+PX/fHsVJzk+o6o0i8kpJmUtwu0OfhJtAzmyoDdxiyOdUdaOI9APW4CTIE/z5M0XkrZIya4GHozK4RZvUNhLyi+r7a6zw9abhFise0mCr/nL5RgO09Oza0gcl788amx+Vwakk2/jvnXCq6uX++/i8+f6zQdOZvPkZrzEep5p2w6nk3YD5Pv2lcvnBeBtsI7i3r1KQpD8GvBnek7QyGfInBfnjSv6e0XvWBsvkzQ/uRTvcpHsnzlxkOO61ybbl8lv6+WnNR9U9FFoDIvJGWhawW7n8LG0Aq1V1NYCqzvDmFQ97sxfBLaLkyQfYqKofAmtFZJqqrvTl3/emEZozP8s1jsAt4lwD/ExVJ4jI+6o6yt+nBvM95dpoJyI74h5qUS+pqOoaEdno2yhXplz+24FU/rqIdFHVsd6M4wN/jYllykzJme+7pB/hFiaeFpHNKKzK/w63gNJQ/scxkmnp2bU5DmAhTtX7VMnRCecO1mB+xjZG4uztwut2AO6lYNpQcb7/3qDpTN78LNcIvkcrlX8hQbotl99QGZwx7nTcgsZ0nJoK7l3lhCxlMuQ3uAofjDm1TN58f43xSffG521VLr+ln63WfLR4B5plkE6cPyYlb0C5/IxtdIweoIT8L+TN958Nms7kzc9yjYTzp+Lc4NLufYP5Wcv4clsDe+cpU5pPA6vwWcvkycevoDbQ3wbz7Ug/2vyCgmEY9YlteWQYRl1ik5thGHWJTW6GYdQlNrkZhlGX2ORmGEZd8v9u/xR2eZDVCAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "#Developing correlation plot\n",
    "\n",
    "# Generate a mask for the upper triangle\n",
    "mask = np.zeros_like(corr, dtype=np.bool)\n",
    "mask[np.triu_indices_from(mask)] = True\n",
    "\n",
    "# Generate a custom diverging colormap\n",
    "cmap = sns.diverging_palette(220, 10, as_cmap=True)\n",
    "\n",
    "# Draw the heatmap with the mask and correct aspect ratio\n",
    "sns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,\n",
    "            square=True, linewidths=.5, cbar_kws={\"shrink\": .5})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = Dataset_2[77]\n",
    "X_org = Dataset_2.drop([77], axis = 1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.19739600730596205"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.sum()/y.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Above value shows that this dataset is biased. Which means accuracy may not a be good evaluation function. For the rest of this project we consider ``AUC`` as a scoring function.** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Taking Random sample with 10% data and pre-processing the data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "_,sample_data, _, sample_target = train_test_split(X_org, y, shuffle = True, test_size = 0.1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_auc_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_unscaled , X_text_unscaled, y_train, y_test = train_test_split(sample_data, sample_target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "scaler = MinMaxScaler()\n",
    "X_train = scaler.fit_transform(X_train_unscaled)\n",
    "X_test = scaler.transform(X_text_unscaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MinMaxScaler rescales the data set such that all feature values are in the range [0,1] . So that it is standardized"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9937240816296908"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "knn = KNeighborsClassifier()\n",
    "param_grid = {'n_neighbors':[1, 5, 10, 15, 20]}\n",
    "\n",
    "grid_knn = GridSearchCV(knn, param_grid=param_grid, cv = 5, scoring='roc_auc')\n",
    "grid_knn.fit(X_train, y_train)\n",
    "grid_knn.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9881345506345507"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_knn.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n_neighbors': 20}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_knn.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.92260221, 0.9702089 , 0.97984543, 0.98323689, 0.9854525 ])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_knn.cv_results_['mean_test_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_knn_predict = grid_knn.predict(X_test)\n",
    "y_knn_train_predict = grid_knn.predict(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train roc_auc_score: 0.96\n",
      "Test roc_auc_score: 0.96 \n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import roc_auc_score\n",
    "print('Train roc_auc_score: %.2f'%roc_auc_score(y_knn_train_predict, y_train))\n",
    "print('Test roc_auc_score: %.2f '%roc_auc_score(y_knn_predict, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_score_list = []\n",
    "test_score_list = []\n",
    "x_range = [1, 5, 10, 15, 20]\n",
    "\n",
    "for alpha in x_range: \n",
    "    model = KNeighborsClassifier(n_neighbors=alpha)\n",
    "    model.fit(X_train,y_train)\n",
    "    train_score_list.append(model.score(X_train,y_train))\n",
    "    test_score_list.append(model.score(X_test, y_test))\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "report_table = [['knn', 'k = 20', grid_knn.score(X_train, y_train), grid_knn.score(X_test, y_test), roc_auc_score(y_knn_train_predict, y_train), roc_auc_score(y_knn_predict, y_test) ]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Linear SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "svc_lin = LinearSVC()\n",
    "param_grid = {'C':[0.001, 0.01, 0.1, 1, 10, 100]}\n",
    "\n",
    "grid_svc_lin = GridSearchCV(svc_lin, param_grid, cv = 5, scoring='roc_auc', return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True,\n",
       "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
       "     verbose=0),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_svc_lin.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 10}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_svc_lin.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.97743359, 0.99015694, 0.99059817, 0.99162916, 0.9919385 ,\n",
       "       0.99173187])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_svc_lin.cv_results_['mean_test_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_svc_lin_predict_train = grid_svc_lin.predict(X_train)\n",
    "y_svc_lin_predict = grid_svc_lin.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_score_list = []\n",
    "test_score_list = []\n",
    "x_range = [0.001, 0.01, 0.1, 1, 10, 100]\n",
    "\n",
    "for alpha in x_range: \n",
    "    model = LinearSVC(C=alpha)\n",
    "    model.fit(X_train,y_train)\n",
    "    train_score_list.append(model.score(X_train,y_train))\n",
    "    test_score_list.append(model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "report_table = report_table + [['LinearSVC', 'C = 10', grid_svc_lin.score(X_train, y_train), grid_svc_lin.score(X_test, y_test), roc_auc_score(y_svc_lin_predict_train, y_train), roc_auc_score(y_svc_lin_predict, y_test)]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "param_grid = {'C': [0.001, 0.01, 0.1, 1, 10, 100] }\n",
    "clf = GridSearchCV(LogisticRegression(penalty='l2'), param_grid)\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 100}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.80346821, 0.83170662, 0.90997821, 0.95195679, 0.96238037,\n",
       "       0.96550744])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.cv_results_['mean_test_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_log_predict_train = clf.predict(X_train)\n",
    "y_log_predict = clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_score_list = []\n",
    "test_score_list = []\n",
    "x_range = [0.001, 0.01, 0.1, 1, 10, 100]\n",
    "\n",
    "for alpha in x_range: \n",
    "    model = LogisticRegression(C=alpha)\n",
    "    model.fit(X_train,y_train)\n",
    "    train_score_list.append(model.score(X_train,y_train))\n",
    "    test_score_list.append(model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "report_table = report_table + [['Logistic Regression', 'C = 100', clf.score(X_train, y_train), clf.score(X_test, y_test), roc_auc_score(y_log_predict_train, y_train), roc_auc_score(y_log_predict, y_test)]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kernalized SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    " from sklearn.svm import SVC\n",
    "kernel = SVC()\n",
    "param_grid = {'C':[0.001, 0.01, 0.1, 1, 10, 100]}\n",
    "\n",
    "grid_kernel = GridSearchCV(kernel, param_grid, cv = 5, scoring='roc_auc', return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_kernel.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 100}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_kernel.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.98978397, 0.98976922, 0.98974509, 0.98976034, 0.98996376,\n",
       "       0.99061965])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid_kernel.cv_results_['mean_test_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_ker_predict_train = grid_kernel.predict(X_train)\n",
    "y_ker_predict = grid_kernel.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_score_list = []\n",
    "test_score_list = []\n",
    "x_range = [0.001, 0.01, 0.1, 1, 10, 100]\n",
    "\n",
    "for alpha in x_range: \n",
    "    model = SVC(C=alpha)\n",
    "    model.fit(X_train,y_train)\n",
    "    train_score_list.append(model.score(X_train,y_train))\n",
    "    test_score_list.append(model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "report_table = report_table + [['Kernalized SVM', 'C = 100', grid_kernel.score(X_train, y_train), grid_kernel.score(X_test, y_test), roc_auc_score(y_ker_predict_train, y_train), roc_auc_score(y_ker_predict, y_test)]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "param_grid = {'max_depth': np.arange(3, 10)}\n",
    "\n",
    "tree = GridSearchCV(DecisionTreeClassifier(), param_grid,cv = 5, scoring='roc_auc', return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best'),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'max_depth': array([3, 4, 5, 6, 7, 8, 9])},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score=True,\n",
       "       scoring='roc_auc', verbose=0)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': 4}"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.98354444, 0.98459662, 0.97770307, 0.9664924 , 0.95247299,\n",
       "       0.93652334, 0.93683623])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.cv_results_['mean_test_score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_tree_predict_train = tree.predict(X_train)\n",
    "y_tree_predict = tree.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_score_list = []\n",
    "test_score_list = []\n",
    "x_range = np.arange(3, 10)\n",
    "\n",
    "for alpha in x_range: \n",
    "    model = DecisionTreeClassifier(max_depth=alpha)\n",
    "    model.fit(X_train,y_train)\n",
    "    train_score_list.append(model.score(X_train,y_train))\n",
    "    test_score_list.append(model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "report_table = report_table + [['Decision Tree', 'd = 4', tree.score(X_train, y_train), tree.score(X_test, y_test), roc_auc_score(y_tree_predict_train, y_train), roc_auc_score(y_tree_predict, y_test)]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "report = pd.DataFrame(report_table,columns = ['Model name', 'Model parameter', 'Train accuracy', 'Test accuracy', 'Train auc score', 'Test auc score'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "report.index = report['Model name']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Model name</th>\n",
       "      <th>Model parameter</th>\n",
       "      <th>Train accuracy</th>\n",
       "      <th>Test accuracy</th>\n",
       "      <th>Train auc score</th>\n",
       "      <th>Test auc score</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Model name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>knn</th>\n",
       "      <td>knn</td>\n",
       "      <td>k = 20</td>\n",
       "      <td>0.993724</td>\n",
       "      <td>0.988135</td>\n",
       "      <td>0.956491</td>\n",
       "      <td>0.960654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>LinearSVC</th>\n",
       "      <td>LinearSVC</td>\n",
       "      <td>C = 10</td>\n",
       "      <td>0.992865</td>\n",
       "      <td>0.993405</td>\n",
       "      <td>0.959129</td>\n",
       "      <td>0.966063</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Logistic Regression</th>\n",
       "      <td>Logistic Regression</td>\n",
       "      <td>C = 100</td>\n",
       "      <td>0.967308</td>\n",
       "      <td>0.968732</td>\n",
       "      <td>0.958571</td>\n",
       "      <td>0.963571</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Kernalized SVM</th>\n",
       "      <td>Kernalized SVM</td>\n",
       "      <td>C = 100</td>\n",
       "      <td>0.991152</td>\n",
       "      <td>0.991533</td>\n",
       "      <td>0.957230</td>\n",
       "      <td>0.957388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Decision Tree</th>\n",
       "      <td>Decision Tree</td>\n",
       "      <td>d = 4</td>\n",
       "      <td>0.988685</td>\n",
       "      <td>0.984661</td>\n",
       "      <td>0.956116</td>\n",
       "      <td>0.950319</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              Model name Model parameter  Train accuracy  \\\n",
       "Model name                                                                 \n",
       "knn                                  knn          k = 20        0.993724   \n",
       "LinearSVC                      LinearSVC          C = 10        0.992865   \n",
       "Logistic Regression  Logistic Regression         C = 100        0.967308   \n",
       "Kernalized SVM            Kernalized SVM         C = 100        0.991152   \n",
       "Decision Tree              Decision Tree           d = 4        0.988685   \n",
       "\n",
       "                     Test accuracy  Train auc score  Test auc score  \n",
       "Model name                                                           \n",
       "knn                       0.988135         0.956491        0.960654  \n",
       "LinearSVC                 0.993405         0.959129        0.966063  \n",
       "Logistic Regression       0.968732         0.958571        0.963571  \n",
       "Kernalized SVM            0.991533         0.957230        0.957388  \n",
       "Decision Tree             0.984661         0.956116        0.950319  "
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAALHCAYAAAB8P9+dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xu4neOB///PLdshjWNEKYrWoU1FEgQNymgR7UwHNU6laR1bHQatabUo1fZXbb+m37aUMY49oGZaPYyqQ9V3qCrRSAiDotFUFYmkThFJ7t8fe9u2O5tsctiJvl7XtS97Pet+nnWvlXXx9uRezyq11gAAAC9apr8nAAAASxqRDAAADZEMAAANkQwAAA2RDAAADZEMAAANkQzAIlFKOaCUcnV/zwPgtSiukwws7Uop1ycZkWStWutz/TwdAF4HnEkGlmqllA2SvCtJTfKPi/mxOxbn4y1NvDbA0k4kA0u7sUluTnJhkg/3vKOUMrCUcnopZXIpZUYp5cZSysCu+7YvpdxUSpleSvljKeUjXduvL6Uc2uMYHyml3Njjdi2l/HMp5b4k93Vt+0bXMf5aSrmtlPKuHuMHlFI+W0q5v5TyZNf9by6lnFlKOb2Z789KKcf09iRLKZuWUq4ppUwrpfyllPLZru3Ll1L+bynl4a6f/1tKWb7rvr8rpUwppXyqlPJoKeXPpZQ9SinvK6Xc23Wsz/Z4jFNKKf9VSvlB11x/V0oZ0eP+43s8j7tKKXs2r9OvSylfL6VMS3JKz9eudPp61zxmlFImllKGdd23SinlO6WUx7r+rE4spSzT8/UvpfyfUsoTpZQHSynvne+7AmABiWRgaTc2yfe7fsaUUtbscd//SbJlkm2TDE7yqSRzSynrJbkyybeSrJFkZJLbX8Vj7pFkmyTv6Lp9a9cxBie5OMl/llJW6LrvE0n2T/K+JCsnOTjJM0kuSrJ/jxgckuQ9SS5pH6yUslKSa5P8IsnaSTZK8suuu09I8s6uxx+RZOskJ/bYfa0kKyRZJ8nnkvxHkgO7Xpd3JflcKeWtPcbvnuQ/ezyXH5dSlu267/6ufVZJ8vkk3yulvKnHvtskeSDJG5N8qXkauybZIckmSVZNsm+SqV33favrmG9NsmM6/0wPao57T5IhSb6a5LxSSmlfJ4CFSSQDS61SyvZJ1k9yWa31tnRG3Ae77lsmnUF6dK31T7XWObXWm7rWLB+Q5Npa6yW11udrrVNrra8mkr9ca51Wa302SWqt3+s6xuxa6+lJlk/ytq6xhyY5sdZ6T+00oWvsLUlmpDOMk2S/JNfXWv/Sy+P9Q5JHaq2n11pn1lqfrLX+tuu+A5KcWmt9tNb6WDrj9UM99n0+yZdqrc8nuTSdofmNrmNMSjIpyfAe42+rtf5X1/h/S2dgv7Pref5nrfXhWuvcWusP0nkmfese+z5ca/1W1+vwbPMcnk+yUpK3p/PzMHfXWv9cShmQzmD+TNec/pDk9OY5TK61/ketdU46/+fiTUnWDMAiJJKBpdmHk1xda3286/bFeXHJxZB0Bt79vez35pfZ3ld/7HmjlPLJUsrdXcsIpqfzrOiQPjzWRek8q5uuf373Zca90jHWTjK5x+3JXdteMLUrLpPkhXDtGeLPJlmxx+3u51ZrnZtkygvHK6WMLaXc3rVEZXqSYXnxeb5k31at9bokZyQ5M8lfSinnlFJW7tp/uV6ewzo9bj/S4zjPdP3ac84AC51IBpZKXWuL90myYynlkVLKI0mOTTKiax3t40lmJtmwl93/+DLbk+TpJG/ocXutXsZ0Xxaoa/3xp7vmslqtddV0niF+YTnAKz3W95Ls3jXfoUl+/DLjXukYD6fzbPoL1uva9lq9+YVfus7Gr5vk4VLK+ulcqnFkktW7nuedefF5Jj1el97UWr9Za90yyabpXHbxr+n8c3q+l+fwpwV4DgALTCQDS6s9ksxJ57rgkV0/Q5PckGRs11nQ85P8Wyll7a4P0I3u+lDb95PsXErZp5TSUUpZvZQysuu4tyf5QCnlDaWUjZIcMp95rJRkdpLHknSUUj6XzrXHLzg3yRdKKRt3fXhteCll9SSptU5J53rm7yb5YS9LFF7w30nWKqUc0/VBvZVKKdt03XdJkhNLKWt0rWv+XDrj+7XaspTygdJ5dYpjkjyXzg9GDkpnBD+WJKWUg9J5JrlPSilblVK26Vrf/HQ6/wdmTtdZ7suSfKnrea2fznXcC/IcABaYSAaWVh9OckGt9aFa6yMv/KTzr/QP6Iq845Lckc4QnZbkK0mWqbU+lM4P0n2ya/vt6fzQW5J8PcmsdC5JuCidQf1KrkrnhwDvTecygZl56bKDf0tnBF6d5K9JzksysMf9FyXZLC+/1CK11ieT7JLk/elcenBfkp267v5iknFJJnY91991bXutfpLONcJPpHNd8Ae61m3flc61wr9J52uzWZJfv4rjrpzOM9FPpPN1mprOD1YmyVHpDOcHktyYzmUz5y/AcwBYYL5MBKAflVJ2SOdZ0w26zn7351xOSbJRrfXA+Y0FeL1zJhmgn3QtPTg6ybn9HcgAvJRIBugHpZShSaan83Jm/7efpwNAw3ILAABoOJMMAAANkQwAAI2O/p5AkgwZMqRusMEG/T0NAABe52677bbHa61rzG/cEhHJG2ywQcaNG9ff0wAA4HWulDK5L+MstwAAgIZIBgCAhkgGAIDGErEmGQBgSfP8889nypQpmTlzZn9PhddghRVWyLrrrptll132Ne0vkgEAejFlypSstNJK2WCDDVJK6e/p8CrUWjN16tRMmTIlb3nLW17TMSy3AADoxcyZM7P66qsL5KVQKSWrr776Av0tgEgGAHgZAnnptaB/diIZAGAJNHXq1IwcOTIjR47MWmutlXXWWaf79qxZs/p0jIMOOij33HPPIp7p65M1yQAAfbDB8Vcs1OP94bS/f8X7V1999dx+++1JklNOOSUrrrhijjvuuJeMqbWm1ppllun9vOcFF1ywcCa7CMyZMycDBgzo72m8LGeSAQCWIr///e8zbNiwfOxjH8sWW2yRP//5zzn88MMzatSobLrppjn11FO7x26//fa5/fbbM3v27Ky66qo5/vjjM2LEiIwePTqPPvroPMe++eabM3r06Gy++ebZbrvtct999yVJZs+enWOPPTbDhg3L8OHD8+1vfztJ8tvf/jajR4/OiBEjss022+SZZ57Jueeem2OOOab7mLvttltuvPHG7jmceOKJ2XrrrXPLLbfk5JNPzlZbbdX9fGqtSZJ777037373uzNixIhsscUW+cMf/pD9998/V1zx4v+o7Lvvvvn5z3++SF7jRCQDACx17rrrrhxyyCEZP3581llnnZx22mkZN25cJkyYkGuuuSZ33XXXPPvMmDEjO+64YyZMmJDRo0fn/PPPn2fM0KFDc+ONN2b8+PE56aSTcuKJJyZJzjrrrDz88MOZMGFCJk6cmP322y8zZ87MfvvtlzPPPDMTJkzI1VdfneWXX/4V5z1jxoxsscUWueWWWzJ69OgcffTRufXWW3PHHXdkxowZ+cUvfpEk2X///XPsscdmwoQJuemmm/LGN74xhx56aPeZ8SeeeCK33nprxowZs6Av5cuy3AIAYCmz4YYbZquttuq+fckll+S8887L7Nmz8/DDD+euu+7KO97xjpfsM3DgwLz3ve9Nkmy55Za54YYb5jnu9OnTM3bs2Nx///0v2X7ttdfmmGOO6V4eMXjw4IwfPz7rrbdetthiiyTJKqusMt95L7fcctlzzz27b//yl7/M1772tcycOTOPP/54ttxyy7zzne/M448/nve///1JOq93nCTvfve7c9RRR2Xq1Km55JJLss8++yzS5RrOJAMALGUGDRrU/ft9992Xb3zjG7nuuusyceLE7Lbbbr1e+my55Zbr/n3AgAGZPXv2PGNOOOGEjBkzJnfeeWd+/OMfdx+n1jrP1SJ625YkHR0dmTt3bvftnnMZOHBg9z7PPPNMjjzyyFx++eWZOHFiDj744O6xvR23lJIDDjggF198cS644IIcdNBBvb84C4lIBgBYiv31r3/NSiutlJVXXjl//vOfc9VVV73mY82YMSPrrLNOkuTCCy/s3r7rrrvmrLPOypw5c5Ik06ZNy6abbprJkyfnd7/7Xfc85syZkw022CDjx49PrTV/+MMfctttt/X6WM8++2yWWWaZDBkyJE8++WR++MMfJklWW221DBkyJD/72c+SdEb2M888k6Tzah1f+9rXssIKK+Rtb3vba36efSGSAQCWYltssUXe8Y53ZNiwYTnssMOy3XbbveZjffrTn86//uu/znOMj370o1lrrbUyfPjwjBgxIpdddlmWX375XHLJJTniiCMyYsSI7Lrrrnnuueey4447Zp111slmm22W448/PiNHjuz1sVZfffV8+MMfzrBhw7Lnnntmm2226b7v+9//fk4//fQMHz4822+/fR577LEkydprr51NNtlkkZ9FTpLywqcI+9OoUaPquHHj+nsaAADd7r777gwdOrS/p0EPTz/9dDbbbLNMmDAhK6200nzH9/ZnWEq5rdY6an77OpMMAMAS76qrrsrQoUNz7LHH9imQF1SfIrmUcmQpZVwp5blSyoXzGXtsKeWRUsqMUsr5pZRXvhYIAADMx5gxY/LQQw/lqKOOWiyP19czyQ8n+WKSeS+o10MpZUyS45O8J8kGSd6a5PMLMD8AAFjs+hTJtdYf1Vp/nGTqfIZ+OMl5tdZJtdYnknwhyUcWbIoAALB4Lew1yZsmmdDj9oQka5ZSVm8HllIO71rCMe6FTywCAMCSYGFH8opJZvS4/cLv86yurrWeU2sdVWsdtcYaayzkaQAAwGu3sCP5qSQr97j9wu9PLuTHAQB4XZs6dWpGjhyZkSNHZq211so666zTfXvWrFl9Ps7555+fRx55ZBHO9PWpYyEfb1KSEUku67o9Islfaq3zW8sMALBkO2WVhXy8Ga949+qrr57bb7+9c+gpp2TFFVfMcccd96of5vzzz88WW2yRtdZa6zVNc2GYPXt2OjoWdnYuWn2abSmlo2vsgCQDSikrJJlda22/9Ps7SS4spXw/yZ+TnJjkwoU33UVrg+OvWKD9/3Da3y+kmbAkWeD3xQofXPBJzOdfpAD8bbnoooty5plnZtasWdl2221zxhlnZO7cuTnooINy++23p9aaww8/PGuuuWZuv/327Lvvvhk4cGBuueWWLLfcct3HOfvss3Peeedl1qxZ2WSTTfKd73wnAwcOzCOPPJKPfvSjefDBB1NKyTnnnJNtttkmF1xwQb7+9a+nlJItttgiF1xwQQ488MD80z/9U/bYY48kyYorrpinnnoq1157bU477bQMGTIkkyZNyh133JH3v//9efjhhzNz5swce+yxOfTQQ5MkV1xxRU466aTMmTMna665Zq688sq87W1vyy233JLBgwdnzpw52XjjjTNu3LgMHjx4sbzGfU36E5Oc3OP2gUk+X0o5P8ldSd5Ra32o1vqLUspXk/wqycAkP2z2A4DXpQX9H+rE/1TTN3feeWcuv/zy3HTTTeno6Mjhhx+eSy+9NBtuuGEef/zx3HHHHUmS6dOnZ9VVV823vvWtnHHGGb1+PfTee++dj33sY0mS448/PhdeeGGOOOKI/PM//3N22WWXHHnkkZk9e3aeeeaZTJgwIV/5yldy0003ZfDgwZk2bdp853rzzTfnrrvuynrrrZekM+4HDx6cZ555JqNGjcpee+2V5557LkcccURuuOGGrL/++pk2bVoGDBiQ/fffPxdffHGOPPLIXHXVVdlqq60WWyAnfYzkWuspSU55mbtXbMb+W5J/W6BZAQC8zk2cMr1P44avu+pLbl977bW59dZbM2pU5zcrP/vss3nzm9+cMWPG5J577snRRx+d973vfdl1113nP4eJE/O5z30u06dPz5NPPpl/+Id/SJJcf/31ufTSS5MkHR0dWXnllXPddddl33337Q7VvgTr6NGjuwM5Sb7+9a/npz/9aZJkypQpuf/++/PHP/4xO+20U9Zff/2XHPeQQw7J3nvvnSOPPDLnn39+91nnxWXpWhyypFsYa5WcAQAAXkGtNQcffHC+8IUvzHPfxIkTc+WVV+ab3/xmfvjDH+acc855xWONHTs2V155ZYYNG5Zzzz03N998c/d9pZR5HrfdlnRG9Ny5c5Mkc+bMyezZL67GHTRoUPfv1157bf7nf/4nN998cwYOHJjtt98+M2fOfNnjbrDBBllttdXyq1/9KuPHj+9T9C9MC/vqFgAALEwPj0+e/HPy1z8lD4/PzsPXyWUXfzeP33ld8vD4TL3zV3no1p/nsTuuS3349uy93Ub5/JEfzO9+e2Py8PistFzNk0/2fqGxp59+OmuttVaef/75XHzxxd3bd9ppp5x99tlJOsP3r3/9a3beeedceuml3cssXvjnBhtskNtuuy1Jcvnll2fOnDm9PtaMGTMyePDgDBw4MJMmTcqtt96aJNluu+1y3XXXZfLkyS85btJ5NvmAAw7Ifvvtl2WWWbzZ6kwyAMBSZLOhG+fkTxyenfc9InPr3Czb0ZGzTzshAwYsk0M+eWr3mdmvnPAvSZKD9vnHHHroob1+cO/UU0/N1ltvnfXWWy/Dhg3LzJkzkyRnnHFGDjvssPz7v/97Ojo68u///u/Zeuut86lPfSo77LBDOjo6suWWW+a8887LRz/60ey+++655pprsuuuu2b55Zfvdd5///d/n3POOScjRozI29/+9myzzTZJkjXXXDNnnXVWdt9999Ras/baa+fKK69Mkuy55545+OCD85GPfGQRvqK9K7XWxf6grVGjRtVx48b19zRcxYBeeV8AfeGDe68/d999d4YOHbpAx+jruuNXMnyZBxf4GFl78wU/Rj+4+eab85nPfCa/+tWvXtP+vf0ZllJuq7WOmt++ziQDALDE+dKXvpRzzjmn+wOEi5s1yQAALHFOOOGETJ48OaNHj+6XxxfJAADQEMkAANAQyQAA0BDJAADQcHULgNfApQGBRW36E9Ny+H67Z4XMyiOPTc2AActkjcGrJUluueK7WW65Zed7jIOOPTnH//NBedtSegm4/iSS4W/E3W9fsGt9JsnQ/717IcwEYOm02UWbLdTjff89N7zi/auuNjiXXXVDhi/zYE45/eysOOgNOe5jY18yptaaWuvLfhvdBV///EKb75Ji9uzZ6ehY9AlruQXA37C73z50gX6Axe/3Dz6UYe/eOx/79JeyxZgP5s9/eTyHf+oLGfXeA7LpTv+UU79+TvfY7fc4OLffeU9mz56dVVddNccff3xGjBiR0aNH59FHH53n2DfffHNGjx6dzTffPNttt13uu+++JMm5556bY445pnvcbrvtlhtvvDFJcsUVV2SLLbbIiBEjsuuuu85zzDvuuCNbbbVVRo4cmeHDh+eBBx5IklxwwQUZPnx4RowYkYMOOihJ8uCDD2annXbK8OHDs8suu2TKlClJkgMPPDCf/OQns9NOO+Wzn/1snnrqqXzkIx/J1ltvnc033zw/+9nPFtKr+yKRDACwlLnr3gdyyP57ZPzVl2SdN70xp33mXzLuyu9nwjWX5pr/uTl33fvAPPvMmDEjO+64YyZMmJDRo0fn/PPPn2fM0KFDc+ONN2b8+PE56aSTcuKJJ77iPB555JEcccQRufzyyzNhwoRev/jj29/+do477rjcfvvtufXWW7P22mtnwoQJ+cpXvpLrr78+EyZMyOmnn54k+fjHP55DDz00EydOzN577/2SML///vvzy1/+Ml/96ldz6qmnZrfddsstt9yS6667Lp/85Ce7v1J7YbHc4nXIX6sDwOvbhuuvm61Gbtp9+5Kf/CLnXfKTzJ4zOw8/8njuuveBvGOTt75kn4EDB+a9731vkmTLLbfMDTfMu9xj+vTpGTt2bO6///4+zeM3v/lNdtppp6y//vpJksGDB88zZtttt80Xv/jFTJ48OR/4wAey0UYb5brrrsu+++7bPf6Ff/72t7/Nf//3fydJxo4dm5NOOqn7OHvvvXf3spKrr746V155ZU477bQkycyZM/PQQw9lk0026dO8+0Ikw1JgYayDu2whzIMli/cF/O0a9IaB3b/f98BD+ca5l+SWK76bVVdZKQcedUJmPvfcPPsst9xy3b8PGDAgs2fPnmfMCSeckDFjxuTjH/94fv/732e33XZLknR0dGTu3Lnd4144a1trTSnlFef6oQ99KKNHj84VV1yRXXbZJRdddFGf9pvnOQ8a1P17rTU//vGPs+GGG76qY7waInkJ4z96ACyIhfLfkS/PG0+vhr+NXLz++tRTWWnFN2TllQblz395LFdd/5vs9nfbvmTMpMcnZW6dm0mPT0qS/PHJP+aJmU90337BlMemZM5KczLp8Un55re/mVlzZmXS45NSV6sZP358aq2ZPHlybrvttiTJdtttl2OOOSaTJ0/O+uuvn2nTps1zNvmBBx7IRhttlKOPPjr33XdfJk6cmJ133jn77LNP/uVf/iWDBw/u3u+d73xnLrvssuy///753ve+lx122KHX5zxmzJh885vfzDe+8Y0kyfjx47P55gv3Ch4iGQBgKbbFZkPzjo3fmmHv3jtvXW+dbLfVyNd8rEOOOiQnHX1Szv/W+dl6+627t2+17Va5ap2rstlmm2XYsGEZObLzMdZcc82cddZZ2X333VNrzdprr50rr7zyJce8+OKLc8kll2TZZZfN2muvnS9+8YtZddVV86lPfSo77LBDOjo6suWWW+a8887LGWeckUMOOSRf/vKXs+aaa+aCCy7odZ4nn3xyjjnmmGy22WaZO3duNtpoo/zkJz95zc+7N6XWulAP+FqMGjWqjhs3rr+nsURc93Szt6y3wMdY0DMAibMAPXlfvMj74kXeF528J160oO+JxPtiSXP33Xdn6NAF+5zPxCnTF3gew5d5cIGPManHUovXatMhm85/0BKmtz/DUspttdZR89vXmWQAAObr2TvvXKD9Bw4btpBmsni4BBwAADREMgAANEQyAAA0RDIAADREMgAANFzdAgBgCTT9iWk5fL/ds0Jm5ZHHpmbAgGWyxuDVkiS3XPHdLLfcsn06zvmX/jgbjnl3hqw5ZFFO93VHJAMA9MHdb3/110x+pYx9/trfvOK+q642OJdddUOGL/NgTjn97Kw46A057mNjX/Uczr/0pzl28+FLbSTPnj07HR2LP1kttwAAWMpcdNnPsvXffygjd9kvH//MlzN37tzMnj07HzrqxGz2nn0y7N1755vnXZIf/OSq3D7pnhx32HHZ6+/2yvOznn/JcX5w4Q+y7y775gN/94F84uBPZOazM5Mknz7i0/nlz3/ZPW6r9bfq/v2r//Ef2WrPPbPNXnvl5G9+c565/eeVV2ZU1/27HXxwks7QPfbYYzNs2LAMHz483/72t5Mk11xzTUaOHJnNNtsshx12WGbNmpUkWXfddfOFL3wh2223XS6//PLcd999GTNmTLbccsvssMMOuffeexfuC9oLZ5IBAJYid/7v73P5L36Vm35yQTo6OnL4p76QS39yVTZcf908/sT03PHLy5Ik02c8mVVXWSnfuuAHOfYrJ+btm719nmON+ccx2fcj+yZJvn7q1/PjS3+c/Q7a72Uf+4rrr8/VN96Y/7nkkgxcYYVMmzFjnjH/31ln5Rfnn581hwzJ9L/+NUnyHz/4QR5++OFMmDAhAwYMyLRp0/LMM8/k4IMPzvXXX58NN9wwBxxwQM4555wceeSRSZJBgwbl17/+dZJkp512yrnnnpsNN9wwv/71r3PkkUfm6quvXrAXcj5EMgDAUuTaG36bWydMyqj3HpgkeXbmc3nzm9bKmB1H5577J+foz30t73v3dtl1x9HzPda9d92bM047I0/+9ck8/dTT2XGXHV9x/K9uvjlj99gjA1dYIUkyeJVV5hnzzs03z2EnnJA9d901u++8c5Lkuptvzic/97kMGDCgc7/Bg3Pbbbdl4403zoYbbpgkGTt2bM4777zuSN533854nz59em6++ebstdde3Y8xe/aCfXV6X4hkAIClSK01B++7e77wqY/Pc9/Ea3+QK6/7db553qX54c9/mXO+etIrHusz//yZnH3p2dl46Mb5r+/+VybeNjFJ0jGgI3Pnzk2SzJkzJ3PmzOl+7FLKKx7z26ecklsnTszP/9//yzZ77ZVbfvjD1GSe/Wqtr3icQYMGdY8bMmRIbr/99lccv7BZkwwAsBTZ+V3b5LKfXZPHpz2RJJk6bXoe+tOf89jUJ1Jrzd7v3yWfP+5j+d0d/5skWWnFN+Tpp57u9VjPPvNshrxxSJ5//vn8/Ec/796+9npr564JdyVJfnnFLzN3Tmcwv2fbbXPR5Zfn2Zmda5d7W27x4JQp2XrEiJx81FFZbeWV8/Cjj+Y9o0fnrLPO6o7tadOm5R3veEfuu+++PPDAA0mS733ve9lxx3nPZK+22mp505velMsvvzxJMnfu3EyYMOHVv3CvkjPJAABLkc2GbpyTP3F4dt73iMytc7NsR0fOPu2EDBiwTA755KndZ3u/csK/JEkO2ucf8+ljT87yKyyfS6++NMv2uHTckZ8+MvuP2T9rrbNWNn77xnnuueeSJPuM3SdHjT0qv/l/v8m2f7dt9z7v23HH3HHPPdl+v/3S0dGR9+24Y04+6qiXzO9TX/1qJk+ZkprkPaNHZ9ONN84mG2yQyRdemOHDh6ejoyNHHHFEPvaxj+W8887LBz7wgcyZMyfbbLNNDjvssF6f86WXXpojjjgip5xySmbNmpUDDzwwI0aMWASv7ovK/E51Lw6jRo2q48aN6+9pZIPjr1ig/f+wwgcXeA6bvWW9BT7GZV9e8HU6Q//37gU+xuuF98WLvC9e5H3RyXviRQv6nki8L5Y0d999d4YOffWXfetp4pTpCzyP4cs8uMDHmLTccgt8jLc+smDNOHDYsAWew6vV259hKeW2Wuuo+e1ruQUAADREMgAANEQyAAA0RDIAwMtYEj67xWuzoH92IhkAoBcrrLBCpk6dKpSXQrXWTJ06NSt0fenJa+EScACtradVAAAgAElEQVQAvVh33XUzZcqUPPbYY6/5GH954tkFnsfd5bU//gse6Vjw5Jvz1wXbf9mub9tbXFZYYYWsu+66r3l/kQwA0Itll102b3nLWxboGO9dQi4NuI9LA75qllsAAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQKNPkVxKGVxKubyU8nQpZXIp5YMvM275UsrZpZS/lFKmlVJ+VkpZZ+FOGQAAFq2+nkk+M8msJGsmOSDJWaWUTXsZd3SS0UmGJ1k7yfQk31oI8wQAgMVmvpFcShmUZK8kJ9Van6q13pjkp0k+1MvwtyS5qtb6l1rrzCSXJuktpgEAYInVlzPJmySZU2u9t8e2Cek9fs9Lsl0pZe1SyhvSedb5ygWfJgAALD4dfRizYpIZzbYZSVbqZey9SR5K8qckc5LckeTI3g5aSjk8yeFJst566/VxugAAsOj15UzyU0lWbratnOTJXsaelWSFJKsnGZTkR3mZM8m11nNqraNqraPWWGONvs8YAAAWsb5E8r1JOkopG/fYNiLJpF7GjkhyYa11Wq31uXR+aG/rUsqQBZ8qAAAsHvON5Frr0+k8I3xqKWVQKWW7JLsn+W4vw29NMraUskopZdkkH0/ycK318YU5aQAAWJT6egm4jycZmOTRJJckOaLWOqmU8q5SylM9xh2XZGaS+5I8luR9SfZciPMFAIBFri8f3EutdVqSPXrZfkM6P9j3wu2p6byiBQAALLV8LTUAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADREMgAANEQyAAA0RDIAADT6FMmllMGllMtLKU+XUiaXUj74CmO3KKX8TynlqVLKX0opRy+86QIAwKLX0cdxZyaZlWTNJCOTXFFKmVBrndRzUCllSJJfJDk2yX8lWS7JugtvugAAsOjN90xyKWVQkr2SnFRrfarWemOSnyb5UC/DP5Hkqlrr92utz9Van6y13r1wpwwAAItWX5ZbbJJkTq313h7bJiTZtJex70wyrZRyUynl0VLKz0op6y2MiQIAwOLSl0heMcmMZtuMJCv1MnbdJB9OcnSS9ZI8mOSS3g5aSjm8lDKulDLuscce6/uMAQBgEetLJD+VZOVm28pJnuxl7LNJLq+13lprnZnk80m2LaWs0g6stZ5Tax1Vax21xhprvNp5AwDAItOXSL43SUcpZeMe20YkmdTL2IlJao/bL/xeXtv0AABg8ZtvJNdan07yoySnllIGlVK2S7J7ku/2MvyCJHuWUkaWUpZNclKSG2ut0xfmpAEAYFHq65eJfDzJwCSPpnON8RG11kmllHeVUp56YVCt9bokn01yRdfYjZK87DWVAQBgSdSn6yTXWqcl2aOX7Tek84N9PbedleSshTI7AADoB76WGgAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGiIZAAAaIhkAABoiGQAAGn2K5FLK4FLK5aWUp0spk0spH5zP+OVKKf9bSpmycKYJAACLT0cfx52ZZFaSNZOMTHJFKWVCrXXSy4z/1ySPJllxwacIAACL13zPJJdSBiXZK8lJtdanaq03Jvlpkg+9zPi3JDkwyZcX5kQBAGBx6ctyi02SzKm13ttj24Qkm77M+G8l+WySZxdwbgAA0C/6EskrJpnRbJuRZKV2YCllzyQdtdbL53fQUsrhpZRxpZRxjz32WJ8mCwAAi0NfIvmpJCs321ZO8mTPDV3LMr6a5Ki+PHCt9Zxa66ha66g11lijL7sAAMBi0ZcP7t2bpKOUsnGt9b6ubSOStB/a2zjJBkluKKUkyXJJVimlPJLknbXWPyyUGQMAwCI230iutT5dSvlRklNLKYem8+oWuyfZthl6Z5I397i9bZIzkmyRxHoKAACWGn39MpGPJxmYzsu6XZLkiFrrpFLKu0opTyVJrXV2rfWRF36STEsyt+v2nEUyewAAWAT6dJ3kWuu0JHv0sv2GvMy1kGut1ydZd0EmBwAA/cHXUgMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAo0+RXEoZXEq5vJTydCllcinlgy8z7l9LKXeWUp4spTxYSvnXhTtdAABY9Dr6OO7MJLOSrJlkZJIrSikTaq2TmnElydgkE5NsmOTqUsofa62XLqwJAwDAojbfM8mllEFJ9kpyUq31qVrrjUl+muRD7dha61drrb+rtc6utd6T5CdJtlvYkwYAgEWpL8stNkkyp9Z6b49tE5Js+ko7lVJKknclac82v3D/4aWUcaWUcY899lhf5wsAAItcXyJ5xSQzmm0zkqw0n/1O6Tr+Bb3dWWs9p9Y6qtY6ao011ujDNAAAYPHoy5rkp5Ks3GxbOcmTL7dDKeXIdK5Nflet9bnXPj0AAFj8+nIm+d4kHaWUjXtsG5GXX0ZxcJLjk7yn1jplwacIAACL13wjudb6dJIfJTm1lDKolLJdkt2TfLcdW0o5IMn/l2SXWusDC3uyAACwOPT1y0Q+nmRgkkeTXJLkiFrrpFLKu0opT/UY98Ukqye5tZTyVNfP2Qt3ygAAsGj16TrJtdZpSfboZfsN6fxg3wu337LwpgYAAP3D11IDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAEBDJAMAQEMkAwBAQyQDAECjT5FcShlcSrm8lPJ0KWVyKeWDLzOulFK+UkqZ2vXz1VJKWbhTBgCARaujj+POTDIryZpJRia5opQyodY6qRl3eJI9koxIUpNck+SBJGcvnOkCAMCiN98zyaWUQUn2SnJSrfWpWuuNSX6a5EO9DP9wktNrrVNqrX9KcnqSjyzE+QIAwCJXaq2vPKCUzZPcVGsd2GPbcUl2rLW+vxk7I8mutdbfdt0eleRXtdaVejnu4ek885wkb0tyz4I8kdeRIUke7+9JsMTxvqA33hf0xvuC3nhfvGj9Wusa8xvUl+UWKyaZ0WybkWSe8O1l7IwkK5ZSSm1qvNZ6TpJz+vD4f1NKKeNqraP6ex4sWbwv6I33Bb3xvqA33hevXl8+uPdUkpWbbSsnebIPY1dO8lQbyAAAsCTrSyTfm6SjlLJxj20jkrQf2kvXthF9GAcAAEus+UZyrfXpJD9KcmopZVApZbskuyf5bi/Dv5PkE6WUdUopayf5ZJILF+J8/xZYgkJvvC/ojfcFvfG+oDfeF6/SfD+4l3ReJznJ+Ul2STI1yfG11otLKe9KcmWtdcWucSXJV5Ic2rXruUk+bbkFAABLkz5FMgAA/C3xtdQAANAQyQAA0Ojr11IDsAQopbwtnVcOWrHn9lrr+f0zI2BJVEpZLcluSdautZ5eSlkryTK11of7eWpLDWuSAZYSpZTPJvlckglJnulxV621vrt/ZkV/KKXMeaW70/meGLC45sOSpevCCj9K578rtqm1rlRK+bskn6i1/mO/Tm4pIpKXEKWUVdL59dzt2aHr+mdG9JdSykeS7FZr3a+X+y5JckWt9XuLfWL0u1LKo0l2rrVO7O+50L+63gvTklyU5CdJnmvH1FrvX9zzYslQSvldOq9EdnUp5Yla62qllBWS/KHWulZ/z29pIZKXAF1RdGY6v7GwPTv01n6ZFP2mlHJzkiNqreN7uW9EkrNrraMX/8zob6WUyUk2rrXO6u+50L9KKQOSvC/J2CTbJ/lpku/UWn/drxNjifBCGHf9Pq3WOriUskySx2qtq/fz9JYaInkJUEr5U5JDa61X9vdc6H+llMdrrUNe6/28fpVSxibZLskpSf7S875a69z+mBP9r+tvIvdL8qEkb0yye6317v6dFf2plPKbJCfVWq/tEck7d23bsb/nt7Twwb0lQ0eSq/t7EiwxBpRSBtdap7V3dH2xj3WGf7su7PrnoT22lSQ13hd/y2rXT+J9QKfjkvyklPKTJANLKWcm2bPrhz5yJnkJUEr5RJKVknzB2SBKKVck+VWt9f/0ct8nk7yn1vq+xT8z+lspZf2Xu6/WOnlxzoX+1fVX5+9N8uEkO6RzucV3a6039OvEWGKUUt6czr9dWD/JH9P5/vDviVdBJC8BSil/TLJWklnp/NrvbrXW9fplUvz/7d15tF1lfcbx7xMkgBDIwGAYEiahBVRUlEGNCIIoTXUpoIiCULVqK5SFYnHAQAtVZCgFsVRAIggRnBAREAwKqCgghYBtlSEMMoYEAoQhkqd/7H3J4XAvuTHxvHvf83zWysq+794391lZZ537O3u/7/srRtLrgZ8C3wC+C9wHTATeA3wI2Mn2dcUCRnF1gbQO8EA+WPcnSfcBjwBnAT8Anuq+xvbtvc4VzSNpbdsPls7RRimSG0DSkPODbP+8l1miGSRtDxwDbE/V9GcR8CvgUNu/KpktypG0OnAy1fzTlwALgRnAgbYfLZktektS54ejgV/k6hzLFnD9q56nfhKwF/Cs7VUlTQW2sf3FsunaI0VyRMNIGjVwd1DSKsA4YJ7tJ8smi9IknUk1Nesw4E6qx6hHAQts71cwWkQ0iKRzgCeAI4Gb6i3g1gautr1Z2XTtkSK5ASSNpnqMvjUv3Cd53xKZopz6MepZVNs53Vw6TzSHpPuBjW0v6BhbDbjN9jrlkkWvSToGONP270pnieap99Fe3/YzA7tb1OOP2l6jcLzWGFU6QADVZvD/BDwG3Nb1J/rPx4CNgGsl/VbSQZLWKh0qGuEpoPu1sCaDNJKIEe/lwG8lXSfpQEnZFjI6zQfGdw7UC/keGPzyGEzuJDeApHnARrYfKZ0lmkPSWOC9VKuTt6HaJnA68EPbC0tmizIkfZ6qecTxLJ5ucTDVqvV/LZktek/SOGBv4APAa4BLgG8CF+Y9or/VLezfDnwWuBDYBfg34GLbx5XM1iYpkhtA0o3ArrbzCS8GJWkjqmL5w8BL00ykP0kSsD/wfmBd4F7gXOAM5828r0l6OdV7xD7AGsAM2/9YNlWUUr9XHAJ8FJhEtQXcqcBxea8YvhTJDVDvfbsncCIv7KI1s0ioaAxJKwHvpNoP9a3AL22/pWyqiGiaen3Lu4AvAxvYTsOwPlS3LN8H+LbtTMVaBimSG0DSHUOcsu2NexomGkPSG6kere8FPMjixXzZDL6PSPqg7bPq4wOGus72Gb1LFU0iaQcWv1c8zOL3itklc0U5WaC3fORTZjP8s+1vdw9KOrJEmChL0jSqx6bjgfOB3W3/omioKGlvqqIHqtfFYAykSO4jHVOwPki1ePM7wN/avrposGiKiyS9w/aPSwdps9xJbgBJtwP/YPvijrGjgbfbfnW5ZFGCpEuAM4Ef2H5BF62ICEkLgcupFup9P+8V0UnSDKqpN1dTzUd+rtizPeQTqXi+3EluhncAl9aPVa+UdDwwBdipcK4owPZu3WOSNge2AG7II9T+VW8F+KTtx+t5h/sCfwK+lfbUfWey7XtLh4jG+gPwldIh2i53khtC0muAC4BfUK1E3c32/LKpogRJx1EVw2fXX+9L9Sh9HlWzmXd3PnWI/iHp18DHbN8g6cvA31C1pr7C9sFl00UvSdoNmG/7l/XXm1JtEbkVVQv7/W3fVzBiFCBpb9vnls4xUqRILkTSYHeJpwB/T9VM4jHI7hb9SNJtwFts31V/fQ9wtO1TJO0HfNz2dkVDRhH1nurjbbt+XewAPA7cYnti2XTRS5KuBQ60/av6658BC4CvAgcAT9nep1zCKEHSfNurl84xUqRILuRFdrTolN0t+lDnm5ykrYBrgbG2n64fsT800GI0+oukOcB6wGZU++BuKWkU8KjtMWXTRS9JmgusbftPktam2jN7su0/1t33brK9btmU0WuSHst7wfKTOcmF2N6odIZorEclrVM3l3kTcF3HXpcrAioXLQq7GDgPmADMqMe2AP5YLFE0wfbAHbYHXgcPU03Niv6zgqS38CK/J/KEevhSJEc0z3nADEnfp+qY9KWOc9sCtxVJFU3wYaqmMgtZvC3cmsC0UoGimGuBAyWdRvW66FynsDEwp0iqKG0l4HSGLpJN9fqIYch0i4iGkbQi8FlgG+AaqvnIrs8dRLW7wX8VjBgNIWkV4Fnbz5TOEr0laQvgQmBD4FaqdQz31uemARva/lCpfFFG5iQvXymSIxqonnv8ReCotBWNAZKOBc6z/RtJu1M1kDDwXtsXlk0XJUiaYPvhrrGxwDO2FxSKFYWkSF6+UiRHNFS9SGvt7H8bAyTdB2xie0G9HdwxwKPACbZfUTZdRJSWhXvLV4rkiIaqm8rcavuU0lmiGSQ9ansNSROA/7W9Vj2eu0cREctZFu5FNNfrgU9KOpQXthWdUixVlPR7SfsAmwKXAdTbfT1ZNFVExAiUIjmiub5e/4kY8AngRKrdLQ6ox94G/KRYooiIESrTLSIiIlpE0rC28LJ9+186S8RIliI5osEkrUM17WJNOva9tH1GsVBRlKRdgPdRLeqcKmkbYPU0COgfkhZRTb8SHdOwur+2vUKPo0XD1N0Yn9dYJh+ehi/TLSIaStK7gLOBPwBbArcAWwFXAymS+5CkTwIHAacBe9TDTwL/AexQKlf0lu1RA8eS9gfeStVQ5k5gMnA48NMi4aIRJO1G1VTkZTy/sYiBfHgaptxJjmgoSTcDR9g+X9I82+PqX4hb2v5U6XzRe5JuA3a2PbvjNbEC8KDtCaXzRe9Jugd4ue0nO8ZeCvze9vrlkkVJ9XvFV4Dpna+NWDqjlnxJRBQyyfb5XWPTgX1LhIlGGEO10wksfqy+IpCOe/1rFFXXvU6Tyd3CfjcOODUF8rJJkRzRXA/Wc5IBZkvaHtiE/PLrZ1cC/9w1diBwRYEs0QwnADMlHS3p45KOpppqcULhXFHW6cD+pUO0XaZbRDSUpM9QNRP5rqR9gf8CFgHH2f5C2XRRgqSJwIVUCznXA24H5gNTbd9fMluUU88/3RNYF7iPqnX5JWVTRUmSrqJa9H0n8Lz3huyzP3wpkiNaQtIkYFXb/1M6S5QhaRTVNIvXUT1Svxv4TVqXR0QnSfsNdc729F5mabMUyRENJmlFYDtgXdvflrQqgO0nyiaLXqsX6D0OjLX9dOk80QySVqLazWJvYELdtnxXYDPbJ5dNF9FumZMc0VCSXgH8nqrr3un18JvJ9m99yfazVK+H7GIRnU6g2hpyHxYv5rwF+HixRNEIkvaXNFPS/9V/Z47yUsqd5IiGknQ11erkszq2+1qVamun9Urni96TdChVI5ETgXt4fuOINBPpQ5LuAza1/YSkubbH1+OP2B5bOF4UIulzVDshHcfi/bMPBs62fVTJbG2SIjmioSTNA8bbdtcvv+eOo79IumOIU7Y9rFbFMbJIuhN4pe1HB94bJK0FXGN7k9L5ooz6vWJH23d2jE0GrrQ9uVyydknHvYjmmg28FrhuYEDS64FbSwWKsmxvVDpDNM75wHRJB8NzO6D8OzCjaKoobVXgoa6xh4FVCmRprcxJjmiuLwAXSToCGC3pMKpfiJ8vGysiGuSzVB+oZwFjqdrY3wscWTBTlHcJ8C1Jm0taRdJfUTWjurRwrlbJdIuIBpP0GuDDLN7u6+u2ry+bKkqRdDcd85A7PE01R/l7wNds/6mnwaIR6mkWc5xf7H1P0urAycBeVF05FwLnAQfafqRktjZJkRwR0RKSPg18APgPqg9Nk4B/oHrCMBc4BPi+7UOLhYyekjQTOMf2aV3jF9nevVCsaIh6b/U1qT48ZT/1pZQiOaKhJI0GPgRsDazWec72viUyRVmSbgF2sX1vx9h6wE9sbylpc+By2xsUCxk9Jekpqs6LM4GD6q0CkTTf9upFw0VPSdrQ9uz6eMiFvLZv71molsvCvYjmmg68iqoN8QOFs0QzTKRqKNLpCap2xFDto5xtv/rLM8C2wLnA5ZLeY3suoLKxooBZwJj6+FaqqVndrwMDK/QyVJulSI5ort2AjTJ/LDpcCFwg6SiqOcjrA4fV4wDbUy3iij5i+zFJU4GjgeskvYvB567HCGZ7TMdxNmZYDjLdIqKhJN0I7Go7d5EDAEkrA9OAPanuHt9LNR/5SNsLJL0MGG37rnIpo5ckPdZZHEl6H9Wc9bG2R5dLFk1ST794tnPf5FiyFMkRDSXpEKpi6ES6pluku1pEAEjay/Z5XWOvBv7W9hGFYkVhks4FTrL9y7od9SnAIqrdLU4vm649UiRHNFS6q8VgJO1C1Zp6bdtTJW0DrJ4PThExQNKDwPq2n5E0C/gY8AjwA9svL5uuPTInOaKh0l0tukn6JHAQcBqwRz38JNXj9R1K5YrekvQ/tv+6Ph5q72xsT+ppsGiS0XWBvB4w3vYvACStUzhXq+ROckRES0i6DdjZ9mxJ82yPk7QC8KDtCaXzRW9IeqPtq+vjNwDzZNoAAAvYSURBVA91ne2f9y5VNImkn1F115sMjLL90bpg/rXt9YuGa5HcSY5okGHeIbLtyb1NFg0xhqqJCCx+baxItQ1Y9ImBArk+TiEcg/k74F+oOu19uh7bHvhWsUQtlDvJEQ0yjDtEKwKH257S22TRBJK+A9xg+yhJc22Pl3QosLXt95fOF70h6cjhXGf78L90loiRLEVyRItIWglYYDubwfchSROp9kReE1iPqtPafGCq7ftLZovekfSN4Vxne/+/dJZoDkkftH1WfXzAUNfZPqN3qdotRXJEi6RIDkkCXkc11/Bu4De2F5VNFRGlSfqx7XfUx1cMcZlt79TDWK2WIjmiRVIkRzdJo4GP2P5q6SxRjqQxVE8YnmtDbPv2coki2i8L9yIaRtKLfcpPB60+JWlnYGvgVtsXSHoJ8AngM8BcIEVyH5K0BdVirFdRLeYUixd15sN0n5K0KzDb9u87xjYDJtu+rFyydkmRHNE8S+qGlJbDfUbSZ4AvALcAW0o6BdgReBr4qO2LCsaLsk4BrgDeAtwBbAj8G/DLgpmivK8C3Qu8H6/HN+t9nHbKdIuIiIaTdDuwp+3rJW0H/AL4lO0TCkeLwiTNo+q+uFDSI7bHSloVuDkNifqXpEdtr9E1JuBR26sXitU6o0oHiIiIJVrT9vUAtq+huoP872UjRUM8RbU1JMAcSZOofrenuUx/u32QqXs7Uj1tiGHKdIuIiBao7wIN/HmqHnvuRkd2uOhbVwF7AWcC3wEupvoQNbNgpihvGvA9SacDtwGbAPvXf2KYMt0iIqLhJC3i+d0XOxdniWpbpyzS6nP1h6Z9gNWAb9p+onCkKEjS64EDgA2otos83fa1ZVO1S4rkiIiGk7TENuS27+xFloiIfpEiOSIioqUkjQc+RbU94Gqd59K+vn/Ve+ofDuwNTLC9Rr0t3Ga2Ty6brj0yJzkiIqK9zgFWAs4DFhTOEs1xAlXr+n2o5qlDtYXkCUCK5GHKneSIiIiWkjQfWMv206WzRHNIug/Y1PYTkubaHl+PP2J7bOF4rZEt4CIiItrrJmD90iGicZ6ha7aApLWAh8vEaadMt4iIaAlJ6wELbM/rGBsHrGL73nLJoqCZwCWSvgHc33nC9hllIkUDnA9Ml3QwgKSJVHurzyiaqmUy3SIioiUkXQscYHtWx9grgNNsb1suWZQi6YohTtl2dzOJ6BOSRgPHAB8GXko1X/3rwGdsP1MyW5ukSI6IaInBWs2+2HiMbHWDmY2Au2z/qXSeaKZ6msUcp+BbapluERHRHg9J2tT2rQMDkjYl8wz7km1LmgWMKZ0lmkXSFsCbgPHAXKrOjL8rGqqFUiRHRLTHGcB3JX0OuJ2q1ey/AKcVTRUl3QBsBvxv6SBRXv104XRgP+Ae4F6qreDWlXQW1XSt3FEephTJERHt8SVgIXAsi1vNngYcXzJUFPUzqoV7Z1K9Hp4rgLJwry99FNgR2K6zBbWk1wHnAn8P/GeZaO2TOckREREtlYV70UnS1cCXbP9okHN/Axxm+w29T9ZOKZIjIhpM0hTbV9bHQxY9tmf2LlVENJGkucBk248Ncm4M1SLPcb1P1k4pkiMiGkzSzba3qo/vGOIy2964h7GiQSRNAN4BvMz2VyStC4yyfU/haNFjS9rpJjvhLJ0UyRERES0l6c3Ad4HrgDfYHlOPfcr21LLpotckLQB2BzTEJRfaXrWHkVotRXJEREtIusD2OwcZ/57td5fIFGVJuoGqIP6ppHm2x0laGbjT9jql80VvSZpNx+LNwdjeqDdp2i9FckRES0iab3v1Qcbn2h5fIlOUNVAY18dzbY+XNAp4yPaEwvEiWi1bwEVENJykI+vD0R3HAzYG7uxxpGiO30l6m+1LO8beCswa6hsiYnhSJEdENN8G9d+jOo6heqx6NzCt14GiMQ4BfiTpImAVSacCU4EXTMuJiKWT6RYRES0h6SO2v146RzRLvZvFB4DJVB+azs7OFhHLLkVyRERLSNoCeNj2A5JWAz4NPAsca3tB2XTRS5LeZPuqFzl/lO3P9TJTxEgzqnSAiIgYtnOAsfXxscAUYHvg1GKJopQLJG072AlJx1PdWY6IZZA7yRERLSHpEdtjJQm4H9gSeBK4w/baZdNFL0l6H3ASsKvtGzrGTwHeBuxse3aheBEjQhbuRUS0x9N1a9ktgLttz5H0EmDlwrmix2zPkLQScKmknWzfLOl04E3AmzMnOWLZpUiOiGiPc4CZwBjg5HrsNcBQ7apjBLM9vS6UL5N0DbAZMMX2/YWjRYwImW4REdEiknYFFtq+ov56G2B12zPLJotekrRTx5efoNob+WPAgwODeU1ELJsUyRERES0jaUlPD2x7456EiRihUiRHRDSYpEts71YfX0XVQOQFbE/pabCIiBEuc5IjIprtmx3HpxVLERHRZ3InOSIiIiKiS+4kR0S0hKQDhjj1NHAPcI3tp3sYKSJixMqd5IiIlpD0M6oOew9QFcXrA+sA1wEb1pe90/Z1JfJFRIwkaUsdEdEetwCftj3J9g62JwGHADdQFcxfo+rCFhERyyh3kiMiWkLSPGCC7UUdYysAc2yPqxtLPGh7jWIhIyJGiNxJjohojweAqV1ju7O4gcTKwMKeJoqIGKGycC8ioj0OBM6XdDNwN7ABsBWwZ31+WzLdIiJiuch0i4iIFpG0JvB2YF3gPuAi2w+XTRURMfKkSI6IaBlJk4D1gD/avqt0noiIkShzkiMiWkLSREk/B/4AfA+4VdKVktYtHC0iYsRJkRwR0R5fA24ExtueCIyj2v7tP4umiogYgTLdIiKiJSTNASbaXtgxthLVtIs1yyWLiBh5cic5IqI95gFbdI1tDjxSIEtExIiWLeAiItrjGOBySacDdwKTgf2BLxRNFRExAmW6RUREi0jaCXg/1RZw9wLn2J5ZNlVExMiTIjkiosXqttRftH146SwRESNJiuSIiBarF+4tsL1C6SwRESNJFu5FRLSfSgeIiBhpUiRHRLRfHglGRCxn2d0iIqLh6sV6QxndsyAREX0kc5IjIhpO0h1Lusb2Rr3IEhHRL1IkR0RERER0yZzkiIiIiIguKZIjIiIiIrqkSI6IiIiI6JIiOSJiGUmypLM6vn6JpIck/Wgp/53ZktZc1msiImLZpUiOiFh2TwBbSVql/noX4I8F80RExDJKkRwRsXxcDOxeH+8NnDtwQtJ4ST+QdJOkayS9sh6fIOknkm6QdCodnfMkfUDSbyT9t6RTJb1o22lJj0s6StKN9c9Ypx6fKunX9c+4vGN8mqTp9c+fLendko6RNEvSJZJWrK97raSfS7pe0qWSJi7P/7SIiKZKkRwRsXzMAN4naWXglcCvO84dAdxg+5XAZ4Fv1uNfBK62/Wrgh8AkAEl/DbwXeIPtrYFngX2W8PNXBa6x/SrgSuAj9fjVwHb1z5gBHNrxPZtQFfbvBM4GrrD9CuBJYPe6UD4J2MP2a4EzgKOG/18SEdFe6bgXEbEc2L5J0oZUd5F/3HX6jcB76utm1neQ1wCmAO+uxy+SNK++fmfgtcC1kgBWAR5cQoRngIE50NdTTfkAWB/4dn0HeDTQ2ZjkYtsLJc0CVgAuqcdnARsCmwNbAZfVOVYA7ltCjoiIESFFckTE8vND4FhgR2BCx7gGudZdf3cSMN32YUvxsxd6cXeoZ1n8/n4ScLztH0raEZjW8T1PA9heJKnz+xfV3y/gFtvbL0WOiIgRIdMtIiKWnzOAI23P6hq/knq6RF2ozrE9v2v87cC4+vqfAntIWrs+N17S5D8z0xosXkS431J+7/8Ba0navs6xoqQt/8wcERGtkiI5ImI5sX2P7RMHOTUN2EbSTcCXWFysHgFMkfRbYFfgrvrf+R3weeAn9fdcBvy5C+amAedLugqYszTfaPsZYA/gy5JuBP4b2OHPzBER0Spa/HQtIiIiIiIgd5IjIiIiIl4gRXJERERERJcUyRERERERXVIkR0RERER0SZEcEREREdElRXJERERERJcUyRERERERXVIkR0RERER0+X8F0HADEHXKNwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x720 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#report.plot(legend=2,kind='bar',figsize=(8, 7));\n",
    "ax = report[['Train accuracy','Test accuracy','Train auc score', 'Test auc score' ]].plot(kind='bar', title =\"Accuracy comparison\", figsize=(12, 10), fontsize=12)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**From the Above chart and Report table, we conclude that Linear SVC is the best model for the data based on the accuracy.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FINAL MODEL"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Training Whole Dataset using Linear SVC**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = Dataset_2[77]\n",
    "X_org = Dataset_2.drop([77], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_unscaled , X_text_unscaled, y_train, y_test = train_test_split(X_org, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "X_train = scaler.fit_transform(X_train_unscaled)\n",
    "X_test = scaler.transform(X_text_unscaled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearSVC(C=10, class_weight=None, dual=True, fit_intercept=True,\n",
       "     intercept_scaling=1, loss='squared_hinge', max_iter=1000,\n",
       "     multi_class='ovr', penalty='l2', random_state=None, tol=0.0001,\n",
       "     verbose=0)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "svc_lin = LinearSVC(C=10)\n",
    "svc_lin.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE: 0.18880755524784607\n",
      "Train score: 0.9657\n",
      "Test score: 0.9644\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "from math import sqrt\n",
    "\n",
    "#Predicting values for the test dataset\n",
    "predict = svc_lin.predict(X_test)\n",
    "\n",
    "#Calulate RMSE Value\n",
    "rmse = sqrt(mean_squared_error(y_test, predict))\n",
    "\n",
    "print('RMSE:', rmse)\n",
    "print('Train score: {:.4f}'.format(svc_lin.score(X_train, y_train)))\n",
    "print('Test score: {:.4f}'.format(svc_lin.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We obtained Training accuracy of 96.5% and Testing accuracy of 96.5% on the whole dataset using Linear SVC**\n",
    "\n",
    "**We Predicted values for Y_test and obtained RMSE of 0.19**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_range = [0.01, 0.1, 1, 10, 100]\n",
    "train_score_list = []\n",
    "test_score_list = []\n",
    "\n",
    "for alpha in x_range: \n",
    "    lins = LinearSVC(C=alpha)\n",
    "    lins.fit(X_train,y_train)\n",
    "    train_score_list.append(lins.score(X_train,y_train))\n",
    "    test_score_list.append(lins.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,0,'$\\\\alpha$')"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEOCAYAAACJlmBtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX+///nOyEhASI10hUELNQQI9ZdlEUEXLt+dVVELPxsHxcFBASVItiww6qsIugqtgVlVaREEEERgoQuRVAMhBYgoSQkk9y/PzLEAAEmjZnJvB7XNVdmTsv73ITzmnOfuc+Ycw4REZEwfxcgIiKBQYEgIiKAAkFERLwUCCIiAigQRETES4EgIiKAAkFERLwUCCIiAigQRETES4EgIiIAVPJ3AcVRp04d16RJE3+XISISVBYvXrzTORd7ouWCKhCaNGlCUlKSv8sQEQkqZva7L8upy0hERAAFgoiIeCkQREQEUCCIiIiXAkFERAAFgoiIeAXVx05FJLjl5uWyJm0NG3dvJDI8kioRVYiOiCa6UvRRPyPCI/xdbshRIIhIucjOzWbVjlX8nPpzwWPptqUcyDng0/rhFn7MsDjq5xHTqkRU8W29Qj8rh1fGzMq5VQKbAkFESi0zJ5Pl25cfdvBfvn052bnZAMRExtC+fnt6x/cmvn48Z9Y+k5y8HDJzMsn0ZB7z54GcA39OO2L+nqw9Ra6Xk5dTon0wjKhKUccNjYKgKWbYHOtnmAVWr70CQUSKZV/2PpK3Jh928F+1YxW5LheAmlE1ObfBufQ5vw/x9eOJrx9Ps1rNTtrBLzcv97ghU2TYHOtnoedpB9JI8aQctUyWJ6vEtUaGR/p8VjPishE0PKVhGbbU0RQIInJMuzN3s2TrksMO/mvT1uJwANStWpdzG5zLNWddU3DwP636aX7tegkPC6daZDWqRVY7Kb8vz+Vx0HPwqEA5kHPAp2A61hlQxsEMtu3bVvB6yF+HlPu+KBBEBIDt+7ezJDX/4L84dTE/p/7Mxj0bC+Y3PqUx8fXjubXNrQUH/wYxDfxYcWAIs7D8d/ER0RDt72pKR4EgEmKcc2zZu+XPd/1b83+mZKQULNOsZjMSGiTQ+9z8Pv/29doTW/WEN8uUIKdAEKnAnHP8tue3ow7+2/dvB/IvpJ5d52w6nt6x4F1/XL04akTV8HPl4g8+BYKZdQVeBcKBt51zzx4x/3RgPBAL7AJud86leOedBrwNNAYc0N0595uZTQA6AunezdzpnEsu9R6JhKg8l8e6tHVHHfz3ZO0BoFJYJVrFtuLKFlcWHPzb1m170vraJfCdMBDMLBwYC1wOpACLzGyqc25VocVGA+855yaaWSfgGaCHd957wEjn3EwzqwbkFVqvv3Pus7LYEZFQ4snzsHrH6sMO/slbk9mXvQ+AyuGVaVu3LTe3urng4N/61NZEVYryc+USyHw5Q+gArHfObQAws4+Aa4DCgdASeMT7fDbwuXfZlkAl59xMAOfcvjKqWyRkHPQcZMX2FYcd/JdtW1bwcceqEVWJqxdHr7heBQf/c+qco5G+Umy+BEJD4I9Cr1OA849YZilwA/ndStcBMWZWGzgT2GNmk4GmwCxgoHPeDyzDSDN7Ekj0Tj9Y4j0RqQD2Z+9n2bZlhx38V2xfgSfPA0D1ytWJrx/Pg+c9WHDwb1GrBeFh4X6uXCoCXwKhqA8UuyNe9wPGmNmdwFxgM+Dxbv8vQHtgE/AxcCfwDjAI2ApEAuOAAcDwo365WW+gN8Bpp53mQ7kiwSE9K/3PAV7e/v5fdv5CnsvvVa1TpQ7n1j+Xbhd1Kzj4N63RNOBur5CbC/v3w759vj8yM8EMwsL+/Fn4uS/zfJ3mj22Ux+9s3BgiI8v339KXQEgh/4LwIY2ALYUXcM5tAa4H8F4nuME5l25mKcCSQt1NnwMXAO8451K9qx80s3fJD5WjOOfGkR8YJCQkHBlEIkEh7UDaURd71+9aXzC/QUwD4uvHc1PLmwoO/g1jGpbpwd85yM4u3oH7yEdRB/7MTN9rqFQJYmIgKiq/HucgL+/Pn4Wfn2ieC7GjwerVcPbZ5fs7fAmERUALM2tK/jv/W4BbCy9gZnWAXc65PPLf+Y8vtG5NM4t1zu0AOgFJ3nXqO+dSLf8v/lpgRVnskIi/pe5NPergvyl9U8H8JjWaEF8/njvb3Zn/Gf/67alXrd5h28jLO/rgW9x34UU9PB7f9yM6GqpVO/pRr17R0315lOU73EOhUJpQKavlT8Y26tcvu7Y7lhMGgnPOY2YPAdPJ/9jpeOfcSjMbDiQ556YClwLPmJkjv8voQe+6uWbWD0j0HvgXA//2bvoDM4slv0sqGbivbHdN5ORYvWM17y6YzOLf1rIsZQM7d2dCdjXIrka9yi1oHNWNdpHNqR1+OqeENSB3RzT7lsCSffD9cd6J+yosrOiD76mnwhln+Hagrlr16NfhAX5Zwiz/AYFfa7AwF0TnXQkJCS4pKcnfZYgAsGP/Dv7vvTf4+IVLYGMnn9apXLl4B2ZfHlFRfx4YRYpiZoudcwknWk4jlUWK6aDnIM/PfoOnR4STPX8Qlatk8/Dj+zmzadUTHuwj9ElQCWAKBBEfOef4dOVnPPT8PHZMGQD7GnDj7bv510s1idVtfqQCCKxvZxAJUAs3L+Tcp+/g5r/HsuP9V2nRpBoLFsCn7ysMpOLQGYLIcWxK30T//43gk3+dDT+9S9UYDy/8K4/evU/RhUypcBQIIkXYe3Avz857jhfe3EzON6Ngf1163eXhheeiqF3b39WJlA8FgkghuXm5TEiewIAP/0PaZ0Ph9460iz/Iv98M47zzynmYqIifKRBEvBI3JPLPL55k5Sc3wsKZVK/ueGEc3H13ZcJ0tU1CgAJBQt4vO3+h34z+fPVZDcJmfY7tq0Pv3jBypKl7SEKKAkFCVtqBNIZ9N4x/fTkPvhoDv19E/Hl5vPEvI+GEQ3hEKh4FgoScg56DjF00lmHTXyXjm77YopepVdN47m3o1StM3UMSshQIEjKcc0z5ZQr9pw9gw3cXEfltMra3Bvffb4wYAbVq+btCEf9SIEhIWLxlMY/OeJS5C9KJnvkJbGhP/AUwdizEx/u7OpHAoECQCi0lI4XB3w7mvQVfEPX9aGzBXVStZYwdDz17ou4hkUIUCFIh7cvexwvzX+D5eaPJWXIrVWankJVRlQe83UM1a/q7QpHAo0CQCiXP5TExeSKDvx1M6tp61J6zhLS1ZxJ3UX73UFycvysUCVwKBKkwZm+cTd8ZfVmycSOn/vRvwubeQFht49134Y471D0kciIKBAl6a9PW8tjMx/hi9VRqre1LzPQR7EyvzIMPGsOHQ40a/q5QJDgoECRo7crcxYjvRjBm0Rgitl1A49l/8Meqhlx8cX73ULt2/q5QJLgoECToZOdm88aiNxj23TD27A7j7KUz+WV6R7JPNSZOhB499JWSIiWhQJCg4Zxj6pqp9J/Zn3U713NOymjc5w+zNr0SDz8Mw4ZB9er+rlIkeCkQJCgsSV1C3xl9mf3bbE7ffwNnTl/I6mU1+MtfYMwYaNvW3xWKBD8FggS0LXu3MOTbIUxInkCNvGZcvHwFP3zekrp1jf/8B269Vd1DImVFgSAB6UDOAUb/MJrn5j9Hdk4unXd/QtL717MgPYw+fWDoUDjlFH9XKVKxKBAkoOS5PD5Y9gGDEgexee9mOkUOZMfnTzEzOYq//jX/00OtW/u7SpGKSYEgAWPu73N5dPqjLE5dTLuYv9H+lyS++rge9erBBx/AP/6h7iGR8qRAEL9bv2s9A2YNYPLqyTSsdhp35izgi6c7sCLDeOQReOopdQ+JnAwKBPGb3Zm7eXru07y+8HUiwyPpXfcdFo67kwlLwrj00vxPD7Vq5e8qRUKHAkFOupzcHN5a/BZD5wxlV+Yu/nHGw7hZoxj3XhUaNIBJk+Dmm9U9JHKyKRDkpHHO8dW6r+g3ox9r0tZw2emduWDreN54uDH79kH//vDEExAT4+9KRUKT7v8oJ8XSrUu5/P3LuWrSVTgcz5/1PXtem8EzgxoTHw9Ll8LzzysMRPxJZwhSrrbu28oT3z7BO0veoWZ0TUZ2+DdrP7mLxyaE0bAhfPwx3HSTuodEAoECQcpFZk4mL/34Es/Me4bs3GwePu9RGvwynGdurcK+fTBgAAwZAtWq+btSETnEpy4jM+tqZmvMbL2ZDSxi/ulmlmhmy8xsjpk1KjTvNDObYWarzWyVmTXxTm9qZj+Z2Toz+9jMIstqp8R/Dg0sO2vMWQyZPYQuzbrw3rm/Mufx0QzoW4WEBFi+HJ59VmEgEmhOGAhmFg6MBboBLYF/mFnLIxYbDbznnGsLDAeeKTTvPeAF59w5QAdgu3f6c8DLzrkWwG7g7tLsiPjf/E3zufCdC7l9yu3EVo1lcvf5nDJtMv+4sjFpafDppzBjBpx9tr8rFZGi+HKG0AFY75zb4JzLBj4CrjlimZZAovf57EPzvcFRyTk3E8A5t885d8DMDOgEfOZdZyJwban2RPxmw+4N/L9P/x+XvHsJKRkpvHPlRO7IXESvyy/iww9h4ED45Re48UZdKxAJZL5cQ2gI/FHodQpw/hHLLAVuAF4FrgNizKw2cCawx8wmA02BWcBAoCawxznnKbTNhiXdCfGP9Kx0Rn4/kld/epVKYZUY2nEoF+Y9Rv+7o1m2DLp0gddeg7PO8nelIuILX84QinpP54543Q/oaGZLgI7AZsBDfuD8xTv/POAM4E4ft5n/y816m1mSmSXt2LHDh3KlvHnyPLyx6A2av96c0T+M5tY2tzLvpvX8Ov4prvhbNLt3w3//C998ozAQCSa+BEIK0LjQ60bAlsILOOe2OOeud861BwZ7p6V7113i7W7yAJ8D8cBOoIaZVTrWNgtte5xzLsE5lxAbG1uMXZPyMG3dNNq+0ZYHvn6AVrGtWHBXEu02vsulCfX5+GN4/HFYvRquv17dQyLBxpcuo0VACzNrSv47/1uAWwsvYGZ1gF3OuTxgEDC+0Lo1zSzWObeD/OsGSc45Z2azgRvJvybRE/iiLHZIyseK7SvoN6Mf03+dTvNazZly8xRqbruGe640li+HK66A11+HFi38XamIlNQJzxC87+wfAqYDq4FPnHMrzWy4mV3tXexSYI2ZrQXqAiO96+aS312UaGbLye8q+rd3nQHAo2a2HqgNvFNmeyVlZtu+bdz35X20e7MdP23+iZeveJlZ163ks6ev5dJLjfR0mDwZpk1TGIgEO3OuyK77gJSQkOCSkpL8XUZIyPJk8cqCVxj1/SgyPZk8kPAAgy56kknja/PUU3DwIDz2GAwaBFWq+LtaETkeM1vsnEs40XIaqSxHmfv7XO6Ycge/p//O1WddzfOdn2fryrPofDGsXAndu8Orr0Lz5v6uVETKkgJBDpPlyaLHlB5UCqtE4h2JnF25E/0ezr8ldZMm8MUXcNVVumAsUhEpEOQwr//0OpvSNzHjH7NZMvlSrhkKOTnw5JP5A8yio/1doYiUFwWCFEg7kMbI70dycaX/o891l7JqFfz97/DKK9Csmb+rE5HypkCQAqO+H0VGZiabP3iB7EyYOjW/e0hEQoMCQQDYuHsjYxaN4YKt4/nx18p8+SVceaW/qxKRk0nfmCYADP52MGEHa/LLZ7fQqVP+J4lEJLQoEISkLUlMWjGJ9us/Yc/ucF58UZ8iEglFCoQQ55yj34x+1MqK5+cpf+GOOyAuzt9ViYg/KBBC3Nfrvua737/jjCWTCAsznn7a3xWJiL8oEEKYJ8/DY7Meo/He60mafiZ9+0KjRideT0QqJn3KKIRNSJ7Aqu2rOGfOd5x6av69iUQkdCkQQtT+7P08OftJzto5gNWL6/DmmxAT4++qRMSfFAgh6qUfXyI1fQeVvn6Kli3h7rv9XZGI+JsCIQRt27eN5394nnZ/vMHSjdF89RVU0l+CSMjTReUQNOy7YWTurczvU3vxt79Bt27+rkhEAoECIcSs2bmGcYvH0XbtJNJ3hzN6tAahiUg+BUKIGZQ4iMr7zmLl/zrTs6cGoYnInxQIIWTepnlM+WUKzX7+mPBwY8QIf1ckIoFEgRAinHP0n9mfOru7szyxtQahichR9NmSEDF59WQW/LGAFt+lEl5Xg9BE5GgKhBCQnZvNwMSBNN7yMOuS6/HWWxqEJiJHU5dRCBi3eBzrt/9O7oxRtGwJd93l74pEJBDpDKGCS89KZ9h3w2ix8UXW/V6Vt7/WIDQRKZoODRXc8/OfZ2daDjlf3U/nztC1q78rEpFApS6jCiwlI4WXFrzEOb/8h4z0ShqEJiLHpUCowJ6c/SS5u07j12lXcued0K6dvysSkUCmLqMKavm25UxInsCZPy9mkwahiYgPdIZQQQ2YNYCq2zuxZk57+vWDhg39XZGIBDqdIVRAiRsSmbZuGk3mpVBVg9BExEcKhAomz+XRf2Z/Yjf15rdlDRk3DqpV83dVIhIMFAgVzIfLP2RJygrqJs6hVSvo1cvfFYlIsFAgVCBZniwGfzuYRmtHkbLpFCZM0yA0EfGdTxeVzayrma0xs/VmNrCI+aebWaKZLTOzOWbWqNC8XDNL9j6mFpo+wcw2FpqnO/OX0piFY9i0NYP0Gf/k8svhiiv8XZGIBJMTvn80s3BgLHA5kAIsMrOpzrlVhRYbDbznnJtoZp2AZ4Ae3nmZzrljHez7O+c+K3n5csiuzF2M/H4kTZf/m9/SI3jhBQ1CE5Hi8eUMoQOw3jm3wTmXDXwEXHPEMi2BRO/z2UXMl3I2cu5I0rfWImXW9RqEJiIl4ksgNAT+KPQ6xTutsKXADd7n1wExZlbb+zrKzJLMbIGZXXvEeiO93Uwvm1nl4hYv+Tbu3siYRWNokjSJiEphGoQmIiXiSyAU1fHgjnjdD+hoZkuAjsBmwOOdd5pzLgG4FXjFzJp5pw8CzgbOA2oBA4r85Wa9vYGStGPHDh/KDT2Dvx2MpVzAxnkdNAhNRErMl0BIARoXet0I2FJ4AefcFufc9c659sBg77T0Q/O8PzcAc4D23tepLt9B4F3yu6aO4pwb55xLcM4lxMbGFmffQkLSliQmLZ9E7Lz3qFcP+vf3d0UiEqx8CYRFQAsza2pmkcAtwNTCC5hZHTM7tK1BwHjv9JqHuoLMrA5wMbDK+7q+96cB1wIrSr87oeXQ9ySfsqEXKStPZ8QIDUITkZI74aeMnHMeM3sImA6EA+OdcyvNbDiQ5JybClwKPGNmDpgLPOhd/RzgLTPLIz98ni306aQPzCyW/C6pZOC+MtyvkPD1uq+Zs34+dWZ/TuvWGoQmIqVjzh15OSBwJSQkuKSkJH+XERA8eR7avdmO7bNuZeeUwUybpi+/EZGimdli77Xc49LdToPUhOQJrNq0hazE/nTpojAQkdJTIASh/dn7eXL2k9RfMpb9e/MHoYmIlJbudBOEXl7wMqmbooiYcwu9ehlt2/q7IhGpCBQIQWbbvm08N/85Gi78ht0ahCYiZUhdRkFm+HfDObChHZsXXEz//tCggb8rEpGKQmcIQWTNzjW8mfQWp85fA/WgXz9/VyQiFYkCIYgMShxE5Jpb2Lq6GW+/rUFoIlK2FAhBYv6m+UxZ8SU1v9tG89Zw553+rkhEKhoFQhAouEXFsoHs3lKTSeMhPNzfVYlIRaNACAKTV0/mx7W/UGXObK64Qt+EJiLlQ58yCnA5uTkMTBxI7aSXyNoXqUFoIlJudIYQ4N5a/Bbr1+dS6fs76NXLaNPG3xWJSEWlQAhgGQczGPbdMGJ//JT9Ecbw4f6uSEQqMgVCAHtu3nPsXNMcFl3K0KEahCYi5UuBEKBSMlJ48ceXqDNvKRH1NQhNRMqfAiFAPTX7KXJXXMvONWfy9ttQtaq/KxKRik6BEICWb1vO+KQPqD43hdPaaBCaiJwcCoQANGDWAKKWPEp6ah1GT9AgNBE5ORQIASZxQyLTli0geu5/ueIK6NLF3xWJSKjQwLQAkufy6D+zPzE/Pc/B/VGMHu3vikQklOgMIYBMWj6JJavSCZ/fi7vuMlq39ndFIhJKFAgBIsuTxePfPk6Nee+QUzlMg9BE5KRTl1GAGLNwDJuWN2TPz5157DGjfn1/VyQioUZnCAFgV+Yunp47khpzfyC6PvTt6++KRCQUKRACwMi5I8n4+Qrc+nN48R0NQhMR/1Ag+NnG3Rt5/cdxVJ27gTPaQs+e/q5IREKVAsHPhsweglv4APu2xTL6fQ1CExH/USD4UdKWJD786Ruivk+hc1e4/HJ/VyQioUyfMvKTQ9+THPXDSLIPROmb0ETE73SG4Cdfr/uaOYv/IHzBvdx9twahiYj/KRD8wJPn4bFZj1F17hioHMawYf6uSEREXUZ+MTF5IqsW12D/0q4ahCYiAUNnCCfZ/uz9DPn2CarNmcEpDRx9+5q/SxIRAXw8QzCzrma2xszWm9nAIuafbmaJZrbMzOaYWaNC83LNLNn7mFpoelMz+8nM1pnZx2YWWTa7FNheXvAyW3/6C/s2tObpp02D0EQkYJwwEMwsHBgLdANaAv8ws5ZHLDYaeM851xYYDjxTaF6mcy7O+7i60PTngJedcy2A3cDdpdiPoLB9/3ae/e5lqnz3Gm3bwh13+LsiEZE/+XKG0AFY75zb4JzLBj4CrjlimZZAovf57CLmH8bMDOgEfOadNBG41teig9WwOcM4MO8uDuyoy4svahCaiAQWXwKhIfBHodcp3mmFLQVu8D6/Dogxs9re11FmlmRmC8zs0EG/NrDHOec5zjYrlDU71/Dm958SOX8Y3bpB587+rkhE5HC+BEJRVz3dEa/7AR3NbAnQEdgMHDrYn+acSwBuBV4xs2Y+bjP/l5v19gZK0o4dO3woNzANShxE+PdPkZMZrUFoIhKQfAmEFKBxodeNgC2FF3DObXHOXe+caw8M9k5LPzTP+3MDMAdoD+wEaphZpWNts9C2xznnEpxzCbGxsb7uV0CZv2k+U+YvJ/en+7jnHqNVK39XJCJyNF8CYRHQwvupoEjgFmBq4QXMrI6ZHdrWIGC8d3pNM6t8aBngYmCVc86Rf63hRu86PYEvSrszgajgFhVzXiE6SoPQRCRwnTAQvP38DwHTgdXAJ865lWY23MwOfWroUmCNma0F6gIjvdPPAZLMbCn5AfCsc26Vd94A4FEzW0/+NYV3ymifAsrk1ZP5cX4YWcuvZMAAo149f1ckIlI0y3+zHhwSEhJcUlKSv8vwWU5uDueMbcmWl/5LzZw2rFtnVKni76pEJNSY2WLvtdzj0kjlcjRu8Th+/T4efm/Lv95FYSAiAU2BUE4yDmYwdNazRM1ZyFntHD166BYVIhLYFAjl5Pn5z7Nzzs2wsz6jJ2kQmogEPgVCOdicsZkXEycQMX8Nl3fXIDQRCQ4KhHLw5Ownyf52IGRV4fnn/V2NiIhvFAhlbPm25bz77Txs0TgNQhORoKJAKGMDZg2gUuILREZrEJqIBBd9Y1oZStyQyLTEveSsvJqBGoQmIkFGZwhlJM/l0X/GACIT36ZOQ8ejj+pjpiISXBQIZWTS8kksmdUcNsUxaoIGoYlI8FEglIEsTxaPTx9O5OzZtIxz3H67zg5EJPgoEMrAmIVj2DTzatjVgNGfaBCaiAQnBUIp7crcxYhv3qDS/KVccSX87W/+rkhEpGT0KaNSGvX9KDKm9yEvq6oGoYlIUNMZQils3L2R177+BluczL33Gi1b+rsiEZGS0xlCKQyZPYTcmSOpokFoIlIBKBBKaPGWxXz4vz/IW3UNgwaGUbeuvysSESkddRmVgHOOftMfo1LiK5zaII9HHlGuikjwUyCUwLT105jz1anwR7wGoYlIhaFAKCZPnod+Xw+h0uyptIrLo0cPnR2ISMWgQCimickTWf1lZ9jViJc+hTDlgYhUEAqEYtifvZ/BX75M+PwFdL3S0amTblEhIhWH3t8Ww8sLXmbbV/8fZFflhRcUBiJSsegMwUfb92/nmc8nY4sXcu+9xjnn+LsiEZGypTMEHw3/bjgHpj1JdFQYQ4f6uxoRkbKnQPDB2rS1vPnZKvjlWgY/rkFoIlIxKRB8MHDm47gZL9CgYS59+vi7GhGR8qFrCCcwf9N8pnwaCZvP5ZmJGoQmIhWXAuE4nHM8+tVgwr/9D63jcrn9dn3zjYhUXEEfCDk5OaSkpJCVlVXm2z6Qc4Bh54yCSXupW3cva9aU+a8IKlFRUTRq1IiIiAh/lyIi5SDoAyElJYWYmBiaNGmCWdmNDchzeazcuoY6OXFUbxhGixahPe7AOUdaWhopKSk0bdrU3+WISDkI+ovKWVlZ1K5du0zDAGDngZ0c3FMbXBiNGoV2GACYGbVr1y6XMzERCQxBHwhAmYdBbl4um3elwf5YYmMhOrpMNx+0yrqdRSSw+BQIZtbVzNaY2XozG1jE/NPNLNHMlpnZHDNrdMT8U8xss5mNKTRtjnebyd7HqaXfnbKxdd9WctPrERYGDRoc/yCYlpZGXFwccXFx1KtXj4YNGxa8zs7O9un39erVizXFuECRmppK9+7dadeuHS1btuTqq6/2eV0RkWM54TUEMwsHxgKXAynAIjOb6pxbVWix0cB7zrmJZtYJeAboUWj+COC7IjZ/m3MuqcTVl4Ps3Gy27toPWQ2p3xBOdP20du3aJCcnAzB06FCqVatGv379DlvGOYdzjrBj3Br13XffLVaNQ4YM4corr+TBBx8EYNmyZcVavygej4dKlYL+kpKIlIIvZwgdgPXOuQ3OuWzgI+CaI5ZpCSR6n88uPN/MzgXqAjNKX27525KxBZfekIgIV6oRyevXr6d169bcd999xMfHk5qaSu/evUlISKBVq1YMHz68YNlLLrmE5ORkPB4PNWrUYODAgbRr144LL7yQ7du3H7Xt1NRUGjX68ySsbdu2Bc9HjRpFmzZtaNeuHYMHDwbg559/5vzzz6dt27bccMMNpKenF/zewYMH89e//pUxY8awbds2rr/+ehISEujQoQMLFiwoeQOISNDxJRAaAn8Uep3inVbYUuAG7/PrgBgzq21mYcCLQP9jbPtdb3fRE3aMDmoz621mSWZGH7GVAAAM5ElEQVSWtGPHDh/KLbkDOQfYuSsPcqrSqJGV+rsOVq1axd13382SJUto2LAhzz77LElJSSxdupSZM2eyatWqo9ZJT0+nY8eOLF26lAsvvJDx48cftcxDDz1Ez5496dSpE6NGjSI1NRWA//3vf0ybNo2FCxeydOlS+vbtC8Dtt9/Oiy++yLJlyzjrrLMYMWJEwbYyMjKYO3cuffr04eGHH+axxx4jKSmJTz75hHvuuad0DSAiQcWXPoKiDtTuiNf9gDFmdicwF9gMeIAHgK+dc38Ucby/zTm32cxigP+S38X03lG/yLlxwDiAhISEI3/vYfp804fkrckn3KFjyczJxHMwirAwo+qK/Glx9eJ4pesrJdpes2bNOO+88wpeT5o0iXfeeQePx8OWLVtYtWoVLVu2PGyd6OhounXrBsC5557L999/f9R2u3fvzq+//so333zDtGnTaN++PStXrmTWrFncddddRHuvgteqVYu0tDSysrK45JJLAOjZsyc9evzZm3fLLbcUPJ81a9Zh1zJ2795NZmZmwfZEpGLzJRBSgMaFXjcCthRewDm3BbgewMyqATc459LN7ELgL2b2AFANiDSzfc65gc65zd5195rZh+R3TR0VCCdLbl4unpwwcEZU5bLZZtWqVQuer1u3jldffZWFCxdSo0YNbr/99iI/whkZGVnwPDw8HI/HU+S2a9euzW233cZtt91G165dmTdvHs65oz4J5NxxM/SwGp1zLFy48LAaRCR0+BIIi4AWZtaU/Hf+twC3Fl7AzOoAu5xzecAgYDyAc+62QsvcCSQ45waaWSWghnNup5lFAH8HZpV2Z0r6Tt45x6pta8nc3Jzqp5TPILSMjAxiYmI45ZRTSE1NZfr06XTt2rVE20pMTOSiiy4iOjqajIwMNm7cyGmnnUaXLl147rnnuPnmm4mOjmbXrl3UqVOH6OhofvjhBy666CLef/99OnbsWOR2O3fuzNixY3nkkUcASE5OJi4ursT7LCLB5YSB4JzzmNlDwHQgHBjvnFtpZsOBJOfcVOBS4Bkzc+R3GT14gs1WBqZ7wyCc/DD4d8l3o3R2Ze4ic3fNch2EFh8fT8uWLWndujVnnHEGF198cYm3tWjRIh566CEiIiLIy8vj/vvvp3379rRv356lS5eSkJBAREQEV111FSNGjOD999/n/vvvJzMzk+bNmx/zU01jx47l/vvv591338Xj8XDZZZcxduzYEtcpIsHFTtSlEEgSEhJcUtLhn1JdvXo155Ti68vyXB7LN68jZ+uZxMbC6adr8NXxlLa9ReTkM7PFzrmEEy1XIUYql8b2/dvJ2VPXp0FoIiIVWUgHgifPw5adeyGrBvXr2wkHoYmIVGQhHQipe1PJS29ARESevhZTREJeyAbCQc9Btu30eAehhZV6EJqISLAL2cNgSvoWyGhIdJU8atXydzUiIv4Xkncz25+9n907IyA3ktMag+7qLCISgmcIzjn+2J0K++pTvXoeMTGl215Z3P4aYPz48WzdurXIefPnz+f8888nLi6Oc84557B7EYmIlJWQO0PIOJjBvl2nlNkgNF9uf+2L8ePHEx8fT7169Y6a17NnTz7//HNat25Nbm5usb474Vhyc3MJDw8v9XZEpOIIqTME5xyb0raftG9CmzhxIh06dCAuLo4HHniAvLw8PB4PPXr0oE2bNrRu3ZrXXnuNjz/+mOTkZG6++eYizyx27NhREBTh4eEFN8Tbu3cvPXv2pE2bNrRt25bPP/8cgP/85z8F23/88ccBCm6tPWTIEDp06MDChQtZtGgRHTt25Nxzz6Vbt25s27atfBtERAJaSJ0h7Dywk4O7YwkLczRoUL5ZuGLFCqZMmcIPP/xApUqV6N27Nx999BHNmjVj586dLF++HIA9e/ZQo0YNXn/9dcaMGVPkvYP69OlDixYtuOyyy+jWrRt33HEHlStXZujQocTGxrJ8+XKcc+zZs4eUlBSGDBlCUlIS1atXp3Pnznz55Zd07dqV9PR04uPjefrppzl48CCXXXYZU6dOpU6dOnzwwQc88cQTjBs3rlzbRUQCV4UKhD59IPkYd792OPZnVcPlRFO5ssPXG3rGxcErJbhn3qxZs1i0aBEJCfmjxTMzM2ncuDFXXHEFa9as4Z///Cfdu3enS5cuJ9zWsGHD6NGjBzNmzOC9997j448/ZtasWcyaNavgrMDMqFmzJt9++y2dOnWiTp06ANx6663MnTuXrl27EhkZyXXXXQfk34Ji5cqVdO7cGcjvQir8pTsiEnoqVCAcT05uNs4TiZkjMrL8P1bknOOuu+4q8gLwsmXLmDZtGq+99hr//e9/fXpX3rx5c5o3b869995L7dq1SU9PL/btrqOjowuWd87Rtm3bIr9vQURCU4UKhGO9k8/JzWHZb6m43U1o2hRq1y7/Wjp37syNN97IP//5T+rUqUNaWhr79+8nOjqaqKgobrrpJpo2bcp9990HQExMDHv37i1yW1999RXdu3fHzFi7di2VK1cmJiaGLl26MGbMGEaPHl3QZXTBBRfQv39/0tLSqF69Oh999FGRF7lbtmzJ5s2bWbhwIR06dCA7O5t169bRqlWrcm0XEQlcFSoQjmVLxlZcRgOio/OoVevkXEdv06YNTz31FJ07dyYvL4+IiAjefPNNwsPDufvuuwve3T/33HMA9OrVi3vuuYfo6OijvqRmwoQJPPLII1SpUoWIiAg+/PBDwsLCeOqpp3jggQdo3bo14eHhjBgxgquvvprhw4dz6aWX4pzjqquu4sorrzzqi3YqV67MZ599xsMPP8zevXvxeDz07dtXgSASwkLi9tdrf9tLxs4YzjqLUo87CHW6/bVI8NHtrwuJJIYaNRQGIiLHExJdRk2aQF6ev6sQEQlsIXGGAOhupiIiJ1AhDpPBdB0kmKmdRSq2oA+EqKgo0tLSdLAqZ8450tLSiIqK8ncpIlJOgv4aQqNGjUhJSWHHjh3+LqXCi4qK0mhmkQos6AMhIiKCpk2b+rsMEZGgF/RdRiIiUjYUCCIiAigQRETEK6huXWFmO4DfvS+rA+lHLHLktMKv6wA7y6m0omopq3WOt9yx5vnSNkVNU3sVb1ogt5ev65VVexU1PdTa63jzi/v3dOTr0rbX6c652BMu5ZwLygcw7kTTCr8Gkk5mLWW1zvGWO9Y8X9pG7VWx28vX9cqqvU7UPqHQXsVts0Bpr8KPYO4y+p8P04papjyU5Pf4us7xljvWPF/apqhpaq/iTQvk9vJ1vbJqr6Kmh1p7HW9+Sf6eTlZ7FQiqLqPSMLMk58Pd/iSf2qt41F7Fo/YqnpPVXsF8hlBc+rLg4lF7FY/aq3jUXsVzUtorZM4QRETk+ELpDEFERI5DgSAiIoACQUREvBQIgJlda2b/NrMvzKyLv+sJdGZ2hpm9Y2af+buWQGVmVc1sovfv6jZ/1xPo9DdVPOV1zAr6QDCz8Wa23cxWHDG9q5mtMbP1ZjbweNtwzn3unLsXuBO4uRzL9bsyaq8Nzrm7y7fSwFPMtrse+Mz7d3X1SS82ABSnvUL1b6qwYrZXuRyzgj4QgAlA18ITzCwcGAt0A1oC/zCzlmbWxsy+POJxaqFVh3jXq8gmUHbtFWom4GPbAY2AP7yL5Z7EGgPJBHxvLylZe5XpMSvovw/BOTfXzJocMbkDsN45twHAzD4CrnHOPQP8/chtmJkBzwLTnHM/l2/F/lUW7RWqitN2QAr5oZBMxXjjVWzFbK9VJ7e6wFOc9jKz1ZTDMaui/qE25M93Z5D/n7PhcZb/P6AzcKOZ3VeehQWoYrWXmdU2szeB9mY2qLyLC3DHarvJwA1m9gZ+uAVBACuyvfQ3dUzH+vsql2NW0J8hHIMVMe2YI/Ccc68Br5VfOQGvuO2VBoRicBalyLZzzu0Hep3sYoLAsdpLf1NFO1Z7lcsxq6KeIaQAjQu9bgRs8VMtwUDtVXJqu+JRexXPSW2vihoIi4AWZtbUzCKBW4Cpfq4pkKm9Sk5tVzxqr+I5qe0V9IFgZpOAH4GzzCzFzO52znmAh4DpwGrgE+fcSn/WGSjUXiWntisetVfxBEJ76eZ2IiICVIAzBBERKRsKBBERARQIIiLipUAQERFAgSAiIl4KBBERARQIIiLipUAQERGg4t7cTuSkMLNWwKvAacD7wKnAe865RX4tTKQENFJZpITMLAr4GbgJ2AD8Aix2zl3v18JESkhnCCIl1xlYcujeMt6bj73o35JESk7XEERKrj35ZwiYWQNgn3Nuvn9LEik5BYJIyR0k//70AM8AkX6sRaTUFAgiJfch8FczWwMsBX40s1f8XJNIiemisoiIADpDEBERLwWCiIgACgQREfFSIIiICKBAEBERLwWCiIgACgQREfFSIIiICAD/PxneqL/vNUI+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_range, train_score_list, c = 'g', label = 'Train Score')\n",
    "plt.plot(x_range, test_score_list, c = 'b', label = 'Test Score')\n",
    "plt.xscale('log')\n",
    "plt.legend(loc = 3)\n",
    "plt.xlabel(r'$\\alpha$')\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}