SVM kernels.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f2ae73bf-33f5-4f01-bbb0-29483df8c32f",
   "metadata": {},
   "source": [
    "Q1. What is the relationship between polynomial functions and kernel functions in machine learning\n",
    "algorithms?\n",
    "\n",
    "Ans.\n",
    "In machine learning, the kernel function is used to transform the input data into a higher dimensional space. This is done by calculating the kernel function between each pair of data points. The resulting higher dimensional space is often more linearly separable than the original space, which can improve the performance of the machine learning algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b367f06f-5327-4ac8-82e7-85cf46112698",
   "metadata": {},
   "source": [
    "Q2. How can we implement an SVM with a polynomial kernel in Python using Scikit-learn?\n",
    "\n",
    "Ans.\n",
    "\n",
    "To implement an SVM with a polynomial kernel in Python using Scikit-learn, we can use the SVC class. The SVC class has a parameter called kernel that can be set to polynomial. The polynomial kernel takes two parameters: degree and gamma. The degree parameter specifies the degree of the polynomial kernel. The gamma parameter specifies the kernel coefficient.\n",
    "\n",
    "eg - \n",
    "import numpy as np\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "#Create the data\n",
    "X = np.array([[1, 2], [3, 4], [5, 6]])\n",
    "y = np.array([0, 1, 1])\n",
    "\n",
    "#Create the SVM classifier\n",
    "clf = SVC(kernel='polynomial', degree=2, gamma=1)\n",
    "\n",
    "#Train the classifier\n",
    "clf.fit(X, y)\n",
    "\n",
    "#Make predictions\n",
    "predictions = clf.predict(X)\n",
    "\n",
    "#Print the predictions\n",
    "print(predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31991896-6172-4264-8b19-0868ac8de53b",
   "metadata": {},
   "source": [
    "Q3. How does increasing the value of epsilon affect the number of support vectors in SVR?\n",
    "\n",
    "Ans.\n",
    "\n",
    "In SVR, the epsilon parameter specifies the amount of error that is allowed for the predictions. When the value of epsilon is increased, the SVR algorithm will allow for more error in the predictions. This will result in a smaller number of support vectors being used."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2a592fd-59ad-42fb-a53c-37a2180c7163",
   "metadata": {},
   "source": [
    "Q4. How does the choice of kernel function, C parameter, epsilon parameter, and gamma parameter\n",
    "affect the performance of Support Vector Regression (SVR)? Can you explain how each parameter works\n",
    "and provide examples of when you might want to increase or decrease its value?\n",
    "\n",
    "Ans.\n",
    "he choice of kernel function, C parameter, epsilon parameter, and gamma parameter all affect the performance of SVR.\n",
    "\n",
    "The kernel function determines how the data is transformed into a higher dimensional space. The C parameter controls the amount of regularization that is applied to the model. The epsilon parameter specifies the amount of error that is allowed for the predictions. The gamma parameter controls the smoothness of the decision boundary.\n",
    "\n",
    "Here are some examples of when we might want to increase or decrease the value of each parameter:\n",
    "\n",
    "1. Kernel function: If the data is not linearly separable, then you might want to use a kernel function that can transform the data into a higher dimensional space where it is linearly separable. For example, you might want to use a polynomial kernel with a high degree.\n",
    "\n",
    "2. C parameter: If you want to reduce the risk of overfitting, then you might want to increase the value of the C parameter. This will make the model more complex and less likely to overfit the data.\n",
    "\n",
    "3. Epsilon parameter: If you want to allow for more error in the predictions, then you might want to increase the value of the epsilon parameter. This will result in a smaller number of support vectors being used and a simpler decision boundary.\n",
    "\n",
    "4. Gamma parameter: If you want to make the decision boundary smoother, then you might want to decrease the value of the gamma parameter. This will make the model less sensitive to noise in the data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aac8300-4b58-450e-aac6-f6973fddc459",
   "metadata": {},
   "source": [
    "Q5. Assignment:\n",
    ". Import the necessary libraries and load the dataset\n",
    "\n",
    ". Split the dataset into training and testing set.\n",
    "\n",
    ". Preprocess the data using any technique of your choice (e.g. scaling, normalization)\n",
    "\n",
    ". Create an instance of the SVC classifier and train it on the training data\n",
    "\n",
    ". hse the trained classifier to predict the labels of the testing datW\n",
    "\n",
    ". Evaluate the performance of the classifier using any metric of your choice (e.g. accuracy,\n",
    "precision, recall, F1-scoreK\n",
    "\n",
    ". Tune the hyperparameters of the SVC classifier using GridSearchCV or RandomiMedSearchCV to\n",
    "improve its performanc_\n",
    "\n",
    ". Train the tuned classifier on the entire dataseg\n",
    "\n",
    ". Save the trained classifier to a file for future use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "id": "199dfc40-30fa-4ace-aba0-b4a7e8e38e50",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "id": "b34e9fa3-832c-419a-b96d-dcdd1ade9dba",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_iris"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "id": "de207d31-5e94-46e8-8afc-2490cef47276",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = load_iris()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 280,
   "id": "26872594-1568-485d-9bfd-95cb5b2ce4ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(dataset.data , columns = dataset.feature_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 281,
   "id": "e12fa689-29e0-49b1-94e7-0b7a43bfa5a5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)\n",
       "0                5.1               3.5                1.4               0.2\n",
       "1                4.9               3.0                1.4               0.2\n",
       "2                4.7               3.2                1.3               0.2\n",
       "3                4.6               3.1                1.5               0.2\n",
       "4                5.0               3.6                1.4               0.2"
      ]
     },
     "execution_count": 281,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "id": "a6729a70-fb03-447c-a048-4d930403c046",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Target'] = dataset.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "id": "4fd47446-dc50-4a0e-b7a4-c7a07e88c8af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sepal length (cm)</th>\n",
       "      <th>sepal width (cm)</th>\n",
       "      <th>petal length (cm)</th>\n",
       "      <th>petal width (cm)</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)  \\\n",
       "0                5.1               3.5                1.4               0.2   \n",
       "1                4.9               3.0                1.4               0.2   \n",
       "2                4.7               3.2                1.3               0.2   \n",
       "3                4.6               3.1                1.5               0.2   \n",
       "4                5.0               3.6                1.4               0.2   \n",
       "\n",
       "   Target  \n",
       "0       0  \n",
       "1       0  \n",
       "2       0  \n",
       "3       0  \n",
       "4       0  "
      ]
     },
     "execution_count": 283,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "id": "39b69a7c-35b3-4fb1-b4f0-3fcafa3015e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df.iloc[:,:-1]\n",
    "y = df.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "id": "eba3bb0b-2b9c-4a41-9a54-f1052d021519",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "id": "4ff09666-863f-4e34-ba52-ed6d39acef5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "id": "2617995e-9eaf-4061-802d-2f3638904b99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-5 {color: black;background-color: white;}#sk-container-id-5 pre{padding: 0;}#sk-container-id-5 div.sk-toggleable {background-color: white;}#sk-container-id-5 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-5 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-5 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-5 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-5 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-5 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-5 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-5 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-5 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-5 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-5 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-5 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-5 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-5 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-5 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-5 div.sk-item {position: relative;z-index: 1;}#sk-container-id-5 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-5 div.sk-item::before, #sk-container-id-5 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-5 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-5 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-5 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-5 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-5 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-5 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-5 div.sk-label-container {text-align: center;}#sk-container-id-5 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-5 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-5\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SVC()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-7\" type=\"checkbox\" checked><label for=\"sk-estimator-id-7\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "SVC()"
      ]
     },
     "execution_count": 287,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "svc = SVC()\n",
    "svc.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "id": "7f72539a-35eb-45e7-9291-cc259d3d766d",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = svc.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 289,
   "id": "2d1e098e-53df-4f96-8643-8387964de820",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 2, 1, 1, 0, 1, 2, 1, 1, 2, 0, 0, 0, 0, 1, 2, 1, 1, 2, 0, 2,\n",
       "       0, 2, 2, 2, 2, 2, 0, 0])"
      ]
     },
     "execution_count": 289,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "id": "f6dcbc72-1ff2-413a-982d-91976cec2cf2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "1.0\n",
      "[[10  0  0]\n",
      " [ 0  9  0]\n",
      " [ 0  0 11]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       1.00      1.00      1.00        10\n",
      "           1       1.00      1.00      1.00         9\n",
      "           2       1.00      1.00      1.00        11\n",
      "\n",
      "    accuracy                           1.00        30\n",
      "   macro avg       1.00      1.00      1.00        30\n",
      "weighted avg       1.00      1.00      1.00        30\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score,confusion_matrix,classification_report\n",
    "score = r2_score(y_test,y_pred)\n",
    "print(score)\n",
    "print(accuracy_score(y_test,y_pred))\n",
    "print(confusion_matrix(y_test,y_pred))\n",
    "print(classification_report(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "id": "3b50fc87-0dc5-4df7-a7e1-66badcdb5534",
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\n",
    "    'C':[1,2,3,4,5],\n",
    "    'gamma':[1,0.1,0.01,0.001],\n",
    "    'kernel':['rbf','poly','linear']\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "id": "bd9f9fd1-5f83-4735-842c-46a5bc43e2b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "grid = GridSearchCV(svc,param_grid = params , cv = 5, verbose = 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 293,
   "id": "62ebbe07-ed2a-4669-abed-de30c5713ac7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 60 candidates, totalling 300 fits\n",
      "[CV 1/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ..........C=1, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.792 total time=   0.0s\n",
      "[CV 4/5] END ..........C=1, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ..........C=1, gamma=1, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 1/5] END .........C=1, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .........C=1, gamma=1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 3/5] END .........C=1, gamma=1, kernel=poly;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .........C=1, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 5/5] END .........C=1, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=1, gamma=1, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .......C=1, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .......C=1, gamma=1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=1, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=1, gamma=1, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ........C=1, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ........C=1, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=1, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .......C=1, gamma=0.1, kernel=poly;, score=0.667 total time=   0.0s\n",
      "[CV 3/5] END .......C=1, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=1, gamma=0.1, kernel=poly;, score=0.750 total time=   0.0s\n",
      "[CV 5/5] END .......C=1, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=1, gamma=0.1, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .....C=1, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .....C=1, gamma=0.1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .....C=1, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .....C=1, gamma=0.1, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.792 total time=   0.0s\n",
      "[CV 2/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 3/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 5/5] END .......C=1, gamma=0.01, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 1/5] END ......C=1, gamma=0.01, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END ......C=1, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=1, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=1, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=1, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ....C=1, gamma=0.01, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ....C=1, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ....C=1, gamma=0.01, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ....C=1, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ....C=1, gamma=0.01, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 2/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=1, gamma=0.001, kernel=rbf;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END .....C=1, gamma=0.001, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .....C=1, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END .....C=1, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END .....C=1, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END .....C=1, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ...C=1, gamma=0.001, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ...C=1, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ...C=1, gamma=0.001, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ...C=1, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ...C=1, gamma=0.001, kernel=linear;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END ..........C=2, gamma=1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ..........C=2, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ..........C=2, gamma=1, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ..........C=2, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ..........C=2, gamma=1, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 1/5] END .........C=2, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .........C=2, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .........C=2, gamma=1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .........C=2, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 5/5] END .........C=2, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=2, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .......C=2, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .......C=2, gamma=1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=2, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=2, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ........C=2, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ........C=2, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ........C=2, gamma=0.1, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ........C=2, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ........C=2, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=2, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .......C=2, gamma=0.1, kernel=poly;, score=0.750 total time=   0.0s\n",
      "[CV 3/5] END .......C=2, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 4/5] END .......C=2, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 5/5] END .......C=2, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=2, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .....C=2, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .....C=2, gamma=0.1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .....C=2, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .....C=2, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END .......C=2, gamma=0.01, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 2/5] END .......C=2, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 3/5] END .......C=2, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .......C=2, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 5/5] END .......C=2, gamma=0.01, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 1/5] END ......C=2, gamma=0.01, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END ......C=2, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=2, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=2, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=2, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ....C=2, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ....C=2, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ....C=2, gamma=0.01, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ....C=2, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ....C=2, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ......C=2, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 2/5] END ......C=2, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 3/5] END ......C=2, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 4/5] END ......C=2, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 5/5] END ......C=2, gamma=0.001, kernel=rbf;, score=0.625 total time=   0.0s\n",
      "[CV 1/5] END .....C=2, gamma=0.001, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .....C=2, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END .....C=2, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END .....C=2, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END .....C=2, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ...C=2, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ...C=2, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ...C=2, gamma=0.001, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ...C=2, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ...C=2, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ..........C=3, gamma=1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ..........C=3, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ..........C=3, gamma=1, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ..........C=3, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ..........C=3, gamma=1, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 1/5] END .........C=3, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .........C=3, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .........C=3, gamma=1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .........C=3, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 5/5] END .........C=3, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=3, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .......C=3, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .......C=3, gamma=1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=3, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=3, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ........C=3, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ........C=3, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ........C=3, gamma=0.1, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ........C=3, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ........C=3, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=3, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .......C=3, gamma=0.1, kernel=poly;, score=0.750 total time=   0.0s\n",
      "[CV 3/5] END .......C=3, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 4/5] END .......C=3, gamma=0.1, kernel=poly;, score=0.875 total time=   0.0s\n",
      "[CV 5/5] END .......C=3, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=3, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .....C=3, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .....C=3, gamma=0.1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .....C=3, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .....C=3, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END .......C=3, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .......C=3, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .......C=3, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .......C=3, gamma=0.01, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=3, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 1/5] END ......C=3, gamma=0.01, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END ......C=3, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=3, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=3, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=3, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ....C=3, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ....C=3, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ....C=3, gamma=0.01, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ....C=3, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ....C=3, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ......C=3, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 2/5] END ......C=3, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 3/5] END ......C=3, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 4/5] END ......C=3, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 5/5] END ......C=3, gamma=0.001, kernel=rbf;, score=0.667 total time=   0.0s\n",
      "[CV 1/5] END .....C=3, gamma=0.001, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .....C=3, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END .....C=3, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END .....C=3, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END .....C=3, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ...C=3, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ...C=3, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ...C=3, gamma=0.001, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ...C=3, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ...C=3, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ..........C=4, gamma=1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ..........C=4, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ..........C=4, gamma=1, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ..........C=4, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ..........C=4, gamma=1, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 1/5] END .........C=4, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .........C=4, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .........C=4, gamma=1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .........C=4, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 5/5] END .........C=4, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=4, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .......C=4, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .......C=4, gamma=1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=4, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=4, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ........C=4, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ........C=4, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ........C=4, gamma=0.1, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ........C=4, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ........C=4, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=4, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .......C=4, gamma=0.1, kernel=poly;, score=0.750 total time=   0.0s\n",
      "[CV 3/5] END .......C=4, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 4/5] END .......C=4, gamma=0.1, kernel=poly;, score=0.875 total time=   0.0s\n",
      "[CV 5/5] END .......C=4, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=4, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .....C=4, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .....C=4, gamma=0.1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .....C=4, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .....C=4, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END .......C=4, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .......C=4, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .......C=4, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .......C=4, gamma=0.01, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=4, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END ......C=4, gamma=0.01, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END ......C=4, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=4, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=4, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=4, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ....C=4, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ....C=4, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ....C=4, gamma=0.01, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ....C=4, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ....C=4, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ......C=4, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 2/5] END ......C=4, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 3/5] END ......C=4, gamma=0.001, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 4/5] END ......C=4, gamma=0.001, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 5/5] END ......C=4, gamma=0.001, kernel=rbf;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=4, gamma=0.001, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .....C=4, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END .....C=4, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END .....C=4, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END .....C=4, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ...C=4, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ...C=4, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ...C=4, gamma=0.001, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ...C=4, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ...C=4, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ..........C=5, gamma=1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ..........C=5, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ..........C=5, gamma=1, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ..........C=5, gamma=1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ..........C=5, gamma=1, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 1/5] END .........C=5, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .........C=5, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .........C=5, gamma=1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .........C=5, gamma=1, kernel=poly;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .........C=5, gamma=1, kernel=poly;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=5, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .......C=5, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .......C=5, gamma=1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .......C=5, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=5, gamma=1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ........C=5, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END ........C=5, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ........C=5, gamma=0.1, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ........C=5, gamma=0.1, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ........C=5, gamma=0.1, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END .......C=5, gamma=0.1, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .......C=5, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 3/5] END .......C=5, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 4/5] END .......C=5, gamma=0.1, kernel=poly;, score=0.917 total time=   0.0s\n",
      "[CV 5/5] END .......C=5, gamma=0.1, kernel=poly;, score=0.792 total time=   0.0s\n",
      "[CV 1/5] END .....C=5, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END .....C=5, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END .....C=5, gamma=0.1, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END .....C=5, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .....C=5, gamma=0.1, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END .......C=5, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 2/5] END .......C=5, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 3/5] END .......C=5, gamma=0.01, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END .......C=5, gamma=0.01, kernel=rbf;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END .......C=5, gamma=0.01, kernel=rbf;, score=0.958 total time=   0.0s\n",
      "[CV 1/5] END ......C=5, gamma=0.01, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END ......C=5, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END ......C=5, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END ......C=5, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END ......C=5, gamma=0.01, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ....C=5, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ....C=5, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ....C=5, gamma=0.01, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ....C=5, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ....C=5, gamma=0.01, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 1/5] END ......C=5, gamma=0.001, kernel=rbf;, score=0.708 total time=   0.0s\n",
      "[CV 2/5] END ......C=5, gamma=0.001, kernel=rbf;, score=0.750 total time=   0.0s\n",
      "[CV 3/5] END ......C=5, gamma=0.001, kernel=rbf;, score=0.875 total time=   0.0s\n",
      "[CV 4/5] END ......C=5, gamma=0.001, kernel=rbf;, score=0.917 total time=   0.0s\n",
      "[CV 5/5] END ......C=5, gamma=0.001, kernel=rbf;, score=0.833 total time=   0.0s\n",
      "[CV 1/5] END .....C=5, gamma=0.001, kernel=poly;, score=0.833 total time=   0.0s\n",
      "[CV 2/5] END .....C=5, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 3/5] END .....C=5, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 4/5] END .....C=5, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 5/5] END .....C=5, gamma=0.001, kernel=poly;, score=0.333 total time=   0.0s\n",
      "[CV 1/5] END ...C=5, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 2/5] END ...C=5, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 3/5] END ...C=5, gamma=0.001, kernel=linear;, score=0.833 total time=   0.0s\n",
      "[CV 4/5] END ...C=5, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n",
      "[CV 5/5] END ...C=5, gamma=0.001, kernel=linear;, score=1.000 total time=   0.0s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-6 {color: black;background-color: white;}#sk-container-id-6 pre{padding: 0;}#sk-container-id-6 div.sk-toggleable {background-color: white;}#sk-container-id-6 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-6 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-6 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-6 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-6 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-6 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-6 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-6 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-6 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-6 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-6 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-6 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-6 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-6 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-6 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-6 div.sk-item {position: relative;z-index: 1;}#sk-container-id-6 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-6 div.sk-item::before, #sk-container-id-6 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-6 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-6 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-6 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-6 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-6 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-6 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-6 div.sk-label-container {text-align: center;}#sk-container-id-6 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-6 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-6\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=5, estimator=SVC(),\n",
       "             param_grid={&#x27;C&#x27;: [1, 2, 3, 4, 5], &#x27;gamma&#x27;: [1, 0.1, 0.01, 0.001],\n",
       "                         &#x27;kernel&#x27;: [&#x27;rbf&#x27;, &#x27;poly&#x27;, &#x27;linear&#x27;]},\n",
       "             verbose=3)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-8\" type=\"checkbox\" ><label for=\"sk-estimator-id-8\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GridSearchCV</label><div class=\"sk-toggleable__content\"><pre>GridSearchCV(cv=5, estimator=SVC(),\n",
       "             param_grid={&#x27;C&#x27;: [1, 2, 3, 4, 5], &#x27;gamma&#x27;: [1, 0.1, 0.01, 0.001],\n",
       "                         &#x27;kernel&#x27;: [&#x27;rbf&#x27;, &#x27;poly&#x27;, &#x27;linear&#x27;]},\n",
       "             verbose=3)</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-9\" type=\"checkbox\" ><label for=\"sk-estimator-id-9\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: SVC</label><div class=\"sk-toggleable__content\"><pre>SVC()</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-10\" type=\"checkbox\" ><label for=\"sk-estimator-id-10\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC()</pre></div></div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "GridSearchCV(cv=5, estimator=SVC(),\n",
       "             param_grid={'C': [1, 2, 3, 4, 5], 'gamma': [1, 0.1, 0.01, 0.001],\n",
       "                         'kernel': ['rbf', 'poly', 'linear']},\n",
       "             verbose=3)"
      ]
     },
     "execution_count": 293,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 294,
   "id": "f19e13bf-4ed9-466d-babc-2887c79d97e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'C': 2, 'gamma': 1, 'kernel': 'linear'}"
      ]
     },
     "execution_count": 294,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "id": "04ba8034-048c-4f54-94c0-25b6e287c8e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "with open('svc.pkl', 'wb') as file:\n",
    "    pickle.dump(svc, file)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}