Exercise02.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.set_option('display.max_colwidth', 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_DIR = 'data/sentiment_labelled_sentences/'\n",
    "IMDB_DATA_FILE = DATA_DIR + 'imdb_labelled.txt'\n",
    "YELP_DATA_FILE = DATA_DIR + 'yelp_labelled.txt'\n",
    "AMAZON_DATA_FILE = DATA_DIR + 'amazon_cells_labelled.txt'\n",
    "COLUMN_NAMES = ['Review', 'Sentiment']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IMDB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "imdb_reviews =  pd.read_table(IMDB_DATA_FILE, names=COLUMN_NAMES)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Review</th>\n",
       "      <th>Sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A very, very, very slow-moving, aimless movie about a distressed, drifting young man.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Not sure who was more lost - the flat characters or the audience, nearly half of whom walked out.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Attempting artiness with black &amp; white and clever camera angles, the movie disappointed - became even more ridiculous - as the acting was poor and the plot and lines almost non-existent.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Very little music or anything to speak of.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The best scene in the movie was when Gerardo is trying to find a song that keeps running through his head.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The rest of the movie lacks art, charm, meaning... If it's about emptiness, it works I guess because it's empty.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Wasted two hours.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Saw the movie today and thought it was a good effort, good messages for kids.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>A bit predictable.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Loved the casting of Jimmy Buffet as the science teacher.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                                                                                                         Review  \\\n",
       "0                                                                                                       A very, very, very slow-moving, aimless movie about a distressed, drifting young man.     \n",
       "1                                                                                           Not sure who was more lost - the flat characters or the audience, nearly half of whom walked out.     \n",
       "2  Attempting artiness with black & white and clever camera angles, the movie disappointed - became even more ridiculous - as the acting was poor and the plot and lines almost non-existent.     \n",
       "3                                                                                                                                                  Very little music or anything to speak of.     \n",
       "4                                                                                  The best scene in the movie was when Gerardo is trying to find a song that keeps running through his head.     \n",
       "5                                                                            The rest of the movie lacks art, charm, meaning... If it's about emptiness, it works I guess because it's empty.     \n",
       "6                                                                                                                                                                           Wasted two hours.     \n",
       "7                                                                                                               Saw the movie today and thought it was a good effort, good messages for kids.     \n",
       "8                                                                                                                                                                          A bit predictable.     \n",
       "9                                                                                                                                   Loved the casting of Jimmy Buffet as the science teacher.     \n",
       "\n",
       "   Sentiment  \n",
       "0          0  \n",
       "1          0  \n",
       "2          0  \n",
       "3          0  \n",
       "4          1  \n",
       "5          0  \n",
       "6          0  \n",
       "7          1  \n",
       "8          0  \n",
       "9          1  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_reviews.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    386\n",
       "0    362\n",
       "Name: Sentiment, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_reviews.Sentiment.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Positive</th>\n",
       "      <td>386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Negative</th>\n",
       "      <td>362</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          Sentiment\n",
       "Positive        386\n",
       "Negative        362"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_counts = imdb_reviews.Sentiment.value_counts().to_frame()\n",
    "imdb_counts.index = pd.Series(['Positive','Negative'])\n",
    "imdb_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Amazon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Review</th>\n",
       "      <th>Sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>So there is no way for me to plug it in here in the US unless I go by a converter.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Good case, Excellent value.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Great for the jawbone.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Tied to charger for conversations lasting more than 45 minutes.MAJOR PROBLEMS!!</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The mic is great.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>I have to jiggle the plug to get it to line up right to get decent volume.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>If you have several dozen or several hundred contacts, then imagine the fun of sending each of them one by one.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>If you are Razr owner...you must have this!</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Needless to say, I wasted my money.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>What a waste of money and time!.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                            Review  \\\n",
       "0                               So there is no way for me to plug it in here in the US unless I go by a converter.   \n",
       "1                                                                                      Good case, Excellent value.   \n",
       "2                                                                                           Great for the jawbone.   \n",
       "3                                  Tied to charger for conversations lasting more than 45 minutes.MAJOR PROBLEMS!!   \n",
       "4                                                                                                The mic is great.   \n",
       "5                                       I have to jiggle the plug to get it to line up right to get decent volume.   \n",
       "6  If you have several dozen or several hundred contacts, then imagine the fun of sending each of them one by one.   \n",
       "7                                                                      If you are Razr owner...you must have this!   \n",
       "8                                                                              Needless to say, I wasted my money.   \n",
       "9                                                                                 What a waste of money and time!.   \n",
       "\n",
       "   Sentiment  \n",
       "0          0  \n",
       "1          1  \n",
       "2          1  \n",
       "3          0  \n",
       "4          1  \n",
       "5          0  \n",
       "6          0  \n",
       "7          1  \n",
       "8          0  \n",
       "9          0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amazon_reviews = pd.read_table(AMAZON_DATA_FILE, names=COLUMN_NAMES)\n",
    "amazon_reviews.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Yelp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Review</th>\n",
       "      <th>Sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Wow... Loved this place.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Crust is not good.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Not tasty and the texture was just nasty.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Stopped by during the late May bank holiday off Rick Steve recommendation and loved it.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The selection on the menu was great and so were the prices.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Now I am getting angry and I want my damn pho.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Honeslty it didn't taste THAT fresh.)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>The potatoes were like rubber and you could tell they had been made up ahead of time being kept under a warmer.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The fries were great too.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>A great touch.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                            Review  \\\n",
       "0                                                                                         Wow... Loved this place.   \n",
       "1                                                                                               Crust is not good.   \n",
       "2                                                                        Not tasty and the texture was just nasty.   \n",
       "3                          Stopped by during the late May bank holiday off Rick Steve recommendation and loved it.   \n",
       "4                                                      The selection on the menu was great and so were the prices.   \n",
       "5                                                                   Now I am getting angry and I want my damn pho.   \n",
       "6                                                                            Honeslty it didn't taste THAT fresh.)   \n",
       "7  The potatoes were like rubber and you could tell they had been made up ahead of time being kept under a warmer.   \n",
       "8                                                                                        The fries were great too.   \n",
       "9                                                                                                   A great touch.   \n",
       "\n",
       "   Sentiment  \n",
       "0          1  \n",
       "1          0  \n",
       "2          0  \n",
       "3          1  \n",
       "4          1  \n",
       "5          0  \n",
       "6          0  \n",
       "7          0  \n",
       "8          1  \n",
       "9          1  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yelp_reviews = pd.read_table(YELP_DATA_FILE, names=COLUMN_NAMES)\n",
    "yelp_reviews.head(10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}