USA Men's Basketball Roster Web Scrape.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "509abd85",
   "metadata": {},
   "source": [
    "# Web Scraping Practice"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "073d60b1",
   "metadata": {},
   "source": [
    "## Step-by-Step Google Chrome\n",
    "1. Import the following packages\n",
    "    - **import scrapy**\n",
    "    - **from scrapy import selector**\n",
    "    - **import requests**\n",
    "2. Visit website [HERE](https://en.wikipedia.org/wiki/2024_United_States_men%27s_Olympic_basketball_team#:~:text=The%20men's%20national%20basketball%20team,gold%20medal%20for%20the%20Americans.)\n",
    "    - Right click page and select \"Inspect\" - HTML code will appear\n",
    "    - Can return the HTML code of a specific object on a web page via hovering over the object right clicking and selecting \"Inspect\" - will return HTML element of the object on the page\n",
    "    - To copy the HTML code, right click select hover over **Copy**\n",
    "        - Choose **Copy XPath**\n",
    "3. Making a GET requests to the page url\n",
    "    - **url = 'https://.....'**\n",
    "    - **html = requests.get(url).content** --> will return HTML of the URL\n",
    "4. Create Selector object sel to interact with the HTML pulled from the GET request\n",
    "    - **sel = Selector(text = html)**\n",
    "5. Paste the xpath notation from the site in the selector object and edit as necessary\n",
    "    - sel.xpath('....')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a6b6f411",
   "metadata": {},
   "outputs": [],
   "source": [
    "# importing packages\n",
    "import scrapy\n",
    "from scrapy import Selector\n",
    "import requests as r\n",
    "import pandas as pd\n",
    "import string as str\n",
    "from datetime import datetime\n",
    "from datetime import timedelta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d45458b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiating web scarper objects via GET request\n",
    "url = \"https://en.wikipedia.org/wiki/2024_United_States_men%27s_Olympic_basketball_team#:~:text=The%20men's%20national%20basketball%20team,gold%20medal%20for%20the%20Americans.\"\n",
    "html = r.get(url).content #--> returns HTML code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f99acf35",
   "metadata": {},
   "outputs": [],
   "source": [
    "# selector object\n",
    "sel = Selector(text = html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "83066a34",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The United States was 5–0 in exhibition games but did not look unbeatable.\n"
     ]
    }
   ],
   "source": [
    "# Web Scraping description of the United States record in the 2024 olympics\n",
    "print(sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/p[10]/text()[1]').extract()[0]) # --> selecting data from web page"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "848471ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the positions of the USA Olympic Men's Basketball Roster\n",
    "positions = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td/span/a/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b1b7aa2c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the jersey numbers of the USA Olympic Men's Basketball Roster\n",
    "jersey_num = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td[2]/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9c76be97",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the player names of the USA Olympic Men's Basketball Roster\n",
    "players = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td[3]/a/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b1b8d0ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the birth dates of the USA Olympic Men's Basketball Roster\n",
    "birth_dates = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td[4]/span/span/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7188e764",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the heigiht(m) of the USA Olympic Men's Basketball Roster\n",
    "height_m = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td[5]/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f79ba5fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using xpath to extract the NBA Clubs/Teams of the USA Olympic Men's Basketball Roster\n",
    "nba_team = sel.xpath('//*[@id=\"mw-content-text\"]/div[1]/table[2]/tbody/tr[3]/td[1]/table/tbody/tr/td[6]/a/text()').extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7317b10f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# creating a dataframe from the scraped data\n",
    "df = pd.DataFrame({'Player Positions':positions,\n",
    "             'Jersey Numbers':jersey_num,\n",
    "             'Players':players,\n",
    "             'Birth Date':birth_dates,\n",
    "             'Height(M)':height_m,\n",
    "             'Club':nba_team})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3b77c364",
   "metadata": {},
   "outputs": [],
   "source": [
    "# data cleaning\n",
    "df['Jersey Numbers'] = df['Jersey Numbers'].astype('int64')\n",
    "df['Height(M)'] = df['Height(M)'].astype('string')\n",
    "df['Birth Date'] = pd.to_datetime(df['Birth Date'])\n",
    "\n",
    "now = datetime.now() #--> creating date time object for age calculation\n",
    "year = timedelta(days = 365.25) #--> creating year object to find age in years\n",
    "df['Age'] = round((now - df['Birth Date'])/year)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "dd066edb",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Player Positions</th>\n",
       "      <th>Jersey Numbers</th>\n",
       "      <th>Players</th>\n",
       "      <th>Birth Date</th>\n",
       "      <th>Height(M)</th>\n",
       "      <th>Club</th>\n",
       "      <th>Age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>PG</td>\n",
       "      <td>4</td>\n",
       "      <td>Stephen Curry</td>\n",
       "      <td>1988-03-14</td>\n",
       "      <td>1.91 m (6 ft 3 in)</td>\n",
       "      <td>Golden State Warriors</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>SG</td>\n",
       "      <td>5</td>\n",
       "      <td>Anthony Edwards</td>\n",
       "      <td>2001-08-05</td>\n",
       "      <td>1.93 m (6 ft 4 in)</td>\n",
       "      <td>Minnesota Timberwolves</td>\n",
       "      <td>23.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>F</td>\n",
       "      <td>6</td>\n",
       "      <td>LeBron James</td>\n",
       "      <td>1984-12-30</td>\n",
       "      <td>2.03 m (6 ft 8 in)</td>\n",
       "      <td>Los Angeles Lakers</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>F</td>\n",
       "      <td>7</td>\n",
       "      <td>Kevin Durant</td>\n",
       "      <td>1988-09-29</td>\n",
       "      <td>2.11 m (6 ft 11 in)</td>\n",
       "      <td>Phoenix Suns</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>G</td>\n",
       "      <td>8</td>\n",
       "      <td>Derrick White</td>\n",
       "      <td>1994-07-02</td>\n",
       "      <td>1.96 m (6 ft 5 in)</td>\n",
       "      <td>Boston Celtics</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>PG</td>\n",
       "      <td>9</td>\n",
       "      <td>Tyrese Haliburton</td>\n",
       "      <td>2000-02-29</td>\n",
       "      <td>1.96 m (6 ft 5 in)</td>\n",
       "      <td>Indiana Pacers</td>\n",
       "      <td>24.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>F</td>\n",
       "      <td>10</td>\n",
       "      <td>Jayson Tatum</td>\n",
       "      <td>1998-03-03</td>\n",
       "      <td>2.03 m (6 ft 8 in)</td>\n",
       "      <td>Boston Celtics</td>\n",
       "      <td>26.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>C</td>\n",
       "      <td>11</td>\n",
       "      <td>Joel Embiid</td>\n",
       "      <td>1994-03-16</td>\n",
       "      <td>2.13 m (7 ft 0 in)</td>\n",
       "      <td>Philadelphia 76ers</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>G</td>\n",
       "      <td>12</td>\n",
       "      <td>Jrue Holiday</td>\n",
       "      <td>1990-06-12</td>\n",
       "      <td>1.93 m (6 ft 4 in)</td>\n",
       "      <td>Boston Celtics</td>\n",
       "      <td>34.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>F/C</td>\n",
       "      <td>13</td>\n",
       "      <td>Bam Adebayo</td>\n",
       "      <td>1997-07-18</td>\n",
       "      <td>2.06 m (6 ft 9 in)</td>\n",
       "      <td>Miami Heat</td>\n",
       "      <td>27.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>F/C</td>\n",
       "      <td>14</td>\n",
       "      <td>Anthony Davis</td>\n",
       "      <td>1993-03-11</td>\n",
       "      <td>2.08 m (6 ft 10 in)</td>\n",
       "      <td>Los Angeles Lakers</td>\n",
       "      <td>31.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>SG</td>\n",
       "      <td>15</td>\n",
       "      <td>Devin Booker</td>\n",
       "      <td>1996-10-30</td>\n",
       "      <td>1.98 m (6 ft 6 in)</td>\n",
       "      <td>Phoenix Suns</td>\n",
       "      <td>28.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Player Positions  Jersey Numbers            Players Birth Date  \\\n",
       "0                PG               4      Stephen Curry 1988-03-14   \n",
       "1                SG               5    Anthony Edwards 2001-08-05   \n",
       "2                 F               6       LeBron James 1984-12-30   \n",
       "3                 F               7       Kevin Durant 1988-09-29   \n",
       "4                 G               8      Derrick White 1994-07-02   \n",
       "5                PG               9  Tyrese Haliburton 2000-02-29   \n",
       "6                 F              10       Jayson Tatum 1998-03-03   \n",
       "7                 C              11        Joel Embiid 1994-03-16   \n",
       "8                 G              12       Jrue Holiday 1990-06-12   \n",
       "9               F/C              13        Bam Adebayo 1997-07-18   \n",
       "10              F/C              14      Anthony Davis 1993-03-11   \n",
       "11               SG              15       Devin Booker 1996-10-30   \n",
       "\n",
       "               Height(M)                    Club   Age  \n",
       "0    1.91 m (6 ft 3 in)\n",
       "   Golden State Warriors  36.0  \n",
       "1    1.93 m (6 ft 4 in)\n",
       "  Minnesota Timberwolves  23.0  \n",
       "2    2.03 m (6 ft 8 in)\n",
       "      Los Angeles Lakers  40.0  \n",
       "3   2.11 m (6 ft 11 in)\n",
       "            Phoenix Suns  36.0  \n",
       "4    1.96 m (6 ft 5 in)\n",
       "          Boston Celtics  30.0  \n",
       "5    1.96 m (6 ft 5 in)\n",
       "          Indiana Pacers  24.0  \n",
       "6    2.03 m (6 ft 8 in)\n",
       "          Boston Celtics  26.0  \n",
       "7    2.13 m (7 ft 0 in)\n",
       "      Philadelphia 76ers  30.0  \n",
       "8    1.93 m (6 ft 4 in)\n",
       "          Boston Celtics  34.0  \n",
       "9    2.06 m (6 ft 9 in)\n",
       "              Miami Heat  27.0  \n",
       "10  2.08 m (6 ft 10 in)\n",
       "      Los Angeles Lakers  31.0  \n",
       "11   1.98 m (6 ft 6 in)\n",
       "            Phoenix Suns  28.0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#displaying df\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}