01_problem_definition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "256a6623-f62c-4cd1-89a7-407cfbd9bf82",
   "metadata": {},
   "source": [
    "# "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37112acf-ff91-482a-af10-ac80f8137c1e",
   "metadata": {},
   "source": [
    "# 01. Problem definition\n",
    "\n",
    "\n",
    "## Define the business problem you are trying to solve and the objectives you want to accomplish.\n",
    "The main goal is to hone skills in ML, particulally when dealing with \"relatevily\" big data files (a few Gb SQL DB, multiple tables, 10-15 million rows). \"Relatevily\" means that this dataset cannot be converted to Pandas DataFrame or, at least, it takes a lot of time. \n",
    "\n",
    "We will use the NBA dataset for this goal.\n",
    "\n",
    "The objectives to answer the following questions:\n",
    "\n",
    "- Efficiency (EFF) Prediction: Use such features as points, assists, rebounds, age, etc. to predict EFF using different ML methods.\n",
    "\n",
    "- Player Performance Prediction: Use linear regression to predict a player's performance in terms of points, assists, rebounds, or any other statistical category based on their previous games, their age, position, minutes played, etc. You can use data from the play_by_play and common_player_info tables for this.\n",
    "\n",
    "- Team Performance Prediction: Predict the number of wins a team might have in a season based on various parameters like team's average points per game, average rebounds, average assists, etc. Data from the team_info_common table can be used here.\n",
    "\n",
    "- Player Improvement Over Time: Analyze how a player's performance (points, rebounds, assists, etc.) improves or declines over time. This could be dependent on variables such as age, experience (number of seasons played), team changes, etc. This would require data from play_by_play, common_player_info, and potentially team_history tables.\n",
    "\n",
    "- Effect of Draft Pick on Career: Examine how a player's draft position (from draft_history) affects their overall career statistics or longevity in the NBA. This analysis could reveal if higher draft picks generally lead to better careers.\n",
    "\n",
    "- Player Attributes and Performance: Investigate relationships between player physical attributes (height, weight, wingspan from draft_combine_stats) and their on-court performance. You can study whether players with certain physical attributes are more likely to excel in specific areas (e.g., taller players and rebounding).\n",
    "\n",
    "- Impact of Home/Away Games: Assess the impact of playing at home vs. away on a team's performance. The play_by_play table might include the necessary data to investigate this aspect.\n",
    "\n",
    "- Etc.\n",
    "\n",
    "## Identify the target audience and stakeholders.\n",
    "It is just me &#x1F603; and other enthusiasts who love basketball and ML.\n",
    "\n",
    "## Determine the data sources required to solve the problem.\n",
    "There are three data sources.\n",
    "\n",
    "1. We will use [the NBA Database](https://github.com/wyattowalsh/nba-db):\n",
    "> This repository contains the associated code base for the creation and updating of [the Kaggle NBA Database](https://www.kaggle.com/datasets/wyattowalsh/basketball). The nba-api is utilized as the API client for stats.nba.com and numerous endpoints are extracted to produce the database tables. .SQLite is the database format of choice for this project. The database is updated daily and monthly via cron scheduled Kaggle Notebooks.\n",
    "\n",
    "    This SQLite DB has a size of about 3 Gb and 16 tables. One of the tables has more than 13 million rows.  \n",
    "\n",
    "    Unfortunatelly, this dataset does not provide a detailed description of all tables and columns. Some names are obvious, while others can be found on the project's GitHub page:\n",
    "\n",
    "    [User Guide](https://github.com/wyattowalsh/nba-db/blob/main/docs/user_guide/endpoints.md)\n",
    "\n",
    "    In addition, it may be useful to visit the following resources to better understand abbreviations and names:\n",
    "    - [The nba_api  project](https://github.com/swar/nba_api), especially: [Examples](https://github.com/swar/nba_api/tree/master/docs/examples) and [Endpoints](https://github.com/swar/nba_api/tree/master/docs/nba_api/stats)\n",
    "    - [Developer Portal](https://gom-uat.ngss.nba.com/ui/developer) and [NBA Game Distribution API](https://developer.geniussports.com/nbangss/rest/index_central.html)\n",
    "\n",
    "\n",
    "2. We will use [the balldontlie project](https://www.balldontlie.io/home.html?shell#introduction) to collect addional data, especially, about advanced statistics.\n",
    "\n",
    "3. We will use [basketball-reference.com](https://www.basketball-reference.com/) to collect addional data, especially, about common player info.\n",
    "\n",
    "   \n",
    "## Define the success criteria and metrics for evaluating the model's performance.\n",
    "The success criteria and performance metrics depend on the task and will be established during the research proces"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74237389-4278-4a0a-8f93-337e4a9fe9c4",
   "metadata": {},
   "source": [
    "# Memo with the main stages"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49baaa5e-33a3-426c-986d-33d331007e27",
   "metadata": {},
   "source": [
    "\n",
    "### Problem Definition:\n",
    "\n",
    "- Define the business problem you are trying to solve and the objectives you want to accomplish.\n",
    "- Identify the target audience and stakeholders.\n",
    "- Determine the data sources required to solve the problem.\n",
    "- Define the success criteria and metrics for evaluating the model's performance.\n",
    "### Data Collection:\n",
    "\n",
    "- Collect and gather the data needed to solve the problem.\n",
    "- Identify the relevant data sources and acquire the data.\n",
    "- Perform data quality checks to ensure the data is accurate and complete.\n",
    "- Store the data in a format that can be easily accessed and analyzed.\n",
    "### Data Exploration:\n",
    "\n",
    "- Perform data profiling to understand the structure, size, and quality of the data.\n",
    "- Use data visualization to explore the data and identify patterns or trends.\n",
    "- Use statistical analysis and hypothesis testing to gain insights into the data.\n",
    "### Data Preparation:\n",
    "\n",
    "- Prepare the data for modeling.\n",
    "- Perform data cleaning to remove missing or inconsistent data, and correct any errors.\n",
    "- Perform feature engineering to extract relevant features from the data.\n",
    "- Transform the data into a format that can be used by the machine learning algorithm.\n",
    "### Model Building:\n",
    "\n",
    "- Develop a machine learning model that can solve the problem.\n",
    "- Select an appropriate algorithm that is suited to the data and the problem.\n",
    "- Split the data into training and testing sets.\n",
    "- Train the model using the training data.\n",
    "- Tune the model by adjusting the hyperparameters to improve its performance.\n",
    "- Test the model using the testing data to evaluate its performance.\n",
    "### Model Evaluation:\n",
    "\n",
    "- Evaluate the performance of the model and assess its ability to solve the problem.\n",
    "- Measure the model's accuracy, precision, recall, F1 score, and other performance metrics.\n",
    "- Visualize the results to gain insights into the model's performance.\n",
    "- Compare the model's performance against the success criteria defined in the problem definition stage.\n",
    "### Model Deployment:\n",
    "\n",
    "- Deploy the model into production.\n",
    "- Integrate the model into the business process and make it available for use by end-users.\n",
    "- Develop a user interface or API to enable end-users to interact with the model.\n",
    "- Test the deployment to ensure that the model is working correctly in the production environment.\n",
    "### Model Maintenance:\n",
    "\n",
    "- Monitor and maintain the model's performance over time.\n",
    "- Track the model's performance using data from the production environment.\n",
    "- Identify any changes in the data or business environment that affect the model's accuracy and update the model as necessary.\n",
    "- Continuously improve the model by retraining it with new data and updating the algorithm or hyperparameters as needed."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9 (env_mc_tf)",
   "language": "python",
   "name": "env_mc_tf"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}