IMD_data_download_amrit_d.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "IMD_data_download_amrit.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyN3n1V97kwXV4jabdvIRQQc",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/amrit-thapa-2044/colab_miscellaneous/blob/main/IMD_data_download_amrit_d.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**<div align=\"center\"><font size=\"20\">Downloading IMD data</font>**\n",
        "\n",
        "Prepared by: Amrit Thapa \\\n",
        "Email: aamritjnu@gmail.com \\\n",
        "GitHub: https://github.com/amrit-thapa-2044\n",
        "\n",
        "---"
      ],
      "metadata": {
        "id": "Dbg3rBPG_RPS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Background"
      ],
      "metadata": {
        "id": "dlPWNn7A6_my"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The purpose of this notebook is to guide you through the process of downloading daily historical precipitation (0.25 by 0.25) and temperature (1 by 1) dataets developed by the Indian Meteorological Division (IMD) for India. More information about the dataset can be found [here](https://www.imdpune.gov.in/Clim_Pred_LRF_New/Grided_Data_Download.html). \n",
        "\n"
      ],
      "metadata": {
        "id": "ay0vcg6_uxEl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We make use of python package named ***IMDLIB*** developed by [Nandi and Patel (2020)](https://doi.org/10.5281/zenodo.4405232)\n",
        "\n",
        "Detail about the package can be obtained from this [link](https://github.com/iamsaswata/imdlib) "
      ],
      "metadata": {
        "id": "Zj3V6gbt5VCs"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Install IMDLIB package\n",
        "Users must install the imdlib package since it is not included in the Python included with Colab Notebook. Let's install the development version."
      ],
      "metadata": {
        "id": "YptWCCCB7EsM"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9lIpOzNBsio_"
      },
      "outputs": [],
      "source": [
        "!pip install git+https://github.com/iamsaswata/imdlib.git"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# load required package\n",
        "import imdlib as imd"
      ],
      "metadata": {
        "id": "KkUSG_Z7yLBm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Define user variables\n",
        "In this section, we define the time frame that we are interested in. Additionally, we will create two variables file_dir and output_filename to specify the input directory and create the output filename to download the data."
      ],
      "metadata": {
        "id": "BRb4UYKk7UvT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "start_yr = 2016\n",
        "end_yr = 2017 # should be equal to or greater than  start_yr\n",
        "variable = 'tmax' # options are ('rain'/'tmin'/ 'tmax')"
      ],
      "metadata": {
        "id": "3nnMWQhRsvKq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "file_dir='/content/'+variable\n",
        "print(file_dir)"
      ],
      "metadata": {
        "id": "AhNodFjytjqL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "output_filename='IMD_data_'+variable+'_'+str(start_yr)+'_'+str(end_yr)+'.nc'\n",
        "print(output_filename)"
      ],
      "metadata": {
        "id": "1r8n2wbb8cLl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Download data within colab\n",
        "Let's download the data using the get_data function from the imdlib package."
      ],
      "metadata": {
        "id": "cVS9PA2m7c0j"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = imd.get_data(variable, start_yr, end_yr, fn_format='yearwise')"
      ],
      "metadata": {
        "id": "0LEDtgmfD_Um"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Convert data to xarray object\n",
        "The output file is in binary format. In the following code, the data is converted into an xarray to make it easier to visualize and download."
      ],
      "metadata": {
        "id": "EpowKYk68GZd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ds = data.get_xarray()\n",
        "print(ds)"
      ],
      "metadata": {
        "id": "ckR1VooRtEZU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Visualize data\n",
        "\n",
        "To get an idea of the data, let's visualize it. We should first remove missing values in the data, which correspond to -999.0 for rain and 99.9 for temperature."
      ],
      "metadata": {
        "id": "ex0dJEC_8Qg9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "if variable == 'rain':\n",
        "    ds = ds.where(ds[variable] != -999.) #Remove NaN values\n",
        "else:\n",
        "    ds = ds.where(ds[variable] < 99.9) #Remove NaN values"
      ],
      "metadata": {
        "id": "7QH9VjStAx7J"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "ds[variable].mean('time').plot()"
      ],
      "metadata": {
        "id": "J6QVMRmzJWEy"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Convert to netCDF format\n",
        "\n",
        "We converted data to xarray in the previous section. To export the file , let's convert xarray to netCDF. "
      ],
      "metadata": {
        "id": "Av5IwgBI7mYm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.to_netcdf(output_filename, file_dir)"
      ],
      "metadata": {
        "id": "kmb6Mk_U2EcQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Dowload data to local machine\n",
        "\n",
        "To download the nc file we just created, we used the file.download function from the google.colab package."
      ],
      "metadata": {
        "id": "xeA6t9a07tDc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import files\n",
        "files.download(file_dir+'/'+output_filename)"
      ],
      "metadata": {
        "id": "PXtqUpqOt2ec"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Keep in Mind**\n",
        "\n",
        "* (-999.0) represents missing values in the rain data downloaded\n",
        "* 99.9 represents missing values in the temperature data downloaded\n",
        "\n",
        "You can further process the data using any GIS software.\n"
      ],
      "metadata": {
        "id": "S4g45vog4UlD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**<div align=\"center\"><font size=\"20\">Good Luck</font>**"
      ],
      "metadata": {
        "id": "oZV0HRx2UC1N"
      }
    }
  ]
}