ann_smscg_ff_calsim3_style.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ANN CALSIM 3 style\n",
    "\n",
    "This notebook creates the ANN structure as mentioned in \n",
    "```\n",
    "Artificial Neural Network for Sacramento–San Joaquin Delta Flow–Salinity Relationship for CalSim 3.0\n",
    "Nimal C. Jayasundara, M.ASCE1; Sanjaya A. Seneviratne2; Erik Reyes3; and Francis I. Chung\n",
    "```\n",
    "\n",
    "The input structure consists of 8 inputs and their antecedent conditions expressed in daily or moving averaged values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Just the imports including the annutils (local) module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from tensorflow.keras.layers.experimental.preprocessing import Normalization\n",
    "from tensorflow.keras import layers\n",
    "#import keras\n",
    "\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.metrics import r2_score\n",
    "\n",
    "import hvplot.pandas\n",
    "import panel as pn\n",
    "import holoviews as hv\n",
    "hv.extension('bokeh')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "path_data = '.'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfin_on = pd.read_csv(os.path.join(path_data, 'smscg_input_on.csv'), index_col=0, parse_dates=True)\n",
    "dfin_off = pd.read_csv(os.path.join(path_data, 'smscg_input_off.csv'), index_col=0, parse_dates=True)\n",
    "\n",
    "dfout_on = pd.read_csv(os.path.join(path_data, 'smscg_output_on.csv'), index_col=0, parse_dates=True)\n",
    "dfout_off = pd.read_csv(os.path.join(path_data, 'smscg_output_off.csv'), index_col=0, parse_dates=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import input (features) and output (labels) data from csv files\n",
    "If you need to see how to build these files see [how to process dss files to create csv files](./read_calsim_and_collate_inputs.ipynb) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensorflow Board Setup\n",
    "A log directory to keep the training logs\n",
    "\n",
    "Tensorboard starts a separate process and is best started from the command line. Open a command window and activate this environment (i.e. keras) and goto the current directory. Then type in\n",
    "```\n",
    "tensorboard --logdir=./tf_training_logs/ --port=6006\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load_ext tensorboard\n",
    "# %tensorboard --logdir=./tf_training_logs/ --port=6006\n",
    "root_logdir = os.path.join(os.curdir, \"tf_training_logs\")\n",
    "tensorboard_cb = keras.callbacks.TensorBoard(root_logdir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Calibration and Validation Periods\n",
    "Calibration is from 1940 - 2015 and Validation from 1923 - 1939 as per the Calsim 3 ANN paper\n",
    "\n",
    "The output locations are names of the columns in the output(labels) csv files. For each location, an ANN is trained on all the specified data sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_locations = ['CVP_INTAKE', 'MIDR_INTAKE', 'OLDR_CCF', 'ROLD024',\n",
    "                    'RSAC081', 'RSAC092', 'RSAN007', 'RSAN018', 'SLMZU003', 'SLMZU011', 'VICT_INTAKE']\n",
    "calib_slice = slice('1940', '2015')\n",
    "valid_slice = slice('1923', '1939')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define Sequential model with 3 layers\n",
    "NFEATURES = 126  # (8 + 10)*7\n",
    "\n",
    "\n",
    "def build_model(nhidden1=8, nhidden2=2, act_func='sigmoid'):\n",
    "    model = keras.Sequential(\n",
    "        [\n",
    "            layers.Input(shape=(NFEATURES)),\n",
    "            layers.Dense(nhidden1, activation=act_func),\n",
    "            layers.Dense(nhidden2, activation=act_func),\n",
    "            layers.Dense(1, activation=keras.activations.linear)\n",
    "        ])\n",
    "    model.compile(optimizer=keras.optimizers.Adam(\n",
    "        learning_rate=0.001), loss=\"mse\")\n",
    "    #model.compile(optimizer=keras.optimizers.RMSprop(), loss=\"mse\")\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import annutils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for location in output_locations:\n",
    "    output_location = '%s_EC' % location\n",
    "    # create tuple of calibration and validation sets and the xscaler and yscaler on the combined inputs\n",
    "    (xallc, yallc), (xallv, yallv), xscaler, yscaler = \\\n",
    "        annutils.create_training_sets([dfin_on, dfin_off],\n",
    "                                      [dfout_on[[output_location]],\n",
    "                                       dfout_off[[output_location]]],\n",
    "                                      calib_slice=slice('1940', '2015'),\n",
    "                                      valid_slice=slice('1923', '1939'))\n",
    "    model = build_model(8, 2, act_func='sigmoid')\n",
    "    display(model.summary())\n",
    "    history = model.fit(\n",
    "        xallc,\n",
    "        yallc,\n",
    "        epochs=1000,\n",
    "        batch_size=128,\n",
    "        validation_data=(xallv, yallv),\n",
    "        callbacks=[\n",
    "            keras.callbacks.EarlyStopping(\n",
    "                monitor=\"val_loss\", patience=50, mode=\"min\", restore_best_weights=True),\n",
    "            tensorboard_cb\n",
    "        ],\n",
    "    )\n",
    "    # pd.DataFrame(history.history).hvplot(logy=True) # if you want to view the graph for calibration/validation training\n",
    "    annutils.save_model(location, model, xscaler, yscaler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Show the performance on the data sets visually\n",
    "\n",
    "Change the location to one of the locations for which the ANN is trained and run cells below to see performance on one or more of the data sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "location = 'RSAN018'\n",
    "output_location = '%s_EC' % location\n",
    "print('Location: ', location)\n",
    "annmodel = annutils.load_model(location)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annutils.show_performance(annmodel.model, dfin_on,\n",
    "                          dfout_on[output_location], annmodel.xscaler, annmodel.yscaler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annutils.show_performance(annmodel.model, dfin_off,\n",
    "                          dfout_off[output_location], annmodel.xscaler, annmodel.yscaler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Display weights and x and y scaling parameters\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annmodel.model.get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annmodel.xscaler.data_min_, annmodel.xscaler.data_max_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annmodel.xscaler.feature_range"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annmodel.xscaler.min_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "annmodel.xscaler.scale_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:anntraining]",
   "language": "python",
   "name": "conda-env-anntraining-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}